@zappie/verify 0.2.9 → 0.3.1

package/CHANGELOG.md

@@ -1,5 +1,25 @@
 # @zappie/verify
 
+## 0.3.1
+
+### Patch Changes
+
+- 0b31798: V03 accepts a card that shows an item's starting price. When `showsFromPrice` from `@zappie/sdk/pricing` is true for an item, its card can show `startingPrice`, with or without "From", as well as the item's price. Before, V03 failed a card that showed "From ₦9,700" for an item priced ₦8,500 with a required choice.
+- 0b31798: V05 finds a required option group by `data-zappie-required="true"` as well as `aria-required="true"`. A required group that takes several choices has no `aria-required`, because ARIA doesn't allow it on `role="group"`, so before V05 failed it with `required_group_missing`. Sites on an older `@zappie/foundations` still pass on `aria-required`. When Add stays enabled and the only unmet required group takes a typed answer, V05 no longer waits for a group of choices to flag.
+- 0b31798: `verify.startPath` and `--start` take `{menu}` and `{branch}`, as `cataloguePath` does. `{menu}` becomes the id of the first menu that holds items, in the order V03 judges menus, and `{branch}` the id of the default branch. Set `/menu?menu={menu}` for a site whose menu page asks the shopper to choose a menu. The report records the filled path in `environment.startPath` and the path as written in `environment.startPathTemplate`. Without `cataloguePath`, V03 judges only the start menu's items on the start page. The item checks still open the start page, so set `cataloguePath` too when their items can sit on another menu. With `--start`, `zappie` 0.1.8 or later checks that verify filled the path.
+
+## 0.3.0
+
+### Minor Changes
+
+- d1c8b1d: V14's `menuJsKbGz` now measures the same build the same way on every run. It counts the JavaScript the menu needs to start: the scripts and module preloads the server's document names, read when the document is parsed. Chunks the running page imports later, when it hydrates, in its effects, or on idle or intent, no longer count toward it. Before, they counted only when they happened to load inside the measurement window, so one build of a site measured anywhere from 163.5 to 194.6 KiB. The 200 KiB budget applies to the new figure. `menuJsLoadedKbGz` counts every script loaded once the network goes quiet, later imports included. V14 fails it past 350 KiB with `loaded_js_too_large`, and `findRegressions` compares it with the baseline, allowing 10% or 10 KiB of growth, whichever is larger. A site's `menuJsKbGz` drops on this release, so record a new baseline from its first run.
+- 5a2bce9: Verify can start somewhere other than `/`, for a site whose root is a directory of its menus. Set `verify.startPath` in `zappie.config.ts`, or pass `startPath` to `verify()` or `verifyProject()`. Every check that opened `/` as the menu opens that route instead. Set `verify.cataloguePath`, such as `/menu?menu={menu}`, when each menu has its own page: V03 then needs 95% of each menu's available items on that menu's page, instead of 95% of the branch's items on one page, and the checks that use an item open the page of a menu that holds it. A route must start with one `/` and contain no backslash. A site that sets neither runs as before. The report's `environment` records `startPath`, and `cataloguePath` when it is set. When `--store` is passed and `zappie.config.ts` exists but can't be read, the report's `warnings` says its verify settings were ignored. The `@zappie/sdk` site config and `zappie check` accept the `verify` settings, so a site that adds them needs `@zappie/sdk` 0.1.14 and `@zappie/check` 0.2.4 in the same change. This is a minor release, so a regression comparison doesn't compare these reports with reports from 0.2.
+
+### Patch Changes
+
+- Updated dependencies [5a2bce9]
+  - @zappie/sdk@0.1.14
+
 ## 0.2.9
 
 ### Patch Changes

package/README.md

@@ -24,6 +24,7 @@ With no URL, it builds your project, starts a local preview, runs the checks and
 | -------------------------------- | ------------------------------------------------------------------------------------- |
 | `--suite standard` or `extended` | The standard checks, or those plus the extended ones                                  |
 | `--only <checks>`                | Run only these checks: IDs such as `V01,V04`, or a set such as `quick`                |
+| `--start <path>`                 | The route verify opens as the menu. Default: `verify.startPath`, then `/`             |
 | `--report <file>`                | Save the full report as JSON                                                          |
 | `--json`                         | Print the report as JSON                                                              |
 | `--no-build`                     | Serve the project's existing build instead of building it again. Local mode only      |
@@ -33,6 +34,23 @@ With no URL, it builds your project, starts a local preview, runs the checks and
 
 The `quick` set is V01 to V07, V09, V10 and V13: the fast journey checks without V14 to V18. V18 stays outside build-blocking quick runs while security headers are report-only. The set is exported as `CHECK_SETS.quick`.
 
+### Where the storefront starts
+
+Verify opens `/` as the menu. A site whose root is something else, such as a directory of its menus, tells verify where to start in `zappie.config.ts`:
+
+```ts
+export default {
+  store: "your-store",
+  theme: "./theme.json",
+  verify: { startPath: "/menu", cataloguePath: "/menu?menu={menu}" },
+} satisfies ZappieSiteConfig;
+```
+
+- `startPath` is the route verify opens wherever it opened `/`. It must list items. `--start` wins over it. It can name a menu with `{menu}` and the branch with `{branch}`, as `cataloguePath` does. `{menu}` becomes the id of the first menu that holds items, in the order V03 judges menus, and `{branch}` the id of the default branch. Use it for a site whose menu page asks the shopper to choose a menu, such as `/menu?menu={menu}`. Without `cataloguePath`, V03 judges only that menu's items on the start page. The item checks still open the start page, so set `cataloguePath` too when the item they use can sit on another menu.
+- `cataloguePath` is the page of each menu: `{menu}` stands for the menu id and `{branch}`, when you use it, for the branch id. With it, V03 opens every menu's page and needs 95% of that menu's available items on it, instead of 95% of the branch's items on one page. V04, V05, V13, V16 and V17 open the page of a menu that holds the item they use. An item that sits in no category of any menu belongs to no menu's page, so V03 doesn't judge it.
+
+Without them, verify runs as before. The report's `environment` records `startPath`, and `cataloguePath` when it is set. A `startPath` with `{menu}` or `{branch}` is recorded filled in, and as written in `startPathTemplate`.
+
 ### Chromium
 
 Verify launches Playwright's own Chromium. To use another one, such as a system Chromium on a build machine, set `ZAPPIE_VERIFY_CHROMIUM` to its executable, or pass `browserExecutablePath` from code, which wins over the variable. The report's `environment.browser` then names the executable after the version.
@@ -63,7 +81,7 @@ const report = await verifyProject({
 console.log(printSummary(report));
 ```
 
-`verifyProject` also takes `build: false`, `port`, `only`, `runner`, `artifactsDir` and `browserExecutablePath`, the options behind the flags above. To verify a site you serve yourself, call `verify({ url, key, store, suite })` instead.
+`verifyProject` also takes `build: false`, `port`, `only`, `startPath`, `cataloguePath`, `runner`, `artifactsDir` and `browserExecutablePath`, the options behind the flags above. It reads `startPath` and `cataloguePath` from `zappie.config.ts` when you don't pass them. To verify a site you serve yourself, call `verify({ url, key, store, suite })` instead, with `startPath` and `cataloguePath` when the site sets them.
 
 The report's shape is the JSON Schema at `@zappie/verify/schema/verify-report.v1.json`.
 
@@ -75,6 +93,8 @@ They accept either way of building a journey where templates differ:
 
 - **Opening an item.** When the element marked `data-zappie-item` is a link or button, as on a card that is the link, the checks tap it. Otherwise they tap the first link inside the marked card, then the first link beside it.
 - **The cart control.** `data-zappie-cart-trigger`, or on a template that marks only its overlays, the visible `data-zappie-overlay-trigger="bag"` control.
+- **Item prices (V03).** A card shows the item's price in `format.money`. When `showsFromPrice` from `@zappie/sdk/pricing` is true, it can show the item's `startingPrice` instead, with or without "From".
+- **Required groups (V05).** A required option group is marked `data-zappie-option-group` and `data-zappie-required="true"`, as `OptionGroup` from `@zappie/foundations` marks single and multi-select groups. `aria-required="true"` on a single-select group also counts.
 - **Required options (V05).** Add can be disabled until every required choice is made. Or it can stay enabled: a tap then adds nothing, keeps the item open and marks the group with `aria-invalid` or `data-invalid` and a visible message, in the group, in its `aria-describedby` or `aria-errormessage` target, or in a `role="alert"` in the dialog.
 - **The quoted total (V06).** `data-zappie-total` in the bag, or, when the bag hands over to a checkout page through `data-zappie-cart-checkout` or a link to `/checkout`, on that page.
 - **A cart column (V13).** On a screen 768 px or wider, a cart the page always shows, with no visible cart control, is scanned with the page. V13 then skips opening it and the focus-return check for it. On a phone the cart control is still required.
@@ -92,6 +112,10 @@ V13 scans each page and overlay once its entry animations have finished, so a fa
 | `data-zappie-action="add-to-cart"` | Checks that the keyboard reaches an Add control. On a menu without quick add, it tabs to an item without required options, opens it with Enter and tabs to its Add. |
 | `data-zappie-checkout`             | Checks that the keyboard reaches the enabled checkout control on `/checkout`.                                                                                       |
 
+## Menu JavaScript
+
+V14 opens the menu on a phone profile and fails when `menuJsKbGz` passes 200 KiB gzipped. It counts the JavaScript the menu needs to start: the scripts and module preloads the server's document names. Chunks the running page imports later, when it hydrates, in its effects, or on idle or intent, don't count, so one build measures the same on every run. `menuJsLoadedKbGz` counts every script loaded once the network goes quiet, those later imports included. It fails V14 past 350 KiB, and `findRegressions` compares it with the baseline, so code moved out of the document into late imports still counts.
+
 ## Overlays
 
 Mark each control that opens an overlay with `data-zappie-overlay-trigger="bag"`, `"branch"`, `"search"` or `"item"`, and the overlay's `role="dialog"` element with `data-zappie-overlay` and the same name. V16 waits until the menu has hydrated and its idle work, such as overlay preloads, and the downloads that work starts are done, as they would be for a shopper who has read the menu. Then it taps each marked trigger on a phone profile with the CPU slowed four times, and fails when the overlay takes more than 100 ms to paint. Anything the tap itself loads counts toward that time. Each tap is a shopper's first open on a first visit: V16 opens every overlay in a new browser context, so the checks before it can't warm it. It opens each overlay up to three times, each in its own context, and judges the median, so one slow sample on a busy machine doesn't fail it. It stops after two samples on the same side of the budget, because the third can't change the median. The CPU slowdown is calibrated to the machine: before the checks run, verify scores the browser's CPU with a short fixed benchmark and sets the throttling rate so that V14, V15 and V16 see the same phone everywhere. A developer laptop gets about four times, a slower CI runner less, between one and eight times. The report's `environment` records `cpuBenchmark` and the `cpuThrottlingRate` the checks used. The report's `overlayOpenMs` holds the medians and `overlayOpenSamplesMs` holds every sample, in the order they ran. V17 opens each one from the keyboard, in a new browser context too, and checks that focus moves in, Tab stays in, the page behind doesn't scroll, Escape closes it and focus returns to the trigger. Both are skipped while the menu has no markers.
@@ -109,6 +133,8 @@ const regressions = findRegressions(baselineReport, report);
 
 A metric regresses when it grows by more than its relative threshold or its absolute floor, whichever is larger. The defaults are in `DEFAULT_REGRESSION_THRESHOLDS`. A metric the baseline records and the new report doesn't also counts as a regression.
 
+Compare only reports with the same `environment.startPath`, and `cataloguePath`. A report without `startPath` started at `/`. Metrics measured from another page don't compare.
+
 ## Docs
 
 - [CLI: zappie verify](https://docs.zappie.ai/cli/verify)

package/dist/index.d.ts

@@ -44,6 +44,18 @@ type VerifyReport = {
         cpuBenchmark?: number;
         /** The CPU throttling rate the timed checks ran at, calibrated from `cpuBenchmark`. */
         cpuThrottlingRate?: number;
+        /**
+         * The route the checks opened as the menu. Reports from before verify 0.3.0 have none, and
+         * started at `/`. Metrics measured from different start paths don't compare.
+         */
+        startPath?: string;
+        /**
+         * The start path as configured, with its `{menu}` or `{branch}`, when it had either.
+         * `startPath` is that path filled in.
+         */
+        startPathTemplate?: string;
+        /** The catalogue path V03 and the item checks used, when the run had one. */
+        cataloguePath?: string;
     };
     startedAt: string;
     durationMs: number;
@@ -54,8 +66,22 @@ type VerifyReport = {
         skipped: number;
     };
     checks: CheckResult[];
+    /** Findings about the run itself, such as settings verify could not read. */
+    warnings?: {
+        code: string;
+        message: string;
+    }[];
     metrics: {
+        /**
+         * Gzipped JavaScript the menu needs to start: the scripts and module preloads its server
+         * document names. Budgeted.
+         */
         menuJsKbGz?: number;
+        /**
+         * Every script loaded once the network goes quiet, idle and intent preloads included. Held to
+         * a loose 350 KiB budget and compared with a baseline, so code moved out of the document counts.
+         */
+        menuJsLoadedKbGz?: number;
         lcpMs?: number;
         cls?: number;
         productCardTappableMs?: number;
@@ -79,6 +105,27 @@ type VerifyOptions = {
     deploymentId?: string;
     /** Check IDs to run, such as `V01`, or a named set such as `quick`. All checks when unset. */
     only?: string[];
+    /**
+     * The route verify opens as the storefront's menu, such as `/menu`, for a site whose root is
+     * a directory rather than a menu. It must list items. Defaults to `/`. `{menu}` stands for
+     * the first menu that holds items, as V03 orders them, and `{branch}` for the default branch,
+     * such as `/menu?menu={menu}`. Without `cataloguePath`, V03 then judges only that menu's items
+     * on the start page, and the item checks still open the start page, so set `cataloguePath`
+     * too when the items they use can sit on another menu.
+     */
+    startPath?: string;
+    /**
+     * The page of each menu, with `{menu}` where the menu id goes and, optionally, `{branch}` for
+     * the branch id, such as `/menu?menu={menu}`. With it, V03 judges each menu on its own page,
+     * and a check that needs an item opens the page of a menu that holds it. Without it, V03
+     * judges every item of the branch on the start page.
+     */
+    cataloguePath?: string;
+    /** Findings about the run to record in the report's `warnings`. */
+    warnings?: {
+        code: string;
+        message: string;
+    }[];
     viewports?: Viewport[];
     artifactsDir?: string;
     timeoutMs?: number;
@@ -147,7 +194,7 @@ type RegressionThreshold = {
     /** Growth always allowed, in the metric's own unit, so small values don't trip on noise. */
     absolute: number;
 };
-type RegressionMetric = "menuJsKbGz" | "menuCssKbGz" | "lcpMs" | "cls" | "productCardTappableMs" | "productCardOpenMs" | "overlayOpenMs";
+type RegressionMetric = "menuJsKbGz" | "menuJsLoadedKbGz" | "menuCssKbGz" | "lcpMs" | "cls" | "productCardTappableMs" | "productCardOpenMs" | "overlayOpenMs";
 /**
  * A metric regresses when `head - base > max(absolute, relative × base)`. Sizes are
  * deterministic, so their allowance is small; timings carry runner noise, so theirs is wider.