@qartisan/playwright-loupe 0.0.1 → 0.0.3

package/CHANGELOG.md

@@ -4,7 +4,46 @@ All notable changes to this project are documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Before 1.0, minor versions may contain breaking changes. The JSON report has its own version (`schemaVersion`), see [`docs/schema/v1.md`](docs/schema/v1.md#versioning-policy).
 
-## [Unreleased]
+## [0.0.3] - 2026-10-16
+
+### Added
+
+- **HTML report application shell.** The renderer is rebuilt with Preact and `@preact/signals`, still one self-contained `index.html` that works from `file://` (both libraries are bundled into it: the package keeps zero runtime dependencies).
+  - Header with the run summary: status, counts per outcome, duration, branch, commit and environment. A partial run shows a banner with the reasons it was detected.
+  - Collapsible sidebar with a test tree: projects → files → describe blocks → tests, single-child levels merged, a status icon per test (passed, failed, flaky, skipped) and failure counts per node. Keyboard navigation (arrows, Home / End, Enter), the current test highlighted and revealed. Collapsed nodes are not rendered, so large suites stay responsive.
+  - Hash routes that work from `file://`: `#/` (overview), `#/errors`, `#/health`, `#/group/<signature>`, `#/test/<id>`. The URL is the single source of navigation state.
+  - Light and dark themes: the system theme by default, or the `theme` option; the viewer's choice is remembered in the browser when storage is available.
+  - The test page is a placeholder for now.
+- **Dashboard, in three views** reached from fixed sidebar entries. Widgets with nothing to show are hidden, except the verdict.
+  - **Overview** (`#/`): the verdict (failed, flaky or passed, a stacked bar of the outcomes with their counts, pass rate and duration; a run with run errors is never green), run errors when there are any, compact error categories (one bar per category, each opening that category in the Errors view, `#/errors/category/<name>`), and project cards (counts, pass rate, test time; a click reveals the project in the sidebar).
+  - **Pass rate**, one definition everywhere: passed / (total − skipped), labeled "pass rate", with a tooltip saying skipped tests are excluded.
+  - **Errors** (`#/errors`): run errors, every error group (most failed tests first, then most flaky), and the categories in detail, each expanding to its groups. Error groups are listed here only. The sidebar entry shows a warning marker and the count when the run has run errors. The former `#/run-errors` route redirects here.
+  - **Suite health** (`#/health`): one widget per dimension. Up to 8 values, one stacked bar per value (passed / failed / flaky / skipped) with the total; more values, a grid of tiles colored by worst status with its icon, problem values first, details on hover and keyboard focus, and a summary line (`38 values · 36 all passed · 2 with failures`). The exact table expands from every dimension widget. Tag hygiene: values outside the declared ones and values differing only by case (`API` / `api`) are flagged, never merged; a hint points to `values` when a dimension declares none. Then the ten slowest tests, the flaky tests with the error of their failing attempt, and the skipped tests with their reason.
+  - An error group row shows its category, its failed / flaky split, the first affected test (`+N others`) and the line that tells the error apart: the `Locator:` line or the HTTP response status rather than a generic `expect(locator).toBeVisible() failed`.
+- **Error group pages** (`#/group/<signature>`): category, signature, failed / flaky split, a full sample message and every affected test with the attempts where the error occurred. Groups are computed in the renderer from the existing report data.
+- **Visual identity.** New light and dark palettes, "Loupe" wordmark in the header, a "Loupe — a reporter for Playwright" footer with the version, and DM Sans (regular and semibold, latin) embedded in the HTML so the report keeps working offline. The font is under the SIL Open Font License, shipped in `licenses/DM-Sans-OFL.txt`. Contrast (4.5:1 for text, 3:1 for borders and icons) is checked by the test suite in both themes. The HTML report is about 52 kB gzip, 29 kB of which are the fonts.
+- **Category colors from a palette.** `categories.rules[].color` now also accepts a palette name: `blue`, `cyan`, `teal`, `violet`, `magenta`, `orange`, `brown`, `gold` (see [Category palette](docs/config.md#category-palette)). An unknown name is a config error listing the valid names. `#rrggbb` colors still work; the HTML report derives light and dark shades from them that meet the contrast rules.
+
+### Changed
+
+- **Error signatures change again: failures are grouped differently from 0.0.2.** The signature now uses only the stable part of Playwright messages. A strict mode violation loses its element list and count (`resolved to <n> elements`), so a test and its retry get the same signature when the page gained an item between attempts (`.first()` vs `.nth(1)`, `li:nth-child(13)` vs `14`). A locator assertion keeps only its `Error:` lines, `Locator:`, and `Expected…:` / `Received…:` lines: the timeout and screenshot pixel counts no longer count. Different locators, actions or expected values still give different signatures. The call log, stack and code frame were already left out. The JSON shape is unchanged: `schemaVersion` stays `1`. See [Error signature](docs/schema/v1.md#error-signature).
+- **Built-in category colors changed** to the new palette, without reds or greens and with no two alike (they were near-duplicate indigos, and Assertion was red like a failure): Ambiguous locator violet `#8656b7`, Locator not found magenta `#ad4681`, Rate limited teal `#04837c`, Server error orange `#ab5803`, Network cyan `#0a7da1`, Assertion blue `#466cc8`, Timeout brown `#745f43`, Uncategorized `#6b6b6b`. New values, same shape: `schemaVersion` stays `1`. `CategoryDefinition.color` is now documented as a reference color that renderers may adapt per theme.
+- **Categories without a configured color** take, in config rule order, the first palette color no other category of the report uses (gold, never used by a built-in category, comes last). Colors used to be assigned over every rule, so with the built-in categories present a configured category could share a color with one shown next to it. The same config with the same categories present always gives the same colors; set `color` for a color that never changes.
+- **New built-in error category: Ambiguous locator**, evaluated first, before "Locator not found". It matches Playwright's `strict mode violation` (a locator resolving to several elements), which was "Uncategorized" or "Assertion" before. See [Error categories](docs/schema/v1.md#error-categories).
+
+### Fixed
+
+- **Run scope with `--no-deps`.** Playwright keeps `project.dependencies` in the config reporters get, so with `--project x --no-deps` the dependencies of `x` counted as running and the run could be reported `full` while a configured project was left out. They no longer count: the run is `partial` with the `project-filter` reason.
+- **Run scope with `default: false` projects** (Playwright 1.64+). The reference for a full run is now the set of default projects: a plain run is `full` even though the non-default projects did not run, and `--project nightly` alone is `partial` because the default projects are missing. See [Default projects](docs/schema/v1.md#default-projects).
+- **Run scope with several `--project` values.** `--project a b` selects two projects, as in Playwright; `b` was taken for a file filter and the run reported a `file-filter` reason.
+
+## [0.0.2] - 2026-10-15
+
+### Changed
+
+- **Error signatures change: failures are grouped differently from 0.0.1.** Normalization now also replaces e-mail addresses with `<email>` and cuid-like and nanoid-like identifiers (20+ characters mixing letters and digits) with `<id>`, so the same failure for different users or records gets one signature. A signature computed by 0.0.1 for a message containing such values no longer matches. The JSON shape is unchanged: `schemaVersion` stays `1`. See [Error signature](docs/schema/v1.md#error-signature).
+- **New built-in error categories**, evaluated after "Locator not found" and before "Network", "Assertion" and "Timeout": **Rate limited** (`Too Many Requests`, `rate limit`, a 429 in an HTTP context such as `status 429`, `HTTP 429` or `Response status: 429`) and **Server error** (a 500–599 status in the same contexts, or with its reason phrase such as `502 Bad Gateway`). Errors from `failOnStatusCode` and failed `expect(response).toBeOK()` that were "Uncategorized" or "Assertion" now land in these categories. A bare number (`429ms`, `Received: 500`) does not match. See [Error categories](docs/schema/v1.md#error-categories).
+- The package description now lists only what exists today.
 
 ## [0.0.1] - 2026-10-13
 

package/README.md

@@ -1,8 +1,10 @@
 # playwright-loupe
-Loupe — a clear, insightful HTML reporter for Playwright. Project overview, history, flakiness and error categories, using native Playwright APIs only.
+Loupe — an HTML reporter for Playwright: a dashboard that shows what failed and why, error groups by normalized signature, tests grouped by project, file and describe block, and dimensions from tags and annotations. Native Playwright APIs only.
 
 > **Status: work in progress — the JSON schema will change before 1.0.**
 
+![Loupe overview](https://raw.githubusercontent.com/qartisan/playwright-loupe/main/docs/images/overview.png)
+
 Loupe is a reporter for Playwright, not affiliated with Microsoft.
 
 ## Installation
@@ -29,10 +31,24 @@ export default defineConfig({
 
 After a run, open `loupe-report/index.html`. The raw data is in `loupe-report/report.json`, with attachments in `loupe-report/data/`. Its format is documented in [`docs/schema/v1.md`](docs/schema/v1.md), with a [JSON Schema](docs/schema/report-v1.schema.json) and TypeScript types (`import type { LoupeReport } from '@qartisan/playwright-loupe'`).
 
-Loupe requires `@playwright/test` 1.52 or later.
-
 A run that broke outside its tests (a spec file that fails to import, a failing global setup) is reported in `run.errors`, even when no test ran.
 
+## The report
+
+The report is a single self-contained HTML file. It works offline, straight from `file://`, in light and dark themes.
+
+- **Overview:** verdict with pass rate, run errors first when the run broke outside its tests, error categories, project cards.
+- **Errors:** every error group, built from normalized error signatures, with the failed / flaky split and links to the affected tests.
+- **Suite health:** dimensions from tags and annotations (with hints for unknown or inconsistent values), slowest, skipped and flaky tests.
+- **Sidebar:** projects, files, `describe` blocks and tests, with status icons, failure counts and keyboard navigation.
+
+### Coming next
+
+- Search and filters (name, suite, status, tag), kept in the URL.
+- Test detail: error, steps, and attachments per attempt (screenshots, video, trace), copyable rerun command and AI prompt.
+
+History and flakiness tracking are planned for a later phase; the JSON model is already prepared for them.
+
 ## Configuration
 
 Put a `loupe.config.ts` next to `playwright.config.ts`:
@@ -44,7 +60,7 @@ import { defineConfig } from '@qartisan/playwright-loupe';
 export default defineConfig({
   title: 'My suite',
   categories: {
-    rules: [{ name: 'Infra', match: [/ECONNRESET/, /ECONNREFUSED/] }],
+    rules: [{ name: 'Infra', match: [/ECONNRESET/, /ECONNREFUSED/], color: 'teal' }],
   },
   dimensions: {
     definitions: [
@@ -78,7 +94,7 @@ Node.js 22.12 is the minimum because ES module config files are loaded with `req
 | `npm run typecheck` | Type-checks the core, tests, fixtures and renderer. |
 | `npm test` | Builds, then runs the Vitest unit tests and the integration tests (fixture suite, and a sharded run merged with `merge-reports`). |
 | `npm run fixtures` | Builds, then runs the fixture suite into `loupe-report/`. Its failures are intentional, see [`fixtures/README.md`](fixtures/README.md). |
-| `npm run sample:update` | Runs the fixture suite and copies its report to `src/renderer/dev/sample.json`. |
-| `npm run dev:renderer` | Vite dev server for the renderer, fed with `src/renderer/dev/sample.json`, with hot reload. |
+| `npm run sample:update` | Builds, then runs the fixture suite with `fixtures/sample.config.ts` and writes its report to `src/renderer/dev/sample.json`: failures, flaky and skipped tests, a partial run and a run error. |
+| `npm run dev:renderer` | Vite dev server for the renderer, fed with `src/renderer/dev/sample.json`, with hot reload. No Playwright run needed for UI work. |
 
 Tests that drive a browser need Chromium: `npx playwright install chromium`.

package/dist/config/resolve.js

@@ -1,4 +1,5 @@
 import path from 'node:path';
+import { paletteReference } from '../shared/palette.js';
 import { validateConfig } from './validate.js';
 export const DEFAULT_OUTPUT_FOLDER = 'loupe-report';
 export const DEFAULT_TITLE = 'Loupe report';
@@ -68,7 +69,8 @@ function resolveRules(config) {
         name: rule.name,
         match: (Array.isArray(rule.match) ? rule.match : [rule.match]).map(stateless),
         on: rule.on ?? 'all',
-        color: rule.color ?? null,
+        // A palette name becomes its reference color: the report only carries #rrggbb colors.
+        color: rule.color === undefined ? null : rule.color.startsWith('#') ? rule.color : paletteReference(rule.color),
     }));
 }
 function resolveDimensions(config) {

package/dist/config/types.d.ts

@@ -17,7 +17,11 @@ export interface CategoryRuleConfig {
     match: RegExp | RegExp[];
     /** What the patterns are tested against. Default: `'all'`. */
     on?: CategoryTarget;
-    /** `#rrggbb`. Default: a color from Loupe's palette. */
+    /**
+     * `#rrggbb`, or the name of a palette color: `blue`, `cyan`, `teal`,
+     * `violet`, `magenta`, `orange`, `brown`, `gold`. Default: the first
+     * palette color no other category of the report uses, in rule order.
+     */
     color?: string;
 }
 export interface CategoriesConfig {
@@ -75,6 +79,7 @@ export interface ResolvedCategoryRule {
     name: string;
     match: RegExp[];
     on: CategoryTarget;
+    /** `#rrggbb` (palette names are resolved to their reference color); null when not configured. */
     color: string | null;
 }
 export type ResolvedDimension = {

package/dist/config/validate.js

@@ -1,3 +1,4 @@
+import { PALETTE_NAMES } from '../shared/palette.js';
 import { CATEGORY_MODES, CATEGORY_TARGETS, SCREENSHOT_VIEWS, THEMES } from './types.js';
 const COLOR = /^#[0-9a-fA-F]{6}$/;
 /** What a tag prefix cannot contain: `@layer:api` has the prefix `layer`. */
@@ -102,10 +103,16 @@ function boolean(ctx) {
             fail(ctx, path, 'a boolean', value, errors);
     };
 }
+/** A `#rrggbb` color or the name of a palette color (`teal`). */
 function color(ctx) {
     return (value, path, errors) => {
-        if (typeof value !== 'string' || !COLOR.test(value))
-            fail(ctx, path, 'a #rrggbb color', value, errors);
+        if (typeof value === 'string' && (COLOR.test(value) || PALETTE_NAMES.includes(value)))
+            return;
+        const expected = `a #rrggbb color or a palette color (${PALETTE_NAMES.map((name) => `"${name}"`).join(', ')})`;
+        const suggestion = typeof value === 'string' && !value.startsWith('#') ? suggestKey(value, PALETTE_NAMES) : null;
+        fail(ctx, path, expected, value, errors);
+        if (suggestion)
+            errors[errors.length - 1] += `. Did you mean "${suggestion}"?`;
     };
 }
 function regexps(ctx) {

package/dist/core/build.js

@@ -87,6 +87,26 @@ function toAnnotation(annotation) {
 function isDefaultGrep(grep) {
     return grep instanceof RegExp && grep.source === '.*' && grep.flags === '';
 }
+/**
+ * Names of the projects with `default: false` (Playwright 1.64+). Only
+ * Playwright's internal config has the flag, under a private symbol; undefined
+ * when it is missing (older versions, merge-reports) or has another shape.
+ */
+function nonDefaultProjects(config) {
+    const symbol = Object.getOwnPropertySymbols(config).find((candidate) => candidate.description === 'configInternalSymbol');
+    const internal = symbol ? config[symbol] : undefined;
+    const projects = internal?.projects;
+    if (!Array.isArray(projects))
+        return undefined;
+    const names = [];
+    for (const project of projects) {
+        if (typeof project.default !== 'boolean' || typeof project.project?.name !== 'string')
+            return undefined;
+        if (!project.default)
+            names.push(project.project.name);
+    }
+    return names;
+}
 /** `test.only` / `describe.only` leave Playwright's internal `_only` flag on the node or an ancestor. */
 function isFocused(test) {
     for (let node = test; node; node = node.parent) {
@@ -263,6 +283,7 @@ export async function buildReport(config, suite, result, options) {
             teardown: project.teardown,
         })),
         filteredProjects: extra.filteredProjects?.map((project) => project.name),
+        nonDefaultProjects: nonDefaultProjects(config),
         runProjects: projects.map((project) => project.name),
         only: suite.allTests().some(isFocused),
     });

package/dist/core/categories.d.ts

@@ -15,17 +15,16 @@ export interface ErrorText {
 type Target = Exclude<CategoryTarget, 'all'>;
 export interface CategoryRule {
     name: string;
-    /** Null: a color is picked from the palette. */
+    /** `#rrggbb`. Null: a color is picked from the palette. */
     color: string | null;
     match: RegExp[];
     targets: readonly Target[];
 }
-export declare const UNCATEGORIZED_COLOR = "#6b7280";
+export declare const UNCATEGORIZED_COLOR: string;
 export declare const BUILT_IN_RULES: readonly CategoryRule[];
 /**
- * Colors for categories without one, assigned in rule order. Distinct from the
- * built-in and uncategorized colors; colors already taken by another category
- * are skipped, and the palette wraps around when exhausted.
+ * Colors for categories without one: the reference colors of the palette, in
+ * palette order (see `src/shared/palette.ts`).
  */
 export declare const PALETTE: readonly string[];
 export interface CategoriesOptions {

package/dist/core/categories.js

@@ -1,18 +1,54 @@
+import { NEUTRAL_ENTRY, PALETTE_ENTRIES, paletteReference } from '../shared/palette.js';
 const ALL_TARGETS = ['message', 'stack', 'step'];
-export const UNCATEGORIZED_COLOR = '#6b7280';
+export const UNCATEGORIZED_COLOR = NEUTRAL_ENTRY.reference;
+/**
+ * An HTTP status in context: `status 429`, `status code: 429`, `"status": 429`,
+ * `statusCode=429`, `HTTP 429`, `HTTP/1.1 429`, `Response status: 429`, or
+ * `← 429`, the response line of Playwright's fetch log.
+ */
+function httpStatus(code) {
+    return new RegExp(`(?:\\b(?:status(?:[ _]?code)?["']?\\s*[:=]?|HTTP(?:/\\d(?:\\.\\d)?)?:?)|\u2190)\\s*${code}(?!\\w|\\.\\d)`, 'i');
+}
 export const BUILT_IN_RULES = [
+    {
+        // Before "Locator not found", "Assertion" and "Timeout": a locator matching
+        // several elements fails strict mode, through an action or an expect.
+        name: 'Ambiguous locator',
+        color: paletteReference('violet'),
+        match: [/\bstrict mode violation\b/i],
+        targets: ['message', 'stack'],
+    },
     {
         // Before "Assertion" and "Timeout": a missing element surfaces as a failed
         // or timed-out expect/action, and "not found" is the useful diagnosis.
         name: 'Locator not found',
-        color: '#7c3aed',
+        color: paletteReference('magenta'),
         match: [/element\(s\) not found/i, /resolved to 0 elements/i],
         targets: ['message', 'stack'],
     },
+    {
+        // Before "Network", "Assertion" and "Timeout": the status is the diagnosis,
+        // whether it surfaces through `failOnStatusCode`, `toBeOK()` or a thrown
+        // error. A bare 429 is not enough (`429ms`, `Received: 429`): it needs an
+        // HTTP context.
+        name: 'Rate limited',
+        color: paletteReference('teal'),
+        match: [/\bToo Many Requests\b/i, /\brate[- ]?limit/i, httpStatus('429')],
+        targets: ['message', 'stack'],
+    },
+    {
+        name: 'Server error',
+        color: paletteReference('orange'),
+        match: [
+            httpStatus('5\\d\\d'),
+            /\b5\d\d (?:Internal Server Error|Not Implemented|Bad Gateway|Service Unavailable|Gateway Time-?out|HTTP Version Not Supported)\b/i,
+        ],
+        targets: ['message', 'stack'],
+    },
     {
         // Before "Timeout" so connection timeouts (ETIMEDOUT) count as network.
         name: 'Network',
-        color: '#0891b2',
+        color: paletteReference('cyan'),
         match: [
             /\bnet::ERR_[A-Z_]+/,
             /\bNS_ERROR_[A-Z_]+/,
@@ -26,34 +62,22 @@ export const BUILT_IN_RULES = [
         // Before "Timeout": a retrying expect that never matched reports its own
         // timeout but is an assertion failure.
         name: 'Assertion',
-        color: '#dc2626',
+        color: paletteReference('blue'),
         match: [/\bexpect\(/, /^\s*Expected\b.*:/m, /\bAssertionError\b/],
         targets: ['message', 'stack'],
     },
     {
         name: 'Timeout',
-        color: '#d97706',
+        color: paletteReference('brown'),
         match: [/\bTest timeout of \d+ms exceeded/i, /\bTimeout \d+ms exceeded/i, /\btimed out\b/i],
         targets: ['message', 'stack'],
     },
 ];
 /**
- * Colors for categories without one, assigned in rule order. Distinct from the
- * built-in and uncategorized colors; colors already taken by another category
- * are skipped, and the palette wraps around when exhausted.
+ * Colors for categories without one: the reference colors of the palette, in
+ * palette order (see `src/shared/palette.ts`).
  */
-export const PALETTE = [
-    '#2563eb',
-    '#16a34a',
-    '#db2777',
-    '#ca8a04',
-    '#0d9488',
-    '#9333ea',
-    '#ea580c',
-    '#4f46e5',
-    '#65a30d',
-    '#be123c',
-];
+export const PALETTE = PALETTE_ENTRIES.map((entry) => entry.reference);
 function matches(rule, error) {
     return rule.targets.some((target) => {
         const text = error[target];
@@ -80,26 +104,43 @@ export function createCategorizer(options) {
         if (color)
             colors.set(name, color);
     }
-    const taken = new Set([...colors.values(), UNCATEGORIZED_COLOR].map((color) => color.toLowerCase()));
-    let next = 0;
-    for (const name of names) {
-        if (colors.has(name))
-            continue;
-        const free = PALETTE.slice(next).concat(PALETTE.slice(0, next)).find((color) => !taken.has(color));
-        const color = free ?? PALETTE[next % PALETTE.length];
-        next = (PALETTE.indexOf(color) + 1) % PALETTE.length;
-        taken.add(color);
-        colors.set(name, color);
-    }
     const builtInNames = new Set(builtIns.map((rule) => rule.name));
+    /**
+     * Colors of the categories in `used` (names in rule order). Explicit colors
+     * (configured or built-in) never change. Each category without one takes,
+     * in config rule order, the first palette color that no category of the
+     * report uses, in palette order (gold last). Once every color is taken,
+     * colors repeat in palette order. The result only depends on the config and
+     * on which categories the report contains.
+     */
+    function assignColors(used) {
+        const assigned = new Map();
+        const taken = new Set([UNCATEGORIZED_COLOR]);
+        for (const name of used) {
+            const color = colors.get(name);
+            if (color) {
+                assigned.set(name, color);
+                taken.add(color.toLowerCase());
+            }
+        }
+        let repeated = 0;
+        for (const name of used) {
+            if (assigned.has(name))
+                continue;
+            const color = PALETTE.find((candidate) => !taken.has(candidate)) ?? PALETTE[repeated++ % PALETTE.length];
+            taken.add(color);
+            assigned.set(name, color);
+        }
+        return assigned;
+    }
     return {
         categorize(error) {
             return rules.find((rule) => matches(rule, error))?.name ?? options.uncategorized;
         },
         definitions(used) {
-            const definitions = names
-                .filter((name) => used.has(name))
-                .map((name) => ({ name, color: colors.get(name), builtIn: builtInNames.has(name) }));
+            const usedNames = names.filter((name) => used.has(name));
+            const assigned = assignColors(usedNames);
+            const definitions = usedNames.map((name) => ({ name, color: assigned.get(name), builtIn: builtInNames.has(name) }));
             if (used.has(options.uncategorized)) {
                 definitions.push({ name: options.uncategorized, color: UNCATEGORIZED_COLOR, builtIn: true });
             }

package/dist/core/scope.d.ts

@@ -8,8 +8,13 @@ import type { RunScope } from '../schema/v1.js';
  *   command line (`config.argv` from 1.61, `process.argv` before).
  * - `--project` is visible as `config.filteredProjects` from 1.64; earlier, the
  *   command line plus the projects present in the suite are used.
+ * - `--no-deps` leaves `project.dependencies` / `teardown` untouched in the
+ *   config reporters get: only the command line tells that they did not run.
  * - A `grep` / `grepInvert` in the config file is visible in the config.
  * - `test.only` is only visible through Playwright's internal `_only` flag.
+ * - `default: false` on a project (1.64+) is only visible through Playwright's
+ *   internal config; a plain run already leaves those projects out of
+ *   `config.filteredProjects`.
  *
  * Limits are documented in docs/schema/v1.md ("Run scope").
  */
@@ -38,6 +43,11 @@ export interface ScopeInput {
     projects: readonly ProjectRef[];
     /** `config.filteredProjects` names when Playwright exposes them (1.64+); empty under merge-reports. */
     filteredProjects: readonly string[] | undefined;
+    /**
+     * Names of the projects with `default: false` (Playwright 1.64+), read from
+     * Playwright's internal config; undefined when it is not available.
+     */
+    nonDefaultProjects: readonly string[] | undefined;
     /** Names of the projects that have a suite in this run. */
     runProjects: readonly string[];
     /** A `test.only` / `describe.only` is in effect. */

package/dist/core/scope.js

@@ -7,8 +7,13 @@
  *   command line (`config.argv` from 1.61, `process.argv` before).
  * - `--project` is visible as `config.filteredProjects` from 1.64; earlier, the
  *   command line plus the projects present in the suite are used.
+ * - `--no-deps` leaves `project.dependencies` / `teardown` untouched in the
+ *   config reporters get: only the command line tells that they did not run.
  * - A `grep` / `grepInvert` in the config file is visible in the config.
  * - `test.only` is only visible through Playwright's internal `_only` flag.
+ * - `default: false` on a project (1.64+) is only visible through Playwright's
+ *   internal config; a plain run already leaves those projects out of
+ *   `config.filteredProjects`.
  *
  * Limits are documented in docs/schema/v1.md ("Run scope").
  */
@@ -49,8 +54,17 @@ const VALUE_OPTIONS = new Set([
     '--ui-host',
     '--ui-port',
 ]);
+/**
+ * Variadic options (`--project <name...>`): like commander, they take every
+ * following token up to the next one starting with '-'. `--project=a` takes one.
+ */
+const VARIADIC_OPTIONS = new Set(['--project']);
 /** Options with an optional value: like commander, they take the next token unless it starts with '-'. */
 const OPTIONAL_VALUE_OPTIONS = new Set(['--only-changed', '-u', '--update-snapshots']);
+/** What commander takes for an option rather than a value: '-' followed by anything. */
+function isOptionLike(arg) {
+    return arg.length > 1 && arg.startsWith('-');
+}
 /** Parses `playwright test` arguments from a full argv; null when the command is not `test`. */
 export function parseTestCommand(argv) {
     const args = argv.slice(2);
@@ -78,9 +92,13 @@ export function parseTestCommand(argv) {
         command.options.add(name);
         if (name !== arg)
             continue;
-        if (VALUE_OPTIONS.has(name))
+        if (VARIADIC_OPTIONS.has(name)) {
+            while (args[i + 1] !== undefined && !isOptionLike(args[i + 1]))
+                i++;
+        }
+        else if (VALUE_OPTIONS.has(name))
             i++;
-        else if (OPTIONAL_VALUE_OPTIONS.has(name) && args[i + 1] !== undefined && !args[i + 1]?.startsWith('-'))
+        else if (OPTIONAL_VALUE_OPTIONS.has(name) && args[i + 1] !== undefined && !isOptionLike(args[i + 1]))
             i++;
     }
     return command;
@@ -104,12 +122,24 @@ function withDependencies(selected, projects) {
     }
     return result;
 }
+/**
+ * The reference for a full run is the set of default projects: a project with
+ * `default: false` only runs when selected with --project, and its absence
+ * never makes a run partial.
+ */
 function hasProjectFilter(input, command) {
     const all = input.projects.map((project) => project.name);
     // merge-reports exposes an empty list: a real run always selects at least one project.
     if (input.filteredProjects && input.filteredProjects.length > 0) {
-        const running = withDependencies(input.filteredProjects, input.projects);
-        return all.some((name) => !running.has(name));
+        // Without the internal flags, a plain `playwright test` already ran every default project.
+        if (!input.nonDefaultProjects && command && !command.options.has('--project'))
+            return false;
+        // With --no-deps, dependencies and teardowns of the selected projects do not run.
+        const running = command?.options.has('--no-deps')
+            ? new Set(input.filteredProjects)
+            : withDependencies(input.filteredProjects, input.projects);
+        const nonDefault = new Set(input.nonDefaultProjects ?? []);
+        return all.some((name) => !nonDefault.has(name) && !running.has(name));
     }
     if (!command?.options.has('--project'))
         return false;

package/dist/core/signature.d.ts

@@ -7,8 +7,9 @@ export interface NormalizeOptions {
 }
 /**
  * Normalizes an error message so the same failure produces the same text
- * across tests and runs: ANSI codes and the call log are removed, volatile
- * values are replaced with placeholders and whitespace is collapsed.
+ * across tests, retries and runs: ANSI codes and the call log are removed,
+ * only the stable part of Playwright messages is kept, volatile values are
+ * replaced with placeholders and whitespace is collapsed.
  */
 export declare function normalizeErrorMessage(message: string, options?: NormalizeOptions): string;
 /** First 12 hex chars of sha256 of the normalized message. Independent of the test. */

package/dist/core/signature.js

@@ -6,6 +6,34 @@ export function stripAnsi(text) {
 }
 /** Playwright appends the action log after a line starting with "Call log:". */
 const CALL_LOG = /^[ \t]*Call log:/m;
+/**
+ * A strict mode violation lists the matching elements, one per line:
+ * `    1) <li>Order 13</li> aka locator('li').first()`. Their positions and
+ * content change when the page gains an item, so the list is dropped and the
+ * count becomes `<n>`. Playwright ends a long list with a `...` line.
+ */
+const STRICT_MODE = /\bstrict mode violation\b/;
+const STRICT_MODE_COUNT = /(\bstrict mode violation:.*?\bresolved to )\d+( elements?\b)/g;
+const ENUMERATED_ELEMENT = /^[ \t]*(?:\d+\) <.*|\.\.\.)[ \t]*$/;
+/**
+ * Lines of a locator assertion that identify the failure: the `Error:` lines
+ * (the matcher line, and `Error: element(s) not found` or a strict mode
+ * violation under it), a matcher line without `Error:` (after a custom
+ * message), `Locator:`, and `Expected…:` / `Received…:`. The rest (timeout,
+ * screenshot pixel counts, diff bodies, element lists) varies between runs.
+ */
+const LOCATOR_LINE = /^[ \t]*Locator:/m;
+const STABLE_LINE = /^[ \t]*(?:Error:|Locator:|(?:Expected|Received)\b[^:\n]*:|expect(?:\.\w+)*\()/;
+/** The stable part of a Playwright message (the call log is already cut). See "Error signature" in docs/schema/v1.md. */
+function stablePart(text) {
+    let lines = text.split('\n');
+    if (STRICT_MODE.test(text)) {
+        lines = lines.filter((line) => !ENUMERATED_ELEMENT.test(line)).map((line) => line.replace(STRICT_MODE_COUNT, '$1<n>$2'));
+    }
+    if (LOCATOR_LINE.test(text))
+        lines = lines.filter((line) => STABLE_LINE.test(line));
+    return lines.join('\n');
+}
 /**
  * `merge-reports` prefixes the run errors of a blob whose config has a `tag`
  * with `(<tags>) `, tags joined by spaces. Config tags always start with `@`.
@@ -16,13 +44,17 @@ export function stripMergeTagPrefix(text) {
     return text.replace(MERGE_TAG_PREFIX, '');
 }
 /**
- * Ordered replacements. The order matters: URLs before paths (a URL contains
- * slashes), paths before hex and numbers (so a path is replaced as a whole),
- * hex before numbers. See "Error signature" in docs/schema/v1.md.
+ * Ordered replacements. The order matters: URLs before e-mails (a URL may
+ * contain `user@host`) and paths (a URL contains slashes), e-mails and paths
+ * before identifiers, identifiers before durations, hex and numbers (so each
+ * is replaced as a whole), hex before numbers. See "Error signature" in
+ * docs/schema/v1.md.
  */
 const REPLACEMENTS = [
     [/\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b/gi, '<uuid>'],
     [/\b[a-z][a-z0-9+.-]*:\/\/[^\s'"`<>()[\]{}]*[^\s'"`<>()[\]{}.,;:!?]/gi, '<url>'],
+    // Local part, then a domain with at least one dot and an alphabetic TLD.
+    [/(?<![\w.%+-])[\w.%+-]+@[a-z0-9](?:[a-z0-9-]*[a-z0-9])?(?:\.[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)*\.[a-z]{2,}(?![\w-])/gi, '<email>'],
     // ISO 8601 date with optional time and zone.
     [/\b\d{4}-\d{2}-\d{2}(?:[T ]\d{2}:\d{2}(?::\d{2}(?:[.,]\d+)?)?(?:Z|[+-]\d{2}:?\d{2})?)?(?![\w:])/g, '<date>'],
     // Time of day.
@@ -32,6 +64,13 @@ const REPLACEMENTS = [
     // Absolute POSIX paths with at least two segments, and Windows drive paths.
     [/(?<![\w.~:/\\-])\/(?:[\w.@+~-]+\/)+[\w.@+~-]+\/?/g, '<path>'],
     [/\b[A-Za-z]:[\\/](?:[^\\/\s'"`:*?<>|]+[\\/])*[^\\/\s'"`:*?<>|]+/g, '<path>'],
+    // nanoid-like: 20+ chars of [A-Za-z0-9_-] with a lowercase letter, an
+    // uppercase letter and a digit. Mixed case keeps kebab-case and snake_case
+    // names out; a digit keeps camelCase words out.
+    [/(?<![\w-])(?=[\w-]*[a-z])(?=[\w-]*[A-Z])(?=[\w-]*\d)[\w-]{20,}(?![\w-])/g, '<id>'],
+    // cuid-like: 20+ lowercase letters and digits, with a digit and a letter
+    // beyond a-f (pure hex strings stay <hex>).
+    [/(?<![0-9A-Za-z])(?=[a-z0-9]*\d)(?=[a-z0-9]*[g-z])[a-z0-9]{20,}(?![0-9A-Za-z])/g, '<id>'],
     [/\b\d+(?:\.\d+)?\s?(?:ms|milliseconds?|s|secs?|seconds?|m|mins?|minutes?|h|hours?)\b/gi, '<duration>'],
     // 8+ hex chars with at least one letter (pure digit runs fall through to <n>).
     [/(?<![0-9A-Za-z])(?=[0-9a-f]*[a-f])[0-9a-f]{8,}(?![0-9A-Za-z])/gi, '<hex>'],
@@ -39,8 +78,9 @@ const REPLACEMENTS = [
 ];
 /**
  * Normalizes an error message so the same failure produces the same text
- * across tests and runs: ANSI codes and the call log are removed, volatile
- * values are replaced with placeholders and whitespace is collapsed.
+ * across tests, retries and runs: ANSI codes and the call log are removed,
+ * only the stable part of Playwright messages is kept, volatile values are
+ * replaced with placeholders and whitespace is collapsed.
  */
 export function normalizeErrorMessage(message, options = {}) {
     let text = stripAnsi(message);
@@ -49,6 +89,7 @@ export function normalizeErrorMessage(message, options = {}) {
     const callLog = CALL_LOG.exec(text);
     if (callLog)
         text = text.slice(0, callLog.index);
+    text = stablePart(text);
     for (const [pattern, placeholder] of REPLACEMENTS) {
         text = text.replace(pattern, placeholder);
     }