@qartisan/playwright-loupe 0.1.1 → 0.1.3

package/CHANGELOG.md

@@ -4,6 +4,42 @@ All notable changes to this project are documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Before 1.0, minor versions may contain breaking changes. The JSON report has its own version (`schemaVersion`), see [`docs/schema/v1.md`](docs/schema/v1.md#versioning-policy).
 
+## [0.1.3] - 2026-10-21
+
+### Added
+
+- **Common failure points.** A new Overview card, after the run errors and before the error categories, lists the functions of your code that appear in the stack of several failures, in different spec files and error groups: when one broken fixture or helper fails many tests with different messages, it shows the function they share. A lead, not a diagnosis. Each point gives its function, file and the lines seen, the functions folded under it ("via …"), its tests, files and error groups, the failed / flaky split and the categories involved, and opens the matching tests. Thresholds: 5 tests (a test in several projects counts once), 3 spec files, 2 error groups; 5 points at most. Spec files, anonymous functions, `node_modules`, Node and Playwright internals are left out; helpers outside the root dir are kept, with `../` paths. See [Common failure points](README.md#common-failure-points).
+- **Failure point filter** in the Tests view and the sidebar, after the error category (`point` in the URL, `#` encoded as `%23`).
+- **Error groups only seen after a test timeout are folded.** When the test timeout stops a test, the pending action fails too, as a second error of the attempt (`Request context disposed`, `Target page, context or browser has been closed`…). A group whose every error is such a secondary error (after `Test timeout of <n>ms exceeded.`, or the fixture and hook variants of the test timeout) now sits in a collapsed disclosure at the end of the Errors view's list and of each category, and group counts leave it out with a mention (`22 error groups · 18 more only seen after a test timeout`). Test counts and test categories are unchanged; `n` / `p` walk these groups last; their pages stay reachable, and the test detail labels the error "Raised after the test timeout". See [Groups only seen after a test timeout](README.md#groups-only-seen-after-a-test-timeout).
+- **Why a test was skipped.** The reason given in the test code (`test.skip(condition, 'reason')`, `test.fixme(…)`, `testInfo.skip(…)`, in a test, a `describe` or a hook) now shows as `Skipped: <reason>` or `Fixme: <reason>` in Suite health, on a muted line under the title in the Tests view (one line, the full text in the tooltip and the accessible name), and next to the outcome in the test detail header. Without a reason: `Skipped (no reason given)` / `Fixme (no reason given)`. A test Playwright skipped without any skip annotation (a failed dependency project or earlier test of a serial group, `--max-failures`, a stopped run) shows `Not run`, with a sentence listing these usual causes, never asserting one. The search (`q`) also matches the reason's text. Read from the existing annotations: no change to the JSON. See [Skip reasons](README.md#skip-reasons).
+- `npm run perf:sample` writes a third dataset, `cascade`, with realistic stacks through invented helpers (some outside the test dir) and 50 spec files, and also times the opening of the Overview.
+
+### Changed
+
+- **Error signatures of `toBeOK()` keep the response status.** For `expect(response).toBeOK()` (and `not.toBeOK()`), the status is only in Playwright's call log, which signatures cut: a 429, a 404 and a 503 shared one error group, shown under the category of its first error. The signature now keeps the final response line of the call log (`← 503 Service Unavailable`, the last one after redirects), so each status gets its own group, matching its category. The same status on different endpoints still shares one group. Other messages keep their signature; `toBeOK()` failures get a new one, so their groups do not match those of earlier reports. No change to the JSON shape (`schemaVersion` stays 1). See [Error signature](docs/schema/v1.md#error-signature).
+- Suite health's skipped tests use the new wording (`Skipped: <reason>`, `Fixme (no reason given)`, `Not run`) instead of a `fixme` badge and "No reason given"; a test with several skip annotations shows the first one with a reason.
+
+## [0.1.2] - 2026-10-21
+
+### Security
+
+- **Secrets are redacted from the report.** Reports are shared (CI artifacts, links), but they could contain secrets in clear: the value typed by `fill` into a password field is part of Playwright's step title (`Fill "<value>"`), and the call log of a failing API call prints the request and response headers, session cookies and bearer tokens included. Loupe now replaces them with `[redacted]` in `report.json`, and therefore in `index.html`, before writing them and before computing error signatures and categories. No change to your tests.
+  - **Input steps** (on by default): the value of `fill`, `type`, `pressSequentially`, `keyboard.type` and `keyboard.insertText` steps whose target has a secret-looking name, read from the step subtitle (Playwright 1.63+) or, on every version, from the source line of the call. That fallback also covers `keyboard.*` and neutral locators when the code names the value (`fill(password)`). The same value is redacted where Playwright repeats it in that attempt's errors (`fill("[redacted]")` in call logs, assertion lines).
+  - **HTTP headers** (on by default): `cookie`, `set-cookie`, `authorization`, `proxy-authorization` and any header with a secret-looking name, in call logs (`- cookie: [redacted]`).
+  - **URL parameters** (on by default): query and fragment parameters with a secret-looking name, wherever URLs appear (navigation steps, call logs, error messages, stdout), keeping the name: `?token=[redacted]`, `#access_token=[redacted]`.
+  - **Environment values** (opt-in): the values of the variables listed in the new `redact.env` option, or added through `LOUPE_REDACT_ENV`, everywhere in the report. Values shorter than 6 characters and unset variables are skipped with a warning naming the variable.
+  - New `redact` option (`patterns`, `mode`, `env`), validated at startup; one list of secret-looking names for input steps, headers and URL parameters, with whole-word matching (`auth` matches `x-auth-token`, not `Author`). The environment can add names, never remove them.
+  - **Not redacted**: attachments (traces, screenshots and videos contain typed values: treat a report with traces as private), snippets (source code from your repository), attachment file names. See [Secret redaction](docs/config.md#secret-redaction). Reports written by earlier versions are not rewritten.
+
+### Changed
+
+- **Error signatures keep HTTP status codes.** A status code (100–599) in an HTTP context is no longer replaced with `<n>`, so `Unable to delete the member with API with error 401` and `… with error 500` are two error groups instead of one. Other numbers are normalized as before. This changes how signatures are computed, not the JSON shape (`schemaVersion` stays 1): signatures of messages with an HTTP status change, so groups do not match across this upgrade for those errors. See [Error signature](docs/schema/v1.md#error-signature).
+- **Wider HTTP context, shared by categories and signatures.** A status code now also counts as an HTTP status after `error` (`with error 503`, `Error: 503`) and before its standard reason phrase (`401 Unauthorized`, previously only for 5xx). Categories change accordingly: an error such as `… with error 503` is now **Server error** and `… error 429` **Rate limited**, where they could be Uncategorized (or Assertion, Timeout) before. See [HTTP context](docs/schema/v1.md#http-context).
+
+### Fixed
+
+- Docs: `Step.subtitle` is provided from Playwright **1.63**, not 1.61, and before that the target of an action is not always in the title: Playwright 1.53 to 1.62 leave it out (`Click`, `Fill "<value>"`). Only 1.52 writes it there (`locator.getByRole('button').click`). The report itself is unchanged.
+
 ## [0.1.1] - 2026-10-18
 
 ### Added

package/README.md

@@ -38,8 +38,8 @@ A run that broke outside its tests (a spec file that fails to import, a failing
 The report is a single self-contained HTML file. It works offline, straight from `file://`, in light and dark themes.
 
 - **Header:** the run's verdict and status counts, its start, duration and environment; the shortcuts and theme controls stay top-right at every width.
-- **Overview:** verdict with pass rate, run errors first when the run broke outside its tests, error categories, project cards, and the run details in three groups: Run (scope, CI, workers, shard), Code (commit, short with a copy button; branch; repository) and Environment (Playwright, Loupe, Node, OS). Empty values are left out.
-- **Errors:** every error group, built from normalized error signatures, with the failed / flaky split and links to the affected tests. A group page has previous / next buttons: from a category, they stay within that category.
+- **Overview:** verdict with pass rate, run errors first when the run broke outside its tests, [common failure points](#common-failure-points), error categories, project cards, and the run details in three groups: Run (scope, CI, workers, shard), Code (commit, short with a copy button; branch; repository) and Environment (Playwright, Loupe, Node, OS). Empty values are left out.
+- **Errors:** every error group, built from normalized error signatures, with the failed / flaky split and links to the affected tests. Groups [only seen after a test timeout](#groups-only-seen-after-a-test-timeout) are folded at the end. A group page has previous / next buttons: from a category, they stay within that category.
 - **Test detail:** title path, outcome, project, `file:line`, tags, annotations and dimensions, then one tab per attempt (the last failing attempt first; `?attempt=` in the URL): errors with Expected / Received and diffs highlighted and the code snippet, steps (`test.step` and failing steps, the failing path expanded; "Show all steps" for hooks, fixtures and API calls), attachments, stdout and stderr. Previous / next walk the Tests view's current list.
 - **Attachments**, per attempt, linked from `data/` next to the report (never inlined in the HTML), all working from `file://`:
   - **Screenshot comparisons**: a failed `toHaveScreenshot` / `toMatchSnapshot` gives one card per image name, before the other screenshots, with four modes: Diff, Side by side (expected | actual, at one scale), Slider (actual over expected, a divider to drag or move with the arrow keys) and Overlay (actual over expected, its opacity to set). Only the modes the images allow are offered; Diff is the default when there is a diff image. The chosen mode applies to every comparison until the report is closed. Each image's size is shown, with a warning when expected and actual differ: slider and overlay align them top-left. "Open in viewer" shows the same modes in the viewer below.
@@ -50,10 +50,40 @@ The report is a single self-contained HTML file. It works offline, straight from
 - **Copy buttons:** rerun commands (this test, the failed tests of its file, the failed tests of the current list) and an AI prompt describing the failure, in English. Both are only copied to the clipboard; nothing is sent anywhere. Projects failing on the same lines share one command (`--project=a --project=b`); otherwise each project gets its own, joined with `&&`.
   - The commands target **POSIX shells** (bash, zsh; not `cmd.exe`) and run from the Playwright root directory: paths in the report are relative to it.
   - A `file:line` argument targets the line where the test is declared and reruns every test declared on that line. A command is only valid while the file has not changed since the run.
-- **Suite health:** dimensions from tags and annotations (with hints for unknown or inconsistent values), slowest, skipped and flaky tests.
-- **Tests:** every test in one sortable list (by default in declaration order, as in the sidebar: project, file, line; the Status header sorts failed, flaky, passed, skipped), filtered by status, error category, project, tag and dimension, with the count each filter would give. Header counts, error categories and dimension values open it pre-filtered. Large lists show their first 500 rows, with "Show 500 more" and "Show all". Under each title, chips show what the test covers: its dimension values first (dimensions in config order, the value alone, `api`, with the dimension in the tooltip), then its other tags (`@smoke`), sorted. A tag a dimension takes appears only as that dimension's value. A chip adds its value to the filters, keeping the others and the sort; an active chip (with a ×) removes it, and Back undoes either. Up to four chips per row, active chips always among them; "+N" shows the rest.
-- **Sidebar:** search (`/` to focus; every word must match, ignoring case and accents, the title path, file, project, tags or dimension values: `smoke`, `api`, `layer:api`), projects, folders, files, `describe` blocks and tests, with status icons, failure counts and keyboard navigation. The folder shared by every file is not repeated.
-- **Filters in the URL:** search, filters and sort live in the link, so a filtered view can be shared or bookmarked, even from `file://`. A project without a `name` (Playwright's default project) cannot be selected in the project filter: an empty name cannot be written in the URL. Give your projects a name to filter by them. Each tag is filterable in one place only: a tag a dimension is built from (`@layer:ui` with a `layer` dimension, configured or automatic) is in that dimension's filter (Layer: `ui`), not in the tag filter, so the two can never be combined into an empty result. Unprefixed tags (`@smoke`) and prefixed tags no dimension uses stay in the tag filter.
+- **Suite health:** dimensions from tags and annotations (with hints for unknown or inconsistent values), slowest, skipped (with why, see [Skip reasons](#skip-reasons)) and flaky tests.
+- **Tests:** every test in one sortable list (by default in declaration order, as in the sidebar: project, file, line; the Status header sorts failed, flaky, passed, skipped), filtered by status, error category, failure point, project, tag and dimension, with the count each filter would give. Header counts, error categories, failure points and dimension values open it pre-filtered. Large lists show their first 500 rows, with "Show 500 more" and "Show all". Under each title, chips show what the test covers: its dimension values first (dimensions in config order, the value alone, `api`, with the dimension in the tooltip), then its other tags (`@smoke`), sorted. A tag a dimension takes appears only as that dimension's value. A chip adds its value to the filters, keeping the others and the sort; an active chip (with a ×) removes it, and Back undoes either. Up to four chips per row, active chips always among them; "+N" shows the rest. A skipped test shows why on a muted line under its title, above the chips.
+- **Sidebar:** search (`/` to focus; every word must match, ignoring case and accents, the title path, file, project, tags, dimension values or the reason a test was skipped: `smoke`, `api`, `layer:api`, `sandbox`), projects, folders, files, `describe` blocks and tests, with status icons, failure counts and keyboard navigation. The folder shared by every file is not repeated.
+- **Filters in the URL:** search, filters and sort live in the link, so a filtered view can be shared or bookmarked, even from `file://`. A project without a `name` (Playwright's default project) cannot be selected in the project filter: an empty name cannot be written in the URL. Give your projects a name to filter by them. Each tag is filterable in one place only: a tag a dimension is built from (`@layer:ui` with a `layer` dimension, configured or automatic) is in that dimension's filter (Layer: `ui`), not in the tag filter, so the two can never be combined into an empty result. Unprefixed tags (`@smoke`) and prefixed tags no dimension uses stay in the tag filter. The failure point filter (`point`, the value `<file>#<function>` encoded like any other, `#` as `%23`: `point=fixtures%2Fsession.ts%23signIn`) offers the points of the Overview card; a test matches when the function is in the stack of one of its failing attempts.
+
+### Skip reasons
+
+A skipped test says why, in Suite health, under its title in the Tests view and next to its outcome in the test detail. The reason is the one given in your test code, read from the `skip` / `fixme` annotation Playwright records for `test.skip(condition, 'reason')`, `test.fixme(condition, 'reason')`, `testInfo.skip(…)`, in a test, a `describe` or a hook:
+
+- `Skipped: <reason>` / `Fixme: <reason>`;
+- `Skipped (no reason given)` / `Fixme (no reason given)` when the call has no reason (`test.skip('title', body)`, `test.describe.fixme`…);
+- `Not run` when Playwright skipped the test without any skip annotation: usually because a dependency project or an earlier test of its serial group failed, the failure limit (`--max-failures`) was reached, or the run was stopped. The report does not say which, so Loupe does not guess.
+
+The search matches the reason's text (not the `Skipped` or `Not run` labels). Reasons go through the same secret redaction as every other string of the report.
+
+### Common failure points
+
+When one problem breaks many tests, their errors rarely share a message: the same function fails at different lines, with different errors, in different categories. The Overview card **Common failure points** lists the functions of your code that appear in the stack of several failures, in different spec files and error groups. **It is a lead, not a diagnosis:** a shared helper also shows up when unrelated failures go through it.
+
+- **What is counted:** every error of every failing attempt, flaky tests included (the card gives the failed / flaky split), read from its stack trace. A point is a function in a file, whatever the line: `fixtures/session.ts#signIn`, with the lines seen. Paths are relative to the Playwright root dir, with `../` for helpers outside it (`../helpers/api.ts`). Spec files (any file holding a test of the report), anonymous functions, `node_modules`, Node internals and Playwright's own frames are left out.
+- **Thresholds:** a point is shown when at least **5** distinct tests (a test run in several projects counts once), **3** spec files and **2** error groups go through it. At most **5** points, the most tests first. A point below a threshold is shown nowhere.
+- **Folding:** a function only reached through another one, by some of its tests, is listed under it (`via checkResponse`) rather than on its own. When their tests only partly overlap, both are shown.
+- Each point opens its tests in the Tests view (the Failure point filter). The root dir is found from the stacks (the frames of each test's own file); when no stack shows it, the card is hidden.
+
+Nothing is configured and nothing changes in your tests: the points are derived when the report opens, from the stacks Playwright records. The thresholds may change while the heuristic is tuned.
+
+### Groups only seen after a test timeout
+
+When a test times out, Playwright stops it and the action still running fails too (`Request context disposed`, `Target page, context or browser has been closed`, `locator.click: Test timeout of 30000ms exceeded`): a second error of the same attempt. Such follow-on errors can make up many error groups of a slow run.
+
+- An error is **secondary** when it is not the first error of its attempt and that first error is Playwright's test timeout: `Test timeout of <n>ms exceeded.`, also while setting up or tearing down a fixture, in a `beforeEach` / `afterEach` hook, a fixture's own timeout and a `beforeAll` / `afterAll` hook timeout.
+- A group is **folded** when every one of its errors is secondary. A group seen at least once as a first error stays visible.
+- In the Errors view, folded groups move to a collapsed "N groups only seen after a test timeout" at the end of the list and of each category. Group counts leave them out and say so (`22 error groups · 18 more only seen after a test timeout`), on the Overview too; test counts are unchanged. `n` / `p` walk the visible groups first, then the folded ones.
+- A folded group's page stays reachable, and in the test detail each secondary error keeps its place, labelled "Raised after the test timeout". The category of a test does not change: the second error is sometimes the most telling one.
 
 ### Keyboard shortcuts
 
@@ -103,6 +133,21 @@ Options can also be passed inline (`['@qartisan/playwright-loupe', { title: 'Nig
 
 Every option, the lookup rules and the environment variables are documented in [`docs/config.md`](docs/config.md).
 
+### Security
+
+Reports are meant to be shared, so Loupe removes secrets from `report.json` and `index.html` before writing them, replacing each with `[redacted]`:
+
+- **typed values** of `fill`, `type`, `pressSequentially` and `keyboard.*` steps on secret-looking fields (a password field, `getByLabel('Password')`, or a source line that names the value, `fill(password)`), and wherever Playwright repeats them in that attempt's errors;
+- **HTTP header values** printed in call logs (`cookie`, `set-cookie`, `authorization`, and any header with a secret-looking name);
+- **URL parameters** with a secret-looking name, wherever URLs appear (navigation steps, call logs, error messages): `?token=[redacted]`, `&access_token=[redacted]`;
+- the **values of the environment variables you list** in `redact.env` (names only, never values), everywhere in the report. It is also the fallback for what the names above miss: a secret in a URL parameter with a neutral name (`?sig=…`), or printed by your code.
+
+```ts
+redact: { env: ['E2E_PASSWORD', 'API_TOKEN'] },
+```
+
+**Attachments are not altered**: traces, screenshots and videos contain typed values and visible secrets, so a report with traces must be treated as private. Snippets (source excerpts around errors) are left as they are: they come from your repository. See [Secret redaction](docs/config.md#secret-redaction).
+
 ## Compatibility
 
 | | Supported | Tested in CI |

package/dist/config/resolve.d.ts

@@ -3,7 +3,7 @@ type Env = Readonly<Record<string, string | undefined>>;
 export declare const DEFAULT_OUTPUT_FOLDER = "loupe-report";
 export declare const DEFAULT_TITLE = "Loupe report";
 export declare const DEFAULT_UNCATEGORIZED = "Uncategorized";
-/** Environment variables mapped onto config options. `LOUPE_CONFIG` and `LOUPE_RUN_SCOPE` are handled apart. */
+/** Environment variables mapped onto config options. `LOUPE_CONFIG`, `LOUPE_RUN_SCOPE` and `LOUPE_REDACT_ENV` are handled apart. */
 export declare const ENV_OPTIONS: {
     readonly LOUPE_OUTPUT_FOLDER: 'outputFolder';
     readonly LOUPE_TITLE: 'title';
@@ -22,6 +22,17 @@ export declare class LoupeConfigError extends Error {
 export declare function stripPlaywrightKeys(options: Record<string, unknown>): Record<string, unknown>;
 /** The environment layer: options set through `LOUPE_*` variables. Empty variables are ignored. */
 export declare function envLayer(env: Env): LoupeConfig;
+/** Comma-separated variable names added to `redact.env`. */
+export declare const REDACT_ENV = "LOUPE_REDACT_ENV";
+/**
+ * Names from `LOUPE_REDACT_ENV`, with a problem for each invalid one (its text
+ * is not echoed: it may be a value pasted by mistake). The environment can only
+ * add names to `redact.env`, never remove or disable redaction.
+ */
+export declare function redactEnvNames(env: Env): {
+    names: string[];
+    problems: string[];
+};
 /**
  * Merges `layer` over `base`: nested objects (`categories`, `dimensions`,
  * `screenshots`, `aiPrompt`) merge key by key; arrays and other values replace. An

package/dist/config/resolve.js

@@ -1,10 +1,10 @@
 import path from 'node:path';
 import { paletteReference } from '../shared/palette.js';
-import { validateConfig } from './validate.js';
+import { ENV_NAME, validateConfig } from './validate.js';
 export const DEFAULT_OUTPUT_FOLDER = 'loupe-report';
 export const DEFAULT_TITLE = 'Loupe report';
 export const DEFAULT_UNCATEGORIZED = 'Uncategorized';
-/** Environment variables mapped onto config options. `LOUPE_CONFIG` and `LOUPE_RUN_SCOPE` are handled apart. */
+/** Environment variables mapped onto config options. `LOUPE_CONFIG`, `LOUPE_RUN_SCOPE` and `LOUPE_REDACT_ENV` are handled apart. */
 export const ENV_OPTIONS = {
     LOUPE_OUTPUT_FOLDER: 'outputFolder',
     LOUPE_TITLE: 'title',
@@ -42,6 +42,30 @@ export function envLayer(env) {
     }
     return layer;
 }
+/** Comma-separated variable names added to `redact.env`. */
+export const REDACT_ENV = 'LOUPE_REDACT_ENV';
+/**
+ * Names from `LOUPE_REDACT_ENV`, with a problem for each invalid one (its text
+ * is not echoed: it may be a value pasted by mistake). The environment can only
+ * add names to `redact.env`, never remove or disable redaction.
+ */
+export function redactEnvNames(env) {
+    const raw = envValue(env, REDACT_ENV);
+    if (raw === undefined)
+        return { names: [], problems: [] };
+    const names = [];
+    const problems = [];
+    raw.split(',').forEach((part, index) => {
+        const name = part.trim();
+        if (name === '')
+            return;
+        if (ENV_NAME.test(name))
+            names.push(name);
+        else
+            problems.push(`${REDACT_ENV}: item ${index + 1} must be an environment variable name (letters, digits, _), got a string that is not a variable name (value not shown)`);
+    });
+    return { names, problems };
+}
 function isObject(value) {
     return value !== null && typeof value === 'object' && !Array.isArray(value) && !(value instanceof RegExp);
 }
@@ -86,9 +110,11 @@ function resolveDimensions(config) {
 export function resolveConfig(layers) {
     const fileSource = layers.file ? path.basename(layers.file.path) : '';
     const env = envLayer(layers.env);
+    const redactEnv = redactEnvNames(layers.env);
     const problems = [
         ...(layers.file ? validateConfig(layers.file.config, fileSource) : []),
         ...validateConfig(layers.inline, 'reporter options', { allowConfigFile: true }),
+        ...redactEnv.problems,
     ];
     if (problems.length > 0)
         throw new LoupeConfigError(problems);
@@ -115,5 +141,11 @@ export function resolveConfig(layers) {
         screenshots: { defaultView: merged.screenshots?.defaultView ?? 'side-by-side' },
         theme: merged.theme ?? 'system',
         aiPrompt: { includeStack: merged.aiPrompt?.includeStack ?? true },
+        redact: {
+            mode: merged.redact?.mode ?? 'extend',
+            patterns: (merged.redact?.patterns ?? []).map((pattern) => (pattern instanceof RegExp ? stateless(pattern) : pattern)),
+            // LOUPE_REDACT_ENV adds to the configured names: a higher layer never removes one.
+            env: [...new Set([...(merged.redact?.env ?? []), ...redactEnv.names])],
+        },
     };
 }

package/dist/config/types.d.ts

@@ -4,6 +4,8 @@
  */
 export declare const CATEGORY_MODES: readonly ['extend', 'replace'];
 export type CategoryMode = (typeof CATEGORY_MODES)[number];
+export declare const REDACT_MODES: readonly ['extend', 'replace'];
+export type RedactMode = (typeof REDACT_MODES)[number];
 export declare const CATEGORY_TARGETS: readonly ['message', 'stack', 'step', 'all'];
 export type CategoryTarget = (typeof CATEGORY_TARGETS)[number];
 export declare const SCREENSHOT_VIEWS: readonly ['side-by-side', 'slider', 'overlay', 'diff'];
@@ -53,6 +55,18 @@ export interface DimensionsConfig {
     auto?: boolean;
     definitions?: DimensionConfig[];
 }
+export interface RedactConfig {
+    /**
+     * Secret-looking names, for input step targets and source lines, and for
+     * HTTP header names. A string matches whole words (`auth` matches
+     * `x-auth-token`, not `Author`); a RegExp is tested as is.
+     */
+    patterns?: Array<string | RegExp>;
+    /** `extend` (default): `patterns` add to the default names. `replace`: only `patterns`; `[]` turns name-based redaction off. */
+    mode?: RedactMode;
+    /** Names of environment variables whose values are redacted everywhere in the report. Names only, never values. */
+    env?: string[];
+}
 export interface LoupeConfig {
     /** Folder the report is written to, relative to the Playwright config file. Default: `loupe-report`. */
     outputFolder?: string;
@@ -72,6 +86,8 @@ export interface LoupeConfig {
         /** Include each error's stack trace in the copyable AI prompt. Default: `true`. */
         includeStack?: boolean;
     };
+    /** Secret redaction in report.json and index.html. `LOUPE_REDACT_ENV` adds variable names. */
+    redact?: RedactConfig;
 }
 /** Inline reporter options: a config, plus the path of the config file. */
 export interface LoupeReporterOptions extends LoupeConfig {
@@ -121,4 +137,10 @@ export interface ResolvedConfig {
     aiPrompt: {
         includeStack: boolean;
     };
+    /** RegExps without the `g` and `y` flags; `env` merged from every layer, without duplicates. */
+    redact: {
+        mode: RedactMode;
+        patterns: Array<string | RegExp>;
+        env: string[];
+    };
 }

package/dist/config/types.js

@@ -3,6 +3,7 @@
  * reporter options. Every option is optional; see docs/config.md.
  */
 export const CATEGORY_MODES = ['extend', 'replace'];
+export const REDACT_MODES = ['extend', 'replace'];
 export const CATEGORY_TARGETS = ['message', 'stack', 'step', 'all'];
 export const SCREENSHOT_VIEWS = ['side-by-side', 'slider', 'overlay', 'diff'];
 export const THEMES = ['system', 'light', 'dark'];

package/dist/config/validate.d.ts

@@ -1,3 +1,5 @@
+/** An environment variable name, as `redact.env` and `LOUPE_REDACT_ENV` take them. */
+export declare const ENV_NAME: RegExp;
 /** Optimal string alignment distance: Levenshtein plus adjacent transpositions (`titel` → `title` is 1). */
 export declare function editDistance(a: string, b: string): number;
 /** The closest known key, when it is close enough to be a typo. */

package/dist/config/validate.js

@@ -1,6 +1,8 @@
 import { PALETTE_NAMES } from '../shared/palette.js';
-import { CATEGORY_MODES, CATEGORY_TARGETS, SCREENSHOT_VIEWS, THEMES } from './types.js';
+import { CATEGORY_MODES, CATEGORY_TARGETS, REDACT_MODES, SCREENSHOT_VIEWS, THEMES } from './types.js';
 const COLOR = /^#[0-9a-fA-F]{6}$/;
+/** An environment variable name, as `redact.env` and `LOUPE_REDACT_ENV` take them. */
+export const ENV_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/;
 /** What a tag prefix cannot contain: `@layer:api` has the prefix `layer`. */
 const INVALID_PREFIX = /[@:\s]/;
 function describe(value) {
@@ -151,6 +153,37 @@ function values(ctx) {
         });
     };
 }
+/** Secret-looking names: non-empty strings or RegExps. An empty list is valid (with `mode: 'replace'`, it turns them off). */
+function namePatterns(ctx) {
+    return (value, path, errors) => {
+        if (!Array.isArray(value))
+            return fail(ctx, path, 'an array of strings or RegExps', value, errors);
+        value.forEach((item, index) => {
+            if (item instanceof RegExp)
+                return;
+            if (typeof item !== 'string' || item.trim() === '')
+                fail(ctx, `${path}[${index}]`, 'a non-empty string or a RegExp', item, errors);
+        });
+    };
+}
+/** Environment variable names. A value with other characters is most likely a secret pasted by mistake: it is not echoed. */
+function envNames(ctx) {
+    return (value, path, errors) => {
+        if (!Array.isArray(value))
+            return fail(ctx, path, 'an array of environment variable names', value, errors);
+        const seen = new Set();
+        value.forEach((item, index) => {
+            if (typeof item !== 'string' || !ENV_NAME.test(item)) {
+                const got = typeof item === 'string' ? 'a string that is not a variable name (value not shown)' : describe(item);
+                errors.push(`${ctx.source}: ${path}[${index}] must be an environment variable name (letters, digits, _), got ${got}`);
+                return;
+            }
+            if (seen.has(item))
+                errors.push(`${ctx.source}: ${path}[${index}] repeats the variable ${item}`);
+            seen.add(item);
+        });
+    };
+}
 function tagPrefix(ctx) {
     return (value, path, errors) => {
         if (typeof value !== 'string' || value === '' || INVALID_PREFIX.test(value)) {
@@ -228,6 +261,7 @@ export function validateConfig(value, source, options = {}) {
         screenshots: object({ defaultView: oneOf(SCREENSHOT_VIEWS, ctx) }, ctx),
         theme: oneOf(THEMES, ctx),
         aiPrompt: object({ includeStack: boolean(ctx) }, ctx),
+        redact: object({ patterns: namePatterns(ctx), mode: oneOf(REDACT_MODES, ctx), env: envNames(ctx) }, ctx),
     };
     if (options.allowConfigFile)
         fields['configFile'] = nonEmptyString(ctx);

package/dist/core/build.d.ts

@@ -2,6 +2,7 @@ import type { FullConfig, FullResult, Suite, TestError, TestStep } from '@playwr
 import type { LoupeReport, RunScope } from '../schema/v1.js';
 import { type CategoriesOptions, type ErrorText } from './categories.js';
 import { type DimensionsOptions } from './dimensions.js';
+import { type RedactOptions } from './redact.js';
 /** Folder, inside the report folder, that holds the copied attachments. */
 export declare const DATA_FOLDER = "data";
 export interface BuildOptions {
@@ -20,12 +21,14 @@ export interface BuildOptions {
         error: TestError;
         project: string | null;
     }>;
+    /** Secret redaction; default: the built-in names, no env values. */
+    redact?: RedactOptions;
 }
 /**
  * Titles of the failing steps leading to `error`, outermost first: the path to
  * the deepest step that ended with the same message. Empty when no step did.
  */
-export declare function failingSteps(steps: readonly TestStep[], error: TestError): string[];
+export declare function failingSteps(steps: readonly TestStep[], error: TestError, title?: (step: TestStep) => string): string[];
 /**
  * Text an error is categorized on. `steps`: titles of the failing steps, outermost first.
  * Run errors lose the `merge-reports` tag prefix, so rules match the same text merged or not.

package/dist/core/build.js

@@ -8,6 +8,7 @@ import { detectEnvironment, detectGit } from './metadata.js';
 import { collectOutput } from './output.js';
 import { esmImportLocation, isUsableLocation } from './runErrors.js';
 import { detectScopeReasons, resolveScope } from './scope.js';
+import { DEFAULT_REDACT, createRedactor, mapReportStrings } from './redact.js';
 import { errorSignature, stripAnsi, stripMergeTagPrefix } from './signature.js';
 /** Folder, inside the report folder, that holds the copied attachments. */
 export const DATA_FOLDER = 'data';
@@ -119,20 +120,20 @@ function isFocused(test) {
  * Titles of the failing steps leading to `error`, outermost first: the path to
  * the deepest step that ended with the same message. Empty when no step did.
  */
-export function failingSteps(steps, error) {
+export function failingSteps(steps, error, title = (step) => step.title) {
     const message = error.message ?? error.value;
     if (message === undefined)
         return [];
     for (const step of [...steps].reverse()) {
         if (!step.error || (step.error.message ?? step.error.value) !== message)
             continue;
-        return [step.title, ...failingSteps(step.steps, error)];
+        return [title(step), ...failingSteps(step.steps, error, title)];
     }
     // A step may fail with a wrapped message while one of its children has the exact one.
     for (const step of [...steps].reverse()) {
-        const inner = failingSteps(step.steps, error);
+        const inner = failingSteps(step.steps, error, title);
         if (inner.length > 0)
-            return [step.title, ...inner];
+            return [title(step), ...inner];
     }
     return [];
 }
@@ -151,6 +152,7 @@ export async function buildReport(config, suite, result, options) {
     const rootDir = config.rootDir;
     const store = new AttachmentStore(options.outputFolder);
     const categorizer = createCategorizer(options.categories ?? DEFAULT_CATEGORIES);
+    const redactor = createRedactor(options.redact ?? DEFAULT_REDACT);
     const dimensionSet = createDimensions(options.dimensions ?? DEFAULT_DIMENSIONS);
     const categories = new Set();
     const usedDimensions = new Set();
@@ -161,10 +163,11 @@ export async function buildReport(config, suite, result, options) {
     /**
      * `steps`: titles of the failing steps leading to the error, outermost first.
      * `runError`: the `merge-reports` tag prefix is ignored by categories and the signature, kept in `message` and `stack`.
+     * `redact`: applied to the message and the stack before the signature and the category, so they never depend on a secret.
      */
-    const toError = (error, steps = [], location = error.location, runError = false) => {
-        const message = stripAnsi(error.message ?? error.value ?? '');
-        const stack = error.stack ? stripAnsi(error.stack) : null;
+    const toError = (error, redact, steps = [], location = error.location, runError = false) => {
+        const message = redact(stripAnsi(error.message ?? error.value ?? ''));
+        const stack = error.stack ? redact(stripAnsi(error.stack)) : null;
         const category = categorizer.categorize(errorText(message, stack, steps, runError));
         categories.add(category);
         return {
@@ -176,17 +179,21 @@ export async function buildReport(config, suite, result, options) {
             category,
         };
     };
-    const toStep = (step, parents) => {
-        const chain = [...parents, step.title];
+    const toStep = (step, parents, redact) => {
+        const title = redact.title(step);
+        const chain = [...parents, title];
+        // `subtitle` exists from Playwright 1.63. Before, the target of an action is in
+        // the title on 1.52 (`locator.getByRole('button').click`), and nowhere on
+        // 1.53–1.62 (`Click`, `Fill "<value>"`).
+        const subtitle = step.subtitle;
         return {
-            title: step.title,
-            // `subtitle` exists from Playwright 1.61; older versions keep the target in the title.
-            subtitle: step.subtitle ?? null,
+            title,
+            subtitle: subtitle === undefined ? null : redact.text(subtitle),
             category: step.category,
             durationMs: toMs(step.duration),
-            error: step.error ? toError(step.error, chain) : null,
+            error: step.error ? toError(step.error, redact.text, chain) : null,
             location: toLocation(step.location),
-            steps: step.steps.map((child) => toStep(child, chain)),
+            steps: step.steps.map((child) => toStep(child, chain, redact)),
         };
     };
     const toAttempt = async (attempt, folder) => {
@@ -194,14 +201,15 @@ export async function buildReport(config, suite, result, options) {
         for (const attachment of attempt.attachments) {
             attachments.push(await store.add(`${folder}/${attempt.retry}`, attachment));
         }
+        const redact = redactor.attempt(attempt.steps);
         return {
             index: attempt.retry,
             status: attempt.status,
             startedAt: attempt.startTime.toISOString(),
             durationMs: toMs(attempt.duration),
             workerIndex: attempt.workerIndex,
-            errors: attempt.errors.map((error) => toError(error, failingSteps(attempt.steps, error))),
-            steps: attempt.steps.map((step) => toStep(step, [])),
+            errors: attempt.errors.map((error) => toError(error, redact.text, failingSteps(attempt.steps, error, redact.title))),
+            steps: attempt.steps.map((step) => toStep(step, [], redact)),
             attachments,
             annotations: attempt.annotations.map(toAnnotation),
             stdout: collectOutput(attempt.stdout),
@@ -292,9 +300,9 @@ export async function buildReport(config, suite, result, options) {
     for (const { error, project } of options.runErrors ?? []) {
         const message = stripAnsi(error.message ?? error.value ?? '');
         const location = isUsableLocation(error.location) ? error.location : ((await esmImportLocation(message)) ?? undefined);
-        errors.push({ ...toError(error, [], location, true), project });
+        errors.push({ ...toError(error, redactor.text, [], location, true), project });
     }
-    return {
+    const report = {
         schemaVersion: 1,
         generator: { name: 'playwright-loupe', version: options.generatorVersion },
         run: {
@@ -318,4 +326,6 @@ export async function buildReport(config, suite, result, options) {
         categories: categorizer.definitions(categories),
         dimensions: dimensionSet.definitions(usedDimensions),
     };
+    // URL parameters and env values (redact.env) are redacted in every other string: test titles, annotations, outputs, attachment bodies...
+    return redactor.rewritesStrings ? mapReportStrings(report, redactor.strings) : report;
 }

package/dist/core/categories.js

@@ -1,14 +1,7 @@
 import { NEUTRAL_ENTRY, PALETTE_ENTRIES, paletteReference } from '../shared/palette.js';
+import { httpStatus } from './http.js';
 const ALL_TARGETS = ['message', 'stack', 'step'];
 export const UNCATEGORIZED_COLOR = NEUTRAL_ENTRY.reference;
-/**
- * An HTTP status in context: `status 429`, `status code: 429`, `"status": 429`,
- * `statusCode=429`, `HTTP 429`, `HTTP/1.1 429`, `Response status: 429`, or
- * `← 429`, the response line of Playwright's fetch log.
- */
-function httpStatus(code) {
-    return new RegExp(`(?:\\b(?:status(?:[ _]?code)?["']?\\s*[:=]?|HTTP(?:/\\d(?:\\.\\d)?)?:?)|\u2190)\\s*${code}(?!\\w|\\.\\d)`, 'i');
-}
 export const BUILT_IN_RULES = [
     {
         // Before "Locator not found", "Assertion" and "Timeout": a locator matching
@@ -30,7 +23,7 @@ export const BUILT_IN_RULES = [
         // Before "Network", "Assertion" and "Timeout": the status is the diagnosis,
         // whether it surfaces through `failOnStatusCode`, `toBeOK()` or a thrown
         // error. A bare 429 is not enough (`429ms`, `Received: 429`): it needs an
-        // HTTP context.
+        // HTTP context (src/core/http.ts), shared with error signatures.
         name: 'Rate limited',
         color: paletteReference('teal'),
         match: [/\bToo Many Requests\b/i, /\brate[- ]?limit/i, httpStatus('429')],
@@ -39,10 +32,8 @@ export const BUILT_IN_RULES = [
     {
         name: 'Server error',
         color: paletteReference('orange'),
-        match: [
-            httpStatus('5\\d\\d'),
-            /\b5\d\d (?:Internal Server Error|Not Implemented|Bad Gateway|Service Unavailable|Gateway Time-?out|HTTP Version Not Supported)\b/i,
-        ],
+        // In an HTTP context, reason phrase included: `500 Internal Server Error` (failOnStatusCode).
+        match: [httpStatus('5\\d\\d')],
         targets: ['message', 'stack'],
     },
     {

package/dist/core/http.d.ts

@@ -0,0 +1,21 @@
+/**
+ * HTTP status codes in context, shared by the "Rate limited" and "Server
+ * error" categories and by error signatures, so both agree on what an HTTP
+ * status is. See "Error categories" and "Error signature" in docs/schema/v1.md.
+ *
+ * A code is in an HTTP context when it is:
+ * - preceded by `status`, `status code`, `statusCode`, `"status":`, `HTTP`,
+ *   `HTTP/1.1`, `error`, or `←` (the response line of Playwright's fetch log),
+ *   with an optional `:` or `=`: `status 429`, `Response status: 503`,
+ *   `HTTP 500`, `statusCode=502`, `with error 401`;
+ * - or followed by its reason phrase: `401 Unauthorized`, `500 Internal Server Error`.
+ *
+ * A bare number is not enough: `Timeout 429ms exceeded` and `Expected: 200
+ * Received: 500` have no HTTP context.
+ */
+/** Any status code from 100 to 599. */
+export declare const STATUS_CODE = "[1-5]\\d\\d";
+/** Source of a pattern matching `code` (a regex source such as `429` or `5\\d\\d`) in an HTTP context, case-insensitive. */
+export declare function httpStatusSource(code: string): string;
+/** `code` in an HTTP context. */
+export declare function httpStatus(code: string): RegExp;

package/dist/core/http.js

@@ -0,0 +1,64 @@
+/**
+ * HTTP status codes in context, shared by the "Rate limited" and "Server
+ * error" categories and by error signatures, so both agree on what an HTTP
+ * status is. See "Error categories" and "Error signature" in docs/schema/v1.md.
+ *
+ * A code is in an HTTP context when it is:
+ * - preceded by `status`, `status code`, `statusCode`, `"status":`, `HTTP`,
+ *   `HTTP/1.1`, `error`, or `←` (the response line of Playwright's fetch log),
+ *   with an optional `:` or `=`: `status 429`, `Response status: 503`,
+ *   `HTTP 500`, `statusCode=502`, `with error 401`;
+ * - or followed by its reason phrase: `401 Unauthorized`, `500 Internal Server Error`.
+ *
+ * A bare number is not enough: `Timeout 429ms exceeded` and `Expected: 200
+ * Received: 500` have no HTTP context.
+ */
+const PREFIX = `(?:\\b(?:status(?:[ _]?code)?["']?\\s*[:=]?|HTTP(?:/\\d(?:\\.\\d)?)?:?|error\\s*[:=]?)|←)`;
+/** Standard reason phrases (RFC 9110, plus 429). */
+const REASON_PHRASES = [
+    'Continue',
+    'Switching Protocols',
+    'OK',
+    'Created',
+    'Accepted',
+    'No Content',
+    'Moved Permanently',
+    'Found',
+    'See Other',
+    'Not Modified',
+    'Temporary Redirect',
+    'Permanent Redirect',
+    'Bad Request',
+    'Unauthorized',
+    'Payment Required',
+    'Forbidden',
+    'Not Found',
+    'Method Not Allowed',
+    'Not Acceptable',
+    'Request Time-?out',
+    'Conflict',
+    'Gone',
+    'Precondition Failed',
+    'Payload Too Large',
+    'Content Too Large',
+    'Unsupported Media Type',
+    'Unprocessable Entity',
+    'Unprocessable Content',
+    'Too Many Requests',
+    'Internal Server Error',
+    'Not Implemented',
+    'Bad Gateway',
+    'Service Unavailable',
+    'Gateway Time-?out',
+    'HTTP Version Not Supported',
+];
+/** Any status code from 100 to 599. */
+export const STATUS_CODE = '[1-5]\\d\\d';
+/** Source of a pattern matching `code` (a regex source such as `429` or `5\\d\\d`) in an HTTP context, case-insensitive. */
+export function httpStatusSource(code) {
+    return `${PREFIX}\\s*${code}(?!\\w|\\.\\d)|\\b${code} (?:${REASON_PHRASES.join('|')})\\b`;
+}
+/** `code` in an HTTP context. */
+export function httpStatus(code) {
+    return new RegExp(httpStatusSource(code), 'i');
+}