@qartisan/playwright-loupe 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +15 -0
- package/dist/config/resolve.d.ts +12 -1
- package/dist/config/resolve.js +34 -2
- package/dist/config/types.d.ts +22 -0
- package/dist/config/types.js +1 -0
- package/dist/config/validate.d.ts +2 -0
- package/dist/config/validate.js +35 -1
- package/dist/core/build.d.ts +4 -1
- package/dist/core/build.js +28 -18
- package/dist/core/categories.js +4 -13
- package/dist/core/http.d.ts +21 -0
- package/dist/core/http.js +64 -0
- package/dist/core/redact.d.ts +101 -0
- package/dist/core/redact.js +254 -0
- package/dist/core/reporter.d.ts +2 -0
- package/dist/core/reporter.js +6 -0
- package/dist/core/signature.js +9 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,27 @@ All notable changes to this project are documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Before 1.0, minor versions may contain breaking changes. The JSON report has its own version (`schemaVersion`), see [`docs/schema/v1.md`](docs/schema/v1.md#versioning-policy).
|
|
6
6
|
|
|
7
|
+
## [0.1.2] - 2026-10-21
|
|
8
|
+
|
|
9
|
+
### Security
|
|
10
|
+
|
|
11
|
+
- **Secrets are redacted from the report.** Reports are shared (CI artifacts, links), but they could contain secrets in clear: the value typed by `fill` into a password field is part of Playwright's step title (`Fill "<value>"`), and the call log of a failing API call prints the request and response headers, session cookies and bearer tokens included. Loupe now replaces them with `[redacted]` in `report.json`, and therefore in `index.html`, before writing them and before computing error signatures and categories. No change to your tests.
|
|
12
|
+
- **Input steps** (on by default): the value of `fill`, `type`, `pressSequentially`, `keyboard.type` and `keyboard.insertText` steps whose target has a secret-looking name, read from the step subtitle (Playwright 1.63+) or, on every version, from the source line of the call. That fallback also covers `keyboard.*` and neutral locators when the code names the value (`fill(password)`). The same value is redacted where Playwright repeats it in that attempt's errors (`fill("[redacted]")` in call logs, assertion lines).
|
|
13
|
+
- **HTTP headers** (on by default): `cookie`, `set-cookie`, `authorization`, `proxy-authorization` and any header with a secret-looking name, in call logs (`- cookie: [redacted]`).
|
|
14
|
+
- **URL parameters** (on by default): query and fragment parameters with a secret-looking name, wherever URLs appear (navigation steps, call logs, error messages, stdout), keeping the name: `?token=[redacted]`, `#access_token=[redacted]`.
|
|
15
|
+
- **Environment values** (opt-in): the values of the variables listed in the new `redact.env` option, or added through `LOUPE_REDACT_ENV`, everywhere in the report. Values shorter than 6 characters and unset variables are skipped with a warning naming the variable.
|
|
16
|
+
- New `redact` option (`patterns`, `mode`, `env`), validated at startup; one list of secret-looking names for input steps, headers and URL parameters, with whole-word matching (`auth` matches `x-auth-token`, not `Author`). The environment can add names, never remove them.
|
|
17
|
+
- **Not redacted**: attachments (traces, screenshots and videos contain typed values: treat a report with traces as private), snippets (source code from your repository), attachment file names. See [Secret redaction](docs/config.md#secret-redaction). Reports written by earlier versions are not rewritten.
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
|
|
21
|
+
- **Error signatures keep HTTP status codes.** A status code (100–599) in an HTTP context is no longer replaced with `<n>`, so `Unable to delete the member with API with error 401` and `… with error 500` are two error groups instead of one. Other numbers are normalized as before. This changes how signatures are computed, not the JSON shape (`schemaVersion` stays 1): signatures of messages with an HTTP status change, so groups do not match across this upgrade for those errors. See [Error signature](docs/schema/v1.md#error-signature).
|
|
22
|
+
- **Wider HTTP context, shared by categories and signatures.** A status code now also counts as an HTTP status after `error` (`with error 503`, `Error: 503`) and before its standard reason phrase (`401 Unauthorized`, previously only for 5xx). Categories change accordingly: an error such as `… with error 503` is now **Server error** and `… error 429` **Rate limited**, where they could be Uncategorized (or Assertion, Timeout) before. See [HTTP context](docs/schema/v1.md#http-context).
|
|
23
|
+
|
|
24
|
+
### Fixed
|
|
25
|
+
|
|
26
|
+
- Docs: `Step.subtitle` is provided from Playwright **1.63**, not 1.61, and before that the target of an action is not always in the title: Playwright 1.53 to 1.62 leave it out (`Click`, `Fill "<value>"`). Only 1.52 writes it there (`locator.getByRole('button').click`). The report itself is unchanged.
|
|
27
|
+
|
|
7
28
|
## [0.1.1] - 2026-10-18
|
|
8
29
|
|
|
9
30
|
### Added
|
package/README.md
CHANGED
|
@@ -103,6 +103,21 @@ Options can also be passed inline (`['@qartisan/playwright-loupe', { title: 'Nig
|
|
|
103
103
|
|
|
104
104
|
Every option, the lookup rules and the environment variables are documented in [`docs/config.md`](docs/config.md).
|
|
105
105
|
|
|
106
|
+
### Security
|
|
107
|
+
|
|
108
|
+
Reports are meant to be shared, so Loupe removes secrets from `report.json` and `index.html` before writing them, replacing each with `[redacted]`:
|
|
109
|
+
|
|
110
|
+
- **typed values** of `fill`, `type`, `pressSequentially` and `keyboard.*` steps on secret-looking fields (a password field, `getByLabel('Password')`, or a source line that names the value, `fill(password)`), and wherever Playwright repeats them in that attempt's errors;
|
|
111
|
+
- **HTTP header values** printed in call logs (`cookie`, `set-cookie`, `authorization`, and any header with a secret-looking name);
|
|
112
|
+
- **URL parameters** with a secret-looking name, wherever URLs appear (navigation steps, call logs, error messages): `?token=[redacted]`, `&access_token=[redacted]`;
|
|
113
|
+
- the **values of the environment variables you list** in `redact.env` (names only, never values), everywhere in the report. It is also the fallback for what the names above miss: a secret in a URL parameter with a neutral name (`?sig=…`), or printed by your code.
|
|
114
|
+
|
|
115
|
+
```ts
|
|
116
|
+
redact: { env: ['E2E_PASSWORD', 'API_TOKEN'] },
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
**Attachments are not altered**: traces, screenshots and videos contain typed values and visible secrets, so a report with traces must be treated as private. Snippets (source excerpts around errors) are left as they are: they come from your repository. See [Secret redaction](docs/config.md#secret-redaction).
|
|
120
|
+
|
|
106
121
|
## Compatibility
|
|
107
122
|
|
|
108
123
|
| | Supported | Tested in CI |
|
package/dist/config/resolve.d.ts
CHANGED
|
@@ -3,7 +3,7 @@ type Env = Readonly<Record<string, string | undefined>>;
|
|
|
3
3
|
export declare const DEFAULT_OUTPUT_FOLDER = "loupe-report";
|
|
4
4
|
export declare const DEFAULT_TITLE = "Loupe report";
|
|
5
5
|
export declare const DEFAULT_UNCATEGORIZED = "Uncategorized";
|
|
6
|
-
/** Environment variables mapped onto config options. `LOUPE_CONFIG` and `
|
|
6
|
+
/** Environment variables mapped onto config options. `LOUPE_CONFIG`, `LOUPE_RUN_SCOPE` and `LOUPE_REDACT_ENV` are handled apart. */
|
|
7
7
|
export declare const ENV_OPTIONS: {
|
|
8
8
|
readonly LOUPE_OUTPUT_FOLDER: 'outputFolder';
|
|
9
9
|
readonly LOUPE_TITLE: 'title';
|
|
@@ -22,6 +22,17 @@ export declare class LoupeConfigError extends Error {
|
|
|
22
22
|
export declare function stripPlaywrightKeys(options: Record<string, unknown>): Record<string, unknown>;
|
|
23
23
|
/** The environment layer: options set through `LOUPE_*` variables. Empty variables are ignored. */
|
|
24
24
|
export declare function envLayer(env: Env): LoupeConfig;
|
|
25
|
+
/** Comma-separated variable names added to `redact.env`. */
|
|
26
|
+
export declare const REDACT_ENV = "LOUPE_REDACT_ENV";
|
|
27
|
+
/**
|
|
28
|
+
* Names from `LOUPE_REDACT_ENV`, with a problem for each invalid one (its text
|
|
29
|
+
* is not echoed: it may be a value pasted by mistake). The environment can only
|
|
30
|
+
* add names to `redact.env`, never remove or disable redaction.
|
|
31
|
+
*/
|
|
32
|
+
export declare function redactEnvNames(env: Env): {
|
|
33
|
+
names: string[];
|
|
34
|
+
problems: string[];
|
|
35
|
+
};
|
|
25
36
|
/**
|
|
26
37
|
* Merges `layer` over `base`: nested objects (`categories`, `dimensions`,
|
|
27
38
|
* `screenshots`, `aiPrompt`) merge key by key; arrays and other values replace. An
|
package/dist/config/resolve.js
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
import path from 'node:path';
|
|
2
2
|
import { paletteReference } from '../shared/palette.js';
|
|
3
|
-
import { validateConfig } from './validate.js';
|
|
3
|
+
import { ENV_NAME, validateConfig } from './validate.js';
|
|
4
4
|
export const DEFAULT_OUTPUT_FOLDER = 'loupe-report';
|
|
5
5
|
export const DEFAULT_TITLE = 'Loupe report';
|
|
6
6
|
export const DEFAULT_UNCATEGORIZED = 'Uncategorized';
|
|
7
|
-
/** Environment variables mapped onto config options. `LOUPE_CONFIG` and `
|
|
7
|
+
/** Environment variables mapped onto config options. `LOUPE_CONFIG`, `LOUPE_RUN_SCOPE` and `LOUPE_REDACT_ENV` are handled apart. */
|
|
8
8
|
export const ENV_OPTIONS = {
|
|
9
9
|
LOUPE_OUTPUT_FOLDER: 'outputFolder',
|
|
10
10
|
LOUPE_TITLE: 'title',
|
|
@@ -42,6 +42,30 @@ export function envLayer(env) {
|
|
|
42
42
|
}
|
|
43
43
|
return layer;
|
|
44
44
|
}
|
|
45
|
+
/** Comma-separated variable names added to `redact.env`. */
|
|
46
|
+
export const REDACT_ENV = 'LOUPE_REDACT_ENV';
|
|
47
|
+
/**
|
|
48
|
+
* Names from `LOUPE_REDACT_ENV`, with a problem for each invalid one (its text
|
|
49
|
+
* is not echoed: it may be a value pasted by mistake). The environment can only
|
|
50
|
+
* add names to `redact.env`, never remove or disable redaction.
|
|
51
|
+
*/
|
|
52
|
+
export function redactEnvNames(env) {
|
|
53
|
+
const raw = envValue(env, REDACT_ENV);
|
|
54
|
+
if (raw === undefined)
|
|
55
|
+
return { names: [], problems: [] };
|
|
56
|
+
const names = [];
|
|
57
|
+
const problems = [];
|
|
58
|
+
raw.split(',').forEach((part, index) => {
|
|
59
|
+
const name = part.trim();
|
|
60
|
+
if (name === '')
|
|
61
|
+
return;
|
|
62
|
+
if (ENV_NAME.test(name))
|
|
63
|
+
names.push(name);
|
|
64
|
+
else
|
|
65
|
+
problems.push(`${REDACT_ENV}: item ${index + 1} must be an environment variable name (letters, digits, _), got a string that is not a variable name (value not shown)`);
|
|
66
|
+
});
|
|
67
|
+
return { names, problems };
|
|
68
|
+
}
|
|
45
69
|
function isObject(value) {
|
|
46
70
|
return value !== null && typeof value === 'object' && !Array.isArray(value) && !(value instanceof RegExp);
|
|
47
71
|
}
|
|
@@ -86,9 +110,11 @@ function resolveDimensions(config) {
|
|
|
86
110
|
export function resolveConfig(layers) {
|
|
87
111
|
const fileSource = layers.file ? path.basename(layers.file.path) : '';
|
|
88
112
|
const env = envLayer(layers.env);
|
|
113
|
+
const redactEnv = redactEnvNames(layers.env);
|
|
89
114
|
const problems = [
|
|
90
115
|
...(layers.file ? validateConfig(layers.file.config, fileSource) : []),
|
|
91
116
|
...validateConfig(layers.inline, 'reporter options', { allowConfigFile: true }),
|
|
117
|
+
...redactEnv.problems,
|
|
92
118
|
];
|
|
93
119
|
if (problems.length > 0)
|
|
94
120
|
throw new LoupeConfigError(problems);
|
|
@@ -115,5 +141,11 @@ export function resolveConfig(layers) {
|
|
|
115
141
|
screenshots: { defaultView: merged.screenshots?.defaultView ?? 'side-by-side' },
|
|
116
142
|
theme: merged.theme ?? 'system',
|
|
117
143
|
aiPrompt: { includeStack: merged.aiPrompt?.includeStack ?? true },
|
|
144
|
+
redact: {
|
|
145
|
+
mode: merged.redact?.mode ?? 'extend',
|
|
146
|
+
patterns: (merged.redact?.patterns ?? []).map((pattern) => (pattern instanceof RegExp ? stateless(pattern) : pattern)),
|
|
147
|
+
// LOUPE_REDACT_ENV adds to the configured names: a higher layer never removes one.
|
|
148
|
+
env: [...new Set([...(merged.redact?.env ?? []), ...redactEnv.names])],
|
|
149
|
+
},
|
|
118
150
|
};
|
|
119
151
|
}
|
package/dist/config/types.d.ts
CHANGED
|
@@ -4,6 +4,8 @@
|
|
|
4
4
|
*/
|
|
5
5
|
export declare const CATEGORY_MODES: readonly ['extend', 'replace'];
|
|
6
6
|
export type CategoryMode = (typeof CATEGORY_MODES)[number];
|
|
7
|
+
export declare const REDACT_MODES: readonly ['extend', 'replace'];
|
|
8
|
+
export type RedactMode = (typeof REDACT_MODES)[number];
|
|
7
9
|
export declare const CATEGORY_TARGETS: readonly ['message', 'stack', 'step', 'all'];
|
|
8
10
|
export type CategoryTarget = (typeof CATEGORY_TARGETS)[number];
|
|
9
11
|
export declare const SCREENSHOT_VIEWS: readonly ['side-by-side', 'slider', 'overlay', 'diff'];
|
|
@@ -53,6 +55,18 @@ export interface DimensionsConfig {
|
|
|
53
55
|
auto?: boolean;
|
|
54
56
|
definitions?: DimensionConfig[];
|
|
55
57
|
}
|
|
58
|
+
export interface RedactConfig {
|
|
59
|
+
/**
|
|
60
|
+
* Secret-looking names, for input step targets and source lines, and for
|
|
61
|
+
* HTTP header names. A string matches whole words (`auth` matches
|
|
62
|
+
* `x-auth-token`, not `Author`); a RegExp is tested as is.
|
|
63
|
+
*/
|
|
64
|
+
patterns?: Array<string | RegExp>;
|
|
65
|
+
/** `extend` (default): `patterns` add to the default names. `replace`: only `patterns`; `[]` turns name-based redaction off. */
|
|
66
|
+
mode?: RedactMode;
|
|
67
|
+
/** Names of environment variables whose values are redacted everywhere in the report. Names only, never values. */
|
|
68
|
+
env?: string[];
|
|
69
|
+
}
|
|
56
70
|
export interface LoupeConfig {
|
|
57
71
|
/** Folder the report is written to, relative to the Playwright config file. Default: `loupe-report`. */
|
|
58
72
|
outputFolder?: string;
|
|
@@ -72,6 +86,8 @@ export interface LoupeConfig {
|
|
|
72
86
|
/** Include each error's stack trace in the copyable AI prompt. Default: `true`. */
|
|
73
87
|
includeStack?: boolean;
|
|
74
88
|
};
|
|
89
|
+
/** Secret redaction in report.json and index.html. `LOUPE_REDACT_ENV` adds variable names. */
|
|
90
|
+
redact?: RedactConfig;
|
|
75
91
|
}
|
|
76
92
|
/** Inline reporter options: a config, plus the path of the config file. */
|
|
77
93
|
export interface LoupeReporterOptions extends LoupeConfig {
|
|
@@ -121,4 +137,10 @@ export interface ResolvedConfig {
|
|
|
121
137
|
aiPrompt: {
|
|
122
138
|
includeStack: boolean;
|
|
123
139
|
};
|
|
140
|
+
/** RegExps without the `g` and `y` flags; `env` merged from every layer, without duplicates. */
|
|
141
|
+
redact: {
|
|
142
|
+
mode: RedactMode;
|
|
143
|
+
patterns: Array<string | RegExp>;
|
|
144
|
+
env: string[];
|
|
145
|
+
};
|
|
124
146
|
}
|
package/dist/config/types.js
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
* reporter options. Every option is optional; see docs/config.md.
|
|
4
4
|
*/
|
|
5
5
|
export const CATEGORY_MODES = ['extend', 'replace'];
|
|
6
|
+
export const REDACT_MODES = ['extend', 'replace'];
|
|
6
7
|
export const CATEGORY_TARGETS = ['message', 'stack', 'step', 'all'];
|
|
7
8
|
export const SCREENSHOT_VIEWS = ['side-by-side', 'slider', 'overlay', 'diff'];
|
|
8
9
|
export const THEMES = ['system', 'light', 'dark'];
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
/** An environment variable name, as `redact.env` and `LOUPE_REDACT_ENV` take them. */
|
|
2
|
+
export declare const ENV_NAME: RegExp;
|
|
1
3
|
/** Optimal string alignment distance: Levenshtein plus adjacent transpositions (`titel` → `title` is 1). */
|
|
2
4
|
export declare function editDistance(a: string, b: string): number;
|
|
3
5
|
/** The closest known key, when it is close enough to be a typo. */
|
package/dist/config/validate.js
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import { PALETTE_NAMES } from '../shared/palette.js';
|
|
2
|
-
import { CATEGORY_MODES, CATEGORY_TARGETS, SCREENSHOT_VIEWS, THEMES } from './types.js';
|
|
2
|
+
import { CATEGORY_MODES, CATEGORY_TARGETS, REDACT_MODES, SCREENSHOT_VIEWS, THEMES } from './types.js';
|
|
3
3
|
const COLOR = /^#[0-9a-fA-F]{6}$/;
|
|
4
|
+
/** An environment variable name, as `redact.env` and `LOUPE_REDACT_ENV` take them. */
|
|
5
|
+
export const ENV_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/;
|
|
4
6
|
/** What a tag prefix cannot contain: `@layer:api` has the prefix `layer`. */
|
|
5
7
|
const INVALID_PREFIX = /[@:\s]/;
|
|
6
8
|
function describe(value) {
|
|
@@ -151,6 +153,37 @@ function values(ctx) {
|
|
|
151
153
|
});
|
|
152
154
|
};
|
|
153
155
|
}
|
|
156
|
+
/** Secret-looking names: non-empty strings or RegExps. An empty list is valid (with `mode: 'replace'`, it turns them off). */
|
|
157
|
+
function namePatterns(ctx) {
|
|
158
|
+
return (value, path, errors) => {
|
|
159
|
+
if (!Array.isArray(value))
|
|
160
|
+
return fail(ctx, path, 'an array of strings or RegExps', value, errors);
|
|
161
|
+
value.forEach((item, index) => {
|
|
162
|
+
if (item instanceof RegExp)
|
|
163
|
+
return;
|
|
164
|
+
if (typeof item !== 'string' || item.trim() === '')
|
|
165
|
+
fail(ctx, `${path}[${index}]`, 'a non-empty string or a RegExp', item, errors);
|
|
166
|
+
});
|
|
167
|
+
};
|
|
168
|
+
}
|
|
169
|
+
/** Environment variable names. A value with other characters is most likely a secret pasted by mistake: it is not echoed. */
|
|
170
|
+
function envNames(ctx) {
|
|
171
|
+
return (value, path, errors) => {
|
|
172
|
+
if (!Array.isArray(value))
|
|
173
|
+
return fail(ctx, path, 'an array of environment variable names', value, errors);
|
|
174
|
+
const seen = new Set();
|
|
175
|
+
value.forEach((item, index) => {
|
|
176
|
+
if (typeof item !== 'string' || !ENV_NAME.test(item)) {
|
|
177
|
+
const got = typeof item === 'string' ? 'a string that is not a variable name (value not shown)' : describe(item);
|
|
178
|
+
errors.push(`${ctx.source}: ${path}[${index}] must be an environment variable name (letters, digits, _), got ${got}`);
|
|
179
|
+
return;
|
|
180
|
+
}
|
|
181
|
+
if (seen.has(item))
|
|
182
|
+
errors.push(`${ctx.source}: ${path}[${index}] repeats the variable ${item}`);
|
|
183
|
+
seen.add(item);
|
|
184
|
+
});
|
|
185
|
+
};
|
|
186
|
+
}
|
|
154
187
|
function tagPrefix(ctx) {
|
|
155
188
|
return (value, path, errors) => {
|
|
156
189
|
if (typeof value !== 'string' || value === '' || INVALID_PREFIX.test(value)) {
|
|
@@ -228,6 +261,7 @@ export function validateConfig(value, source, options = {}) {
|
|
|
228
261
|
screenshots: object({ defaultView: oneOf(SCREENSHOT_VIEWS, ctx) }, ctx),
|
|
229
262
|
theme: oneOf(THEMES, ctx),
|
|
230
263
|
aiPrompt: object({ includeStack: boolean(ctx) }, ctx),
|
|
264
|
+
redact: object({ patterns: namePatterns(ctx), mode: oneOf(REDACT_MODES, ctx), env: envNames(ctx) }, ctx),
|
|
231
265
|
};
|
|
232
266
|
if (options.allowConfigFile)
|
|
233
267
|
fields['configFile'] = nonEmptyString(ctx);
|
package/dist/core/build.d.ts
CHANGED
|
@@ -2,6 +2,7 @@ import type { FullConfig, FullResult, Suite, TestError, TestStep } from '@playwr
|
|
|
2
2
|
import type { LoupeReport, RunScope } from '../schema/v1.js';
|
|
3
3
|
import { type CategoriesOptions, type ErrorText } from './categories.js';
|
|
4
4
|
import { type DimensionsOptions } from './dimensions.js';
|
|
5
|
+
import { type RedactOptions } from './redact.js';
|
|
5
6
|
/** Folder, inside the report folder, that holds the copied attachments. */
|
|
6
7
|
export declare const DATA_FOLDER = "data";
|
|
7
8
|
export interface BuildOptions {
|
|
@@ -20,12 +21,14 @@ export interface BuildOptions {
|
|
|
20
21
|
error: TestError;
|
|
21
22
|
project: string | null;
|
|
22
23
|
}>;
|
|
24
|
+
/** Secret redaction; default: the built-in names, no env values. */
|
|
25
|
+
redact?: RedactOptions;
|
|
23
26
|
}
|
|
24
27
|
/**
|
|
25
28
|
* Titles of the failing steps leading to `error`, outermost first: the path to
|
|
26
29
|
* the deepest step that ended with the same message. Empty when no step did.
|
|
27
30
|
*/
|
|
28
|
-
export declare function failingSteps(steps: readonly TestStep[], error: TestError): string[];
|
|
31
|
+
export declare function failingSteps(steps: readonly TestStep[], error: TestError, title?: (step: TestStep) => string): string[];
|
|
29
32
|
/**
|
|
30
33
|
* Text an error is categorized on. `steps`: titles of the failing steps, outermost first.
|
|
31
34
|
* Run errors lose the `merge-reports` tag prefix, so rules match the same text merged or not.
|
package/dist/core/build.js
CHANGED
|
@@ -8,6 +8,7 @@ import { detectEnvironment, detectGit } from './metadata.js';
|
|
|
8
8
|
import { collectOutput } from './output.js';
|
|
9
9
|
import { esmImportLocation, isUsableLocation } from './runErrors.js';
|
|
10
10
|
import { detectScopeReasons, resolveScope } from './scope.js';
|
|
11
|
+
import { DEFAULT_REDACT, createRedactor, mapReportStrings } from './redact.js';
|
|
11
12
|
import { errorSignature, stripAnsi, stripMergeTagPrefix } from './signature.js';
|
|
12
13
|
/** Folder, inside the report folder, that holds the copied attachments. */
|
|
13
14
|
export const DATA_FOLDER = 'data';
|
|
@@ -119,20 +120,20 @@ function isFocused(test) {
|
|
|
119
120
|
* Titles of the failing steps leading to `error`, outermost first: the path to
|
|
120
121
|
* the deepest step that ended with the same message. Empty when no step did.
|
|
121
122
|
*/
|
|
122
|
-
export function failingSteps(steps, error) {
|
|
123
|
+
export function failingSteps(steps, error, title = (step) => step.title) {
|
|
123
124
|
const message = error.message ?? error.value;
|
|
124
125
|
if (message === undefined)
|
|
125
126
|
return [];
|
|
126
127
|
for (const step of [...steps].reverse()) {
|
|
127
128
|
if (!step.error || (step.error.message ?? step.error.value) !== message)
|
|
128
129
|
continue;
|
|
129
|
-
return [step
|
|
130
|
+
return [title(step), ...failingSteps(step.steps, error, title)];
|
|
130
131
|
}
|
|
131
132
|
// A step may fail with a wrapped message while one of its children has the exact one.
|
|
132
133
|
for (const step of [...steps].reverse()) {
|
|
133
|
-
const inner = failingSteps(step.steps, error);
|
|
134
|
+
const inner = failingSteps(step.steps, error, title);
|
|
134
135
|
if (inner.length > 0)
|
|
135
|
-
return [step
|
|
136
|
+
return [title(step), ...inner];
|
|
136
137
|
}
|
|
137
138
|
return [];
|
|
138
139
|
}
|
|
@@ -151,6 +152,7 @@ export async function buildReport(config, suite, result, options) {
|
|
|
151
152
|
const rootDir = config.rootDir;
|
|
152
153
|
const store = new AttachmentStore(options.outputFolder);
|
|
153
154
|
const categorizer = createCategorizer(options.categories ?? DEFAULT_CATEGORIES);
|
|
155
|
+
const redactor = createRedactor(options.redact ?? DEFAULT_REDACT);
|
|
154
156
|
const dimensionSet = createDimensions(options.dimensions ?? DEFAULT_DIMENSIONS);
|
|
155
157
|
const categories = new Set();
|
|
156
158
|
const usedDimensions = new Set();
|
|
@@ -161,10 +163,11 @@ export async function buildReport(config, suite, result, options) {
|
|
|
161
163
|
/**
|
|
162
164
|
* `steps`: titles of the failing steps leading to the error, outermost first.
|
|
163
165
|
* `runError`: the `merge-reports` tag prefix is ignored by categories and the signature, kept in `message` and `stack`.
|
|
166
|
+
* `redact`: applied to the message and the stack before the signature and the category, so they never depend on a secret.
|
|
164
167
|
*/
|
|
165
|
-
const toError = (error, steps = [], location = error.location, runError = false) => {
|
|
166
|
-
const message = stripAnsi(error.message ?? error.value ?? '');
|
|
167
|
-
const stack = error.stack ? stripAnsi(error.stack) : null;
|
|
168
|
+
const toError = (error, redact, steps = [], location = error.location, runError = false) => {
|
|
169
|
+
const message = redact(stripAnsi(error.message ?? error.value ?? ''));
|
|
170
|
+
const stack = error.stack ? redact(stripAnsi(error.stack)) : null;
|
|
168
171
|
const category = categorizer.categorize(errorText(message, stack, steps, runError));
|
|
169
172
|
categories.add(category);
|
|
170
173
|
return {
|
|
@@ -176,17 +179,21 @@ export async function buildReport(config, suite, result, options) {
|
|
|
176
179
|
category,
|
|
177
180
|
};
|
|
178
181
|
};
|
|
179
|
-
const toStep = (step, parents) => {
|
|
180
|
-
const
|
|
182
|
+
const toStep = (step, parents, redact) => {
|
|
183
|
+
const title = redact.title(step);
|
|
184
|
+
const chain = [...parents, title];
|
|
185
|
+
// `subtitle` exists from Playwright 1.63. Before, the target of an action is in
|
|
186
|
+
// the title on 1.52 (`locator.getByRole('button').click`), and nowhere on
|
|
187
|
+
// 1.53–1.62 (`Click`, `Fill "<value>"`).
|
|
188
|
+
const subtitle = step.subtitle;
|
|
181
189
|
return {
|
|
182
|
-
title
|
|
183
|
-
|
|
184
|
-
subtitle: step.subtitle ?? null,
|
|
190
|
+
title,
|
|
191
|
+
subtitle: subtitle === undefined ? null : redact.text(subtitle),
|
|
185
192
|
category: step.category,
|
|
186
193
|
durationMs: toMs(step.duration),
|
|
187
|
-
error: step.error ? toError(step.error, chain) : null,
|
|
194
|
+
error: step.error ? toError(step.error, redact.text, chain) : null,
|
|
188
195
|
location: toLocation(step.location),
|
|
189
|
-
steps: step.steps.map((child) => toStep(child, chain)),
|
|
196
|
+
steps: step.steps.map((child) => toStep(child, chain, redact)),
|
|
190
197
|
};
|
|
191
198
|
};
|
|
192
199
|
const toAttempt = async (attempt, folder) => {
|
|
@@ -194,14 +201,15 @@ export async function buildReport(config, suite, result, options) {
|
|
|
194
201
|
for (const attachment of attempt.attachments) {
|
|
195
202
|
attachments.push(await store.add(`${folder}/${attempt.retry}`, attachment));
|
|
196
203
|
}
|
|
204
|
+
const redact = redactor.attempt(attempt.steps);
|
|
197
205
|
return {
|
|
198
206
|
index: attempt.retry,
|
|
199
207
|
status: attempt.status,
|
|
200
208
|
startedAt: attempt.startTime.toISOString(),
|
|
201
209
|
durationMs: toMs(attempt.duration),
|
|
202
210
|
workerIndex: attempt.workerIndex,
|
|
203
|
-
errors: attempt.errors.map((error) => toError(error, failingSteps(attempt.steps, error))),
|
|
204
|
-
steps: attempt.steps.map((step) => toStep(step, [])),
|
|
211
|
+
errors: attempt.errors.map((error) => toError(error, redact.text, failingSteps(attempt.steps, error, redact.title))),
|
|
212
|
+
steps: attempt.steps.map((step) => toStep(step, [], redact)),
|
|
205
213
|
attachments,
|
|
206
214
|
annotations: attempt.annotations.map(toAnnotation),
|
|
207
215
|
stdout: collectOutput(attempt.stdout),
|
|
@@ -292,9 +300,9 @@ export async function buildReport(config, suite, result, options) {
|
|
|
292
300
|
for (const { error, project } of options.runErrors ?? []) {
|
|
293
301
|
const message = stripAnsi(error.message ?? error.value ?? '');
|
|
294
302
|
const location = isUsableLocation(error.location) ? error.location : ((await esmImportLocation(message)) ?? undefined);
|
|
295
|
-
errors.push({ ...toError(error, [], location, true), project });
|
|
303
|
+
errors.push({ ...toError(error, redactor.text, [], location, true), project });
|
|
296
304
|
}
|
|
297
|
-
|
|
305
|
+
const report = {
|
|
298
306
|
schemaVersion: 1,
|
|
299
307
|
generator: { name: 'playwright-loupe', version: options.generatorVersion },
|
|
300
308
|
run: {
|
|
@@ -318,4 +326,6 @@ export async function buildReport(config, suite, result, options) {
|
|
|
318
326
|
categories: categorizer.definitions(categories),
|
|
319
327
|
dimensions: dimensionSet.definitions(usedDimensions),
|
|
320
328
|
};
|
|
329
|
+
// URL parameters and env values (redact.env) are redacted in every other string: test titles, annotations, outputs, attachment bodies...
|
|
330
|
+
return redactor.rewritesStrings ? mapReportStrings(report, redactor.strings) : report;
|
|
321
331
|
}
|
package/dist/core/categories.js
CHANGED
|
@@ -1,14 +1,7 @@
|
|
|
1
1
|
import { NEUTRAL_ENTRY, PALETTE_ENTRIES, paletteReference } from '../shared/palette.js';
|
|
2
|
+
import { httpStatus } from './http.js';
|
|
2
3
|
const ALL_TARGETS = ['message', 'stack', 'step'];
|
|
3
4
|
export const UNCATEGORIZED_COLOR = NEUTRAL_ENTRY.reference;
|
|
4
|
-
/**
|
|
5
|
-
* An HTTP status in context: `status 429`, `status code: 429`, `"status": 429`,
|
|
6
|
-
* `statusCode=429`, `HTTP 429`, `HTTP/1.1 429`, `Response status: 429`, or
|
|
7
|
-
* `← 429`, the response line of Playwright's fetch log.
|
|
8
|
-
*/
|
|
9
|
-
function httpStatus(code) {
|
|
10
|
-
return new RegExp(`(?:\\b(?:status(?:[ _]?code)?["']?\\s*[:=]?|HTTP(?:/\\d(?:\\.\\d)?)?:?)|\u2190)\\s*${code}(?!\\w|\\.\\d)`, 'i');
|
|
11
|
-
}
|
|
12
5
|
export const BUILT_IN_RULES = [
|
|
13
6
|
{
|
|
14
7
|
// Before "Locator not found", "Assertion" and "Timeout": a locator matching
|
|
@@ -30,7 +23,7 @@ export const BUILT_IN_RULES = [
|
|
|
30
23
|
// Before "Network", "Assertion" and "Timeout": the status is the diagnosis,
|
|
31
24
|
// whether it surfaces through `failOnStatusCode`, `toBeOK()` or a thrown
|
|
32
25
|
// error. A bare 429 is not enough (`429ms`, `Received: 429`): it needs an
|
|
33
|
-
// HTTP context.
|
|
26
|
+
// HTTP context (src/core/http.ts), shared with error signatures.
|
|
34
27
|
name: 'Rate limited',
|
|
35
28
|
color: paletteReference('teal'),
|
|
36
29
|
match: [/\bToo Many Requests\b/i, /\brate[- ]?limit/i, httpStatus('429')],
|
|
@@ -39,10 +32,8 @@ export const BUILT_IN_RULES = [
|
|
|
39
32
|
{
|
|
40
33
|
name: 'Server error',
|
|
41
34
|
color: paletteReference('orange'),
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
/\b5\d\d (?:Internal Server Error|Not Implemented|Bad Gateway|Service Unavailable|Gateway Time-?out|HTTP Version Not Supported)\b/i,
|
|
45
|
-
],
|
|
35
|
+
// In an HTTP context, reason phrase included: `500 Internal Server Error` (failOnStatusCode).
|
|
36
|
+
match: [httpStatus('5\\d\\d')],
|
|
46
37
|
targets: ['message', 'stack'],
|
|
47
38
|
},
|
|
48
39
|
{
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* HTTP status codes in context, shared by the "Rate limited" and "Server
|
|
3
|
+
* error" categories and by error signatures, so both agree on what an HTTP
|
|
4
|
+
* status is. See "Error categories" and "Error signature" in docs/schema/v1.md.
|
|
5
|
+
*
|
|
6
|
+
* A code is in an HTTP context when it is:
|
|
7
|
+
* - preceded by `status`, `status code`, `statusCode`, `"status":`, `HTTP`,
|
|
8
|
+
* `HTTP/1.1`, `error`, or `←` (the response line of Playwright's fetch log),
|
|
9
|
+
* with an optional `:` or `=`: `status 429`, `Response status: 503`,
|
|
10
|
+
* `HTTP 500`, `statusCode=502`, `with error 401`;
|
|
11
|
+
* - or followed by its reason phrase: `401 Unauthorized`, `500 Internal Server Error`.
|
|
12
|
+
*
|
|
13
|
+
* A bare number is not enough: `Timeout 429ms exceeded` and `Expected: 200
|
|
14
|
+
* Received: 500` have no HTTP context.
|
|
15
|
+
*/
|
|
16
|
+
/** Any status code from 100 to 599. */
|
|
17
|
+
export declare const STATUS_CODE = "[1-5]\\d\\d";
|
|
18
|
+
/** Source of a pattern matching `code` (a regex source such as `429` or `5\\d\\d`) in an HTTP context, case-insensitive. */
|
|
19
|
+
export declare function httpStatusSource(code: string): string;
|
|
20
|
+
/** `code` in an HTTP context. */
|
|
21
|
+
export declare function httpStatus(code: string): RegExp;
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* HTTP status codes in context, shared by the "Rate limited" and "Server
|
|
3
|
+
* error" categories and by error signatures, so both agree on what an HTTP
|
|
4
|
+
* status is. See "Error categories" and "Error signature" in docs/schema/v1.md.
|
|
5
|
+
*
|
|
6
|
+
* A code is in an HTTP context when it is:
|
|
7
|
+
* - preceded by `status`, `status code`, `statusCode`, `"status":`, `HTTP`,
|
|
8
|
+
* `HTTP/1.1`, `error`, or `←` (the response line of Playwright's fetch log),
|
|
9
|
+
* with an optional `:` or `=`: `status 429`, `Response status: 503`,
|
|
10
|
+
* `HTTP 500`, `statusCode=502`, `with error 401`;
|
|
11
|
+
* - or followed by its reason phrase: `401 Unauthorized`, `500 Internal Server Error`.
|
|
12
|
+
*
|
|
13
|
+
* A bare number is not enough: `Timeout 429ms exceeded` and `Expected: 200
|
|
14
|
+
* Received: 500` have no HTTP context.
|
|
15
|
+
*/
|
|
16
|
+
const PREFIX = `(?:\\b(?:status(?:[ _]?code)?["']?\\s*[:=]?|HTTP(?:/\\d(?:\\.\\d)?)?:?|error\\s*[:=]?)|←)`;
|
|
17
|
+
/** Standard reason phrases (RFC 9110, plus 429). */
|
|
18
|
+
const REASON_PHRASES = [
|
|
19
|
+
'Continue',
|
|
20
|
+
'Switching Protocols',
|
|
21
|
+
'OK',
|
|
22
|
+
'Created',
|
|
23
|
+
'Accepted',
|
|
24
|
+
'No Content',
|
|
25
|
+
'Moved Permanently',
|
|
26
|
+
'Found',
|
|
27
|
+
'See Other',
|
|
28
|
+
'Not Modified',
|
|
29
|
+
'Temporary Redirect',
|
|
30
|
+
'Permanent Redirect',
|
|
31
|
+
'Bad Request',
|
|
32
|
+
'Unauthorized',
|
|
33
|
+
'Payment Required',
|
|
34
|
+
'Forbidden',
|
|
35
|
+
'Not Found',
|
|
36
|
+
'Method Not Allowed',
|
|
37
|
+
'Not Acceptable',
|
|
38
|
+
'Request Time-?out',
|
|
39
|
+
'Conflict',
|
|
40
|
+
'Gone',
|
|
41
|
+
'Precondition Failed',
|
|
42
|
+
'Payload Too Large',
|
|
43
|
+
'Content Too Large',
|
|
44
|
+
'Unsupported Media Type',
|
|
45
|
+
'Unprocessable Entity',
|
|
46
|
+
'Unprocessable Content',
|
|
47
|
+
'Too Many Requests',
|
|
48
|
+
'Internal Server Error',
|
|
49
|
+
'Not Implemented',
|
|
50
|
+
'Bad Gateway',
|
|
51
|
+
'Service Unavailable',
|
|
52
|
+
'Gateway Time-?out',
|
|
53
|
+
'HTTP Version Not Supported',
|
|
54
|
+
];
|
|
55
|
+
/** Any status code from 100 to 599. */
|
|
56
|
+
export const STATUS_CODE = '[1-5]\\d\\d';
|
|
57
|
+
/** Source of a pattern matching `code` (a regex source such as `429` or `5\\d\\d`) in an HTTP context, case-insensitive. */
|
|
58
|
+
export function httpStatusSource(code) {
|
|
59
|
+
return `${PREFIX}\\s*${code}(?!\\w|\\.\\d)|\\b${code} (?:${REASON_PHRASES.join('|')})\\b`;
|
|
60
|
+
}
|
|
61
|
+
/** `code` in an HTTP context. */
|
|
62
|
+
export function httpStatus(code) {
|
|
63
|
+
return new RegExp(httpStatusSource(code), 'i');
|
|
64
|
+
}
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
import type { RedactMode } from '../config/types.js';
|
|
2
|
+
/**
|
|
3
|
+
* Secret redaction, applied by the core before signatures and categories are
|
|
4
|
+
* computed and before anything is written. Three mechanisms (see "Secret
|
|
5
|
+
* redaction" in docs/config.md):
|
|
6
|
+
*
|
|
7
|
+
* - A, input steps: the value typed by `fill`, `type`, `pressSequentially`,
|
|
8
|
+
* `keyboard.type` or `keyboard.insertText` is replaced when the target
|
|
9
|
+
* (step subtitle, from Playwright 1.63) or the source line of the call has a
|
|
10
|
+
* secret-looking name. The same value is then redacted in the attempt's
|
|
11
|
+
* errors (Playwright repeats it in the call log: `fill("<value>")`).
|
|
12
|
+
* - C, HTTP headers and URL parameters: values of header lines with a
|
|
13
|
+
* secret-looking name, as Playwright prints them in call logs
|
|
14
|
+
* (`- cookie: <value>`), and of URL query or fragment parameters with a
|
|
15
|
+
* secret-looking name (`?token=<value>`), wherever URLs appear.
|
|
16
|
+
* - B, environment values: the values of the variables listed in
|
|
17
|
+
* `redact.env`, everywhere in the report except snippets and attachment paths.
|
|
18
|
+
*
|
|
19
|
+
* A and C share one list of secret-looking names. Attachment files (traces,
|
|
20
|
+
* screenshots, videos) are never altered.
|
|
21
|
+
*/
|
|
22
|
+
/** What replaces a secret. */
|
|
23
|
+
export declare const REDACTED = "[redacted]";
|
|
24
|
+
/** Values shorter than this are only redacted in quoted form (A), or not used at all (B): too many false matches. */
|
|
25
|
+
export declare const MIN_SECRET_LENGTH = 6;
|
|
26
|
+
/**
|
|
27
|
+
* Default secret-looking names, for A (locators, source lines) and C (header
|
|
28
|
+
* and URL parameter names). A string matches whole words, see `createNameMatcher`; a RegExp is
|
|
29
|
+
* tested as is.
|
|
30
|
+
*/
|
|
31
|
+
export declare const DEFAULT_SECRET_NAMES: ReadonlyArray<string | RegExp>;
|
|
32
|
+
/**
|
|
33
|
+
* Lowercase words of a name or a line: split on anything but ASCII letters and
|
|
34
|
+
* digits, and on camelCase boundaries (`passwordInput` → password, input;
|
|
35
|
+
* `APIKey` → api, key).
|
|
36
|
+
*/
|
|
37
|
+
export declare function words(text: string): string[];
|
|
38
|
+
/**
|
|
39
|
+
* Tells whether a text has a secret-looking name. A string pattern matches
|
|
40
|
+
* consecutive whole words, case-insensitively, each word allowing a trailing
|
|
41
|
+
* "s": `auth` matches `x-auth-token` and `authToken` but not `Author`; `pin`
|
|
42
|
+
* matches `PIN code` but not `spinner`; `api key` matches `x-api-key`, `apiKey`
|
|
43
|
+
* and `API_KEY`. A RegExp is tested on the text as is.
|
|
44
|
+
*/
|
|
45
|
+
export declare function createNameMatcher(patterns: ReadonlyArray<string | RegExp>): (text: string) => boolean;
|
|
46
|
+
export interface InputStep {
|
|
47
|
+
/** The title with the value replaced by `[redacted]`. */
|
|
48
|
+
redacted: string;
|
|
49
|
+
value: string;
|
|
50
|
+
/** Target written in the title (Playwright 1.52), null otherwise. */
|
|
51
|
+
target: string | null;
|
|
52
|
+
}
|
|
53
|
+
export declare function parseInputStep(title: string): InputStep | null;
|
|
54
|
+
/** The part of a Playwright step redaction reads. */
|
|
55
|
+
export interface StepLike {
|
|
56
|
+
title: string;
|
|
57
|
+
category: string;
|
|
58
|
+
location?: {
|
|
59
|
+
file: string;
|
|
60
|
+
line: number;
|
|
61
|
+
} | undefined;
|
|
62
|
+
steps: readonly StepLike[];
|
|
63
|
+
}
|
|
64
|
+
export interface RedactOptions {
|
|
65
|
+
/** `extend`: `patterns` add to the default names. `replace`: only `patterns`; `[]` turns A and C off. */
|
|
66
|
+
mode: RedactMode;
|
|
67
|
+
patterns: ReadonlyArray<string | RegExp>;
|
|
68
|
+
/** Values of the `redact.env` variables, already filtered by `secretValues`. */
|
|
69
|
+
values: readonly string[];
|
|
70
|
+
/** Reads a source file; null when it cannot be read. Default: from disk. */
|
|
71
|
+
readSource?: (file: string) => string | null;
|
|
72
|
+
}
|
|
73
|
+
export declare const DEFAULT_REDACT: RedactOptions;
|
|
74
|
+
/** Redaction for the steps and errors of one attempt. */
|
|
75
|
+
export interface AttemptRedactor {
|
|
76
|
+
/** A step title: input value (A), values typed in this attempt, URL parameters (C), env values (B). */
|
|
77
|
+
title(step: StepLike): string;
|
|
78
|
+
/** An error message or stack, a subtitle: values typed in this attempt, header values and URL parameters (C), env values (B). */
|
|
79
|
+
text(text: string): string;
|
|
80
|
+
}
|
|
81
|
+
export interface Redactor {
|
|
82
|
+
/** Finds the secret input steps of an attempt (A) and returns the redaction for its steps and errors. */
|
|
83
|
+
attempt(steps: readonly StepLike[]): AttemptRedactor;
|
|
84
|
+
/** Text outside any attempt (run errors): header values and URL parameters (C), env values (B). */
|
|
85
|
+
text(text: string): string;
|
|
86
|
+
/** Env values only (B). */
|
|
87
|
+
values(text: string): string;
|
|
88
|
+
/** URL parameters (C) and env values (B), for every other string of the report: outputs, annotations, test titles... */
|
|
89
|
+
strings(text: string): string;
|
|
90
|
+
/** False when `strings` never changes anything (no names and no env values): the report needs no rewriting pass. */
|
|
91
|
+
rewritesStrings: boolean;
|
|
92
|
+
}
|
|
93
|
+
export declare function createRedactor(options?: RedactOptions): Redactor;
|
|
94
|
+
/**
|
|
95
|
+
* Values of the `redact.env` variables. A variable that is unset or empty, or
|
|
96
|
+
* whose value is shorter than MIN_SECRET_LENGTH, is left out with a warning
|
|
97
|
+
* that names the variable, never its value.
|
|
98
|
+
*/
|
|
99
|
+
export declare function secretValues(names: readonly string[], env: Readonly<Record<string, string | undefined>>, warn: (message: string) => void): string[];
|
|
100
|
+
/** Applies `replace` to every string of a JSON value, except under KEPT_KEYS. Returns a copy. */
|
|
101
|
+
export declare function mapReportStrings<T>(value: T, replace: (text: string) => string): T;
|
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
import { readFileSync } from 'node:fs';
|
|
2
|
+
/**
|
|
3
|
+
* Secret redaction, applied by the core before signatures and categories are
|
|
4
|
+
* computed and before anything is written. Three mechanisms (see "Secret
|
|
5
|
+
* redaction" in docs/config.md):
|
|
6
|
+
*
|
|
7
|
+
* - A, input steps: the value typed by `fill`, `type`, `pressSequentially`,
|
|
8
|
+
* `keyboard.type` or `keyboard.insertText` is replaced when the target
|
|
9
|
+
* (step subtitle, from Playwright 1.63) or the source line of the call has a
|
|
10
|
+
* secret-looking name. The same value is then redacted in the attempt's
|
|
11
|
+
* errors (Playwright repeats it in the call log: `fill("<value>")`).
|
|
12
|
+
* - C, HTTP headers and URL parameters: values of header lines with a
|
|
13
|
+
* secret-looking name, as Playwright prints them in call logs
|
|
14
|
+
* (`- cookie: <value>`), and of URL query or fragment parameters with a
|
|
15
|
+
* secret-looking name (`?token=<value>`), wherever URLs appear.
|
|
16
|
+
* - B, environment values: the values of the variables listed in
|
|
17
|
+
* `redact.env`, everywhere in the report except snippets and attachment paths.
|
|
18
|
+
*
|
|
19
|
+
* A and C share one list of secret-looking names. Attachment files (traces,
|
|
20
|
+
* screenshots, videos) are never altered.
|
|
21
|
+
*/
|
|
22
|
+
/** What replaces a secret. */
|
|
23
|
+
export const REDACTED = '[redacted]';
|
|
24
|
+
/** Values shorter than this are only redacted in quoted form (A), or not used at all (B): too many false matches. */
|
|
25
|
+
export const MIN_SECRET_LENGTH = 6;
|
|
26
|
+
/**
|
|
27
|
+
* Default secret-looking names, for A (locators, source lines) and C (header
|
|
28
|
+
* and URL parameter names). A string matches whole words, see `createNameMatcher`; a RegExp is
|
|
29
|
+
* tested as is.
|
|
30
|
+
*/
|
|
31
|
+
export const DEFAULT_SECRET_NAMES = [
|
|
32
|
+
// As a substring: newPassword, passwordConfirm, Passphrase, passwd.
|
|
33
|
+
/pass(?:word|wd|phrase)/i,
|
|
34
|
+
'pwd',
|
|
35
|
+
'passcode',
|
|
36
|
+
'secret',
|
|
37
|
+
'token',
|
|
38
|
+
'otp',
|
|
39
|
+
'totp',
|
|
40
|
+
'pin',
|
|
41
|
+
'api key',
|
|
42
|
+
'apikey',
|
|
43
|
+
'private key',
|
|
44
|
+
'credential',
|
|
45
|
+
'cookie',
|
|
46
|
+
'session',
|
|
47
|
+
'authorization',
|
|
48
|
+
'auth',
|
|
49
|
+
'bearer',
|
|
50
|
+
'jwt',
|
|
51
|
+
'csrf',
|
|
52
|
+
'xsrf',
|
|
53
|
+
];
|
|
54
|
+
/**
|
|
55
|
+
* Lowercase words of a name or a line: split on anything but ASCII letters and
|
|
56
|
+
* digits, and on camelCase boundaries (`passwordInput` → password, input;
|
|
57
|
+
* `APIKey` → api, key).
|
|
58
|
+
*/
|
|
59
|
+
export function words(text) {
|
|
60
|
+
return text
|
|
61
|
+
.replace(/([a-z0-9])([A-Z])/g, '$1 $2')
|
|
62
|
+
.replace(/([A-Z]+)([A-Z][a-z])/g, '$1 $2')
|
|
63
|
+
.toLowerCase()
|
|
64
|
+
.split(/[^a-z0-9]+/)
|
|
65
|
+
.filter((word) => word !== '');
|
|
66
|
+
}
|
|
67
|
+
/**
|
|
68
|
+
* Tells whether a text has a secret-looking name. A string pattern matches
|
|
69
|
+
* consecutive whole words, case-insensitively, each word allowing a trailing
|
|
70
|
+
* "s": `auth` matches `x-auth-token` and `authToken` but not `Author`; `pin`
|
|
71
|
+
* matches `PIN code` but not `spinner`; `api key` matches `x-api-key`, `apiKey`
|
|
72
|
+
* and `API_KEY`. A RegExp is tested on the text as is.
|
|
73
|
+
*/
|
|
74
|
+
export function createNameMatcher(patterns) {
|
|
75
|
+
const regexps = patterns.filter((pattern) => pattern instanceof RegExp);
|
|
76
|
+
const sequences = patterns.filter((pattern) => typeof pattern === 'string').map(words).filter((list) => list.length > 0);
|
|
77
|
+
if (regexps.length === 0 && sequences.length === 0)
|
|
78
|
+
return () => false;
|
|
79
|
+
return (text) => {
|
|
80
|
+
if (regexps.some((pattern) => pattern.test(text)))
|
|
81
|
+
return true;
|
|
82
|
+
if (sequences.length === 0)
|
|
83
|
+
return false;
|
|
84
|
+
const textWords = words(text);
|
|
85
|
+
return sequences.some((sequence) => textWords.some((_, start) => sequence.every((word, offset) => {
|
|
86
|
+
const candidate = textWords[start + offset];
|
|
87
|
+
return candidate === word || candidate === `${word}s`;
|
|
88
|
+
})));
|
|
89
|
+
};
|
|
90
|
+
}
|
|
91
|
+
/**
|
|
92
|
+
* Value input steps, as Playwright titles them:
|
|
93
|
+
* - 1.53+: `Fill "<value>"`, `Type "<value>"` (`type`, `pressSequentially`,
|
|
94
|
+
* `keyboard.type`), `Insert "<value>"` (`keyboard.insertText`). Quotes in
|
|
95
|
+
* the value are not escaped, so the value runs to the last quote; newlines
|
|
96
|
+
* are written `\n`.
|
|
97
|
+
* - 1.52: `locator.fill(<value>)`, `locator.getByLabel('Password').type(<value>)`,
|
|
98
|
+
* `locator.getByLabel('Password').pressSequentially(<value>)`,
|
|
99
|
+
* `keyboard.type(<value>)`, `keyboard.insertText(<value>)`. The target is
|
|
100
|
+
* part of the title for `type` and `pressSequentially` only.
|
|
101
|
+
*/
|
|
102
|
+
const INPUT_TITLE = /^(Fill|Type|Insert) "([\s\S]*)"$/;
|
|
103
|
+
const LEGACY_INPUT_TITLE = /^((?:locator|page|frame|keyboard|elementHandle)\b.*?)\.(fill|type|pressSequentially|insertText)\(([\s\S]*)\)$/;
|
|
104
|
+
export function parseInputStep(title) {
|
|
105
|
+
const current = INPUT_TITLE.exec(title);
|
|
106
|
+
if (current)
|
|
107
|
+
return { redacted: `${current[1]} "${REDACTED}"`, value: current[2], target: null };
|
|
108
|
+
const legacy = LEGACY_INPUT_TITLE.exec(title);
|
|
109
|
+
if (legacy) {
|
|
110
|
+
const [, target, action, value] = legacy;
|
|
111
|
+
return { redacted: `${target}.${action}(${REDACTED})`, value, target };
|
|
112
|
+
}
|
|
113
|
+
return null;
|
|
114
|
+
}
|
|
115
|
+
/**
|
|
116
|
+
* A header line as Playwright prints it in a call log (` - cookie: <value>`),
|
|
117
|
+
* or a bare `Name: value` line. Header names are tokens: no spaces.
|
|
118
|
+
*/
|
|
119
|
+
const HEADER_LINE = /^([ \t]*(?:-[ \t]+)?)([A-Za-z0-9!#$%&'*+.^_`|~-]+):[ \t]+\S.*$/gm;
|
|
120
|
+
/**
|
|
121
|
+
* A URL query or fragment parameter: `?token=<value>`, `&access_token=<value>`,
|
|
122
|
+
* `#access_token=<value>` (OAuth implicit flow). Only after `?`, `&` or `#`, so
|
|
123
|
+
* a bare `token=abc` in prose is left alone. The value stops at the next
|
|
124
|
+
* parameter, the fragment, whitespace, a quote, a bracket or a parenthesis.
|
|
125
|
+
*/
|
|
126
|
+
const URL_PARAM = /([?&#])([A-Za-z0-9_.~%-]+)=(\[redacted\]|[^&#\s"'`<>()[\]]+)/g;
|
|
127
|
+
function escapeRegExp(text) {
|
|
128
|
+
return text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
129
|
+
}
|
|
130
|
+
/**
|
|
131
|
+
* Replaces every occurrence of `values` with `[redacted]`, longest first. An
|
|
132
|
+
* existing `[redacted]` is left alone, so applying it twice changes nothing
|
|
133
|
+
* even when a value is part of the word "redacted".
|
|
134
|
+
*/
|
|
135
|
+
function valueReplacer(values) {
|
|
136
|
+
const unique = [...new Set(values)].filter((value) => value !== '').sort((a, b) => b.length - a.length);
|
|
137
|
+
if (unique.length === 0)
|
|
138
|
+
return (text) => text;
|
|
139
|
+
const pattern = new RegExp([escapeRegExp(REDACTED), ...unique.map(escapeRegExp)].join('|'), 'g');
|
|
140
|
+
return (text) => text.replace(pattern, REDACTED);
|
|
141
|
+
}
|
|
142
|
+
export const DEFAULT_REDACT = { mode: 'extend', patterns: [], values: [] };
|
|
143
|
+
function readFromDisk(file) {
|
|
144
|
+
try {
|
|
145
|
+
return readFileSync(file, 'utf8');
|
|
146
|
+
}
|
|
147
|
+
catch {
|
|
148
|
+
return null;
|
|
149
|
+
}
|
|
150
|
+
}
|
|
151
|
+
export function createRedactor(options = DEFAULT_REDACT) {
|
|
152
|
+
const namePatterns = options.mode === 'extend' ? [...DEFAULT_SECRET_NAMES, ...options.patterns] : options.patterns;
|
|
153
|
+
const names = createNameMatcher(namePatterns);
|
|
154
|
+
const envValues = valueReplacer(options.values);
|
|
155
|
+
const readSource = options.readSource ?? readFromDisk;
|
|
156
|
+
const sources = new Map();
|
|
157
|
+
/** The source line of a step, from disk; null when the file cannot be read (merge-reports without the code). */
|
|
158
|
+
const sourceLine = (location) => {
|
|
159
|
+
if (!location)
|
|
160
|
+
return null;
|
|
161
|
+
if (!sources.has(location.file))
|
|
162
|
+
sources.set(location.file, readSource(location.file)?.split(/\r?\n/) ?? null);
|
|
163
|
+
return sources.get(location.file)?.[location.line - 1] ?? null;
|
|
164
|
+
};
|
|
165
|
+
const headers = (text) => text.replace(HEADER_LINE, (line, indent, name) => (names(name) ? `${indent}${name}: ${REDACTED}` : line));
|
|
166
|
+
const urlParams = (text) => text.replace(URL_PARAM, (param, separator, name) => (names(name) ? `${separator}${name}=${REDACTED}` : param));
|
|
167
|
+
/** A: the target (subtitle from 1.63, title on 1.52), else the source line of the call, has a secret-looking name. */
|
|
168
|
+
const isSecret = (step, input) => {
|
|
169
|
+
const subtitle = step.subtitle;
|
|
170
|
+
if (typeof subtitle === 'string' && names(subtitle))
|
|
171
|
+
return true;
|
|
172
|
+
if (input.target !== null && names(input.target))
|
|
173
|
+
return true;
|
|
174
|
+
const line = sourceLine(step.location);
|
|
175
|
+
return line !== null && names(line);
|
|
176
|
+
};
|
|
177
|
+
return {
|
|
178
|
+
rewritesStrings: namePatterns.length > 0 || options.values.length > 0,
|
|
179
|
+
text: (text) => envValues(urlParams(headers(text))),
|
|
180
|
+
values: envValues,
|
|
181
|
+
strings: (text) => envValues(urlParams(text)),
|
|
182
|
+
attempt(steps) {
|
|
183
|
+
const titles = new Map();
|
|
184
|
+
const typed = new Set();
|
|
185
|
+
const visit = (list) => {
|
|
186
|
+
for (const step of list) {
|
|
187
|
+
const input = step.category === 'pw:api' ? parseInputStep(step.title) : null;
|
|
188
|
+
if (input && isSecret(step, input)) {
|
|
189
|
+
titles.set(step, input.redacted);
|
|
190
|
+
typed.add(input.value);
|
|
191
|
+
// Titles from 1.53 write newlines as `\n`; call logs keep the real ones.
|
|
192
|
+
typed.add(input.value.replaceAll('\\n', '\n'));
|
|
193
|
+
}
|
|
194
|
+
visit(step.steps);
|
|
195
|
+
}
|
|
196
|
+
};
|
|
197
|
+
visit(steps);
|
|
198
|
+
// A short value (a PIN) would also match line numbers and other text: only its quoted form is redacted.
|
|
199
|
+
const long = [...typed].filter((value) => value.length >= MIN_SECRET_LENGTH);
|
|
200
|
+
const short = [...typed].filter((value) => value !== '' && value.length < MIN_SECRET_LENGTH).map((value) => `"${value}"`);
|
|
201
|
+
const replaceLong = valueReplacer(long);
|
|
202
|
+
const shortPattern = short.length > 0 ? new RegExp(short.map(escapeRegExp).join('|'), 'g') : null;
|
|
203
|
+
const typedValues = (text) => {
|
|
204
|
+
const replaced = replaceLong(text);
|
|
205
|
+
return shortPattern ? replaced.replace(shortPattern, `"${REDACTED}"`) : replaced;
|
|
206
|
+
};
|
|
207
|
+
return {
|
|
208
|
+
title: (step) => envValues(urlParams(titles.get(step) ?? typedValues(step.title))),
|
|
209
|
+
text: (text) => envValues(urlParams(headers(typedValues(text)))),
|
|
210
|
+
};
|
|
211
|
+
},
|
|
212
|
+
};
|
|
213
|
+
}
|
|
214
|
+
/**
|
|
215
|
+
* Values of the `redact.env` variables. A variable that is unset or empty, or
|
|
216
|
+
* whose value is shorter than MIN_SECRET_LENGTH, is left out with a warning
|
|
217
|
+
* that names the variable, never its value.
|
|
218
|
+
*/
|
|
219
|
+
export function secretValues(names, env, warn) {
|
|
220
|
+
const values = [];
|
|
221
|
+
for (const name of names) {
|
|
222
|
+
const value = env[name];
|
|
223
|
+
if (value === undefined || value === '') {
|
|
224
|
+
warn(`Loupe: redact.env: ${name} is not set or empty; nothing to redact for it.`);
|
|
225
|
+
}
|
|
226
|
+
else if (value.length < MIN_SECRET_LENGTH) {
|
|
227
|
+
warn(`Loupe: redact.env: the value of ${name} is shorter than ${MIN_SECRET_LENGTH} characters and is not redacted (too many false matches).`);
|
|
228
|
+
}
|
|
229
|
+
else {
|
|
230
|
+
values.push(value);
|
|
231
|
+
}
|
|
232
|
+
}
|
|
233
|
+
return values;
|
|
234
|
+
}
|
|
235
|
+
/**
|
|
236
|
+
* Keys whose strings B never rewrites: identifiers and signatures (hex that a
|
|
237
|
+
* short value could match), snippets (source code, already in the
|
|
238
|
+
* repository) and attachment paths (the files keep their names).
|
|
239
|
+
*/
|
|
240
|
+
const KEPT_KEYS = new Set(['id', 'signature', 'errorSignature', 'snippet', 'path']);
|
|
241
|
+
/** Applies `replace` to every string of a JSON value, except under KEPT_KEYS. Returns a copy. */
|
|
242
|
+
export function mapReportStrings(value, replace) {
|
|
243
|
+
const visit = (node) => {
|
|
244
|
+
if (typeof node === 'string')
|
|
245
|
+
return replace(node);
|
|
246
|
+
if (Array.isArray(node))
|
|
247
|
+
return node.map(visit);
|
|
248
|
+
if (node !== null && typeof node === 'object') {
|
|
249
|
+
return Object.fromEntries(Object.entries(node).map(([key, child]) => [key, KEPT_KEYS.has(key) ? child : visit(child)]));
|
|
250
|
+
}
|
|
251
|
+
return node;
|
|
252
|
+
};
|
|
253
|
+
return visit(value);
|
|
254
|
+
}
|
package/dist/core/reporter.d.ts
CHANGED
|
@@ -11,6 +11,8 @@ interface ErrorWorkerInfo {
|
|
|
11
11
|
}
|
|
12
12
|
export declare class LoupeReporter implements Reporter {
|
|
13
13
|
private readonly loupeConfig;
|
|
14
|
+
/** Values of the `redact.env` variables, read once at startup. */
|
|
15
|
+
private readonly secretValues;
|
|
14
16
|
private readonly scopeOverride;
|
|
15
17
|
private readonly runErrors;
|
|
16
18
|
private config;
|
package/dist/core/reporter.js
CHANGED
|
@@ -4,12 +4,15 @@ import path from 'node:path';
|
|
|
4
4
|
import { loadConfig } from '../config/index.js';
|
|
5
5
|
import { buildReport } from './build.js';
|
|
6
6
|
import { injectData, injectSettings } from './inject.js';
|
|
7
|
+
import { secretValues } from './redact.js';
|
|
7
8
|
import { SCOPE_ENV, parseScopeOverride } from './scope.js';
|
|
8
9
|
import { generatorVersion } from './version.js';
|
|
9
10
|
/** Built renderer template, shipped next to the compiled core (`dist/renderer/index.html`). */
|
|
10
11
|
const TEMPLATE_URL = new URL('../renderer/index.html', import.meta.url);
|
|
11
12
|
export class LoupeReporter {
|
|
12
13
|
loupeConfig;
|
|
14
|
+
/** Values of the `redact.env` variables, read once at startup. */
|
|
15
|
+
secretValues;
|
|
13
16
|
scopeOverride;
|
|
14
17
|
runErrors = [];
|
|
15
18
|
config;
|
|
@@ -20,6 +23,8 @@ export class LoupeReporter {
|
|
|
20
23
|
// run with its message, which no later reporter hook can do.
|
|
21
24
|
this.scopeOverride = parseScopeOverride(process.env[SCOPE_ENV]);
|
|
22
25
|
this.loupeConfig = loadConfig(options);
|
|
26
|
+
// After the config is loaded; warnings name a variable, never its value.
|
|
27
|
+
this.secretValues = secretValues(this.loupeConfig.redact.env, process.env, (message) => console.warn(message));
|
|
23
28
|
}
|
|
24
29
|
printsToStdio() {
|
|
25
30
|
return false;
|
|
@@ -47,6 +52,7 @@ export class LoupeReporter {
|
|
|
47
52
|
categories: this.loupeConfig.categories,
|
|
48
53
|
dimensions: this.loupeConfig.dimensions,
|
|
49
54
|
runErrors: this.runErrors,
|
|
55
|
+
redact: { mode: this.loupeConfig.redact.mode, patterns: this.loupeConfig.redact.patterns, values: this.secretValues },
|
|
50
56
|
});
|
|
51
57
|
const settings = {
|
|
52
58
|
title: this.loupeConfig.title,
|
package/dist/core/signature.js
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { createHash } from 'node:crypto';
|
|
2
|
+
import { STATUS_CODE, httpStatusSource } from './http.js';
|
|
2
3
|
// Same pattern as the `ansi-regex` package (CSI and OSC sequences).
|
|
3
4
|
const ANSI_PATTERN = /[\u001B\u009B][[\]()#;?]*(?:(?:(?:(?:;[-a-zA-Z\d/#&.:=?%@~_]+)*|[a-zA-Z\d]+(?:;[-a-zA-Z\d/#&.:=?%@~_]*)*)?\u0007)|(?:(?:\d{1,4}(?:;\d{0,4})*)?[\dA-PR-TZcf-ntqry=><~]))/g;
|
|
4
5
|
export function stripAnsi(text) {
|
|
@@ -78,8 +79,14 @@ const REPLACEMENTS = [
|
|
|
78
79
|
[/\b\d+(?:\.\d+)?\s?(?:ms|milliseconds?|s|secs?|seconds?|m|mins?|minutes?|h|hours?)\b/gi, '<duration>'],
|
|
79
80
|
// 8+ hex chars with at least one letter (pure digit runs fall through to <n>).
|
|
80
81
|
[/(?<![0-9A-Za-z])(?=[0-9a-f]*[a-f])[0-9a-f]{8,}(?![0-9A-Za-z])/gi, '<hex>'],
|
|
81
|
-
[/\d+(?:\.\d+)?/g, '<n>'],
|
|
82
82
|
];
|
|
83
|
+
/**
|
|
84
|
+
* Last replacement: remaining numbers become `<n>`, except HTTP status codes
|
|
85
|
+
* (100–599) in an HTTP context, kept with their context so `error 401` and
|
|
86
|
+
* `error 500` stay apart. Same context as the "Rate limited" and "Server
|
|
87
|
+
* error" categories (src/core/http.ts).
|
|
88
|
+
*/
|
|
89
|
+
const STATUS_OR_NUMBER = new RegExp(`(${httpStatusSource(STATUS_CODE)})|\\d+(?:\\.\\d+)?`, 'gi');
|
|
83
90
|
/**
|
|
84
91
|
* Normalizes an error message so the same failure produces the same text
|
|
85
92
|
* across tests, retries and runs: ANSI codes and the call log are removed,
|
|
@@ -97,6 +104,7 @@ export function normalizeErrorMessage(message, options = {}) {
|
|
|
97
104
|
for (const [pattern, placeholder] of REPLACEMENTS) {
|
|
98
105
|
text = text.replace(pattern, placeholder);
|
|
99
106
|
}
|
|
107
|
+
text = text.replace(STATUS_OR_NUMBER, (_match, status) => status ?? '<n>');
|
|
100
108
|
return text.replace(/\s+/g, ' ').trim();
|
|
101
109
|
}
|
|
102
110
|
/** First 12 hex chars of sha256 of the normalized message. Independent of the test. */
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qartisan/playwright-loupe",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "HTML reporter for Playwright: dashboard, error groups by normalized signature, tests by project and suite, dimensions from tags and annotations.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"playwright",
|