@qaguardian/playwright-score 2.1.1 → 2.2.0

package/CHANGELOG.md

@@ -10,6 +10,79 @@ parsing accuracy, not grading policy — can still move a suite's score
 without itself being a new model version; 2.1.0 below is exactly that
 case, and says so explicitly.
 
+## 2.2.0 — 2026-11-14
+
+False-positive fixes found by hand-checking error-level findings on public
+suites (7 of 18 were wrong), then tightened after review so the fixes cannot be used to hide a test with no assertion. No rule's weight or severity changed, so this
+is not a new scoring model (still v4), but **scores can rise**: findings
+that were wrong are no longer reported. Nothing gets stricter.
+
+- `playwright/expect-expect`: a test that asserts through a fixture
+  function injected via the test args is no longer reported as having no
+  assertion, but only when the fixture is a callable that really asserts:
+  a `test.extend({ check: async ({ page }, use) => { await use(async () =>
+  { await expect(...) }) } })` entry whose function handed to `use(...)`
+  runs `expect`, the test destructures it AND calls it, and the fixtures
+  module is reachable from that spec through relative imports. An `expect`
+  in a fixture's setup or teardown does not count, nor does any fixture
+  named like a built-in one (`page`, `context`, `request`, ...). A test
+  also counts as asserting through an object-literal method / class-property
+  arrow function in a file it imports (a fixture factory's returned object,
+  a page object), except that a helper whose name collides with a Playwright
+  method (`goto`, `click`, `fill`, `check`, `get`, `post`, ...) is never
+  treated as an assertion helper, so it cannot turn every `page.goto()` into
+  an assertion. When a test has no `expect` of its own but calls an
+  assertion-named method (`assert`, `verify`, `check`, `validate`, `expect`,
+  `ensure`, `should`, `confirm` followed by an upper-case letter, `_`, a digit
+  or nothing, such as `checkTotals` but not `checkout`; or `toBe*`, `toHave*`,
+  `toContain*`, `toMatch*`, `toEqual*`, `toThrow*`) on an object this scan
+  cannot read (an injected fixture from another package, an unresolvable
+  import, a third-party package), it is no longer an error: it is reported
+  as the report-only finding `playwright-score/assertion-unverified`
+  ("needs a human check"), which does not lower the score and is not
+  counted in `summary.unassertedTests`. Those tests are counted separately
+  in `summary.unverifiedTests`, a new field that is always present in the
+  JSON output (`0` when nothing is unverified; the text and Markdown reports
+  show those tests as unverified, never as asserting): they are not known to
+  assert. A fixture a test calls, whose `test` comes from a module the scan
+  does not resolve (a tsconfig `paths` alias, a workspace package), is
+  unverified the same way. Names bound by destructuring
+  (`const { channelsPage } = await pw.login(user)`) take the kind of the
+  expression they come from. Query-style names (`is...`, `has...`,
+  `wait...`), helper methods named like Playwright actions (`fill`, `check`,
+  `goto`, ...), built-in calls (`new Date().toISOString()`,
+  `user.email.toLowerCase()`, `x.toString()`), `shop.checkout()` and
+  `loginPage.rememberMe.check()` are not treated as assertions: a call whose
+  name is not assertion-like, a test that only calls Playwright API on
+  `page`/`request`/..., and helpers whose body we can read and which do not
+  assert are still `expect-expect` errors.
+- `pwscore/no-trivial-assertion`: `expect(pageErrors).toEqual([])` where
+  `pageErrors` is a `const` array that may be mutated (passed to a call as
+  an argument, the receiver of `push`/`pop`/`splice`/`sort`/..., an
+  assignment target, aliased, or captured by a closure that does one of
+  those, such as a listener, or an inner array reached through an index:
+  `rows[0].push(9)`) is no longer flagged. A `.length` or index
+  read, a non-mutating method such as `.map()`, and an unrelated property
+  with the same name (`obj.expected`) do not count as mutation; literals
+  and a `const` array that is only read are still flagged.
+- `pwscore/no-timer-sleep`: a retry/backoff pause inside a plain helper
+  function (declaration, `const f = () =>`, class method) is no longer
+  flagged, but only when the sleep sits in a `catch` clause or in a loop
+  whose body contains a non-empty `try` with a `catch`, and the helper is not declared inside
+  a test, hook, describe or step callback. Sleeps directly in a test body or
+  hook, any loop without `try`/`catch` (including a hand-rolled
+  `while (await x.isVisible()) sleep()` polling wait), helpers declared
+  inside a test callback, and bare `sleep()` helpers are still flagged.
+- `playwright/no-conditional-in-test`: a guard whose only body is `throw`
+  (`if (!id) throw new Error(...)`, no `else`) is no longer flagged.
+- `playwright/valid-expect`: a helper whose name merely ends in `Expect`
+  (`clickSortableHeaderAndExpect(...)`) is no longer reported as an
+  `expect()` without a matcher.
+- Known and not fixed: `playwright/no-standalone-expect` on tests declared
+  through a factory function that returns a `test.extend(...)` result
+  (`const t = testWithConfig({...}); t('name', ...)`). The alias detection
+  only follows direct `test.extend` chains.
+
 ## 2.1.1 — 2026-09-26
 
 Scoring model stays v4. Reviewer follow-ups from the 2.1.0 release review

package/METHODOLOGY.md

@@ -82,7 +82,7 @@ owns this dimension; counting them as demerits too would double-penalize.)
 | Dimension | Weight | Demerit sources |
 |---|---:|---|
 | playwrightHygiene | 40 | hard waits (`page.waitForTimeout`, `new Promise`+`setTimeout` and its respellings, `node:timers/promises` setTimeout), networkidle, force, coordinate-based mouse interaction (`*.mouse.click/dblclick/move/down/up(x, y)`, partial demerit — see below), missing await, element handles, eval, conditionals, page.pause, useless await |
-| assertions | 25 | a test with no recognized assertion is fully flawed (`expect-expect`, after all delegation resolution); an assertion whose subject/matcher combination can only ever pass (`expect(true).toBe(true)`, `expect([]).toEqual([])`, never a deliberate force-fail like `expect(true).toBe(false)`); `valid-expect`, `no-standalone-expect`, `prefer-web-first-assertions`, a soft-assertion-only test (`expect.poll` counts as hard, not soft) all add partial demerits |
+| assertions | 25 | a test with no recognized assertion is fully flawed (`expect-expect`, after all delegation resolution; a test with no `expect` of its own that calls an assertion-named method (`assert`, `verify`, `check`, `validate`, `expect`, `ensure`, `should`, `confirm` followed by an upper-case letter, `_`, a digit or nothing; or `toBe*`, `toHave*`, `toContain*`, `toMatch*`, `toEqual*`, `toThrow*`) on an object the scan cannot read, or calls a fixture that comes from a module the scan cannot resolve is a report-only `playwright-score/assertion-unverified` "needs a human check" finding, not a demerit, and is counted in `summary.unverifiedTests` rather than as asserting; any other call on such an object does not excuse the test, so query-style names (`is...`, `has...`, `wait...`), helper methods named like Playwright actions (`fill`, `check`, `goto`) and built-in calls are never treated as assertions; an assertion helper or fixture only counts when its body is readable, it is called by the test, and its name does not collide with a Playwright method); an assertion whose subject/matcher combination can only ever pass (`expect(true).toBe(true)`, `expect([]).toEqual([])`, never a deliberate force-fail like `expect(true).toBe(false)`, and never an array that may be mutated elsewhere: passed to a call, the receiver of a mutating method such as `push`, an assignment target, or captured by a closure such as an event listener doing so, including an inner array mutated through an index such as `rows[0].push(9)`); `valid-expect`, `no-standalone-expect`, `prefer-web-first-assertions`, a soft-assertion-only test (`expect.poll` counts as hard, not soft) all add partial demerits |
 | locators | 20 | usage ratio (above) |
 | structure | 15 | focused tests, always-skipped declarations, nested-describe depth, oversized files (module scope), an inline `eslint-disable` suppression comment (partial demerit — see "Known limitations" in VALIDATION.md) |
 

package/dist/finding-triage.d.ts

@@ -0,0 +1,13 @@
+import type { Finding } from './types.js';
+export declare const ASSERTION_UNVERIFIED_RULE = "playwright-score/assertion-unverified";
+export interface TriageContext {
+    /** Directory the findings' `file` paths are relative to. */
+    filesBase: string;
+    /** Scored spec files, absolute path -> source. */
+    sources: Map<string, string>;
+    /** Locally imported dependency files (page objects, fixtures), absolute path -> source. */
+    importedFiles: Map<string, string>;
+    /** Directory relative imports may not climb out of (defaults to filesBase). */
+    importBoundary?: string;
+}
+export declare function triageFindings(findings: Finding[], ctx: TriageContext): Finding[];

package/dist/finding-triage.js

@@ -0,0 +1,470 @@
+/**
+ * Post-lint triage of two upstream rules whose own checks cannot see across
+ * files or tell a guard from real branching (QAG-287):
+ *
+ * - `playwright/expect-expect` only sees `expect(...)` written in the test
+ *   body. A test that asserts through a page-object method, an injected
+ *   fixture or an imported helper was reported as "no assertion" at error
+ *   severity even though the assertion runs. Each such finding is resolved
+ *   here: dropped when a fixture the test uses is known to assert, turned
+ *   into a report-only "needs a human check" finding when the test calls
+ *   something this static scan cannot read, and kept as the error it is
+ *   when every call resolves to code we could read (or is plain Playwright
+ *   API on `page`, `request`, ...).
+ * - `playwright/valid-expect` fires on a helper whose name ends in
+ *   "Expect" (`clickSortableHeaderAndExpect(...)`) as if it were expect().
+ * - `playwright/no-conditional-in-test` fires on a guard whose only body is
+ *   `throw`. That is an assertion in disguise, not branching test logic.
+ *
+ * Purely syntactic and name-based, like the rest of this package: a
+ * resolved-by-name match is the same trade-off `assertFunctionNames` makes.
+ */
+import path from 'node:path';
+import { parseForESLint } from '@typescript-eslint/parser';
+import { resolveRelativeImport } from './import-graph.js';
+import { PLAYWRIGHT_API_METHODS } from './playwright-api-names.js';
+export const ASSERTION_UNVERIFIED_RULE = 'playwright-score/assertion-unverified';
+/** Identifiers whose calls are Playwright API or the language runtime, never a hidden assertion helper. */
+const NEUTRAL_ROOTS = new Set([
+    'page', 'context', 'request', 'browser', 'browserName', 'baseURL', 'testInfo', 'playwright',
+    'expect', 'test', 'it', 'describe', 'console', 'Promise', 'JSON', 'Math', 'Date', 'Object',
+    'Array', 'String', 'Number', 'Boolean', 'process', 'setTimeout', 'setInterval', 'require',
+    'URL', 'Buffer', 'fetch', 'undefined', 'globalThis', 'window', 'document',
+]);
+// Word-boundary, case-sensitive (same idea as the assertFunctionPatterns in
+// eslint-runner.ts): `checkout`, `toISOString`, `toLowerCase` or `shop.checkout()`
+// are not assertions, `checkFoo`, `assert_ok` and `toBeVisible` are.
+const ASSERTIONISH_PREFIX = /^(assert|verify|check|validate|expect|ensure|should|confirm)([A-Z_0-9]|$)/;
+const ASSERTIONISH_TO = /^to(Be|Have|Contain|Match|Equal|Throw)/;
+/** JS / Node built-ins: `new Date().toISOString()` is never a page-object call. */
+const BUILTIN_CLASSES = new Set([
+    'Date', 'Map', 'Set', 'WeakMap', 'WeakSet', 'URL', 'URLSearchParams', 'Array', 'Error', 'TypeError',
+    'RangeError', 'RegExp', 'Promise', 'Object', 'String', 'Number', 'Boolean', 'Buffer', 'Uint8Array',
+    'ArrayBuffer', 'Intl', 'AbortController', 'TextEncoder', 'TextDecoder', 'FormData', 'Headers', 'Blob',
+]);
+/**
+ * Only a method whose NAME reads like an assertion may be a hidden one:
+ * `app.waitForLoad()` or `users.list()` on an unreadable object is just an
+ * unverified call, and the test stays an expect-expect error. A name that is
+ * a Playwright API verb (`check`, `fill`, `goto`) is an action whatever the
+ * receiver: `loginPage.rememberMe.check()` is a locator held in a property.
+ */
+function isAssertionLikeCall(name) {
+    if (!ASSERTIONISH_PREFIX.test(name) && !ASSERTIONISH_TO.test(name))
+        return false;
+    return !PLAYWRIGHT_API_METHODS.has(name);
+}
+function walk(node, visit, parent) {
+    if (!node || typeof node !== 'object')
+        return;
+    if (Array.isArray(node)) {
+        for (const item of node)
+            walk(item, visit, parent);
+        return;
+    }
+    if (typeof node.type !== 'string')
+        return;
+    visit(node, parent);
+    for (const key of Object.keys(node)) {
+        if (key === 'parent')
+            continue;
+        const v = node[key];
+        if (v && typeof v === 'object')
+            walk(v, visit, node);
+    }
+}
+function parse(astCache, source, file) {
+    const hit = astCache.get(file);
+    if (hit && hit.source === source)
+        return hit.ast;
+    let ast;
+    try {
+        ast = parseForESLint(source, {
+            ecmaVersion: 2022,
+            sourceType: 'module',
+            ecmaFeatures: { jsx: /\.(tsx|jsx)$/.test(file) },
+        }).ast;
+    }
+    catch {
+        ast = undefined;
+    }
+    astCache.set(file, { source, ast });
+    return ast;
+}
+function isExpectCall(n) {
+    if (n?.type !== 'CallExpression')
+        return false;
+    const c = n.callee;
+    if (c?.type === 'Identifier' && c.name === 'expect')
+        return true;
+    return c?.type === 'MemberExpression' && c.object?.type === 'Identifier' && c.object.name === 'expect';
+}
+function containsExpect(node) {
+    let found = false;
+    walk(node, (n) => {
+        if (!found && isExpectCall(n))
+            found = true;
+    });
+    return found;
+}
+function isFn(n) {
+    return n?.type === 'ArrowFunctionExpression' || n?.type === 'FunctionExpression';
+}
+function propKeyName(p) {
+    if (p?.type !== 'Property' && p?.type !== 'ObjectProperty')
+        return undefined;
+    if (p.key?.type === 'Identifier' && !p.computed)
+        return p.key.name;
+    if (p.key?.type === 'Literal' && typeof p.key.value === 'string')
+        return p.key.value;
+    return undefined;
+}
+/** Does the fixture function hand a function containing expect(...) to use(...)? */
+function usesAssertingCallable(fixtureFn) {
+    const p1 = fixtureFn.params?.[1];
+    const useName = p1?.type === 'Identifier' ? p1.name : 'use';
+    let found = false;
+    walk(fixtureFn.body, (n) => {
+        if (found || n.type !== 'CallExpression' || n.callee?.type !== 'Identifier' || n.callee.name !== useName)
+            return;
+        if ((n.arguments ?? []).some((a) => isFn(a) && containsExpect(a)))
+            found = true;
+    });
+    return found;
+}
+function gather(ctx, cache) {
+    const defined = new Set();
+    const assertingFixtures = new Map();
+    const fixtureKeys = new Map();
+    const all = [...ctx.sources, ...ctx.importedFiles];
+    for (const [file, source] of all) {
+        const ast = parse(cache, source, file);
+        if (!ast)
+            continue;
+        walk(ast, (n) => {
+            if (n.type === 'FunctionDeclaration' && n.id?.name)
+                defined.add(n.id.name);
+            else if (n.type === 'ClassDeclaration' && n.id?.name)
+                defined.add(n.id.name);
+            else if (n.type === 'VariableDeclarator' && n.id?.type === 'Identifier' && isFn(n.init)) {
+                defined.add(n.id.name);
+            }
+            else if (n.type === 'MethodDefinition' && !n.computed && n.key?.type === 'Identifier') {
+                defined.add(n.key.name);
+            }
+            else if (n.type === 'PropertyDefinition' && !n.computed && n.key?.type === 'Identifier' && isFn(n.value)) {
+                defined.add(n.key.name);
+            }
+            else if (n.type === 'Property' && isFn(n.value)) {
+                const k = propKeyName(n);
+                if (k)
+                    defined.add(k);
+            }
+            else if (n.type === 'CallExpression' &&
+                n.callee?.type === 'MemberExpression' &&
+                n.callee.property?.name === 'extend' &&
+                n.arguments?.[0]?.type === 'ObjectExpression') {
+                for (const p of n.arguments[0].properties ?? []) {
+                    const k = propKeyName(p);
+                    if (k) {
+                        if (!fixtureKeys.has(file))
+                            fixtureKeys.set(file, new Set());
+                        fixtureKeys.get(file).add(k);
+                    }
+                    if (!k || NEUTRAL_ROOTS.has(k) || !isFn(p.value) || !usesAssertingCallable(p.value))
+                        continue;
+                    if (!assertingFixtures.has(file))
+                        assertingFixtures.set(file, new Set());
+                    assertingFixtures.get(file).add(k);
+                }
+            }
+        });
+    }
+    return { defined, assertingFixtures, fixtureKeys, cache };
+}
+/** Names bound by an identifier, object or array pattern. */
+function boundNames(pat, out = []) {
+    if (!pat)
+        return out;
+    if (pat.type === 'Identifier')
+        out.push(pat.name);
+    else if (pat.type === 'ObjectPattern')
+        for (const p of pat.properties ?? [])
+            boundNames(p.type === 'RestElement' ? p.argument : p.value, out);
+    else if (pat.type === 'ArrayPattern')
+        for (const e of pat.elements ?? [])
+            boundNames(e, out);
+    else if (pat.type === 'AssignmentPattern')
+        boundNames(pat.left, out);
+    else if (pat.type === 'RestElement')
+        boundNames(pat.argument, out);
+    return out;
+}
+/** Root identifier / expression of a call or member chain. */
+function chainRoot(node) {
+    let cur = node;
+    for (let i = 0; i < 64 && cur; i++) {
+        if (cur.type === 'AwaitExpression')
+            cur = cur.argument;
+        else if (cur.type === 'TSNonNullExpression' || cur.type === 'ChainExpression')
+            cur = cur.expression;
+        else if (cur.type === 'TSAsExpression' || cur.type === 'TSTypeAssertion')
+            cur = cur.expression;
+        else if (cur.type === 'CallExpression')
+            cur = cur.callee;
+        else if (cur.type === 'MemberExpression')
+            cur = cur.object;
+        else
+            break;
+    }
+    return cur;
+}
+function importKind(specifier) {
+    if (specifier.startsWith('.') || /^(@\/|~\/|#|src\/|tests?\/|e2e\/|pages\/|fixtures\/|utils?\/|lib\/)/.test(specifier)) {
+        return 'external';
+    }
+    return 'lib';
+}
+/**
+ * Calls inside a test callback whose receiver lives somewhere this scan
+ * cannot read: names of those calls (empty when every call is plain
+ * Playwright API, a same-file/readable helper, or a utility library).
+ */
+function unresolvedCalls(testCall, ast, know) {
+    const fn = [...(testCall.arguments ?? [])].reverse().find(isFn);
+    if (!fn)
+        return [];
+    const imports = new Map();
+    for (const stmt of ast.body ?? []) {
+        if (stmt.type !== 'ImportDeclaration' || typeof stmt.source?.value !== 'string')
+            continue;
+        for (const s of stmt.specifiers ?? []) {
+            if (s.local?.name)
+                imports.set(s.local.name, importKind(stmt.source.value));
+        }
+    }
+    const kinds = new Map();
+    const first = fn.params?.[0];
+    if (first?.type === 'ObjectPattern') {
+        for (const p of first.properties ?? []) {
+            const local = p.value?.type === 'AssignmentPattern' ? p.value.left : p.value;
+            const name = local?.type === 'Identifier' ? local.name : undefined;
+            if (name)
+                kinds.set(name, NEUTRAL_ROOTS.has(propKeyName(p) ?? name) ? 'neutral' : 'external');
+        }
+    }
+    const rootKind = (root) => {
+        if (root?.type === 'NewExpression') {
+            const cls = root.callee?.name;
+            if (cls && (BUILTIN_CLASSES.has(cls) || NEUTRAL_ROOTS.has(cls)))
+                return 'neutral';
+            return cls && know.defined.has(cls) ? 'neutral' : 'external';
+        }
+        if (root?.type !== 'Identifier')
+            return 'neutral';
+        const local = kinds.get(root.name);
+        if (local)
+            return local;
+        if (NEUTRAL_ROOTS.has(root.name))
+            return 'neutral';
+        return imports.get(root.name) ?? 'neutral';
+    };
+    const unresolved = [];
+    walk(fn.body, (n) => {
+        if (n.type === 'VariableDeclarator' && n.init) {
+            // `const x = ...` and destructuring (`const { a, b: [c] } = await ...`):
+            // every bound name takes the kind of the initializer's root.
+            const kind = rootKind(chainRoot(n.init));
+            for (const bound of boundNames(n.id))
+                kinds.set(bound, kind);
+            return;
+        }
+        if (n.type !== 'CallExpression')
+            return;
+        const c = n.callee;
+        let name;
+        if (c?.type === 'Identifier')
+            name = c.name;
+        else if (c?.type === 'MemberExpression' && !c.computed && c.property?.type === 'Identifier')
+            name = c.property.name;
+        if (!name || know.defined.has(name))
+            return;
+        const kind = rootKind(chainRoot(c));
+        if ((kind === 'external' || kind === 'lib') && isAssertionLikeCall(name))
+            unresolved.push(name);
+    });
+    return unresolved;
+}
+/** The spec file plus every file reachable from it through relative imports / re-exports. */
+function reachableFiles(spec, ctx, know) {
+    const boundary = ctx.importBoundary ?? ctx.filesBase;
+    const seen = new Set([spec]);
+    const queue = [spec];
+    while (queue.length) {
+        const file = queue.pop();
+        const source = ctx.sources.get(file) ?? ctx.importedFiles.get(file);
+        const ast = source === undefined ? undefined : parse(know.cache, source, file);
+        for (const stmt of ast?.body ?? []) {
+            const spec2 = stmt.source?.value;
+            if (typeof spec2 !== 'string' || !spec2.startsWith('.'))
+                continue;
+            const target = resolveRelativeImport(file, spec2, boundary);
+            if (target && !seen.has(target)) {
+                seen.add(target);
+                queue.push(target);
+            }
+        }
+    }
+    return seen;
+}
+/** Local names of destructured test args that are asserting fixtures reachable from this spec. */
+function assertingFixtureLocals(testFn, reachable, know) {
+    const locals = new Set();
+    for (const p of testFn?.params?.[0]?.properties ?? []) {
+        const key = propKeyName(p);
+        if (!key || NEUTRAL_ROOTS.has(key))
+            continue;
+        if (![...reachable].some((f) => know.assertingFixtures.get(f)?.has(key)))
+            continue;
+        const local = p.value?.type === 'AssignmentPattern' ? p.value.left : p.value;
+        if (local?.type === 'Identifier')
+            locals.add(local.name);
+    }
+    return locals;
+}
+/**
+ * Fixtures the test destructures AND calls (`a11y()`) that no reachable file
+ * defines, while its `test` comes from a non-relative module this scan does
+ * not resolve (a tsconfig `paths` alias such as `@fx/base`, a workspace
+ * package). The fixture may well assert there, so it is "unverified", not
+ * "no assertion". Relative imports are followed by reachableFiles instead.
+ */
+function unresolvedFixtureCalls(call, ast, fn, reachable, know) {
+    const testRoot = chainRoot(call.callee);
+    if (testRoot?.type !== 'Identifier')
+        return [];
+    const fromUnresolvedModule = (ast.body ?? []).some((stmt) => stmt.type === 'ImportDeclaration' &&
+        typeof stmt.source?.value === 'string' &&
+        !stmt.source.value.startsWith('.') &&
+        !/^(@playwright\/|playwright|node:)/.test(stmt.source.value) &&
+        (stmt.specifiers ?? []).some((sp) => sp.local?.name === testRoot.name));
+    if (!fromUnresolvedModule)
+        return [];
+    const out = [];
+    for (const p of fn?.params?.[0]?.properties ?? []) {
+        const key = propKeyName(p);
+        if (!key || NEUTRAL_ROOTS.has(key))
+            continue;
+        if ([...reachable].some((f) => know.fixtureKeys.get(f)?.has(key)))
+            continue;
+        const local = p.value?.type === 'AssignmentPattern' ? p.value.left : p.value;
+        if (local?.type === 'Identifier' && callsAny(fn.body, new Set([local.name])))
+            out.push(key);
+    }
+    return out;
+}
+/** Does the body call `name(...)` directly (the fixture is invoked, not merely destructured)? */
+function callsAny(body, names) {
+    let found = false;
+    walk(body, (n) => {
+        if (!found && n.type === 'CallExpression' && n.callee?.type === 'Identifier' && names.has(n.callee.name))
+            found = true;
+    });
+    return found;
+}
+function findCall(ast, line, col0) {
+    let hit;
+    walk(ast, (n) => {
+        if (!hit && n.type === 'CallExpression' && n.loc?.start.line === line && n.loc.start.column === col0)
+            hit = n;
+    });
+    return hit;
+}
+function findIf(ast, line, col0) {
+    let hit;
+    walk(ast, (n) => {
+        if (!hit && n.type === 'IfStatement' && n.loc?.start.line === line && n.loc.start.column === col0)
+            hit = n;
+    });
+    return hit;
+}
+function isThrowOnly(stmt) {
+    if (stmt?.type === 'ThrowStatement')
+        return true;
+    return stmt?.type === 'BlockStatement' && stmt.body?.length === 1 && stmt.body[0].type === 'ThrowStatement';
+}
+export function triageFindings(findings, ctx) {
+    const wants = findings.some((f) => f.rule === 'playwright/expect-expect' ||
+        f.rule === 'playwright/no-conditional-in-test' ||
+        f.rule === 'playwright/valid-expect');
+    if (!wants)
+        return findings;
+    const cache = new Map(); // per call: nothing leaks between scorePaths runs
+    const know = gather(ctx, cache);
+    const out = [];
+    for (const f of findings) {
+        if ((f.rule !== 'playwright/expect-expect' &&
+            f.rule !== 'playwright/no-conditional-in-test' &&
+            f.rule !== 'playwright/valid-expect') ||
+            f.line === undefined ||
+            f.column === undefined) {
+            out.push(f);
+            continue;
+        }
+        const abs = path.resolve(ctx.filesBase, f.file);
+        const source = ctx.sources.get(abs);
+        const ast = source === undefined ? undefined : parse(cache, source, abs);
+        if (!ast) {
+            out.push(f);
+            continue;
+        }
+        if (f.rule === 'playwright/valid-expect') {
+            // Upstream treats any callee whose name merely ends in "Expect"
+            // (`clickSortableHeaderAndExpect(header, 'ascending')`) as an expect()
+            // call missing its matcher. Only the real `expect` is one.
+            const node = findCall(ast, f.line, f.column - 1);
+            const callee = node?.callee;
+            const isRealExpect = callee?.type === 'Identifier'
+                ? callee.name === 'expect'
+                : callee?.type === 'MemberExpression' && callee.object?.name === 'expect';
+            if (node && !isRealExpect)
+                continue;
+            out.push(f);
+            continue;
+        }
+        if (f.rule === 'playwright/no-conditional-in-test') {
+            const node = findIf(ast, f.line, f.column - 1);
+            if (node && !node.alternate && isThrowOnly(node.consequent))
+                continue; // a guard, not branching
+            out.push(f);
+            continue;
+        }
+        const call = findCall(ast, f.line, f.column - 1);
+        if (!call) {
+            out.push(f);
+            continue;
+        }
+        const fn = [...(call.arguments ?? [])].reverse().find(isFn);
+        const reachable = reachableFiles(abs, ctx, know);
+        const fixtureLocals = assertingFixtureLocals(fn, reachable, know);
+        if (fixtureLocals.size > 0 && callsAny(fn.body, fixtureLocals))
+            continue; // a callable fixture that asserts is invoked
+        const calls = [...new Set([...unresolvedCalls(call, ast, know), ...unresolvedFixtureCalls(call, ast, fn, reachable, know)])];
+        if (calls.length === 0) {
+            out.push(f);
+            continue;
+        }
+        out.push({
+            ...f,
+            rule: ASSERTION_UNVERIFIED_RULE,
+            severity: 'info',
+            reportOnly: true,
+            message: `Needs a human check: no expect(...) in this test body, but it calls ${calls
+                .slice(0, 3)
+                .map((c) => `${c}()`)
+                .join(', ')}${calls.length > 3 ? ', ...' : ''} on an object defined elsewhere (page object, injected fixture or ` +
+                'imported helper) that this scan cannot read, so the assertion may live there. Not counted against the score.',
+        });
+    }
+    return out;
+}

package/dist/formatters/markdown.js

@@ -12,7 +12,8 @@ export function formatMarkdown(result) {
         lines.push(`| ${k} | ${v} |`);
     }
     lines.push('');
-    lines.push(`Files: ${result.summary.files} · Tests: ${result.summary.tests} · SLOC: ${result.summary.sloc} · Findings: ${result.summary.findings}`);
+    lines.push(`Files: ${result.summary.files} · Tests: ${result.summary.tests} · SLOC: ${result.summary.sloc} · Findings: ${result.summary.findings}` +
+        (result.summary.unverifiedTests ? ` · Unverified assertions: ${result.summary.unverifiedTests}` : ''));
     if (result.findings.length) {
         lines.push('');
         lines.push('## Findings');

package/dist/formatters/text.js

@@ -13,7 +13,8 @@ export function formatText(result) {
     lines.push(`Summary: files=${result.summary.files} tests=${result.summary.tests} sloc=${result.summary.sloc} ` +
         `findings=${result.summary.findings} (errors=${result.summary.errors} warnings=${result.summary.warnings}) ` +
         `locators native/raw=${result.summary.nativeLocators}/${result.summary.rawLocators} ` +
-        `tests asserting=${result.summary.tests - result.summary.unassertedTests}/${result.summary.tests} ` +
+        `tests asserting=${result.summary.tests - result.summary.unassertedTests - result.summary.unverifiedTests}/${result.summary.tests}` +
+        `${result.summary.unverifiedTests ? ` (unverified=${result.summary.unverifiedTests})` : ''} ` +
         `clean tests=${result.summary.cleanTests}/${result.summary.tests}`);
     if (result.findings.length) {
         lines.push('');

package/dist/index.js

@@ -1,3 +1,4 @@
+import { triageFindings } from './finding-triage.js';
 import fs from 'node:fs';
 import path from 'node:path';
 import { globSync, hasMagic, Ignore } from 'glob';
@@ -407,6 +408,7 @@ function hardFail(profile, threshold, findings, extra = {}, skippedFiles, config
             nativeLocators: 0,
             rawLocators: 0,
             unassertedTests: 0,
+            unverifiedTests: 0,
             cleanTests: 0,
             ...extra,
         },
@@ -514,13 +516,13 @@ export async function scorePaths(options) {
     // below) and this package's own AST checks (getTestSpans/countTests,
     // base-plugin.ts).
     const testAliasNames = detectExtendedTestAliases(sources, importedFiles, importBoundary);
-    const eslintFindings = await runEslint({
+    const eslintFindings = triageFindings(await runEslint({
         files,
         profile,
         cwd,
         assertFunctionNames: [...assertFunctionNames],
         testAliasNames,
-    });
+    }), { filesBase, sources, importedFiles, importBoundary });
     let totalSloc = 0;
     let totalTests = 0;
     let native = 0;

package/dist/metrics.js

@@ -1,5 +1,6 @@
 import { parseForESLint } from '@typescript-eslint/parser';
 import { countSloc } from './sloc.js';
+import { PLAYWRIGHT_API_METHODS } from './playwright-api-names.js';
 // A positive "this IS a Playwright spec" check turned out to be unsafe:
 // Playwright's own custom-fixtures pattern (test/expect re-exported from a
 // project-local module built with base.extend()) is extremely common in
@@ -269,10 +270,25 @@ export function findLocalAssertionHelperNames(source, file) {
     if (!ast)
         return [];
     const candidates = [];
+    // `test.extend({ login: async ({page}, use) => {...} })` entries are
+    // fixtures, not callable helpers: an expect in their setup says nothing
+    // about whoever uses the name, so they never register as helper names
+    // (src/finding-triage.ts resolves a fixture that hands `use()` an
+    // asserting callable). Parents are visited before children, so marking
+    // the entries when the extend call is seen is early enough.
+    const fixtureEntries = new Set();
     walk(ast, (node) => {
         let name;
         let body;
-        if (node.type === 'FunctionDeclaration') {
+        if (node.type === 'CallExpression') {
+            const callee = node.callee;
+            const arg0 = node.arguments?.[0];
+            if (callee?.type === 'MemberExpression' && callee.property?.name === 'extend' && arg0?.type === 'ObjectExpression') {
+                for (const p of arg0.properties ?? [])
+                    fixtureEntries.add(p);
+            }
+        }
+        else if (node.type === 'FunctionDeclaration') {
             const id = node.id;
             if (id?.type === 'Identifier') {
                 name = id.name;
@@ -288,6 +304,21 @@ export function findLocalAssertionHelperNames(source, file) {
                 body = init;
             }
         }
+        else if (
+        // `{ checkTab: async () => { await expect(...) } }` — an object-literal
+        // method (a fixture factory's returned object, a test.extend entry) or
+        // a class property holding an arrow function (`check = async () => {}`).
+        (node.type === 'Property' || node.type === 'PropertyDefinition') &&
+            !node.computed &&
+            !fixtureEntries.has(node)) {
+            const key = node.key;
+            const value = node.value;
+            if (key?.type === 'Identifier' &&
+                (value?.type === 'ArrowFunctionExpression' || value?.type === 'FunctionExpression')) {
+                name = key.name;
+                body = value;
+            }
+        }
         else if (node.type === 'MethodDefinition') {
             const key = node.key;
             if (!node.computed && key?.type === 'Identifier' && node.kind === 'method') {
@@ -295,7 +326,7 @@ export function findLocalAssertionHelperNames(source, file) {
                 body = node.value;
             }
         }
-        if (name && body)
+        if (name && body && !PLAYWRIGHT_API_METHODS.has(name))
             candidates.push({ name, body });
     });
     const names = new Set();

package/dist/playwright-api-names.d.ts

@@ -0,0 +1,10 @@
+/**
+ * Method names of the Playwright objects a spec calls directly (Page,
+ * Locator, FrameLocator, Frame, APIRequestContext, Keyboard, Mouse,
+ * Touchscreen, BrowserContext). A user-defined helper that shares one of
+ * these names must never be registered as an "assertion helper": upstream
+ * `expect-expect` matches the LAST property of any call chain, so a helper
+ * `nav.goto` that asserts would make every `page.goto()` count as an
+ * assertion. One exported set, used by metrics.ts and finding-triage.ts.
+ */
+export declare const PLAYWRIGHT_API_METHODS: ReadonlySet<string>;

package/dist/playwright-api-names.js

@@ -0,0 +1,35 @@
+/**
+ * Method names of the Playwright objects a spec calls directly (Page,
+ * Locator, FrameLocator, Frame, APIRequestContext, Keyboard, Mouse,
+ * Touchscreen, BrowserContext). A user-defined helper that shares one of
+ * these names must never be registered as an "assertion helper": upstream
+ * `expect-expect` matches the LAST property of any call chain, so a helper
+ * `nav.goto` that asserts would make every `page.goto()` count as an
+ * assertion. One exported set, used by metrics.ts and finding-triage.ts.
+ */
+export const PLAYWRIGHT_API_METHODS = new Set([
+    // navigation / page
+    'goto', 'goBack', 'goForward', 'reload', 'close', 'bringToFront', 'evaluate', 'evaluateHandle',
+    'evaluateAll', 'setContent', 'content', 'title', 'url', 'screenshot', 'pdf', 'route', 'unroute',
+    'routeFromHAR', 'on', 'once', 'off', 'addInitScript', 'addScriptTag', 'addStyleTag', 'exposeFunction',
+    'exposeBinding', 'pause', 'frame', 'frames', 'mainFrame', 'opener', 'setViewportSize',
+    'setExtraHTTPHeaders', 'emulateMedia', 'dragAndDrop', 'setDefaultTimeout',
+    'setDefaultNavigationTimeout', 'storageState', 'newPage', 'cookies', 'addCookies', 'clearCookies',
+    'grantPermissions', 'clearPermissions', 'setGeolocation', 'setOffline', 'newContext', 'newCDPSession',
+    // locators
+    'locator', 'getByRole', 'getByText', 'getByLabel', 'getByPlaceholder', 'getByAltText', 'getByTitle',
+    'getByTestId', 'frameLocator', 'filter', 'and', 'or', 'first', 'last', 'nth', 'all', 'count',
+    'contentFrame', 'elementHandle', 'elementHandles', 'highlight', 'describe',
+    // actions
+    'click', 'dblclick', 'tap', 'fill', 'clear', 'type', 'press', 'pressSequentially', 'check', 'uncheck',
+    'setChecked', 'hover', 'focus', 'blur', 'selectOption', 'selectText', 'setInputFiles', 'dispatchEvent',
+    'scrollIntoViewIfNeeded', 'down', 'up', 'insertText', 'move', 'wheel',
+    // reads / waits
+    'textContent', 'innerText', 'innerHTML', 'inputValue', 'getAttribute', 'allTextContents',
+    'allInnerTexts', 'isVisible', 'isHidden', 'isEnabled', 'isDisabled', 'isEditable', 'isChecked',
+    'boundingBox', 'ariaSnapshot', 'waitFor', 'waitForSelector', 'waitForTimeout', 'waitForLoadState',
+    'waitForURL', 'waitForEvent', 'waitForFunction', 'waitForRequest', 'waitForResponse',
+    'waitForNavigation',
+    // APIRequestContext
+    'get', 'post', 'put', 'patch', 'delete', 'head', 'fetch', 'dispose',
+]);

package/dist/profiles.js

@@ -52,6 +52,8 @@ export const ESLINT_RULE_MAP = {
     },
     // Assertions
     'playwright/expect-expect': { dimension: 'assertions' },
+    // Report-only: see finding-triage.ts. A test that calls into code the scan cannot read.
+    'playwright-score/assertion-unverified': { dimension: 'assertions', severityOverride: 'info', reportOnly: true },
     // No severityOverride: matches eslint-plugin-playwright's own
     // recommended severity ('error'), set via eslint-runner.ts's rule config.
     'playwright/prefer-web-first-assertions': { dimension: 'assertions' },

package/dist/rules/base-plugin.js

@@ -175,6 +175,64 @@ function callsResolve(fn, resolveName) {
         call.callee?.type === 'Identifier' &&
         call.callee.name === resolveName);
 }
+const LOOP_TYPES = new Set(['ForStatement', 'ForInStatement', 'ForOfStatement', 'WhileStatement', 'DoWhileStatement']);
+/**
+ * A retry/backoff pause inside a plain helper function (an API retry loop,
+ * `for (...) { try { await call() } catch { await sleep(500) } }`) is not a
+ * test waiting on the UI: it paces calls to a service. Only that shape is
+ * exempt: the sleep sits inside a catch clause, or inside a loop whose body
+ * contains a try/catch, and the enclosing function is *defined* (function
+ * declaration, `const f = () => ...`, class method) and is not itself nested
+ * in a callback handed to a call (a test, hook, describe or step body). A
+ * hand-rolled polling wait (`while (await x.isVisible()) sleep()`), a sleep
+ * directly in a test body or hook, a helper declared inside a test callback
+ * and a bare `sleep()` helper are all still flagged. `from` is the ancestor
+ * index to start climbing from (the Promise executor's own parent).
+ */
+function isRetryBackoffInHelper(ancestors, from) {
+    let retryShape = false;
+    let helperFound = false;
+    for (let i = from; i >= 0; i--) {
+        const anc = ancestors[i];
+        if (!helperFound) {
+            if (anc.type === 'CatchClause')
+                retryShape = true;
+            if (LOOP_TYPES.has(anc.type) && containsTryCatch(anc.body))
+                retryShape = true;
+        }
+        if (!FUNCTION_TYPES.has(anc.type) && anc.type !== 'FunctionDeclaration')
+            continue;
+        const parent = ancestors[i - 1];
+        if (parent?.type === 'CallExpression' || parent?.type === 'NewExpression') {
+            // Callback handed to a call (test body, hook, describe, .then, step):
+            // either the sleep is not in a free-standing helper at all, or the
+            // helper lives inside such a callback. Not exempt either way.
+            return false;
+        }
+        if (helperFound)
+            continue; // a function around the helper: keep checking the callback chain
+        if (anc.type === 'FunctionDeclaration' ||
+            parent?.type === 'MethodDefinition' ||
+            parent?.type === 'PropertyDefinition' ||
+            (parent?.type === 'VariableDeclarator' && parent.init === anc)) {
+            if (!retryShape)
+                return false;
+            helperFound = true;
+        }
+    }
+    return helperFound;
+}
+function containsTryCatch(root) {
+    let found = false;
+    walkNode(root, (n) => {
+        if (n.type !== 'TryStatement' || !n.handler)
+            return;
+        // An empty `try {}` is not a retry shape.
+        if ((n.block?.body ?? []).length > 0)
+            found = true;
+    });
+    return found;
+}
 /**
  * `await new Promise((resolve) => setTimeout(resolve, ms))` is the same
  * hard-coded sleep as `page.waitForTimeout(ms)` (already flagged, as an
@@ -289,7 +347,7 @@ const noTimerSleep = {
                             firstArg.name === resolveParam.name;
                         const resolveName = resolveParam?.type === 'Identifier' ? resolveParam.name : undefined;
                         const isWrappedResolve = typeof resolveName === 'string' && callsResolve(firstArg, resolveName);
-                        if (isDirectResolve || isWrappedResolve) {
+                        if ((isDirectResolve || isWrappedResolve) && !isRetryBackoffInHelper(ancestors, i - 1)) {
                             context.report({ node: node, messageId: 'sleep' });
                         }
                     }
@@ -517,8 +575,119 @@ function collectConstBindings(root) {
                 bindings.set(d.id.name, value);
         }
     });
+    // A `const` array is still mutable: `const errors = []; page.on('pageerror',
+    // (e) => errors.push(e)); expect(errors).toEqual([])` is a real assertion
+    // (the listener fills the array), not `expect([]).toEqual([])`. Only keep
+    // an array binding when every reference to it is the plain subject of an
+    // expect(...) call; any other use (method call, passed along, spread,
+    // reassigned element) might mutate it, so it is not a constant.
+    for (const [name, value] of [...bindings]) {
+        if (value.ok && Array.isArray(value.value) && !isOnlyReadAsExpectSubject(root, name)) {
+            bindings.delete(name);
+        }
+    }
     return bindings;
 }
+const MUTATING_METHODS = new Set([
+    'push', 'unshift', 'splice', 'pop', 'shift', 'sort', 'reverse', 'fill', 'copyWithin',
+]);
+/**
+ * True when no reference to `name` under `root` could change the array:
+ * every reference is a declaration, the subject of expect(...), a `.length`
+ * or index read, or the receiver of a non-mutating method. A reference that
+ * is a call argument (it may be handed to code that fills it), the
+ * receiver of a mutating method, an assignment/update target, or any other
+ * use that aliases the array (returned, assigned on, spread, put in an
+ * object) counts as possibly mutating, wherever it sits — including inside
+ * a closure such as a listener. Same-named properties (`obj.name`,
+ * `{ name: 1 }`) are not references. Shadowing is not resolved: a shadowing
+ * binding is read like the outer one, which can only keep a finding quiet.
+ */
+function isOnlyReadAsExpectSubject(root, name) {
+    let ok = true;
+    const visit = (node, stack) => {
+        if (!ok || !node || typeof node !== 'object')
+            return;
+        if (Array.isArray(node)) {
+            for (const item of node)
+                visit(item, stack);
+            return;
+        }
+        const rec = node;
+        if (typeof rec.type !== 'string')
+            return;
+        const n = node;
+        if (n.type === 'Identifier' && n.name === name) {
+            if (referenceMayMutate(n, stack))
+                ok = false;
+            return;
+        }
+        const next = [...stack, n];
+        for (const key of Object.keys(rec)) {
+            if (key === 'parent')
+                continue;
+            const v = rec[key];
+            if (v && typeof v === 'object')
+                visit(v, next);
+        }
+    };
+    visit(root, []);
+    return ok;
+}
+/** Is the member expression `m` itself written, deleted or the receiver of a mutating method? */
+function memberMutates(m, above) {
+    if (above.type === 'AssignmentExpression' && above.left === m)
+        return true;
+    if (above.type === 'UpdateExpression')
+        return true;
+    if (above.type === 'UnaryExpression' && above.operator === 'delete')
+        return true;
+    const prop = m.property;
+    return (!m.computed &&
+        prop?.type === 'Identifier' &&
+        !!prop.name &&
+        MUTATING_METHODS.has(prop.name) &&
+        above.type === 'CallExpression' &&
+        above.callee === m);
+}
+function referenceMayMutate(id, stack) {
+    const parent = stack[stack.length - 1];
+    const grand = stack[stack.length - 2];
+    if (!parent)
+        return true;
+    switch (parent.type) {
+        case 'VariableDeclarator':
+            return parent.id !== id; // `const b = expected` aliases it
+        case 'CallExpression':
+            if (parent.arguments?.[0] === id && isExpectCallNode(parent))
+                return false;
+            return parent.callee !== id || false; // an argument, or the array called
+        case 'MemberExpression': {
+            if (parent.object !== id)
+                return false; // `obj.expected`: a property name, not this binding
+            // Mutation of the member itself (`a.x = 1`, `a.push(1)`), then, for a
+            // computed access chain (`rows[0].push(9)`, `rows[0][1] = 2`), of
+            // the element it reaches: the inner array is part of the asserted one.
+            let member = parent;
+            for (let up = stack.length - 2; up >= 0; up--) {
+                const above = stack[up];
+                if (memberMutates(member, above))
+                    return true;
+                if (!member.computed || above.type !== 'MemberExpression' || above.object !== member)
+                    break;
+                member = above;
+            }
+            return false; // `.length`, `[0]`, `.map(...)`, ... are reads
+        }
+        case 'Property':
+        case 'PropertyDefinition':
+            // Non-computed key position is a name, not a reference; a value (or
+            // shorthand) puts the array inside an object.
+            return !((parent.key === id || false) && !parent.computed && !parent.shorthand);
+        default:
+            return true;
+    }
+}
 /**
  * Climbs from `node` to the body of the nearest enclosing function (the
  * test/hook callback in the common case) — the scope `noTrivialAssertion`

package/dist/score-engine.js

@@ -188,6 +188,7 @@ export function computeScore(input) {
             nativeLocators: input.nativeLocators,
             rawLocators: input.rawLocators,
             unassertedTests: input.findings.filter((f) => f.rule === 'playwright/expect-expect' && !f.reportOnly).length,
+            unverifiedTests: input.findings.filter((f) => f.rule === 'playwright-score/assertion-unverified').length,
             cleanTests,
         },
         dimensions: dims,

package/dist/types.d.ts

@@ -51,6 +51,10 @@ export interface ScoreSummary {
     /** Tests with no recognized assertion (uncapped playwright/expect-expect
      * census, after delegation resolution). */
     unassertedTests: number;
+    /** Tests with no expect(...) that call something this scan cannot read
+     * (`playwright-score/assertion-unverified`, report-only). They are NOT in
+     * `unassertedTests` and not penalised, but they are not known to assert. */
+    unverifiedTests: number;
     /** Tests with no demerit in any dimension — the headline the score is
      * built from: the weighted share of tests that are clean. */
     cleanTests: number;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qaguardian/playwright-score",
-  "version": "2.1.1",
+  "version": "2.2.0",
   "description": "Deterministic, AI-free Playwright spec quality score — the weighted share of your tests that are clean.",
   "type": "module",
   "main": "./dist/index.js",