@qaguardian/playwright-score 2.1.0 → 2.1.1

package/CHANGELOG.md

@@ -10,6 +10,114 @@ parsing accuracy, not grading policy — can still move a suite's score
 without itself being a new model version; 2.1.0 below is exactly that
 case, and says so explicitly.
 
+## 2.1.1 — 2026-09-26
+
+Scoring model stays v4. Reviewer follow-ups from the 2.1.0 release review
+(QAG-230) — none of these blocked that release. They close symlink-escape
+gaps a crafted repo could otherwise use, quiet a false warning, and fix a
+false-positive drop the first round of these same fixes introduced. None
+of that changes what an ordinary suite with no symlinked spec files
+scores; the realpath dedupe fix below is the one exception — it can
+change the file/test count for a suite that has a symlinked spec file
+matching discovery alongside its own real target, and says so.
+
+### Fixed
+- **A match escaping the *repository* through a symlink is now dropped —
+  scoped to the repo root, not the narrower directory the caller pointed
+  at.** `testDirEscapesRepo` (added in 2.1.0) only clamps a config's
+  `testDir` itself before it becomes a glob root — it couldn't see a
+  symlinked directory the glob then walked *through* on the way to an
+  individual match. A `testMatch` entry that names a symlinked directory
+  by a literal path segment (not a `**` wildcard) is still followed even
+  with glob's `follow: false` (that option only stops `**` from
+  *expanding into* a symlinked directory, not a pattern from naming one
+  explicitly), so a crafted repo (`tests/evil -> /`, `testMatch:
+  ['evil/**/*.ts']`) could make discovery walk arbitrary parts of the
+  filesystem while every match still lexically looked like it lived
+  under the scanned directory. The same gap let an individually
+  symlinked spec file sitting directly in the repo (no `testMatch`
+  trickery needed) resolve to, and get scored from, a file outside it.
+  `src/index.ts`'s match loop realpath's every candidate match and drops
+  it unless it's still contained within the realpath of the **repository
+  root** (`safeRealpath(findRepoBoundary(abs))`) — not, as this package
+  first shipped it, the narrower directory the caller actually scanned.
+  That first version silently dropped a real, common monorepo shape too:
+  a spec reachable through an in-repo symlink that resolves to a sibling
+  directory *outside* the caller's scanned subpath (one package's specs
+  shared with another via a symlink) is legitimate and is kept; only a
+  match whose realpath lands outside the repository itself is dropped.
+  The same repo-root containment now also applies to a bare glob-pattern
+  input (e.g. `tests/*.spec.ts` passed directly to `scorePaths`, not a
+  directory it walks itself) — the first version of this fix only
+  covered the directory-scan branch, so switching an otherwise-identical
+  call from a directory argument to an equivalent glob argument silently
+  bypassed the guard entirely.
+- **The walk itself is now pruned at a symlink that escapes the repo
+  root, not just filtered after the fact.** The realpath containment
+  check above still has to pay to walk all the way through a symlinked
+  directory before dropping every match inside it one by one — for a
+  large or slow target (`tests/evil -> /`) that's a real cost, not just
+  a theoretical one (measured: ~6.4s to fully walk a large symlinked
+  directory before this fix). `RepoBoundaryIgnore` (`src/index.ts`) wraps
+  glob's own `ignore` matching with a `childrenIgnored` check that stops
+  the walk from ever descending into a symlinked directory whose
+  realpath lands outside the repo root, only paying the realpath cost at
+  an actual symlink (a plain nested directory can't walk outside a root
+  its own parent is already inside, so it's never checked).
+- **An in-repo symlinked spec and its own real target are no longer both
+  scored, when both independently match discovery under the same scan.**
+  Pre-existing, not new to the fixes above: `expandPaths` deduped a
+  literal path only against `explicit` inputs, never against another
+  `expanded` match that happens to resolve to the same realpath. A
+  symlink sitting alongside its target inside the same scanned directory
+  (or reachable via two different scan roots in one `scorePaths` call)
+  was counted, and scored, twice. Now deduped by realpath, keeping the
+  first lexical path in sorted order. **This can change file/test counts
+  for a repo that has such a link** — said plainly here rather than
+  folded into "doesn't change what a correctly-shaped suite scores",
+  since a suite with this exact shape is exactly what changes.
+- **False "testDir resolves outside the repository" warning on a
+  nonexistent `testDir` under a symlinked repo path.** `safeRealpath`
+  only resolves the portion of a path that exists on disk, so a
+  nonexistent `testDir` kept its literal, unresolved form while
+  `repoRoot` (which does exist) got fully resolved — on a host where the
+  repo itself sits under a symlinked prefix (macOS: `/tmp` ->
+  `/private/tmp`), that asymmetry alone made the two sides disagree and
+  fired a false escape warning for an ordinary missing `testDir`.
+  `testDirEscapesRepo` now checks existence first and returns "does not
+  escape" immediately for a `testDir` that isn't on disk — consistent
+  with `resolveConfigScopedRoot`'s own existing "nothing to score there"
+  handling for the same case.
+- **Escape warning now prints cwd-relative paths**, matching every other
+  `configWarning` in this package, instead of the runner's own absolute
+  filesystem paths.
+- **A raw glob-pattern input pointing at a different repo than `cwd` no
+  longer hard-fails with 0 matches.** The repo-root containment fix above
+  used `cwd`'s own repo boundary for a bare glob pattern regardless of
+  where the pattern actually pointed — an absolute pattern into a sibling
+  repo (`['/abs/repoB/e2e/*.spec.ts']`), or an equivalent `../`-relative
+  one, had every match rejected as an "escape" of `cwd`'s repo, where the
+  same target passed as a directory already scored fine. `src/index.ts`'s
+  glob-input branch now derives its containment boundary from *the
+  pattern's own* literal (non-magic) leading-directory prefix, resolved
+  against `cwd`, not from `cwd` itself — a literal prefix that lands
+  inside another repo entirely is scored against that repo's own
+  boundary, while a symlink an in-bounds prefix still reaches that escapes
+  *that* repo is still dropped.
+- **An explicit target and a directory scan that reaches it again through
+  a symlink are no longer scored twice.** The realpath dedupe added above
+  only ever populated its map from `expanded` matches; an `explicit` file
+  and a same-realpath `expanded` symlink are two different literal paths,
+  so the separate "explicit wins" step (an exact literal-path delete)
+  never caught that pairing. The dedupe map is now seeded with
+  `explicit`'s own realpaths first, so a same-realpath `expanded` match is
+  recognized as a duplicate instead of surviving as a second, independent
+  file.
+
+Corpus rerun (100 public repos, directory inputs, run on the round-2 build;
+the round-3 fixes only touch glob-pattern inputs): 96/100 repos score
+identically, 4 moved upstream, 0 changed.
+
 ## 2.1.0 — 2026-09-24
 
 Scoring model stays v4 — nothing here changes a rule's weight or

package/dist/index.js

@@ -1,12 +1,12 @@
 import fs from 'node:fs';
 import path from 'node:path';
-import { globSync } from 'glob';
+import { globSync, hasMagic, Ignore } from 'glob';
 import { runEslint } from './eslint-runner.js';
 import { commonAncestorDir } from './fs-util.js';
 import { collectLocallyImportedFiles } from './import-graph.js';
 import { analyzeSource, attributeFindingScopes, countLocators, findLocalAssertionHelperNames, getTestSpans, looksLikeNonPlaywrightTest, } from './metrics.js';
 import { detectExtendedTestAliases } from './test-aliases.js';
-import { findPlaywrightConfig, findRepoBoundary, importsPlaywrightTestTransitively, parsePlaywrightConfig, resolveConfigScopedRoot, testDirEscapesRepo, } from './playwright-config.js';
+import { findPlaywrightConfig, findRepoBoundary, importsPlaywrightTestTransitively, isAncestorOrSame, parsePlaywrightConfig, resolveConfigScopedRoot, safeRealpath, testDirEscapesRepo, } from './playwright-config.js';
 import { DEFAULT_THRESHOLDS } from './profiles.js';
 import { computeScore } from './score-engine.js';
 export { computeScore, MODEL, locatorsScore, gradeFromScore, ratioDimensionScore, } from './score-engine.js';
@@ -57,6 +57,86 @@ const DEFAULT_IGNORE_GLOBS = [
     '**/playwright-report/**',
     '**/test-results/**',
 ];
+/**
+ * Wraps glob's own default `Ignore` (the string-pattern matcher behind
+ * DEFAULT_IGNORE_GLOBS/testIgnore) so this package can *also* prune the
+ * walk at a symlinked directory whose realpath escapes `realRoot`,
+ * without giving up the string-pattern ignores glob's `ignore` option
+ * only accepts one or the other of (a plain string[] XOR an IgnoreLike
+ * object — see glob's own types).
+ *
+ * Only `childrenIgnored` does the extra realpath work, and only for an
+ * actual symlink: a plain (non-symlinked) directory can't walk outside a
+ * root its own parent is already inside, so resolving its realpath would
+ * just be wasted syscalls on every directory in a large tree. This is
+ * what stops discovery from following a `testDir -> /` (or any other
+ * huge, real) symlinked directory arbitrarily far (QAG-230) — the
+ * per-match realpath containment check in expandPaths below still
+ * catches an escaping *match*, but only after the walk already paid to
+ * get there; this stops the walk itself from ever descending.
+ */
+class RepoBoundaryIgnore {
+    realRoot;
+    inner;
+    constructor(patterns, realRoot) {
+        this.realRoot = realRoot;
+        this.inner = new Ignore(patterns, {});
+    }
+    ignored(p) {
+        return this.inner.ignored(p);
+    }
+    // Forwarded, not required by any globSync call this package makes today
+    // (includeChildMatches defaults to true, which never checks for this) —
+    // present anyway so relying on that default doesn't quietly become a
+    // hard requirement if it's ever overridden.
+    add(pattern) {
+        this.inner.add(pattern);
+    }
+    childrenIgnored(p) {
+        if (this.inner.childrenIgnored(p))
+            return true;
+        if (!p.isSymbolicLink())
+            return false;
+        return !isAncestorOrSame(this.realRoot, safeRealpath(p.fullpath()));
+    }
+}
+/**
+ * The literal (non-magic) leading-directory prefix of a raw glob pattern
+ * string, resolved against `cwd` — e.g. `/abs/repoB/e2e/*.spec.ts` ->
+ * `/abs/repoB/e2e`; `../repoB/e2e/*.spec.ts` -> `<cwd>/../repoB/e2e`
+ * (collapsed by path.resolve). Used to find *that pattern's own* repo
+ * boundary (QAG-230 round 3): a bare glob-pattern input isn't necessarily
+ * anywhere near `cwd`'s own repo — it can be an absolute path, or a
+ * relative one that climbs out via `../` — into a completely different
+ * repo, and computing containment from cwd's repo root regardless made
+ * every match there look like an escape (hard-failing a perfectly legit
+ * `['/abs/repoB/e2e/*.spec.ts']` down to 0 files, where the equivalent
+ * directory input `../repoB/e2e` already scored fine).
+ *
+ * Segments are checked with glob's own `hasMagic` — with `magicalBraces`
+ * on, since an unexpanded `{a,b}` segment isn't a real, literal directory
+ * name any more than a `*` is — so this only ever returns a directory
+ * that's guaranteed to be a plain path segment, never a piece the glob
+ * engine itself still has to interpret.
+ */
+function literalPrefixDir(pattern, cwd) {
+    const segments = pattern.split('/');
+    const prefix = [];
+    let sawMagic = false;
+    for (const segment of segments) {
+        if (hasMagic(segment, { magicalBraces: true })) {
+            sawMagic = true;
+            break;
+        }
+        prefix.push(segment);
+    }
+    // No magic anywhere in the pattern: it's a literal path (only reachable
+    // here because it didn't exist on disk — an existing path is handled by
+    // the file/directory branches above) — its directory is the parent of
+    // the last (literal) segment, not the whole joined string.
+    const dirSegments = sawMagic ? prefix : prefix.slice(0, -1);
+    return path.resolve(cwd, dirSegments.join('/') || '.');
+}
 /**
  * `explicit`: paths the caller named directly (a literal existing file) —
  * always scored, regardless of whether they look like a Playwright spec.
@@ -84,6 +164,19 @@ function expandPaths(inputs, cwd) {
             // needs this wider boundary, not the narrower common ancestor of
             // only the matched spec files.
             scanRootDirs.add(abs);
+            // Repo boundary for this scan's per-match realpath containment check
+            // and glob-walk pruning below (see the containment check itself, and
+            // RepoBoundaryIgnore, for why this must be the *repo root*, not the
+            // narrower `abs`): a scan pointed at one monorepo package's specs
+            // must still keep an in-repo symlink that resolves to a sibling
+            // directory outside that subpath, while still rejecting anything
+            // that resolves outside the repo entirely (QAG-230). Starts with the
+            // same conservative fallback-to-`abs` findRepoBoundary uses
+            // elsewhere in this file; replaced below with the wider boundary a
+            // successfully parsed config already computes for testDirEscapesRepo,
+            // so both checks share one repo boundary instead of computing it
+            // twice.
+            let repoRootForContainment = findRepoBoundary(abs, abs);
             // A playwright.config.* at or above this directory, when it can be
             // read statically, is authoritative about what this target itself
             // considers a spec — see playwright-config.ts. Falls back to the
@@ -120,13 +213,14 @@ function expandPaths(inputs, cwd) {
                     // same of it), else `abs` itself. Never the filesystem root or
                     // 20 levels up — see findRepoBoundary's doc comment.
                     const repoRoot = findRepoBoundary(abs, path.dirname(configFile));
+                    repoRootForContainment = repoRoot;
                     if (testDirEscapesRepo(parsedConfig.testDirAbs, repoRoot)) {
                         // The config's testDir resolves outside the repo boundary
                         // (directly, or via a symlink — see testDirEscapesRepo). Its
                         // testMatch/testIgnore were written for that escaped testDir,
                         // so applying them to the clamped repoRoot instead would be a
                         // silent mismatch; fall back to filename-based discovery.
-                        configWarnings.push(`Found ${path.relative(cwd, configFile) || configFile} but its testDir (${parsedConfig.testDirAbs}) resolves outside the repository (${repoRoot}) — falling back to filename-based spec discovery for ${path.relative(cwd, abs) || abs}.`);
+                        configWarnings.push(`Found ${path.relative(cwd, configFile) || configFile} but its testDir (${path.relative(cwd, parsedConfig.testDirAbs) || parsedConfig.testDirAbs}) resolves outside the repository (${path.relative(cwd, repoRoot) || repoRoot}) — falling back to filename-based spec discovery for ${path.relative(cwd, abs) || abs}.`);
                     }
                     else {
                         const scopedRoot = resolveConfigScopedRoot(abs, parsedConfig.testDirAbs, repoRoot);
@@ -143,12 +237,21 @@ function expandPaths(inputs, cwd) {
                     }
                 }
             }
+            const realRepoRoot = safeRealpath(repoRootForContainment);
             for (const g of specGlobs) {
                 for (const f of globSync(g, {
                     cwd: globCwd,
                     absolute: true,
                     nodir: true,
-                    ignore: ignoreGlobs,
+                    // RepoBoundaryIgnore keeps ignoreGlobs' own string-pattern
+                    // matching (node_modules/, testIgnore, ...) and *additionally*
+                    // stops the walk from ever descending into a symlinked directory
+                    // whose realpath escapes realRepoRoot — see its own doc comment.
+                    // Pruning during the walk (not just filtering matches after the
+                    // fact, below) is what keeps a `testDir/evil -> /` (or any
+                    // other huge, real symlinked target) from making discovery
+                    // actually walk arbitrary parts of the filesystem (QAG-230).
+                    ignore: new RepoBoundaryIgnore(ignoreGlobs, realRepoRoot),
                     // glob's own default for `follow` is already false (don't
                     // descend into a symlinked directory while expanding `**`), but
                     // this is untrusted-repo territory (see testDirEscapesRepo
@@ -157,6 +260,31 @@ function expandPaths(inputs, cwd) {
                     follow: false,
                 })) {
                     const resolved = path.resolve(f);
+                    // testDirEscapesRepo (above) only clamps testDir itself before
+                    // it becomes a glob root — it can't see what the glob then
+                    // actually walks *through* on the way to a match. A testMatch
+                    // entry that names a symlinked directory by a literal path
+                    // segment (not a `**` wildcard) still gets followed even with
+                    // `follow: false` on the globSync call above, since that option
+                    // only stops `**` from *expanding into* a symlinked directory —
+                    // it doesn't stop a pattern from naming one explicitly (e.g.
+                    // `tests/evil -> /` with `testMatch: ['evil/**/*.ts']` walks
+                    // `/`, and every match still lexically looks like it's under
+                    // `abs`). The same gap lets an individually symlinked spec file
+                    // sitting directly in the repo (no testMatch trickery needed)
+                    // resolve to a file outside it. A lexical containment check
+                    // can't catch either case — both matches still "look" like they
+                    // live under `abs` right up until the symlink is resolved — so
+                    // this compares realpaths on both sides. Checked against the
+                    // *repo root* (realRepoRoot), not `abs` itself: a scan pointed
+                    // at one monorepo package's specs must still keep a match that
+                    // resolves, in-repo, to a sibling directory outside that
+                    // subpath — only a true escape of the repo itself is dropped
+                    // here (QAG-230; the config-scoped re-narrowing to `abs` right
+                    // below handles the "don't silently widen past what the caller
+                    // asked for" half of this separately).
+                    if (!isAncestorOrSame(realRepoRoot, safeRealpath(resolved)))
+                        continue;
                     // resolveConfigScopedRoot always globs from testDir (Playwright's
                     // own testMatch/testIgnore semantics — see its doc comment), which
                     // can be wider than the directory the caller actually pointed
@@ -191,16 +319,65 @@ function expandPaths(inputs, cwd) {
             }
             continue;
         }
-        // treat as glob
+        // treat as glob — a raw pattern the caller passed directly (e.g.
+        // `tests/*.spec.ts`), resolved relative to `cwd` rather than a
+        // directory this function itself walked. Same realpath containment
+        // and walk-pruning as the directory branch above applies here too
+        // (QAG-230): a bare glob input is just as capable of naming a
+        // symlinked path segment explicitly, and skipping this check here
+        // would have made switching from a directory argument to an
+        // equivalent glob argument silently bypass it.
+        //
+        // The boundary itself comes from *this pattern's own* literal prefix
+        // directory (literalPrefixDir), not from `cwd` (QAG-230 round 3) — see
+        // that function's doc comment for why cwd's repo root is the wrong
+        // boundary for a pattern that points somewhere else entirely.
+        const prefixDir = literalPrefixDir(input, cwd);
+        const repoRootForPattern = safeRealpath(findRepoBoundary(prefixDir, prefixDir));
         for (const f of globSync(input, {
             cwd,
             absolute: true,
             nodir: true,
-            ignore: DEFAULT_IGNORE_GLOBS,
+            ignore: new RepoBoundaryIgnore(DEFAULT_IGNORE_GLOBS, repoRootForPattern),
+            follow: false,
         })) {
-            expanded.add(path.resolve(f));
+            const resolved = path.resolve(f);
+            if (!isAncestorOrSame(repoRootForPattern, safeRealpath(resolved)))
+                continue;
+            expanded.add(resolved);
         }
     }
+    // De-dupe by realpath: an in-repo symlinked spec and its real target can
+    // both independently match discovery (the symlink itself sitting under
+    // one scanned root, its target sitting somewhere else under the same
+    // repo) and would otherwise be scored twice, as if they were two
+    // different files. Keeps the first lexical path per realpath, in sorted
+    // order, so which one "wins" is deterministic rather than depending on
+    // Set insertion order. Pre-existing gap, not new to the escape fixes
+    // above — those only ever drop a true escape, never a legitimate
+    // in-repo duplicate (see CHANGELOG: this can change file counts for a
+    // repo that happens to have such a link).
+    const firstPathByRealpath = new Map();
+    // Seeded with `explicit`'s own realpaths first: an explicit target and a
+    // directory scan containing a symlink to that same file are two
+    // different literal paths for the same file, so the plain "explicit
+    // wins" delete below (which only matches identical literal paths) can't
+    // catch that pairing — without this seeding, the symlinked match survives
+    // dedup (its realpath hadn't been seen yet) and gets scored a second
+    // time alongside the explicit path.
+    for (const f of [...explicit].sort()) {
+        const real = safeRealpath(f);
+        if (!firstPathByRealpath.has(real))
+            firstPathByRealpath.set(real, f);
+    }
+    for (const f of [...expanded].sort()) {
+        const real = safeRealpath(f);
+        if (!firstPathByRealpath.has(real))
+            firstPathByRealpath.set(real, f);
+    }
+    expanded.clear();
+    for (const f of firstPathByRealpath.values())
+        expanded.add(f);
     // A file can be reached both explicitly and via expansion (e.g. an
     // explicit path plus an overlapping glob); explicit intent wins.
     for (const f of explicit)

package/dist/playwright-config.d.ts

@@ -57,6 +57,19 @@ export interface ParsedPlaywrightConfig {
  * partial or guessed reading.
  */
 export declare function parsePlaywrightConfig(configFile: string): ParsedPlaywrightConfig | undefined;
+/** Exported for index.ts's own realpath containment check on matched spec
+ * files (see expandPaths) — a config-scoped `testDir`/`testMatch` can be
+ * clamped to the repo boundary here, but the glob it drives can still walk
+ * through a symlinked directory *inside* that boundary whose real target
+ * is outside it (see testDirEscapesRepo's doc comment for why a lexical
+ * check alone isn't enough); index.ts needs the same primitive to catch
+ * that per-match, not just per-testDir. */
+export declare function isAncestorOrSame(ancestor: string, descendant: string): boolean;
+/** Resolves symlinks; falls back to the input path unchanged when it
+ * doesn't exist on disk (matches the existing "testDir doesn't exist"
+ * behavior elsewhere in this file rather than throwing). Exported for the
+ * same reason as isAncestorOrSame above. */
+export declare function safeRealpath(p: string): string;
 /**
  * True when `testDirAbs`, resolved through any symlinks, is NOT inside
  * (or equal to) `repoRoot`, also resolved through symlinks. A scored
@@ -67,6 +80,19 @@ export declare function parsePlaywrightConfig(configFile: string): ParsedPlaywri
  * this public tool glob outside its own boundary. Both sides are
  * realpath'd (never just testDirAbs) because `repoRoot` itself could in
  * principle be reached through a symlinked path.
+ *
+ * A nonexistent `testDirAbs` (nothing to score there — resolveConfigScopedRoot's
+ * own existence check downstream already handles that) short-circuits to
+ * "doesn't escape" *before* either side is realpath'd: `safeRealpath` only
+ * resolves symlinks up to the point where a path stops existing on disk,
+ * so a testDir that doesn't exist keeps its literal (unresolved) form
+ * while `repoRoot` — which does exist — gets fully resolved. On a host
+ * where the repo itself sits under a symlinked prefix (macOS: `/tmp` ->
+ * `/private/tmp`), that asymmetry alone made the two sides disagree and
+ * fired a false "escapes the repository" warning for a perfectly ordinary
+ * missing testDir (QAG-230). Checked with a plain `fs.existsSync`, not
+ * `safeRealpath`'s own try/catch, so this stays a pure existence check
+ * with no symlink resolution of its own to get wrong.
  */
 export declare function testDirEscapesRepo(testDirAbs: string, repoRoot: string): boolean;
 /**

package/dist/playwright-config.js

@@ -276,14 +276,22 @@ export function parsePlaywrightConfig(configFile) {
         testIgnore: sawTestIgnoreKey ? testIgnore : undefined,
     };
 }
-function isAncestorOrSame(ancestor, descendant) {
+/** Exported for index.ts's own realpath containment check on matched spec
+ * files (see expandPaths) — a config-scoped `testDir`/`testMatch` can be
+ * clamped to the repo boundary here, but the glob it drives can still walk
+ * through a symlinked directory *inside* that boundary whose real target
+ * is outside it (see testDirEscapesRepo's doc comment for why a lexical
+ * check alone isn't enough); index.ts needs the same primitive to catch
+ * that per-match, not just per-testDir. */
+export function isAncestorOrSame(ancestor, descendant) {
     const rel = path.relative(ancestor, descendant);
     return rel === '' || (!rel.startsWith('..') && !path.isAbsolute(rel));
 }
 /** Resolves symlinks; falls back to the input path unchanged when it
  * doesn't exist on disk (matches the existing "testDir doesn't exist"
- * behavior elsewhere in this file rather than throwing). */
-function safeRealpath(p) {
+ * behavior elsewhere in this file rather than throwing). Exported for the
+ * same reason as isAncestorOrSame above. */
+export function safeRealpath(p) {
     try {
         return fs.realpathSync(p);
     }
@@ -301,8 +309,23 @@ function safeRealpath(p) {
  * this public tool glob outside its own boundary. Both sides are
  * realpath'd (never just testDirAbs) because `repoRoot` itself could in
  * principle be reached through a symlinked path.
+ *
+ * A nonexistent `testDirAbs` (nothing to score there — resolveConfigScopedRoot's
+ * own existence check downstream already handles that) short-circuits to
+ * "doesn't escape" *before* either side is realpath'd: `safeRealpath` only
+ * resolves symlinks up to the point where a path stops existing on disk,
+ * so a testDir that doesn't exist keeps its literal (unresolved) form
+ * while `repoRoot` — which does exist — gets fully resolved. On a host
+ * where the repo itself sits under a symlinked prefix (macOS: `/tmp` ->
+ * `/private/tmp`), that asymmetry alone made the two sides disagree and
+ * fired a false "escapes the repository" warning for a perfectly ordinary
+ * missing testDir (QAG-230). Checked with a plain `fs.existsSync`, not
+ * `safeRealpath`'s own try/catch, so this stays a pure existence check
+ * with no symlink resolution of its own to get wrong.
  */
 export function testDirEscapesRepo(testDirAbs, repoRoot) {
+    if (!fs.existsSync(testDirAbs))
+        return false;
     const realRepoRoot = safeRealpath(repoRoot);
     const realTestDirAbs = safeRealpath(testDirAbs);
     return !isAncestorOrSame(realRepoRoot, realTestDirAbs);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qaguardian/playwright-score",
-  "version": "2.1.0",
+  "version": "2.1.1",
   "description": "Deterministic, AI-free Playwright spec quality score — the weighted share of your tests that are clean.",
   "type": "module",
   "main": "./dist/index.js",