@qaguardian/playwright-score 0.3.0 → 0.5.0

package/METHODOLOGY.md

@@ -1,174 +1,145 @@
-# Playwright Spec Score (SQS) Methodology
+# Playwright Spec Score Methodology
 
-**Score version:** `sqs-v1`  
+**Scoring model:** v3 (2026-08-27)  
 **Package:** `playwright-score`  
 **Landing page:** https://qaguardian.com/open-source/playwright-score  
 **Maintainer:** [QA Guardian](https://qaguardian.com)  
 **Deterministic:** same inputs → same score  
 **AI-free:** no LLM calls in the scoring path
 
-Any change to formulas, weights, SLOC definition, caps, or constants requires a new version (`sqs-v2`, …).
+Any change that alters what a given suite scores is a new model version
+(v4, …), documented in the CHANGELOG. Model history: v1/v2 (published as
+"sqs-v1"/"sqs-v2" through package 0.4.0) scored finding *density per line
+of code*; v3 replaced that entirely — see "Why v3" at the bottom.
 
 ---
 
-## Profile
+## The model in one sentence
 
-`standard` (the only profile): community Playwright best practices
-(`eslint-plugin-playwright`) plus suite-level metrics (locator ratio,
-assertion-delegation tracing, Page Object Model import resolution).
+**Your score is the weighted share of your tests that are clean.**
 
 ---
 
-## Output
+## How it works
 
-- **score:** integer 0–100  
-- **grade:** A (90–100), B (80–89), C (70–79), D (60–69), F (0–59)  
-- **pass:** `score >= threshold`  
-- **dimensions:** per-dimension 0–100  
-- **findings:** full list (uncapped) for humans/agents  
+### 1. Findings are attributed to tests
 
----
-
-## Dimension weights
+Every file is parsed and each test declaration's source span is recorded
+(`test(...)`, `test.only/skip/fixme/slow(...)` — any form carrying a
+function body). Each lint/metric finding is then attributed to:
 
-| Dimension | Weight |
-|---|---|
-| playwrightHygiene | 40 |
-| assertions | 25 |
-| locators | 20 |
-| structure | 15 |
-
----
+- **the test it sits in**, when its line falls inside a test span;
+- **every test in the file**, when it sits in a `beforeEach`/`afterEach`/
+  `beforeAll`/`afterAll` hook — setup problems affect every test that runs
+  through them;
+- **the file, once** (module scope), for anything else — imports,
+  `describe` bodies, file-level metrics like oversized-file.
 
-## Locators dimension (ratio)
+### 2. Tests accumulate demerits
 
-Independent of finding-penalty math:
-
-```
-native = count(getByRole|getByLabel|getByTestId|getByText|getByPlaceholder)
-raw    = count(page.locator(|frame.locator()
-total  = native + raw
-
-locatorsScore = total === 0 ? 100 : round(100 * native / total)
-```
+Within each scored dimension, a test's demerit is the sum of its findings'
+units, capped at 1:
 
-ESLint `no-raw-locators` / `prefer-native-locators` still appear in `findings[]` for repair guidance.  
-They are **not** double-counted into the locators dimension via exponential penalties.  
-For sqs-v1 they map to **report-only** for scoring (severity still shown in findings; units = 0 for penalty dimensions).
+| Severity | Demerit |
+|---|---|
+| error | 1.0 |
+| warning | 0.4 |
+| info / report-only | 0 |
 
----
+A test is at worst *fully flawed* (demerit 1) — never more. That cap is
+both the anti-nuke mechanism (one loop-generated repeated finding cannot
+zero a suite) and the anti-gaming mechanism (there is no per-line density
+left to dilute by padding files with clean code). Module-scope findings
+are capped at 1 demerit per file per dimension.
 
-## SLOC (source lines of code)
+### 3. Dimensions are ratios
 
 ```
-sloc = lines that are NOT:
-  - blank / whitespace-only
-  - pure comment lines (// or block-comment-only lines)
-sloc = max(sloc, 1)
+dimensionScore = round(100 × (1 − totalDemerit / tests))     // clamped 0–100
 ```
 
-Used for penalty density. JSDoc / decorative comments do not inflate slots.
-
----
-
-## Penalty dimensions
+With zero tests the denominator is 1, so a findings-bearing zero-test
+input still cannot score clean.
 
-Applies to: `playwrightHygiene`, `assertions`, `structure`.
+The **locators** dimension is independent of demerits — it is the usage
+ratio it always was:
 
-### Constants (frozen sqs-v1)
+```
+native = getByRole|getByLabel|getByTestId|getByText|getByPlaceholder calls
+raw    = .locator(...) calls + legacy string-selector actions (page.click('#x'), …)
 
-| Constant | Value |
-|---|---|
-| `ERROR_UNIT` | 1.0 |
-| `WARNING_UNIT` | 0.4 |
-| `INFO_UNIT` | 0.0 |
-| `MAX_FINDINGS_PER_RULE_PER_FILE` | 3 (for penalty math only) |
-| `SLOT_DIVISOR` | 25 |
-| `MIN_SLOTS` | 4 |
-| `K` | 0.4 |
+locatorsScore = total === 0 ? 100 : round(100 × native / total)
+```
 
-### Steps
+(`no-raw-locators` / `prefer-native-locators` / `prefer-locator` findings
+appear in `findings[]` for repair guidance but are report-only — the ratio
+owns this dimension; counting them as demerits too would double-penalize.)
 
-1. **Cap** findings per `(file, ruleId)` to 3 for penalty math (report all in `findings[]`).
-2. **Units:** `rawUnits = E * 1.0 + W * 0.4`
-3. **Slots:** `slots = max(sloc / 25, 4)`
-4. **Load:** `load = rawUnits / slots`
-5. **Score:** `dimensionScore = clamp(0, 100, round(100 * exp(-0.4 * load)))`
+### 4. Weighted sum
 
-### Final score
+| Dimension | Weight | Demerit sources |
+|---|---:|---|
+| playwrightHygiene | 40 | hard waits, networkidle, force, missing await, element handles, eval, conditionals, page.pause, useless await |
+| assertions | 25 | a test with no recognized assertion is fully flawed (`expect-expect`, after all delegation resolution); `valid-expect`, `no-standalone-expect`, `prefer-web-first-assertions` add partial demerits |
+| locators | 20 | usage ratio (above) |
+| structure | 15 | focused tests, always-skipped declarations, nested-describe depth, oversized files (module scope) |
 
 ```
-score = round(Σ (weight_d / 100) * dimensionScore_d)
-pass  = score >= threshold
+score = round(Σ (weight_d / 100) × dimensionScore_d)
+pass  = score >= threshold        // CLI default 80
+grade = A ≥90 · B ≥80 · C ≥70 · D ≥60 · F <60
 ```
 
 ---
 
-## Worked examples (Phase 0 validation)
+## Worked example
 
-Assumptions: single-file runs; weights as above.
+3 tests. Test 1 has two hard-wait errors and no assertion. Test 2 is an
+always-skipped declaration with a conditional and no assertion. Test 3 is
+clean. Locators: 4 native, 4 raw.
 
-### 1. Tiny clean (~15 SLOC, 0 findings, all native locators)
+- **hygiene**: test 1 → min(1, 2×1.0) = 1; test 2 → 0.4 → total 1.4/3 → **53**
+- **assertions**: tests 1, 2 unasserted → 2/3 → **33**
+- **locators**: 4/8 → **50**
+- **structure**: skip warning 0.4/3 → **87**
+- **score** = 0.40·53 + 0.25·33 + 0.20·50 + 0.15·87 = **53 → F**
 
-- All penalty dims = 100  
-- Locators = 100  
-- **standard score ≈ 100**
-
-### 2. Tiny file, 1 hygiene error (~20 SLOC)
-
-- slots = max(20/25, 4) = 4  
-- rawUnits = 1.0 → load = 0.25 → hygiene = round(100 * e^(-0.1)) ≈ **90**  
-- Other dims 100  
-- standard ≈ 0.4*90 + 0.25*100 + 0.2*100 + 0.15*100 = **96**
-
-### 3. Medium file, 5 distinct hygiene errors (~100 SLOC)
-
-- slots = max(100/25, 4) = 4  
-- rawUnits = 5 → load = 1.25 → hygiene = round(100 * e^(-0.5)) ≈ **61**  
-- standard ≈ 0.4*61 + 60 = **84** if other dims perfect  
-
-### 4. Large file, 20× same warning (capped to 3)
-
-- 20 warnings same rule → cap 3 → rawUnits = 3 * 0.4 = 1.2  
-- sloc = 500 → slots = 20 → load = 0.06 → score ≈ **98** on that dim  
-- Cap prevents auto-zero  
-
-### 5. Locator ratio 2 native / 10 total
-
-- locatorsScore = round(100 * 2/10) = **20**  
-- Independent of caps  
-
-These examples feel fair: tiny single error stays high; multi-error drops materialy; repeated same-rule does not nuke large files; locator mix is a pure ratio.
+The same suite scored 78/C under the v1 density model. "Two of your three
+tests are broken" should not read as a near-B.
 
 ---
 
-## Default thresholds
+## SLOC
 
-| Context | Threshold |
-|---|---|
-| CLI default | 80 |
-
-QA Guardian's own internal codegen pipeline (`playwright_runner`) layers a
-private house-rules gate on top of `standard` — see the README's
-"QA Guardian integration" section. That layer isn't part of this package.
+`summary.sloc` (non-blank, non-comment lines) is reported for context and
+feeds the oversized-file check (>400 SLOC per file → structure warning,
+module scope). It plays no other role in scoring.
 
 ---
 
-## Rule → dimension map (summary)
-
-See package rule docs / source `profiles.ts`. High level:
-
-- **playwrightHygiene:** waits, force, networkidle, missing await, handles, conditionals  
-- **assertions:** expect-expect, empty tests, web-first preference  
-- **locators:** ratio metric only (for score); raw-locator ESLint in findings  
-- **structure:** focused/skipped, oversized file, describe shape
+## Active rule set (frozen)
 
-### Active rule set (frozen)
-
-The exact ESLint rules that produce sqs-v1 findings are enabled explicitly
-in `eslint-runner.ts` with fixed severities — the score never inherits
+The exact ESLint rules that produce findings are enabled explicitly in
+`eslint-runner.ts` with fixed severities — the score never inherits
 eslint-plugin-playwright's `recommended` set at install time, so identical
 code scores identically regardless of which plugin version npm resolves.
-(Versions ≤0.2.0 intended to inherit `recommended` but a config-shape bug
-dropped it entirely; 0.3.0 froze the intended set explicitly. See
-CHANGELOG.)  
+
+---
+
+## Why v3
+
+v1/v2 scored findings per 25 lines of code through an exponential decay
+with a minimum-slots floor. Three structural problems, each verified with
+real inputs:
+
+1. **Density is not how anyone judges a test suite.** "2 of 3 tests have
+   hard waits" is what a reviewer says; "0.5 penalty units per slot" is
+   not. v2 already fixed this for assertions (coverage ratio); v3 applies
+   the same per-test framing everywhere.
+2. **Small suites were structurally under-penalized** — the minimum-slots
+   floor meant a 3-test suite with 2 errors, 2 assertion-free tests, and a
+   skipped test scored 78/C.
+3. **Density is gameable**: adding clean lines of code (or clean files)
+   diluted `findings/SLOC` and raised the score without fixing anything.
+   A per-test ratio has no such lever — padding with fake tests costs
+   assertions coverage instead.

package/README.md

@@ -4,7 +4,8 @@
 [![npm version](https://img.shields.io/npm/v/@qaguardian/playwright-score.svg)](https://www.npmjs.com/package/@qaguardian/playwright-score)
 [![license](https://img.shields.io/npm/l/@qaguardian/playwright-score.svg)](./LICENSE)
 
-**Deterministic, AI-free quality score for Playwright specs** (`sqs-v1`).
+**Deterministic, AI-free quality score for Playwright specs** — the
+weighted share of your tests that are clean.
 
 **Website:** [qaguardian.com/open-source/playwright-score](https://qaguardian.com/open-source/playwright-score)  
 **Built by:** [QA Guardian](https://qaguardian.com) — managed Playwright E2E (AI drafts, engineers verify, you own the code)
@@ -21,30 +22,30 @@ The score never calls an LLM. AI may *generate* or *repair* code using findings;
 17 public Playwright suites, scored with the published package against each
 project's actual source — chosen to include both well-known,
 heavily-engineered platforms and smaller, less mature projects. Not curated
-to look good: 3 of 17 fail the default threshold, for real, verifiable
+to look good: 5 of 17 fail the default threshold, for real, verifiable
 reasons (see [VALIDATION.md](./VALIDATION.md)).
 
-**17 suites · 1,228 files · 6,037 tests · 14/17 passed (80% threshold)**
+**17 suites · 1,228 files · 5,664 tests · 12/17 passed (80% threshold)**
 
 | Repo (source scanned) | Score | Grade | Result |
 |---|---:|:-:|:-:|
 | [Playwright (own TodoMVC example)](https://github.com/microsoft/playwright/tree/main/examples/todomvc) | 98/100 | A | PASS |
-| [Supabase](https://github.com/supabase/supabase/tree/master/e2e) | 97/100 | A | PASS |
-| [Documenso](https://github.com/documenso/documenso/tree/main/packages/app-tests) | 96/100 | A | PASS |
-| [Storybook](https://github.com/storybookjs/storybook/tree/next/code/e2e-internal) | 96/100 | A | PASS |
-| [freeCodeCamp](https://github.com/freeCodeCamp/freeCodeCamp/tree/main/e2e) | 96/100 | A | PASS |
+| [Storybook](https://github.com/storybookjs/storybook/tree/next/code/e2e-internal) | 95/100 | A | PASS |
 | [n8n](https://github.com/n8n-io/n8n/tree/master/packages/testing/playwright) | 95/100 | A | PASS |
-| [dub](https://github.com/dubinc/dub/tree/main/apps/web/playwright) | 95/100 | A | PASS |
-| [novu](https://github.com/novuhq/novu/tree/next/apps/dashboard/tests) | 93/100 | A | PASS |
-| [Grafana](https://github.com/grafana/grafana/tree/main/e2e-playwright) | 91/100 | A | PASS |
-| [PostHog](https://github.com/PostHog/posthog/tree/master/playwright) | 90/100 | A | PASS |
-| [sencho](https://github.com/Studio-Saelix/sencho/tree/main/e2e) | 90/100 | A | PASS |
-| [Mattermost](https://github.com/mattermost/mattermost/tree/master/e2e-tests/playwright) | 89/100 | B | PASS |
-| [Immich](https://github.com/immich-app/immich/tree/main/e2e/src/specs) | 89/100 | B | PASS |
-| [cal.com](https://github.com/calcom/cal.com/tree/main/apps/web/playwright) | 84/100 | B | PASS |
-| [livecodes](https://github.com/live-codes/livecodes/tree/develop/e2e) | 78/100 | C | FAIL |
-| [openplayerjs](https://github.com/openplayerjs/openplayerjs/tree/master/e2e) | 77/100 | C | FAIL |
-| [TheCyberHub](https://github.com/th3cyb3rhub/TheCyberHub/tree/dev/e2e) | 68/100 | D | FAIL |
+| [freeCodeCamp](https://github.com/freeCodeCamp/freeCodeCamp/tree/main/e2e) | 94/100 | A | PASS |
+| [dub](https://github.com/dubinc/dub/tree/main/apps/web/playwright) | 94/100 | A | PASS |
+| [Documenso](https://github.com/documenso/documenso/tree/main/packages/app-tests) | 93/100 | A | PASS |
+| [Supabase](https://github.com/supabase/supabase/tree/master/e2e) | 91/100 | A | PASS |
+| [PostHog](https://github.com/PostHog/posthog/tree/master/playwright) | 88/100 | B | PASS |
+| [Grafana](https://github.com/grafana/grafana/tree/main/e2e-playwright) | 87/100 | B | PASS |
+| [sencho](https://github.com/Studio-Saelix/sencho/tree/main/e2e) | 85/100 | B | PASS |
+| [Mattermost](https://github.com/mattermost/mattermost/tree/master/e2e-tests/playwright) | 83/100 | B | PASS |
+| [novu](https://github.com/novuhq/novu/tree/next/apps/dashboard/tests) | 82/100 | B | PASS |
+| [Immich](https://github.com/immich-app/immich/tree/main/e2e/src/specs) | 78/100 | C | FAIL |
+| [cal.com](https://github.com/calcom/cal.com/tree/main/apps/web/playwright) | 73/100 | C | FAIL |
+| [openplayerjs](https://github.com/openplayerjs/openplayerjs/tree/master/e2e) | 73/100 | C | FAIL |
+| [livecodes](https://github.com/live-codes/livecodes/tree/develop/e2e) | 70/100 | C | FAIL |
+| [TheCyberHub](https://github.com/th3cyb3rhub/TheCyberHub/tree/dev/e2e) | 56/100 | F | FAIL |
 
 Repo names link to the exact source scanned. Full methodology, findings
 breakdown, and `scripts/validate-corpus.sh` to reproduce every number here
@@ -53,7 +54,7 @@ report of an earlier five-suite audit lives at
 [`docs/scorecard.html`](./docs/scorecard.html) — host it wherever
 (GitHub Pages, qaguardian.com, ...).
 
-See [METHODOLOGY.md](./METHODOLOGY.md) for the frozen formula, or the full product write-up on the [landing page](https://qaguardian.com/open-source/playwright-score).
+See [METHODOLOGY.md](./METHODOLOGY.md) for the full scoring model, or the product write-up on the [landing page](https://qaguardian.com/open-source/playwright-score).
 
 ## Install
 

package/dist/cli.js

@@ -10,7 +10,7 @@ import { formatText } from './formatters/text.js';
 import { formatMarkdown } from './formatters/markdown.js';
 import { formatSarif } from './formatters/sarif.js';
 function printHelp() {
-    console.log(`@qaguardian/playwright-score — deterministic Playwright spec quality score (sqs-v1)
+    console.log(`@qaguardian/playwright-score — deterministic Playwright spec quality score
 
 Usage:
   npx -p @qaguardian/playwright-score playwright-score <paths...> [options]
@@ -31,7 +31,7 @@ Exit codes:
   2  tool error
 
 Website: https://qaguardian.com/open-source/playwright-score
-Methodology: METHODOLOGY.md (sqs-v1)
+Methodology: METHODOLOGY.md (scoring model v3)
 `);
 }
 // Deliberately never process.exit() right after a console.log/console.error

package/dist/eslint-runner.js

@@ -23,7 +23,7 @@ function buildConfig(assertFunctionNames) {
                 // no-wait-for-selector, expect-expect) are intentionally stricter
                 // than eslint-plugin-playwright's own "recommended" severity —
                 // these are patterns Playwright's own docs call out as anti-
-                // patterns (see playwright.dev/docs/best-practices), so sqs-v1
+                // patterns (see playwright.dev/docs/best-practices), so this scorer
                 // treats them as errors rather than warnings. no-networkidle and
                 // no-raw-locators aren't part of upstream "recommended" at all but
                 // are enabled here for the same reason.
@@ -87,7 +87,7 @@ function buildConfig(assertFunctionNames) {
                 'playwright/prefer-native-locators': 'warn',
                 // Matches eslint-plugin-playwright's own recommended severity.
                 'playwright/prefer-web-first-assertions': 'error',
-                // The rules below complete the sqs-v1 rule set from
+                // The rules below complete the intended rule set from
                 // profiles.ts's ESLINT_RULE_MAP. Earlier versions (<=0.2.0)
                 // expected them to arrive via eslint-plugin-playwright's
                 // flat/recommended config, but the spread only handled an array

package/dist/formatters/text.js

@@ -12,7 +12,9 @@ export function formatText(result) {
     lines.push('');
     lines.push(`Summary: files=${result.summary.files} tests=${result.summary.tests} sloc=${result.summary.sloc} ` +
         `findings=${result.summary.findings} (errors=${result.summary.errors} warnings=${result.summary.warnings}) ` +
-        `locators native/raw=${result.summary.nativeLocators}/${result.summary.rawLocators}`);
+        `locators native/raw=${result.summary.nativeLocators}/${result.summary.rawLocators} ` +
+        `tests asserting=${result.summary.tests - result.summary.unassertedTests}/${result.summary.tests} ` +
+        `clean tests=${result.summary.cleanTests}/${result.summary.tests}`);
     if (result.findings.length) {
         lines.push('');
         lines.push('Findings:');

package/dist/index.d.ts

@@ -1,6 +1,6 @@
 import type { ScoreOptions, ScoreResult } from './types.js';
 export type { ScoreResult, ScoreOptions, Finding, ProfileName } from './types.js';
-export { computeScore, SQS_V1, locatorsScore, gradeFromScore, penaltyDimensionScore, applyPerRuleCap, } from './score-engine.js';
+export { computeScore, MODEL, locatorsScore, gradeFromScore, ratioDimensionScore, } from './score-engine.js';
 export { countSloc } from './sloc.js';
 export { countLocators, analyzeSource } from './metrics.js';
 export { formatJson } from './formatters/json.js';
@@ -8,6 +8,6 @@ export { formatText } from './formatters/text.js';
 export { formatMarkdown } from './formatters/markdown.js';
 export { formatSarif } from './formatters/sarif.js';
 /**
- * Score Playwright spec files. Deterministic and AI-free (sqs-v1).
+ * Score Playwright spec files. Deterministic and AI-free.
  */
 export declare function scorePaths(options: ScoreOptions): Promise<ScoreResult>;

package/dist/index.js

@@ -4,10 +4,10 @@ import { globSync } from 'glob';
 import { runEslint } from './eslint-runner.js';
 import { commonAncestorDir } from './fs-util.js';
 import { collectLocallyImportedFiles } from './import-graph.js';
-import { analyzeSource, countLocators, findLocalAssertionHelperNames, looksLikeNonPlaywrightTest, } from './metrics.js';
+import { analyzeSource, countLocators, findLocalAssertionHelperNames, getTestSpans, looksLikeNonPlaywrightTest, } from './metrics.js';
 import { DEFAULT_THRESHOLDS } from './profiles.js';
 import { computeScore } from './score-engine.js';
-export { computeScore, SQS_V1, locatorsScore, gradeFromScore, penaltyDimensionScore, applyPerRuleCap, } from './score-engine.js';
+export { computeScore, MODEL, locatorsScore, gradeFromScore, ratioDimensionScore, } from './score-engine.js';
 export { countSloc } from './sloc.js';
 export { countLocators, analyzeSource } from './metrics.js';
 export { formatJson } from './formatters/json.js';
@@ -110,7 +110,7 @@ function expandPaths(inputs, cwd) {
 }
 function hardFail(profile, threshold, findings, extra = {}, skippedFiles) {
     return {
-        scoreVersion: 'sqs-v1',
+        scoreVersion: 'v3',
         profile,
         score: 0,
         grade: 'F',
@@ -125,6 +125,8 @@ function hardFail(profile, threshold, findings, extra = {}, skippedFiles) {
             warnings: findings.filter((f) => f.severity === 'warning').length,
             nativeLocators: 0,
             rawLocators: 0,
+            unassertedTests: 0,
+            cleanTests: 0,
             ...extra,
         },
         dimensions: {
@@ -138,7 +140,7 @@ function hardFail(profile, threshold, findings, extra = {}, skippedFiles) {
     };
 }
 /**
- * Score Playwright spec files. Deterministic and AI-free (sqs-v1).
+ * Score Playwright spec files. Deterministic and AI-free.
  */
 export async function scorePaths(options) {
     const cwd = options.cwd ?? process.cwd();
@@ -233,15 +235,24 @@ export async function scorePaths(options) {
     let native = 0;
     let raw = 0;
     const metricFindings = [];
+    const fileTests = {};
+    const fileSpans = new Map();
     for (const file of files) {
         const source = sources.get(file) ?? '';
         const relFile = path.relative(filesBase, file) || path.basename(file);
         const m = analyzeSource(source, relFile);
         totalSloc += m.sloc;
-        totalTests += m.tests;
         native += m.locators.native;
         raw += m.locators.raw;
         metricFindings.push(...m.findings);
+        const spans = getTestSpans(source, relFile);
+        fileSpans.set(relFile, spans);
+        // AST spans are the authoritative test count (they are also the
+        // attribution targets); the regex count is the fallback for a file
+        // the parser can't read — which hard-fails below anyway.
+        const testCount = spans.tests.length > 0 ? spans.tests.length : m.tests;
+        fileTests[relFile] = testCount;
+        totalTests += testCount;
     }
     for (const [file, source] of importedFiles) {
         const counts = countLocators(source, file);
@@ -249,6 +260,29 @@ export async function scorePaths(options) {
         raw += counts.raw;
     }
     const findings = [...eslintFindings, ...metricFindings];
+    // Attribute each finding to the test it sits in (model v3): a finding
+    // inside a test span belongs to that test; inside a hook span it affects
+    // every test in the file; anything else — imports, describe bodies,
+    // file-level metrics like oversized-file — is module scope, counted once
+    // per file.
+    for (const f of findings) {
+        const spans = fileSpans.get(f.file);
+        if (!spans || f.line === undefined) {
+            f.scope = 'module';
+            continue;
+        }
+        const idx = spans.tests.findIndex((t) => f.line !== undefined && f.line >= t.startLine && f.line <= t.endLine);
+        if (idx >= 0) {
+            f.scope = 'test';
+            f.testKey = `${f.file}#${idx}`;
+        }
+        else if (spans.hooks.some((h) => f.line !== undefined && f.line >= h.startLine && f.line <= h.endLine)) {
+            f.scope = 'hook';
+        }
+        else {
+            f.scope = 'module';
+        }
+    }
     const parseErrors = findings.filter((f) => f.rule === 'playwright-score/parse-error');
     if (parseErrors.length > 0) {
         // A file that isn't valid JS/TS can't be executed, let alone scored on
@@ -270,6 +304,7 @@ export async function scorePaths(options) {
         sloc: Math.max(totalSloc, 1),
         files: files.length,
         tests: totalTests,
+        fileTests,
         nativeLocators: native,
         rawLocators: raw,
     });

package/dist/metrics.d.ts

@@ -38,6 +38,31 @@ export declare function countLocators(source: string, file?: string): LocatorCou
  * function against each imported dependency file too).
  */
 export declare function findLocalAssertionHelperNames(source: string, file?: string): string[];
+export interface SourceSpan {
+    startLine: number;
+    endLine: number;
+}
+export interface TestSpans {
+    /** Outermost test declaration spans (test/test.only/test.skip/test.fixme/
+     * test.slow calls that carry a function body). Order is source order and
+     * is the stable per-file test index used for finding attribution. */
+    tests: SourceSpan[];
+    /** Hook spans (test.beforeEach/afterEach/beforeAll/afterAll and their
+     * bare-identifier forms) — findings inside these affect every test in
+     * the file, not one. */
+    hooks: SourceSpan[];
+}
+/**
+ * AST spans of test declarations and hooks, for attributing findings to
+ * the test they actually sit in. Only calls carrying a function argument
+ * count — that is what makes `test.skip('t', async () => {...})` (an
+ * always-skipped declaration) a span while the documented runtime form
+ * `test.skip(condition, reason)` inside a test body is not, and likewise
+ * for `test.fixme()` used as a statement. Spans nested inside another test
+ * span (rare, but e.g. a helper declaring a test inside a test would be
+ * malformed anyway) are dropped so each line maps to at most one test.
+ */
+export declare function getTestSpans(source: string, file?: string): TestSpans;
 /** Count test( / test.only( / test.skip( / test.fixme( style declarations (approx). */
 export declare function countTests(source: string): number;
 export declare function findOversizedFile(source: string, file: string, maxSloc?: number): Finding[];

package/dist/metrics.js

@@ -313,6 +313,75 @@ export function findLocalAssertionHelperNames(source, file) {
     }
     return [...names];
 }
+const TEST_MODIFIERS = new Set(['only', 'skip', 'fixme', 'slow']);
+const HOOK_NAMES = new Set(['beforeEach', 'afterEach', 'beforeAll', 'afterAll']);
+function hasFunctionArg(node) {
+    const args = node.arguments;
+    return (args ?? []).some((a) => a?.type === 'ArrowFunctionExpression' || a?.type === 'FunctionExpression');
+}
+/**
+ * AST spans of test declarations and hooks, for attributing findings to
+ * the test they actually sit in. Only calls carrying a function argument
+ * count — that is what makes `test.skip('t', async () => {...})` (an
+ * always-skipped declaration) a span while the documented runtime form
+ * `test.skip(condition, reason)` inside a test body is not, and likewise
+ * for `test.fixme()` used as a statement. Spans nested inside another test
+ * span (rare, but e.g. a helper declaring a test inside a test would be
+ * malformed anyway) are dropped so each line maps to at most one test.
+ */
+export function getTestSpans(source, file) {
+    const tests = [];
+    const hooks = [];
+    const ast = parseSource(source, file);
+    if (!ast)
+        return { tests, hooks };
+    walk(ast, (node) => {
+        if (node.type !== 'CallExpression')
+            return;
+        const callee = node.callee;
+        const loc = node.loc;
+        if (!loc?.start?.line || !loc?.end?.line)
+            return;
+        const span = { startLine: loc.start.line, endLine: loc.end.line };
+        let isTest = false;
+        let isHook = false;
+        if (callee?.type === 'Identifier') {
+            const name = callee.name;
+            if (name === 'test' || name === 'it')
+                isTest = true;
+            else if (name && HOOK_NAMES.has(name))
+                isHook = true;
+        }
+        else if (callee?.type === 'MemberExpression' && !callee.computed) {
+            const obj = callee.object;
+            const prop = callee.property;
+            if (obj?.type === 'Identifier' && (obj.name === 'test' || obj.name === 'it') && prop?.type === 'Identifier') {
+                if (TEST_MODIFIERS.has(prop.name ?? ''))
+                    isTest = true;
+                else if (HOOK_NAMES.has(prop.name ?? ''))
+                    isHook = true;
+            }
+        }
+        if (!isTest && !isHook)
+            return;
+        if (!hasFunctionArg(node))
+            return;
+        if (isTest)
+            tests.push(span);
+        else
+            hooks.push(span);
+    });
+    // Keep outermost test spans only, in source order.
+    tests.sort((a, b) => a.startLine - b.startLine || b.endLine - a.endLine);
+    const outer = [];
+    for (const s of tests) {
+        const last = outer[outer.length - 1];
+        if (last && s.startLine >= last.startLine && s.endLine <= last.endLine)
+            continue;
+        outer.push(s);
+    }
+    return { tests: outer, hooks };
+}
 const TEST_NON_DECL_RE = /\btest\.(describe|step|beforeEach|afterEach|beforeAll|afterAll|use|setTimeout|info|expect)\b/;
 /** Count test( / test.only( / test.skip( / test.fixme( style declarations (approx). */
 export function countTests(source) {

package/dist/score-engine.d.ts

@@ -1,21 +1,51 @@
 import type { DimensionName, Finding, Grade, ProfileName, ScoreResult } from './types.js';
-/** sqs-v1 frozen constants — change requires sqs-v2 */
-export declare const SQS_V1: {
-    readonly scoreVersion: "sqs-v1";
-    readonly ERROR_UNIT: 1;
-    readonly WARNING_UNIT: 0.4;
-    readonly INFO_UNIT: 0;
-    readonly MAX_FINDINGS_PER_RULE_PER_FILE: 3;
-    readonly SLOT_DIVISOR: 25;
-    readonly MIN_SLOTS: 4;
-    readonly K: 0.4;
+/**
+ * Scoring model v3 — "the weighted share of your tests that are clean."
+ *
+ * Every finding is attributed to the test it actually sits in (AST test
+ * spans, resolved in index.ts). Each test accumulates demerit units for a
+ * dimension — an error is a full demerit (1.0), a warning 0.4 — capped at
+ * 1 per test: a test is at worst fully flawed, never more. Findings in a
+ * before/after hook demerit every test in that file (setup problems affect
+ * every test that runs through them); module-level findings count once per
+ * file. A dimension's score is simply:
+ *
+ *   score = 100 × (1 − totalDemerit / tests)
+ *
+ * The locators dimension stays the native/(native+raw) usage ratio.
+ * The final score is the weighted sum of dimensions.
+ *
+ * There is deliberately no per-SLOC density, no slot divisor, no minimum
+ * slots, no exponential decay, and no per-rule finding caps (the per-test
+ * cap of 1 is the anti-nuke mechanism, and it is also the anti-gaming
+ * mechanism: padding a suite with clean lines no longer dilutes anything,
+ * and padding it with fake clean tests costs assertions coverage). Earlier
+ * models (≤0.4.0, published as "sqs-v1"/"sqs-v2") used density math and
+ * scored e.g. a 3-test suite with two hard-wait errors, two assertion-free
+ * tests, and a skipped test at 78/C; model v3 scores it 53/F, which is
+ * what it is.
+ */
+export declare const MODEL: {
+    readonly scoreVersion: "v3";
+    readonly ERROR_DEMERIT: 1;
+    readonly WARNING_DEMERIT: 0.4;
+    readonly INFO_DEMERIT: 0;
+    /** A single test's demerit ceiling per dimension. */
+    readonly MAX_DEMERIT_PER_TEST: 1;
+    /** Module-level findings' demerit ceiling per (file, dimension). */
+    readonly MAX_DEMERIT_PER_FILE_MODULE: 1;
 };
 declare const PENALTY_DIMENSIONS: DimensionName[];
 export declare function gradeFromScore(score: number): Grade;
 export declare function locatorsScore(native: number, raw: number): number;
-/** Cap findings per (file, rule) for penalty math only */
-export declare function applyPerRuleCap(findings: Finding[]): Finding[];
-export declare function penaltyDimensionScore(findings: Finding[], dimension: DimensionName, sloc: number): number;
+/** Tests per file, keyed by the same relative file path findings carry. */
+export type FileTestCounts = Record<string, number>;
+/**
+ * 100 × (1 − demerit share). With zero tests there is nothing to take a
+ * share of — module/hook demerits then act on a denominator of 1 so a
+ * findings-bearing zero-test input still can't score clean.
+ */
+export declare function ratioDimensionScore(findings: Finding[], dimension: DimensionName, fileTests: FileTestCounts, tests: number): number;
 export declare function computeScore(input: {
     profile: ProfileName;
     threshold: number;
@@ -23,6 +53,7 @@ export declare function computeScore(input: {
     sloc: number;
     files: number;
     tests: number;
+    fileTests: FileTestCounts;
     nativeLocators: number;
     rawLocators: number;
 }): ScoreResult;

package/dist/score-engine.js

@@ -1,14 +1,39 @@
 import { PROFILE_WEIGHTS } from './profiles.js';
-/** sqs-v1 frozen constants — change requires sqs-v2 */
-export const SQS_V1 = {
-    scoreVersion: 'sqs-v1',
-    ERROR_UNIT: 1.0,
-    WARNING_UNIT: 0.4,
-    INFO_UNIT: 0.0,
-    MAX_FINDINGS_PER_RULE_PER_FILE: 3,
-    SLOT_DIVISOR: 25,
-    MIN_SLOTS: 4,
-    K: 0.4,
+/**
+ * Scoring model v3 — "the weighted share of your tests that are clean."
+ *
+ * Every finding is attributed to the test it actually sits in (AST test
+ * spans, resolved in index.ts). Each test accumulates demerit units for a
+ * dimension — an error is a full demerit (1.0), a warning 0.4 — capped at
+ * 1 per test: a test is at worst fully flawed, never more. Findings in a
+ * before/after hook demerit every test in that file (setup problems affect
+ * every test that runs through them); module-level findings count once per
+ * file. A dimension's score is simply:
+ *
+ *   score = 100 × (1 − totalDemerit / tests)
+ *
+ * The locators dimension stays the native/(native+raw) usage ratio.
+ * The final score is the weighted sum of dimensions.
+ *
+ * There is deliberately no per-SLOC density, no slot divisor, no minimum
+ * slots, no exponential decay, and no per-rule finding caps (the per-test
+ * cap of 1 is the anti-nuke mechanism, and it is also the anti-gaming
+ * mechanism: padding a suite with clean lines no longer dilutes anything,
+ * and padding it with fake clean tests costs assertions coverage). Earlier
+ * models (≤0.4.0, published as "sqs-v1"/"sqs-v2") used density math and
+ * scored e.g. a 3-test suite with two hard-wait errors, two assertion-free
+ * tests, and a skipped test at 78/C; model v3 scores it 53/F, which is
+ * what it is.
+ */
+export const MODEL = {
+    scoreVersion: 'v3',
+    ERROR_DEMERIT: 1.0,
+    WARNING_DEMERIT: 0.4,
+    INFO_DEMERIT: 0.0,
+    /** A single test's demerit ceiling per dimension. */
+    MAX_DEMERIT_PER_TEST: 1.0,
+    /** Module-level findings' demerit ceiling per (file, dimension). */
+    MAX_DEMERIT_PER_FILE_MODULE: 1.0,
 };
 const PENALTY_DIMENSIONS = ['playwrightHygiene', 'assertions', 'structure'];
 export function gradeFromScore(score) {
@@ -28,39 +53,80 @@ export function locatorsScore(native, raw) {
         return 100;
     return Math.round((100 * native) / total);
 }
-function severityUnit(severity) {
+function severityDemerit(severity) {
     if (severity === 'error')
-        return SQS_V1.ERROR_UNIT;
+        return MODEL.ERROR_DEMERIT;
     if (severity === 'warning')
-        return SQS_V1.WARNING_UNIT;
-    return SQS_V1.INFO_UNIT;
+        return MODEL.WARNING_DEMERIT;
+    return MODEL.INFO_DEMERIT;
 }
-/** Cap findings per (file, rule) for penalty math only */
-export function applyPerRuleCap(findings) {
-    const counts = new Map();
-    const out = [];
+function demeritsForDimension(findings, dimension, fileTests) {
+    const perTest = new Map();
+    const perFileHook = new Map();
+    const perFileModule = new Map();
     for (const f of findings) {
-        if (f.reportOnly)
+        if (f.dimension !== dimension || f.reportOnly)
             continue;
-        const key = `${f.file}::${f.rule}`;
-        const n = counts.get(key) ?? 0;
-        if (n >= SQS_V1.MAX_FINDINGS_PER_RULE_PER_FILE)
+        const d = severityDemerit(f.severity);
+        if (d === 0)
             continue;
-        counts.set(key, n + 1);
-        out.push(f);
+        if (f.scope === 'test' && f.testKey) {
+            perTest.set(f.testKey, (perTest.get(f.testKey) ?? 0) + d);
+        }
+        else if (f.scope === 'hook') {
+            perFileHook.set(f.file, (perFileHook.get(f.file) ?? 0) + d);
+        }
+        else {
+            perFileModule.set(f.file, (perFileModule.get(f.file) ?? 0) + d);
+        }
     }
-    return out;
-}
-export function penaltyDimensionScore(findings, dimension, sloc) {
-    const dimFindings = findings.filter((f) => f.dimension === dimension);
-    const capped = applyPerRuleCap(dimFindings);
-    let rawUnits = 0;
-    for (const f of capped) {
-        rawUnits += severityUnit(f.severity);
+    let total = 0;
+    const flawedTestKeys = new Set();
+    // Per-test demerits: each test's own findings plus its file's hook
+    // findings, capped at 1 — a test is at worst fully flawed.
+    for (const [file, count] of Object.entries(fileTests)) {
+        const hookUnits = perFileHook.get(file) ?? 0;
+        for (let i = 0; i < count; i++) {
+            const key = `${file}#${i}`;
+            const units = (perTest.get(key) ?? 0) + hookUnits;
+            if (units <= 0)
+                continue;
+            total += Math.min(MODEL.MAX_DEMERIT_PER_TEST, units);
+            flawedTestKeys.add(key);
+        }
+    }
+    // A test-scoped finding whose file has no counted tests (should not
+    // happen, but attribution and counting are separate passes) still
+    // counts, capped the same way.
+    for (const [key, units] of perTest) {
+        const file = key.slice(0, key.lastIndexOf('#'));
+        if (fileTests[file] !== undefined)
+            continue;
+        total += Math.min(MODEL.MAX_DEMERIT_PER_TEST, units);
+        flawedTestKeys.add(key);
+    }
+    // Hook findings in a file with zero tests (setup-only file swept in):
+    // count once per file, like module scope.
+    for (const [file, units] of perFileHook) {
+        if ((fileTests[file] ?? 0) > 0)
+            continue;
+        total += Math.min(MODEL.MAX_DEMERIT_PER_FILE_MODULE, units);
+    }
+    // Module-level findings count once per file.
+    for (const units of perFileModule.values()) {
+        total += Math.min(MODEL.MAX_DEMERIT_PER_FILE_MODULE, units);
     }
-    const slots = Math.max(sloc / SQS_V1.SLOT_DIVISOR, SQS_V1.MIN_SLOTS);
-    const load = rawUnits / slots;
-    const score = Math.round(100 * Math.exp(-SQS_V1.K * load));
+    return { total, flawedTestKeys };
+}
+/**
+ * 100 × (1 − demerit share). With zero tests there is nothing to take a
+ * share of — module/hook demerits then act on a denominator of 1 so a
+ * findings-bearing zero-test input still can't score clean.
+ */
+export function ratioDimensionScore(findings, dimension, fileTests, tests) {
+    const { total } = demeritsForDimension(findings, dimension, fileTests);
+    const denominator = Math.max(tests, 1);
+    const score = Math.round(100 * (1 - total / denominator));
     return Math.min(100, Math.max(0, score));
 }
 export function computeScore(input) {
@@ -71,10 +137,10 @@ export function computeScore(input) {
     // score instead of a hard crash.
     const weights = PROFILE_WEIGHTS[input.profile] ?? PROFILE_WEIGHTS.standard;
     const dims = {
-        playwrightHygiene: penaltyDimensionScore(input.findings, 'playwrightHygiene', input.sloc),
-        assertions: penaltyDimensionScore(input.findings, 'assertions', input.sloc),
+        playwrightHygiene: ratioDimensionScore(input.findings, 'playwrightHygiene', input.fileTests, input.tests),
+        assertions: ratioDimensionScore(input.findings, 'assertions', input.fileTests, input.tests),
         locators: locatorsScore(input.nativeLocators, input.rawLocators),
-        structure: penaltyDimensionScore(input.findings, 'structure', input.sloc),
+        structure: ratioDimensionScore(input.findings, 'structure', input.fileTests, input.tests),
     };
     let score = 0;
     for (const [dim, weight] of Object.entries(weights)) {
@@ -87,8 +153,18 @@ export function computeScore(input) {
     score = Math.round(score);
     const errors = input.findings.filter((f) => f.severity === 'error').length;
     const warnings = input.findings.filter((f) => f.severity === 'warning').length;
+    // Clean tests: no demerit in any penalty dimension. (Locator usage is a
+    // suite-level ratio, not a per-test property.)
+    const flawed = new Set();
+    for (const dim of PENALTY_DIMENSIONS) {
+        for (const key of demeritsForDimension(input.findings, dim, input.fileTests)
+            .flawedTestKeys) {
+            flawed.add(key);
+        }
+    }
+    const cleanTests = Math.max(0, input.tests - flawed.size);
     return {
-        scoreVersion: SQS_V1.scoreVersion,
+        scoreVersion: MODEL.scoreVersion,
         profile: input.profile,
         score,
         grade: gradeFromScore(score),
@@ -103,6 +179,8 @@ export function computeScore(input) {
             warnings,
             nativeLocators: input.nativeLocators,
             rawLocators: input.rawLocators,
+            unassertedTests: input.findings.filter((f) => f.rule === 'playwright/expect-expect' && !f.reportOnly).length,
+            cleanTests,
         },
         dimensions: dims,
         findings: input.findings,

package/dist/types.d.ts

@@ -1,4 +1,4 @@
-export type ScoreVersion = 'sqs-v1';
+export type ScoreVersion = 'v3';
 export type ProfileName = 'standard';
 export type Grade = 'A' | 'B' | 'C' | 'D' | 'F';
 export type Severity = 'error' | 'warning' | 'info';
@@ -13,6 +13,13 @@ export interface Finding {
     dimension: DimensionName;
     /** When true, finding is listed but does not contribute penalty units */
     reportOnly?: boolean;
+    /** Where the finding sits: inside a specific test ('test', with testKey),
+     * inside a before/after hook ('hook' — affects every test in the file),
+     * or at module level ('module' — counted once per file). Set during
+     * attribution in index.ts; findings without it are treated as 'module'. */
+    scope?: 'test' | 'hook' | 'module';
+    /** `${file}#${testIndex}` when scope === 'test'. */
+    testKey?: string;
 }
 export interface ScoreDimensions {
     playwrightHygiene: number;
@@ -29,6 +36,12 @@ export interface ScoreSummary {
     warnings: number;
     nativeLocators: number;
     rawLocators: number;
+    /** Tests with no recognized assertion (uncapped playwright/expect-expect
+     * census, after delegation resolution). */
+    unassertedTests: number;
+    /** Tests with no demerit in any dimension — the headline the score is
+     * built from: the weighted share of tests that are clean. */
+    cleanTests: number;
 }
 export interface ScoreResult {
     scoreVersion: ScoreVersion;

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@qaguardian/playwright-score",
-  "version": "0.3.0",
-  "description": "Deterministic, AI-free Playwright spec quality score (sqs-v1).",
+  "version": "0.5.0",
+  "description": "Deterministic, AI-free Playwright spec quality score (sqs-v2).",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",