@qaguardian/playwright-score 0.2.0 → 0.3.0

package/METHODOLOGY.md

@@ -161,4 +161,14 @@ See package rule docs / source `profiles.ts`. High level:
 - **playwrightHygiene:** waits, force, networkidle, missing await, handles, conditionals  
 - **assertions:** expect-expect, empty tests, web-first preference  
 - **locators:** ratio metric only (for score); raw-locator ESLint in findings  
-- **structure:** focused/skipped, oversized file, describe shape  
+- **structure:** focused/skipped, oversized file, describe shape
+
+### Active rule set (frozen)
+
+The exact ESLint rules that produce sqs-v1 findings are enabled explicitly
+in `eslint-runner.ts` with fixed severities — the score never inherits
+eslint-plugin-playwright's `recommended` set at install time, so identical
+code scores identically regardless of which plugin version npm resolves.
+(Versions ≤0.2.0 intended to inherit `recommended` but a config-shape bug
+dropped it entirely; 0.3.0 froze the intended set explicitly. See
+CHANGELOG.)  

package/README.md

@@ -16,7 +16,44 @@ Page Object Model import resolution, ...).
 
 The score never calls an LLM. AI may *generate* or *repair* code using findings; **rules grade code**.
 
-See [METHODOLOGY.md](./METHODOLOGY.md) for the frozen formula, [VALIDATION.md](./VALIDATION.md) for real-world results against public repos (cal.com, Grafana, Mattermost, ...) and a re-runnable script to reproduce them, or the full product write-up on the [landing page](https://qaguardian.com/open-source/playwright-score).
+## Real-world results
+
+17 public Playwright suites, scored with the published package against each
+project's actual source — chosen to include both well-known,
+heavily-engineered platforms and smaller, less mature projects. Not curated
+to look good: 3 of 17 fail the default threshold, for real, verifiable
+reasons (see [VALIDATION.md](./VALIDATION.md)).
+
+**17 suites · 1,228 files · 6,037 tests · 14/17 passed (80% threshold)**
+
+| Repo (source scanned) | Score | Grade | Result |
+|---|---:|:-:|:-:|
+| [Playwright (own TodoMVC example)](https://github.com/microsoft/playwright/tree/main/examples/todomvc) | 98/100 | A | PASS |
+| [Supabase](https://github.com/supabase/supabase/tree/master/e2e) | 97/100 | A | PASS |
+| [Documenso](https://github.com/documenso/documenso/tree/main/packages/app-tests) | 96/100 | A | PASS |
+| [Storybook](https://github.com/storybookjs/storybook/tree/next/code/e2e-internal) | 96/100 | A | PASS |
+| [freeCodeCamp](https://github.com/freeCodeCamp/freeCodeCamp/tree/main/e2e) | 96/100 | A | PASS |
+| [n8n](https://github.com/n8n-io/n8n/tree/master/packages/testing/playwright) | 95/100 | A | PASS |
+| [dub](https://github.com/dubinc/dub/tree/main/apps/web/playwright) | 95/100 | A | PASS |
+| [novu](https://github.com/novuhq/novu/tree/next/apps/dashboard/tests) | 93/100 | A | PASS |
+| [Grafana](https://github.com/grafana/grafana/tree/main/e2e-playwright) | 91/100 | A | PASS |
+| [PostHog](https://github.com/PostHog/posthog/tree/master/playwright) | 90/100 | A | PASS |
+| [sencho](https://github.com/Studio-Saelix/sencho/tree/main/e2e) | 90/100 | A | PASS |
+| [Mattermost](https://github.com/mattermost/mattermost/tree/master/e2e-tests/playwright) | 89/100 | B | PASS |
+| [Immich](https://github.com/immich-app/immich/tree/main/e2e/src/specs) | 89/100 | B | PASS |
+| [cal.com](https://github.com/calcom/cal.com/tree/main/apps/web/playwright) | 84/100 | B | PASS |
+| [livecodes](https://github.com/live-codes/livecodes/tree/develop/e2e) | 78/100 | C | FAIL |
+| [openplayerjs](https://github.com/openplayerjs/openplayerjs/tree/master/e2e) | 77/100 | C | FAIL |
+| [TheCyberHub](https://github.com/th3cyb3rhub/TheCyberHub/tree/dev/e2e) | 68/100 | D | FAIL |
+
+Repo names link to the exact source scanned. Full methodology, findings
+breakdown, and `scripts/validate-corpus.sh` to reproduce every number here
+yourself live in [VALIDATION.md](./VALIDATION.md). A self-contained visual
+report of an earlier five-suite audit lives at
+[`docs/scorecard.html`](./docs/scorecard.html) — host it wherever
+(GitHub Pages, qaguardian.com, ...).
+
+See [METHODOLOGY.md](./METHODOLOGY.md) for the frozen formula, or the full product write-up on the [landing page](https://qaguardian.com/open-source/playwright-score).
 
 ## Install
 

package/dist/eslint-runner.js

@@ -5,8 +5,6 @@ import playwright from 'eslint-plugin-playwright';
 import { basePlugin } from './rules/base-plugin.js';
 import { mapRule } from './profiles.js';
 function buildConfig(assertFunctionNames) {
-    const playwrightRecommended = playwright.configs?.['flat/recommended'] ??
-        playwright.configs?.recommended;
     const base = [
         {
             files: ['**/*.{ts,tsx,js,jsx,mjs,cjs}'],
@@ -89,14 +87,34 @@ function buildConfig(assertFunctionNames) {
                 'playwright/prefer-native-locators': 'warn',
                 // Matches eslint-plugin-playwright's own recommended severity.
                 'playwright/prefer-web-first-assertions': 'error',
+                // The rules below complete the sqs-v1 rule set from
+                // profiles.ts's ESLINT_RULE_MAP. Earlier versions (<=0.2.0)
+                // expected them to arrive via eslint-plugin-playwright's
+                // flat/recommended config, but the spread only handled an array
+                // shape while the plugin exports a single config object — so the
+                // entire recommended set was silently dropped and these rules
+                // never ran (METHODOLOGY.md's hygiene dimension lists
+                // "conditionals" and "handles", yet neither could ever fire).
+                // They are enabled explicitly and frozen here on purpose:
+                // eslint-plugin-playwright is a ^range dependency, and inheriting
+                // whatever its recommended set says at install time would make
+                // identical code score differently across installs. Severities
+                // mirror upstream flat/recommended as of eslint-plugin-playwright
+                // 2.x. require-top-level-describe stays disabled deliberately —
+                // it is stylistic, not in upstream recommended, and would
+                // penalize idiomatic describe-less suites.
+                'playwright/no-conditional-in-test': 'warn',
+                'playwright/no-conditional-expect': 'warn',
+                'playwright/no-element-handle': 'warn',
+                'playwright/no-eval': 'warn',
+                'playwright/no-page-pause': 'warn',
+                'playwright/no-useless-await': 'warn',
+                'playwright/valid-expect': 'error',
+                'playwright/no-standalone-expect': 'error',
+                'playwright/max-nested-describe': 'warn',
             },
         },
     ];
-    // Prefer spreading official recommended flat config when available
-    if (Array.isArray(playwrightRecommended)) {
-        const configs = playwrightRecommended;
-        return [...configs, base[0]];
-    }
     return base;
 }
 function toSeverity(eslintSeverity, override) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qaguardian/playwright-score",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Deterministic, AI-free Playwright spec quality score (sqs-v1).",
   "type": "module",
   "main": "./dist/index.js",