@sentinelqa/playwright-reporter 0.1.54 → 0.1.56

package/README.md

@@ -1,68 +1,38 @@
-# Playwright Reporter
+# @sentinelqa/playwright-reporter
 
-After every failed Playwright run, Sentinel gives you:
-
-- a clear CLI diagnosis
-- a shareable debugging link
-- grouped failures by likely root cause
-
-Sample run: https://app.sentinelqa.com/share/1f343d91-be17-4c14-b1b9-2d4e8ef448d2
+Deterministic Playwright failure triage: CLI diagnosis, root-cause grouping, and a shareable debugging report.
 
 [![npm](https://img.shields.io/npm/v/@sentinelqa/playwright-reporter)](https://www.npmjs.com/package/@sentinelqa/playwright-reporter)
 [![downloads](https://img.shields.io/npm/dm/@sentinelqa/playwright-reporter)](https://www.npmjs.com/package/@sentinelqa/playwright-reporter)
 [![license](https://img.shields.io/npm/l/@sentinelqa/playwright-reporter)](./LICENSE)
 
-From failed CI run to root cause in seconds.
-
-Works with no account and no API key.
-
-![Sentinel Report Example](./docs/screenshot2.png)
-![CLI Quick Diagnosis](./docs/CLI2.png)
-
-## What It Does
-
-Sentinel explains Playwright failures locally, groups them into real root causes, and gives you a shareable link to inspect the run.
-
-The open-source reporter is focused on one job:
+## What You Get
 
-- tell you what broke
-- show why it failed
-- point to the likely root cause
-- tell you what to check next
+SAMPLE REPORT: https://sentinelqa.com/share/run/permanent-demo-playwright-report
 
-## Features
+![CLI Output](./docs/cli.png)
 
-- Free hosted debugging links by default
-- Deterministic CLI diagnosis after failed runs
-- Failure grouping by likely root cause
-- Shared run page with traces, screenshots, videos, and logs
-- Failure pages that explain what broke, why, where, and what to do next
-- Better timeout, actionability, assertion, navigation, and backend failure wording
-- Public share links that are easy to paste into Slack, PRs, and issues
-- Works with existing Playwright reporter setup
+After a failed Playwright run, Sentinel prints:
 
-## Why Teams Use It
-
-- You do not need to open raw logs first
-- Repeated failures collapse into a smaller number of real issues
-- The CLI explains the failure immediately in CI
-- Anyone on the team can open the same shared debugging link
-- The hosted report keeps the explanation, artifacts, and grouped failures in one place
+- a deterministic CLI diagnosis (1–3 root causes)
+- how many tests are affected per root cause
+- what to inspect first (usually trace)
+- a report link (hosted by default, local in offline mode)
 
 ## Requirements
 
 - Node.js 18+
-- `@playwright/test` 1.40+
-
-## Quick Start
+- `@playwright/test` 1.40+ (newer is recommended)
 
-Install:
+## Install
 
 ```bash
 npm install -D @sentinelqa/playwright-reporter
 ```
 
-Add Sentinel to your Playwright config:
+## Setup (Recommended)
+
+Wrap your `playwright.config.ts`:
 
 ```ts
 import { defineConfig } from "@playwright/test";
@@ -72,11 +42,6 @@ export default withSentinel(
   defineConfig({
     reporter: [["line"]],
     outputDir: "test-results",
-    use: {
-      trace: "retain-on-failure",
-      screenshot: "only-on-failure",
-      video: "retain-on-failure",
-    },
   }),
   {
     project: "my-app",
@@ -84,91 +49,144 @@ export default withSentinel(
 );
 ```
 
-Run your tests:
+Run tests normally:
 
 ```bash
 npx playwright test
 ```
 
-If tests fail, Sentinel uploads the run and prints a shareable link in the terminal.
+## How It Works (High Level)
+
+Sentinel is a Playwright reporter that:
+
+1. Ensures a Playwright JSON report exists (adds a JSON reporter if needed).
+2. Collects the artifacts Playwright already produces (`trace.zip`, screenshots, video, logs, report.json).
+3. Builds a **deterministic** failure summary:
+   - groups repeated failures into 1–3 canonical root causes
+   - extracts normalized evidence (where, blocker, target state, expected/received)
+4. Publishes a report:
+   - default: a hosted share link
+   - offline: a local HTML report folder (no upload)
+
+This project is intentionally heuristic-driven (not “AI guesses”) for root-cause grouping.
+
+## Modes
+
+Sentinel behavior is controlled by `SENTINEL_MODE`.
+
+## Pricing Tiers (How This Maps)
+
+- **Community (Open Source)**
+  - “Everything local”: generate the same report UI fully offline, no upload required.
+  - “Bring your own storage + DB”: artifacts and metadata stay in your VPC (S3/GCS + Postgres).
+  - “Deterministic root-cause clustering”: same grouping logic as the hosted product, no AI guessing.
+  - “No limits”: unlimited runs because you own infra.
+  - “Security posture”: secrets masking happens before anything leaves your environment (and in offline, nothing leaves).
+  - Share links are optional: run in hosted mode when you want a public URL, or offline mode for zero egress.
+
+- **Personal ($0)**
+  - “Zero setup”: install + run, auto-upload on failures.
+  - “Share links for PRs”: reviewers open a read-only report without reproducing locally.
+  - “Retention”: keep enough history to see patterns (200 runs / 14 days).
+  - “Safe-by-default”: automatic secret masking before upload.
+  - “Works with shards”: parallel jobs still collapse into root-cause clusters.
+
+- **Pro ($50/mo)**
+  - “Stop repeating fixes”: recurring failure tracking across commits (first seen, frequency, impact).
+  - “Compare runs”: last known good vs now, isolate what changed.
+  - “Team workflow”: shared workspace, multiple projects, roles.
+  - “High volume”: 10,000 run history and longer retention.
+  - “Faster decision making”: regressions vs flakes becomes obvious (trend + history).
+
+The reporter package works in all tiers; the difference is where reports live and how much history you get.
+
+### Hosted Mode (Default)
+
+Do nothing. Sentinel uploads and prints a share link when failures happen.
+
+Environment:
+
+- `SENTINEL_MODE` unset (or `SENTINEL_MODE=hosted`)
+
+### Workspace Mode (Private History)
+
+If you have a workspace token:
+
+```bash
+SENTINEL_TOKEN=your_project_ingest_token npx playwright test
+```
+
+Environment:
 
-## CLI Output
+- `SENTINEL_MODE` unset (or `SENTINEL_MODE=hosted`)
+- `SENTINEL_TOKEN=...`
 
-On failed runs, Sentinel prints a compact diagnosis that answers:
+### Offline Mode (No Uploads)
 
-1. What failed?
-2. What caused it?
-3. Where is it?
-4. What should I check next?
+To keep everything local:
+
+```bash
+SENTINEL_MODE=offline npx playwright test
+```
 
-Typical output:
+Offline mode is strict:
 
-```text
-Sentinel diagnosis
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+- uploads are skipped
+- a local report is generated (default `./sentinel-report/index.html`)
+- the CLI prints a local `file://` link (or a relative path)
 
-RECURRING FAILURE (12 previous runs)
+If a hosted upload fails, Sentinel falls back to generating the local report automatically.
 
-❌ 4 failures (grouped)
-Collapsed into 2 real issues
+### Local Workspace Uploads (When Not In CI)
 
-Issue 1: Blocked click on getByRole('button', { name: 'Save' }) (3 tests)
-  Cause:        .modal-backdrop intercepted pointer events before the click completed
-  Where:        checkout.spec.ts:22; billing.spec.ts:41
-  Failing step: Click on getByRole('button', { name: 'Save' }) selector
-  Blocker:      .modal-backdrop intercepting pointer events
-  Likely fix:   dismiss or wait for .modal-backdrop to disappear before clicking Save
-  Impact:       3 tests failing with same root cause
+If you set `SENTINEL_TOKEN` locally, Sentinel will _not_ upload by default (to avoid accidental data egress).
+To allow a local upload to your workspace, set:
 
-Issue 2: Assertion mismatch (getByTestId('metric-pass-rate')) (1 test)
-  Cause:        getByTestId('metric-pass-rate') showed "88%" instead of "100%"
-  Where:        app.spec.ts:68
-  Failing step: Expect text on getByTestId('metric-pass-rate') selector
-  Expected:     100%
-  Received:     88%
-  Likely fix:   fix the UI state or data behind getByTestId('metric-pass-rate') so it shows "100%"
-  Impact:       1 test failing with this root cause
+```bash
+SENTINEL_UPLOAD_LOCAL=1 SENTINEL_TOKEN=your_project_ingest_token npx playwright test
 ```
 
-On passed runs, Sentinel stays quiet unless there is a strong risk signal worth surfacing.
+This is useful for quickly generating a private run history entry from your laptop.
+
+### Implicit Local Public Upload (No Token, Not In CI)
 
-## What The Shared Report Shows
+If you are not in CI and you did not set `SENTINEL_TOKEN`, Sentinel can still upload in public mode by enabling:
 
-The hosted report is simple on purpose.
+```bash
+SENTINEL_UPLOAD_LOCAL=1 npx playwright test
+```
 
-It focuses on:
+### Share Links Are Optional (Open Source / Self-Hosted)
 
-- grouped failures
-- explanation of why each issue failed
-- affected tests
-- artifacts
-- recent history
+- Hosted mode prints a share URL on failures.
+- Offline mode skips uploads and produces only a local report.
 
-Open the link to inspect:
+Pick the behavior explicitly with `SENTINEL_MODE` (and `SENTINEL_UPLOAD_LOCAL` for local uploads).
 
-- failures grouped by likely root cause
-- whether multiple failures appear related
-- the best available explanation for each issue
-- screenshots, traces, videos, and logs
-- direct links you can share with the rest of the team
+## Serving Local Reports
 
-## Free Hosted Mode
+Local reports are static HTML + copied artifacts. You can open `index.html` directly, or run a tiny local server:
+
+```bash
+cd sentinel-report
+npx --yes serve -p 4173 .
+```
 
-If `SENTINEL_TOKEN` is not set, the reporter uploads the run to a free hosted Sentinel report and prints the public share URL.
+Then open `http://localhost:4173`.
 
-This free flow includes:
+## Secrets Masking
 
-- CLI diagnosis
-- hosted run page
-- grouped failures
-- failure explanation
-- public share links
+Sentinel masks common secret patterns before data is shared. This includes:
 
-## Optional Richer Capture
+- environment-variable looking strings
+- common credential/token patterns
+- internal URLs/hosts (where possible)
 
-`withSentinel()` works on its own and is the simplest setup.
+If you find something that should be masked but isn’t, file an issue with a minimal reproducible example (redact the secret).
 
-If you want richer failure evidence, you can also attach the failure capture fixture:
+## Optional: Richer Failure Evidence
+
+`withSentinel()` is enough for most setups. If you want richer UI evidence for actionability/timeouts, you can attach the failure capture fixture:
 
 ```ts
 // tests/test.ts
@@ -185,24 +203,7 @@ Then import from that file in your specs:
 import { test, expect } from "./test";
 ```
 
-This improves the quality of:
-
-- blocked action diagnosis
-- assertion mismatch diagnosis
-- timeout explanations
-- DOM and state evidence in the report
-
-## What `withSentinel()` Does
-
-- Preserves your existing reporter configuration
-- Injects a Playwright JSON reporter if one is missing
-- Sets sensible artifact defaults:
-  - trace: `retain-on-failure`
-  - screenshot: `only-on-failure`
-  - video: `retain-on-failure`
-- Uploads the run to Sentinel at the end of the test run
-
-## Options
+## Configuration Options
 
 ```ts
 withSentinel(config, {
@@ -215,12 +216,23 @@ withSentinel(config, {
 });
 ```
 
-## Workspace
+## Troubleshooting
 
-If you want private run history, recurring issue tracking, and team access, sign up for a Sentinel workspace and set `SENTINEL_TOKEN`.
+### “POST /api/runs failed …”
 
-```bash
-SENTINEL_TOKEN=your_project_ingest_token npx playwright test
-```
+- Check `SENTINEL_MODE` (offline skips uploads).
+- If using a workspace token, confirm `SENTINEL_TOKEN` is correct.
+- If a server schema changed, upgrade the reporter/uploader and/or run DB migrations for self-hosted deployments.
+
+### “Local report generation failed …”
+
+Make sure Playwright produced:
+
+- `playwright-report/report.json`
+- `test-results/` (or your configured `outputDir`)
+
+Then rerun with `SENTINEL_MODE=offline` to force local generation.
+
+## License
 
-Create a workspace at [sentinelqa.com/register](https://sentinelqa.com/register).
+See [LICENSE](./LICENSE).