@testrelic/playwright-analytics 2.0.0 → 2.1.0

package/README.md

@@ -1,42 +1,29 @@
-# @testrelic/playwright-analytics
+# TestRelic
 
-A Playwright custom reporter and navigation-tracking fixture that produces a structured JSON timeline of every page visit, network request, and test result in your test suite.
+A test analytics toolkit that generates structured JSON and HTML reports from your Playwright test runs — capturing navigation timelines, API call tracking, network statistics, failure diagnostics, and CI metadata. Supports browser E2E tests, API-only tests, and unified workflows that combine both.
 
 ## What It Does
 
-When you run your Playwright tests with this SDK, it generates a single JSON report that captures:
+When you run your Playwright tests with the TestRelic reporter and fixture, you get a JSON + HTML report that captures:
 
-- **Navigation timeline** — Every URL visited during each test, in chronological order
-- **Network statistics** — Total requests, bytes transferred, and breakdowns by resource type (scripts, images, stylesheets, fonts, XHR, etc.)
-- **Test results** — Pass/fail status, duration, retry count, and tags for every test
-- **Failure diagnostics** — Error messages, stack traces, and source code snippets pointing to the exact line that failed
-- **CI metadata** — Automatic detection of GitHub Actions, GitLab CI, Jenkins, and CircleCI with build ID, commit SHA, and branch
-- **SPA detection** — Tracks `pushState`, `replaceState`, `popstate`, and `hashchange` navigations in single-page applications
-- **Sensitive data redaction** — Automatically scrubs AWS keys, Bearer tokens, private keys, and credential URLs from the report
+- **Navigation timeline** — Every URL visited during each test, in chronological order, with navigation type detection (goto, link click, back/forward, SPA route changes, hash changes)
+- **API call tracking** — Captures HTTP method, URL, status code, request/response headers and bodies, response times, and assertions for every API call
+- **Network statistics** — Total requests, failed requests, bytes transferred, and resource type breakdowns (scripts, images, stylesheets, fonts, XHR)
+- **Test results** — Pass/fail/flaky status, duration, retry count, and tags for every test
+- **Failure diagnostics** — Error messages, source code snippets pointing to the exact failure line, and optional stack traces
+- **CI metadata** — Auto-detection of GitHub Actions, GitLab CI, Jenkins, and CircleCI with build ID, commit SHA, and branch
+- **Sensitive data redaction** — Automatically scrubs AWS keys, Bearer tokens, private keys, credential URLs, and sensitive API headers/body fields
+- **HTML report** — Self-contained interactive HTML report with test grid, filter bar, and expandable drawer showing navigation timeline, API call details, and artifacts
 
-## Prerequisites
-
-- **Node.js** >= 18
-- **Playwright** >= 1.35.0
-- **Package manager**: npm, pnpm, or yarn
+## Quick Start
 
-## Installation
+### 1. Install
 
 ```bash
 npm install @testrelic/playwright-analytics
 ```
 
-```bash
-pnpm add @testrelic/playwright-analytics
-```
-
-```bash
-yarn add @testrelic/playwright-analytics
-```
-
-## Quick Start
-
-### 1. Add the reporter to your Playwright config
+### 2. Add the reporter to your Playwright config
 
 ```typescript
 // playwright.config.ts
@@ -55,68 +42,61 @@ export default defineConfig({
 });
 ```
 
-### 2. Use the fixture in your tests
+### 3. Use the fixture in your tests
+
+TestRelic provides two fixture modes depending on your testing needs:
 
-Replace your Playwright `test` import with the TestRelic fixture to enable automatic navigation tracking:
+**E2E tests (browser)** — uses `page` for navigation tracking:
 
 ```typescript
-// my-test.spec.ts
 import { test, expect } from '@testrelic/playwright-analytics/fixture';
 
-test('homepage loads correctly', async ({ page }) => {
+test('homepage loads correctly', { tag: ['@e2e'] }, async ({ page }) => {
   await page.goto('https://example.com');
   await expect(page.locator('h1')).toBeVisible();
 });
 ```
 
-That's it. Run your tests as usual:
+**API tests** — uses `request` for API call tracking:
 
-```bash
-npx playwright test
+```typescript
+import { test as base } from '@playwright/test';
+import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture';
+import { expect } from '@testrelic/playwright-analytics/fixture';
+
+const test = base.extend(testRelicApiFixture);
+
+test('fetch posts', { tag: ['@api'] }, async ({ request }) => {
+  const response = await request.get('https://api.example.com/posts');
+  expect(response.status()).toBe(200);
+});
 ```
 
-The JSON report will be written to `./test-results/analytics-timeline.json` (or wherever you set `outputPath`).
+**Unified tests (browser + API)** — uses both `page` and `request` together:
 
-## Configuration Options
+```typescript
+import { test, expect } from '@testrelic/playwright-analytics/fixture';
 
-All options are passed as the second element of the reporter tuple in your Playwright config:
+test('API data matches UI', { tag: ['@e2e', '@api'] }, async ({ page, request }) => {
+  // API call
+  const apiResponse = await request.get('https://api.example.com/user/1');
+  const user = await apiResponse.json();
 
-| Option | Type | Default | Description |
-|---|---|---|---|
-| `outputPath` | `string` | `./test-results/analytics-timeline.json` | Where to write the JSON report |
-| `includeStackTrace` | `boolean` | `false` | Include full stack traces in failure diagnostics |
-| `includeCodeSnippets` | `boolean` | `true` | Include source code snippets around the failure line |
-| `codeContextLines` | `number` | `3` | Number of lines above/below the failure line to include |
-| `includeNetworkStats` | `boolean` | `true` | Track network requests and byte sizes per navigation |
-| `navigationTypes` | `NavigationType[] \| null` | `null` (all) | Filter timeline to specific navigation types only |
-| `redactPatterns` | `(string \| RegExp)[]` | Built-in patterns | Additional patterns to redact from error messages and stacks |
-| `testRunId` | `string \| null` | `null` (auto UUID) | Override the test run ID |
-| `metadata` | `Record<string, unknown> \| null` | `null` | Attach custom metadata to the report |
+  // Browser navigation
+  await page.goto('https://example.com/profile');
+  await expect(page.locator('.user-name')).toHaveText(user.name);
+});
+```
 
-### Configuration Example
+### 4. Run your tests
 
-```typescript
-['@testrelic/playwright-analytics', {
-  outputPath: './reports/timeline.json',
-  includeStackTrace: true,
-  includeCodeSnippets: true,
-  codeContextLines: 5,
-  includeNetworkStats: true,
-  navigationTypes: ['goto', 'navigation', 'back', 'forward'],
-  metadata: {
-    team: 'frontend',
-    environment: 'staging',
-  },
-  redactPatterns: [
-    /my-secret-pattern/g,
-    'literal-string-to-redact',
-  ],
-}]
+```bash
+npx playwright test
 ```
 
-## Output Format
+The JSON report will be written to `./test-results/analytics-timeline.json` and the HTML report to `./test-results/analytics-timeline.html`.
 
-The report is a single JSON file with this structure:
+## Output Example
 
 ```jsonc
 {
@@ -132,17 +112,20 @@ The report is a single JSON file with this structure:
     "flaky": 0,
     "skipped": 0
   },
-  "ci": null,                // Auto-populated in CI environments
-  "metadata": null,          // Custom metadata from config
+  "ci": {
+    "provider": "github-actions",
+    "buildId": "12345678",
+    "commitSha": "abc123def456",
+    "branch": "main"
+  },
   "timeline": [
     {
       "url": "https://en.wikipedia.org/wiki/Main_Page",
       "navigationType": "goto",
       "visitedAt": "2026-02-07T10:41:29.844Z",
       "duration": 216,
-      "specFile": "sdk-validation.spec.ts",
+      "specFile": "tests/homepage.spec.ts",
       "domContentLoadedAt": "2026-02-07T10:41:30.200Z",
-      "networkIdleAt": null,
       "networkStats": {
         "totalRequests": 40,
         "failedRequests": 0,
@@ -159,103 +142,120 @@ The report is a single JSON file with this structure:
       },
       "tests": [
         {
-          "title": "sdk-validation.spec.ts > SDK E2E Validation > homepage visit",
+          "title": "homepage.spec.ts > Homepage > loads correctly",
           "status": "passed",
           "duration": 1028,
-          "startedAt": "2026-02-07T10:41:29.133Z",
-          "completedAt": "2026-02-07T10:41:30.161Z",
-          "retryCount": 0,
-          "tags": [],
           "failure": null
         }
       ]
     }
-    // ... more timeline entries
-  ],
-  "shardRunIds": null        // Populated when merging shard reports
+  ]
 }
 ```
 
-### Timeline Entry Fields
-
-| Field | Type | Description |
-|---|---|---|
-| `url` | `string` | The URL that was navigated to |
-| `navigationType` | `string` | How the navigation happened (see navigation types below) |
-| `visitedAt` | `string` | ISO-8601 timestamp of the navigation |
-| `duration` | `number` | Milliseconds until the next navigation or test end |
-| `specFile` | `string` | Relative path to the spec file |
-| `domContentLoadedAt` | `string \| null` | When DOMContentLoaded fired |
-| `networkIdleAt` | `string \| null` | When network became idle |
-| `networkStats` | `object \| null` | Network request statistics for this navigation |
-| `tests` | `array` | Test results associated with this navigation |
-
-### Navigation Types
-
-| Type | Description |
-|---|---|
-| `goto` | `page.goto()` call |
-| `navigation` | Browser frame navigation (link click, form submit) |
-| `back` | `page.goBack()` call |
-| `forward` | `page.goForward()` call |
-| `refresh` | `page.reload()` call |
-| `spa_route` | `history.pushState()` detected |
-| `spa_replace` | `history.replaceState()` detected |
-| `hash_change` | URL hash change |
-| `popstate` | Browser back/forward via popstate event |
-
-### Network Stats
-
-When `includeNetworkStats` is enabled, each timeline entry includes:
-
-```json
-{
-  "totalRequests": 40,
-  "failedRequests": 0,
-  "totalBytes": 1289736,
-  "byType": {
-    "xhr": 0,
-    "document": 1,
-    "script": 9,
-    "stylesheet": 2,
-    "image": 27,
-    "font": 2,
-    "other": 0
-  }
-}
-```
+## Configuration
 
-### Failure Diagnostics
+All options are passed as the second element of the reporter tuple:
 
-When a test fails, the report includes diagnostic details:
+### General Options
 
-```json
-{
-  "message": "expect(received).toBe(expected) // Object.is equality\n\nExpected: 3\nReceived: 2",
-  "line": 70,
-  "code": "  67 |   test('deliberate failure', async ({ page }) => {\n  68 |     await page.goto('https://example.com');\n  69 |     // This assertion will fail\n> 70 |     expect(1 + 1).toBe(3);\n  71 |   });",
-  "stack": "Error: expect(received).toBe(expected)\n    at /path/to/spec.ts:70:19"
-}
+| Option | Type | Default | Description |
+|---|---|---|---|
+| `outputPath` | `string` | `./test-results/analytics-timeline.json` | Where to write the JSON report |
+| `includeStackTrace` | `boolean` | `false` | Include full stack traces in failure diagnostics |
+| `includeCodeSnippets` | `boolean` | `true` | Include source code snippets around the failure line |
+| `codeContextLines` | `number` | `3` | Lines of context above/below the failure line |
+| `includeNetworkStats` | `boolean` | `true` | Track network requests and bytes per navigation |
+| `includeArtifacts` | `boolean` | `true` | Include screenshots and video in the report |
+| `navigationTypes` | `NavigationType[] \| null` | `null` (all) | Filter timeline to specific navigation types |
+| `redactPatterns` | `(string \| RegExp)[]` | Built-in patterns | Additional patterns to redact from error output |
+| `testRunId` | `string \| null` | `null` (auto UUID) | Override the test run ID |
+| `metadata` | `Record<string, unknown> \| null` | `null` | Attach custom metadata to the report |
+| `openReport` | `boolean` | `true` | Auto-open the HTML report after test run |
+| `quiet` | `boolean` | `false` | Suppress TestRelic banner output |
+
+### API Tracking Options
+
+| Option | Type | Default | Description |
+|---|---|---|---|
+| `trackApiCalls` | `boolean` | `true` | Enable/disable API call tracking entirely |
+| `captureRequestHeaders` | `boolean` | `true` | Capture request headers for each API call |
+| `captureResponseHeaders` | `boolean` | `true` | Capture response headers for each API call |
+| `captureRequestBody` | `boolean` | `true` | Capture request body/payload for each API call |
+| `captureResponseBody` | `boolean` | `true` | Capture response body for each API call |
+| `captureAssertions` | `boolean` | `true` | Track assertion results (pass/fail with expected/actual values) |
+| `redactHeaders` | `string[]` | `['authorization', 'cookie', 'set-cookie', 'x-api-key']` | Header names to redact (case-insensitive) |
+| `redactBodyFields` | `string[]` | `['password', 'secret', 'token', 'apiKey', 'api_key']` | Body field names whose values are replaced with `[REDACTED]` |
+| `apiIncludeUrls` | `(string \| RegExp)[]` | `[]` (all URLs) | Only track API calls matching these patterns |
+| `apiExcludeUrls` | `(string \| RegExp)[]` | `[]` (none excluded) | Exclude API calls matching these patterns |
+
+### Configuration Examples
+
+**Default (zero config)** — captures everything with built-in redaction:
+
+```typescript
+reporter: [
+  ['list'],
+  ['@testrelic/playwright-analytics', {
+    outputPath: './test-results/analytics-timeline.json',
+  }],
+]
 ```
 
-### CI Metadata
+**Minimal capture (no bodies)** — reduces report size, tracks only method/URL/status/timing:
 
-When running in a supported CI environment, the `ci` field is auto-populated:
+```typescript
+reporter: [
+  ['list'],
+  ['@testrelic/playwright-analytics', {
+    outputPath: './test-results/analytics-timeline.json',
+    captureRequestBody: false,
+    captureResponseBody: false,
+  }],
+]
+```
 
-```json
-{
-  "provider": "github-actions",
-  "buildId": "12345678",
-  "commitSha": "abc123def456",
-  "branch": "main"
-}
+**URL filtering** — only track specific endpoints:
+
+```typescript
+reporter: [
+  ['list'],
+  ['@testrelic/playwright-analytics', {
+    outputPath: './test-results/analytics-timeline.json',
+    apiIncludeUrls: ['**/api/**'],
+    apiExcludeUrls: ['**/health', '**/metrics'],
+  }],
+]
 ```
 
-Supported providers: **GitHub Actions**, **GitLab CI**, **Jenkins**, **CircleCI**.
+**Custom redaction** — add your own sensitive header/body field names:
+
+```typescript
+reporter: [
+  ['list'],
+  ['@testrelic/playwright-analytics', {
+    outputPath: './test-results/analytics-timeline.json',
+    redactHeaders: ['authorization', 'cookie', 'set-cookie', 'x-api-key', 'x-custom-secret'],
+    redactBodyFields: ['password', 'secret', 'token', 'ssn', 'creditCard'],
+  }],
+]
+```
+
+**Disable API tracking** — browser-only report with no API call data:
+
+```typescript
+reporter: [
+  ['list'],
+  ['@testrelic/playwright-analytics', {
+    outputPath: './test-results/analytics-timeline.json',
+    trackApiCalls: false,
+  }],
+]
+```
 
 ## Merging Shard Reports
 
-When running Playwright with sharding, each shard produces its own report. Use the CLI to merge them:
+When running Playwright with sharding, each shard produces its own report. Merge them with the CLI:
 
 ```bash
 npx testrelic merge shard-1.json shard-2.json shard-3.json -o merged-report.json
@@ -266,38 +266,75 @@ Or programmatically:
 ```typescript
 import { mergeReports } from '@testrelic/playwright-analytics/merge';
 
-const merged = await mergeReports(
+await mergeReports(
   ['shard-1.json', 'shard-2.json'],
   { output: 'merged-report.json' }
 );
 ```
 
-The merged report combines all timelines chronologically, recalculates the summary, and records the original shard run IDs.
+## Testing Modes
+
+TestRelic supports three testing modes, each with its own fixture:
+
+### E2E Testing (Browser)
 
-## Security
+Use the unified fixture which provides `page` with navigation tracking — page load timing, DOM content loaded, network idle detection, and network request statistics.
+
+```typescript
+import { test, expect } from '@testrelic/playwright-analytics/fixture';
 
-The reporter automatically redacts sensitive data from error messages, stack traces, and code snippets before writing the report:
+test('search works', { tag: ['@e2e'] }, async ({ page }) => {
+  await page.goto('https://example.com');
+  await page.fill('#search', 'query');
+  await page.click('button[type=submit]');
+});
+```
 
-- AWS access key IDs (`AKIA...`)
-- Bearer tokens (`Bearer eyJ...`)
-- Private keys (`-----BEGIN PRIVATE KEY-----`)
-- Credential URLs (`//user:password@host`)
+### API Testing
 
-You can add custom redaction patterns via the `redactPatterns` config option.
+Use the API-only fixture which provides `request` without any browser dependency — tracks HTTP method, URL, status code, headers, bodies, response times, and assertions.
 
-## How It Works
+```typescript
+import { test as base } from '@playwright/test';
+import { testRelicApiFixture } from '@testrelic/playwright-analytics/api-fixture';
+import { expect } from '@testrelic/playwright-analytics/fixture';
 
-The SDK has two components that work together:
+const test = base.extend(testRelicApiFixture);
 
-1. **Fixture** (`@testrelic/playwright-analytics/fixture`) — Wraps the Playwright `page` object with a `NavigationTracker` that intercepts `goto`, `goBack`, `goForward`, and `reload` calls, listens for `framenavigated` events, injects a script to detect SPA navigations, and optionally tracks network responses. All tracked data is written as test annotations.
+test('CRUD operations', { tag: ['@api'] }, async ({ request }) => {
+  const response = await request.post('https://api.example.com/posts', {
+    data: { title: 'New Post', body: 'Content' },
+  });
+  expect(response.status()).toBe(201);
+});
+```
 
-2. **Reporter** (`@testrelic/playwright-analytics`) — A Playwright custom reporter that collects test results and navigation annotations in `onTestEnd`, then on `onEnd` builds the final JSON timeline, detects CI metadata, applies redaction, and writes the report atomically (write to temp file, then rename).
+### Unified Testing (Browser + API)
 
-The reporter never throws during test execution — all hooks are wrapped in try/catch to ensure it never crashes your test run.
+Use the unified fixture which provides **both** `page` and `request` — the TestRelic report shows navigation timeline AND API call details for the same test. This is ideal for:
 
-## Using Without the Fixture
+- **Contract testing**: Verify API data matches what the browser renders
+- **Setup/teardown via API**: Use API calls to set up test data, then verify in the browser
+- **Full-stack workflows**: Test end-to-end flows that span both UI and API layers
 
-The reporter works without the fixture, but navigation tracking will not be available. Tests will still be captured with their pass/fail status, duration, and failure diagnostics. The timeline entries will show `about:blank` as the URL with `dummy` navigation type.
+```typescript
+import { test, expect } from '@testrelic/playwright-analytics/fixture';
+
+test('API data matches UI', { tag: ['@e2e', '@api'] }, async ({ page, request }) => {
+  // Fetch data via API
+  const apiResponse = await request.get('https://api.example.com/user/1');
+  const user = await apiResponse.json();
+
+  // Verify it renders correctly in the browser
+  await page.goto('https://example.com/profile');
+  await expect(page.locator('.user-name')).toHaveText(user.name);
+});
+```
+
+## Prerequisites
+
+- **Node.js** >= 18
+- **Playwright** >= 1.35.0
 
 ## License
 

package/package.json

@@ -1,7 +1,35 @@
 {
   "name": "@testrelic/playwright-analytics",
-  "version": "2.0.0",
-  "description": "Playwright custom reporter and navigation-tracking fixture for test analytics",
+  "version": "2.1.0",
+  "description": "Playwright test analytics reporter with E2E navigation tracking, API call capture, network stats, failure diagnostics, and interactive HTML reports",
+  "keywords": [
+    "playwright",
+    "test",
+    "analytics",
+    "reporter",
+    "api-testing",
+    "e2e-testing",
+    "navigation",
+    "network",
+    "html-report",
+    "test-results",
+    "api-tracking",
+    "assertions",
+    "redaction",
+    "ci",
+    "testing",
+    "test-automation",
+    "playwright-reporter"
+  ],
+  "author": "TestRelic AI <hello@testrelic.ai>",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/testrelic-ai/testrelic-js-sdk.git",
+    "directory": "packages/playwright-analytics"
+  },
+  "bugs": {
+    "url": "https://github.com/testrelic-ai/testrelic-js-sdk/issues"
+  },
   "type": "module",
   "main": "./dist/index.cjs",
   "module": "./dist/index.js",
@@ -54,7 +82,8 @@
   },
   "files": [
     "dist",
-    "timeline-schema.json"
+    "timeline-schema.json",
+    "README.md"
   ],
   "scripts": {
     "build": "tsup",