@alternative-path/testlens-playwright-reporter 0.4.8 → 0.4.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/README.md +248 -163
  2. package/cross-env.js +2 -2
  3. package/index.d.ts +22 -1
  4. package/index.js +1538 -1434
  5. package/index.ts +1865 -1714
  6. package/package.json +75 -75
package/README.md CHANGED
@@ -1,163 +1,248 @@
1
- # TestLens Playwright Reporter
2
-
3
- A Playwright reporter for [TestLens](https://testlens.qa-path.com) - real-time test monitoring dashboard.
4
-
5
- ## Features
6
-
7
- - 🚀 **Real-time streaming** - Watch test results as they happen in the dashboard
8
- - 📸 **Artifact support** - Shows screenshots, videos, and traces
9
- - 🔄 **Retry tracking** - Monitor test retries and identify flaky tests
10
- - ⚡ **Cross-platform** - Works on Windows, macOS, and Linux
11
-
12
- ## Installation
13
-
14
- ```bash
15
- npm i @alternative-path/testlens-playwright-reporter
16
- ```
17
-
18
- ## Configuration
19
-
20
- ### Quick Start (Recommended)
21
-
22
- The simplest config - API key and metadata are passed via environment variables:
23
-
24
- **TypeScript (`playwright.config.ts`)**
25
- ```typescript
26
- import { defineConfig } from '@playwright/test';
27
-
28
- export default defineConfig({
29
- use: {
30
- screenshot: 'on',
31
- video: 'on',
32
- trace: 'on',
33
- },
34
- reporter: [
35
- ['testlens-playwright-reporter']
36
- // API key is auto-detected from TESTLENS_API_KEY env var
37
- // Build metadata is auto-detected from testlensBuildName/testlensBuildTag env vars
38
- ],
39
- });
40
- ```
41
-
42
- **JavaScript (`playwright.config.js`)**
43
- ```javascript
44
- const { defineConfig } = require('@playwright/test');
45
-
46
- module.exports = defineConfig({
47
- use: {
48
- screenshot: 'on',
49
- video: 'on',
50
- trace: 'on',
51
- },
52
- reporter: [
53
- ['testlens-playwright-reporter']
54
- // API key is auto-detected from TESTLENS_API_KEY env var
55
- // Build metadata is auto-detected from testlensBuildName/testlensBuildTag env vars
56
- ],
57
- });
58
- ```
59
-
60
- ### Advanced Configuration (Optional)
61
-
62
- If you prefer to set API key or metadata explicitly in config:
63
-
64
- **TypeScript (`playwright.config.ts`)**
65
- ```typescript
66
- import { defineConfig } from '@playwright/test';
67
-
68
- export default defineConfig({
69
- use: {
70
- screenshot: 'on',
71
- video: 'on',
72
- trace: 'on',
73
- },
74
- reporter: [
75
- ['@alternative-path/testlens-playwright-reporter', {
76
- apiKey: process.env.TESTLENS_API_KEY || 'your-api-key-here',
77
-
78
- // Optional: explicitly forward build metadata from env vars
79
- customMetadata: {
80
- testlensBuildName: process.env.testlensBuildName,
81
- testlensBuildTag: process.env.testlensBuildTag,
82
- },
83
- }]
84
- ],
85
- });
86
- ```
87
-
88
- **JavaScript (`playwright.config.js`)**
89
- ```javascript
90
- const { defineConfig } = require('@playwright/test');
91
-
92
- module.exports = defineConfig({
93
- use: {
94
- screenshot: 'on',
95
- video: 'on',
96
- trace: 'on',
97
- },
98
- reporter: [
99
- ['@alternative-path/testlens-playwright-reporter', {
100
- apiKey: process.env.TESTLENS_API_KEY || 'your-api-key-here',
101
-
102
- // Optional: explicitly forward build metadata from env vars
103
- customMetadata: {
104
- testlensBuildName: process.env.testlensBuildName,
105
- testlensBuildTag: process.env.testlensBuildTag,
106
- },
107
- }]
108
- ],
109
- });
110
- ```
111
-
112
- ## Run With Build Name & Tag (UI Mode)
113
-
114
- Use `testlens-cross-env` (bundled with this package) to set API key and metadata cross-platform, including Windows.
115
-
116
- ### Command
117
-
118
- ```bash
119
- npx testlens-cross-env TESTLENS_API_KEY="<your-api-key>" testlensBuildName="Testing Build Local Environment" testlensBuildTag="smoke" playwright test --ui
120
- ```
121
-
122
- ### What Gets Auto-Detected
123
-
124
- The reporter automatically reads these environment variables (no config changes needed):
125
-
126
- - **API Key**: `TESTLENS_API_KEY` (also checks: `testlens_api_key`, `TESTLENS_KEY`, `testlensApiKey`, `PLAYWRIGHT_API_KEY`, `PW_API_KEY`)
127
- - **Build Name**: `testlensBuildName` (also checks: `TESTLENS_BUILD_NAME`, `BUILDNAME`, `BUILD_NAME`)
128
- - **Build Tag**: `testlensBuildTag` (also checks: `TESTLENS_BUILD_TAG`, `BUILDTAG`, `BUILD_TAG`)
129
-
130
- ### Notes
131
-
132
- - No config changes required - just pass env vars in the command
133
- - `testlensBuildName` and `testlensBuildTag` are sent to TestLens as run `custom_metadata`
134
- - Multiple tags: use comma-separated list `testlensBuildTag="smoke,regression"`
135
-
136
- > 💡 **Tip:** Keep `screenshot`, `video`, and `trace` set to `'on'` for better debugging experience. TestLens automatically uploads these artifacts for failed tests, making it easier to identify issues.
137
-
138
- ### Configuration Options
139
-
140
- | Option | Type | Default | Description |
141
- |--------|------|---------|-------------|
142
- | `apiKey` | `string` | **Required** | Your TestLens API key |
143
-
144
- ## Artifacts
145
-
146
- TestLens automatically captures and uploads:
147
-
148
- | Artifact | Description |
149
- |----------|-------------|
150
- | **Screenshots** | Visual snapshots of test failures |
151
- | **Videos** | Full video recording of test execution |
152
- | **Traces** | Playwright trace files for step-by-step debugging |
153
-
154
- These artifacts are viewable directly in the TestLens dashboard for easy debugging.
155
-
156
- ## Requirements
157
-
158
- - Node.js >= 16.0.0
159
- - Playwright >= 1.40.0
160
-
161
- ## License
162
-
163
- MIT License
1
+ # TestLens Playwright Reporter
2
+
3
+ A Playwright reporter for [TestLens](https://testlens.qa-path.com) - real-time test monitoring dashboard.
4
+
5
+ ## Features
6
+
7
+ - 🚀 **Real-time streaming** - Watch test results as they happen in the dashboard
8
+ - 📸 **Artifact support** - Shows screenshots, videos, and traces
9
+ - 🔄 **Retry tracking** - Monitor test retries and identify flaky tests
10
+ - ⚡ **Cross-platform** - Works on Windows, macOS, and Linux
11
+
12
+ ## Installation
13
+
14
+ ```bash
15
+ npm i @alternative-path/testlens-playwright-reporter
16
+ ```
17
+
18
+ ## Configuration
19
+
20
+ ### Quick Start (Recommended)
21
+
22
+ The simplest config - API key and metadata are passed via environment variables:
23
+
24
+ **TypeScript (`playwright.config.ts`)**
25
+ ```typescript
26
+ import { defineConfig } from '@playwright/test';
27
+
28
+ export default defineConfig({
29
+ use: {
30
+ screenshot: 'on',
31
+ video: 'on',
32
+ trace: 'on',
33
+ },
34
+ reporter: [
35
+ ['testlens-playwright-reporter']
36
+ // API key is auto-detected from TESTLENS_API_KEY env var
37
+ // Build metadata is auto-detected from testlensBuildName/testlensBuildTag env vars
38
+ ],
39
+ });
40
+ ```
41
+
42
+ **JavaScript (`playwright.config.js`)**
43
+ ```javascript
44
+ const { defineConfig } = require('@playwright/test');
45
+
46
+ module.exports = defineConfig({
47
+ use: {
48
+ screenshot: 'on',
49
+ video: 'on',
50
+ trace: 'on',
51
+ },
52
+ reporter: [
53
+ ['testlens-playwright-reporter']
54
+ // API key is auto-detected from TESTLENS_API_KEY env var
55
+ // Build metadata is auto-detected from testlensBuildName/testlensBuildTag env vars
56
+ ],
57
+ });
58
+ ```
59
+
60
+ ### Advanced Configuration (Optional)
61
+
62
+ If you prefer to set API key or metadata explicitly in config:
63
+
64
+ **TypeScript (`playwright.config.ts`)**
65
+ ```typescript
66
+ import { defineConfig } from '@playwright/test';
67
+
68
+ export default defineConfig({
69
+ use: {
70
+ screenshot: 'on',
71
+ video: 'on',
72
+ trace: 'on',
73
+ },
74
+ reporter: [
75
+ ['@alternative-path/testlens-playwright-reporter', {
76
+ apiKey: process.env.TESTLENS_API_KEY || 'your-api-key-here',
77
+
78
+ // Optional: explicitly forward build metadata from env vars
79
+ customMetadata: {
80
+ testlensBuildName: process.env.testlensBuildName,
81
+ testlensBuildTag: process.env.testlensBuildTag,
82
+ },
83
+ }]
84
+ ],
85
+ });
86
+ ```
87
+
88
+ **JavaScript (`playwright.config.js`)**
89
+ ```javascript
90
+ const { defineConfig } = require('@playwright/test');
91
+
92
+ module.exports = defineConfig({
93
+ use: {
94
+ screenshot: 'on',
95
+ video: 'on',
96
+ trace: 'on',
97
+ },
98
+ reporter: [
99
+ ['@alternative-path/testlens-playwright-reporter', {
100
+ apiKey: process.env.TESTLENS_API_KEY || 'your-api-key-here',
101
+
102
+ // Optional: explicitly forward build metadata from env vars
103
+ customMetadata: {
104
+ testlensBuildName: process.env.testlensBuildName,
105
+ testlensBuildTag: process.env.testlensBuildTag,
106
+ },
107
+ }]
108
+ ],
109
+ });
110
+ ```
111
+
112
+ ## Run With Build Name & Tag (UI Mode)
113
+
114
+ Use `TESTLENS-REPORTER` (bundled with this package) to set API key and metadata cross-platform, including Windows.
115
+
116
+ ### Command
117
+
118
+ ```bash
119
+ npx TESTLENS-REPORTER TESTLENS_API_KEY="<your-api-key>" testlensBuildName="Testing Build Local Environment" testlensBuildTag="smoke" playwright test --ui
120
+ ```
121
+
122
+ ### What Gets Auto-Detected
123
+
124
+ The reporter automatically reads these environment variables (no config changes needed):
125
+
126
+ - **API Key**: `TESTLENS_API_KEY` (also checks: `testlens_api_key`, `TESTLENS_KEY`, `testlensApiKey`, `PLAYWRIGHT_API_KEY`, `PW_API_KEY`)
127
+ - **Build Name**: `TL_BUILDNAME` or `testlensBuildName` (also checks: `TESTLENS_BUILD_NAME`, `TESTLENS_BUILDNAME`, `BUILDNAME`, `BUILD_NAME`). Env keys are matched **case-insensitively** (e.g. `tl_buildname` works).
128
+ - **Build Tag**: `TL_BUILDTAG` or `testlensBuildTag` (also checks: `TESTLENS_BUILD_TAG`, `TESTLENS_BUILDTAG`, `BUILDTAG`, `BUILD_TAG`). Env keys are matched **case-insensitively** (e.g. `tl_buildtag` works).
129
+ - **Execution ID** (one run per pipeline): see [One test run per pipeline execution](#one-test-run-per-pipeline-execution) below.
130
+
131
+ ### One test run per pipeline execution
132
+
133
+ If your pipeline runs `npx playwright test` in **multiple steps**, you get one TestLens run per step. To group all steps into **one** run, set **`TESTLENS_EXECUTION_ID`** to your pipeline’s run identifier (e.g. build number or run ID) in every step. Use one of the three methods below. For per-CI examples (Bitbucket, GitHub, GitLab, Azure DevOps, Jenkins), see **[PIPELINE_EXECUTION_ID.md](./PIPELINE_EXECUTION_ID.md)**.
134
+
135
+ #### 1. Environment variable (recommended in CI)
136
+
137
+ Set one of these env vars **before** each Playwright step. The reporter uses the **first one it finds** (in this order):
138
+
139
+ | Env var | When to use |
140
+ |---------|--------------|
141
+ | `TL_EXECUTIONID` | Short alias (matched case-insensitively) |
142
+ | `TESTLENS_EXECUTION_ID` | Any pipeline; set this to your build/run ID |
143
+ | `TestlensExecutionId`, `TestLensExecutionId`, `testlensexecutionid` | Alternative spellings |
144
+ | `BITBUCKET_BUILD_UUID` | Bitbucket Pipelines (auto-set; no config needed) |
145
+ | `GITHUB_RUN_ID` | GitHub Actions (auto-set) |
146
+ | `CI_PIPELINE_ID`, `CI_JOB_ID` | GitLab CI |
147
+ | `BUILD_BUILDID`, `SYSTEM_JOBID` | Azure DevOps |
148
+ | `BUILD_ID`, `BUILD_NUMBER` | Jenkins |
149
+ | `CIRCLE_WORKFLOW_ID`, `CIRCLE_BUILD_NUM` | CircleCI |
150
+
151
+ **Command examples:**
152
+
153
+ ```bash
154
+ # Linux / macOS
155
+ export TESTLENS_EXECUTION_ID="${BITBUCKET_BUILD_UUID}"
156
+ npx playwright test
157
+
158
+ # Or inline
159
+ TESTLENS_EXECUTION_ID="my-build-123" npx playwright test
160
+ ```
161
+
162
+ ```powershell
163
+ # Windows PowerShell
164
+ $env:TESTLENS_EXECUTION_ID = $env:BITBUCKET_BUILD_UUID
165
+ npx playwright test
166
+
167
+ # Or inline
168
+ $env:TESTLENS_EXECUTION_ID="my-build-123"; npx playwright test
169
+ ```
170
+
171
+ ```cmd
172
+ REM Windows CMD
173
+ set TESTLENS_EXECUTION_ID=my-build-123 && npx playwright test
174
+ ```
175
+
176
+ #### 2. Config option `executionId`
177
+
178
+ In your Playwright config, set the top-level reporter option:
179
+
180
+ ```typescript
181
+ // playwright.config.ts
182
+ reporter: [
183
+ ['@alternative-path/testlens-playwright-reporter', {
184
+ apiKey: process.env.TESTLENS_API_KEY,
185
+ executionId: process.env.BITBUCKET_BUILD_UUID, // or any string
186
+ }]
187
+ ],
188
+ ```
189
+
190
+ #### 3. Config option `customMetadata`
191
+
192
+ You can pass the execution ID inside `customMetadata` (as `executionId` or `TESTLENS_EXECUTION_ID`):
193
+
194
+ ```typescript
195
+ // playwright.config.ts
196
+ reporter: [
197
+ ['@alternative-path/testlens-playwright-reporter', {
198
+ apiKey: process.env.TESTLENS_API_KEY,
199
+ customMetadata: {
200
+ executionId: process.env.BITBUCKET_BUILD_UUID,
201
+ // or: TESTLENS_EXECUTION_ID: process.env.BITBUCKET_BUILD_UUID,
202
+ testlensBuildName: 'Build123',
203
+ testlensBuildTag: 'smoke',
204
+ },
205
+ }]
206
+ ],
207
+ ```
208
+
209
+ #### Recommendation
210
+
211
+ Prefer a **UUID or globally unique ID** (e.g. `BITBUCKET_BUILD_UUID`, `GITHUB_RUN_ID`) so different repos or pipelines never share the same run. Build numbers like `BUILD_NUMBER` reset per repo and can collide. See **[PIPELINE_EXECUTION_ID.md](./PIPELINE_EXECUTION_ID.md)** for per-CI examples.
212
+
213
+ ### Notes
214
+
215
+ - No config changes required - just pass env vars in the command
216
+ - `testlensBuildName` and `testlensBuildTag` are sent to TestLens as run `custom_metadata`
217
+ - Multiple tags: use comma-separated list `testlensBuildTag="smoke,regression"`
218
+
219
+ > 💡 **Tip:** Keep `screenshot`, `video`, and `trace` set to `'on'` for better debugging experience. TestLens automatically uploads these artifacts for failed tests, making it easier to identify issues.
220
+
221
+ ### Configuration Options
222
+
223
+ | Option | Type | Default | Description |
224
+ |--------|------|---------|-------------|
225
+ | `apiKey` | `string` | **Required** | Your TestLens API key |
226
+ | `executionId` | `string` | — | Optional. When set (or use env `TESTLENS_EXECUTION_ID`, any auto-detected CI run ID, or `customMetadata.executionId` / `customMetadata.TESTLENS_EXECUTION_ID`), used as run ID so multiple pipeline steps share one run. See [PIPELINE_EXECUTION_ID.md](./PIPELINE_EXECUTION_ID.md). |
227
+ | `logLevel` | `'info' \| 'debug'` | `'info'` | Logging level. `'info'` shows test start/completion with status and errors. `'debug'` shows all internal operations. Can also be set via `TESTLENS_LOG_LEVEL` env var. |
228
+
229
+ ## Artifacts
230
+
231
+ TestLens automatically captures and uploads:
232
+
233
+ | Artifact | Description |
234
+ |----------|-------------|
235
+ | **Screenshots** | Visual snapshots of test failures |
236
+ | **Videos** | Full video recording of test execution |
237
+ | **Traces** | Playwright trace files for step-by-step debugging |
238
+
239
+ These artifacts are viewable directly in the TestLens dashboard for easy debugging.
240
+
241
+ ## Requirements
242
+
243
+ - Node.js >= 16.0.0
244
+ - Playwright >= 1.40.0
245
+
246
+ ## License
247
+
248
+ MIT License
package/cross-env.js CHANGED
@@ -12,10 +12,10 @@
12
12
  * - Any platform that runs Node.js
13
13
  *
14
14
  * Usage:
15
- * npx testlens-cross-env VAR1=value1 VAR2=value2 command args...
15
+ * npx TESTLENS-REPORTER VAR1=value1 VAR2=value2 command args...
16
16
  *
17
17
  * Example:
18
- * npx testlens-cross-env testlensApiKey="abc123" playwright test
18
+ * npx TESTLENS-REPORTER testlensApiKey="abc123" playwright test
19
19
  */
20
20
 
21
21
  const crossEnv = require('cross-env');
package/index.d.ts CHANGED
@@ -28,6 +28,14 @@ export interface TestLensReporterConfig {
28
28
  ignoreSslErrors?: boolean;
29
29
  /** Custom metadata from CLI arguments (automatically parsed from --key=value arguments) */
30
30
  customMetadata?: Record<string, string | string[]>;
31
+ /** Execution ID for one run per pipeline (e.g. from TESTLENS_EXECUTION_ID or CI build UUID). When set, used as runId so multiple steps share one run. */
32
+ executionId?: string;
33
+ /**
34
+ * Log level for the reporter.
35
+ * - 'info' (default): Shows test start/completion with status, artifact status, and errors
36
+ * - 'debug': Shows all logs including detailed internal operations
37
+ */
38
+ logLevel?: 'info' | 'debug';
31
39
  }
32
40
  export interface TestLensReporterOptions {
33
41
  /** TestLens API endpoint URL */
@@ -58,6 +66,14 @@ export interface TestLensReporterOptions {
58
66
  ignoreSslErrors?: boolean;
59
67
  /** Custom metadata from CLI arguments (automatically parsed from --key=value arguments) */
60
68
  customMetadata?: Record<string, string | string[]>;
69
+ /** Execution ID for one run per pipeline (e.g. from TESTLENS_EXECUTION_ID or CI build UUID). When set, used as runId so multiple steps share one run. */
70
+ executionId?: string;
71
+ /**
72
+ * Log level for the reporter.
73
+ * - 'info' (default): Shows test start/completion with status, artifact status, and errors
74
+ * - 'debug': Shows all logs including detailed internal operations
75
+ */
76
+ logLevel?: 'info' | 'debug';
61
77
  }
62
78
  export interface GitInfo {
63
79
  branch: string;
@@ -157,14 +173,19 @@ export declare class TestLensReporter implements Reporter {
157
173
  private axiosInstance;
158
174
  private runId;
159
175
  private runMetadata;
176
+ private usedExecutionId;
160
177
  private specMap;
161
178
  private testMap;
162
179
  private runCreationFailed;
163
180
  private cliArgs;
164
181
  private pendingUploads;
165
- private traceNetworkRows;
166
182
  private artifactStats;
167
183
  private artifactsSeen;
184
+ private logger;
185
+ /**
186
+ * Get environment variable value with case-insensitive key match (e.g. TL_BUILDNAME, tl_buildname).
187
+ */
188
+ private static getEnvCaseInsensitive;
168
189
  /**
169
190
  * Parse custom metadata from environment variables
170
191
  * Checks for common metadata environment variables