@appliqation/automation-sdk 2.5.1 → 2.8.0

package/README.md

@@ -1,169 +1,94 @@
 # Appliqation Automation SDK
 
-A powerful SDK for integrating Playwright test results with the Appliqation test management portal.
+Playwright reporter and CLI for Appliqation: authenticate against a gated
+application, run your suite, and report results back to Appliqation with
+zero manual login scripts and no separate test-runner integration.
 
-**What it does:**
-- ✅ Automatically creates test runs in Appliqation
-- ✅ Handles browser authentication with JWT tokens
-- ✅ Reports test results (passed/failed/skipped) to your portal
-- ✅ Tracks browser, OS, and device information
-- ✅ Manages test case mappings via UUIDs
+## What it does
 
-**Perfect for:**
-- QA teams running automated regression tests
-- CI/CD pipelines needing test result tracking
-- Teams using Appliqation for test management
+- Creates and reports to Appliqation test runs directly from `playwright test`
+- Authenticates against your application under test — either via a portable,
+  customer-defined login flow (`defineLogin`/`setupAuth`) or JWT-based
+  browser authentication, both usable in CI and locally
+- Maps Playwright tests to Appliqation test cases via UUID, and reports
+  pass/fail/skip results, browser/OS/device metadata, and orphan tests
+- Scopes a run to a single Scenario or an entire Test Set (smoke, sanity,
+  regression), with pre-execution validation that a tag-based test selection
+  hasn't accidentally drifted outside that scope
+- Auto-tags test cases as automation-covered the first time a result for
+  them is accepted
 
----
+Reporting to Appliqation is **opt-in** per run (`--appq` flag or
+`APPQ_ENABLED=1`) — without it, tests run and results appear in Playwright's
+own reporters as normal, nothing is sent to Appliqation.
 
-## Prerequisites
+## Requirements
 
-Before you begin, ensure you have:
+- Node.js ≥ 18
+- `@playwright/test` ≥ 1.30 (peer dependency — install it yourself if you
+  haven't already)
+- An Appliqation API key and project key (Project Settings → API Keys)
 
-1. **Node.js 16+** installed
-   ```bash
-   node --version  # Should be v16.0.0 or higher
-   ```
-
-2. **Playwright** installed in your project
-   ```bash
-   npm install @playwright/test
-   ```
-
-3. **Appliqation Account** with:
-   - Access to Appliqation portal
-   - API key (get from your project settings or ask your project admin)
-   - Project key (get from your project settings or ask your project admin)
-
----
-
-## Quick Start
-
-Follow these steps in order to integrate the SDK with your Playwright tests.
-
-**📌 Important:** Appliqation reporting is **opt-in**. You must either set `APPQ_ENABLE=1` environment variable or add `-- --appq` flag to your test command to send results to Appliqation. Without enabling reporting, tests run normally but results are not reported.
-
-### Step 1: Install the SDK
-
-Navigate to your Playwright project and install the SDK:
+## Install
 
 ```bash
-npm install @appliqation/automation-sdk dotenv
+npm install @appliqation/automation-sdk --save-dev
 ```
 
-**Why dotenv?** The SDK uses environment variables for configuration, and dotenv loads them from your `.env` file.
+No other runtime dependency is required. The SDK does not use `dotenv` —
+in CI, inject configuration as environment variables/secrets the normal way
+for your platform; locally, export them in your shell or use whatever
+`.env` loader you already prefer (nothing the SDK does depends on it).
 
----
+## Quick start
 
-### Step 2: Create .env File
+### 1. Configure credentials
 
-Create a `.env` file in your project root with your Appliqation credentials:
+Set these as environment variables (CI secrets, shell exports, or your own
+`.env` tooling):
 
-**File:** `.env`
 ```bash
-# IMPORTANT: No spaces around = signs, no trailing slashes on URLs
-
 APPLIQATION_API_KEY=appq_live_xxxxxxxxxxxxxxxxxxxx
-APPLIQATION_PROJECT_KEY=your-project-key-here
-TEST_APP_URL=https://www.example.com    # App under test (Playwright baseURL)
-```
-
-**How to get these values:**
-- `APPLIQATION_API_KEY` -     Found in your project settings
-- `APPLIQATION_PROJECT_KEY` - Found in your project settings
-- `TEST_APP_URL` - The site you’re testing (e.g., https://www.amazon.in). Used as Playwright `baseURL`.
-- 
-
-**⚠️ Add .env to .gitignore** to keep credentials secret:
-```bash
-echo ".env" >> .gitignore
+APPLIQATION_PROJECT_KEY=your-project-key
+TEST_APP_URL=https://staging.your-app.com   # used as Playwright's baseURL
 ```
 
----
-
-### Step 3: Update playwright.config.js
-
-Update your Playwright configuration to use the SDK's built-in features:
-
-**File:** `playwright.config.js`
+### 2. Wire up `playwright.config.js`
 
 ```javascript
 const { defineConfig } = require('@playwright/test');
-require('dotenv').config();  // Load .env variables
 
-// SDK configuration
 const appliqationConfig = {
   apiKey: process.env.APPLIQATION_API_KEY,
   projectKey: process.env.APPLIQATION_PROJECT_KEY,
-
-  // Optional settings
-  autoCreateRun: true,           // Automatically create run on test start
-  batchSubmit: true,             // Batch results for efficiency
-  batchSize: 50,                 // Results per batch
-  logOrphans: true,              // Log tests without UUID mappings
-  logLevel: 'INFO',              // INFO, DEBUG, ERROR
-  rejectUnauthorized: false      // Set to false for self-signed certificates in dev
+  environment: process.env.APPLIQATION_ENVIRONMENT, // e.g. 'Staging'
 };
 
 module.exports = defineConfig({
   testDir: './tests',
 
-  // SDK's built-in global setup (handles auth + run creation)
-  globalSetup: require.resolve('@appliqation/automation-sdk/playwright/global-setup'),
-  globalTeardown: require.resolve('@appliqation/automation-sdk/playwright/global-teardown'),
-
   use: {
     baseURL: process.env.TEST_APP_URL,
-
-    // Use SDK-managed authentication state
-    storageState: '.auth/jwt.json',
-
-    // Ignore HTTPS errors for local development
-    ignoreHTTPSErrors: true,
   },
 
   reporter: [
-    ['list'],  // Console output
-    ['html'],  // HTML report
-
-    // Appliqation reporter for result submission
-    ['@appliqation/automation-sdk/playwright/reporter', appliqationConfig]
+    ['list'],
+    ['@appliqation/automation-sdk/playwright/reporter', appliqationConfig],
   ],
-
-  projects: [
-    {
-      name: 'chromium',
-      use: {
-        browserName: 'chromium',
-        storageState: '.auth/jwt.json'  // Authenticated browser state
-      }
-    }
-  ]
 });
 ```
 
-**What this does:**
-- ✅ Loads environment variables from `.env`
-- ✅ Configures SDK with your credentials
-- ✅ Uses SDK's built-in `global-setup.js` to handle authentication automatically
-- ✅ Saves authenticated browser state to `.auth/jwt.json`
-- ✅ All tests use authenticated sessions (no login required!)
-- ✅ Reports results to Appliqation portal
-
----
-
-### Step 4: Add UUIDs to Your Tests
+This is enough to report results for an **ungated** application. If your
+app requires login, see [Authenticated apps](#authenticated-apps-gated-suts)
+below — it adds a `globalSetup`/`storageState` wiring on top of this.
 
-Map your automated tests to Appliqation test cases using `mapAppqUuid()`:
-
-**File:** `tests/example.spec.js`
+### 3. Map tests to Appliqation test cases
 
 ```javascript
 const { test, expect } = require('@playwright/test');
 const { mapAppqUuid } = require('@appliqation/automation-sdk/utils');
 
-test('should login successfully', async ({ page }, testInfo) => {
-  // Map this test to Appliqation test case UUID
+test('should log in successfully', async ({ page }, testInfo) => {
   mapAppqUuid(testInfo, '1154-7a17b809-0ff9-4ba1-9322-4eb2a49abfc5');
 
   await page.goto('/login');
@@ -173,1172 +98,408 @@ test('should login successfully', async ({ page }, testInfo) => {
 
   await expect(page).toHaveURL('/dashboard');
 });
-
-test('should display user profile', async ({ page }, testInfo) => {
-  // Different test case, different UUID
-  mapAppqUuid(testInfo, '1154-8b28c910-1aa0-5cc2-a433-5fc6a4d82dc6');
-
-  await page.goto('/profile');
-  await expect(page.locator('h1')).toContainText('My Profile');
-});
 ```
 
-**How to get UUIDs:**
-1. Login to your Appliqation portal
-2. Navigate to your test cases
-3. Copy the UUID for each test case (format: `nid-uuid`)
-4. You can get UUID from downloading the CSV from the test scenario
-
-**Important:**
-- ⚠️ Always include `testInfo` as second parameter: `async ({ page }, testInfo) =>`
-- ⚠️ Call `mapAppqUuid()` at the start of each test
-- ⚠️ UUID format must be: `nid-uuid` (e.g., `1154-7a17b809-...`)
+- `testInfo` must be the test function's second parameter.
+- The UUID is `{test_case_nid}-{uuid}`; copy it from the test case in the
+  Appliqation portal, or export the scenario's CSV.
+- Tests without a mapped UUID still run normally — they're reported as
+  **orphan tests** rather than failing the run. See
+  [Orphan tests](#orphan-test-handling).
 
----
+### 4. Run with reporting enabled
 
-### Step 5: Run Your Tests
-
-**IMPORTANT:** To enable Appliqation reporting, you must either set `APPQ_ENABLE=1` environment variable or use the `-- --appq` flag.
-
-**Reporting DISABLED (Default):**
 ```bash
-# Tests run normally, but results are NOT sent to Appliqation
+# Reporting disabled by default — results stay local
 npx playwright test
-```
-
-**Reporting ENABLED (Method 1 - Environment Variable - Recommended):**
-```bash
-# Set APPQ_ENABLE to enable reporting (simpler approach)
-APPQ_ENABLE=1 APPLIQATION_RUN_TITLE=yourruntitle APPLIQATION_ENVIRONMENT=test_env_name npx playwright test
-```
-
-**Reporting ENABLED (Method 2 - CLI Flag):**
-```bash
-# Use -- --appq flag to enable reporting (requires separator)
-APPLIQATION_RUN_TITLE=yourruntitle APPLIQATION_ENVIRONMENT=test_env_name npx playwright test -- --appq
-```
-
-**Examples:**
-```bash
-# Method 1: Using environment variable (recommended)
-APPQ_ENABLE=1 npx playwright test
-APPQ_ENABLE=1 npx playwright test tests/login.spec.js
-APPQ_ENABLE=1 npx playwright test --project=chromium
-APPQ_ENABLE=1 npx playwright test --headed
-
-# Method 2: Using CLI flag (requires -- separator)
-npx playwright test -- --appq
-npx playwright test tests/login.spec.js -- --appq
-npx playwright test --project=chromium -- --appq
-npx playwright test --headed -- --appq
-
-# Bonus: Set run title via CLI flag
-npx playwright test -- --appq --appq_run_title="My Custom Run"
-npx playwright test tests/login.spec.js -- --appq --appq_run_title="Login Tests Sprint 23"
-```
-
-**What happens when `--appq` flag is present:**
-
-1. **Global Setup Runs** (once before all tests):
-   - Reads `.env` configuration
-   - Creates automation run in Appliqation portal
-   - Requests browser JWT token from portal
-   - Sets up authenticated browser session
-   - Saves authentication to `.auth/jwt.json`
-
-2. **Tests Execute**:
-   - Each test loads authenticated browser state
-   - No login required - JWT cookie already set!
-   - Tests run normally
-   - Results collected by SDK reporter
-
-3. **Results Submitted**:
-   - SDK batches results (50 per batch by default)
-   - Submits to Appliqation portal
-   - Logs any orphan tests (tests without UUIDs)
-
-4. **Global Teardown Runs** (once after all tests):
-   - Cleanup temporary files
-   - Log final summary
-
-**Check your results:**
-- Visit your Appliqation portal
-- Navigate to the scenario you configured
-- See test results in the run matrix!
-
-**Remember:** Results only appear in Appliqation when you enable reporting with `APPQ_ENABLE=1` or `-- --appq` flag!
-
----
-
-✅ **That's it!** Your tests are now integrated with Appliqation. Remember to enable reporting with `APPQ_ENABLE=1` or `-- --appq`!
-
----
-
-## Enabling Appliqation Reporting (--appq Flag)
-
-By default, Appliqation reporting is **DISABLED**. You must explicitly enable it by adding the `--appq` flag to your test command.
-
-### Why Opt-In?
-
-This design allows you to:
-- ✅ Run tests locally without creating runs in Appliqation
-- ✅ Debug and develop tests without affecting production data
-- ✅ Control when results are sent to Appliqation portal
-- ✅ Avoid accidental test runs in your dashboard
-
-### How to Enable
-
-You can enable Appliqation reporting in two ways:
-
-#### Method 1: Environment Variable (Recommended - Simpler)
 
-Set the `APPQ_ENABLE` environment variable to `1` or `true`:
+# Enabled via env var (recommended)
+APPQ_ENABLED=1 npx playwright test
 
-```bash
-# Enable reporting with environment variable
-APPQ_ENABLE=1 npx playwright test
-
-# Works with any other flags
-APPQ_ENABLE=1 npx playwright test tests/login.spec.js --headed --project=chromium
-
-# Also works with APPLIQATION_ENABLE
-APPLIQATION_ENABLE=true npx playwright test
-```
-
-#### Method 2: CLI Flag (Requires `--` Separator)
-
-Add `--appq` flag after the `--` separator:
-
-```bash
-# Enable reporting with CLI flag (note the -- separator)
+# Enabled via CLI flag (note the -- separator, required by Playwright)
 npx playwright test -- --appq
-
-# Works with any other flags
-npx playwright test tests/login.spec.js --headed --project=chromium -- --appq
 ```
 
-**Note:** The `--` separator is required because Playwright doesn't recognize custom flags directly. The environment variable method is simpler and doesn't require the separator.
-
-### Setting Run Title via CLI
-
-You can also specify the run title directly in the command line using the `--appq_run_title` flag:
+Both forms accept the usual Playwright flags (`--project`, `--headed`,
+`-g`, a file path, etc.).
 
-```bash
-# Set run title via CLI flag
-npx playwright test -- --appq --appq_run_title="My Custom Run Title"
-
-# Works with other flags
-npx playwright test tests/login.spec.js --headed --project=chromium -- --appq --appq_run_title="Login Tests"
-
-# With quotes for titles containing spaces
-npx playwright test -- --appq --appq_run_title="Regression Suite - Sprint 23"
-```
-
-**Priority Order for Run Title:**
-1. Config file `title` option (if specified)
-2. CLI flag `--appq_run_title` (new!)
-3. Environment variable `APPLIQATION_RUN_TITLE`
-4. Auto-generated timestamp (default)
-
-**Examples:**
-```bash
-# Method 1: Environment variable
-APPLIQATION_RUN_TITLE="My Run" npx playwright test -- --appq
-
-# Method 2: CLI flag (overrides environment variable)
-APPLIQATION_RUN_TITLE="Old Title" npx playwright test -- --appq --appq_run_title="New Title"
-# Result: Run title will be "New Title"
-
-# Method 3: Combined usage
-npx playwright test -- --appq --appq_run_title="Sprint 23 Regression"
-```
+## Scoping runs: Scenario or Test Set
 
-### What Happens When Reporting is Disabled
-
-When you run tests **without** enabling reporting (no flag, no env var):
-- ✅ Tests execute normally
-- ✅ Execution summary file is still created
-- ❌ No run is created in Appliqation
-- ❌ No results are sent to Appliqation portal
-- ℹ️  Console shows: "Appliqation reporting disabled: Add --appq flag to send results"
-
-**Example:**
-```bash
-# Reporting disabled - no results sent to Appliqation
-npx playwright test
-```
-
-### What Happens When Reporting is Enabled
-
-When you run tests **with** reporting enabled (using flag or env var):
-- ✅ Tests execute normally
-- ✅ Run matrix created in Appliqation
-- ✅ Results sent to Appliqation portal
-- ✅ Execution summary file created
-- ✅ Console shows: "Appliqation reporting enabled: Results will be sent to Appliqation portal"
-
-**Examples:**
-```bash
-# Reporting enabled with environment variable
-APPQ_ENABLE=1 npx playwright test
-
-# Reporting enabled with CLI flag
-npx playwright test -- --appq
-```
-
----
-
-## Execution Summary Files
-
-After each test run, the SDK automatically creates a timestamped summary file with comprehensive results.
-
-**Location:** `test-results/AppQ_Execution_Summary/`
-
-**Filename Format:** `{run_title}_2025-01-21_14-30-45.txt`
-
-**What's Included:**
-- ✅ Execution time (start, end, duration)
-- ✅ Test counts (submitted, accepted, rejected, passed, failed, skipped)
-- ✅ Run IDs for all created matrices
-- ✅ Detailed error information (duplicates, orphans, backend rejections)
-- ✅ Complete ASCII table (same as terminal output)
-
-**Example:**
-```
-test-results/
-└── AppQ_Execution_Summary/
-    ├── My_Test_Run_2025-01-21_09-30-15.txt
-    ├── My_Test_Run_2025-01-21_14-45-30.txt
-    └── Regression_Suite_2025-01-22_08-00-00.txt
-```
-
-**Features:**
-- Always enabled automatically (no configuration needed)
-- Each run creates a NEW file (never overwrites)
-- If file writing fails, tests continue normally (non-blocking)
-
-**Sample File Content:**
-```
-═══════════════════════════════════════════════════════════
-          APPLIQATION TEST EXECUTION SUMMARY
-═══════════════════════════════════════════════════════════
-
-EXECUTION METADATA:
-─────────────────────────────────────────────────────────
-Start Time:     2025-01-21T14:30:15.123Z
-End Time:       2025-01-21T14:35:45.456Z
-Duration:       5m 30s
-Run Title:      My_Test_Run
-
-╔═══════════════════════════════════════════════════════════╗
-║         Appliqation Test Results Summary                 ║
-╠═══════════════════════════════════════════════════════════╣
-║  Submitted to Backend:                                    ║
-║    Total Submitted:    10                                 ║
-║    ✅ Accepted:         8                                 ║
-║    ❌ Rejected:         2                                 ║
-║                                                           ║
-║  Test Execution Results (Playwright):                    ║
-║    Passed:             8                                  ║
-║    Failed:             0                                  ║
-║    Skipped:            0                                  ║
-║                                                           ║
-║  Not Submitted:                                           ║
-║    Orphan (No UUID):   3                                  ║
-║    Duplicates:         2                                  ║
-╠═══════════════════════════════════════════════════════════╣
-║  Run Matrices Created:     2                              ║
-║    Desktop-Windows            : run_abc123_1234567890     ║
-║    Desktop-Linux              : run_xyz789_0987654321     ║
-╚═══════════════════════════════════════════════════════════╝
-
-DETAILED ERRORS & WARNINGS:
-═══════════════════════════════════════════════════════════
-[Duplicate UUIDs, Orphan Tests, and Backend Rejections details...]
-```
-
----
-
-## Auto-Tagging Test Cases
-
-The SDK automatically tags test cases with "Appq_automated" (configurable) after their **first successful run**. This helps you track which test cases have been automated and are actively running in your test suite.
-
-### How It Works
-
-1. **Test runs and passes** → SDK submits result to Appliqation
-2. **Backend accepts result** → SDK triggers auto-tagging (fire-and-forget)
-3. **Check if already tagged** → Skip if test case already has the tag
-4. **Add tag** → Test case gets tagged in Appliqation UI
-
-**Key Features:**
-- ✅ **Enabled by default** when Appliqation reporting is enabled
-- ✅ **Fire-and-forget** - tagging failures never block your test runs
-- ✅ **Smart deduplication** - checks before tagging, won't create duplicate tags
-- ✅ **Only accepted results** - backend-rejected results are NOT tagged
-- ✅ **Works for both** single and batch result submissions
-- ✅ **Async execution** - zero impact on test execution performance
-
-### Configuration
-
-#### Environment Variables
-
-Add to your `.env` file:
-
-```bash
-# Auto-Tagging Configuration (optional - all have sensible defaults)
-APPLIQATION_AUTO_TAG_ENABLED=true              # Enable/disable (default: true)
-APPLIQATION_AUTO_TAG_NAME=Appq_automated       # Custom tag name (default: Appq_automated)
-```
-
-#### Playwright Reporter Config
-
-Configure in `playwright.config.js`:
+Add `scenarioId` to link a run to one scenario, or `testSetId` to link it
+to a Test Set — the grouping you'd use for a smoke, sanity, or regression
+suite that spans multiple scenarios. They're mutually exclusive;
+`scenarioId` wins if both are set.
 
 ```javascript
-reporter: [
-  ['@appliqation/automation-sdk/playwright/reporter', {
-    apiKey: process.env.APPLIQATION_API_KEY,
-    projectKey: process.env.APPLIQATION_PROJECT_KEY,
-
-    // Auto-tagging options (optional)
-    autoTag: true,                    // Enable auto-tagging (default: true)
-    autoTagName: 'My_Custom_Tag'      // Custom tag name (default: 'Appq_automated')
-  }]
-]
-```
-
-#### Programmatic Configuration
-
-When using the SDK directly:
-
-```javascript
-const { AppliqationClient } = require('@appliqation/automation-sdk');
-
-const client = new AppliqationClient({
-  apiKey: 'your_api_key',
-  projectKey: 'your_project_key',
-
-  // Auto-tagging options
-  options: {
-    autoTag: true,                    // Enable auto-tagging (default: true)
-    autoTagName: 'Automated_Test'     // Custom tag name (default: 'Appq_automated')
-  }
-});
-```
-
-### Disabling Auto-Tagging
-
-If you want to disable auto-tagging:
-
-**Option 1: Environment Variable**
-```bash
-APPLIQATION_AUTO_TAG_ENABLED=false
-```
-
-**Option 2: Config**
-```javascript
-{
-  options: {
-    autoTag: false
-  }
-}
-```
-
-### What You'll See
-
-When auto-tagging is working:
-
-```
-✅ Auto-tagged 3 test case(s) with "Appq_automated"
-```
-
-When test cases are already tagged (second run):
-
-```
-DEBUG: Skipped 3 already-tagged test case(s)
-```
-
-If tagging fails (non-blocking):
-
-```
-⚠️  Auto-tagging failed (non-blocking): Connection timeout
-```
-
-### Troubleshooting
-
-**Q: I don't see the tag in Appliqation UI**
-
-Check:
-1. Is `APPQ_ENABLE=1` set? (Auto-tagging only works when reporting is enabled)
-2. Did the test result get accepted by backend? (Check for backend validation errors)
-3. Check SDK logs for "Auto-tagged X test case(s)" message
-
-**Q: Can I use a custom tag name?**
-
-Yes! Set `APPLIQATION_AUTO_TAG_NAME=Your_Tag_Name` in your `.env` file or use the config options shown above.
-
-**Q: Does tagging failure affect my test results?**
-
-No! Auto-tagging is fire-and-forget. If tagging fails, it's logged as a warning but your test run continues normally and results are still submitted successfully.
-
----
-
-## Handling Orphan Tests
-
-### What are Orphan Tests?
-
-**Orphan tests** are tests that execute successfully but **cannot be mapped to Appliqation test cases** because they're missing UUID annotations. When a test runs without a UUID, the SDK cannot link it to a specific test case in your Appliqation project, making the result "orphaned."
-
-```javascript
-// ❌ This test will be orphaned (no UUID annotation)
-test('Login with valid credentials', async ({ page }) => {
-  await page.goto('/login');
-  await page.fill('#username', 'user@example.com');
-  // ... test code
-});
-
-// ✅ This test will be properly mapped (has UUID annotation)
-test('Login with valid credentials', { tag: '@uuid:1154-abc-def' }, async ({ page }) => {
-  await page.goto('/login');
-  await page.fill('#username', 'user@example.com');
-  // ... test code
-});
-```
-
-### Automatic Orphan Run Cleanup
-
-By default, the SDK **automatically prevents corrupted runs** from being created when ALL tests in a run are orphaned:
-
-**Default Behavior:**
-- ✅ If ALL tests lack UUIDs → Run is **deleted** and a clear error message is shown
-- ✅ If SOME tests have UUIDs → Run is **kept**, valid results are submitted, orphans are logged as warnings
-- ✅ CI/CD pipeline **fails with exit code 1** when orphan-only runs are detected
-- ✅ Clear, actionable error message guides users on how to fix the issue
-
-**Why?** Orphan-only runs create empty entries in Appliqation with "N/A" pass rates, which corrupts your analytics and dashboards.
-
-### Error Message Example
-
-When all tests are orphaned, you'll see:
-
-```
-╔════════════════════════════════════════════════════════════════════╗
-║  ❌ RUN CREATION FAILED - ALL TESTS MISSING UUID ANNOTATIONS      ║
-╠════════════════════════════════════════════════════════════════════╣
-║  Project: 1162-MyProject                                           ║
-║  Orphan Tests: 6                                                   ║
-║                                                                    ║
-║  ⚠️  NO RESULTS WERE SUBMITTED TO APPLIQATION                      ║
-║  The test run was automatically deleted to prevent analytics       ║
-║  corruption. All tests are missing UUID annotations.               ║
-╠════════════════════════════════════════════════════════════════════╣
-║  ✅ ACTION REQUIRED: Add UUID Annotations                          ║
-╠════════════════════════════════════════════════════════════════════╣
-║  Option 1: Using test tags (Recommended)                          ║
-║  test('My Test', { tag: '@uuid:123-xxx' }, async ({ page }) => {  ║
-║    // your test code                                               ║
-║  });                                                               ║
-╚════════════════════════════════════════════════════════════════════╝
-```
-
-### Configuration Options
-
-You can customize orphan handling behavior via environment variables:
-
-```env
-# .env file
-
-# Delete runs with only orphan tests (default: true)
-APPLIQATION_DELETE_ORPHAN_RUNS=true
-
-# Exit with error code 1 for orphan-only runs (default: true)
-APPLIQATION_FAIL_ON_ORPHAN_RUNS=true
-```
-
-**Configuration via playwright.config.js:**
-
-```javascript
-reporter: [
-  [
-    '@appliqation/automation-sdk-js/playwright',
-    {
-      deleteOrphanOnlyRuns: true,   // Delete orphan-only runs (default: true)
-      failOnOrphanOnlyRuns: true,   // Fail CI/CD for orphan-only runs (default: true)
-    }
-  ]
-]
-```
-
-### Mixed Scenarios (Some Tests Have UUIDs)
-
-When your test suite has **both valid and orphan tests**, the SDK handles it gracefully:
-
-```javascript
-// Project 1162: 3 tests total
-test('Valid Test 1', { tag: '@uuid:1154-abc' }, async ({ page }) => {
-  // ✅ Will be submitted to Appliqation
-});
-
-test('Orphan Test 1', async ({ page }) => {
-  // ⚠️ Logged as warning, not submitted
-});
-
-test('Valid Test 2', { tag: '@uuid:1155-def' }, async ({ page }) => {
-  // ✅ Will be submitted to Appliqation
-});
-```
-
-**Result:**
-- ✅ Run is kept (because 2 tests have UUIDs)
-- ✅ 2 valid results submitted to Appliqation
-- ⚠️ 1 orphan logged in console and summary file
-- ✅ CI/CD passes (because at least some tests were valid)
-- ✅ Analytics remain accurate (only valid tests counted)
-
-### Troubleshooting FAQ
-
-**Q: Why does my run get deleted?**
-
-Your run is deleted only when **100% of your tests lack UUID annotations**. This prevents corrupted analytics. Add UUIDs to at least one test to keep the run.
-
-**Q: How do I disable automatic deletion?**
-
-Set `APPLIQATION_DELETE_ORPHAN_RUNS=false` in your `.env` file. However, this is not recommended as it will corrupt your analytics with N/A pass rates.
-
-**Q: Can I keep orphan-only runs but still fail CI/CD?**
-
-Yes! Set:
-```env
-APPLIQATION_DELETE_ORPHAN_RUNS=false
-APPLIQATION_FAIL_ON_ORPHAN_RUNS=true
-```
-
-This will create the run in Appliqation but still fail your pipeline, forcing developers to fix UUIDs.
-
-**Q: Where do I find UUIDs for my tests?**
-
-1. Log into Appliqation portal
-2. Navigate to your project
-3. Go to "Test Cases" tab
-4. Find your test case
-5. The UUID is in the format: `{test_nid}-{uuid}` (e.g., `1154-c1f9559c-b978-43cc-9c76-fd539c717cb4`)
-
-**Q: Does orphan cleanup affect test execution?**
-
-No! Cleanup happens **after** all tests complete in the `onEnd()` hook. Test execution is never blocked or interrupted.
-
-**Q: What if deletion fails?**
-
-Deletion is fire-and-forget with error handling. If deletion fails (network issue, permission, etc.), it's logged as an error but doesn't crash your test run. The corrupted run may remain in Appliqation in this rare case.
-
----
-
-## How It Works
-
-Understanding the SDK's authentication and result submission flow helps troubleshoot issues.
-
-### Authentication & Test Execution Flow
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│                    GLOBAL SETUP (Runs Once)                     │
-├─────────────────────────────────────────────────────────────────┤
-│                                                                   │
-│  1. Read .env configuration                                     │
-│     ├─ APPLIQATION_API_KEY                                      │
-│     ├─ APPLIQATION_PROJECT_KEY                                  │
-│     ├─ TEST_APP_URL                                     │
-                                  │
-│                   │                                              │
-│                   ▼                                              │
-│  2. Create automation run via API                               │
-│     POST /api/automation/run/create                             │
-│     ├─ Returns: run_id, api_token                               │
-│     └─ Saves to: process.env.APPLIQATION_RUN_ID                │
-│                   │                                              │
-│                   ▼                                              │
-│  3. Request browser JWT token                                   │
-│     POST /api/auth/jwt/browser                                  │
-│     ├─ Send: api_key                                            │
-│     └─ Returns: jwt_token, expires_in                           │
-│                   │                                              │
-│                   ▼                                              │
-│  4. Setup browser authentication                                │
-│     ├─ Launch headless browser                                  │
-│     ├─ Navigate to baseURL                                      │
-│     ├─ Set cookie: appliqation_jwt = jwt_token                 │
-│     ├─ Save storage state to: .auth/jwt.json                   │
-│     └─ Close browser                                            │
-│                                                                   │
-│  ✅ Setup complete - All tests will use authenticated state     │
-│                                                                   │
-└─────────────────────────────────────────────────────────────────┘
-                               │
-                               ▼
-┌─────────────────────────────────────────────────────────────────┐
-│                   TEST EXECUTION (Per Worker)                   │
-├─────────────────────────────────────────────────────────────────┤
-│                                                                   │
-│  For each test:                                                  │
-│                                                                   │
-│  1. Load storage state from .auth/jwt.json                      │
-│     └─ Browser starts with JWT cookie already set              │
-│                   │                                              │
-│                   ▼                                              │
-│  2. Run test code                                               │
-│     ├─ mapAppqUuid(testInfo, '1154-uuid-here')                 │
-│     ├─ Execute test steps (page.goto, clicks, etc.)            │
-│     └─ Collect result: passed/failed/skipped                    │
-│                   │                                              │
-│                   ▼                                              │
-│  3. Reporter collects result                                    │
-│     ├─ Extract UUID from test.info().annotations               │
-│     ├─ Capture browser/OS/device metadata                      │
-│     ├─ Add to batch queue                                       │
-│     └─ (Batch submitted when size reached or tests complete)   │
-│                                                                   │
-└─────────────────────────────────────────────────────────────────┘
-                               │
-                               ▼
-┌─────────────────────────────────────────────────────────────────┐
-│                    REPORTER (After All Tests)                   │
-├─────────────────────────────────────────────────────────────────┤
-│                                                                   │
-│  1. Batch all test results                                      │
-│     └─ Group by 50 results per batch (configurable)            │
-│                   │                                              │
-│                   ▼                                              │
-│  2. Submit to portal                                            │
-│     POST /api/automation/result/batch                           │
-│     ├─ Send: run_id, results[], api_key                        │
-│     └─ Retry on failure (exponential backoff)                  │
-│                   │                                              │
-│                   ▼                                              │
-│  3. Handle orphan tests (tests without UUIDs)                  │
-│     POST /api/automation/orphans                                │
-│     └─ Logs tests that need UUID mapping                       │
-│                   │                                              │
-│                   ▼                                              │
-│  ✅ All results submitted to portal                             │
-│                                                                   │
-└─────────────────────────────────────────────────────────────────┘
-                               │
-                               ▼
-┌─────────────────────────────────────────────────────────────────┐
-│                 GLOBAL TEARDOWN (Runs Once)                     │
-├─────────────────────────────────────────────────────────────────┤
-│                                                                   │
-│  1. Cleanup temporary files (optional)                          │
-│  2. Log final summary                                           │
-│                                                                   │
-│  ✅ Test run complete                                           │
-│                                                                   │
-└─────────────────────────────────────────────────────────────────┘
-```
-
-**Key Points:**
-- Authentication happens **once** in global-setup
-- JWT token saved to `.auth/jwt.json`
-- All test workers reuse the same authenticated state
-- No per-test login required!
-
-### SDK Architecture
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│                     Playwright Test Suite                       │
-│                                                                   │
-│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐      │
-│  │  Test File 1  │    │  Test File 2  │    │  Test File 3  │      │
-│  │              │    │              │    │              │      │
-│  │ mapAppqUuid()│    │ mapAppqUuid()│    │ mapAppqUuid()│      │
-│  └──────────────┘    └──────────────┘    └──────────────┘      │
-│         │                    │                    │              │
-└─────────┼────────────────────┼────────────────────┼──────────────┘
-          │                    │                    │
-          └────────────────────┴────────────────────┘
-                               │
-                               ▼
-          ┌─────────────────────────────────────────┐
-          │     Appliqation SDK Reporter            │
-          │                                          │
-          │  • Collects test results                │
-          │  • Extracts UUIDs from annotations      │
-          │  • Batches submissions                  │
-          │  • Handles browser/OS/device metadata   │
-          └─────────────────────────────────────────┘
-                               │
-                               ▼
-          ┌─────────────────────────────────────────┐
-          │     Appliqation HTTP Client             │
-          │                                          │
-          │  • API key authentication               │
-          │  • Connection pooling                   │
-          │  • Exponential backoff retry            │
-          └─────────────────────────────────────────┘
-                               │
-                               ▼
-          ┌─────────────────────────────────────────┐
-          │     Appliqation Portal API              │
-          │                                          │
-          │  • Validates API key                    │
-          │  • Creates run matrices                 │
-          │  • Stores test results                  │
-          │  • Generates reports                    │
-          └─────────────────────────────────────────┘
-```
-
-### File Structure After Setup
-
-```
-your-playwright-project/
-├── .auth/                          # Auto-created by SDK global-setup
-│   └── jwt.json                    # Browser authentication state
-│
-├── .env                            # YOU CREATE THIS (Step 2)
-│   ├── APPLIQATION_API_KEY=appq_live_xxxxx
-│   ├── APPLIQATION_PROJECT_KEY=your-project-key
-│   ├── TEST_APP_URL=https://your-domain.com
-│   
-│
-├── playwright.config.js            # YOU UPDATE THIS (Step 3)
-│   ├── globalSetup: require.resolve('@appliqation/automation-sdk/playwright/global-setup')
-│   ├── globalTeardown: require.resolve('@appliqation/automation-sdk/playwright/global-teardown')
-│   ├── use: { storageState: '.auth/jwt.json' }
-│   └── reporter: ['@appliqation/automation-sdk/playwright/reporter', config]
-│
-├── tests/
-│   ├── login.spec.js               # YOUR TESTS (Step 4)
-│   │   ├── const { mapAppqUuid } = require('@appliqation/automation-sdk/utils');
-│   │   └── mapAppqUuid(testInfo, '1154-uuid-here');
-│   │
-│   └── checkout.spec.js
-│
-├── package.json
-│   └── dependencies:
-│       ├── @playwright/test
-│       ├── @appliqation/automation-sdk
-│       └── dotenv                  # Required for .env loading
-│
-└── node_modules/
-    └── @appliqation/automation-sdk/
-        └── playwright/
-            ├── global-setup.js     # SDK provides this!
-            ├── global-teardown.js  # SDK provides this!
-            └── index.js            # Reporter
-```
-
----
-
-## Configuration Reference
-
-### Environment Variables (.env)
-
-All environment variables and their purposes:
-
-| Variable | Required | Description | Example |
-|----------|----------|-------------|---------|
-| `TEST_APP_URL` | ✅ Yes | Your Test App  URL (no trailing slash) | `https://example.com` |
-| `APPLIQATION_API_KEY` | ✅ Yes | API key from `/admin/config/appliqation/api-keys` | `appq_live_abc123xyz...` |
-| `APPLIQATION_PROJECT_KEY` | ✅ Yes | Project key from project settings | `my-project-key` |
-
-
-**Common Configurations:**
-
-**CI/CD Pipeline:**
-```javascript
-{
-  environment: process.env.CI_ENVIRONMENT || 'CI',
-  title: `Build #${process.env.CI_BUILD_NUMBER}`,
-  logLevel: 'ERROR',  // Less verbose in CI
-  batchSize: 100      // Larger batches for speed
-}
-```
-
-**Local Development:**
-```javascript
-{
-  environment: 'Local',
-  logLevel: 'DEBUG',          // Detailed logs
-  logOrphans: true,           // See unmapped tests
-  rejectUnauthorized: false   // Allow local SSL
-}
-```
-
----
-
-## Adding Test Case UUIDs
-
-Map your automated tests to Appliqation test cases to track results.
-
-### Method: mapAppqUuid()
-
-**Import the utility:**
-```javascript
-const { mapAppqUuid } = require('@appliqation/automation-sdk/utils');
-```
-
-**Use in tests:**
-```javascript
-test('test description', async ({ page }, testInfo) => {
-  //                                        ^^^^^^^^ Required parameter!
+const appliqationConfig = {
+  apiKey: process.env.APPLIQATION_API_KEY,
+  projectKey: process.env.APPLIQATION_PROJECT_KEY,
 
-  // Map to test case UUID
-  mapAppqUuid(testInfo, '1154-7a17b809-0ff9-4ba1-9322-4eb2a49abfc5');
+  scenarioId: parseInt(process.env.APPLIQATION_SCENARIO_ID) || undefined,
+  // testSetId: parseInt(process.env.MY_SMOKE_TEST_SET_ID) || undefined,
 
-  // Your test code...
-});
+  // How to resolve a run whose executed tests fall outside the configured
+  // scope — 'cancel' (default, fail-safe), 'adhoc', or 'filter'.
+  onScopeMismatch: process.env.APPLIQATION_ON_SCOPE_MISMATCH,
+};
 ```
 
-**Important:**
-- ⚠️ Always include `testInfo` as second parameter
-- ⚠️ UUID format: `nid-uuid` (e.g., `1154-abc123...`)
-- ⚠️ Get UUIDs from Appliqation portal test cases
-
-### Getting UUIDs from Portal
+Before the suite runs, the reporter checks every UUID it's about to execute
+against the real membership of the configured scenario or test set, so a
+tag-based selection (`-g`, `--grep`) that drifts outside it doesn't silently
+write results into the wrong run:
 
-1. Login to Appliqation portal
-2. Navigate to **Test Cases** section
-3. Open the test case you want to map
-4. Copy the UUID (shown in test case details)
-5. Use format: `nid-uuid` where:
-   - `nid` = Test case node ID (e.g., `1154`)
-   - `uuid` = Test case UUID (e.g., `7a17b809-0ff9-4ba1-9322-4eb2a49abfc5`)
+| Strategy | Behavior |
+|---|---|
+| `cancel` (default in CI) | Nothing submitted, exit code 1. Tests still run locally so you keep your own Playwright output. |
+| `adhoc` | Overrides the configured scope and creates an ad-hoc run instead — everything is submitted. |
+| `filter` | Submits only the in-scope results; out-of-scope tests still execute but aren't uploaded. |
 
-### What About Tests Without UUIDs?
+Outside CI, with an interactive terminal, you're prompted to choose one of
+the three before the run starts.
 
-Tests without `mapAppqUuid()` calls are called **orphan tests**. The SDK will:
-- ✅ Still run them normally
-- ✅ Collect results
-- ✅ Log them separately (if `logOrphans: true`)
-- ❌ Won't report to portal (no UUID to map to)
+## Authenticated apps (gated SUTs)
 
-**Check orphan logs to see which tests need UUID mapping.**
+If your application requires login, the SDK ships a portable auth setup
+that works the same way in Appliqation's own executor and in your CI/local
+runs. The login flow itself is **your code, in your repo**
+(`tests/automan/auth/login.ts` by default — the path is configurable in
+your project's Appliqation settings) — both environments run that same
+function, so SSO, MFA, multi-step login, OAuth, and custom redirects all
+work because you write the Playwright code that handles them.
 
----
+### One-time setup per project
 
-## Troubleshooting
-
-### Common Issues and Solutions
+1. **In Appliqation**: configure roles and credentials under **Project
+   Settings → Auth Config**. Credentials are encrypted server-side. This is
+   also where you'd change the login file's path/location if your project
+   doesn't use the default.
 
-#### ❌ Error: "APPLIQATION_API_KEY is required"
+2. **In your repo**, create `tests/automan/auth/login.ts`:
 
-**Cause:** `.env` file not loaded or missing variable
+   ```typescript
+   import { defineLogin } from '@appliqation/automation-sdk/login';
 
-**Fix:**
-1. Verify `.env` file exists in project root
-2. Check variable name spelling (no typos!)
-3. Ensure `require('dotenv').config()` at top of `playwright.config.js`
-4. No spaces around `=` sign:
-   ```bash
-   # ❌ Wrong
-   APPLIQATION_API_KEY = appq_live_xxx
+   export default defineLogin(async (page, { username, password, role, baseURL }) => {
+     await page.goto('/login'); // baseURL handles env (staging/prod)
+     await page.getByLabel('Email').fill(username);
+     await page.getByLabel('Password').fill(password);
+     await page.getByRole('button', { name: 'Sign in' }).click();
+     await page.waitForURL('**/dashboard');
 
-   # ✅ Correct
-   APPLIQATION_API_KEY=appq_live_xxx
+     if (role === 'admin') {
+       await page.getByRole('button', { name: 'Switch to admin view' }).click();
+     }
+   });
    ```
 
----
+   Push to your project's default branch — Appliqation's GitHub webhook
+   ingests the file automatically.
 
-#### ❌ Tests redirecting to login page (authentication failed)
-
-**Cause:** JWT authentication not working, storage state not loading
-
-**Fix:**
-1. Verify `globalSetup` configured correctly:
-   ```javascript
-   globalSetup: require.resolve('@appliqation/automation-sdk/playwright/global-setup')
-   ```
+3. **In Appliqation**, use the **download .env for CI** link on the project
+   settings page and add the values to your CI secrets:
 
-2. Check `.auth/jwt.json` file exists after global-setup runs
-
-3. Verify all projects use storage state:
-   ```javascript
-   projects: [{
-     name: 'chromium',
-     use: { storageState: '.auth/jwt.json' }  // Must be set!
-   }]
-   ```
-
-4. Check `.env` has correct `TEST_APP_URL` (no trailing slash):
    ```bash
-   # ❌ Wrong
-   TEST_APP_URL=https://portal.com/
+   APPLIQATION_API_KEY="<project API key>"
+   APPLIQATION_BASE_URL="https://appliqation.io"
+   APPLIQATION_PROJECT_KEY=126
+   APPLIQATION_SUT_BASE_URL="<your SUT URL, e.g. https://staging.acme.com>"
 
-   # ✅ Correct
-   TEST_APP_URL=https://portal.com
+   APPQ_PROJECT_126_DEFAULT_USERNAME="<username>"
+   APPQ_PROJECT_126_DEFAULT_PASSWORD="<password>"
    ```
 
----
+   `APPLIQATION_SUT_BASE_URL` is what your login function's
+   `page.goto('/login')` resolves against, so the same `login.ts` works
+   across staging/preprod/prod by changing one env var.
 
-#### ❌ Error: "testInfo is not defined"
+4. **(Optional)** Use **Test login** on the project settings page to
+   validate `login.ts` against the live SUT before wiring up CI.
 
-**Cause:** Missing `testInfo` parameter in test function
+### In your test files
 
-**Fix:**
-```javascript
-// ❌ Wrong - missing testInfo
-test('my test', async ({ page }) => {
-  mapAppqUuid(testInfo, 'uuid-here');  // testInfo undefined!
-});
+```typescript
+const { mapAppqUuid, setupAuth } = require('@appliqation/automation-sdk/utils');
+const { test, expect } = require('@playwright/test');
 
-// ✅ Correct - include testInfo
-test('my test', async ({ page }, testInfo) => {
-  mapAppqUuid(testInfo, 'uuid-here');
+test.use({
+  storageState: setupAuth({ project_id: 126, role: 'default' }),
 });
-```
-
----
 
-#### ❌ Error: "Invalid UUID format"
-
-**Cause:** UUID not in `nid-uuid` format
-
-**Fix:**
-```javascript
-// ❌ Wrong formats
-mapAppqUuid(testInfo, '7a17b809-0ff9-4ba1-9322-4eb2a49abfc5');  // Missing nid
-mapAppqUuid(testInfo, '1154');  // Missing UUID
-mapAppqUuid(testInfo, 'test-case-123');  // Wrong format
-
-// ✅ Correct format
-mapAppqUuid(testInfo, '1154-7a17b809-0ff9-4ba1-9322-4eb2a49abfc5');
+test('manager dashboard loads', async ({ page }, testInfo) => {
+  mapAppqUuid(testInfo, '1141-...');
+  await page.goto('/dashboard'); // already authenticated
+});
 ```
 
----
-
-#### ❌ Error: "Module 'dotenv' not found"
+`setupAuth()` returns a deterministic storage-state file path — it does
+**not** perform login itself:
 
-**Cause:** `dotenv` package not installed
+- **In Appliqation's own executor**, the login helper imports `login.ts`
+  from the canonical store and writes that state before your test runs.
+  Nothing else to do.
+- **In your CI/local runs**, run `npx appq-auth-setup` once before
+  `playwright test`.
 
-**Fix:**
 ```bash
-npm install dotenv
+npx appq-auth-setup --project-id 126 --role default
+npx playwright test
 ```
 
----
-
-#### ❌ Error: "unable to verify the first certificate" or SSL errors
+Re-run the CLI whenever sessions expire (typically once per CI run) — it's
+idempotent and fast on cache hits.
 
-**Cause:** Using self-signed SSL certificates in development environment
+If `login.ts` is TypeScript, the CLI needs a TS loader as a dev dependency
+(`npm install --save-dev tsx`, or `ts-node`). Plain `.js`/`.mjs`/`.cjs`
+login files need neither. The `appq-auth-setup` CLI itself resolves the
+login file path via `--login-file`, then `APPQ_LOGIN_FILE`, then its own
+local default — set the path in Project Settings if you want Appliqation's
+executor and your CLI/CI runs to agree on a non-default location.
 
-**Fix:**
+**Multiple roles**: configure each in Project Settings, then run one
+`appq-auth-setup` call per role and reference the matching role in
+`setupAuth({ project_id, role })`:
 
-Add `rejectUnauthorized: false` to your SDK configuration in `playwright.config.js`:
+```bash
+npx appq-auth-setup --project-id 126 --role default
+npx appq-auth-setup --project-id 126 --role manager
+```
+
+## CI/CD
+
+Any CI system works — the SDK just needs the same environment variables
+your local run uses, provided as secrets. GitHub Actions example:
+
+```yaml
+# .github/workflows/playwright.yml
+name: Playwright Tests
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+      - run: npm ci
+      - run: npx playwright install --with-deps chromium
+
+      # Only if your app requires login — see "Authenticated apps" above
+      - run: npx appq-auth-setup --project-id 126 --role default
+        env:
+          APPLIQATION_SUT_BASE_URL: ${{ vars.STAGING_URL }}
+          APPQ_PROJECT_126_DEFAULT_USERNAME: ${{ secrets.APPQ_DEFAULT_USERNAME }}
+          APPQ_PROJECT_126_DEFAULT_PASSWORD: ${{ secrets.APPQ_DEFAULT_PASSWORD }}
+
+      - name: Run tests and report to Appliqation
+        env:
+          APPQ_ENABLED: '1'
+          APPLIQATION_API_KEY: ${{ secrets.APPLIQATION_API_KEY }}
+          APPLIQATION_PROJECT_KEY: ${{ secrets.APPLIQATION_PROJECT_KEY }}
+          APPLIQATION_ENVIRONMENT: Staging
+          APPLIQATION_RUN_TITLE: "CI Build #${{ github.run_number }}"
+        run: npx playwright test
+```
+
+Nothing here is GitHub-specific beyond the workflow syntax — the same
+`npm ci && npx playwright install && APPQ_ENABLED=1 npx playwright test`
+sequence, with the env vars below set as secrets, works on GitLab CI,
+CircleCI, Jenkins, or any other runner.
+
+## Configuration reference
+
+### Environment variables
+
+| Variable | Required | Description |
+|---|---|---|
+| `APPLIQATION_API_KEY` | Yes | API key from Project Settings → API Keys |
+| `APPLIQATION_PROJECT_KEY` | Yes | Project key from Project Settings |
+| `TEST_APP_URL` | Recommended | Your own convention for Playwright's `baseURL` — read it in your `playwright.config.js`, the SDK doesn't read it directly |
+| `APPLIQATION_ENVIRONMENT` | No | Environment label for the run (e.g. `Staging`) — also settable via `--appq-env` |
+| `APPLIQATION_RUN_TITLE` | No | Custom run title — also settable via `--appq_run_title` |
+| `APPLIQATION_RUN_ID` | No | Reuse an existing run instead of creating one (TDD iteration mode) |
+| `APPQ_ENABLED` | No | `1`/`true` enables reporting — alternative to the `--appq` CLI flag |
+| `APPLIQATION_ON_SCOPE_MISMATCH` | No | `cancel` (default) / `adhoc` / `filter` — see [Scoping runs](#scoping-runs-scenario-or-test-set) |
+| `APPLIQATION_AUTO_TAG_ENABLED` | No | `false` disables auto-tagging (default: enabled). The tag name itself (`Appq_Auto`) is fixed, not configurable — see [Auto-tagging](#auto-tagging-test-cases) |
+| `APPLIQATION_INSECURE` | No | Set to `allow` to permit `rejectUnauthorized: false` in CI/production — see [Troubleshooting](#tls-certificate-errors) |
+| `LOG_LEVEL` | No | `ERROR` / `WARN` / `INFO` (default) / `DEBUG` |
+
+`scenarioId`/`testSetId` and orphan-run handling (below) are reporter
+constructor options, not environment variables — read your own env var of
+choice into them in `playwright.config.js`, the same way the examples above
+do for `scenarioId`.
+
+### Reporter options (`playwright.config.js`)
 
 ```javascript
-const appliqationConfig = {
+['@appliqation/automation-sdk/playwright/reporter', {
   apiKey: process.env.APPLIQATION_API_KEY,
   projectKey: process.env.APPLIQATION_PROJECT_KEY,
+  environment: process.env.APPLIQATION_ENVIRONMENT,
 
-  // Add this for self-signed certificates
-  rejectUnauthorized: false  // ⚠️ Only use in development!
-};
-```
-
-**⚠️ Security Note:** Only use `rejectUnauthorized: false` in development environments with self-signed certificates. In production with valid SSL certificates, remove this setting or set it to `true`.
-
----
-
-#### ❌ Results not appearing in portal
-
-**Possible causes:**
-
-1. **Reporting not enabled** - Missing `APPQ_ENABLE` env var or `--appq` flag (MOST COMMON!)
-   - **Fix (Method 1 - Recommended):** Set `APPQ_ENABLE=1` environment variable
-     - Example: `APPQ_ENABLE=1 npx playwright test`
-   - **Fix (Method 2):** Add `-- --appq` flag to your test command
-     - Example: `npx playwright test -- --appq`
-   - Check console output for: "Appliqation reporting disabled"
-
-2. **Orphan tests** - Tests don't have `mapAppqUuid()` calls
-   - **Fix:** Add UUID mapping to all tests
-   - Enable `logOrphans: true` to see which tests are orphans
+  scenarioId: parseInt(process.env.APPLIQATION_SCENARIO_ID) || undefined,
+  onScopeMismatch: process.env.APPLIQATION_ON_SCOPE_MISMATCH,
 
-3. **API key invalid** - Authentication failed
-   - **Fix:** Regenerate API key from portal
+  autoCreateRun: true,     // default: true
+  batchSubmit: true,       // default: true
+  batchSize: 50,           // results per batch
+  logOrphans: true,        // log tests with no mapped UUID
+  logLevel: 'INFO',
 
-4. **Network issues** - Portal unreachable
-   - **Fix:** Check `TEST_APP_URL` is accessible
+  deleteOrphanOnlyRuns: true,  // delete a run if every test in it was orphaned
+  failOnOrphanOnlyRuns: true,  // exit 1 in that case, so CI catches it
 
----
-
-### Debug Mode
-
-Enable detailed logging to troubleshoot issues:
-
-**In .env:**
-```bash
-LOG_LEVEL=DEBUG
+  rejectUnauthorized: true,    // set false only for local self-signed certs
+}]
 ```
 
-**Or in config:**
-```javascript
-const appliqationConfig = {
-  // ... other config
-  logLevel: 'DEBUG'
-};
-```
-
-**Debug output shows:**
-- ✅ API requests and responses
-- ✅ Authentication flow details
-- ✅ Result submission batches
-- ✅ Orphan test detection
-- ✅ Error stack traces
+## Orphan test handling
 
----
+A test with no `mapAppqUuid()` call still runs, but its result can't be
+linked to an Appliqation test case — it's logged as an **orphan** instead
+of being submitted.
 
-### Test Connection
+If **every** test in a run is orphaned, the run is deleted automatically
+(`deleteOrphanOnlyRuns`, default `true`) and the process exits non-zero
+(`failOnOrphanOnlyRuns`, default `true`), so a fully-unmapped suite fails
+CI loudly instead of quietly writing an empty run with an N/A pass rate
+into your dashboards. If only *some* tests are orphaned, the run is kept,
+mapped results are submitted normally, and the orphans are logged as
+warnings.
 
-Verify SDK can reach your portal:
-
-**Create test file:** `test-connection.js`
-```javascript
-const AppliqationClient = require('@appliqation/automation-sdk');
-require('dotenv').config();
-
-async function testConnection() {
-  const client = new AppliqationClient({
-    apiKey: process.env.APPLIQATION_API_KEY,
-    projectKey: process.env.APPLIQATION_PROJECT_KEY
-  });
-
-  const result = await client.testConnection();
-  console.log('Connection test:', result);
-}
-
-testConnection();
-```
-
-**Run:**
-```bash
-node test-connection.js
-```
-
-**Expected output:**
-```
-Connection test: { success: true, message: 'Connected to Appliqation portal' }
-```
+## Auto-tagging test cases
 
----
+The first time the SDK successfully submits a result for a test case
+(pass, fail, or skip — any result the backend accepts), it tags that test
+case with `Appq_Auto` in Appliqation, so you can see which test cases have
+real automation coverage. The tag name is fixed, not configurable — appq's
+own backend writes this exact same tag when it auto-tags at run creation,
+so the two stay consistent. Tagging is fire-and-forget: a tagging failure
+is logged as a warning and never affects your test results or exit code.
+Disable it entirely with `APPLIQATION_AUTO_TAG_ENABLED=false`.
 
-## Advanced Features
+## Advanced
 
-### Custom Playwright Fixture (Auto JWT Refresh)
+### Auto-refreshing JWT fixture
 
-For long-running test suites, use the SDK's custom fixture to automatically refresh JWT tokens:
+For long-running suites, use the SDK's Playwright fixture in place of
+`@playwright/test` directly — it checks JWT expiry before each test and
+refreshes if less than 5 minutes remain, updating the saved storage state:
 
-**File:** `tests/example.spec.js`
 ```javascript
-// Use the SDK's test fixture instead of Playwright's default
 const { test, expect } = require('@appliqation/automation-sdk/playwright');
 
-test.describe('Authenticated Tests', () => {
-  test('should have valid JWT throughout test', async ({ page }) => {
-    // JWT is automatically refreshed if < 5 minutes remaining
-    await page.goto('/dashboard');
-    // Your test logic...
-  });
+test('long-running suite stays authenticated', async ({ page }) => {
+  await page.goto('/dashboard');
 });
 ```
 
-**What it does:**
-- ✅ Checks JWT expiry before each test
-- ✅ Automatically refreshes if < 5 minutes remaining
-- ✅ Updates `.auth/jwt.json` with new token
-- ✅ Prevents authentication failures in long test runs
-
-### Using Core SDK Directly
+### Using the client directly
 
-For custom integrations or non-Playwright frameworks:
+For custom integrations outside Playwright's reporter lifecycle:
 
 ```javascript
 const AppliqationClient = require('@appliqation/automation-sdk');
 
-// Initialize client
 const client = new AppliqationClient({
   apiKey: process.env.APPLIQATION_API_KEY,
-  projectKey: process.env.APPLIQATION_PROJECT_KEY
+  projectKey: process.env.APPLIQATION_PROJECT_KEY,
 });
 
-// Create run
 const run = await client.createRun({
-  environment: 'Production',
+  environment: 'Staging',
   browsers: ['Chrome'],
   device: 'Desktop',
-  os: 'Windows 11'
+  os: 'Windows 11',
 });
 
-console.log('Run created:', run.runId);
-
-// Submit single result
 await client.submitResult(run.runId, {
   uuid: '1154-7a17b809-0ff9-4ba1-9322-4eb2a49abfc5',
   status: 'passed',
   browser: 'Chrome',
-  comment: 'Test passed successfully'
 });
 
-// Submit batch results
-const results = [
+const summary = await client.submitBatch([
   { uuid: '1154-uuid-1', runId: run.runId, status: 'passed', browser: 'Chrome' },
-  { uuid: '1154-uuid-2', runId: run.runId, status: 'failed', browser: 'Chrome' }
-];
-
-const summary = await client.submitBatch(results);
-console.log(`Submitted: ${summary.success}/${summary.total} successful`);
+  { uuid: '1154-uuid-2', runId: run.runId, status: 'failed', browser: 'Chrome' },
+]);
+console.log(`Submitted: ${summary.success} succeeded, ${summary.failed} failed`);
 ```
 
-### Custom Run Titles
+## How it works
 
-Set custom titles for test runs:
+1. **Global setup** (once, if you're using JWT auth for a gated app):
+   creates the run via `POST /api/automation/run/create`, requests a
+   browser JWT via `POST /api/auth/jwt/browser`, and saves authenticated
+   storage state so every worker starts already logged in.
+2. **Per test**: the reporter reads the UUID off `testInfo`'s annotations,
+   captures browser/OS/device metadata, and queues the result.
+3. **After the run**: results are submitted in batches
+   (`POST /api/automation/result/batch`), orphan tests are logged, and — if
+   configured — an orphan-only run is deleted and the process exits
+   non-zero.
 
-**Method 1: Environment Variable**
-```bash
-APPLIQATION_RUN_TITLE="Sprint 24 - Regression Tests" npx playwright test
-```
----
-
-## API Reference
+A timestamped execution summary (counts, run IDs, and any rejected/orphan
+detail) is always written to
+`test-results/AppQ_Execution_Summary/` after each run, independent of
+whatever Playwright reporters you also have configured.
 
-### Quick Reference
+## Troubleshooting
 
-Most commonly used SDK exports:
+**Results not appearing in the portal** — almost always reporting wasn't
+enabled: confirm `APPQ_ENABLED=1` or `-- --appq` was actually passed (the
+console prints "Appliqation reporting enabled" when it is). Next most
+common cause is orphan tests — check for a "no mapped UUID" warning in the
+output. If neither, verify the API key and `APPLIQATION_PROJECT_KEY` are
+correct with:
 
-**Reporter (for playwright.config.js):**
 ```javascript
-['@appliqation/automation-sdk/playwright/reporter', config]
+const AppliqationClient = require('@appliqation/automation-sdk');
+new AppliqationClient({
+  apiKey: process.env.APPLIQATION_API_KEY,
+  projectKey: process.env.APPLIQATION_PROJECT_KEY,
+}).testConnection().then(() => console.log('OK')).catch(console.error);
 ```
 
-**Global Setup/Teardown:**
-```javascript
-require.resolve('@appliqation/automation-sdk/playwright/global-setup')
-require.resolve('@appliqation/automation-sdk/playwright/global-teardown')
-```
+**`APPLIQATION_API_KEY is required`** — the reporter only reads
+`process.env`, so make sure the variable is actually exported/injected
+into the process running `playwright test` (a CI secret not mapped to an
+`env:` key, or a `.env` file your own tooling never loaded, are the usual
+causes).
 
-**Utilities:**
-```javascript
-const { mapAppqUuid } = require('@appliqation/automation-sdk/utils');
-```
+**Tests redirect to a login page** — for JWT-based auth, confirm
+`globalSetup` is `require.resolve('@appliqation/automation-sdk/playwright/global-setup')`
+and every project's `use.storageState` points at the same file
+(`.auth/jwt.json` by default) that setup wrote. For `defineLogin`-based
+auth, confirm `npx appq-auth-setup` ran before the tests and its
+`--project-id`/`--role` match what `setupAuth()` is called with.
 
-**Core Client:**
-```javascript
-const AppliqationClient = require('@appliqation/automation-sdk');
-```
+**`Invalid UUID format`** — UUIDs must be `{nid}-{uuid}`, e.g.
+`1154-7a17b809-0ff9-4ba1-9322-4eb2a49abfc5`. A bare UUID with no leading
+node ID, or a bare node ID with no UUID, both fail validation.
+
+### TLS certificate errors
 
-### Full API Documentation
+For local self-signed certificates, set `rejectUnauthorized: false` in the
+reporter config. In CI or production, this throws a configuration error by
+design — TLS bypass there would let a MITM attacker intercept result
+submissions. If you genuinely need it in CI, set
+`APPLIQATION_INSECURE=allow` to explicitly acknowledge the risk.
 
-For complete API reference including all methods, parameters, and return types, see the source code documentation in the `/src` directory.
+## Further reading
 
----
+- [docs/SETUP.md](./docs/SETUP.md) — step-by-step setup guide
+- [docs/JWT-AUTHENTICATION.md](./docs/JWT-AUTHENTICATION.md) — JWT auth flow in depth
 
 ## Support
 
-- 📧 **Email**: support@appliqation.com
-- 📖 **Documentation**: https://docs.appliqation.com
-- 🐛 **Issues**: https://github.com/appliqation/automation-sdk-js/issues
+- Issues: https://github.com/appliqation/automation-sdk-js/issues
 
 ## License
 
-MIT License - see LICENSE file for details.
+MIT — see [LICENSE](./LICENSE).