@appliqation/automation-sdk 2.2.0 → 2.3.0

package/README.md

@@ -459,15 +459,277 @@ DETAILED ERRORS & WARNINGS:
 
 ---
 
-## Auto-tag Accepted Tests
+## Auto-Tagging Test Cases
 
-After a test is automated, the SDK can tag its UUID (default tag: `Appq_automated`) for accepted results only. Tagging is fire-and-forget: failures are logged as warnings and never fail your run.
+The SDK automatically tags test cases with "Appq_automated" (configurable) after their **first successful run**. This helps you track which test cases have been automated and are actively running in your test suite.
 
-- Enabled by default when Appq is enabled.
-- Overrides:
-  - Tag name: `APPLIQATION_TAG_NAME=MyTag`
-  - Endpoint (if customized): `APPLIQATION_TAG_ENDPOINT=/api/automation/testcase/tag`
-- Works for both batch and single submissions; backend-rejected results are skipped automatically.
+### How It Works
+
+1. **Test runs and passes** → SDK submits result to Appliqation
+2. **Backend accepts result** → SDK triggers auto-tagging (fire-and-forget)
+3. **Check if already tagged** → Skip if test case already has the tag
+4. **Add tag** → Test case gets tagged in Appliqation UI
+
+**Key Features:**
+- ✅ **Enabled by default** when Appliqation reporting is enabled
+- ✅ **Fire-and-forget** - tagging failures never block your test runs
+- ✅ **Smart deduplication** - checks before tagging, won't create duplicate tags
+- ✅ **Only accepted results** - backend-rejected results are NOT tagged
+- ✅ **Works for both** single and batch result submissions
+- ✅ **Async execution** - zero impact on test execution performance
+
+### Configuration
+
+#### Environment Variables
+
+Add to your `.env` file:
+
+```bash
+# Auto-Tagging Configuration (optional - all have sensible defaults)
+APPLIQATION_AUTO_TAG_ENABLED=true              # Enable/disable (default: true)
+APPLIQATION_AUTO_TAG_NAME=Appq_automated       # Custom tag name (default: Appq_automated)
+```
+
+#### Playwright Reporter Config
+
+Configure in `playwright.config.js`:
+
+```javascript
+reporter: [
+  ['@appliqation/automation-sdk/playwright/reporter', {
+    apiKey: process.env.APPLIQATION_API_KEY,
+    projectKey: process.env.APPLIQATION_PROJECT_KEY,
+
+    // Auto-tagging options (optional)
+    autoTag: true,                    // Enable auto-tagging (default: true)
+    autoTagName: 'My_Custom_Tag'      // Custom tag name (default: 'Appq_automated')
+  }]
+]
+```
+
+#### Programmatic Configuration
+
+When using the SDK directly:
+
+```javascript
+const { AppliqationClient } = require('@appliqation/automation-sdk');
+
+const client = new AppliqationClient({
+  apiKey: 'your_api_key',
+  projectKey: 'your_project_key',
+
+  // Auto-tagging options
+  options: {
+    autoTag: true,                    // Enable auto-tagging (default: true)
+    autoTagName: 'Automated_Test'     // Custom tag name (default: 'Appq_automated')
+  }
+});
+```
+
+### Disabling Auto-Tagging
+
+If you want to disable auto-tagging:
+
+**Option 1: Environment Variable**
+```bash
+APPLIQATION_AUTO_TAG_ENABLED=false
+```
+
+**Option 2: Config**
+```javascript
+{
+  options: {
+    autoTag: false
+  }
+}
+```
+
+### What You'll See
+
+When auto-tagging is working:
+
+```
+✅ Auto-tagged 3 test case(s) with "Appq_automated"
+```
+
+When test cases are already tagged (second run):
+
+```
+DEBUG: Skipped 3 already-tagged test case(s)
+```
+
+If tagging fails (non-blocking):
+
+```
+⚠️  Auto-tagging failed (non-blocking): Connection timeout
+```
+
+### Troubleshooting
+
+**Q: I don't see the tag in Appliqation UI**
+
+Check:
+1. Is `APPQ_ENABLE=1` set? (Auto-tagging only works when reporting is enabled)
+2. Did the test result get accepted by backend? (Check for backend validation errors)
+3. Check SDK logs for "Auto-tagged X test case(s)" message
+
+**Q: Can I use a custom tag name?**
+
+Yes! Set `APPLIQATION_AUTO_TAG_NAME=Your_Tag_Name` in your `.env` file or use the config options shown above.
+
+**Q: Does tagging failure affect my test results?**
+
+No! Auto-tagging is fire-and-forget. If tagging fails, it's logged as a warning but your test run continues normally and results are still submitted successfully.
+
+---
+
+## Handling Orphan Tests
+
+### What are Orphan Tests?
+
+**Orphan tests** are tests that execute successfully but **cannot be mapped to Appliqation test cases** because they're missing UUID annotations. When a test runs without a UUID, the SDK cannot link it to a specific test case in your Appliqation project, making the result "orphaned."
+
+```javascript
+// ❌ This test will be orphaned (no UUID annotation)
+test('Login with valid credentials', async ({ page }) => {
+  await page.goto('/login');
+  await page.fill('#username', 'user@example.com');
+  // ... test code
+});
+
+// ✅ This test will be properly mapped (has UUID annotation)
+test('Login with valid credentials', { tag: '@uuid:1154-abc-def' }, async ({ page }) => {
+  await page.goto('/login');
+  await page.fill('#username', 'user@example.com');
+  // ... test code
+});
+```
+
+### Automatic Orphan Run Cleanup
+
+By default, the SDK **automatically prevents corrupted runs** from being created when ALL tests in a run are orphaned:
+
+**Default Behavior:**
+- ✅ If ALL tests lack UUIDs → Run is **deleted** and a clear error message is shown
+- ✅ If SOME tests have UUIDs → Run is **kept**, valid results are submitted, orphans are logged as warnings
+- ✅ CI/CD pipeline **fails with exit code 1** when orphan-only runs are detected
+- ✅ Clear, actionable error message guides users on how to fix the issue
+
+**Why?** Orphan-only runs create empty entries in Appliqation with "N/A" pass rates, which corrupts your analytics and dashboards.
+
+### Error Message Example
+
+When all tests are orphaned, you'll see:
+
+```
+╔════════════════════════════════════════════════════════════════════╗
+║  ❌ RUN CREATION FAILED - ALL TESTS MISSING UUID ANNOTATIONS      ║
+╠════════════════════════════════════════════════════════════════════╣
+║  Project: 1162-MyProject                                           ║
+║  Orphan Tests: 6                                                   ║
+║                                                                    ║
+║  ⚠️  NO RESULTS WERE SUBMITTED TO APPLIQATION                      ║
+║  The test run was automatically deleted to prevent analytics       ║
+║  corruption. All tests are missing UUID annotations.               ║
+╠════════════════════════════════════════════════════════════════════╣
+║  ✅ ACTION REQUIRED: Add UUID Annotations                          ║
+╠════════════════════════════════════════════════════════════════════╣
+║  Option 1: Using test tags (Recommended)                          ║
+║  test('My Test', { tag: '@uuid:123-xxx' }, async ({ page }) => {  ║
+║    // your test code                                               ║
+║  });                                                               ║
+╚════════════════════════════════════════════════════════════════════╝
+```
+
+### Configuration Options
+
+You can customize orphan handling behavior via environment variables:
+
+```env
+# .env file
+
+# Delete runs with only orphan tests (default: true)
+APPLIQATION_DELETE_ORPHAN_RUNS=true
+
+# Exit with error code 1 for orphan-only runs (default: true)
+APPLIQATION_FAIL_ON_ORPHAN_RUNS=true
+```
+
+**Configuration via playwright.config.js:**
+
+```javascript
+reporter: [
+  [
+    '@appliqation/automation-sdk-js/playwright',
+    {
+      deleteOrphanOnlyRuns: true,   // Delete orphan-only runs (default: true)
+      failOnOrphanOnlyRuns: true,   // Fail CI/CD for orphan-only runs (default: true)
+    }
+  ]
+]
+```
+
+### Mixed Scenarios (Some Tests Have UUIDs)
+
+When your test suite has **both valid and orphan tests**, the SDK handles it gracefully:
+
+```javascript
+// Project 1162: 3 tests total
+test('Valid Test 1', { tag: '@uuid:1154-abc' }, async ({ page }) => {
+  // ✅ Will be submitted to Appliqation
+});
+
+test('Orphan Test 1', async ({ page }) => {
+  // ⚠️ Logged as warning, not submitted
+});
+
+test('Valid Test 2', { tag: '@uuid:1155-def' }, async ({ page }) => {
+  // ✅ Will be submitted to Appliqation
+});
+```
+
+**Result:**
+- ✅ Run is kept (because 2 tests have UUIDs)
+- ✅ 2 valid results submitted to Appliqation
+- ⚠️ 1 orphan logged in console and summary file
+- ✅ CI/CD passes (because at least some tests were valid)
+- ✅ Analytics remain accurate (only valid tests counted)
+
+### Troubleshooting FAQ
+
+**Q: Why does my run get deleted?**
+
+Your run is deleted only when **100% of your tests lack UUID annotations**. This prevents corrupted analytics. Add UUIDs to at least one test to keep the run.
+
+**Q: How do I disable automatic deletion?**
+
+Set `APPLIQATION_DELETE_ORPHAN_RUNS=false` in your `.env` file. However, this is not recommended as it will corrupt your analytics with N/A pass rates.
+
+**Q: Can I keep orphan-only runs but still fail CI/CD?**
+
+Yes! Set:
+```env
+APPLIQATION_DELETE_ORPHAN_RUNS=false
+APPLIQATION_FAIL_ON_ORPHAN_RUNS=true
+```
+
+This will create the run in Appliqation but still fail your pipeline, forcing developers to fix UUIDs.
+
+**Q: Where do I find UUIDs for my tests?**
+
+1. Log into Appliqation portal
+2. Navigate to your project
+3. Go to "Test Cases" tab
+4. Find your test case
+5. The UUID is in the format: `{test_nid}-{uuid}` (e.g., `1154-c1f9559c-b978-43cc-9c76-fd539c717cb4`)
+
+**Q: Does orphan cleanup affect test execution?**
+
+No! Cleanup happens **after** all tests complete in the `onEnd()` hook. Test execution is never blocked or interrupted.
+
+**Q: What if deletion fails?**
+
+Deletion is fire-and-forget with error handling. If deletion fails (network issue, permission, etc.), it's logged as an error but doesn't crash your test run. The corrupted run may remain in Appliqation in this rare case.
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@appliqation/automation-sdk",
-  "version": "2.2.0",
+  "version": "2.3.0",
   "description": "Appliqation Automation SDK with API key authentication, custom run titles, and framework-specific reporters",
   "main": "src/index.js",
   "types": "src/index.d.ts",
@@ -114,7 +114,7 @@
   },
   "repository": {
     "type": "git",
-    "url": "https://github.com/appliqation/automation-sdk-js"
+    "url": "git+https://github.com/appliqation/automation-sdk-js.git"
   },
   "bugs": {
     "url": "https://github.com/appliqation/automation-sdk-js/issues"

package/src/AppliqationClient.js

@@ -2,6 +2,7 @@ const HttpClient = require('./core/HttpClient');
 const AuthManager = require('./core/AuthManager');
 const RunMatrixService = require('./services/RunMatrixService');
 const ResultService = require('./services/ResultService');
+const TaggingService = require('./services/TaggingService');
 const OrphanTestService = require('./services/OrphanTestService');
 const UuidValidator = require('./utils/UuidValidator');
 const PayloadBuilder = require('./utils/PayloadBuilder');
@@ -99,7 +100,15 @@ class AppliqationClient {
         timeout: normalizedConfig.options?.timeout || 30000,
         retries: normalizedConfig.options?.retries || 3,
         logOrphans: normalizedConfig.options?.logOrphans !== false,
-        logLevel: normalizedConfig.options?.logLevel || 'info'
+        logLevel: normalizedConfig.options?.logLevel || 'info',
+        // Auto-tagging configuration
+        autoTag: normalizedConfig.options?.autoTag !== false,
+        autoTagName: normalizedConfig.options?.autoTagName ||
+                     normalizedConfig.autoTagName ||
+                     process.env.APPLIQATION_AUTO_TAG_NAME ||
+                     'Appq_automated',
+        autoTagBatchSize: normalizedConfig.options?.autoTagBatchSize || 50,
+        autoTagRetries: normalizedConfig.options?.autoTagRetries || 2
       }
     };
 
@@ -112,9 +121,16 @@ class AppliqationClient {
 
     // Initialize services
     this.runMatrix = new RunMatrixService(this.http, this.config);
-    this.results = new ResultService(this.http, this.config);
+    this.tagging = new TaggingService(this.http, this.config);
+    this.results = new ResultService(this.http, this.tagging, this.config);
     this.orphans = new OrphanTestService(this.http);
 
+    // Log tagging configuration
+    logger.debug('TaggingService initialized', {
+      enabled: this.tagging.isEnabled(),
+      tagName: this.tagging.tagName
+    });
+
     // Track current run context
     this.currentRun = null;
 
@@ -405,6 +421,35 @@ class AppliqationClient {
     this.currentRun = run;
   }
 
+  /**
+   * Delete a test run
+   * @param {string} runId - Run ID to delete
+   * @param {string} reason - Deletion reason (for audit logging)
+   * @returns {Promise<Object>} Deletion result
+   */
+  async deleteRun(runId, reason = 'orphan_cleanup') {
+    try {
+      logger.info('Deleting run...', { runId, reason });
+
+      const result = await this.runMatrix.delete(runId, reason);
+
+      // Clear from current run if it matches
+      if (this.currentRun && this.currentRun.runId === runId) {
+        this.currentRun = null;
+      }
+
+      logger.info('Run deleted successfully', { runId });
+
+      return result;
+    } catch (error) {
+      logger.error('Failed to delete run', {
+        error: error.message,
+        runId
+      });
+      throw error;
+    }
+  }
+
   /**
    * Validate UUID format
    * @param {string} uuid - UUID to validate

package/src/core/HttpClient.js

@@ -254,6 +254,58 @@ class HttpClient {
     }
   }
 
+  /**
+   * Delete a test run
+   * @param {string} runId - Run ID to delete
+   * @param {string} reason - Reason for deletion (for audit logging)
+   * @returns {Promise<Object>} Deletion result
+   */
+  async deleteRun(runId, reason = 'sdk_cleanup') {
+    const maxRetries = 3;
+    const retryDelays = [500, 1000, 2000]; // ms - exponential backoff
+
+    for (let attempt = 0; attempt <= maxRetries; attempt++) {
+      try {
+        logger.debug('Deleting run', { runId, reason, attempt: attempt + 1 });
+
+        const response = await this.delete(`/api/automation/run/${runId}/delete`, {
+          data: { reason }
+        });
+
+        if (attempt > 0) {
+          logger.info('Run deletion succeeded after retry', { runId, attempt: attempt + 1 });
+        }
+
+        return response;
+      } catch (error) {
+        const is404 = error.response?.status === 404;
+        const isLastAttempt = attempt === maxRetries;
+
+        // Only retry on 404 (run not found yet - possible race condition)
+        if (is404 && !isLastAttempt) {
+          const delay = retryDelays[attempt];
+          logger.warn('Run not found, retrying deletion...', {
+            runId,
+            attempt: attempt + 1,
+            maxRetries: maxRetries + 1,
+            retryAfterMs: delay
+          });
+          await new Promise(resolve => setTimeout(resolve, delay));
+          continue;
+        }
+
+        // Final attempt failed or non-404 error - throw
+        logger.error('Failed to delete run', {
+          error: error.message,
+          runId,
+          status: error.response?.status,
+          attempts: attempt + 1
+        });
+        throw error;
+      }
+    }
+  }
+
   /**
    * Set authorization header
    * @param {string} token - Authorization token