@ash-mallick/browserstack-sync 1.1.3 → 1.3.0

package/README.md

@@ -1,51 +1,130 @@
 # @ash-mallick/browserstack-sync
 
-Sync **Playwright** and **Cypress** e2e specs to CSV (TC-001, TC-002, …) and optionally to **BrowserStack Test Management** (one folder per spec, create/update test cases).
+Sync **Playwright** and **Cypress** e2e specs to **BrowserStack Test Management** with **AI-powered test step extraction**.
 
-**🦙 FREE AI-powered test step extraction** using **Ollama** — runs 100% locally, no data sent to cloud!
+**🦙 FREE AI analysis** using **Ollama** — runs 100% locally, no data sent to cloud!
 
 **By Ashutosh Mallick**
 
 ---
 
-## Install
+## 🚀 Quick Start (3 Steps)
 
+### Step 1: Install
 ```bash
 npm install @ash-mallick/browserstack-sync
 ```
 
-Or run without installing: `npx @ash-mallick/browserstack-sync --csv-only`
+### Step 2: Setup Environment
+Create `.env` file in your project root:
+```env
+BROWSERSTACK_USERNAME=your_username
+BROWSERSTACK_ACCESS_KEY=your_access_key
+BROWSERSTACK_PROJECT_ID=PR-XXXX
+```
+
+### Step 3: Onboard Tests to BrowserStack
+```bash
+npx am-browserstack-sync --all
+```
+
+This will:
+- ✅ Generate CSV files with test cases
+- ✅ Create folders in BrowserStack (one per spec file)
+- ✅ Create/update test cases with AI-generated steps
 
 ---
 
-## Run
+## 📊 Sync Test Run Results (CI Pipeline)
 
-From your project root (where your e2e specs live):
+Add this single line to your existing CI pipeline to sync test results:
 
 ```bash
-# Generate CSVs only
-npx am-browserstack-sync --csv-only
+npx am-browserstack-sync-runs
+```
+
+### GitLab CI Example
+
+```yaml
+test:
+  script:
+    # Your existing test command (add JSON reporter)
+    - npx playwright test --reporter=json --output-file=test-results.json
+    # Sync results to BrowserStack (add this line)
+    - npx am-browserstack-sync-runs --run-name="Pipeline #$CI_PIPELINE_ID"
+```
+
+### GitHub Actions Example
+
+```yaml
+- run: npx playwright test --reporter=json --output-file=test-results.json
+- run: npx am-browserstack-sync-runs --run-name="Build #${{ github.run_number }}"
+```
+
+The command will:
+- 📄 Auto-detect your test report file
+- 📊 Parse test results (pass/fail/skip)
+- 🔗 Create a Test Run in BrowserStack
+- ✅ Sync each test result
+- 🔗 Provide a link to view results
+
+---
+
+## Commands Overview
+
+| Command | Purpose |
+|---------|---------|
+| `npx am-browserstack-sync --all` | Onboard tests to BrowserStack (one-time setup) |
+| `npx am-browserstack-sync-runs` | Sync test run results (add to CI pipeline) |
+| `npx am-browserstack-sync --csv-only` | Generate CSVs only (no BrowserStack sync) |
+
+---
+
+## Detailed Usage
 
-# Sync all specs, no prompt (e.g. CI)
+### Onboard Tests (am-browserstack-sync)
+
+```bash
+# Onboard all spec files
 npx am-browserstack-sync --all
 
-# Sync only certain specs
+# Onboard specific specs
 npx am-browserstack-sync --spec=login.spec,checkout.spec
 
-# Disable AI analysis (use regex extraction only)
+# Generate CSVs only (no BrowserStack)
+npx am-browserstack-sync --csv-only
+
+# Disable AI (use regex extraction)
 npx am-browserstack-sync --no-ai
 
-# Use a specific Ollama model
+# Use specific Ollama model
 npx am-browserstack-sync --model=llama3.2
 ```
 
-**Scripts** in `package.json`:
+### Sync Run Results (am-browserstack-sync-runs)
+
+```bash
+# Auto-detect report and sync
+npx am-browserstack-sync-runs
+
+# Specify report file
+npx am-browserstack-sync-runs --report=test-results.json
+
+# Custom run name
+npx am-browserstack-sync-runs --run-name="Nightly Regression"
+
+# JUnit XML format
+npx am-browserstack-sync-runs --report=junit.xml --format=junit
+```
+
+### Scripts in package.json
 
 ```json
 {
   "scripts": {
-    "sync:e2e": "am-browserstack-sync",
-    "sync:e2e-csv": "am-browserstack-sync --csv-only"
+    "test": "playwright test --reporter=json --output-file=test-results.json",
+    "sync:onboard": "am-browserstack-sync --all",
+    "sync:runs": "am-browserstack-sync-runs"
   }
 }
 ```
@@ -167,6 +246,186 @@ Sync pushes your e2e tests into **BrowserStack Test Management** so you can trac
 - Enriches with state (Active), type (Functional), automation (automated), tags (from spec + title).
 - Writes **one CSV per spec** (test_case_id, title, state, case_type, steps, expected_results, jira_issues, automation_status, tags, description, spec_file).
 - Optionally syncs to BrowserStack with description, steps, and tags.
+
+---
+
+## 📊 Test Run Tracking (NEW!)
+
+Run your Playwright tests and **automatically track results on BrowserStack**. Each test run is recorded with pass/fail status, duration, and links back to your test cases.
+
+### Quick Start
+
+```bash
+# Run all tests with BrowserStack tracking
+npx am-browserstack-sync --run
+
+# Run with a custom run name
+npx am-browserstack-sync --run --run-name="Nightly Regression"
+
+# Run specific spec files
+npx am-browserstack-sync --run --spec=login.spec.js,checkout.spec.js
+
+# Run tests matching a pattern
+npx am-browserstack-sync --run --grep="login"
+```
+
+### Setup
+
+1. First, **sync your test cases** to BrowserStack (one-time or when tests change):
+   ```bash
+   npx am-browserstack-sync --all
+   ```
+
+2. Ensure your `.env` has the required credentials:
+   ```env
+   BROWSERSTACK_USERNAME=your_username
+   BROWSERSTACK_ACCESS_KEY=your_access_key
+   BROWSERSTACK_PROJECT_ID=PR-XXXX
+   ```
+
+3. Run tests with tracking:
+   ```bash
+   npx am-browserstack-sync --run
+   ```
+
+### What happens during a test run
+
+1. **Creates a Test Run** in BrowserStack with your specified name
+2. **Maps tests** to existing test cases by title
+3. **Updates results in real-time** as each test passes/fails/skips
+4. **Completes the run** with a summary and link to view results
+
+### Example Output
+
+```
+🚀 Running Playwright tests with BrowserStack tracking...
+   Run Name: Nightly Regression
+
+🔗 BrowserStack Test Management - Initializing...
+   Fetching test cases from BrowserStack...
+   Found 15 test case(s) in project PR-1234
+   Creating test run: "Nightly Regression"...
+   ✓ Test run created: TR-42 (ID: 12345)
+   Adding 15 test case(s) to the run...
+   ✓ Test cases added to run
+   🚀 Starting test execution with BrowserStack tracking
+
+   ✓ [BS] should display login form → passed
+   ✓ [BS] should log in successfully → passed
+   ✗ [BS] should show error for invalid credentials → failed
+
+🔗 BrowserStack Test Management - Completing run...
+
+   ✓ Test run completed: TR-42
+   📊 Results: 2 passed, 1 failed, 0 skipped (3 total)
+   🔗 View in BrowserStack: https://test-management.browserstack.com/projects/PR-1234/runs/12345
+```
+
+### Using as a Playwright Reporter
+
+You can also use the BrowserStack reporter directly in your `playwright.config.js`:
+
+```javascript
+// playwright.config.js
+export default {
+  reporter: [
+    ['list'],
+    ['@ash-mallick/browserstack-sync/reporter', {
+      projectId: 'PR-1234',
+      runName: 'CI Pipeline Run',
+    }],
+  ],
+};
+```
+
+### CI/CD Integration - GitLab
+
+**Sync test results from GitLab CI pipelines to BrowserStack!**
+
+#### Quick Setup
+
+1. Add CI/CD variables in GitLab (Settings > CI/CD > Variables):
+   - `BROWSERSTACK_USERNAME`
+   - `BROWSERSTACK_ACCESS_KEY`
+   - `BROWSERSTACK_PROJECT_ID`
+
+2. Run tests with JSON reporter and sync results:
+
+```yaml
+# .gitlab-ci.yml
+test_and_sync:
+  image: mcr.microsoft.com/playwright:v1.40.0-jammy
+  script:
+    - npm ci
+    - npm install @ash-mallick/browserstack-sync
+    # Run tests with JSON reporter
+    - npx playwright test --reporter=json --output-file=test-results.json || true
+    # Sync results to BrowserStack
+    - npx am-browserstack-sync-runs --run-name="CI Pipeline #$CI_PIPELINE_ID"
+  artifacts:
+    paths:
+      - test-results.json
+```
+
+#### Scheduled Runs (Nightly/Weekly)
+
+```yaml
+nightly_tests:
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "schedule"
+  variables:
+    BROWSERSTACK_RUN_NAME: "Nightly Regression - $CI_COMMIT_SHORT_SHA"
+  script:
+    - npx playwright test --reporter=json --output-file=test-results.json || true
+    - npx am-browserstack-sync-runs
+```
+
+#### CLI Options
+
+```bash
+# Auto-detect report and sync
+npx am-browserstack-sync-runs
+
+# Specify report file
+npx am-browserstack-sync-runs --report=custom-results.json
+
+# Custom run name
+npx am-browserstack-sync-runs --run-name="Nightly Build"
+
+# JUnit XML format
+npx am-browserstack-sync-runs --report=junit.xml --format=junit
+```
+
+### GitHub Actions Example
+
+```yaml
+- name: Run tests and sync to BrowserStack
+  env:
+    BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
+    BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
+    BROWSERSTACK_PROJECT_ID: PR-1234
+  run: |
+    npx playwright test --reporter=json --output-file=test-results.json || true
+    npx am-browserstack-sync-runs --run-name="Build #${{ github.run_number }}"
+```
+
+---
+
+## Programmatic API
+
+```javascript
+import { runSync } from '@ash-mallick/browserstack-sync';
+
+await runSync({
+  cwd: '/path/to/project',
+  csvOnly: true,
+  all: true,
+  spec: ['login.spec'],
+  useAI: true,
+  model: 'llama3.2',
+});
+```
+
 ---
 
 **Author:** Ashutosh Mallick

package/bin/cli.js

@@ -2,31 +2,67 @@
 /**
  * CLI for @ash-mallick/browserstack-sync (am-browserstack-sync).
  * By Ashutosh Mallick. Run from your project root: npx am-browserstack-sync [options]
- * Uses cwd as project root; config from .env, config file, or package.json.
+ * 
+ * This command onboards your Playwright/Cypress tests to BrowserStack:
+ * - Generates CSV files with test cases
+ * - Creates folders in BrowserStack (one per spec file)
+ * - Creates/updates test cases with AI-generated steps
  *
- * Options:
+ * OPTIONS:
  *   --csv-only      Generate CSVs only, do not sync to BrowserStack
  *   --all           Sync all spec files (no interactive prompt)
- *   --spec=name     Sync only these specs (comma-separated). e.g. --spec=login.spec,checkout.spec
- *   --no-ai         Disable AI-powered step analysis (use regex extraction only)
- *   --model=name    Ollama model to use (default: llama3.2). e.g. --model=codellama
+ *   --spec=name     Sync only these specs (comma-separated)
+ *   --no-ai         Disable AI-powered step analysis
+ *   --model=name    Ollama model to use (default: llama3.2)
  *
- * AI Analysis (FREE, Local with Ollama):
- *   Install Ollama from https://ollama.ai, then:
- *     ollama pull llama3.2
- *     ollama serve
+ * EXAMPLES:
+ *   npx am-browserstack-sync --all           # Onboard all tests
+ *   npx am-browserstack-sync --csv-only      # Generate CSVs only
+ *   npx am-browserstack-sync --spec=login    # Onboard specific spec
  *
- *   AI analysis is automatic when Ollama is running! No API key needed.
- *   The AI will analyze your test code and generate human-readable steps like:
- *     - "Navigate to /login page"
- *     - "Enter 'user@test.com' in the Email field"
- *     - "Click the 'Sign In' button"
- *     - "Verify URL matches /dashboard"
+ * To sync test run results, use:
+ *   npx am-browserstack-sync-runs
  */
 
 import { runSync } from '../lib/index.js';
+import 'dotenv/config';
 
 const argv = process.argv.slice(2);
+
+// Show help
+if (argv.includes('--help') || argv.includes('-h')) {
+  console.log(`
+📦 BrowserStack Test Case Sync
+
+Onboard your Playwright/Cypress tests to BrowserStack Test Management.
+
+USAGE:
+  npx am-browserstack-sync [options]
+
+OPTIONS:
+  --csv-only      Generate CSVs only, do not sync to BrowserStack
+  --all           Sync all spec files without prompting
+  --spec=<names>  Sync specific specs (comma-separated)
+  --no-ai         Disable AI step analysis (use regex)
+  --model=<name>  Ollama model (default: llama3.2)
+  --help, -h      Show this help
+
+EXAMPLES:
+  npx am-browserstack-sync --all
+  npx am-browserstack-sync --csv-only
+  npx am-browserstack-sync --spec=login.spec,checkout.spec
+
+TO SYNC TEST RUN RESULTS:
+  npx am-browserstack-sync-runs
+
+ENVIRONMENT VARIABLES:
+  BROWSERSTACK_USERNAME     Your BrowserStack username
+  BROWSERSTACK_ACCESS_KEY   Your BrowserStack access key
+  BROWSERSTACK_PROJECT_ID   Project ID (e.g., PR-1234)
+`);
+  process.exit(0);
+}
+
 const csvOnly = argv.includes('--csv-only');
 const all = argv.includes('--all');
 const noAI = argv.includes('--no-ai');

package/bin/sync-runs.js

@@ -0,0 +1,189 @@
+#!/usr/bin/env node
+/**
+ * Simple CLI to sync test run results to BrowserStack.
+ * Add this to your existing CI pipeline after tests run.
+ * 
+ * Usage:
+ *   npx am-browserstack-sync-runs
+ *   npx am-browserstack-sync-runs --report=custom-results.json
+ *   npx am-browserstack-sync-runs --run-name="Nightly Build #123"
+ * 
+ * Environment Variables (required):
+ *   BROWSERSTACK_USERNAME
+ *   BROWSERSTACK_ACCESS_KEY
+ *   BROWSERSTACK_PROJECT_ID
+ * 
+ * The command will:
+ *   1. Auto-detect Playwright report files (results.json, test-results.json, etc.)
+ *   2. Parse test results
+ *   3. Create a Test Run in BrowserStack
+ *   4. Sync pass/fail status for each test
+ *   5. Show a link to view results
+ */
+
+import 'dotenv/config';
+import fs from 'fs';
+import path from 'path';
+import { syncResultsFromReport } from '../lib/sync-results.js';
+
+const argv = process.argv.slice(2);
+
+// Show help
+if (argv.includes('--help') || argv.includes('-h')) {
+  console.log(`
+📊 BrowserStack Test Run Sync
+
+Sync your Playwright/Cypress test results to BrowserStack Test Management.
+Add this command to your existing CI pipeline after tests complete.
+
+USAGE:
+  npx am-browserstack-sync-runs [options]
+
+OPTIONS:
+  --report=<path>     Path to report file (auto-detects if not specified)
+  --run-name=<name>   Name for the test run (default: "CI Run - <timestamp>")
+  --format=<fmt>      Report format: playwright-json, junit (default: auto)
+  --help, -h          Show this help
+
+ENVIRONMENT VARIABLES (required):
+  BROWSERSTACK_USERNAME     Your BrowserStack username
+  BROWSERSTACK_ACCESS_KEY   Your BrowserStack access key
+  BROWSERSTACK_PROJECT_ID   Your project ID (e.g., PR-1234)
+
+EXAMPLES:
+  # Auto-detect report and sync
+  npx am-browserstack-sync-runs
+
+  # Specify report file
+  npx am-browserstack-sync-runs --report=test-results.json
+
+  # Custom run name (great for CI)
+  npx am-browserstack-sync-runs --run-name="Build #\${CI_PIPELINE_ID}"
+
+  # Sync JUnit XML results
+  npx am-browserstack-sync-runs --report=junit.xml --format=junit
+
+ADD TO YOUR CI PIPELINE:
+  # GitLab CI
+  script:
+    - npm test  # Your existing test command
+    - npx am-browserstack-sync-runs --run-name="Pipeline #\$CI_PIPELINE_ID"
+
+  # GitHub Actions  
+  - run: npm test
+  - run: npx am-browserstack-sync-runs --run-name="Build #\${{ github.run_number }}"
+
+PREREQUISITES:
+  1. First, onboard your tests to BrowserStack:
+     npx am-browserstack-sync --all
+
+  2. Then add run sync to your pipeline:
+     npx am-browserstack-sync-runs
+`);
+  process.exit(0);
+}
+
+// Auto-detect report file
+function findReportFile() {
+  const cwd = process.cwd();
+  const possibleFiles = [
+    'test-results.json',
+    'results.json',
+    'playwright-report/results.json',
+    'playwright-report.json',
+    'report.json',
+    'junit.xml',
+    'test-results.xml',
+  ];
+  
+  for (const file of possibleFiles) {
+    const fullPath = path.join(cwd, file);
+    if (fs.existsSync(fullPath)) {
+      return fullPath;
+    }
+  }
+  
+  // Check for playwright-report directory
+  const htmlReportDir = path.join(cwd, 'playwright-report');
+  if (fs.existsSync(htmlReportDir) && fs.statSync(htmlReportDir).isDirectory()) {
+    return htmlReportDir;
+  }
+  
+  return null;
+}
+
+// Auto-detect format from file
+function detectFormat(reportPath) {
+  if (reportPath.endsWith('.xml')) {
+    return 'junit';
+  }
+  return 'playwright-json';
+}
+
+// Parse arguments
+const reportArg = argv.find((a) => a.startsWith('--report='));
+const formatArg = argv.find((a) => a.startsWith('--format='));
+const runNameArg = argv.find((a) => a.startsWith('--run-name='));
+
+let reportPath = reportArg ? reportArg.slice('--report='.length).trim() : null;
+let format = formatArg ? formatArg.slice('--format='.length).trim() : null;
+const runName = runNameArg 
+  ? runNameArg.slice('--run-name='.length).trim() 
+  : process.env.BROWSERSTACK_RUN_NAME || `CI Run - ${new Date().toISOString().slice(0, 16).replace('T', ' ')}`;
+
+// Auto-detect report if not specified
+if (!reportPath) {
+  reportPath = findReportFile();
+  if (!reportPath) {
+    console.error(`
+❌ No test report found!
+
+Make sure your test framework generates a report file:
+
+  # Playwright - add to your test command:
+  npx playwright test --reporter=json --output-file=test-results.json
+
+  # Or specify the report path:
+  npx am-browserstack-sync-runs --report=path/to/results.json
+`);
+    process.exit(1);
+  }
+  console.log(`📄 Auto-detected report: ${reportPath}`);
+}
+
+// Auto-detect format if not specified
+if (!format) {
+  format = detectFormat(reportPath);
+}
+
+// Check required env vars
+const missing = [];
+if (!process.env.BROWSERSTACK_USERNAME && !process.env.BROWSERSTACK_API_TOKEN) {
+  missing.push('BROWSERSTACK_USERNAME');
+}
+if (!process.env.BROWSERSTACK_ACCESS_KEY && !process.env.BROWSERSTACK_API_TOKEN) {
+  missing.push('BROWSERSTACK_ACCESS_KEY');
+}
+if (!process.env.BROWSERSTACK_PROJECT_ID) {
+  missing.push('BROWSERSTACK_PROJECT_ID');
+}
+
+if (missing.length > 0) {
+  console.error(`
+❌ Missing required environment variables:
+   ${missing.join('\n   ')}
+
+Set these in your CI/CD settings or .env file.
+`);
+  process.exit(1);
+}
+
+// Run sync
+syncResultsFromReport({
+  reportPath,
+  format,
+  runName,
+}).catch((err) => {
+  console.error('\n❌ Error:', err.message);
+  process.exit(1);
+});

package/lib/ai-analyzer.js

@@ -14,74 +14,78 @@ const OLLAMA_HOST = process.env.OLLAMA_HOST || 'http://localhost:11434';
 /**
  * System prompt for the AI to understand what we want.
  */
-const SYSTEM_PROMPT = `You are a QA expert creating detailed manual test steps from automated test code. Your goal is to write steps so clear and detailed that ANY tester can execute the test manually by just reading them.
-
-IMPORTANT: Output ONLY a valid JSON array. No markdown, no explanation, no extra text.
-
-Each step must have:
-- "step": A detailed, numbered action description. Include:
-  - Exact UI element to interact with (button name, field label, section title)
-  - Exact text/values to enter or look for
-  - Location hints (e.g., "in the header", "in the Solutions section", "at the top of the page")
-- "result": The specific expected outcome that can be visually verified. Include:
-  - Exact text that should be displayed
-  - UI state changes (visible, enabled, selected, etc.)
-  - URL changes if applicable
-
-RULES FOR DETAILED STEPS:
-1. NAVIGATION: "Open the browser and navigate to [exact URL]. Wait for the page to fully load."
-2. VISIBILITY CHECK: "Locate the [section name] section on the page. Verify it is visible and contains the text '[exact text]'."
-3. ELEMENT VERIFICATION: "In the [section name], find the [element type] with text '[text]'. Verify it is displayed."
-4. LINK VALIDATION: "Find the '[link text]' link in the [location]. Verify it has href pointing to '[URL pattern]'."
-5. BUTTON CHECK: "Locate the '[button text]' button in the [section]. Verify it is visible and clickable."
-6. CARD/TILE VALIDATION: "In the [section], locate the first card/tile. Verify it displays: title, image, and description."
-7. SCROLL ACTION: "Scroll down to the [section name] section to bring it into view."
-
-EXAMPLE OUTPUT:
+const SYSTEM_PROMPT = `You are a test automation expert. Your job is to analyze Playwright or Cypress test code and extract human-readable test steps.
+
+CRITICAL RULES:
+1. Generate steps ONLY for what is explicitly in the test code provided
+2. DO NOT add steps for things not in the code
+3. DO NOT hallucinate or assume content - use ONLY text/values from the code
+4. Each expect() or assertion = one verification step
+5. Output ONLY valid JSON array, nothing else
+
+STEP FORMAT:
+{
+  "step": "Numbered action description with exact element and location",
+  "result": "Expected outcome using exact text from the code"
+}
+
+MAPPING CODE TO STEPS:
+
+1. getByTestId('section-name') → "Locate the section with test ID 'section-name'"
+2. locator('.class-name') → "Find the element with class 'class-name'"
+3. getByRole('link', { name: /text/i }) → "Find the link labeled 'text'"
+4. expect(element).toBeVisible() → Result: "The element is visible"
+5. expect(element).toContainText('exact text') → Result: "The element displays 'exact text'"
+6. expect(element).toHaveText('exact text') → Result: "The element shows exactly 'exact text'"
+7. expect(link).toHaveAttribute('href', '/path') → Result: "The link href is '/path'"
+8. scrollIntoViewIfNeeded() → "Scroll to bring the element into view"
+
+EXAMPLE - For this code:
+  const scope = page.getByTestId("success-stories");
+  await expect(scope.locator(".band__headline")).toHaveText("Success stories");
+  
+OUTPUT:
 [
-  {"step": "1. Open the browser and navigate to the application homepage (base URL + '/'). Wait for the page to fully load.", "result": "The homepage loads successfully. The masthead banner is visible at the top of the page."},
-  {"step": "2. Locate the masthead section at the top of the page. Verify the main heading is displayed.", "result": "The heading 'Accelerate AI with an open ecosystem' is visible in the masthead."},
-  {"step": "3. In the masthead section, find the 'Explore our AI ecosystem' call-to-action link.", "result": "The 'Explore our AI ecosystem' link is visible and has an href containing '/ai'."},
-  {"step": "4. Scroll down to the 'Solutions' section. Verify the section headline is displayed.", "result": "The Solutions section is visible with headline 'Find the solution you're looking for, in less time'."},
-  {"step": "5. In the Solutions section, locate the product slider/carousel. Verify the first product card is displayed.", "result": "The first product card shows: product title, product image, and product description."}
+  {"step": "1. Locate the section with test ID 'success-stories' on the page.", "result": "The success-stories section is found."},
+  {"step": "2. In the success-stories section, find the element with class 'band__headline'. Verify its text content.", "result": "The headline displays exactly 'Success stories'."}
 ]
 
-IMPORTANT: 
-- Number each step sequentially (1, 2, 3...)
-- Be specific about WHAT to look for and WHERE
-- Include exact text strings from the code in quotes
-- Make expected results measurable and verifiable
-- Output ONLY the JSON array, nothing else`;
+REMEMBER:
+- ONLY use text/values that appear in the test code
+- DO NOT make up text or values
+- Number steps 1, 2, 3...
+- Be precise and specific`;
 
 /**
  * Build the user prompt for AI analysis.
  */
 function buildUserPrompt(testTitle, testCode) {
-  return `Create detailed manual test steps from this automated test. A QA tester should be able to execute this test manually by reading your steps.
+  return `Convert this automated test into manual test steps.
 
-TEST NAME: "${testTitle}"
+TEST: "${testTitle}"
 
-AUTOMATED TEST CODE:
-\`\`\`javascript
+CODE:
 ${testCode}
-\`\`\`
-
-ANALYZE THE CODE CAREFULLY:
-1. Look at page.goto() calls to identify which page/URL to navigate to
-2. Look at getByTestId(), locator(), getByRole(), getByLabel() to identify UI elements
-3. Look at expect() assertions to understand what needs to be verified
-4. Look at .toContainText(), .toHaveText(), .toBeVisible() for expected text/state
-5. Look at .toHaveAttribute('href', ...) for link URL validations
-6. Identify ALL assertions and convert them to expected results
-
-CREATE COMPREHENSIVE STEPS:
-- Include EVERY action from the test code
-- Include EVERY assertion/verification from the test code
-- Use exact text strings from .toContainText() and .toHaveText() in your expected results
-- Number steps sequentially (1, 2, 3...)
-- Make each step detailed enough for a manual tester
-
-OUTPUT: Return ONLY a valid JSON array, no other text.`;
+
+INSTRUCTIONS:
+1. Create one step for EACH expect() assertion in the code
+2. Use ONLY the exact text strings that appear in the code (in quotes or regex)
+3. For each getByTestId(), locator(), or getByRole() - describe THAT element
+4. For each .toContainText('X') or .toHaveText('X') - the expected result is "displays 'X'"
+5. For each .toHaveAttribute('href', 'Y') - the expected result is "href is 'Y'"
+6. DO NOT add extra text or values that are not in the code
+
+Example mapping:
+- expect(page.getByTestId("masthead")).toBeVisible() 
+  → Step: "Locate the element with test ID 'masthead'." Result: "The masthead element is visible."
+
+- expect(el).toContainText("Hello World")
+  → Step: "Verify the element's text content." Result: "The element contains 'Hello World'."
+
+- expect(link).toHaveAttribute("href", "/contact")
+  → Step: "Check the link's href attribute." Result: "The link href is '/contact'."
+
+OUTPUT: Return ONLY a JSON array of {step, result} objects. No other text.`;
 }
 
 /**

package/lib/browserstack.js

@@ -112,32 +112,91 @@ export async function updateTestCase(projectId, testCaseId, testCase, auth) {
 
 /** Match existing BS test case to our local case: by title or by our TC-xxx tag */
 function findExisting(existingList, localCase) {
-  const byTitle = existingList.find((t) => (t.title || '').trim() === (localCase.title || '').trim());
+  const byTitle = existingList.find((t) => (t.name || t.title || '').trim() === (localCase.title || '').trim());
   if (byTitle) return byTitle;
   const byTag = existingList.find((t) => (t.tags || []).includes(localCase.id));
   return byTag;
 }
 
+/**
+ * Compare local test case with existing BrowserStack test case.
+ * Returns true if content has changed and update is needed.
+ */
+function hasContentChanged(existing, localCase) {
+  // Compare steps - this is the main content that changes
+  const existingSteps = existing.test_case_steps || [];
+  const localSteps = Array.isArray(localCase.steps) ? localCase.steps : [];
+  
+  // Different number of steps = changed
+  if (existingSteps.length !== localSteps.length) {
+    return true;
+  }
+  
+  // Compare each step
+  for (let i = 0; i < localSteps.length; i++) {
+    const existingStep = existingSteps[i] || {};
+    const localStep = localSteps[i] || {};
+    
+    const existingStepText = (existingStep.step || '').trim();
+    const localStepText = (localStep.step || '').trim();
+    const existingResult = (existingStep.result || '').trim();
+    const localResult = (localStep.result || '').trim();
+    
+    if (existingStepText !== localStepText || existingResult !== localResult) {
+      return true;
+    }
+  }
+  
+  // Compare title
+  const existingTitle = (existing.name || existing.title || '').trim();
+  const localTitle = (localCase.title || '').trim();
+  if (existingTitle !== localTitle) {
+    return true;
+  }
+  
+  // No changes detected
+  return false;
+}
+
 export async function syncToBrowserStack(specsMap, projectId, auth) {
   console.log('\nSyncing to BrowserStack project', projectId);
+  
+  let totalCreated = 0;
+  let totalUpdated = 0;
+  let totalSkipped = 0;
+  
   for (const [baseName, { specFile, cases }] of specsMap) {
     const folderName = baseName;
     const folderId = await ensureFolder(projectId, folderName, null, auth);
     const existing = await getTestCasesInFolder(projectId, folderId, auth);
-    console.log('Folder:', folderName, '(id:', folderId, ')');
+    console.log('Folder:', folderName, '(id:', folderId, ') -', cases.length, 'test(s)');
+    
     for (const tc of cases) {
       try {
         const found = findExisting(existing, tc);
         if (found) {
-          await updateTestCase(projectId, found.identifier, tc, auth);
-          console.log('  Updated', tc.id, tc.title, '->', found.identifier);
+          // Check if content has actually changed before updating
+          if (hasContentChanged(found, tc)) {
+            await updateTestCase(projectId, found.identifier, tc, auth);
+            console.log('  ✓ Updated', tc.id, tc.title, '->', found.identifier);
+            totalUpdated++;
+          } else {
+            console.log('  ○ Skipped', tc.id, tc.title, '(no changes)');
+            totalSkipped++;
+          }
         } else {
           const id = await createTestCase(projectId, folderId, tc, auth);
-          console.log('  Created', tc.id, tc.title, '->', id);
+          console.log('  + Created', tc.id, tc.title, '->', id);
+          totalCreated++;
         }
       } catch (e) {
-        console.error('  Failed', tc.id, tc.title, e.message);
+        console.error('  ✗ Failed', tc.id, tc.title, e.message);
       }
     }
   }
+  
+  console.log('\nSync summary:');
+  console.log(`  Created: ${totalCreated} test case(s)`);
+  console.log(`  Updated: ${totalUpdated} test case(s)`);
+  console.log(`  Skipped: ${totalSkipped} test case(s) (no changes)`);
 }

package/lib/enrich.js

@@ -40,6 +40,46 @@ function inferTags(specBaseName, title, isCypress) {
   return tags;
 }
 
+/**
+ * Count the number of expect() assertions in test code.
+ */
+function countAssertions(code) {
+  const expectMatches = code.match(/expect\s*\(/g) || [];
+  return expectMatches.length;
+}
+
+/**
+ * Merge AI steps with regex steps to ensure completeness.
+ * If AI generated fewer steps than assertions, supplement with unique regex steps.
+ */
+function mergeStepsForCompleteness(aiSteps, regexSteps, assertionCount) {
+  if (!aiSteps || aiSteps.length === 0) {
+    return regexSteps;
+  }
+  
+  // If AI generated enough steps, use AI steps
+  if (aiSteps.length >= assertionCount * 0.7) { // 70% coverage threshold
+    return aiSteps;
+  }
+  
+  // If AI generated too few steps, supplement with unique regex steps
+  const aiStepTexts = new Set(aiSteps.map(s => s.step.toLowerCase()));
+  const supplementalSteps = regexSteps.filter(rs => {
+    // Only add regex steps that aren't similar to existing AI steps
+    const rsLower = rs.step.toLowerCase();
+    return !Array.from(aiStepTexts).some(aiText => 
+      aiText.includes(rsLower.slice(0, 30)) || rsLower.includes(aiText.slice(0, 30))
+    );
+  });
+  
+  // Combine and renumber
+  const combined = [...aiSteps, ...supplementalSteps];
+  return combined.map((step, idx) => ({
+    step: step.step.replace(/^\d+\.\s*/, `${idx + 1}. `),
+    result: step.result,
+  }));
+}
+
 /**
  * Enrich a single case with state, case_type, steps, expected_results, jira_issues, automation_status, tags, description.
  * @param {Object} case_ - Test case with id, title, code
@@ -54,13 +94,19 @@ export function enrichCase(case_, specBaseName, isCypress = false, aiSteps = nul
   const fw = isCypress ? FRAMEWORK.cypress : FRAMEWORK.playwright;
   const tags = inferTags(specBaseName, title, isCypress);
 
-  // Determine steps: AI steps > regex-extracted steps > default
+  // Count assertions in code for completeness check
+  const assertionCount = code ? countAssertions(code) : 0;
+  
+  // Get regex-based steps as fallback/supplement
+  const regexSteps = code ? extractStepsWithRegex(code, isCypress) : [];
+
+  // Determine steps: merge AI + regex for completeness, or use regex only
   let steps;
   if (aiSteps && aiSteps.length > 0) {
-    steps = aiSteps;
-  } else if (code) {
-    // Try regex-based extraction from code
-    steps = extractStepsWithRegex(code, isCypress);
+    // Merge AI steps with regex steps to ensure we don't miss assertions
+    steps = mergeStepsForCompleteness(aiSteps, regexSteps, assertionCount);
+  } else if (regexSteps.length > 0) {
+    steps = regexSteps;
   } else {
     // Fallback to generic step
     steps = [{ step: fw.step, result: DEFAULT_EXPECTED_RESULT }];

package/lib/sync-results.js

@@ -0,0 +1,329 @@
+/**
+ * Sync test results from CI pipeline to BrowserStack Test Management.
+ * 
+ * This module reads test results from Playwright JSON report or JUnit XML
+ * and syncs them to BrowserStack, creating a Test Run with pass/fail status.
+ * 
+ * Usage:
+ *   npx am-browserstack-sync --sync-results --report=playwright-report.json
+ *   npx am-browserstack-sync --sync-results --report=junit.xml --format=junit
+ */
+
+import fs from 'fs';
+import path from 'path';
+import {
+  createTestRun,
+  addTestCasesToRun,
+  updateTestResult,
+  completeTestRun,
+  getAllTestCases,
+  findTestCaseByTitle,
+} from './browserstack.js';
+
+/**
+ * Parse Playwright JSON report format.
+ * @param {string} reportPath - Path to the JSON report file
+ * @returns {Array<{title: string, status: string, duration: number, error?: string}>}
+ */
+function parsePlaywrightJson(reportPath) {
+  const report = JSON.parse(fs.readFileSync(reportPath, 'utf-8'));
+  const results = [];
+  
+  // Playwright JSON report structure: { suites: [...], stats: {...} }
+  function extractTests(suites) {
+    for (const suite of suites || []) {
+      for (const spec of suite.specs || []) {
+        for (const test of spec.tests || []) {
+          // Get the result from the test results array
+          const result = test.results?.[0] || {};
+          results.push({
+            title: spec.title || test.title,
+            fullTitle: `${suite.title} > ${spec.title}`,
+            status: mapPlaywrightStatus(test.status || result.status),
+            duration: result.duration || 0,
+            error: result.error?.message || result.errors?.[0]?.message || '',
+            file: suite.file || spec.file,
+          });
+        }
+      }
+      
+      // Recursively process nested suites
+      if (suite.suites) {
+        extractTests(suite.suites);
+      }
+    }
+  }
+  
+  extractTests(report.suites);
+  return results;
+}
+
+/**
+ * Parse JUnit XML report format.
+ * @param {string} reportPath - Path to the JUnit XML file
+ * @returns {Array<{title: string, status: string, duration: number, error?: string}>}
+ */
+function parseJunitXml(reportPath) {
+  const xml = fs.readFileSync(reportPath, 'utf-8');
+  const results = [];
+  
+  // Simple regex-based XML parsing for JUnit format
+  // <testcase name="..." classname="..." time="...">
+  //   <failure message="...">...</failure>
+  //   <skipped/>
+  // </testcase>
+  
+  const testcaseRegex = /<testcase\s+[^>]*name="([^"]+)"[^>]*time="([^"]+)"[^>]*>([\s\S]*?)<\/testcase>/gi;
+  const failureRegex = /<failure[^>]*message="([^"]*)"[^>]*>/i;
+  const skippedRegex = /<skipped\s*\/?>/i;
+  const errorRegex = /<error[^>]*message="([^"]*)"[^>]*>/i;
+  
+  let match;
+  while ((match = testcaseRegex.exec(xml)) !== null) {
+    const [, name, time, content] = match;
+    
+    let status = 'passed';
+    let error = '';
+    
+    if (skippedRegex.test(content)) {
+      status = 'skipped';
+    } else if (failureRegex.test(content)) {
+      status = 'failed';
+      const failMatch = content.match(failureRegex);
+      error = failMatch?.[1] || 'Test failed';
+    } else if (errorRegex.test(content)) {
+      status = 'failed';
+      const errMatch = content.match(errorRegex);
+      error = errMatch?.[1] || 'Test error';
+    }
+    
+    results.push({
+      title: name,
+      status,
+      duration: Math.round(parseFloat(time) * 1000), // Convert seconds to ms
+      error,
+    });
+  }
+  
+  return results;
+}
+
+/**
+ * Parse Playwright HTML report's index.html or results.json
+ */
+function parsePlaywrightHtmlReport(reportDir) {
+  // Try to find the JSON data in the HTML report directory
+  const possiblePaths = [
+    path.join(reportDir, 'report.json'),
+    path.join(reportDir, 'data', 'report.json'),
+    path.join(reportDir, 'results.json'),
+  ];
+  
+  for (const p of possiblePaths) {
+    if (fs.existsSync(p)) {
+      return parsePlaywrightJson(p);
+    }
+  }
+  
+  throw new Error(`Could not find JSON data in Playwright HTML report directory: ${reportDir}`);
+}
+
+/**
+ * Map Playwright status to BrowserStack status.
+ */
+function mapPlaywrightStatus(status) {
+  const map = {
+    passed: 'passed',
+    failed: 'failed',
+    timedOut: 'failed',
+    skipped: 'skipped',
+    interrupted: 'blocked',
+    expected: 'passed',
+    unexpected: 'failed',
+    flaky: 'retest',
+  };
+  return map[status] || status;
+}
+
+/**
+ * Load authentication from environment.
+ */
+function loadAuth() {
+  const username = process.env.BROWSERSTACK_USERNAME;
+  const accessKey = process.env.BROWSERSTACK_ACCESS_KEY;
+  const apiToken = process.env.BROWSERSTACK_API_TOKEN;
+  
+  if (apiToken) {
+    return { apiToken };
+  }
+  
+  if (username && accessKey) {
+    return { username, accessKey };
+  }
+  
+  return null;
+}
+
+/**
+ * Sync test results from a report file to BrowserStack.
+ * 
+ * @param {Object} options
+ * @param {string} options.reportPath - Path to the report file or directory
+ * @param {string} options.format - Report format: 'playwright-json', 'junit', 'playwright-html'
+ * @param {string} options.projectId - BrowserStack project ID
+ * @param {string} options.runName - Name for the test run
+ * @param {string} options.runDescription - Description for the test run
+ */
+export async function syncResultsFromReport(options = {}) {
+  const {
+    reportPath,
+    format = 'playwright-json',
+    projectId = process.env.BROWSERSTACK_PROJECT_ID,
+    runName = process.env.BROWSERSTACK_RUN_NAME || `CI Run - ${new Date().toISOString().slice(0, 16).replace('T', ' ')}`,
+    runDescription = process.env.BROWSERSTACK_RUN_DESCRIPTION || 'Test results synced from CI pipeline',
+  } = options;
+  
+  // Validate inputs
+  if (!reportPath) {
+    throw new Error('Report path is required. Use --report=<path>');
+  }
+  
+  if (!projectId) {
+    throw new Error('BROWSERSTACK_PROJECT_ID is required');
+  }
+  
+  const auth = loadAuth();
+  if (!auth) {
+    throw new Error('BrowserStack credentials not found. Set BROWSERSTACK_USERNAME/ACCESS_KEY or API_TOKEN');
+  }
+  
+  // Check if report exists
+  if (!fs.existsSync(reportPath)) {
+    throw new Error(`Report file not found: ${reportPath}`);
+  }
+  
+  console.log('\n📊 BrowserStack Results Sync');
+  console.log('   Report:', reportPath);
+  console.log('   Format:', format);
+  console.log('   Project:', projectId);
+  console.log('   Run Name:', runName);
+  
+  // Parse the report based on format
+  console.log('\n   Parsing test results...');
+  let testResults;
+  
+  const stat = fs.statSync(reportPath);
+  if (stat.isDirectory()) {
+    // Assume it's a Playwright HTML report directory
+    testResults = parsePlaywrightHtmlReport(reportPath);
+  } else if (format === 'junit' || reportPath.endsWith('.xml')) {
+    testResults = parseJunitXml(reportPath);
+  } else {
+    testResults = parsePlaywrightJson(reportPath);
+  }
+  
+  console.log(`   Found ${testResults.length} test result(s)`);
+  
+  if (testResults.length === 0) {
+    console.log('\n   ⚠️ No test results found in report');
+    return;
+  }
+  
+  // Fetch all test cases from BrowserStack
+  console.log('\n   Fetching test cases from BrowserStack...');
+  const allBsTestCases = await getAllTestCases(projectId, auth);
+  console.log(`   Found ${allBsTestCases.length} test case(s) in project`);
+  
+  // Map results to BrowserStack test cases
+  const mappedResults = [];
+  const unmappedResults = [];
+  
+  for (const result of testResults) {
+    const bsTestCase = findTestCaseByTitle(allBsTestCases, result.title);
+    if (bsTestCase) {
+      mappedResults.push({ ...result, bsTestCase });
+    } else {
+      unmappedResults.push(result);
+    }
+  }
+  
+  console.log(`   Mapped: ${mappedResults.length} test(s)`);
+  if (unmappedResults.length > 0) {
+    console.log(`   Unmapped: ${unmappedResults.length} test(s) (not found in BrowserStack)`);
+  }
+  
+  if (mappedResults.length === 0) {
+    console.log('\n   ⚠️ No tests matched. Make sure tests are synced first:');
+    console.log('      npx am-browserstack-sync --all');
+    return;
+  }
+  
+  // Create test run
+  console.log(`\n   Creating test run: "${runName}"...`);
+  const run = await createTestRun(projectId, runName, runDescription, auth);
+  const runId = run.runId;
+  const runIdentifier = run.identifier;
+  console.log(`   ✓ Test run created: ${runIdentifier}`);
+  
+  // Add test cases to run
+  const testCaseIds = mappedResults.map(r => r.bsTestCase.identifier);
+  console.log(`   Adding ${testCaseIds.length} test case(s) to run...`);
+  await addTestCasesToRun(projectId, runId, testCaseIds, auth);
+  
+  // Update results
+  console.log('\n   Syncing results:');
+  const stats = { passed: 0, failed: 0, skipped: 0 };
+  
+  for (const result of mappedResults) {
+    try {
+      await updateTestResult(
+        projectId,
+        runId,
+        result.bsTestCase.identifier,
+        result.status,
+        {
+          duration: result.duration,
+          comment: result.error || '',
+        },
+        auth
+      );
+      
+      const icon = result.status === 'passed' ? '✓' : result.status === 'failed' ? '✗' : '○';
+      console.log(`   ${icon} ${result.title} → ${result.status}`);
+      
+      if (result.status === 'passed') stats.passed++;
+      else if (result.status === 'failed') stats.failed++;
+      else stats.skipped++;
+    } catch (error) {
+      console.error(`   ✗ Failed to update ${result.title}: ${error.message}`);
+    }
+  }
+  
+  // Complete the run
+  console.log('\n   Completing test run...');
+  await completeTestRun(projectId, runId, auth);
+  
+  const total = stats.passed + stats.failed + stats.skipped;
+  console.log(`\n   ✓ Test run completed: ${runIdentifier}`);
+  console.log(`   📊 Results: ${stats.passed} passed, ${stats.failed} failed, ${stats.skipped} skipped (${total} total)`);
+  console.log(`   🔗 View: https://test-management.browserstack.com/projects/${projectId}/runs/${runId}`);
+  
+  // Show unmapped tests
+  if (unmappedResults.length > 0) {
+    console.log('\n   ⚠️ Tests not found in BrowserStack (sync them first):');
+    unmappedResults.slice(0, 5).forEach(r => console.log(`      - ${r.title}`));
+    if (unmappedResults.length > 5) {
+      console.log(`      ... and ${unmappedResults.length - 5} more`);
+    }
+  }
+  
+  return {
+    runId,
+    runIdentifier,
+    stats,
+    mappedCount: mappedResults.length,
+    unmappedCount: unmappedResults.length,
+  };
+}
+
+export { parsePlaywrightJson, parseJunitXml };

package/package.json

@@ -1,12 +1,13 @@
 {
   "name": "@ash-mallick/browserstack-sync",
-  "version": "1.1.3",
-  "description": "Sync Playwright & Cypress e2e specs to CSV and BrowserStack Test Management with FREE AI-powered test step extraction using Ollama (local)",
+  "version": "1.3.0",
+  "description": "Sync Playwright & Cypress e2e specs to CSV and BrowserStack Test Management with FREE AI-powered test step extraction using Ollama (local).",
   "author": "Ashutosh Mallick",
   "type": "module",
   "main": "lib/index.js",
   "bin": {
-    "am-browserstack-sync": "bin/cli.js"
+    "am-browserstack-sync": "bin/cli.js",
+    "am-browserstack-sync-runs": "bin/sync-runs.js"
   },
   "files": [
     "bin",
@@ -14,10 +15,9 @@
   ],
   "scripts": {
     "test": "playwright test",
-    "test:e2e": "playwright test",
     "sync": "node bin/cli.js",
-    "sync:csv": "node bin/cli.js",
     "sync:csv-only": "node bin/cli.js --csv-only",
+    "sync:runs": "node bin/sync-runs.js",
     "prepublishOnly": "node bin/cli.js --csv-only",
     "publish:public": "npm publish --access public"
   },
@@ -28,9 +28,11 @@
     "e2e",
     "test-management",
     "sync",
+    "ci-cd",
+    "gitlab",
+    "github-actions",
     "ai",
     "ollama",
-    "llama",
     "test-steps",
     "free",
     "local",
@@ -46,10 +48,8 @@
     "ollama": "^0.6.3"
   },
   "devDependencies": {
-    "@playwright/test": "^1.49.0",
-    "playwright": "^1.49.0"
+    "@playwright/test": "^1.49.0"
   },
-  "peerDependenciesMeta": {},
   "engines": {
     "node": ">=18"
   }