@msalaam/xray-qe-toolkit 1.1.0 → 1.2.0

package/README.md

@@ -31,6 +31,7 @@
 - [Jira/Xray Project Setup](#jiraxray-project-setup)
 - [QE Workflow](#qe-workflow)
 - [Building Regression Packs](#building-regression-packs)
+- [Working with Existing Xray Tests](#working-with-existing-xray-tests)
 - [Idempotent Push](#idempotent-push)
 - [Programmatic API](#programmatic-api)
 - [Troubleshooting](#troubleshooting)
@@ -46,8 +47,9 @@
 3. **Idempotent push** — updates existing tests, creates new ones, skips duplicates
 4. **Postman collection generation** from test definitions
 5. **CI pipeline template** for Azure DevOps (Newman + Xray import)
-6. **Modular architecture** — every function is importable for programmatic use
-7. **AI-ready scaffolds** — optional AI assistance for test generation (manual workflow fully supported)
+6. **Playwright integration** — import Playwright JSON results with automatic test mapping
+7. **Modular architecture** — every function is importable for programmatic use
+8. **AI-ready scaffolds** — optional AI assistance for test generation (manual workflow fully supported)
 
 ### QE Review Gate Philosophy
 
@@ -242,7 +244,7 @@ All commands support these global options:
 
 ---
 
-### `xray-qe init`
+### `xqt init`
 
 Scaffold a new project with starter templates.
 
@@ -261,7 +263,7 @@ Existing files are **never overwritten** — the command skips them with a warni
 
 ---
 
-### `xray-qe gen-tests`
+### `xqt gen-tests`
 
 Generate test cases from `knowledge/` folder documentation (AI-assisted or manual guidance).
 
@@ -300,7 +302,7 @@ Existing files are **never overwritten** — the command skips them with a warni
 
 ---
 
-### `xray-qe edit-json`
+### `xqt edit-json`
 
 Launch the browser-based QE review gate editor.
 
@@ -325,7 +327,7 @@ npx xqt edit-json --port 3000
 
 ---
 
-### `xray-qe push-tests`
+### `xqt push-tests`
 
 Push or update tests in Xray Cloud from `tests.json`.
 
@@ -354,7 +356,7 @@ npx xqt push-tests --skip-exec
 
 ---
 
-### `xray-qe gen-postman`
+### `xqt gen-postman`
 
 Generate a Postman Collection v2.1 JSON from `tests.json` (works with or without AI).
 
@@ -398,7 +400,7 @@ npx xqt gen-postman --knowledge ./docs
 
 ---
 
-### `xray-qe create-execution`
+### `xqt create-execution`
 
 Create a standalone Test Execution issue in JIRA.
 
@@ -415,22 +417,58 @@ npx xqt create-execution --summary "Feature XYZ" --issue QE-123
 
 ---
 
-### `xray-qe import-results`
+### `xqt import-results`
 
-Import JUnit/Newman XML results into Xray Cloud. Designed for CI — no human interaction.
+Import test results into Xray Cloud. Supports JUnit XML and Playwright JSON formats. Designed for CI — no human interaction.
 
+**JUnit XML (JUnit, Newman, etc.):**
 ```bash
 npx xqt import-results --file results.xml --testExecKey QE-123
 ```
 
-| Option              | Description                                | Required |
-|---------------------|--------------------------------------------|----------|
-| `--file <path>`     | Path to the JUnit/Newman results XML file  | Yes      |
-| `--testExecKey <key>` | Test Execution to associate results with | Yes      |
+**Playwright JSON:**
+```bash
+# Generate JSON report in Playwright
+npx playwright test --reporter=json > playwright-results.json
+
+# Import to Xray (links to existing tests or creates new ones)
+npx xqt import-results --file playwright-results.json --testExecKey QE-123
+
+# Create new Test Execution automatically
+npx xqt import-results --file playwright-results.json --summary "Regression Suite"
+```
+
+| Option                      | Description                                       | Required |
+|-----------------------------|---------------------------------------------------|----------|
+| `--file <path>`             | Path to results file (.xml or .json)             | Yes      |
+| `--testExecKey <key>`       | Test Execution to link results to                | No*      |
+| `--summary <text>`          | Summary for new Test Execution (if no testExecKey) | No    |
+| `--description <text>`      | Description for new Test Execution               | No       |
+
+\* If `--testExecKey` is omitted, a new Test Execution will be created automatically.
+
+**Mapping Playwright Tests to Xray:**
+
+To link Playwright test results to existing Xray test cases, use annotations or include test keys in titles:
+
+```typescript
+// Option 1: Using annotations (recommended)
+test('User login flow', async ({ page }) => {
+  test.info().annotations.push({ type: 'xray', description: 'PROJ-123' });
+  // ... test code
+});
+
+// Option 2: Include test key in title
+test('[PROJ-456] User registration validates email', async ({ page }) => {
+  // ... test code
+});
+```
+
+If no test key is found, a new test will be created in Xray with the test title as the summary.
 
 ---
 
-### `xray-qe gen-pipeline`
+### `xqt gen-pipeline`
 
 Generate an Azure Pipelines YAML template.
 
@@ -453,7 +491,7 @@ The generated pipeline:
 
 ---
 
-### `xray-qe mcp-server`
+### `xqt mcp-server`
 
 Start a Model Context Protocol server for GitHub Copilot agent-mode integration.
 
@@ -682,6 +720,34 @@ Use this checklist to align a new team's board and Xray configuration with the t
 └─────────────────────────────────────────────────────────────────────────┘
 ```
 
+### Playwright Integration Workflow
+
+For teams using Playwright for API or E2E testing:
+
+```bash
+# 1. Write Playwright tests with Xray annotations
+# tests/login.spec.ts
+test('User login with valid credentials', async ({ page }) => {
+  test.info().annotations.push({ type: 'xray', description: 'PROJ-123' });
+  // ... test implementation
+});
+
+# 2. Run tests and generate JSON report
+npx playwright test --reporter=json > playwright-results.json
+
+# 3. Import results to Xray (links to existing tests or creates new)
+npx xqt import-results --file playwright-results.json --testExecKey PROJ-456
+
+# Or create new execution automatically
+npx xqt import-results --file playwright-results.json --summary "Playwright Regression"
+```
+
+**Benefits:**
+- Automatically maps Playwright tests to Xray test cases using annotations or `[TEST-123]` in titles
+- Creates new test cases in Xray if no mapping found
+- Supports Playwright retries, worker info, and detailed error reporting
+- Works in CI/CD pipelines for continuous test reporting
+
 ---
 
 ## Building Regression Packs
@@ -823,6 +889,173 @@ newman run collection.postman.json  # Full regression weekly
 
 ---
 
+## Working with Existing Xray Tests
+
+If your team already has test cases in Xray Cloud that were created manually or by another tool, you can set up this toolkit to manage and update those existing tests.
+
+### Step 1: Query Existing Tests from Xray
+
+Use the Xray GraphQL API to fetch your existing tests. Here's a script to generate the mapping file:
+
+**fetch-existing-tests.js:**
+```javascript
+import { authenticate, loadConfig } from "@msalaam/xray-qe-toolkit";
+import fs from "fs";
+
+const cfg = loadConfig();
+const token = await authenticate(cfg);
+
+// GraphQL query to fetch all tests in your project
+const query = `
+  query {
+    getTests(jql: "project = ${cfg.jiraProjectKey} AND issuetype = Test", limit: 1000) {
+      total
+      results {
+        issueId
+        jira(fields: ["key", "summary", "description", "priority", "labels"])
+      }
+    }
+  }
+`;
+
+const response = await fetch(cfg.xrayGraphQLUrl || "https://us.xray.cloud.getxray.app/api/v2/graphql", {
+  method: "POST",
+  headers: {
+    "Content-Type": "application/json",
+    Authorization: `Bearer ${token}`,
+  },
+  body: JSON.stringify({ query }),
+});
+
+const data = await response.json();
+const tests = data.data.getTests.results;
+
+// Generate xray-mapping.json
+const mapping = {};
+tests.forEach((test) => {
+  const testId = test.jira.key; // Use JIRA key as test_id initially
+  mapping[testId] = {
+    key: test.jira.key,
+    id: test.issueId,
+  };
+});
+
+fs.writeFileSync("xray-mapping.json", JSON.stringify(mapping, null, 2));
+console.log(`✓ Created xray-mapping.json with ${tests.length} tests`);
+
+// Generate tests.json scaffold
+const testsJson = {
+  testExecution: {
+    summary: "Existing Tests - Managed by Toolkit",
+    description: "Imported from existing Xray tests",
+  },
+  tests: tests.map((test) => ({
+    test_id: test.jira.key,
+    skip: false,
+    tags: [],
+    xray: {
+      summary: test.jira.summary,
+      description: test.jira.description || "",
+      priority: test.jira.priority?.name || "Medium",
+      labels: test.jira.labels || [],
+      steps: [
+        // You'll need to fetch test steps separately via another API call
+        {
+          action: "PLACEHOLDER - Edit this in npx xqt edit-json",
+          data: "",
+          expected_result: "PLACEHOLDER",
+        },
+      ],
+    },
+  })),
+};
+
+fs.writeFileSync("tests.json", JSON.stringify(testsJson, null, 2));
+console.log(`✓ Created tests.json with ${tests.length} test scaffolds`);
+console.log("\nNext steps:");
+console.log("1. Run 'npx xqt edit-json' to review and complete test steps");
+console.log("2. Run 'npx xqt push-tests' to update tests in Xray");
+```
+
+### Step 2: Run the Script
+
+```bash
+# Save the script above as fetch-existing-tests.js
+node fetch-existing-tests.js
+```
+
+This generates:
+- `xray-mapping.json` — maps your existing JIRA test keys to their IDs
+- `tests.json` — scaffold with summaries, descriptions, priorities, labels
+
+### Step 3: Complete Test Steps
+
+The script can't fetch test steps automatically (requires additional API calls). Complete them in the editor:
+
+```bash
+npx xqt edit-json
+# Edit each test to add proper steps
+# Save when done
+```
+
+### Step 4: Update Existing Tests
+
+Now you can update your existing Xray tests:
+
+```bash
+npx xqt push-tests
+```
+
+Because the tests are in `xray-mapping.json`, they'll be **updated** (not duplicated).
+
+### Alternative: Manual Mapping Creation
+
+If you have a small number of tests, create the mapping manually:
+
+**xray-mapping.json:**
+```json
+{
+  "APIEE-6933": { "key": "APIEE-6933", "id": "1865623" },
+  "APIEE-6934": { "key": "APIEE-6934", "id": "1865627" },
+  "APIEE-6935": { "key": "APIEE-6935", "id": "1865628" }
+}
+```
+
+**tests.json:**
+```json
+{
+  "tests": [
+    {
+      "test_id": "APIEE-6933",
+      "xray": {
+        "summary": "Test User Login",
+        "steps": [
+          { "action": "...", "expected_result": "..." }
+        ]
+      }
+    }
+  ]
+}
+```
+
+Then run `npx xqt push-tests` to update.
+
+### Future Enhancement: Pull Command
+
+A `pull-tests` command to automatically fetch and sync existing tests is planned:
+
+```bash
+# Future feature (not yet implemented)
+npx xqt pull-tests --project APIEE
+npx xqt pull-tests --jql "project = APIEE AND labels = regression"
+```
+
+This would automatically generate both `tests.json` and `xray-mapping.json` from Xray.
+
+**Want this feature?** [Open an issue on GitHub](https://github.com/Muhammad-Salaam_omit/xray-qe-toolkit/issues) or contribute via PR.
+
+---
+
 ## Idempotent Push
 
 `push-tests` checks `xray-mapping.json` before each operation:

package/bin/cli.js

@@ -13,7 +13,7 @@
  *   push-tests         Push / update tests in Xray Cloud
  *   gen-postman        Generate Postman collection from tests.json (with optional AI)
  *   create-execution   Create a new Test Execution in JIRA
- *   import-results     Import JUnit/Newman results into Xray
+ *   import-results     Import test results (JUnit XML or Playwright JSON) into Xray
  *   gen-pipeline       Generate an Azure Pipelines YAML template
  *   mcp-server         Start Model Context Protocol server for agent integration
  */
@@ -103,9 +103,11 @@ program
 
 program
   .command("import-results")
-  .description("Import JUnit/Newman XML results into Xray Cloud")
-  .requiredOption("--file <path>", "Path to the results XML file")
-  .requiredOption("--testExecKey <key>", "Test Execution key to associate results with")
+  .description("Import test results into Xray Cloud (JUnit XML or Playwright JSON)")
+  .requiredOption("--file <path>", "Path to the results file (.xml for JUnit, .json for Playwright)")
+  .option("--testExecKey <key>", "Test Execution key to associate results with (creates new if omitted)")
+  .option("--summary <text>", "Test Execution summary (for new executions)")
+  .option("--description <text>", "Test Execution description (for new executions)")
   .action(async (opts, cmd) => {
     const mod = await import("../commands/importResults.js");
     await mod.default({ ...opts, ...cmd.optsWithGlobals() });

package/commands/importResults.js

@@ -1,8 +1,11 @@
 /**
- * Command: xray-qe import-results --file <path> --testExecKey <key>
+ * Command: xqt import-results --file <path> [--testExecKey <key>]
  *
- * Imports JUnit/Newman XML results into Xray Cloud and associates
- * them with the specified Test Execution.
+ * Imports test results into Xray Cloud and associates them with a Test Execution.
+ *
+ * Supported formats:
+ * - JUnit XML (.xml) - Standard JUnit or XUnit format
+ * - Playwright JSON (.json) - Playwright JSON reporter output
  *
  * Designed to run in CI — no human interaction required.
  */
@@ -11,7 +14,8 @@ import fs from "node:fs";
 import path from "node:path";
 import logger, { setVerbose } from "../lib/logger.js";
 import { loadConfig, validateConfig } from "../lib/config.js";
-import { authenticate, importResults as importResultsApi } from "../lib/xrayClient.js";
+import { authenticate, importResults as importResultsXml, importResultsXrayJson } from "../lib/xrayClient.js";
+import { convertPlaywrightToXray } from "../lib/playwrightConverter.js";
 
 export default async function importResults(opts = {}) {
   if (opts.verbose) setVerbose(true);
@@ -27,23 +31,53 @@ export default async function importResults(opts = {}) {
     process.exit(1);
   }
 
-  logger.info(`File: ${path.basename(filePath)}`);
-  logger.info(`Test Execution: ${opts.testExecKey}`);
+  const fileName = path.basename(filePath);
+  const fileExt = path.extname(filePath).toLowerCase();
+
+  logger.info(`File: ${fileName}`);
+  if (opts.testExecKey) {
+    logger.info(`Test Execution: ${opts.testExecKey}`);
+  }
   logger.blank();
 
   // Authenticate
   const xrayToken = await authenticate(cfg);
 
-  // Read XML file
-  const xmlBuffer = fs.readFileSync(filePath);
-  logger.send(`Importing ${xmlBuffer.length} bytes to Xray...`);
+  // Detect format and import
+  let result;
+
+  if (fileExt === '.json') {
+    // Playwright JSON format
+    logger.send(`Importing Playwright JSON results...`);
 
-  const result = await importResultsApi(cfg, xrayToken, xmlBuffer, opts.testExecKey);
+    const playwrightJson = JSON.parse(fs.readFileSync(filePath, 'utf-8'));
+    
+    // Convert to Xray format
+    const xrayJson = convertPlaywrightToXray(playwrightJson, {
+      testExecutionKey: opts.testExecKey,
+      projectKey: cfg.jiraProjectKey,
+      summary: opts.summary || `Playwright Test Execution - ${new Date().toLocaleString()}`,
+      description: opts.description,
+    });
+
+    logger.step(`Converted ${xrayJson.tests.length} test results to Xray format`);
+
+    // Import to Xray
+    result = await importResultsXrayJson(cfg, xrayToken, xrayJson);
+  } else {
+    // JUnit XML format (default)
+    logger.send(`Importing JUnit XML results...`);
+
+    const xmlBuffer = fs.readFileSync(filePath);
+    result = await importResultsXml(cfg, xrayToken, xmlBuffer, opts.testExecKey);
+  }
 
   logger.success("Results imported successfully");
 
   if (result?.key) {
     logger.link(`View: ${cfg.jiraUrl}/browse/${result.key}`);
+  } else if (result?.testExecIssue?.key) {
+    logger.link(`View: ${cfg.jiraUrl}/browse/${result.testExecIssue.key}`);
   }
 
   logger.blank();