@msalaam/xray-qe-toolkit 1.5.0 → 1.6.0

package/schema/tests.schema.json

@@ -60,8 +60,27 @@
             "pattern": "^/"
           },
           "testSet": {
-            "type": "string",
-            "description": "Name of the Xray Test Set this test belongs to. Tests sharing the same value are grouped into a single Test Set issue in JIRA. xqt push creates the Test Set automatically if it doesn't exist and adds the test to it. Test Set → JIRA key mappings are stored in xray-mapping.json under _testSets."
+            "oneOf": [
+              {
+                "type": "string",
+                "description": "Name of a single Xray Test Set this test belongs to."
+              },
+              {
+                "type": "array",
+                "items": { "type": "string" },
+                "minItems": 1,
+                "description": "Names of multiple Xray Test Sets this test belongs to."
+              }
+            ],
+            "description": "Xray Test Set name(s) — string or array of strings. Tests are grouped into Test Set issues in JIRA. xqt push creates sets automatically and adds the test to each."
+          },
+          "preconditions": {
+            "type": "array",
+            "items": {
+              "type": "string",
+              "pattern": "^[A-Z][A-Z0-9_]+-[0-9]+$"
+            },
+            "description": "JIRA issue keys of Xray Precondition issues linked to this test (e.g. [\"PROJ-5\"]). xqt push calls addPreconditionsToTest on each push."
           },
           "requirementKeys": {
             "type": "array",

package/templates/README.template.md

@@ -152,6 +152,7 @@ After running `npx xqt init`:
       "tags": ["smoke", "regression"],
       "folder": "/{{SERVICE_NAME}}/HealthCheck/Validation",
       "testSet": "Health Check",
+      "preconditions": ["APIEE-50"],
       "requirementKeys": [],
       "xray": {
         "summary": "Service returns 200 OK when healthy",
@@ -175,7 +176,8 @@ After running `npx xqt init`:
 | `skip` | boolean | — | `true` to exclude from push-tests |
 | `tags` | string[] | — | QE tags for categorisation and filtering |
 | `folder` | string | — | Xray repository folder path — must start with `/` |
-| `testSet` | string | — | Test Set name in Jira — groups tests by feature |
+| `testSet` | string **or** string[] | — | Test Set name(s) in Jira — use a string or an array to assign to multiple sets |
+| `preconditions` | string[] | — | JIRA keys of Xray Precondition issues to link (e.g. `["APIEE-50"]`) |
 | `requirementKeys` | string[] | — | JIRA issue keys this test covers (creates coverage links) |
 | `xray.summary` | string | Yes | Test case title (JIRA issue summary) |
 | `xray.description` | string | — | Detailed description |
@@ -213,14 +215,19 @@ npx xqt create-plan --summary "{{SERVICE_NAME}} v2 Regression"
 npx xqt create-plan --summary "Sprint 12 Smoke Tests" --version "2024.12"
 ```
 
-### `xqt push-tests`
+### `xqt push-tests` (alias: `push`)
 
 Create or update tests in Xray Cloud, then sync the Test Plan membership and folder structure.
 
+- New tests are **bulk-imported** via the Xray REST API when there are ≥ `bulkImportThreshold` (default: 50) to create — significantly faster for large suites
+- Test Plan membership is **bi-directionally synced**: new tests are added and tests removed from `tests.json` are removed from the plan
+- Test Sets are **deduplicated by name** — safe to run on a fresh clone, existing sets are found in JIRA before creating new ones
+- `preconditions` are linked after every create or update
+
 ```bash
-npx xqt push-tests
-npx xqt push-tests --plan APIEE-1234   # override plan key
-npx xqt push-tests --verbose
+npx xqt push
+npx xqt push --plan APIEE-1234   # override plan key
+npx xqt push --verbose
 ```
 
 ### `xqt pull-tests`
@@ -232,15 +239,15 @@ npx xqt pull-tests --plan APIEE-1234
 npx xqt pull-tests --project APIEE --limit 200
 ```
 
-### `xqt import-results`
+### `xqt import-results` (alias: `import`)
 
 Import test execution results into Xray. Creates a **new** Test Execution each time.
 
 ```bash
-npx xqt import-results --file test-results/results.json --env IOP-QA
-npx xqt import-results --file test-results/results.xml   --env IOP-PROD
-npx xqt import-results --file test-results/results.json --env IOP-DEV \
-  --plan APIEE-1234 --version "2024.12" --revision "a1b2c3d"
+npx xqt import --file test-results/results.json --env IOP-QA
+npx xqt import --file test-results/results.xml   --env IOP-PROD
+npx xqt import --file test-results/results.json --env IOP-DEV \
+  --plan APIEE-1234 --fix-version "2.5.0" --revision "a1b2c3d"
 ```
 
 **Options:**
@@ -250,10 +257,31 @@ npx xqt import-results --file test-results/results.json --env IOP-DEV \
 | `--file <path>` | Path to results file (`.json` = Playwright, `.xml` = JUnit) |
 | `--env <label>` | Environment label: `IOP-DEV`, `IOP-QA`, `IOP-PROD` |
 | `--plan <key>` | Test Plan key (overrides `.xrayrc`) |
-| `--version <ver>` | Fix version / release label |
-| `--revision <sha>` | Build number or git SHA |
+| `--exec <key>` | Import INTO an existing execution (from `xqt exec`) |
+| `--fix-version <ver>` | JIRA Fix Version name to stamp on the execution (e.g. `2.5.0`) |
+| `--version <ver>` | Xray version label (free-form, separate from Fix Version) |
+| `--revision <sha>` | Build number or git SHA (enables traceability to a specific commit) |
 | `--summary <text>` | Custom execution summary |
 
+### `xqt create-execution` (alias: `exec`)
+
+Pre-create a Test Execution before running tests (for controlled test selection).
+
+```bash
+EXEC_KEY=$(npx xqt exec --env IOP-QA --quiet)
+npx playwright test
+npx xqt import --file test-results/results.json --exec $EXEC_KEY
+```
+
+| Flag | Description |
+|---|---|
+| `--env <label>` | Environment label |
+| `--plan <key>` | Test Plan to link to |
+| `--tests <ids>` | Comma-separated testIds or JIRA keys |
+| `--summary <text>` | Custom execution title |
+| `--fix-version <ver>` | JIRA Fix Version to stamp on the execution |
+| `--quiet` | Print only the execution key |
+
 ### `xqt sync-folders`
 
 Sync the Xray repository folder structure from `folder` fields in tests.json.
@@ -306,7 +334,10 @@ npx xqt gen-pipeline --output .azure/ci.yml
 
 Tests live in **Test Sets** in Jira, grouped by feature (matching the feature name in `business-rules.yaml`).
 
-- Created automatically by `push-tests` from the `testSet` field in `tests.json`
+- Assign a test to one set: `"testSet": "Health Check"`
+- Assign to multiple sets: `"testSet": ["Health Check", "Smoke"]`
+- Created automatically by `push` from the `testSet` field in `tests.json`
+- **Deduplicated by name** — if `xray-mapping.json` is lost (e.g. fresh clone), sets are found by searching JIRA before creating a new one
 - Persistent across sprints — they represent what tests exist
 - Link Test Sets to Test Plans when entering a sprint
 
@@ -314,17 +345,20 @@ Tests live in **Test Sets** in Jira, grouped by feature (matching the feature na
 
 A Test Plan scopes testing for a specific sprint or release.
 
-- Create per sprint: `npx xqt create-plan --summary "Sprint 12 — {{SERVICE_NAME}}"`
+- Create per sprint: `npx xqt plan --summary "Sprint 12 — {{SERVICE_NAME}}"`
 - Link relevant Test Sets to the Test Plan in Jira
 - The key is stored in `.xrayrc` (`testPlanKey`)
-- `import-results` links executions to the active plan
+- `import` links executions to the active plan
+- **`xqt push` performs a bi-directional sync** — new tests are added and tests removed from `tests.json` are removed from the plan
 
 ### Test Executions (ephemeral)
 
 A Test Execution represents one CI run.
-- Created automatically by `xqt import-results`
+- Created automatically by `xqt import`
+- **Pre-created** by `xqt exec` for controlled test selection (filter which tests run)
 - Tagged with the environment (`IOP-DEV`, `IOP-QA`, `IOP-PROD`)
 - Linked to the Test Plan
+- Stamped with `--fix-version` and `--revision` for release traceability
 
 ### Summary
 
@@ -348,7 +382,7 @@ Set the default environment and allowed values in `.xrayrc`:
 Override per run with `--env`:
 
 ```bash
-npx xqt import-results --file results.json --env IOP-PROD
+npx xqt import --file results.json --env IOP-PROD
 ```
 
 ---
@@ -387,12 +421,12 @@ The generated `playwright.config.ts` configures three reporters:
 
 ```bash
 # After running: npx playwright test
-npx xqt import-results --file test-results/results.json --env IOP-QA
+npx xqt import --file test-results/results.json --env IOP-QA
 ```
 
 ### Test steps in Xray
 
-Steps defined with `test.step()` are automatically mapped to Xray step results:
+Steps defined with `test.step()` are automatically mapped to Xray step results. **Screenshots captured on a failing step are attached directly to that step's evidence** in Xray — giving per-step failure screenshots in the Xray UI. Traces and other attachments are attached at the test level.
 
 ```typescript
 test('Create and retrieve user', async ({ request }) => {
@@ -481,9 +515,9 @@ Key import step (runs even on test failure):
 ```yaml
 - script: |
     REVISION=$(echo "$(Build.SourceVersion)" | cut -c1-8)
-    npx xqt import-results \
+    npx xqt import \
       --file test-results/results.json \
-      --version "$(Build.BuildNumber)" \
+      --fix-version "$(Build.BuildNumber)" \
       --revision "$REVISION"
   condition: succeededOrFailed()
   env:
@@ -551,11 +585,34 @@ Auto-managed by `push-tests` — do not edit manually.
   "defaultEnvironment": "IOP-DEV",
   "environments":       ["IOP-DEV", "IOP-QA", "IOP-PROD"],
   "folderRoot":         "/{{SERVICE_NAME}}",
-  "xrayRegion":         "us"
+  "xrayRegion":         "us",
+  "bulkImportThreshold": 50,
+  "statusMapping": {
+    "interrupted": "ABORTED",
+    "skipped":     "TODO"
+  }
 }
 ```
 
-### xrayRegion values
+| Field | Description |
+|---|---|
+| `testPlanKey` | Default Test Plan key |
+| `defaultEnvironment` | Fallback environment when `--env` is not provided |
+| `environments` | Allowed environment labels |
+| `folderRoot` | Base folder path in Xray Test Repository |
+| `xrayRegion` | `us` (default), `eu`, or `au` |
+| `bulkImportThreshold` | Min new-test count to switch to bulk REST import (default: `50`) |
+| `statusMapping` | Override default Playwright → Xray status mapping |
+
+#### Playwright status mapping
+
+By default: `passed→PASSED`, `failed→FAILED`, `timedOut→FAILED`, `interrupted→ABORTED`, `skipped→TODO`. Override any value:
+
+```json
+{ "statusMapping": { "interrupted": "FAILED", "skipped": "TODO" } }
+```
+
+You can also set `XQT_BULK_THRESHOLD=<n>` as an environment variable to override the bulk threshold at runtime.
 
 | Value | GraphQL endpoint |
 |---|---|