@msalaam/xray-qe-toolkit 1.4.0 → 1.5.0

package/templates/resources-README.md

@@ -0,0 +1,112 @@
+# Resources — AI Test Generation Sources
+
+This folder contains documentation and specifications used by the AI agent to generate test cases for Xray.
+
+---
+
+## Folder Structure
+
+```
+resources/
+├── api-specs/          OpenAPI/Swagger specifications (YAML/JSON)
+├── requirements/       Business requirements, acceptance criteria, logic docs
+└── tickets/            Exported JIRA tickets, Confluence pages, user stories
+```
+
+---
+
+## Supported File Types
+
+### `api-specs/`
+- **OpenAPI 3.x / Swagger 2.0** — `.yaml`, `.yml`, `.json`
+- **GraphQL schemas** — `.graphql`, `.gql`
+
+Files in this folder drive model, service, assertion, and spec generation via the spec-driven workflow.
+
+### `requirements/`
+- **Markdown** — `.md`
+- **Plain text** — `.txt`
+- **Word documents** — `.docx`
+- **PDFs** — `.pdf`
+
+Business logic, acceptance criteria, workflow descriptions, and domain rules. Used to generate test scenarios, edge cases, and validation logic.
+
+### `tickets/`
+- **JIRA exports** — `.json`, `.xml`
+- **Confluence HTML exports** — `.html`
+- **User stories** — `.md`, `.txt`
+
+Ticket acceptance criteria and linked documentation. Used to ensure test coverage aligns with stories and epics.
+
+---
+
+## How the AI Agent Uses These Files
+
+1. **Test Case Generation** (AI agent reads these files and populates `tests.json`)
+   - Reads `openapi.yaml` (API shape) and `business-rules.yaml` (domain rules)
+   - Produces `tests.json` entries with stable `test_id` values for Xray traceability
+   - Each `business-rules.yaml` rule maps 1:1 to a `tests.json` entry
+   - See **[SPEC-DRIVEN-APPROACH.md](../SPEC-DRIVEN-APPROACH.md)** for the full workflow
+
+2. **Xray Sync** (after `tests.json` is generated)
+   - `npx xqt validate` then `npx xqt push-tests`
+   - Creates Xray Test issues + Test Sets in Jira
+   - Populates `xray-mapping.json` with issue keys
+
+3. **Playwright Test Creation** (after Xray sync)
+   - Copilot skill reads `tests.json` + `xray-mapping.json`
+   - Creates one Playwright `test()` call per `tests.json` entry
+   - Real Jira keys in test titles from day one
+
+4. **Continuous Refinement**
+   - As you add/update specs and business rules, re-run the generation workflow
+   - The toolkit is **idempotent** — existing tests are updated, retired rules get `skip: true`
+   - Always validate before pushing: `npx xqt validate` then `npx xqt push-tests`
+
+---
+
+## Best Practices
+
+✅ **DO:**
+- Keep specs up to date with your API implementation
+- Organize files by feature/domain for easier tracking
+- Use descriptive filenames (e.g., `auth-api-v2.yaml`, `payment-flows.md`)
+- Commit this folder to source control (Git)
+
+⚠️ **DON'T:**
+- Store credentials or secrets in these files
+- Include massive binary files (>10MB) — extract relevant sections
+- Mix unrelated specs in the same file
+- Forget to update specs when the API changes
+
+---
+
+## Example Workflow
+
+```bash
+# 1. Add your OpenAPI spec
+cp ~/Downloads/my-api.yaml resources/openapi.yaml
+
+# 2. Populate business rules (or generate a draft from the spec)
+# Use Copilot + SPEC-DRIVEN-APPROACH.md to generate resources/business-rules.yaml
+
+# 3. Validate knowledge artifacts
+npx xqt validate --schema business-rules
+
+# 4. Use Copilot (spec-driven workflow) to generate models, services, assertions, and specs
+# See SPEC-DRIVEN-APPROACH.md for the full step-by-step
+
+# 5. Validate tests.json
+npx xqt validate
+
+# 6. Push tests to Xray
+npx xqt push-tests
+
+# 7. Run tests and import results
+npx playwright test
+npx xqt import-results --file test-results/results.json
+```
+
+---
+
+**Note:** This folder is the knowledge input for the spec-driven workflow. See **[SPEC-DRIVEN-APPROACH.md](../SPEC-DRIVEN-APPROACH.md)** for the complete greenfield/brownfield generation process.

package/templates/tests.json

@@ -1,74 +1,245 @@
 {
-  "testExecution": {
-    "summary": "My Project - Regression Suite",
-    "description": "Automated regression tests for My Project API"
+  "testPlan": {
+    "summary": "Death_Claims_Portfolio_API — Test Plan"
+  },
+  "planWorkflow": {
+    "mode": "dual-track",
+    "createPlan": {
+      "tool": "xqt",
+      "enabled": true
+    },
+    "linkTestSets": {
+      "mode": "auto"
+    },
+    "execution": {
+      "contractStage": "separate"
+    }
   },
   "tests": [
     {
-      "test_id": "TC-API-001",
+      "test_id": "TC-DEATHCLA-BR-001",
+      "testKind": "business-rule",
       "type": "api",
       "skip": false,
-      "tags": ["regression", "smoke"],
+      "tags": ["smoke", "regression"],
+      "folder": "/Death_Claims_Portfolio_API/HealthCheck/Validation",
+      "testSet": "Health Check",
+      "requirementKeys": [],
       "xray": {
-        "summary": "Verify successful response for valid request",
-        "description": "Test that the API returns the expected data and status code for a valid, well-formed request.",
+        "summary": "Service returns 200 OK when healthy",
+        "testType": "Manual",
         "priority": "High",
-        "labels": ["API", "Positive", "Regression"],
+        "labels": ["smoke", "regression"],
         "steps": [
           {
-            "action": "Send GET request to /api/resource with valid parameters",
-            "data": "Method: GET, Headers: Authorization: Bearer {token}, Params: id=1",
-            "expected_result": "API returns HTTP 200 with resource object"
-          },
+            "action": "Send GET /health with no request body",
+            "data": "",
+            "expected_result": "HTTP 200, Content-Type: application/json, body: { \"status\": \"ok\" }"
+          }
+        ]
+      },
+      "spec": {
+        "operation": {
+          "method": "GET",
+          "path": "/health",
+          "operationId": ""
+        },
+        "expectations": {
+          "statusCodes": [200],
+          "contentType": "application/json",
+          "responseSchemaHints": []
+        },
+        "dataHints": {
+          "pathParams": [],
+          "queryParams": [],
+          "body": "",
+          "env": []
+        }
+      }
+    },
+    {
+      "test_id": "TC-DEATHCLA-BR-002",
+      "testKind": "business-rule",
+      "type": "api",
+      "skip": false,
+      "tags": ["smoke", "regression"],
+      "folder": "/Death_Claims_Portfolio_API/ClientLookup/BusinessLogic",
+      "testSet": "Client Lookup",
+      "requirementKeys": [],
+      "xray": {
+        "summary": "Valid client number returns portfolio details",
+        "testType": "Manual",
+        "priority": "High",
+        "labels": ["smoke", "regression"],
+        "steps": [
           {
-            "action": "Validate response body structure",
-            "data": "Response body JSON",
-            "expected_result": "Response contains expected fields: id, name, status"
+            "action": "Send GET /v1/death-claims/portfolio/{clientNo} with a valid client number",
+            "data": "clientNo: env.TEST_CLIENT_NO",
+            "expected_result": "HTTP 200, Content-Type: application/json, body matches ED43Response schema"
           }
         ]
+      },
+      "spec": {
+        "operation": {
+          "method": "GET",
+          "path": "/v1/death-claims/portfolio/{clientNo}",
+          "operationId": "getPortfolioByClientNo"
+        },
+        "expectations": {
+          "statusCodes": [200],
+          "contentType": "application/json",
+          "responseSchemaHints": ["#/components/schemas/ED43Response"]
+        },
+        "dataHints": {
+          "pathParams": ["clientNo"],
+          "queryParams": [],
+          "body": "",
+          "env": ["TEST_CLIENT_NO"]
+        }
       }
     },
     {
-      "test_id": "TC-API-002",
+      "test_id": "TC-DEATHCLA-CT-GET-V1_DEATH_CLAIMS_PORTFOLIO_CLIENTNO-200",
+      "testKind": "contract",
       "type": "api",
       "skip": false,
-      "tags": ["regression", "negative"],
+      "tags": ["regression", "integration"],
+      "folder": "/Death_Claims_Portfolio_API/Contract/ClientLookup",
+      "testSet": "Contract - Client Lookup",
+      "requirementKeys": [],
       "xray": {
-        "summary": "Verify error response for invalid request",
-        "description": "Test that the API returns a descriptive error when invalid or missing input is provided.",
+        "summary": "Contract: GET /v1/death-claims/portfolio/{clientNo} → 200",
+        "testType": "Manual",
         "priority": "Medium",
-        "labels": ["API", "Negative", "Validation"],
+        "labels": ["regression", "integration"],
         "steps": [
           {
-            "action": "Send POST request to /api/resource with missing required fields",
-            "data": "Method: POST, Headers: Authorization: Bearer {token}, Body: {}",
-            "expected_result": "API returns HTTP 400 Bad Request"
-          },
+            "action": "Send GET /v1/death-claims/portfolio/{clientNo} with a valid client number",
+            "data": "clientNo: env.TEST_CLIENT_NO",
+            "expected_result": "HTTP 200, Content-Type: application/json, response validates against ED43Response schema (status-code, content-type, required-fields, type-checks)"
+          }
+        ]
+      },
+      "spec": {
+        "operation": {
+          "method": "GET",
+          "path": "/v1/death-claims/portfolio/{clientNo}",
+          "operationId": "getPortfolioByClientNo"
+        },
+        "expectations": {
+          "statusCodes": [200],
+          "contentType": "application/json",
+          "responseSchemaHints": ["#/components/schemas/ED43Response"]
+        },
+        "dataHints": {
+          "pathParams": ["clientNo"],
+          "queryParams": [],
+          "body": "",
+          "env": ["TEST_CLIENT_NO"]
+        }
+      },
+      "contract": {
+        "operationId": "getPortfolioByClientNo",
+        "expectedStatus": 200,
+        "expectedContentType": "application/json",
+        "responseSchemaRef": "#/components/schemas/ED43Response",
+        "validationScope": ["status-code", "content-type", "required-fields", "type-checks"]
+      }
+    },
+    {
+      "test_id": "TC-DEATHCLA-CT-GET-V1_DEATH_CLAIMS_PORTFOLIO_CLIENTNO-400",
+      "testKind": "contract",
+      "type": "api",
+      "skip": false,
+      "tags": ["regression", "integration"],
+      "folder": "/Death_Claims_Portfolio_API/Contract/ClientLookup",
+      "testSet": "Contract - Client Lookup",
+      "requirementKeys": [],
+      "xray": {
+        "summary": "Contract: GET /v1/death-claims/portfolio/{clientNo} → 400",
+        "testType": "Manual",
+        "priority": "High",
+        "labels": ["regression", "integration"],
+        "steps": [
           {
-            "action": "Validate error response body",
-            "data": "Response body JSON",
-            "expected_result": "Response contains 'error' field with a descriptive message string"
+            "action": "Send GET /v1/death-claims/portfolio/{clientNo} with an invalid client number format",
+            "data": "clientNo: env.TEST_INVALID_CLIENT_NO",
+            "expected_result": "HTTP 400, Content-Type: application/json, response validates against ErrorResponse schema (status-code, content-type, required-fields, type-checks)"
           }
         ]
+      },
+      "spec": {
+        "operation": {
+          "method": "GET",
+          "path": "/v1/death-claims/portfolio/{clientNo}",
+          "operationId": "getPortfolioByClientNo"
+        },
+        "expectations": {
+          "statusCodes": [400],
+          "contentType": "application/json",
+          "responseSchemaHints": ["#/components/schemas/ErrorResponse"]
+        },
+        "dataHints": {
+          "pathParams": ["clientNo"],
+          "queryParams": [],
+          "body": "",
+          "env": ["TEST_INVALID_CLIENT_NO"]
+        }
+      },
+      "contract": {
+        "operationId": "getPortfolioByClientNo",
+        "expectedStatus": 400,
+        "expectedContentType": "application/json",
+        "responseSchemaRef": "#/components/schemas/ErrorResponse",
+        "validationScope": ["status-code", "content-type", "required-fields", "type-checks"]
       }
     },
     {
-      "test_id": "TC-API-003",
+      "test_id": "TC-DEATHCLA-CT-GET-V1_DEATH_CLAIMS_PORTFOLIO_CLIENTNO-404",
+      "testKind": "contract",
       "type": "api",
-      "skip": true,
-      "tags": ["edge", "performance"],
+      "skip": false,
+      "tags": ["regression", "integration"],
+      "folder": "/Death_Claims_Portfolio_API/Contract/ClientLookup",
+      "testSet": "Contract - Client Lookup",
+      "requirementKeys": [],
       "xray": {
-        "summary": "Verify API handles large payload within acceptable time",
-        "description": "Test that the API processes a large request payload without timeout or degraded response.",
-        "priority": "Low",
-        "labels": ["API", "Performance", "Edge"],
+        "summary": "Contract: GET /v1/death-claims/portfolio/{clientNo} → 404",
+        "testType": "Manual",
+        "priority": "High",
+        "labels": ["regression", "integration"],
         "steps": [
           {
-            "action": "Send POST request to /api/resource with large payload",
-            "data": "Method: POST, Headers: Authorization: Bearer {token}, Body: large JSON array with 1000 items",
-            "expected_result": "API returns HTTP 200 within 3000ms"
+            "action": "Send GET /v1/death-claims/portfolio/{clientNo} with an unknown client number",
+            "data": "clientNo: env.TEST_UNKNOWN_CLIENT_NO",
+            "expected_result": "HTTP 404, Content-Type: application/json, response validates against ErrorResponse schema (status-code, content-type, required-fields, type-checks)"
           }
         ]
+      },
+      "spec": {
+        "operation": {
+          "method": "GET",
+          "path": "/v1/death-claims/portfolio/{clientNo}",
+          "operationId": "getPortfolioByClientNo"
+        },
+        "expectations": {
+          "statusCodes": [404],
+          "contentType": "application/json",
+          "responseSchemaHints": ["#/components/schemas/ErrorResponse"]
+        },
+        "dataHints": {
+          "pathParams": ["clientNo"],
+          "queryParams": [],
+          "body": "",
+          "env": ["TEST_UNKNOWN_CLIENT_NO"]
+        }
+      },
+      "contract": {
+        "operationId": "getPortfolioByClientNo",
+        "expectedStatus": 404,
+        "expectedContentType": "application/json",
+        "responseSchemaRef": "#/components/schemas/ErrorResponse",
+        "validationScope": ["status-code", "content-type", "required-fields", "type-checks"]
       }
     }
   ]

package/commands/compareOpenapi.js

@@ -1,78 +0,0 @@
-import { diff } from 'openapi-diff';
-import fs from 'fs';
-import path from 'path';
-
-/**
- * Compare two OpenAPI specs and detect breaking changes.
- * @param {string} currentPath  - Path to the current (live) OpenAPI spec.
- * @param {string} snapshotPath - Path to the approved QA snapshot.
- * @param {string|null} reportPath - Optional path to write a JSON diff report.
- */
-export async function compareOpenApiSpecs(currentPath, snapshotPath, reportPath = null) {
-  const resolvedCurrent = path.resolve(currentPath);
-  const resolvedSnapshot = path.resolve(snapshotPath);
-
-  if (!fs.existsSync(resolvedCurrent)) {
-    throw new Error(`Current OpenAPI spec not found: ${resolvedCurrent}`);
-  }
-
-  if (!fs.existsSync(resolvedSnapshot)) {
-    throw new Error(`Snapshot OpenAPI spec not found: ${resolvedSnapshot}`);
-  }
-
-  const currentSpec = fs.readFileSync(resolvedCurrent, 'utf-8');
-  const snapshotSpec = fs.readFileSync(resolvedSnapshot, 'utf-8');
-
-  const result = await diff({
-    sourceSpec: { content: snapshotSpec },      // approved baseline
-    destinationSpec: { content: currentSpec }   // proposed change
-  });
-
-  const summary = {
-    breaking: result.breakingDifferencesFound,
-    breakingDifferences: result.breakingDifferences,
-    nonBreakingDifferences: result.nonBreakingDifferences
-  };
-
-  if (reportPath && result.breakingDifferencesFound) {
-    const resolvedReport = path.resolve(reportPath);
-    fs.writeFileSync(resolvedReport, JSON.stringify(summary, null, 2), 'utf-8');
-    console.log(`📄 Diff report written to: ${resolvedReport}`);
-  }
-
-  return summary;
-}
-
-export default async function compareOpenapi(opts) {
-  const { current, snapshot, report } = opts;
-
-  try {
-    const result = await compareOpenApiSpecs(current, snapshot, report || null);
-
-    if (result.breaking) {
-      console.error('\n❌ Breaking changes detected:\n');
-      console.error(JSON.stringify(result.breakingDifferences, null, 2));
-
-      const count = result.breakingDifferences?.length ?? 0;
-      console.error(`\n${count} breaking difference(s) found.`);
-
-      if (!report) {
-        console.error('\nTip: use --report <path> to save a full diff report as a pipeline artifact.');
-      }
-
-      process.exit(1);
-    }
-
-    const nonBreaking = result.nonBreakingDifferences?.length ?? 0;
-    console.log('\n✅ No breaking changes detected.');
-
-    if (nonBreaking > 0) {
-      console.log(`ℹ️  ${nonBreaking} non-breaking difference(s) found (safe to proceed).`);
-    }
-
-    process.exit(0);
-  } catch (err) {
-    console.error(`\n❌ ${err.message}`);
-    process.exit(1);
-  }
-}

package/commands/genPostman.js

@@ -1,70 +0,0 @@
-/**
- * Command: xray-qe gen-postman [--base-url <url>] [--ai] [--spec <path>] [--knowledge <path>]
- *
- * Generates a Postman Collection v2.1 JSON file from tests.json.
- * Optionally uses xray-mapping.json to embed JIRA keys for better traceability.
- * With --ai flag, enables AI-assisted generation (requires connected provider).
- */
-
-import fs from "node:fs";
-import logger, { setVerbose } from "../lib/logger.js";
-import { loadConfig } from "../lib/config.js";
-import { generate } from "../lib/postmanGenerator.js";
-import { loadMapping } from "../lib/testCaseBuilder.js";
-
-export default async function genPostman(opts = {}) {
-  if (opts.verbose) setVerbose(true);
-
-  logger.rocket("@msalaam/xray-qe-toolkit — Generate Postman Collection\n");
-
-  const cfg = loadConfig({ envPath: opts.env });
-
-  if (!fs.existsSync(cfg.testsPath)) {
-    logger.error(`tests.json not found at ${cfg.testsPath}`);
-    logger.info("Run 'npx xray-qe init' to create a starter tests.json");
-    process.exit(1);
-  }
-
-  const testsConfig = JSON.parse(fs.readFileSync(cfg.testsPath, "utf8"));
-
-  // Load mapping to embed JIRA keys when available
-  const mapping = loadMapping(cfg.mappingPath);
-
-  // Filter out skipped tests
-  const filteredConfig = {
-    ...testsConfig,
-    tests: (testsConfig.tests || []).filter((t) => !t.skip),
-  };
-
-  // AI flag handling
-  if (opts.ai) {
-    logger.info("🤖 AI-assisted generation enabled");
-    logger.warn("AI provider not yet connected — using schema-driven generation as fallback");
-    logger.blank();
-  }
-
-  // Knowledge path override
-  const knowledgePath = opts.knowledge || cfg.knowledgePath;
-  if (opts.knowledge) {
-    logger.info(`Using custom knowledge path: ${knowledgePath}`);
-  }
-
-  // Spec file handling (schema-driven generation without AI)
-  if (opts.spec && fs.existsSync(opts.spec)) {
-    logger.info(`Using OpenAPI spec: ${opts.spec}`);
-    logger.info("📝 Schema-driven endpoint inference enabled");
-    // TODO: Parse OpenAPI spec and enhance endpoint/method/body inference
-    logger.blank();
-  }
-
-  generate(filteredConfig, cfg.collectionPath, {
-    baseUrl: opts.baseUrl || "{{baseUrl}}",
-    mapping: mapping,
-  });
-
-  logger.blank();
-  logger.success(`Collection saved to ${cfg.collectionPath}`);
-  logger.info("Import into Postman or run with Newman:");
-  console.log(`   npx newman run ${cfg.collectionPath}`);
-  logger.blank();
-}