digiqagent 1.2.0

package/skills/postman-test-suite-generator/SKILL.md

@@ -0,0 +1,653 @@
+---
+name: postman-test-suite-generator
+description: Use when generating Postman collection test scripts with positive and negative test scenarios that developers can run manually to verify API behavior
+---
+
+# Postman Test Suite Generator
+
+## Overview
+
+Produce a Postman collection with comprehensive test scripts covering both positive (happy path) and negative (error/edge case) scenarios for every endpoint. Developers import and run it in Postman to verify API behavior with zero test-writing effort.
+
+**Core principle:** Every endpoint gets tested for what should work AND what should fail. If you can't prove the API rejects bad input, you don't know if it validates anything.
+
+## When to Use
+
+- API needs test coverage that developers can run manually
+- After generating a Postman collection with `digiqagent:postman-collection-generator`
+- QA handoff — provide runnable test scenarios
+- Pre-release verification — structured test pass before deployment
+- Onboarding — new developers see expected API behavior through tests
+
+## Input Detection
+
+### Preferred — Existing Postman Collection
+
+Look for `postman_collection.json` in the project. Build test scripts on top of existing requests.
+
+If no collection exists, recommend running `digiqagent:postman-collection-generator` first.
+
+### Alternative — OpenAPI Spec
+
+Parse `openapi.yaml` / `swagger.yaml` directly to build both requests and tests in one pass.
+
+### Alternative — Codebase Scan
+
+If neither exists, scan the codebase for endpoints, validation rules, and auth requirements. Generate a complete test collection from code analysis.
+
+## Generation Process
+
+### Step 1 — Analyze Endpoints
+
+For each endpoint, identify:
+- **Expected success behavior** — status code, response shape, side effects
+- **Required fields** — which fields trigger 400 if missing
+- **Field validations** — types, formats, min/max, patterns, enums
+- **Auth requirements** — which roles/permissions are needed
+- **Business rules** — uniqueness constraints, state transitions, dependencies
+- **Edge cases** — empty collections, boundary values, concurrent access
+
+Build a **scenario matrix** per endpoint:
+
+| Endpoint | Positive Scenarios | Negative Scenarios |
+|----------|-------------------|-------------------|
+| POST /users | Valid creation, minimal fields, all fields | Missing email, invalid email, duplicate email, short password, missing name, invalid role |
+| GET /users/:id | Valid ID returns user | Non-existent ID, invalid ID format, unauthorized |
+| PUT /users/:id | Valid update, partial update | Non-existent ID, invalid fields, unauthorized, forbidden |
+| DELETE /users/:id | Valid deletion | Non-existent ID, unauthorized, already deleted |
+
+### Step 2 — Generate Positive Test Scenarios
+
+For each endpoint, create happy-path tests that verify:
+
+**Status code:**
+```javascript
+pm.test("Status code is 201", function () {
+    pm.response.to.have.status(201);
+});
+```
+
+**Response time:**
+```javascript
+pm.test("Response time is under 2000ms", function () {
+    pm.expect(pm.response.responseTime).to.be.below(2000);
+});
+```
+
+**Response body structure:**
+```javascript
+pm.test("Response has correct structure", function () {
+    const response = pm.response.json();
+    pm.expect(response).to.have.property("id");
+    pm.expect(response).to.have.property("name");
+    pm.expect(response).to.have.property("email");
+    pm.expect(response).to.have.property("createdAt");
+});
+```
+
+**Response data types:**
+```javascript
+pm.test("Response fields have correct types", function () {
+    const response = pm.response.json();
+    pm.expect(response.id).to.be.a("string");
+    pm.expect(response.name).to.be.a("string");
+    pm.expect(response.email).to.be.a("string");
+    pm.expect(response.active).to.be.a("boolean");
+});
+```
+
+**Response content matches request:**
+```javascript
+pm.test("Created user matches request data", function () {
+    const request = JSON.parse(pm.request.body.raw);
+    const response = pm.response.json();
+    pm.expect(response.name).to.eql(request.name);
+    pm.expect(response.email).to.eql(request.email);
+});
+```
+
+**Headers:**
+```javascript
+pm.test("Content-Type is application/json", function () {
+    pm.response.to.have.header("Content-Type");
+    pm.expect(pm.response.headers.get("Content-Type")).to.include("application/json");
+});
+```
+
+**Pagination (for list endpoints):**
+```javascript
+pm.test("Response has pagination metadata", function () {
+    const response = pm.response.json();
+    pm.expect(response).to.have.property("meta");
+    pm.expect(response.meta).to.have.property("page");
+    pm.expect(response.meta).to.have.property("limit");
+    pm.expect(response.meta).to.have.property("totalItems");
+    pm.expect(response.meta).to.have.property("totalPages");
+    pm.expect(response.meta.page).to.be.a("number");
+    pm.expect(response.data).to.be.an("array");
+});
+```
+
+**Variable capture for chaining:**
+```javascript
+if (pm.response.code === 201) {
+    const response = pm.response.json();
+    pm.environment.set("createdUserId", response.id);
+}
+```
+
+### Step 3 — Generate Negative Test Scenarios
+
+For each endpoint, **systematically** cover these categories:
+
+#### 3a. Missing Required Fields (one at a time)
+
+For a POST /users endpoint with required fields `name`, `email`, `password`:
+
+**Missing name:**
+```json
+{
+  "name": "Negative - Create User - Missing Name",
+  "request": {
+    "method": "POST",
+    "body": {
+      "mode": "raw",
+      "raw": "{\n  \"email\": \"jane@example.com\",\n  \"password\": \"SecureP@ss123\"\n}"
+    }
+  },
+  "event": [{
+    "listen": "test",
+    "script": {
+      "exec": [
+        "pm.test('Returns 400 for missing name', function () {",
+        "    pm.response.to.have.status(400);",
+        "});",
+        "pm.test('Error message mentions name field', function () {",
+        "    const response = pm.response.json();",
+        "    const message = Array.isArray(response.message) ? response.message.join(' ') : response.message;",
+        "    pm.expect(message.toLowerCase()).to.include('name');",
+        "});"
+      ]
+    }
+  }]
+}
+```
+
+**Repeat for each required field:** Remove one field at a time, verify 400 response and field-specific error message.
+
+#### 3b. Invalid Data Types
+
+```javascript
+// String where number expected
+pm.test("Returns 400 for string price", function () {
+    pm.response.to.have.status(400);
+});
+// Body: { "price": "not-a-number" }
+
+// Number where string expected
+// Body: { "name": 12345 }
+
+// Boolean where string expected
+// Body: { "email": true }
+
+// Array where object expected
+// Body: []
+```
+
+#### 3c. Boundary Values
+
+```javascript
+// Empty string
+pm.test("Returns 400 for empty name", function () {
+    pm.response.to.have.status(400);
+});
+// Body: { "name": "", "email": "jane@example.com", "password": "SecureP@ss123" }
+
+// Exceeds max length (if maxLength: 100)
+// Body: { "name": "A".repeat(101) }
+
+// Below minimum (if minimum: 0)
+// Body: { "price": -1 }
+
+// Zero value
+// Body: { "quantity": 0 }
+
+// Whitespace only
+// Body: { "name": "   " }
+```
+
+#### 3d. Unauthorized Access
+
+**No token:**
+```json
+{
+  "name": "Negative - Get Users - No Auth Token",
+  "request": {
+    "method": "GET",
+    "header": [],
+    "url": { "raw": "{{baseUrl}}/api/users" }
+  },
+  "event": [{
+    "listen": "test",
+    "script": {
+      "exec": [
+        "pm.test('Returns 401 without auth token', function () {",
+        "    pm.response.to.have.status(401);",
+        "});",
+        "pm.test('Error indicates authentication required', function () {",
+        "    const response = pm.response.json();",
+        "    pm.expect(response.error || response.message).to.be.a('string');",
+        "});"
+      ]
+    }
+  }]
+}
+```
+
+**Invalid token:**
+```json
+{
+  "name": "Negative - Get Users - Invalid Token",
+  "request": {
+    "method": "GET",
+    "header": [
+      { "key": "Authorization", "value": "Bearer invalid.token.value" }
+    ]
+  }
+}
+```
+
+**Expired token (simulated):**
+```json
+{
+  "name": "Negative - Get Users - Expired Token",
+  "request": {
+    "method": "GET",
+    "header": [
+      { "key": "Authorization", "value": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwiZXhwIjoxMDAwMDAwMDAwfQ.invalid" }
+    ]
+  }
+}
+```
+
+**Wrong role/insufficient permissions:**
+```json
+{
+  "name": "Negative - Delete User - Insufficient Permissions",
+  "request": {
+    "method": "DELETE",
+    "header": [
+      { "key": "Authorization", "value": "Bearer {{regularUserToken}}" }
+    ]
+  },
+  "event": [{
+    "listen": "test",
+    "script": {
+      "exec": [
+        "pm.test('Returns 403 for insufficient permissions', function () {",
+        "    pm.response.to.have.status(403);",
+        "});"
+      ]
+    }
+  }]
+}
+```
+
+#### 3e. Not Found
+
+```json
+{
+  "name": "Negative - Get User - Non-Existent ID",
+  "request": {
+    "method": "GET",
+    "url": { "raw": "{{baseUrl}}/api/users/00000000-0000-0000-0000-000000000000" }
+  },
+  "event": [{
+    "listen": "test",
+    "script": {
+      "exec": [
+        "pm.test('Returns 404 for non-existent user', function () {",
+        "    pm.response.to.have.status(404);",
+        "});",
+        "pm.test('Error message indicates resource not found', function () {",
+        "    const response = pm.response.json();",
+        "    const msg = (response.message || response.error || '').toLowerCase();",
+        "    pm.expect(msg).to.include('not found');",
+        "});"
+      ]
+    }
+  }]
+}
+```
+
+**Invalid ID format:**
+```json
+{
+  "name": "Negative - Get User - Invalid ID Format",
+  "request": {
+    "method": "GET",
+    "url": { "raw": "{{baseUrl}}/api/users/not-a-valid-id!" }
+  },
+  "event": [{
+    "listen": "test",
+    "script": {
+      "exec": [
+        "pm.test('Returns 400 or 404 for invalid ID format', function () {",
+        "    pm.expect(pm.response.code).to.be.oneOf([400, 404]);",
+        "});"
+      ]
+    }
+  }]
+}
+```
+
+#### 3f. Duplicate / Conflict
+
+```json
+{
+  "name": "Negative - Create User - Duplicate Email",
+  "event": [
+    {
+      "listen": "prerequest",
+      "script": {
+        "exec": [
+          "// Uses the same email as the previously created user"
+        ]
+      }
+    },
+    {
+      "listen": "test",
+      "script": {
+        "exec": [
+          "pm.test('Returns 409 for duplicate email', function () {",
+          "    pm.response.to.have.status(409);",
+          "});",
+          "pm.test('Error indicates duplicate resource', function () {",
+          "    const response = pm.response.json();",
+          "    const msg = (response.message || '').toLowerCase();",
+          "    pm.expect(msg).to.satisfy(function(m) {",
+          "        return m.includes('already exists') || m.includes('duplicate') || m.includes('conflict');",
+          "    });",
+          "});"
+        ]
+      }
+    }
+  ],
+  "request": {
+    "method": "POST",
+    "body": {
+      "mode": "raw",
+      "raw": "{\n  \"name\": \"Another User\",\n  \"email\": \"jane.smith@example.com\",\n  \"password\": \"SecureP@ss123\"\n}"
+    }
+  }
+}
+```
+
+#### 3g. Invalid Content Type
+
+```json
+{
+  "name": "Negative - Create User - Wrong Content-Type",
+  "request": {
+    "method": "POST",
+    "header": [
+      { "key": "Content-Type", "value": "text/plain" },
+      { "key": "Authorization", "value": "Bearer {{authToken}}" }
+    ],
+    "body": {
+      "mode": "raw",
+      "raw": "this is not json"
+    }
+  },
+  "event": [{
+    "listen": "test",
+    "script": {
+      "exec": [
+        "pm.test('Returns 400 or 415 for wrong content type', function () {",
+        "    pm.expect(pm.response.code).to.be.oneOf([400, 415]);",
+        "});"
+      ]
+    }
+  }]
+}
+```
+
+#### 3h. Malformed Request Body
+
+```json
+{
+  "name": "Negative - Create User - Malformed JSON",
+  "request": {
+    "method": "POST",
+    "header": [
+      { "key": "Content-Type", "value": "application/json" },
+      { "key": "Authorization", "value": "Bearer {{authToken}}" }
+    ],
+    "body": {
+      "mode": "raw",
+      "raw": "{invalid json content"
+    }
+  },
+  "event": [{
+    "listen": "test",
+    "script": {
+      "exec": [
+        "pm.test('Returns 400 for malformed JSON', function () {",
+        "    pm.response.to.have.status(400);",
+        "});"
+      ]
+    }
+  }]
+}
+```
+
+### Step 4 — Write Test Scripts
+
+Embed `pm.test()` blocks in the `event` array of each request item:
+
+```json
+{
+  "event": [
+    {
+      "listen": "test",
+      "script": {
+        "type": "text/javascript",
+        "exec": [
+          "// line 1",
+          "// line 2"
+        ]
+      }
+    }
+  ]
+}
+```
+
+Each `exec` array entry is a line of JavaScript. Keep lines short and readable.
+
+### Step 5 — Organize into Folders
+
+Structure the collection with clear separation:
+
+```
+Test Suite Root
+├── Setup
+│   ├── Register Test User
+│   └── Login (capture token)
+├── Users
+│   ├── Positive Tests
+│   │   ├── Create User - Valid Data
+│   │   ├── List Users - Default Pagination
+│   │   ├── List Users - Custom Page Size
+│   │   ├── Get User by ID
+│   │   ├── Update User - Full Update
+│   │   ├── Update User - Partial Update
+│   │   └── Delete User
+│   └── Negative Tests
+│       ├── Create User - Missing Name
+│       ├── Create User - Missing Email
+│       ├── Create User - Invalid Email Format
+│       ├── Create User - Short Password
+│       ├── Create User - Duplicate Email
+│       ├── Get User - Non-Existent ID
+│       ├── Get User - Invalid ID Format
+│       ├── Update User - Not Found
+│       ├── Delete User - Not Found
+│       ├── Delete User - Insufficient Permissions
+│       ├── Access Without Auth Token
+│       ├── Access With Invalid Token
+│       ├── Create User - Malformed JSON
+│       └── Create User - Wrong Content-Type
+├── Products
+│   ├── Positive Tests
+│   │   └── ...
+│   └── Negative Tests
+│       └── ...
+└── Teardown
+    └── Delete Test User
+```
+
+### Step 6 — Add Data-Driven Tests
+
+For endpoints with many validation scenarios, suggest data file usage:
+
+**CSV data file format (`test_data_users.csv`):**
+```csv
+name,email,password,expectedStatus,testDescription
+,jane@example.com,SecureP@ss123,400,Missing name
+Jane Smith,,SecureP@ss123,400,Missing email
+Jane Smith,not-an-email,SecureP@ss123,400,Invalid email format
+Jane Smith,jane@example.com,short,400,Password too short
+Jane Smith,jane@example.com,SecureP@ss123,201,Valid data
+```
+
+**Data-driven test script:**
+```javascript
+pm.test(pm.iterationData.get("testDescription"), function () {
+    pm.response.to.have.status(parseInt(pm.iterationData.get("expectedStatus")));
+});
+```
+
+Include a note in the collection description explaining how to use data files with Postman Runner.
+
+### Step 7 — Write File
+
+Output as: `postman_test_collection.json` at project root (or user-specified path).
+
+The file follows the same Postman Collection v2.1 schema as the base collection, with test scripts embedded in each request's `event` array.
+
+### Step 8 — Verify
+
+Before claiming the test suite is complete:
+
+1. **Coverage check** — every endpoint has at least one positive and multiple negative scenarios
+2. **JSON validity** — file parses without errors
+3. **Script syntax** — all `pm.test()` blocks are syntactically correct JavaScript
+4. **Variable consistency** — all `{{variables}}` used in tests are defined or captured by prior requests
+5. **Ordering** — setup (auth) runs before tests that need tokens; create runs before get-by-id
+6. Apply `digiqagent:verification-before-completion` — evidence before claims
+
+## Negative Scenario Matrix
+
+Use this matrix to ensure systematic coverage. Every cell should have at least one test:
+
+| Scenario Category | POST (Create) | GET (Read) | PUT (Update) | DELETE |
+|-------------------|--------------|------------|-------------|--------|
+| Missing required fields | One test per field | N/A | One test per field | N/A |
+| Invalid data types | Per typed field | N/A | Per typed field | N/A |
+| Empty/blank values | Per string field | N/A | Per string field | N/A |
+| Boundary values | Per constrained field | N/A | Per constrained field | N/A |
+| Invalid format | Email, date, UUID | Invalid ID format | Email, date, UUID | Invalid ID format |
+| No auth token | Yes | Yes | Yes | Yes |
+| Invalid auth token | Yes | Yes | Yes | Yes |
+| Insufficient permissions | If role-based | If role-based | If role-based | If role-based |
+| Resource not found | N/A | Yes | Yes | Yes |
+| Duplicate/conflict | If unique constraints | N/A | If unique constraints | N/A |
+| Wrong content type | Yes | N/A | Yes | N/A |
+| Malformed body | Yes | N/A | Yes | N/A |
+
+## Test Assertion Quick Reference
+
+| What to Assert | pm.test Code |
+|---------------|-------------|
+| Status code | `pm.response.to.have.status(200)` |
+| Status code one of | `pm.expect(pm.response.code).to.be.oneOf([200, 201])` |
+| Response time | `pm.expect(pm.response.responseTime).to.be.below(2000)` |
+| Has property | `pm.expect(json).to.have.property("id")` |
+| Property type | `pm.expect(json.id).to.be.a("string")` |
+| Property value | `pm.expect(json.status).to.eql("active")` |
+| Array length | `pm.expect(json.data).to.have.lengthOf.at.least(1)` |
+| Array type | `pm.expect(json.data).to.be.an("array")` |
+| String contains | `pm.expect(json.message).to.include("not found")` |
+| Header exists | `pm.response.to.have.header("Content-Type")` |
+| Header value | `pm.expect(pm.response.headers.get("Content-Type")).to.include("json")` |
+| Not empty | `pm.expect(json.id).to.not.be.empty` |
+| Null check | `pm.expect(json.deletedAt).to.be.null` |
+| Nested property | `pm.expect(json.user.profile.name).to.eql("Jane")` |
+| JSON Schema | `pm.response.to.have.jsonSchema(schema)` |
+| Set variable | `pm.environment.set("key", json.value)` |
+| Get variable | `pm.environment.get("key")` |
+
+## Red Flags — STOP and Fix
+
+| Problem | Fix |
+|---------|-----|
+| Endpoint has only positive tests | Add negative scenarios from the matrix |
+| Negative test doesn't check error message | Add assertion on error body content |
+| Tests depend on hardcoded IDs | Use environment variables from setup requests |
+| No setup folder for auth | Add login request at the start |
+| Test scripts have syntax errors | Validate JavaScript before writing file |
+| Missing coverage for auth scenarios | Add no-token, invalid-token, wrong-role tests |
+| All negative tests in one request | Separate: one scenario per request for clear reporting |
+| Test names are generic | Use descriptive names: "Returns 400 when email is missing" |
+
+## Verification Checklist
+
+Before marking generation complete:
+
+- [ ] Every endpoint has at least one positive test scenario
+- [ ] Every mutating endpoint (POST/PUT/PATCH) has missing-field tests
+- [ ] Every authenticated endpoint has no-auth and invalid-auth tests
+- [ ] Every endpoint with path params has not-found and invalid-ID tests
+- [ ] Tests organized into Positive/Negative folders per resource
+- [ ] Setup folder with auth requests runs first
+- [ ] All `pm.test()` scripts are syntactically valid JavaScript
+- [ ] Environment variables are captured before they're used
+- [ ] Collection JSON is valid and imports into Postman
+- [ ] File written and path confirmed to user
+
+Can't check all boxes? Fix the test suite. Incomplete negative coverage means unknown API behavior.
+
+## Common Patterns
+
+See @test-scenario-patterns.md for:
+- Complete CRUD test suite example
+- Auth test patterns
+- Schema validation patterns
+- Chained test patterns
+- Idempotency and rate limiting tests
+
+## Developer Usage
+
+Include in the collection description:
+
+```
+## Running the Test Suite
+
+1. Import `postman_test_collection.json` into Postman
+2. Import `postman_environment.json` (from collection generator)
+3. Select the environment in the top-right dropdown
+4. Ensure your API server is running at the configured baseUrl
+5. Run the "Setup" folder first to populate auth tokens
+6. Run individual folders or the entire collection via Postman Runner
+
+## Understanding Results
+- Green: Test passed — API behaves as expected
+- Red: Test failed — investigate the endpoint
+- Positive test failures: API is broken
+- Negative test failures: API may be missing validation
+
+## Using Data-Driven Tests
+1. Open Postman Runner
+2. Select a data file (CSV/JSON) for parameterized scenarios
+3. Run — each row becomes a separate test iteration
+```