@testsmith/api-spector 0.6.0 → 0.6.2

package/docs/ai/api-spector-scripting-reference.md

@@ -0,0 +1,149 @@
+# API Spector: Scripting Reference
+
+This document describes the scripting API available in API Spector's pre-request and post-request scripts. Use this as a reference when generating test scripts.
+
+## Script Context
+
+Scripts run in a sandboxed environment. The global `sp` object provides all testing, assertion, and variable functionality.
+
+## Assertions
+
+### sp.test(name, fn)
+
+Wrap assertions in a named test block:
+
+```javascript
+sp.test('status is 200', function() {
+  sp.expect(sp.response.code).to.equal(200);
+});
+```
+
+### sp.expect(value)
+
+Returns a chainable asserter. Available matchers:
+
+| Matcher | Example | Description |
+|---|---|---|
+| `.to.equal(v)` | `sp.expect(json.name).to.equal("Alice")` | Strict equality |
+| `.to.eql(v)` | `sp.expect(json).to.eql({id: 1})` | Deep equality |
+| `.to.include(v)` | `sp.expect(json.name).to.include("Ali")` | String/array contains |
+| `.to.be.a(type)` | `sp.expect(json.age).to.be.a("number")` | Type check |
+| `.to.be.above(n)` | `sp.expect(json.count).to.be.above(0)` | Greater than |
+| `.to.be.below(n)` | `sp.expect(json.count).to.be.below(100)` | Less than |
+| `.to.be.least(n)` | `sp.expect(json.count).to.be.least(1)` | >= |
+| `.to.be.most(n)` | `sp.expect(json.count).to.be.most(50)` | <= |
+| `.to.have.property(k)` | `sp.expect(json).to.have.property("id")` | Key exists |
+| `.to.have.lengthOf(n)` | `sp.expect(json.items).to.have.lengthOf(3)` | Array/string length |
+| `.to.match(re)` | `sp.expect(json.email).to.match(/^.+@.+$/)` | Regex match |
+| `.to.be.oneOf([...])` | `sp.expect(json.role).to.be.oneOf(["admin","user"])` | Value in list |
+| `.to.not.xxx` | `sp.expect(json.id).to.not.be.oneOf([null, undefined])` | Negation |
+
+Chainable words (no-op, for readability): `.to`, `.be`, `.been`, `.have`, `.that`, `.and`, `.is`, `.deep`
+
+Flag assertions (property-style): `.ok`, `.true`, `.false`, `.null`, `.undefined`
+
+## Response Object
+
+Available in post-request scripts when a response has been received:
+
+```javascript
+sp.response.code           // HTTP status code (number)
+sp.response.status         // "200 OK" (string)
+sp.response.statusText     // "OK" (string)
+sp.response.responseTime   // Duration in ms
+sp.response.responseSize   // Body size in bytes
+sp.response.json()         // Parse body as JSON (cached)
+sp.response.text()         // Raw body as string
+sp.response.xmlText(sel)   // Extract text from XML/HTML using CSS selector
+sp.response.headers.get(n) // Get response header by name
+sp.response.headers.toObject() // All headers as {key: value}
+sp.response.to.have.status(code) // Assert status code
+```
+
+## Variables
+
+Four variable scopes, from most-local to most-global:
+
+```javascript
+// Local variables (per-request, not persisted)
+sp.variables.get("key")
+sp.variables.set("key", "value")
+sp.variables.has("key")
+sp.variables.clear("key")
+
+// Environment variables (persisted to the active environment)
+sp.environment.get("key")
+sp.environment.set("key", "value")
+
+// Collection variables (persisted to the collection)
+sp.collectionVariables.get("key")
+sp.collectionVariables.set("key", "value")
+
+// Global variables (persisted across collections)
+sp.globals.get("key")
+sp.globals.set("key", "value")
+
+// Shorthand: read from all scopes (local wins)
+sp.variables_get("key")
+```
+
+### Variable interpolation in scripts
+
+Scripts support `{{variable}}` interpolation. Before execution, `{{var}}` tokens are replaced with the variable's value:
+
+```javascript
+sp.variables.set("otp", sp.totp("{{secret}}"));
+```
+
+## JSONPath
+
+Query JSON data with JSONPath expressions:
+
+```javascript
+const matches = sp.jsonPath(sp.response.json(), '$.users[?(@.role=="admin")].name');
+sp.expect(matches.length).to.be.above(0);
+sp.expect(matches[0]).to.equal("Alice");
+```
+
+## TOTP (Time-based One-Time Password)
+
+Generate TOTP codes from a base32 secret:
+
+```javascript
+const code = sp.totp("JBSWY3DPEHPK3PXP");
+sp.environment.set("otp", code);
+
+// With options:
+const code8 = sp.totp("JBSWY3DPEHPK3PXP", { digits: 8, period: 30, algorithm: "sha1" });
+```
+
+## Available globals
+
+Scripts have access to: `JSON`, `Math`, `Date`, `parseInt`, `parseFloat`, `isNaN`, `isFinite`, `encodeURIComponent`, `decodeURIComponent`, `btoa`, `atob`, `console` (log/warn/error/info), `dayjs` (date library), `faker` (test data generation), `tv4` (JSON Schema validation), `DOMParser` (XML parsing).
+
+## Hooks
+
+Requests can be designated as lifecycle hooks within their folder:
+
+| Hook type | When it runs |
+|---|---|
+| `beforeAll` | Once before any request in the folder |
+| `before` | Before each request in the folder |
+| `after` | After each request in the folder |
+| `afterAll` | Once after all requests in the folder |
+
+Hooks from parent folders wrap child folders (outer-to-inner for before, inner-to-outer for after).
+
+A common pattern is a login hook:
+
+```javascript
+// Post-request script on a beforeAll hook that calls POST /login:
+const json = sp.response.json();
+sp.environment.set("access_token", json.access_token);
+```
+
+Then other requests use `{{access_token}}` in their Bearer token auth field.
+
+## Schema validation
+
+The `request.schema` field (JSON Schema) is automatically validated against every response. Failed validations appear as `[schema]` test results. The schema uses standard JSON Schema draft-07+ syntax.

package/docs/ai/collection-file-format.md

@@ -0,0 +1,110 @@
+# API Spector: Collection File Format
+
+Collections are stored as `.spector` JSON files. This document describes the format so AI tools can read and modify collections programmatically.
+
+## File structure
+
+```json
+{
+  "version": "1.0",
+  "id": "uuid",
+  "name": "My API",
+  "description": "Optional description",
+  "rootFolder": { ... },
+  "requests": { "request-id": { ... }, ... },
+  "collectionVariables": { "key": "value" },
+  "auth": { "type": "bearer", "token": "{{access_token}}" },
+  "headers": [{ "key": "Accept", "value": "application/json", "enabled": true }]
+}
+```
+
+## Request object
+
+```json
+{
+  "id": "unique-id",
+  "name": "Get Users",
+  "method": "GET",
+  "url": "{{baseUrl}}/users",
+  "headers": [
+    { "key": "Accept", "value": "application/json", "enabled": true }
+  ],
+  "params": [
+    { "key": "page", "value": "1", "enabled": true, "paramType": "query" },
+    { "key": "id", "value": "42", "enabled": true, "paramType": "path" }
+  ],
+  "auth": { "type": "bearer", "token": "{{access_token}}" },
+  "body": {
+    "mode": "json",
+    "json": "{ \"name\": \"Alice\" }"
+  },
+  "preRequestScript": "sp.environment.set('otp', sp.totp('{{secret}}'));",
+  "postRequestScript": "sp.test('status', function() { ... });",
+  "schema": "{ \"type\": \"object\", ... }",
+  "hookType": "beforeAll",
+  "disabled": false
+}
+```
+
+### Auth types
+
+```json
+{ "type": "none" }
+{ "type": "bearer", "token": "{{access_token}}" }
+{ "type": "basic", "username": "admin", "password": "secret" }
+{ "type": "apikey", "apiKeyName": "X-API-Key", "apiKeyValue": "{{key}}", "apiKeyIn": "header" }
+```
+
+### Body modes
+
+- `"none"`: no body
+- `"json"`: `body.json` contains the JSON string
+- `"form"`: `body.form` is an array of `{ key, value, enabled }` pairs
+- `"raw"`: `body.raw` contains raw text, `body.rawContentType` is the MIME type
+- `"graphql"`: `body.graphql` has `{ query, variables, operationName }`
+
+## Folder structure
+
+```json
+{
+  "id": "folder-id",
+  "name": "Users",
+  "description": "",
+  "folders": [ ... ],
+  "requestIds": ["req-1", "req-2"],
+  "auth": { "type": "bearer", "token": "{{admin_token}}" },
+  "headers": [{ "key": "X-Tenant", "value": "{{tenant}}", "enabled": true }],
+  "tags": ["smoke"]
+}
+```
+
+Folders can nest. Each folder can define auth and headers that are inherited by all requests within it (overridable at the request level). The root folder is synthetic (name "root"); its subfolders are the visible top-level groups.
+
+## Environment files
+
+Stored as `.env.json`:
+
+```json
+{
+  "version": "1.0",
+  "id": "uuid",
+  "name": "Staging",
+  "variables": [
+    { "key": "baseUrl", "value": "https://staging.api.com", "enabled": true },
+    { "key": "admin_password", "value": "", "enabled": true, "secret": true }
+  ]
+}
+```
+
+## Workspace file
+
+`workspace.json` ties collections and environments together:
+
+```json
+{
+  "version": "1.0",
+  "collections": ["collections/users.spector", "collections/orders.spector"],
+  "environments": ["environments/staging.env.json"],
+  "activeEnvironmentId": "env-uuid"
+}
+```

package/docs/ai/functional-testing-guide.md

@@ -0,0 +1,145 @@
+# API Spector: Functional API Testing Guide
+
+Use this guide to generate functional test cases for REST APIs. Each endpoint should be tested across multiple categories.
+
+## Test categories per endpoint
+
+### 1. Happy path (positive)
+- Send valid data, assert correct status code and response structure
+- Verify all required response fields are present and correctly typed
+
+```javascript
+sp.test('returns 200 with user data', function() {
+  const json = sp.response.json();
+  sp.expect(sp.response.code).to.equal(200);
+  sp.expect(json.id).to.not.be.oneOf([null, undefined]);
+  sp.expect(json.name).to.be.a("string");
+  sp.expect(json.email).to.match(/^.+@.+\..+$/);
+});
+```
+
+### 2. Missing required fields (negative)
+- Omit each required field one at a time
+- Expect 422 or 400 with a descriptive error
+
+```javascript
+sp.test('returns 422 when email is missing', function() {
+  sp.expect(sp.response.code).to.equal(422);
+  const json = sp.response.json();
+  sp.expect(JSON.stringify(json)).to.include("email");
+});
+```
+
+### 3. Wrong types (negative)
+- Send a string where a number is expected, an object where a string is expected, etc.
+- Expect 422 or 400
+
+### 4. Boundary values
+- Empty strings for string fields
+- 0, -1, MAX_INT for numeric fields
+- Very long strings (1000+ chars)
+- Special characters: `<script>alert(1)</script>`, `'; DROP TABLE--`, unicode
+
+### 5. Authentication
+- Missing auth header → 401
+- Invalid/expired token → 401
+- Valid token, wrong role → 403
+
+### 6. Resource not found
+- Request a non-existent ID → 404
+- Request with a malformed ID format → 400 or 404
+
+### 7. List/pagination endpoints
+- Default pagination returns reasonable count
+- `page` and `limit` params work correctly
+- Out-of-range page returns empty list (not error)
+- Sorting works if supported
+
+### 8. Idempotency
+- PUT with same data twice → same result
+- DELETE then DELETE again → 404 on second call
+
+## Test naming convention
+
+Use descriptive names that include the endpoint and expected behavior:
+
+```javascript
+sp.test('POST /users - valid data returns 201', function() { ... });
+sp.test('POST /users - missing email returns 422', function() { ... });
+sp.test('GET /users/:id - non-existent ID returns 404', function() { ... });
+sp.test('GET /users - unauthorized returns 401', function() { ... });
+```
+
+## Variable extraction pattern
+
+When a test creates a resource, extract its ID for subsequent tests:
+
+```javascript
+// Post-request script on POST /users
+const json = sp.response.json();
+sp.environment.set("userId", String(json.id));
+
+// Then GET /users/{{userId}} in a subsequent request uses the created ID
+```
+
+## Hook pattern for auth
+
+Create a `beforeAll` hook that logs in and stores the token:
+
+```javascript
+// Post-request script on the login hook (POST /auth/login)
+const json = sp.response.json();
+sp.environment.set("access_token", json.access_token);
+```
+
+Then configure the folder/collection auth as Bearer with token `{{access_token}}`.
+
+## Test plan structure
+
+When generating a test plan, organize by endpoint and include:
+
+1. **Endpoint**: method + path
+2. **Test cases**: list of scenarios with:
+   - Name (descriptive)
+   - Category (positive/negative/boundary/auth/edge)
+   - Request body (if applicable)
+   - Expected status code
+   - Assertions to make
+3. **Dependencies**: which tests depend on data from earlier tests
+4. **Hooks needed**: login, setup data, cleanup
+
+## Example test plan output
+
+```
+## POST /users
+
+### Positive
+- [x] Valid user creation → 201, response has id, name, email
+- [x] Create user with all optional fields → 201
+
+### Negative
+- [x] Missing name → 422, error mentions "name"
+- [x] Missing email → 422, error mentions "email"
+- [x] Invalid email format → 422
+- [x] Duplicate email → 409
+
+### Boundary
+- [x] Name with 255 characters → 201 (or 422 if max exceeded)
+- [x] Empty name → 422
+- [x] Price = 0 → 201 or 422 depending on business rule
+- [x] Price = -1 → 422
+
+### Auth
+- [x] No token → 401
+- [x] Expired token → 401
+
+## GET /users
+
+### Positive
+- [x] List users → 200, returns array
+- [x] Filter by name → 200, filtered results
+
+### Pagination
+- [x] ?page=1&limit=10 → 200, max 10 results
+- [x] ?page=9999 → 200, empty array
+```

package/docs/ai/security-testing-guide.md

@@ -0,0 +1,253 @@
+# API Spector: Security API Testing Guide (OWASP API Top 10)
+
+Use this guide to generate security test cases based on the OWASP API Security Top 10 (2023). Each test case should use the API Spector `sp.*` scripting API.
+
+## Prerequisites
+
+Security tests typically require multiple user identities:
+- **User A** (attacker): a regular user trying to access resources they shouldn't
+- **User B** (victim): the owner of the resources being targeted
+- **Admin**: for testing admin-only endpoints with regular credentials
+
+Store tokens in environment variables: `{{user_a_token}}`, `{{user_b_token}}`, `{{admin_token}}`
+
+## API1:2023 - Broken Object-Level Authorization (BOLA)
+
+**What to test**: Can User A access User B's resources by changing the ID?
+
+```javascript
+// Request: GET /users/{{user_b_id}} with User A's token
+sp.test('[BOLA] cannot access other user profile', function() {
+  sp.expect(sp.response.code).to.be.oneOf([403, 404]);
+});
+
+// Request: PUT /users/{{user_b_id}} with User A's token
+sp.test('[BOLA] cannot modify other user profile', function() {
+  sp.expect(sp.response.code).to.be.oneOf([403, 404]);
+});
+
+// Request: DELETE /orders/{{user_b_order_id}} with User A's token
+sp.test('[BOLA] cannot delete other user order', function() {
+  sp.expect(sp.response.code).to.be.oneOf([403, 404]);
+});
+```
+
+**For each endpoint with an ID parameter**: create a request where User A tries to access User B's resource.
+
+## API2:2023 - Broken Authentication
+
+**What to test**: Authentication bypass, weak tokens, missing validation.
+
+```javascript
+// Request with no Authorization header
+sp.test('[AUTH] missing token returns 401', function() {
+  sp.expect(sp.response.code).to.equal(401);
+});
+
+// Request with an invalid/garbage token
+sp.test('[AUTH] invalid token returns 401', function() {
+  sp.expect(sp.response.code).to.equal(401);
+});
+
+// Request with expired token
+sp.test('[AUTH] expired token returns 401', function() {
+  sp.expect(sp.response.code).to.equal(401);
+});
+```
+
+## API3:2023 - Broken Object Property-Level Authorization
+
+**What to test**: Mass assignment. Can a user set fields they shouldn't?
+
+```javascript
+// Request: PUT /users/{{user_a_id}} with body including role: "admin"
+sp.test('[MASS-ASSIGN] cannot escalate own role', function() {
+  sp.expect(sp.response.code).to.be.oneOf([200, 422]);
+  // If 200, verify the role wasn't actually changed
+  const json = sp.response.json();
+  if (sp.response.code === 200) {
+    sp.expect(json.role).to.not.equal("admin");
+  }
+});
+
+// Request: POST /users with is_admin: true
+sp.test('[MASS-ASSIGN] cannot create admin user', function() {
+  const json = sp.response.json();
+  if (json.is_admin !== undefined) {
+    sp.expect(json.is_admin).to.equal(false);
+  }
+});
+```
+
+**For each write endpoint**: include fields that shouldn't be user-settable (role, is_admin, balance, permissions).
+
+## API4:2023 - Unrestricted Resource Consumption
+
+**What to test**: Rate limits, pagination abuse, large payloads.
+
+```javascript
+// Request: GET /users?limit=999999
+sp.test('[RESOURCE] excessive limit is capped or rejected', function() {
+  sp.expect(sp.response.code).to.be.oneOf([200, 400, 422]);
+  if (sp.response.code === 200) {
+    const json = sp.response.json();
+    const items = Array.isArray(json) ? json : json.data ?? [];
+    sp.expect(items.length).to.be.most(100);
+  }
+});
+
+// Request: POST /upload with an extremely large body
+sp.test('[RESOURCE] rejects oversized payload', function() {
+  sp.expect(sp.response.code).to.be.oneOf([413, 400, 422]);
+});
+```
+
+## API5:2023 - Broken Function-Level Authorization
+
+**What to test**: Admin endpoints accessible by regular users.
+
+```javascript
+// Request: GET /admin/users with regular user token
+sp.test('[FUNC-AUTH] admin endpoint rejects regular user', function() {
+  sp.expect(sp.response.code).to.be.oneOf([403, 404]);
+});
+
+// Request: DELETE /users/{{user_b_id}} with regular user token
+sp.test('[FUNC-AUTH] user cannot delete other users', function() {
+  sp.expect(sp.response.code).to.be.oneOf([403, 404]);
+});
+```
+
+**For each endpoint**: determine the expected minimum role and test with a lower-privilege token.
+
+## API6:2023 - Unrestricted Access to Sensitive Business Flows
+
+**What to test**: Business logic abuse.
+
+```javascript
+// Request: POST /orders (repeat the same order)
+sp.test('[BIZ-FLOW] duplicate order is rejected', function() {
+  // Second call with same data should not create a duplicate
+  sp.expect(sp.response.code).to.be.oneOf([409, 422, 429]);
+});
+```
+
+## API7:2023 - Server-Side Request Forgery (SSRF)
+
+**What to test**: URL parameters that the server fetches.
+
+```javascript
+// Request with URL field pointing to internal metadata
+// e.g. POST /import with { "url": "http://169.254.169.254/latest/meta-data/" }
+sp.test('[SSRF] internal URL is blocked', function() {
+  sp.expect(sp.response.code).to.be.oneOf([400, 403, 422]);
+});
+
+// Same with http://localhost, http://127.0.0.1, http://[::1]
+```
+
+**For each endpoint that accepts a URL parameter**: test with internal addresses.
+
+## API8:2023 - Security Misconfiguration
+
+**What to test**: Headers, error verbosity, CORS.
+
+```javascript
+// Any endpoint: check security headers
+sp.test('[CONFIG] has security headers', function() {
+  sp.expect(sp.response.headers.get('x-content-type-options')).to.equal('nosniff');
+  // Optional: check for other headers
+  // sp.expect(sp.response.headers.get('x-frame-options')).to.not.equal(null);
+  // sp.expect(sp.response.headers.get('strict-transport-security')).to.not.equal(null);
+});
+
+// Invalid endpoint: check that error doesn't leak stack traces
+sp.test('[CONFIG] error response does not expose stack trace', function() {
+  const body = sp.response.text();
+  sp.expect(body).to.not.include("at Object.");
+  sp.expect(body).to.not.include("node_modules");
+  sp.expect(body).to.not.include("Exception in thread");
+});
+
+// OPTIONS request: check CORS
+sp.test('[CONFIG] CORS does not allow *', function() {
+  const allow = sp.response.headers.get('access-control-allow-origin');
+  if (allow) {
+    sp.expect(allow).to.not.equal('*');
+  }
+});
+```
+
+## API9:2023 - Improper Inventory Management
+
+**What to test**: Deprecated or undocumented endpoints still accessible.
+
+This is best tested by comparing the spec against actual behavior:
+- Hit known deprecated endpoints → should return 404 or 410
+- Hit versioned paths (v1 when v2 is current) → should be blocked or redirected
+
+## API10:2023 - Unsafe Consumption of APIs
+
+**What to test**: Injection through fields that the API passes downstream.
+
+```javascript
+// SQL injection in search/filter parameters
+// Request: GET /users?search=' OR '1'='1
+sp.test('[INJECTION] SQL injection in search returns 400 or safe results', function() {
+  sp.expect(sp.response.code).to.be.oneOf([200, 400, 422]);
+  if (sp.response.code === 200) {
+    const json = sp.response.json();
+    const items = Array.isArray(json) ? json : json.data ?? [];
+    sp.expect(items.length).to.be.below(50); // should not dump entire table
+  }
+});
+
+// XSS payload in input fields
+// Request: POST /users with name: "<script>alert(1)</script>"
+sp.test('[INJECTION] XSS payload is sanitized or rejected', function() {
+  sp.expect(sp.response.code).to.be.oneOf([201, 200, 400, 422]);
+  if (sp.response.code <= 201) {
+    const json = sp.response.json();
+    sp.expect(json.name).to.not.include("<script>");
+  }
+});
+
+// NoSQL injection
+// Request: POST /login with { "email": {"$gt": ""}, "password": {"$gt": ""} }
+sp.test('[INJECTION] NoSQL injection is rejected', function() {
+  sp.expect(sp.response.code).to.be.oneOf([400, 401, 422]);
+});
+```
+
+## Test plan structure
+
+When generating security tests, organize by OWASP category:
+
+```
+Security/
+├── BOLA/
+│   ├── [BOLA] GET /users/:id - access other user
+│   ├── [BOLA] PUT /orders/:id - modify other user order
+│   └── ...
+├── Auth/
+│   ├── [AUTH] No token
+│   ├── [AUTH] Invalid token
+│   └── [AUTH] Expired token
+├── Mass Assignment/
+│   ├── [MASS-ASSIGN] Escalate role via PUT /users
+│   └── ...
+├── Injection/
+│   ├── [INJECTION] SQL in search param
+│   ├── [INJECTION] XSS in name field
+│   └── ...
+└── Config/
+    ├── [CONFIG] Security headers
+    ├── [CONFIG] Error verbosity
+    └── [CONFIG] CORS policy
+```
+
+Each request should:
+1. Have a descriptive name prefixed with the OWASP category
+2. Be pre-filled with the attack payload in the body/params/headers
+3. Have a post-request script that asserts the expected safe behavior
+4. Use the attacker's token (User A) unless testing missing-auth scenarios

package/out/main/agents.js

@@ -2,7 +2,8 @@
 "use strict";
 const promises = require("fs/promises");
 const path = require("path");
-const cliCommon = require("./chunks/cli-common-BKYgsbGB.js");
+const cliCommon = require("./chunks/cli-common-vASoDfQH.js");
+require("./chunks/data-files-Br2h3Dt2.js");
 require("crypto");
 const AGENTS = {
   claude: {