@luckystack/test-runner 0.2.7 → 0.3.0

package/docs/auth-tests.md

@@ -2,7 +2,7 @@
 
 The auth-enforcement layer answers one question per endpoint: "If a logged-out caller hits this route, does the framework reject them?" It is the cheapest insurance policy against a `_api/*` file that forgets to set `auth.login: true`, an `auth.login` guard that was silently swapped to `false` during a refactor, or a middleware change in `@luckystack/api` that stops short-circuiting unauthenticated requests.
 
-Pass criterion: the endpoint returns `{ status: 'error', errorCode: 'auth.required' }`. Anything else — `success`, a different `errorCode`, a missing envelope, a network error — is a failure.
+Pass criterion: the endpoint returns `{ status: 'error', errorCode }` where `errorCode` is one of the auth-rejection codes — `auth.required` (an `auth.login: true` route) or `auth.forbidden` (a route guarded only by `auth.additional[]` predicates). Anything else — `success`, a different `errorCode`, a missing envelope, a network error — is a failure.
 
 ## Functions
 
@@ -32,14 +32,14 @@ Flow:
 5. Branch on `parsed.status`:
    - `'success'` -> fail with `reason: 'auth.login endpoint returned success without a session'`. This is the headline case: the endpoint is open when it should be closed.
    - Not `'error'` (missing, weird) -> fail with `'Response missing standard `status` envelope'`.
-   - `'error'` with `errorCode !== 'auth.required'` -> fail with the expected/actual codes inline in the `reason`. A different error code (e.g. validation rejecting `{}`) means the auth guard never ran — the request was killed earlier in the pipeline.
-   - `'error'` with `errorCode === 'auth.required'` -> pass.
+   - `'error'` with an `errorCode` outside the auth-rejection set (`auth.required` | `auth.forbidden`) -> fail with the expected/actual codes inline in the `reason` (`expected an auth-rejection errorCode (auth.required | auth.forbidden) but got '<actual>'`). A different error code (e.g. validation rejecting `{}`) means the auth guard never ran — the request was killed earlier in the pipeline.
+   - `'error'` with `errorCode === 'auth.required'` OR `errorCode === 'auth.forbidden'` -> pass.
 
-The canonical error code is imported from the framework constant `auth.required` (see `packages/api/src/handleHttpApiRequest.ts`). It is intentionally hard-coded in the runner rather than fed from config because the whole point is to detect drift: if `@luckystack/api` ever renames the code, this layer should fail every protected endpoint until consumers acknowledge the change.
+The accepted error codes are the framework's auth-rejection set, hard-coded in the runner as `EXPECTED_ERROR_CODES = new Set(['auth.required', 'auth.forbidden'])` (see `packages/api/src/handleHttpApiRequest.ts` + `packages/core/src/validateRequest.ts`). `auth.required` is what an `auth.login: true` route emits for an anonymous caller; `auth.forbidden` is what an `auth.additional[]`-only route (`login: false`) emits when the predicate rejects the anonymous caller. Either one is a PASS — both mean the guard fired. They are intentionally hard-coded rather than fed from config because the whole point is to detect drift: if `@luckystack/api` ever renames a code, this layer should fail every protected endpoint until consumers acknowledge the change.
 
 ### `runAuthEnforcementTests(input)`
 
-Full sweep. Iterates `walkEndpoints(input.apiMethodMap)` and only checks endpoints where `apiMetaMap[page][name][version].auth.login === true`. Public endpoints (`auth.login: false` or missing meta) are skipped **silently** — they do not appear in the summary at all. This keeps the report focused on what this layer actually covers; public endpoints are already exercised by the contract layer.
+Full sweep. Iterates `walkEndpoints(input.apiMethodMap)` and checks every endpoint that declares an authorization requirement — `apiMetaMap[page][name][version].auth.login === true` OR a non-empty `auth.additional[]` predicate list (`hasAdditional`). Truly-public endpoints (`auth.login: false` AND no `additional[]`, or missing meta) are skipped **silently** — they do not appear in the summary at all. This keeps the report focused on what this layer actually covers; public endpoints are already exercised by the contract layer.
 
 Signature:
 
@@ -66,7 +66,7 @@ Behavior notes:
 
 - **`skip` uses the same two-tier matching as `runContractTests`**: pass `'<page>/<name>'` for version-agnostic skip, or `'<page>/<name>/<version>'` for a specific version.
 - **`onResult` fires only for endpoints that pass `requiresLogin`.** Public endpoints don't trigger it; they don't exist for this layer.
-- **`additional` auth checks (e.g. role-based gates) are NOT covered.** Only the `login: true` flag is read. An endpoint with `auth.login: true, additional: [{ role: 'admin' }]` is asserted by this layer to reject unauthenticated requests with `auth.required`; the role check is out of scope and lives in a custom layer (see `extension-hooks.md`).
+- **Unauthenticated enforcement covers `additional[]`-only routes too.** An endpoint guarded only by `auth.additional[]` predicates (`login: false`) is probed and must reject an anonymous caller with `auth.forbidden`. The DEEPER role/scope *value* assertion (e.g. "this admin-only route rejects a logged-in non-admin") is out of scope and lives in a custom layer (see `extension-hooks.md`).
 
 Returns the same `RunContractSummary` shape as the contract layer; `total` is the number of protected endpoints actually probed (after public-endpoint filtering).
 
@@ -94,12 +94,12 @@ interface RunAuthEnforcementTestsInput {
 
 ## Result shapes by branch
 
-- **Pass**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: 'auth.required', durationMs }`.
+- **Pass (`auth.login` route)**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: 'auth.required', durationMs }`.
+- **Pass (`additional[]`-only route)**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: 'auth.forbidden', durationMs }`.
 - **Fail / open endpoint**: `{ status: 'fail', httpStatus, responseStatus: 'success', reason: 'auth.login endpoint returned success without a session', durationMs }`.
-- **Fail / wrong errorCode**: `{ status: 'fail', httpStatus, responseStatus: 'error', errorCode: '<actual>', reason: "expected errorCode 'auth.required' but got '<actual>'", durationMs }`.
+- **Fail / wrong errorCode**: `{ status: 'fail', httpStatus, responseStatus: 'error', errorCode: '<actual>', reason: "expected an auth-rejection errorCode (auth.required | auth.forbidden) but got '<actual>'", durationMs }`.
 - **Fail / non-envelope**: `{ status: 'fail', httpStatus, responseStatus: 'unknown', reason: 'Response missing standard `status` envelope', durationMs }`.
-- **Fail / JSON parse**: `{ status: 'fail', httpStatus, responseStatus: 'unknown', reason: 'JSON parse failed: ...', durationMs }`.
-- **Fail / fetch error**: `{ status: 'fail', reason: <Error.message>, durationMs }`.
+- **Fail / fetch no response**: `{ status: 'fail', reason: 'fetch returned no response', durationMs }`.
 - **Skip**: `{ status: 'skipped', durationMs: 0, reason: 'Explicitly skipped' }`.
 
 ## Examples
@@ -187,8 +187,8 @@ Prefer fixing the meta over skipping — a wrong `auth.login` flag breaks runtim
 ## Edge cases and gotchas
 
 - **CSRF.** The framework enforces CSRF on state-changing requests. The runner sends no `X-CSRF-Token`, so for a brief window the request can be rejected on CSRF grounds before reaching auth. The current framework returns `auth.required` ahead of CSRF for unauthenticated sessions, so this layer passes; if you reorder middleware to CSRF-first, this layer will need updating.
-- **`additional` auth checks** are not covered here. Put role/scope checks in a `registerTestLayer({ name: 'role-enforcement', run })` plug-in (see `extension-hooks.md`).
-- **Endpoints absent from `apiMetaMap`.** The runner reads `apiMetaMap[page]?.[name]?.[version]?.auth.login`. If any link in that chain is missing, `requiresLogin` returns `false` and the endpoint is silently skipped. Regenerate the meta map if your sweep reports `total: 0`.
+- **Deeper `additional` role/scope *value* checks** are not covered here (the layer only asserts an anonymous caller is rejected with `auth.forbidden`). Put authenticated-but-wrong-role checks in a `registerTestLayer({ name: 'role-enforcement', run })` plug-in (see `extension-hooks.md`).
+- **Endpoints absent from `apiMetaMap`.** The runner reads `apiMetaMap[page]?.[name]?.[version]?.auth`. If any link in that chain is missing, `hasAuthRequirement` returns `false` and the endpoint is silently skipped. Regenerate the meta map if your sweep reports `total: 0`.
 - **Session-cookie names are project-specific.** This layer never sets one, so the name doesn't matter to the runner. It does matter for the auth layer's mirror image: a custom "logged-in sweep" you might write to confirm protected endpoints work *with* a valid session.
 - **Wrong errorCode is a real signal.** When you see `expected 'auth.required' but got 'auth.sessionExpired'`, the framework is detecting and reporting the missing session — just with a stricter code. Either align the runner expectation (fork or PR) or align the framework code. Do not skip.
 - **`@luckystack/login` session-token interaction.** Sessions live in Redis under `${projectName}-session:*` keys. The runner does not touch them. If a logged-in regression suite must run before this layer, use `resetServerState()` to clear sessions and rate limits in one shot (see `rate-limit-tests.md`).

package/docs/extension-hooks.md

@@ -29,7 +29,7 @@ Output format:
 - Pass success: `[PASS] POST /api/billing/getInvoice/v1 12ms http=200`
 - Pass typed error: `[PASS] POST /api/auth/login/v1 8ms http=200 (error: auth.required)`
 - Skip: `[SKIP] POST /api/uploads/fileUpload/v1 0ms Explicitly skipped`
-- Fail: `[FAIL] POST /api/billing/cancelSubscription/v1 14ms http=500 reason=fuzz payload produced 5xx: ...`
+- Fail: `[FAIL] POST /api/billing/cancelSubscription/v1 14ms http=500 reason=fuzz payload produced non-envelope response: ...`
 
 Status tag is one of `[PASS]`, `[FAIL]`, `[SKIP]`. `http=-` appears when no HTTP status is captured (network error, abort). `durationMs` is always present.
 
@@ -49,7 +49,7 @@ Output:
 Contract tests: 47/50 passed, 2 failed, 1 skipped
 
 Failures:
-  - /api/billing/cancelSubscription/v1: fuzz payload produced 5xx: { nested: ...
+  - /api/billing/cancelSubscription/v1: fuzz payload produced non-envelope response: { nested: ...
   - /api/integrations/stripeWebhook/v1: Response missing standard `status` envelope
 ```
 
@@ -84,7 +84,7 @@ interface TestLayerResult {
 
 Replace-by-name: registering a layer with a name that already exists overwrites the previous entry. Use this to update a layer mid-run, e.g. after fetching a fresh auth token.
 
-The runner does **not** currently wire registered layers into the built-in sweeps automatically. Today they live in the registry as a coordination surface for custom test harnesses — read them with `listTestLayers()`, iterate `walkEndpoints`, and call `layer.run(input)` per endpoint. The roadmap is for a future `runRegisteredLayers({ apiMethodMap, baseUrl })` entry point to walk the registry as a fifth (or N-th) built-in layer.
+The built-in sweeps (contract/auth/rate-limit/fuzz) do **not** run registered layers — they only cover the framework envelope, and `runAllTests` does not call into the registry. To execute the registered layers, call **`runRegisteredLayers({ apiMethodMap })`** (documented below): it walks every registered layer against every endpoint, fans each result through the registered reporter, and POSTs the webhook. You can still drive the registry by hand — read it with `listTestLayers()`, iterate `walkEndpoints`, and call `layer.run(input)` per endpoint — but `runRegisteredLayers` is the supported entry point and adds error-isolation the manual loop does not.
 
 ### `listTestLayers()`
 
@@ -98,6 +98,30 @@ listTestLayers(): TestLayer[]
 
 Order is registration order (Map iteration order). The returned array is a fresh copy — mutating it does not affect the registry.
 
+### `runRegisteredLayers(input)`
+
+Run every registered layer against every endpoint in `apiMethodMap` and emit the aggregate through the registered reporter. This is the shipped entry point for the layer registry — the built-in sweeps never touch it, and it never runs the built-in sweeps.
+
+Signature:
+
+```ts
+runRegisteredLayers(input: RunRegisteredLayersInput): Promise<TestSummary>
+
+interface RunRegisteredLayersInput {
+  apiMethodMap: ApiMethodMap;
+  authToken?: string;   // forwarded to each layer's run({ authToken })
+  skip?: string[];      // '<page>/<name>' or '<page>/<name>/<version>'
+}
+```
+
+Behavior (read from `src/runRegisteredLayers.ts`):
+
+- **Cartesian walk.** For each layer from `listTestLayers()` (registration order) × each endpoint from `walkEndpoints(apiMethodMap)`, it calls `layer.run({ endpoint: endpoint.fullPath, method: endpoint.method, authToken })`. `skip` uses the same two-tier matching as the built-in sweeps; skipped endpoints produce no result.
+- **Error isolation.** Each `layer.run` is wrapped in `tryCatch`. A throwing layer (or one returning a falsy result) does not abort the walk — it is recorded as a failed `TestResult` whose `message` is the thrown error's message, or `'layer returned no result'`. Each result's `durationMs` is measured around the call. This is the behavior the manual `listTestLayers()` loop lacks, where a throw propagates and skips the rest.
+- **Reporter fan-out.** After each result it calls `reporter.onResult?.(result)`; after the walk it calls `reporter.onSummary?.(summary)`. Both are wrapped in `tryCatch`, so a throwing reporter callback never fails the run — unlike the built-in sweeps, whose `onResult` is called un-wrapped.
+- **Webhook.** If the reporter declares a `webhookUrl`, the summary is POSTed to it: `Content-Type: application/json`, plus `Authorization: Bearer <token>` when `webhookAuth?.type === 'bearer'`, body `JSON.stringify(summary)`, with a 10s `AbortController` timeout. The POST is best-effort (`tryCatch`-wrapped) — a failure never fails the run. A plaintext-`http:` URL pointed at a non-loopback host logs a `console.warn` (the summary carries endpoint paths, error codes, and any bearer token in clear) but is never blocked, since the URL is consumer-self-registered.
+- **Result.** Returns the `TestSummary` (`totalLayers`, `totalEndpoints`, `passed`, `failed`, `results`). Unlike the built-in sweeps, the per-result shape here is `TestResult` (the registry shape), not `ContractCheckResult`.
+
 ### `registerTestReporter(reporter)`
 
 Set the reporter slot, or clear it by passing `null`.
@@ -134,7 +158,7 @@ interface TestSummary {
 
 Replace-by-overwrite: only one reporter is active. To compose multiple sinks, write a single reporter that fans out internally.
 
-Like layers, the reporter is a coordination surface today: the built-in sweeps emit `ContractCheckResult` shapes, not `TestResult`. The registered reporter is intended for the cross-layer aggregator that lives in your test harness — collect results from each built-in sweep, map them into `TestResult`, and emit through this reporter so all sinks see one stream.
+`runRegisteredLayers` drives this reporter directly: it emits a `TestResult` per layer×endpoint through `onResult`, calls `onSummary` once at the end, and POSTs `webhookUrl`. The built-in sweeps do NOT — they emit `ContractCheckResult` shapes, not `TestResult`, so for them the reporter is a coordination surface your cross-layer aggregator consumes: collect each built-in sweep's results, map them into `TestResult`, and emit through this reporter so all sinks see one stream.
 
 ### `getTestReporter()`
 
@@ -160,13 +184,14 @@ The name advertises the intent: do not import this from production paths. There
 
 ## Webhook contract
 
-When `reporter.webhookUrl` is set, the harness that drives the reporter is expected to POST the JSON-serialised `TestSummary` to that URL at the end of the sweep. The runner's own built-in sweeps do not call the webhook themselves — that's the job of the consumer's aggregator. Recommended client behavior:
+When `reporter.webhookUrl` is set, **`runRegisteredLayers` POSTs the JSON-serialised `TestSummary` to that URL automatically** at the end of its run — you do not wire it yourself. The built-in sweeps (contract/auth/rate-limit/fuzz) do NOT: if you drive only those, the POST is your aggregator's job. The behavior `runRegisteredLayers` implements (and a hand-rolled aggregator should mirror):
 
 - **Method:** `POST`.
 - **Headers:** `Content-Type: application/json`, plus `Authorization: Bearer <token>` when `webhookAuth?.type === 'bearer'`.
 - **Body:** `JSON.stringify(summary)`. `TestSummary.results` can be large — clip or compress on the producer side if the receiver has body limits.
-- **Timeout:** wrap the POST in `tryCatch` with a sensible per-call timeout (e.g. 5s). A webhook failure should not fail the sweep.
-- **Retries:** none by default. Add idempotency at the receiver if you want retries; the runner is a one-shot test driver, not a queue.
+- **Timeout:** a 10s `AbortController` timeout, with the POST wrapped in `tryCatch`. A webhook failure does not fail the run.
+- **Plaintext warning:** a `webhookUrl` whose protocol is `http:` and whose host is not loopback logs a `console.warn` before POSTing — the summary (endpoint paths, error codes, metadata) and any bearer token go over the wire unencrypted. The warning never blocks the POST.
+- **Retries:** none. Add idempotency at the receiver if you want retries; the runner is a one-shot test driver, not a queue.
 
 Concrete example of a webhook-emitting reporter:
 

package/docs/fuzz-tests.md

@@ -1,8 +1,8 @@
 # Fuzz Tests
 
-The fuzz layer is the crash-resistance net. It feeds every endpoint a fixed catalogue of malformed payloads — wrong types, oversized strings, deeply nested objects, prototype-pollution attempts — and asserts the server never returns 5xx with a stack and never returns a body that escapes the framework envelope. It is intentionally exhaustive and slow; wire it into nightly CI, not into every PR.
+The fuzz layer is the crash-resistance net. It feeds every endpoint a fixed catalogue of malformed payloads — wrong types, oversized strings, deeply nested objects, prototype-pollution attempts — and asserts the server never *crashes* (returns no response at all) and never returns a body that escapes the framework envelope. A 5xx status is **tolerated as long as it still carries a valid `{ status: 'error', errorCode }` envelope** — what the layer pins is that the framework caught and shaped the error, not the numeric status code. It is intentionally exhaustive and slow; wire it into nightly CI, not into every PR.
 
-Pass criterion: across every probe, the server replies with a 1xx/2xx/3xx/4xx status code AND a JSON body of `{ status: 'success' | 'error', errorCode? }`. Any 5xx, any non-JSON body, any non-envelope JSON, or any `error` without an `errorCode` is a fail and the probe loop short-circuits to report the offending payload.
+Pass criterion: across every probe, the server returns *some* response (no fetch crash) whose body is a valid framework envelope `{ status: 'success' | 'error', errorCode? }` (with `errorCode` a string whenever `status === 'error'`). The HTTP status code itself is NOT asserted — a 5xx that still returns a valid envelope passes. A crashed connection (no response at all), a non-JSON / non-envelope body, or an `error` envelope missing a string `errorCode` is a fail, and the probe loop short-circuits to report the offending payload.
 
 ## Functions
 
@@ -29,8 +29,8 @@ Flow:
 2. For each payload, POST/PUT/DELETE it to `${baseUrl}/${endpoint.fullPath}` (or send no body on GET). The body is `JSON.stringify(payload)` so values that lose information under stringify (e.g. `undefined` keys, `NaN`) round-trip through the wire encoding the same as in production.
 3. After each request:
    - **No response (fetch error, abort, DNS):** fail with `reason: 'fuzz probe crashed (no response) with payload: ...'`. The reason includes the first 80 chars of the payload JSON.
-   - **`httpStatus >= 500`:** fail with `reason: 'fuzz payload produced 5xx: ...'`. Concrete: the server threw an unhandled error and the HTTP layer surfaced it as 500.
-   - **Body fails JSON parse or `status` is not `'success' | 'error'`:** fail with `reason: 'fuzz payload produced non-envelope response: ...'`.
+   - **`httpStatus >= 500` WITH a valid envelope:** **pass**. A 5xx is deliberately tolerated as long as the body is still a `{ status: 'error', errorCode }` envelope — that means the framework caught the error and shaped it, which is exactly the crash-resistance property this layer checks. Only the envelope checks below can fail a probe; the status code alone never does.
+   - **Body fails JSON parse or `status` is not `'success' | 'error'`:** fail with `reason: 'fuzz payload produced non-envelope response: ...'`. This is the check that actually catches a raw 5xx stack — an unhandled error that escaped the envelope arrives as a non-JSON / non-envelope body.
    - **`status === 'error'` and `errorCode` is not a string:** fail with `reason: 'fuzz error response missing errorCode for payload: ...'`.
 4. If every payload passes, return pass.
 
@@ -144,8 +144,7 @@ interface TestFixture<TPayload = unknown> {
 
 - **Pass**: `{ status: 'pass', durationMs }`. No `httpStatus` — the layer does not report which payloads passed, only that none failed.
 - **Fail / crash**: `{ status: 'fail', reason: 'fuzz probe crashed (no response) with payload: <preview>', durationMs }`.
-- **Fail / 5xx**: `{ status: 'fail', httpStatus, reason: 'fuzz payload produced 5xx: <preview>', durationMs }`.
-- **Fail / non-envelope**: `{ status: 'fail', httpStatus, reason: 'fuzz payload produced non-envelope response: <preview>', durationMs }`.
+- **Fail / non-envelope**: `{ status: 'fail', httpStatus, reason: 'fuzz payload produced non-envelope response: <preview>', durationMs }`. A bare 5xx (raw stack, HTML error page, truncated body) lands here — a 5xx is NOT a fail on its own; only the missing/invalid envelope is.
 - **Fail / error missing code**: `{ status: 'fail', httpStatus, reason: 'fuzz error response missing errorCode for payload: <preview>', durationMs }`.
 - **Skip**: `{ status: 'skipped', durationMs: 0, reason: 'Explicitly skipped' }`.
 

package/docs/rate-limit-tests.md

@@ -25,14 +25,14 @@ interface RateLimitCheckInput {
 
 Flow:
 
-1. Fire `rateLimit` requests back-to-back. Their responses are ignored — they exist only to drain the bucket. The framework records each one against the limiter, regardless of whether the request itself succeeded or failed.
+1. Fire `rateLimit` requests back-to-back to drain the bucket. Each drain response IS inspected — not for success/failure, but for a *pre-limiter* rejection. If any drain response carries an `errorCode` of `auth.required`, `auth.forbidden`, or `auth.csrfMismatch` (codes the server emits *before* the rate-limit pipeline runs: the auth guard and the CSRF middleware), the limiter bucket was never incremented for that request, so the check short-circuits to `status: 'skipped'` with `reason: 'drain request ${i+1} rejected pre-limiter (${errorCode}); bucket was never incremented — check CSRF/auth headers'`. Otherwise the drain response value is not asserted — the framework records every non-pre-limiter request against the limiter regardless of whether it succeeded or failed (e.g. validation errors, which run AFTER the limiter, still consume budget).
 2. Fire one final request. This is the probe.
 3. Parse the probe response. If `status === 'error'` AND `errorCode === 'api.rateLimitExceeded'`, pass. Anything else — including a `success` response, a different error code, or a network failure on the final probe — is a fail.
 
 Important details:
 
 - **Serial, not parallel.** Concurrent requests would race and the limiter could see them in any order. Serial keeps the math predictable: request N+1 is the one that crosses the threshold.
-- **The first `rateLimit` requests are not asserted.** They can be `success`, validation-errors, anything. The whole point is to count, not to validate intermediate behavior. The contract layer already covers correctness of those.
+- **The first `rateLimit` requests are not asserted for success** — but each is checked for a *pre-limiter* rejection (`auth.required` / `auth.forbidden` / `auth.csrfMismatch`). Such a rejection means the limiter bucket never incremented, so the check returns `skipped` rather than a misleading fail on the final probe. Any other response (`success`, a validation error, etc.) is fine — the point is to count, not to validate intermediate behavior; the contract layer already covers correctness of those. (Validation, `api.invalidInputType`, runs AFTER the limiter and is deliberately NOT treated as pre-limiter.)
 - **Default timeout is 10 seconds**, double the contract layer. Rate limiters at the upper bound of `maxRateLimitToTest` (default 50) generate ~50 requests per endpoint, which adds latency.
 - **Headers and `inputFor` are forwarded to every request**, including the drain requests. Use them when the validator rejects empty input and you want every probe to hit the limiter rather than fail at validation. (Validation failures still consume rate-limit budget in the framework.)
 
@@ -143,6 +143,7 @@ interface ResetServerStateInput {
 ## Result shapes by branch
 
 - **Pass**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: 'api.rateLimitExceeded', durationMs }`.
+- **Skip / pre-limiter rejection**: `{ status: 'skipped', durationMs, reason: 'drain request ${i+1} rejected pre-limiter (${errorCode}); bucket was never incremented — check CSRF/auth headers' }`. Emitted when a drain request is rejected by the auth guard or CSRF middleware (`auth.required` / `auth.forbidden` / `auth.csrfMismatch`) before the limiter runs — the bucket never filled, so the `N+1` assertion would be meaningless.
 - **Fail / final probe never returned**: `{ status: 'fail', reason: 'Final rate-limit probe request failed to return a response', durationMs }`.
 - **Fail / wrong status or code**: `{ status: 'fail', httpStatus, responseStatus, errorCode, reason: "expected 'api.rateLimitExceeded' on request ${N+1} but got <status>/<code>", durationMs }`.
 - **Skip / over max**: `{ status: 'skipped', durationMs: 0, reason: 'rateLimit ${N} exceeds maxRateLimitToTest=${max}' }`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@luckystack/test-runner",
-  "version": "0.2.7",
+  "version": "0.3.0",
   "private": false,
   "publishConfig": {
     "access": "public",
@@ -54,10 +54,10 @@
     "test": "vitest run"
   },
   "dependencies": {
-    "@luckystack/core": "^0.2.7"
+    "@luckystack/core": "^0.3.0"
   },
   "peerDependencies": {
-    "@luckystack/login": "^0.2.7",
+    "@luckystack/login": "^0.3.0",
     "socket.io-client": "^4.8.0",
     "zod": "^4.0.0"
   },