@luckystack/test-runner 0.1.0

package/docs/fuzz-tests.md

@@ -0,0 +1,249 @@
+# Fuzz Tests
+
+The fuzz layer is the crash-resistance net. It feeds every endpoint a fixed catalogue of malformed payloads — wrong types, oversized strings, deeply nested objects, prototype-pollution attempts — and asserts the server never returns 5xx with a stack and never returns a body that escapes the framework envelope. It is intentionally exhaustive and slow; wire it into nightly CI, not into every PR.
+
+Pass criterion: across every probe, the server replies with a 1xx/2xx/3xx/4xx status code AND a JSON body of `{ status: 'success' | 'error', errorCode? }`. Any 5xx, any non-JSON body, any non-envelope JSON, or any `error` without an `errorCode` is a fail and the probe loop short-circuits to report the offending payload.
+
+## Functions
+
+### `runFuzzCheck(input)`
+
+Single-endpoint variant.
+
+Signature:
+
+```ts
+runFuzzCheck(input: FuzzCheckInput): Promise<ContractCheckResult>
+
+interface FuzzCheckInput {
+  endpoint: EndpointDescriptor;
+  baseUrl: string;
+  headers?: Record<string, string>;
+  requestTimeoutMs?: number; // default 5000
+}
+```
+
+Flow:
+
+1. Iterate the built-in `JUNK_PAYLOADS` list serially.
+2. For each payload, POST/PUT/DELETE it to `${baseUrl}/${endpoint.fullPath}` (or send no body on GET). The body is `JSON.stringify(payload)` so values that lose information under stringify (e.g. `undefined` keys, `NaN`) round-trip through the wire encoding the same as in production.
+3. After each request:
+   - **No response (fetch error, abort, DNS):** fail with `reason: 'fuzz probe crashed (no response) with payload: ...'`. The reason includes the first 80 chars of the payload JSON.
+   - **`httpStatus >= 500`:** fail with `reason: 'fuzz payload produced 5xx: ...'`. Concrete: the server threw an unhandled error and the HTTP layer surfaced it as 500.
+   - **Body fails JSON parse or `status` is not `'success' | 'error'`:** fail with `reason: 'fuzz payload produced non-envelope response: ...'`.
+   - **`status === 'error'` and `errorCode` is not a string:** fail with `reason: 'fuzz error response missing errorCode for payload: ...'`.
+4. If every payload passes, return pass.
+
+The loop stops at the first failing probe. The result captures only that probe's response details — earlier passes are not aggregated, this layer is binary per endpoint.
+
+### `runFuzzTests(input)`
+
+Full sweep. Iterates `walkEndpoints(input.apiMethodMap)` and runs `runFuzzCheck` against each.
+
+Signature:
+
+```ts
+runFuzzTests(input: RunFuzzTestsInput): Promise<RunContractSummary>
+
+interface RunFuzzTestsInput {
+  apiMethodMap: ApiMethodMap;
+  baseUrl: string;
+  skip?: string[];
+  headers?: Record<string, string>;
+  onResult?: (result: ContractCheckResult) => void;
+}
+```
+
+There is no per-endpoint metadata filtering here — every endpoint is fuzzed, regardless of `auth.login` or `rateLimit`. `skip` matching follows the same two-tier convention as the other sweeps: `'<page>/<name>'` for version-agnostic, `'<page>/<name>/<version>'` for specific.
+
+`onResult` fires once per endpoint (after all junk payloads for that endpoint have run, or at the first failing probe). It is NOT called per junk payload.
+
+## Built-in junk payloads
+
+The current catalogue lives in `src/fuzzCheck.ts:JUNK_PAYLOADS`. It is intentionally small and stable so a flake here is meaningful:
+
+| Payload | What it stresses |
+| --- | --- |
+| `null` | Validator that assumes the body is an object. |
+| `[]` | Empty-array confusion (validators that destructure as object). |
+| `[1, 2, 3]` | Array where an object is expected. |
+| `'string-instead-of-object'` | String-as-body. |
+| `1234567890` | Number-as-body. |
+| `true` | Boolean-as-body. |
+| `{ nested: { deeply: { nested: { value: 'x'.repeat(10000) } } } }` | Oversized nested strings. Catches handlers that log the entire body without truncation. |
+| `{ __proto__: { polluted: true } }` | Prototype-pollution attempt. Detects parsers that merge into a shared object. |
+| `{ key: null, other: undefined, third: Number.NaN }` | Edge values that survive JSON encoding in non-obvious ways (`undefined` keys drop, `NaN` becomes `null`). |
+
+This is a lightweight fuzzer, not a property-based one. Real schema-driven fuzzing (TS type -> random valid input via something like `fast-check`) is deferred until the devkit emits Zod schemas alongside the runtime types. When that lands, the fuzz layer will preferentially use registered fixtures (see below), fall back to schema-random, and finally fall back to this fixed catalogue.
+
+## Fixture registry
+
+Realistic-payload fixtures plug into the fuzz layer's roadmap. Today they are registered but the current implementation of `runFuzzCheck` does not consume them automatically — they are designed to be read by custom test layers and by the upcoming schema-aware fuzz mode. Registering them now keeps fixtures co-located with the endpoint code that owns the type.
+
+### `registerTestFixture(typeKey, fixture)`
+
+Store a fixture for a given type key. Re-registering replaces the previous entry. Typically called at module load from a `_api/*` file or a central registration module.
+
+Signature:
+
+```ts
+registerTestFixture<TPayload = unknown>(typeKey: string, fixture: TestFixture<TPayload>): void
+
+interface TestFixture<TPayload = unknown> {
+  valid: TPayload[];
+  invalid: TPayload[];
+}
+```
+
+Conventions:
+
+- **`typeKey` is a project-defined string** that uniquely identifies the input type. Common patterns: the route literal (`api/billing/getInvoice/v1`), or the named TypeScript interface (`GetInvoiceInput`). Pick one convention per project.
+- **`valid` payloads** are shapes the validator accepts. Used to drive happy-path behavior (e.g. by a custom layer running each valid fixture and asserting `status === 'success'`).
+- **`invalid` payloads** are shapes the validator should reject. Used to probe the error envelope: each invalid fixture should return `{ status: 'error', errorCode: '<validation code>' }`.
+- **Re-registration replaces.** No merging — pass the full list each time.
+
+### `getTestFixture(typeKey)`
+
+Read back the fixture. Returns `undefined` when no fixture is registered for that key.
+
+Signature:
+
+```ts
+getTestFixture(typeKey: string): TestFixture | undefined
+```
+
+Use this from a custom layer (see `extension-hooks.md`) to iterate per-endpoint fixtures.
+
+## Types
+
+```ts
+interface FuzzCheckInput {
+  endpoint: EndpointDescriptor;
+  baseUrl: string;
+  headers?: Record<string, string>;
+  requestTimeoutMs?: number;
+}
+
+interface RunFuzzTestsInput {
+  apiMethodMap: ApiMethodMap;
+  baseUrl: string;
+  skip?: string[];
+  headers?: Record<string, string>;
+  onResult?: (result: ContractCheckResult) => void;
+}
+
+interface TestFixture<TPayload = unknown> {
+  valid: TPayload[];
+  invalid: TPayload[];
+}
+```
+
+`ContractCheckResult` and `RunContractSummary` are shared across layers — see `contract-tests.md`.
+
+## Result shapes by branch
+
+- **Pass**: `{ status: 'pass', durationMs }`. No `httpStatus` — the layer does not report which payloads passed, only that none failed.
+- **Fail / crash**: `{ status: 'fail', reason: 'fuzz probe crashed (no response) with payload: <preview>', durationMs }`.
+- **Fail / 5xx**: `{ status: 'fail', httpStatus, reason: 'fuzz payload produced 5xx: <preview>', durationMs }`.
+- **Fail / non-envelope**: `{ status: 'fail', httpStatus, reason: 'fuzz payload produced non-envelope response: <preview>', durationMs }`.
+- **Fail / error missing code**: `{ status: 'fail', httpStatus, reason: 'fuzz error response missing errorCode for payload: <preview>', durationMs }`.
+- **Skip**: `{ status: 'skipped', durationMs: 0, reason: 'Explicitly skipped' }`.
+
+`durationMs` covers the time to the first failing probe (fail branches) or the time to walk the whole catalogue (pass branch).
+
+## Examples
+
+### Nightly CI sweep
+
+```ts
+import { apiMethodMap } from './generated/apiMethodMap.generated';
+import {
+  runFuzzTests,
+  logContractResult,
+  logContractSummary,
+} from '@luckystack/test-runner';
+
+const summary = await runFuzzTests({
+  apiMethodMap,
+  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
+  onResult: logContractResult,
+});
+
+logContractSummary(summary);
+
+if (summary.failed > 0) {
+  process.exit(3);
+}
+```
+
+### Authenticated fuzz (cookie-protected endpoints)
+
+The fuzz layer does not strip credentials the way the auth layer does. Pass a session header so probes reach the handler instead of being killed at the auth guard:
+
+```ts
+await runFuzzTests({
+  apiMethodMap,
+  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
+  headers: {
+    Cookie: 'session=<test-session-cookie>',
+  },
+  skip: ['integrations/stripeWebhook'], // signed payloads, would always fail
+});
+```
+
+### Single-endpoint debug
+
+```ts
+import { runFuzzCheck } from '@luckystack/test-runner';
+
+const result = await runFuzzCheck({
+  endpoint: {
+    page: 'organization-settings',
+    name: 'sendInvite',
+    version: 'v1',
+    method: 'POST',
+    fullPath: 'api/organization-settings/sendInvite/v1',
+  },
+  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
+  headers: { Cookie: 'session=<test-session>' },
+  requestTimeoutMs: 8000,
+});
+
+console.log(result);
+```
+
+### Register a fixture for a critical endpoint
+
+Co-locate the fixture next to the endpoint so it travels with refactors:
+
+```ts
+// src/billing/_api/getInvoice_v1.ts
+import { registerTestFixture } from '@luckystack/test-runner';
+
+registerTestFixture<{ invoiceId: string; expand?: string[] }>('api/billing/getInvoice/v1', {
+  valid: [
+    { invoiceId: 'inv_1' },
+    { invoiceId: 'inv_1', expand: ['line_items'] },
+  ],
+  invalid: [
+    { invoiceId: '' },
+    { invoiceId: 'inv_1', expand: ['malicious; DROP TABLE'] },
+  ],
+});
+
+export const main = async ({ data }: ApiParams) => { /* ... */ };
+```
+
+A nightly custom layer can then walk every endpoint, look up the fixture by route, and run the valid/invalid lists against the live endpoint to verify expected behavior beyond the framework envelope. The fuzz layer itself does not yet auto-pick these up — that wiring lands with the schema-aware fuzz mode.
+
+## Edge cases and gotchas
+
+- **Binary uploads.** Endpoints expecting `multipart/form-data` will fail every JUNK payload because the Content-Type header is `application/json`. Skip these endpoints explicitly; they need a separate, format-aware fuzzer.
+- **Streaming endpoints.** Endpoints that hold the connection open (SSE, long-poll) will hit `requestTimeoutMs` and report as crashed. Skip them or add support for streaming responses in a custom layer.
+- **Endpoints with filesystem side-effects.** A fuzz probe that triggers a real file write happens on every endpoint where the validator is permissive. Add cleanup in a custom layer or run the fuzz sweep against a disposable test instance.
+- **Prototype pollution.** The runner sends `{ __proto__: { polluted: true } }`. Node's `JSON.parse` already strips `__proto__` to a regular own-property in modern runtimes, so this probe is mostly defensive. A failure here means a handler is using a custom merge function that does not honor that hardening.
+- **`NaN` and `undefined` after stringify.** `Number.NaN` JSON-stringifies as `null`; `undefined`-valued keys are dropped. The payload that hits the server is `{ key: null, third: null }`. This is intentional — it mirrors what a malicious client can actually transmit.
+- **Auth-protected endpoints with no session header.** They will all fail with `errorCode: 'auth.required'` (which passes the fuzz envelope check, since the response is a typed error). The fuzz sweep will report them as `pass`. Whether that is what you want depends on whether you trust the auth layer's earlier verdict — most projects pass headers to fuzz so the handler logic is actually reached.
+- **The `<preview>` in the `reason`** is `JSON.stringify(payload).slice(0, 80)`. Long payloads (oversized nested) are truncated. Look at the source `JUNK_PAYLOADS` to identify which probe caused the failure when the preview is cut.
+- **Adding payloads.** Today the catalogue is hard-coded. To extend it, fork the package or wrap `runFuzzCheck` in a custom layer (`registerTestLayer({ name: 'extra-fuzz', run })`) that issues additional probes. The roadmap is to make this list registry-driven once the schema-aware mode lands.
+- **`onResult` is per-endpoint.** If you need per-probe visibility for a single endpoint, instrument `runFuzzCheck` directly or call it once per payload yourself. The exported function is the smallest stable unit.

package/docs/rate-limit-tests.md

@@ -0,0 +1,256 @@
+# Rate-Limit Tests
+
+The rate-limit layer proves one thing per endpoint: an endpoint declared `rateLimit: N` rejects the `(N+1)`th request inside its window with `errorCode: 'api.rateLimitExceeded'`. It is the layer that mutates the most server-side state — every probe drains a real bucket. Run it after the contract and auth layers, against a dedicated test instance, and ideally with `resetBetweenEndpoints: true`.
+
+## Functions
+
+### `runRateLimitCheck(input)`
+
+Single-endpoint variant.
+
+Signature:
+
+```ts
+runRateLimitCheck(input: RateLimitCheckInput): Promise<ContractCheckResult>
+
+interface RateLimitCheckInput {
+  endpoint: EndpointDescriptor;
+  baseUrl: string;
+  rateLimit: number;
+  inputFor?: (endpoint: EndpointDescriptor) => unknown;
+  headers?: Record<string, string>;
+  requestTimeoutMs?: number; // default 10000
+}
+```
+
+Flow:
+
+1. Fire `rateLimit` requests back-to-back. Their responses are ignored — they exist only to drain the bucket. The framework records each one against the limiter, regardless of whether the request itself succeeded or failed.
+2. Fire one final request. This is the probe.
+3. Parse the probe response. If `status === 'error'` AND `errorCode === 'api.rateLimitExceeded'`, pass. Anything else — including a `success` response, a different error code, or a network failure on the final probe — is a fail.
+
+Important details:
+
+- **Serial, not parallel.** Concurrent requests would race and the limiter could see them in any order. Serial keeps the math predictable: request N+1 is the one that crosses the threshold.
+- **The first `rateLimit` requests are not asserted.** They can be `success`, validation-errors, anything. The whole point is to count, not to validate intermediate behavior. The contract layer already covers correctness of those.
+- **Default timeout is 10 seconds**, double the contract layer. Rate limiters at the upper bound of `maxRateLimitToTest` (default 50) generate ~50 requests per endpoint, which adds latency.
+- **Headers and `inputFor` are forwarded to every request**, including the drain requests. Use them when the validator rejects empty input and you want every probe to hit the limiter rather than fail at validation. (Validation failures still consume rate-limit budget in the framework.)
+
+### `runRateLimitTests(input)`
+
+Full sweep. Reads `rateLimit` from `apiMetaMap[page][name][version].rateLimit`, skips endpoints where it is `false`, `undefined`, or above `maxRateLimitToTest`, then runs `runRateLimitCheck` against the rest.
+
+Signature:
+
+```ts
+runRateLimitTests(input: RunRateLimitTestsInput): Promise<RunContractSummary>
+
+interface RunRateLimitTestsInput {
+  apiMethodMap: ApiMethodMap;
+  apiMetaMap: ApiMetaMap;
+  baseUrl: string;
+  skip?: string[];
+  inputFor?: (endpoint: EndpointDescriptor) => unknown;
+  headers?: Record<string, string>;
+  maxRateLimitToTest?: number;     // default 50
+  resetBetweenEndpoints?: boolean; // default false
+  resetToken?: string;
+  onResult?: (result: ContractCheckResult) => void;
+}
+```
+
+Behavior notes:
+
+- **Endpoints with `rateLimit: false` are silently skipped.** No `skipped` result, no `onResult` call. The summary's `total` reflects only endpoints with a numeric rate limit.
+- **Endpoints with `rateLimit > maxRateLimitToTest` are skipped loudly.** They appear with `status: 'skipped'` and `reason: 'rateLimit ${N} exceeds maxRateLimitToTest=${max}'`. Bump `maxRateLimitToTest` to cover them, but be aware each endpoint then fires `N+1` requests serially.
+- **`resetBetweenEndpoints: true` calls `resetServerState({ baseUrl, token: resetToken })` before every endpoint** that survives the skip filter. Without this, the limiter bucket on a shared IP (the test runner's host) is not flushed between endpoints — endpoint A's drain requests bleed into endpoint B's window and the N+1 assertion fires too early or too late depending on key collisions.
+- **`resetToken` is required when `TEST_RESET_TOKEN` is set on the server.** Always set this in any non-prod environment that is reachable over the network (staging, preview). The server is fail-closed: missing token = 403, unset env var server-side = 403 (never "open").
+- **`onResult` fires for tested AND `maxRateLimitToTest`-skipped endpoints**, but not for `rateLimit: false` endpoints (those don't show up at all).
+
+### `resetServerState(input)`
+
+POST to the server's `/_test/reset` endpoint. Used between rate-limit probes and at the start of a CI run to ensure a clean slate.
+
+Signature:
+
+```ts
+resetServerState(input: ResetServerStateInput): Promise<boolean>
+
+interface ResetServerStateInput {
+  baseUrl: string;
+  token?: string;
+}
+```
+
+Returns `true` when the server responds 2xx, `false` on network error, parse error, or non-2xx. The error is swallowed deliberately — the most common reason for `false` is "the route is disabled in this environment", and the caller usually wants to proceed anyway.
+
+URL: `${baseUrl}/_test/reset` (POST). When `token` is set, the request includes `X-Test-Reset-Token: ${token}`. The header name is part of the server contract.
+
+## `/_test/reset` server contract
+
+The endpoint is served by `@luckystack/server` (see `packages/server/src/httpRoutes/testResetRoute.ts`). It is mounted at `projectConfig.http.testResetEndpoint` (default `/_test/reset`).
+
+Gating, in order:
+
+1. `process.env.NODE_ENV` must be exactly `'development'` or `'test'`. Anything else — including missing — returns 404. "Anything but production" was rejected as too loose: a misconfigured env var must not expose this route.
+2. `process.env.TEST_RESET_TOKEN` must be set on the server AND `req.headers['x-test-reset-token']` must match exactly. Empty server-side token = 403 unconditionally. The header check happens after the NODE_ENV check, so leaking the token name from a 403 only happens in dev/test environments.
+
+What it clears (when both gates pass):
+
+- **All rate-limit buckets** via `clearAllRateLimits()` from `@luckystack/core`. Every key under the rate-limit Redis namespace is dropped. Always cleared.
+- **All sessions** under `${projectName}-session:*` and active-user records under `${projectName}-activeUsers:*`. Scanned in batches of 200 keys via `redis.scan + redis.del`. Project name comes from `getProjectName()` for consistency across `session.ts`, `rateLimiter.ts`, and this endpoint.
+- **Registered runtime hooks** when `?include=hooks` is in the query string. NOT included by default because framework-internal handlers (e.g. presence postLogout) register at boot — clearing them would require a server restart to recover.
+
+Response:
+
+```json
+{ "status": "success", "cleared": ["rateLimits", "sessions", "activeUsers"] }
+```
+
+The `cleared` array reflects what actually had keys to drop. It is informational; `runRateLimitTests` only checks `response.ok`.
+
+## Types
+
+```ts
+interface RateLimitCheckInput {
+  endpoint: EndpointDescriptor;
+  baseUrl: string;
+  rateLimit: number;
+  inputFor?: (endpoint: EndpointDescriptor) => unknown;
+  headers?: Record<string, string>;
+  requestTimeoutMs?: number;
+}
+
+interface RunRateLimitTestsInput {
+  apiMethodMap: ApiMethodMap;
+  apiMetaMap: ApiMetaMap;
+  baseUrl: string;
+  skip?: string[];
+  inputFor?: (endpoint: EndpointDescriptor) => unknown;
+  headers?: Record<string, string>;
+  maxRateLimitToTest?: number;
+  resetBetweenEndpoints?: boolean;
+  resetToken?: string;
+  onResult?: (result: ContractCheckResult) => void;
+}
+
+interface ResetServerStateInput {
+  baseUrl: string;
+  token?: string;
+}
+```
+
+## Result shapes by branch
+
+- **Pass**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: 'api.rateLimitExceeded', durationMs }`.
+- **Fail / final probe never returned**: `{ status: 'fail', reason: 'Final rate-limit probe request failed to return a response', durationMs }`.
+- **Fail / wrong status or code**: `{ status: 'fail', httpStatus, responseStatus, errorCode, reason: "expected 'api.rateLimitExceeded' on request ${N+1} but got <status>/<code>", durationMs }`.
+- **Skip / over max**: `{ status: 'skipped', durationMs: 0, reason: 'rateLimit ${N} exceeds maxRateLimitToTest=${max}' }`.
+- **Skip / explicit**: `{ status: 'skipped', durationMs: 0, reason: 'Explicitly skipped' }`.
+
+`durationMs` covers the entire `N+1` request sequence, not just the final probe.
+
+## Examples
+
+### Full sweep with state reset between endpoints
+
+```ts
+import { apiMethodMap } from './generated/apiMethodMap.generated';
+import { apiMetaMap } from './generated/apiMetaMap.generated';
+import {
+  resetServerState,
+  runRateLimitTests,
+  logContractResult,
+  logContractSummary,
+} from '@luckystack/test-runner';
+
+const baseUrl = process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80';
+const resetToken = process.env.TEST_RESET_TOKEN;
+
+await resetServerState({ baseUrl, token: resetToken });
+
+const summary = await runRateLimitTests({
+  apiMethodMap,
+  apiMetaMap,
+  baseUrl,
+  resetBetweenEndpoints: true,
+  resetToken,
+  onResult: logContractResult,
+});
+
+logContractSummary(summary);
+```
+
+### Local dev: skip the reset (faster, less hygienic)
+
+```ts
+await runRateLimitTests({
+  apiMethodMap,
+  apiMetaMap,
+  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
+  // No resetBetweenEndpoints — fine if all endpoints have separate limiter keys.
+});
+```
+
+This works when every endpoint has its own limiter key (the framework default is `<ip>:<endpoint>`). It fails when limiter keys are coarsened to IP-only.
+
+### Single-endpoint debug
+
+```ts
+import { runRateLimitCheck } from '@luckystack/test-runner';
+
+const result = await runRateLimitCheck({
+  endpoint: {
+    page: 'auth',
+    name: 'login',
+    version: 'v1',
+    method: 'POST',
+    fullPath: 'api/auth/login/v1',
+  },
+  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
+  rateLimit: 5, // matches the value in src/auth/_api/login_v1.ts
+  inputFor: () => ({ email: 'test@example.com', password: 'wrong' }),
+});
+
+console.log(result);
+```
+
+### Staging deploy with a token
+
+```ts
+await runRateLimitTests({
+  apiMethodMap,
+  apiMetaMap,
+  baseUrl: 'https://staging.example.com',
+  resetBetweenEndpoints: true,
+  resetToken: process.env.STAGING_TEST_RESET_TOKEN,
+  maxRateLimitToTest: 100, // wider net for nightly sweep
+});
+```
+
+Set `TEST_RESET_TOKEN` on the staging server to the same value. Without both halves of the token, the reset returns 403 and the rate-limit windows leak across endpoints.
+
+### Tighten the budget
+
+```ts
+await runRateLimitTests({
+  apiMethodMap,
+  apiMetaMap,
+  baseUrl,
+  maxRateLimitToTest: 20, // skip everything noisier than 20/window
+});
+```
+
+Endpoints with `rateLimit: 100` will appear as skipped with the reason inline.
+
+## Edge cases and gotchas
+
+- **Time-window granularity.** The framework rate-limiter uses Redis-backed sliding windows. Firing `N+1` serial requests typically completes in well under the window, so the assertion is reliable. If your endpoint sets `rateLimit: 1` and the window is sub-second, latency between the drain and the probe can let the bucket reset — bump `requestTimeoutMs` only buys per-call slack; the window itself is server-controlled.
+- **Distributed rate-limit storage.** The runner assumes one limiter authority (one Redis instance, or a single cluster). Multi-region setups with eventual consistency can produce false negatives: the N+1 probe lands on a different shard than the drain. Run this layer against a single-region staging.
+- **Multi-instance servers.** Without sticky sessions, the drain requests fan out to several pods and each pod sees fewer than N. The probe lands on a pod that has not yet seen the bucket fill. Either pin the runner to one upstream, or set the limiter keying to include the user-ip header consistently.
+- **Limiter keys.** The framework keys on a combination of route + IP (or session, when present). Two endpoints can share a coarser key (e.g. `ip:<all>`). When they do, `resetBetweenEndpoints` is mandatory. When they do not, the option is optional and the test runs faster without it.
+- **The `validation.failed` trap.** If the validator rejects every drain request before the limiter sees them, the framework still counts the request — but verifying this assumption is the user's responsibility. Use `inputFor` to provide payloads that survive validation when in doubt.
+- **`maxRateLimitToTest` accidentally too high.** With `maxRateLimitToTest: 1000` and 30 endpoints, the sweep fires `~30000` requests serially. This will time out CI long before it completes. Default 50 is deliberate.
+- **`/_test/reset` returning false from `resetServerState`.** Most common cause: `NODE_ENV !== 'development' | 'test'` on the server. Second most common: missing token. Third: route disabled via custom routing. The runner does not retry — it returns `false` and proceeds. The next endpoint's window will be polluted; consider failing the sweep early.
+- **Sessions cleared as collateral damage.** `resetServerState` always drops sessions and active users in addition to limiter buckets. If you have a long-running logged-in test alongside this layer, run the rate-limit sweep first, then re-establish the session.
+- **`?include=hooks` is opt-in for a reason.** Do not call `resetServerState` with this flag in a long-running test process — framework-internal hooks register only at boot, and dropping them mid-run will break presence, post-logout cleanup, and similar side-effects until restart.

package/package.json

@@ -0,0 +1,66 @@
+{
+  "name": "@luckystack/test-runner",
+  "version": "0.1.0",
+  "private": false,
+  "publishConfig": {
+    "access": "public"
+  },
+  "type": "module",
+  "description": "Generated-type-driven test layers for LuckyStack: contract smoke (Zod-valid input), auth enforcement, rate-limit, and crash-resistance fuzz. Walks every endpoint via the generated apiMethodMap.",
+  "license": "MIT",
+  "author": "Mathijs van Melick <mathijsvanmelick3@gmail.com>",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/ItsLucky23/LuckyStack-v2.git",
+    "directory": "packages/test-runner"
+  },
+  "homepage": "https://github.com/ItsLucky23/LuckyStack-v2/tree/master/packages/test-runner#readme",
+  "bugs": {
+    "url": "https://github.com/ItsLucky23/LuckyStack-v2/issues"
+  },
+  "keywords": [
+    "luckystack",
+    "testing",
+    "contract-testing",
+    "fuzz",
+    "rate-limit",
+    "api-testing",
+    "auth-testing",
+    "zod"
+  ],
+  "engines": {
+    "node": ">=20.0.0"
+  },
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    }
+  },
+  "files": [
+    "dist",
+    "README.md",
+    "CLAUDE.md",
+    "docs",
+    "LICENSE",
+    "CHANGELOG.md"
+  ],
+  "scripts": {
+    "build": "tsup",
+    "build:watch": "tsup --watch",
+    "test": "vitest run"
+  },
+  "dependencies": {
+    "@luckystack/core": "^0.1.0"
+  },
+  "peerDependencies": {
+    "@luckystack/login": "^0.1.0",
+    "socket.io-client": "^4.8.0",
+    "zod": "^4.0.0"
+  },
+  "peerDependenciesMeta": {
+    "@luckystack/login": { "optional": true }
+  }
+}