@luckystack/test-runner 0.10.2 → 0.11.0

package/docs/auth-tests.md

@@ -1,226 +1,7 @@
-# Auth Enforcement Tests
+# Auth enforcement tests
 
-The auth-enforcement layer answers one question per endpoint: "If a logged-out caller hits this route, does the framework reject them?" It is the cheapest insurance policy against a `_api/*` file that forgets to set `auth.login: true`, an `auth.login` guard that was silently swapped to `false` during a refactor, or a middleware change in `@luckystack/api` that stops short-circuiting unauthenticated requests.
+This layer deliberately calls protected endpoints **without credentials** to verify that the guard rejects an anonymous request. A rejection from validation or an unrelated handler is not evidence that authorization ran. Routes with additional role or scope predicates still need project-specific checks using authenticated lower-privilege users; anonymous tests alone cannot prove tenant isolation.
 
-Pass criterion: the endpoint returns `{ status: 'error', errorCode }` where `errorCode` is one of the auth-rejection codes — `auth.required` (an `auth.login: true` route) or `auth.forbidden` (a route guarded only by `auth.additional[]` predicates). Anything else — `success`, a different `errorCode`, a missing envelope, a network error — is a failure.
+Sync routes are covered only when the sweep receives the generated sync metadata, which is also where their auth requirement is read from. Their probes travel over the project's sync transport, so the check exercises the same guard path a real client hits (ADR 0064). GET probes carry their sample payload in the query string like the real client (ADR 0062), so a GET result is not distorted by a payload that never arrived.
 
-## Functions
-
-### `runAuthEnforcementCheck(input)`
-
-Single-endpoint variant.
-
-Signature:
-
-```ts
-runAuthEnforcementCheck(input: AuthEnforcementCheckInput): Promise<ContractCheckResult>
-
-interface AuthEnforcementCheckInput {
-  endpoint: EndpointDescriptor;
-  baseUrl: string;
-  inputFor?: (endpoint: EndpointDescriptor) => unknown;
-  requestTimeoutMs?: number; // default 5000
-}
-```
-
-Flow:
-
-1. Build URL the same way the contract layer does.
-2. Build body via `inputFor?.(endpoint) ?? {}`. On `GET` it travels in the query string, as in the contract layer.
-3. **Send the request with no session cookie, no `Authorization` header, no `X-CSRF-Token`.** The runner deliberately omits every form of credential — that is the whole point of this layer. `headers` is not a supported field for this reason; passing one would defeat the test.
-4. Parse the JSON. Parse failure is a fail.
-5. Branch on `parsed.status`:
-   - `'success'` -> fail with `reason: 'auth.login endpoint returned success without a session'`. This is the headline case: the endpoint is open when it should be closed.
-   - Not `'error'` (missing, weird) -> fail with `'Response missing standard `status` envelope'`.
-   - `'error'` with an `errorCode` outside the auth-rejection set (`auth.required` | `auth.forbidden`) -> fail with the expected/actual codes inline in the `reason` (`expected an auth-rejection errorCode (auth.required | auth.forbidden) but got '<actual>'`). A different error code (e.g. validation rejecting `{}`) means the auth guard never ran — the request was killed earlier in the pipeline.
-   - `'error'` with `errorCode === 'auth.required'` OR `errorCode === 'auth.forbidden'` -> pass.
-
-The accepted error codes are the framework's auth-rejection set, hard-coded in the runner as `EXPECTED_ERROR_CODES = new Set(['auth.required', 'auth.forbidden'])` (see `packages/api/src/handleHttpApiRequest.ts` + `packages/core/src/validateRequest.ts`). `auth.required` is what an `auth.login: true` route emits for an anonymous caller; `auth.forbidden` is what an `auth.additional[]`-only route (`login: false`) emits when the predicate rejects the anonymous caller. Either one is a PASS — both mean the guard fired. They are intentionally hard-coded rather than fed from config because the whole point is to detect drift: if `@luckystack/api` ever renames a code, this layer should fail every protected endpoint until consumers acknowledge the change.
-
-### `runAuthEnforcementTests(input)`
-
-Full sweep. Iterates every API route (plus every sync route when `syncMetaMap` is given — its requirement is read from `syncMetaMap`), narrowed by `filter` before any request, and checks every endpoint that declares an authorization requirement — `apiMetaMap[page][name][version].auth.login === true` OR a non-empty `auth.additional[]` predicate list (`hasAdditional`). Truly-public endpoints (`auth.login: false` AND no `additional[]`, or missing meta) are skipped **silently** — they do not appear in the summary at all. This keeps the report focused on what this layer actually covers; public endpoints are already exercised by the contract layer.
-
-Signature:
-
-```ts
-runAuthEnforcementTests(input: RunAuthEnforcementTestsInput): Promise<RunContractSummary>
-
-interface RunAuthEnforcementTestsInput {
-  apiMethodMap: ApiMethodMap;
-  apiMetaMap: ApiMetaMap;
-  baseUrl: string;
-  skip?: string[];
-  inputFor?: (endpoint: EndpointDescriptor) => unknown;
-  onResult?: (result: ContractCheckResult) => void;
-}
-
-type ApiMetaMap = Record<string, Record<string, Record<string, {
-  method: string;
-  auth: { login: boolean; additional?: Record<string, unknown>[] };
-  rateLimit?: number | false;
-}>>>;
-```
-
-Behavior notes:
-
-- **`skip` uses the same two-tier matching as `runContractTests`**: pass `'<page>/<name>'` for version-agnostic skip, or `'<page>/<name>/<version>'` for a specific version.
-- **`onResult` fires only for endpoints that pass `requiresLogin`.** Public endpoints don't trigger it; they don't exist for this layer.
-- **Unauthenticated enforcement covers `additional[]`-only routes too.** An endpoint guarded only by `auth.additional[]` predicates (`login: false`) is probed and must reject an anonymous caller with `auth.forbidden`. The DEEPER role/scope *value* assertion (e.g. "this admin-only route rejects a logged-in non-admin") is out of scope and lives in a custom layer (see `extension-hooks.md`).
-
-Returns the same `RunContractSummary` shape as the contract layer; `total` is the number of protected endpoints actually probed (after public-endpoint filtering).
-
-## Types
-
-```ts
-interface AuthEnforcementCheckInput {
-  endpoint: EndpointDescriptor;
-  baseUrl: string;
-  inputFor?: (endpoint: EndpointDescriptor) => unknown;
-  requestTimeoutMs?: number;
-}
-
-interface RunAuthEnforcementTestsInput {
-  apiMethodMap: ApiMethodMap;
-  apiMetaMap: ApiMetaMap;
-  baseUrl: string;
-  skip?: string[];
-  inputFor?: (endpoint: EndpointDescriptor) => unknown;
-  onResult?: (result: ContractCheckResult) => void;
-}
-```
-
-`ContractCheckResult` and `RunContractSummary` are shared with the contract layer — see `contract-tests.md`.
-
-## Result shapes by branch
-
-- **Pass (`auth.login` route)**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: 'auth.required', durationMs }`.
-- **Pass (`additional[]`-only route)**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: 'auth.forbidden', durationMs }`.
-- **Fail / open endpoint**: `{ status: 'fail', httpStatus, responseStatus: 'success', reason: 'auth.login endpoint returned success without a session', durationMs }`.
-- **Fail / wrong errorCode**: `{ status: 'fail', httpStatus, responseStatus: 'error', errorCode: '<actual>', reason: "expected an auth-rejection errorCode (auth.required | auth.forbidden) but got '<actual>'", durationMs }`.
-- **Fail / non-envelope**: `{ status: 'fail', httpStatus, responseStatus: 'unknown', reason: 'Response missing standard `status` envelope', durationMs }`.
-- **Fail / fetch no response**: `{ status: 'fail', reason: 'fetch returned no response', durationMs }`.
-- **Skip**: `{ status: 'skipped', durationMs: 0, reason: 'Explicitly skipped' }`.
-
-## Examples
-
-### Full sweep in CI
-
-```ts
-import { apiMethodMap } from './generated/apiMethodMap.generated';
-import { apiMetaMap } from './generated/apiMetaMap.generated';
-import {
-  runAuthEnforcementTests,
-  logContractResult,
-  logContractSummary,
-} from '@luckystack/test-runner';
-
-const summary = await runAuthEnforcementTests({
-  apiMethodMap,
-  apiMetaMap,
-  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
-  onResult: logContractResult,
-});
-
-logContractSummary(summary);
-
-if (summary.failed > 0) {
-  // A failure here is a security finding, not a flaky test.
-  process.exit(2);
-}
-```
-
-### Single endpoint debug
-
-```ts
-import { runAuthEnforcementCheck, logContractResult } from '@luckystack/test-runner';
-
-const result = await runAuthEnforcementCheck({
-  endpoint: {
-    page: 'billing',
-    name: 'cancelSubscription',
-    version: 'v1',
-    method: 'POST',
-    fullPath: 'api/billing/cancelSubscription/v1',
-  },
-  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
-});
-
-logContractResult(result);
-```
-
-### Bypass an over-eager validator
-
-Some validators reject `{}` before the auth guard runs. The framework runs auth before validation, but a custom middleware (or a custom error formatter) can flip that order. If you see `errorCode: 'validation.failed'` instead of `auth.required`, supply a payload that the validator accepts so the request reaches the auth check:
-
-```ts
-await runAuthEnforcementTests({
-  apiMethodMap,
-  apiMetaMap,
-  baseUrl,
-  inputFor: (endpoint) => {
-    if (endpoint.fullPath === 'api/billing/cancelSubscription/v1') {
-      return { subscriptionId: 'sub_test' };
-    }
-    return {};
-  },
-});
-```
-
-When you see this happen for many endpoints, the right fix is usually in the middleware order in `@luckystack/api`, not in the runner.
-
-### Skip endpoints with custom auth schemes
-
-Endpoints that authenticate via a non-session mechanism (e.g. a webhook with HMAC signature, an internal cron token) declare `auth.login: false` and live outside this layer. If for some reason a webhook endpoint is registered as `auth.login: true` (and you cannot fix the source), skip it:
-
-```ts
-await runAuthEnforcementTests({
-  apiMethodMap,
-  apiMetaMap,
-  baseUrl,
-  skip: ['integrations/stripeWebhook'],
-});
-```
-
-Prefer fixing the meta over skipping — a wrong `auth.login` flag breaks runtime behavior, not just this test.
-
-## Edge cases and gotchas
-
-- **CSRF.** The framework enforces CSRF on state-changing requests. The runner sends no `X-CSRF-Token`, so for a brief window the request can be rejected on CSRF grounds before reaching auth. The current framework returns `auth.required` ahead of CSRF for unauthenticated sessions, so this layer passes; if you reorder middleware to CSRF-first, this layer will need updating.
-- **Deeper `additional` role/scope *value* checks** are not covered here (the layer only asserts an anonymous caller is rejected with `auth.forbidden`). Put authenticated-but-wrong-role checks in a `registerTestLayer({ name: 'role-enforcement', run })` plug-in (see `extension-hooks.md`).
-- **Endpoints absent from `apiMetaMap`.** The runner reads `apiMetaMap[page]?.[name]?.[version]?.auth`. If any link in that chain is missing, `hasAuthRequirement` returns `false` and the endpoint is silently skipped. Regenerate the meta map if your sweep reports `total: 0`.
-- **Session-cookie names are project-specific.** This layer never sets one, so the name doesn't matter to the runner. It does matter for the auth layer's mirror image: a custom "logged-in sweep" you might write to confirm protected endpoints work *with* a valid session.
-- **Wrong errorCode is a real signal.** When you see `expected 'auth.required' but got 'auth.sessionExpired'`, the framework is detecting and reporting the missing session — just with a stricter code. Either align the runner expectation (fork or PR) or align the framework code. Do not skip.
-- **`@luckystack/login` session-token interaction.** Sessions live in Redis under `${projectName}-session:*` keys. The runner does not touch them. If a logged-in regression suite must run before this layer, use `resetServerState()` to clear sessions and rate limits in one shot (see `rate-limit-tests.md`).
-- **`onResult` async handlers.** The runner does `input.onResult?.(result)` without awaiting. Sync side-effects fire in order; async work is fire-and-forget. Push results into an array and process after the sweep if you need ordering guarantees.
-- **Empty `summary.total`.** When the protected-endpoint filter excludes every endpoint, the sweep returns `{ total: 0, passed: 0, failed: 0, skipped: 0, results: [] }`. This is the expected "no protected endpoints" outcome, not a failure mode. Distinguish from network failure by inspecting `apiMetaMap` directly before running the sweep when you want to assert there *should* be protected endpoints.
-
-## Pairing with the contract layer
-
-The auth-enforcement layer covers the unauthenticated path. It does NOT cover the authenticated path — an endpoint that's protected, accepts a valid session, but crashes on real input would pass this layer cleanly. Run the contract layer with real session headers as the second half of the matrix:
-
-```ts
-await runContractTests({
-  apiMethodMap,
-  baseUrl,
-  headers: { Cookie: 'session=<test-session-cookie>' },
-});
-
-await runAuthEnforcementTests({
-  apiMethodMap,
-  apiMetaMap,
-  baseUrl,
-  // No headers — the layer deliberately omits credentials.
-});
-```
-
-The first sweep proves "logged-in callers get sane responses"; the second proves "logged-out callers get blocked". A custom role-enforcement layer (registered via `registerTestLayer`) is the natural third leg.
-
-## Interaction with `@luckystack/login`
-
-Sessions written by `@luckystack/login` live in Redis under `${projectName}-session:*`. The auth layer does not read them and never sets a `Cookie` header. When wiring an end-to-end matrix:
-
-1. Use `resetServerState()` from `rate-limit-tests.md` to flush sessions before kicking off any logged-in sweep.
-2. Establish one session through the project's `/auth/login` flow (or call `setSession` directly in your harness).
-3. Pass that session as `Cookie: session=<token>` to the contract or fuzz layer.
-4. Run the auth-enforcement layer last (or before step 2) — it does its own thing with no credentials, so order vs the logged-in sweep does not matter for correctness, only for test runtime.
+Do not add shared auth headers or a cookie to this sweep: that would defeat its purpose. Treat an unexpected rejection code as a security finding to investigate, not a reason to skip or loosen the assertion. Source and generated metadata define which endpoints participate and the expected response envelope; avoid copying a route-by-route inventory here.

package/docs/contract-tests.md

@@ -1,314 +1,18 @@
-# Contract Tests
+# Endpoint contract tests
 
-The contract layer is the entry point of `@luckystack/test-runner`. It walks every endpoint that the devkit emits into `apiMethodMap.generated.ts` (and every sync route in `syncMetaMap` when one is passed), fires a single well-formed request, and asserts the response conforms to the framework envelope. It is the cheapest and broadest of the four layers — it does NOT prove business logic, it proves the route exists, decodes input without crashing, and replies with `{ status: 'success' | 'error', errorCode? }`.
+Generated route maps drive a smoke check over the API surface so tests catch mismatches between generated input schemas and live handlers. Endpoint-specific side effects and business rules still require application tests. Serial execution reduces interference from shared rate-limit state and makes failures easier to attribute; do not assume a parallel sweep is interchangeable.
 
-## When to use it
+## Probes follow the project transport
 
-- CI on every PR: catches `_api/<page>/<name>_v<n>.ts` files that have a typo in `main`, a broken import, or a renamed Zod field.
-- After regenerating the type map (`npx @luckystack/devkit gen`): confirms the routing table the runtime resolves still matches the file system.
-- During refactors of `@luckystack/api` request handling: contract failures here are a fast signal that the envelope shape regressed.
+A route is only meaningfully tested over the path real users hit (ADR 0064). API routes go over HTTP. Sync routes go over the project's own `transport.invocation`: Socket.io in a socket-mode project (the default), the routed `/sync/*` HTTP path in a routed-HTTP project; the caller may override this. The CSRF layer always probes sync over HTTP, because CSRF is an HTTP-only guard. Layer-5 `ctx.callSync` follows the same rule, so a socket-mode project's custom sync tests run through the socket pipeline and differences between the two pipelines surface as failures instead of being masked.
 
-## Functions
+- Sync routes are swept only when the sweep receives the generated sync metadata (and sync input schemas for valid samples). Without it the sweep warns and covers API routes only.
+- All sync probes in one run target a single throwaway room that the sweep's own session joins, so room-membership enforcement sees a member rather than rejecting every probe.
+- Sockets the runner opens on the caller's session close as an intentional disconnect, the signal a real client sends; a plain close would let presence log that session out shortly after the sweep.
+- GET probes carry their payload in the query string under the reserved key the real client uses (ADR 0062); `fetch` cannot send a GET body, and dropping the payload made every GET route with a required field fail validation.
 
-### `walkEndpoints(apiMethodMap)`
+## Sweep boundaries
 
-Flatten the three-level generated map (`page -> name -> version -> method`) into a flat list of endpoint descriptors. Used internally by every `run*Tests` sweep and exported so consumers can build custom layers on top of the same iteration order.
+A route filter narrows the selection before any request is sent; filtered-out routes are never called. An authenticated sweep calls every selected route with its own session, logout-type routes included: a route that destroys the current session (logout, account deletion, session revoke) leaves every later layer running anonymously, so skip such routes explicitly.
 
-Signature:
-
-```ts
-walkEndpoints(apiMethodMap: ApiMethodMap): EndpointDescriptor[]
-
-type ApiMethodMap = Record<string, Record<string, Record<string, string>>>;
-```
-
-The order is deterministic: it follows `Object.entries` over the generated map, which the devkit emits sorted by page name, then api name, then version. Empty maps return `[]` (not an error).
-
-`walkSyncEndpoints(syncMap)` is the sync counterpart: it takes the generated `syncMetaMap` (or a `syncMethodMap`-shaped map), yields `fullPath: sync/<page>/<name>/<version>`, and marks each descriptor `kind: 'sync'`. A descriptor without `kind` is an API route.
-
-### Sync routes and probe transports
-
-The checks only know "this endpoint, this payload"; a `ProbeTransport` owns the wire. API routes always go over HTTP. Sync routes go over the transport the sweep passes as `syncTransport`: `runAllTests` picks Socket.io (`createSocketProbeTransport` — the same `sync` emit a real client makes) unless the project's `transport.invocation` is `routed-http` or the caller sets `syncTransport: 'http'`, in which case `createHttpProbeTransport` posts the `{ data, receiver }` envelope to `/sync/*`. The CSRF layer always probes sync over HTTP, because CSRF is an HTTP-only guard. Every sync probe in a run targets one throwaway room; with an `authToken` the sweep session joins it first, so room-membership enforcement sees a member rather than rejecting every probe. Without `syncMetaMap`, `runAllTests` warns that sync routes are not swept.
-
-Layer-5 `ctx.callSync` follows the same rule (project `transport.invocation`, per-call `opts.transport` override) — see `docs/ARCHITECTURE_TESTING.md`.
-
-### `runContractCheck(input)`
-
-Run the contract assertion against one endpoint.
-
-Signature:
-
-```ts
-runContractCheck(input: ContractCheckInput): Promise<ContractCheckResult>
-
-interface ContractCheckInput {
-  endpoint: EndpointDescriptor;
-  baseUrl: string;
-  inputFor?: (endpoint: EndpointDescriptor) => unknown;
-  headers?: Record<string, string>;
-  requestTimeoutMs?: number; // default 5000
-  transport?: ProbeTransport; // default: HTTP
-}
-```
-
-Flow:
-
-1. Build URL: `${baseUrl.replace(/\/$/, '')}/${endpoint.fullPath}` so trailing slashes on `baseUrl` are tolerated. (HTTP transport; a socket transport emits instead — see above.)
-2. Build body: `inputFor?.(endpoint) ?? {}`. For `GET` it rides the query string under the reserved key the real client uses (core `buildRoutedGetUrl`); otherwise it is `JSON.stringify`d.
-3. Send the request with `AbortController` wired to `requestTimeoutMs`. A timeout surfaces as a `fetch` error rather than hanging the sweep.
-4. Parse the JSON body. Any parse error is a contract failure (the framework always returns JSON).
-5. Assert `parsed.status` is exactly `'success'` or `'error'`. Anything else (`null`, `undefined`, missing key, a custom shape) fails with `reason: 'Response missing standard `status` envelope'`.
-6. If `parsed.status === 'error'`, assert `parsed.errorCode` is truthy. Missing `errorCode` on an error response is a contract violation — every framework error path is required to set one.
-
-Pass criterion: a response that round-trips through the framework envelope, regardless of whether it's success or a typed error.
-
-### `runContractTests(input)`
-
-Full sweep version of `runContractCheck`. Iterates every API route (plus every sync route when `syncMetaMap` is given), narrowed by `filter` before any request is sent, serially, and aggregates results.
-
-Signature:
-
-```ts
-runContractTests(input: RunContractTestsInput): Promise<RunContractSummary>
-
-interface RunContractTestsInput {
-  apiMethodMap: ApiMethodMap;
-  syncMetaMap?: SyncMetaMap;       // sweep sync routes too
-  filter?: string;                 // substring on `<page>/<name>/<version>`
-  syncTransport?: ProbeTransport;  // HTTP when omitted
-  baseUrl: string;
-  skip?: string[];
-  inputFor?: (endpoint: EndpointDescriptor) => unknown;
-  headers?: Record<string, string>;
-  onResult?: (result: ContractCheckResult) => void;
-}
-```
-
-Behavior:
-
-- **Every route is called with the sweep's session — logout-type routes included.** An authenticated sweep that hits a route which destroys the current session (a logout, account deletion, session revoke) loses its session for every layer after it. Put such routes in `skip` (`TEST_SKIP` in the scaffolded script).
-
-- **Serial, not parallel.** The runner walks endpoints one by one. This avoids tripping shared rate-limit buckets, keeps log output linear, and stops endpoint-ordering bugs from being masked by races. A pool option may land later; do not assume concurrency.
-- **`skip` matching is two-tier.** `skip: ['billing/getInvoice']` skips every version of that endpoint. `skip: ['billing/getInvoice/v2']` skips only v2. Both forms compose: pass both to skip every version AND keep the specific one explicit.
-- **`onResult` is fired after every endpoint** including skipped ones, so progress reporters update in real time. Throwing inside `onResult` propagates out of the sweep — wrap your own handler in `tryCatch` if you want it isolated.
-- **Skipped results carry `status: 'skipped'` with `reason: 'Explicitly skipped'`** and `durationMs: 0`. They count toward `summary.skipped`, not `passed` or `failed`.
-
-Returned summary:
-
-```ts
-interface RunContractSummary {
-  total: number;     // results.length, including skipped
-  passed: number;
-  failed: number;
-  skipped: number;
-  results: ContractCheckResult[];
-}
-```
-
-### `sampleSchemaInput(schema)`
-
-Walk a Zod schema and produce a deterministic minimal-valid value. The contract layer does not call this automatically — consumers use it inside a custom `inputFor` when an endpoint's Zod validator rejects an empty object.
-
-Signature:
-
-```ts
-sampleSchemaInput(schema: z.ZodTypeAny): unknown
-```
-
-Type-to-sample table:
-
-| Zod node | Sample |
-| --- | --- |
-| `ZodString` | `'test'` |
-| `ZodNumber` | `0` |
-| `ZodBoolean` | `false` |
-| `ZodNull` | `null` |
-| `ZodUndefined` | `undefined` |
-| `ZodAny` / `ZodUnknown` | `null` |
-| `ZodLiteral` | the literal value |
-| `ZodOptional` / `ZodNullable` | `undefined` (field omitted in objects) |
-| `ZodUnion` | first option, recursively sampled |
-| `ZodArray` | `[]` |
-| `ZodObject` | object with every non-`undefined` child sampled |
-| `ZodRecord` | `{}` |
-| `ZodDate` | `new Date().toISOString()` |
-| anything else | `null` |
-
-`ZodOptional`/`ZodNullable` returning `undefined` is deliberate — the object branch drops keys whose sample is `undefined`, so optional fields stay absent rather than being sent as `null`. The result is the smallest payload that the validator will accept.
-
-This is NOT a property-based generator. It always returns the same value for the same schema. Swap in `fast-check` if you need randomized fuzz that respects the schema.
-
-### `resolveTestBaseUrl(options?)`
-
-Resolve the live server target for test entrypoints. Priority is: non-empty `TEST_BASE_URL`, a valid integer port from `<cwd>/node_modules/.luckystack/dev-server.json` whose owner PID is still alive, then `fallbackUrl` (default `http://localhost:80`). Generated projects pass `http://localhost:${ports.backend}` as the fallback, so config changes, dev auto-increment hops, and stale crash leftovers remain truthful.
-
-```ts
-resolveTestBaseUrl(options?: {
-  cwd?: string;
-  fallbackUrl?: string;
-}): string
-```
-
-This helper does not start or probe a server; connection failures remain visible to the actual test layer.
-
-### `resolveTestEnvironment(input?)`
-
-Prepare the **test process** before DB/Redis-backed integration tests run. It first calls core's `loadEnvFiles()`. When `loadProjectConfig` returns a config with a non-empty `secretManager.url`, it dynamically loads the optional `@luckystack/secret-manager` peer and runs `initSecretManager({ ...config.secretManager, source: 'remote' })`.
-
-```ts
-await resolveTestEnvironment({
-  loadProjectConfig: async () => (await import('./config')).default,
-});
-```
-
-`runAllTests` calls this automatically through its required `loadProjectConfig` callback. The official scaffolded `scripts/testAll.ts` supplies it. The callback is lazy deliberately: `.env` and `.env.local` must load before `config.ts` reads `LUCKYSTACK_SECRET_MANAGER_URL`.
-
-This is a separate process from the live server. Resolving `DATABASE_URL` during server boot does **not** resolve it for Layer-5 `ctx.prisma` calls. If a secret-manager URL is configured but the package cannot load, resolution fails loudly before any layer runs instead of handing a raw `DATABASE_URL_V<n>` pointer to Prisma.
-
-For a custom Vitest integration setup, call `resolveTestEnvironment` from a setup file before importing modules that construct env-backed clients. Direct Layer-5 callers must pass the same lazy loader to `runCustomTests({ ..., loadProjectConfig })`; it resolves before discovery/import. Both orchestrator APIs reject a missing loader at runtime, including from untyped JavaScript. `runAllTests` uses an internal already-prepared Layer-5 entrypoint, so the custom layer does not reload `.env.local` pointers over freshly resolved values.
-
-## Types
-
-```ts
-type HttpMethod = 'GET' | 'POST' | 'PUT' | 'DELETE';
-
-interface EndpointDescriptor {
-  page: string;
-  name: string;
-  version: string;
-  method: HttpMethod;
-  fullPath: string; // `api/<page>/<name>/<version>` or `sync/<page>/<name>/<version>`
-  kind?: 'api' | 'sync'; // absent = 'api'
-}
-
-interface ContractCheckResult {
-  endpoint: EndpointDescriptor;
-  status: 'pass' | 'fail' | 'skipped';
-  httpStatus?: number;
-  responseStatus?: 'success' | 'error' | 'unknown';
-  errorCode?: string;
-  reason?: string;
-  durationMs: number;
-}
-
-interface RunContractSummary {
-  total: number;
-  passed: number;
-  failed: number;
-  skipped: number;
-  results: ContractCheckResult[];
-}
-
-interface ResolveTestEnvironmentInput {
-  loadProjectConfig?: () => unknown | Promise<unknown>;
-}
-
-interface RunCustomTestsInput {
-  // other Layer-5 options omitted
-  loadProjectConfig: NonNullable<ResolveTestEnvironmentInput['loadProjectConfig']>;
-}
-```
-
-`responseStatus: 'unknown'` only appears on failed results when the JSON either failed to parse or did not carry a `status` field. `errorCode` is set on pass results when `responseStatus === 'error'` and on fail results when the auth/rate-limit layers' `errorCode` mismatch path runs (contract layer never sets it on fail).
-
-## Result shapes by branch
-
-- **Pass / success**: `{ status: 'pass', httpStatus, responseStatus: 'success', errorCode: undefined, durationMs }`.
-- **Pass / typed error**: `{ status: 'pass', httpStatus, responseStatus: 'error', errorCode: '<code>', durationMs }`. A typed framework error still satisfies the contract.
-- **Fail / fetch error**: `{ status: 'fail', reason: <Error.message>, durationMs }`. No `httpStatus`. Network refused, DNS failure, or `AbortController` firing all land here.
-- **Fail / JSON parse**: `{ status: 'fail', httpStatus, responseStatus: 'unknown', reason: 'JSON parse failed: ...', durationMs }`.
-- **Fail / missing envelope**: `{ status: 'fail', httpStatus, responseStatus: 'unknown', reason: 'Response missing standard `status` envelope', durationMs }`.
-- **Fail / error without code**: `{ status: 'fail', httpStatus, responseStatus: 'error', reason: 'Error response missing `errorCode`', durationMs }`.
-- **Skip**: `{ status: 'skipped', durationMs: 0, reason: 'Explicitly skipped' }`.
-
-## Examples
-
-### CI sweep with default inputs
-
-```ts
-import { apiMethodMap } from './generated/apiMethodMap.generated';
-import {
-  runContractTests,
-  logContractResult,
-  logContractSummary,
-  resolveTestBaseUrl,
-} from '@luckystack/test-runner';
-import { ports } from './config.ports';
-
-const summary = await runContractTests({
-  apiMethodMap,
-  baseUrl: resolveTestBaseUrl({ fallbackUrl: `http://localhost:${ports.backend}` }),
-  onResult: logContractResult,
-});
-
-logContractSummary(summary);
-
-if (summary.failed > 0) {
-  process.exit(1);
-}
-```
-
-### Endpoint-specific input via `inputFor`
-
-Most validators reject `{}`. Use `inputFor` to return a shape the validator will accept for the specific endpoints that need it; fall through to `{}` for everything else.
-
-```ts
-import { sampleSchemaInput } from '@luckystack/test-runner';
-import { sendInviteSchema } from 'src/organization-settings/_api/sendInvite_v1';
-
-await runContractTests({
-  apiMethodMap,
-  baseUrl,
-  inputFor: (endpoint) => {
-    if (endpoint.fullPath === 'api/organization-settings/sendInvite/v1') {
-      return sampleSchemaInput(sendInviteSchema);
-    }
-    return {}; // default
-  },
-});
-```
-
-### Single-endpoint debug loop
-
-```ts
-import { runContractCheck, logContractResult } from '@luckystack/test-runner';
-
-const result = await runContractCheck({
-  endpoint: {
-    page: 'billing',
-    name: 'getInvoice',
-    version: 'v1',
-    method: 'POST',
-    fullPath: 'api/billing/getInvoice/v1',
-  },
-  baseUrl: process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80',
-  inputFor: () => ({ invoiceId: 'inv_1' }),
-  headers: { Cookie: 'session=abc' },
-  requestTimeoutMs: 15000, // bump for slow endpoints
-});
-
-logContractResult(result);
-```
-
-### Skip noisy endpoints
-
-```ts
-await runContractTests({
-  apiMethodMap,
-  baseUrl,
-  skip: [
-    'uploads/fileUpload',         // needs multipart, not JSON
-    'billing/getInvoice/v1',      // deprecated, kept for backwards compat
-  ],
-});
-```
-
-## Edge cases and gotchas
-
-- **`GET` endpoints receive `inputFor` through the query string.** `fetch` cannot send a GET body, so the payload is JSON-encoded under the same reserved query key the real client sends and the server reads. Before this, the payload was dropped and every GET route with a required field failed validation. Layer 5's `ctx.callApi` uses the same encoding.
-- **`headers` are merged after `Content-Type: application/json`.** If you pass `Content-Type` yourself you will override the default; the framework parses based on the header so this is intentional.
-- **`baseUrl` trailing slash is normalized once.** `http://localhost/` and `http://localhost` produce identical requests. Embedded `//` further down the path is not collapsed.
-- **`requestTimeoutMs` is per-request, not per-sweep.** A 30-endpoint sweep with the default 5000ms timeout can run for over two minutes if every endpoint stalls. The sweep itself has no overall timeout.
-- **An endpoint that returns `204 No Content`** fails the contract because there is no JSON to parse. The framework never emits 204 from `_api/*`; this only happens if you've added a custom route handler that intercepts the path.
-- **A response with `status: 'success'` and an extra `errorCode`** still passes. The contract layer only validates the envelope keys it cares about; unknown extras are ignored.
-- **Endpoints absent from `apiMetaMap` are still walked.** The contract layer never reads `apiMetaMap`. Auth and rate-limit layers do — see those docs.
+The test process must load environment files and resolve configured secret pointers before importing code that creates database clients. The live server's resolved environment is isolated from the runner process. The loader callback is intentionally deferred until after local env layers are available. Consult the runner source for current return shapes and schema sampling; a sampled valid payload is not exhaustive input coverage.