@luckystack/test-runner 0.1.9 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (8) hide show
  1. package/CHANGELOG.md +14 -14
  2. package/CLAUDE.md +98 -98
  3. package/LICENSE +21 -21
  4. package/README.md +83 -83
  5. package/dist/index.d.ts +76 -41
  6. package/dist/index.js +586 -283
  7. package/dist/index.js.map +1 -1
  8. package/package.json +5 -4
package/CHANGELOG.md CHANGED
@@ -1,14 +1,14 @@
1
- # Changelog
2
-
3
- All notable changes to `@luckystack/test-runner` are documented in this file.
4
-
5
- The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
- and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
-
8
- ## [Unreleased]
9
-
10
- ## [0.1.0]
11
-
12
- ### Added
13
-
14
- - Initial public release as part of the LuckyStack package split.
1
+ # Changelog
2
+
3
+ All notable changes to `@luckystack/test-runner` are documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ ## [Unreleased]
9
+
10
+ ## [0.1.0]
11
+
12
+ ### Added
13
+
14
+ - Initial public release as part of the LuckyStack package split.
package/CLAUDE.md CHANGED
@@ -1,98 +1,98 @@
1
- # @luckystack/test-runner
2
-
3
- > AI summary + function INDEX. For deep specs see `docs/` next to this file.
4
-
5
- ## What this package does
6
-
7
- Generated-type-driven test layers voor LuckyStack APIs. Walkt elke endpoint uit de generated `apiMethodMap` en draait vijf progressive layers: contract smoke (Zod-valid input), auth enforcement, rate-limit, crash-resistance fuzz, en per-route custom tests (`<name>_v<N>.tests.ts` naast `_api/`/`_sync/` sources). Elke layer is independent runnable; `runAllTests` orchestreert alles in volgorde en levert een geaggregeerde summary op. Reporter/fixture/layer-extension registries laten consumers eigen checks plug-in zonder de framework te forken.
8
-
9
- ## When to USE this package
10
-
11
- - Je wil elke `_api/*` route automatisch valideren tegen de generated contract zonder per endpoint een testfile te schrijven.
12
- - Je wil in CI verifiëren dat endpoints met `auth.login: true` daadwerkelijk 401 returneren voor unauthenticated requests.
13
- - Je wil rate-limit gedrag (`rateLimit: N`) confirmeren zonder handmatig de `N+1`th request te bouwen.
14
- - Je wil nightly crash-resistance fuzz draaien op alle endpoints om 500-stacks te vangen.
15
- - Je wil custom test-layers, fixtures of een webhook reporter toevoegen aan de standard sweep.
16
-
17
- ## When to NOT suggest this (yet)
18
-
19
- - Voor unit tests van pure functions binnen `_api/*` — gebruik vitest/jest direct, niet deze runner.
20
- - Voor frontend-UI tests — gebruik Playwright/`webapp-testing` skill, deze runner is server-only.
21
- - Wanneer er geen generated `apiMethodMap` beschikbaar is (project gebruikt geen `@luckystack/devkit`).
22
- - Wanneer de target server productie is — fuzz/rate-limit layers maken honderden requests en `/_test/reset` is daar uit.
23
- - Voor sync-event testing via de auto-sweep — die walkt momenteel alleen API endpoints, niet de sync map (sync support voor de auto-layers staat op de roadmap maar is nog niet geleverd). Layer 5 (`runCustomTests` / `<name>_v<N>.tests.ts`) ondersteunt al wél sync routes via per-route testfiles naast `_sync/` sources.
24
-
25
- ## Function Index
26
-
27
- | Function / Export | 1-regel | Deep doc |
28
- |---|---|---|
29
- | `runAllTests({ apiMethodMap, apiMetaMap, apiInputSchemas, baseUrl, ... })` | Orchestrator: draait contract + auth + rate-limit + fuzz + custom (Layer 5) in volgorde, levert een `RunAllTestsSummary`. | → docs/contract-tests.md |
30
- | `runCustomTests(input)` | Layer 5: draait per-route `<name>_v<N>.tests.ts` files naast de `_api/` / `_sync/` sources, levert een `RunCustomTestsSummary`. | → docs/contract-tests.md |
31
- | `discoverCustomTestFiles(srcDir?)` | Walker die elke `<name>_v<N>.tests.ts` onder de configured src dir yields. | → docs/contract-tests.md |
32
- | `logRunAllSummary(summary)` | Pretty-print één `RunAllTestsSummary`. | → docs/extension-hooks.md |
33
- | `walkEndpoints(apiMethodMap)` | Genereert `EndpointDescriptor[]` uit de generated map | → docs/contract-tests.md |
34
- | `runContractCheck({ endpoint, baseUrl, inputFor?, headers? })` | Single-endpoint contract smoke | → docs/contract-tests.md |
35
- | `runContractTests({ apiMethodMap, baseUrl, skip?, inputFor?, headers?, onResult? })` | Full sweep contract layer | → docs/contract-tests.md |
36
- | `runAuthEnforcementCheck({ endpoint, baseUrl, inputFor? })` | Single-endpoint auth check (verwacht 401 zonder token) | → docs/auth-tests.md |
37
- | `runAuthEnforcementTests({ apiMethodMap, apiMetaMap, baseUrl, skip?, inputFor?, onResult? })` | Sweep alle `auth.login: true` endpoints | → docs/auth-tests.md |
38
- | `runRateLimitCheck({ endpoint, baseUrl, rateLimit, inputFor?, headers? })` | Single-endpoint rate-limit assertie | → docs/rate-limit-tests.md |
39
- | `runRateLimitTests({ apiMethodMap, apiMetaMap, baseUrl, skip?, inputFor?, headers?, maxRateLimitToTest?, resetBetweenEndpoints?, resetToken?, onResult? })` | Sweep alle `rateLimit: N` endpoints | → docs/rate-limit-tests.md |
40
- | `runFuzzCheck({ endpoint, baseUrl, headers? })` | Single-endpoint crash-resistance fuzz | → docs/fuzz-tests.md |
41
- | `runFuzzTests({ apiMethodMap, baseUrl, skip?, headers?, onResult? })` | Sweep fuzz layer (nightly CI) | → docs/fuzz-tests.md |
42
- | `resetServerState({ baseUrl, token? })` | POST naar `/_test/reset` om DB+Redis schoon te maken | → docs/rate-limit-tests.md |
43
- | `sampleSchemaInput(schema)` | Genereert Zod-valid sample payload uit een schema | → docs/contract-tests.md |
44
- | `logContractResult(result)` | Pretty-print één `ContractCheckResult` | → docs/extension-hooks.md |
45
- | `logContractSummary(summary)` | Pretty-print aggregate `RunContractSummary` | → docs/extension-hooks.md |
46
- | `registerTestLayer(layer)` | Plug-in custom test-layer (CORS, business rules, multi-tenant) | → docs/extension-hooks.md |
47
- | `listTestLayers()` | Lees alle geregistreerde custom layers | → docs/extension-hooks.md |
48
- | `registerTestFixture(typeKey, fixture)` | Realistic-payload fixtures per typeKey (valid + invalid) | → docs/fuzz-tests.md |
49
- | `getTestFixture(typeKey)` | Lees fixture voor een typeKey | → docs/fuzz-tests.md |
50
- | `registerTestReporter(reporter)` | Plug-in onResult/onSummary/webhookUrl callbacks | → docs/extension-hooks.md |
51
- | `getTestReporter()` | Lees actieve reporter callbacks | → docs/extension-hooks.md |
52
- | `resetTestExtensionsForTests()` | Clear alle registries (alleen voor interne tests) | → docs/extension-hooks.md |
53
- | Type: `EndpointDescriptor` | `{ page, name, version, method, fullPath }` | → docs/contract-tests.md |
54
- | Type: `HttpMethod` | `'GET' \| 'POST' \| 'PUT' \| 'DELETE'` | → docs/contract-tests.md |
55
- | Type: `ContractCheckResult` | Single-result shape (status, httpStatus, responseStatus, errorCode, reason, durationMs) | → docs/contract-tests.md |
56
- | Type: `RunContractSummary` | Aggregate shape (total, passed, failed, skipped, results[]) | → docs/contract-tests.md |
57
- | Type: `TestLayer` / `TestLayerInput` / `TestLayerResult` | Custom-layer interface | → docs/extension-hooks.md |
58
- | Type: `TestFixture` | `{ valid: T[], invalid: T[] }` per typeKey | → docs/fuzz-tests.md |
59
- | Type: `TestResult` / `TestSummary` | Aggregate shapes voor extension registry | → docs/extension-hooks.md |
60
- | Type: `TestReporter` | `{ onResult?, onSummary?, webhookUrl?, webhookAuth? }` | → docs/extension-hooks.md |
61
- | Type: `RunContractTestsInput` / `ContractCheckInput` | Inputs voor contract layer | → docs/contract-tests.md |
62
- | Type: `RunAuthEnforcementTestsInput` / `AuthEnforcementCheckInput` | Inputs voor auth layer | → docs/auth-tests.md |
63
- | Type: `RunRateLimitTestsInput` / `RateLimitCheckInput` | Inputs voor rate-limit layer | → docs/rate-limit-tests.md |
64
- | Type: `RunFuzzTestsInput` / `FuzzCheckInput` | Inputs voor fuzz layer | → docs/fuzz-tests.md |
65
- | Type: `ResetServerStateInput` | Input voor `resetServerState` | → docs/rate-limit-tests.md |
66
- | Type: `TestContext` | `{ callApi, callSync, watchStream, session, prisma, expect }` — handed naar elke Layer-5 custom-tests case. | → docs/contract-tests.md |
67
- | `ctx.watchStream(roomCode)` | Open een second-socket (socket B) joined aan `roomCode` om de chunk stream van `broadcastStream`/`streamTo`/`_client_v{N}` te observeren. Returnt `StreamWatcher<TChunk>` met `.chunks[]`, `.stopAt(predicate, timeoutMs?)`, `.waitForCount(n, timeoutMs?)`, `.close()`. Watchers worden auto-closed na elke case. | → docs/contract-tests.md |
68
- | `openStreamWatcher(input)` | Lower-level factory voor `watchStream` — direct bruikbaar buiten een `TestContext` (custom layers, ad-hoc harness scripts). | → docs/contract-tests.md |
69
- | Type: `StreamWatcher<TChunk>` | `{ readonly chunks, stopAt, waitForCount, close }` — observable chunk stream voor één route. | → docs/contract-tests.md |
70
- | Type: `StreamChunkFrame` | Wire-shape van een sync chunk frame (`{ cb, fullName, status: 'stream', ...payload }`). Default chunk type voor `StreamWatcher`. | → docs/contract-tests.md |
71
- | Type: `OpenStreamWatcherInput` | `{ baseUrl, roomCode, token, routeFullName, defaultTimeoutMs? }` voor `openStreamWatcher`. | → docs/contract-tests.md |
72
- | Type: `CustomTestCase` | `{ name, run(ctx) }` — shape die consumers exporteren uit `<name>_v<N>.tests.ts`. | → docs/contract-tests.md |
73
- | Type: `RunAllTestsSummary` | Aggregate result shape voor `runAllTests` (per-layer summaries + totals). | → docs/contract-tests.md |
74
- | Type: `RunCustomTestsSummary` | Aggregate result shape voor `runCustomTests` (per-file resultaten + totals). | → docs/contract-tests.md |
75
-
76
- ## Config keys (env vars + register* slots)
77
-
78
- - `NODE_ENV` (env, server-side) — `/_test/reset` is automatisch beschikbaar wanneer NIET `production`.
79
- - `TEST_RESET_TOKEN` (env, optional, server-side) — wanneer gezet moet `resetServerState({ token })` deze meesturen als `X-Test-Reset-Token`. Verplicht voor staging/preview deploys die het endpoint over het netwerk exposen.
80
- - `RunRateLimitTestsInput.maxRateLimitToTest` (default `50`) — endpoints met hogere `rateLimit` worden geskipt om duizenden requests in CI te vermijden.
81
- - `RunRateLimitTestsInput.resetBetweenEndpoints` (default `false`) — hit `/_test/reset` voor elke endpoint zodat het shared IP-bucket schoon is.
82
- - `registerTestLayer({ name, run })` — custom check per endpoint (CORS, business rules, custom auth-schemes, multi-tenant isolation).
83
- - `registerTestFixture(typeKey, { valid, invalid })` — realistic payloads die de fuzz layer prefereert boven schema-random.
84
- - `registerTestReporter({ onResult?, onSummary?, webhookUrl?, webhookAuth? })` — wire per-result/per-summary callbacks of POST de summary naar een webhook.
85
-
86
- ## Peer dependencies
87
-
88
- - **Required**: `zod@^4.0.0` (peer), `socket.io-client@^4.8.0` (peer — `streamWatcher.ts` opent een tweede socket voor `ctx.watchStream`), `@luckystack/core@^0.1.0` (dependency — leest `getApiMethodMapPath()` voor de default map-locatie).
89
- - **Required server-side**: `@luckystack/server` met `/_test/reset` endpoint gemount (default ingebouwd, gated op `NODE_ENV !== 'production'`).
90
- - **Required generated artefact**: `apiMethodMap.generated.ts` (en `apiMetaMap.generated.ts` voor de auth/rate-limit layers), geproduceerd door `@luckystack/devkit`.
91
- - **Optional**: geen — fixtures, reporters en custom layers zijn pure-JS opt-in.
92
-
93
- ## Related
94
-
95
- - Architecture deep-dives: `/docs/ARCHITECTURE_API.md` (het contract dat deze runner asserteert), `/docs/ARCHITECTURE_PACKAGING.md` (hoe generated maps per preset worden geëmit).
96
- - Consumer quickstart: `./README.md`.
97
- - Devkit (genereert de maps die deze runner consumeert): `packages/devkit/CLAUDE.md`.
98
- - Server (`/_test/reset` endpoint): `packages/server/CLAUDE.md`.
1
+ # @luckystack/test-runner
2
+
3
+ > AI summary + function INDEX. For deep specs see `docs/` next to this file.
4
+
5
+ ## What this package does
6
+
7
+ Generated-type-driven test layers voor LuckyStack APIs. Walkt elke endpoint uit de generated `apiMethodMap` en draait vijf progressive layers: contract smoke (Zod-valid input), auth enforcement, rate-limit, crash-resistance fuzz, en per-route custom tests (`<name>_v<N>.tests.ts` naast `_api/`/`_sync/` sources). Elke layer is independent runnable; `runAllTests` orchestreert alles in volgorde en levert een geaggregeerde summary op. Reporter/fixture/layer-extension registries laten consumers eigen checks plug-in zonder de framework te forken.
8
+
9
+ ## When to USE this package
10
+
11
+ - Je wil elke `_api/*` route automatisch valideren tegen de generated contract zonder per endpoint een testfile te schrijven.
12
+ - Je wil in CI verifiëren dat endpoints met `auth.login: true` daadwerkelijk 401 returneren voor unauthenticated requests.
13
+ - Je wil rate-limit gedrag (`rateLimit: N`) confirmeren zonder handmatig de `N+1`th request te bouwen.
14
+ - Je wil nightly crash-resistance fuzz draaien op alle endpoints om 500-stacks te vangen.
15
+ - Je wil custom test-layers, fixtures of een webhook reporter toevoegen aan de standard sweep.
16
+
17
+ ## When to NOT suggest this (yet)
18
+
19
+ - Voor unit tests van pure functions binnen `_api/*` — gebruik vitest/jest direct, niet deze runner.
20
+ - Voor frontend-UI tests — gebruik Playwright/`webapp-testing` skill, deze runner is server-only.
21
+ - Wanneer er geen generated `apiMethodMap` beschikbaar is (project gebruikt geen `@luckystack/devkit`).
22
+ - Wanneer de target server productie is — fuzz/rate-limit layers maken honderden requests en `/_test/reset` is daar uit.
23
+ - Voor sync-event testing via de auto-sweep — die walkt momenteel alleen API endpoints, niet de sync map (sync support voor de auto-layers staat op de roadmap maar is nog niet geleverd). Layer 5 (`runCustomTests` / `<name>_v<N>.tests.ts`) ondersteunt al wél sync routes via per-route testfiles naast `_sync/` sources.
24
+
25
+ ## Function Index
26
+
27
+ | Function / Export | 1-regel | Deep doc |
28
+ |---|---|---|
29
+ | `runAllTests({ apiMethodMap, apiMetaMap, apiInputSchemas, baseUrl, ... })` | Orchestrator: draait contract + auth + rate-limit + fuzz + custom (Layer 5) in volgorde, levert een `RunAllTestsSummary`. | → docs/contract-tests.md |
30
+ | `runCustomTests(input)` | Layer 5: draait per-route `<name>_v<N>.tests.ts` files naast de `_api/` / `_sync/` sources, levert een `RunCustomTestsSummary`. | → docs/contract-tests.md |
31
+ | `discoverCustomTestFiles(srcDir?)` | Walker die elke `<name>_v<N>.tests.ts` onder de configured src dir yields. | → docs/contract-tests.md |
32
+ | `logRunAllSummary(summary)` | Pretty-print één `RunAllTestsSummary`. | → docs/extension-hooks.md |
33
+ | `walkEndpoints(apiMethodMap)` | Genereert `EndpointDescriptor[]` uit de generated map | → docs/contract-tests.md |
34
+ | `runContractCheck({ endpoint, baseUrl, inputFor?, headers? })` | Single-endpoint contract smoke | → docs/contract-tests.md |
35
+ | `runContractTests({ apiMethodMap, baseUrl, skip?, inputFor?, headers?, onResult? })` | Full sweep contract layer | → docs/contract-tests.md |
36
+ | `runAuthEnforcementCheck({ endpoint, baseUrl, inputFor? })` | Single-endpoint auth check (verwacht 401 zonder token) | → docs/auth-tests.md |
37
+ | `runAuthEnforcementTests({ apiMethodMap, apiMetaMap, baseUrl, skip?, inputFor?, onResult? })` | Sweep alle `auth.login: true` endpoints | → docs/auth-tests.md |
38
+ | `runRateLimitCheck({ endpoint, baseUrl, rateLimit, inputFor?, headers? })` | Single-endpoint rate-limit assertie | → docs/rate-limit-tests.md |
39
+ | `runRateLimitTests({ apiMethodMap, apiMetaMap, baseUrl, skip?, inputFor?, headers?, maxRateLimitToTest?, resetBetweenEndpoints?, resetToken?, onResult? })` | Sweep alle `rateLimit: N` endpoints | → docs/rate-limit-tests.md |
40
+ | `runFuzzCheck({ endpoint, baseUrl, headers? })` | Single-endpoint crash-resistance fuzz | → docs/fuzz-tests.md |
41
+ | `runFuzzTests({ apiMethodMap, baseUrl, skip?, headers?, onResult? })` | Sweep fuzz layer (nightly CI) | → docs/fuzz-tests.md |
42
+ | `resetServerState({ baseUrl, token? })` | POST naar `/_test/reset` om DB+Redis schoon te maken | → docs/rate-limit-tests.md |
43
+ | `sampleSchemaInput(schema)` | Genereert Zod-valid sample payload uit een schema | → docs/contract-tests.md |
44
+ | `logContractResult(result)` | Pretty-print één `ContractCheckResult` | → docs/extension-hooks.md |
45
+ | `logContractSummary(summary)` | Pretty-print aggregate `RunContractSummary` | → docs/extension-hooks.md |
46
+ | `registerTestLayer(layer)` | Plug-in custom test-layer (CORS, business rules, multi-tenant) | → docs/extension-hooks.md |
47
+ | `listTestLayers()` | Lees alle geregistreerde custom layers | → docs/extension-hooks.md |
48
+ | `registerTestFixture(typeKey, fixture)` | Realistic-payload fixtures per typeKey (valid + invalid) | → docs/fuzz-tests.md |
49
+ | `getTestFixture(typeKey)` | Lees fixture voor een typeKey | → docs/fuzz-tests.md |
50
+ | `registerTestReporter(reporter)` | Plug-in onResult/onSummary/webhookUrl callbacks | → docs/extension-hooks.md |
51
+ | `getTestReporter()` | Lees actieve reporter callbacks | → docs/extension-hooks.md |
52
+ | `resetTestExtensionsForTests()` | Clear alle registries (alleen voor interne tests) | → docs/extension-hooks.md |
53
+ | Type: `EndpointDescriptor` | `{ page, name, version, method, fullPath }` | → docs/contract-tests.md |
54
+ | Type: `HttpMethod` | `'GET' \| 'POST' \| 'PUT' \| 'DELETE'` | → docs/contract-tests.md |
55
+ | Type: `ContractCheckResult` | Single-result shape (status, httpStatus, responseStatus, errorCode, reason, durationMs) | → docs/contract-tests.md |
56
+ | Type: `RunContractSummary` | Aggregate shape (total, passed, failed, skipped, results[]) | → docs/contract-tests.md |
57
+ | Type: `TestLayer` / `TestLayerInput` / `TestLayerResult` | Custom-layer interface | → docs/extension-hooks.md |
58
+ | Type: `TestFixture` | `{ valid: T[], invalid: T[] }` per typeKey | → docs/fuzz-tests.md |
59
+ | Type: `TestResult` / `TestSummary` | Aggregate shapes voor extension registry | → docs/extension-hooks.md |
60
+ | Type: `TestReporter` | `{ onResult?, onSummary?, webhookUrl?, webhookAuth? }` | → docs/extension-hooks.md |
61
+ | Type: `RunContractTestsInput` / `ContractCheckInput` | Inputs voor contract layer | → docs/contract-tests.md |
62
+ | Type: `RunAuthEnforcementTestsInput` / `AuthEnforcementCheckInput` | Inputs voor auth layer | → docs/auth-tests.md |
63
+ | Type: `RunRateLimitTestsInput` / `RateLimitCheckInput` | Inputs voor rate-limit layer | → docs/rate-limit-tests.md |
64
+ | Type: `RunFuzzTestsInput` / `FuzzCheckInput` | Inputs voor fuzz layer | → docs/fuzz-tests.md |
65
+ | Type: `ResetServerStateInput` | Input voor `resetServerState` | → docs/rate-limit-tests.md |
66
+ | Type: `TestContext` | `{ callApi, callSync, watchStream, session, prisma, expect }` — handed naar elke Layer-5 custom-tests case. | → docs/contract-tests.md |
67
+ | `ctx.watchStream(roomCode)` | Open een second-socket (socket B) joined aan `roomCode` om de chunk stream van `broadcastStream`/`streamTo`/`_client_v{N}` te observeren. Returnt `StreamWatcher<TChunk>` met `.chunks[]`, `.stopAt(predicate, timeoutMs?)`, `.waitForCount(n, timeoutMs?)`, `.close()`. Watchers worden auto-closed na elke case. | → docs/contract-tests.md |
68
+ | `openStreamWatcher(input)` | Lower-level factory voor `watchStream` — direct bruikbaar buiten een `TestContext` (custom layers, ad-hoc harness scripts). | → docs/contract-tests.md |
69
+ | Type: `StreamWatcher<TChunk>` | `{ readonly chunks, stopAt, waitForCount, close }` — observable chunk stream voor één route. | → docs/contract-tests.md |
70
+ | Type: `StreamChunkFrame` | Wire-shape van een sync chunk frame (`{ cb, fullName, status: 'stream', ...payload }`). Default chunk type voor `StreamWatcher`. | → docs/contract-tests.md |
71
+ | Type: `OpenStreamWatcherInput` | `{ baseUrl, roomCode, token, routeFullName, defaultTimeoutMs? }` voor `openStreamWatcher`. | → docs/contract-tests.md |
72
+ | Type: `CustomTestCase` | `{ name, run(ctx) }` — shape die consumers exporteren uit `<name>_v<N>.tests.ts`. | → docs/contract-tests.md |
73
+ | Type: `RunAllTestsSummary` | Aggregate result shape voor `runAllTests` (per-layer summaries + totals). | → docs/contract-tests.md |
74
+ | Type: `RunCustomTestsSummary` | Aggregate result shape voor `runCustomTests` (per-file resultaten + totals). | → docs/contract-tests.md |
75
+
76
+ ## Config keys (env vars + register* slots)
77
+
78
+ - `NODE_ENV` (env, server-side) — `/_test/reset` is automatisch beschikbaar wanneer NIET `production`.
79
+ - `TEST_RESET_TOKEN` (env, optional, server-side) — wanneer gezet moet `resetServerState({ token })` deze meesturen als `X-Test-Reset-Token`. Verplicht voor staging/preview deploys die het endpoint over het netwerk exposen.
80
+ - `RunRateLimitTestsInput.maxRateLimitToTest` (default `50`) — endpoints met hogere `rateLimit` worden geskipt om duizenden requests in CI te vermijden.
81
+ - `RunRateLimitTestsInput.resetBetweenEndpoints` (default `false`) — hit `/_test/reset` voor elke endpoint zodat het shared IP-bucket schoon is.
82
+ - `registerTestLayer({ name, run })` — custom check per endpoint (CORS, business rules, custom auth-schemes, multi-tenant isolation).
83
+ - `registerTestFixture(typeKey, { valid, invalid })` — realistic payloads die de fuzz layer prefereert boven schema-random.
84
+ - `registerTestReporter({ onResult?, onSummary?, webhookUrl?, webhookAuth? })` — wire per-result/per-summary callbacks of POST de summary naar een webhook.
85
+
86
+ ## Peer dependencies
87
+
88
+ - **Required**: `zod@^4.0.0` (peer), `socket.io-client@^4.8.0` (peer — `streamWatcher.ts` opent een tweede socket voor `ctx.watchStream`), `@luckystack/core@^0.1.0` (dependency — leest `getApiMethodMapPath()` voor de default map-locatie).
89
+ - **Required server-side**: `@luckystack/server` met `/_test/reset` endpoint gemount (default ingebouwd, gated op `NODE_ENV !== 'production'`).
90
+ - **Required generated artefact**: `apiMethodMap.generated.ts` (en `apiMetaMap.generated.ts` voor de auth/rate-limit layers), geproduceerd door `@luckystack/devkit`.
91
+ - **Optional**: geen — fixtures, reporters en custom layers zijn pure-JS opt-in.
92
+
93
+ ## Related
94
+
95
+ - Architecture deep-dives: `/docs/ARCHITECTURE_API.md` (het contract dat deze runner asserteert), `/docs/ARCHITECTURE_PACKAGING.md` (hoe generated maps per preset worden geëmit).
96
+ - Consumer quickstart: `./README.md`.
97
+ - Devkit (genereert de maps die deze runner consumeert): `packages/devkit/CLAUDE.md`.
98
+ - Server (`/_test/reset` endpoint): `packages/server/CLAUDE.md`.
package/LICENSE CHANGED
@@ -1,21 +1,21 @@
1
- MIT License
2
-
3
- Copyright (c) 2026 Mathijs van Melick
4
-
5
- Permission is hereby granted, free of charge, to any person obtaining a copy
6
- of this software and associated documentation files (the "Software"), to deal
7
- in the Software without restriction, including without limitation the rights
8
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
- copies of the Software, and to permit persons to whom the Software is
10
- furnished to do so, subject to the following conditions:
11
-
12
- The above copyright notice and this permission notice shall be included in all
13
- copies or substantial portions of the Software.
14
-
15
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
- SOFTWARE.
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Mathijs van Melick
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,83 +1,83 @@
1
- # @luckystack/test-runner
2
-
3
- > Generated-type-driven test layers for [LuckyStack](https://github.com/ItsLucky23/LuckyStack-v2). Walks every endpoint in your `apiMethodMap.generated.ts` and runs four progressive test layers: contract smoke (Zod-valid input), auth enforcement, rate-limit, and crash-resistance fuzz.
4
-
5
- ## Install
6
-
7
- ```bash
8
- npm install --save-dev @luckystack/test-runner zod
9
- ```
10
-
11
- ## Quickstart
12
-
13
- Run all four layers against a running server:
14
-
15
- ```ts
16
- import {
17
- runContractTests,
18
- runAuthEnforcementTests,
19
- runRateLimitTests,
20
- runFuzzTests,
21
- resetServerState,
22
- logContractSummary,
23
- } from '@luckystack/test-runner';
24
- import { apiMethodMap } from './src/_sockets/apiMethodMap.generated';
25
- import { apiMetaMap } from './src/_sockets/apiMetaMap.generated';
26
-
27
- const baseUrl = process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80';
28
- const contract = await runContractTests({ apiMethodMap, baseUrl });
29
- logContractSummary(contract);
30
-
31
- const auth = await runAuthEnforcementTests({ apiMethodMap, apiMetaMap, baseUrl });
32
- const rate = await runRateLimitTests({ apiMethodMap, apiMetaMap, baseUrl });
33
- const fuzz = await runFuzzTests({ apiMethodMap, baseUrl });
34
-
35
- await resetServerState({ baseUrl });
36
- ```
37
-
38
- Each layer is independently runnable — you can skip the ones that don't apply or rearrange them.
39
-
40
- ## Layers
41
-
42
- | Layer | What it asserts |
43
- | --- | --- |
44
- | **Contract** | A Zod-valid input payload reaches `main` and returns `{ status: 'success' }` (or a typed failure status). Catches schema/route mismatches. |
45
- | **Auth enforcement** | Endpoints flagged `auth.login: true` reject unauthenticated requests with the framework's standard 401 shape. |
46
- | **Rate limit** | Endpoints with `rateLimit: <N>` reject the `(N + 1)`th request inside the window. |
47
- | **Fuzz** | Bad/random inputs never crash the server (still returns a typed error response, never 500 with a stack). |
48
-
49
- The fuzz layer is intentionally exhaustive and slow — wire it into nightly CI rather than every PR.
50
-
51
- ## Server hooks it depends on
52
-
53
- - `/_test/reset` endpoint — served by `@luckystack/server` outside production (gated on `NODE_ENV !== 'production'` and an optional `TEST_RESET_TOKEN`). Used by `resetServerState` to clear DB + Redis between layers. Make sure `TEST_RESET_TOKEN` is set in any non-prod environment that is reachable over the network.
54
- - `apiMethodMap.generated.ts` — produced by `@luckystack/devkit` from your `_api/*` files. Defaults are read via `getApiMethodMapPath()` from `@luckystack/core`.
55
-
56
- ## Public API
57
-
58
- | Export | Purpose |
59
- | --- | --- |
60
- | `walkEndpoints(apiMethodMap)` | Returns `EndpointDescriptor[]` from the generated map. |
61
- | `runContractCheck(input)` / `runContractTests(input)` | Single endpoint / full sweep. |
62
- | `runAuthEnforcementCheck(input)` / `runAuthEnforcementTests(input)` | Auth layer. |
63
- | `runRateLimitCheck(input)` / `runRateLimitTests(input)` | Rate-limit layer. |
64
- | `runFuzzCheck(input)` / `runFuzzTests(input)` | Fuzz layer. |
65
- | `resetServerState(input)` | POST to `/_test/reset`. |
66
- | `sampleSchemaInput(schema)` | Generate a Zod-valid sample payload. |
67
- | `logContractResult` / `logContractSummary` | Pretty-printers. |
68
-
69
- Types: `EndpointDescriptor`, `HttpMethod`, `ContractCheckResult`, `RunContractSummary`, plus per-layer input types.
70
-
71
- ## Related architecture docs
72
-
73
- - [`docs/ARCHITECTURE_API.md`](../../docs/ARCHITECTURE_API.md) — the contract this runner asserts against.
74
- - [`docs/ARCHITECTURE_PACKAGING.md`](../../docs/ARCHITECTURE_PACKAGING.md) — how generated maps are emitted per preset.
75
-
76
- ## Dependencies
77
-
78
- - Peer (canonical ranges, standardized 2026-05-07):
79
- - `zod@^3.25.0`
80
-
81
- ## License
82
-
83
- MIT — see [LICENSE](../../LICENSE).
1
+ # @luckystack/test-runner
2
+
3
+ > Generated-type-driven test layers for [LuckyStack](https://github.com/ItsLucky23/LuckyStack-v2). Walks every endpoint in your `apiMethodMap.generated.ts` and runs four progressive test layers: contract smoke (Zod-valid input), auth enforcement, rate-limit, and crash-resistance fuzz.
4
+
5
+ ## Install
6
+
7
+ ```bash
8
+ npm install --save-dev @luckystack/test-runner zod
9
+ ```
10
+
11
+ ## Quickstart
12
+
13
+ Run all four layers against a running server:
14
+
15
+ ```ts
16
+ import {
17
+ runContractTests,
18
+ runAuthEnforcementTests,
19
+ runRateLimitTests,
20
+ runFuzzTests,
21
+ resetServerState,
22
+ logContractSummary,
23
+ } from '@luckystack/test-runner';
24
+ import { apiMethodMap } from './src/_sockets/apiMethodMap.generated';
25
+ import { apiMetaMap } from './src/_sockets/apiMetaMap.generated';
26
+
27
+ const baseUrl = process.env.TEST_BASE_URL ?? 'http://127.0.0.1:80';
28
+ const contract = await runContractTests({ apiMethodMap, baseUrl });
29
+ logContractSummary(contract);
30
+
31
+ const auth = await runAuthEnforcementTests({ apiMethodMap, apiMetaMap, baseUrl });
32
+ const rate = await runRateLimitTests({ apiMethodMap, apiMetaMap, baseUrl });
33
+ const fuzz = await runFuzzTests({ apiMethodMap, baseUrl });
34
+
35
+ await resetServerState({ baseUrl });
36
+ ```
37
+
38
+ Each layer is independently runnable — you can skip the ones that don't apply or rearrange them.
39
+
40
+ ## Layers
41
+
42
+ | Layer | What it asserts |
43
+ | --- | --- |
44
+ | **Contract** | A Zod-valid input payload reaches `main` and returns `{ status: 'success' }` (or a typed failure status). Catches schema/route mismatches. |
45
+ | **Auth enforcement** | Endpoints flagged `auth.login: true` reject unauthenticated requests with the framework's standard 401 shape. |
46
+ | **Rate limit** | Endpoints with `rateLimit: <N>` reject the `(N + 1)`th request inside the window. |
47
+ | **Fuzz** | Bad/random inputs never crash the server (still returns a typed error response, never 500 with a stack). |
48
+
49
+ The fuzz layer is intentionally exhaustive and slow — wire it into nightly CI rather than every PR.
50
+
51
+ ## Server hooks it depends on
52
+
53
+ - `/_test/reset` endpoint — served by `@luckystack/server` outside production (gated on `NODE_ENV !== 'production'` and an optional `TEST_RESET_TOKEN`). Used by `resetServerState` to clear DB + Redis between layers. Make sure `TEST_RESET_TOKEN` is set in any non-prod environment that is reachable over the network.
54
+ - `apiMethodMap.generated.ts` — produced by `@luckystack/devkit` from your `_api/*` files. Defaults are read via `getApiMethodMapPath()` from `@luckystack/core`.
55
+
56
+ ## Public API
57
+
58
+ | Export | Purpose |
59
+ | --- | --- |
60
+ | `walkEndpoints(apiMethodMap)` | Returns `EndpointDescriptor[]` from the generated map. |
61
+ | `runContractCheck(input)` / `runContractTests(input)` | Single endpoint / full sweep. |
62
+ | `runAuthEnforcementCheck(input)` / `runAuthEnforcementTests(input)` | Auth layer. |
63
+ | `runRateLimitCheck(input)` / `runRateLimitTests(input)` | Rate-limit layer. |
64
+ | `runFuzzCheck(input)` / `runFuzzTests(input)` | Fuzz layer. |
65
+ | `resetServerState(input)` | POST to `/_test/reset`. |
66
+ | `sampleSchemaInput(schema)` | Generate a Zod-valid sample payload. |
67
+ | `logContractResult` / `logContractSummary` | Pretty-printers. |
68
+
69
+ Types: `EndpointDescriptor`, `HttpMethod`, `ContractCheckResult`, `RunContractSummary`, plus per-layer input types.
70
+
71
+ ## Related architecture docs
72
+
73
+ - [`docs/ARCHITECTURE_API.md`](../../docs/ARCHITECTURE_API.md) — the contract this runner asserts against.
74
+ - [`docs/ARCHITECTURE_PACKAGING.md`](../../docs/ARCHITECTURE_PACKAGING.md) — how generated maps are emitted per preset.
75
+
76
+ ## Dependencies
77
+
78
+ - Peer (canonical ranges, standardized 2026-05-07):
79
+ - `zod@^3.25.0`
80
+
81
+ ## License
82
+
83
+ MIT — see [LICENSE](../../LICENSE).