@mindbricks/genesis-set 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/CHANGELOG.md +34 -0
  2. package/LICENSE +202 -0
  3. package/README.md +147 -2
  4. package/bin/genesis-set.mjs +153 -0
  5. package/corpus/README.md +25 -0
  6. package/corpus/genesis/compile-results.md +246 -0
  7. package/corpus/genesis/compiler-api.md +265 -0
  8. package/corpus/genesis/custom-code.md +519 -0
  9. package/corpus/genesis/deploy.md +190 -0
  10. package/corpus/genesis/pitfalls.md +231 -0
  11. package/corpus/genesis/procedures/README.md +17 -0
  12. package/corpus/genesis/procedures/agent-hub-blueprint.md +13 -0
  13. package/corpus/genesis/procedures/auth-blueprint.md +14 -0
  14. package/corpus/genesis/procedures/bff-blueprint.md +16 -0
  15. package/corpus/genesis/procedures/blueprint-authoring.md +16 -0
  16. package/corpus/genesis/procedures/howto-mail-to-tasks.md +26 -0
  17. package/corpus/genesis/procedures/howto-payment-flow.md +112 -0
  18. package/corpus/genesis/procedures/howto-subscription.md +33 -0
  19. package/corpus/genesis/procedures/notifications-blueprint.md +13 -0
  20. package/corpus/genesis/procedures/patterns-catalog.md +12 -0
  21. package/corpus/genesis/procedures/project-lifecycle.md +27 -0
  22. package/corpus/genesis/run-locally.md +310 -0
  23. package/corpus/genesis/testing.md +532 -0
  24. package/lib/config-cli.mjs +179 -0
  25. package/lib/install.mjs +639 -0
  26. package/lib/mcp-config.mjs +220 -0
  27. package/lib/semver.mjs +94 -0
  28. package/mcp/blueprint.mjs +64 -0
  29. package/mcp/client.mjs +141 -0
  30. package/mcp/config.mjs +166 -0
  31. package/mcp/key-page-html.mjs +275 -0
  32. package/mcp/key-page.mjs +217 -0
  33. package/mcp/key-store.mjs +204 -0
  34. package/mcp/key-tools.mjs +225 -0
  35. package/mcp/server.mjs +164 -0
  36. package/mcp/tools.mjs +270 -0
  37. package/package.json +43 -4
  38. package/skills/genesis-compiler-api/SKILL.md +56 -0
  39. package/skills/genesis-deploy/SKILL.md +36 -0
  40. package/skills/genesis-generated-code/SKILL.md +48 -0
  41. package/skills/genesis-run-and-test/SKILL.md +43 -0
@@ -0,0 +1,532 @@
1
+ # Testing the generated services and seeding sample data
2
+
3
+ How to prove that the services Genesis generated do what the blueprint says, and how to fill
4
+ a running stack with a small, believable dataset. Genesis runs nothing for you: the suite and
5
+ the seed are code in your own repo, run against your own stack (see [run-locally.md](run-locally.md)),
6
+ and they read your own logs and database when a case goes red.
7
+
8
+ For the plan and the report documents that mind-set writes, see
9
+ `procedures/design/writers/test-plan.md` and `procedures/design/writers/test-report.md`
10
+ (schema: `procedures/design/schemas/test-report.v1.json`). This page does not repeat them; it
11
+ covers the suite itself.
12
+
13
+ ## 1. What the suite is
14
+
15
+ - Plain Node (>= 20) ES modules under `tests/` in a repo you own (for example
16
+ `<codename>-tests`). No test framework, no dependencies.
17
+ - It drives the public HTTP surface the way a client does: real registration and login, real
18
+ fixtures built through the APIs, one observable assertion per designer API.
19
+ - It is not a unit test of generated code, and it never writes rows straight into the database.
20
+ A row the API could not have created proves nothing and pollutes later cases.
21
+ - It proves: every designer API has a case that passed, or a skip with a stated reason, and the
22
+ number of APIs with neither is zero. Completeness comes from a script, never from your own claim.
23
+
24
+ Layout:
25
+
26
+ ```
27
+ tests/
28
+ helpers/config.mjs base URLs per service, User-Agent, random suffix, credential resolution
29
+ helpers/http.mjs JSON in/out, bearer token, brief() for evidence
30
+ helpers/asserts.mjs status checks, envelope unwrap, list/get envelope checks
31
+ helpers/runner.mjs Suite(service).case(api, title, fn) / .skip(api, reason)
32
+ helpers/session.mjs identity bootstrap (section 5), SkipError
33
+ helpers/fixtures.mjs cross-service fixtures, built once per process (section 6)
34
+ <service>.flow.mjs ONE per service, runnable alone
35
+ run-all.mjs every flow in one process, in dependency order -> report.json
36
+ coverage-gate.mjs API list vs report.json; exit 0 only at 0 MISSING
37
+ inventory/<service>.json the committed route lists the gate reads
38
+ seed/seed-sample-data.mjs the sample-data seed (section 10)
39
+ ```
40
+
41
+ Commit `inventory/` so a revision's gate can be reproduced. Git-ignore `report.json`.
42
+
43
+ ## 2. Conventions that make the gate computable
44
+
45
+ - **Case key = `(service, apiName)`**, with `apiName` copied verbatim from the blueprint (and the
46
+ service spec's Business APIs section). One case per API is the minimum; extra cases are fine
47
+ and count as "extra".
48
+ - **The population** is every designer-authored API of every service, including APIs authored on
49
+ `auth`. A framework built-in of a system service (auth's `login`, `register`, `currentUser`,
50
+ `mintSessionForUser`, the api-key and machine-session routes, the notification, agent-hub and
51
+ bff surfaces) counts only when the test plan lists that flow; a listed built-in the product
52
+ does not use gets a `BUILT-IN-UNUSED` skip (section 8). An unlisted built-in is out of scope,
53
+ never MISSING.
54
+ - **Copy routes, never compose them.** The method and path of a case come from the route list the
55
+ generated service ships, not from the API name or a guess:
56
+ - `_api.json` at the root of the service repo: one JSON file listing every HTTP, socket and SSE
57
+ endpoint (`operationId`, `method`, `path`, auth, path/query parameters, request and response
58
+ shapes, the tenant header where the API reads the session tenant). Read it without booting the
59
+ service.
60
+ - `GET /openapi.json` (and `/swagger` for a browser) on the running service: the same surface
61
+ as OpenAPI 3.
62
+ Framework auth routes have their own shapes (`/login`, `/register`, ...); custom business APIs
63
+ mount under `/v1/`. `serviceBase(service)` in `config.mjs` is the only place that knows how a
64
+ service is addressed on your stack (local ports, a gateway, or the container URLs your deployer
65
+ set).
66
+ - **Send a `User-Agent` header on every request.** Many front doors refuse the default
67
+ `curl` / `python` / `node` agents (Cloudflare error 1010); without one the suite fails before
68
+ the first assertion and looks like an outage.
69
+ - **Anchor dates to the real today** (`procedures/pitfalls.md`). Put a random per-process suffix in
70
+ every id, SKU, slug and email so re-runs never hit a unique index.
71
+ - **Accept the 200 family** where the API answers 201 (register, login, most creates):
72
+ `expectStatus(res, [200, 201])`. The session and list envelopes are in `procedures/pitfalls.md`.
73
+ - **No secret in a tracked file.** `brief()` truncates bodies and never prints headers. No token,
74
+ password or provider key is logged, reported or committed. Fake tokens in negative auth cases
75
+ are fine.
76
+
77
+ ## 3. Scaffold, then fill
78
+
79
+ Never author the whole suite in one output. The order follows what each piece reads:
80
+
81
+ 1. **Route list.** For every service (system services too), copy the route rows from `_api.json`
82
+ into `inventory/<service>.json` next to the blueprint's API names. The dump is stale after any
83
+ rebuild of that service: refresh it before you gate again.
84
+ 2. **Helpers once, then `auth.flow.mjs`.** Every other flow gets its sessions from it. Run it
85
+ alone (`node tests/auth.flow.mjs`) until the identity bootstrap of section 5 is green.
86
+ 3. **One flow per service in dependency order** (catalog before order before review; bff and
87
+ notification last). Per API: read the route row, write the case (fixtures, one call, one
88
+ observable assertion), run that flow, fix, next API. Kafka-only, cron-only, internal-only and
89
+ machine-only rows are driven through the test transport of section 7, not skipped.
90
+ 4. **`node tests/run-all.mjs`**, then **`node tests/coverage-gate.mjs`**. The gate's MISSING list
91
+ is the fill queue; the loop ends at `GATE: PASS`.
92
+
93
+ Every case has one shape: get a session (`ctx.customer()`, `ctx.staff()`, `ctx.superAdmin()`),
94
+ get the fixture it needs, make ONE call, assert the status and the one thing the API promises
95
+ (an `id` in the list rows, a transition's resulting state, a 403 for the wrong role on a
96
+ role-gated API).
97
+
98
+ ### A 2xx is not the whole verdict on a write
99
+
100
+ A create, update or delete whose row committed but whose post-write actions did not all
101
+ complete answers `200` / `201` WITH an `actionFailures: [{ action, error, retryable?, errorCode?, outbox? }]`
102
+ member. So:
103
+
104
+ - Every write case asserts `body.actionFailures === undefined` next to the status; do it once in
105
+ the helper that reads a single-item body.
106
+ - When it is present, **the row exists. Never re-run a create to "retry" it**: that makes a second
107
+ row (one pilot stranded 17 that way). The case fails, naming each `action` and `error`; triage
108
+ reads the named action, not the status.
109
+ - `outbox: "queued"` (a dropped event the service republishes itself) is the one entry a case may
110
+ tolerate, and only when it asserts nothing downstream of that event.
111
+ - `errorCode: "ACTION_NOT_RUN"` entries are consequences: fix the first entry and they go away.
112
+
113
+ ### An approval-gated API answers `403 APPROVAL_REQUIRED` first
114
+
115
+ An API whose blueprint sets `authOptions.requireApproval` answers its first call with `403` and
116
+ `errorCode: "APPROVAL_REQUIRED"`, sends a one-time code to the caller's verified email or mobile,
117
+ and runs only when the SAME request (same parameters, same user) is repeated with the headers
118
+ `api-approval-id` and `api-approval-code`. That is a step, never a red.
119
+
120
+ The challenge is in the problem body's `errors` member: `{ approvalId, channel, codeIndex,
121
+ expireTime, expiresAt, actionName, actionDescription }`, plus `secretCode` only where the service
122
+ runs in a dev-like environment (the dev slot). On stage and prod the code is never echoed.
123
+
124
+ Keep one helper, `approvedCall(method, url, body, token)`:
125
+
126
+ 1. Make the call. Anything other than `403` + `APPROVAL_REQUIRED` is returned as is.
127
+ 2. On the challenge, read `errors.approvalId` and `errors.secretCode`. With no `secretCode`, assert
128
+ the challenge shape and throw `SkipError('ENVIRONMENT: approval code not echoed on <env> - the challenge is the evidence')`.
129
+ Never guess a code.
130
+ 3. Repeat the identical call with the two headers and return that response, plus the challenge
131
+ facts for the case to assert on.
132
+
133
+ A case on such an API asserts both legs: the first answer was the challenge, and the retry is the
134
+ API's normal success. Never widen a status set to `[200, 403]`; never retry with changed
135
+ parameters (that is `403 APPROVAL_INVALID` by design, worth one negative case). The other answers:
136
+
137
+ | Answer | Meaning |
138
+ |---|---|
139
+ | `403 APPROVAL_INVALID` | wrong, used or expired code, or changed input; the approval is gone after 5 attempts |
140
+ | `429 APPROVAL_RESEND_WINDOW` | a code for a different input went out under a minute ago: wait, or finish the first approval |
141
+ | `403 APPROVAL_NOT_AVAILABLE` | the actor has no verified email or mobile: an identity finding, like `BLOCKED-IDENTITY` |
142
+
143
+ Send an idempotency key unchanged on both legs.
144
+
145
+ ## 4. Running and reporting
146
+
147
+ - `node tests/<service>.flow.mjs` runs one flow: the authoring unit.
148
+ - `node tests/run-all.mjs` runs all flows in ONE process, so sessions and fixtures are shared,
149
+ and writes `tests/report.json`. Exit code 1 on any fail.
150
+ - `node tests/coverage-gate.mjs` takes the best verdict per `(service, api)` (pass > fail > skip),
151
+ prints per service covered / failing / skipped (with reasons) / MISSING / extra, and exits 0 only
152
+ at 0 MISSING. A gate PASS with fails is still a red suite: the gate measures coverage, the run
153
+ measures health. Report both numbers.
154
+ - Run one suite at a time. Two concurrent runs race each other's fixtures (unique slugs, cart
155
+ state, order transitions). Do not run the suite while a rebuild or a redeploy is in flight, and
156
+ make sure the stack runs the build you think it runs (check `/health` after a restart): a run
157
+ against the previous build "proves" a fix that is not live.
158
+
159
+ If you want the run recorded as a test report, make `report.json` match the schema of
160
+ `procedures/design/schemas/test-report.v1.json`. The shape the schema expects:
161
+
162
+ ```json
163
+ { "project": "<codename>", "env": "dev", "base": "<base url>",
164
+ "ranAt": "<iso>", "durationMs": 86498,
165
+ "summary": { "pass": 84, "fail": 0, "skip": 40 },
166
+ "services": { "auth": { "pass": 44, "fail": 0, "skip": 4 } },
167
+ "results": [ { "service": "auth", "api": "login", "title": "fresh user can log in",
168
+ "status": "pass", "ms": 1287 },
169
+ { "service": "order", "api": "shipOrder", "title": "...",
170
+ "status": "skip", "reason": "KNOWN-DEFECT <ref>: ..." } ] }
171
+ ```
172
+
173
+ `summary` and every `services.<service>` must equal the counts of `results[]`; `api` is the API
174
+ name verbatim; `reason` carries the class prefix of section 8. Keep headers, bodies and tokens out
175
+ of it.
176
+
177
+ ## 5. Identity bootstrap (`session.mjs`)
178
+
179
+ Three actors, all through the PUBLIC auth surface:
180
+
181
+ - **customer.** `POST /register` (the user shape from the auth spec: custom user properties are
182
+ required when the blueprint says so). On `emailVerificationNeeded`, run the two-call verify chain
183
+ (`POST /api/auth/verify/email-verification/start`, then `complete` with `secretCode` as a
184
+ STRING; `procedures/pitfalls.md`). On the dev slot the start call echoes `secretCode`; on stage
185
+ and prod it never does (a real mailbox, or a sink you read yourself). Then `POST /login`.
186
+ Register and login answer 201 with the flat session envelope: `accessToken` at the top level,
187
+ `userId` beside it. Read the token from the body first, `set-cookie` only as a fallback.
188
+ - **staff** (any custom role). A customer, then the super-admin grants the role with
189
+ `PATCH /userrole/:userId { roleId: "<role>" }`, then a FRESH login: a session minted before the
190
+ grant does not carry it. Under `usersHaveMultipleRoles` the stored `roleId` is an array: send and
191
+ assert the array form. A role-gated API answers 403 to the wrong actor; that 403 is a valid
192
+ assertion, not a reason to add the role to `absoluteRoles`.
193
+ - **super-admin.** The blueprint's bootstrap credential
194
+ (`authentication` -> `loginDefinition.userSettings.superAdminIdentifier` / `superAdminPassword`),
195
+ inserted once when absent. In a Genesis project that value should be an `$ENV:` reference
196
+ (`compiler-api.md`). The suite resolves it at runtime from environment variables you name (the
197
+ same variables the blueprint references) and nowhere else: not a literal in `config.mjs`, not
198
+ a line in a report, not a committed file. Never print it, `cat` it or paste it into a prompt.
199
+
200
+ Sessions are memoised per process, and a bootstrap failure degrades instead of aborting:
201
+ `superAdmin()` that cannot log in throws `SkipError('BLOCKED-IDENTITY: ...')`, `staff()` inherits
202
+ it, and every case that needs either actor records a reasoned skip. A blocked identity is a
203
+ FINDING to report with the exact 403 body, never a reason to reset a password in the database.
204
+
205
+ `registerUser(tag, extra)` is the one-call composite (register, verify if needed, login)
206
+ every fixture uses for a throwaway actor. Give it a `tag` so the data is traceable to the case.
207
+
208
+ ## 6. Fixtures across services
209
+
210
+ Fixtures are built THROUGH the APIs, once per process, as a chain that mirrors the product's own
211
+ flow, for example: staff creates a category and a published product with stock, then the customer
212
+ checks out and pays, then staff ships and delivers, then the customer writes the review.
213
+
214
+ - **`once()` memoises the rejection too, classified.** A memo helper that caches only success
215
+ re-runs a permanently blocked fixture for every case that asks for it (one pilot lane burned
216
+ 16 cases x 2 throwaway users x ~15 s on one blocked checkout, another left 17 stranded
217
+ rows). Cache the ERROR OBJECT of a permanent failure, so the second caller sees the same
218
+ evidence; never cache a transient one.
219
+
220
+ | Class | Tell | Memo |
221
+ |---|---|---|
222
+ | permanent | a `SkipError`; a 4xx about the request or the actor (401, 403, 404 on an advertised route, a validation 422); a precondition the suite cannot create | cache the rejection |
223
+ | transient | a 5xx, a network error, a 429, a not-ready state the fixture polls for | do not memoise; the next caller retries |
224
+
225
+ - **A fixture failure is a real failure.** It throws with `brief(res)` and the cases that need it
226
+ fail; only a classified skip (section 8) turns it into a skip.
227
+ - **Keep parents open until the children exist** (`procedures/pitfalls.md`). Destructive cases
228
+ (cancel, refund, reject) get a fresh fixture, never the shared one. A state-shaped 422 means
229
+ "pick another open parent", not "reopen".
230
+ - **Fixtures carry their evidence** (for a paid order, each payment call's status and body), so a
231
+ red downstream can be read without re-running.
232
+ - **Restate the precondition before reusing a fixture** in another service: the same-shape failure
233
+ in review (no delivered fact for THIS user and product) has a different root than in order.
234
+
235
+ ### Mock payment discipline
236
+
237
+ The suite never sends card data, not a real card, not a "test card number": the mock adapter needs
238
+ none, and a payment-method API that stores or echoes such fields is a security finding.
239
+ The payment ladders are in `genesis/procedures/howto-payment-flow.md` and
240
+ `genesis/procedures/howto-subscription.md`; the test
241
+ discipline on top of them:
242
+
243
+ 1. **Probe first, once per run:** `GET /<order-service>-api/v1/payments/mock-status` (or the same
244
+ path on your service address). For your provider's row, `mock: true` with `transport: "present"`
245
+ is the expected answer when the provider keys are blank. **`mock: false` means the service is
246
+ bound to live credentials: stop and do not run the paid chain.** `transport: "absent"` (the row
247
+ is also in the top-level `blocking[]`) is the one case where the paid chain is unrunnable.
248
+ 2. **Path A** (the customer's product path): `PATCH /v1/startorderpayment/:orderId`, then
249
+ `POST /v1/ordermockauthenticate/:orderId` (no body). Copy both routes from your own route list;
250
+ the names here belong to one pilot's blueprint.
251
+ 3. **Path B** (operator fallback): `POST /v1/payments/admin/simulate-capture/order/:orderId` as
252
+ super-admin. A staff session answers `403 OPERATOR_REQUIRED`; do not grant the capability to
253
+ the test role to make it pass. It needs the payment ticket Path A creates (else 404 "no payment
254
+ ticket").
255
+ 4. **Poll the order** (a handful of 1.5 s reads) until its status leaves the pending family, and
256
+ assert on the ORDER's state, not on the payment call's 200.
257
+ 5. **Read `transport`, not the brand.** `mock: true` says the credentials are blank; `transport`
258
+ says whether the mock path answers. Only a brand whose row says `absent` makes the fixture
259
+ throw a `SkipError` at the payment step, so every dependent case records the same reason.
260
+
261
+ ## 7. The test transport: driving what HTTP cannot reach
262
+
263
+ Four API kinds have no public caller: **kafka-only** (runs when a message lands on a topic),
264
+ **cron-only** (runs on a schedule; `0 4 1 * *` is a month), **internal-only**
265
+ (`controllers: ["internal"]`, reachable only from inside the service) and **machine-only**
266
+ (`requiredCallerKind: "machine"` answers `403 M2M_CALLER_REQUIRED` to every session, super-admin
267
+ included). Every generated nodejs service mounts a drive surface for them under its own prefix,
268
+ for example `/<service>-api/admin/test-transport`.
269
+
270
+ **When it is open.** Only on an explicit non-production slot: `CONFIG_ENV` and `SERVICE_CONFIG`
271
+ both set, both non-production and agreeing (every generated `Dockerfile.<slot>` sets both), and
272
+ `MB_TEST_TRANSPORT` unset. Anything else answers `TEST_TRANSPORT_DISABLED`: production, or a plain
273
+ `npm start` with no `SERVICE_CONFIG` (the loader then falls back to `.test.env`), so set it yourself
274
+ for a local run. A non-empty `MB_TEST_TRANSPORT` switches the surface off outright.
275
+ That means a stage deployment is open too: set `MB_TEST_TRANSPORT=off` there if the surface must
276
+ not be reachable. Every caller that is not super-admin gets `TEST_TRANSPORT_FORBIDDEN`, so the
277
+ suite calls it with `ctx.superAdmin()`.
278
+
279
+ | Route | What it does |
280
+ |---|---|
281
+ | `GET /admin/test-transport` | census: declared and subscribed kafka topics, every cron job (with `registrationFailed`), the internal-call API names. Read it instead of guessing a name |
282
+ | `POST /admin/test-transport/kafka/publish` `{ topic, data, type?, old?, tenantCodename?, tenantId? }` | publishes through the service's own publisher (topic scope, envelope, scrub), so the consumer sees what production hands it. Call it on the service that OWNS the consumer: a topic outside its census is `400 TEST_TRANSPORT_TOPIC_NOT_CONSUMED`; a dropped publish is `503 TEST_TRANSPORT_PUBLISH_DROPPED`. `subscribed: false` means delivered but nobody listens. On a multi-tenant project the event goes out in the root tenant unless you name the tenant; unknown codename is `404 TEST_TRANSPORT_TENANT_NOT_FOUND` |
283
+ | `POST /admin/test-transport/crons/:name/run-now` `{ inputData? }` | runs the job once and waits. `ran: true` is the transport's verdict; assert on `outcome.httpStatus` / `outcome.output`, because a sweep that refused (400) also "ran". A handler that threw reports `outcome.httpStatus: 500` |
284
+ | `POST /admin/test-transport/internal/:apiName` `{ params? }` | invokes the internal-call API through the same registry a designer script uses. `invoked: true` is the transport's verdict; the API's own is `outcome.failed` / `outcome.httpStatus` / `outcome.errorCode` |
285
+ | `POST /admin/test-transport/machine-token` `{ targetService?, ttlSeconds? }` | mints a service-signed token (default 120 s, ceiling 300 s; a longer one is refused with `TEST_TRANSPORT_TTL_TOO_LONG`, never clamped). Send it as `x-m2m-token` to the machine-only API. It verifies only on the slot that minted it, so mint and call on the same deployment. A target in a language whose verifier does not check that binding answers `422 TEST_TRANSPORT_TARGET_UNVERIFIED` |
286
+
287
+ Other refusals: `TEST_TRANSPORT_LOGIN_REQUIRED` (no session), `TEST_TRANSPORT_CRON_UNKNOWN`,
288
+ `TEST_TRANSPORT_NO_SCHEDULER`, `TEST_TRANSPORT_API_UNKNOWN`. The minting service is the issuer, so
289
+ minting on a service the callee's `allowedM2MClients` does not list is how `M2M_CLIENT_NOT_ALLOWED`
290
+ is produced.
291
+
292
+ A kafka- or cron-driven case asserts on the EFFECT (the row the consumer wrote, the state the sweep
293
+ moved), polled a handful of times like the order in section 6. The publish or run-now answer only
294
+ says the drive happened.
295
+
296
+ **Auth's `mintSessionForUser`.** It is a trusted proxy-login route: it answers `403
297
+ errMsg_UntrustedProxyLoginCaller` to every user session, super-admin included, because it wants a
298
+ verified `x-m2m-token`. That is not a reason to skip it: mint a token on auth itself
299
+ (`POST <auth base>/admin/test-transport/machine-token` as super-admin, body `{}`), then call
300
+ `POST <auth base>/mintsessionforuser` with `x-m2m-token` and `{ userId, reason }`. Cases to write:
301
+ the `200` whose `body.userId` is the requested user, plus a `/currentuser` read on the returned
302
+ `accessToken` naming the same user; a `404` for an unknown `userId`; the `403` for a plain user
303
+ session (that negative is the contract); a `401` with no credential. If the deployment sets
304
+ `PROXY_SESSION_TRUSTED_SERVICES` to a list without `auth`, or auth is built in a language the mint
305
+ refuses, throw `SkipError('ENVIRONMENT: ...')` from the live response.
306
+
307
+ **Steering the decision mock.** A decision provider with no key (or the `mock` brand) answers from
308
+ a hash; `GET /<service>-api/v1/decisions/mock-status` says which providers are on it. To drive every
309
+ branch of an `objectModeration` flow (one `choice` question named `verdict` over `accept`, `reject`,
310
+ `review`), put a directive in any text field of the content the decision reads:
311
+
312
+ [[mock-decision verdict=reject]]
313
+ [[mock-decision verdict=accept@0.6]] (low confidence, below the threshold: routes to review)
314
+ [[mock-decision *=no]] (every question; other decision kinds)
315
+
316
+ Question names and legal values come from the blueprint's decision questions. A value the question
317
+ cannot take is refused with `MOCK_DECISION_DIRECTIVE_INVALID`, never silently replaced. Directives
318
+ are inert on production and on the real providers.
319
+
320
+ **A failed fiscal document.** The fiscal `mock` provider takes `configuration.behavior` in the
321
+ blueprint (a suite cannot pick it per document): `accept` (default), `reject`, `pending`, `fail`
322
+ (the first submit the service process serves throws, every later one accepts; a restart re-arms
323
+ it) and `failAlways`. A `failed` document is reachable no other way.
324
+
325
+ ## 8. Reasoned skips: the taxonomy
326
+
327
+ A skip is a covered-with-reason verdict. The reason is a string with a class prefix, so the gate
328
+ counts it as covered and a human can audit every one. Two mechanics:
329
+
330
+ - **static** `suite.skip(api, reason)`: decided from the route row before any call;
331
+ - **thrown** `throw new SkipError(reason)` inside a case or fixture: decided from live evidence, so
332
+ when the cause disappears the case PASSES with no code change.
333
+
334
+ | Class (prefix) | When | Mechanic | Must carry |
335
+ |---|---|---|---|
336
+ | `STATIC-TRANSPORT` / `cron-only` / `internal-call-only` | no external transport AND the test transport cannot reach it either (a service built before the transport existed: the census route answers 404) | static | the transport the row declares, and why section 7 does not reach it |
337
+ | `KNOWN-DEFECT <ref>: <symptom>` | a filed product defect blocks the case | thrown | the issue ref AND `brief(res)` evidence (status, body head, requestId) |
338
+ | `ENVIRONMENT: <what>` | your stack cannot support it (a capacity cap, an unbound provider, no echoed approval code) | static or thrown | the evidence and what would unblock it |
339
+ | `BLOCKED-IDENTITY: <why>` | a section 5 actor could not be bootstrapped | thrown by `session.mjs`, cascades | the login failure `brief` |
340
+ | `BUILT-IN-UNUSED: <what the product does instead>` | a built-in row the plan looked at and the product does not use. Never on a designer API: an unused designer API is a design question, not a skip | static | the one-line product reason |
341
+ | precondition (`no rows yet`, `provider boundary: ...`) | the case's data precondition did not hold on this run | thrown | the response that showed it |
342
+
343
+ Watch the precondition class. It is legitimate when the product allows the empty state, and a
344
+ hidden fail when the precondition SHOULD have held (a fixture that silently did not create the
345
+ row). A precondition skip on a fixture-backed case is re-triaged as a fail.
346
+
347
+ **A gate carries its true root's class, never a bare `precondition`.** The fixture holds the
348
+ response that caused its failure; no downstream case does. So at the point a fixture gives up it
349
+ classifies its own first failure (section 9 ladder) and throws that class with evidence. Every
350
+ dependent case inherits the same string, so the skip ledger counts one classified family, not N
351
+ unexplained preconditions. A `precondition` skip on a case whose fixture ran and failed is a
352
+ mislabelled gate: fix the gate's class, do not re-triage the case.
353
+
354
+ **Never:** weaken an assertion, widen a status set to turn a red green, delete a case, mark a fail
355
+ as a skip without a class and evidence, or change the product to fit the test. 95 % passing with
356
+ real defects accurately flagged beats 100 % with failures skipped.
357
+
358
+ ## 9. Triage: the ladder and the fix policy
359
+
360
+ After every run, classify EVERY fail before touching anything. In the pilot, 27 of 32 first-round
361
+ reds were one root (a `whereClause` that named `userId` where the pattern's column is `ownerId`),
362
+ and the second round's 27 were one root again (an inter-service 404 on an unmounted route family).
363
+
364
+ 1. **Group by first-failing fixture.** A red fixture takes every dependent case with it. Read the
365
+ first failure's `brief`, fix that, re-run, then look at the survivors. Never fix a cascade case
366
+ by case.
367
+ 2. **Read the evidence you have:** `brief(res)` (status, body, `requestId`, `errMsg_*`), the
368
+ fixture's trace, then the service's own log around the request time. The log line with the same
369
+ `requestId` names the stage that threw. If the log does not show it, reproduce the request
370
+ against your local stack with the same case. Query the database read-only to compare a row with
371
+ the API's answer; never write to it.
372
+ 3. **Classify.** The class decides the remedy:
373
+
374
+ | Class | Tell | Remedy |
375
+ |---|---|---|
376
+ | Test fault | wrong parameter name, wrong envelope key, a precondition the case never set up, a validator message that contradicts the doc (the live error outranks the doc) | fix the test; the generated service's source or `_api.json` settles a shape question in one read |
377
+ | Design fault (blueprint) | an authored slot is wrong: a `whereClause` column, an MScript expression, a missing role, a validation the spec never asked for | fix the BLUEPRINT, validate, rebuild, redeploy, refresh the route list, re-run. Cap 3 attempts per failure class, each changing something; then surface |
378
+ | Codegen fault | generated framework code fails: a 500 with a stack in generated code, a route the list advertises but the app does not mount, a sibling call that 404s on a generated path | collect the evidence (route, `brief`, `requestId`, log excerpt, route row, blueprint path), report it to the Genesis team, and make the case a `KNOWN-DEFECT <ref>` skip thrown from the live response |
379
+ | Environment | your stack cannot do it (capacity, an unbound provider, a missing external surface) | `ENVIRONMENT` skip with evidence and the unblock condition |
380
+ | Partial write | a `200` / `201` carrying `actionFailures` | classify by the FIRST entry's `action` + `error` like a red of that action; never by the status, never by re-sending a create |
381
+
382
+ Restate a fix's precondition before applying it to a second case: same-shape reds have different
383
+ roots.
384
+
385
+ **Fix policy, non-negotiable.**
386
+
387
+ - A failing test is not license to change the product. A design fault is fixed in the blueprint
388
+ and rebuilt (`procedures/blueprint-authoring.md`); the suite re-runs against the new build.
389
+ Never hand-edit generated files to pass a test (they are overwritten on the next build; see
390
+ [custom-code.md](custom-code.md) for where your own code lives), and never write rows into the
391
+ database to set up state the API could not create.
392
+ - A workaround you ship (a fixture that routes around a broken generated path, a widened status
393
+ set "because the service answers 500 sometimes") is a defect report's trigger: report first,
394
+ then decide whether it stays as a tripwire skip.
395
+ - Cap and surface: three attempts per failure class, then the user decides, with the reds quoted,
396
+ the classification and the evidence trail, never summarised away.
397
+
398
+ ## 10. Sample data: the committed seed
399
+
400
+ After testing, a stack is full of `test1`, `aaa` and half-finished flows. The seed is a written,
401
+ committed script that produces the opposite, a small image-carrying dataset that makes the product
402
+ legible. When tests dirty the data again, the cycle is "reset, seed", not a person doing it by
403
+ hand. It doubles as the cheapest end-to-end smoke test.
404
+
405
+ The seed is NOT the blueprint's `SeedData` pattern. `SeedData` upserts declared rows at service
406
+ startup, at database level, per data object, for reference and config rows. The sample-data seed is
407
+ post-test, cosmetic, cross-service, and goes through the public API so that validation, ownership,
408
+ derived fields, events and the search index all really turn.
409
+
410
+ ### The five rules
411
+
412
+ 1. **Public API only.** Every record is created with `fetch` against the service's API, never
413
+ through the database. A database-level seed leaves search empty and events unfired.
414
+ 2. **Idempotent and deterministic.** Every record carries a stable key in a natural field (slug,
415
+ sku, email, name), and the script looks the record up before creating it (`ensure(kind, key,
416
+ { find, create })`). A second run reports `created: 0`. Note the contrast with the suite, whose
417
+ ids carry random suffixes so re-runs never collide: the seed uses stable keys so re-runs
418
+ converge. If an object has no natural field for the key, put a recognisably stable value in a
419
+ unique field and match on it; "create blindly" is how a seed stops being re-runnable.
420
+ 3. **Seeded stock images, never drawn or generated.**
421
+ - An image URL field gets `https://picsum.photos/seed/<stable-key>/<w>/<h>`: the seeded path
422
+ keeps the same picture per record on every run. Never the unseeded `picsum.photos/<w>/<h>`.
423
+ - An image stored as a bucket key (the create call takes a storage key, not a URL) gets the same
424
+ seeded photo downloaded once, cached under `seed/images/` (commit the cache: a re-run needs no
425
+ network), and uploaded through the product's own bucket upload route inside the record's
426
+ `create()`.
427
+ - Never draw images with code and never generate them with an image model: both cost more than
428
+ a download and look worse.
429
+ - For topical photos use a stock library whose licence allows commercial use without
430
+ attribution (Unsplash, Pexels, Pixabay). Their search APIs need a free key: keep it in the
431
+ environment only, never in the seed, and cache the downloads the same way. Avoid Wikimedia
432
+ Commons unless every file's attribution travels with the picture.
433
+ 4. **A real login.** The seed authenticates through the same public login the frontend uses,
434
+ registers its demo users through the real register, verify, login chain and creates the tenant(s)
435
+ it needs. On a multi-tenant project it sends the tenant codename header; copy the header name
436
+ from the route list (`_api.json` carries it per endpoint) and the auth spec, never guess.
437
+ 5. **Fail loud.** Any non-2xx stops the seed and names the API, the call and the payload. Nothing
438
+ is swallowed or blind-retried. A red seed is a FINDING: triage it with the ladder of section 9
439
+ before touching the seed.
440
+
441
+ ### Where it lives, and what it reads first
442
+
443
+ - It is ONE self-contained script, `seed/seed-sample-data.mjs` in the same tests repo (`npm run
444
+ seed`), with one README line on what it seeds, how to run it and the idempotence check.
445
+ - Self-contained: Node >= 18, built-ins only, no imports from `helpers/`, base URL from an
446
+ environment variable, because it runs alone, in its own phase, long after the suite.
447
+ - Before filling it, read: the route lists of every service (routes are copied, never composed),
448
+ the auth spec (roles, required register fields, tenant shape), the data model (which objects have
449
+ states, which fields are images) and `procedures/pitfalls.md` (the verify chain with `secretCode`
450
+ as a string, real-today dates, keep-parents-open).
451
+
452
+ ### Filling the story
453
+
454
+ The data is cosmetic: a small story the UI can walk through, not a fixture dump.
455
+
456
+ - **Per service, per state.** Every meaningful state of every object the UI shows gets at least one
457
+ record (a draft AND a published product; an open, a shipped AND a delivered order). Reach states
458
+ through the PUBLIC transitions in a second pass: create everything open first, seed children
459
+ while parents are open, then transition a few parents.
460
+ - **Plausible and coherent.** Real-looking names, sensible relations (the review is written by the
461
+ customer whose order was delivered), prices and dates that could be true, anchored to the real
462
+ today.
463
+ - **Enough volume:** roughly 5 to 15 records per primary object, at least one page where the
464
+ frontend paginates, at least two distinct values for every field the UI filters on.
465
+ - **Images everywhere the UI shows one**, sized for the slot (card, hero, avatar).
466
+ - **A small, reusable cast:** one owner or staff actor and one or two customers, with stable
467
+ seed-key emails and a deterministic demo password from the environment, so the second run logs in
468
+ instead of registering and a person can log in AS a demo user. The super-admin credential is
469
+ resolved at runtime only (section 5) and never appears in the committed file.
470
+ - Use reserved non-deliverable addresses for demo users (`.invalid`, `.test`, `.example`,
471
+ `example.com`) so no real provider ever receives a demo message.
472
+
473
+ ### Running and verifying
474
+
475
+ ```
476
+ cd tests
477
+ BASE=<your stack url> node seed/seed-sample-data.mjs # seed (idempotent)
478
+ BASE=<your stack url> node seed/seed-sample-data.mjs --expect-existing # the twice-run check
479
+ ```
480
+
481
+ (Name the base-URL variable whatever your `config.mjs` uses; keep it the same in both files.)
482
+
483
+ - The stack must be up and healthy first (`/health`; [run-locally.md](run-locally.md)).
484
+ - First run: expect `created: N, existing: 0` and a dataset a person can navigate.
485
+ - Second run straight after: `--expect-existing` exits non-zero if ANYTHING was created. That is
486
+ the acceptance check for idempotence; run it every time the seed changes.
487
+ - Seeding never deletes. To start clean, drop your local stack's data volumes and bring the stack
488
+ up again (the generated services re-seed the bootstrap super-admin and every blueprint `seedData`
489
+ row at boot), check you can log in as super-admin, then seed. A seed on top of test data without
490
+ a reset is allowed but said out loud in the run report: the dataset is additive.
491
+ - Never delete rows by hand to "help": the database and the search index must keep agreeing. To
492
+ remove one row, use the product's own delete or archive API; if the object has none, that is a
493
+ blueprint gap to fix there.
494
+ - After a payment-bearing seed step, provision plans and webhooks first as described in
495
+ `genesis/procedures/howto-subscription.md`.
496
+ - Verify by read-back, not by the exit code: count rows per service in the database and compare
497
+ with what the API lists, and check the search index moved with the database (database up and
498
+ index flat means the seed wrote past the search layer: a finding, never something to seed over).
499
+ Put the counts in the report as numbers.
500
+
501
+ ### Seeding a shared stage
502
+
503
+ Only when the owner asks, after a stage deploy, and never as an automatic step:
504
+
505
+ - Confirm the base URL with the owner before the first call. No reset, ever; the seed stays
506
+ additive and idempotent.
507
+ - Catalog only: definition data (categories, products, brands, settings). Anything bound to a user
508
+ (orders, carts, reviews, profiles) is skipped and named in the report; no demo end users are
509
+ registered, because stage runs live verification and codes never ride the answer.
510
+ - A multi-tenant catalog lives inside a tenant, so write it as an owner of ONE example tenant,
511
+ reusing it when it exists. Payment steps run only when the stage payment provider holds
512
+ test-mode keys; otherwise skip them with the reason.
513
+ - Read back every created record through the public API and fail loud on one that is missing.
514
+
515
+ ## 11. Checklist
516
+
517
+ - [ ] Test plan written (`procedures/design/writers/test-plan.md`) before the first script.
518
+ - [ ] `inventory/<service>.json` fresh for every service (re-copied after each rebuild).
519
+ - [ ] Helpers and `auth.flow.mjs` green; three actors bootstrap, or a `BLOCKED-IDENTITY` finding
520
+ is reported with the 403 body.
521
+ - [ ] One flow per service; every designer API has a case or a class-prefixed skip; built-ins only
522
+ where the plan lists them.
523
+ - [ ] Routes copied from `_api.json` / `/openapi.json`, a `User-Agent` on every call.
524
+ - [ ] Every write case asserts no `actionFailures`; every approval-gated API goes through
525
+ `approvedCall` and asserts both legs.
526
+ - [ ] `mock-status` probed (`mock: true`, `transport: "present"`); no card data anywhere.
527
+ - [ ] Kafka, cron, internal and machine-only rows driven through the test transport.
528
+ - [ ] `run-all` then `coverage-gate`: `GATE: PASS`, and fails are 0 or each classified and capped.
529
+ - [ ] Every skip carries its class and evidence; fixture gates throw their true root's class.
530
+ - [ ] Design faults fixed in the blueprint and rebuilt; codegen faults reported with evidence.
531
+ - [ ] Seed: public API only, stable keys, seeded stock images, real login, fail loud, run twice
532
+ with `created: 0` on the second run, counts verified by read-back.