@mindbricks/genesis-set 0.0.0-stage → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +44 -0
- package/LICENSE +202 -0
- package/README.md +148 -2
- package/bin/genesis-set.mjs +153 -0
- package/corpus/README.md +25 -0
- package/corpus/genesis/compile-results.md +246 -0
- package/corpus/genesis/compiler-api.md +265 -0
- package/corpus/genesis/custom-code.md +519 -0
- package/corpus/genesis/deploy.md +190 -0
- package/corpus/genesis/pitfalls.md +231 -0
- package/corpus/genesis/procedures/README.md +17 -0
- package/corpus/genesis/procedures/agent-hub-blueprint.md +13 -0
- package/corpus/genesis/procedures/auth-blueprint.md +14 -0
- package/corpus/genesis/procedures/bff-blueprint.md +16 -0
- package/corpus/genesis/procedures/blueprint-authoring.md +16 -0
- package/corpus/genesis/procedures/howto-mail-to-tasks.md +26 -0
- package/corpus/genesis/procedures/howto-payment-flow.md +112 -0
- package/corpus/genesis/procedures/howto-subscription.md +33 -0
- package/corpus/genesis/procedures/notifications-blueprint.md +13 -0
- package/corpus/genesis/procedures/patterns-catalog.md +12 -0
- package/corpus/genesis/procedures/project-lifecycle.md +27 -0
- package/corpus/genesis/run-locally.md +310 -0
- package/corpus/genesis/testing.md +532 -0
- package/lib/config-cli.mjs +179 -0
- package/lib/install.mjs +639 -0
- package/lib/mcp-config.mjs +222 -0
- package/lib/semver.mjs +94 -0
- package/mcp/blueprint.mjs +64 -0
- package/mcp/client.mjs +141 -0
- package/mcp/config.mjs +166 -0
- package/mcp/key-page-html.mjs +275 -0
- package/mcp/key-page.mjs +217 -0
- package/mcp/key-store.mjs +204 -0
- package/mcp/key-tools.mjs +225 -0
- package/mcp/server.mjs +164 -0
- package/mcp/tools.mjs +270 -0
- package/package.json +44 -4
- package/skills/genesis-compiler-api/SKILL.md +56 -0
- package/skills/genesis-deploy/SKILL.md +36 -0
- package/skills/genesis-generated-code/SKILL.md +48 -0
- package/skills/genesis-run-and-test/SKILL.md +43 -0
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
# Genesis additions: How-to: the payment flow
|
|
2
|
+
|
|
3
|
+
These notes add to the mind-set procedure `procedures/howto-payment-flow.md` (installed at
|
|
4
|
+
`~/.mind-set/current/procedures/howto-payment-flow.md`). Follow that procedure; at each place below,
|
|
5
|
+
do what this file says. They hold what is true of the Genesis compiler and the code it
|
|
6
|
+
generates, and the Genesis Compiler API steps.
|
|
7
|
+
|
|
8
|
+
## 1. In "9. Validate"
|
|
9
|
+
|
|
10
|
+
Add:
|
|
11
|
+
|
|
12
|
+
**Runtime prerequisites.** The service compiled and running; the AUTH
|
|
13
|
+
service compiled AFTER the payment provider was bound (its `sys_paymentCustomer` / `sys_paymentMethod` family
|
|
14
|
+
is generated only then: a `startorderpayment` call 500s through a missing
|
|
15
|
+
`/auth-api/paymentcustomers` until auth is compiled again); a published,
|
|
16
|
+
in-stock product; a verified BUYER session (`pitfalls.md` section 1); a
|
|
17
|
+
`superAdmin` session for the operator rung; a staff-role session for
|
|
18
|
+
ship/deliver. Base paths below are an example (`/order-api`, `<order>` =
|
|
19
|
+
`order`); read yours off the generated service's route list.
|
|
20
|
+
|
|
21
|
+
**Webhook secret step.** After the FIRST compile of a service with a
|
|
22
|
+
payment provider, and after every reset of its database, call once as
|
|
23
|
+
superAdmin:
|
|
24
|
+
`POST /<service>-api/v1/payments/admin/webhooks/provision`, body `{}`.
|
|
25
|
+
It registers the webhook endpoint and stores its signing secret encrypted
|
|
26
|
+
in the service's database (restarts keep it); without it every webhook is
|
|
27
|
+
refused 401 `SIGNATURE_UNVERIFIABLE` and plan invoices / credit notes never
|
|
28
|
+
issue. Done = each `providers[]` entry `provisioned` / `rotated` with
|
|
29
|
+
`persisted: true`, or `already-registered` with `liveInThisProcess: true`
|
|
30
|
+
(after a reset it rotates by itself: `autoRotated: true`). Skip while the
|
|
31
|
+
provider is mock (`mock-status` `mock: true`). Re-run it after every framework
|
|
32
|
+
upgrade too: on a matched endpoint it adds the events a newer framework maps
|
|
33
|
+
and the endpoint lacks (`eventsAdded`, fw #6369; id and secret unchanged,
|
|
34
|
+
nothing removed); `eventsNotAdded` names events the provider refused to add,
|
|
35
|
+
report them to the owner. A call that creates or rotates the endpoint
|
|
36
|
+
also deletes this service's own endpoints at an older URL (`staleRemoved`; a
|
|
37
|
+
plain `already-registered` match deletes nothing), except the endpoint
|
|
38
|
+
of the stored secret and anything in a dev slot (`CONFIG_ENV=dev`): those are
|
|
39
|
+
only listed in `staleNotRemoved` with the reason. An endpoint at the same URL
|
|
40
|
+
with another owner's `mb_*` tags is never touched and is listed in
|
|
41
|
+
`foreignEndpointIds`: report it to the owner; an entry `failed`
|
|
42
|
+
with `errorCode: "WEBHOOK_ENDPOINT_LIMIT"` means the provider account is full:
|
|
43
|
+
report its `endpointsByProject` list to the owner (endpoints are pruned at the
|
|
44
|
+
provider, never by you) and re-run once they are removed.
|
|
45
|
+
|
|
46
|
+
**Payment re-provision after a database reset.** A reset of the service's
|
|
47
|
+
database empties the subscription plan catalog
|
|
48
|
+
(`sys_<X>SubscriptionPlans`), and the boot seed deliberately never writes
|
|
49
|
+
`providerPlanId`, so after EVERY reset each subscription purchase answers
|
|
50
|
+
409 `SUBSCRIPTION_PLAN_NOT_PROVISIONED`. For each payment service, as
|
|
51
|
+
superAdmin and BEFORE any purchase: first
|
|
52
|
+
`POST /<service>-api/v1/payments/admin/plans/provision`, body `{}` (a
|
|
53
|
+
service with subscription plans), THEN the webhook step above
|
|
54
|
+
(`POST /<service>-api/v1/payments/admin/webhooks/provision`, body `{}`).
|
|
55
|
+
The plan sweep finds the provider prices the earlier provisioning created
|
|
56
|
+
and adopts them; it creates nothing new. Done = `totals.failed: 0` with
|
|
57
|
+
every plan counted as `adopted` (the usual answer after a reset) or
|
|
58
|
+
`provisioned`, then the webhook done-check above.
|
|
59
|
+
|
|
60
|
+
| Rung | Call | Expect |
|
|
61
|
+
|---|---|---|
|
|
62
|
+
| 0 | `GET /order-api/v1/payments/mock-status` | `{ providers: [{ providerName, brand, mock: true, reason, transport: "present", blocking: false, mode }], anyMock: true, blocking: [], health: "ok" }` on a deployment with blank credentials — the mock adapter is the payment path; never call the live provider from tests. `transport`, not `mock`, is the field that says the mock path ANSWERS (see the brand caveat below): `transport: "absent"` puts the provider in `blocking[]` and flips `health` to `"blocked"`. A row that carries `transportFamilies` (Adyen) is covered only for the families with `coverage: "present"`: a call of an `absent` family is refused `501 MOCK_UNHANDLED_PATH` although `transport` reads `"present"` |
|
|
63
|
+
| 1 | buyer: `POST /order-api/cart/items { productId, quantity }`, `GET /order-api/cart`, `POST /order-api/checkout` | 201 with the order (`status: placed`, `paymentStatus: unpaid`, `expiresAt`), items priced from the catalog; `GET /order-api/cart` empty afterwards; an unavailable line answers 422 and creates NO order |
|
|
64
|
+
| 2 | buyer: `PATCH /order-api/v1/startorderpayment/:orderId` body `{ paymentUserParams: { return_url: "<app url>" } }` (add `providerOptions: { mockScenario: "requires_action" }` to exercise the 3-D Secure branch) | 200 with the ticket (`paymentStatus`, `statusLiteral`); the buyer must OWN the order (`403 NOT_OWNER` otherwise). With the default mock scenario the mock settles synchronously and the capture hooks run before the answer (`lifecyclePending: true` when they exceed `PAYMENT_SYNC_SETTLE_TIMEOUT_MS`, default 10 s) |
|
|
65
|
+
| 3a | `POST /order-api/v1/ordermockauthenticate/:orderId` | Mock mode only (404 otherwise): stands in for the customer authenticating and confirms the intent — needed in the `requires_action` scenario |
|
|
66
|
+
| 3b | superAdmin: `POST /order-api/v1/payments/admin/simulate-capture/order/:orderId` | Operator-gated like every `/v1/payments/admin/*` route: `401 LOGIN_REQUIRED` without a session, `403 OPERATOR_REQUIRED` without the `paymentOperator` capability (a plain `staff` role is refused — pilot-verified; `superAdmin` passes). The route is platform-operator-only (`superAdmin` / `admin` / `saasAdmin`; `superAdmin` / `saasAdmin` in a multi-tenant project, which has no role named `admin`, #5701), and that second gate answers differently HERE: a tenant-level operator (`tenantOwner` / `tenantAdmin`), or a caller admitted only through the `paymentOperator` permission, PASSES the operator gate and is then refused `404` rather than the console's `403 PLATFORM_OPERATOR_REQUIRED`, so this sandbox route reads as unmounted - assert 404, not 403, for that actor. Also: 404 with no payment ticket yet ("start the payment first"), 404 on a production deployment or non-test credentials, `409 PAYMENT_NOT_CAPTURABLE` on a terminal ticket; repeat answers 200 `outcome: "already-recorded"` when its event-store lookup completes within the wait budget; if recording has not completed by that deadline, the 200 body instead carries `outcome: "record-pending"` and `lifecyclePending: true` (including on a repeat). Recording continues after the answer; the existing row is reused when found. `PAYMENT_SYNC_SETTLE_TIMEOUT_MS=0` skips waiting and answers `record-pending` immediately |
|
|
67
|
+
| 4 | `GET /order-api/v1/orderpaymentbyorderid/:orderId`; `GET /order-api/my-orders/:orderId` | ticket `statusLiteral` captured (or `paymentStatus: simulated_capture` after 3b); the ORDER shows `status: paid`, `paymentStatus: succeeded`, `paidAt`, `paymentReference` = the provider intent id — proof the `afterPaymentCaptured` hook ran your API. If the ticket is captured but the order did not move, read the service log for `PaymentLifecycleHookFailed` (hook name, trigger, error) — the hook ran and your API refused |
|
|
68
|
+
| 5 | a Kafka consumer on the wire topic (`<codename>-order-service-payment-captured`); the notification service's log or its notification rows in the database | the captured event with your payload keys; the `paymentReceived` mail resolved to `customerUserId` |
|
|
69
|
+
| 6 | staff: `POST /order-api/orders/:orderId/ship { trackingNumber }`, then `.../deliver` | `shipped` with `shippedAt` + tracking; an unpaid or claim-incomplete order answers 422 (static message); `paid` -> `shipped` -> `delivered` are one-way — do NOT retry on a state-shaped 422, pick another open order |
|
|
70
|
+
| 7 | buyer: `POST /order-api/my-orders/:orderId/cancel` on a PAID, unshipped order | 200 `status: cancelled`, `paymentStatus: refund_pending`, the `sys_orderRefund` row exists (`GET /order-api/v1/payments/refunds`), stock released once at the catalog; a shipped order answers 422 with no refund and no release. Then re-read the order: `paymentStatus: refunded` once `afterPaymentRefunded` converged it — if it stays `refund_pending`, the sweep retries every 5 min; record what you observed (the pilot could not climb this rung because the iyzico mock transport did not exist yet; it does now — climb it) |
|
|
71
|
+
| 8 | wait past `expiresAt` on an UNPAID order (or seed one with a past `expiresAt`) | the minute cron flips it to `expired`, publishes `order-expired`, releases nothing; a capture racing the sweep wins (`applyPaymentCapture` refunds a late capture in full and keeps `expired`) |
|
|
72
|
+
|
|
73
|
+
**Brand caveat — READ THE `transport` FIELD (fw #4473, SHIPPED).**
|
|
74
|
+
`mock: true` only says the credentials are blank; whether the mock path
|
|
75
|
+
WORKS is the per-provider `transport` field, and `stripe`, `iyzico` and
|
|
76
|
+
`adyen` all answer `transport: "present"` now. An iyzico-bound object climbs the whole
|
|
77
|
+
ladder on its mock — rungs 2-8 must NOT be logged as skips, and a lane that
|
|
78
|
+
records them as "blocked by #4473" is reporting a condition that no longer
|
|
79
|
+
exists. An Adyen-bound object's mock answers too. Adyen has no
|
|
80
|
+
`GET /payments/{pspReference}`, and since fw #5808 the adapter never calls it:
|
|
81
|
+
a capture, cancel or refund sends the amount and currency recorded on the
|
|
82
|
+
payment (or the refund) itself, and a refresh answers from the ticket and the
|
|
83
|
+
state the notifications already persisted, so the refund of rung 7 and a
|
|
84
|
+
refresh climb on the mock like every other brand - do NOT log them as a
|
|
85
|
+
reasoned skip. Note what stays true of Adyen: a webhook AUTHORISATION leaves
|
|
86
|
+
the payment `authorized` (only a CAPTURE pays), and a shopper-action challenge
|
|
87
|
+
you abandon is not auto-cancelled - a restart inside the action window
|
|
88
|
+
(`ADYEN_ACTION_TTL_MS`, 30 minutes) resumes the same attempt from the ticket,
|
|
89
|
+
and after it the restart answers 409 (there is no automatic unlock yet). An
|
|
90
|
+
iyzico restart of an open payment (fw #6343) answers the SAME step again
|
|
91
|
+
(`pendingIntentReused: true`, `next_action.paymentPageUrl` or
|
|
92
|
+
`next_action.threeDSHtmlContent`, with `next_action.actionExpiresAt`) while
|
|
93
|
+
it has not expired; a restart that names another card (or switches between
|
|
94
|
+
no card and a card) opens a new attempt while the open payment provably
|
|
95
|
+
cannot capture (an expired hosted page nobody started paying on, a 3-D
|
|
96
|
+
Secure payment at `INIT_THREEDS`) and answers `409 PAYMENT_START_INTENT_PENDING` otherwise - on
|
|
97
|
+
Stripe a changed `paymentMethodId` is always that 409. Assert these, do not
|
|
98
|
+
log them as defects. A brand with no mock of its own (paypal, mangopay, ...) still answers
|
|
99
|
+
`transport: "absent"`: the adapter constructs and then refuses every call by
|
|
100
|
+
name, `blocking` lists that provider and `health` is `"blocked"`. Only THAT
|
|
101
|
+
is a reasoned skip, and it is named by the payload rather than assumed.
|
|
102
|
+
|
|
103
|
+
## 2. In "9. Validate"
|
|
104
|
+
|
|
105
|
+
Add:
|
|
106
|
+
|
|
107
|
+
**Rung fails — where the fix goes.** A 500 on your own route is a design
|
|
108
|
+
fault: fix the blueprint, rebuild (`iteration.md`), never the generated
|
|
109
|
+
code. A pattern route misbehaving with your blueprint clean is a compiler
|
|
110
|
+
fault: capture `requestId` + the exact error from the service log
|
|
111
|
+
and report it with evidence. `column "x" does not exist` after a blueprint
|
|
112
|
+
change is schema drift — rebuild, never hand DDL (`pitfalls.md` section 3).
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Genesis additions: How-to: a recurring pass
|
|
2
|
+
|
|
3
|
+
These notes add to the mind-set procedure `procedures/howto-subscription.md` (installed at
|
|
4
|
+
`~/.mind-set/current/procedures/howto-subscription.md`). Follow that procedure; at each place below,
|
|
5
|
+
do what this file says. They hold what is true of the Genesis compiler and the code it
|
|
6
|
+
generates, and the Genesis Compiler API steps.
|
|
7
|
+
|
|
8
|
+
## 1. In "7. Renewal versus charging"
|
|
9
|
+
|
|
10
|
+
Instead of the mind-set step that starts "## 8. Validate Validate the blueprint after every module you touched: ...":
|
|
11
|
+
|
|
12
|
+
#### 8. Validate, then verify the running service - the ladder
|
|
13
|
+
|
|
14
|
+
Validate every module you touched (`mindset validate`, then the Genesis
|
|
15
|
+
Compiler API's validation), compile, and start the service. Then climb,
|
|
16
|
+
recording what you OBSERVED per rung; log a rung you could not climb as a
|
|
17
|
+
finding, not as success.
|
|
18
|
+
|
|
19
|
+
| Rung | Call | Expect |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| 0 | `GET /<svc>-api/v1/payments/mock-status` | the provider listed with `mock: true`, `transport: "present"`, `blocking: false`. `transport: "absent"` = the brand has no mock; stop and report |
|
|
22
|
+
| 1 | catalog present: the generated service's route list shows the subscribe route; the plan seed rows exist | the seed was applied (otherwise compile and start the service again) |
|
|
23
|
+
| 2 | `POST .../v1/payments/admin/plans/provision` as superAdmin | `totals.provisioned` >= number of seeded plans; a second call reports them `skipped` |
|
|
24
|
+
| 3 | member: subscribe with the plan code (injected `POST /v1/subscribe<x>/:orderId` or your authored API) | 200 with `statusLiteral` (`active` or `trialing`; `pending_provider` plus a checkout form on iyzico: complete it per `howto-payment-flow.md`) and a `sys_<X>Subscription` row |
|
|
25
|
+
| 4 | read the allowance (`getBalance` or your wrapper) | the first period's allowance is present after an eligible confirmation (section 5); a still-unpaid first payment grants nothing |
|
|
26
|
+
| 5 | member: spend through the wrapper API; spend past zero | decrements; the overspend is refused with the wrapper's own error, the balance unchanged |
|
|
27
|
+
| 6 | trigger the renewal path (call the cron's business API by its manual route, or replay a renewal event) | the allowance is refreshed once; a second trigger with the same period is a no-op (`alreadyProcessed`) |
|
|
28
|
+
| 7 | member: `DELETE /v1/<x>subscription/:subscriptionId` | `canceled` (or cancel-at-period-end); the cancel hook revoked the allowance with a `consume` |
|
|
29
|
+
|
|
30
|
+
Failure reading: `409 SUBSCRIPTION_PLAN_NOT_FOUND` = seed missing;
|
|
31
|
+
`409 SUBSCRIPTION_PLAN_NOT_PROVISIONED` = run rung 2; a hook that "did
|
|
32
|
+
nothing" = read the service log for `PaymentLifecycleHookFailed`; an
|
|
33
|
+
overspend that succeeded = the wrapper reads the wrong owner row.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Genesis additions: Notifications module authoring
|
|
2
|
+
|
|
3
|
+
These notes add to the mind-set procedure `procedures/notifications-blueprint.md` (installed at
|
|
4
|
+
`~/.mind-set/current/procedures/notifications-blueprint.md`). Follow that procedure; at each place below,
|
|
5
|
+
do what this file says. They hold what is true of the Genesis compiler and the code it
|
|
6
|
+
generates, and the Genesis Compiler API steps.
|
|
7
|
+
|
|
8
|
+
## 1. In "4. Validate, build, gate"
|
|
9
|
+
|
|
10
|
+
Instead of the mind-set step that starts "2. Building the system service **`notification`** and checking its ...":
|
|
11
|
+
|
|
12
|
+
2. Compile the system service **`notification`** with the Genesis Compiler
|
|
13
|
+
API (the genesis-compiler-api skill) and check its health where you run it.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# Genesis additions: Patterns catalog
|
|
2
|
+
|
|
3
|
+
These notes add to the mind-set procedure `procedures/patterns-catalog.md` (installed at
|
|
4
|
+
`~/.mind-set/current/procedures/patterns-catalog.md`). Follow that procedure; at each place below,
|
|
5
|
+
do what this file says. They hold what is true of the Genesis compiler and the code it
|
|
6
|
+
generates, and the Genesis Compiler API steps.
|
|
7
|
+
|
|
8
|
+
## 1. In "0. Need -> pattern selector"
|
|
9
|
+
|
|
10
|
+
Instead of the mind-set step that starts "| Each user or tenant brings their OWN AI key (BYOK) | ...":
|
|
11
|
+
|
|
12
|
+
| Each user or tenant brings their OWN AI key (BYOK) | `AiProvider.credentialScopes` | nodejs, python, rust (user, tenant, workspace levels); dotnet, java, golang (user and tenant levels); never a key column or custom variable; `pitfalls.md` section 7 |
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Genesis additions: Project lifecycle
|
|
2
|
+
|
|
3
|
+
These notes add to the mind-set procedure `procedures/project-lifecycle.md` (installed at
|
|
4
|
+
`~/.mind-set/current/procedures/project-lifecycle.md`). Follow that procedure; at each place below,
|
|
5
|
+
do what this file says. They hold what is true of the Genesis compiler and the code it
|
|
6
|
+
generates, and the Genesis Compiler API steps.
|
|
7
|
+
|
|
8
|
+
## 1. In "1. Dependency graph — always respect, even on partial requests"
|
|
9
|
+
|
|
10
|
+
Add:
|
|
11
|
+
|
|
12
|
+
**Compile once, after authoring.** With the single exception of auth
|
|
13
|
+
(compiled first because domain authoring reads its live shape), do not
|
|
14
|
+
compile a service just because the Compiler API is at hand right after you
|
|
15
|
+
authored it. Every build is charged by the lines it adds, and compiling a
|
|
16
|
+
domain service before the bff wires `dataViewName` into its APIs costs a
|
|
17
|
+
second, avoidable round: author and validate everything first, THEN compile
|
|
18
|
+
it once.
|
|
19
|
+
|
|
20
|
+
## 2. In "2. The phases > Phase D — Design (the interview, the requirement docs, the specs)"
|
|
21
|
+
|
|
22
|
+
Add:
|
|
23
|
+
|
|
24
|
+
With genesis-set, that compiler is the Genesis compiler, reached through the
|
|
25
|
+
Genesis Compiler API (the genesis-compiler-api skill): compile `auth` first,
|
|
26
|
+
then every other service once, after the whole blueprint validates, and run
|
|
27
|
+
the result locally or deploy it yourself; the Compiler API only writes code.
|
|
@@ -0,0 +1,310 @@
|
|
|
1
|
+
# Running and debugging the generated services locally
|
|
2
|
+
|
|
3
|
+
This page is for the moment after a build: the service repos are checked out and you want to start
|
|
4
|
+
them, reach one, read its health, reproduce a failure and decide who owns the fault. Everything runs
|
|
5
|
+
on your machine with your own tools. Genesis never runs or deploys anything for you. For shipping
|
|
6
|
+
to a real slot see [deploy.md](deploy.md); for the build answers see [compile-results.md](compile-results.md).
|
|
7
|
+
|
|
8
|
+
## 1. What you have
|
|
9
|
+
|
|
10
|
+
- One git repo per service, `<codename>-<service>-service`, in your own account. Check them out
|
|
11
|
+
side by side in one parent folder; the compose stack of section 4 expects that layout.
|
|
12
|
+
- The repo is the compiler's output plus your own files. You never edit generated files: the next
|
|
13
|
+
build overwrites them. Extension goes through custom code, see [custom-code.md](custom-code.md).
|
|
14
|
+
- Env files (nodejs): `.dev.env`, `.test.env`, `.stage.env`, `.prod.env` are committed with secret
|
|
15
|
+
keys left blank; `.<slot>.secrets.env` is your gitignored keystore. There is no `.exp.*` file.
|
|
16
|
+
- `Dockerfile.dev` builds the local image; `Dockerfile.prod` is the production image.
|
|
17
|
+
- Treat the checkout as a build artifact: after any new build of that service, pull it before you
|
|
18
|
+
debug. A checkout at an older commit cannot reproduce a failure of a newer one.
|
|
19
|
+
|
|
20
|
+
## 2. Env resolution (why a value is not what the file says)
|
|
21
|
+
|
|
22
|
+
Order, strongest first:
|
|
23
|
+
|
|
24
|
+
```
|
|
25
|
+
non-empty variable in the process environment (shell, compose, orchestrator)
|
|
26
|
+
> .<slot>.secrets.env (your local keystore)
|
|
27
|
+
> .<slot>.env (committed slot file)
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
- `SERVICE_CONFIG` selects the slot file at boot; the default is `test`. The nodejs scripts set it:
|
|
31
|
+
`npm run dev` (ts-node) and `npm run start-dev` (built `dist/`) run the `dev` slot,
|
|
32
|
+
`start-stage` / `start-prod` the others; plain `npm start` runs `test`.
|
|
33
|
+
- The env files load through dotenv, which does not override a variable that is already set. That
|
|
34
|
+
is why one tree runs natively (file values) and in compose (container values) unchanged.
|
|
35
|
+
- A variable the process holds EMPTY is not filled from the slot file, but the keystore still
|
|
36
|
+
fills it. A secret you put in the keystore therefore does not win over a non-empty injected one.
|
|
37
|
+
- If a key looks ignored, print what the process really holds before editing files: the shell
|
|
38
|
+
value or the compose `environment:` entry is usually the one that won.
|
|
39
|
+
- Slots `test`, `stage` and `prod` carry EMPTY peer addresses (`SERVICE_URL`, `AUTH_SERVICE_URL`,
|
|
40
|
+
`SERVICE_<NAME>_URL`, `SERVICE_MCPBFF_URL`); the deployer fills them. `dev` keeps local ones.
|
|
41
|
+
The JWT public key URL is `AUTH_SERVICE_URL` + `/publickey`.
|
|
42
|
+
- Never copy a secrets file into a commit, an issue, a log or a bug report.
|
|
43
|
+
|
|
44
|
+
## 3. Ports and signing keys
|
|
45
|
+
|
|
46
|
+
Dev ports (every `.dev.env` sibling URL is built from them):
|
|
47
|
+
|
|
48
|
+
| Service | Port |
|
|
49
|
+
|---|---|
|
|
50
|
+
| `auth` | 3001 |
|
|
51
|
+
| `bff` | 3002 |
|
|
52
|
+
| `notification` | 3003 |
|
|
53
|
+
| `document` | 3004 |
|
|
54
|
+
| `agenthub` | 3006 |
|
|
55
|
+
| `mcpbff` | 3600 |
|
|
56
|
+
| business services | 3050, 3051, ... in blueprint order |
|
|
57
|
+
|
|
58
|
+
Check the real value in the service's `.dev.env` (`HTTP_PORT`) before you rely on the table.
|
|
59
|
+
|
|
60
|
+
Signing keys: each service signs its tokens with its own Ed25519 key, in `SERVICE_PRIVATE_KEY_PEM`
|
|
61
|
+
and `SERVICE_KID` (in compose: `<SVC>_SERVICE_PRIVATE_KEY_PEM` and `<SVC>_SERVICE_KID`).
|
|
62
|
+
Without one the service starts degraded and the login service cannot issue sessions.
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
openssl genpkey -algorithm ed25519 -out key.pem # PKCS#8 PEM
|
|
66
|
+
uuidgen # any UUID works as the key id
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Put the PEM in double quotes when it goes into a `.env`. Keys are yours; the compiler never
|
|
70
|
+
holds them.
|
|
71
|
+
|
|
72
|
+
## 4. Start the stack
|
|
73
|
+
|
|
74
|
+
Two ways. Pick by what you need.
|
|
75
|
+
|
|
76
|
+
### 4a. The `docker-compose` deploy target (all services, one command)
|
|
77
|
+
|
|
78
|
+
Ask for it in the build request (`deployTargets`, see [compiler-api.md](compiler-api.md)). It lands
|
|
79
|
+
in every service repo under `deploy/docker-compose/`:
|
|
80
|
+
|
|
81
|
+
- `docker-compose.yml`, `.env.example`, `.providers.secrets.env.example`, `README.md`.
|
|
82
|
+
- Every deployable service of the build, each built from its own repo at `../../../<repo>`
|
|
83
|
+
(the repo you stand in is `../..`), running its DEV slot image.
|
|
84
|
+
- Infrastructure by need: PostgreSQL (PostGIS 16) and/or MongoDB 7 by the service's `dbType`;
|
|
85
|
+
Redis 7 and Elasticsearch for every backend service; Kafka 3.7 (KRaft) for all.
|
|
86
|
+
They are reachable inside the compose network only; nothing but the services publishes a port.
|
|
87
|
+
- Every password is a `${VAR}` reference; no secret value is written into the files.
|
|
88
|
+
|
|
89
|
+
```sh
|
|
90
|
+
cd deploy/docker-compose
|
|
91
|
+
cp .env.example .env
|
|
92
|
+
cp .providers.secrets.env.example .providers.secrets.env
|
|
93
|
+
# fill .env: POSTGRES_PASSWORD (and MONGO_PASSWORD), the signing keys per service
|
|
94
|
+
docker compose up --build
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
- Needs Docker Compose 2.24 or later (optional `env_file`). `.env` and `.providers.secrets.env`
|
|
98
|
+
are gitignored.
|
|
99
|
+
- The MongoDB password goes into a connection URL: use a URL-safe one.
|
|
100
|
+
- Container env wins over the baked slot file. The stack sets, per service: `DB_HOST`, `DB_PORT`,
|
|
101
|
+
`DB_USER`, `DB_PASSWORD` (or `MONGO_URL`), `REDIS_URL`, `ELASTIC_HOST`, `KAFKA_BROKERS`,
|
|
102
|
+
`KAFKA_ENABLED_IN_DEV=true` (the dev slot keeps Kafka off otherwise), `AUTH_SERVICE_URL` and
|
|
103
|
+
`SERVICE_<NAME>_URL` as `http://<service>:<port>` for peers inside the stack.
|
|
104
|
+
- A peer the build does not include is `${KEY:-}`, an empty line you fill in `.env` if you run
|
|
105
|
+
that peer somewhere else. A missing address shows up later as an inter-service error.
|
|
106
|
+
- Provider credentials (payment, mail, storage) are passed to every service from one file. Add the
|
|
107
|
+
keys the other repos list in their own example file. A provider without a credential runs
|
|
108
|
+
degraded; that is expected until you add one.
|
|
109
|
+
- Never ask the user to paste a credential into the chat. Ask for it on the local key page instead:
|
|
110
|
+
1. `genesis_requestKey` with a name, a label and where the user gets the key.
|
|
111
|
+
2. Give the user the link it answers.
|
|
112
|
+
3. Check `genesis_listKeys` until the key is set.
|
|
113
|
+
4. `genesis_writeKeyToEnvFile` puts it into the gitignored env file (`.env`,
|
|
114
|
+
`.providers.secrets.env` or another `*.secrets.env`) without the value passing through you.
|
|
115
|
+
- Compose project name is the codename. Use `docker compose ps` and `docker compose logs -f <service>`
|
|
116
|
+
with the compose service names from `docker-compose.yml`.
|
|
117
|
+
- Stop with `docker compose down` (volumes kept); `down -v` wipes the data, which is your decision.
|
|
118
|
+
|
|
119
|
+
### 4b. One service as a native process (fastest loop)
|
|
120
|
+
|
|
121
|
+
Use it when you reproduce one service and its siblings run elsewhere.
|
|
122
|
+
|
|
123
|
+
1. Start the infrastructure the service's `.dev.env` names (`DB_HOST`, `REDIS_URL`, `ELASTIC_HOST`,
|
|
124
|
+
`KAFKA_BROKERS`, or `MONGO_URL`). The simplest source is the same compose file: bring up only
|
|
125
|
+
the infra services (`docker compose up -d postgres redis elasticsearch kafka`), then publish
|
|
126
|
+
their ports yourself if the compose file does not (it does not by default).
|
|
127
|
+
2. `npm install` in the repo once, then `npm run dev` (or `npm run build` and `npm run start-dev`).
|
|
128
|
+
3. The service binds its own port. Every sibling the request path calls must also be running, on
|
|
129
|
+
its own dev port or in compose, and its address must be in this service's env.
|
|
130
|
+
4. Kafka advertises the in-network name `kafka:9092`. A native process on the host cannot resolve
|
|
131
|
+
it unless `kafka` maps to `127.0.0.1` in `/etc/hosts`. Say so when a Kafka-consuming service runs
|
|
132
|
+
natively and does not get its events.
|
|
133
|
+
|
|
134
|
+
### 4c. No container runtime on the machine
|
|
135
|
+
|
|
136
|
+
Both ways above need a container runtime for the infrastructure. Check it first:
|
|
137
|
+
`docker info` (or `docker compose version`). When it is missing or the daemon is not running,
|
|
138
|
+
do not guess. Tell the user and offer the two ways below.
|
|
139
|
+
|
|
140
|
+
**1. Install a runtime (recommended).**
|
|
141
|
+
|
|
142
|
+
The user installs it; the installer asks for an administrator password and a licence acceptance,
|
|
143
|
+
which are the user's to give. You may name the download page or prepare the package-manager
|
|
144
|
+
command; run it only when the user says so, and never type their password.
|
|
145
|
+
|
|
146
|
+
| OS | Runtime | Notes |
|
|
147
|
+
|---|---|---|
|
|
148
|
+
| macOS | OrbStack (`brew install --cask orbstack`) or Docker Desktop | OrbStack is lighter; it is paid for commercial use. |
|
|
149
|
+
| Linux | Docker Engine from the distribution's or Docker's packages | The user then joins the `docker` group or uses `sudo`. |
|
|
150
|
+
| Windows | Docker Desktop with the WSL 2 backend | Run the stack inside WSL 2. |
|
|
151
|
+
|
|
152
|
+
- Docker Desktop is paid for larger companies (its subscription terms decide); say so, and
|
|
153
|
+
let the user choose.
|
|
154
|
+
- After the install: start the runtime and run `docker info` again. Then continue with 4a or 4b.
|
|
155
|
+
|
|
156
|
+
**2. One service natively, with the database only.**
|
|
157
|
+
|
|
158
|
+
Only the database is critical at boot (section 6). Redis, Elasticsearch and Kafka are optional,
|
|
159
|
+
so one service can run with just a PostgreSQL (or MongoDB) and no container at all.
|
|
160
|
+
|
|
161
|
+
1. **A database.** Either one installed on the machine (for example `brew install postgresql@16`
|
|
162
|
+
and `brew services start postgresql@16`) or a hosted one the user creates in their own
|
|
163
|
+
account.
|
|
164
|
+
2. **Its address.** Put `DB_HOST`, `DB_PORT`, `DB_USER` and `DB_PASSWORD` (for MongoDB,
|
|
165
|
+
`MONGO_URL`) in the gitignored `.dev.secrets.env` or in the shell environment. Both win over
|
|
166
|
+
the committed `.dev.env` (section 2), which you never edit. The service creates its own
|
|
167
|
+
database on the server.
|
|
168
|
+
3. **Start it.** `npm install`, then `npm run dev` (4b).
|
|
169
|
+
|
|
170
|
+
Say what is missing before you report anything:
|
|
171
|
+
|
|
172
|
+
- Without Redis there is no cache.
|
|
173
|
+
- Without Elasticsearch the reads that use it answer `503 ELASTIC_UNAVAILABLE`.
|
|
174
|
+
- Without Kafka no event is published or consumed, so event-driven flows and their tests cannot
|
|
175
|
+
pass.
|
|
176
|
+
|
|
177
|
+
A test that needs a missing dependency is reported as skipped with the reason, never as passed.
|
|
178
|
+
|
|
179
|
+
## 5. Resource checklist (before `up`)
|
|
180
|
+
|
|
181
|
+
- RAM: the infrastructure alone takes about 3 GB (Elasticsearch heap is set to 512 MB); each node
|
|
182
|
+
service needs 200-400 MB more. Give Docker 6 GB or more and run only the services the
|
|
183
|
+
reproduction needs.
|
|
184
|
+
- Ports: each service publishes its own dev port; free them first (`lsof -i :<port>`). Two stacks
|
|
185
|
+
of different projects on one machine collide on the same dev ports: run one at a time or change
|
|
186
|
+
the published port in the compose file you own.
|
|
187
|
+
- One heavy job at a time on one machine: not a full stack, a test run and a build together.
|
|
188
|
+
- Report what is left running when you stop.
|
|
189
|
+
|
|
190
|
+
## 6. Health: alive is not ready
|
|
191
|
+
|
|
192
|
+
`GET http://localhost:<port>/health` answers 200 `{"status":"ok"}` as soon as the port is bound, and
|
|
193
|
+
in every boot state. That is liveness only. Read the whole body:
|
|
194
|
+
|
|
195
|
+
| Field | Meaning |
|
|
196
|
+
|---|---|
|
|
197
|
+
| `readiness.ready` | `false` until the boot chain finished |
|
|
198
|
+
| `readiness.schema` | the schema step outcome; `ready:false` with a failed schema means every request that touches a missing table fails |
|
|
199
|
+
| `boot.state` | `starting`, `ready` or `degraded` |
|
|
200
|
+
| `boot.bootErrors[]` | `step`, `code`, `critical`, `retryable`, `attempts`, `lastAttemptAt`, `nextRetryAt` |
|
|
201
|
+
|
|
202
|
+
- Nothing is "running" until `boot.state` is `ready` and `readiness.ready` is `true`.
|
|
203
|
+
- While starting, every other route answers `503 SERVICE_STARTING`. When a critical step fails,
|
|
204
|
+
every other route answers `503 SERVICE_DEGRADED` with a `requestId` and `Retry-After`, and the
|
|
205
|
+
body does not name the cause. The cause is in `/health` (`code`) and in the service log.
|
|
206
|
+
- The service never exits on a boot error. It logs and keeps answering. Do not wait for a crash.
|
|
207
|
+
- Public codes: `DB_UNREACHABLE`, `DB_AUTH_FAILED`, `SCHEMA_FAILED`, `CONFIG_ENV_INVALID`,
|
|
208
|
+
`CONFIG_INVALID`, `IDENTITY_KEY_MISSING`, `PORT_IN_USE`, `BUILD_INCOMPLETE`,
|
|
209
|
+
`DEPENDENCY_UNSUPPORTED`, `STEP_FAILED`.
|
|
210
|
+
- `SchemaBootstrapFailed` and `ModelSyncFailed` in the log mean the schema did not materialise
|
|
211
|
+
(for example a permission or missing-table fault). Waiting or restarting does not fix it: fix the
|
|
212
|
+
database or the design, then rebuild.
|
|
213
|
+
- A pending migration, an absent optional Elasticsearch or Kafka, never make `ready` false.
|
|
214
|
+
- Do not probe business endpoints to "verify" the boot. They need auth, bodies and fixtures, and
|
|
215
|
+
answer 401/403/400/404 for reasons that teach nothing. Use the health body, then the test suite.
|
|
216
|
+
|
|
217
|
+
Boot verdicts by dependency (nodejs):
|
|
218
|
+
|
|
219
|
+
| Step | Failure | Verdict | Retried |
|
|
220
|
+
|---|---|---|---|
|
|
221
|
+
| `postgres`, `mongodb` | refused or timed out | critical `DB_UNREACHABLE` | yes, background |
|
|
222
|
+
| `postgres`, `mongodb` | credential refused | critical `DB_AUTH_FAILED` | yes |
|
|
223
|
+
| `mongodb` | malformed `MONGO_URL` | critical `CONFIG_INVALID` | no |
|
|
224
|
+
| `migrations` | lock or schema step failed | critical `SCHEMA_FAILED` | yes |
|
|
225
|
+
| `service-identity` | no `SERVICE_KID` / `SERVICE_PRIVATE_KEY_PEM` on a login-minting service | critical `IDENTITY_KEY_MISSING` | no |
|
|
226
|
+
| `models` | a model definition throws | critical `STEP_FAILED` | no (a generated-code defect: report it) |
|
|
227
|
+
| `redis` | unreachable after its attempts | optional; runs without cache | no |
|
|
228
|
+
| `elastic`, `kafka` | unreachable | optional; heals on its own | yes |
|
|
229
|
+
| `elastic` | endpoint is not Elasticsearch | optional `DEPENDENCY_UNSUPPORTED`; reads answer `503 ELASTIC_UNAVAILABLE` | no |
|
|
230
|
+
|
|
231
|
+
`/admin/health` carries the full, redacted detail but is gated; use the log if you cannot reach it.
|
|
232
|
+
|
|
233
|
+
## 7. Dependencies that start late
|
|
234
|
+
|
|
235
|
+
A cold start where the database is a second behind is normal; the service heals without a restart.
|
|
236
|
+
|
|
237
|
+
- Kafka, Elasticsearch: unbounded background retry, boot never blocked. Env: `KAFKA_RECONNECT_INTERVAL_MS`
|
|
238
|
+
(5000) and `KAFKA_RECONNECT_MAX_INTERVAL_MS` (60000); `ELASTIC_RECONNECT_INTERVAL_MS` (5000) and
|
|
239
|
+
`ELASTIC_RECONNECT_MAX_INTERVAL_MS` (60000). A literal `0` on a `*_MAX_INTERVAL_MS` opts out.
|
|
240
|
+
`/health` echoes `{connected, reason?}` for each, so "still converging" differs from "misconfigured".
|
|
241
|
+
- Redis, Postgres: a short bounded retry inside boot (`REDIS_RECONNECT_*`, `DB_RECONNECT_*`; defaults
|
|
242
|
+
3 attempts, 1s initial, 8s cap; `*_MAX_ATTEMPTS=1` tries once; the budget is capped at 20). When it is
|
|
243
|
+
spent, Postgres runs degraded and the boot supervisor retries in the background
|
|
244
|
+
(`BOOTSTRAP_RETRY_INTERVAL_MS` 5000 rising to `BOOTSTRAP_RETRY_MAX_INTERVAL_MS` 60000).
|
|
245
|
+
- Blank, negative or non-decimal values count as unset; intervals have a 1000 ms floor.
|
|
246
|
+
- Close budgets (`DB_CLOSE_TIMEOUT_MS`, `ELASTIC_CLOSE_TIMEOUT_MS`, `KAFKA_CLOSE_TIMEOUT_MS`,
|
|
247
|
+
`REDIS_CLOSE_TIMEOUT_MS`, `MONGO_CLOSE_TIMEOUT_MS`, default 5000) only matter for slow shutdowns.
|
|
248
|
+
- Retry lines are `warn`, not `error`; the verdict lines to alert on are `redisStartFailed`,
|
|
249
|
+
`postgresStartFailed` and `BootstrapDegraded`. A permanent Kafka/Elasticsearch outage logs one loud
|
|
250
|
+
line about every tenth attempt once the backoff has settled.
|
|
251
|
+
- Shutdown on `SIGTERM` is graceful and exits 0; that is the only exit.
|
|
252
|
+
|
|
253
|
+
## 8. Reproduce a failure
|
|
254
|
+
|
|
255
|
+
1. Capture the failing call exactly: method, path, body, which actor, the response (`status`, the
|
|
256
|
+
error code and message) and its `requestId`. Find the log line with the same `requestId`: it
|
|
257
|
+
names the stage and the generated file that threw. Keep the excerpt as the comparison target.
|
|
258
|
+
2. Confirm the commit: the checkout you run is the commit of the build you are debugging.
|
|
259
|
+
3. Decide the running set: every sibling the request path calls must run too. An inter-service error
|
|
260
|
+
that your original failure did not show is your setup, not the defect.
|
|
261
|
+
4. Bootstrap identity locally: the service seeds its bootstrap super admin from the env it booted
|
|
262
|
+
with; the password is the one in your blueprint or `$ENV:` reference. Customer and staff actors
|
|
263
|
+
come from your own register, verify, login and role-grant calls.
|
|
264
|
+
5. Replay the one failing case, ideally from your test suite ([testing.md](testing.md)), not by hand.
|
|
265
|
+
Same error and same stack in the local log means reproduced. A different symptom means your local
|
|
266
|
+
setup differs (env, data, a missing sibling); do not go on until they match.
|
|
267
|
+
6. Instrument on a throwaway copy: log the assembled `dataClause` / `whereClause` in the api manager
|
|
268
|
+
the stack names, run `node --inspect`, rerun the one case. Narrow it to one of three owners
|
|
269
|
+
(section 9).
|
|
270
|
+
7. Read logs the same way each time: native, the process stdout; compose, `docker compose logs -f`.
|
|
271
|
+
Quote the original line and the local line together.
|
|
272
|
+
8. Inspect state directly when the log is not enough: query the service's database, its Redis keys,
|
|
273
|
+
its Kafka topic or Elasticsearch index with your own client against the local infrastructure.
|
|
274
|
+
Compose publishes no infra ports by default, so run the client inside the network
|
|
275
|
+
(`docker compose exec <infra> ...`).
|
|
276
|
+
|
|
277
|
+
Payment and subscription runtime checks (webhook, mock provider, state ladder) have their own
|
|
278
|
+
ladders: follow the Genesis additions in the `genesis/procedures/` folder of this corpus (the
|
|
279
|
+
payment and subscription files, and its index) instead of repeating them here.
|
|
280
|
+
|
|
281
|
+
## 9. Whose fault is it, and where the fix goes
|
|
282
|
+
|
|
283
|
+
| You found | Owner | Action |
|
|
284
|
+
|---|---|---|
|
|
285
|
+
| An authored slot is wrong: an MScript expression, a where column, an action's parameters, a role or route | Blueprint | Fix the blueprint, validate, rebuild, pull, rerun. Discard the local experiment. |
|
|
286
|
+
| Your custom code (a library or edge function body in the blueprint) | Custom code | Fix it in the blueprint ([custom-code.md](custom-code.md)), validate, rebuild. |
|
|
287
|
+
| A file you added to the repo, or under `service-files/` | You | Fix it in place; it survives rebuilds. |
|
|
288
|
+
| A generated framework line mishandles a legal input, or a model/route fails to compile or load | Compiler | Report it with the evidence below. Do not patch the generated file. |
|
|
289
|
+
| Missing env, key, provider credential or peer address | Your deployment | Fix env or keys; nothing to rebuild. |
|
|
290
|
+
|
|
291
|
+
Report a compiler fault with: the request id or build job id of the build, the repo and commit, the
|
|
292
|
+
generated file and line, the input that reaches it, the local log line and the original `requestId`.
|
|
293
|
+
A local reproduction is the strongest report; file it before you ship any workaround.
|
|
294
|
+
|
|
295
|
+
Never:
|
|
296
|
+
|
|
297
|
+
- hand-edit a generated file as a durable fix (it is overwritten and counted at the next build);
|
|
298
|
+
- weaken a blueprint or test just to get green; classify the red first;
|
|
299
|
+
- copy an env or secrets file into a commit, a report or a chat;
|
|
300
|
+
- treat a local edit that confirmed the diagnosis as the fix. The fix lands in the blueprint or in
|
|
301
|
+
your custom code, then a rebuild ships it.
|
|
302
|
+
|
|
303
|
+
## 10. Checklist
|
|
304
|
+
|
|
305
|
+
- [ ] Checkouts at the commit of the build being debugged; container runtime (`docker info`), RAM and ports checked, or the no-runtime way of 4c chosen with the user.
|
|
306
|
+
- [ ] `.env` filled: passwords and one Ed25519 key and key id per service.
|
|
307
|
+
- [ ] Every service `boot.state: ready` and `readiness.ready: true`, not just 200 OK.
|
|
308
|
+
- [ ] Failure reproduced with the same symptom, or reported as not reproduced.
|
|
309
|
+
- [ ] Owner decided: blueprint, custom code, compiler, deployment; fix routed there.
|
|
310
|
+
- [ ] Stack stopped, or the running state reported.
|