@mindbricks/genesis-set 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -0
- package/LICENSE +202 -0
- package/README.md +147 -2
- package/bin/genesis-set.mjs +153 -0
- package/corpus/README.md +25 -0
- package/corpus/genesis/compile-results.md +246 -0
- package/corpus/genesis/compiler-api.md +265 -0
- package/corpus/genesis/custom-code.md +519 -0
- package/corpus/genesis/deploy.md +190 -0
- package/corpus/genesis/pitfalls.md +231 -0
- package/corpus/genesis/procedures/README.md +17 -0
- package/corpus/genesis/procedures/agent-hub-blueprint.md +13 -0
- package/corpus/genesis/procedures/auth-blueprint.md +14 -0
- package/corpus/genesis/procedures/bff-blueprint.md +16 -0
- package/corpus/genesis/procedures/blueprint-authoring.md +16 -0
- package/corpus/genesis/procedures/howto-mail-to-tasks.md +26 -0
- package/corpus/genesis/procedures/howto-payment-flow.md +112 -0
- package/corpus/genesis/procedures/howto-subscription.md +33 -0
- package/corpus/genesis/procedures/notifications-blueprint.md +13 -0
- package/corpus/genesis/procedures/patterns-catalog.md +12 -0
- package/corpus/genesis/procedures/project-lifecycle.md +27 -0
- package/corpus/genesis/run-locally.md +310 -0
- package/corpus/genesis/testing.md +532 -0
- package/lib/config-cli.mjs +179 -0
- package/lib/install.mjs +639 -0
- package/lib/mcp-config.mjs +220 -0
- package/lib/semver.mjs +94 -0
- package/mcp/blueprint.mjs +64 -0
- package/mcp/client.mjs +141 -0
- package/mcp/config.mjs +166 -0
- package/mcp/key-page-html.mjs +275 -0
- package/mcp/key-page.mjs +217 -0
- package/mcp/key-store.mjs +204 -0
- package/mcp/key-tools.mjs +225 -0
- package/mcp/server.mjs +164 -0
- package/mcp/tools.mjs +270 -0
- package/package.json +43 -4
- package/skills/genesis-compiler-api/SKILL.md +56 -0
- package/skills/genesis-deploy/SKILL.md +36 -0
- package/skills/genesis-generated-code/SKILL.md +48 -0
- package/skills/genesis-run-and-test/SKILL.md +43 -0
|
@@ -0,0 +1,519 @@
|
|
|
1
|
+
# Extending the generated code: custom code
|
|
2
|
+
|
|
3
|
+
This page is for the moment a blueprint needs behaviour that its patterns cannot express. It says
|
|
4
|
+
where custom code lives, how the compiler turns it into the generated service, what it can call at
|
|
5
|
+
run time, and what survives a rebuild. How to design the library and edge patterns themselves is
|
|
6
|
+
in mind-set (`procedures/blueprint-authoring.md`, section "Library functions"); this page covers
|
|
7
|
+
what the compiler does with them. Related pages: [compiler-api.md](compiler-api.md),
|
|
8
|
+
[compile-results.md](compile-results.md), [testing.md](testing.md), [pitfalls.md](pitfalls.md).
|
|
9
|
+
|
|
10
|
+
## The central rules
|
|
11
|
+
|
|
12
|
+
1. **Custom code is part of the blueprint.** Library function bodies and edge function bodies are
|
|
13
|
+
fields of the blueprint (`moduleBody`). The compiler re-injects them into the service at every
|
|
14
|
+
build. To change them, change the blueprint and compile again.
|
|
15
|
+
2. **Never edit a generated file to change behaviour.** The next compile overwrites it, and the
|
|
16
|
+
overwrite is counted in that build's `addedLines`.
|
|
17
|
+
3. **Files you add survive.** A file you add to a service repo outside the generated paths stays
|
|
18
|
+
through every rebuild. Service files under `service-files/` are always yours.
|
|
19
|
+
4. **Where service-file bytes live.** The bytes stay in your service repo. The blueprint
|
|
20
|
+
holds only the manifest (`path`, `visibility`, `contentType`, `size`, `sha256`); the compiler
|
|
21
|
+
reads the bytes from your repo once builds compile onto your repositories. Until then (dry runs)
|
|
22
|
+
a build skips the declared service files and its log says so (`service files skipped
|
|
23
|
+
in Genesis builds`).
|
|
24
|
+
5. **Custom code runs with the service's full privileges.** There is no sandbox for MScript, arrow
|
|
25
|
+
functions, library functions or edge functions. Review them like any code you ship.
|
|
26
|
+
|
|
27
|
+
## The four escape hatches
|
|
28
|
+
|
|
29
|
+
Pick the lowest layer that carries the requirement.
|
|
30
|
+
|
|
31
|
+
| Hatch | Scope | Language |
|
|
32
|
+
|---|---|---|
|
|
33
|
+
| MScript | one pattern field (condition, where clause, default, response value) | JavaScript, transpiled to the service language |
|
|
34
|
+
| `ArrowFunctionAction` | one step inside one Business API | JavaScript body, spliced into the service |
|
|
35
|
+
| `ServiceLibrary` function | reused across the service, callable as `LIB.<moduleName>(...)` | the service's own language |
|
|
36
|
+
| `EdgeController` + edge function | a whole route (REST, Kafka) outside the action workflow | the service's own language |
|
|
37
|
+
|
|
38
|
+
Choice rules:
|
|
39
|
+
|
|
40
|
+
- A predicate, derived value or payload field: MScript.
|
|
41
|
+
- A transform of 1 to 10 lines used in one API that MScript cannot express: `ArrowFunctionAction`.
|
|
42
|
+
- Logic used in three places, or longer than about 10 to 30 lines: a library function.
|
|
43
|
+
- A route that must own the whole request (provider webhook with its own signature, binary
|
|
44
|
+
download, a bespoke report, a Kafka consumer): an edge controller. If the operation reads and
|
|
45
|
+
writes data objects in the request, validate, act, respond shape, declare a Business API instead.
|
|
46
|
+
- If it is persisted, declare it on the data object. Custom code is behaviour, not data. Tables
|
|
47
|
+
that a library reaches into outside the blueprint are invisible to migrations and will drift. For a
|
|
48
|
+
derived counter, declare the column and keep it up to date with an update action on a cron API.
|
|
49
|
+
|
|
50
|
+
Only MScript is portable. Library and edge bodies are never transpiled and never bridged: they are
|
|
51
|
+
written in the service's language.
|
|
52
|
+
|
|
53
|
+
## `LIB` and `API`: the ambient containers
|
|
54
|
+
|
|
55
|
+
In nodejs every hatch reaches the same two globals. No import is needed.
|
|
56
|
+
|
|
57
|
+
| Container | Holds | Call |
|
|
58
|
+
|---|---|---|
|
|
59
|
+
| `LIB` | this service's `library.functions[]` modules (plus native kernels) | `LIB.<moduleName>(...)` |
|
|
60
|
+
| `API` | this service's Business APIs that set `controllers.internalCallSettings.hasInternalCallController: true` | `await API.<apiName>({ ...params })` |
|
|
61
|
+
|
|
62
|
+
They are installed on `globalThis` at boot (`src/index.ts`) and typed by `src/lib/index.ts` and
|
|
63
|
+
`src/global.d.ts`. Rules:
|
|
64
|
+
|
|
65
|
+
- Call them inside the exported function, never at module load time. A top-level `API.x()` runs
|
|
66
|
+
before the install.
|
|
67
|
+
- `API` names are exact and case-sensitive. An unknown name is `INTERNAL_API_NOT_FOUND` and an API
|
|
68
|
+
without the controller is `INTERNAL_API_NO_CONTROLLER`, both at design time. Computed access
|
|
69
|
+
(`API[expr]`) cannot be checked and draws a warning.
|
|
70
|
+
- An internal call runs as the system with a null session. On nodejs and rust the second argument
|
|
71
|
+
carries a session: `await API.getOrder(params, { session: this.session })`. Even then role and
|
|
72
|
+
ownership gates stay bypassed, so the calling code must enforce access. A `session` key inside
|
|
73
|
+
`params` is plain input data.
|
|
74
|
+
- An internal call bypasses object moderation: a get or list of an `onPlace` object returns every
|
|
75
|
+
row, unapproved ones too. The same holds for direct `dbLayer` reads. Filter before relaying to a
|
|
76
|
+
user.
|
|
77
|
+
- Nested internal calls fail with `INTERNAL_CALL_DEPTH_EXCEEDED`.
|
|
78
|
+
|
|
79
|
+
## Library functions
|
|
80
|
+
|
|
81
|
+
`services.library` holds one library per service, with these partitions:
|
|
82
|
+
|
|
83
|
+
| Partition | Holds | Body key |
|
|
84
|
+
|---|---|---|
|
|
85
|
+
| `functions[]` | reusable functions | `ts` (preferred) or `js` on nodejs; the language key elsewhere |
|
|
86
|
+
| `edgeFunctions[]` | request handlers | same |
|
|
87
|
+
| `templates[]` | render templates | `ejs` |
|
|
88
|
+
| `assets[]` | server-side static text | `json`, `svg`, `txt`, `html` |
|
|
89
|
+
| `public[]` | files served at a public URL | same |
|
|
90
|
+
| `pdfTemplates[]` | PDF templates (page setup, header, footer, metadata) | EJS body plus layout fields |
|
|
91
|
+
| `hooks[]` | binds a `functions[]` module to service startup or shutdown | none |
|
|
92
|
+
|
|
93
|
+
### How the compiler picks a body
|
|
94
|
+
|
|
95
|
+
`moduleBody` is an object keyed by language, not file extension: `ts`, `js`, `python`, `java`,
|
|
96
|
+
`csharp`, `go`, `rust`. The compiler takes the key of the service's language (nodejs: `ts`, then
|
|
97
|
+
`js`). A module with no body for that language does not fail the build. It is emitted as a stub that
|
|
98
|
+
throws at runtime (for example "lib 'computeShipping' has no nodejs (js/ts) body") and the build
|
|
99
|
+
prints a warning. So read every build warning, and give each language you generate its own body.
|
|
100
|
+
A `php` key is kept but no service reads it.
|
|
101
|
+
|
|
102
|
+
Name rules: `moduleName` is camelCase, unique in its partition, and is the name MScript uses and
|
|
103
|
+
the file base name.
|
|
104
|
+
|
|
105
|
+
### A nodejs function
|
|
106
|
+
|
|
107
|
+
A module must export one function. `module.exports = { a, b }` and `exports.a = ...` are refused
|
|
108
|
+
(`NO_EXPORT`), because the runtime binds the module itself as `LIB.<moduleName>`.
|
|
109
|
+
|
|
110
|
+
```ts
|
|
111
|
+
module.exports = function shippingFee(order: { weightKg: number; zone: string; subtotal: number }): number {
|
|
112
|
+
if (!order || order.weightKg <= 0) throw new Error('order weight required');
|
|
113
|
+
if (order.subtotal >= 250) return 0;
|
|
114
|
+
return Math.round(order.weightKg * 12 * 100) / 100;
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
Call it from a `FunctionCallAction` with `callScript: { "js": "LIB.shippingFee({ ... })" }`. The
|
|
119
|
+
result lands on the workflow context as `this.<contextPropertyName>` (the action `name` when no
|
|
120
|
+
property name is set). Schedule that action before the step that reads it. For example, a data
|
|
121
|
+
clause item that reads it needs the action at `afterCheckBasicAuth`, because the data clause is
|
|
122
|
+
built after that step.
|
|
123
|
+
|
|
124
|
+
Several helpers in one module: declare the function, attach members, export it.
|
|
125
|
+
|
|
126
|
+
```js
|
|
127
|
+
function textTools() {}
|
|
128
|
+
textTools.slug = (s) => String(s).toLowerCase().replace(/\s+/g, '-');
|
|
129
|
+
module.exports = textTools;
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Then `LIB.textTools.slug(this.title)` works. Write `await` yourself where an attached member is
|
|
133
|
+
async; the validator does not check it. The validator trusts attached members only for this plain
|
|
134
|
+
shape (no `this`, `arguments`, `eval`, prototype tricks or reads of `process`). Anything else is
|
|
135
|
+
valid JavaScript, but the members are then not counted and `LIB_BARE_FUNCTION_MEMBER_ACCESS`
|
|
136
|
+
fires.
|
|
137
|
+
|
|
138
|
+
Undeclared names fail the build before anything is committed. The failed build names the function,
|
|
139
|
+
the identifier and the line. Allowed without declaration: `LIB`, `API`, the CommonJS names, Node
|
|
140
|
+
globals and JavaScript built-ins. Reach a runtime-specific name through `globalThis.Deno?.exit()`,
|
|
141
|
+
not `Deno.exit()`. The same check runs in the repo as the `correctness/noUndeclaredVariables` lint rule on
|
|
142
|
+
`src/lib/functions/**` and `src/lib/edge/**`: `npm run lint` shows it.
|
|
143
|
+
|
|
144
|
+
To give a function the whole workflow context (API params, the session, earlier action results),
|
|
145
|
+
pass `this`: `LIB.quoteOrder(this)`. Pass plain fields instead for a pure helper that is easy to
|
|
146
|
+
test. A function that writes rows forwards `{ session, requestId }` to the `dbLayer` helper.
|
|
147
|
+
|
|
148
|
+
### Publishing Kafka events from a function
|
|
149
|
+
|
|
150
|
+
A function that publishes with `ServicePublisher` lists every topic in `publishes` on the library
|
|
151
|
+
module, named as it goes on the wire (`<codename>-<service>-service-<event>`). The compiler cannot see
|
|
152
|
+
inside a body. Without the declaration, a listener on that topic is reported producerless at boot
|
|
153
|
+
(`KafkaSubscriptionsWithoutProducer`, ERROR) and at design time (`KAFKA_TOPIC_NO_PRODUCER`). Code that
|
|
154
|
+
is not a library module declares on `serviceSettings.serviceOptions.publishes`. A lifecycle hook that must
|
|
155
|
+
announce a fact calls a library function that publishes. An `API.*` call from a hook that already
|
|
156
|
+
runs inside an internal call fails with `INTERNAL_CALL_DEPTH_EXCEEDED`.
|
|
157
|
+
|
|
158
|
+
### Startup and shutdown hooks
|
|
159
|
+
|
|
160
|
+
`hooks[]` entries (`name`, `trigger` of `startup` or `shutdown`, `moduleName`) run a `functions[]`
|
|
161
|
+
module automatically. Only `functions[]` modules are valid (`HOOK_MODULE_NOT_FOUND`).
|
|
162
|
+
|
|
163
|
+
- Startup hooks run on every boot, so the function must be idempotent (upserts, existence checks,
|
|
164
|
+
version stamps; never blind inserts).
|
|
165
|
+
- They run after the database, migrations, seed data and Kafka wiring, and before the HTTP listener binds.
|
|
166
|
+
- A hook that throws never blocks `/health` and never crashes boot. It is logged as
|
|
167
|
+
`StartupHookFailed` and the service comes up degraded. Success is `StartupHookCompleted`.
|
|
168
|
+
- A startup hook is awaited for at most `LIBRARY_STARTUP_HOOK_BUDGET_MS` (default 30000). After
|
|
169
|
+
that boot proceeds (`StartupHookStillRunning`) while the hook keeps running.
|
|
170
|
+
- Shutdown hooks run best effort on SIGTERM and SIGINT, bounded by
|
|
171
|
+
`LIBRARY_SHUTDOWN_HOOK_BUDGET_MS` (default 10000). A killed process runs nothing, so never
|
|
172
|
+
depend on one for correctness.
|
|
173
|
+
|
|
174
|
+
### Service files in code
|
|
175
|
+
|
|
176
|
+
On a nodejs service that has at least one service file, `LIB.serviceFiles` exists:
|
|
177
|
+
`text(path)`, `json(path)`, `bytes(path)`, `info(path)`, `has(path)`. All are synchronous (no
|
|
178
|
+
`await`). An unlisted path throws `ServiceFileNotFoundError`; a missing or altered file throws
|
|
179
|
+
`ServiceFileIntegrityError`. Do not name a library module `serviceFiles`. A private file is not secret:
|
|
180
|
+
it sits in the repo and in every image built from it. The manifest design is in mind-set.
|
|
181
|
+
|
|
182
|
+
## Edge controllers
|
|
183
|
+
|
|
184
|
+
Two blueprint pieces that reference each other by name: an entry in `services.edgeControllers[]`
|
|
185
|
+
and an `edgeFunctions[]` module. `edgeControllerOptions.functionName` must equal the module's
|
|
186
|
+
`moduleName`. The compiler emits an adapter that authenticates, builds the context and calls your
|
|
187
|
+
function.
|
|
188
|
+
|
|
189
|
+
### Triggers
|
|
190
|
+
|
|
191
|
+
| Block | Fields | Generated |
|
|
192
|
+
|---|---|---|
|
|
193
|
+
| `edgeRestSettings` | `restEnabled`, `method`, `path` | a REST route |
|
|
194
|
+
| `edgeKafkaSettings` | `kafkaEnabled`, `requestTopic`, `responseTopic?` | a Kafka consumer; the return value goes to `responseTopic` |
|
|
195
|
+
| `edgeCronSettings` | `cronEnabled`, `cronExpression` | nothing: no generator schedules it |
|
|
196
|
+
|
|
197
|
+
`cronEnabled: true` draws the warning `EDGE_CRON_NOT_SCHEDULED` and the function is never called by
|
|
198
|
+
a clock. For scheduled work use a service cron job (`Service.cronJobs[]`), a cron-triggered
|
|
199
|
+
Business API, or an external scheduler that calls the REST route.
|
|
200
|
+
|
|
201
|
+
### Where the route lives
|
|
202
|
+
|
|
203
|
+
`edgeRestSettings.path` is relative to the edge sub-router. The mount prefix is `/edge`, so
|
|
204
|
+
`/generate-invoice` is served at `/edge/generate-invoice`. Never write the prefix into `path`
|
|
205
|
+
(`/edge/x` would be served at `/edge/edge/x`). A deployment can change the prefix with the env var
|
|
206
|
+
`EDGE_MOUNT_PATH` (unset or blank is `/edge`; `/` mounts at the root). The declared paths never
|
|
207
|
+
change. Edge routes stay out of the `/v1` business API routes.
|
|
208
|
+
|
|
209
|
+
### Kafka consumer group
|
|
210
|
+
|
|
211
|
+
A Kafka edge controller consumes in the group `<SERVICE_CODENAME>-edge-<functionName>`: keyed by
|
|
212
|
+
the function, not the topic. Two functions on the same `requestTopic` both receive every message
|
|
213
|
+
and each publishes its own response. If you want one handler, use one edge controller and branch
|
|
214
|
+
inside it. `requestTopic` defaults to `<functionName>-requests`. Replicas of the same function share
|
|
215
|
+
the partitions.
|
|
216
|
+
|
|
217
|
+
### The handler context
|
|
218
|
+
|
|
219
|
+
One async export that receives one normalized context, not an Express request:
|
|
220
|
+
|
|
221
|
+
| Field | Meaning |
|
|
222
|
+
|---|---|
|
|
223
|
+
| `body` | REST: `{ ...req.body, ...req.query }`; Kafka: the payload |
|
|
224
|
+
| `headers` | request headers (REST); `{}` on Kafka |
|
|
225
|
+
| `session` | REST: the user session, or `null` on a public or service-token-only route; Kafka: the identity the event arrived with, or `null` |
|
|
226
|
+
| `isM2M`, `m2mIssuer` | true only for a verified, admitted service token, and the issuing service name from the token; always false on Kafka |
|
|
227
|
+
| `requestId` | the correlation id, never empty (a fresh UUID when none came in) |
|
|
228
|
+
| `service`, `function` | for logging |
|
|
229
|
+
|
|
230
|
+
### Return contract
|
|
231
|
+
|
|
232
|
+
- A plain value is sent as JSON with HTTP 200, even an object that has a `status` member (it is
|
|
233
|
+
just a JSON field).
|
|
234
|
+
- A response envelope `{ status, headers?, body? }` is honoured when a numeric `status` comes with a
|
|
235
|
+
`body` and/or an object `headers`. A `Buffer` or string `body` is sent raw (images, CSV, PDF);
|
|
236
|
+
anything else is JSON. A bare `{ status: 200, ... }` stays a JSON body.
|
|
237
|
+
- To refuse, throw an error with a numeric `.status`; the adapter answers `application/problem+json`
|
|
238
|
+
with that code (500 by default).
|
|
239
|
+
- On a Kafka controller with a `responseTopic`, a plain return is published as is and an envelope
|
|
240
|
+
publishes its `body`. An envelope with `status >= 400` is a refusal: it is logged as
|
|
241
|
+
`EdgeKafkaMessageRejected` and never published, and the offset commits. Without a `responseTopic`
|
|
242
|
+
the return value is ignored.
|
|
243
|
+
|
|
244
|
+
### Auth
|
|
245
|
+
|
|
246
|
+
The adapter enforces auth before your function runs:
|
|
247
|
+
|
|
248
|
+
| `loginRequired` | `M2MAllowed` | Mode | Needs | Refusal |
|
|
249
|
+
|---|---|---|---|---|
|
|
250
|
+
| true | false | session | a user session | 401 `SESSION_REQUIRED` |
|
|
251
|
+
| false | true | m2m | a verified, admitted service token | 401 `M2M_TOKEN_REQUIRED`, 403 `M2M_CLIENT_NOT_ALLOWED` |
|
|
252
|
+
| true | true | both | a session or a service token | 401 `SESSION_REQUIRED`, 403 `M2M_CLIENT_NOT_ALLOWED` |
|
|
253
|
+
| false | false | public | nothing | none |
|
|
254
|
+
|
|
255
|
+
The credential is the `x-m2m-token` header (an `Authorization: Bearer` token is tried only when
|
|
256
|
+
`x-m2m-token` is absent, and only if it verifies as a service token). Verified means an EdDSA
|
|
257
|
+
service JWT from a service of this project, signed by that service's key and addressed to this
|
|
258
|
+
service. Admitted means it passes `serviceOptions.allowedM2MClients` / `blockedM2MClients`. Kafka
|
|
259
|
+
has no per-message check. A webhook with a provider signature is a public (or M2M) route that
|
|
260
|
+
verifies `ctx.headers[...]` itself and throws a `.status = 401` error on a bad signature.
|
|
261
|
+
|
|
262
|
+
### Other languages
|
|
263
|
+
|
|
264
|
+
- Library and edge bodies are never bridged. Without a native body there is a warning and a stub
|
|
265
|
+
that fails at runtime.
|
|
266
|
+
- .NET: `moduleBody.csharp`, a global static PascalCase class with
|
|
267
|
+
`Task<Dictionary<string, object?>> Invoke(IDictionary<string, object?> ctx)`. For typed `LIB`
|
|
268
|
+
forwarders a leading comment `// @mindbricks-lib-invoke {"returnType":...,"parameters":[...]}` declares each
|
|
269
|
+
`Invoke` overload (bad metadata is `DOTNET_LIB_INVOKE_METADATA_INVALID`; missing metadata is the warning
|
|
270
|
+
`DOTNET_LIB_INVOKE_METADATA_MISSING`). Call `global::LIB.echo(...)`; `API.getOrder(input)` returns a
|
|
271
|
+
`Task`.
|
|
272
|
+
- Java: `moduleBody.java`; static `API` and `LIB` in the service's `lib` package;
|
|
273
|
+
`API.<name>(params)`. An API name outside the membership is `JAVA_NATIVE_API_UNBOUND`.
|
|
274
|
+
- Rust: `moduleBody.rust`. A function is `src/lib_layer/<snake_name>.rs`; an edge function is
|
|
275
|
+
`src/edge_functions/<snake_name>.rs` with
|
|
276
|
+
`pub async fn handle(ctx: Value, pool: Arc<PgPool>) -> Result<Value, EdgeError>`. There is no
|
|
277
|
+
`LIB` or `API` identifier: use `crate::lib_layer::<snake_name>(...)` and
|
|
278
|
+
`crate::api::<snake_name>(params, session).await` (`crate::api::blocking::...` in a synchronous
|
|
279
|
+
body). Errors `RUST_INTERNAL_API_UNBOUND`, `RUST_INTERNAL_API_VALUE_UNSUPPORTED`,
|
|
280
|
+
`RUST_NATIVE_API_UNBOUND`. Rust edge routes exist for PostgreSQL only. A binary body is a base64 string
|
|
281
|
+
with `"bodyEncoding": "base64"` (`EDGE_BODY_ENCODING_INVALID`, `EDGE_BODY_BASE64_INVALID`).
|
|
282
|
+
- Python: `moduleBody.python`. A function is `src/<package>/lib/functions/<moduleName>.py` and must define a
|
|
283
|
+
top-level function named `<moduleName>`; an edge function is `src/<package>/lib/edge/<moduleName>.py`
|
|
284
|
+
with `handler(ctx)`. `API` and `LIB` are bare names. `await API.<apiName>(params)`; there is no
|
|
285
|
+
session argument. A wrong name raises an error naming `INTERNAL_API_NOT_REGISTERED`.
|
|
286
|
+
- Go: the edge REST route is mounted with auth and answers `501 EDGE_FN_UNSUPPORTED`; its Kafka edge
|
|
287
|
+
transport is absent.
|
|
288
|
+
- Payment lifecycle hooks are not lowered on Rust or Python (`PAYMENT_LIFECYCLE_HOOK_NOT_LOWERED`),
|
|
289
|
+
so an `API` call in a hook does not run there.
|
|
290
|
+
- When converting an existing nodejs blueprint, keep the original `ts`/`js` bodies, add the native
|
|
291
|
+
body, check each missing-body warning, and test the real route. A build that succeeds is not proof
|
|
292
|
+
of behaviour.
|
|
293
|
+
|
|
294
|
+
## Native MScript expression slots
|
|
295
|
+
|
|
296
|
+
Use the portable `js` expression for a blueprint that targets several languages. A
|
|
297
|
+
nonempty native expression is inserted into its target's generated evaluator.
|
|
298
|
+
Expression slots and library module bodies have different contracts.
|
|
299
|
+
|
|
300
|
+
| Target | Native expression contract |
|
|
301
|
+
| --- | --- |
|
|
302
|
+
| Java | The generated manager evaluates the expression in a synchronous `Supplier`. `this` is the manager. Use a map lookup or `MScriptRuntime.prop(this.order, "status")` for nested values. `prop` returns null for a null receiver. |
|
|
303
|
+
| .NET | The manager evaluates the expression in a synchronous `Func`. Read custom context from `Context["order"]` or `Ms.Get(this, "order")`. The lambda has no top-level `await`. The wrapper completes a returned `Task` or `ValueTask` before using its value. |
|
|
304
|
+
| Rust | In API managers, `this` is the JSON snapshot from `self.this_view()`. Read members with JSON indexing or the MScript `member` helper. The synchronous closure returns the value directly. Value-model slots require a `serde_json::Value`, such as `serde_json::json!(true)`. Other slots can wrap the result in JSON themselves. |
|
|
305
|
+
|
|
306
|
+
For .NET expressions, the first non-nullish key wins in this order
|
|
307
|
+
`csharp`, `cs`, `dotnet`. An empty winning key falls back to translating `js`
|
|
308
|
+
(or `value`) and shadows subsequent aliases. For example, in
|
|
309
|
+
`{ "js": "40 + 2", "csharp": "", "cs": "6 * 7" }`, .NET translates
|
|
310
|
+
`40 + 2`. Remove the empty alias to use `cs`. A .NET library module body
|
|
311
|
+
uses only `moduleBody.csharp`.
|
|
312
|
+
|
|
313
|
+
Design validation, native syntax checking and translation during generation
|
|
314
|
+
each report their own findings. A valid design can still encounter a per-slot
|
|
315
|
+
translation refusal. For example, fiscal lifecycle conditions and scripts
|
|
316
|
+
refuse translator warnings and unsupported waits, while JobStream reports
|
|
317
|
+
translator warnings. Use the failed build's `buildErrors` entry to locate the
|
|
318
|
+
service, slot and source expression. Read [the compile result guide](compile-results.md)
|
|
319
|
+
for diagnosis and repair the exact expression that failed.
|
|
320
|
+
|
|
321
|
+
## What custom code can call (nodejs)
|
|
322
|
+
|
|
323
|
+
Import aliases, resolved at compile time, at run time and under test:
|
|
324
|
+
|
|
325
|
+
| Alias | Is |
|
|
326
|
+
|---|---|
|
|
327
|
+
| `common` | the framework runtime barrel |
|
|
328
|
+
| `dbLayer` | per-data-object CRUD helpers generated from this service's `dataObjects[]` |
|
|
329
|
+
| `models` | raw Sequelize or Mongoose models (advanced; skips indexing, events, soft delete) |
|
|
330
|
+
| `apiLayer`, `controllers`, `agentLayer` | internal runtime; rarely imported by hand |
|
|
331
|
+
| `bucketLayer` | bucket layer; `bucketLayer/bucket-router` exports `readBucketObject`, `writeBucketObject`, `deleteBucketObject` |
|
|
332
|
+
|
|
333
|
+
There is no `serviceCommon` package and no such alias. `ServicePublisher` and `ElasticIndexer`
|
|
334
|
+
ship from `common`. Code that imports `serviceCommon` must import from `common`.
|
|
335
|
+
|
|
336
|
+
Useful `common` exports: `ServicePublisher` (Kafka event with the caller's session:
|
|
337
|
+
`new ServicePublisher(eventName, data, session, requestId).publish()`), `KafkaListener`,
|
|
338
|
+
`dbEventTopic`, `ServiceIndexer`, `fetchRemote*`, `getRedisClient`, `EntityCache`,
|
|
339
|
+
`QueryCache` and `QueryCacheInvalidator`, `getLogger`, `newUUID`, `runMScript`, and the
|
|
340
|
+
Ed25519 helpers `signServiceToken` / `verifyServiceToken`. There is no OpenAI or Stripe client
|
|
341
|
+
export: AI and payments go through their blueprint settings. The authoritative export list is the
|
|
342
|
+
generated `src/common/index.ts` in the service repo; check there before relying on a name.
|
|
343
|
+
|
|
344
|
+
Two helpers live on the API manager, so MScript and arrow functions reach them as `this.*`:
|
|
345
|
+
`this.hashPassword(plain)` (bcrypt, cost 10) and `this.hashCompare(plain, hashed)`. Both return
|
|
346
|
+
Promises; inside a larger expression write `await` yourself, or the design is refused with
|
|
347
|
+
`MSCRIPT_ASYNC_HELPER_NOT_AWAITED`.
|
|
348
|
+
|
|
349
|
+
### `dbLayer` helpers
|
|
350
|
+
|
|
351
|
+
Always prefer `dbLayer` to `models`. For an object `order` the names are `<operation><Order>`:
|
|
352
|
+
|
|
353
|
+
- writes: `createOrder(data, context?)`, `createBulkOrder(rows, context?)`,
|
|
354
|
+
`updateOrderById(id, data, context?)`, `updateOrderByIdList(ids, data, context?)`,
|
|
355
|
+
`updateOrderByMQuery(data, mQuery, context?, includeArchived?)` (data clause FIRST),
|
|
356
|
+
`deleteOrderById(id, context?)`, `deleteOrderByMQuery(mQuery, context?, includeArchived?)`;
|
|
357
|
+
- reads: `getOrderById`, `getOrderByMQuery`, `getOrderAggById`, `listOrder`, `listOrderByMQuery`
|
|
358
|
+
(alias `getOrderListByMQuery`), `countOrder`, `getOrderStatsByMQuery`, `getIdListOfOrderByField`,
|
|
359
|
+
`getIdPageOfOrder`, plus `getOrderBy<Field>` for each indexed non-key field.
|
|
360
|
+
|
|
361
|
+
Rules worth knowing:
|
|
362
|
+
|
|
363
|
+
- Two families, one suffix apart. `listOrder`, `countOrder`, `getOrderByQuery` and
|
|
364
|
+
`deleteOrderByQuery` take a raw driver `where`; MQuery `$` operators fail there on a SQL service.
|
|
365
|
+
Only the `*ByMQuery` family lowers MQuery. Validation refuses the raw form in library and edge code
|
|
366
|
+
with `LIB_DBLAYER_RAW_WHERE_USES_MQUERY`. `getOrderListByMQuery` takes a bare MQuery and returns an
|
|
367
|
+
array, not `{ rows, count }`. The `*ByQuery` helpers are deprecated.
|
|
368
|
+
- Return shapes: `createOrder` gives `{ success, data?, error?, existing? }`. `updateOrderById`
|
|
369
|
+
gives `{ success, affectedRows, data }` and answers `success: false` (it does not throw) when no live row
|
|
370
|
+
matches. `deleteOrderById` gives the deleted row, or `null`; there is no `success` flag.
|
|
371
|
+
- Stats: `getOrderStatsByMQuery(mQuery, stats?, bucketBy?)`. `stats` is an ARRAY (`'count'`,
|
|
372
|
+
`'sum(f)'`, `'avg(f)'`, `'min(f)'`, `'max(f)'`). Without `bucketBy` the result is a flat object
|
|
373
|
+
keyed `op-field` (`{ count, 'sum-total' }`, an object even for one stat). With `bucketBy` it is an
|
|
374
|
+
array of buckets, each with `count`.
|
|
375
|
+
- `context` is `{ session?, requestId? }`. Pass the caller's so the Kafka events carry the session.
|
|
376
|
+
Omit it for cron and system work.
|
|
377
|
+
- A bulk or by-query update or delete with no selector besides `isActive` (after the soft-delete gate is
|
|
378
|
+
waived) is refused.
|
|
379
|
+
- All delete helpers honour `onDeleteAction` relations; a live `restrict` child refuses the delete
|
|
380
|
+
with 409 `RELATION_RESTRICTED`.
|
|
381
|
+
- Soft delete: with it on, a delete flips `isActive` to false and stamps `_archivedAt`; reads hide
|
|
382
|
+
those rows unless `includeArchived` is true. Without it, a delete removes the row.
|
|
383
|
+
- `dbLayer` skips the Business API permission and validation chain. Use it only on paths that have
|
|
384
|
+
made their own checks (an edge function with explicit authorization, a cron library call). In a
|
|
385
|
+
Business API use the CRUD actions.
|
|
386
|
+
|
|
387
|
+
Every write is followed by, in order: the Elasticsearch index write (awaited for at most
|
|
388
|
+
`ELASTIC_PROJECTION_WAIT_MS`, default 3000; a timeout is logged `ElasticProjectionWaitExpired` and is
|
|
389
|
+
not fatal), the `elastic-index-<codename>_<object>-<verb>` event, and the
|
|
390
|
+
`<SERVICE_CODENAME>-dbevent-<objectname>-<verb>` event (always emitted). A failed index or publish
|
|
391
|
+
never rolls back the database write. Because the db event is published after the index attempt, a
|
|
392
|
+
consumer that re-reads the projection normally sees the new row.
|
|
393
|
+
|
|
394
|
+
Example (library function for a cron API):
|
|
395
|
+
|
|
396
|
+
```js
|
|
397
|
+
const { listOrder, updateOrderByIdList } = require('dbLayer');
|
|
398
|
+
const { Op } = require('sequelize');
|
|
399
|
+
|
|
400
|
+
module.exports = async function closeStaleOrders(cutoffDate, context) {
|
|
401
|
+
const { rows } = await listOrder({
|
|
402
|
+
where: { status: 'awaiting_payment', createdAt: { [Op.lt]: cutoffDate } },
|
|
403
|
+
limit: 1000,
|
|
404
|
+
});
|
|
405
|
+
if (rows.length === 0) return { closed: 0 };
|
|
406
|
+
const result = await updateOrderByIdList(rows.map(r => r.id), { status: 'canceled' }, context);
|
|
407
|
+
return { closed: result.count };
|
|
408
|
+
};
|
|
409
|
+
```
|
|
410
|
+
|
|
411
|
+
## Reading another service's data
|
|
412
|
+
|
|
413
|
+
Never read another service's database. Read its Elasticsearch projection instead; every
|
|
414
|
+
`dbLayer` write keeps it current. Pick by call site:
|
|
415
|
+
|
|
416
|
+
| Where | Use |
|
|
417
|
+
|---|---|
|
|
418
|
+
| Inside a Business API | `InterserviceCallAction` for one typed call, or a static join that copies the field at write time |
|
|
419
|
+
| Inside a BFF data view | the view's `joins[]` |
|
|
420
|
+
| Library, edge or cron code | `ServiceIndexer` or `fetchRemote*` from `common` |
|
|
421
|
+
|
|
422
|
+
Rules for code you write:
|
|
423
|
+
|
|
424
|
+
- `new ServiceIndexer('User', session, requestId)` takes the bare object name; it adds the project
|
|
425
|
+
codename (`<codename>_user`). `session` and `requestId` are only diagnostic context. They do not
|
|
426
|
+
filter the read.
|
|
427
|
+
- Read methods: `getById`, `getOne`, `getByPage(from, size, query?, sort?, cached?)`, `getCount`
|
|
428
|
+
(each takes `{ onError: 'throw' }`), `getAggregations` (degrades to `null`), `rawSearch` (always
|
|
429
|
+
throws), `checkIndexState()` (`present`, `absent`, `unknown`). By default a failure degrades to
|
|
430
|
+
`null`, `[]` or `0`, which is indistinguishable from "nothing there". Pass `onError: 'throw'`, use
|
|
431
|
+
the `*WithStatus` helpers (`{ data, failure }`), or check `checkIndexState()` whenever the degraded
|
|
432
|
+
value is an answer you act on. An unconfigured Elasticsearch client still returns the empty value
|
|
433
|
+
even with `onError: 'throw'`.
|
|
434
|
+
- `fetchRemoteObjectByMQuery(object, query, ctx?)` and the list, stats and `*ByElasticQuery` variants
|
|
435
|
+
take an optional trailing `{ session, requestId }`. The plain helpers collapse "absent" and "not
|
|
436
|
+
answered" into `null` or `[]`.
|
|
437
|
+
- Not tenant-scoped. The index is namespaced by the project codename only and the session is never
|
|
438
|
+
consulted on a read. Add the tenant column to your query yourself, failing closed:
|
|
439
|
+
`convertMQueryToElasticQuery({ companyId: ctx.session?.tenantId ?? null, status: 'active' })`.
|
|
440
|
+
- Not access-scoped. Every data object is projected, whatever its `dataObjectAccess`, and the
|
|
441
|
+
owner's Business API gates (such as `ownershipCheck`) do not run. For a read that must honour the
|
|
442
|
+
owner's rules, or must see the database, use an `InterserviceCallAction` to the owner's API.
|
|
443
|
+
Custom reads also add no moderation gate: filter on `moderationStatus` yourself.
|
|
444
|
+
- Rows from `fetchRemote*` never carry a property the project marks `hashed` or `isSecret`.
|
|
445
|
+
- Paginate every list read. Prefer one `getByPage` or aggregation to a loop of `getById`.
|
|
446
|
+
- Cross-service reads are async. Wrap them in a library function and call it through a
|
|
447
|
+
`FunctionCallAction`; do not inline them in MScript.
|
|
448
|
+
- The `projection-index-missing` reason: the owning service never indexed that object, or its
|
|
449
|
+
Elasticsearch bootstrap or startup sync failed. Read the owner's boot log for `[SMART-SYNC]` lines, fix the
|
|
450
|
+
cause, restart the owner (its startup sync rebuilds an absent or empty index). A row inserted
|
|
451
|
+
straight into the database by a script is not projected until the next rebuild. Other reasons:
|
|
452
|
+
`elastic-not-configured`, `invalid-arguments`, `query-conversion-failed`, `result-window`,
|
|
453
|
+
`partial-timeout`, `transport-error`.
|
|
454
|
+
- A cross-service `FetchObjectAction` that cannot be answered logs `RemoteFetchNotAnswered` and
|
|
455
|
+
answers 503 when required (it is fail-closed: a gate behind it does not run).
|
|
456
|
+
|
|
457
|
+
## Packages
|
|
458
|
+
|
|
459
|
+
Third-party dependencies are `serviceSettings.serviceBasics.packages[]` entries: `manager`,
|
|
460
|
+
`name`, `version` (`"*"` is latest), optional `importAs`, optional `options`.
|
|
461
|
+
|
|
462
|
+
| `manager` | Written into |
|
|
463
|
+
|---|---|
|
|
464
|
+
| `npm` | `package.json` dependencies |
|
|
465
|
+
| `pip` | `pyproject.toml` |
|
|
466
|
+
| `maven` (`name` is `groupId:artifactId`) | `pom.xml` |
|
|
467
|
+
| `nuget` | `.csproj` |
|
|
468
|
+
| `gomod` (`name` is the module path) | `go.mod` |
|
|
469
|
+
| `cargo` | `Cargo.toml` (`options` carry crate features) |
|
|
470
|
+
| `composer` | nowhere (no service language generates it) |
|
|
471
|
+
| `vcpkg`, `conan`, `apt` | the build image, for native libraries a kernel links |
|
|
472
|
+
|
|
473
|
+
Each target takes only the entries whose manager matches its language. An npm package used only by
|
|
474
|
+
a `ts` body is not loaded by a non-nodejs service: declare the target language's equivalent and use
|
|
475
|
+
it from the native body. `importAs` also binds the package into MScript, so an expression can say
|
|
476
|
+
`dayjs(this.createdAt)`; without it the package is only installed and the function imports it itself.
|
|
477
|
+
The nodejs compiler also merges `import` and `require` references in your library modules into
|
|
478
|
+
`package.json`. Older `nodejsPackages[]` items are migrated into `packages[]` (`defaultImportConst`
|
|
479
|
+
becomes `importAs`). For `apt`, `name` matches `^[a-z0-9][a-z0-9+.-]+$` and `version` is `"*"` or
|
|
480
|
+
matches `^[0-9][A-Za-z0-9.+:~-]*$`.
|
|
481
|
+
|
|
482
|
+
A native C or C++ library is a `NativeKernel` on `library.nativeKernels[]`, called as
|
|
483
|
+
`LIB.<name>(...)` and linked into the service binary through FFI. Use it only when a specific
|
|
484
|
+
native library must be linked: it adds a native build dependency, a crash takes the process down, and it
|
|
485
|
+
has no database or session access. Keep calls coarse. Its design is in the kernel pattern of mind-set.
|
|
486
|
+
|
|
487
|
+
## Names a custom-code author needs
|
|
488
|
+
|
|
489
|
+
The compiler's naming rules (the generated repo itself holds the per-generator detail; read the
|
|
490
|
+
generated files for the exact names before you call them):
|
|
491
|
+
|
|
492
|
+
- Data object: model name is PascalCase of the object name (`userGroup` becomes `UserGroup`).
|
|
493
|
+
Cross-service reference is `<service>:<object>`. Postgres table is the English plural of the object
|
|
494
|
+
name with its camel case kept (`order` becomes `orders`, `person` becomes `people`). MongoDB
|
|
495
|
+
collection is the object name verbatim. Objects generated by higher-order patterns (payment, agent hub)
|
|
496
|
+
start with `sys_`.
|
|
497
|
+
- `dbLayer`: `<operation><ModelName>`, imported from `dbLayer`.
|
|
498
|
+
- REST routes sit behind `/<serviceName>-api/v<version>/`. Standard action with its own resource:
|
|
499
|
+
`/<pluralObject>` and `/<pluralObject>/:<object>Id`. Non-standard action: `/<action><Resource>`.
|
|
500
|
+
An explicit `restSettings.routePath` overrides. Standard actions are get, list, create, update,
|
|
501
|
+
delete; there is no automatic CRUD, only declared APIs.
|
|
502
|
+
- Kafka: DB events `<serviceCodename>-dbevent-<objectname>-<verb>` with
|
|
503
|
+
`serviceCodename = <projectCodename>-<serviceName>-service` and verb `created`, `updated` or
|
|
504
|
+
`deleted`; API events `<projectCodename>-<serviceName>-service-<raisedEvent>`, lowercase, raised
|
|
505
|
+
only when `apiOptions.raiseApiEvent` is true; index events `elastic-index-<codename>_<object>-<verb>`.
|
|
506
|
+
- Elasticsearch index: `<codename>_<objectname>` lowercased.
|
|
507
|
+
- Every service that lets users log in also serves `GET /publickey` (its Ed25519 public key) and
|
|
508
|
+
`POST /rawsearch/:index`. The latter needs a login, and stays off (403) until the operator sets
|
|
509
|
+
`RAWSEARCH_INDEXES`; it accepts only a small query subset (no scripts, aggregations or wildcards).
|
|
510
|
+
|
|
511
|
+
## Testing custom code
|
|
512
|
+
|
|
513
|
+
A generated service collects `**/__tests__/**/*.test.ts` and `**/*.spec.ts`, and nothing else.
|
|
514
|
+
`__tests__/mytest.ts` and `my.test.js` are silently never run. Keep helper modules out of
|
|
515
|
+
`__tests__/` (the convention is a sibling `__testutils__/`), because jest would collect them as
|
|
516
|
+
empty suites and fail the run. Test an edge function by passing a mocked context
|
|
517
|
+
(`{ body, headers, session }`) and asserting the return value; mock `dbLayer` so unit tests never
|
|
518
|
+
touch a database, and mock `ServiceIndexer` for cross-service reads. The aliases `common`,
|
|
519
|
+
`dbLayer` and `models` resolve under jest. More in [testing.md](testing.md).
|