@starci/hfs 4.7.38 → 4.7.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +4 -0
- package/package.json +1 -1
- package/runtime/knowledge/hfs/canon-pins.yaml +7 -7
- package/runtime/knowledge/hfs/rules.yaml +2 -1
- package/runtime/knowledge/hfs/slots.yaml +19 -1
- package/runtime/knowledge/patterns/be/queues.yaml +1 -1
- package/runtime/scripts/hfs/declaration-shape.mjs +14 -2
- package/runtime/scripts/hfs/rules/database-queue-outbox.mjs +102 -0
- package/runtime/scripts/hfs/rules/database-queues.mjs +35 -17
- package/scaffold/add-queue-lite.mjs +3 -1
- package/scaffold/app.mjs +1 -0
- package/scaffold/lite-exports.mjs +8 -2
- package/templates/app/skeleton-lite/.starcistacks/dev/README.md +1 -1
- package/templates/be/skeleton-lite/src/modules/platform/logging/index.ts +2 -13
- package/templates/be/skeleton-lite/src/modules/platform/primitives/index.ts +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 4.7.39 - 2026-10-26
|
|
4
|
+
- Added: R258 follows the active cron schedule and accepts a declared outbox (`sides.be.queueOutbox`), and the optional slot `app.scripts.integration` carries integration scripts.
|
|
5
|
+
- Fixed: the lite scaffold templates, barrels and dev README follow the lite scaffold end-to-end fixes.
|
|
6
|
+
|
|
3
7
|
## 4.7.38 - 2026-10-26
|
|
4
8
|
- Changed: the bundled canon pins select @starci/stylelint-canon 2.0.13, @starci/eslint-canon-be 3.4.46, @starci/eslint-canon-fe 9.1.1, the Grammar lint families 0.12.5 and @starci/grammar 0.12.5.
|
|
5
9
|
- Fixed: the full and lite scaffold templates, skeletons and specs follow the scaffold end-to-end and repaint fixes made after 4.7.37.
|
package/package.json
CHANGED
|
@@ -23,35 +23,35 @@ pins:
|
|
|
23
23
|
source: packages/heroicons/package.json
|
|
24
24
|
why: 'StarCi custom Heroicons-compatible cuts; eslint-grammar-starci (icon.mjs) admits @starci/heroicons/24/outline and /16/solid as icon sources, so an app that uses them gets this exact version.'
|
|
25
25
|
'@starci/grammar':
|
|
26
|
-
version: 0.12.
|
|
26
|
+
version: 0.12.6
|
|
27
27
|
group: starci
|
|
28
28
|
install: registry
|
|
29
29
|
side: fe
|
|
30
30
|
source: packages/grammar/package.json
|
|
31
31
|
why: 'the grammar front ends read: the brand layer sets `--font-sans` and `--font-mono` and the grammar reads them; it carries the Input tel kind, the copyable and read-only Input, the IconButton disclosure props and count badge, ChatBubble, FillViewport, Textarea mentions and Enter-to-send, the leading PrimaryRailLayout rail, the plate-less Image, ListRow, the soft Alert, Progress steps, the floating primary IconButton, FlowGraph, the STATE-8 loading views and the bordered selectable TagGroup chips. The runtime source version wins for a @starci package.'
|
|
32
32
|
'@starci/eslint-canon-be':
|
|
33
|
-
version: 3.4.
|
|
33
|
+
version: 3.4.47
|
|
34
34
|
group: starci
|
|
35
35
|
install: registry
|
|
36
36
|
side: be
|
|
37
37
|
source: packages/eslint/be/package.json
|
|
38
38
|
why: 'the back-end ESLint canon: `loadHfs(import.meta.url)` of be/eslint.config.mjs finds the app-root hfs.json (kind app) and gives every linted file the view of the be side, and the project graph is built per side; its bundled runtime copies (slots, canon-pins, failure codes, the architecture machine) put .starcistacks and .sops.yaml at the app root; `starciBeConfig({ hfs: loadHfs(import.meta.url) })` is the typed factory and the BE-CONVENTION laws are its rules.'
|
|
39
39
|
'@starci/eslint-canon-fe':
|
|
40
|
-
version: 9.1.
|
|
40
|
+
version: 9.1.2
|
|
41
41
|
group: starci
|
|
42
42
|
install: registry
|
|
43
43
|
side: fe
|
|
44
44
|
source: packages/eslint/fe/package.json
|
|
45
45
|
why: 'the front-end ESLint canon: `loadHfs(import.meta.url)` of fe/eslint.config.mjs finds the app-root hfs.json (kind app) and gives every linted file the view of the fe side, and the project graph is built per side; its bundled runtime copies (slots, canon-pins, failure codes, the architecture machine) put .starcistacks and .sops.yaml at the app root; the front end has no tests (FE_NO_TESTS R97 in hfs) and no-vietnamese-in-source (R91) applies.'
|
|
46
46
|
'@starci/eslint-grammar-starci':
|
|
47
|
-
version: 0.12.
|
|
47
|
+
version: 0.12.6
|
|
48
48
|
group: starci
|
|
49
49
|
install: registry
|
|
50
50
|
side: fe
|
|
51
51
|
source: packages/eslint/grammar-starci/package.json
|
|
52
52
|
why: 'the StarCi Grammar component API lint, versioned with @starci/grammar; front ends compose this family package with the universal FE canon.'
|
|
53
53
|
'@starci/eslint-grammar-offset-pop':
|
|
54
|
-
version: 0.12.
|
|
54
|
+
version: 0.12.6
|
|
55
55
|
group: starci
|
|
56
56
|
install: registry
|
|
57
57
|
side: fe
|
|
@@ -91,14 +91,14 @@ pins:
|
|
|
91
91
|
source: packages/test-world/package.json
|
|
92
92
|
why: 'the shared e2e library of every back end (R47, R48), a devDependency of every back end: the warm stack behind toxiproxy, the network-edge fakes, the Nest boot, the typed useTestWorld handle, useSandbox for contract specs and the outage lock. Kafka is real own infra (the one digest-pinned apache/kafka KRaft image, a broker listener and proxy per slot, slot-prefixed topics, consumer groups and client ids) and Postgres is schema-per-context (a schema and a login role per context in a shared database). Each worker has its own data slot (<ns>_w<k>) with its own databases, Keycloak realm, leased Redis DB, bucket/collection/topic prefixes, fakes host, toxiproxy proxies and outage lock; a run provisions min(jest workers, workers cap, default 2) slots, and state file protocol 2 pairs exactly with @starci/jest-preset (TEST_WORLD_PAIR_MISMATCH otherwise). A modules world boots real peer apps beside its modules, world.apps.<name>.during(fn) is the outage of a peer app, w.keycloak.clientSecret(client) answers the run-generated secret of a confidential realm client, world.keycloak.events/sessions read the realm user events and live sessions, world.infra.postgresql.connection(name) takes one database connection down while the others serve, and world.resolve and scope.resolve take any Nest token. Every path a declaration names (`stack`, seeds, the realm, Dockerfiles) resolves from the app root (the directory of hfs.json); the globalSetup registers the path aliases as TypeScript resolves them. Its CLI main is exported at ./cli for starci app stack.'
|
|
93
93
|
'@starci/cli':
|
|
94
|
-
version: 1.7.
|
|
94
|
+
version: 1.7.39
|
|
95
95
|
group: starci
|
|
96
96
|
install: registry
|
|
97
97
|
side: both
|
|
98
98
|
source: packages/cli/package.json
|
|
99
99
|
why: 'the only StarCi command package; product repositories install it for starci app, while @starci/hfs remains its pinned transitive implementation.'
|
|
100
100
|
'@starci/hfs':
|
|
101
|
-
version: 4.7.
|
|
101
|
+
version: 4.7.39
|
|
102
102
|
group: starci
|
|
103
103
|
install: registry
|
|
104
104
|
side: both
|
|
@@ -4405,7 +4405,7 @@ rules:
|
|
|
4405
4405
|
editionWhy: "owner:lite-backing-services makes Supabase lite's database backing and ruling:R212/case-10 moves the lite queue to Redis; the pgmq retirement and pg_cron ownership requirements apply to that backing."
|
|
4406
4406
|
docs: [docs/microservices.md#r258]
|
|
4407
4407
|
templateNa: "Queue declarations and enqueue-only cron commands depend on product-owned work and are authored in its Supabase migrations."
|
|
4408
|
-
law: "DB_QUEUE_CRON: the queue is BullMQ on Redis (ruling:R212/case-10), so the active migrations leave no pgmq queue named by a queue constant under modules/queues; a pgmq queue a migration created for one is dropped by a later migration. Every pgmq queue the active migrations leave is an event-bus topic queue named by a literal *_TOPIC_QUEUE constant under platform/event-bus, every such constant has its pgmq queue, and pgmq objects are granted only to the application role while they exist. A
|
|
4408
|
+
law: "DB_QUEUE_CRON: the queue is BullMQ on Redis (ruling:R212/case-10), so the active migrations leave no pgmq queue named by a queue constant under modules/queues; a pgmq queue a migration created for one is dropped by a later migration. Every pgmq queue the active migrations leave is an event-bus topic queue named by a literal *_TOPIC_QUEUE constant under platform/event-bus, every such constant has its pgmq queue, and pgmq objects are granted only to the application role while they exist. A pg_cron job belongs to a migration and only enqueues; its name is scheduled once while it is active, so a later migration schedules it again only after its own cron.unschedule of that name, and one tick function has one live job at a time. Its command, directly or through one enqueue-only *_tick function, inserts one row of a literal queue constant into the queue outbox table of ruleParams.db.queueOutbox, writing only its declared columns (the relay hands it to BullMQ). An app whose hfs.json sides.be.backing declares redis may declare its own outbox in sides.be.queueOutbox ({table, enqueue, queueParameter}); the command, or the tick, then calls the declared enqueue function with a literal queue constant for the declared parameter and pure other arguments, while the active definition of that function only inserts one row into the declared table with its queue column bound to that parameter (an ON CONFLICT DO NOTHING, a read of the declared table and pure expressions are the only other work). A direct insert into the app's outbox table, or any other function, stays refused. The job executes no business DML or HTTP request and shares no tick with an app scheduler. A pgmq.send validates only in the original body of a job an older migration scheduled while that queue existed; the live job and its current tick never enqueue that way. Removing a tick unschedules its job."
|
|
4409
4409
|
kinds: [check]
|
|
4410
4410
|
gates: [settle, land, ci]
|
|
4411
4411
|
failureCodes: ["DB_PGMQ_DECLARED", "DB_CRON_OWNED"]
|
|
@@ -4413,6 +4413,7 @@ rules:
|
|
|
4413
4413
|
- {kind: hfs, id: db-queue-cron, at: scripts/hfs/rules/database-queues.mjs}
|
|
4414
4414
|
proves:
|
|
4415
4415
|
- {path: packages/hfs/src/hfs-database-queues.spec.mjs, edition: lite, codes: [DB_PGMQ_DECLARED, DB_CRON_OWNED]}
|
|
4416
|
+
- {path: packages/hfs/src/hfs-queue-outbox-declaration.spec.mjs, edition: lite, codes: [DB_CRON_OWNED]}
|
|
4416
4417
|
exemplifiedBy:
|
|
4417
4418
|
- {path: packages/hfs/src/hfs-database-queues.spec.mjs, edition: lite, codes: [DB_PGMQ_DECLARED, DB_CRON_OWNED]}
|
|
4418
4419
|
witness:
|
|
@@ -43,7 +43,7 @@ versioning:
|
|
|
43
43
|
# journey (enabled with app.browser by `browser: true`).
|
|
44
44
|
# 3.6.0 is a minor (alpha.30): it ADDS the opt-in be.feature.realtime.sse slot, the server-sent-events door of the realtime kind (owner decision
|
|
45
45
|
# 2026-10-24: lite may serve SSE, and SSE is the default door for one-way push), and lets fe.modules.api hold the SSE stream operations (sse/).
|
|
46
|
-
# 3.7.0 is a minor (alpha.31): it ADDS the optional fe.transport.stream slot, the fetch reader of server-sent-events streams (the stream half of the
|
|
46
|
+
# 3.7.0 is a minor (alpha.31): it ADDS the optional app.scripts.integration slot (scripts/integration/*.mjs, the product test-support scripts that only the integration harness runs) and the optional fe.transport.stream slot, the fetch reader of server-sent-events streams (the stream half of the
|
|
47
47
|
# transport client layer), and lets fe.hooks.swr hold the SWR subscription hooks (api/sse/subscriptions/useSubscribe<X>Swr.ts, one useSWRSubscription per stream operation).
|
|
48
48
|
|
|
49
49
|
# Field meaning, per slot.
|
|
@@ -727,6 +727,24 @@ slots:
|
|
|
727
727
|
rules: [HFS_PLAINTEXT_SECRET, HFS_SCRIPT_ONE_OFF, BE_SPEC_PLACEMENT, HFS_REPO_LOCAL_CHECK]
|
|
728
728
|
exemplifiedBy:
|
|
729
729
|
- {path: packages/hfs/src/hfs-slot-ownership.spec.mjs, edition: lite}
|
|
730
|
+
- id: app.scripts.integration
|
|
731
|
+
since: 3.7.0
|
|
732
|
+
profiles: [app]
|
|
733
|
+
editionWhy: "Lite and full share the slot: a product's integration proof that needs a throwaway, unmigrated database has the same need in both editions, and lite's generic scripts path is closed to anything but codegen and delivery tooling."
|
|
734
|
+
# Product test-support scripts, run only by the integration harness (the integration suites call them; no package script, workflow, hook
|
|
735
|
+
# or deploy step does). A proof that must create a throwaway database that no migration has touched and run DDL on it keeps that work here,
|
|
736
|
+
# because the back-end lint refuses runtime DDL and raw query text in test sources. The folder is `integration`, not `test`:
|
|
737
|
+
# R252's script scan skips test folders, and a support script that creates databases must stay inside it. A spec or test is not a script
|
|
738
|
+
# (BE_SPEC_PLACEMENT), a check or lint source has no place (HFS_REPO_LOCAL_CHECK), and a script that reaches a host other than the private
|
|
739
|
+
# test stack is refused by its own connection guard, which the integration spec of the product proves.
|
|
740
|
+
path: "scripts/integration/*.mjs"
|
|
741
|
+
presence: optional
|
|
742
|
+
tracked: tracked
|
|
743
|
+
tier: none
|
|
744
|
+
tests: none
|
|
745
|
+
rules: [HFS_PLAINTEXT_SECRET, HFS_SCRIPT_ONE_OFF, BE_SPEC_PLACEMENT, HFS_REPO_LOCAL_CHECK]
|
|
746
|
+
exemplifiedBy:
|
|
747
|
+
- {path: packages/hfs/src/hfs-slot-ownership.spec.mjs, editions: [lite, full]}
|
|
730
748
|
- id: app.sides
|
|
731
749
|
since: 3.1.0
|
|
732
750
|
profiles: [app]
|
|
@@ -85,7 +85,7 @@ rules:
|
|
|
85
85
|
A recurring task is a job scheduler (`upsertJobScheduler`) that the queue module declares in its `schedulers` list and
|
|
86
86
|
`platform/queue` registers at boot, or, in an app on Supabase, a uniquely named pg_cron job of a migration whose command
|
|
87
87
|
(directly or through one `*_tick` function) only inserts one row of a queue constant into the queue outbox table of
|
|
88
|
-
`ruleParams.db.queueOutbox` (`public.queue`), which the relay hands to BullMQ; a tick has exactly one of the two owners. Its job is processed by the same
|
|
88
|
+
`ruleParams.db.queueOutbox` (`public.queue`), which the relay hands to BullMQ, or by calling the enqueue function that the app declares with its own outbox table in `hfs.json` `sides.be.queueOutbox` (redis declared in `sides.be.backing`; the function's active definition only inserts into that table); a tick has exactly one of the two owners. A job name is scheduled once while it is active: a later migration that replaces a tick's command calls `cron.unschedule` of the name first. Its job is processed by the same
|
|
89
89
|
`features/jobs/<job>/<job>.processor.ts` as any enqueued job. `@nestjs/schedule`, `@Cron`, `@Interval`, `@Timeout` and
|
|
90
90
|
`setInterval` for work are not used anywhere.
|
|
91
91
|
rationale: |-
|
|
@@ -73,6 +73,18 @@ function extraPackageProblems(side, entries) {
|
|
|
73
73
|
return [];
|
|
74
74
|
}
|
|
75
75
|
|
|
76
|
+
/** `sides.be.queueOutbox`: the app's own queue outbox ({table, enqueue, queueParameter}), which a pg_cron tick may enqueue through (R258); it needs the redis backing the relay hands rows to. */
|
|
77
|
+
function queueOutboxProblems(side, value) {
|
|
78
|
+
const declared = value.queueOutbox;
|
|
79
|
+
if (declared === undefined) return [];
|
|
80
|
+
if (side !== 'be') return ['queueOutbox belongs to the be side'];
|
|
81
|
+
const name = (item) => typeof item === 'string' && /^[a-z_][a-z0-9_]*\.[a-z_][a-z0-9_]*$/.test(item);
|
|
82
|
+
const bad = [];
|
|
83
|
+
if (!isPlainObject(declared) || Object.keys(declared).sort().join() !== 'enqueue,queueParameter,table' || !name(declared.table) || !name(declared.enqueue) || !/^[a-z_][a-z0-9_]*$/.test(String(declared.queueParameter))) bad.push('sides.be.queueOutbox must be {table: <schema>.<table>, enqueue: <schema>.<function>, queueParameter: <parameter name>}');
|
|
84
|
+
if (!(value.backing ?? []).includes('redis')) bad.push('sides.be.queueOutbox needs sides.be.backing to declare redis, which the relay hands its rows to');
|
|
85
|
+
return bad;
|
|
86
|
+
}
|
|
87
|
+
|
|
76
88
|
function grammarProblems(side, grammar) {
|
|
77
89
|
if (grammar === undefined) return [];
|
|
78
90
|
if (side !== 'fe') return ['grammar belongs to the fe side'];
|
|
@@ -83,7 +95,7 @@ function declarationSideProblems(side, value, names) {
|
|
|
83
95
|
const bad = [];
|
|
84
96
|
const at = `sides.${side}`;
|
|
85
97
|
if (!isPlainObject(value)) return [`${at} must be an object`];
|
|
86
|
-
for (const key of Object.keys(value)) if (!['apps', 'optionalSlots', 'patterns', 'kinds', 'connections', 'reads', 'extraPackages', 'grammar', 'backing'].includes(key)) bad.push(`${at} has unknown key ${key}`);
|
|
98
|
+
for (const key of Object.keys(value)) if (!['apps', 'optionalSlots', 'patterns', 'kinds', 'connections', 'reads', 'extraPackages', 'grammar', 'backing', 'queueOutbox'].includes(key)) bad.push(`${at} has unknown key ${key}`);
|
|
87
99
|
bad.push(...appShapeProblems(value.apps, at, side, names));
|
|
88
100
|
if (isBadUniqueList(value.optionalSlots, (slot) => SLOT_ID.test(String(slot)))) bad.push(`${at}.optionalSlots must be a unique list of slot ids`);
|
|
89
101
|
if (value.backing !== undefined && side !== 'be') bad.push('backing belongs to the be side');
|
|
@@ -91,7 +103,7 @@ function declarationSideProblems(side, value, names) {
|
|
|
91
103
|
bad.push(...patternShapeProblems(value, at, NAME), ...kindShapeProblems(value, at, NAME));
|
|
92
104
|
if (isBadUniqueList(value.reads, (read) => typeof read === 'string' && read.length > 0)) bad.push(`${at}.reads must be a unique list of paths`);
|
|
93
105
|
if (value.connections !== undefined) bad.push(...connectionProblems(side, value));
|
|
94
|
-
bad.push(...extraPackageProblems(side, value.extraPackages), ...grammarProblems(side, value.grammar));
|
|
106
|
+
bad.push(...extraPackageProblems(side, value.extraPackages), ...grammarProblems(side, value.grammar), ...queueOutboxProblems(side, value));
|
|
95
107
|
return bad;
|
|
96
108
|
}
|
|
97
109
|
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
// R258: an app whose queue is BullMQ on a declared Redis may keep its own outbox beside the canon `ruleParams.db.queueOutbox` table. The
|
|
2
|
+
// app declares it in hfs.json `sides.be.queueOutbox` ({table, enqueue, queueParameter}), and a pg_cron tick enqueues through the declared
|
|
3
|
+
// `enqueue` function only while that function's active definition does nothing but write the declared table with its queue column bound
|
|
4
|
+
// to the declared parameter. The checker reads the declaration and the function body; it names no app table or function.
|
|
5
|
+
import pg from 'libpg-query';
|
|
6
|
+
import { sqlStringParts as strings, walkSqlAst as walk, sqlParser } from '../../lib/sql-ast.mjs';
|
|
7
|
+
|
|
8
|
+
await pg.loadModule();
|
|
9
|
+
|
|
10
|
+
const sqlTree = sqlParser(pg);
|
|
11
|
+
const STATEMENTS = new Set(['PLpgSQL_stmt_block', 'PLpgSQL_stmt_execsql', 'PLpgSQL_stmt_if', 'PLpgSQL_stmt_assign', 'PLpgSQL_stmt_return']);
|
|
12
|
+
const OTHER_DML = ['UpdateStmt', 'DeleteStmt', 'MergeStmt'];
|
|
13
|
+
const qualified = (name) => name.includes('.') ? name : `public.${name}`;
|
|
14
|
+
|
|
15
|
+
/** The outbox an app declares, or null: it counts only while the back end declares the redis backing the relay hands rows to. */
|
|
16
|
+
export function queueOutboxDeclaration(declaration) {
|
|
17
|
+
const be = declaration?.sides?.be;
|
|
18
|
+
const declared = be?.queueOutbox;
|
|
19
|
+
if (!declared || !(be.backing ?? []).includes('redis')) return null;
|
|
20
|
+
return { table: qualified(declared.table), enqueue: qualified(declared.enqueue), parameter: declared.queueParameter };
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
function plpgsqlStatements(fn, body) {
|
|
24
|
+
const parameters = (fn.parameters ?? []).map((item) => item.FunctionParameter?.name).filter(Boolean).map((name) => `"${name.replaceAll('"', '""')}" text`).join(',');
|
|
25
|
+
let tag = '$hfs_outbox$';
|
|
26
|
+
while (body.includes(tag)) tag = `${tag.slice(0, -1)}_$`;
|
|
27
|
+
let tree;
|
|
28
|
+
try { tree = pg.parsePlPgSQLSync(`create function hfs_probe(${parameters}) returns text language plpgsql as ${tag}${body}${tag};`); } catch { return null; }
|
|
29
|
+
const trees = [];
|
|
30
|
+
let safe = true;
|
|
31
|
+
walk(tree, (node) => {
|
|
32
|
+
for (const key of Object.keys(node)) if (key.startsWith('PLpgSQL_stmt_') && !STATEMENTS.has(key)) safe = false;
|
|
33
|
+
const expr = node.PLpgSQL_expr;
|
|
34
|
+
if (expr) trees.push(sqlTree(expr.parseMode === 0 ? expr.query : `select ${expr.query}`));
|
|
35
|
+
});
|
|
36
|
+
return safe ? trees : null;
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
function bodyStatements(fn) {
|
|
40
|
+
const option = (name) => (fn.options ?? []).find((item) => item.DefElem?.defname === name)?.DefElem?.arg;
|
|
41
|
+
const language = option('language')?.String?.sval ?? 'sql';
|
|
42
|
+
const body = strings(option('as')?.List?.items).join('');
|
|
43
|
+
if (language === 'plpgsql') return plpgsqlStatements(fn, body);
|
|
44
|
+
if (language !== 'sql') return null;
|
|
45
|
+
if (fn.sql_body) return (fn.sql_body.List ? fn.sql_body.List.items : [fn.sql_body]).map((stmt) => ({ stmts: [{ stmt }] }));
|
|
46
|
+
return sqlTree(body)?.stmts?.map((entry) => ({ stmts: [entry] })) ?? null;
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
/** True when the nodes hold no sub-select or other statement and call only functions `pureCall` admits. */
|
|
50
|
+
function pureNodes(nodes, pureCall) {
|
|
51
|
+
let ok = true;
|
|
52
|
+
walk(nodes, (node) => {
|
|
53
|
+
if (node.SubLink || node.SelectStmt || node.InsertStmt || OTHER_DML.some((key) => node[key])) ok = false;
|
|
54
|
+
if (node.FuncCall && !pureCall(node.FuncCall)) ok = false;
|
|
55
|
+
});
|
|
56
|
+
return ok;
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
/** `insert into <table> (..., queue, ...) values (..., <parameter>, ...) [on conflict .. do nothing] [returning ..]` with pure other values. */
|
|
60
|
+
function insertOk(declared, insert, pureCall) {
|
|
61
|
+
if (insert.withClause || `${insert.relation?.schemaname || 'public'}.${insert.relation?.relname}` !== declared.table) return false;
|
|
62
|
+
if (insert.onConflictClause && insert.onConflictClause.action !== 'ONCONFLICT_NOTHING') return false;
|
|
63
|
+
const columns = (insert.cols ?? []).map((item) => item.ResTarget?.name);
|
|
64
|
+
const rows = insert.selectStmt?.SelectStmt?.valuesLists ?? [];
|
|
65
|
+
const values = rows.length === 1 ? rows[0].List?.items ?? [] : [];
|
|
66
|
+
const at = columns.indexOf('queue');
|
|
67
|
+
if (at < 0 || new Set(columns).size !== columns.length || values.length !== columns.length) return false;
|
|
68
|
+
const bound = values[at].ColumnRef?.fields;
|
|
69
|
+
if (!bound || strings(bound).join('.') !== declared.parameter || bound.length !== 1) return false;
|
|
70
|
+
return pureNodes(values.filter((_, index) => index !== at), pureCall);
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
/** A read of the declared table with pure expressions, or an expression over variables alone. */
|
|
74
|
+
function readOk(declared, tree, pureCall) {
|
|
75
|
+
let ok = true;
|
|
76
|
+
walk(tree, (node) => {
|
|
77
|
+
if (OTHER_DML.some((key) => node[key]) || node.InsertStmt || node.SelectStmt?.intoClause) ok = false;
|
|
78
|
+
if (node.FuncCall && !pureCall(node.FuncCall)) ok = false;
|
|
79
|
+
if (node.RangeVar && `${node.RangeVar.schemaname || 'public'}.${node.RangeVar.relname}` !== declared.table) ok = false;
|
|
80
|
+
});
|
|
81
|
+
return ok;
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
function statementOk(declared, tree, pureCall) {
|
|
85
|
+
const stmt = tree?.stmts?.length === 1 ? tree.stmts[0].stmt : null;
|
|
86
|
+
if (stmt?.InsertStmt) return insertOk(declared, stmt.InsertStmt, pureCall);
|
|
87
|
+
return Boolean(stmt?.SelectStmt) && readOk(declared, tree, pureCall);
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
/**
|
|
91
|
+
* The position of the queue parameter when the active definition of the declared enqueue function only writes the declared outbox
|
|
92
|
+
* (inserts, reads of that table and pure expressions), or -1. `pureCall` admits a function call that does no work.
|
|
93
|
+
*/
|
|
94
|
+
export function outboxEnqueueQueueIndex(declared, entry, pureCall) {
|
|
95
|
+
const fn = entry?.node;
|
|
96
|
+
if (!fn) return -1;
|
|
97
|
+
const index = (fn.parameters ?? []).findIndex((item) => item.FunctionParameter?.name === declared.parameter);
|
|
98
|
+
const trees = bodyStatements(fn);
|
|
99
|
+
if (index < 0 || !trees?.length || !trees.every((tree) => statementOk(declared, tree, pureCall))) return -1;
|
|
100
|
+
const inserts = trees.filter((tree) => tree?.stmts?.[0]?.stmt?.InsertStmt).length;
|
|
101
|
+
return inserts === 1 ? index : -1;
|
|
102
|
+
}
|
|
@@ -11,10 +11,11 @@ import { queueSourceFacts, queueSchedulers, queueProblem as problem } from './da
|
|
|
11
11
|
import { readJson, readText } from './read.mjs';
|
|
12
12
|
import { DB_CRON_OWNED, DB_PGMQ_DECLARED, isActiveMigration } from './database-constants.mjs';
|
|
13
13
|
import { loadSlotManifest } from '../slots.mjs';
|
|
14
|
+
import { queueOutboxDeclaration, outboxEnqueueQueueIndex } from './database-queue-outbox.mjs';
|
|
14
15
|
|
|
15
16
|
await pg.loadModule();
|
|
16
17
|
|
|
17
|
-
const PURE_CALLS = new Set(['jsonb_build_object', 'json_build_object', 'jsonb_build_array', 'json_build_array', 'to_jsonb', 'now', 'clock_timestamp']);
|
|
18
|
+
const PURE_CALLS = new Set(['jsonb_build_object', 'json_build_object', 'jsonb_build_array', 'json_build_array', 'to_jsonb', 'now', 'clock_timestamp', 'date_trunc']);
|
|
18
19
|
const TABLE_PRIVILEGES = ['select', 'insert', 'update', 'delete', 'truncate', 'references', 'trigger', 'maintain'];
|
|
19
20
|
const sqlName = (nodes) => strings(nodes).join('.');
|
|
20
21
|
const callName = (call) => sqlName(call.funcname);
|
|
@@ -54,9 +55,11 @@ function inspectQueueCall(state, call, file, migration) {
|
|
|
54
55
|
}
|
|
55
56
|
|
|
56
57
|
function recordSchedule(state, call, file) {
|
|
57
|
-
const entry = { call, file, created: new Map(state.created), functions: new Map(state.functions) };
|
|
58
|
-
state.schedules.push(entry);
|
|
59
58
|
const job = sqlLiteral(argument(call, 0, ['job_name']));
|
|
59
|
+
// The names scheduled and not unscheduled since, in migration order: a name is a duplicate only while it is still scheduled.
|
|
60
|
+
const entry = { call, file, created: new Map(state.created), functions: new Map(state.functions), active: new Set(state.scheduledNames) };
|
|
61
|
+
state.schedules.push(entry);
|
|
62
|
+
if (job !== null) state.scheduledNames.add(job);
|
|
60
63
|
const schedule = sqlLiteral(argument(call, 1, ['schedule']));
|
|
61
64
|
const command = sqlLiteral(argument(call, 2, ['command']));
|
|
62
65
|
if (job !== null && schedule !== null && command !== null) {
|
|
@@ -77,6 +80,7 @@ function inspectCronCall(state, call, file, migration) {
|
|
|
77
80
|
if (job !== null) {
|
|
78
81
|
state.unschedules.push({ name: job, file });
|
|
79
82
|
state.activeCron.delete(job);
|
|
83
|
+
state.scheduledNames.delete(job);
|
|
80
84
|
state.liveSchedules.delete(job);
|
|
81
85
|
}
|
|
82
86
|
else problem(state, DB_CRON_OWNED, file, 'cron.unschedule must identify the migration-owned job by literal name');
|
|
@@ -205,12 +209,16 @@ function enqueue(state, call) {
|
|
|
205
209
|
return queue !== null && state.created.has(queue) && safe ? queue : null;
|
|
206
210
|
}
|
|
207
211
|
|
|
212
|
+
function pureCall(call) {
|
|
213
|
+
return uuidConstructor(call) || PURE_CALLS.has(callName(call).replace(/^pg_catalog\./, ''));
|
|
214
|
+
}
|
|
215
|
+
|
|
208
216
|
function pureArguments(call) {
|
|
209
217
|
return pureExpressions(call.args);
|
|
210
218
|
}
|
|
211
219
|
|
|
212
220
|
function pureExpressions(nodes) {
|
|
213
|
-
let safe = callsOf(nodes).every(
|
|
221
|
+
let safe = callsOf(nodes).every(pureCall);
|
|
214
222
|
walk(nodes, (node) => {
|
|
215
223
|
if (['SubLink', 'SelectStmt', 'InsertStmt', 'UpdateStmt', 'DeleteStmt', 'MergeStmt'].some((key) => node[key])) safe = false;
|
|
216
224
|
});
|
|
@@ -223,6 +231,15 @@ function queueOutbox(manifest) {
|
|
|
223
231
|
return declared ? { table: declared.table, columns: new Set(declared.columns) } : null;
|
|
224
232
|
}
|
|
225
233
|
|
|
234
|
+
/** The queue constant a call of the app's declared outbox enqueue function enqueues (hfs.json sides.be.queueOutbox), or null. */
|
|
235
|
+
function declaredEnqueue(state, call) {
|
|
236
|
+
const declared = state.declaredOutbox;
|
|
237
|
+
if (!declared || qualified(callName(call)) !== declared.enqueue) return null;
|
|
238
|
+
const index = outboxEnqueueQueueIndex(declared, state.functions.get(declared.enqueue), pureCall);
|
|
239
|
+
const queue = index < 0 ? null : sqlLiteral(argument(call, index, [declared.parameter]));
|
|
240
|
+
return queue !== null && state.constants.has(queue) && pureArguments(call) ? queue : null;
|
|
241
|
+
}
|
|
242
|
+
|
|
226
243
|
/** The queue constant one outbox row enqueues: `insert into <outbox> (queue, ...) values ('<queue constant>', ...)` with pure values. */
|
|
227
244
|
function outboxEnqueue(state, insert) {
|
|
228
245
|
if (!state.outbox || insert.returningClause || insert.returningList || insert.onConflictClause || insert.withClause) return null;
|
|
@@ -242,7 +259,8 @@ function singleEnqueue(state, tree, legacy) {
|
|
|
242
259
|
const insert = tree?.stmts?.length === 1 ? tree.stmts[0].stmt?.InsertStmt : undefined;
|
|
243
260
|
if (insert) return outboxEnqueue(state, insert);
|
|
244
261
|
const call = singleCall(tree);
|
|
245
|
-
|
|
262
|
+
if (!call) return null;
|
|
263
|
+
return declaredEnqueue(state, call) ?? (legacy ? enqueue(state, call) : null);
|
|
246
264
|
}
|
|
247
265
|
|
|
248
266
|
/** PostgreSQL's catalog-owned scalar UUID constructor adds an envelope identity without doing work in the tick. */
|
|
@@ -311,7 +329,7 @@ function scheduleTarget(state, command, legacy) {
|
|
|
311
329
|
const tree = sqlTree(command);
|
|
312
330
|
const target = singleCall(tree);
|
|
313
331
|
if (!target) return { tick: null, queues: singleEnqueue(state, tree, legacy) };
|
|
314
|
-
if (!callName(target).endsWith('_tick')) return { tick: null, queues: legacy ? enqueue(state, target) : null };
|
|
332
|
+
if (!callName(target).endsWith('_tick')) return { tick: null, queues: declaredEnqueue(state, target) ?? (legacy ? enqueue(state, target) : null) };
|
|
315
333
|
const tick = qualified(callName(target));
|
|
316
334
|
const fn = state.functions.get(tick);
|
|
317
335
|
const queues = fn && pureArguments(target) && tickQueues(state, fn, legacy);
|
|
@@ -325,7 +343,7 @@ function tickRemoval(state, tick, name) {
|
|
|
325
343
|
}
|
|
326
344
|
}
|
|
327
345
|
|
|
328
|
-
function scheduleCheck(state, schedule,
|
|
346
|
+
function scheduleCheck(state, schedule, schedulers, ticks) {
|
|
329
347
|
const { call, file } = schedule;
|
|
330
348
|
const name = sqlLiteral(argument(call, 0, ['job_name']));
|
|
331
349
|
const command = sqlLiteral(argument(call, 2, ['command']));
|
|
@@ -333,22 +351,23 @@ function scheduleCheck(state, schedule, names, schedulers, ticks) {
|
|
|
333
351
|
problem(state, DB_CRON_OWNED, file, 'cron.schedule requires literal job name, cadence and command for migration ownership');
|
|
334
352
|
return;
|
|
335
353
|
}
|
|
336
|
-
if (
|
|
337
|
-
names.add(name);
|
|
354
|
+
if (schedule.active.has(name)) problem(state, DB_CRON_OWNED, file, 'a cron job name is scheduled once while it is active: cron.unschedule it before a later migration schedules it again');
|
|
338
355
|
const historical = { ...state, created: schedule.created, functions: schedule.functions };
|
|
339
356
|
const { tick, queues } = scheduleTarget(historical, command, true);
|
|
340
357
|
if (!queues) problem(state, DB_CRON_OWNED, file, 'a cron command inserts one row of a queue constant into the queue outbox, directly or through one existing *_tick function that only does that; business DML, dynamic SQL and network calls are refused');
|
|
341
358
|
if (state.liveSchedules.get(name) === schedule && !scheduleTarget(state, command, false).queues) problem(state, DB_CRON_OWNED, file, 'the live cron job must insert one outbox row of a queue constant (ruling:R212/case-10): it targets a missing queue or tick, still enqueues with pgmq.send, or its current tick no longer only does that');
|
|
342
|
-
tickOwner(state, tick,
|
|
359
|
+
tickOwner(state, tick, name, schedule, ticks);
|
|
343
360
|
const owners = [name, ...(tick ? [tick, tick.split('.').at(-1)] : []), ...[queues ?? []].flat()];
|
|
344
361
|
if (owners.some((owner) => schedulers.has(owner))) problem(state, DB_CRON_OWNED, file, 'the tick is also declared in an application scheduler list; give it one owner');
|
|
345
362
|
tickRemoval(state, tick, name);
|
|
346
363
|
}
|
|
347
364
|
|
|
348
|
-
function
|
|
365
|
+
/** One tick function has one cron owner at a time: an earlier job of the tick that is still scheduled is a second owner. */
|
|
366
|
+
function tickOwner(state, tick, name, schedule, ticks) {
|
|
349
367
|
if (!tick) return;
|
|
350
|
-
|
|
351
|
-
|
|
368
|
+
const owners = ticks.get(tick) ?? [];
|
|
369
|
+
if (owners.some((owner) => schedule.active.has(owner))) problem(state, DB_CRON_OWNED, schedule.file, 'a tick function has more than one cron owner');
|
|
370
|
+
ticks.set(tick, [...owners, name]);
|
|
352
371
|
}
|
|
353
372
|
|
|
354
373
|
function queueState({ repoRoot, files = [], all = files, repo = {}, manifest }) {
|
|
@@ -356,7 +375,7 @@ function queueState({ repoRoot, files = [], all = files, repo = {}, manifest })
|
|
|
356
375
|
const declaration = readJson(repoRoot, 'hfs.json') ?? repo;
|
|
357
376
|
const connected = ['be', 'fe'].some((side) => (declaration.sides?.[side]?.connections ?? []).some((item) => item.provider === 'supabase'));
|
|
358
377
|
if (repo.edition !== 'lite' && declaration.edition !== 'lite' && !declaration.supabase && !connected && !paths.some((file) => file.startsWith('supabase/'))) return null;
|
|
359
|
-
const state = { repoRoot, appRole: declaration.supabase?.appRole ?? 'app_be', findings: [], outbox: queueOutbox(manifest ?? loadSlotManifest()), constants: new Map(), topics: new Map(), created: new Map(), functions: new Map(), schedules: [], activeCron: new Map(), liveSchedules: new Map(), unschedules: [], drops: [], grants: [], queueAcls: new Map() };
|
|
378
|
+
const state = { repoRoot, appRole: declaration.supabase?.appRole ?? 'app_be', findings: [], outbox: queueOutbox(manifest ?? loadSlotManifest()), declaredOutbox: queueOutboxDeclaration(declaration), scheduledNames: new Set(), constants: new Map(), topics: new Map(), created: new Map(), functions: new Map(), schedules: [], activeCron: new Map(), liveSchedules: new Map(), unschedules: [], drops: [], grants: [], queueAcls: new Map() };
|
|
360
379
|
state.sources = queueSourceFacts(state, paths, inspectCalls);
|
|
361
380
|
migrationFacts(state, paths);
|
|
362
381
|
return state;
|
|
@@ -369,10 +388,9 @@ function validateQueues(state, inScope) {
|
|
|
369
388
|
}
|
|
370
389
|
for (const [topic, file] of state.topics) if (!state.created.has(topic)) problem(state, DB_PGMQ_DECLARED, file, `event-bus topic queue ${topic} has no pgmq.create in an active migration`);
|
|
371
390
|
for (const { node, file, created, executed } of state.grants) grantCheck({ ...state, created: executed ? created : state.created }, node, file);
|
|
372
|
-
const
|
|
373
|
-
const ticks = new Set();
|
|
391
|
+
const ticks = new Map();
|
|
374
392
|
const schedulers = queueSchedulers(state.sources);
|
|
375
|
-
for (const schedule of state.schedules) scheduleCheck(state, schedule,
|
|
393
|
+
for (const schedule of state.schedules) scheduleCheck(state, schedule, schedulers, ticks);
|
|
376
394
|
return state.findings.filter((finding) => inScope(finding.path));
|
|
377
395
|
}
|
|
378
396
|
|
|
@@ -65,9 +65,11 @@ const DATABASE_ERROR = 'be/src/modules/platform/database/errors/database.error.t
|
|
|
65
65
|
/** Skeleton files the lite starter leaves out and the queue capabilities use, each from its skeleton template. */
|
|
66
66
|
const SKELETON_FILES = Object.freeze([
|
|
67
67
|
['be/src/modules/platform/database/database.sql.ts', 'skeleton-lite/src/modules/platform/database/database.sql.ts'],
|
|
68
|
+
['be/src/modules/platform/primitives/sequence.policy.ts', 'skeleton/src/modules/platform/primitives/sequence.policy.ts'],
|
|
69
|
+
['be/src/modules/platform/primitives/sequence.policy.spec.ts', 'skeleton/src/modules/platform/primitives/sequence.policy.spec.ts'],
|
|
68
70
|
]);
|
|
69
71
|
|
|
70
|
-
/** The sql tag of the outbox and claim statements (with its identifier error) and the in-order relay
|
|
72
|
+
/** The sql tag of the outbox and claim statements (with its identifier error) and the in-order relay the queue adapters walk their rows with. */
|
|
71
73
|
function skeletonFiles(repoRoot) {
|
|
72
74
|
const created = SKELETON_FILES.filter(([relative]) => !fs.existsSync(path.join(repoRoot, ...relative.split('/'))))
|
|
73
75
|
.map(([relative, source]) => ({ relative, text: render(template(`be/${source}`), {}) }));
|
package/scaffold/app.mjs
CHANGED
|
@@ -412,6 +412,7 @@ const LITE_DEFERRED_GENERATOR_FILES = new Set([
|
|
|
412
412
|
'src/modules/platform/database/database.sql.ts',
|
|
413
413
|
'src/modules/platform/database/migration-runner.service.ts',
|
|
414
414
|
'src/modules/platform/database/seed-runner.service.ts',
|
|
415
|
+
'src/modules/platform/primitives/sequence.policy.ts',
|
|
415
416
|
]);
|
|
416
417
|
const liteBaseSource = (rel) => !rel.startsWith('apps/cli/')
|
|
417
418
|
&& !rel.startsWith('src/features/cli/')
|
|
@@ -36,9 +36,15 @@ const REGISTRATIONS = Object.freeze({
|
|
|
36
36
|
names: ["InjectPrimaryEntityManager", "PRIMARY_ENTITY_MANAGER"],
|
|
37
37
|
},
|
|
38
38
|
],
|
|
39
|
-
// The queue and jobs capabilities state their outbox and claim statements with the sql tag
|
|
40
|
-
//
|
|
39
|
+
// The queue and jobs capabilities state their outbox and claim statements with the sql tag and walk rows with the in-order relay
|
|
40
|
+
// (the lite starter already carries the record guard the BullMQ client reads stored jobs with).
|
|
41
41
|
queue: [
|
|
42
|
+
{
|
|
43
|
+
file: "be/src/modules/platform/primitives/index.ts",
|
|
44
|
+
source: "./sequence.policy",
|
|
45
|
+
names: ["eachInOrder"],
|
|
46
|
+
beforeSource: "./record.policy",
|
|
47
|
+
},
|
|
42
48
|
{
|
|
43
49
|
file: "be/src/modules/platform/database/index.ts",
|
|
44
50
|
source: "./database.sql",
|
|
@@ -7,7 +7,7 @@ The runtime template tracks no encrypted members. Scaffolding generates the two
|
|
|
7
7
|
|
|
8
8
|
| Command | Operation |
|
|
9
9
|
| --- | --- |
|
|
10
|
-
| prepare | Run `npm run db:start` at the app root; obtain the local Supabase connection and Auth configuration. For a declared Redis
|
|
10
|
+
| prepare | Run `npm run db:start` at the app root; obtain the local Supabase connection and Auth configuration. For a declared Redis (its `redis:create`, `redis:start` and `redis:stop` scripts arrive in package.json with the first queue or job), run the create script once, then the start script when using the native app profile. An owner who chooses local Swarm initializes it, labels its node `{{project}}=dev` and provisions the ingress volumes; the model starts its declared Redis with the app. Replace the reserved development host consistently in the manifest, Caddyfile and public configuration; resolve it to the node and record this in `.starcistacks/dev/infra/swarm/owner-actions.md`. Create the external development secrets named in the model from real sealed members. |
|
|
11
11
|
| doctor | From this directory, run `docker stack config -c infra/swarm/stack.yaml`. Confirm image, secret and local Supabase availability. |
|
|
12
12
|
| up | Run `docker stack deploy -c infra/swarm/stack.yaml {{project}}-dev` from this directory after preparation. |
|
|
13
13
|
| status | Run `docker stack services {{project}}-dev`. |
|
|
@@ -1,17 +1,6 @@
|
|
|
1
1
|
export { createJsonLogger } from "./json-logger.service"
|
|
2
|
-
export {
|
|
3
|
-
|
|
4
|
-
LOGGER,
|
|
5
|
-
CORRELATION_IDS,
|
|
6
|
-
CORRELATION_RUNNER,
|
|
7
|
-
InjectCorrelationIds,
|
|
8
|
-
InjectCorrelationRunner,
|
|
9
|
-
} from "./logging.decorators"
|
|
10
|
-
export { applyCorrelation, correlationMiddleware } from "./correlation-request.policy"
|
|
11
|
-
export type { CorrelationIds, CorrelationRunner } from "./correlation.port"
|
|
12
|
-
export type { CorrelationHeaders, CorrelationRequest } from "./correlation.contracts"
|
|
13
|
-
export { createCorrelationRunner } from "./correlation.service"
|
|
14
|
-
export { isCorrelationId } from "./correlation.policy"
|
|
2
|
+
export { InjectLogger, LOGGER } from "./logging.decorators"
|
|
3
|
+
export { applyCorrelation } from "./correlation-request.policy"
|
|
15
4
|
export { LoggingLogEvent } from "./logging.log-events"
|
|
16
5
|
export { LoggingModule } from "./logging.module"
|
|
17
6
|
export type { Logger } from "./logging.port"
|