@starci/hfs 4.9.11 → 4.10.0

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 4.10.0 - 2026-10-29
+
+Breaking for lite apps (ruling lite-self-hosted-postgres-passport-minio; the migration note is contexts/project/lite.md "Moving an app off Supabase").
+
+- Changed: the lite edition leaves Supabase. `starci app db` runs against a self-hosted PostgreSQL named by `--url-env <NAME>` (default `DATABASE_URL`) through psql and pg_dump (`db/postgres.mjs`): `migrate` applies each `db/migrations/<ts14>_<kebab>.sql` with its history row in one transaction (the `supabase_migrations.schema_migrations` layout, so a restored history carries over), runs `db/bootstrap.sql` first on an empty database and needs no `STARCI_DB_ARMED` on the loopback; `dump` writes roles, schema and data with pg_dumpall and pg_dump; `plan`, `baseline`, `restamp` and `post-cutover` query and repair the history through psql. New `db new <kebab-name>`. Removed: `db reset`, `--linked`, `--local`, `--target-binding` of the db verbs, the bundled Supabase CLI wrappers and the generated database types (`emit/db-types.mjs`, `app check --db-types`, `add --no-types`).
+- Changed: the lite templates. Managed: the db package scripts (`db:create|start|stop|reset` of the pinned `supabase/postgres` image, `db:new`, `db:push`, `db:plan`, `db:dump`), `scripts/codegen.mjs` (GraphQL only), `ci.yml` (no database-types job), `db-deploy.yml` (`starci app db migrate --url-env DATABASE_URL`), `.gitignore`, `.prettierignore` and `db/bootstrap.sql`; `starci app sync --write` deletes a leftover `supabase/types/` of a lite app. Scaffolded: `db/` instead of `supabase/`, a `db` service in both Swarm stacks with its volume, backup and sealed password, `backing.db`, the passport (Google OAuth with state and PKCE, email and password with bcryptjs, own ES256 tokens and JWKS, the refresh cookie) and storage (S3 client, presigned PUT and GET) integrations, the identity guard on the own-token verifier, `platform/database` entities option, `PRIMARY_ENTITIES` and the `callDbFunction`, `advisoryXactLock` and `notify` helpers, and the front end's `modules/auth` over the app's own auth doors (`auth-door.ts`, `token-store.ts`).
+- Changed: the checks. `LITE_DB_MIGRATION` (R332) refuses a `supabase/` tree or SQL outside `db/`; the database rules read `db/migrations` and hfs.json `db` (`forceRls`, `forceRlsExemptions`, `appRole`, `urlEnv`, `image`) and drop the config.toml, browser-access and realtime-publication checks; R276 admits the lite `db` component of image `supabase/postgres`; `backing` entries are `{provider: postgres, component}`; the architecture machine's identity check (`identity.mjs`) holds R329 and R260; SQL table ownership reads the tables the migrations create; hfs.json `sides.be.dbFunctions` joins `ruleParams.be.dbFunctions`.
+- Changed: `starci app add table` in lite writes a TypeORM entity (`persistence/entities/<table>.entity.ts`, `@Entity({ schema: "public", name })`) and the capability's `persistence/entities/index.ts` (`<c>Entities`), exports it from the capability entry and names it in `PRIMARY_ENTITIES` of `platform/database/entities.ts`; the service reads through `requireOwnedEntity` of `platform/database` (the entity's query builder, named parameters) and inserts with `save`. No `<name>.rows.ts` or `<name>.sql.ts` is written (ruling lite-persistence-typeorm-entities). The machine `schema-owner` (R35) checks that lite shape: the entity list in `persistence/entities/index.ts` and its name in `PRIMARY_ENTITIES`.
+- Changed: the lite queue, jobs and webhook generators write no SQL text outside platform/database (R334): their statements are functions of the generated db migrations (job_claim, job_advance, job_complete, job_fail, queue_write, queue_waiting, queue_mark_sent, inbox_claim_event, inbox_claim_client, inbox_save_client_outcome, inbox_sweep_clients, event_bus_write) called through `callDbFunction`, listed with their reasons in the canon `ruleParams.be.dbFunctions`; no `persistence/*.sql.ts` or `*.rows.ts` is written, and the webhook receipt insert is `webhookReceiptInsert(table)` of `platform/database`. The storage integration is `be.integrations.object-storage` (`src/modules/integrations/object-storage/`, `parseObjectStorageConfig`, `ObjectStorageOptions`, options member `objectStorage`). `db/entity-list.mjs` reads the lite entity list at `be/src/modules/platform/database/entities.ts`, where the skeleton keeps it.
+- Changed: the bundled lite pattern knowledge (event-bus, idempotency, jobs, projections, queues, reactors, realtime, saga, webhooks) names db migrations and the local db service instead of Supabase; the command policy points `supabase` at `starci app db` and the db scripts.
+- Changed: the bundled slot manifest 3.9.0 replaces the lite `app.supabase*` slots with `app.db` (`db/migrations`, `db/migrations/archive`, `db/seed.sql`, the managed `db/bootstrap.sql`), `be.integrations.supabase` with `be.integrations.passport` and `be.integrations.object-storage`, removes `fe.route.callback`, sets lite `schemaAuthority: sql`, gives lite TypeORM entities in `be.persistence`, adds `ruleParams.be.dbFunctions`, and moves the S3 SDKs to `integrations/object-storage` and the identity libraries to `integrations/passport` in `infraOwners` (`@supabase/*`, `minio` and `bcrypt` nowhere); the bundled rule catalog retires R216, R217, R219, R254 and R287 and adds R328-R334; the bundled canon pins name hfs 4.10.0, eslint-canon-be 3.7.0, eslint-canon-fe 9.5.0 and the Grammar family 0.15.11, pin `@nestjs/passport` 11.0.5 (CommonJS on the Nest 11 line), `passport` 0.7.0, `passport-google-oauth20` 2.0.0, `passport-local` 1.0.0, `bcryptjs` 3.0.3 and their types (tokens are signed and verified with `jose`; `@nestjs/jwt` stays allowed in `integrations/passport` but unpinned), and drop `@supabase/ssr`, `@supabase/supabase-js` and the `supabase` CLI.
+
+
 ## 4.9.11 - 2026-10-29
 
 - Added: `starci app db drift [--url-env <NAME>] [--json]` (`db/drift.mjs`, owner ruling 2026-10-29: lite tables are mapped by TypeORM entities while `db/migrations/*.sql` stays the only DDL and `synchronize` is false). It builds TypeORM's metadata from `be/src/platform/database/entities.ts` with the app's own typescript, typeorm and pg (a `DataSource` that never connects), applies `db/bootstrap.sql` and every active migration to a scratch container of the app's `components.db` image (or the empty database `--url-env` names), and reports every missing table or column, type mismatch and nullability mismatch as `BE_ENTITY_SCHEMA_DRIFT` (R333 in the bundled rule catalog, with its failure-code entry). No Docker, no app packages or an unreadable target is no verdict (`DB_DRIFT_NEEDS_DATABASE`, exit 2), never a green; a migration that does not apply to an empty database is `DB_DRIFT_MIGRATION_FAILED`. An app with an entity list gets a `db-drift` row in its release gate (default or declared gates) and `npx starci app db drift` after `npm run db:lint` in its managed CI's build job. Specs: `src/db-drift.spec.mjs` (real TypeORM metadata over a fixture app, stubbed schema) and `src/db-drift-docker.spec.mjs` (a real postgres:17-alpine scratch database where Docker runs, else the no-verdict answer); typeorm 0.3.31 and pg 8.23.1 are dev dependencies for them.

package/README.md

@@ -55,9 +55,9 @@ Lite is the smaller edition of the same HFS standard, not another profile or eng
 the same slot manifest, rule catalog, lint canons, grammar, security rules and Docker rules apply, while edition fields filter
 out the test world and event machinery.
 
-`starci app scaffold <name> --edition lite` creates the minimum single-api, single-Next-app Supabase product. It generates the
-`app.supabase.types` artifact from the app's local Supabase stack; if that generation cannot run, scaffolding fails and removes
-the app it began instead of leaving a partial tree.
+`starci app scaffold <name> --edition lite` creates the minimum single-api, single-Next-app product on its own PostgreSQL (`db/`,
+the `db` service of its stack), its Passport identity and its S3 storage; a failed step removes the app it began instead of leaving a
+partial tree. `starci app db new|plan|migrate|dump|post-cutover --url-env <NAME>` drives that database (default `DATABASE_URL`).
 
 The lite `app.starciwork` slot is optional when the app has no brand records. When present, it holds only `.gitignore`,
 optional `workspace.yaml` metadata and `brand/**` (R212): the existing brand record, drawing records, references and assets.
@@ -67,22 +67,20 @@ and `workspace.yaml` and retains `brand/` only when present.
 
 The lite additions are deliberately narrow:
 
-- `starci app add table <name> [--fe] [--no-types]` adds the migration and matching back-end capability. `--fe` also adds the table's
+- `starci app add table <name> [--fe]` adds the migration and matching back-end capability. `--fe` also adds the table's
   api feature over GraphQL (get-<singular> query, create-<singular> mutation, the GraphQL server capability on first use) and, in
   every product Next app, one operation file per operation under `modules/api/graphql` and its `useQuery*Swr` / `useMutate*Swr`
-  hook under `hooks/swr/api/graphql`; never a Server Action or a Supabase reader. Types regenerate by default, while `--no-types` is
-  the explicit opt-out.
+  hook under `hooks/swr/api/graphql`; never a Server Action or a database reader.
 - `starci app add cli <group>` creates the optional cli app on first use, including its built-in migration and seed groups, then adds
   the requested group without full-edition specs.
 - `starci app add app <name>` adds another Next app in either edition: the edition's app skeleton plus the shared app core
   (modules/api, auth, toast, redux, the providers and the hooks), one fe structure for lite and full (owner ruling 2026-10-13).
 - An `add` or `new` whose target slot is unavailable in lite stops before writing with `<verb> <noun>: full edition only`.
 
-`starci app emit` continues to emit the enabled back-end snapshots and also writes `supabase/types/database.types.ts` when
-the `app.supabase.types` slot is enabled. The local Supabase stack must be running for generation.
+`starci app emit` emits the enabled back-end contract snapshots; a lite app generates no database types.
 
 `starci app db drift` holds a lite app's TypeORM entities to the schema its migrations build (R333, `BE_ENTITY_SCHEMA_DRIFT`; `db/drift.mjs`).
-It loads the entity list `be/src/platform/database/entities.ts` with the app's own `typescript`, `typeorm` and `pg` (its `be/tsconfig.json`
+It loads the entity list `be/src/modules/platform/database/entities.ts` with the app's own `typescript`, `typeorm` and `pg` (its `be/tsconfig.json`
 path aliases resolved) and builds TypeORM's metadata through a postgres `DataSource` with `synchronize` false that never connects. It applies
 `db/bootstrap.sql`, when present, and every active `db/migrations/*.sql`, each file in one transaction, to a scratch database: a throwaway
 container of the image `.starcistacks/application-stacks.yaml` declares for `components.db`, or with `--url-env <NAME>` the database that

package/db/backup-receipt.mjs

@@ -1,11 +1,10 @@
-// backup-receipt.mjs - encrypted database backup custody, shared by dump and reset; tool output never enters diagnostics.
+// backup-receipt.mjs - encrypted database backup custody of `starci app db dump`; tool output never enters diagnostics.
 import fs from 'node:fs';
 import path from 'node:path';
 import { createHash } from 'node:crypto';
 import { parseYaml } from '../runtime/engine/yaml.mjs';
-import { readDeclaration } from '../release/index.mjs';
-import { resolveTarget, runCli as migrationCli, targetArgs } from './migrate.mjs';
 import { failure, refusal } from '../stack/errors.mjs';
+import { DEFAULT_DB_URL_ENV, dbTarget, pgTool, queryRows as psqlRows } from './postgres.mjs';
 
 export const RECEIPT_SCHEMA = 'starci/db-backup@1';
 export function sha256(file) {
@@ -20,10 +19,9 @@ export function sha256(file) {
 }
 const inside = (root, file) => { const rel = path.relative(root, file); return rel === '' || (!rel.startsWith(`..${path.sep}`) && rel !== '..' && !path.isAbsolute(rel)); };
 
-/** Backup/restore SQL can hold row values, so even the migration runner's scrubbed stderr must be withheld. */
-export function runCli(session, args) {
-  try { return migrationCli(session, args); }
-  catch { throw failure('DB_BACKUP_CLI_FAILED', `supabase ${args.slice(0, 2).join(' ')} failed; output withheld`); }
+/** Backup SQL can hold row values, so a client tool's stderr is withheld whole. */
+export function runPgTool(session, command, args) {
+  return pgTool(session.ctx, session.target, command, args, { withhold: true, code: 'DB_BACKUP_CLI_FAILED' });
 }
 
 /** Resolves existing ancestors before checking containment, including symlinked output directories and Git worktree marker files. */
@@ -40,27 +38,17 @@ export function outsideRepository(root, output) {
   return resolved;
 }
 
-/** Uses migrate's linked/password/URL resolution; local is explicitly selected and never silently defaults to linked. */
+/** The database of `--url-env` (default DATABASE_URL) and the project name the archive carries. */
 export function backupSession(ctx, options) {
-  const database = readDeclaration(ctx.cwd).database ?? {};
-  const selections = [options.linked === true, options.local === true, options.urlEnv !== undefined].filter(Boolean).length;
-  if (selections !== 1) throw refusal('DB_USAGE', 'select exactly one of --linked, --local or --url-env');
-  const target = options.local ? { kind: 'local', ref: null, env: {}, secrets: [], how: 'local database' }
-    : resolveTarget(ctx, { projectRef: null, password: null, ...database }, options);
-  if (target.kind === 'linked' && !/^[a-z0-9]{10,40}$/u.test(target.ref)) throw refusal('DB_NOT_LINKED', 'linked project ref is invalid');
+  const target = dbTarget(ctx, options.urlEnv ?? DEFAULT_DB_URL_ENV);
   const project = path.basename(ctx.cwd).replace(/[^A-Za-z0-9_-]/gu, '-') || 'database';
-  return { ctx, target, database, project };
+  return { ctx, target, project };
 }
 
-export const backupArgs = (session) => session.target.kind === 'local' ? ['--local', '--workdir', session.ctx.cwd] : targetArgs(session.target, session.ctx.cwd);
-
-/** Queries via the same injected Supabase runner; malformed output is withheld. */
+/** One read-only query through psql; failures and malformed output are withheld. */
 export function queryRows(session, sql) {
-  let rows;
-  try { rows = JSON.parse(runCli(session, ['db', 'query', sql, '--output', 'json', '--agent', 'no', ...backupArgs(session)]).stdout); }
+  try { return psqlRows(session.ctx, session.target, sql, { withhold: true }); }
   catch { throw failure('DB_BACKUP_QUERY_FAILED', 'database inventory query failed; output withheld'); }
-  if (!Array.isArray(rows) || rows.some((row) => !row || typeof row !== 'object' || Array.isArray(row))) throw failure('DB_BACKUP_QUERY_FAILED', 'database inventory is not a rows array');
-  return rows;
 }
 
 /** Executes local tar/age through ctx.spawn, the same seam used by the stack's age installer; never relays tool errors containing data. */
@@ -102,28 +90,3 @@ export function encryptFile(ctx, input, output, recipients) {
   return checkArchive(output);
 }
 
-/** Validates a complete, recent, full-data receipt and its encrypted bytes before any destructive reset call. */
-export function validateReceipt(session, receiptPath) {
-  if (!receiptPath) throw refusal('DB_RESET_BACKUP_MISSING', '--backup must name an encrypted backup receipt');
-  let receipt;
-  try { receipt = JSON.parse(fs.readFileSync(receiptPath, 'utf8')); } catch { throw refusal('DB_RESET_BACKUP_INVALID', 'backup receipt is missing or invalid JSON'); }
-  const valid = receipt && receipt.schema === RECEIPT_SCHEMA && receipt.target === 'linked' && receipt.schemaOnly === false
-    && typeof receipt.project === 'string' && typeof receipt.ref === 'string' && typeof receipt.file === 'string'
-    && path.basename(receipt.file) === receipt.file && receipt.file.endsWith('.age') && Number.isSafeInteger(receipt.bytes) && receipt.bytes >= 100
-    && /^[a-f0-9]{64}$/u.test(receipt.sha256 ?? '') && Array.isArray(receipt.parts)
-    && ['roles.sql', 'schema.sql', 'auth-storage-schema.sql', 'data.sql', 'auth-storage-data.sql', 'rowcounts.json'].every((part) => receipt.parts.includes(part))
-    && Number.isSafeInteger(receipt.rowcounts?.tables) && receipt.rowcounts.tables >= 0 && /^\d+$/u.test(receipt.rowcounts?.rows ?? '')
-    && typeof receipt.at === 'string' && Number.isFinite(Date.parse(receipt.at));
-  if (!valid) throw refusal('DB_RESET_BACKUP_INVALID', 'backup receipt is not a full starci/db-backup@1 linked backup');
-  if (receipt.ref !== session.target.ref) throw refusal('DB_RESET_BACKUP_PROJECT', 'backup targets a different project ref');
-  const maxAge = session.database.reset?.backupMaxAgeHours ?? 24;
-  if (!Number.isFinite(maxAge) || maxAge <= 0) throw refusal('DB_CONFIG_INVALID', 'database.reset.backupMaxAgeHours must be positive');
-  const age = session.ctx.now() - Date.parse(receipt.at);
-  if (age < 0 || age > maxAge * 3600000) throw refusal('DB_RESET_BACKUP_STALE', 'backup is future-dated or older than the allowed backup age');
-  const file = path.resolve(path.dirname(receiptPath), receipt.file);
-  if (!fs.existsSync(file)) throw refusal('DB_RESET_BACKUP_ARCHIVE_MISSING', 'backup archive is absent');
-  try {
-    if (checkArchive(file) !== receipt.bytes || sha256(file) !== receipt.sha256) throw new Error('mismatch');
-  } catch { throw refusal('DB_RESET_BACKUP_HASH', 'backup archive size, header or sha256 differs from the receipt'); }
-  return receipt;
-}

package/db/drift.mjs

@@ -1,7 +1,7 @@
 // drift.mjs - `starci app db drift`: the TypeORM entities of an app against the schema its SQL migrations build.
 // The migrations (db/migrations/*.sql, after db/bootstrap.sql when present) are the only DDL and `synchronize` is false, so an entity only
 // maps tables that already exist and can drift from them. The verb builds TypeORM's metadata from the app's entity list
-// (be/src/platform/database/entities.ts, compiled in-process with the app's own typescript and loaded with the app's own typeorm) through a
+// (be/src/modules/platform/database/entities.ts, compiled in-process with the app's own typescript and loaded with the app's own typeorm) through a
 // DataSource that never connects, applies the migrations to a scratch database (a throwaway container of the image the app's
 // .starcistacks declares for its db component, or the empty database named by --url-env), reads that schema from
 // information_schema.columns and pg_catalog, and reports every entity column whose table or column is missing, whose type differs (both

package/db/dump.mjs

@@ -1,17 +1,18 @@
-// dump.mjs - full Supabase backup, encrypted outside Git; every plaintext part is removed even when a tool fails.
+// dump.mjs - `starci app db dump`: a full backup of the database --url-env names (pg_dumpall roles, pg_dump schema and data),
+// encrypted outside Git; every plaintext part is removed even when a tool fails.
 import fs from 'node:fs';
 import path from 'node:path';
 import { runVerb } from '../release/index.mjs';
 import { failure, refusal } from '../stack/errors.mjs';
-import { RECEIPT_SCHEMA, outsideRepository, backupSession, backupArgs, backupRecipients, backupTool, encryptFile, queryRows, sha256, runCli } from './backup-receipt.mjs';
+import { RECEIPT_SCHEMA, outsideRepository, backupSession, backupRecipients, backupTool, encryptFile, queryRows, sha256, runPgTool } from './backup-receipt.mjs';
 
-/** Counts every ordinary table with quoted identifiers in a read-only transaction, including auth and storage. */
+/** Counts every ordinary table with quoted identifiers, every schema but the catalogs (auth included). */
 export function rowcounts(session) {
   const tables = queryRows(session, "select schemaname, tablename from pg_catalog.pg_tables where schemaname not in ('pg_catalog', 'information_schema') order by schemaname, tablename;");
   const quote = (name) => `"${String(name).replaceAll('"', '""')}"`;
   return tables.map((table) => {
     if (typeof table.schemaname !== 'string' || typeof table.tablename !== 'string') throw failure('DB_BACKUP_QUERY_FAILED', 'invalid table inventory');
-    const rows = queryRows(session, `begin read only; select count(*)::text as count from ${quote(table.schemaname)}.${quote(table.tablename)}; commit;`);
+    const rows = queryRows(session, `select count(*)::text as count from ${quote(table.schemaname)}.${quote(table.tablename)}`);
     if (rows.length !== 1 || !/^\d+$/u.test(String(rows[0].count))) throw failure('DB_BACKUP_QUERY_FAILED', 'invalid rowcount');
     return { schema: table.schemaname, table: table.tablename, count: String(rows[0].count) };
   });
@@ -48,15 +49,15 @@ export function dumpDatabase(ctx, options = {}) {
   let complete = false;
   try {
     fs.mkdirSync(scratch, { mode: 0o700 });
-    const jobs = [['roles.sql', ['--role-only']], ['schema.sql', []], ['auth-storage-schema.sql', ['--schema', 'auth,storage']]];
-    if (!options.schemaOnly) jobs.push(['data.sql', ['--data-only']], ['auth-storage-data.sql', ['--data-only', '--schema', 'auth,storage']]);
-    for (const [file, flags] of jobs) runCli(session, ['db', 'dump', ...flags, '--file', path.join(scratch, file), ...backupArgs(session)]);
+    const jobs = [['roles.sql', 'pg_dumpall', ['--roles-only', '--no-role-passwords']], ['schema.sql', 'pg_dump', ['--schema-only']]];
+    if (!options.schemaOnly) jobs.push(['data.sql', 'pg_dump', ['--data-only']]);
+    for (const [file, command, flags] of jobs) runPgTool(session, command, [...flags, '--file', path.join(scratch, file), '--dbname', session.target.url]);
     const counts = rowcounts(session);
     fs.writeFileSync(path.join(scratch, 'rowcounts.json'), `${JSON.stringify(counts, null, 2)}\n`, { mode: 0o600 });
     const parts = [...jobs.map(([file]) => file), 'rowcounts.json'];
     const tar = archiveParts(ctx, scratch, parts);
     const bytes = encryptFile(ctx, tar, archive, recipients);
-    const receipt = { schema: RECEIPT_SCHEMA, project: session.project, target: session.target.kind, ref: session.target.ref ?? null,
+    const receipt = { schema: RECEIPT_SCHEMA, project: session.project, target: session.target.kind, ref: session.target.urlEnv,
       at, file: path.basename(archive), bytes, sha256: sha256(archive), parts,
       rowcounts: { tables: counts.length, rows: counts.reduce((sum, row) => sum + BigInt(row.count), 0n).toString(), tablesByName: counts }, schemaOnly: options.schemaOnly === true };
     fs.writeFileSync(receiptPath, `${JSON.stringify(receipt, null, 2)}\n`, { mode: 0o600 });
@@ -70,9 +71,9 @@ export function dumpDatabase(ctx, options = {}) {
 
 export function dbDumpMain(argv, io = {}) {
   return runVerb({ label: 'starci app db dump', name: 'db dump', code: 'DB_USAGE', io, argv,
-    spec: { boolean: ['--linked', '--local', '--schema-only', '--json'], value: ['--url-env', '--out'] },
+    spec: { boolean: ['--schema-only', '--json'], value: ['--url-env', '--out'] },
     run: (ctx, flags, positional) => {
       if (positional.length) throw refusal('DB_USAGE', 'dump takes flags only');
-      return dumpDatabase(ctx, { linked: flags.linked, local: flags.local, urlEnv: flags['url-env'], out: flags.out, schemaOnly: flags['schema-only'] });
+      return dumpDatabase(ctx, { urlEnv: flags['url-env'], out: flags.out, schemaOnly: flags['schema-only'] });
     } });
 }

package/db/entity-list.mjs

@@ -3,7 +3,7 @@ import fs from 'node:fs';
 import path from 'node:path';
 
 /** The app's one entity list: it re-exports the arrays each capability's persistence/entities/index.ts exports. */
-export const ENTITY_LIST = 'be/src/platform/database/entities.ts';
+export const ENTITY_LIST = 'be/src/modules/platform/database/entities.ts';
 
 /** True when the app at `root` keeps an entity list: the apps whose entities `starci app db drift` checks and whose release gate runs it. */
 export const hasEntityList = (root) => fs.existsSync(path.join(root, ENTITY_LIST));