fastify-auto-sqlapi 0.2.0 → 0.2.2

package/BREAKING_CHANGES.md

@@ -455,3 +455,135 @@ returns a `400` naming the reason.
 
 Use `joinMustExist` on the same relation when you need an extra filter on the parent — that path
 delegates to the target's own `filters()` and does run `extendedCondition`.
+
+---
+
+# Breaking Change — the tenant follows the request, not the table it addresses
+
+`tenantScope` on a table is now enforced even when the request was addressed to a **different**
+table that reaches it through a relation. Previously the tenant was resolved from the addressed
+table alone: if that table declared no `tenantScope`, no predicate was applied anywhere — not to
+it, and not to the scoped tables it joined or wrote into.
+
+## Why this changed
+
+A relation in `allowedReadJoins` is a read grant on its target, and one in `allowedWriteJoins` is
+a write grant (see [ADR 0010](./docs/adr/0010-joins-do-not-run-route-guards.md)). `tenantScope`
+was documented as the one request-derived protection that crosses a join — the cap that makes a
+relation safe to declare. It did not cross when the *host* was open:
+
+```typescript
+// shift is public on purpose: everyone may see who works when.
+shift: {
+  ...exportTableInfo(SchemaShift),
+  allowedReadJoins: [buildRelation(SchemaShift, 'id', SchemaSwapRequest, 'shiftId', { alias: 'swap' })],
+},
+// …but why somebody wants to change the roster is not public.
+shift_swap_request: {
+  ...exportTableInfo(SchemaSwapRequest),
+  tenantScope: { column: 'agent_id' },
+},
+```
+
+```sql
+-- POST /search/shift  { "joinMultiple": { "swap": {} } }   — before
+SELECT … FROM shift_swap_request WHERE (shift_id IN ($1))
+-- after
+SELECT … FROM shift_swap_request WHERE (shift_id IN ($1) AND (agent_id IN ($2)))
+```
+
+Every scoped row of the neighbour was readable through the open host, and writable through it as
+a secondary. The declaration said "scoped"; the deployment was not. Nothing in the response said
+so, which is the dangerous direction — the same reasoning as unknown filter keys above.
+
+## What changed
+
+- `TenantContext.scope` is now **optional**. `ids` is request-level and travels with the request;
+  `scope` is table-level and absent when the addressed table declares none. Absent `scope` means
+  "no filter on *this* table", never "no tenant" — related tables still enforce their own.
+- `getTenantId` is now called for a request addressed to an unscoped table **when one of that
+  table's declared relations points at a scoped table**. It is still not called when nothing
+  scoped is in reach, so genuinely tenant-free routes are unaffected. Reach is one level: joins
+  and secondaries resolve from the addressed table's own relations and never chain further.
+
+## Who is affected
+
+- **You declare `tenantScope` on a table that another table joins or writes into, where that
+  other table has no scope of its own.** Those requests now return fewer rows and can now answer
+  `403`. This is the fix: the rows were never yours to serve. Check any UI that displayed them.
+- **Your `getTenantId` assumes an authenticated request.** It now runs on routes that previously
+  skipped it — a public host with a scoped neighbour. A body like
+  `(request) => request.user.organizationId` throws there, turning a `200` into a `500`.
+- **You read `tenant.scope` in TypeScript** (custom routes using the exported `TenantContext`).
+  Its type is now `TenantScope | undefined`.
+
+Not affected: tables with no `tenantScope` anywhere in reach, deployments without `getTenantId`,
+admins (`getTenantId` → `null`), and every table that already declared its own scope — their SQL
+is unchanged.
+
+## Migration
+
+Make `getTenantId` total. It already had to return `null` for admins, so the null case exists:
+
+```typescript
+// Before — throws on a route that never had a user
+getTenantId: (request) => request.user.organizationId,
+
+// After — an absent user is simply an unscoped caller
+getTenantId: (request) => request.user?.organizationId ?? null,
+```
+
+Then decide, for each open table that relates to a scoped one, whether the relation was meant to
+be a read grant at all. If it was not, remove it from `allowedReadJoins` — a relation is the
+grant, and the scope is only the cap on it.
+
+---
+
+# Breaking Change — secondaries run the child table's `beforeInsert`
+
+## Why this changed
+
+`processSecondaries` consulted the child table's configuration for its `excludeFromCreation` and
+its `tenantScope`, but skipped its `beforeInsert`. A child table whose hook *transformed* a value
+rather than adding one — encrypting a column, normalising a unit — therefore stored the raw
+client value whenever its rows arrived as another table's secondaries, and correctly only through
+its own route. Nothing failed at write time; the row read back as garbage later. See
+[ADR 0014](./docs/adr/0014-what-follows-a-write-join.md).
+
+## What changed
+
+- A child table's **`beforeInsert` now runs** for rows written as `secondaries`, after that
+  table's `excludeFromCreation` strip and before the engine's FK auto-fill (which stays
+  authoritative). The child's `validate` and `afterInsert` still do not run.
+- The **get response schema honours `readExclude`**: it previously advertised fields the engine
+  never selects. `schemaOverrides` is unchanged — still write bodies only.
+- The **update body now requires the primary key**. It identifies the row (`PUT /rest/:table`
+  carries it in `main`), but the generated PK field is `Optional` and that modifier reached the
+  update body unchanged, so a request omitting `main.id` built its `WHERE` on `undefined`
+  instead of being rejected with a 400.
+
+## Who is affected
+
+- **A table of yours declares `beforeInsert` and is also a write-join target.** The hook now runs
+  on that path. If it was written assuming the table's own route — reading `req.params`, counting
+  invocations, calling an external service per record — it now runs where it did not before. It
+  receives the transaction connection, so what it writes rolls back with the host's write.
+- **You send an update without `main.<pk>`.** It never identified a row — the `WHERE` was built
+  on `undefined` — and is now a 400.
+- **A client of yours read a `readExclude`d field out of the get route's Swagger.** It was never
+  returned; only the documentation claimed it.
+
+Not affected: tables with no `beforeInsert`, or none that are write-join targets. Nothing about
+`schemaOverrides` changes.
+
+## Migration
+
+For a `beforeInsert` that must keep running on the table's own route only, guard on the context
+it needs rather than on the path:
+
+```typescript
+beforeInsert: async (db, req, record) => {
+  record.slug = slugify(record.name);           // shaping: correct on both paths, leave it
+  if (req.routeOptions?.url?.startsWith('/rest/')) await notify(record);  // side effect: narrow it
+},
+```

package/CHANGELOG.md

@@ -9,8 +9,128 @@ Migration instructions for breaking changes live in **[BREAKING_CHANGES.md](./BR
 
 ## [Unreleased]
 
+### Added
+
+- **`ApiRequest` — one seam between the core and the HTTP framework.** The engines, the schema
+  builders and the write pipeline never read a property off the request: it arrives from the
+  route, is forwarded to `getTenantId`, `validate` and the hooks, and is otherwise untouched. That
+  dependency was spread over thirteen files as a `FastifyRequest` type import; it is now declared
+  once in `src/types/request.ts` and exported from the package. `ApiRequest` *is* `FastifyRequest`,
+  so nothing changes for consumers — hooks keep their exact types. A new dependency-cruiser rule
+  fails the build on a `fastify` import (type-only included) anywhere under `src/lib/`,
+  `src/types/` or `src/bin/` but that file, and the two Fastify bridge helpers that lived in
+  `lib/` (`setupSwagger`, the `sqlApi` decorator) moved to `src/routes/` so the rule needs no
+  exceptions. See [ADR 0018](./docs/adr/0018-one-seam-to-the-http-framework.md).
+
+- **Views are supported explicitly, and PostgreSQL materialized views at all.** A view was
+  already usable — both introspections read `information_schema.columns`, which lists view
+  columns like any other — but the generator treated it as a base table and **invented a
+  primary key from the column types**: on an aggregating view that takes a count as the key,
+  and `get/:id`, `update` and `delete` then address rows by it, silently. Introspection now
+  reports `is_view` on both engines, the generated Schema carries `isView`, and the table
+  generator refuses to guess: it uses `id` when the view has one and says the key was *inferred,
+  not read from the database*, and otherwise writes a `TODO_pick_a_unique_column` placeholder
+  with `operations: ['search']` and a real `defaultOrder` — a config that works as generated,
+  since search never reads the primary key. Writes start off in the template but are **not**
+  blocked at runtime: a view the engine can make updatable is a legitimate projection layer.
+  PostgreSQL **materialized views** are now introspected from `pg_catalog`; they are absent from
+  `information_schema` and were being skipped without a word. `sqlapi-generate-schema` prints
+  the views it found and the assumptions it had to make. See
+  [ADR 0016](./docs/adr/0016-a-view-is-a-table-config.md).
+
+- **Columns the database computes are no longer offered as writable.** A
+  `GENERATED ALWAYS AS (<expr>)` column — PostgreSQL STORED, MySQL VIRTUAL or STORED — is
+  rejected by both engines when a write so much as names it, and the whole statement fails.
+  The plugin used to advertise those columns as ordinary writable fields, so a client sending
+  back what a read had just returned got a `500` for a payload nobody could have made valid.
+  `sqlapi-generate-schema` now detects them and records them in the generated Schema's
+  `generatedFields`; the runtime drops them from the insert, update and bulk-upsert bodies
+  (main and secondaries) and again inside the engines, which `sqlApi.*` reaches without those
+  schemas. **Regenerating the Schema files is enough** — the exclusion does not live in the
+  `Table*.ts` the generator never overwrites. The CLI also prints the columns it found, per
+  table, since a generated source file is not always read. See
+  [ADR 0015](./docs/adr/0015-non-writable-columns.md).
+- **`writeExclude` on `defineTable`** — the write-side counterpart of `readExclude`, for the
+  non-writable columns introspection cannot know about (one a trigger owns, one a migration is
+  about to drop). Reads are untouched: the field stays projected, filterable and orderable. It
+  runs **after** the write hooks, unlike `excludeFromCreation` which is sanitized before them
+  (ADR 0005): a hook cannot put back a value the database will refuse. `defineTable` rejects an
+  unknown field, the primary key, and a field that is also `readExclude`d. It is static and
+  applies to everyone — a rule that depends on *who* is asking is still product logic (ADR 0004).
+- **The agent manifest marks non-writable fields** `readOnly`, alongside the existing
+  `writeOnly` for `readExclude`.
+
+- **`afterRead` — the read-side hook.** The counterpart of `beforeInsert`/`beforeUpdate`, for
+  turning a stored representation back into the API one: decrypting a column, unpacking a blob,
+  rescaling a unit. Called **once per result set** with camelCase rows mutated **in place** and
+  awaited, so a hook that calls a remote KMS gets one invocation per page instead of one per row.
+  Declared on the table that **owns** the column and it follows a join: the hook runs wherever
+  that table's rows surface — `get`, `search`, and a `joinMultiple`/`joinLeft` declared on
+  another table, with `ctx.source` and `ctx.alias` saying which — so a relation elsewhere needs
+  no extra wiring, the way `readExclude` and `tenantScope` already work (ADR 0010). It does not
+  run on `joinGroup` (aggregates, not table rows) nor on an empty result, and rows cannot be
+  added or removed: the pagination `COUNT` has already run. On the main table it runs *after* the
+  join side queries, which correlate on the values the database returned, so a hook rewriting the
+  column a relation joins on cannot break the correlation. `SearchParams`/`GetParams` now carry
+  the optional `request`, which `sqlApi.search()`/`sqlApi.get()` forward when given one.
+
+- **`tenantScope` over several owner columns — `{ anyOf: [...] }`.** A row that belongs to two
+  parties and is visible to either — a message (`sender_id` / `recipient_id`), a transfer
+  (`from_account_id` / `to_account_id`), a shift swap — could not be expressed before, leaving
+  the table open or the route hand-written. Reads add
+  `AND (colA IN (…) OR colB IN (…))`, alias-qualified, and the predicate crosses every join
+  family including `joinLeft`, where it lands in the `ON` clause so `LEFT` semantics survive.
+  A `NULL` column does not match and does not stop the other one from matching.
+  Writes cannot auto-inject an owner, so the payload **anchors** the row instead: at least one
+  listed column present and holding a tenant id, otherwise `400` (nothing to anchor to) or `403`
+  (present but foreign); the other party is stored exactly as sent. Updates strip every listed
+  column from the `SET`, and upserts exclude them from the `DO UPDATE` — neither party can be
+  re-assigned. The upsert conflict guard is null-safe for this shape: a row whose parties are
+  all `NULL` is visible to nobody, where a plain `NOT IN` would have read "unknown" as "not
+  foreign" and handed it over. Purely additive: `{ column }` and `{ column, through }` produce
+  byte-identical SQL. The three forms do not mix — `defineTable` throws at startup on `anyOf`
+  combined with `column` or `through`, on an empty or malformed `anyOf`, and on a scope
+  declaring no owner column at all.
+- `QueryClient.bulkInsertOrUpdate` takes an optional trailing `immutableCols`: columns written
+  on insert but left untouched when the statement lands on an existing row. Additive.
+
 ### Changed
 
+- **BREAKING: the tenant follows the request, not the table it addresses.** `tenantScope` on a
+  table is now enforced even when the request was addressed to a different table that reaches it
+  through a relation. Previously the tenant was resolved from the addressed table alone, so a
+  host declaring no scope of its own carried no predicate into the scoped tables it joined or
+  wrote into — every scoped row of the neighbour was readable through the open host, and writable
+  through it as a secondary, while the declaration said otherwise. A relation is a read grant and
+  a write join a write grant ([ADR 0010](./docs/adr/0010-joins-do-not-run-route-guards.md)), and
+  `tenantScope` is documented as the cap that makes them safe to declare; it now actually is.
+  `TenantContext.scope` becomes optional — `ids` is request-level, `scope` is table-level and
+  absent means "no filter on this table", never "no tenant". `getTenantId` is now consulted for a
+  request addressed to an unscoped table when one of its declared relations points at a scoped
+  one; it is still never called when nothing scoped is in reach. Tables that already declared
+  their own scope emit unchanged SQL. Migration — including the `getTenantId` body that now needs
+  to tolerate an unauthenticated request — in
+  [BREAKING_CHANGES.md](./BREAKING_CHANGES.md#breaking-change--the-tenant-follows-the-request-not-the-table-it-addresses).
+- **Database errors no longer reach clients verbatim.** Any error escaping an auto-generated
+  route that the plugin did not raise itself now answers `{"statusCode":500,"error":"Internal
+  Server Error","message":"Internal Server Error"}` instead of the driver message, which names
+  tables, columns and constraints (`duplicate key value violates unique constraint
+  "users_email_key"`). Errors the plugin raises are deliberate `4xx` written for clients and
+  are untouched; **no status code is remapped** — a unique violation is still a `500`, per
+  [ADR 0006](./docs/adr/0006-raw-db-errors.md). The original error is logged through
+  `request.log.error` and attached as `err.cause`, so a consumer `setErrorHandler` can still
+  apply its own policy (`23505` → `409`). The guard covers route plugins registered standalone
+  as well. The sanitized body carries `requestId` — the `reqId` Fastify logs on every line — so
+  the entry holding the real error is one grep away. While developing — where a client building
+  against the API (an agent especially) may have no way to read the server log — the new
+  **`exposeDebugInfo: true`** attaches the driver error (message, `code`, `constraint`, stack)
+  as a `debugInfo` payload. It is additive: `statusCode`, `error`, `message` and `requestId`
+  keep their production shape, so client-side error handling does not fork between
+  environments. Nothing else flips it — not `debug`, not `NODE_ENV`: `debug` belongs to the
+  consumer, who may wire it to an environment variable, and it governs the log (access
+  controlled) rather than the response body (not, and these routes are open by default). See
+  [ADR 0013](./docs/adr/0013-sanitized-db-errors.md).
+
 - **Unknown `filters` keys are now rejected with `400 Unknown filter field: <key>`** instead
   of being dropped in silence — on the main table and on every join family. The engine only
   ever visited keys matching a schema field, an `extraFilters` entry or a computed field, so a
@@ -26,8 +146,58 @@ Migration instructions for breaking changes live in **[BREAKING_CHANGES.md](./BR
   `LEFT JOIN` alias), so the filter applied nothing. They are now rejected with a `400` naming
   the reason and pointing at `joinMustExist` on the same relation.
 
+- **The update body requires the primary key**, which identifies the row it updates
+  (`PUT /rest/:table` carries it in `main`). The generated PK field is `Optional` — it is
+  absent on insert — and that modifier reached the update body unchanged, so a request omitting
+  `main.id` built its `WHERE` on `undefined` instead of being rejected. The code comment said
+  "PK required" all along.
+- **A secondary now runs the child table's `beforeInsert`.** `processSecondaries` already applied
+  the child's `excludeFromCreation` and its `tenantScope` but skipped its `beforeInsert`, so a
+  table whose hook *transformed* a value — encrypting a column, normalising a unit — stored the
+  raw client value whenever its rows arrived as another table's children, and the correct one
+  through its own route. Nothing failed at write time; the row read back as garbage later. The
+  hook runs after the exclusion strip and before the engine's FK auto-fill, which stays
+  authoritative, and receives the transaction connection. The child's `validate` and
+  `afterInsert` deliberately still do not run — see
+  [ADR 0014](./docs/adr/0014-what-follows-a-write-join.md). **Breaking** for a `beforeInsert`
+  written assuming it only ever ran on its own table's route.
+- **The get response schema honours `readExclude`**, like the search response already did; it
+  previously advertised fields the engine never selects. It now lives in `lib/schema/get.ts`
+  (`GetTableResponse`) instead of inline in the route, which is how it had drifted.
+  `schemaOverrides` stays where it was — write bodies only: an override rules what the API
+  accepts from now on, while the rows already stored predate it, so narrowing the response
+  would promise something no one can retroactively make true.
+
 ### Fixed
 
+- **A `tenantScope` naming a schema field instead of a column now fails at startup.**
+  `tenantScope` names real DB columns — the scope is applied to the SQL, after the payload has
+  been converted, and `through.foreignField` belongs to a parent table that may have no config of
+  its own. Nothing checked it, so writing `organizationId` where the column is `organization_id`
+  passed `defineTable` and surfaced as `column "organizationId" does not exist` on every request
+  to the table: an opaque `500`, failing closed but saying nothing. All four positions are now
+  checked (`column`, each `anyOf` entry, `through.localField`, and `through.foreignField` against
+  the parent schema). The check is narrow by design — it fires only when the name *is* a schema
+  field whose column differs, so a column absent from the schema raises nothing and a camelCase
+  database, where `col()` is the identity, raises nothing either.
+- **The `no-static-optional-peer` gate had never been able to fire.** `includeOnly: '^src/'` in
+  `.dependency-cruiser.mjs` dropped every npm edge from the dependency graph, so the rule meant to
+  stop a static import of `pg` / `mysql2` / `@fastify/*` — which breaks installs that use the other
+  dialect — matched nothing, ever. The graph now keeps npm dependencies as leaves, and the rule is
+  qualified to what its name says: static, value-level imports only, since a type-only import is
+  erased at compile time and a dynamic one is what the rule asks for. Both gates were verified by
+  introducing a violation and watching them fail.
+- **`defineTable` never validated `primary`.** A primary key naming a field the schema does not
+  have passed startup untouched and became a raw SQL error on the first request — a `500`. It
+  now throws at `defineTable` time, naming the field and the operations that need it. The one
+  exception is exact: a table exposing only `search` and declaring its own `defaultOrder` never
+  reads `primary`, which is the shape generated for a view whose key could not be inferred.
+- **PostgreSQL identity columns made a table impossible to insert into.**
+  `GENERATED ... AS IDENTITY` reports no `column_default`, which is what the generator read to
+  decide that the database fills a column in — so the primary key came out **mandatory** in the
+  write body while PostgreSQL rejected any explicit value for it (`Column "id" is an identity
+  column defined as GENERATED ALWAYS`). Identity columns are now flagged like MySQL's
+  `AUTO_INCREMENT`: the field is Optional and the CLI lists it under `excludeFromCreation`.
 - **Join validation no longer depends on the main result set.** `joinMultiple` / `joinLeft` /
   `joinGroup` skip their side query entirely when the main query matched no rows, so everything
   validated inside it was skipped with it — the same request answered 400 or 200 depending on