fastify-auto-sqlapi 0.1.11 → 0.1.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +37 -0
- package/README.md +91 -35
- package/dist/bin/generate-schema.js +9 -1
- package/dist/lib/cli/schema-codegen.d.ts +7 -0
- package/dist/lib/cli/schema-codegen.d.ts.map +1 -1
- package/dist/lib/cli/schema-codegen.js +25 -0
- package/dist/lib/engine/search/search.d.ts.map +1 -1
- package/dist/lib/engine/search/search.js +8 -2
- package/dist/types/cli.d.ts +6 -0
- package/dist/types/cli.d.ts.map +1 -1
- package/docs/adr/0008-agent-client-interface.md +66 -0
- package/docs/adr/0009-schema-exclusion-config-blacklist.md +47 -0
- package/docs/adr/README.md +2 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,43 @@ All notable changes to this project are documented here. This project follows
|
|
|
7
7
|
|
|
8
8
|
Migration instructions for breaking changes live in **[BREAKING_CHANGES.md](./BREAKING_CHANGES.md)**.
|
|
9
9
|
|
|
10
|
+
## [Unreleased]
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- **`excludeTables` config option** — blacklist for `sqlapi-generate-schema`. Tables
|
|
15
|
+
listed in `excludeTables` in `sqlapi.config.ts` (exact names or `*` globs, e.g.
|
|
16
|
+
`knex_*`) are skipped during schema generation; their previously generated schema files
|
|
17
|
+
are removed as orphans on the next full run, like those of dropped tables (ADR 0009).
|
|
18
|
+
Does not apply to `sqlapi-generate-tables`, which is already explicit (ADR 0007).
|
|
19
|
+
|
|
20
|
+
## [0.1.12]
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- **Security: `aggregations.by` bypassed `readExclude`.** Every other reference to a
|
|
25
|
+
read-excluded field (filters, conditions, orderBy, aggregation fields, join selections)
|
|
26
|
+
is rejected with 400, but the `joinGroup` GROUP BY field was resolved without the check —
|
|
27
|
+
`aggregations: { by: '<excludedField>', count: [...] }` returned the hidden field's
|
|
28
|
+
distinct values verbatim in `rows[].by`. The field now goes through the same
|
|
29
|
+
`validateSchemaField` guard as aggregation fields (400).
|
|
30
|
+
- **Security: HAVING-style conditions and 3-part aggregation `orderBy` could probe
|
|
31
|
+
read-excluded fields.** `buildAggOrderExpr` validated the aggregation field without the
|
|
32
|
+
join table's config, skipping the `readExclude` check. The downstream validation in the
|
|
33
|
+
joinGroup execution masked it — except when the main result set was empty, where that
|
|
34
|
+
validation is skipped entirely: a condition like `player.sum.<hiddenField> > X` answered
|
|
35
|
+
400 when at least one row matched and 200 when none did, a binary oracle allowing
|
|
36
|
+
bisection of the hidden field's aggregates. The check now fires at condition-build time.
|
|
37
|
+
Found by a full audit of every field-referencing input against `readExclude` (all other
|
|
38
|
+
paths verified guarded: filters, conditions, orderBy 1/2-part, selections, aggregation
|
|
39
|
+
fields, compute*, GET).
|
|
40
|
+
|
|
41
|
+
### Added
|
|
42
|
+
|
|
43
|
+
- ADR 0008: agent client interface — static grammar (`AGENTS_FRONTEND.md`, same filename by
|
|
44
|
+
design) + runtime manifest split, density as a feature, loose validation with the
|
|
45
|
+
structured 400 as the default tool strategy.
|
|
46
|
+
|
|
10
47
|
## [0.1.11]
|
|
11
48
|
|
|
12
49
|
### Breaking
|
package/README.md
CHANGED
|
@@ -1,14 +1,63 @@
|
|
|
1
1
|
# fastify-auto-sqlapi
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Your database already knows its tables, columns and relations. This plugin turns that knowledge into a complete REST API inside your [Fastify](https://fastify.dev/) app.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Two CLI commands introspect PostgreSQL / MySQL / MariaDB and generate typed schemas; one `register()` call turns each table *you choose to expose* into seven endpoints: search with filters, advanced conditions, relation joins and aggregations; validated writes with nested child records; bulk upsert/delete; multi-tenant isolation; Swagger docs. Related tables don't need endpoints of their own — reads reach them through joins, writes through nested children — so a whole subgraph can live behind a single exposed table. What's left to write is what makes your product yours — auth, business rules, custom logic — as plain TypeScript hooks next to your other routes.
|
|
6
|
+
|
|
7
|
+
One request, to show the point — *"active customers with over 500 total in completed orders, with their order stats, biggest spenders first"*:
|
|
8
|
+
|
|
9
|
+
```jsonc
|
|
10
|
+
// POST /api/search/customer?orderBy=orders.sum.total DESC&page=1&itemsPerPage=20
|
|
11
|
+
{
|
|
12
|
+
"filters": { "status": "active" },
|
|
13
|
+
"joinGroup": {
|
|
14
|
+
"orders": {
|
|
15
|
+
"filters": { "status": "completed" },
|
|
16
|
+
"aggregations": { "sum": ["total"], "count": ["id"] }
|
|
17
|
+
}
|
|
18
|
+
},
|
|
19
|
+
"conditions": [{ "field": "orders.sum.total", "method": "isGreater", "params": [500] }]
|
|
20
|
+
}
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Under the hood this is a fixed, request-shaped number of set-based queries — the main select plus one per requested join alias (EXISTS clauses and correlated subqueries ride inside them), **never one per returned row**. There are no lazy relations to accidentally loop over: the N+1 problem is structurally absent, not something you remember to avoid.
|
|
24
|
+
|
|
25
|
+
Writes read the same way — *"sync these clients and their tags: update the ones that exist, insert the missing ones, don't duplicate a tag"*:
|
|
26
|
+
|
|
27
|
+
```jsonc
|
|
28
|
+
// PUT /api/bulk/customer — all mains in one SQL; child FKs auto-filled from each parent
|
|
29
|
+
[
|
|
30
|
+
{ "main": { "email": "mario@acme.it", "name": "Mario" },
|
|
31
|
+
"secondaries": { "tags": [{ "tag": "vip" }, { "tag": "newsletter" }] } },
|
|
32
|
+
{ "main": { "email": "luigi@acme.it", "name": "Luigi" } }
|
|
33
|
+
]
|
|
34
|
+
// → 200 [{ "main": { "id": 1 }, "secondaries": { "tags": [{ "id": 10 }, { "id": 11 }] } },
|
|
35
|
+
// { "main": { "id": 2 } }]
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Conflict keys are declared server-side (`upsertMap`: here customer on `email`, tag on `(customerId, tag)`), so re-sending the same payload updates rows instead of duplicating them — client-side sync without hand-written diffing.
|
|
39
|
+
|
|
40
|
+
And multi-tenancy is one plugin option (`getTenantId: (req) => req.user.organizationId`), enforced on reads *and* writes with zero code in handlers. Rows of other tenants are simply invisible (`404`); here a caller in org B tries to re-point one of their own orders to a customer belonging to org A:
|
|
41
|
+
|
|
42
|
+
```jsonc
|
|
43
|
+
// PUT /api/rest/order — order 7 is mine, customer 41 belongs to another tenant
|
|
44
|
+
{ "main": { "id": 7, "customerId": 41 } }
|
|
45
|
+
// → 403 Forbidden — cross-tenant references are rejected server-side
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
No endpoint written by hand, no resolvers, no query language on the server — and these requests are schema-validated, Swagger-documented, size-capped and tenant-isolated like every other one.
|
|
49
|
+
|
|
50
|
+
## Not an ORM, not GraphQL — a third thing
|
|
51
|
+
|
|
52
|
+
- **Not an ORM.** An ORM is a library *your* code uses to talk to the database — you still hand-write every endpoint on top of it. This plugin generates the endpoints themselves. And there is no ORM underneath either: no models, no migrations, no second schema to keep in sync. The database is the single source of truth (the CLI reads it from `information_schema`), and every request runs as plain parameterized SQL you can read with `debug: true`.
|
|
53
|
+
- **Not GraphQL.** GraphQL buys client-driven flexibility with a heavy toolchain: a schema layer, hand-written resolvers (and their N+1 traps), mutations written one by one, client libraries, and query-cost analysis to stop hostile requests. This plugin covers what most projects actually reach for GraphQL for — filter, paginate, join related data, aggregate, in one round trip — with a fixed JSON grammar over plain REST: curl-able, Swagger-documented, bounded by design (join depth is fixed, page and bulk sizes are capped). The write side — nested children, upserts, bulk — is generated too, which GraphQL never gives you for free. What you give up: arbitrarily deep nesting, per-field selection on the main table, subscriptions. If you need those, you need GraphQL; most CRUD backends don't.
|
|
54
|
+
- **Not a hosted black box.** PostgREST / Hasura / Supabase give you an instant API as a separate service, configured from the outside. This is a plugin inside your own app: hooks, validation and auth are TypeScript functions in your codebase, and you can always drop down to `app.sqlApi.*` or raw SQL in a custom route — same engine, no lock-in.
|
|
6
55
|
|
|
7
56
|
## Features
|
|
8
57
|
|
|
58
|
+
- **Zero boilerplate** — 7 endpoints per exposed table; related tables can stay behind joins and nested writes instead of getting endpoints of their own
|
|
59
|
+
- **No ORM** — raw SQL via `pg` + parameterized queries; set-based joins, so no N+1 by construction
|
|
9
60
|
- **Agent-ready** — `AGENTS.md` and the ADRs ship *inside* the npm package, so a coding agent can configure the library without reading the source or guessing intent
|
|
10
|
-
- **Zero boilerplate** — define your tables, get 7 endpoints each
|
|
11
|
-
- **No ORM** — raw SQL via `pg` + parameterized queries
|
|
12
61
|
- **TypeBox validation** — request/response schemas auto-generated from your DB
|
|
13
62
|
- **Joins** — four explicit families (`joinMustExist`, `joinMultiple`, `joinGroup`, `joinLeft`) with alias support
|
|
14
63
|
- **Computed fields** — declare virtual fields as SQL expressions in `defineTable`; usable like schema fields in filters/orderBy/conditions, and opt-in projected in the response via `selectComputed`. Covers JSON extraction, derived columns, dialect-aware date bucketing
|
|
@@ -25,37 +74,6 @@ Point it at your database, and get a full REST API with search, advanced conditi
|
|
|
25
74
|
> with bound values, read the 0.1.6 entry — earlier versions could return wrong rows.
|
|
26
75
|
|
|
27
76
|
|
|
28
|
-
## Working with coding agents
|
|
29
|
-
|
|
30
|
-
The library is documented for two readers. Humans get this README; agents get task-oriented
|
|
31
|
-
files that ship in the published package, so they are already on disk after `npm install`:
|
|
32
|
-
|
|
33
|
-
| File | Contents |
|
|
34
|
-
| -------------------------------------- | ----------------------------------------------------------------- |
|
|
35
|
-
| `AGENTS.md` | Entry point: what the library does, which file to read for what |
|
|
36
|
-
| `AGENTS_BACKEND.md` | `defineTable` reference, hooks, tenancy, computed fields, CLI |
|
|
37
|
-
| `AGENTS_FRONTEND.md` | Request/response shapes for every endpoint |
|
|
38
|
-
| `docs/adr/` | Why the non-obvious behaviours are what they are |
|
|
39
|
-
|
|
40
|
-
Point your agent at them once:
|
|
41
|
-
|
|
42
|
-
> Read `node_modules/fastify-auto-sqlapi/AGENTS.md` before touching anything under `src/db/`.
|
|
43
|
-
|
|
44
|
-
Three design choices make the generated code predictable enough for an agent to write it
|
|
45
|
-
unsupervised:
|
|
46
|
-
|
|
47
|
-
- **Configuration is declarative and local.** A table is one `defineTable()` call — nothing to
|
|
48
|
-
wire across files, so a diff is reviewable at a glance.
|
|
49
|
-
- **One naming convention.** camelCase in requests, responses, hooks and validators; the
|
|
50
|
-
mapping to DB columns is automatic. There is no second convention to remember, and no
|
|
51
|
-
layer where the agent has to guess which one applies.
|
|
52
|
-
- **Errors are machine-readable.** A 400 carries `fields: [{ path, code, message }]`, so a
|
|
53
|
-
failing request tells the agent exactly what to fix instead of requiring a guess.
|
|
54
|
-
|
|
55
|
-
The ADRs matter more than they look: they answer the "why is it like this?" questions
|
|
56
|
-
(open-by-default, non-transactional bulk, always-updatable fields) that an agent would
|
|
57
|
-
otherwise resolve by inventing a workaround — or by proposing to change intentional behaviour.
|
|
58
|
-
|
|
59
77
|
## Quick Start
|
|
60
78
|
|
|
61
79
|
### 1. Install
|
|
@@ -76,6 +94,7 @@ export default {
|
|
|
76
94
|
schema: 'public', // PostgreSQL schema to introspect (default: 'public')
|
|
77
95
|
// dialect: 'mysql', // 'postgres' (default) | 'mysql' | 'mariadb'
|
|
78
96
|
// envFile: '.env.local', // env file to load (default: '.env')
|
|
97
|
+
// excludeTables: ['knex_*'], // tables to skip in schema generation ('*' = wildcard)
|
|
79
98
|
};
|
|
80
99
|
```
|
|
81
100
|
|
|
@@ -113,6 +132,12 @@ This introspects your tables and generates one TypeBox schema file per table in
|
|
|
113
132
|
`<outputDir>/schemas/` (e.g. `SchemaCustomer.ts`, `SchemaOrder.ts`). These files are
|
|
114
133
|
auto-generated — don't edit them; re-run the command after a DB change.
|
|
115
134
|
|
|
135
|
+
To keep some tables out of schema generation (migration bookkeeping, PostGIS internals, …)
|
|
136
|
+
list them in `excludeTables` in the config — exact table names or `*` globs (e.g.
|
|
137
|
+
`knex_*`). Excluded tables are invisible to the generator: their schemas are not created,
|
|
138
|
+
and any previously generated schema file is removed as an orphan on the next full run,
|
|
139
|
+
like that of a dropped table.
|
|
140
|
+
|
|
116
141
|
### 4. Generate the tables template
|
|
117
142
|
|
|
118
143
|
```bash
|
|
@@ -161,6 +186,37 @@ That's it. For a table called `customer`, you now have:
|
|
|
161
186
|
|
|
162
187
|
> **Note:** Search uses `POST` because filters are passed as JSON in the request body.
|
|
163
188
|
|
|
189
|
+
## Working with coding agents
|
|
190
|
+
|
|
191
|
+
The library is documented for two readers. Humans get this README; agents get task-oriented
|
|
192
|
+
files that ship in the published package, so they are already on disk after `npm install`:
|
|
193
|
+
|
|
194
|
+
| File | Contents |
|
|
195
|
+
| -------------------------------------- | ----------------------------------------------------------------- |
|
|
196
|
+
| `AGENTS.md` | Entry point: what the library does, which file to read for what |
|
|
197
|
+
| `AGENTS_BACKEND.md` | `defineTable` reference, hooks, tenancy, computed fields, CLI |
|
|
198
|
+
| `AGENTS_FRONTEND.md` | Request/response shapes for every endpoint |
|
|
199
|
+
| `docs/adr/` | Why the non-obvious behaviours are what they are |
|
|
200
|
+
|
|
201
|
+
Point your agent at them once:
|
|
202
|
+
|
|
203
|
+
> Read `node_modules/fastify-auto-sqlapi/AGENTS.md` before touching anything under `src/db/`.
|
|
204
|
+
|
|
205
|
+
Three design choices make the generated code predictable enough for an agent to write it
|
|
206
|
+
unsupervised:
|
|
207
|
+
|
|
208
|
+
- **Configuration is declarative and local.** A table is one `defineTable()` call — nothing to
|
|
209
|
+
wire across files, so a diff is reviewable at a glance.
|
|
210
|
+
- **One naming convention.** camelCase in requests, responses, hooks and validators; the
|
|
211
|
+
mapping to DB columns is automatic. There is no second convention to remember, and no
|
|
212
|
+
layer where the agent has to guess which one applies.
|
|
213
|
+
- **Errors are machine-readable.** A 400 carries `fields: [{ path, code, message }]`, so a
|
|
214
|
+
failing request tells the agent exactly what to fix instead of requiring a guess.
|
|
215
|
+
|
|
216
|
+
The ADRs matter more than they look: they answer the "why is it like this?" questions
|
|
217
|
+
(open-by-default, non-transactional bulk, always-updatable fields) that an agent would
|
|
218
|
+
otherwise resolve by inventing a workaround — or by proposing to change intentional behaviour.
|
|
219
|
+
|
|
164
220
|
## Table Configuration
|
|
165
221
|
|
|
166
222
|
Tables are configured with `defineTable()`. The only required fields are `primary` and the output of `exportTableInfo()`:
|
|
@@ -4,7 +4,7 @@ import * as path from 'node:path';
|
|
|
4
4
|
import { loadConfig } from '../lib/cli/config.js';
|
|
5
5
|
import { buildConnectionString, introspectTables } from '../lib/cli/pg-introspect.js';
|
|
6
6
|
import { buildMysqlConnectionConfig, introspectMysqlTables } from '../lib/cli/mysql-introspect.js';
|
|
7
|
-
import { buildTableMap, generateSchemaFile } from '../lib/cli/schema-codegen.js';
|
|
7
|
+
import { applyExcludeTables, buildTableMap, generateSchemaFile } from '../lib/cli/schema-codegen.js';
|
|
8
8
|
import { loadEnvFile, CONSOLE_COLORS, display, displayAsTableRow, parseArgs, runCli } from './utils.js';
|
|
9
9
|
await runCli('fastify-auto-sqlapi: generating schemas', async () => {
|
|
10
10
|
const cliArgs = parseArgs({
|
|
@@ -36,6 +36,14 @@ await runCli('fastify-auto-sqlapi: generating schemas', async () => {
|
|
|
36
36
|
return;
|
|
37
37
|
}
|
|
38
38
|
const tableMap = buildTableMap(rows);
|
|
39
|
+
// Config blacklist: excluded tables are invisible to the generator, so their
|
|
40
|
+
// schema files (if any) fall through to the orphan cleanup below.
|
|
41
|
+
const excluded = applyExcludeTables(tableMap, config.excludeTables);
|
|
42
|
+
if (excluded.length) {
|
|
43
|
+
display(`Excluded by config (excludeTables): ${excluded.join(', ')}`, CONSOLE_COLORS.gray);
|
|
44
|
+
}
|
|
45
|
+
// Note: no early return when everything is excluded — the orphan cleanup below
|
|
46
|
+
// must still run to remove the schema files of freshly excluded tables.
|
|
39
47
|
// Filter tables if --tables flag is provided
|
|
40
48
|
const tableNames = cliArgs.tables;
|
|
41
49
|
if (tableNames.length) {
|
|
@@ -5,5 +5,12 @@ export declare function convertColType(udtName: string, columnInfo: {
|
|
|
5
5
|
is_auto_increment?: boolean;
|
|
6
6
|
}): string;
|
|
7
7
|
export declare function buildTableMap(rows: ColumnInfo[]): TableMap;
|
|
8
|
+
/**
|
|
9
|
+
* Remove from `tableMap` the tables matching any of the `excludeTables` config patterns.
|
|
10
|
+
* Patterns are matched against the DB table name (not the generated schema name);
|
|
11
|
+
* `*` is the only wildcard — everything else is literal.
|
|
12
|
+
* Returns the names of the excluded tables, in map order.
|
|
13
|
+
*/
|
|
14
|
+
export declare function applyExcludeTables(tableMap: TableMap, excludeTables?: string[]): string[];
|
|
8
15
|
export declare function generateSchemaFile(schemaName: string, tableName: string, fields: Record<string, string>, colMap: Record<string, string>, primary?: string[]): string;
|
|
9
16
|
//# sourceMappingURL=schema-codegen.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"schema-codegen.d.ts","sourceRoot":"","sources":["../../../src/lib/cli/schema-codegen.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,UAAU,EAAE,QAAQ,EAAE,MAAM,gBAAgB,CAAC;AAE3D,wBAAgB,cAAc,CAC5B,OAAO,EAAE,MAAM,EACf,UAAU,EAAE;IAAE,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAAC,WAAW,EAAE,MAAM,CAAC;IAAC,iBAAiB,CAAC,EAAE,OAAO,CAAA;CAAE,GAC9F,MAAM,CA0DR;AAED,wBAAgB,aAAa,CAAC,IAAI,EAAE,UAAU,EAAE,GAAG,QAAQ,CAqC1D;AAED,wBAAgB,kBAAkB,CAChC,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAC9B,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAC9B,OAAO,CAAC,EAAE,MAAM,EAAE,GACjB,MAAM,CA8CR"}
|
|
1
|
+
{"version":3,"file":"schema-codegen.d.ts","sourceRoot":"","sources":["../../../src/lib/cli/schema-codegen.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,UAAU,EAAE,QAAQ,EAAE,MAAM,gBAAgB,CAAC;AAE3D,wBAAgB,cAAc,CAC5B,OAAO,EAAE,MAAM,EACf,UAAU,EAAE;IAAE,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAAC,WAAW,EAAE,MAAM,CAAC;IAAC,iBAAiB,CAAC,EAAE,OAAO,CAAA;CAAE,GAC9F,MAAM,CA0DR;AAED,wBAAgB,aAAa,CAAC,IAAI,EAAE,UAAU,EAAE,GAAG,QAAQ,CAqC1D;AAED;;;;;GAKG;AACH,wBAAgB,kBAAkB,CAAC,QAAQ,EAAE,QAAQ,EAAE,aAAa,CAAC,EAAE,MAAM,EAAE,GAAG,MAAM,EAAE,CAoBzF;AAED,wBAAgB,kBAAkB,CAChC,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAC9B,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,EAC9B,OAAO,CAAC,EAAE,MAAM,EAAE,GACjB,MAAM,CA8CR"}
|
|
@@ -85,6 +85,31 @@ export function buildTableMap(rows) {
|
|
|
85
85
|
return acc;
|
|
86
86
|
}, {});
|
|
87
87
|
}
|
|
88
|
+
/**
|
|
89
|
+
* Remove from `tableMap` the tables matching any of the `excludeTables` config patterns.
|
|
90
|
+
* Patterns are matched against the DB table name (not the generated schema name);
|
|
91
|
+
* `*` is the only wildcard — everything else is literal.
|
|
92
|
+
* Returns the names of the excluded tables, in map order.
|
|
93
|
+
*/
|
|
94
|
+
export function applyExcludeTables(tableMap, excludeTables) {
|
|
95
|
+
if (excludeTables === undefined)
|
|
96
|
+
return [];
|
|
97
|
+
if (!Array.isArray(excludeTables) || excludeTables.some((p) => typeof p !== 'string')) {
|
|
98
|
+
throw new Error('Invalid config: "excludeTables" must be an array of strings.');
|
|
99
|
+
}
|
|
100
|
+
if (excludeTables.length === 0)
|
|
101
|
+
return [];
|
|
102
|
+
const regexes = excludeTables.map((pattern) => new RegExp(`^${pattern.replace(/[.*+?^${}()|[\]\\]/g, '\\$&').replaceAll('\\*', '.*')}$`));
|
|
103
|
+
const excluded = [];
|
|
104
|
+
for (const schemaName of Object.keys(tableMap)) {
|
|
105
|
+
const tableName = tableMap[schemaName].name;
|
|
106
|
+
if (regexes.some((re) => re.test(tableName))) {
|
|
107
|
+
excluded.push(tableName);
|
|
108
|
+
delete tableMap[schemaName];
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
return excluded;
|
|
112
|
+
}
|
|
88
113
|
export function generateSchemaFile(schemaName, tableName, fields, colMap, primary) {
|
|
89
114
|
const fieldsStr = JSON.stringify(fields, null, 2).replace(/"(.*?)"\s*:\s*"Type(.*?)"/g, '$1: Type$2');
|
|
90
115
|
const colMapStr = JSON.stringify(colMap, null, 2);
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"search.d.ts","sourceRoot":"","sources":["../../../../src/lib/engine/search/search.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EACV,QAAQ,EACR,YAAY,EACZ,YAAY,EAcb,MAAM,mBAAmB,CAAC;
|
|
1
|
+
{"version":3,"file":"search.d.ts","sourceRoot":"","sources":["../../../../src/lib/engine/search/search.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EACV,QAAQ,EACR,YAAY,EACZ,YAAY,EAcb,MAAM,mBAAmB,CAAC;AA8lC3B,wBAAsB,YAAY,CAChC,QAAQ,EAAE,QAAQ,EAClB,MAAM,EAAE,YAAY,GACnB,OAAO,CAAC,YAAY,CAAC,CA4HvB"}
|
|
@@ -219,7 +219,10 @@ function buildAggOrderExpr(db, dbTables, tableConf, alias, fn, field, joinGroup)
|
|
|
219
219
|
if (groupReq.aggregations.by && groupReq.aggregations.by !== joinField) {
|
|
220
220
|
err400(`Cannot order by aggregation on joinGroup with 'by' clause on non-FK column: ${alias} (grouped by '${groupReq.aggregations.by}', correlation FK is '${joinField}')`);
|
|
221
221
|
}
|
|
222
|
-
|
|
222
|
+
// Pass the join table's conf: without it validateSchemaField skips the readExclude
|
|
223
|
+
// check, and a HAVING-style condition on a hidden field becomes a 400-vs-200 oracle
|
|
224
|
+
// (executeJoinGroup's own validation is skipped when the main result set is empty).
|
|
225
|
+
const fieldCol = validateSchemaField(field, joinSchema, dbTables[joinSchema.tableName]);
|
|
223
226
|
const refs = extractJoinRefs(db, tableConf, joinDef);
|
|
224
227
|
// The subquery FROM is aliased so the correlation to the outer table survives a
|
|
225
228
|
// self-referencing relation; everything inside references the alias.
|
|
@@ -555,7 +558,10 @@ function buildByExpression(by, joinSchema, joinTableConf, db) {
|
|
|
555
558
|
err400(`Invalid 'by' specification: expected a field or computed name`);
|
|
556
559
|
}
|
|
557
560
|
if (by in joinSchema.fields) {
|
|
558
|
-
|
|
561
|
+
// validateSchemaField also enforces readExclude: GROUP BY on a hidden field
|
|
562
|
+
// would return its distinct values verbatim in `rows[].by`.
|
|
563
|
+
const col = validateSchemaField(by, joinSchema, joinTableConf);
|
|
564
|
+
return `${db.qi(joinSchema.tableName)}.${db.qi(col)}`;
|
|
559
565
|
}
|
|
560
566
|
const fn = joinTableConf?.computedFields?.[by];
|
|
561
567
|
if (fn) {
|
package/dist/types/cli.d.ts
CHANGED
|
@@ -4,6 +4,12 @@ export interface SqlApiConfig {
|
|
|
4
4
|
schema?: string;
|
|
5
5
|
dialect?: DialectName;
|
|
6
6
|
envFile?: string;
|
|
7
|
+
/**
|
|
8
|
+
* Tables to skip during schema generation (`sqlapi-generate-schema`).
|
|
9
|
+
* Matched against the DB table name; `*` is a wildcard (e.g. "knex_*").
|
|
10
|
+
* Excluded schemas already on disk are removed as orphans on the next full run.
|
|
11
|
+
*/
|
|
12
|
+
excludeTables?: string[];
|
|
7
13
|
}
|
|
8
14
|
export interface ColumnInfo {
|
|
9
15
|
table_name: string;
|
package/dist/types/cli.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../../src/types/cli.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAErD,MAAM,WAAW,YAAY;IAC3B,SAAS,EAAE,MAAM,CAAC;IAClB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,WAAW,CAAC;IACtB,OAAO,CAAC,EAAE,MAAM,CAAC;
|
|
1
|
+
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../../src/types/cli.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAErD,MAAM,WAAW,YAAY;IAC3B,SAAS,EAAE,MAAM,CAAC;IAClB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,WAAW,CAAC;IACtB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;;OAIG;IACH,aAAa,CAAC,EAAE,MAAM,EAAE,CAAC;CAC1B;AAED,MAAM,WAAW,UAAU;IACzB,UAAU,EAAE,MAAM,CAAC;IACnB,WAAW,EAAE,MAAM,CAAC;IACpB,QAAQ,EAAE,MAAM,CAAC;IACjB,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,WAAW,EAAE,MAAM,CAAC;IACpB,yFAAyF;IACzF,UAAU,CAAC,EAAE,OAAO,CAAC;IACrB,wFAAwF;IACxF,iBAAiB,CAAC,EAAE,OAAO,CAAC;CAC7B;AAED,MAAM,WAAW,QAAQ;IACvB,CAAC,UAAU,EAAE,MAAM,GAAG;QACpB,IAAI,EAAE,MAAM,CAAC;QACb,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QAC/B,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QAC/B,iFAAiF;QACjF,OAAO,EAAE,MAAM,EAAE,CAAC;KACnB,CAAC;CACH"}
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# 0008. Agent client interface: static grammar + runtime manifest
|
|
2
|
+
|
|
3
|
+
- **Status**: accepted
|
|
4
|
+
- **Date**: 2026-07-26
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
The plugin doubles as the enforcement layer for LLM clients that drive the API at runtime
|
|
9
|
+
(a chat agent operating on backoffice data: whatever requests the model invents, tenant
|
|
10
|
+
scoping, `readExclude`, `operations` and caps still apply). Such a client needs to be
|
|
11
|
+
told how to call the API — but an LLM prompt has different constraints than human
|
|
12
|
+
documentation: every line costs tokens on every call, and anything deployment-specific
|
|
13
|
+
goes stale the moment the schema changes.
|
|
14
|
+
|
|
15
|
+
## Decision
|
|
16
|
+
|
|
17
|
+
The client-facing documentation is split into two halves with different lifecycles:
|
|
18
|
+
|
|
19
|
+
- **Grammar** — `AGENTS_FRONTEND.md`: HOW to call the API (endpoints, search with the four
|
|
20
|
+
join families, condition methods with params arity, ordering notations, pagination,
|
|
21
|
+
writes, error shapes). Static, identical for every deployment, ~110 dense lines designed
|
|
22
|
+
for a system prompt. It deliberately contains **no tables, fields or aliases**.
|
|
23
|
+
- **Vocabulary** — the manifest (`agentManifest: true` → `GET {prefix}/agent/manifest(.md)`,
|
|
24
|
+
or `buildAgentManifest()` programmatically): WHAT this deployment exposes — tables,
|
|
25
|
+
fields with type/required/nullable, enabled operations, join aliases with direction,
|
|
26
|
+
computed fields. Generated from `DbTables` at startup, so it can never drift from the
|
|
27
|
+
running configuration.
|
|
28
|
+
|
|
29
|
+
Three deliberate properties:
|
|
30
|
+
|
|
31
|
+
1. **The grammar file replaced the old tutorial-style `AGENTS_FRONTEND.md` under the same
|
|
32
|
+
filename.** The path `node_modules/fastify-auto-sqlapi/AGENTS_FRONTEND.md` is known to
|
|
33
|
+
consumers' coding agents; a new filename would have broken every existing reference for
|
|
34
|
+
zero benefit. Do not rename it.
|
|
35
|
+
2. **Density is a feature, not a documentation gap.** The file lives in LLM system
|
|
36
|
+
prompts: jsonc examples with terse end-of-line comments, no rationale, no repetition —
|
|
37
|
+
each concept appears exactly once. Normative rules (what returns 400) belong in it;
|
|
38
|
+
explanations do not. Expanding it with prose and tutorials would revert this decision.
|
|
39
|
+
3. **Loose validation with the structured 400 as retry signal is the default strategy**
|
|
40
|
+
for LLM tools; strict per-table JSON-Schema tools are opt-in via `agentToolSchemas()`.
|
|
41
|
+
The plugin's 400 (`fields: [{path, code, message}]`) was designed machine-readable, so
|
|
42
|
+
a loose generic tool self-corrects in one round trip. This is also why the plugin does
|
|
43
|
+
not ship an opinionated MCP/tool layer: it provides the pieces (grammar, manifest,
|
|
44
|
+
schemas), the consumer composes them.
|
|
45
|
+
|
|
46
|
+
## Alternatives considered
|
|
47
|
+
|
|
48
|
+
- **One complete document (grammar + tables of a given deployment)** — rejected: the
|
|
49
|
+
deployment-specific half goes stale on every schema change; the manifest endpoint is
|
|
50
|
+
the only source that cannot drift.
|
|
51
|
+
- **A new `AGENT_CLIENT.md` alongside the old `AGENTS_FRONTEND.md`** — rejected: near-total
|
|
52
|
+
content overlap, proven drift risk (a fix had to be applied to both files during the
|
|
53
|
+
same session that created the second one), and known-path breakage on removal.
|
|
54
|
+
- **Strict JSON-Schema tools as the default** — rejected: the full search schema per
|
|
55
|
+
table is enormous (conditions, four join families, per-alias shapes); across N tables
|
|
56
|
+
it explodes prompt size and degrades model performance. Kept as opt-in for hot tables.
|
|
57
|
+
|
|
58
|
+
## Consequences
|
|
59
|
+
|
|
60
|
+
- `AGENTS_FRONTEND.md` serves both coding agents (writing client code) and runtime LLM
|
|
61
|
+
clients; the verbose tutorial content it replaced is covered by the grammar plus
|
|
62
|
+
Swagger.
|
|
63
|
+
- Contributions that add tables/fields/examples-with-prose to the grammar file, or rename
|
|
64
|
+
it, should be declined with a pointer to this ADR.
|
|
65
|
+
- New request-shape features must update the grammar (a rule line, not a tutorial) and,
|
|
66
|
+
when they touch per-table capabilities, the manifest builder.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# 0009. Schema exclusion is a config blacklist — excluded schemas are removed as orphans
|
|
2
|
+
|
|
3
|
+
- **Status**: accepted
|
|
4
|
+
- **Date**: 2026-07-27
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
`sqlapi-generate-schema` regenerates every table by default (ADR 0007: schemas are
|
|
9
|
+
disposable mirrors). There was no way to keep unwanted tables — migration bookkeeping
|
|
10
|
+
(`knex_migrations*`), PostGIS internals (`spatial_ref_sys`), vendor tables — out of the
|
|
11
|
+
generated schemas short of deleting the files after every run.
|
|
12
|
+
|
|
13
|
+
## Decision
|
|
14
|
+
|
|
15
|
+
A blacklist lives in the existing config file: `excludeTables: string[]` in
|
|
16
|
+
`sqlapi.config.ts`, matched against the DB table name, with `*` as the only wildcard.
|
|
17
|
+
|
|
18
|
+
Excluded tables are **invisible to the generator**: their schemas are not generated, and a
|
|
19
|
+
previously generated `Schema*.ts` for an excluded table is removed by the orphan cleanup
|
|
20
|
+
on the next full run — exactly as if the table had been dropped. This keeps the ADR 0007
|
|
21
|
+
invariant intact: everything under `schemas/` is a disposable mirror; excluding a table
|
|
22
|
+
must not turn its schema file into a hand-maintained exception.
|
|
23
|
+
|
|
24
|
+
This does not contradict ADR 0007's rejection of a skip-list for *tables*: there, explicit
|
|
25
|
+
invocation solves exclusion for free. For schemas the default is "all", so a blacklist is
|
|
26
|
+
the only way to exclude — and the same ADR is why `excludeTables` does **not** apply to
|
|
27
|
+
`sqlapi-generate-tables`.
|
|
28
|
+
|
|
29
|
+
## Alternatives considered
|
|
30
|
+
|
|
31
|
+
- **`.schemaignore` file next to the schemas** — rejected: a second config format and a
|
|
32
|
+
second source of truth to parse and document, when a typed config file already exists.
|
|
33
|
+
- **CLI-only `--exclude` flag** — rejected as the primary mechanism: the exclusion is a
|
|
34
|
+
durable property of the project, not of one invocation; forgetting the flag once would
|
|
35
|
+
resurrect the unwanted schemas.
|
|
36
|
+
- **Keeping excluded schema files on disk (generator "never touches" them)** — rejected:
|
|
37
|
+
it would silently support hand-maintained files under `schemas/`, breaking the "always
|
|
38
|
+
safe to regenerate everything" property of ADR 0007.
|
|
39
|
+
|
|
40
|
+
## Consequences
|
|
41
|
+
|
|
42
|
+
- Adding a table to `excludeTables` deletes its generated schema file on the next full
|
|
43
|
+
run. This is intended: excluded means gone, not frozen.
|
|
44
|
+
- Hand-written schemas do not belong in `schemas/`; exclusion is not a mechanism to
|
|
45
|
+
protect local edits.
|
|
46
|
+
- An invalid `excludeTables` value (not an array of strings) fails the run with an
|
|
47
|
+
explicit error rather than being silently ignored.
|
package/docs/adr/README.md
CHANGED
|
@@ -22,6 +22,8 @@ not by editing history.
|
|
|
22
22
|
| [0005](./0005-insert-pipeline-sanitize-before-hooks.md) | Insert pipeline: client payload sanitized before `beforeInsert` |
|
|
23
23
|
| [0006](./0006-raw-db-errors.md) | Raw DB errors surface as 500 — no SQLSTATE→HTTP mapping |
|
|
24
24
|
| [0007](./0007-explicit-table-generation.md) | Table generation is explicit — `--all` is a migration helper, not the default |
|
|
25
|
+
| [0008](./0008-agent-client-interface.md) | Agent client interface: static grammar + runtime manifest |
|
|
26
|
+
| [0009](./0009-schema-exclusion-config-blacklist.md) | Schema exclusion is a config blacklist — excluded schemas are removed as orphans |
|
|
25
27
|
|
|
26
28
|
## When to write one
|
|
27
29
|
|
package/package.json
CHANGED