json-rest-api 1.0.29 → 2.1.0

package/docs/GUIDE/01-getting-started.md

@@ -0,0 +1,338 @@
+---
+title: "Getting started"
+chapter: 1
+chapter_label: "01"
+---
+
+# 01. Getting started
+
+The guide uses a book catalog with countries, publishers, authors, books and
+book-author links. Start with countries here; subsequent chapters add the
+related resources. The [quickstart](../QUICKSTART.md) provides a complete
+Express server with related publishers and authors.
+
+Use Node 24 or newer; development verification runs on Node 24. These examples
+use the revised contract in the [migration guide](33-migrating-to-v2.md). Until
+its release is published, install the intended library tarball instead of the
+registry package.
+
+## Install and create a resource
+
+```bash
+npm init -y
+npm pkg set type=module
+npm install json-rest-api knex better-sqlite3
+```
+
+Save this as `index.js`. The database is in memory, so records disappear when
+the process exits. Resource schemas are flat field maps. Related resources are
+not declared in this first example because their definitions come later.
+
+```javascript
+import { JsonRestApi, RestApiPlugin, RestApiKnexPlugin } from 'json-rest-api'
+import knexLib from 'knex'
+
+const knex = knexLib({
+  client: 'better-sqlite3',
+  connection: { filename: ':memory:' },
+  useNullAsDefault: true
+})
+const api = new JsonRestApi({
+  name: 'book-catalog-api',
+})
+
+try {
+  await api.use(RestApiPlugin)
+  await api.use(RestApiKnexPlugin, { knex })
+  await api.addResource('countries', {
+    schema: {
+      name: { type: 'string', required: true, max: 100, search: true },
+      code: { type: 'string', max: 2, unique: true, search: true }
+    }
+  })
+  await api.resources.countries.createKnexTable()
+
+  const country = await api.resources.countries.post({
+    data: { name: 'United States', code: 'US' }
+  })
+  const refetched = await api.resources.countries.get({ id: country.id })
+  console.log('Country:', refetched)
+
+  const matches = await api.resources.countries.query({
+    queryParams: {
+      filters: { code: 'US' },
+      sort: ['name'],
+      page: { number: 1, size: 10 }
+    }
+  })
+  console.log('Matches:', matches.data)
+
+  const changed = await api.resources.countries.patch({
+    id: country.id,
+    data: { name: 'United States of America' },
+    returning: 'minimal'
+  })
+  console.log('Changed identifier:', changed)
+
+  const document = await api.resources.countries.get({
+    id: country.id,
+    format: 'jsonapi'
+  })
+  console.log('JSON:API attributes:', document.data.attributes)
+} finally {
+  await knex.destroy()
+}
+```
+
+Run `node index.js`. The first record has name `United States` and code `US`;
+the query returns that record. PATCH returns `{ type: 'countries', id: country.id }`.
+The final JSON:API read has `data.attributes.name` equal to
+`United States of America`.
+
+When adapting later examples to this script, insert their operations inside the
+`try` block after resource registration and before the database is destroyed.
+
+## Logging
+
+Logging is silent unless you supply a logger. For example:
+
+```javascript
+const api = new JsonRestApi({
+  name: 'app',
+  logger: { warn: console.warn, error: console.error }
+})
+```
+
+The logger can supply `trace`, `debug`, `info`, `warn`, `error` and `fatal` methods.
+Missing methods are silent. Your logger controls formatting and filtering.
+Resource and connector diagnostics use the library's bounded error formatting;
+see the [API reference](../API.md) for diagnostic and error contracts.
+The old `log` and `logging` constructor options are rejected.
+
+## Database configuration
+
+`RestApiKnexPlugin` uses the supplied Knex instance. Install the driver for the
+selected database. For PostgreSQL, use `npm install pg` and configure:
+
+```javascript
+const knex = knexLib({
+  client: 'pg',
+  connection: {
+    host: '127.0.0.1',
+    user: 'your_username',
+    password: 'your_password',
+    database: 'your_database_name',
+    port: 5432
+  }
+})
+```
+
+For MySQL 8, use `npm install mysql2` and configure:
+
+```javascript
+const knex = knexLib({
+  client: 'mysql2',
+  connection: {
+    host: '127.0.0.1',
+    user: 'your_username',
+    password: 'your_password',
+    database: 'your_database_name',
+    port: 3306
+  }
+})
+```
+
+The verified combinations are SQLite through better-sqlite3, PostgreSQL 16
+through pg and MySQL 8 through mysql2. Review the
+[backend capabilities](30-backend-capabilities.md) before switching: precision,
+collation, schema evolution and transaction behavior have backend-specific
+limits. Other Knex clients and server versions are unverified.
+
+## Method arguments and response options
+
+Call resources through `api.resources.countries`. Methods accept operation
+parameters first and optional application context second. Plain write records
+go in `data`; JSON:API write documents go in `document`. Supply exactly one of
+these keys. The target ID, transaction handle and response options
+remain outside the record. Query selection goes in `queryParams`.
+
+| Option | Values | Programmatic default |
+| --- | --- | --- |
+| `format` | `'plain'`, `'jsonapi'` | `'plain'` |
+| `returning` | `'none'`, `'minimal'`, `'full'` | `'full'` |
+
+`format` selects the output representation independently of input. Plain `get`
+returns a record; plain `query` returns a collection whose `data` contains
+records. JSON:API output is a document with `data`. Either output can be requested
+with either input key. Record attributes never become top-level method parameters.
+
+For POST, PUT and PATCH, `returning: 'minimal'` returns `{ type, id }` in plain
+format or `{ data: { type, id } }` in JSON:API format. `returning: 'none'` returns
+undefined. Full responses run the resource read/enrichment path, including its
+visibility rules and selected fields. DELETE and relationship writes return
+undefined. `returning` does not change read results.
+
+You can set programmatic defaults when installing `RestApiPlugin`, override
+them in a resource definition, and override those defaults for a particular
+call. Each setting accepts a single value, not a per-method map or boolean.
+
+```javascript
+await api.use(RestApiPlugin, { format: 'plain', returning: 'minimal' })
+await api.addResource('countries', {
+  returning: 'full',
+  schema: { name: { type: 'string', required: true } }
+})
+const result = await api.resources.countries.post({
+  data: { name: 'Canada' },
+  returning: 'none'
+})
+// result is undefined
+```
+
+This is an alternative configuration for a fresh API instance; do not install
+the plugin or register the same resource twice in the initial script.
+
+PUT creates a missing target or replaces a visible existing record according
+to schema requirements, defaults and nullability. PATCH changes supplied fields.
+Both accept the target ID as `params.id` or in the input record; conflicting
+IDs are rejected. For exact payloads, relationships and pagination, see the
+[API reference](../API.md).
+
+## HTTP endpoints
+
+The library provides [Express](../QUICKSTART.md) and
+[Fastify](23-fastify.md) connectors. Install the connector before resource
+registration. Express exposes `api.http.express.router` and
+`api.http.express.notFoundRouter`, which your application mounts with `app.use`.
+The quickstart contains the complete setup and shutdown code.
+
+Generated HTTP routes explicitly use JSON:API format and full write responses.
+They do not inherit a resource's programmatic plain/minimal defaults. There is
+no separate simplified-transport switch.
+
+For a server exposing the countries resource at `/api`, create a country with:
+
+```bash
+curl -i -X POST http://localhost:3000/api/countries \
+  -H 'Content-Type: application/vnd.api+json' \
+  -d '{"data":{"type":"countries","attributes":{"name":"United Kingdom","code":"GB"}}}'
+```
+
+Successful POST returns 201 with a JSON:API document. Use its `data.id` in a
+subsequent GET, for example `curl http://localhost:3000/api/countries/1` on an
+otherwise empty database. Successful PUT/PATCH with a response body return 200;
+DELETE returns 204 with no body. A custom endpoint selecting no write result
+returns 204; it must not send a JSON body with that status. These describe normal
+successful responses; application hooks can reject requests or customize responses.
+
+## Limits and resource configuration
+
+Configure limits through documented plugin/resource options, rather than
+mutating compiled resource metadata after registration.
+
+| Option | Default | Purpose |
+| --- | --- | --- |
+| `queryDefaultLimit` | `20` | Default collection size |
+| `queryMaxLimit` | `100` | Maximum requested collection size |
+| `includeDepthLimit` | `3` | Maximum nested include depth |
+| `enablePaginationCounts` | `true` | Counts for numbered pagination |
+| `idProperty` | `'id'` | Physical primary-key column for table-backed resources |
+| `normalizeId` | Built-in normalizer | Canonical resource identifiers |
+
+Counts do not turn cursor pagination into numbered pagination. Omitted or empty
+page options use the default collection cap without pagination metadata.
+`sortableFields` and `defaultSort` control resource sorting; see
+[pagination and ordering](09-pagination-and-sorting.md) for their contracts.
+
+## Logical IDs and Physical Columns
+
+For table-backed resources, `idProperty` defines the physical primary-key column. The API surface still uses the logical resource id.
+
+- plain writes use `data.id`
+- JSON:API writes use `document.data.id`
+- responses expose `id` or `data.id`
+- the resource id is not part of `attributes`
+
+```javascript
+await api.addResource('profiles', {
+  idProperty: 'user_id',
+  schema: {
+    id: { type: 'id', required: true, storage: { column: 'user_id' } },
+    displayName: { type: 'string', required: true },
+    loginCount: { type: 'number', defaultTo: 0 }
+  },
+  tableName: 'profiles'
+});
+```
+
+For table-backed resources, storage columns default to snake_case, so `displayName` maps to `display_name` and `loginCount` maps to `login_count`. Use `storage.column` for a per-field override, or `storage: { naming: 'exact' }` when physical column names should match logical field names exactly.
+
+## ID normalization
+
+`normalizeId` lets you canonicalize resource identifiers before the library reads, writes, validates, or links records.
+
+The default normalizer already handles the common cases:
+
+- trims surrounding whitespace from strings
+- converts finite numbers and bigints to strings
+- rejects values that normalize to an empty identifier
+
+You only need to provide `normalizeId` when your resource ids need additional canonicalization, such as uppercasing, lowercasing, or removing formatting characters.
+
+```javascript
+await api.use(RestApiPlugin, {
+  normalizeId: (value) => {
+    if (value === null || value === undefined) return null
+
+    const normalized = String(value).trim()
+    return normalized ? normalized.toUpperCase() : null
+  }
+})
+
+await api.addResource('countries', {
+  // Override the plugin-wide normalizer for this one resource
+  normalizeId: (value) => {
+    if (value === null || value === undefined) return null
+
+    const normalized = String(value).trim()
+    return normalized ? normalized.toLowerCase() : null
+  },
+  schema: {
+    name: { type: 'string', required: true }
+  }
+})
+```
+
+Normalization is applied consistently to:
+
+- `get`, `put`, `patch`, `delete`, `getRelated`, and `getRelationship` parent ids
+- explicit resource ids in `POST`, `PUT`, and `PATCH` documents
+- relationship identifiers supplied to `postRelationship`, `patchRelationship`, and `deleteRelationship`
+
+The resource-level `normalizeId` override also applies to relationship identifiers when that resource appears as the related target.
+
+If an id normalizes to an empty value:
+
+- existing-resource operations behave as `not_found`
+- explicit `POST` document ids are rejected as validation errors on `data.id`
+
+For full examples of table creation, physical column naming, and migration generation, see [Knex Schema and Migrations](21-schema-and-migrations.md).
+
+## Transactions and database helpers
+
+Use `api.transaction(callback, context?)` for a unit of library writes and pass
+its managed handle as each operation's `transaction` parameter. Raw Knex
+transactions are available for direct SQL and reads; they are not accepted as
+owners of library writes. See [managed transactions](19-managed-transactions.md)
+and [transaction outcomes](20-transaction-outcomes.md) before composing operations.
+
+`api.knex.instance` exposes the supplied Knex instance and
+`api.knex.capabilities` describes detected backend capabilities. Table-backed
+resources provide `createKnexTable`, `introspectKnexTableSnapshot`,
+`generateKnexMigration`, `generateKnexMigrationDiff`, `addKnexFields` and
+`alterKnexFields`. These helpers have different applicability to canonical
+storage; use the [schema and migration guide](21-schema-and-migrations.md)
+for examples and limitations.
+
+Continue with [the starting point](03-running-example.md) or return to
+the [guide index](index.md).

package/docs/GUIDE/02-resources-and-relationships.md

@@ -0,0 +1,33 @@
+---
+title: "Resources and relationships"
+chapter: 2
+chapter_label: "02"
+---
+
+# 02. Resources and relationships
+
+These tutorials cover resource CRUD, relationship linkage and related-record
+queries. Use the same public relationship names in plain input and JSON:API
+relationships; backing foreign-key columns are storage details.
+
+Each runnable scenario starts from a fresh database unless it explicitly says
+otherwise. Do not concatenate different chapters' resource declarations into
+one API: several deliberately reuse the same resource names.
+
+| Tutorial | What it demonstrates |
+| --- | --- |
+| [Starting point](03-running-example.md) | API, database and Express setup. |
+| [Manipulating and searching tables](04-creating-and-querying.md) | CRUD, declared filters, custom search and sparse fields. |
+| [Belongs-to relationships](05-belongs-to.md) | Public relationship input, linkage, includes, filtering and clearing. |
+| [Has-many relationships](06-has-many.md) | Reverse membership, includes, related queries, attachment and removal. |
+| [Polymorphic relationships](07-polymorphic-relationships.md) | Typed identifiers, distinct resources sharing an ID, includes and search. |
+| [Many-to-many relationships](08-many-to-many.md) | Membership and ordinary pivot metadata, with canonical-storage distinctions. |
+| [Pagination and ordering](09-pagination-and-sorting.md) | Default caps, numbered/cursor pages, sorting and sparse results. |
+| [PUT and PATCH](10-put-and-patch.md) | Explicit replacement, omitted-value rejection and relationship-object boundaries. |
+| [Relationship URLs](11-relationship-endpoints.md) | Identifier linkage versus related records, and relationship mutation endpoints. |
+
+For query projections and their SQL ordering, use
+[Query Projections](15-query-projections.md). For mandatory visibility before
+counts and pagination, use [Row Policies](17-row-policies.md) or
+[Autofiltering](16-autofiltering.md), according to the required policy.
+See [backend capabilities](30-backend-capabilities.md) for database differences.

package/docs/GUIDE/03-running-example.md

@@ -0,0 +1,87 @@
+---
+title: "Running example"
+chapter: 3
+chapter_label: "03"
+---
+
+# 03. Running example
+
+```javascript
+import { JsonRestApi, RestApiPlugin, RestApiKnexPlugin, ExpressPlugin } from 'json-rest-api';
+import knexLib from 'knex';
+import util from 'util';
+import express from 'express';
+
+// Utility used throughout this guide
+const inspect = (obj) => util.inspect(obj, { depth: 8 })
+
+// Create a Knex instance connected to SQLite in-memory database
+const knex = knexLib({
+  client: 'better-sqlite3',
+  connection: {
+    filename: ':memory:'
+  },
+  useNullAsDefault: true
+});
+
+// Create API instance
+const api = new JsonRestApi({ name: 'book-catalog-api' });
+
+// Install plugins
+await api.use(RestApiPlugin);
+await api.use(RestApiKnexPlugin, { knex });
+await api.use(ExpressPlugin, { mountPath: '/api' }); // Links are generated under /api
+
+/// *** ...programmatic calls here... ***
+
+// Create the express server and add the API's routes 
+const app = express();
+app.use(api.http.express.router);
+app.use(api.http.express.notFoundRouter);
+
+app.listen(3000, () => {
+  console.log('Express server started on port 3000. API available at http://localhost:3000/api');
+}).on('error', (err) => {
+  console.error('Failed to start server:', err);
+  process.exit(1)
+});
+```
+
+Note that every time we provide a snippet of code, it will be assumed that
+
+1. The script is edited in the section `/// *** ...programmatic calls here... ***`
+2. The code is stopped with CTRL-C and then restarted. 
+3. The code proposed in each snippet _replaces_ the code provided earlier.
+
+This will ensure that each example has a fresh start.
+
+Each example will be introduced programmatically first, and then via HTTP. The HTTP calls will be run assuming that the API calls (and any data created with them) stay. The use of the in-memory database will be assumed, which means that the data will start afresh each time.
+
+## Logical fields and storage columns
+
+Resource definitions describe logical fields and relationships. Table-backed
+storage maps those fields to physical columns: snake_case by default, explicit
+`storage.column` overrides, or exact naming when configured. Canonical storage
+uses its own allocation and descriptor mapping. A logical field name is not a
+portable physical SQL column name.
+
+For a definition such as `country_id: { type: 'id', belongsTo: 'countries',
+as: 'country' }`, callers write the relationship through
+`data: { country: countryId }`. Plain output contains a `country` object
+with its ID; an unassigned to-one relationship is omitted from plain output.
+Including the country adds its selected attributes.
+JSON:API output represents linkage under `data.relationships.country` and
+included records under `included`.
+
+Hooks operate at specific processing stages. The write pipeline converts plain
+input into its internal JSON:API representation before validation and storage.
+Do not assume every hook receives raw columns or the original public payload.
+Use the [hook guide](13-hooks-and-lifecycle.md) and the
+[API reference](../API.md) for the context available at each stage.
+
+Search schemas name the filters callers may use and map them to logical fields
+or declared related-resource fields. Raw SQL remains available through Knex,
+but code that uses physical tables/columns must account for the selected storage
+backend and does not automatically inherit resource access policies.
+
+---

package/docs/GUIDE/04-creating-and-querying.md

@@ -0,0 +1,181 @@
+---
+title: "Creating and querying records"
+chapter: 4
+chapter_label: "04"
+---
+
+# 04. Creating and querying records
+
+This example uses countries without relationships. Insert the blocks below in
+order into the [starting script](03-running-example.md), with a fresh
+database and before starting the HTTP server.
+
+## Declare the record and search schemas
+
+```javascript
+await api.addResource('countries', {
+  schema: {
+    name: { type: 'string', required: true, max: 100, search: true },
+    code: { type: 'string', max: 2, nullable: true, unique: true, search: true },
+    population: { type: 'number', nullable: true }
+  },
+  searchSchema: {
+    name: { type: 'string', filterOperator: '=' },
+    nameContains: { type: 'string', actualField: 'name', filterOperator: 'contains' },
+    populationRange: { type: 'array', actualField: 'population', filterOperator: 'between' },
+    words: {
+      type: 'string', oneOf: ['name', 'code'],
+      filterOperator: 'contains', splitBy: ' ', matchAll: true
+    },
+    nameOrCode: {
+      type: 'string',
+      applyFilter (query, input, { column }) {
+        query.where(function () {
+          this.whereRaw('LOWER(??) LIKE LOWER(?)', [column('name'), `%${input}%`])
+            .orWhereRaw('LOWER(??) = LOWER(?)', [column('code'), input])
+        })
+      }
+    }
+  }
+})
+await api.resources.countries.createKnexTable()
+```
+
+The record schema validates stored data. `searchSchema` declares public filter
+names and their meaning. An explicit search field overrides the same field
+inferred from `search: true`; other inferred fields remain available. Here,
+`name` explicitly uses equality and `code` remains searchable through its marker.
+The record's `population` field is searchable through `populationRange` only.
+
+Filter names need not be column names. `actualField` maps an alias to a logical
+resource field. `oneOf` applies a filter across several fields. Search schemas
+are an allowlist and input-validation contract, not arbitrary client-supplied SQL.
+
+## Create, patch, replace and delete records
+
+```javascript
+const france = await api.resources.countries.post({
+  data: { name: 'France', code: 'FR', population: 68 }
+})
+const italy = await api.resources.countries.post({
+  data: { name: 'Italyy', code: 'IT', population: 59 }
+})
+const germany = await api.resources.countries.post({
+  data: { name: 'Germ', code: 'DE', population: 84 }
+})
+await api.resources.countries.patch({
+  id: germany.id, data: { name: 'Germany' }, returning: 'none'
+})
+const patched = await api.resources.countries.get({ id: germany.id })
+const replaced = await api.resources.countries.put({
+  id: italy.id, data: { name: 'Italy', code: null, population: null }
+})
+console.log('Patched:', patched)
+console.log('Replaced:', replaced)
+await api.resources.countries.delete({ id: italy.id })
+```
+
+These population values are illustrative numbers for demonstrating filters.
+PATCH changes the supplied name while retaining Germany's code and population.
+PUT replaces Italy's record and explicitly clears its nullable fields. A
+replacement must include attributes that already have values; omitting them
+rejects with a complete-replacement validation error. Use null explicitly when
+the schema permits clearing a value. PUT can also create a missing target. DELETE returns
+undefined. Writes always place record attributes inside `data`.
+
+## Query and filter
+
+```javascript
+await api.resources.countries.post({
+  data: { name: 'United States', code: 'US', population: 330 }
+})
+await api.resources.countries.post({
+  data: { name: 'United Kingdom', code: 'GB', population: 67 }
+})
+await api.resources.countries.post({
+  data: { name: 'Austria', code: 'AT', population: 9 }
+})
+const exact = await api.resources.countries.query({
+  queryParams: { filters: { name: 'France' } }
+})
+const byCode = await api.resources.countries.query({
+  queryParams: { filters: { code: 'DE' } }
+})
+const contains = await api.resources.countries.query({
+  queryParams: { filters: { nameContains: 'United' }, sort: ['name'] }
+})
+const range = await api.resources.countries.query({
+  queryParams: { filters: { populationRange: [60, 90] }, sort: ['name'] }
+})
+console.log('Exact:', exact.data)
+console.log('By code:', byCode.data)
+console.log('Contains:', contains.data)
+console.log('Range:', range.data)
+```
+
+The results are France, Germany, the two United countries, and France/Germany/
+United Kingdom respectively. Query returns an object with a `data` array, not a
+bare array. Pagination metadata is supplied for explicit pagination requests;
+it is not guaranteed on every collection. Multiple filter keys combine as
+constraints. The selected operator and database collation determine string
+comparison behavior; avoid assuming all databases ignore case.
+
+## Search multiple fields and words
+
+```javascript
+const allWords = await api.resources.countries.query({
+  queryParams: { filters: { words: 'United US' } }
+})
+const custom = await api.resources.countries.query({
+  queryParams: { filters: { nameOrCode: 'at' }, sort: ['name'] }
+})
+console.log('All words:', allWords.data)
+console.log('Custom filter:', custom.data)
+```
+
+`words` splits on spaces and requires every term to match at least one of name
+or code. `United US` matches United States: one term matches its name, the other
+its code. Setting `matchAll: false` would allow any term to match.
+
+The custom filter searches a case-folded name substring or an exact case-folded
+code. It finds Austria and United States for `at`, not United Kingdom.
+`applyFilter(query, input, { column, value, context, scopeName })` runs
+synchronously and mutates the supplied Knex builder. `column(field)` translates
+logical fields and `value(field, input)` converts values for their storage
+representation. Group OR conditions so they do not bypass other filters or
+access constraints. This example binds user input as values; SQL LIKE wildcard
+characters in that input retain their SQL meaning. Custom SQL functions and
+collations must be verified on the database you deploy.
+
+## Select fields and paginate
+
+```javascript
+const page = await api.resources.countries.query({
+  queryParams: {
+    fields: { countries: 'name,code' },
+    sort: ['name'],
+    page: { number: 1, size: 2 }
+  }
+})
+console.log('Selected fields:', page.data)
+console.log('Pagination:', page.meta.pagination)
+```
+
+This returns Austria and France, with IDs, names and codes but no populations.
+The numbered pagination total is five. Fieldsets select output and do not grant
+access to hidden fields. See [pagination and ordering](09-pagination-and-sorting.md)
+for cursors and collection limits.
+
+## HTTP queries
+
+With the example server running:
+
+```bash
+curl --globoff 'http://localhost:3000/api/countries?filter[nameContains]=United'
+curl --globoff 'http://localhost:3000/api/countries?filter[words]=United%20US'
+curl --globoff 'http://localhost:3000/api/countries?fields[countries]=name,code&page[number]=1&page[size]=2&sort=name'
+```
+
+HTTP filters use `filter[...]`; programmatic queries use `queryParams.filters`.
+Generated HTTP responses are JSON:API documents. For payload changes from the
+older API, use the [migration guide](33-migrating-to-v2.md).