json-rest-api 1.0.22 → 1.0.24

package/docs/GUIDE/GUIDE_X_Knex_Schema_And_Migrations.md

@@ -0,0 +1,370 @@
+# Knex Schema and Migrations
+
+`RestApiKnexPlugin` exposes a small schema-management surface for table-backed resources.
+
+Use it when you want to:
+
+- create tables directly from resource definitions
+- add or alter plain columns on existing tables
+- inspect a live table snapshot
+- generate a create migration from a resource
+- generate an additive diff migration from a live table
+
+This is a Knex/table feature surface. It does not exist for non-table resources.
+
+## Overview
+
+Once a resource is backed by `RestApiKnexPlugin`, these scope methods are available:
+
+- `createKnexTable()`
+- `addKnexFields({ fields })`
+- `alterKnexFields({ fields })`
+- `introspectKnexTableSnapshot()`
+- `generateKnexMigration()`
+- `generateKnexMigrationDiff()`
+
+Example:
+
+```js
+await api.addResource('memberships', {
+  schema: {
+    id: { type: 'id' },
+    workspaceId: { type: 'id', required: true },
+    userId: { type: 'id', required: true },
+    role: { type: 'string', enum: ['owner', 'member'], required: true, defaultTo: 'member' }
+  },
+
+  indexes: [
+    {
+      name: 'uq_memberships_workspace_user',
+      unique: true,
+      columns: ['workspaceId', 'userId']
+    }
+  ],
+
+  foreignKeys: [
+    {
+      name: 'fk_memberships_workspace_user',
+      columns: ['workspaceId', 'userId'],
+      referencedTableName: 'workspace_users',
+      referencedColumns: ['workspace_id', 'user_id'],
+      deleteRule: 'CASCADE',
+      updateRule: 'RESTRICT'
+    }
+  ],
+
+  checkConstraints: [
+    {
+      name: 'chk_memberships_workspace_positive',
+      clause: 'workspace_id > 0'
+    }
+  ],
+
+  tableName: 'memberships'
+})
+```
+
+## Default Column Naming
+
+Table-backed resources keep logical field names in the resource schema and map them to physical columns for Knex operations.
+
+By default, physical columns use snake_case:
+
+- `workspaceId` -> `workspace_id`
+- `userId` -> `user_id`
+- `createdAt` -> `created_at`
+
+This keeps the API surface expressive without repeating `storage.column` on every camelCase field.
+
+You only need `storage.column` when a field should use a non-standard column name:
+
+```js
+await api.addResource('profiles', {
+  schema: {
+    id: { type: 'id' },
+    displayName: { type: 'string', required: true },
+    legacyRef: { type: 'string', storage: { column: 'legacy_profile_ref' } }
+  },
+  tableName: 'profiles'
+})
+```
+
+If you need physical columns to match the logical field names exactly for a whole resource, opt out explicitly:
+
+```js
+await api.addResource('verbatim_profiles', {
+  storage: { naming: 'exact' },
+  schema: {
+    id: { type: 'id' },
+    displayName: { type: 'string', required: true }
+  },
+  tableName: 'verbatim_profiles'
+})
+```
+
+## Logical IDs and `idProperty`
+
+`idProperty` names the physical primary-key column for table-backed resources. The API contract still uses the logical resource id.
+
+```js
+await api.addResource('profiles', {
+  idProperty: 'user_id',
+  schema: {
+    id: { type: 'id', required: true, storage: { column: 'user_id' } },
+    displayName: { type: 'string', required: true },
+    loginCount: { type: 'number', defaultTo: 0 }
+  },
+  tableName: 'profiles'
+})
+```
+
+With this definition:
+
+- writes send the resource id as `id` or `data.id`
+- reads return the resource id as `id` or `data.id`
+- the primary key is stored in `user_id`
+- `displayName` and `loginCount` are stored in `display_name` and `login_count`
+
+The resource id is not part of `attributes`:
+
+```js
+await api.resources.profiles.post({
+  inputRecord: {
+    data: {
+      type: 'profiles',
+      id: '42',
+      attributes: {
+        displayName: 'Mercury'
+      }
+    }
+  }
+})
+```
+
+### ID normalization
+
+If your primary keys need canonicalization beyond the default trim-and-stringify behavior, configure `normalizeId`.
+
+```js
+await api.use(RestApiPlugin, {
+  normalizeId: (value) => {
+    if (value === null || value === undefined) return null
+
+    const normalized = String(value).trim()
+    return normalized ? normalized.toUpperCase() : null
+  }
+})
+```
+
+You can override that per resource:
+
+```js
+await api.addResource('profiles', {
+  idProperty: 'user_id',
+  normalizeId: (value) => {
+    if (value === null || value === undefined) return null
+
+    const normalized = String(value).trim()
+    return normalized ? normalized.toLowerCase() : null
+  },
+  schema: {
+    id: { type: 'id', required: true, storage: { column: 'user_id' } },
+    displayName: { type: 'string', required: true }
+  },
+  tableName: 'profiles'
+})
+```
+
+For table-backed resources, `normalizeId` is applied before:
+
+- reading a record by id
+- replacing, patching, or deleting a record
+- persisting an explicit `POST` resource id
+- validating or writing relationship identifiers that point at the resource
+
+If the normalizer returns an empty value, existing-resource operations fail as `not_found`, while explicit `POST` document ids fail validation on `data.id`.
+
+## Create Tables
+
+Create the table directly from the resource definition:
+
+```js
+await api.resources.memberships.createKnexTable()
+```
+
+`createKnexTable()` understands:
+
+- storage-mapped column names
+- mapped logical id columns
+- top-level `indexes`
+- top-level `foreignKeys`
+- top-level `checkConstraints`
+- enum columns
+- MySQL-style `setValues`
+
+This is the same table metadata used by migration generation and diffing.
+
+## Field-Only Helpers
+
+Two helpers only operate on columns:
+
+```js
+await api.resources.memberships.addKnexFields({
+  fields: {
+    noteCount: { type: 'number', defaultTo: 0, storage: { column: 'note_count' } }
+  }
+})
+
+await api.resources.memberships.alterKnexFields({
+  fields: {
+    role: { type: 'string', enum: ['owner', 'member', 'viewer'] }
+  }
+})
+```
+
+Important:
+
+- these helpers are field-only
+- they reject top-level `indexes`, `foreignKeys`, and `checkConstraints`
+- use `createKnexTable()`, `generateKnexMigration()`, or `generateKnexMigrationDiff()` for table metadata
+
+## Live Table Snapshots
+
+Use `introspectKnexTableSnapshot()` to inspect the physical table shape:
+
+```js
+const snapshot = await api.resources.memberships.introspectKnexTableSnapshot()
+```
+
+Returned shape:
+
+```js
+{
+  dialect: 'sqlite',
+  schemaName: 'main',
+  tableName: 'memberships',
+  tableCollation: '',
+  idColumn: 'id',
+  primaryKeyColumns: ['id'],
+  hasWorkspaceIdColumn: true,
+  hasUserIdColumn: true,
+  columns: [...],
+  indexes: [...],
+  foreignKeys: [...],
+  checkConstraints: [...]
+}
+```
+
+Column entries are normalized and include information such as:
+
+- physical column name
+- logical key
+- type kind
+- nullability
+- default value
+- precision / scale / datetime precision
+- enum values
+- set values
+
+Current live introspection support:
+
+- SQLite
+- MySQL / MariaDB via `mysql2`
+
+## Create Migrations
+
+Use `generateKnexMigration()` to emit a Knex migration string from the resource definition:
+
+```js
+const migration = await api.resources.memberships.generateKnexMigration()
+console.log(migration)
+```
+
+This generates `exports.up` and `exports.down` for full table creation/drop.
+
+It includes the same table metadata as `createKnexTable()`:
+
+- mapped columns
+- mapped id column
+- indexes
+- foreign keys
+- check constraints
+- enum columns
+- supported MySQL-specific set columns
+
+## Diff Migrations
+
+Use `generateKnexMigrationDiff()` to compare the desired schema against the live table snapshot:
+
+```js
+const diff = await api.resources.memberships.generateKnexMigrationDiff()
+```
+
+Returned shape:
+
+```js
+{
+  migration,
+  warnings,
+  plan
+}
+```
+
+`plan` is normalized into:
+
+- `addColumns`
+- `alterColumns`
+- `dropColumns`
+- `addIndexes`
+- `dropIndexes`
+- `addForeignKeys`
+- `dropForeignKeys`
+- `addCheckConstraints`
+
+The generated migration is intentionally additive-first:
+
+- create missing columns
+- alter columns when supported
+- recreate changed indexes when needed
+- add missing foreign keys
+- add supported check constraints
+
+## Important Limits
+
+This is a schema-vs-live-table diff, not a migration-history engine.
+
+It answers:
+
+- what the table looks like now
+- what the resource says it should look like
+- what Knex migration can safely move the table toward that shape
+
+It does not answer:
+
+- what changed since a previous migration file
+- how to reconstruct migration history
+
+Other important limits:
+
+- destructive changes are surfaced as warnings
+- dropped columns are skipped unless explicitly allowed by diff options
+- SQLite check-constraint add/drop/alter support is warning-only
+- `setValues` diffing is only emitted for MySQL-compatible targets
+- diff migrations do not auto-generate a destructive `down`
+
+## When to Use Which Helper
+
+- `createKnexTable()`
+  - bootstrap a table directly from the resource
+
+- `addKnexFields()` / `alterKnexFields()`
+  - small column-only local changes
+
+- `introspectKnexTableSnapshot()`
+  - inspect the real physical table
+
+- `generateKnexMigration()`
+  - emit a full create migration from the current resource definition
+
+- `generateKnexMigrationDiff()`
+  - emit an additive migration against the live table

package/docs/GUIDE/GUIDE_X_Query_Projections.md

@@ -0,0 +1,159 @@
+# Query Projections
+
+Query projections are SQL-backed, output-only query fields provided by the optional `QueryProjectionsPlugin`. They are different from normal computed fields.
+
+If you want to build your own plugin on the same query-field seam, see [Writing Plugins](GUIDE_X_Writing_Plugins.md).
+
+- **Computed fields** run after the row is fetched, in JavaScript.
+- **Projected fields** are selected inside the query, so they can participate in `SELECT`, `ORDER BY`, and cursor pagination.
+
+Use query projection fields when a derived value must behave like a real list field.
+
+Typical cases:
+- sorting by a derived label such as `full_name`
+- cursor pagination over a derived value
+- returning a derived SQL value without storing it
+
+Do **not** use query projection fields for normal response decoration. Regular computed fields are simpler and remain the right default for that.
+
+## Setup
+
+```javascript
+import { RestApiPlugin, QueryProjectionsPlugin } from 'json-rest-api'
+
+await api.use(RestApiPlugin, { simplifiedApi: false, simplifiedTransport: false })
+await api.use(QueryProjectionsPlugin)
+```
+
+## Defining a query field
+
+```javascript
+await api.addResource('authors', {
+  schema: {
+    id: { type: 'id' },
+    first_name: { type: 'string', required: true },
+    last_name: { type: 'string', required: true }
+  },
+
+  queryFields: {
+    full_name: {
+      type: 'string',
+      sortable: true,
+      select: ({ knex, column }) => knex.raw(
+        "trim(coalesce(??, '') || ' ' || coalesce(??, ''))",
+        [column('first_name'), column('last_name')]
+      )
+    }
+  },
+
+  sortableFields: ['id', 'first_name', 'last_name'],
+  defaultSort: ['full_name']
+})
+```
+
+`select()` is called at query time. It can return:
+
+- a `knex.raw(...)` expression
+- a Knex reference
+- a Knex query/subquery expression
+
+The callback receives:
+
+```javascript
+{
+  knex,      // current knex/db handle
+  db,        // same as knex
+  context,   // request context
+  scopeName,
+  tableName,
+  fieldName,
+  schemaInfo,
+  adapter,   // storage adapter
+  column,    // helper: logical field -> qualified storage column
+  ref        // helper: logical field -> knex ref
+}
+```
+
+## Behavior
+
+Query projection fields are:
+
+- returned by default in `get()` and `query()`
+- available in sparse fieldsets
+- usable in sorting and cursor pagination
+- available on included resources
+- ignored on writes
+
+They are **not** stored columns, and they are **not** part of write validation.
+
+## Sparse fieldsets
+
+Query projection fields can be requested explicitly:
+
+```javascript
+const result = await api.resources.authors.get({
+  id: '1',
+  queryParams: {
+    fields: {
+      authors: 'first_name,full_name'
+    }
+  }
+})
+```
+
+If a query field is needed only for sorting, it is still selected internally, but it is removed from the response unless the sparse fieldset asked for it.
+
+## Sorting and cursor pagination
+
+Query projection fields are useful when a derived value must drive list behavior:
+
+```javascript
+const result = await api.resources.authors.query({
+  queryParams: {
+    sort: ['full_name'],
+    page: { size: 20 }
+  }
+})
+```
+
+`json-rest-api` automatically appends `id` as a stable tie-breaker for cursor ordering. That prevents duplicate or skipped rows when multiple records share the same projected value.
+
+## Included resources
+
+Query projection fields can be selected for included resources too:
+
+```javascript
+const result = await api.resources.books.get({
+  id: '1',
+  queryParams: {
+    include: ['author'],
+    fields: {
+      books: 'title',
+      authors: 'full_name'
+    }
+  }
+})
+```
+
+## Write behavior
+
+Query projection fields are output-only. If a client sends them in `POST`, `PUT`, or `PATCH`, they are ignored.
+
+That keeps the boundary clean:
+
+- persisted fields own storage
+- computed fields own response-layer enrichment
+- query projection fields own query-layer derivation
+
+## Limits
+
+Projected fields are an advanced query feature. They are intentionally **backend-sensitive**.
+
+Important limits:
+
+- projection SQL can be dialect-specific
+- this is not a portability layer across every SQL engine
+- query projection fields are not auto-searchable
+- if you want to sort by one, mark it `sortable: true`
+
+For display-only derived values, prefer normal computed fields from [Field Transformations](GUIDE_3_Field_Transformations.md).

package/docs/GUIDE/GUIDE_X_Writing_Plugins.md

@@ -0,0 +1,199 @@
+# Writing Plugins
+
+`json-rest-api` is built as a set of plugins on top of `hooked-api`. This guide covers the public extension surface for plugin authors.
+
+Use this guide when you want to:
+
+- add resource-level behavior across many resources
+- add new scope methods
+- compile resource metadata into reusable runtime state
+- hook into query or write processing
+
+For internal architecture notes and maintainer-oriented details, see [docs/ONBOARDING.md](../ONBOARDING.md).
+
+## Minimal plugin shape
+
+```javascript
+export const MyPlugin = {
+  name: 'my-plugin',
+  dependencies: ['rest-api'],
+
+  install ({ addHook, addScopeMethod, helpers, log, pluginOptions = {} }) {
+    addHook('scope:added', 'compile-my-plugin', {}, ({ context, scopes }) => {
+      const scope = scopes[context.scopeName]
+      scope.vars.myPlugin = { enabled: true }
+    })
+
+    addScopeMethod('doSomething', async ({ scopeName, scope, params, context }) => {
+      return { scopeName, enabled: scope.vars.myPlugin?.enabled === true }
+    })
+  }
+}
+```
+
+The usual pattern is:
+
+1. compile resource metadata during `scope:added`
+2. store normalized state in `scope.vars`
+3. use hooks or scope methods to apply behavior at runtime
+
+## The stable extension points
+
+### `scope:added`
+
+Use this to inspect `scopeOptions`, validate configuration, and compile resource-specific runtime state into `scope.vars`.
+
+Typical uses:
+
+- compile autofilter presets
+- compile query-field definitions
+- validate resource-specific options
+
+Example:
+
+```javascript
+addHook('scope:added', 'compile-example', {}, ({ context, scopes }) => {
+  const scope = scopes[context.scopeName]
+  const options = scope.scopeOptions || {}
+
+  scope.vars.example = {
+    flag: options.exampleFlag === true
+  }
+})
+```
+
+### `beforeSchemaValidate`
+
+Use this to normalize or strip write input before schema validation runs.
+
+Typical uses:
+
+- inject derived input values
+- remove output-only fields from `POST`/`PUT`/`PATCH`
+- translate alternate request forms into canonical attributes/relationships
+
+Example:
+
+```javascript
+addHook('beforeSchemaValidate', 'strip-output-only-input', {}, ({ context }) => {
+  const attributes = context.inputRecord?.data?.attributes
+  if (!attributes) return
+
+  delete attributes.output_only_field
+})
+```
+
+### `knexQueryFiltering`
+
+Use this to add query constraints to the generated Knex query.
+
+Typical uses:
+
+- automatic scoping
+- cross-table filters
+- custom public filter semantics
+
+Example:
+
+```javascript
+addHook('knexQueryFiltering', 'scope-by-workspace', {}, async ({ context }) => {
+  const query = context.knexQuery?.query
+  if (!query) return
+
+  query.where('workspace_id', context.session.workspaceId)
+})
+```
+
+`knexQueryFiltering` is the right seam for filtering. It is **not** the right seam for turning ad hoc SQL aliases into first-class fields.
+
+### `addScopeMethod`
+
+Use this when a plugin needs a reusable method on every resource or selected resources.
+
+Example:
+
+```javascript
+addScopeMethod('introspect', async ({ vars }) => {
+  return {
+    tableName: vars.schemaInfo?.tableName,
+    fields: Object.keys(vars.schemaInfo?.schemaStructure || {})
+  }
+})
+```
+
+## The query-field seam
+
+`json-rest-api` now supports a small, explicit seam for **query-only read fields**.
+
+This is the seam used by `QueryProjectionsPlugin`, and it is the recommended pattern for plugins that need derived SQL-backed fields.
+
+### What a plugin should provide
+
+Compile resource-level definitions into:
+
+```javascript
+scope.vars.queryFields = {
+  full_name: {
+    type: 'string',
+    sortable: true,
+    hidden: false,
+    normallyHidden: false,
+    select: ({ knex, db, context, scopeName, tableName, fieldName, schemaInfo, adapter, column, ref }) => {
+      return knex.raw(
+        "trim(coalesce(??, '') || ' ' || coalesce(??, ''))",
+        [column('first_name'), column('last_name')]
+      )
+    }
+  }
+}
+```
+
+### What core will do with `scope.vars.queryFields`
+
+Once a plugin sets `scope.vars.queryFields`, core will:
+
+- include visible query fields by default in `get()` and `query()`
+- allow them in sparse fieldsets
+- allow them in sorting when the plugin marks them sortable
+- use them in cursor pagination with a stable `id` tie-breaker
+- carry them through included-resource selection
+- only expose declared query-field aliases on the AnyAPI path
+
+### What this seam is for
+
+Use it for:
+
+- SQL-backed projections
+- aggregate list fields
+- joined display labels
+- derived fields that must participate in `ORDER BY`
+
+Do **not** use it for:
+
+- writable fields
+- storage/schema ownership
+- migrations or introspection
+- normal response-only computed fields
+
+For the concrete projection example, see [Query Projections](GUIDE_X_Query_Projections.md).
+
+## Recommended plugin pattern
+
+When you add a new plugin feature, prefer this flow:
+
+1. read plugin options at install time
+2. validate and compile per-resource config in `scope:added`
+3. store only normalized runtime state in `scope.vars`
+4. use hooks or scope methods to apply behavior
+
+That keeps the plugin declarative and avoids re-parsing configuration during requests.
+
+## Boundaries to keep clean
+
+- Keep **query-layer** extensions out of `schema`.
+- Keep **write/input** reshaping in `beforeSchemaValidate`.
+- Keep **filtering** in `knexQueryFiltering`.
+- Keep **response-only** enrichment in `enrichAttributes`.
+- Do not rely on arbitrary internal helper filenames as public API.
+
+If a feature requires deeper integration than the public seams provide, treat that as a core extension discussion rather than a plugin hack.