json-rest-api 1.0.22 → 1.0.23

package/docs/GUIDE/GUIDE_X_Knex_Schema_And_Migrations.md

@@ -0,0 +1,250 @@
+# Knex Schema and Migrations
+
+`RestApiKnexPlugin` exposes a small schema-management surface for table-backed resources.
+
+Use it when you want to:
+
+- create tables directly from resource definitions
+- add or alter plain columns on existing tables
+- inspect a live table snapshot
+- generate a create migration from a resource
+- generate an additive diff migration from a live table
+
+This is a Knex/table feature surface. It does not exist for non-table resources.
+
+## Overview
+
+Once a resource is backed by `RestApiKnexPlugin`, these scope methods are available:
+
+- `createKnexTable()`
+- `addKnexFields({ fields })`
+- `alterKnexFields({ fields })`
+- `introspectKnexTableSnapshot()`
+- `generateKnexMigration()`
+- `generateKnexMigrationDiff()`
+
+Example:
+
+```js
+await api.addResource('memberships', {
+  schema: {
+    id: { type: 'id' },
+    workspaceId: { type: 'id', required: true, storage: { column: 'workspace_id' } },
+    userId: { type: 'id', required: true, storage: { column: 'user_id' } },
+    role: { type: 'string', enum: ['owner', 'member'], required: true, defaultTo: 'member' }
+  },
+
+  indexes: [
+    {
+      name: 'uq_memberships_workspace_user',
+      unique: true,
+      columns: ['workspaceId', 'userId']
+    }
+  ],
+
+  foreignKeys: [
+    {
+      name: 'fk_memberships_workspace_user',
+      columns: ['workspaceId', 'userId'],
+      referencedTableName: 'workspace_users',
+      referencedColumns: ['workspace_id', 'user_id'],
+      deleteRule: 'CASCADE',
+      updateRule: 'RESTRICT'
+    }
+  ],
+
+  checkConstraints: [
+    {
+      name: 'chk_memberships_workspace_positive',
+      clause: 'workspace_id > 0'
+    }
+  ],
+
+  tableName: 'memberships'
+})
+```
+
+## Create Tables
+
+Create the table directly from the resource definition:
+
+```js
+await api.resources.memberships.createKnexTable()
+```
+
+`createKnexTable()` understands:
+
+- storage-mapped column names
+- mapped logical id columns
+- top-level `indexes`
+- top-level `foreignKeys`
+- top-level `checkConstraints`
+- enum columns
+- MySQL-style `setValues`
+
+This is the same table metadata used by migration generation and diffing.
+
+## Field-Only Helpers
+
+Two helpers only operate on columns:
+
+```js
+await api.resources.memberships.addKnexFields({
+  fields: {
+    noteCount: { type: 'number', defaultTo: 0, storage: { column: 'note_count' } }
+  }
+})
+
+await api.resources.memberships.alterKnexFields({
+  fields: {
+    role: { type: 'string', enum: ['owner', 'member', 'viewer'] }
+  }
+})
+```
+
+Important:
+
+- these helpers are field-only
+- they reject top-level `indexes`, `foreignKeys`, and `checkConstraints`
+- use `createKnexTable()`, `generateKnexMigration()`, or `generateKnexMigrationDiff()` for table metadata
+
+## Live Table Snapshots
+
+Use `introspectKnexTableSnapshot()` to inspect the physical table shape:
+
+```js
+const snapshot = await api.resources.memberships.introspectKnexTableSnapshot()
+```
+
+Returned shape:
+
+```js
+{
+  dialect: 'sqlite',
+  schemaName: 'main',
+  tableName: 'memberships',
+  tableCollation: '',
+  idColumn: 'id',
+  primaryKeyColumns: ['id'],
+  hasWorkspaceIdColumn: true,
+  hasUserIdColumn: true,
+  columns: [...],
+  indexes: [...],
+  foreignKeys: [...],
+  checkConstraints: [...]
+}
+```
+
+Column entries are normalized and include information such as:
+
+- physical column name
+- logical key
+- type kind
+- nullability
+- default value
+- precision / scale / datetime precision
+- enum values
+- set values
+
+Current live introspection support:
+
+- SQLite
+- MySQL / MariaDB via `mysql2`
+
+## Create Migrations
+
+Use `generateKnexMigration()` to emit a Knex migration string from the resource definition:
+
+```js
+const migration = await api.resources.memberships.generateKnexMigration()
+console.log(migration)
+```
+
+This generates `exports.up` and `exports.down` for full table creation/drop.
+
+It includes the same table metadata as `createKnexTable()`:
+
+- mapped columns
+- mapped id column
+- indexes
+- foreign keys
+- check constraints
+- enum columns
+- supported MySQL-specific set columns
+
+## Diff Migrations
+
+Use `generateKnexMigrationDiff()` to compare the desired schema against the live table snapshot:
+
+```js
+const diff = await api.resources.memberships.generateKnexMigrationDiff()
+```
+
+Returned shape:
+
+```js
+{
+  migration,
+  warnings,
+  plan
+}
+```
+
+`plan` is normalized into:
+
+- `addColumns`
+- `alterColumns`
+- `dropColumns`
+- `addIndexes`
+- `dropIndexes`
+- `addForeignKeys`
+- `dropForeignKeys`
+- `addCheckConstraints`
+
+The generated migration is intentionally additive-first:
+
+- create missing columns
+- alter columns when supported
+- recreate changed indexes when needed
+- add missing foreign keys
+- add supported check constraints
+
+## Important Limits
+
+This is a schema-vs-live-table diff, not a migration-history engine.
+
+It answers:
+
+- what the table looks like now
+- what the resource says it should look like
+- what Knex migration can safely move the table toward that shape
+
+It does not answer:
+
+- what changed since a previous migration file
+- how to reconstruct migration history
+
+Other important limits:
+
+- destructive changes are surfaced as warnings
+- dropped columns are skipped unless explicitly allowed by diff options
+- SQLite check-constraint add/drop/alter support is warning-only
+- `setValues` diffing is only emitted for MySQL-compatible targets
+- diff migrations do not auto-generate a destructive `down`
+
+## When to Use Which Helper
+
+- `createKnexTable()`
+  - bootstrap a table directly from the resource
+
+- `addKnexFields()` / `alterKnexFields()`
+  - small column-only local changes
+
+- `introspectKnexTableSnapshot()`
+  - inspect the real physical table
+
+- `generateKnexMigration()`
+  - emit a full create migration from the current resource definition
+
+- `generateKnexMigrationDiff()`
+  - emit an additive migration against the live table

package/docs/GUIDE/GUIDE_X_Query_Projections.md

@@ -0,0 +1,159 @@
+# Query Projections
+
+Query projections are SQL-backed, output-only query fields provided by the optional `QueryProjectionsPlugin`. They are different from normal computed fields.
+
+If you want to build your own plugin on the same query-field seam, see [Writing Plugins](GUIDE_X_Writing_Plugins.md).
+
+- **Computed fields** run after the row is fetched, in JavaScript.
+- **Projected fields** are selected inside the query, so they can participate in `SELECT`, `ORDER BY`, and cursor pagination.
+
+Use query projection fields when a derived value must behave like a real list field.
+
+Typical cases:
+- sorting by a derived label such as `full_name`
+- cursor pagination over a derived value
+- returning a derived SQL value without storing it
+
+Do **not** use query projection fields for normal response decoration. Regular computed fields are simpler and remain the right default for that.
+
+## Setup
+
+```javascript
+import { RestApiPlugin, QueryProjectionsPlugin } from 'json-rest-api'
+
+await api.use(RestApiPlugin, { simplifiedApi: false, simplifiedTransport: false })
+await api.use(QueryProjectionsPlugin)
+```
+
+## Defining a query field
+
+```javascript
+await api.addResource('authors', {
+  schema: {
+    id: { type: 'id' },
+    first_name: { type: 'string', required: true },
+    last_name: { type: 'string', required: true }
+  },
+
+  queryFields: {
+    full_name: {
+      type: 'string',
+      sortable: true,
+      select: ({ knex, column }) => knex.raw(
+        "trim(coalesce(??, '') || ' ' || coalesce(??, ''))",
+        [column('first_name'), column('last_name')]
+      )
+    }
+  },
+
+  sortableFields: ['id', 'first_name', 'last_name'],
+  defaultSort: ['full_name']
+})
+```
+
+`select()` is called at query time. It can return:
+
+- a `knex.raw(...)` expression
+- a Knex reference
+- a Knex query/subquery expression
+
+The callback receives:
+
+```javascript
+{
+  knex,      // current knex/db handle
+  db,        // same as knex
+  context,   // request context
+  scopeName,
+  tableName,
+  fieldName,
+  schemaInfo,
+  adapter,   // storage adapter
+  column,    // helper: logical field -> qualified storage column
+  ref        // helper: logical field -> knex ref
+}
+```
+
+## Behavior
+
+Query projection fields are:
+
+- returned by default in `get()` and `query()`
+- available in sparse fieldsets
+- usable in sorting and cursor pagination
+- available on included resources
+- ignored on writes
+
+They are **not** stored columns, and they are **not** part of write validation.
+
+## Sparse fieldsets
+
+Query projection fields can be requested explicitly:
+
+```javascript
+const result = await api.resources.authors.get({
+  id: '1',
+  queryParams: {
+    fields: {
+      authors: 'first_name,full_name'
+    }
+  }
+})
+```
+
+If a query field is needed only for sorting, it is still selected internally, but it is removed from the response unless the sparse fieldset asked for it.
+
+## Sorting and cursor pagination
+
+Query projection fields are useful when a derived value must drive list behavior:
+
+```javascript
+const result = await api.resources.authors.query({
+  queryParams: {
+    sort: ['full_name'],
+    page: { size: 20 }
+  }
+})
+```
+
+`json-rest-api` automatically appends `id` as a stable tie-breaker for cursor ordering. That prevents duplicate or skipped rows when multiple records share the same projected value.
+
+## Included resources
+
+Query projection fields can be selected for included resources too:
+
+```javascript
+const result = await api.resources.books.get({
+  id: '1',
+  queryParams: {
+    include: ['author'],
+    fields: {
+      books: 'title',
+      authors: 'full_name'
+    }
+  }
+})
+```
+
+## Write behavior
+
+Query projection fields are output-only. If a client sends them in `POST`, `PUT`, or `PATCH`, they are ignored.
+
+That keeps the boundary clean:
+
+- persisted fields own storage
+- computed fields own response-layer enrichment
+- query projection fields own query-layer derivation
+
+## Limits
+
+Projected fields are an advanced query feature. They are intentionally **backend-sensitive**.
+
+Important limits:
+
+- projection SQL can be dialect-specific
+- this is not a portability layer across every SQL engine
+- query projection fields are not auto-searchable
+- if you want to sort by one, mark it `sortable: true`
+
+For display-only derived values, prefer normal computed fields from [Field Transformations](GUIDE_3_Field_Transformations.md).

package/docs/GUIDE/GUIDE_X_Writing_Plugins.md

@@ -0,0 +1,199 @@
+# Writing Plugins
+
+`json-rest-api` is built as a set of plugins on top of `hooked-api`. This guide covers the public extension surface for plugin authors.
+
+Use this guide when you want to:
+
+- add resource-level behavior across many resources
+- add new scope methods
+- compile resource metadata into reusable runtime state
+- hook into query or write processing
+
+For internal architecture notes and maintainer-oriented details, see [docs/ONBOARDING.md](../ONBOARDING.md).
+
+## Minimal plugin shape
+
+```javascript
+export const MyPlugin = {
+  name: 'my-plugin',
+  dependencies: ['rest-api'],
+
+  install ({ addHook, addScopeMethod, helpers, log, pluginOptions = {} }) {
+    addHook('scope:added', 'compile-my-plugin', {}, ({ context, scopes }) => {
+      const scope = scopes[context.scopeName]
+      scope.vars.myPlugin = { enabled: true }
+    })
+
+    addScopeMethod('doSomething', async ({ scopeName, scope, params, context }) => {
+      return { scopeName, enabled: scope.vars.myPlugin?.enabled === true }
+    })
+  }
+}
+```
+
+The usual pattern is:
+
+1. compile resource metadata during `scope:added`
+2. store normalized state in `scope.vars`
+3. use hooks or scope methods to apply behavior at runtime
+
+## The stable extension points
+
+### `scope:added`
+
+Use this to inspect `scopeOptions`, validate configuration, and compile resource-specific runtime state into `scope.vars`.
+
+Typical uses:
+
+- compile autofilter presets
+- compile query-field definitions
+- validate resource-specific options
+
+Example:
+
+```javascript
+addHook('scope:added', 'compile-example', {}, ({ context, scopes }) => {
+  const scope = scopes[context.scopeName]
+  const options = scope.scopeOptions || {}
+
+  scope.vars.example = {
+    flag: options.exampleFlag === true
+  }
+})
+```
+
+### `beforeSchemaValidate`
+
+Use this to normalize or strip write input before schema validation runs.
+
+Typical uses:
+
+- inject derived input values
+- remove output-only fields from `POST`/`PUT`/`PATCH`
+- translate alternate request forms into canonical attributes/relationships
+
+Example:
+
+```javascript
+addHook('beforeSchemaValidate', 'strip-output-only-input', {}, ({ context }) => {
+  const attributes = context.inputRecord?.data?.attributes
+  if (!attributes) return
+
+  delete attributes.output_only_field
+})
+```
+
+### `knexQueryFiltering`
+
+Use this to add query constraints to the generated Knex query.
+
+Typical uses:
+
+- automatic scoping
+- cross-table filters
+- custom public filter semantics
+
+Example:
+
+```javascript
+addHook('knexQueryFiltering', 'scope-by-workspace', {}, async ({ context }) => {
+  const query = context.knexQuery?.query
+  if (!query) return
+
+  query.where('workspace_id', context.session.workspaceId)
+})
+```
+
+`knexQueryFiltering` is the right seam for filtering. It is **not** the right seam for turning ad hoc SQL aliases into first-class fields.
+
+### `addScopeMethod`
+
+Use this when a plugin needs a reusable method on every resource or selected resources.
+
+Example:
+
+```javascript
+addScopeMethod('introspect', async ({ vars }) => {
+  return {
+    tableName: vars.schemaInfo?.tableName,
+    fields: Object.keys(vars.schemaInfo?.schemaStructure || {})
+  }
+})
+```
+
+## The query-field seam
+
+`json-rest-api` now supports a small, explicit seam for **query-only read fields**.
+
+This is the seam used by `QueryProjectionsPlugin`, and it is the recommended pattern for plugins that need derived SQL-backed fields.
+
+### What a plugin should provide
+
+Compile resource-level definitions into:
+
+```javascript
+scope.vars.queryFields = {
+  full_name: {
+    type: 'string',
+    sortable: true,
+    hidden: false,
+    normallyHidden: false,
+    select: ({ knex, db, context, scopeName, tableName, fieldName, schemaInfo, adapter, column, ref }) => {
+      return knex.raw(
+        "trim(coalesce(??, '') || ' ' || coalesce(??, ''))",
+        [column('first_name'), column('last_name')]
+      )
+    }
+  }
+}
+```
+
+### What core will do with `scope.vars.queryFields`
+
+Once a plugin sets `scope.vars.queryFields`, core will:
+
+- include visible query fields by default in `get()` and `query()`
+- allow them in sparse fieldsets
+- allow them in sorting when the plugin marks them sortable
+- use them in cursor pagination with a stable `id` tie-breaker
+- carry them through included-resource selection
+- only expose declared query-field aliases on the AnyAPI path
+
+### What this seam is for
+
+Use it for:
+
+- SQL-backed projections
+- aggregate list fields
+- joined display labels
+- derived fields that must participate in `ORDER BY`
+
+Do **not** use it for:
+
+- writable fields
+- storage/schema ownership
+- migrations or introspection
+- normal response-only computed fields
+
+For the concrete projection example, see [Query Projections](GUIDE_X_Query_Projections.md).
+
+## Recommended plugin pattern
+
+When you add a new plugin feature, prefer this flow:
+
+1. read plugin options at install time
+2. validate and compile per-resource config in `scope:added`
+3. store only normalized runtime state in `scope.vars`
+4. use hooks or scope methods to apply behavior
+
+That keeps the plugin declarative and avoids re-parsing configuration during requests.
+
+## Boundaries to keep clean
+
+- Keep **query-layer** extensions out of `schema`.
+- Keep **write/input** reshaping in `beforeSchemaValidate`.
+- Keep **filtering** in `knexQueryFiltering`.
+- Keep **response-only** enrichment in `enrichAttributes`.
+- Do not rely on arbitrary internal helper filenames as public API.
+
+If a feature requires deeper integration than the public seams provide, treat that as a core extension discussion rather than a plugin hack.

package/docs/GUIDE/index.md

@@ -15,16 +15,13 @@ Welcome to the comprehensive guide for JSON REST API. This guide will walk you t
 3. **[Field Transformations](GUIDE_3_Field_Transformations.md)**  
    Transform data with virtual fields, getters, setters, computed fields, and visibility control.
 
-4. **[Authentication and Permissions](GUIDE_4_Authentication_And_Permissions.md)**  
-   Implement security with authentication strategies and fine-grained permissions.
-
-5. **[Non-Database Resources](GUIDE_5_Non-Db_Resources.md)**  
+4. **[Non-Database Resources](GUIDE_5_Non-Db_Resources.md)**  
    Connect your API to alternative data sources beyond traditional databases.
 
-6. **[Positioning](GUIDE_6_Positioning.md)**  
+5. **[Positioning](GUIDE_6_Positioning.md)**  
    Implement drag-and-drop functionality with automatic position management.
 
-7. **[Hooks, Data Management, and Plugins](GUIDE_7_Hooks_Data_Management_And_Plugins.md)**  
+6. **[Hooks, Data Management, and Plugins](GUIDE_7_Hooks_Data_Management_And_Plugins.md)**  
    Master the hook system for data validation, transformation, and business logic.
 
 
@@ -39,8 +36,20 @@ Welcome to the comprehensive guide for JSON REST API. This guide will walk you t
 - **[CORS Configuration](GUIDE_X_Cors.md)**  
   Set up Cross-Origin Resource Sharing for browser-based applications.
 
-- **[Multihome Support](GUIDE_X_Multihome.md)**  
-  Run multiple API instances or serve different domains from a single server.
+- **[Autofiltering](GUIDE_X_Autofiltering.md)**  
+  Scope datasets automatically using configured context values, without baking in auth semantics.
+
+- **[Knex Schema and Migrations](GUIDE_X_Knex_Schema_And_Migrations.md)**  
+  Create tables, inspect live table snapshots, and generate create or additive diff migrations from table-backed resource scopes.
+
+- **[Query Projections](GUIDE_X_Query_Projections.md)**  
+  Use the optional query-projections plugin to add SQL-backed, output-only derived fields that can participate in sorting and cursor pagination.
+
+- **[Writing Plugins](GUIDE_X_Writing_Plugins.md)**  
+  Build plugins against the supported extension surface, including hooks, scope methods, and query-only read fields.
+
+- **[Fastify Integration](GUIDE_X_Fastify.md)**  
+  Register REST routes on Fastify and reject malformed write payloads at the transport layer using exported resource schemas.
 
 - **[Socket.IO Integration](GUIDE_X_SocketIO.md)**  
   Add real-time capabilities to your API with WebSocket support.
@@ -61,4 +70,4 @@ Before starting this guide, you should have:
 
 ---
 
-Ready to get started? Head to [Chapter 1: Initial Setup](GUIDE_1_Initial_Setup.md) →
+Ready to get started? Head to [Chapter 1: Initial Setup](GUIDE_1_Initial_Setup.md) →

package/docs/QUICKSTART.md

@@ -103,6 +103,10 @@ await api.addResource('authors', {
 await api.resources.authors.createKnexTable();
 ```
 
+`searchSchema` defines the public filter names your API accepts. Those keys do not need to match database columns or even resource field names. Keep the quickstart mental model simple: `searchSchema` is the resource's public filter contract. For advanced mapping, multi-field search, and custom backend filtering, see [Manipulating and Searching Tables](GUIDE/GUIDE_2_2_Manipulating_And_Searching_Tables.md).
+
+If you want to inspect the physical table shape or generate Knex migrations directly from your resource definitions, see [Knex Schema and Migrations](GUIDE/GUIDE_X_Knex_Schema_And_Migrations.md).
+
 ## Run the program
 
 ```bash
@@ -332,7 +336,7 @@ You've successfully set up a basic JSON REST API! This quickstart covered the es
 
 Check out the [full guide](./GUIDE) to learn about:
 - Advanced relationships (many-to-many, polymorphic)
-- Authentication and authorization
+- Permission hooks, request auth context, and autofiltering
 - Hooks and middleware
 - Custom validation
 - Pagination strategies

package/docs/index.md

@@ -27,7 +27,7 @@ JSON REST API is a lightweight, plugin-based framework that makes building REST
   
   <div class="feature-card">
     <h3>🔌 Plugin Architecture</h3>
-    <p>Extend your API with powerful plugins. Authentication, validation, CORS, and more - just plug and play.</p>
+    <p>Extend your API with focused plugins for transports, dataset scoping, query fields, CORS, and more.</p>
   </div>
   
   <div class="feature-card">

package/index.js

@@ -10,9 +10,10 @@
 
 // Core plugins
 export { RestApiPlugin } from './plugins/core/rest-api-plugin.js'
+export { AutoFilterPlugin } from './plugins/core/rest-api-autofilter-plugin.js'
+export { QueryProjectionsPlugin } from './plugins/core/rest-api-query-projections-plugin.js'
 export { FileHandlingPlugin } from './plugins/core/file-handling-plugin.js'
 export { CorsPlugin } from './plugins/core/rest-api-cors-plugin.js'
-export { AccessPlugin } from './plugins/core/rest-api-access.js'
 export { LabelPlugin } from './plugins/core/rest-api-label-plugin.js'
 
 // Database plugins
@@ -21,7 +22,7 @@ export { RestApiAnyapiKnexPlugin } from './plugins/core/rest-api-anyapi-knex-plu
 
 // Connector plugins
 export { ExpressPlugin } from './plugins/core/connectors/express-plugin.js'
-// Future: export { FastifyPlugin } from './plugins/core/connectors/fastify-plugin.js';
+export { FastifyPlugin } from './plugins/core/connectors/fastify-plugin.js'
 // Future: export { KoaPlugin } from './plugins/core/connectors/koa-plugin.js';
 
 // Storage plugins for file handling