json-rest-api 1.0.21 → 1.0.23

package/CLAUDE.md

@@ -71,7 +71,7 @@ This separation ensures backend developers have full control and visibility into
    - `url-helpers.js` - Centralized URL generation functions
    - `knex-json-api-transformers-querying.js` - JSON:API transformations
    - `relationship-processor.js` - Handles relationship updates
-   - `payload-validators.js` - Request validation
+   - `request-contracts.js` - Compiled request validation contracts
    - `knex-query-helpers.js` - Database query building
    - `simplified-helpers.js` - Simplified API mode transformations
 
@@ -158,4 +158,3 @@ const book = await api.resources.books.post({
 3. **ID Type Confusion**: SQLite returns numeric IDs, but JSON:API expects strings
 4. **Context Placement**: Context is a separate parameter, not inside the first parameter
 5. **Cleanup**: Always destroy knex connection in `after()` hook
-

package/JSKIT_BACKEND_MIGRATION_TRACKER.txt

@@ -0,0 +1,180 @@
+JSKIT Backend Migration Tracker
+
+Scope
+- Move DB, schema, casting, and normalization ownership out of jskit-ai.
+- Enrich json-rest-schema and json-rest-api so they can natively cover the required capability set.
+- Do not add compatibility runtimes.
+- Do not bend json-rest-api or json-rest-schema around jskit-ai shapes unless the capability is a clean native addition.
+
+Status Legend
+- done
+- in_progress
+- pending
+- blocked
+
+Execution Rule
+- Complete one numbered point at a time.
+- After each point, write a full report:
+  - what changed
+  - how it worked before
+  - how it works now
+  - relevant code snippets
+- Stop and wait for the next user prompt before starting the next numbered point.
+
+Work List
+
+0. [done] Create and maintain this tracker as the canonical work ledger.
+   Notes:
+   - This file is the progress source of truth for the migration work.
+   - Status changes must be recorded here before moving to the next point.
+
+1. [done] Tighten json-rest-schema scalar semantics.
+   Deliverables:
+   - make `id` represent the real external/resource ID contract
+   - tighten boolean parsing
+   - tighten number/integer parsing where schema should reject invalid inputs instead of silently coercing
+
+2. [done] Define a native json-rest-schema create/replace/patch validation contract.
+   Deliverables:
+   - explicit operation mode semantics
+   - presence-aware field handling
+   - patch behavior that preserves omission and only normalizes touched fields
+   - switch json-rest-api call sites to the explicit schema contract
+   Notes:
+   - verification blocker removed by deleting stale DB-helper tests from `json-rest-schema`
+   - DB-helper coverage now lives in `json-rest-api`, which owns `createKnexTable()` and `generateKnexMigration()`
+   - verified end-to-end against a local symlinked `json-rest-schema@1.0.13`; `json-rest-api` test suite passes with the new schema contract
+
+3. [done] Add transport-schema export from json-rest-schema.
+   Deliverables:
+   - export usable by JSON Schema / Ajv style consumers
+   - enough structure for pre-handler validation in connectors such as Fastify
+   Notes:
+   - added `schema.toJsonSchema({ mode, additionalProperties })` in `json-rest-schema`
+   - export is draft-07, mode-aware, and transport-facing rather than a clone of every in-process coercion path
+   - custom transport export hooks are supported through `handler.toJsonSchema`
+   - unknown external metadata keys are ignored; custom registered validators without a transport hook fail loudly
+
+4. [done] Extend json-rest-schema field metadata only where it remains schema-level meaning.
+   Deliverables:
+   - preserve useful passive metadata such as nullability, defaults, precision, scale, unsigned, enum values, and temporal precision
+   Notes:
+   - made `enum` a first-class built-in rule in `json-rest-schema`
+   - added canonical passive metadata support for `temporalPrecision`
+   - transport export now preserves passive schema metadata under `x-json-rest-schema.metadata`
+   - passive metadata remains schema-owned metadata; no DB-specific runtime logic was added to `json-rest-schema`
+
+5. [done] Add repository/storage mapping in json-rest-api.
+   Deliverables:
+   - API field to DB column mapping
+   - virtual/non-column fields
+   - write serializers where persistence formatting is required
+   Notes:
+   - added `field.storage = { column, serialize }` as the native repository/storage surface in `json-rest-api`
+   - storage metadata is compiled once into `schemaInfo.storageInfo`
+   - Knex writes now translate logical attributes to storage columns through the compiled mapping
+   - table creation and migration generation now respect storage column overrides
+   - JSON:API transformers translate storage rows back to logical field names before response shaping
+   - storage adapters now own row/id/select translation used by Knex callers, reducing repeated mapping helpers
+   - runtime row access now uses the same logical field path for `id` as for other fields: `getFieldValue(record, 'id')`
+   - include handling and relationship links now use logical record IDs consistently, including custom-id resources
+   - direct DB helper surfaces now support logical `id` fields mapped to custom storage columns without duplicate-column drift
+   - verified with dedicated storage-mapping tests and the full `json-rest-api` suite
+
+6. [done] Add DB introspection snapshot support in json-rest-api.
+   Deliverables:
+   - column type and shape introspection
+   - enum/set values
+   - indexes
+   - foreign keys
+   - check constraints
+   - precision/scale/temporal precision
+   Notes:
+   - added `introspectKnexTableSnapshot(knex, { tableName, idColumn })` as the native normalized snapshot helper
+   - snapshot support is dialect-aware for `better-sqlite3`/SQLite and `mysql2`
+   - SQLite captures named check constraints and foreign keys from table SQL, while indexes/columns/foreign-key actions come from PRAGMA metadata
+   - `RestApiKnexPlugin` now exposes `api.resources.{resource}.introspectKnexTableSnapshot()` for table-backed resources
+   - verified with real SQLite snapshot tests, MySQL raw-double snapshot tests, and a public scope-method test
+
+7. [done] Add deterministic migration generation and diffing in json-rest-api.
+   Deliverables:
+   - create and alter support
+   - named/composite indexes
+   - FK rules
+   - enum/set handling
+   - defaults and safety warnings for destructive changes
+   Notes:
+   - `dbTablesOperations` now normalizes table-level metadata (`indexes`, `foreignKeys`, `checkConstraints`) and uses one contract for live table creation, create-migration generation, and snapshot diffing
+   - field-level `unique` / `index` / `references` are compiled into deterministic named indexes and foreign keys instead of relying on ad hoc column builder side effects
+   - added `generateKnexMigrationDiff(tableName, currentSnapshot, schema, options)` returning `{ migration, warnings, plan }`
+   - additive diff generation supports columns, indexes, and foreign keys; destructive or unsupported changes stay explicit as warnings
+   - `RestApiKnexPlugin` now exposes `generateKnexMigration()` and `generateKnexMigrationDiff()` on table-backed scopes and threads table-level metadata through `schemaInfo`
+
+8. [done] Preserve and clarify the pass-through searchSchema contract in json-rest-api.
+   Deliverables:
+   - searchSchema validates allowed filter keys and normalizes values only
+   - filter meaning remains backend-owned and server-defined
+   - arbitrary public filter keys are supported when the backend maps them intentionally
+   - regression coverage for shared merge/validation behavior and invalid filter keys
+   Notes:
+   - did not introduce a generic filter DSL
+   - documented that searchSchema keys are public filter names, not a portable operator language
+   - verified that alias filters (`oneOf`) and custom backend filters (`applyFilter`) work as intended in the Knex path, where their execution semantics are owned by the backend
+
+9. [done] Add declarative autofiltering/scoping in json-rest-api.
+   Deliverables:
+   - public
+   - workspace
+   - user
+   - workspace_user
+   Notes:
+   - replaced the purpose-specific multihome surface with a generic `AutoFilterPlugin`
+   - resources now declare `autofilter` presets or inline filters, while the plugin resolves scope values from configured resolver functions
+   - `workspace`, `user`, `workspace_user`, and `public` are example presets, not built-in auth meanings
+   - the plugin scopes collection queries and single-record lookups, stamps scoped fields on `POST`/`PUT`, and rejects inconsistent scoped updates
+   - scoped `belongsTo` relationships are supported by injecting or validating JSON:API relationship identifiers before write validation
+   - auth/admission semantics remain out of scope for `json-rest-api`; core keeps `checkPermissions` hooks as a neutral extension point only
+   - shipped `AccessPlugin` was removed before this point so scoping does not inherit built-in auth assumptions
+
+10. [done] Add SQL-backed query projections and stable list behavior in json-rest-api.
+    Deliverables:
+    - SQL-backed output-only query fields via an optional plugin
+    - sparse-field and include support for declared query fields
+    - deterministic ordered-list / cursor behavior with stable tie-breakers
+    Notes:
+    - added an optional `QueryProjectionsPlugin` that compiles resource-level `queryFields`, separate from normal schema fields and response-layer computed fields
+    - query fields are selected by default, can be requested in sparse fieldsets, and are stripped from write input
+    - query fields can drive sorting and cursor pagination; `id` is appended as a deterministic tie-breaker
+    - query-field selection is supported on the Knex path and the AnyAPI Knex path, including included resources
+    - documented as a separate advanced guide so it does not blur with response-layer computed fields
+
+11. [done] Add a Fastify-oriented connector layer in json-rest-api.
+    Deliverables:
+    - consume exported schema before request handling
+    - no jskit-ai compatibility shim
+    Notes:
+    - added `FastifyPlugin` as a transport connector that registers REST routes on a supplied Fastify instance
+    - resource `POST`/`PUT`/`PATCH` routes now attach transport-facing JSON Schemas built from `schema.toJsonSchema({ mode })`, wrapped in the JSON:API document envelope
+    - connector-level transport schemas strip output-only/computed fields and direct `belongsTo` foreign keys from `data.attributes`, matching the existing transport write contract
+    - connector registers an `application/vnd.api+json` parser and rejects unsupported write content types before route handlers run
+    - no jskit-ai compatibility shim or duplicate hand-written field validation layer was added
+
+12. [pending] Remove the corresponding DB/schema/normalization ownership from jskit-ai and verify parity.
+    Deliverables:
+    - no jskit-ai-owned DB/schema normalization path remains for the migrated surface
+    - verification notes for anything still intentionally left outside scope
+
+ADDENDUM ITEMS TO POINT 12:
+
+* This is the reason we did points 1 to 11 (!).
+* Needs to be done with a lot of care, since it's harder than it looks 
+* You MUST read AGENTS.md in jskit-ai VERY carefully, follow it VERY carefully. One of the previous attempts (4 failed in total) was unsuccessful because this step was missed. That file has important instructions on how to avoid repeated helpers and a pointer to important function maps
+* There must be NOTHING left of the in house DB stuff. jskit is only a user of what json-rest-api offers.
+* NO SHIMS. NO COMPATIBILITY LAYERS. NO LAYERS AT ALL. NO RUNTIMES.
+* In an earlier attempt, you made a runtime that mimiked the repository's functions. NO. Repository functions will use json-rest-api's functions to do what they need to do
+* A repository should be simple. It can have helper functions, but if it has a LOT of helper functions, there is a problem -- interrupt and let's talk. This happened in the workspace repository
+* Multi homing is to be done with the generic filtering option of json-rest-api
+* Take a LOT of care with the definiton of the resource file. Do NOT keep the old Type + Normalisation flow. That is GONE. We now have json-rest-schema, and that's what you need to use. One of the 4 failed attempts failed because you were stubborn about reusing that paradigm
+* The resource file (which is what is used by the server, AND by the client) needs to be clean, and contain the right information (schemas, relationships, etc). It's a DATA ONLY file.
+* The goal is to devoid jskit of any db logic/layering/
+

package/SINGLE_VALIDATION_CONTRACT_TODO.txt

@@ -0,0 +1,172 @@
+Single Validation Contract TODO
+
+Goal
+- Make `json-rest-schema` the single authored validation contract in `json-rest-api`.
+- Keep plain JSON Schema only as a derived transport artifact for connectors.
+- Remove the split between:
+  - resource/search validation via `json-rest-schema`
+  - hand-written JSON:API/query validators in `plugins/core/lib/querying-writing/payload-validators.js`
+
+Current audit status
+- [done] Audit the live validation paths and identify the actual indirection.
+
+Root cause
+- Validation is split across three places:
+  1. `plugins/core/lib/querying-writing/compile-schemas.js`
+     - compiles resource and search schemas with `createSchema(...)`
+  2. `plugins/core/lib/querying-writing/payload-validators.js`
+     - manually validates JSON:API request envelopes and query parameter objects
+  3. `plugins/core/connectors/lib/transport-route-schemas.js`
+     - separately builds connector-facing JSON Schemas from `schemaInfo`
+- This duplicates contract meaning across runtime and transport.
+- Route methods still call imperative payload validators before calling
+  `schemaInstance.create()`, `schemaInstance.replace()`, or `schemaInstance.patch()`.
+- `setupCommonRequest()` in `plugins/core/rest-api-plugin-methods/common.js`
+  also repeats some checks that are already transport-contract concerns.
+
+What the neat end state looks like
+- Authored contract:
+  - `json-rest-schema` only
+- Derived transport schema:
+  - plain JSON Schema, generated from the same compiled contract
+- Runtime write validation:
+  - uses the same compiled contract as Fastify pre-validation
+- Remaining imperative checks:
+  - only stateful/business checks that are not schema concerns
+  - examples: permission checks, record existence, relationship access, complete-replace semantics
+
+What should stay imperative
+- `validateRelationshipAccess()` in `plugins/core/rest-api-plugin-methods/common.js`
+- `validateCompleteReplacePayload()` in `plugins/core/rest-api-plugin-methods/common.js`
+- path/body cross-source checks such as URL id vs body id consistency
+- permission and data existence checks
+
+What should stop existing
+- `validateGetPayload()`
+- `validateQueryPayload()`
+- `validatePostPayload()`
+- `validatePutPayload()`
+- `validatePatchPayload()`
+- duplicate route/body/query syntax checks spread across route methods and `setupCommonRequest()`
+
+Work list
+
+1. [done] Define one compiled request-contract surface for resource `get/query/post/put/patch`.
+   Target files:
+   - `plugins/core/lib/querying-writing/compile-schemas.js`
+   - `plugins/core/connectors/lib/transport-route-schemas.js`
+   Deliverables:
+   - compile request contracts once per scope
+   - store them on `scope.vars.schemaInfo` or an adjacent compiled-contract object
+   - separate:
+     - write document contracts (`post`, `put`, `patch`)
+     - query/get contracts (`get`, `query`)
+     - relationship route contracts
+   Status:
+   - resource route contracts now compile once in `request-contracts.js`
+   - relationship route contracts are still separate and remain pending work
+
+2. [done] Replace manual write-document validation with compiled schema contracts.
+   Target files:
+   - `plugins/core/lib/querying-writing/payload-validators.js`
+   - `plugins/core/rest-api-plugin-methods/post.js`
+   - `plugins/core/rest-api-plugin-methods/put.js`
+   - `plugins/core/rest-api-plugin-methods/patch.js`
+   Deliverables:
+   - one compiled JSON:API document contract per write mode
+   - `data.type` constrained to the scope name inside the contract
+   - `data.id` required/optional according to mode
+   - `attributes` contract derived from `schemaInstance.toJsonSchema({ mode })`
+   - relationship identifier contracts derived from `schemaRelationships` and `belongsTo` metadata
+   - `included` rules expressed once in the contract rather than repeated manually
+
+3. [done] Replace manual GET/query parameter validation with compiled schema contracts.
+   Target files:
+   - `plugins/core/lib/querying-writing/payload-validators.js`
+   - `plugins/core/rest-api-plugin-methods/get.js`
+   - `plugins/core/rest-api-plugin-methods/query.js`
+   Deliverables:
+   - a compiled `get` contract for:
+     - `id`
+     - `queryParams.include`
+     - `queryParams.fields`
+   - a compiled `query` contract for:
+     - `include`
+     - `fields`
+     - `filters`
+     - `sort`
+     - `page`
+   - include-depth validation and sortable-field restrictions represented in the compiled contract, not duplicated in ad hoc code
+   Notes:
+   - `searchSchemaInstance` should remain the value contract for `filters`
+   - the query contract should validate the envelope/shape, not duplicate filter semantics
+
+4. [done] Collapse Fastify pre-validation and in-process validation onto the same source for resource write routes.
+   Target files:
+   - `plugins/core/connectors/lib/transport-route-schemas.js`
+   - `plugins/core/connectors/fastify-plugin.js`
+   - route methods in `plugins/core/rest-api-plugin-methods/`
+   Deliverables:
+   - Fastify consumes transport JSON Schema exported from the compiled request contract
+   - route methods consume the same compiled contract directly for runtime validation
+   - no second hand-written envelope contract path
+
+5. [done] Shrink `setupCommonRequest()` to non-schema responsibilities.
+   Target file:
+   - `plugins/core/rest-api-plugin-methods/common.js`
+   Deliverables:
+   - keep simplified-mode conversion, transaction setup, context wiring
+   - keep stateful/business checks only
+   - remove syntax/envelope checks that belong in the compiled contract
+   - keep only cross-input checks that a single schema cannot express cleanly
+
+6. [done] Delete `payload-validators.js`.
+   Target file:
+   - `plugins/core/lib/querying-writing/payload-validators.js`
+   Deliverables:
+   - no route method imports it anymore
+   - helper logic worth keeping is either:
+     - absorbed into compiled contracts
+     - or moved into narrowly named stateful helpers
+
+7. [done] Tighten tests around the single-contract flow.
+   Target areas:
+   - `tests/fastify-plugin.test.js`
+   - route method coverage exercised by the existing REST suites
+   - add one focused contract test suite if needed
+   Deliverables:
+   - prove Fastify pre-validation and runtime validation reject the same bad inputs
+   - cover:
+     - wrong `data.type`
+     - missing/invalid `data.id`
+     - invalid relationships
+     - invalid query/include/page/sort shapes
+   - verify there is no drift between transport schema and runtime schema behavior
+
+8. [done] Proof cleanup.
+   Deliverables:
+   - `rg` proof that route methods no longer import `payload-validators.js`
+   - `rg` proof that `validateGetPayload`, `validateQueryPayload`, `validatePostPayload`,
+     `validatePutPayload`, and `validatePatchPayload` are gone
+   - `npm test`
+
+Current state
+- Resource `get/query/post/put/patch` now use one compiled contract source.
+- Fastify transport schemas for those resource routes are derived from the same compiled contract.
+- `payload-validators.js` is deleted.
+- `npm test` passes.
+- The main remaining simplification is the separate relationship-route contract path.
+
+File map for the next implementation pass
+- Remaining relationship-route cleanup:
+  - `plugins/core/rest-api-plugin-methods/post-relationship.js`
+  - `plugins/core/rest-api-plugin-methods/patch-relationship.js`
+  - `plugins/core/rest-api-plugin-methods/delete-relationship.js`
+- Transport schema generator to reuse, not duplicate:
+  - `plugins/core/connectors/lib/transport-route-schemas.js`
+
+Explicit non-goals
+- Do not move permission checks into schemas.
+- Do not move DB existence checks into schemas.
+- Do not turn `searchSchema` into a generic filter DSL.
+- Do not add a compatibility layer that preserves both manual validators and compiled contracts.

package/docs/API.md

@@ -4,13 +4,15 @@ This reference provides comprehensive documentation for all methods, parameters,
 
 ## Important Notes
 
-1. **Context Parameter**: The `context` parameter shown in method signatures is typically managed internally by the framework. When using the API programmatically, you usually don't need to provide it unless you're implementing custom authentication or request-specific data.
+1. **Context Parameter**: The `context` parameter shown in method signatures is typically managed internally by the framework. When using the API programmatically, you usually don't need to provide it unless you're implementing custom request identity/auth context or other request-specific data.
 
-2. **Simplified Mode Defaults**: 
+2. **App-Owned Auth Context**: `json-rest-api` does not define authentication semantics. If you use `context.auth` in hooks, treat it as application-owned data supplied by your transport or surrounding stack.
+
+3. **Simplified Mode Defaults**: 
    - `simplifiedApi`: `true` (default for programmatic API calls)
    - `simplifiedTransport`: `false` (default for HTTP transport)
 
-3. **All parameters must be passed within a single params object** as the first argument to each method.
+4. **All parameters must be passed within a single params object** as the first argument to each method.
 
 ## Table of Contents
 
@@ -1400,7 +1402,7 @@ Each API method has its own specific hook execution order. Here's the exact sequ
 
 1. **Processing Hooks**: Only POST, PUT, and PATCH have `beforeProcessing` hooks
 2. **Schema Validation**: Only POST, PUT, and PATCH have schema validation hooks
-3. **Permission Checks**: GET uses `checkDataPermissions`, while relationship methods use `checkPermissions`
+3. **Permission Hooks**: GET uses `checkDataPermissions`, while relationship methods use `checkPermissions`
 4. **Query Filtering**: Only QUERY method triggers `knexQueryFiltering` hooks
 5. **Enrichment**: Only GET and QUERY have `enrichRecord` hooks
 6. **Relationships**: Only GET has `enrichRecordWithRelationships`
@@ -1421,7 +1423,7 @@ Each hook receives a context object with different properties based on the hook
     queryParams: {...},
     simplified: true
   },
-  auth: {...},                 // Authentication info
+  auth: {...},                 // App-provided auth/identity context if present
   transaction: {...},          // Database transaction
   schemaInfo: {...},           // Resource schema
   db: {...}                    // Database connection
@@ -1490,7 +1492,7 @@ api.resource('articles').hook('afterSchemaValidate', async (context) => {
 }
 ```
 
-**Purpose:** Authorization checks - throw error to deny access
+**Purpose:** Permission or visibility checks implemented by your app hooks. Throw an error to deny the operation.
 
 **Example:**
 ```javascript
@@ -2243,13 +2245,9 @@ api.addResource({
       }
     },
     
-    // Permissions
-    permissions: {
-      create: ['author', 'admin'],
-      read: ['*'],
-      update: ['author', 'editor', 'admin'],
-      delete: ['admin']
-    },
+    // No built-in permissions option exists here.
+    // Use permission hooks for access decisions, and optional plugins
+    // such as AutoFilterPlugin for dataset scoping.
     
     // Soft delete configuration
     softDelete: {
@@ -2499,6 +2497,45 @@ try {
 
 **Important:** When you provide a transaction, you're responsible for committing/rolling back. When you don't provide one, the library auto-commits.
 
+### Knex Schema Helpers
+
+When a resource is backed by `RestApiKnexPlugin`, its scope also exposes table-management helpers:
+
+```javascript
+await api.resources.articles.createKnexTable()
+
+const snapshot = await api.resources.articles.introspectKnexTableSnapshot()
+const createMigration = await api.resources.articles.generateKnexMigration()
+const diff = await api.resources.articles.generateKnexMigrationDiff()
+```
+
+Available helpers:
+
+- `createKnexTable()`
+- `addKnexFields({ fields })`
+- `alterKnexFields({ fields })`
+- `introspectKnexTableSnapshot()`
+- `generateKnexMigration()`
+- `generateKnexMigrationDiff()`
+
+`generateKnexMigrationDiff()` returns:
+
+```javascript
+{
+  migration,
+  warnings,
+  plan
+}
+```
+
+Important boundaries:
+
+- this is a live-table snapshot diff, not migration-history reconstruction
+- `addKnexFields()` and `alterKnexFields()` are field-only helpers
+- indexes, foreign keys, and check constraints belong on the full table schema surface
+
+For full examples and dialect notes, see [Knex Schema and Migrations](GUIDE/GUIDE_X_Knex_Schema_And_Migrations.md).
+
 ### Batch Operations
 
 Process multiple operations efficiently:
@@ -2746,4 +2783,4 @@ api.resource('articles').hook('knexQueryFiltering', async (context) => {
     query.whereRaw("metadata->>'key' = ?", [filters.metadata_key]);
   }
 });
-```
+```

package/docs/COMPARISON.md

@@ -30,7 +30,7 @@ json-rest-api is a full-featured, plugin-based JSON:API implementation built on
 | WebSocket support | ✅ | ❌ | ❌ | ❌ | ❌ |
 | Bulk operations | ✅ | ❌ | ❌ | ❌ | ❌ |
 | Positioning/sorting | ✅ | ❌ | ❌ | ❌ | ❌ |
-| Authentication | ✅ | ❌ | ⚠️ | ❌ | ❌ |
+| Permission hooks / app-owned auth integration | ✅ | ❌ | ⚠️ | ❌ | ❌ |
 | **Developer Experience** |
 | TypeScript support | ✅ | ✅ | ✅ | ❌ | ❌ |
 | Active maintenance | ✅ (2025) | ✅ (2024) | ✅ (2024) | 🟡 (2022) | ❌ (2022) |
@@ -430,4 +430,4 @@ json-rest-api stands out as the most comprehensive JSON:API solution for Node.js
 4. **Active Development**: Regular updates and community support
 5. **Production Ready**: Battle-tested with extensive test coverage
 
-For teams building JSON:API services in 2025, json-rest-api offers the best combination of compliance, features, and developer experience.
+For teams building JSON:API services in 2025, json-rest-api offers the best combination of compliance, features, and developer experience.

package/docs/GUIDE/GUIDE_1_Initial_Setup.md

@@ -928,6 +928,32 @@ Each resource (added via `api.addResource()`) gets these methods automatically:
   await api.resources.countries.createKnexTable();
   ```
 
+- **`api.resources.{resourceName}.introspectKnexTableSnapshot()`** - Returns a normalized snapshot of the live physical table
+  ```javascript
+  const snapshot = await api.resources.countries.introspectKnexTableSnapshot();
+  ```
+
+- **`api.resources.{resourceName}.generateKnexMigration()`** - Generates a Knex create migration string for the resource
+  ```javascript
+  const migration = await api.resources.countries.generateKnexMigration();
+  ```
+
+- **`api.resources.{resourceName}.generateKnexMigrationDiff()`** - Generates an additive diff migration from the live table snapshot
+  ```javascript
+  const diff = await api.resources.countries.generateKnexMigrationDiff();
+  ```
+
+- **`api.resources.{resourceName}.addKnexFields({ fields })`** / **`alterKnexFields({ fields })`** - Column-only helpers for existing tables
+  ```javascript
+  await api.resources.countries.addKnexFields({
+    fields: {
+      iso3: { type: 'string', maxLength: 3 }
+    }
+  });
+  ```
+
+For full examples of snapshots, migration generation, diffing, and table metadata such as indexes / foreign keys / checks, see [Knex Schema and Migrations](GUIDE_X_Knex_Schema_And_Migrations.md).
+
 
 ## API Namespaces (internal, for plugin developers)
 
@@ -943,6 +969,7 @@ These helpers and methods provide a complete toolkit for:
 - Managing database transactions
 - Generating proper URLs and links
 - Accessing the underlying database when needed
+- Inspecting live tables and generating Knex migrations when using table-backed resources
 - Integrating with web frameworks like Express
 
-The architecture ensures clean separation between HTTP transport, business logic, and data persistence layers.
+The architecture ensures clean separation between HTTP transport, business logic, and data persistence layers.

package/docs/GUIDE/GUIDE_2_2_Manipulating_And_Searching_Tables.md

@@ -306,6 +306,10 @@ The `searchSchema` gives you:
 * Flexibility to extend: Add virtual fields or complex filters via searchSchema while basic fields use `search: true`
 * Best of both worlds: Explicit control for complex cases, automatic setup for simple ones
 
+`searchSchema` also defines the public filter names your API accepts. Those keys do not need to match database columns or even resource field names. Treat `searchSchema` as an allowlist plus normalization layer for filter input: the actual meaning of each key is still implemented by the backend through mappings such as `actualField`, multi-field rules such as `oneOf`, standard operators, or custom `applyFilter` logic. It is not a portable filter DSL.
+
+The merge rules and validation surface are shared across storage backends. Advanced execution semantics are still backend-owned. In practice, that means alias filters such as `oneOf` and custom `applyFilter` functions should be validated against the backend you actually run, and the SQL-heavy examples in this chapter assume the Knex-backed query layer.
+
 `searchSchema` also gives you the ability to define a search field that will search in multiple fields. For example:
 
 ```javascript
@@ -801,4 +805,4 @@ GET /api/countries?filter[search]=united+states&fields[countries]=code
 
 ---
 
-[Previous: 2.1 The starting point](./GUIDE_2_1_The_Starting_Point.md) | [Back to Guide](./README.md) | [Next: 2.3 `belongsTo` Relationships](./GUIDE_2_3_BelongsTo_Relationships.md)
+[Previous: 2.1 The starting point](./GUIDE_2_1_The_Starting_Point.md) | [Back to Guide](./README.md) | [Next: 2.3 `belongsTo` Relationships](./GUIDE_2_3_BelongsTo_Relationships.md)

package/docs/GUIDE/GUIDE_2_7_Pagination_And_Ordering.md

@@ -2,6 +2,8 @@
 
 Pagination and ordering are essential features for working with large datasets. The json-rest-api library provides powerful pagination that applies not just to main resources, but also to included relationships. This means you can limit and order both parent records AND their children independently.
 
+For derived fields that must participate in `ORDER BY` and cursor pagination, see [Query Projections](GUIDE_X_Query_Projections.md). Regular computed fields are response-layer only and are not the right tool for stable SQL ordering.
+
 Let's create a comprehensive example with posts, comments, and tags to demonstrate all pagination and ordering features:
 
 ```javascript
@@ -1233,4 +1235,4 @@ The pagination and ordering system is designed to be efficient and predictable:
 
 ---
 
-[Previous: 2.6 Many to Many Relationships](./GUIDE_2_6_Many_To_Many.md) | [Back to Guide](./README.md)
+[Previous: 2.6 Many to Many Relationships](./GUIDE_2_6_Many_To_Many.md) | [Back to Guide](./README.md)