json-rest-api 1.0.29 → 2.1.0

package/docs/GUIDE/15-query-projections.md

@@ -0,0 +1,266 @@
+---
+title: "Query projections"
+chapter: 15
+chapter_label: "15"
+---
+
+# 15. Query projections
+
+Query projections are SQL-backed, output-only query fields provided by the optional `QueryProjectionsPlugin`. They are different from normal computed fields.
+
+If you want to build your own plugin on the same query-field seam, see [Writing Plugins](29-writing-plugins.md).
+
+- **Computed fields** run after the row is fetched, in JavaScript.
+- **Projected fields** are selected inside the query, so they can participate in `SELECT`, `ORDER BY`, and cursor pagination.
+
+Use query projection fields when a derived value must behave like a real list field.
+
+Typical cases:
+- sorting by a derived label such as `full_name`
+- cursor pagination over a derived value
+- returning a derived SQL value without storing it
+
+Do **not** use query projection fields for normal response decoration. Regular computed fields are simpler and remain the right default for that.
+
+## Setup
+
+The six executable blocks below form one scenario. Add them to the
+[starting script](03-running-example.md) after REST/storage installation,
+before starting the server, on a fresh database. Other code blocks are
+configuration fragments.
+
+```javascript
+import { QueryProjectionsPlugin } from 'json-rest-api'
+await api.use(QueryProjectionsPlugin)
+```
+
+## Defining a query field
+
+```javascript
+await api.addResource('authors', {
+  schema: {
+    id: { type: 'id' },
+    first_name: { type: 'string', required: true },
+    last_name: { type: 'string', required: true }
+  },
+
+  queryFields: {
+    full_name: {
+      type: 'string',
+      sortable: true,
+      select: ({ knex, column }) => knex.raw(
+        knex.client.config.client === 'mysql2'
+          ? "trim(concat(coalesce(??, ''), ' ', coalesce(??, '')))"
+          : "trim(coalesce(??, '') || ' ' || coalesce(??, ''))",
+        [column('first_name'), column('last_name')]
+      )
+    }
+  },
+
+  sortableFields: ['id', 'first_name', 'last_name'],
+  defaultSort: ['full_name']
+})
+await api.addResource('books', {
+  schema: {
+    title: { type: 'string', required: true },
+    author_id: { type: 'id', belongsTo: 'authors', as: 'author', nullable: true }
+  }
+})
+await api.resources.authors.createKnexTable()
+await api.resources.books.createKnexTable()
+```
+
+The expression uses `concat` for MySQL and `||` for SQLite/PostgreSQL. `column`
+resolves logical fields to the selected storage representation.
+
+```javascript
+const jane = await api.resources.authors.post({ format: 'plain', data: { first_name: 'Jane', last_name: 'Doe' } })
+await api.resources.authors.post({ format: 'plain', data: { first_name: 'John', last_name: 'Adams' } })
+await api.resources.authors.post({ format: 'plain', data: { first_name: 'Jane', last_name: 'Smith' } })
+const book = await api.resources.books.post({ format: 'plain', data: { title: 'Projection example', author: jane.id } })
+```
+
+`select()` is called at query time. It can return:
+
+- a `knex.raw(...)` expression
+- a Knex reference
+- a Knex query/subquery expression
+
+Return the expression directly from a synchronous callback. Knex raw values,
+references and query builders are thenable: returning them from an `async`
+callback or awaiting them can execute SQL instead of supplying an expression.
+For example, use `select: ({ knex }) => knex.raw('?', [1])`, without `async`.
+
+Definitions are compiled into `scope.vars.schemaInfo.queryFields` during schema
+enrichment. Projection names cannot collide with stored/computed fields,
+relationship names or logical ID names. Canonical field additions repeat these
+checks before publishing a replacement schema. SQL expressions remain local to
+each query; compiled definitions do not cache request-dependent SQL.
+
+Getters and computed fields can declare projection names as dependencies. The
+projection is selected before JavaScript callbacks run, including through a
+chain of dependencies in a sparse fieldset. A projection needed only as an input
+is removed from output unless selected and visible. Install the declaration
+plugin before resource registration so dependencies can be validated.
+
+The callback receives:
+
+```js
+{
+  knex,      // current knex/db handle
+  db,        // same as knex
+  context,   // request context
+  scopeName,
+  tableName,
+  fieldName,
+  schemaInfo,
+  adapter,   // storage adapter
+  column,    // helper: logical field -> qualified storage column
+  ref        // helper: logical field -> knex ref
+}
+```
+
+## Behavior
+
+Query projection fields are:
+
+- returned by default in `get()` and `query()`
+- available in sparse fieldsets
+- usable in sorting and cursor pagination
+- available on included resources
+- ignored on writes
+
+They are **not** stored columns, and they are **not** part of write validation.
+
+A failed `select()` callback or rejected promise rejects the operation. On reads,
+typed API errors retain their identity; unexpected errors are wrapped with the
+resource/field context and the original thrown value as `cause`, including
+non-Error values. Included projection failures also reject GET/query instead of
+returning a partial response. During a full write response, the failure occurs
+before an owned transaction commits; the public `RestApiWriteError` retains the
+failure as its cause and reports the transaction outcome. A borrowed transaction
+remains its owner's responsibility.
+
+## Sparse fieldsets
+
+Query projection fields can be requested explicitly:
+
+```javascript
+const sparse = await api.resources.authors.get({
+  id: jane.id, format: 'jsonapi',
+  queryParams: {
+    fields: {
+      authors: 'first_name,full_name'
+    }
+  }
+})
+```
+
+If a query field is needed only for sorting, it is still selected internally, but it is removed from the response unless the sparse fieldset asked for it.
+
+## Sorting and cursor pagination
+
+Query projection fields are useful when a derived value must drive list behavior:
+
+```javascript
+const firstPage = await api.resources.authors.query({
+  format: 'jsonapi',
+  queryParams: { sort: ['full_name'], page: { size: 2 } }
+})
+const nextPage = await api.resources.authors.query({
+  format: 'jsonapi',
+  queryParams: { sort: ['full_name'], page: { size: 2, after: firstPage.meta.pagination.cursor.next } }
+})
+```
+
+`json-rest-api` automatically appends `id` as a stable tie-breaker for cursor ordering. That prevents duplicate or skipped rows when multiple records share the same projected value.
+
+Projection cursors retain the SQL expression's value before output formatting
+and bind it directly on the next request. This also preserves precise timestamps
+beyond the built-in millisecond write limit:
+
+```js
+queryFields: {
+  preciseTimestamp: {
+    type: 'dateTime',
+    temporalPrecision: 6,
+    sortable: true,
+    select: ({ knex, column }) => knex.raw('??', [column('recordedAt')])
+  }
+}
+```
+
+Projections cannot declare `storage`, `setter`, or `runSetterAfter`; compilation
+rejects those options. Use the stored field's serializer when writing that field.
+For projection output formatting, use `getter` and optional `runGetterAfter`.
+The shared dependency order awaits projection getters before dependent getters
+or computed fields, and sparse reads fetch their declared dependencies. Hidden
+dependencies remain hidden in the response. Getter failures propagate as read
+errors. Neither getters nor response formatting change SQL ordering or cursors.
+Always follow returned cursors/links without rebuilding them from attributes.
+See [temporal migration requirements](33-migrating-to-v2.md#temporal-formatting-and-native-databases)
+for UTC bindings, fractional precision and existing AnyAPI data.
+
+Structured (`object`/`array`) projections are output fields and cannot be marked
+sortable. Select a scalar JSON key in a separate typed projection when it should
+drive ordering or cursors. PostgreSQL structured selections use `to_jsonb` to
+support `DISTINCT` parent queries; this does not alter application tables.
+See [querying structured attributes](33-migrating-to-v2.md#querying-structured-attributes)
+for bound SQL examples and custom-filter migration.
+
+Hidden projections cannot be sortable or appear in resource/include sort
+configuration: cursors contain the selected SQL value. Use `normallyHidden`
+for a publicly readable projection that should be omitted by default but remain
+sortable. A `hidden` projection can still supply a getter/computed dependency.
+See [hidden sort migration](33-migrating-to-v2.md#hidden-sort-fields).
+
+## Included resources
+
+Without a target fieldset, includes select all visible projections and the
+dependencies of getters and computed fields. This applies to to-one, to-many
+and polymorphic includes in both storage modes, even before the target's first
+direct read. A sparse fieldset returns only the requested fields; dependencies
+selected solely for a callback are removed from the response.
+
+Query projection fields can be selected for included resources too:
+
+```javascript
+const included = await api.resources.books.get({
+  id: book.id, format: 'jsonapi',
+  queryParams: {
+    include: ['author'],
+    fields: {
+      books: 'title,author',
+      authors: 'full_name'
+    }
+  }
+})
+```
+
+## Write behavior
+
+Query projection fields are output-only. If a client sends them in `POST`, `PUT`, or `PATCH`, they are ignored.
+
+The configured logger receives a bounded warning with the operation, resource,
+validation phase and total ignored-field count. It does not include submitted
+attribute values. A failure to write this advisory warning does not reject the
+operation.
+
+That keeps the boundary clean:
+
+- persisted fields own storage
+- computed fields own response-layer enrichment
+- query projection fields own query-layer derivation
+
+## Limits
+
+Projected fields are an advanced query feature. They are intentionally **backend-sensitive**.
+
+Important limits:
+
+- projection SQL can be dialect-specific
+- this is not a portability layer across every SQL engine
+- query projection fields are not auto-searchable
+- if you want to sort by one, mark it `sortable: true`
+
+For display-only derived values, prefer normal computed fields from [Field Transformations](12-field-transformations.md).

package/docs/GUIDE/{GUIDE_X_Autofiltering.md → 16-autofiltering.md}

@@ -1,4 +1,10 @@
-# Autofiltering
+---
+title: "Autofiltering"
+chapter: 16
+chapter_label: "16"
+---
+
+# 16. Autofiltering
 
 The `AutoFilterPlugin` provides declarative dataset scoping for `json-rest-api`.
 
@@ -31,17 +37,75 @@ The plugin is generic. It does not assume:
 
 Your application decides where scope values come from. The plugin only consumes configured resolver functions.
 
+## Runnable workspace example
+
+Insert these three blocks into the [starting script](03-running-example.md)
+after installing storage and before starting the server, on a fresh database.
+They also work with canonical storage after its normal initialization. The
+remaining examples below are configuration alternatives and fragments.
+
+```javascript
+import { AutoFilterPlugin } from 'json-rest-api'
+
+await api.use(AutoFilterPlugin, {
+  resolvers: { workspace: ({ context }) => context.session?.workspaceId },
+  presets: { workspace: { filters: [{ field: 'workspace_id', resolver: 'workspace' }] } }
+})
+await api.addResource('projects', {
+  schema: {
+    name: { type: 'string', required: true },
+    workspace_id: { type: 'string', required: true }
+  },
+  autofilter: 'workspace'
+})
+await api.resources.projects.createKnexTable()
+const workspaceContext = workspaceId => ({ session: { workspaceId } })
+```
+
+Supply trusted application context as the second argument. Here the workspace
+values are illustrative; a server should derive them from its authenticated
+request context.
+
+```javascript
+const acmeProject = await api.resources.projects.post({
+  data: { name: 'Roadmap' }, format: 'plain'
+}, workspaceContext('acme'))
+await api.resources.projects.post({
+  data: { name: 'Other workspace' }, format: 'plain'
+}, workspaceContext('other'))
+const acmePage = await api.resources.projects.query({
+  format: 'jsonapi', queryParams: { page: { number: 1, size: 10 } }
+}, workspaceContext('acme'))
+console.log(acmeProject.workspace_id, acmePage.data, acmePage.meta.pagination.total)
+```
+
+The created record has `workspace_id: 'acme'`. The page contains only Roadmap
+and its total is 1, even though another workspace has a stored project.
+
+```javascript
+const replacedProject = await api.resources.projects.put({
+  id: acmeProject.id, data: { name: 'Updated roadmap' }, format: 'plain'
+}, workspaceContext('acme'))
+const otherPage = await api.resources.projects.query({ format: 'plain' }, workspaceContext('other'))
+console.log(replacedProject.workspace_id, otherPage.data.map(project => project.name))
+```
+
+PUT supplies the complete writable record, with the scoped workspace field
+injected by the plugin. The record remains in acme, and the other workspace
+still sees only its own project. Explicitly submitting a different workspace
+value rejects; omitting required application scope context also rejects.
+
 ## Installation
 
 ```js
-import { Api } from 'hooked-api'
+import { JsonRestApi } from 'json-rest-api'
 import {
   RestApiPlugin,
   RestApiKnexPlugin,
   AutoFilterPlugin
 } from 'json-rest-api'
 
-const api = new Api({ name: 'scoped-api' })
+const api = new JsonRestApi({ name: 'scoped-api' })
 
 await api.use(RestApiPlugin)
 await api.use(RestApiKnexPlugin, { knex })
@@ -84,7 +148,7 @@ await api.use(AutoFilterPlugin, {
 - `RestApiKnexPlugin` for resources backed by normal database tables
 - `RestApiAnyapiKnexPlugin` for logical resources backed by AnyAPI canonical storage
 
-An `Api` instance has one storage engine. Install either storage plugin, then install `AutoFilterPlugin`. If an application keeps core resources in normal tables and user-defined resources in AnyAPI, create one `Api` instance for each storage engine. Each instance owns its resources and autofilter configuration.
+A `JsonRestApi` instance has one storage engine. Install either storage plugin, then install `AutoFilterPlugin`. If an application keeps core resources in normal tables and user-defined resources in AnyAPI, create one `JsonRestApi` instance for each storage engine. Each instance owns its resources and autofilter configuration.
 
 Autofilter definitions always use logical schema field names. The selected storage plugin translates those fields to normal table columns or AnyAPI canonical slots.
 
@@ -93,14 +157,14 @@ Autofilter definitions always use logical schema field names. The selected stora
 AnyAPI's `tenantId` is fixed when `RestApiAnyapiKnexPlugin` is installed. It identifies the metadata and canonical-storage namespace; it is not a per-request workspace value. For one AnyAPI instance serving multiple application workspaces, use a stable internal `tenantId` and persist `workspace_id` on every workspace-scoped resource:
 
 ```js
-import { Api } from 'hooked-api'
+import { JsonRestApi } from 'json-rest-api'
 import {
   RestApiPlugin,
   RestApiAnyapiKnexPlugin,
   AutoFilterPlugin
 } from 'json-rest-api'
 
-const api = new Api({ name: 'form-data-api' })
+const api = new JsonRestApi({ name: 'form-data-api' })
 
 await api.use(RestApiPlugin)
 await api.use(RestApiAnyapiKnexPlugin, {
@@ -222,7 +286,7 @@ For a `workspace_user` resource:
 
 ```js
 await api.resources.projects.query(
-  { simplified: false },
+  { format: 'jsonapi' },
   {
     session: { workspaceId: 'acme' },
     subject: { id: 101 }
@@ -252,7 +316,7 @@ Example:
 
 ```js
 await api.resources.projects.post({
-  inputRecord: {
+  document: {
     data: {
       type: 'projects',
       attributes: {
@@ -260,7 +324,7 @@ await api.resources.projects.post({
       }
     }
   },
-  simplified: false
+  format: 'jsonapi'
 }, {
   session: { workspaceId: 'acme' },
   subject: { id: 101 }
@@ -297,7 +361,7 @@ So this will fail if the referenced project is outside scope:
 
 ```js
 await api.resources.tasks.post({
-  inputRecord: {
+  document: {
     data: {
       type: 'tasks',
       attributes: { title: 'Cross-scope task' },
@@ -308,12 +372,10 @@ await api.resources.tasks.post({
       }
     }
   },
-  simplified: false
+  format: 'jsonapi'
 }, {
-  scopeValues: {
-    workspaceId: 'workspace-a',
-    userId: 101
-  }
+  session: { workspaceId: 'workspace-a' },
+  subject: { id: 101 }
 })
 ```
 

package/docs/GUIDE/{GUIDE_X_Row_Policies.md → 17-row-policies.md}

@@ -1,4 +1,10 @@
-# Row Policies
+---
+title: "Row policies"
+chapter: 17
+chapter_label: "17"
+---
+
+# 17. Row policies
 
 `RowPolicyPlugin` is the server-side visibility seam for resources whose readable rows depend on more than a fixed equality filter.
 
@@ -18,7 +24,7 @@ This is unsafe and paginates the wrong dataset:
 ```js
 const page = await api.resources.organisation_units.query({
   queryParams: { page: { number: 1, size: 20 } },
-  simplified: false
+  format: 'jsonapi'
 }, context)
 
 page.data = page.data.filter((unit) => canSeeUnit(context, unit))
@@ -66,7 +72,7 @@ The page size and count now describe the visible dataset.
 | Requirement | Use |
 | --- | --- |
 | The caller asks to filter by a public search field | Normal `queryParams.filters` / `searchSchema` |
-| Every row must match one or more context-derived persisted fields, and those fields should be stamped on writes | [`AutoFilterPlugin`](GUIDE_X_Autofiltering.md) |
+| Every row must match one or more context-derived persisted fields, and those fields should be stamped on writes | [`AutoFilterPlugin`](16-autofiltering.md) |
 | Visibility needs `OR`, `EXISTS`, joins, hierarchy traversal, or domain-composed SQL | `RowPolicyPlugin` |
 | The caller may perform an action such as update, approve, or delete | Permission hooks, usually in addition to a row policy |
 | A response-only field must be calculated | Computed fields or query projections |
@@ -77,46 +83,14 @@ The page size and count now describe the visible dataset.
 
 Install the REST plugin, one supported Knex storage plugin, and then the row-policy plugin. Install `RowPolicyPlugin` before adding resources that declare `rowPolicy`.
 
-### Normal table storage
-
-```js
-import { Api } from 'hooked-api'
-import {
-  RestApiPlugin,
-  RestApiKnexPlugin,
-  RowPolicyPlugin
-} from 'json-rest-api'
-
-const api = new Api({ name: 'application-api' })
-
-await api.use(RestApiPlugin)
-await api.use(RestApiKnexPlugin, { knex })
-await api.use(RowPolicyPlugin, {
-  policies: {
-    workspaceMember: ({ query, context, column, value }) => {
-      const workspaceId = context.session?.workspaceId
-      if (!workspaceId) return false
-
-      query.where(
-        column('workspace_id'),
-        value('workspace_id', workspaceId)
-      )
-      return true
-    }
-  }
-})
-```
-
-### AnyAPI canonical storage
+The following example starts with an `api` that already has `RestApiPlugin` and
+one of `RestApiKnexPlugin` or `RestApiAnyapiKnexPlugin` installed, as described in
+[initial setup](01-getting-started.md). Use a fresh database for this example.
+The policy code is the same for both storage modes:
 
-Use the same plugin and logical field names:
+```javascript
+import { RowPolicyPlugin } from 'json-rest-api'
 
-```js
-await api.use(RestApiPlugin)
-await api.use(RestApiAnyapiKnexPlugin, {
-  knex,
-  tenantId: 'application'
-})
 await api.use(RowPolicyPlugin, {
   policies: {
     workspaceMember: ({ query, context, column, value }) => {
@@ -133,7 +107,8 @@ await api.use(RowPolicyPlugin, {
 })
 ```
 
-`column()` and `value()` translate logical schema fields and values through the active storage adapter. The example therefore works with both normal tables and AnyAPI canonical slots.
+`column()` and `value()` translate logical schema fields and values through the
+active storage adapter, including canonical slots.
 
 ## Resource configuration
 
@@ -141,7 +116,7 @@ await api.use(RowPolicyPlugin, {
 
 Registered policies are preferred for shared or package-owned behavior:
 
-```js
+```javascript
 await api.addResource('documents', {
   schema: {
     id: { type: 'id' },
@@ -151,10 +126,44 @@ await api.addResource('documents', {
   rowPolicy: 'workspaceMember',
   tableName: 'documents'
 })
+await api.resources.documents.createKnexTable()
 ```
 
 The string must match a function in `RowPolicyPlugin`'s `policies` option. Unknown names fail while the resource is being added.
 
+Create two documents in one workspace and one in another. Supply trusted
+application context separately from the call parameters:
+
+```javascript
+const acmeContext = { session: { workspaceId: 'acme' } }
+const otherContext = { session: { workspaceId: 'other' } }
+for (const title of ['Alpha', 'Beta']) {
+  await api.resources.documents.post({
+    data: { title, workspace_id: 'acme' }, returning: 'none'
+  }, acmeContext)
+}
+await api.resources.documents.post({
+  data: { title: 'Other workspace', workspace_id: 'other' }, returning: 'none'
+}, otherContext)
+
+const acmePage = await api.resources.documents.query({
+  format: 'jsonapi',
+  queryParams: { sort: 'title', page: { number: 1, size: 1 } }
+}, acmeContext)
+const otherPage = await api.resources.documents.query({}, otherContext)
+const deniedPage = await api.resources.documents.query({})
+
+console.log(acmePage.data[0].attributes.title) // Alpha
+console.log(acmePage.meta.pagination.total) // 2, before the page limit
+console.log(otherPage.data.map(document => document.title)) // ['Other workspace']
+console.log(deniedPage.data) // []: no workspace in the caller context
+```
+
+These inserts explicitly supply `workspace_id`. This policy does not stamp or
+validate ownership on writes; use autofilter and write permissions for that
+contract. The example demonstrates read visibility, counts and missing-context
+denial.
+
 ### Inline policy
 
 An inline function is supported for a policy that belongs to one resource:
@@ -279,8 +288,9 @@ Policies are reused for the resource selections that define externally visible d
 | `count` | A separate count query, including native offset-pagination counts and standalone count helpers. |
 | `single` | Minimal lookup used by `get`, and by existing-record preflights for `put`, `patch`, and `delete`. |
 | `include` | A target resource loaded into JSON:API `included` data. |
+| `search-join` | A related resource used by a cross-resource or polymorphic search, reference filter, or reference sort, including intermediate search path segments. |
 | `relationship-identifiers` | Child identifiers attached to a relationship, including has-many and AnyAPI many-to-many links. |
-| `relationship-validation` | A target record referenced by a write payload. |
+| `relationship-validation` | A batch of up to 100 distinct target IDs referenced by a write payload, or a scoped lookup retaining database ID equality. |
 | `relationship-parent` | The parent of a relationship route such as `getRelated`. |
 | `unspecified` | A custom call to the lower-level query-filtering seam did not provide a purpose. |
 
@@ -288,12 +298,74 @@ The important invariant is not the label. It is that the predicate is attached b
 
 Treat `queryPurpose` as diagnostic context, not as permission to weaken a policy. A policy should normally add the same visibility predicate for every purpose. If an advanced policy uses a purpose-specific SQL shape, each branch must enforce equivalent visibility and must be tested independently; never skip a predicate for counts, includes, identifiers, or relationship validation.
 
+Relationship identifier visibility is selected in batches of at most 100 unique
+IDs per target resource type. Target query permissions and filtering hooks run
+for each batch, with the caller's authorization and transaction context. Always
+attach the complete predicate to the supplied builder; a request-level
+"already applied" flag must not skip later queries. Keep ordering, pagination
+and include limits in their operation/resource settings. Visibility hooks add
+row predicates and retain the query's selected ID and mandatory constraints.
+Their invocation count is not a per-request or per-record contract.
+
+Regular and polymorphic belongs-to includes also load at most 100 target IDs
+per query, with target permission and filtering hooks applied to every batch
+under `queryPurpose: 'include'`. SQL projection callbacks run for each query.
+Nested collection includes run after all target batches have been collected,
+so a global collection limit is not restarted for each batch.
+
+Relationship validation uses the same batch bound. Its query context has no
+individual `id` and no parent client filters. Retain the selected ID and scope
+predicates; filtering supplies an ID subquery used to fetch complete minimal
+records. Read permission hooks then receive each target's ID and minimal record,
+once per distinct identifier per relationship. Link writers do not run target
+response GET hooks. See the [migration steps](33-migrating-to-v2.md#relationship-target-validation).
+
+Large collection and relationship-metadata reads keep their selection in one
+SQL query. PostgreSQL uses an array predicate and SQLite uses a JSON1 ID
+subquery; their policies and projections still run before the original sort
+and limit. Apply predicates using the supplied query/context rather than
+assuming the internal ID constraint is always a Knex `whereIn` value array.
+
+The identifier selection uses `DISTINCT` so predicate joins do not multiply its
+returned ID rows. The library retains the input linkage order and occurrences
+after visibility filtering. A permission or filter failure in a later batch
+rejects the read rather than returning a partial set. No visibility results are
+shared between calls, resource types or tenant APIs.
+
 ### Collections and counts
 
 Offset pagination applies the same mandatory policy to the data query and its count. This remains true when the caller supplies no `filters` object.
 
 Cursor predicates are added after the row policy, so the cursor moves through visible rows rather than through a page that will later be filtered.
 
+Related search rows are filtered inside each joined selection. A hidden related
+name cannot match a search or contribute to its count. A visible primary record
+can still match its own attributes in an OR search when its related row is hidden
+or absent. These selections check the target's `query` permission and use the
+caller's authorization and transaction context, without inheriting the primary
+resource's client filters. Explicit permission or query-hook failures reject the
+query. Keep mandatory policy predicates active for `search-join` too.
+
+Built-in filters on a belongs-to foreign key, its relationship alias, or a
+polymorphic ID/type pair also require a visible target. A hidden or missing
+reference behaves as null when evaluating the declared filter operator. This
+applies to `actualField`, `oneOf` and joined target fields as well as direct
+filters. Comparisons retain the stored column's type and collation. Custom
+`applyFilter` callbacks still own the visibility of any related SQL they write;
+the library cannot infer relationships from arbitrary custom predicates.
+
+Built-in sorting on these reference fields and their aliases uses the same
+visibility boundary, including `defaultSort`. Hidden and absent references
+share a null sort value; later fields and the primary ID resolve ties. Ordering,
+cursor comparisons and cursor values all use the same visible value, so a
+hidden target's ID cannot appear in pagination metadata or links. Visible values
+retain their storage column's type and collation. Target lookup predicates use
+`search-join`, the current caller and the caller's transaction.
+
+For HTTP null filters, send `filter[field][json]=null`; plain
+`filter[field]=null` is the literal string. Generated links preserve typed nulls
+and arrays using this explicit JSON form.
+
 ### Single records and writes
 
 An existing row outside the policy behaves as not found for `get`, `patch`, and `delete`. Existing-record `put` authorization uses the same scoped minimal lookup.
@@ -310,6 +382,27 @@ The target resource's policy is used when the framework loads:
 - per-parent windowed include queries
 - a relationship target during write validation
 
+These secondary reads also check the target resource's `query` action permission.
+A permission denial rejects the read, whereas a row policy filters individual
+rows. Default relationship identifiers are subject to the same permission check
+as requested includes. The target permission context retains caller auth and
+transaction, with target scope/schema and no inherited parent ID or filters.
+
+Regular-storage many-to-many reads also apply the pivot resource's permission and
+policy to membership before standard/windowed include limits and linkage. AnyAPI
+uses tenant-scoped canonical links for these associations; a declared pivot
+resource's row policy does not apply to canonical links.
+
+Both storage backends apply include ordering and limits after target visibility.
+`include: { strategy: 'window', limit: 2, orderBy: ['name'] }` selects up to two
+visible children for each parent. The default/`standard` strategy applies a global
+limit to the included target set; a shared many-to-many child counts once and
+remains linked to every selected parent it belongs to. Duplicate physical links
+do not consume extra slots. Sparse fields and SQL query projections retain the
+configured strategy. Reverse polymorphic collections follow the same rules.
+Included linkage lists the selected subset; use the relationship/related endpoint
+to read or paginate the full visible membership.
+
 Filtering the parent resource does not automatically define child visibility. Put `rowPolicy` on every child resource that has its own visibility rule.
 
 Relationship routes preserve the caller's trusted context when they invoke the target resource. Filters, sorting, and pagination supplied for a related-resource request are applied to that target query; they are deliberately withheld from the preliminary parent-visibility lookup. This keeps a target-only filter from being misinterpreted as a parent field while still enforcing the parent's policy.