json-rest-api 2.4.0 → 3.0.0

package/docs/GUIDE/35-migrating-to-v3.md

@@ -0,0 +1,325 @@
+---
+title: "Migrating to v3"
+chapter: 35
+chapter_label: "35"
+---
+
+# 35. Migrating to v3
+
+Version 3 adds a data-access layer for work that does not fit one record at a
+time: reports, imports and background jobs. It replaces the 2.x bulk methods,
+which wrote one row per call, with many-row writes that run one statement per
+chunk. Single-record methods, relationship methods, HTTP routes other than
+`/bulk` and stored data are unchanged. There is no compatibility mode; the old
+bulk options are rejected with a message that names their replacement.
+
+## What is new
+
+- **Many-row writes:** `postMany`, `patchMany`, `patchWhereIds`, `patchWhere`,
+  `deleteMany` and `deleteWhere`, all-or-nothing, with upserts (`onConflict`)
+  and compare-and-set (`onlyIf`). See [many-row writes](27-many-row-writes.md).
+- **Scoped reads:** `scopedQuery`, a read-only Knex builder limited to the
+  caller's rows, and `queryAll`, every matching row in cursor pages.
+- **Row locks:** `lockByIds`, `lockWhere` and `claim` (`SKIP LOCKED`), inside a
+  transaction.
+- **Lock versions:** `lockVersionField`, a counter every update advances.
+- **Shared rows:** an autofilter filter with `shared: true` lets every owner
+  read rows whose owner field is NULL.
+- **System contexts:** `createSystemContext(reason)` for work across every
+  owner; `isSystemContext(context)`; `readOnlyQueryBuilder(builder)`.
+- **Batch hooks:** `beforeProcessingPostMany`, `beforeDataCallPostMany`,
+  `afterDataCallPostMany` and the `PatchMany` and `DeleteMany` forms; hook
+  registrations take `{ batch: true }`.
+- **Aggregates:** measures take a `where` condition; `groupBy` takes date
+  buckets (`day`, `week`, `month`, `year`) with a time zone; date fields with a
+  storage serializer can be bucketed and take `min` and `max`. In a system
+  context, a hidden ownership field may be grouped and counted distinct. See
+  [AGGREGATE](../API.md#aggregate---grouped-totals).
+- **One read for `get`:** a resource with no `checkPermissions`,
+  `checkDataPermissions`, `checkDataPermissionsGet`, `beforeData` or
+  `beforeDataGet` hook reads its record in one statement; such a get no
+  longer sets `context.minimalRecord` (read `context.record` in later hooks).
+- **Skipping writes in one statement:** `patchMany`, `patchWhereIds` and
+  `deleteMany` with `missing: 'skip'` put the scope in the UPDATE or DELETE
+  instead of reading the visible ids first, when no permission or
+  before-data-call batch hook needs those rows (not on MySQL and MariaDB,
+  where InnoDB would lock every listed row before checking the scope; there a
+  patch of one visible row runs its UPDATE without a locking read first).
+- **Fewer reads before single writes:** on table and AnyAPI storage, `patch`
+  and `delete` read their row once, scoped, before the UPDATE or DELETE, and a
+  `put` that replaces a row reads it twice (whether it exists, then scoped)
+  instead of three times. The unscoped read that checked the row still existed
+  is gone; the write's own row count answers a row that vanished in between
+  (`not_found`, as before).
+- **Single writes keep to their owner:** on table and AnyAPI storage, the
+  UPDATE or DELETE of a single `patch`, `put` or `delete` carries the
+  ownership (autofilter) conditions its scoped read found, as many-row writes
+  already did. A row that another writer moved to a different owner after the
+  read answers `not_found` instead of being written. On AnyAPI storage a row
+  that vanished after the read now answers `not_found` too, where the write
+  used to change nothing and report success.
+- **Links to hidden targets survive a write-back:** a `patch`, `put`,
+  many-row patch or merge upsert that sets a belongs-to to null keeps the
+  stored link when its target exists but is hidden from the writer, as the read
+  showed the link as unassigned. It used to clear it. Clear such links in a
+  system context.
+- **Polymorphic links are set through the relationship only:** the type and id
+  fields behind a polymorphic belongs-to (`owner_type`, `owner_id`) are refused
+  in attributes, as foreign keys already were (rule
+  `foreign_key_in_attributes`), so a write cannot skip the target check or half
+  clear a link. Send `owner: { _type, id }` (plain) or `relationships.owner`
+  (JSON:API) instead.
+- **Selection writes:** `patchWhere` and `deleteWhere` carry the scope and the
+  filters in every UPDATE or DELETE and return the rows they wrote; a row that
+  vanishes or stops matching in the meantime is left out, never an error.
+- **Counts:** `returning: 'count'` on every many-row write returns
+  `{ meta: { count } }`, the number of rows written, without reading rows back.
+- **Upserts on the id:** on a resource with no ownership fields and no row
+  policy, `onConflict: { keys: ['id'], action }` (or the `idProperty`, such as
+  an email natural key) creates the rows that do not exist yet, atomically.
+- **Integer fields over BIGINT columns read as numbers:** an `integer` or
+  `number` field whose driver returns a string (PostgreSQL's BIGINT, MySQL's
+  BIGINT past 2^53) reads as a number when a JavaScript number holds it exactly,
+  on every read and write response, as aggregates already did. It used to stay
+  a string, such as `"4400"`. A longer integer, or a fraction a number cannot
+  hold exactly, keeps its digits as a string, where an aggregate used to round
+  it; decimal fields (`precision` and `scale`) keep their strings.
+- **Numbered pages without counts:** with `enablePaginationCounts: false` an
+  offset page reads one extra row and reports `hasMore` and a `next` link,
+  still without a count query.
+
+Scoped reads, row locks, lock versions, shared rows and system contexts are
+described in [scoped reads, locks and system work](34-scoped-reads-and-locks.md).
+
+## Many-row writes replace the bulk methods
+
+`bulkPost`, `bulkPatch` and `bulkDelete` are removed. Their replacements run
+the per-row checks in memory and one INSERT, UPDATE or DELETE per chunk.
+
+| 2.x | 3.0 |
+| --- | --- |
+| `bulkPost({ data: [...] })` | `postMany({ data: [...] })` |
+| `bulkPost({ document: { data: [...] } })` | `postMany({ document: { data: [...] } })` |
+| `bulkPatch({ operations: [{ id, data: { ... } }] })` | `patchMany({ data: [{ id, ... }] })` |
+| `bulkPatch({ operations: [{ id, document: { data } }] })` | `patchMany({ document: { data: [{ type, id, attributes }] } })` |
+| the same change for many ids | `patchWhereIds({ ids, data })` or `patchWhere({ filters, data })` |
+| `bulkDelete({ ids })` | `deleteMany({ ids })` or `deleteWhere({ filters })` |
+| the same change for, or deletion of, every row | `patchWhere({ all: true, data })`, `deleteWhere({ all: true })`: without `filters`, `all: true` is required |
+| `atomic: true` | remove: every call is all-or-nothing |
+| `atomic: false` | remove; for rows that must succeed independently, call single writes in a loop |
+| `expectedVersion`, `expectedVersions` | removed: resources with a `versionField` refuse many-row writes. Use single writes with `expectedVersion`, or a `lockVersionField` with `onlyIf` |
+| result `{ meta: { total, succeeded, failed, atomic }, errors, data }` | nothing by default; `returning: 'minimal'` gives `{ data: [{ type, id }] }`, `'full'` gives `{ data: [records] }`, `'count'` gives `{ meta: { count } }` |
+| `errors[i].index` for a failed entry | the call throws; a validation error names the row in `details.index`, and its field paths start with `data.<index>.` |
+| `meta.deleted` | `deleteMany({ ids, returning: 'minimal' }).data` |
+| default `returning` of the resource | `'none'` unless the call says otherwise |
+
+```js
+// Before (2.x)
+await api.resources.books.bulkPatch({
+  operations: rows.map(row => ({ id: row.id, data: { title: row.title } })),
+  atomic: true,
+  format: 'plain'
+})
+
+// After (3.0)
+await api.resources.books.patchMany({
+  data: rows.map(row => ({ id: row.id, title: row.title })),
+  format: 'plain'
+})
+```
+
+Ids that are missing or hidden from the caller now fail the whole call with
+`RestApiRecordsNotFoundError` (404) before anything is written; pass
+`missing: 'skip'` to ignore them instead.
+
+### Behaviour that bulk writes had and many-row writes refuse
+
+The 2.x methods ran the full single-record pipeline for every entry, so they
+accepted anything a single write did. Many-row writes throw
+`RestApiBulkWriteUnsupportedError` (`REST_API_BULK_WRITE_UNSUPPORTED`, 422)
+before writing in these cases; keep single writes for them:
+
+| Case | Single-write alternative |
+| --- | --- |
+| AnyAPI storage | single writes |
+| a resource with a `versionField`, or whose writes advance another resource's version | single writes with `expectedVersion` |
+| rows that set many-to-many or has-many relationships | single writes or the relationship methods |
+| rows that set file fields | single writes with uploads |
+| creating positioned records, or moving them | single writes |
+| per-record write hooks without a batch form | add `{ batch: true }` and a batch hook (below) |
+
+### Hooks
+
+Many-row writes do not run the per-record write hooks (`beforeProcessing`,
+`beforeSchemaValidate*`, `beforeDataCall*`, `afterDataCall*`, `finish*` for
+post, patch and delete). If a resource has such a handler, its many-row writes
+are refused, naming the handler. For each one, either:
+
+- register it with `{ batch: true }` and add the matching batch hook
+  (`beforeProcessingPostMany`, `beforeDataCallPostMany`,
+  `afterDataCallPostMany`, `beforeProcessingPatchMany`,
+  `beforeDataCallPatchMany`, `afterDataCallPatchMany`,
+  `beforeDataCallDeleteMany`, `afterDataCallDeleteMany`); or
+- register it with `{ batch: true }` alone when it does not apply to many-row
+  writes.
+
+```js
+// Before (2.x): ran once for every bulk entry
+api.addHook('afterDataCallPost', 'audit', {}, auditOne)
+
+// After (3.0)
+api.addHook('afterDataCallPost', 'audit', { batch: true }, auditOne)
+api.addHook('afterDataCallPostMany', 'audit-many', {}, ({ context }) => auditMany(context.ids))
+```
+
+`context.bulkOperation` is gone. `checkPermissions` still runs once per row, with
+the row's position in `originalContext.bulkIndex`. `afterCommit` and
+`afterRollback` run once per call. The built-in autofilter, Socket.IO and query
+projection plugins already have batch forms.
+
+### BulkOperationsPlugin and the HTTP routes
+
+The plugin now only adds the HTTP routes; programs call the resource methods.
+
+| 2.x | 3.0 |
+| --- | --- |
+| `{ maxBulkOperations, defaultAtomic }` | `{ maxBulkOperations }`; `defaultAtomic` is rejected |
+| `maxBulkOperations` default 100 | default 1,000 |
+| `POST /bulk` body `{ data: [...] }` | unchanged; runs `postMany` |
+| `PATCH /bulk` body `{ operations: [{ id, data }] }` | `{ data: [{ type, id, attributes }] }`; runs `patchMany` |
+| `DELETE /bulk` body `{ ids: [...] }` | `{ data: [{ type, id }] }`; runs `deleteMany` |
+| `?atomic=true` / `?atomic=false` | removed; any query parameter is a 422 |
+| `expectedVersions` in the body | removed; versioned resources answer 422 |
+| 200/201 with `{ meta, errors, data }`, even when entries failed | 201 or 200 with `{ data }` when every row was written; otherwise an error response (404, 403 or 422) and nothing is written |
+
+### Types
+
+`types/bulk-methods` and its exports (`BulkResourceMethods`, `BulkPostInput`,
+`BulkPatchInput`, `BulkWriteResult`, `BulkDeleteResult`, `BulkMeta`,
+`BulkError`, `BulkTransactionOptions`) are removed. Resource handles carry the
+new methods; their parameter types are exported from the package root
+(`ManyRowWriteOptions`, `ManyRowCreateInput`, `ManyRowPatchInput`,
+`SharedPatchInput`, `UpsertOptions`, `ManyRowResult`, `ScopedQueryParams`,
+`RecordsResult` and others). `BulkOperationsPluginOptions` has only
+`maxBulkOperations`.
+
+## Objects in single-value fields
+
+Only `object` and `array` fields store a structure. A write, filter or cursor
+that gives any other field an object or array, as a `none` or `blob` field
+accepted from JSON, now fails with `RestApiValidationError` (rule
+`single_value`) before any SQL runs. In 2.x, MySQL drivers that format queries
+on the client (mysql2 before 3.17, mysql) expanded such an object into
+`column = value` pairs, so a value for one column could set others; PostgreSQL
+stored the JSON text and SQLite refused it. This also ships in 2.4.1.
+
+| 2.x | 3.0 |
+| --- | --- |
+| `{ type: 'none' }` holding an object | `{ type: 'object' }`, or a `storage.serialize` function |
+| `{ type: 'blob' }` given JSON | a `Buffer` or `Uint8Array` from server code |
+
+## `get` reads once
+
+A `get` on a resource with no `checkPermissions`, `checkDataPermissions`,
+`checkDataPermissionsGet`, `beforeData` or `beforeDataGet` hook (for the
+resource or global) reads its record in one statement, scope and row policy
+included, instead of a minimal read followed by a full read. Not found stays
+`not_found` and still answers before an invalid fieldset. Such a get no longer
+sets `context.minimalRecord`.
+
+| 2.x | 3.0 |
+| --- | --- |
+| `enrichRecord`, `finish` or `finishGet` reading `context.minimalRecord` | read `context.record`, or register a `checkPermissions` or `beforeDataGet` hook to keep the minimal read |
+
+## Write failure logs
+
+The `Error in METHOD method` entry that a failed write logs at error level now
+carries only the resource, the operation and error codes. In 2.x it also held
+the request's `inputRecord` (hidden fields redacted, every other value kept)
+and the formatted error, whose message and SQL can repeat the values being
+written, so logs collected payment state, ciphertext and other stored secrets.
+The error the call throws is unchanged and still carries its `cause`.
+
+| 2.x log field | 3.0 |
+| --- | --- |
+| `inputRecord` | removed; log what you need from the input yourself, in your own code |
+| `error`, `errorSummary` (message, stack, details) | `code`: the error's `code`, or its `name` when it has none |
+| `cleanupErrors: [{ phase, error }]` | `cleanupErrors: [{ phase, code }]` |
+| `method`, `scopeName`, `phase`, `backend`, `transactionOutcome` | unchanged |
+
+## HTTP error answers and logs
+
+A 500 answer for an error that is not one of the library's own `RestApi…Error`
+classes, such as a database error or a plain `Error` thrown by a hook, now has
+the detail `An unexpected error occurred`. In 2.x it carried the error's
+message, and a database error's message repeats the SQL with its values (Knex
+puts the statement and its bindings in it), so a duplicate key answered with
+the value that clashed. For a database error, the connector's `HTTP request
+error` or `Fastify request error` log entry now records only its `name` and
+`code`; PostgreSQL's `detail` and MySQL's `sql` repeat the values too.
+
+| 2.x | 3.0 |
+| --- | --- |
+| 500 `detail` is the error's message | the message for a `RestApi…Error`; otherwise `An unexpected error occurred` |
+| a hook throws `new Error(message)` to tell the client why | throw a `RestApiResourceError` or `RestApiValidationError`, or call `context.reject` in a transport hook |
+| the HTTP error log entry holds the database error (message, SQL, values) | `{ name, code }` of the database error |
+
+## System contexts and `context.system`
+
+The autofilter and row-policy plugins now reserve `context.system`. A context
+with `system: true` that was not made by `createSystemContext` throws
+`REST_API_SYSTEM_CONTEXT_FORGED`. If your application used its own `system`
+flag, rename it; if it used one to skip ownership filters, use
+`createSystemContext(reason)`.
+
+```js
+// Before (2.x): a hand-made flag that each hook had to check
+await api.resources.entries.query({}, { system: true })
+
+// After (3.0)
+await api.resources.entries.query({}, createSystemContext('monthly report'))
+```
+
+A system create must set every autofilter field explicitly; NULL creates a
+shared row on a `shared` filter. Row policies receive `system` alongside
+`context`.
+
+## Autofilter `shared`
+
+Filter definitions take `shared: true | false` (default false). A value other
+than a boolean, or `shared: true` on a field that is not `nullable`, fails when
+the resource is added. `getScopeConfig` reports `shared` for each filter.
+Outside a system context, shared rows refuse `patch`, `put`, `delete`,
+`postRelationship` and `deleteRelationship`, and a create may not leave the
+field NULL, also when an optional (`required: false`) filter does not resolve.
+
+## Values and links that read or write differently
+
+| 2.x | 3.0 |
+| --- | --- |
+| an `integer` field over a PostgreSQL BIGINT column reads `"4400"` | `4400`; a value past 2^53 stays a string |
+| an aggregate group, `min` or `max` of a BIGINT past 2^53 is a rounded number | the exact digits, as a string |
+| a `patch` or `put` of `accountId: null` by a writer who cannot see the account clears the link | keeps it; clear it in a system context |
+| `owner_type` and `owner_id` in attributes set a polymorphic link | refused; send `owner: { _type, id }` or `relationships.owner` |
+
+## Custom storage plugins
+
+The storage helpers gain `dataScopedQuery`, `dataVisibleIds`, `dataIdsByKeys`,
+`dataInsertMany`, `dataUpdateByIds`, `dataDeleteByIds`, `dataLockByIds` and
+`dataClaim`. They are optional: a storage plugin without them keeps working,
+and the new methods refuse its resources with
+`RestApiBulkWriteUnsupportedError`. `RestApiKnexPlugin` implements all of them;
+`RestApiAnyapiKnexPlugin` implements only `dataScopedQuery`.
+
+## Verify the port
+
+1. Search for `bulkPost`, `bulkPatch`, `bulkDelete`, `atomic`,
+   `expectedVersions`, `defaultAtomic` and `operations:`, and port each call.
+2. Search for `system: true` in contexts.
+3. Run the application's tests. A many-row write on a resource with per-record
+   hooks fails with `RestApiBulkWriteUnsupportedError`; its `details.hooks`
+   names the hooks to give a batch form.
+4. Search for `patchWhere` and `deleteWhere` calls without `filters`, and add
+   `all: true` where every visible row is meant.
+5. Update HTTP clients of `/RESOURCE/bulk` to the new PATCH and DELETE bodies
+   and to error responses for failed batches.

package/docs/GUIDE/index.md

@@ -9,7 +9,8 @@ resources over HTTP. Start with chapters 01–04, then follow the relationship
 tutorials or jump to the contract you need. Each runnable chapter uses a fresh
 database unless it explicitly says otherwise.
 
-Upgrading an application? Start with [33. Migrating to v2](33-migrating-to-v2.md).
+Upgrading an application? From 2.x, read [35. Migrating to v3](35-migrating-to-v3.md);
+from 1.x, start with [33. Migrating to v2](33-migrating-to-v2.md).
 Before choosing storage, read [30. Backend capabilities and limits](30-backend-capabilities.md).
 
 The examples require Node.js 24 or newer and familiarity with JavaScript and npm.
@@ -56,7 +57,7 @@ Development verification targets Node 24.
 - [24. CORS](24-cors.md)
 - [25. URL management](25-url-management.md)
 - [26. File uploads](26-file-uploads.md)
-- [27. Bulk operations](27-bulk-operations.md)
+- [27. Many-row writes](27-many-row-writes.md)
 - [28. Socket.IO integration](28-socketio.md)
 - [29. Writing plugins](29-writing-plugins.md)
 
@@ -66,3 +67,5 @@ Development verification targets Node 24.
 - [31. Positioning (experimental)](31-positioning.md)
 - [32. Date and time](32-date-and-time.md)
 - [33. Migrating to v2](33-migrating-to-v2.md)
+- [34. Scoped reads, locks and system work](34-scoped-reads-and-locks.md)
+- [35. Migrating to v3](35-migrating-to-v3.md)

package/docs/QUICKSTART.md

@@ -17,8 +17,8 @@ npm pkg set type=module
 npm install json-rest-api knex better-sqlite3 express
 ```
 
-These examples use the revised API described in the
-[migration guide](GUIDE/33-migrating-to-v2.md). Until that release is published,
+These examples use the v3 API; upgrading applications follow the
+[migration guide](GUIDE/35-migrating-to-v3.md). Until that release is published,
 install the intended library tarball in place of the registry package.
 
 ## Define resources

package/docs/contributing.md

@@ -70,7 +70,7 @@ application migration.
 ## Resource stress checks
 
 For smaller repeatable measurements, `node scripts/measure-write-lifecycles.js`
-records SQL counts and checks child-hook order for bulk and reverse-relationship
+records SQL counts and checks child-hook order for reverse-relationship
 writes. It supports both storage modes and the database runner.
 `node scripts/measure-diagnostics.js` reports bounded logger formatting cost;
 its local timings are observations, not pass/fail performance thresholds.

package/index.d.ts

@@ -3,7 +3,7 @@ export * from './types/file-storage.js'
 export type * from './types/representations.js'
 export type * from './types/resource-methods.js'
 export type * from './types/relationship-methods.js'
-export type * from './types/bulk-methods.js'
+export type * from './types/data-access-methods.js'
 export type { ManagedTransaction, ManagedTransactionParam, TransactionMethods } from './types/transactions.js'
 export type * from './types/plugin-options.js'
 export * from './types/anyapi.js'
@@ -31,6 +31,13 @@ export const RestApiAnyapiKnexPlugin: LibraryPlugin<'rest-api-anyapi-knex'>
 export const ExpressPlugin: LibraryPlugin<'express'>
 export const FastifyPlugin: LibraryPlugin<'fastify'>
 
+// A context for population-wide work: ownership filters are skipped, row policies get system: true.
+export function createSystemContext<Values extends Record<string, unknown> = Record<string, never>>(
+  reason: string, values?: Values
+): Values & { system: true; systemReason: string }
+export function isSystemContext(context: unknown): boolean
+export function readOnlyQueryBuilder<Builder extends import('knex').Knex.QueryBuilder>(builder: Builder, options?: { resourceType?: string }): Builder
+
 export function getUrlPrefix(
   context?: { urlPrefixOverride?: string; urlPrefix?: string } | null,
   scope?: { vars?: { transport?: { publicBaseUrl?: string; mountPath?: string } } } | null

package/index.js

@@ -25,4 +25,8 @@ export { LocalStorage } from './plugins/storage/local-storage.js'
 
 export * from './lib/rest-api-errors.js'
 
+// Population-wide work (jobs) and the SELECT-only builder of scopedQuery
+export { createSystemContext, isSystemContext } from './lib/system-context.js'
+export { readOnlyQueryBuilder } from './plugins/core/lib/querying/read-only-query.js'
+
 export { getUrlPrefix } from './plugins/core/lib/querying/url-helpers.js'

package/lib/rest-api-errors.js

@@ -255,3 +255,82 @@ export class RestApiTemporalDataError extends RestApiError {
     }
   }
 }
+
+/**
+ * A many-row operation cannot run on this resource as one statement per chunk.
+ * It never falls back to a per-row loop; the reason names what to change.
+ */
+export class RestApiBulkWriteUnsupportedError extends RestApiError {
+  constructor ({ resourceType, operation, reason, hooks } = {}) {
+    super(`${operation} on '${resourceType}' is not supported: ${reason}`, 'REST_API_BULK_WRITE_UNSUPPORTED')
+    this.type = 'rest_api_bulk_write_unsupported'
+    this.statusCode = 422
+    this.details = { resourceType, operation, reason, ...(hooks ? { hooks } : {}) }
+  }
+
+  toJSON () {
+    return { name: this.name, message: this.message, code: this.code, type: this.type, statusCode: this.statusCode, details: this.details, stack: this.stack }
+  }
+}
+
+/**
+ * Requested records are missing or outside the caller's scope. Only the count
+ * is reported, so the error does not reveal which identifiers exist elsewhere.
+ */
+export class RestApiRecordsNotFoundError extends RestApiResourceError {
+  constructor ({ resourceType, missing } = {}) {
+    super(`${missing} requested ${resourceType} record(s) not found`, { subtype: 'not_found', resourceType })
+    this.details = { resourceType, missing }
+  }
+}
+
+/** A MySQL multi-row insert could not prove which identifiers it generated; the transaction rolls back and the call is safe to retry. */
+export class RestApiBulkInsertIdsError extends RestApiError {
+  constructor ({ resourceType, expected, verified } = {}) {
+    super(`Could not verify the identifiers of ${expected} inserted ${resourceType} record(s)`, 'REST_API_BULK_INSERT_IDS')
+    this.type = 'rest_api_bulk_insert_ids'
+    this.details = { resourceType, expected, verified }
+  }
+
+  toJSON () {
+    return { name: this.name, message: this.message, code: this.code, type: this.type, details: this.details, stack: this.stack }
+  }
+}
+
+/** Row locks only last as long as a transaction; a lock taken outside one would end immediately. */
+export class RestApiLockOutsideTransactionError extends RestApiValidationError {
+  constructor ({ resourceType, operation } = {}) {
+    super(`${operation} on '${resourceType}' needs a transaction from api.transaction`, {
+      fields: ['transaction'],
+      violations: [{ field: 'transaction', rule: 'lock_transaction', message: 'Row locks need an active transaction' }]
+    })
+    this.type = 'rest_api_lock_outside_transaction'
+  }
+}
+
+/** A scoped query is SELECT-only; writes and locks go through the resource's write and lock methods. */
+export class RestApiScopedQueryReadOnlyError extends RestApiError {
+  constructor ({ resourceType, method } = {}) {
+    super(`The scoped query of '${resourceType}' is read-only: ${method}() is not allowed`, 'REST_API_SCOPED_QUERY_READ_ONLY')
+    this.type = 'rest_api_scoped_query_read_only'
+    this.details = { resourceType, method }
+  }
+
+  toJSON () {
+    return { name: this.name, message: this.message, code: this.code, type: this.type, details: this.details, stack: this.stack }
+  }
+}
+
+/** queryAll stopped at the caller's rowLimit instead of reading an unbounded set. */
+export class RestApiRowLimitError extends RestApiError {
+  constructor ({ resourceType, rowLimit, operation = 'queryAll' } = {}) {
+    const advice = operation === 'queryAll' ? 'read them in a background job or narrow the filters' : 'narrow the filters'
+    super(`${operation} on '${resourceType}' matched more than ${rowLimit} rows; ${advice}`, 'REST_API_ROW_LIMIT')
+    this.type = 'rest_api_row_limit'
+    this.details = { resourceType, rowLimit, operation }
+  }
+
+  toJSON () {
+    return { name: this.name, message: this.message, code: this.code, type: this.type, details: this.details, stack: this.stack }
+  }
+}

package/lib/runtime/json-rest-api.js

@@ -1,12 +1,15 @@
 export class Hooks {
   #events = new Map()
 
+  // batch: true marks a per-record handler whose work many-row operations do
+  // through a batch hook (or that does not apply to them), so they may skip it.
   add (event, name, options, handler, resource) {
     if (typeof handler !== 'function') throw new TypeError(`Hook '${name}' needs a function`)
-    const { beforeFunction, afterFunction, ...unsupported } = options
+    const { beforeFunction, afterFunction, batch = false, ...unsupported } = options
     if (Object.keys(unsupported).length || (beforeFunction && afterFunction)) {
       throw new TypeError(`Hook '${name}' requires at most one beforeFunction or afterFunction`)
     }
+    if (typeof batch !== 'boolean') throw new TypeError(`Hook '${name}' option batch must be a boolean`)
     const handlers = this.#events.get(event) || []
     const anchor = beforeFunction || afterFunction
     let index = handlers.length
@@ -15,7 +18,7 @@ export class Hooks {
       if (index < 0) throw new Error(`Hook '${event}' has no handler '${anchor}'`)
       if (afterFunction) index++
     }
-    handlers.splice(index, 0, { name, handler, resource })
+    handlers.splice(index, 0, { name, handler, resource, batch })
     this.#events.set(event, handlers)
   }
 
@@ -24,6 +27,17 @@ export class Hooks {
     return (this.#events.get(event) || []).some(entry => !entry.resource || entry.resource === resource)
   }
 
+  // Handlers of these events that would run for this resource and have no batch form.
+  perRecord (events, resource) {
+    const found = []
+    for (const event of events) {
+      for (const entry of this.#events.get(event) || []) {
+        if (!entry.batch && (!entry.resource || entry.resource === resource)) found.push({ event, name: entry.name })
+      }
+    }
+    return found
+  }
+
   async run (event, args) {
     for (const entry of [...(this.#events.get(event) || [])]) {
       if (entry.resource && entry.resource !== args.scope) continue
@@ -87,7 +101,8 @@ export class JsonRestApi {
       helpers: resource?.helpers || this.helpers,
       log: this.log,
       runHooks: (event, nextContext = context) => this.#hooks.run(event, this.#arguments(resource, {}, nextContext)),
-      hasHooks: (event, scope = resource) => this.#hooks.has(event, scope)
+      hasHooks: (event, scope = resource) => this.#hooks.has(event, scope),
+      perRecordHooks: (events, scope = resource) => this.#hooks.perRecord(events, scope)
     }
   }
 

package/lib/system-context.js

@@ -0,0 +1,42 @@
+// @ts-check
+
+// Module-private: only createSystemContext can attach the marker. Object spread
+// copies it, so contexts the library derives from a system context stay system.
+const systemMarker = Symbol('jsonRestApiSystemContext')
+const marker = Object.freeze({})
+
+/**
+ * A context for population-wide work, such as background jobs. Ownership
+ * predicates (autofilter) are skipped for it, row policies receive
+ * `system: true`, and creates must name their ownership values explicitly.
+ * Request input cannot produce one: the marker is a private symbol.
+ * @param {string} reason Why this work may see every owner's rows; shown in diagnostics
+ * @param {Record<string, unknown>} [values] Extra context fields for hooks and policies
+ */
+export function createSystemContext (reason, values = {}) {
+  if (typeof reason !== 'string' || !reason.trim()) {
+    throw new TypeError('createSystemContext needs a reason: a non-empty string naming the work')
+  }
+  if (values === null || typeof values !== 'object' || Array.isArray(values)) {
+    throw new TypeError('createSystemContext values must be an object')
+  }
+  return { ...values, system: true, systemReason: reason, [systemMarker]: marker }
+}
+
+/** @param {unknown} context */
+export function isSystemContext (context) {
+  return context !== null && typeof context === 'object' && /** @type {Record<symbol, unknown>} */ (context)[systemMarker] === marker
+}
+
+/**
+ * A context claiming `system: true` without the marker was not made by
+ * createSystemContext; trusting it would let a copied flag skip ownership.
+ * @param {unknown} context
+ */
+export function assertGenuineSystemFlag (context) {
+  if (context !== null && typeof context === 'object' && /** @type {{ system?: unknown }} */ (context).system === true && !isSystemContext(context)) {
+    const error = new Error('context.system is set, but the context was not created by createSystemContext')
+    Object.assign(error, { code: 'REST_API_SYSTEM_CONTEXT_FORGED' })
+    throw error
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "json-rest-api",
-  "version": "2.4.0",
+  "version": "3.0.0",
   "main": "index.js",
   "types": "index.d.ts",
   "type": "module",
@@ -83,6 +83,7 @@
     "test:databases:sqlite": "node scripts/test-databases.js better-sqlite3",
     "test:databases:pg": "node scripts/test-databases.js pg",
     "test:databases:mysql": "node scripts/test-databases.js mysql2",
+    "test:databases:mariadb": "node scripts/test-databases.js mariadb",
     "typecheck": "tsc --project tsconfig.json"
   },
   "repository": {
@@ -141,7 +142,7 @@
   "devDependencies": {
     "@socket.io/redis-adapter": "^8.3.0",
     "@types/node": "24.0.14",
-    "better-sqlite3": "^11.10.0",
+    "better-sqlite3": "^13.0.3",
     "busboy": "^1.6.0",
     "eslint": "^9.37.0",
     "express": "^5.1.0",