json-rest-api 2.1.0 → 2.2.0

package/docs/API.md

@@ -78,6 +78,7 @@ a separate context for each enlisted operation. See [transaction support](#trans
    - [PUT - Replace Resource](#put---replace-resource)
    - [PATCH - Update Resource](#patch---update-resource)
    - [DELETE - Remove Resource](#delete---remove-resource)
+   - [AGGREGATE - Grouped Totals](#aggregate---grouped-totals)
 3. [Relationship Methods](#relationship-methods)
    - [getRelated - Retrieve Related Resources](#getrelated---retrieve-related-resources)
    - [getRelationship - Retrieve Relationship Identifiers](#getrelationship---retrieve-relationship-identifiers)
@@ -1136,6 +1137,68 @@ savepoints cannot own library writes.
 
 ---
 
+### AGGREGATE - Grouped Totals
+
+Groups the records a collection query would select and computes measures for
+each group, on ordinary and AnyAPI storage alike.
+
+#### Method Signature
+
+```javascript
+await api.resources.sales.aggregate(params, context)
+```
+
+#### Parameters
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `groupBy` | string[] | Fields to group by. Omit for one row over all selected records. |
+| `measures` | object | Output names mapped to `{ count \| countDistinct \| sum \| avg \| min \| max: field }`. Only `count` accepts `'*'`. |
+| `queryParams.filters` | object | Search schema filters, validated as for `query`. |
+| `queryParams.sort` | string[] | Grouped field or measure names, `-` for descending. Grouped fields break ties. |
+| `queryParams.page` | object | `size` and `number`, for grouped results only. `size` defaults to `queryDefaultLimit` and cannot exceed `queryMaxLimit`. |
+| `transaction` | Knex transaction | Optional transaction to read in. |
+
+`sum` and `avg` take numeric fields; `min` and `max` also take strings, dates and
+times. Grouping and counting accept any scalar stored field, including belongs-to
+foreign keys. Fields that are `hidden`, virtual, computed, or that have a getter or
+`storage.serialize` are rejected, because SQL would group or sum values other than
+the ones clients read. `normallyHidden` fields are allowed.
+
+#### Return Value
+
+```javascript
+{
+  data: [{ salesperson: 'Ann', total: 150.5, count: 2 }],
+  meta: { page: { size: 20, number: 1, hasMore: false } } // grouped results only
+}
+```
+
+Both formats return these plain rows. Group values and `min`/`max` use the
+field's public representation; counts are numbers, and `sum`/`avg` are numbers
+except on decimal fields (`precision` and `scale`), which keep the driver's value.
+NULL sorts as the lowest value on every database.
+
+#### Security
+
+`aggregate` checks permissions as a `query` (`method: 'query'`, with
+`originalContext.method === 'aggregate'`). Row policies, autofilters and other
+`knexQueryFiltering` hooks run with `queryPurpose: 'aggregate'`. The aggregate reads
+only the IDs the filtered query selects, so joins added by filters or policies
+never count a record twice.
+
+#### Examples
+
+```javascript
+const totals = await api.resources.sales.aggregate({
+  groupBy: ['salesperson'],
+  measures: { total: { sum: 'amount' }, count: { count: '*' }, avg: { avg: 'amount' } },
+  queryParams: { filters: { closed: true }, sort: ['-total'], page: { size: 10 } }
+}, { subject })
+```
+
+---
+
 ## Relationship Methods
 
 ### getRelated - Retrieve Related Resources
@@ -2039,6 +2102,23 @@ database writes; it is not a persisted UI-state field. Use a setter when a
 virtual input needs to prepare other stored attributes. A `virtualFields` array
 is not the resource declaration contract. See [virtual-field behavior](GUIDE/12-field-transformations.md).
 
+#### Email and UUID Fields
+
+`type: 'email'` and `type: 'uuid'` store trimmed strings and reject values that
+are not an email address or a UUID. Filters with a text operator (`like`,
+`contains`, `startsWith`, `endsWith`) accept fragments. Registration rejects any
+type json-rest-schema cannot cast, so a misspelt type fails at `addResource`
+rather than on the first write.
+
+#### Retired Fields
+
+`retired: true` keeps a stored field readable, filterable and sortable while
+rejecting every value written to it, including `null`, with the `retired_field`
+rule. `required` and `defaultTo` no longer apply, and PUT leaves its stored value
+alone. Only stored attributes can be retired, not belongs-to fields. AnyAPI
+resources can retire fields at run time with
+[`retireKnexFields`](GUIDE/21-schema-and-migrations.md#retiring-and-migrating-anyapi-fields).
+
 #### Field Transformations
 
 Getters and setters receive a value and an injected context. Computed fields

package/docs/GUIDE/17-row-policies.md

@@ -286,6 +286,7 @@ Policies are reused for the resource selections that define externally visible d
 | --- | --- |
 | `collection` | Top-level collection query, before sorting and pagination. On AnyAPI this filtered builder is also cloned for the matching count. |
 | `count` | A separate count query, including native offset-pagination counts and standalone count helpers. |
+| `aggregate` | The records a resource `aggregate()` call groups and measures. |
 | `single` | Minimal lookup used by `get`, and by existing-record preflights for `put`, `patch`, and `delete`. |
 | `include` | A target resource loaded into JSON:API `included` data. |
 | `search-join` | A related resource used by a cross-resource or polymorphic search, reference filter, or reference sort, including intermediate search path segments. |

package/docs/GUIDE/21-schema-and-migrations.md

@@ -297,7 +297,8 @@ This optional map may cover a subset of the new stored fields; remaining fields
 use the existing allocator. An initial resource map must still cover all stored
 fields. Invalid, occupied or unknown slots and existing field names reject the
 addition. Computed and virtual fields need no slots and cannot have overrides.
-AnyAPI `alterKnexFields` remains explicitly unsupported.
+AnyAPI `alterKnexFields` remains explicitly unsupported; change a field's type with
+[`migrateField`](#retiring-and-migrating-anyapi-fields).
 
 One call uses one owned metadata transaction. Allocation failure rolls back the
 batch and leaves the runtime schema unchanged. Added getters, setters, defaults,
@@ -365,6 +366,7 @@ await api.anyapi.handleSchemaChange(change)
 
 `handleSchemaChange({ tenant, resource, version })` reloads the resource only when
 the version is newer than the published one, and returns whether it did.
+`api.anyapi.getSchemaVersion('items')` reads the stored version with one query.
 `api.resources.items.reloadSchema()` reloads unconditionally and resolves to the
 published version. Reloads are read-only: they combine the stored definition
 with this server's declaration and never re-register it.
@@ -384,6 +386,45 @@ of the tenant. A stored resource whose non-field options needed callbacks, such
 `normalizeId`, must be declared in code. To load resources on demand over HTTP,
 call `loadResource` from middleware placed before the API router.
 
+### Retiring and migrating AnyAPI fields
+
+Slots are never released while records exist, so AnyAPI retires fields instead of
+dropping them:
+
+```js
+await api.resources.contacts.retireKnexFields(['phone'])
+```
+
+A retired field keeps its slot and its values, stays readable, filterable and
+sortable, and rejects writes as a [`retired: true`](../API.md#retired-fields) field
+does. The mark is stored with the resource definition, so a server that registers
+an older declaration still listing the field keeps it retired. Its name stays taken.
+
+`migrateField` changes a field's type by copying its values into a new field:
+
+```js
+await api.resources.contacts.migrateField({
+  from: 'phone',
+  to: { name: 'phoneNumber', type: 'number', nullable: true, search: true },
+  transform: (value, { id }) => value === null ? null : Number(value.replace(/\D/g, ''))
+})
+```
+
+In one transaction it allocates a slot for the new field, converts every record's
+value, and retires the old field. Each converted value must pass the new field's
+validation; without `transform`, that validation casts the old value directly. A
+value it rejects names the record and rolls everything back. The new field must be
+a plain stored attribute: add fields with callbacks with `addKnexFields` and
+backfill them. Values are written directly, so setters, hooks and record versions
+do not run. Records are converted one by one, which suits per-tenant tables
+better than very large ones. Other servers keep accepting writes to the old field
+until the migration commits, and those writes do not reach the new field, so run
+it while nothing writes the field.
+
+Both methods bump `schema_version` and call `onSchemaChanged` once after commit.
+`AnyapiRegistry.retireFields({ tenant, resource, fields }, { transaction })` is the
+deep form of retirement.
+
 ### AnyAPI tenant instances
 
 One API instance per tenant stays cheap when the instances share their start-up work:

package/docs/GUIDE/33-migrating-to-v2.md

@@ -1038,6 +1038,15 @@ Use `HookHandler<Event, AppContext>` for known hooks and
 `RuntimeHook<PluginContext>` for arbitrary plugin events; see the
 [hook guide](13-hooks-and-lifecycle.md).
 
+## Field type names
+
+`addResource` now rejects a field whose type json-rest-schema cannot cast, naming
+the field and type. Such fields used to register and then fail on their first
+write with `No casting function for type`. AnyAPI no longer lists `text`, `float`,
+`decimal`, `datetime` or `json`, which no storage could write; use `string`,
+`number`, `dateTime` and `object` or `array`. `email` and `uuid` are now real
+string types with format checks.
+
 ## Object and array attributes
 
 Declare structured fields using their public JSON type:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "json-rest-api",
-  "version": "2.1.0",
+  "version": "2.2.0",
   "main": "index.js",
   "types": "index.d.ts",
   "type": "module",

package/plugins/core/lib/anyapi/anyapi-registry.js

@@ -243,22 +243,16 @@ export class AnyapiRegistry {
     })
   }
 
-  async #allocate (options, method, readParams) {
-    const context = { method, [readCommittedTransaction]: true }
-    let trx; let managed = false
-    try {
-      const { tenant, resource, fields, searchSchema } = readParams()
-      Object.assign(context, { tenant, resource })
+  #allocate (options, method, readParams) {
+    return this.#changeResource(options, method, () => {
+      const params = readParams()
+      const { resource, fields, searchSchema } = params
       for (const [fieldName, { definition }] of Object.entries(fields)) {
         validateSchemaFieldNames({ [fieldName]: definition }, resource)
       }
       assertFieldNameMap(searchSchema, `added search schema in '${resource}'`)
-
-      managed = !options.transaction
-      trx = await beginWriteTransaction(context, options.transaction, ownerContext => createKnexTransaction(this.knex, ownerContext))
-      const resourceRow = await this.#lockResourceRow(trx, tenant, resource)
-      const descriptor = resourceRow && await this.#loadDescriptor(tenant, resource, trx)
-      if (!descriptor) throw new Error(`Resource '${resource}' not registered for tenant '${tenant}'`)
+      return params
+    }, async ({ trx, resourceRow, descriptor }, { fields, searchSchema }) => {
       const plan = await this.#withSlotPools(trx, pools => this.#planAllocation(descriptor, fields, pools))
 
       for (const row of plan.fieldRows) {
@@ -280,11 +274,53 @@ export class AnyapiRegistry {
           runtime.callbacks.push(...callbacks.map(path => [kind, name, ...path]))
         }
       }
-      await this.#bumpVersion(trx, resourceRow, {
+      return {
         schema_json: JSON.stringify({ ...descriptor.schema, ...plan.definitions }),
-        definition_json: JSON.stringify({ ...stored, runtime }),
-        updated_at: this.knex.fn.now()
-      })
+        definition_json: JSON.stringify({ ...stored, runtime })
+      }
+    })
+  }
+
+  /**
+   * Mark stored fields retired: they keep their slots and values, and the
+   * published schema stops accepting writes to them. Registration keeps the mark.
+   */
+  async retireFields (params, options = {}) {
+    return this.#changeResource(options, 'retireFields', () => {
+      const { tenant, resource, fields } = params || {}
+      if (!tenant || !resource || !Array.isArray(fields) || !fields.length || fields.some(field => typeof field !== 'string')) {
+        throw new Error('retireFields requires tenant, resource, and an array of field names')
+      }
+      return { tenant, resource, fields }
+    }, ({ descriptor }, { fields }) => {
+      for (const fieldName of fields) {
+        const field = descriptor.fields[fieldName]
+        if (!field) throw new Error(`Field '${descriptor.resource}.${fieldName}' has no stored slot to retire`)
+        if (field.slotType === 'belongsTo') throw new Error(`Field '${descriptor.resource}.${fieldName}' is a relationship and cannot be retired`)
+      }
+      const stored = descriptor.definition || { options: null, callbacks: [] }
+      const retired = [...new Set([...(stored.retired || []), ...fields])]
+      return { definition_json: JSON.stringify({ ...stored, retired }) }
+    })
+  }
+
+  // Run one schema change under the resource's row lock and bump its version once.
+  // `change` returns the any_resource_configs values to write.
+  async #changeResource (options, method, readParams, change) {
+    const context = { method, [readCommittedTransaction]: true }
+    let trx; let managed = false
+    try {
+      const params = readParams()
+      const { tenant, resource } = params
+      Object.assign(context, { tenant, resource })
+
+      managed = !options.transaction
+      trx = await beginWriteTransaction(context, options.transaction, ownerContext => createKnexTransaction(this.knex, ownerContext))
+      const resourceRow = await this.#lockResourceRow(trx, tenant, resource)
+      const descriptor = resourceRow && await this.#loadDescriptor(tenant, resource, trx)
+      if (!descriptor) throw new Error(`Resource '${resource}' not registered for tenant '${tenant}'`)
+      const values = await change({ trx, resourceRow, descriptor }, params)
+      await this.#bumpVersion(trx, resourceRow, { ...values, updated_at: this.knex.fn.now() })
       const updated = await this.#loadDescriptor(tenant, resource, trx)
       if (managed) {
         await commitTransaction(trx, context)
@@ -551,7 +587,13 @@ export class AnyapiRegistry {
       const row = {
         schema_json: JSON.stringify(storageSchema),
         relationships_json: JSON.stringify(relationships || {}),
-        definition_json: JSON.stringify({ options: declaration.json, callbacks: declaration.callbacks, runtime }),
+        definition_json: JSON.stringify({
+          options: declaration.json,
+          callbacks: declaration.callbacks,
+          runtime,
+          // Retirement outlives declarations that still list the field.
+          ...(stored?.retired?.length ? { retired: stored.retired } : {})
+        }),
       }
       const unchanged = previous &&
         Object.entries(row).every(([column, value]) => previous.resourceRow[column] === value) &&

package/plugins/core/lib/anyapi/schema-utils.js

@@ -270,22 +270,17 @@ export const readSlotPools = async (db) => {
 
 export const TYPE_TO_POOL = new Map([
   ['string', 'string'],
-  ['text', 'string'],
   ['uuid', 'string'],
   ['email', 'string'],
   ['file', 'string'],
   ['number', 'number'],
   ['integer', 'number'],
-  ['float', 'number'],
-  ['decimal', 'number'],
   ['boolean', 'boolean'],
   ['date', 'string'],
-  ['datetime', 'date'],
   ['dateTime', 'date'],
   ['time', 'string'],
   ['epochMilliseconds', 'number'],
   ['epochSeconds', 'number'],
-  ['json', 'json'],
   ['object', 'json'],
   ['array', 'json'],
 ])

package/plugins/core/lib/anyapi/stored-definition.js

@@ -72,6 +72,9 @@ export const composeResourceOptions = ({ declaration = {}, stored = null, local
   merge('searchSchema', searchSchema, declaredSearch, local.searchSchema, hiddenFilters)
 
   for (const name of hidden) delete schema[name]
+  for (const name of stored?.retired || []) {
+    if (Object.hasOwn(schema, name)) schema[name] = { ...schema[name], retired: true }
+  }
   for (const [name, definition] of Object.entries(searchSchema)) {
     if (hiddenFilters.has(name) || targetsOf(definition).some(target => hidden.has(target))) {
       hiddenFilters.add(name)

package/plugins/core/lib/querying/aggregate-query.js

@@ -0,0 +1,170 @@
+import { RestApiValidationError } from '../../../../lib/rest-api-errors.js'
+import { applyDatabaseReadOptions, normalizeStoredFieldValue } from '../querying-writing/database-value-normalizers.js'
+
+const NUMERIC_TYPES = new Set(['number', 'integer', 'epochMilliseconds', 'epochSeconds'])
+const ORDERED_TYPES = new Set([...NUMERIC_TYPES, 'string', 'email', 'uuid', 'date', 'dateTime', 'time'])
+const SCALAR_TYPES = new Set([...ORDERED_TYPES, 'boolean', 'id'])
+// Measure functions and the field types each accepts.
+const MEASURES = {
+  count: SCALAR_TYPES,
+  countDistinct: SCALAR_TYPES,
+  sum: NUMERIC_TYPES,
+  avg: NUMERIC_TYPES,
+  min: ORDERED_TYPES,
+  max: ORDERED_TYPES,
+}
+const PARAMS = new Set(['groupBy', 'measures', 'queryParams', 'transaction'])
+const QUERY_PARAMS = new Set(['filters', 'sort', 'page'])
+const OUTPUT_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/
+const AGGREGATE_ALIAS = '__jra_aggregate'
+
+const invalid = (field, message) => {
+  throw new RestApiValidationError(message, { fields: [field], violations: [{ field, rule: 'aggregate', message }] })
+}
+
+const isPlainObject = value => value !== null && typeof value === 'object' && !Array.isArray(value)
+
+const isDecimal = definition => definition.type === 'number' && definition.precision !== undefined && definition.scale !== undefined
+
+const positiveInteger = (value, field) => {
+  const number = typeof value === 'string' && value.trim() !== '' ? Number(value) : value
+  if (!Number.isSafeInteger(number) || number < 1) invalid(field, `${field} must be a positive integer`)
+  return number
+}
+
+/**
+ * Validate an aggregate request against a resource's compiled schema.
+ * Only stored scalar fields whose stored values are their public values qualify:
+ * getters and storage serializers would make SQL group or sum other values than clients see.
+ * @param {{ params: Record<string, any>, schemaInfo: Record<string, any>, scopeName: string, defaultLimit: number, maxLimit: number }} request
+ */
+export function compileAggregateRequest ({ params, schemaInfo, scopeName, defaultLimit, maxLimit }) {
+  if (!isPlainObject(params)) invalid('params', 'aggregate requires a parameter object')
+  for (const key of Object.keys(params)) {
+    if (!PARAMS.has(key)) invalid(key, `Unknown aggregate parameter '${key}'`)
+  }
+  const { groupBy = [], measures, queryParams = {} } = params
+  if (!isPlainObject(queryParams)) invalid('queryParams', 'queryParams must be an object')
+  for (const key of Object.keys(queryParams)) {
+    if (!QUERY_PARAMS.has(key)) invalid(`queryParams.${key}`, `aggregate does not support queryParams.${key}`)
+  }
+
+  const fields = schemaInfo.schemaStructure || {}
+  const fieldDefinition = (name, parameter) => {
+    const definition = typeof name === 'string' && Object.hasOwn(fields, name) ? fields[name] : null
+    if (!definition || name === 'id' || name === schemaInfo.idProperty || definition.virtual === true || definition.hidden === true) {
+      invalid(parameter, `'${name}' is not an attribute of '${scopeName}' that can be aggregated`)
+    }
+    if (typeof definition.getter === 'function' || definition.storage?.serialize) {
+      invalid(parameter, `Field '${name}' transforms its stored values, so it cannot be aggregated`)
+    }
+    return definition
+  }
+
+  if (!Array.isArray(groupBy)) invalid('groupBy', 'groupBy must be an array of field names')
+  const groups = groupBy.map((field, index) => {
+    const definition = fieldDefinition(field, `groupBy.${index}`)
+    if (!SCALAR_TYPES.has(definition.type)) invalid(`groupBy.${index}`, `Field '${field}' has type '${definition.type}' and cannot be grouped`)
+    return { name: field, field, definition, alias: `g${index}` }
+  })
+  if (new Set(groupBy).size !== groupBy.length) invalid('groupBy', 'groupBy lists a field twice')
+
+  if (!isPlainObject(measures) || !Object.keys(measures).length) invalid('measures', 'measures must name at least one measure')
+  const names = new Set(groupBy)
+  const measureList = Object.entries(measures).map(([name, spec], index) => {
+    const parameter = `measures.${name}`
+    if (!OUTPUT_NAME.test(name)) invalid(parameter, `Measure name '${name}' must be a letter or underscore followed by letters, digits or underscores`)
+    if (names.has(name)) invalid(parameter, `Measure '${name}' has the name of a grouped field`)
+    names.add(name)
+    const entries = isPlainObject(spec) ? Object.entries(spec) : []
+    if (entries.length !== 1 || !Object.hasOwn(MEASURES, entries[0][0])) {
+      invalid(parameter, `Measure '${name}' must be one of { ${Object.keys(MEASURES).join(' | ')}: field }`)
+    }
+    const [fn, field] = entries[0]
+    if (field === '*') {
+      if (fn !== 'count') invalid(parameter, "Only count accepts '*'")
+      return { name, fn, field, definition: null, alias: `m${index}` }
+    }
+    const definition = fieldDefinition(field, parameter)
+    if (!MEASURES[fn].has(definition.type)) invalid(parameter, `${fn} cannot be applied to field '${field}' of type '${definition.type}'`)
+    return { name, fn, field, definition, alias: `m${index}` }
+  })
+
+  const outputs = new Map([...groups, ...measureList].map(output => [output.name, output.alias]))
+  const sortParam = queryParams.sort ?? []
+  if (!Array.isArray(sortParam)) invalid('queryParams.sort', 'queryParams.sort must be an array')
+  const sort = sortParam.map((entry, index) => {
+    const name = typeof entry === 'string' ? entry.replace(/^-/, '') : null
+    if (!outputs.has(name)) invalid(`queryParams.sort.${index}`, `Sort '${entry}' must name a grouped field or a measure`)
+    return { alias: outputs.get(name), direction: entry.startsWith('-') ? 'desc' : 'asc' }
+  })
+  // Grouped fields break ties, so pages never overlap.
+  for (const { alias } of groups) {
+    if (!sort.some(entry => entry.alias === alias)) sort.push({ alias, direction: 'asc' })
+  }
+
+  let page = null
+  if (groups.length) {
+    const pageParam = queryParams.page ?? {}
+    if (!isPlainObject(pageParam)) invalid('queryParams.page', 'queryParams.page must be an object')
+    for (const key of Object.keys(pageParam)) {
+      if (!['size', 'number'].includes(key)) invalid(`queryParams.page.${key}`, `aggregate pages support only size and number, not ${key}`)
+    }
+    const size = pageParam.size === undefined ? defaultLimit : positiveInteger(pageParam.size, 'queryParams.page.size')
+    if (size > maxLimit) invalid('queryParams.page.size', `queryParams.page.size cannot exceed ${maxLimit}`)
+    const number = pageParam.number === undefined ? 1 : positiveInteger(pageParam.number, 'queryParams.page.number')
+    page = { size, number }
+  } else if (queryParams.page !== undefined) {
+    invalid('queryParams.page', 'An aggregate without groupBy returns one row and takes no page')
+  }
+
+  return { groups, measures: measureList, sort, page, scopeName }
+}
+
+const measureValue = (value, measure, resourceType) => {
+  if (measure.fn === 'count' || measure.fn === 'countDistinct') return Number(value ?? 0)
+  if (value == null) return null
+  if (measure.fn === 'min' || measure.fn === 'max') return normalizeStoredFieldValue(value, measure.definition, { fieldName: measure.field, resourceType })
+  // Drivers return numeric aggregates as strings; decimals keep theirs to avoid rounding.
+  return isDecimal(measure.definition) ? value : Number(value)
+}
+
+/**
+ * Run a compiled aggregate over the rows a filtered query selects.
+ * The filtered query only supplies identifiers, so joins added by filters or
+ * policies cannot repeat a row inside a group.
+ * @param {{ source: import('knex').Knex.QueryBuilder, sourceTable: string, storageAdapter: Record<string, any>, db: import('knex').Knex | import('knex').Knex.Transaction, plan: ReturnType<typeof compileAggregateRequest> }} request
+ */
+export async function runAggregateQuery ({ source, sourceTable, storageAdapter, db, plan }) {
+  const idColumn = storageAdapter.getIdColumn()
+  const ids = source.clearSelect().clearOrder().select(`${sourceTable}.${idColumn}`)
+  const query = storageAdapter.buildBaseQuery({ transaction: db, tableAlias: AGGREGATE_ALIAS })
+    .whereIn(`${AGGREGATE_ALIAS}.${idColumn}`, ids)
+  const column = field => `${AGGREGATE_ALIAS}.${storageAdapter.translateColumn(field)}`
+
+  for (const group of plan.groups) {
+    query.select({ [group.alias]: column(group.field) }).groupBy(column(group.field))
+  }
+  for (const measure of plan.measures) {
+    query[measure.fn]({ [measure.alias]: measure.field === '*' ? '*' : column(measure.field) })
+  }
+  // NULL sorts as the lowest value, as MySQL and SQLite order it by default.
+  const pg = ['pg', 'postgresql'].includes(db.client.config.client)
+  for (const { alias, direction } of plan.sort) {
+    if (pg) query.orderBy(alias, direction, direction === 'asc' ? 'first' : 'last')
+    else query.orderBy(alias, direction)
+  }
+  if (plan.page) query.limit(plan.page.size + 1).offset((plan.page.number - 1) * plan.page.size)
+
+  const rows = await applyDatabaseReadOptions(query)
+  const hasMore = Boolean(plan.page && rows.length > plan.page.size)
+  if (hasMore) rows.pop()
+
+  const data = rows.map(row => {
+    const result = {}
+    for (const group of plan.groups) result[group.name] = normalizeStoredFieldValue(row[group.alias], group.definition, { fieldName: group.field, resourceType: plan.scopeName })
+    for (const measure of plan.measures) result[measure.name] = measureValue(row[measure.alias], measure, plan.scopeName)
+    return result
+  })
+  return plan.page ? { data, meta: { page: { ...plan.page, hasMore } } } : { data }
+}

package/plugins/core/lib/querying-writing/compile-schemas.js

@@ -5,6 +5,7 @@ import { compileQueryFields } from './query-field-helpers.js'
 import { buildEffectiveSortList } from '../querying/query-field-sort-helpers.js'
 import { assertScalarQueryField, assertFieldNameMap } from './field-utils.js'
 import { validateRelationshipFieldNames, validateSchemaFieldNames } from './scope-validations.js'
+import { assertKnownFieldTypes, installFieldTypes } from './field-types.js'
 
 /**
  * Compile one resource and publish its completed metadata in vars.schemaInfo.
@@ -74,7 +75,22 @@ export async function compileSchemas (scope, deps) {
   assertFieldNameMap(schemaContext.queryFields, `query fields in '${scopeName}'`)
 
   // Create schema object
+  // A retired field keeps its stored values readable and accepts no new ones.
+  for (const [fieldName, fieldDef] of Object.entries(schemaContext.fields)) {
+    if (fieldDef.retired === undefined) continue
+    if (typeof fieldDef.retired !== 'boolean') throw new Error(`Field '${fieldName}' in resource '${scopeName}' has a non-boolean retired option`)
+    if (fieldDef.retired && (fieldDef.belongsTo || fieldDef.virtual === true)) {
+      throw new Error(`Field '${fieldName}' in resource '${scopeName}' cannot be retired: only stored attributes can`)
+    }
+    if (fieldDef.retired) {
+      delete fieldDef.required
+      delete fieldDef.defaultTo
+    }
+  }
+
+  installFieldTypes()
   const schemaObject = createSchema(schemaContext.fields)
+  assertKnownFieldTypes(schemaObject, `resource '${scopeName}'`)
 
   // Generate searchSchema by merging explicit searchSchema with fields marked search:true.
   // This allows two ways to define searchable fields: either mark fields with search:true
@@ -98,6 +114,7 @@ export async function compileSchemas (scope, deps) {
   assertFieldNameMap(searchSchemaContext.fields, `search schema in '${scopeName}'`)
   ensureSearchFieldsAreIndexed(searchSchemaContext.fields)
   const searchSchemaObject = createSchema(searchSchemaContext.fields)
+  assertKnownFieldTypes(searchSchemaObject, `the search schema of '${scopeName}'`)
 
   // Derived fields must be rebuilt from the current candidate before publication.
   const idProperty = scopeOptions.idProperty || scope.vars.idProperty || 'id'

package/plugins/core/lib/querying-writing/database-value-normalizers.js

@@ -416,6 +416,22 @@ export function normalizeAttributes (attributes, schemaStructure, {
   return normalized
 }
 
+/**
+ * Convert one value read from a field's storage column to its public value,
+ * without the field's getter.
+ * @param {unknown} value
+ * @param {Record<string, any>} definition
+ * @param {{ fieldName: string, resourceType?: string }} options
+ */
+export function normalizeStoredFieldValue (value, definition, { fieldName, resourceType }) {
+  if (value == null) return null
+  if (definition.type === 'id') return String(value)
+  const decimal = definition.precision !== undefined && definition.scale !== undefined
+  // Drivers return some numeric columns as strings; decimals keep theirs to avoid rounding.
+  if (['number', 'integer'].includes(definition.type) && typeof value === 'string' && !decimal) return Number(value)
+  return normalizeAttributes({ [fieldName]: value }, { [fieldName]: definition }, { resourceType })[fieldName]
+}
+
 /**
  * Normalize a JSON:API document or one plain resource using compiled output fields.
  * JSON:API primary/included attributes are replaced on the supplied document;

package/plugins/core/lib/querying-writing/default-data-helpers.js

@@ -24,6 +24,10 @@ export const defaultDataHelpers = {
     throw new Error('No storage implementation for query. Install a storage plugin.')
   },
 
+  dataAggregate: async function (request) {
+    throw new Error('No storage implementation for aggregate. Install a storage plugin.')
+  },
+
   dataPost: async function (request) {
     throw new Error('No storage implementation for post. Install a storage plugin.')
   },

package/plugins/core/lib/querying-writing/field-types.js

@@ -0,0 +1,43 @@
+import { addType } from 'json-rest-schema'
+
+const EMAIL_PATTERN = /^[^\s@]+@[^\s@]+\.[^\s@]+$/
+const UUID_PATTERN = /^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$/
+const TEXT_SEARCH_OPERATORS = new Set(['like', 'contains', 'startswith', 'endswith'])
+
+// Partial-match filters search fragments of a value, not whole values.
+const isTextSearch = definition => TEXT_SEARCH_OPERATORS.has(String(definition?.filterOperator).toLowerCase())
+
+const formattedStringType = (pattern, format) => {
+  const type = (context) => {
+    if (typeof context.value !== 'string') context.throwTypeError()
+    const value = context.definition.noTrim ? context.value : context.value.trim()
+    if (!isTextSearch(context.definition) && !pattern.test(value)) context.throwTypeError()
+    return value
+  }
+  type.toJsonSchema = ({ definition }) => isTextSearch(definition) ? { type: 'string' } : { type: 'string', format, pattern: pattern.source }
+  return type
+}
+
+let installed = false
+
+/** Register the string types json-rest-schema lacks; both storages keep them in string columns. */
+export function installFieldTypes () {
+  if (installed) return
+  installed = true
+  addType('email', formattedStringType(EMAIL_PATTERN, 'email'))
+  addType('uuid', formattedStringType(UUID_PATTERN, 'uuid'))
+}
+
+/**
+ * Reject field types no casting function exists for, at registration rather than on the first write.
+ * @param {{ structure: Record<string, any>, types: Record<string, unknown> }} schema
+ * @param {string} label
+ */
+export function assertKnownFieldTypes (schema, label) {
+  for (const [fieldName, definition] of Object.entries(schema.structure)) {
+    const type = definition?.type
+    if (typeof type === 'string' && !Object.hasOwn(schema.types, type)) {
+      throw new Error(`Field '${fieldName}' in ${label} has unknown type '${type}'`)
+    }
+  }
+}

package/plugins/core/lib/querying-writing/request-contracts.js

@@ -354,7 +354,7 @@ function buildWritableAttributesStructure (schemaInfo = {}) {
     if (!fieldDef) continue
     if (fieldName === idProperty) continue
     if (excludedFieldNames.has(fieldName)) continue
-    if (fieldDef.computed === true) continue
+    if (fieldDef.computed === true || fieldDef.retired === true) continue
     if (fieldDef.type === undefined) continue
     attributesStructure[fieldName] = { ...fieldDef }
   }

package/plugins/core/lib/storage/canonical-data-helpers.js

@@ -5,6 +5,7 @@ import { applySortDescriptorOrder, buildEffectiveSortList, parseSortEntry, resol
 import { normalizeId, resolveFieldInfo } from '../anyapi/utils/descriptor-helpers.js'
 import { decorateResourceLinks, getAllowedQueryFieldNames } from '../anyapi/canonical-relationship-reader.js'
 import { applyQueryConstraint } from '../querying/query-constraint.js'
+import { runAggregateQuery } from '../querying/aggregate-query.js'
 import { calculatePaginationMeta, generatePaginationLinks, generateCursorPaginationLinks, buildCursorMeta, applyPaginationToQuery } from '../querying/knex-pagination-helpers.js'
 import { getUrlPrefix, buildJsonApiLink } from '../querying/url-helpers.js'
 import { applyFieldSelectionToQuery, buildFieldSelection } from '../querying-writing/knex-field-helpers.js'
@@ -670,6 +671,33 @@ export function createCanonicalDataHelpers ({
     return response
   }
 
+  /** @type {CanonicalDataReadHelpers['dataAggregate']} */
+  const dataAggregate = async ({ scopeName, context, runHooks, plan }) => {
+    const storageAdapter = getScopeStorageAdapter(scopeName)
+    if (!storageAdapter) throw new Error(`Storage adapter for resource '${scopeName}' not found`)
+    context.storageAdapter = storageAdapter
+    const descriptor = getDescriptor(scopeName)
+    const db = context.db || context.transaction || api.knex.instance
+    const tableAlias = context.schemaInfo.tableName || descriptor.resource
+
+    const filteredState = await withQueryFilteringContext(context, {
+      query: storageAdapter.buildBaseQuery({ transaction: db, tableAlias }),
+      filters: context.queryParams.filters,
+      schemaInfo: context.schemaInfo,
+      scopeName,
+      tableName: tableAlias,
+      db,
+      queryPurpose: 'aggregate',
+      isAnyApi: true,
+      adapter: storageAdapter,
+      storageAdapter,
+    }, async () => {
+      await applyBuiltInAnyApiQueryFilters(context)
+      await runHooks('knexQueryFiltering')
+    })
+    return runAggregateQuery({ source: filteredState.query, sourceTable: tableAlias, storageAdapter, db, plan })
+  }
+
   return {
     dataExists,
     dataPost,
@@ -678,6 +706,7 @@ export function createCanonicalDataHelpers ({
     dataDelete,
     dataGet,
     dataQuery,
+    dataAggregate,
     dataGetMinimal
   }
 }

package/plugins/core/lib/storage/ordinary-data-helpers.js

@@ -9,6 +9,7 @@ import { toJsonApiRecordWithBelongsTo } from '../querying-writing/knex-json-api-
 import { processIncludes } from '../querying/knex-process-includes.js'
 import { loadRelationshipIdentifiers } from '../querying/relationship-identifiers.js'
 import { applyQueryConstraint } from '../querying/query-constraint.js'
+import { runAggregateQuery } from '../querying/aggregate-query.js'
 import { RestApiResourceError } from '../../../../lib/rest-api-errors.js'
 import { getOperationDiagnosticContext } from '../../../../lib/error-context.js'
 import { applyInsertReturning } from '../querying-writing/database-capabilities.js'
@@ -505,6 +506,29 @@ export function createOrdinaryDataHelpers ({ api, scopes, knex, log, getScopeSto
     return buildJsonApiResponse(scope, records, included, false, scopeName, context)
   }
 
+  /** @type {OrdinaryDataReadHelpers['dataAggregate']} */
+  const dataAggregate = async ({ scopeName, context, runHooks, plan }) => {
+    const storageAdapter = getScopeStorageAdapter(scopeName)
+    context.storageAdapter = storageAdapter
+    const tableName = storageAdapter.getTableName()
+    const db = context.db || api.knex.instance
+
+    const filteredState = await withQueryFilteringContext(context, {
+      query: db(tableName),
+      filters: context.queryParams.filters,
+      schemaInfo: context.schemaInfo,
+      scopeName,
+      tableName,
+      db,
+      queryPurpose: 'aggregate',
+      adapter: storageAdapter,
+      storageAdapter,
+    }, async () => {
+      await runHooks('knexQueryFiltering')
+    })
+    return runAggregateQuery({ source: filteredState.query, sourceTable: tableName, storageAdapter, db, plan })
+  }
+
   /** @type {OrdinaryDataWriteHelpers['dataPost']} */
   const dataPost = async ({ scopeName, context }) => {
     const storageAdapter = getScopeStorageAdapter(scopeName)
@@ -701,6 +725,7 @@ export function createOrdinaryDataHelpers ({ api, scopes, knex, log, getScopeSto
     dataExists,
     dataGet,
     dataQuery,
+    dataAggregate,
     dataPost,
     dataPut,
     dataPatch,

package/plugins/core/lib/storage/storage-types.d.ts

@@ -307,6 +307,27 @@ export interface DataReadHelpers<GetResult> {
       queryParams: NonNullable<DataReadContext['queryParams']>
     }
   }) => Promise<DataDocument<DataResource[]>>
+  // The plan comes from compileAggregateRequest; filters are in context.queryParams.
+  dataAggregate: (request: DataReadRequest & {
+    runHooks: NonNullable<DataReadRequest['runHooks']>
+    context: DataReadContext & {
+      queryParams: NonNullable<DataReadContext['queryParams']>
+    }
+    plan: AggregatePlan
+  }) => Promise<AggregateResult>
+}
+
+// Compiled by compileAggregateRequest: outputs carry SQL aliases.
+export interface AggregatePlan {
+  scopeName: string
+  groups: Array<{ name: string; field: string; definition: StorageFieldDefinition; alias: string }>
+  measures: Array<{ name: string; fn: 'count' | 'countDistinct' | 'sum' | 'avg' | 'min' | 'max'; field: string; definition: StorageFieldDefinition | null; alias: string }>
+  sort: Array<{ alias: string; direction: 'asc' | 'desc' }>
+  page: { size: number; number: number } | null
+}
+export interface AggregateResult {
+  data: Array<Record<string, unknown>>
+  meta?: { page: { size: number; number: number; hasMore: boolean } }
 }
 
 export type OrdinaryDataReadHelpers = DataReadHelpers<DataDocument<DataResource>>

package/plugins/core/rest-api-anyapi-knex-plugin.js

@@ -6,18 +6,25 @@ import { createEnhancedLogger } from '../../lib/enhanced-logger.js'
 import { ensureAnyApiSchema } from './lib/anyapi/schema-utils.js'
 import { AnyapiRegistry } from './lib/anyapi/anyapi-registry.js'
 import { composeResourceOptions, relatedResources } from './lib/anyapi/stored-definition.js'
-import { RestApiResourceError } from '../../lib/rest-api-errors.js'
+import { RestApiResourceError, RestApiValidationError } from '../../lib/rest-api-errors.js'
 import { compileSchemas } from './lib/querying-writing/compile-schemas.js'
 import { snapshotResourceConfiguration } from './lib/querying-writing/schema-helpers.js'
 import { assertFieldNameMap } from './lib/querying-writing/field-utils.js'
 import { polymorphicFiltersHook, crossTableFiltersHook, basicFiltersHook } from './lib/querying/knex-query-helpers.js'
 import { createStorageAdapterLookup } from './lib/storage/storage-adapter.js'
+import { translateCanonicalAttributesForStorage } from './lib/storage/canonical-storage-mapping.js'
+import { applyDatabaseReadOptions, normalizeStoredFieldValue } from './lib/querying-writing/database-value-normalizers.js'
 
 const DEFAULT_TENANT = 'default'
+const MIGRATION_BATCH_SIZE = 500
 // Methods that read a published schema; schema checks wrap these.
-const REQUEST_METHODS = ['query', 'get', 'post', 'put', 'patch', 'delete', 'getRelated', 'getRelationship', 'postRelationship', 'patchRelationship', 'deleteRelationship']
+const REQUEST_METHODS = ['query', 'aggregate', 'get', 'post', 'put', 'patch', 'delete', 'getRelated', 'getRelationship', 'postRelationship', 'patchRelationship', 'deleteRelationship']
 
-const optionsSignature = options => JSON.stringify([Object.keys(options.schema || {}).sort(), Object.keys(options.searchSchema || {}).sort()])
+const optionsSignature = options => JSON.stringify([
+  Object.keys(options.schema || {}).sort(),
+  Object.keys(options.searchSchema || {}).sort(),
+  Object.entries(options.schema || {}).filter(([, definition]) => definition?.retired === true).map(([name]) => name).sort()
+])
 
 // A validation failure may come from a schema another server already changed.
 // Writes that may have committed are never repeated.
@@ -252,6 +259,7 @@ export const RestApiAnyapiKnexPlugin = {
       for (const { resource } of await registry.listResources(tenantId)) await loadResource(resource)
       return api.resources
     }
+    api.anyapi.getSchemaVersion = resource => registry.getSchemaVersion(tenantId, resource)
     api.anyapi.handleSchemaChange = async ({ tenant, resource, version } = {}) => {
       if ((tenant !== undefined && tenant !== tenantId) || typeof resource !== 'string' || !Object.hasOwn(api.resources, resource)) return false
       if (version !== undefined && publishedVersion(resource) >= version) return false
@@ -330,49 +338,134 @@ export const RestApiAnyapiKnexPlugin = {
 
     addResourceMethod('reloadSchema', async ({ scopeName }) => reload(scopeName))
 
-    addResourceMethod('addKnexFields', async ({ scopeName, params }) => {
-      if (!params?.fields || typeof params.fields !== 'object' || Array.isArray(params.fields)) {
-        throw new Error('fields parameter is required for addKnexFields')
-      }
-      assertFieldNameMap(params.fields, `added fields in '${scopeName}'`)
-      assertFieldNameMap(params.searchSchema, `added search schema in '${scopeName}'`)
-      assertFieldNameMap(params.canonicalFieldsMap, `added canonical fields in '${scopeName}'`)
-      const scope = api.resources[scopeName]
-      const unknown = Object.keys(params.canonicalFieldsMap || {}).filter(field => !Object.hasOwn(params.fields, field))
-      if (unknown.length) {
-        throw new Error(`canonicalFieldsMap contains unknown fields for addKnexFields: ${unknown}`)
-      }
-      const fields = snapshotResourceConfiguration(params.fields)
-      const searchSchema = snapshotResourceConfiguration(params.searchSchema || {})
-      const current = composeOptions(scopeName, scope.vars.schemaInfo.descriptor).options
+    // Compile the resource with the additions, so allocation stores enriched definitions.
+    const prepareAdditions = async (scopeName, { fields, searchSchema = {}, canonicalFieldsMap }) => {
+      const current = composeOptions(scopeName, api.resources[scopeName].vars.schemaInfo.descriptor).options
       const candidate = snapshotResourceConfiguration({
         ...current,
         schema: { ...current.schema, ...fields },
         searchSchema: { ...current.searchSchema, ...searchSchema }
       })
       const schemaInfo = await compileOptions(scopeName, candidate)
-      await api.transaction(transaction => registry.allocateFields({
+      const allocation = {
         tenant: tenantId,
         resource: scopeName,
         fields: Object.fromEntries(Object.keys(fields).map(fieldName => [fieldName, {
           definition: schemaInfo.schemaStructure[fieldName] || schemaInfo.computed[fieldName],
           declaration: fields[fieldName],
-          canonicalField: params.canonicalFieldsMap?.[fieldName]
+          canonicalField: canonicalFieldsMap?.[fieldName]
         }])),
         searchSchema
-      }, { transaction }))
+      }
+      return { candidate, schemaInfo, allocation }
+    }
+
+    const publishChange = async (scopeName, { fields = {}, searchSchema = {}, compiled } = {}) => {
       registry.invalidateDescriptor(tenantId, scopeName)
-      const local = runtimeDeclarations.get(scopeName) || {}
-      runtimeDeclarations.set(scopeName, {
-        schema: { ...local.schema, ...fields },
-        searchSchema: { ...local.searchSchema, ...searchSchema }
-      })
-      // The fresh descriptor also carries fields other servers added meanwhile.
+      if (Object.keys(fields).length || Object.keys(searchSchema).length) {
+        const local = runtimeDeclarations.get(scopeName) || {}
+        runtimeDeclarations.set(scopeName, {
+          schema: { ...local.schema, ...fields },
+          searchSchema: { ...local.searchSchema, ...searchSchema }
+        })
+      }
+      // The fresh descriptor also carries changes other servers made meanwhile.
       const descriptor = await registry.getDescriptor(tenantId, scopeName)
-      await publish(scopeName, descriptor, { compiled: { signature: optionsSignature(candidate), schemaInfo } })
+      await publish(scopeName, descriptor, { compiled })
       await notifySchemaChanged(scopeName, descriptor.schemaVersion)
+    }
+
+    addResourceMethod('addKnexFields', async ({ scopeName, params }) => {
+      if (!params?.fields || typeof params.fields !== 'object' || Array.isArray(params.fields)) {
+        throw new Error('fields parameter is required for addKnexFields')
+      }
+      assertFieldNameMap(params.fields, `added fields in '${scopeName}'`)
+      assertFieldNameMap(params.searchSchema, `added search schema in '${scopeName}'`)
+      assertFieldNameMap(params.canonicalFieldsMap, `added canonical fields in '${scopeName}'`)
+      const unknown = Object.keys(params.canonicalFieldsMap || {}).filter(field => !Object.hasOwn(params.fields, field))
+      if (unknown.length) {
+        throw new Error(`canonicalFieldsMap contains unknown fields for addKnexFields: ${unknown}`)
+      }
+      const fields = snapshotResourceConfiguration(params.fields)
+      const searchSchema = snapshotResourceConfiguration(params.searchSchema || {})
+      const { candidate, schemaInfo, allocation } = await prepareAdditions(scopeName, { fields, searchSchema, canonicalFieldsMap: params.canonicalFieldsMap })
+      await api.transaction(transaction => registry.allocateFields(allocation, { transaction }))
+      await publishChange(scopeName, { fields, searchSchema, compiled: { signature: optionsSignature(candidate), schemaInfo } })
     })
 
+    const retirable = (scopeName, fieldName) => {
+      const definition = api.resources[scopeName].vars.schemaInfo.schemaStructure[fieldName]
+      if (!definition || fieldName === 'id' || definition.virtual === true || definition.belongsTo) {
+        throw new Error(`'${scopeName}.${fieldName}' is not a stored attribute`)
+      }
+      return definition
+    }
+
+    addResourceMethod('retireKnexFields', async ({ scopeName, params }) => {
+      const fields = Array.isArray(params) ? params : params?.fields
+      if (!Array.isArray(fields) || !fields.length) throw new Error('retireKnexFields requires an array of field names')
+      fields.forEach(fieldName => retirable(scopeName, fieldName))
+      await registry.retireFields({ tenant: tenantId, resource: scopeName, fields })
+      await publishChange(scopeName)
+    })
+
+    // Change a field's type: copy its values into a new field, then retire it.
+    addResourceMethod('migrateField', async ({ scopeName, params }) => {
+      const { from, to, transform } = params || {}
+      const { name, ...declaration } = to || {}
+      if (typeof from !== 'string' || typeof name !== 'string' || !name) {
+        throw new Error('migrateField requires from and to: { name, ...definition }')
+      }
+      if (transform !== undefined && typeof transform !== 'function') throw new Error('migrateField transform must be a function')
+      const fromDefinition = retirable(scopeName, from)
+      if (fromDefinition.retired) throw new Error(`Field '${scopeName}.${from}' is already retired`)
+      if (declaration.computed || declaration.virtual || declaration.belongsTo || declaration.getter || declaration.setter || declaration.storage) {
+        throw new Error(`migrateField target '${name}' must be a plain stored attribute; add other fields with addKnexFields`)
+      }
+      assertFieldNameMap({ [name]: declaration }, `migrated field in '${scopeName}'`)
+      const fields = snapshotResourceConfiguration({ [name]: declaration })
+      const { schemaInfo, allocation } = await prepareAdditions(scopeName, { fields })
+
+      await api.transaction(async (transaction) => {
+        const descriptor = await registry.allocateFields(allocation, { transaction })
+        await copyFieldValues({ transaction, descriptor, schemaInfo, from, fromDefinition, to: name, transform })
+        await registry.retireFields({ tenant: tenantId, resource: scopeName, fields: [from] }, { transaction })
+      })
+      await publishChange(scopeName, { fields })
+    })
+
+    const copyFieldValues = async ({ transaction, descriptor, schemaInfo, from, fromDefinition, to, transform }) => {
+      const { canonical } = descriptor
+      const source = descriptor.fields[from].slot
+      const target = descriptor.fields[to].slot
+      const idColumn = canonical.logicalIdColumn
+      for (let lastId = null; ;) {
+        const query = transaction(canonical.tableName)
+          .where(canonical.tenantColumn, tenantId)
+          .where(canonical.resourceColumn, descriptor.resource)
+          .orderBy('id')
+          .limit(MIGRATION_BATCH_SIZE)
+          .select('id', idColumn, source)
+        if (lastId !== null) query.where('id', '>', lastId)
+        const rows = await applyDatabaseReadOptions(query)
+        if (!rows.length) return
+        for (const row of rows) {
+          lastId = row.id
+          const value = normalizeStoredFieldValue(row[source], fromDefinition, { fieldName: from, resourceType: descriptor.resource })
+          const next = transform ? await transform(value, { id: String(row[idColumn]) }) : value
+          const { validatedObject, errors } = await schemaInfo.schemaInstance.patch({ [to]: next ?? null })
+          if (Object.keys(errors).length) {
+            const message = `Record '${row[idColumn]}' cannot migrate '${from}' to '${to}': ${Object.values(errors)[0].message}`
+            throw new RestApiValidationError(message, { fields: [to], violations: [{ field: to, rule: 'field_migration', message }] })
+          }
+          const stored = translateCanonicalAttributesForStorage({ [to]: validatedObject[to] ?? null }, descriptor, {
+            schemaInfo, databaseClient: knex.client.config.client
+          })
+          await transaction(canonical.tableName).where('id', row.id).update({ [target]: stored[target] ?? null })
+        }
+      }
+    }
+
     addResourceMethod('alterKnexFields', async () => {
       throw new Error('alterKnexFields is not supported by AnyAPI Knex plugin yet')
     })

package/plugins/core/rest-api-plugin-methods/aggregate.js

@@ -0,0 +1,41 @@
+import { getRequestContracts, validateRequestContractOrThrow } from '../lib/querying-writing/request-contracts.js'
+import { getEffectiveSortableFields } from '../lib/querying/query-field-sort-helpers.js'
+import { compileAggregateRequest } from '../lib/querying/aggregate-query.js'
+
+/**
+ * Group a filtered collection and compute measures for each group.
+ * Filters are validated like query filters; permissions are checked as a
+ * query, and row policies and autofilters see queryPurpose 'aggregate'.
+ * Returns plain rows in either format; grouped results are paged.
+ */
+export default async function aggregateMethod ({ params, context, vars, helpers, scope, runHooks, scopeName, api }) {
+  context.method = 'aggregate'
+  context.scopeName = scopeName
+  context.schemaInfo = scope.vars.schemaInfo
+
+  const plan = compileAggregateRequest({
+    params,
+    schemaInfo: context.schemaInfo,
+    scopeName,
+    defaultLimit: vars.queryDefaultLimit,
+    maxLimit: vars.queryMaxLimit
+  })
+  const requestContracts = getRequestContracts({
+    scopeName,
+    schemaInfo: context.schemaInfo,
+    includeDepthLimit: vars.includeDepthLimit,
+    sortableFields: getEffectiveSortableFields(vars)
+  })
+  const validated = validateRequestContractOrThrow(
+    requestContracts.query,
+    { queryParams: { filters: params.queryParams?.filters ?? {} } },
+    'Aggregate filters are invalid'
+  )
+  context.queryParams = { filters: validated.queryParams?.filters || {} }
+
+  context.transaction = params.transaction
+  context.db = context.transaction || api.knex.instance
+
+  await scope.checkPermissions({ method: 'query', originalContext: context })
+  return helpers.dataAggregate({ scopeName, context, runHooks, plan })
+}

package/plugins/core/rest-api-plugin-methods/common.js

@@ -213,7 +213,7 @@ export const validateCompleteReplacePayload = ({
     if (!fieldDef) continue
     if (fieldName === idProperty) continue
     if (Object.hasOwn(belongsToUpdates, fieldName)) continue
-    if (fieldDef.computed === true || fieldDef.virtual === true) continue
+    if (fieldDef.computed === true || fieldDef.virtual === true || fieldDef.retired === true) continue
     if (fieldDef.type === undefined) continue
 
     if (fieldDef.belongsTo && fieldDef.as) {
@@ -296,8 +296,16 @@ export const validateResourceAttributesBeforeWrite = async ({
   /** @type {StorageRow} */
   const attributesForValidation = { ...attributesToValidate }
   const foreignKeyViolations = []
+  const retiredViolations = []
   for (const [field, value] of Object.entries(attributesToValidate)) {
     const definition = schemaStructure[field]
+    if (definition?.retired === true && value !== undefined) {
+      retiredViolations.push({
+        field: `data.attributes.${field}`,
+        rule: 'retired_field',
+        message: `Field '${field}' is retired and accepts no new values`
+      })
+    }
     if (!definition?.belongsTo || !definition.as || Object.hasOwn(belongsToUpdates, field)) continue
     delete attributesForValidation[field]
     if (value === undefined) continue
@@ -307,6 +315,13 @@ export const validateResourceAttributesBeforeWrite = async ({
       message: `Foreign key field '${field}' should not be in attributes. Use 'data.relationships.${definition.as}' instead.`
     })
   }
+  const [firstRetired] = retiredViolations
+  if (firstRetired) {
+    throw new RestApiValidationError(firstRetired.message, {
+      fields: retiredViolations.map(violation => violation.field),
+      violations: retiredViolations
+    })
+  }
   if (foreignKeyViolations.length) {
     throw new RestApiValidationError('Foreign key fields cannot be set directly in attributes', {
       fields: foreignKeyViolations.map(violation => violation.field),

package/plugins/core/rest-api-plugin.js

@@ -10,6 +10,7 @@ import registerRelationshipRoutes from './rest-api-plugin-hooks/register-relatio
 
 // Import method functions
 import queryMethod from './rest-api-plugin-methods/query.js'
+import aggregateMethod from './rest-api-plugin-methods/aggregate.js'
 import getMethod from './rest-api-plugin-methods/get.js'
 import postMethod from './rest-api-plugin-methods/post.js'
 import putMethod from './rest-api-plugin-methods/put.js'
@@ -90,6 +91,7 @@ export const RestApiPlugin = {
 
     // Main REST methods
     addResourceMethod('query', withAvailableContext(queryMethod))
+    addResourceMethod('aggregate', withAvailableContext(aggregateMethod))
     addResourceMethod('get', withAvailableContext(getMethod))
     addResourceMethod('post', withWriteOutcome(postMethod))
     addResourceMethod('put', withWriteOutcome(putMethod))

package/types/anyapi.d.ts

@@ -23,6 +23,8 @@ export class AnyapiRegistry {
   readonly cacheSize: number
   ensureSchema(options?: { slots?: AnyapiSlotCounts }): Promise<void>
   getSchemaVersion(tenant: string, resource: string, options?: { transaction?: Knex.Transaction }): Promise<number | null>
+  // Retired fields keep their slots and values and reject new writes.
+  retireFields(params: { tenant: string; resource: string; fields: string[] }, options?: { transaction?: Knex.Transaction }): Promise<Record<string, unknown>>
   invalidateDescriptor(tenant: string, resource: string): void
 }
 

package/types/resource-methods.d.ts

@@ -23,6 +23,24 @@ export interface QueryParams extends SelectionParams {
   sort?: string[]
   page?: PageParams
 }
+export type AggregateFunction = 'count' | 'countDistinct' | 'sum' | 'avg' | 'min' | 'max'
+// One function per measure; only count accepts '*'.
+export type AggregateMeasure = { count: string } | { [Function in Exclude<AggregateFunction, 'count'>]: { [Name in Function]: string } }[Exclude<AggregateFunction, 'count'>]
+export interface AggregateParams extends TransactionParam {
+  groupBy?: string[]
+  measures: Record<string, AggregateMeasure>
+  queryParams?: {
+    filters?: Record<string, unknown>
+    // Grouped field or measure names, '-' for descending.
+    sort?: string[]
+    // Grouped results only.
+    page?: { size?: number; number?: number }
+  }
+}
+export interface AggregateResult {
+  data: Array<Record<string, unknown>>
+  meta?: { page: { size: number; number: number; hasMore: boolean } }
+}
 export interface TransactionParam {
   transaction?: Knex.Transaction
 }
@@ -78,6 +96,7 @@ export interface ResourceCoreMethods<
   query<Format extends ResourceFormat = DefaultFormat>(
     params?: RemovedResourceOptions & TransactionParam & { format?: Format; queryParams?: QueryParams }, context?: Context
   ): Promise<CollectionResult<Format, Fields, Type>>
+  aggregate(params: AggregateParams, context?: Context): Promise<AggregateResult>
   post<Format extends ResourceFormat = DefaultFormat, Returning extends WriteReturning = DefaultReturning>(
     params: WriteOptions<Format, Returning> & ResourceWriteInput<Input, Type> & {
       expectedVersion?: never

package/types/resource-schema.d.ts

@@ -1,7 +1,7 @@
 import type { StorageFieldDefinition, StorageQuery } from '../plugins/core/lib/storage/storage-types.js'
 import type { FileStorage } from './file-storage.js'
 
-export type ResourceFieldType = 'string' | 'number' | 'integer' | 'boolean' | 'id' |
+export type ResourceFieldType = 'string' | 'email' | 'uuid' | 'number' | 'integer' | 'boolean' | 'id' |
   'date' | 'time' | 'dateTime' | 'epochSeconds' | 'epochMilliseconds' |
   'array' | 'object' | 'file' | 'blob' | 'serialize' | 'none'
 
@@ -46,6 +46,8 @@ export interface ResourceFieldOptions {
   normallyHidden?: boolean
   virtual?: boolean
   computed?: boolean
+  // Readable and filterable, but rejects new values.
+  retired?: boolean
   include?: ResourceRelationshipOptions['include']
   search?: boolean | ResourceSearchOptions | Record<string, ResourceSearchOptions>
   dependencies?: readonly string[]
@@ -113,7 +115,7 @@ type ObjectValue<Field, Mode extends Direction> =
     : Field extends { values: infer Values } ? Record<string, NestedValue<Values, Mode>> : Record<string, unknown>
 
 type BaseValue<Field, Mode extends Direction> =
-  Field extends { type: 'string' | 'file' | 'date' | 'time' | 'dateTime' } ? string
+  Field extends { type: 'string' | 'email' | 'uuid' | 'file' | 'date' | 'time' | 'dateTime' } ? string
   : Field extends { type: 'number'; precision: number; scale: number } ? Mode extends 'output' ? string | number : number
   : Field extends { type: 'number' | 'integer' | 'epochSeconds' | 'epochMilliseconds' } ? number
   : Field extends { type: 'boolean' } ? boolean
@@ -147,7 +149,7 @@ type StoredNull<Field> = Field extends { required: true } | { nullable: false }
 // Attribute helpers exclude the conventional ID and relationship backing fields.
 // Hooks/defaults may supply required values, so caller input stays optional.
 export type InferInput<Schema extends object> = {
-  -readonly [Key in keyof Schema as Schema[Key] extends { computed: true } ? never : AttributeKey<Schema, Key>]?: FieldValue<Schema[Key], 'input'>
+  -readonly [Key in keyof Schema as Schema[Key] extends { computed: true } | { retired: true } ? never : AttributeKey<Schema, Key>]?: FieldValue<Schema[Key], 'input'>
 }
 
 // Resource result types add optionality for sparse fields and permissions.