json-rest-api 1.0.29 → 2.1.0

package/docs/architecture.md

@@ -0,0 +1,75 @@
+---
+title: "Architecture"
+---
+
+# Architecture
+
+The library exposes resource methods directly and uses connectors to translate
+HTTP requests into those same calls. Storage and transaction rules are shared
+between direct calls and transports.
+
+## Runtime
+
+`lib/runtime/json-rest-api.js` contains the API object and its sequential hook list.
+It registers ordinary resource methods, installs plugins and constructs the argument
+object passed to existing operation handlers. There are no proxy-based resource
+objects, generic scope aliases or automatic dispatcher logs.
+
+A resource inherits shared methods; its vars and helpers inherit API defaults.
+Local properties override those defaults. The internal operation argument names
+`scope`, `scopes` and `scopeName` refer to resources. The public collection is
+`api.resources`.
+
+## Resource operations and schemas
+
+`plugins/core/rest-api-plugin.js` installs CRUD, relationship and transaction methods.
+The operation handlers live in `plugins/core/rest-api-plugin-methods/`. They validate
+requests, run permission/lifecycle hooks and call storage helpers.
+
+Writes select plain input with `data` or JSON:API input with `document`.
+`request-contracts.js` validates that selection; plain conversion produces the
+normalized `context.inputRecord` used by the existing write lifecycle. `format`
+controls output only. Both input forms run through the same resource operation,
+including response preparation before transaction completion. Connectors submit
+documents through this same boundary.
+
+Hooks deliberately share mutable operation context for caches, added information
+and phase-specific field changes. The caller observes those mutations. See the
+[hook contract](GUIDE/13-hooks-and-lifecycle.md) for wrapper hooks, nested calls,
+writable values and library-owned bookkeeping. Independent calls use separate
+contexts; this is not an immutable-context API.
+
+`plugins/core/lib/querying-writing/compile-schemas.js` compiles resource declarations.
+Its neighboring modules own response options, field selection, ID normalization and
+representation conversion. The JSON schema package remains a separate dependency;
+it is unrelated to the removed hooked-api runtime.
+
+## Storage
+
+`rest-api-knex-plugin.js` maps resources to ordinary tables.
+`rest-api-anyapi-knex-plugin.js` maps them to canonical records and relationship links.
+Both use native Knex queries and the common storage interfaces under
+`plugins/core/lib/storage/`.
+
+Canonical links and relationship reads have separate modules under `lib/anyapi/`
+within the core plugin. Include traversal and to-one/to-many loaders live under
+`plugins/core/lib/querying/`. Keep query ownership and visibility checks in those
+modules rather than duplicating them in connectors.
+
+## Transactions and errors
+
+`lib/error-context.js` owns managed transactions, participating operations,
+completion hooks and write outcomes. `lib/knex-transaction.js` handles connection
+completion. Resource methods delegate to these modules.
+
+`lib/rest-api-errors.js` defines the local error hierarchy. The runtime preserves
+thrown values; operation-level code attaches transaction outcomes and secondary
+cleanup diagnostics. See the [transaction contract](GUIDE/19-managed-transactions.md)
+and [error outcomes](GUIDE/20-transaction-outcomes.md).
+
+## Transports and files
+
+Express and Fastify connectors share parsing, route and response helpers under
+`plugins/core/connectors/lib/`. Socket.IO remains an optional integration.
+File handling owns upload lifetime; storage plugins under `plugins/storage/` own
+file bytes. Their effects participate in the documented completion-hook lifecycle.

package/docs/contributing.md

@@ -0,0 +1,178 @@
+---
+title: "Contributing"
+---
+
+# Contributing
+
+Use **Node 24** for development and verification. The library requires Node 24 or
+newer; `.nvmrc` pins the version used by this repository and CI. Do not add a
+second Node-version matrix without a concrete compatibility requirement.
+
+## Setup
+
+```sh
+nvm install
+nvm use
+npm ci
+```
+
+SQLite fixtures need no external database or credentials. If the native SQLite
+driver compiles from source, install Python 3, make and a C/C++ compiler.
+Documentation uses Ruby 3.2 and Bundler 2.6.9. Install Bundler and run `npm run docs`;
+the script installs missing gems into `docs/vendor/bundle` before building.
+`npm run docs:dev -- --no-open` serves the site without opening a browser.
+
+## Repository layout
+
+| Path | Purpose |
+| --- | --- |
+| `index.js`, `index.d.ts` | Public exports |
+| `lib/` | Runtime, errors, diagnostics and transaction ownership |
+| `plugins/` | Resource operations, storage and optional integrations |
+| `types/` | Public TypeScript contracts |
+| `tests/fixtures/`, `tests/helpers/` | Resource factories and test infrastructure |
+| `tests/*.test.js` | Library regression suites |
+| `examples/` | Maintained runnable examples and data migration utilities |
+| `scripts/` | Documentation, package and verification tools |
+| `docs/GUIDE/` | User guides and the application migration guide |
+| `old/` | Inactive history; excluded from packaging and automated discovery |
+
+[Architecture](architecture.md) explains the main execution boundaries.
+`tests/README.md` in a source checkout contains fixture conventions and the command matrix.
+
+## Verification
+
+Run focused suites for changed behavior. Reserve the comprehensive gate for
+changes that need it and release preparation:
+
+```sh
+npm run verify
+```
+
+This runs checked JavaScript/types, packed public declarations and imports,
+query budgets, both complete SQLite storage suites, Express 4 checks, lint and
+the documentation build. A storage-specific test may be skipped in one mode
+and executed in the other; inspect both results before calling it untested.
+
+`npm run test:clean-package` installs the actual tarball in a fresh temporary
+consumer and checks optional dependencies plus ordinary/canonical CRUD.
+The `test:*guide`, `test:quickstart`, `test:api-reference` and `test:server-examples`
+commands execute maintained documentation examples. Run those when their code changes.
+`npm run test:migration-guide:package` executes the migration guide's selected
+examples against an extracted tarball and reports its hash. It reuses local test
+dependencies; the separate clean-package check verifies fresh installation.
+
+Tests and package tools use temporary directories and dispose of their databases,
+servers and sockets. They do not operate on application databases. Consumer checks
+require an explicit path and command; library verification does not establish an
+application migration.
+
+## Resource stress checks
+
+For smaller repeatable measurements, `node scripts/measure-write-lifecycles.js`
+records SQL counts and checks child-hook order for bulk and reverse-relationship
+writes. It supports both storage modes and the database runner.
+`node scripts/measure-diagnostics.js` reports bounded logger formatting cost;
+its local timings are observations, not pass/fail performance thresholds.
+
+`npm run test:stress` runs a bounded workload in both SQLite storage modes,
+separately from the full suite. It repeats authorized, paginated `tasks` and
+`shared_tasks` relationship reads with polymorphic `subject` includes. The
+fixture contains equal visible, policy-hidden and foreign-workspace groups;
+every response must contain exactly the expected visible records and includes.
+
+Four sequential write scenarios check owned and managed commit/rollback,
+persisted values and exact completion-hook counts. SQL is attributed to each
+operation with `AsyncLocalStorage`; fixed per-operation query ceilings and zero
+runtime metadata lookups are enforced. SQL attribution failures fail the run.
+
+To save one JSON report directly:
+
+```sh
+node scripts/stress-resources.js > /tmp/json-rest-api-stress-knex.json
+JSON_REST_API_STORAGE=anyapi node scripts/stress-resources.js > /tmp/json-rest-api-stress-anyapi.json
+```
+
+Store reports outside the tracked source tree. The report includes configuration,
+environment, per-scenario query counts, latency p50/p95/p99, throughput and memory
+samples. Timing includes correctness checks; writes include a persisted-state
+read. There are no machine-dependent latency or memory pass thresholds.
+
+| Environment variable | Default | Allowed values |
+| --- | ---: | --- |
+| `JSON_REST_API_STRESS_ROWS` | 100 | 1–1000 per visibility group |
+| `JSON_REST_API_STRESS_READS` | 200 | 2–100000 per batch |
+| `JSON_REST_API_STRESS_WRITES` | 32 | 4–20000 per batch, alternating four scenarios |
+| `JSON_REST_API_STRESS_CONCURRENCY` | 4 | 1–32 overlapping read operations |
+| `JSON_REST_API_STRESS_BATCHES` | 1 | 1–100 repetitions against the same fixture |
+| `JSON_REST_API_STRESS_PAGE_SIZE` | 20 | 1–100 |
+
+Totals across batches are capped at 100000 reads and 20000 writes. Defaults seed
+300 linked tasks plus a project and a task used as a polymorphic subject.
+Each independent operation receives fresh context. SQLite uses one in-memory
+connection, so overlapping reads queue their SQL; this does not test concurrent
+SQLite writers. Writes are sequential on every backend.
+
+For a longer bounded run with explicit retained-heap sampling:
+
+```sh
+JSON_REST_API_STRESS_BATCHES=10 node --expose-gc scripts/stress-resources.js > /tmp/json-rest-api-stress-soak.json
+```
+
+Without `--expose-gc`, memory samples describe ordinary heap/RSS fluctuations.
+With it, samples additionally record heap retained after requested garbage
+collection. Both are observations; a bounded run cannot establish leak freedom.
+The driver-aware fixture also supports native checks through the existing runner:
+
+```sh
+node scripts/test-databases.js pg scripts/stress-resources.js
+node scripts/test-databases.js mysql2 scripts/stress-resources.js
+```
+
+## Real databases and Redis
+
+The database runner starts disposable servers and creates isolated test databases.
+Supply PostgreSQL's `initdb`/`postgres` and MySQL's `mysqld`, then run:
+
+```sh
+npm run test:databases:pg
+npm run test:databases:mysql
+npm run test:redis
+```
+
+`JSON_REST_API_POSTGRES_BIN` selects the PostgreSQL executable directory; otherwise
+the runner uses `pg_config --bindir`. `JSON_REST_API_MYSQL_BIN` selects the MySQL
+executable; otherwise it uses `mysqld` on PATH. MariaDB does not replace this MySQL
+check. `JSON_REST_API_REDIS_BIN` selects Redis; otherwise the runner uses PATH.
+
+For an affected subset, pass test paths directly:
+
+```sh
+node scripts/test-databases.js pg tests/conformance-managed-transactions.test.js
+```
+
+The runner checks both storage modes by default. Select one explicitly when
+rechecking a backend-specific failure:
+
+```sh
+JSON_REST_API_RUNNER_STORAGE=anyapi npm run test:databases:pg
+```
+
+A complete backend/storage selection has a 30-minute limit; explicitly selected
+test files retain a 15-minute limit. Both limits terminate the owned process group
+and dispose of the database. They do not retry tests or relax assertions.
+
+See [backend capabilities](GUIDE/30-backend-capabilities.md) for supported semantics.
+The CI workflow documents its disposable binary setup and database matrix.
+
+## Releases
+
+1. Review the public API and update the [migration guide](GUIDE/33-migrating-to-v2.md).
+2. Use a new major version for breaking changes. Keep the manifest and lock aligned.
+3. Run the agreed verification scope and inspect failures, cancellations and skips.
+4. Inspect the tarball contents and confirm that `old/`, tests, local configuration
+   and development dependencies are absent from the installed runtime package.
+5. Commit the release changes. Publishing and tagging are explicit maintainer actions.
+
+Keep secrets in ignored local configuration. Record release evidence in the release
+or pull request; do not add another execution ledger or TODO document to the active tree.

package/index.d.ts

@@ -0,0 +1,37 @@
+export * from './types/errors.js'
+export * from './types/file-storage.js'
+export type * from './types/representations.js'
+export type * from './types/resource-methods.js'
+export type * from './types/relationship-methods.js'
+export type * from './types/bulk-methods.js'
+export type { ManagedTransaction, ManagedTransactionParam, TransactionMethods } from './types/transactions.js'
+export type * from './types/plugin-options.js'
+export * from './types/anyapi.js'
+export type * from './types/file-detectors.js'
+export type * from './types/fastify-options.js'
+export type * from './types/row-policy.js'
+export type * from './types/autofilter.js'
+
+export { JsonRestApi } from './types/runtime.js'
+export type * from './types/runtime.js'
+export type * from './types/resource-schema.js'
+export type * from './types/hook-context.js'
+import type { LibraryPlugin } from './types/runtime.js'
+
+export const RestApiPlugin: LibraryPlugin<'rest-api'>
+export const AutoFilterPlugin: LibraryPlugin<'autofilter'>
+export const RowPolicyPlugin: LibraryPlugin<'row-policy'>
+export const QueryProjectionsPlugin: LibraryPlugin<'query-projections'>
+export const FileHandlingPlugin: LibraryPlugin<'file-handling'>
+export const CorsPlugin: LibraryPlugin<'rest-api-cors'>
+export const LabelPlugin: LibraryPlugin<'rest-api-label'>
+export const SocketIOPlugin: LibraryPlugin<'socketio'>
+export const RestApiKnexPlugin: LibraryPlugin<'rest-api-knex'>
+export const RestApiAnyapiKnexPlugin: LibraryPlugin<'rest-api-anyapi-knex'>
+export const ExpressPlugin: LibraryPlugin<'express'>
+export const FastifyPlugin: LibraryPlugin<'fastify'>
+
+export function getUrlPrefix(
+  context?: { urlPrefixOverride?: string; urlPrefix?: string } | null,
+  scope?: { vars?: { transport?: { publicBaseUrl?: string; mountPath?: string } } } | null
+): string

package/index.js

@@ -1,12 +1,4 @@
-/**
- * JSON REST API - REST API plugin for hooked-api with JSON:API compliance
- *
- * This package provides:
- * - REST API plugin that adds query, get, post, put, patch, delete methods to scopes
- * - Connector plugins for various HTTP frameworks (Express, etc.)
- * - JSON:API compliant request/response handling
- * - Comprehensive error handling and validation
- */
+export { JsonRestApi } from './lib/runtime/json-rest-api.js'
 
 // Core plugins
 export { RestApiPlugin } from './plugins/core/rest-api-plugin.js'
@@ -21,27 +13,16 @@ export { SocketIOPlugin } from './plugins/core/socketio-plugin.js'
 // Database plugins
 export { RestApiKnexPlugin } from './plugins/core/rest-api-knex-plugin.js'
 export { RestApiAnyapiKnexPlugin } from './plugins/core/rest-api-anyapi-knex-plugin.js'
+export { AnyapiRegistry } from './plugins/core/lib/anyapi/anyapi-registry.js'
+export { ensureAnyApiSchema } from './plugins/core/lib/anyapi/schema-utils.js'
 
 // Connector plugins
 export { ExpressPlugin } from './plugins/core/connectors/express-plugin.js'
 export { FastifyPlugin } from './plugins/core/connectors/fastify-plugin.js'
-// Future: export { KoaPlugin } from './plugins/core/connectors/koa-plugin.js';
 
 // Storage plugins for file handling
 export { LocalStorage } from './plugins/storage/local-storage.js'
-export { S3Storage } from './plugins/storage/s3-storage.js'
 
-// Error classes for consumers who need them
-export {
-  RestApiFieldsetError,
-  RestApiTemporalDataError,
-  RestApiValidationError,
-  RestApiResourceError,
-  RestApiPayloadError
-} from './lib/rest-api-errors.js'
-
-// Re-export everything for backward compatibility
 export * from './lib/rest-api-errors.js'
 
-// URL helper function for generating API URL prefix
 export { getUrlPrefix } from './plugins/core/lib/querying/url-helpers.js'

package/lib/enhanced-logger.js

@@ -1,124 +1,130 @@
-/**
- * Enhanced logger wrapper that automatically serializes errors with full details
- * Wraps any logger to provide comprehensive error logging
- */
+// @ts-check
+/** @import { DiagnosticLogger, DiagnosticOptions, EnhancedDiagnosticLogger, DiagnosticWriter } from './logger-types.js' */
 
-import { formatError, formatErrorSummary } from './error-formatter.js'
+import { isErrorObject } from './error-context.js'
+import { formatDiagnosticValue, formatError, formatErrorSummary, readDiagnosticProperty } from './error-formatter.js'
 
 /**
- * Creates an enhanced logger that automatically formats errors
- * @param {Object} baseLogger - The base logger to wrap (must have debug, info, warn, error methods)
- * @param {Object} options - Configuration options
- * @param {boolean} options.logFullErrors - Whether to log full error details (default: true)
- * @param {boolean} options.includeStack - Whether to include stack traces (default: true)
- * @returns {Object} Enhanced logger with automatic error formatting
+ * @template {DiagnosticLogger} Logger
+ * @overload
+ * @param {Logger & { error: DiagnosticWriter }} baseLogger
+ * @param {DiagnosticOptions} [options]
+ * @returns {EnhancedDiagnosticLogger & { error: DiagnosticWriter }}
+ */
+/**
+ * @template {DiagnosticLogger} Logger
+ * @overload
+ * @param {Logger} baseLogger
+ * @param {DiagnosticOptions} [options]
+ * @returns {EnhancedDiagnosticLogger}
+ */
+/**
+ * Bound diagnostic previews and apply compiled field visibility to supplied levels.
+ * @param {DiagnosticLogger} baseLogger
+ * @param {DiagnosticOptions} [options]
+ * @returns {EnhancedDiagnosticLogger}
  */
 export function createEnhancedLogger (baseLogger, options = {}) {
   const {
     logFullErrors = true,
-    includeStack = true
+    includeStack = true,
+    redactFields = [],
+    schemaInfo
   } = options
-
-  // Helper to process arguments and format errors
-  function processArgs (args) {
-    return args.map(arg => {
-      // If it's an error object, format it
-      if (arg instanceof Error) {
-        if (logFullErrors) {
-          return formatError(arg, { includeStack })
-        } else {
-          return formatErrorSummary(arg)
-        }
-      }
-
-      // If it's an object with an error property, format the error
-      if (arg && typeof arg === 'object' && arg.error instanceof Error) {
-        return {
-          ...arg,
-          error: logFullErrors
-            ? formatError(arg.error, { includeStack })
-            : formatErrorSummary(arg.error)
-        }
+  const hiddenFields = Object.entries(schemaInfo?.outputFields || {})
+    .filter(([, definition]) => definition.hidden === true || definition.normallyHidden === true)
+    .map(([field]) => field)
+  const diagnosticOptions = { includeStack, redactFields: [...redactFields, ...hiddenFields] }
+  const formatLoggedError = logFullErrors ? formatError : formatErrorSummary
+
+  /** @param {unknown} arg */
+  function formatArgument (arg) {
+    try {
+      if (isErrorObject(arg)) return formatLoggedError(arg, diagnosticOptions)
+      if (!arg || typeof arg !== 'object' || diagnosticOptions.redactFields.includes('error')) return arg
+
+      /** @type {unknown} */
+      const nestedError = Object.getOwnPropertyDescriptor(arg, 'error')?.value
+      if (!isErrorObject(nestedError)) return arg
+      /** @type {object} */
+      const metadata = Object(formatDiagnosticValue(arg, diagnosticOptions))
+      if (!Object.hasOwn(metadata, 'error')) return metadata
+      return {
+        ...metadata,
+        error: formatLoggedError(nestedError, diagnosticOptions)
       }
-
+    } catch {
+      // The final shared formatter contains values that cannot even be classified.
       return arg
-    })
+    }
   }
 
   // Create wrapper methods for each log level
+  /** @type {DiagnosticLogger & Record<string, unknown>} */
   const enhancedLogger = {}
 
-  // Common log levels
-  const logLevels = ['debug', 'info', 'warn', 'error', 'trace', 'fatal', 'log']
+  const logLevels = /** @type {const} */ (['debug', 'info', 'warn', 'error', 'trace', 'fatal', 'log'])
 
   for (const level of logLevels) {
-    if (typeof baseLogger[level] === 'function') {
-      enhancedLogger[level] = function (...args) {
-        const processedArgs = processArgs(args)
-
-        // Special handling for error level - always log full details
-        if (level === 'error' && args.some(arg => arg instanceof Error || (arg?.error instanceof Error))) {
-          // Log a formatted version as well for better readability
-          const errors = args.filter(arg => arg instanceof Error || arg?.error instanceof Error)
-          for (const errorArg of errors) {
-            const error = errorArg instanceof Error ? errorArg : errorArg.error
-            if (error.details?.violations && Array.isArray(error.details.violations) && error.details.violations.length > 0) {
-              // Log violations in a readable format
-              baseLogger[level]('Validation error details:', {
-                message: error.message,
-                violations: error.details.violations.map(v => ({
-                  field: v.field,
-                  rule: v.rule,
-                  message: v.message
-                }))
-              })
-            }
-          }
-        }
-
-        return baseLogger[level](...processedArgs)
-      }
+    if (typeof baseLogger[level] !== 'function') continue
+    // Capture before enhanceLogger replaces methods on the same object.
+    const write = baseLogger[level].bind(baseLogger)
+    enhancedLogger[level] = function (...args) {
+      const processedArgs = args.slice(0, 50).map(formatArgument)
+      if (args.length > 50) processedArgs.push('[Truncated]')
+      const formatted = formatDiagnosticValue(processedArgs, diagnosticOptions)
+      return write(...(Array.isArray(formatted) ? formatted : [formatted]))
     }
   }
 
   // Preserve any other properties/methods from the base logger
   for (const key in baseLogger) {
     if (!(key in enhancedLogger)) {
-      enhancedLogger[key] = baseLogger[key]
+      enhancedLogger[key] = Reflect.get(baseLogger, key)
     }
   }
 
-  // Add utility methods
-  enhancedLogger.logError = function (message, error, additionalData = {}) {
-    const errorData = formatError(error, { includeStack })
-    this.error(message, {
-      ...additionalData,
+  /** @type {EnhancedDiagnosticLogger['logError']} */
+  const logError = function (message, error, additionalData = {}) {
+    const errorData = formatError(error, diagnosticOptions)
+    /** @type {object} */
+    const metadata = Object(formatDiagnosticValue(additionalData, diagnosticOptions))
+    return this.error(message, {
+      ...metadata,
       error: errorData,
-      errorSummary: formatErrorSummary(error)
+      errorSummary: formatErrorSummary(error, diagnosticOptions)
     })
   }
 
-  enhancedLogger.logValidationError = function (message, error, additionalData = {}) {
-    if (error.details?.violations && Array.isArray(error.details.violations)) {
-      this.error(message, {
-        ...additionalData,
-        message: error.message,
-        code: error.code,
-        violations: error.details.violations,
-        fields: error.details?.fields || [],
-        fullError: formatError(error, { includeStack: false })
-      })
-    } else {
-      this.logError(message, error, additionalData)
-    }
+  /** @type {EnhancedDiagnosticLogger['logValidationError']} */
+  const logValidationError = function (message, error, additionalData = {}) {
+    const details = diagnosticOptions.redactFields.includes('details') ? undefined : readDiagnosticProperty(error, 'details')
+    const violations = diagnosticOptions.redactFields.includes('violations') ? undefined : readDiagnosticProperty(details, 'violations')
+    if (!Array.isArray(violations)) return this.logError(message, error, additionalData)
+    const formatted = formatError(error, { ...diagnosticOptions, includeStack: false })
+    const formattedDetails = readDiagnosticProperty(formatted, 'details')
+    /** @type {object} */
+    const metadata = Object(formatDiagnosticValue(additionalData, diagnosticOptions))
+    return this.error(message, {
+      ...metadata,
+      message: readDiagnosticProperty(formatted, 'message'),
+      code: readDiagnosticProperty(formatted, 'code'),
+      violations: readDiagnosticProperty(formattedDetails, 'violations'),
+      fields: readDiagnosticProperty(formattedDetails, 'fields') || [],
+      fullError: formatted
+    })
   }
 
-  return enhancedLogger
+  return Object.assign(enhancedLogger, { logError, logValidationError })
 }
 
 /**
  * Middleware to enhance an existing logger instance
  * Can be used to patch global loggers
+ * @template {DiagnosticLogger} Logger
+ * @param {Logger} logger
+ * @param {DiagnosticOptions} [options]
+ * @returns {Logger}
  */
 export function enhanceLogger (logger, options = {}) {
   const enhanced = createEnhancedLogger(logger, options)
@@ -126,7 +132,7 @@ export function enhanceLogger (logger, options = {}) {
   // Replace methods on the original logger
   for (const key in enhanced) {
     if (typeof enhanced[key] === 'function') {
-      logger[key] = enhanced[key]
+      Object.assign(logger, { [key]: enhanced[key] })
     }
   }