@jskit-ai/agent-docs 0.1.283 → 0.1.284

package/skills/jskit/references/database-layer.md

@@ -0,0 +1,411 @@
+<!-- Generated from `packages/agent-docs/site/guide/app-setup/database-layer.md`. -->
+
+# Database layer
+
+JSKIT applications use Knex through one deliberately selected database driver.
+Install either `@jskit-ai/database-runtime-mysql` or
+`@jskit-ai/database-runtime-postgres`; do not install both unless the product
+genuinely chooses a driver at runtime.
+
+## Install the selected driver
+
+For MySQL or MariaDB:
+
+```bash
+npm install @jskit-ai/database-runtime @jskit-ai/database-runtime-mysql
+```
+
+For PostgreSQL:
+
+```bash
+npm install @jskit-ai/database-runtime @jskit-ai/database-runtime-postgres
+```
+
+The application declares `@jskit-ai/database-runtime` itself: each driver takes
+it as a peer, so the application holds its one copy, and brings its own Knex
+driver through ordinary npm dependencies. There is no JSKIT install wizard.
+
+## Connection environment
+
+Keep credentials outside Git. Supply either `DATABASE_URL` or the individual
+values:
+
+```dotenv
+DB_HOST=127.0.0.1
+DB_PORT=3306
+DB_NAME=exampleapp
+DB_USER=exampleapp
+DB_PASSWORD=secret
+```
+
+Use port `5432` for PostgreSQL. A single-driver app fixes its dialect in
+`knexfile.js`, so it does not need `DB_CLIENT`. The running database provider
+also derives its dialect from the installed driver and rejects a conflicting
+`DB_CLIENT` value when one is supplied.
+
+The application or hosting environment creates the database and provides its
+credentials. JSKIT never guesses or commits them.
+
+The running database provider registers a `database` health check that runs
+`select 1` within two seconds. The foundations' `GET /api/health` answers `503`
+while it fails; see [Health endpoint](/guide/app-setup/initial-scaffolding#health-endpoint).
+
+## Migration configuration
+
+Use the fixed-dialect pattern owned by the selected driver. The important
+application file is small:
+
+```js
+import { createKnexMigrationConfigFromApp } from
+  "@jskit-ai/database-runtime/server/knexMigrationConfig";
+
+export default await createKnexMigrationConfigFromApp({ client: "mysql2", environment: process.env });
+```
+
+Use `client: "pg"` for PostgreSQL. The environment is always passed: JSKIT reads
+the connection values from what the application gives it, never from
+`process.env` on its own. The complete examples are in:
+
+- `database/mysql-application`
+- `database/postgres-application`
+
+Each example is complete for its own engine. Copy files only from the selected
+driver's pattern and never merge the two; for example, the CI workflow must
+contain only that engine's database service.
+
+The config discovers migrations directly from the current installed package
+graph. Application migrations live in the app's `migrations/` directory;
+package-owned migrations live in directories declared by the installed
+package's `package.json#jskit.migrations.directories`.
+
+There is no migration sync command and no copied migration projection. Knex
+runs the authoritative files where their owners ship them.
+
+## What runs migrations
+
+The application owns normal npm scripts:
+
+```json
+{
+  "scripts": {
+    "db:migrate": "knex --knexfile ./knexfile.js migrate:latest",
+    "db:migrate:rollback": "knex --knexfile ./knexfile.js migrate:rollback",
+    "db:migrate:status": "knex --knexfile ./knexfile.js migrate:list"
+  }
+}
+```
+
+`npm run db:migrate` runs the Knex CLI. Knex loads `knexfile.js`, discovers the
+application and installed-package migration directories, connects using the
+environment, and applies pending migrations.
+
+Deployment or a managed development environment may invoke this app-owned
+script as a release step. JSKIT itself does not maintain a background migration
+service.
+
+## Authoring schema changes
+
+Write a new immutable migration in the package that owns the schema. Never edit
+an already-applied migration and never change a live database without recording
+the equivalent source-controlled migration.
+
+Package migrations are normal `.cjs` Knex migrations and are declared in that
+package's metadata:
+
+```json
+{
+  "jskit": {
+    "migrations": {
+      "directories": ["migrations"]
+    }
+  }
+}
+```
+
+A `constraints/` subdirectory of a declared directory, such as
+`migrations/constraints/`, holds the package's constraints migrations; it runs
+later, as described below.
+
+A package whose table migrations use another package's tables names that
+package in `after`, and its migrations, and its constraints, then run after
+that package's:
+
+```json
+{
+  "jskit": {
+    "migrations": {
+      "directories": ["migrations"],
+      "after": ["@jskit-ai/users-core"]
+    }
+  }
+}
+```
+
+Each name in `after` must be another installed JSKIT package; a name that is
+not installed, the package's own name, or packages that each name the other
+stop discovery with an error. A named package without migrations orders
+nothing.
+
+Names must remain unique across the effective migration directories. Preserve
+the basename when relocating a migration: Knex records that name in its ledger.
+Do not rename it to force execution or edit an already-applied migration's body;
+changed schema behavior requires a new migration.
+
+### Directory order comes before filenames
+
+`createKnexMigrationConfigFromApp()` sets `sortDirsSeparately: true` and runs
+pending migrations in this order, using the default `migrationsDirectory`:
+
+1. Discovered package migration directories, sorted by their absolute paths
+   using `localeCompare`, except that a package's directories follow those of
+   every package its `after` names, which follow theirs in turn, each in the
+   order listed. Without `after` the path order is unchanged.
+2. Application `migrations/`.
+3. The `constraints/` subdirectory of each discovered package migration
+   directory that has one, in the same order.
+4. Application `migrations/constraints/`, when it exists.
+
+Knex sorts filenames within each directory, not across the whole graph; see
+its [`sortDirsSeparately` option](https://knexjs.org/guide/migrations.html#migration-api).
+Package discovery does not read tables or foreign keys; only `after` changes
+the package order. A newer timestamp in an earlier package still runs before
+every file in a later package.
+Inspect the effective `knexfile.js` configuration if the app customizes it.
+
+Every package table exists before the application's own migrations run, so a
+root `migrations/` file can reference a package table directly, for example a
+`books.user_id` foreign key to `users.id`. A package's table migrations must
+never depend on an application table.
+
+Keep table and column creation with the owning package. Put foreign keys
+between two packages, and other constraints that need the complete schema, in a
+constraints phase: the referencing package's own `constraints/` subdirectory
+when it ships the key, or application `migrations/constraints/`. A package
+constraints migration runs after the application's `migrations/`, so it can also
+reference a table that may belong to the application, such as `users`; for
+example, `@jskit-ai/assistant-runtime` adds its `users` and `workspaces` keys
+from `migrations/constraints/`. Similarly, a `pet_notes.product_id` column can be
+added by the pet-notes package, but its foreign key to the products package's
+`products.id` belongs in a constraints phase. Do not rely on the
+package-directory path order to make a cross-package reference work; when a
+package's table migrations need another package's table, declare `after`.
+
+Until `@jskit-ai/database-runtime` 0.1.252, root `migrations/` ran before the
+package directories, so a fresh database failed on any application migration
+that referenced a package table. Applied migrations stay in the Knex ledger;
+only pending ones follow the new order. An application migration that still
+creates a package-owned table belongs in that package.
+
+The assistant-runtime, console-core and rewarded-core table migrations used to
+add their keys to `users`, `workspaces` and later tables inline. A fresh MySQL
+or MariaDB database failed when the parent table did not exist yet (errno 150),
+and the assistant tables silently skipped their `workspaces` keys. They now add
+them from `migrations/constraints/`. A database migrated earlier keeps the keys
+it already has, and gains any that are missing.
+
+`@jskit-ai/database-runtime/server/foreignKeyMigrations` serves constraints
+migrations that must run on fresh and already-migrated databases alike.
+`addMissingForeignKey(knex, key)` adds a single-column key only when no key
+already links that column to the parent column. It fails, naming the table,
+column and count, when rows have no parent, and it deletes nothing; repair
+those rows and migrate again. `dropForeignKeyIfPresent(knex, key)` serves
+`down()`.
+
+Before adding the key, check that both tables and columns exist, the referenced
+key is indexed appropriately, column types match (including integer signedness),
+existing values have valid parents, and `ON DELETE SET NULL` uses a nullable
+column. An incorrectly formed foreign key is not always an ordering failure.
+Do not suppress the failure, skip a missing parent, or disable foreign-key
+checks to make preparation appear successful.
+
+### MySQL and MariaDB failures can leave partial schema
+
+MySQL and MariaDB DDL can commit implicitly. If adding a column succeeds and a
+later statement adding its foreign key fails, the column can remain even though
+Knex has not recorded that migration as completed. A Knex transaction does not
+make a multi-statement DDL migration atomic. See the
+[MySQL implicit-commit rules](https://dev.mysql.com/doc/refman/8.4/en/implicit-commit.html)
+and [MariaDB implicit-commit rules](https://mariadb.com/docs/server/reference/sql-statements/transactions/sql-statements-that-cause-an-implicit-commit).
+
+Design new multi-step DDL migrations for their concrete partial states. Inspect
+the column and foreign key independently; `hasColumn()` alone cannot prove the
+constraint exists. Verify the definition of an existing object before skipping
+its creation, complete only missing work, and fail clearly on incompatible
+schema. Do not catch arbitrary SQL errors as success or mark incomplete work
+as applied. Keep recovery local to the affected migration or an explicit repair;
+do not add schema repair to request handlers or normal server startup.
+
+For an existing failure, inspect the ledger and actual schema read-only first.
+Preserve applied migration bodies and use a new corrective migration for
+databases that already recorded them. If an earlier pending migration prevents
+that correction from running, describe the required backed-up, explicit repair
+before retrying; a later file alone cannot repair a blocked earlier step. Any
+relocation must retain its basename and be checked against both fresh and
+existing ledgers. Never drop valuable data or delete ledger entries to force
+a replay.
+
+## Seed data is not a migration
+
+Migrations establish schema and invariant framework data. Product fixtures,
+sample accounts, catalog content, and other environment-specific starting data
+belong in an explicit, idempotent application seed operation that runs after
+migrations. Do not hide product seeding in schema migrations.
+
+Managed editors must be able to create one isolated database per development
+session, apply the full migration graph, then invoke that explicit seed
+operation. JSKIT supplies portable migration and seed seams; the editor owns
+database allocation, credentials, lifetime, and environment injection.
+
+## Resource services and custom operations
+
+Conventional persisted resources use `defineTableResource()`,
+`defineCrudContract()` and `defineCrudFeature()` so the framework owns
+repeated table ownership, service, action, permission, JSON API, and route
+mechanics. This does not make product CRUD behavior fixed.
+
+- `decorateRepository` adds resource-specific queries, locks, or writes.
+- `decorateService` overrides a standard method or adds domain methods such as
+  `confirm`, `publish`, `cancel`, or `sendReminder`.
+- `operationLifecycle` surrounds a standard operation with `before`, `execute`,
+  `after`, and mutation-only `afterCommit` phases. Create, update, and delete
+  phases before commit share one repository transaction, and `execute` receives
+  `standard(nextInput)` for retaining the normal framework write. A deadlock
+  runs that transaction again, so `before`, `execute` and `after` can run twice
+  and touch only the database; an outside effect (an email, an HTTP call) goes
+  in `afterCommit`, which runs once after commit, or in an outbox row.
+- Named `actions` expose non-CRUD service methods through normal input,
+  permission, event, and optional HTTP route contracts.
+
+The feature that owns a table reads and writes it through the handles its
+`owns` entry gives its repository; services, hooks and other features call
+that repository or the owner's API, never the table. External delivery belongs after commit; when it must be durable,
+write an outbox record inside the transaction and deliver it separately.
+A separate Feature is warranted when an operation belongs to another domain,
+not merely because a useful resource has behavior beyond list and save. Write
+it as a [typed feature](typed-features.md).
+
+## Writes, versions, locks and deadlocks
+
+Writes that skip their read-back (`returning`), optimistic locking
+(`lockVersionField`), row locks and deadlock retries belong to the table
+resource's handle: see [Data access](data-access.md), Layers 1
+and 4 and [Transactions](data-access.md#transactions).
+
+## Verification
+
+- For every schema change, run the normal preparation entrypoint against a
+  fresh disposable database on the selected engine with the complete package
+  graph. An existing development database can hide ordering failures.
+- In a separate disposable fixture at the previous schema and ledger, apply
+  the upgrade and prove representative existing rows survive.
+- For multi-step MySQL or MariaDB DDL, reproduce partial completion (for
+  example, column present, foreign key absent, migration still pending), retry
+  preparation, and verify the final column and constraint definitions.
+- Run preparation again and `npm run db:migrate:status`; require no pending
+  migrations. Prove invalid references are rejected and the declared delete
+  behavior works. A stubbed Knex test or successful process exit is insufficient.
+- Exercise a real transaction and one invalid-connection case.
+- When a seed operation exists, run it twice and require the second run to be
+  safe.
+- Use the application's selected engine; exercise each supported engine when
+  changing a shared pattern. Report unavailable database verification explicitly.
+
+Do not add migration receipts, sync ledgers, generator provenance, or dialect
+questionnaires. The installed graph, migration source, environment, and
+database migration table are sufficient.
+
+## Integration test databases
+
+Disposable integration and migration tests use `TEST_DB_NAME`, never the
+application's `DB_NAME`. A test run prepares that schema once and restores it
+from a stored snapshot. `jskit test --databases` (the database patterns' `test`
+script) does this with `@jskit-ai/database-runtime/server/testDatabases`, from
+the application's `knexfile.js`:
+
+- Each test file running at once gets a database of its own for as long as it
+  runs, as both `DB_NAME` and `TEST_DB_NAME`: `TEST_DB_NAME` itself, then
+  copies named `<TEST_DB_NAME>_1`, `_2`, ... up to the machine's test budget,
+  as many as the database user may create. With no usable `TEST_DB_NAME`
+  (unset, the connected database itself, or one the database user may not
+  prepare), the files run one at a time on the connected database. A copy
+  named like the application's database or `PROD_DB_NAME`, or a server that
+  cannot be reached, fails the run instead.
+- The snapshot holds the schema as the engine reports it and the rows the
+  migrations wrote, including the migration ledger. On MySQL and MariaDB it is
+  each table's `SHOW CREATE TABLE` and rows, in
+  `node_modules/.cache/jskit-test-databases/`; on PostgreSQL it is the database
+  `<TEST_DB_NAME>_snapshot`, which `CREATE DATABASE <name> TEMPLATE <snapshot>`
+  copies while nothing is connected to it (the user needs `CREATEDB`), file by
+  file (`STRATEGY FILE_COPY`) from PostgreSQL 15. Its key
+  is a digest of the dialect, of the name and contents of every migration file
+  that the application's `migrations/` and its installed packages supply, and
+  of `package-lock.json`, since an upgraded package can change a helper its
+  migrations import. MySQL test databases hold tables only: a schema with
+  views, triggers, routines or events is refused.
+- Each test file gets the environment `jskit test` was given, plus its
+  database's connection (`DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASSWORD`); the
+  rest of the `.env` that `knexfile.js` loads is read for the databases only.
+  The run's settings, connection password included, are in a file only the
+  user can read; the runner's environment names only its directory.
+- When that digest changes, the harness replays the migrations into the
+  designated database once, outside the timed tests, and stores a new
+  snapshot. Otherwise it restores the stored one.
+- A schema the migrations alone do not build, such as migrations that start
+  from a baseline schema file, or reference rows every test reads, comes from
+  the application's own build: `knexfile.js` also exports
+  `testDatabaseSchema = { files, build }`, and a browser `prepare` module passes
+  the same `schema` to `prepareTestDatabases()`. `build(knex)` runs in the empty
+  designated database instead of `knex.migrate.latest()` (the baseline, then
+  `knex.migrate.latest()`), and `files` lists every file it reads, its own
+  module included, relative to the application root: their contents join the
+  digest, so changing one takes a new snapshot.
+- A test that runs migrations up or down, or otherwise changes the tables,
+  leaves its database dirty: its tables or migration ledger differ from the
+  snapshot's. The harness restores the snapshot before another test file uses
+  that database, instead of running `migrate.latest()` over the whole history.
+- A browser run's seed is not part of the snapshot: it runs in each restored
+  database, in one transaction ([Browser test preparation](#browser-test-preparation)).
+- Only the fresh-schema check in [Verification](#verification) replays the
+  whole history, as its own step.
+- The harness reports each phase (digest, restore or rebuild, seed) with its
+  duration, and those durations are read, so slow preparation is visible
+  instead of hiding inside the next test that waits for it. A slow phase is
+  fixed, never absorbed by a longer time limit.
+
+Endpoints and batch operations get their own database tests at realistic row
+counts that compare their query counts, counted with `countQueries()` from
+`@jskit-ai/database-runtime/server/testDatabases`, or `countRuntimeQueries()`
+for a route of the server the test boots; see the typed features guide's
+[data access tests](data-access.md#tests).
+
+## Browser test preparation
+
+A browser run gives every Playwright worker its own server and its own copy of
+`BROWSER_TEST_DB_NAME`. The database patterns' `example/tests/e2e/databases.js`
+is the `prepare` module of `browserTestConfig({ prepare: "tests/e2e/databases.js" })`
+from `@jskit-ai/jskit-catalog/browser-tests`: it calls `prepareTestDatabases()`
+with `variable: "BROWSER_TEST_DB_NAME"`, the worker count and the app's fixture
+seed, and returns each worker's `DB_NAME`. The worker's own process gets the
+same environment as its server, so a test that opens the database itself (with
+the app's `knexfile.js`) reaches that worker's copy. The copies (`<name>`, `<name>_1`,
+...) are restored from the same stored snapshot as the Node test databases
+([Integration test databases](#integration-test-databases)), then seeded in one
+transaction each, so stale rows never survive a run and the migrations are
+replayed only when one changes. It refuses the application's database,
+`PROD_DB_NAME`, `TEST_DB_NAME` and their copies.
+
+Give disposable integration and migration tests their own `TEST_DB_NAME`;
+the two names, and their copies, must differ, and the application account
+needs grants for each (a `<app>_%` grant covers the copies). Each preparation
+reports how many databases it made, whether it replayed the migrations for a
+new snapshot, and its restore and seed durations, so a cold rebuild is visible
+to the caller.
+
+Run related checks in one suite invocation and prepare only their required
+fixtures. A local browser run serves the production build, never the
+development launcher; fresh-schema migration proofs are separate. A managed
+suite (`PLAYWRIGHT_BASE_URL`) uses the host's server URL and identity, as the
+host serves it, starts no second server, and restores normal Preview after the
+batch. Database isolation alone does not
+disable email, payments or other external effects: configure those test controls
+and verify the actual server identity before destructive requests.

package/skills/jskit/references/existing-application-migration.md

@@ -96,31 +96,33 @@ application's normal verification script, and Playwright.
 
 ## 3. Choose the narrowest server seam that owns each behavior
 
-Conventional resources use `defineCrudResource()` and
-`defineCrudJsonApiFeature()` so JSKIT owns the repeated repository, service,
-permission, action, JSON API, and route mechanics. Extend that contract instead
+Conventional resources use `defineTableResource()`, `defineCrudContract()`
+and `defineCrudFeature()` so JSKIT owns the repeated repository,
+service, permission, action, JSON API, and route mechanics. Extend that contract instead
 of copying it.
 
 | Product need | Normal owner |
 | --- | --- |
 | Behavior before, around, or after one standard CRUD operation | `operationLifecycle` |
 | A resource-specific domain method or override | `decorateService` |
-| A resource-specific query, lock, or persistence operation | `decorateRepository` |
+| A resource-specific query, lock, or many-row write | `decorateRepository`, on the table's handle |
 | A named non-CRUD command on the resource | a named CRUD `action` backed by the service |
 | A separate aggregate, workflow, or capability domain | `defineFeature()` with named actions |
-| Persistence that the high-level resource APIs cannot express | an explicit custom repository owned by that Feature |
+| A report the handle cannot express | one `scopedQuery()` repository method, under `// Knex: <kind> - <reason>` ([data access](data-access.md#layer-5-reports-the-resource-cannot-express)) |
 
 For mutations, lifecycle phases before commit share one repository
 transaction. An `execute` hook that only normalizes input calls
-`standard(nextInput)` rather than reimplementing the write. Repositories own
-database access; services and hooks orchestrate repositories. External work
+`standard(nextInput)` rather than reimplementing the write. The feature that
+owns a table reads and writes it through the handles its `owns` entry gives
+its repository; services, hooks and other features call that repository or
+the owner's API, never the table. External work
 runs after commit, or a durable outbox record is written inside the transaction
 and delivered separately.
 
 Give commands product names such as `confirm`, `publish`, or `cancel`, with
 explicit input, permissions, routes where needed, audit behavior, events, and
 tests. Do not hide unrelated workflows behind a generic `execute(anything)`
-action, and do not create a custom repository merely because SQL is familiar.
+action, and do not write Knex merely because SQL is familiar.
 
 ### Reuse response serializers
 
@@ -143,9 +145,11 @@ upgrading JSKIT and declaring the directly used
 ```js
 import Fastify from "fastify";
 import SerializerSelector from "@fastify/fast-json-stringify-compiler";
-import { createCachedResponseSerializerFactory } from "@jskit-ai/kernel/server/http";
+import { createCachedResponseSerializerFactory, resolveTrustedProxies } from "@jskit-ai/kernel/server/http";
 
 const app = Fastify({
+  // TRUSTED_PROXIES: the proxies whose X-Forwarded-For is believed (loopback without it).
+  trustProxy: resolveTrustedProxies(runtimeEnv),
   schemaController: {
     compilersFactory: {
       buildSerializer: createCachedResponseSerializerFactory(SerializerSelector())
@@ -155,84 +159,113 @@ const app = Fastify({
 ```
 
 Keep the application's other Fastify options, including serializer options and
-validator configuration. This setup wraps the native default serializer builder;
+validator configuration, but replace an application's own `trustProxy` setting
+with `resolveTrustedProxies(runtimeEnv)` (`runtimeEnv` is the environment the
+server starts with, `process.env` if it has no other), and its environment
+variable with `TRUSTED_PROXIES`; see
+[sign-in protection](https://mobily-enterprises.github.io/jskit-ai/guide/app-setup/authentication#sign-in-protection)
+for the setting. This setup wraps the native default serializer builder;
 retain any deliberately chosen custom compiler instead of replacing or wrapping
 it. Native route-level compiler overrides continue to apply. Update the lockfile
 through the application's normal installation flow and restart the backend to
 activate the new setup. Updating packages or starter examples alone does not
 change an existing application's startup source.
 
-### JSON REST v2 integration
+### Data access on table resources (json-rest-api 3)
 
-Applications using `@jskit-ai/json-rest-api-core` require Node.js 24 or newer.
+Applications using `@jskit-ai/resource-runtime` require Node.js 24 or newer.
 JSKIT's repository and application default is Node.js 26. Upgrade dependencies,
-custom resource calls, and hooks together; rebuild native database dependencies
-after changing Node.
-
-The standard `defineCrudJsonApiFeature()` API keeps `queryDocuments`,
-`getDocumentById`, `createDocument`, `patchDocumentById`, and
-`deleteDocumentById`, with `{ trx, context }` repository/service options.
-Read, create, and patch services still return tagged JSON:API documents.
-Application-owned hosts use `JsonRestApi` from `json-rest-api`; the JSKIT host
-already owns that setup.
-
-For direct resource calls, replace `inputRecord` with exactly one of `data`
-(plain values) or `document` (a JSON:API document). `format` selects output
-independently: `"plain"` or `"jsonapi"`. Replace `simplified` and
-`returnFullRecord` with `format` and `returning: "full" | "minimal" | "none"`.
-The JSKIT host defaults to plain output and full write responses; standard
-document repositories explicitly request JSON:API output and full create/patch
-responses. A plain collection is still `{ data: [...], meta?, links? }`; use
-`extractJsonRestCollectionRows()` when a custom repository needs its rows.
-A query without a page returns only the resource's first 20 rows; a
-server-side read that needs every row uses `queryAllJsonRestCollectionRows()`
-([every row](typed-features.md#every-row)).
+resource calls and hooks together; rebuild native database dependencies after
+changing Node.
 
-```js
-await jsonRestApi.resources.books.patch({
-  id: recordId,
-  data: { title: "Revised title" },
-  format: "jsonapi",
-  returning: "full",
-  transaction: trx
-}, createJsonRestContext(context));
-```
+Every table has one table resource and one owner feature, and every read and
+write goes through that table's handle
+([data access](data-access.md)). Rename as you go:
 
-Keep the target ID outside the patch data. Direct plain writes use the
-belongs-to relationship name and ID, such as `author: authorId`; the standard
-repository maps a resource field's declared `as` alias. Do not send both names.
-
-Resource hooks share the mutable context supplied as the second argument
-within one operation. The normalized hook field remains
-`context.inputRecord.data`; only the public input argument changed.
-`createJsonRestContext()` clones the JSKIT execution context and its
-`visibilityContext` and `scopeValues`, allowing frozen caller contexts; other
-nested objects remain shared references. Permission hooks use
-`context.originalContext ?? context` for operation-wide caches; enrichment
-hooks use `context.parentContext`. Built-in nested operations get separate
-contexts with shallow inheritance. For an application-initiated child call,
-choose the application properties to share in a fresh context and pass
-`transaction` explicitly; reusing or blindly spreading an active operation
-context carries its working state into the child.
-
-The standard repository's `withTransaction(work)` delegates to
-`jsonRestApi.transaction(work)`. Its callback and mutation lifecycle phases
-receive the actual callable Knex transaction: custom repository SQL can use
-`await trx("audit_entries").insert(...)`, while resource calls receive it as
-`transaction`. Await every participant before returning. The helper owns
-commit, rollback, and completion hooks. Raw `knex.transaction()` handles cannot
-own resource writes, and calling `transaction()` again starts an independent
-unit; forward the existing handle to compose work.
+| Before | Now |
+| --- | --- |
+| `defineCrudResource()` | `defineTableResource()` plus `defineCrudContract()` |
+| `defineCrudJsonApiFeature()`, `createCrudResourceRuntime()` | `defineCrudFeature({ crud })` |
+| the CRUD runtime's `listByIds()`, `listByForeignIds()` | `queryAll({ filters: { [key]: ids } })` on a filter declared `{ type: "array", filterOperator: "in" }`; a page's children with `createHooksToCollectChildren()` |
+| the CRUD runtime's lookup hydration | `include` on the CRUD list or view |
+| the CRUD runtime's `virtualFields` | `queryFields` in the `owns` entry, or `decorateService` for a value SQL cannot compute |
+| the CRUD runtime's `createdAt`/`updatedAt` stamping | every handle write stamps them; only a system context sets its own |
+| the CRUD runtime's output validation | the table resource's schema, whose types json-rest-api reads each field as |
+| `defineCrudFeature`'s `ownershipFilter`, `searchSchema` | `visibility`; an `owns` entry |
+| a lifecycle hook's `trx`; a decorator's or CRUD capability's `resource` | `options`; `crud` |
+| `decorateRepository`'s `database`, `http`, `jsonRestApi` | its `repository` and `resources`, each owned table's handle |
+| `@jskit-ai/crud-core/server/fieldAccess`, `/server/lookups` | removed (nothing called them); restrict fields in the CRUD contract and its `decorateService` |
+| `runtime.json-rest-api`, `addResourceIfMissing()`, `createJsonRestResourceScopeOptions()` | `defineFeature({ owns })`; a row policy, search schema or query fields go in the `owns` entry |
+| `queryDocuments()`, `getDocumentById()` | `query({ page: { size } })`, `get(id)` |
+| `queryAllJsonRestCollectionRows()`, `queryEveryDocument()` | `queryAll(params)` |
+| `createDocument()`, `createDocuments()` | `create()`, `createMany()` |
+| `patchDocumentById()`, `patchDocumentIfLockVersion()` | `patch(id, changes, options)`, `patch(id, changes, { ...options, lockVersion })` |
+| `deleteDocumentById()` | `delete(id)` |
+| `getDocumentById(…, { forUpdate: true })`, `applyForUpdate()`, `lockScopedRecordId()` | `lock(id)` inside `withTransaction` |
+| `runInTransaction()`, `createWithTransaction()`, `joinOrOpenTransaction()`, `resolveTransaction()`, `resolveQueryOptions()`, `resolveRepoClient()` | `withTransaction(options, (txOptions) => …)` |
+| `returning: "full" \| "minimal"` | `returning: "record" \| "id"` (`"none"` is unchanged) |
+| a hand-built `visibilityContext` or scope values, `applyVisibility()`, `applyVisibilityOwners()` | the caller's context, or `systemContext(reason).forWorkspace(id)` |
+| a field's `default:` | `defaultTo:` (`default` is refused, since json-rest-api never read it) |
+| a `createdAt` or `updatedAt` default of now | nothing: drop it, since every handle write stamps them |
+| a CRUD contract that showed every column | a contract shows only its `fields`, so list every field the API showed |
+| reading every row | `queryAll`, which refuses above 10,000 rows, so page with `query()` |
+| an app's own wrappers around old calls | the handle method they wrapped, then delete the wrapper |
+
+Handles answer records, `{ id, ...fields }`, with a belongs-to field as the
+related id; the CRUD service alone answers JSON:API documents. Every call takes
+`options` last: the caller's context, and inside a transaction the `txOptions`
+`withTransaction` handed its work. A call without a context throws on a table
+that is not public.
+
+json-rest-api 3 changes what a failed write means: any failed write inside a
+managed transaction fails the whole transaction. "Insert, catch the duplicate,
+then read" inside a transaction no longer works; use
+`createMany(rows, { ...options, onConflict: { keys, action: "ignore" } })` and then read,
+or read first. A `get`, `patch`, `delete` or `lock` of a missing or hidden
+record throws json-rest-api's not-found error (a 404); with `missing: "skip"`
+it answers `null`, without failing the caller's transaction. Upsert rows carry no
+`id`, never repeat their conflict keys, and name the ownership field among
+those keys; only a table with no owners and no row policy may key the upsert on
+the id alone, `onConflict: { keys: ["id"] }`, with an id on every row. `patchWhere` and `deleteWhere` need filters, or `all: true`.
 
 Preserve `error.transactionOutcome` and its cause when translating failures:
 `none` means no transaction was acquired or accepted, `pending` leaves
 completion to the owner, `rolledBack` confirms rollback, `committed` confirms
 commit, and `unknown` means the outcome is unresolved. These describe database
-transaction evidence, not external effects. A failed JSKIT mutation
-`afterCommit` hook raises `RestApiWriteError` with outcome `committed` and does
-not undo the write. Do not turn pending, committed, or unknown write failures
-into successful missing-record results, or automatically replay a committed
-or unresolved write.
+transaction evidence, not external effects. A failed `afterCommit` hook raises
+`RestApiWriteError` with outcome `committed` and does not undo the write. Do
+not turn pending, committed, or unknown write failures into successful
+missing-record results, or automatically replay a committed or unresolved
+write.
+
+### Moving hand-written SQL behind resources
+
+An application with Knex in its repositories moves it behind table resources
+one table at a time:
+
+1. **Inventory.** List every statement that reads or writes a table: the
+   repository methods, raw queries, migrations' seed data, jobs and scripts.
+   `jskit check` and the data-access lint rules (`jskit/no-direct-knex`,
+   `jskit/no-hand-scope`) find most of them.
+2. **One owner per table.** Pick the feature that owns each table and give the
+   table one table resource with its `ownership`. Two features writing one
+   table means one of them asks the other.
+3. **Owner APIs.** Code outside the owner calls the owner's service or
+   repository capability. JSKIT's own tables already work this way: users
+   (`users.profiles`, `users.settings`), workspaces, local sign-in and console
+   settings.
+4. **Map each statement to a layer**
+   ([data access](data-access.md)): one record or page →
+   Layer 1; screens over HTTP → Layer 2; a loop of writes → a many-row write;
+   a `forUpdate` read → a lock; totals and reports → `aggregate`, a window
+   include, or one `scopedQuery()` method; work over every workspace → a job
+   with `systemScope`.
+5. **Two tests per path.** `expectScoped` proves the path never reads or
+   changes a second owner's rows; `expectFlatQueries` proves its statements do
+   not grow with its rows. Both come from `@jskit-ai/resource-runtime/test`.
+
+Keep each move behaviour-preserving
+([refactors](sanity-checks.md#refactors)).
 
 ### Workspace invitation transaction compatibility (workspaces-core 0.1.198)