@kequtech/tenon-orm 0.4.0 → 0.4.3

package/docs/human-guide.md

@@ -1,49 +1,74 @@
-# Using Tenon: a human guide
+# Getting started
 
-Tenon lets you use PostgreSQL directly from TypeScript. You write SQL; Tenon binds its inputs and
-executes it through pgclear. This guide follows a small application from its first table to reusable
-queries, transactions, and result checking.
+This guide creates a table, inserts and reads a user, reuses a query in a transaction, and generates
+TypeScript types. You need an existing PostgreSQL database. If your application already has tables
+and a migration runner, keep them and start at [Write and read a row](#write-and-read-a-row).
 
-Use the [reference](reference.md) for exact contracts and [adoption](adoption-guide.md) when
-replacing an existing ORM. Code blocks build on the connection and table introduced below;
-query-function examples belong in their own module.
+## Install and connect
 
-## Connect to your database
-
-In an existing Node.js 24+ TypeScript project using ESM:
+Use Node.js 24+, PostgreSQL 16+, ESM, and TypeScript 6+ with Node.js development typings. In an
+existing TypeScript project:
 
 ```sh
 npm install @kequtech/tenon-orm
 ```
 
-Use PostgreSQL 16+ and TypeScript 6+. Keep your existing strict TypeScript configuration and Node
-typings. Tenon includes pgclear 0.3.x. Supply `DATABASE_URL` through your application's
-environment setup; Tenon does not load `.env` files. Import `sql`, `json`, `param`, and codec helpers
-from `@kequtech/tenon-orm` for application queries. See [driver updates](reference.md#driver-updates)
-when updating Pgclear independently.
+Keep the project's TypeScript configuration; a typical Node.js application uses `module: "NodeNext"`
+and `strict: true`. Tenon includes its PostgreSQL driver, pgclear.
+
+Supply `DATABASE_URL` through your environment setup. For a local example database:
+
+```sh
+export DATABASE_URL=postgresql://postgres:postgres@localhost:5432/your_database
+```
+
+Tenon does not create the database or load `.env` files. Create shared configuration:
+
+```sh
+npx tenon init
+```
+
+The command writes `tenon.config.ts` without opening a database or overwriting existing config:
+
+```ts
+// tenon.config.ts
+import type { TenonConfig } from '@kequtech/tenon-orm';
+
+const connectionString = process.env.DATABASE_URL;
+if (!connectionString) throw new Error('DATABASE_URL is required');
+export default {
+  connection: { connectionString },
+  migrations: { directory: 'migrations', namespace: 'public' },
+  generate: { out: 'src/generated-schema.d.ts' },
+  check: { project: 'tsconfig.json' },
+} satisfies TenonConfig;
+```
+
+Use it in the application too:
 
 ```ts
 // src/db.ts
 import { createTenon } from '@kequtech/tenon-orm';
+import config from '../tenon.config.ts';
 
-const connection = process.env.DATABASE_URL;
-if (!connection) throw new Error('DATABASE_URL is required');
-export const db = createTenon({ connection });
+export const db = createTenon({ connection: config.connection });
 ```
 
-Reuse this instance across the application and call `await db.close()` at shutdown. You can also
-supply an existing pgclear `Pool`; your application then owns its shutdown.
+These `.ts` imports use Node.js 24+ TypeScript support. See
+[shared configuration](configuration.md) for import conventions, custom codecs, discovery, and overrides.
+
+Reuse `db` across requests. Call `await db.close()` when a standalone script finishes or a server
+shuts down. See [connections](connections.md) for pool options and application-owned pools.
 
-## Create a table
+## Create and apply a migration
 
-If you already have a schema and migration runner, keep them. For a fresh example database, create a
-migration:
+In a fresh migrations directory:
 
 ```sh
 npx tenon new create_users
 ```
 
-Fill the generated files:
+This creates two empty files. Put the table definition in the up file:
 
 ```sql
 -- migrations/0001_create_users.up.sql
@@ -54,91 +79,74 @@ CREATE TABLE public.users (
 );
 ```
 
+Put the reverse operation in the down file:
+
 ```sql
 -- migrations/0001_create_users.down.sql
 DROP TABLE public.users;
 ```
 
-For development, add a script to `package.json` that applies migrations and updates generated types:
-
-```json
-{
-  "scripts": {
-    "db:migrate": "tenon migrate && tenon generate --out src/generated-schema.d.ts"
-  }
-}
-```
+Apply it to the database selected by `DATABASE_URL`:
 
 ```sh
-npm run db:migrate
+npx tenon migrate
+npx tenon status
 ```
 
-Commit migrations and generated types together. In production, run only `npx tenon migrate`; deploy
-the already-built application. If you do not use generated table types, omit generation from the
-development script. For later schema changes, add a new migration instead of editing an applied one.
-If a change cannot be reversed, omit its down file. A down migration is not a backup.
+For later changes, create a new migration. Applied files are checked against recorded checksums and
+must stay unchanged. If a migration cannot be reversed, remove its down file before applying it.
+See [migrations](migrations.md) for rollback and deployment behavior.
 
-## Write a query
-
-Create a row, then read the fields you need:
+## Write and read a row
 
 ```ts
+// src/example.ts
 import { sql } from '@kequtech/tenon-orm';
 import { db } from './db.js';
 
-const email = 'reader@example.test';
-await db.query(sql`INSERT INTO public.users (email) VALUES (${email})`);
-const user = await db.query(
-  sql<{ id: number; email: string }>`
-    SELECT id, email FROM public.users WHERE email = ${email}
-  `,
-  { returns: 'oneOrNone' },
-);
-```
-
-`${email}` becomes a bound parameter. Do not put quotes around it: the driver passes the value
-separately from the SQL text, even when the value contains SQL.
-
-The type describes one selected row. `returns: 'oneOrNone'` gives you that row or `undefined`; it
-throws if the query returns more than one row. Use `'one'` when absence should also be an error,
-`'many'` for an array, or omit `returns` for the native result object with `.rows`, `.rowCount`, and
-`.fields`. These options do not change your SQL.
-
-`rowCount` is a number or null when the command has no count. SQL `count(*)` and bigint/bigserial
-column values are exact strings, including values beyond JavaScript's safe integer range. Use
-strings, bigint, or pgclear `param(text, builtin.int8)` for int8 parameters. Plain Date and supported
-Temporal inputs also work; they do not change the default string outputs. Date inputs encode in UTC.
-Opt-in codecs can return Date, bigint, or Temporal values when the application needs them.
-
-Generated table types can help describe projections without repeating every field:
-
-```ts
-import type { Database } from './generated-schema.js';
-type UserSummary = Pick<Database['users']['row'], 'id' | 'email'>;
+try {
+  const email = 'reader@example.test';
+  const created = await db.query(
+    sql<{ id: number }>`INSERT INTO public.users (email) VALUES (${email}) RETURNING id`,
+    { returns: 'one' },
+  );
+
+  const user = await db.query(
+    sql<{ id: number; email: string; active: boolean }>`
+      SELECT id, email, active FROM public.users WHERE id = ${created.id}
+    `,
+    { returns: 'oneOrNone' },
+  );
+  console.log(user);
+} finally {
+  await db.close();
+}
 ```
 
-For frequently used types, define [application aliases](reference.md#application-aliases) such as
-`UserRow`, `UserInsert`, and `UserUpdate` in a handwritten module.
+This inserts a new email; rerunning it with the same email will fail the unique constraint. Run it
+through your project's TypeScript runner or compile it with your application. The `.js` imports
+follow NodeNext's convention for compiled TypeScript modules.
 
-You can then write `sql<UserSummary>`. A table type describes the full table; a join or aggregate
-needs a type for what that particular query selects. An annotation is a declaration, not proof that
-the SQL matches. Without one, result fields are `unknown`. Use the inspection and database-test
-workflow below to check the declared contract against the query's behavior.
+The template binds `${email}` separately from the SQL. Do not put quotes around an interpolation.
+`sql<{ id: number }>` describes one returned row; `RETURNING id` asks PostgreSQL to return it.
+`returns: 'one'` requires exactly one row, while `oneOrNone` permits absence as `undefined`.
+Use `many` for an array or omit `returns` for the driver result with `.rows` and `.rowCount`.
 
-Keep storage types conservative and validate them when projecting domain values. The
-[persistence-to-domain example](reference.md#persistence-and-domain-values) shows validated JSON
-writes and reads, temporal strings, and an SQL predicate that rejects infinite session expiry.
+The annotation does not check SQL or validate the row at runtime. Without an annotation, field values
+are `unknown`. Bigint columns and SQL `count(*)` return strings by default; the integer ID in this
+example returns a number. See [value mappings](schema-types.md#value-mappings) for JSON, arrays,
+decimals, and dates.
 
-## Reuse a query with a function
+## Reuse a query
 
-Use an ordinary function when several callers need the same query. An object input makes the
-supplied values easy to read:
+Put reusable queries in ordinary functions. Accept `QueryClient` when a caller may supply either
+`db` or a transaction's `tx`:
 
 ```ts
+// src/queries.ts
 import { sql, type QueryClient } from '@kequtech/tenon-orm';
-import type { Database } from './generated-schema.js';
 
-type UserSummary = Pick<Database['users']['row'], 'id' | 'email'>;
+type UserSummary = { id: number; email: string };
 
 export function userByEmail(client: QueryClient, { email }: { email: string }) {
   return client.query(sql<UserSummary>`SELECT id, email FROM public.users WHERE email = ${email}`, {
@@ -147,12 +155,15 @@ export function userByEmail(client: QueryClient, { email }: { email: string }) {
 }
 ```
 
-Call `userByEmail(db, { email: 'reader@example.test' })` at the call site, or pass a transaction's
-`tx`. Keep the SQL and execution options together in the function.
+Call `userByEmail(db, { email: 'reader@example.test' })`. Keep the SQL and result options together in
+the function. Joins and aggregates work the same way; see [writing queries](sql.md).
+
+## Use a transaction and a write guard
 
-## Change data in a transaction
+In code using the shared `db`:
 
 ```ts
+import { sql } from '@kequtech/tenon-orm';
 import { userByEmail } from './queries.js';
 
 await db.transaction(async (tx) => {
@@ -165,117 +176,77 @@ await db.transaction(async (tx) => {
 });
 ```
 
-Use `tx` for all work in the callback and await it. Tenon commits when the callback succeeds and
-rolls back when an error escapes.
-
-`affectedRows: { exact: 1 }` requires exactly one affected row. `{ max: 1 }` permits zero or one.
-Tenon rolls back a write that violates the bound. You can also use these guards on `db.query`
-outside a callback transaction. The guard checks a count; you still need the correct `WHERE` clause
-and application authorization. Checking returned rows with `returns` alone does not undo an
-autocommitted write.
-
-## Write objects without building column lists by hand
-
-Use `sql.insertValues(rowOrRows)` inside an authored `INSERT` and `sql.assignments(patch)` after
-`SET`. Inserts use database defaults for missing/undefined fields; updates omit those fields.
-Explicit `null` stays null. Empty batches and updates throw. Heterogeneous insert rows remain
-aligned, with `DEFAULT` for missing cells.
-
-```ts
-import type { Database } from './generated-schema.js';
-
-const input: Database['users']['insert'] = { email: 'another@example.test' };
-await db.query(sql`INSERT INTO public.users ${sql.insertValues(input, ['email'])}`);
-
-const patch: Pick<Database['users']['update'], 'active'> = { active: false };
-await db.query(sql`UPDATE public.users SET ${sql.assignments(patch, ['active'])}
-  WHERE email = ${input.email}`);
-```
+Run all participating queries through `tx` and await them. The callback commits on success and rolls
+back when an error escapes.
 
-Use `Pick` to limit editable fields. Keep expressions, predicates, conflict handling, and
-`RETURNING` in SQL. Selected columns choose fields; they do not validate the complete object. See
-[input types](reference.md#insert-and-update-inputs) for mappings and the
-[mutation cookbook](mutations.md) for defaults, batches, and upserts.
+The guard requires exactly one affected row and rolls back the update if the count differs. Use
+`{ max: 1 }` to allow zero or one. Guards also work with `db.query` outside a callback. Result modes
+such as `one` check returned rows; by themselves they do not undo an autocommitted write.
 
-## Check your SQL and result types
+You still author the `WHERE` clause and application authorization. The
+[transaction reference](reference.md#transactions-guarded-writes-and-errors) covers caught errors,
+savepoints, and retry limits. For object-based inserts and patches, see [writing data](mutations.md).
 
-The three database tools answer different questions: `tenon check` compares authored queries with
-their TypeScript contracts, `tenon describe` reports metadata for finished SQL, and
-`tenon generate --check` checks whether saved database declarations are current. See
-[validation tools](reference.md#choosing-a-validation-tool) for their limits.
+## Generate types
 
-After migrating a development/test database, run the optional source-aware checker:
+Generation is optional. It reads the migrated database and writes table read/insert/update types.
+The CLI uses the shared connection and codecs from `tenon.config.ts`:
 
 ```sh
-npx tenon check src/queries.ts
-npx tenon check src/queries.ts:23
+npx tenon generate
+npx tenon generate --check
 ```
 
-A line selects one complete query. Omit the file to check the TypeScript project. The checker reads
-source and types without importing the application or executing SQL. It reports definite
-contradictions and unsupported composition; nullability, JSON members, and other inconclusive
-contracts receive the benefit of the declared type and remain unproven. See
-[query checking](query-checking.md) for supported fragments and explicit object expansion shapes.
+The second command checks committed output without rewriting it. The namespace comes from
+`generate.namespace`, then `migrations.namespace`, then `public`, independently of runtime
+`search_path`. Generate after migrations have committed.
 
-Use describeQuery on finished SQL, or tenon describe query.sql, when you only need metadata and a
-conservative row declaration. Match runtime role/session settings. Retain TypeScript and database
-tests for actual rows, effects, permissions, and rollback. The checker does not replace those tests.
-Generated patch types contain optional fields and can therefore be unchecked even when execution is
-valid. Keep optional patches when needed; see [responding to findings](query-checking.md#responding-to-findings).
-For rejected values, result-count errors, and other common failures, see
-[troubleshooting](reference.md#troubleshooting).
+Add these commands to your existing `package.json`:
 
-## Reuse or assemble parts of SQL when needed
+```json
+{
+  "scripts": {
+    "db:generate": "tenon generate",
+    "db:check-types": "tenon generate --check",
+    "db:check": "tenon check",
+    "db:migrate": "tenon migrate && tenon generate"
+  }
+}
+```
 
-A fragment is a piece of SQL that carries its bound values with it. It is useful when multiple
-queries share a condition. For example, a search can use the same predicate for its result page and
-its count:
+Run `npm run db:generate` now. The output exports a `Database` type:
 
 ```ts
-const pattern = '%example%';
-const limit = 20;
-const search = sql.fragment`active = ${true} AND email ILIKE ${pattern}`;
-const page = sql<UserSummary>`SELECT id, email FROM public.users
-  WHERE ${search} ORDER BY id LIMIT ${limit}`;
-const count = sql<{ count: string }>`SELECT count(*) AS count FROM public.users
-  WHERE ${search}`;
-```
+import type { Database } from './generated-schema.js';
 
-Execute each query with `db.query`. Validate application-supplied patterns and page sizes before
-building queries. The complete query supplies the result type.
-The SQL aggregate's `count` is a string; the native result's `rowCount` counts returned rows.
+type UserRow = Database['users']['row'];
+type UserSummary = Pick<UserRow, 'id' | 'email'>;
+type UserInsert = Database['users']['insert'];
+type UserUpdate = Database['users']['update'];
+```
 
-Use `sql.join` when the number of fragments varies, such as one search condition per word or a set
-of optional filters. It combines fragments with `'AND'`, `'OR'`, or `','` and preserves their bound
-values:
+Use `UserSummary` with `sql<UserSummary>` or `UserInsert` to type an insert input. These types
+follow table columns, defaults, nullability, and codec representations. They do not validate
+application inputs. Keep aliases in handwritten files and leave generated output unchanged.
+See [schema types](schema-types.md) for an output example and mapping details.
 
-```ts
-const terms = ['reader', 'example'];
-const filters = [sql.fragment`active = ${true}`];
-for (const term of terms) {
-  filters.push(sql.fragment`email ILIKE ${'%' + term + '%'}`);
-}
+## Keep schema and code in step
 
-const query = sql<UserSummary>`SELECT id, email FROM public.users
-  WHERE ${sql.join(filters, 'AND')} ORDER BY id`;
-```
+During development, run `npm run db:migrate`, review the generated diff, and commit migrations and
+generated declarations together. Exclude the generated file from automatic formatting.
 
-This matches active users whose email contains every term. With no terms, the active condition still
-applies. For an entirely optional set of filters, omit the whole `WHERE` clause when the list is
-empty:
+In CI, migrate a dedicated test database and check committed output without rewriting it:
 
-```ts
-const optionalFilters = terms.map((term) => sql.fragment`email ILIKE ${'%' + term + '%'}`);
-const unfilteredOrMatching = optionalFilters.length
-  ? sql<UserSummary>`SELECT id, email FROM public.users WHERE ${sql.join(optionalFilters, 'AND')} ORDER BY id`
-  : sql<UserSummary>`SELECT id, email FROM public.users ORDER BY id`;
+```sh
+npx tenon migrate
+npm run db:check-types
+npx tsc --noEmit
 ```
 
-`sql.join` and fragments require nonempty input. Keep mandatory authorization predicates in the
-initial list when applicable; an empty optional-filter list must not remove them. Use parentheses
-when mixing `AND` and `OR`. See the [composition reference](reference.md#sql-fragments-and-joins).
+Do not run the regenerating `db:migrate` script before the freshness check: it would replace the file
+you meant to verify. Production runs migrations and deploys the already-built application.
 
-Use fixed SQL when composition is unnecessary. See the [reference](reference.md) for arrays,
-identifiers, value mappings, CLI options, and error behavior.
-Runtime loops and dynamic fragment lists work during execution but remain unchecked by `tenon check`;
-use database tests for those queries.
+Optionally run `npx tenon check src/queries.ts` against the migrated database to compare source query
+types with PostgreSQL metadata. Keep database tests for rows, permissions, and rollback. See
+[choosing a validation tool](reference.md#choosing-a-validation-tool) and
+[troubleshooting](reference.md#troubleshooting) as needed.

package/docs/migrations.md

@@ -0,0 +1,140 @@
+# SQL migrations
+
+Tenon's optional runner applies numbered SQL files and records their checksums in the database. It
+can be used independently of query execution and type generation. Keep an existing runner unless
+you intend to [transfer migration ownership](adoption-guide.md#transfer-migration-ownership).
+
+## Create and apply a change
+
+```sh
+npx tenon new add_user_display_name
+```
+
+`new` opens no database connection. It creates the next numeric pair in the configured migration
+directory, defaulting to `migrations/`, and starts at `0001` in an empty directory. As with other
+commands, importing your config may require its environment values. For a directory whose latest
+migration is `0001`:
+
+```sql
+-- migrations/0002_add_user_display_name.up.sql
+ALTER TABLE public.users ADD COLUMN display_name text;
+```
+
+```sql
+-- migrations/0002_add_user_display_name.down.sql
+ALTER TABLE public.users DROP COLUMN display_name;
+```
+
+Configure `connection` and migration directory/namespace in [shared configuration](configuration.md).
+Without config, migration commands retain the `DATABASE_URL` workflow. Then run:
+
+```sh
+npx tenon status
+npx tenon migrate
+```
+
+`status` prints JSON with `applied` and `pending` ID arrays. `migrate` applies every pending up file
+in numeric order and reports completed IDs. No pending files means nothing changes. No command
+loads `.env` automatically.
+
+After migrating a development database, [generate types](schema-types.md#generate-and-check-a-file),
+review the diff, and commit migrations with generated declarations. Production applies migrations
+and deploys the already-built application. Use a dedicated migrated database for tests.
+
+## File rules and history
+
+Files use `<numeric-id>_<name>.up.sql` and optional matching `.down.sql`. Names contain letters,
+digits, underscores, or hyphens. Each ID has one name and an up file. SQL files with invalid names,
+empty contents, duplicate numeric IDs, or down files without an up file fail validation.
+
+The runner records ID, name, checksum, and application time in `<namespace>.__tenon_migrations`.
+Checksums cover both up SQL and the presence/content of down SQL. Never edit, rename, remove, or
+add a down file to an already-applied migration. Add a new forward migration instead. History must
+match the beginning of the ordered file list; inserting an older migration also causes a conflict.
+
+Migrations may contain multiple statements. Tenon owns the transaction, so authored files must not
+contain transaction-control statements. Use dollar-quoted bodies for functions and explicit `E'...'`
+for backslash escapes. A scaffold's files are empty until you fill them; both are checked even if
+only the up direction is about to run.
+
+## Transaction and failure behavior
+
+Each migration file and its history entry commit together. The whole run is not one transaction:
+if file three fails, files one and two may already have committed. Correct the cause, inspect
+`status`, and rerun. The failed file is rolled back and remains pending. Restore original applied
+files if history validation reports a conflict.
+
+SQL that cannot run in a transaction, such as `CREATE INDEX CONCURRENTLY`, needs a separate,
+application-owned procedure. Tenon has no nontransactional migration flag. It also does not infer
+schema changes or generate migration SQL from TypeScript.
+
+## Roll back
+
+```sh
+npx tenon rollback
+npx tenon rollback --steps 2
+```
+
+Rollback runs the most recent down files in reverse order and removes their history records. It
+prechecks that every selected migration has down SQL before executing any. `steps` must be a
+positive safe integer; requesting more steps than exist selects all applied migrations.
+
+For an irreversible change, omit the down file before first application. The missing file prevents
+rollback across that migration. A down migration is not a backup: dropped data and sequence changes
+may not be recoverable. Use the application's backup and recovery procedure when needed.
+
+## Directory, schema, and locking
+
+Set `migrations: { directory: 'db/migrations', namespace: 'app' }` in shared configuration to use
+the same defaults for creation, status, migration, and rollback. Generation inherits that namespace
+unless `generate.namespace` is set. Without configured values, defaults are directory `migrations`
+and namespace `public`.
+
+CLI flags override the config for an individual command:
+
+```sh
+npx tenon new add_setting --directory db/migrations
+npx tenon migrate --directory db/migrations --namespace app
+npx tenon status --directory db/migrations --namespace app
+npx tenon rollback --directory db/migrations --namespace app --steps 1
+```
+
+The namespace selects history location, migration search path, and lock scope. During a file,
+`search_path` is `<namespace>, pg_catalog`; qualify cross-schema names explicitly. The runner creates
+the namespace when applying pending migrations. It does not configure runtime application queries.
+Keep generation on the same namespace, through the shared default or an explicit override.
+
+Migration and adoption operations use an exclusive session advisory lock for this database and
+namespace. Status and generation use a shared lock. Different namespaces have independent lock
+scopes. These locks do not coordinate with a legacy runner or arbitrary DDL.
+
+Use direct or session-pooled connections for these tools; transaction pooling does not preserve
+their session locks. Configure server timeouts on the tooling connection to bound waits. Migration
+commands do not accept Describe's timeout flags.
+
+Offline file creation uses `.tenon-create.lock` in the directory. Remove a stale lock only after
+confirming that the process which created it has stopped. Baseline capture uses this local lock
+and `pg_dump`; it does not take the runner's database advisory lock.
+
+## Existing databases
+
+An existing schema needs no baseline merely to execute queries or generate types. A baseline is
+needed when Tenon takes over migration history: capture one reviewed schema at the old runner's
+final checkpoint, record it on existing databases without execution, and execute it on fresh ones.
+Follow [the adoption workflow](adoption-guide.md#transfer-migration-ownership).
+
+## Programmatic API
+
+```ts
+import { migrate, migrationStatus, rollback } from '@kequtech/tenon-orm';
+
+const options = { connection, directory: 'migrations', namespace: 'public' };
+const before = await migrationStatus(options); // { applied, pending }
+const applied = await migrate(options); // readonly string[] of completed IDs
+const reverted = await rollback({ ...options, steps: 1 });
+```
+
+`connection` accepts the same URI/options/pool forms as `createTenon`. A supplied pool retains
+application ownership. `createMigration({ name, directory? })` returns `{ id, upPath, downPath }`;
+`readMigrations(directory?)` returns validated `{ id, name, up, down?, checksum }` records.
+See [the command reference](reference.md#command-reference) for all CLI forms.