@kequtech/tenon-orm 0.3.1 → 0.4.2

package/docs/human-guide.md

@@ -1,24 +1,28 @@
-# Using Tenon: a human guide
+# Getting started
 
-Tenon lets you use PostgreSQL directly from TypeScript. You write SQL; Tenon binds its inputs and
-executes it through pg. This guide follows a small application from its first table to reusable
-queries, transactions, and result checking.
+This guide creates a table, inserts and reads a user, reuses a query in a transaction, and generates
+TypeScript types. You need an existing PostgreSQL database. If your application already has tables
+and a migration runner, keep them and start at [Write and read a row](#write-and-read-a-row).
 
-Use the [reference](reference.md) for exact contracts and [adoption](adoption-guide.md) when
-replacing an existing ORM. Code blocks build on the connection and table introduced below;
-query-function examples belong in their own module.
+## Install and connect
 
-## Connect to your database
-
-In an existing Node.js 24+ TypeScript project using ESM:
+Use Node.js 24+, PostgreSQL 16+, ESM, and TypeScript 6+ with Node.js development typings. In an
+existing TypeScript project:
 
 ```sh
 npm install @kequtech/tenon-orm
 ```
 
-Use PostgreSQL 16+ and TypeScript 6+. Keep your existing strict TypeScript configuration and Node
-typings. Supply `DATABASE_URL` through your application's environment setup; Tenon does not load
-`.env` files.
+Keep the project's TypeScript configuration; a typical Node.js application uses `module: "NodeNext"`
+and `strict: true`. Tenon includes its PostgreSQL driver, pgclear.
+
+Supply `DATABASE_URL` through your environment setup. For a local example database:
+
+```sh
+export DATABASE_URL=postgresql://postgres:postgres@localhost:5432/your_database
+```
+
+Tenon does not create the database or load `.env` files. Create one shared database module:
 
 ```ts
 // src/db.ts
@@ -29,19 +33,18 @@ if (!connection) throw new Error('DATABASE_URL is required');
 export const db = createTenon({ connection });
 ```
 
-Reuse this instance across the application and call `await db.close()` at shutdown. You can also
-supply an existing `pg.Pool`; your application then owns its shutdown.
+Reuse `db` across requests. Call `await db.close()` when a standalone script finishes or a server
+shuts down. See [connections](connections.md) for pool options and application-owned pools.
 
-## Create a table
+## Create and apply a migration
 
-If you already have a schema and migration runner, keep them. For a fresh example database, create a
-migration:
+In a fresh migrations directory:
 
 ```sh
 npx tenon new create_users
 ```
 
-Fill the generated files:
+This creates two empty files. Put the table definition in the up file:
 
 ```sql
 -- migrations/0001_create_users.up.sql
@@ -52,82 +55,74 @@ CREATE TABLE public.users (
 );
 ```
 
+Put the reverse operation in the down file:
+
 ```sql
 -- migrations/0001_create_users.down.sql
 DROP TABLE public.users;
 ```
 
-For development, add a script to `package.json` that applies migrations and updates generated types:
-
-```json
-{
-  "scripts": {
-    "db:migrate": "tenon migrate && tenon generate --out src/generated-schema.d.ts"
-  }
-}
-```
+Apply it to the database selected by `DATABASE_URL`:
 
 ```sh
-npm run db:migrate
+npx tenon migrate
+npx tenon status
 ```
 
-Commit migrations and generated types together. In production, run only `npx tenon migrate`; deploy
-the already-built application. If you do not use generated table types, omit generation from the
-development script. For later schema changes, add a new migration instead of editing an applied one.
-If a change cannot be reversed, omit its down file. A down migration is not a backup.
+For later changes, create a new migration. Applied files are checked against recorded checksums and
+must stay unchanged. If a migration cannot be reversed, remove its down file before applying it.
+See [migrations](migrations.md) for rollback and deployment behavior.
 
-## Write a query
-
-Create a row, then read the fields you need:
+## Write and read a row
 
 ```ts
+// src/example.ts
 import { sql } from '@kequtech/tenon-orm';
 import { db } from './db.js';
 
-const email = 'reader@example.test';
-await db.query(sql`INSERT INTO public.users (email) VALUES (${email})`);
-const user = await db.query(
-  sql<{ id: number; email: string }>`
-    SELECT id, email FROM public.users WHERE email = ${email}
-  `,
-  { returns: 'oneOrNone' },
-);
+try {
+  const email = 'reader@example.test';
+  const created = await db.query(
+    sql<{ id: number }>`INSERT INTO public.users (email) VALUES (${email}) RETURNING id`,
+    { returns: 'one' },
+  );
+
+  const user = await db.query(
+    sql<{ id: number; email: string; active: boolean }>`
+      SELECT id, email, active FROM public.users WHERE id = ${created.id}
+    `,
+    { returns: 'oneOrNone' },
+  );
+  console.log(user);
+} finally {
+  await db.close();
+}
 ```
 
-`${email}` becomes a bound parameter. Do not put quotes around it: the driver passes the value
-separately from the SQL text, even when the value contains SQL.
-
-The type describes one selected row. `returns: 'oneOrNone'` gives you that row or `undefined`; it
-throws if the query returns more than one row. Use `'one'` when absence should also be an error,
-`'many'` for an array, or omit `returns` for the native result object with `.rows`, `.rowCount`, and
-`.fields`. These options do not change your SQL.
-
-Generated table types can help describe projections without repeating every field:
-
-```ts
-import type { Rows } from './generated-schema.js';
-type UserSummary = Pick<Rows['users'], 'id' | 'email'>;
-```
+This inserts a new email; rerunning it with the same email will fail the unique constraint. Run it
+through your project's TypeScript runner or compile it with your application. The `.js` imports
+follow NodeNext's convention for compiled TypeScript modules.
 
-You can then write `sql<UserSummary>`. A table type describes the full table; a join or aggregate
-needs a type for what that particular query selects. An annotation is a declaration, not proof that
-the SQL matches. Without one, result fields are `unknown`. Use the inspection and database-test
-workflow below to check the declared contract against the query's behavior.
+The template binds `${email}` separately from the SQL. Do not put quotes around an interpolation.
+`sql<{ id: number }>` describes one returned row; `RETURNING id` asks PostgreSQL to return it.
+`returns: 'one'` requires exactly one row, while `oneOrNone` permits absence as `undefined`.
+Use `many` for an array or omit `returns` for the driver result with `.rows` and `.rowCount`.
 
-Keep storage types conservative and validate them when projecting domain values. The
-[persistence-to-domain example](reference.md#persistence-and-domain-values) shows validated JSON
-writes and reads, finite dates, and an SQL predicate that rejects infinite session expiry.
+The annotation does not check SQL or validate the row at runtime. Without an annotation, field values
+are `unknown`. Bigint columns and SQL `count(*)` return strings by default; the integer ID in this
+example returns a number. See [value mappings](schema-types.md#value-mappings) for JSON, arrays,
+decimals, and dates.
 
-## Reuse a query with a function
+## Reuse a query
 
-Use an ordinary function when several callers need the same query. An object input makes the
-supplied values easy to read:
+Put reusable queries in ordinary functions. Accept `QueryClient` when a caller may supply either
+`db` or a transaction's `tx`:
 
 ```ts
+// src/queries.ts
 import { sql, type QueryClient } from '@kequtech/tenon-orm';
-import type { Rows } from './generated-schema.js';
 
-type UserSummary = Pick<Rows['users'], 'id' | 'email'>;
+type UserSummary = { id: number; email: string };
 
 export function userByEmail(client: QueryClient, { email }: { email: string }) {
   return client.query(sql<UserSummary>`SELECT id, email FROM public.users WHERE email = ${email}`, {
@@ -136,12 +131,15 @@ export function userByEmail(client: QueryClient, { email }: { email: string }) {
 }
 ```
 
-Call `userByEmail(db, { email: 'reader@example.test' })` at the call site, or pass a transaction's
-`tx`. Keep the SQL and execution options together in the function.
+Call `userByEmail(db, { email: 'reader@example.test' })`. Keep the SQL and result options together in
+the function. Joins and aggregates work the same way; see [writing queries](sql.md).
 
-## Change data in a transaction
+## Use a transaction and a write guard
+
+In code using the shared `db`:
 
 ```ts
+import { sql } from '@kequtech/tenon-orm';
 import { userByEmail } from './queries.js';
 
 await db.transaction(async (tx) => {
@@ -154,97 +152,86 @@ await db.transaction(async (tx) => {
 });
 ```
 
-Use `tx` for all work in the callback and await it. Tenon commits when the callback succeeds and
-rolls back when an error escapes.
-
-`affectedRows: { exact: 1 }` requires exactly one affected row. `{ max: 1 }` permits zero or one.
-Tenon rolls back a write that violates the bound. You can also use these guards on `db.query`
-outside a callback transaction. The guard checks a count; you still need the correct `WHERE` clause
-and application authorization. Checking returned rows with `returns` alone does not undo an
-autocommitted write.
+Run all participating queries through `tx` and await them. The callback commits on success and rolls
+back when an error escapes.
 
-## Write objects without building column lists by hand
+The guard requires exactly one affected row and rolls back the update if the count differs. Use
+`{ max: 1 }` to allow zero or one. Guards also work with `db.query` outside a callback. Result modes
+such as `one` check returned rows; by themselves they do not undo an autocommitted write.
 
-Use `sql.insertValues(rowOrRows)` inside an authored `INSERT` and `sql.assignments(patch)` after
-`SET`. Inserts use database defaults for missing/undefined fields; updates omit those fields.
-Explicit `null` stays null. Empty batches and updates throw. Heterogeneous insert rows remain
-aligned, with `DEFAULT` for missing cells.
+You still author the `WHERE` clause and application authorization. The
+[transaction reference](reference.md#transactions-guarded-writes-and-errors) covers caught errors,
+savepoints, and retry limits. For object-based inserts and patches, see [writing data](mutations.md).
 
-```ts
-import type { Rows } from './generated-schema.js';
+## Generate types
 
-const input: Rows.Insert['users'] = { email: 'another@example.test' };
-await db.query(sql`INSERT INTO public.users ${sql.insertValues(input, ['email'])}`);
+Generation is optional. It reads the migrated database and writes table read/insert/update types.
+Run it through the configured instance so the types use the same value conversions as your queries:
 
-const patch: Pick<Rows.Update['users'], 'active'> = { active: false };
-await db.query(sql`UPDATE public.users SET ${sql.assignments(patch, ['active'])}
-  WHERE email = ${input.email}`);
+```ts
+// scripts/generate-types.ts
+import { parseArgs } from 'node:util';
+import { db } from '../src/db.ts';
+
+const { values } = parseArgs({ options: { check: { type: 'boolean', default: false } } });
+try {
+  await db.generateTypes({ out: 'src/generated-schema.d.ts', check: values.check });
+} finally {
+  await db.close();
+}
 ```
 
-Use `Pick` to limit editable fields. Keep expressions, predicates, conflict handling, and
-`RETURNING` in SQL. Selected columns choose fields; they do not validate the complete object. See
-[input types](reference.md#insert-and-update-inputs) for mappings and the
-[mutation cookbook](mutations.md) for defaults, batches, and upserts.
+This script runs directly on Node.js 24+ using its TypeScript support. For this execution path,
+`src/db.ts` must be runnable without build-time transformations or extension rewriting. If your
+project compiles scripts, use its normal runner and import conventions instead. To typecheck `.ts`
+imports with `noEmit`, enable `allowImportingTsExtensions`; emitting projects can use
+`rewriteRelativeImportExtensions`.
 
-## Check your SQL and result types
+Add these scripts to your existing `package.json`:
 
-Use `describeQuery({ connection, query })` on completed SQL, or `npx tenon describe query.sql` on a
-file. It reports PostgreSQL metadata and a conservative `rowType` declaration; it does not update or
-verify the application's `sql<Row>` annotation. Inspection uses a separate session, so match the
-runtime role and settings. See [query inspection](query-inspection.md) for a worked test.
-
-After migrating the development/test database, inspect meaningful query variants, run TypeScript,
-and test actual rows and effects. Keep those checks in CI. Execution assertions cover values, nulls,
-JSON, authorization, and rollback that description alone cannot establish.
-
-## Reuse or assemble parts of SQL when needed
+```json
+{
+  "scripts": {
+    "db:generate": "node scripts/generate-types.ts",
+    "db:check-types": "npm run db:generate -- --check",
+    "db:migrate": "tenon migrate && npm run db:generate"
+  }
+}
+```
 
-A fragment is a piece of SQL that carries its bound values with it. It is useful when multiple
-queries share a condition. For example, a search can use the same predicate for its result page and
-its count:
+Run `npm run db:generate` now. The output exports a `Database` type:
 
 ```ts
-const pattern = '%example%';
-const limit = 20;
-const search = sql.fragment`active = ${true} AND email ILIKE ${pattern}`;
-const page = sql<UserSummary>`SELECT id, email FROM public.users
-  WHERE ${search} ORDER BY id LIMIT ${limit}`;
-const count = sql<{ count: string }>`SELECT count(*) AS count FROM public.users
-  WHERE ${search}`;
-```
+import type { Database } from './generated-schema.js';
 
-Execute each query with `db.query`. Validate application-supplied patterns and page sizes before
-building queries. The complete query supplies the result type.
+type UserRow = Database['users']['row'];
+type UserSummary = Pick<UserRow, 'id' | 'email'>;
+type UserInsert = Database['users']['insert'];
+type UserUpdate = Database['users']['update'];
+```
 
-Use `sql.join` when the number of fragments varies, such as one search condition per word or a set
-of optional filters. It combines fragments with `'AND'`, `'OR'`, or `','` and preserves their bound
-values:
+Use `UserSummary` with `sql<UserSummary>` or `UserInsert` to type an insert input. These types
+follow table columns, defaults, nullability, and codec representations. They do not validate
+application inputs. Keep aliases in handwritten files and leave generated output unchanged.
+See [schema types](schema-types.md) for an output example and mapping details.
 
-```ts
-const terms = ['reader', 'example'];
-const filters = [sql.fragment`active = ${true}`];
-for (const term of terms) {
-  filters.push(sql.fragment`email ILIKE ${'%' + term + '%'}`);
-}
+## Keep schema and code in step
 
-const query = sql<UserSummary>`SELECT id, email FROM public.users
-  WHERE ${sql.join(filters, 'AND')} ORDER BY id`;
-```
+During development, run `npm run db:migrate`, review the generated diff, and commit migrations and
+generated declarations together. Exclude the generated file from automatic formatting.
 
-This matches active users whose email contains every term. With no terms, the active condition still
-applies. For an entirely optional set of filters, omit the whole `WHERE` clause when the list is
-empty:
+In CI, migrate a dedicated test database and check committed output without rewriting it:
 
-```ts
-const optionalFilters = terms.map((term) => sql.fragment`email ILIKE ${'%' + term + '%'}`);
-const unfilteredOrMatching = optionalFilters.length
-  ? sql<UserSummary>`SELECT id, email FROM public.users WHERE ${sql.join(optionalFilters, 'AND')} ORDER BY id`
-  : sql<UserSummary>`SELECT id, email FROM public.users ORDER BY id`;
+```sh
+npx tenon migrate
+npm run db:check-types
+npx tsc --noEmit
 ```
 
-`sql.join` and fragments require nonempty input. Keep mandatory authorization predicates in the
-initial list when applicable; an empty optional-filter list must not remove them. Use parentheses
-when mixing `AND` and `OR`. See the [composition reference](reference.md#sql-fragments-and-joins).
+Do not run the regenerating `db:migrate` script before the freshness check: it would replace the file
+you meant to verify. Production runs migrations and deploys the already-built application.
 
-Use fixed SQL when composition is unnecessary. See the [reference](reference.md) for arrays,
-identifiers, value mappings, CLI options, and error behavior.
+Optionally run `npx tenon check src/queries.ts` against the migrated database to compare source query
+types with PostgreSQL metadata. Keep database tests for rows, permissions, and rollback. See
+[choosing a validation tool](reference.md#choosing-a-validation-tool) and
+[troubleshooting](reference.md#troubleshooting) as needed.

package/docs/migrations.md

@@ -0,0 +1,133 @@
+# SQL migrations
+
+Tenon's optional runner applies numbered SQL files and records their checksums in the database. It
+can be used independently of query execution and type generation. Keep an existing runner unless
+you intend to [transfer migration ownership](adoption-guide.md#transfer-migration-ownership).
+
+## Create and apply a change
+
+```sh
+npx tenon new add_user_display_name
+```
+
+`new` works offline. It creates the next numeric pair in `migrations/`, starting at `0001` in an empty
+directory. For a directory whose latest migration is `0001`:
+
+```sql
+-- migrations/0002_add_user_display_name.up.sql
+ALTER TABLE public.users ADD COLUMN display_name text;
+```
+
+```sql
+-- migrations/0002_add_user_display_name.down.sql
+ALTER TABLE public.users DROP COLUMN display_name;
+```
+
+Set `DATABASE_URL` through your environment, then run:
+
+```sh
+npx tenon status
+npx tenon migrate
+```
+
+`status` prints JSON with `applied` and `pending` ID arrays. `migrate` applies every pending up file
+in numeric order and reports completed IDs. No pending files means nothing changes. No command
+loads `.env` automatically.
+
+After migrating a development database, [generate types](schema-types.md#generate-and-check-a-file),
+review the diff, and commit migrations with generated declarations. Production applies migrations
+and deploys the already-built application. Use a dedicated migrated database for tests.
+
+## File rules and history
+
+Files use `<numeric-id>_<name>.up.sql` and optional matching `.down.sql`. Names contain letters,
+digits, underscores, or hyphens. Each ID has one name and an up file. SQL files with invalid names,
+empty contents, duplicate numeric IDs, or down files without an up file fail validation.
+
+The runner records ID, name, checksum, and application time in `<namespace>.__tenon_migrations`.
+Checksums cover both up SQL and the presence/content of down SQL. Never edit, rename, remove, or
+add a down file to an already-applied migration. Add a new forward migration instead. History must
+match the beginning of the ordered file list; inserting an older migration also causes a conflict.
+
+Migrations may contain multiple statements. Tenon owns the transaction, so authored files must not
+contain transaction-control statements. Use dollar-quoted bodies for functions and explicit `E'...'`
+for backslash escapes. A scaffold's files are empty until you fill them; both are checked even if
+only the up direction is about to run.
+
+## Transaction and failure behavior
+
+Each migration file and its history entry commit together. The whole run is not one transaction:
+if file three fails, files one and two may already have committed. Correct the cause, inspect
+`status`, and rerun. The failed file is rolled back and remains pending. Restore original applied
+files if history validation reports a conflict.
+
+SQL that cannot run in a transaction, such as `CREATE INDEX CONCURRENTLY`, needs a separate,
+application-owned procedure. Tenon has no nontransactional migration flag. It also does not infer
+schema changes or generate migration SQL from TypeScript.
+
+## Roll back
+
+```sh
+npx tenon rollback
+npx tenon rollback --steps 2
+```
+
+Rollback runs the most recent down files in reverse order and removes their history records. It
+prechecks that every selected migration has down SQL before executing any. `steps` must be a
+positive safe integer; requesting more steps than exist selects all applied migrations.
+
+For an irreversible change, omit the down file before first application. The missing file prevents
+rollback across that migration. A down migration is not a backup: dropped data and sequence changes
+may not be recoverable. Use the application's backup and recovery procedure when needed.
+
+## Directory, schema, and locking
+
+Defaults are `--directory migrations` and `--namespace public`. For a custom schema, use the same
+options consistently:
+
+```sh
+npx tenon new add_setting --directory db/migrations
+npx tenon migrate --directory db/migrations --namespace app
+npx tenon status --directory db/migrations --namespace app
+npx tenon rollback --directory db/migrations --namespace app --steps 1
+```
+
+The namespace selects history location, migration search path, and lock scope. During a file,
+`search_path` is `<namespace>, pg_catalog`; qualify cross-schema names explicitly. The runner creates
+the namespace when applying pending migrations. It does not configure runtime application queries.
+Pass the matching namespace to type generation when generating that schema.
+
+Migration and adoption operations use an exclusive session advisory lock for this database and
+namespace. Status and generation use a shared lock. Different namespaces have independent lock
+scopes. These locks do not coordinate with a legacy runner or arbitrary DDL.
+
+Use direct or session-pooled connections for these tools; transaction pooling does not preserve
+their session locks. Configure server timeouts on the tooling connection to bound waits. Migration
+commands do not accept Describe's timeout flags.
+
+Offline file creation uses `.tenon-create.lock` in the directory. Remove a stale lock only after
+confirming that the process which created it has stopped. Baseline capture uses this local lock
+and `pg_dump`; it does not take the runner's database advisory lock.
+
+## Existing databases
+
+An existing schema needs no baseline merely to execute queries or generate types. A baseline is
+needed when Tenon takes over migration history: capture one reviewed schema at the old runner's
+final checkpoint, record it on existing databases without execution, and execute it on fresh ones.
+Follow [the adoption workflow](adoption-guide.md#transfer-migration-ownership).
+
+## Programmatic API
+
+```ts
+import { migrate, migrationStatus, rollback } from '@kequtech/tenon-orm';
+
+const options = { connection, directory: 'migrations', namespace: 'public' };
+const before = await migrationStatus(options); // { applied, pending }
+const applied = await migrate(options); // readonly string[] of completed IDs
+const reverted = await rollback({ ...options, steps: 1 });
+```
+
+`connection` accepts the same URI/options/pool forms as `createTenon`. A supplied pool retains
+application ownership. `createMigration({ name, directory? })` returns `{ id, upPath, downPath }`;
+`readMigrations(directory?)` returns validated `{ id, name, up, down?, checksum }` records.
+See [the command reference](reference.md#command-reference) for all CLI forms.