@intentius/chant-lexicon-sql 0.0.0-stage → 0.112.1

package/src/skills/chant-sql-postgres.md

@@ -0,0 +1,165 @@
+---
+skill: chant-sql-postgres
+description: Declare Postgres schemas, tables, constraints, indexes, views, sequences, enum and domain types and extensions as SQL-shaped tagged templates, with references, lineage and the SQLPG lint rules
+user-invocable: true
+---
+# Declaring a Postgres schema with chant
+
+Use this skill when a project holds Postgres DDL. The sql lexicon's Postgres dialect is spec-true: the statements are Postgres's own, parsed at build time without running them, and names fold and quote as Postgres folds and quotes them. It is not a database-agnostic model. A project holds one dialect; the ClickHouse dialect has its own skills (`chant-sql` and the two after it) and none of them applies here.
+
+Import the tags from the `/postgres` subpath. Each template is one `CREATE` statement, optionally followed by `COMMENT ON` statements for the same object.
+
+```ts
+import { schema, table } from "@intentius/chant-lexicon-sql/postgres";
+
+export const app = schema`CREATE SCHEMA app`;
+
+export const users = table`
+  CREATE TABLE ${app}.users (
+    id         bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+    email      text NOT NULL UNIQUE,
+    created_at timestamptz NOT NULL DEFAULT now()
+  );
+  COMMENT ON TABLE ${app}.users IS 'One row per account'`;
+```
+
+The export name (`users`) is the object's identity in chant. The name in the SQL (`app.users`) is its name in the database. An unquoted name folds to lower case, so `CREATE TABLE Users` is `users`, and `.columns` is keyed by the folded name.
+
+## The tags
+
+| Tag | Statement | Entity type |
+|---|---|---|
+| `schema` | `CREATE SCHEMA` | `Postgres::Schema` |
+| `table` | `CREATE TABLE` | `Postgres::Table` |
+| `index` | `CREATE INDEX` | `Postgres::Index` |
+| `view` | `CREATE VIEW`, `CREATE MATERIALIZED VIEW` | `Postgres::View`, `Postgres::MaterializedView` |
+| `sequence` | `CREATE SEQUENCE` | `Postgres::Sequence` |
+| `type` | `CREATE TYPE ... AS ENUM` | `Postgres::Enum` |
+| `domain` | `CREATE DOMAIN` | `Postgres::Domain` |
+| `extension` | `CREATE EXTENSION` | `Postgres::Extension` |
+
+`table` and `view` imported from the package root are ClickHouse's. Import every tag from `@intentius/chant-lexicon-sql/postgres`.
+
+## Interpolations
+
+| Value | Means | Renders as |
+|---|---|---|
+| a Postgres entity | a reference to that object | its schema-qualified name; a schema renders as its name |
+| `entity.columns.<name>` | a reference to one column | the column's name |
+| a sequence inside `nextval(...)`, `currval(...)`, `setval(...)`, or any object before `::regclass` | a reference to the object | a `regclass` literal |
+| a string | SQL text, spliced in before the statement is parsed | itself |
+| a number, bigint, boolean or `null` | a literal | `42`, `true`, `NULL` |
+| `literal(value)` | a string literal | `'quoted and escaped'` |
+
+Reach columns through `.columns` and only there. `${users.kind}` is the entity's own field (the string `"resource"`) and `${users.id}` is `undefined`; SQLPG002 flags the first. In a join, qualify each reference: `u.${users.columns.id}`.
+
+A plain string is SQL text. When a string is a value, wrap it in `literal()`: `DEFAULT ${literal(plan)}` writes `DEFAULT 'free'`, where `DEFAULT ${plan}` writes `DEFAULT free`, a reference to a column or function of that name. A single quote inside is doubled and a backslash is left alone, as `standard_conforming_strings` reads it.
+
+## Tables and constraints
+
+A table's props hold its columns (type, `NOT NULL`, default, generated and identity, collation, storage, comment) and every constraint kind: primary key, unique (including `NULLS NOT DISTINCT` and `INCLUDE`), check, foreign key, exclusion. Column constraints and table constraints are filed together. Name a constraint (`CONSTRAINT orders_amount_ck CHECK (...)`) when a later migration must refer to it, because a generated name is not stable.
+
+```ts
+export const orders = table`
+  CREATE TABLE ${app}.orders (
+    id         bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+    user_id    bigint NOT NULL REFERENCES ${users} (${users.columns.id}) ON DELETE CASCADE,
+    amount     numeric(12, 2) NOT NULL,
+    placed_at  timestamptz NOT NULL DEFAULT now(),
+    CONSTRAINT orders_amount_ck CHECK (amount >= 0)
+  )`;
+```
+
+Partitioning (`PARTITION BY`, `PARTITION OF ... FOR VALUES`), `INHERITS`, `LIKE`, `OF type`, storage parameters and `USING` are parsed too. Each partition is its own `table` template that names its parent as an entity. A primary key or unique constraint on a partitioned table must include every partition key column.
+
+## Indexes
+
+An index needs a name, unqualified (it lives in its table's schema), because the name is its identity on the server. SQLPG001 reports an unnamed one.
+
+```ts
+import { index } from "@intentius/chant-lexicon-sql/postgres";
+
+export const ordersUser = index`
+  CREATE INDEX orders_user_id_idx ON ${orders} (${orders.columns.user_id}, ${orders.columns.placed_at} DESC)`;
+```
+
+The index records `CONCURRENTLY`, its method, each element (with the column when the element is one), `INCLUDE` and `WHERE`. An expression element such as `lower(${users.columns.email})` is recorded as an expression.
+
+## Views
+
+```ts
+import { view } from "@intentius/chant-lexicon-sql/postgres";
+
+export const orderTotals = view`
+  CREATE VIEW ${app}.order_totals AS
+  SELECT u.${users.columns.id} AS user_id, count(o.${orders.columns.id}) AS order_count
+  FROM ${users} u
+  LEFT JOIN ${orders} o ON o.${orders.columns.user_id} = u.${users.columns.id}
+  GROUP BY u.${users.columns.id}`;
+```
+
+A view's lineage is recorded per output column of its top-level select list, from the column references in it. Write `${users.columns.id}` rather than bare `id` where you want that edge. A column named inside SQL text is not a reference. An unaliased item takes Postgres's own name (`count(x)` is `count`, `x::text` is `x`, anything else `?column?`), so alias what you want to name. A `CREATE MATERIALIZED VIEW` goes in a `view` template as well.
+
+## Sequences
+
+```ts
+import { sequence } from "@intentius/chant-lexicon-sql/postgres";
+
+export const invoiceSeq = sequence`CREATE SEQUENCE ${app}.invoice_seq AS bigint START WITH 1000`;
+
+// in a table: invoice_no bigint NOT NULL DEFAULT nextval(${invoiceSeq})
+```
+
+`nextval(${invoiceSeq})` renders as `nextval('app.invoice_seq'::regclass)`, which is how the catalog prints the default, so the declaration and the live database read the same, and the table depends on the sequence. A name typed in a string (`nextval('app.invoice_seq')`) records no edge and the create order can put the table first; SQLPG003 warns about it. A sequence `OWNED BY` a column whose default calls `nextval` on that sequence is a reference cycle, and the build's cycle error names it. Use an identity column for that case.
+
+## Enum and domain types, extensions
+
+```ts
+import { domain, extension, type } from "@intentius/chant-lexicon-sql/postgres";
+
+export const status = type`CREATE TYPE ${app}.order_status AS ENUM ('placed', 'paid', 'cancelled')`;
+export const email = domain`CREATE DOMAIN ${app}.email AS text CONSTRAINT email_shape CHECK (VALUE ~ '^[^@]+@')`;
+export const citext = extension`CREATE EXTENSION IF NOT EXISTS citext WITH SCHEMA ${app}`;
+```
+
+Use the entity as a column type: `status ${status} NOT NULL DEFAULT 'placed'`. The table then depends on the type. Enum labels are values of the type, so a label cannot be removed or reordered later without a rebuild; add labels at the end. An extension is an object of its own with no references to it from a column type, so a table using a type it provides does not wait for it; keep extensions in a project the database already has, or apply them first.
+
+## Comments
+
+Postgres has no inline comment clause. Follow the `CREATE` with `COMMENT ON` for the object, its columns and its named constraints, separated by `;`:
+
+```ts
+export const users = table`
+  CREATE TABLE ${app}.users (id bigint PRIMARY KEY, email text NOT NULL);
+  COMMENT ON TABLE ${app}.users IS 'One row per account';
+  COMMENT ON COLUMN ${app}.users.email IS 'Unique; lower-cased by the app'`;
+```
+
+The comments fold into the entity's `comment`, each column's and each constraint's, and stay in the DDL. A template that starts with `COMMENT ON` is refused.
+
+## What the build records
+
+References become dependency edges: a table after its schema, its enum and domain types, its sequences and the tables it references; an index after its table; a view after what it reads. The build lists objects in that order as `applyOrder` and writes every statement, in that order, to `postgres.sql` next to the JSON document. A reference cycle is an error naming the cycle (a pair of tables that reference each other needs one foreign key added after both exist, which is a migration, not a declaration).
+
+## Checks
+
+| Id | Severity | What it flags |
+|---|---|---|
+| SQLPG001 | error | the DDL does not parse, the template holds another statement than its tag or a second `CREATE`, or an index has no name |
+| SQLPG002 | error | `${users.kind}` where `${users.columns.kind}` was meant |
+| SQLPG003 | warning | an object named in a regclass string instead of interpolated |
+
+SQLPG001 is reported at the token, as a line and column of the `.ts` file, by `chant lint` and in the editor. An error inside text a string interpolation supplies is left to the build.
+
+## Commands
+
+```bash
+chant lint src
+chant build src --lexicon sql -o dist/schema.json
+```
+
+`chant init --lexicon sql --template postgres` scaffolds a default app schema: a schema, two tables with a foreign key, an index and a view. `--template postgres-tenant` adds a multi-tenant schema whose keys and indexes lead with the tenant column, and `--template postgres-events` an events table partitioned by month.
+
+## Not covered
+
+`ALTER`, `CREATE POLICY`, `GRANT`, functions, triggers, rules and the composite and range forms of `CREATE TYPE` are not declared with the tags. Planning a schema change (`chant sql diff`, `chant sql plan`, the lock classes) is the `chant-sql-postgres-plan` skill. Apply and the expand-and-contract migration Op have their own skills, which arrive with the code they describe.

package/src/skills/chant-sql-rebuild.md

@@ -0,0 +1,68 @@
+---
+skill: chant-sql-rebuild
+description: Run a ClickHouse table rebuild (a sorting key, primary key, partition key, engine or key column type change) with ClickHouseRebuildOp, a gated Op with backfill receipts, verification, an exchange swap and onFailure cleanup
+user-invocable: true
+---
+# Running a ClickHouse rebuild migration
+
+Use this skill when `chant sql plan` or `chant sql diff` exits 2 because a table's change is a rebuild: a sorting key rename or reorder (SQLCH220), a primary key change (SQLCH221), a partition key change (SQLCH222), an engine or engine argument change such as a ReplacingMergeTree version column (SQLCH223), or a key column's type (SQLCH211). ClickHouse has no ALTER for these. The data is copied into a new table, and the old one is swapped out.
+
+`ClickHouseRebuildOp` rebuilds tables. A view's or materialized view's rebuild-class change (SQLCH224, SQLCH242, SQLCH243) is a drop and a create of that view, not this Op.
+
+## Declare the Op
+
+The plan names the Op for each refused table, with the table filled in and a dual-write mode suggested. Copy it into an `*.op.ts` file in the same pull request as the schema change:
+
+```typescript
+// ops/rebuild-events.op.ts
+import { ClickHouseRebuildOp } from "@intentius/chant-lexicon-sql/clickhouse";
+
+export const { op } = ClickHouseRebuildOp({
+  name: "rebuild-shop-events",
+  env: "prod",                       // sql.profiles.prod
+  table: "shop.events",              // database.name on the server
+  dualWrite: { mode: "materialized-view", cutoverColumn: "ts" },
+  retain: "7d",                      // how long the old table is kept
+});
+```
+
+Options: `name`, `env`, `table`, `dualWrite` (required); `output` (default `dist/schema.json`), `path`, `build` (default true runs `chant build` first), `retain` (default `7d`), `gate` (the swap gate: `gate`, `timeout`, `description`, `approval`), `dropGate`, `writesGate` (app mode), `backfillTimeout` (default `6h`), `mutationTimeout` (default `10m`), `replicaTimeout` (default `2m`, Replicated databases only), `stack`, `ownershipEnv`.
+
+Pick the dual-write mode:
+
+- `{ mode: "materialized-view", cutoverColumn: "ts", cutoverDelay?: "1m" }` when the table has a time column rows arrive in order of. Writes keep flowing: a materialized view sends rows at or after the cut-over to the new table, the backfill copies the rows before it.
+- `{ mode: "app" }` otherwise. A gate, `<name>-writes-stopped`, waits for someone to confirm the application stopped writing to the table; writes resume after the swap.
+
+## Run it
+
+```bash
+chant run rebuild-shop-events        # new table, dual write, backfill, verify; exits 3 at approve-rebuild-shop-events
+chant run status rebuild-shop-events # the Verification, VerifiedPartitions and VerifiedRows outcomes
+chant approve rebuild-shop-events approve-rebuild-shop-events --actor <you>
+chant run rebuild-shop-events        # swaps, recreates the views that read the table; exits 3 at approve-rebuild-shop-events-drop
+chant approve rebuild-shop-events approve-rebuild-shop-events-drop --actor <you>
+chant run rebuild-shop-events        # drops the old table once retain has passed
+```
+
+Each run goes as far as the next gate. Every step reads the server again, so re-running after a gate, a crash or an approval carries on where the last run stopped.
+
+The swap gate is bound to a digest of the plan and the verification (row counts and checksums per partition). If the next run verifies different numbers, the gate asks again. The drop gate is bound to the old table's UUID and retention date; approving it early is fine, nothing is dropped before the date.
+
+## When something goes wrong
+
+- A run stopped during the backfill (Ctrl-C, a killed job) is not a failure. The next run resumes from the receipts in `chant_receipts.receipts` on the same server (in `<db>.__chant_receipts` on a Replicated database), and a partition whose copy was cut off before its receipt is cleared and copied again, so no row is copied twice.
+- A failed step (a failed copy after its retries, a verification mismatch, a refused declaration) runs onFailure: the new table `<t>__chant_new` and the dual-write view `<t>__chant_dual` are dropped, and the next run starts from a new table. The original table is untouched until the swap.
+- A verification mismatch in materialized-view mode usually means rows arrived with a time before the cut-over later than `cutoverDelay`. Raise the delay and run again.
+- The Op refuses a database that is neither Atomic nor Replicated (`EXCHANGE TABLES` needs one of them), a table whose change is not a rebuild (use the applier), and an object already under one of its working names that is not its own. In a Replicated database it also refuses a table, or a declaration, whose engine is not `Replicated*MergeTree`: a plain MergeTree keeps different rows on each replica and one replica's copy would lose the others'.
+
+## On a Replicated database
+
+The Op runs on a `Replicated` database (two replicas with Keeper are covered by the e2e test). Declare the tables `ENGINE = ReplicatedMergeTree` with no arguments, so each table, the new one included, gets its own Keeper path from its UUID; an explicit path without `{uuid}` is refused. `chant run` can reach any replica, and a different one on each run: the DDL reaches every replica through the database, the receipts live in `<db>.__chant_receipts` and replicate, each step that reads rows first waits for its replica to catch up (`SYSTEM SYNC REPLICA`), and a copy still running on another replica is killed with `KILL QUERY ON CLUSTER '<db>'`. A dependent view is detached `PERMANENTLY`, the only form the database takes, and attached again.
+
+A replica that is down does not stop the rebuild by itself. Every DDL statement runs with `distributed_ddl_output_mode = throw_only_active`: it goes on with the live replicas, and the down one runs the statements from the database's log in Keeper when it comes back, in order, then fetches the rows. Right after a replica goes down, Keeper still counts it active until its session expires (about 30s), so the next statement waits that long once. A step that reads rows stops only when the down replica holds parts that no live replica has fetched: it waits `replicaTimeout` per attempt for them, then fails with a message naming the replica and the parts, and onFailure drops the new table. Bring that replica back and run again. Point `chant run` at a live replica, and keep Keeper itself up: without Keeper nothing replicates.
+
+## Checking it is done
+
+`chant sql plan <env> dist/schema.json` reports no changes for the table, `<t>__chant_old` is gone after the drop, and a later `chant run` of the Op reports `RebuildState` `done`. Then remove the `*.op.ts` file.
+
+The rebuild's working tables carry chant's comment marker with `rebuild=<db.t>` and a role, and plans, imports and prunes leave them out, so an `ApplyOp` for the rest of the schema can run beside a rebuild.

package/src/skills/chant-sql.md

@@ -0,0 +1,123 @@
+---
+skill: chant-sql
+description: Declare ClickHouse databases, tables, views and materialized views as SQL-shaped tagged templates, with references, lineage and the checks that run on them
+user-invocable: true
+---
+# Declaring a ClickHouse schema with chant
+
+Use this skill when a project holds ClickHouse DDL: a database, tables, views, materialized views. The sql lexicon is spec-true to ClickHouse at its pinned server release. It is not a database-agnostic model: engine names, column types, codecs and settings are the pinned server's own, and a name the server does not have fails at build.
+
+An object is declared as its own DDL inside a `database`, `table` or `view` tagged template from the dialect subpath. `chant build` parses the template when it folds the file, without running it, and turns it into one entity.
+
+```ts
+import { database, table, view } from "@intentius/chant-lexicon-sql/clickhouse";
+
+export const analytics = database`CREATE DATABASE analytics ENGINE = Atomic`;
+
+export const events = table`
+  CREATE TABLE ${analytics}.events (
+    user_id  UUID,
+    kind     LowCardinality(String),
+    ts       DateTime CODEC(Delta, ZSTD(3))
+  )
+  ENGINE = MergeTree
+  PARTITION BY toYYYYMM(ts)
+  ORDER BY (user_id, kind, ts)`;
+
+export const byKind = view`
+  CREATE VIEW ${analytics}.by_kind AS
+  SELECT ${events.columns.kind} AS kind, count() AS n
+  FROM ${events}
+  GROUP BY kind`;
+```
+
+The export name (`events`) is the object's identity in chant. The name in the SQL (`analytics.events`) is its name in the database. A materialized view is a `view` template holding `CREATE MATERIALIZED VIEW`.
+
+## Interpolations
+
+| Value | Means | Renders as |
+|---|---|---|
+| a `database`, `table` or `view` entity | a reference to that object | its name, database-qualified when the DDL qualifies it |
+| `entity.columns.<name>` | a reference to one column | the column's name |
+| a string | SQL text, spliced in before the statement is parsed | itself |
+| a number, bigint, boolean or `null` | a literal | `42`, `true`, `NULL` |
+| `literal(value)` | a string literal | `'quoted and escaped'` |
+
+Reach columns through `.columns` and only there. `${events.kind}` is the entity's own field (the string `"resource"`), and `${events.user_id}` is `undefined`, which the tag refuses.
+
+A plain string is SQL text, so a composite can supply a type, an engine or an expression. When a string is a value, wrap it in `literal()`: `DEFAULT ${literal(plan)}` writes `DEFAULT 'free'`, where `DEFAULT ${plan}` writes `DEFAULT free`, a column called `free`.
+
+## What the build records
+
+References become dependency edges. A view is created after the tables it reads, a materialized view after its `TO` target, a table after its database. The build output lists objects in that order as `applyOrder` and writes every statement, in the same order, to `clickhouse.sql` next to the JSON document.
+
+A view's lineage is recorded per output column of its top-level select list, from the column references in it. Write `${events.columns.kind}` rather than bare `kind` where you want that edge recorded; a column named inside the SQL text is not a reference.
+
+## Materialized views
+
+A rollup is a target table plus a materialized view writing to it with `TO`:
+
+```ts
+export const daily = table`
+  CREATE TABLE ${analytics}.daily (
+    day    Date,
+    kind   LowCardinality(String),
+    users  AggregateFunction(uniq, UUID)
+  )
+  ENGINE = AggregatingMergeTree
+  ORDER BY (day, kind)`;
+
+export const dailyMv = view`
+  CREATE MATERIALIZED VIEW ${analytics}.daily_mv TO ${daily} AS
+  SELECT toDate(${events.columns.ts}) AS day, ${events.columns.kind} AS kind, uniqState(${events.columns.user_id}) AS users
+  FROM ${events}
+  GROUP BY day, kind`;
+```
+
+A materialized view's `TO` target is fixed when the view is created. Changing it later is a rebuild, see the `chant-sql-plan` skill.
+
+## Composites
+
+`@intentius/chant-lexicon-sql/clickhouse` exports five composites for tables that are usually declared the same way. Each is built from the `table` and `view` tags, and each prop is SQL text spliced into the template:
+
+| Composite | Members | What it is |
+|---|---|---|
+| `ReplacingTable` | `table` | ReplacingMergeTree with a version column (`version`, UInt64 by default) |
+| `EventsTable` | `table` | MergeTree partitioned by its timestamp (`ts`), with a TTL (`ttlDays`, default 90) and `ttl_only_drop_parts` |
+| `RollupView` | `table`, `view` | A target table (SummingMergeTree by default) and a materialized view writing to it from `source` |
+| `CdcMirror` | `table`, `current` | ReplacingMergeTree(`_version`, `_is_deleted`) for a CDC feed, and a view of its live rows |
+| `ShardedTable` | `local`, `distributed` | ReplicatedMergeTree `ON CLUSTER` and the Distributed table over it |
+
+```ts
+import { EventsTable, RollupView } from "@intentius/chant-lexicon-sql/clickhouse";
+
+export const events = EventsTable({ name: "events", columns: "user_id UUID, kind LowCardinality(String)", orderBy: "(kind, user_id, ts)", ttlDays: 30 });
+```
+
+Put a composite whose props reference another call's member (`source: events.table`) in its own file; a same-file reference to a composite call's member makes the file run instead of fold. Alias each item of a `RollupView` select list to a target column, or SQLCH110 reports it.
+
+When chant interprets a composite's factory at build time, each field records the parameter it came from, so drift on `ttl` is reported as a change to `ttlDays`. Chant interprets a factory only from a project file, or from a package path a tsconfig `paths` entry maps to source, and only from the module that defines it. A composite imported from the installed package records its fields as `unknown`.
+
+## Checks
+
+| Id | When | What it flags |
+|---|---|---|
+| SQLCH001 | lint | the DDL does not parse, or the tag holds a different statement |
+| SQLCH002 | lint | a `Nullable` column in the sort key or primary key, which ClickHouse refuses without `allow_nullable_key` |
+| SQLCH003 | lint | `${events.kind}` where `${events.columns.kind}` was meant |
+| SQLCH101 | build | an engine the pinned ClickHouse server does not have |
+
+## Commands
+
+```bash
+chant lint src
+chant build src --lexicon sql -o dist/schema.json
+```
+
+`chant init --lexicon sql` scaffolds a project. `--template events` adds an events table with a TTL and a rollup, and `--template cdc` a ReplacingMergeTree mirror table.
+
+## Not covered
+
+`CREATE TABLE ... AS` and `CREATE TABLE ... AS SELECT` are not declared with the tags. Dictionaries, functions, named collections and access control are not modelled. A second dialect is not supported.
+
+Note that `chant migrate` does not run schema migrations. Schema changes go through `chant sql plan`, apply and the rebuild Op, described in the `chant-sql-plan` and `chant-sql-rebuild` skills.

package/src/spec/catalog.e2e.test.ts

@@ -0,0 +1,29 @@
+/**
+ * The committed snapshot is what the pinned server reports, byte for byte.
+ *
+ * Opt-in: it pulls the ~850 MB server image. Run it on a pin move, or when the
+ * snapshot is edited by hand, with Docker available:
+ *
+ *     CHANT_SQL_LIVE=1 npx vitest run lexicons/sql/src/spec/catalog.e2e.test.ts
+ */
+import { readFileSync } from "node:fs";
+import { afterAll, describe, expect, test } from "vitest";
+import { dockerAvailable, startScratchServer, type ScratchServer } from "../clickhouse/container";
+import { catalogFromServer, SNAPSHOT_FILE } from "./fetch";
+import { stringifyCatalog } from "./catalog";
+import { clickhouseImage } from "./pin";
+
+const enabled = process.env.CHANT_SQL_LIVE === "1" && (await dockerAvailable());
+let server: ScratchServer | undefined;
+
+afterAll(async () => {
+  await server?.stop();
+});
+
+describe.skipIf(!enabled)("the snapshot against the pinned server", () => {
+  test("reads back byte-identical", async () => {
+    server = await startScratchServer(clickhouseImage(), { namePrefix: "chant-sql-catalog-e2e" });
+    const fresh = await catalogFromServer(server.endpoint, clickhouseImage());
+    expect(stringifyCatalog(fresh)).toBe(readFileSync(SNAPSHOT_FILE, "utf-8"));
+  }, 600_000);
+});

package/src/spec/catalog.test.ts

@@ -0,0 +1,56 @@
+import { readFileSync } from "node:fs";
+import { describe, expect, test } from "vitest";
+import { CATALOG_SECTIONS, normalizeDefault, parseCatalog, stringifyCatalog, summaryOf } from "./catalog";
+import { SNAPSHOT_FILE } from "./fetch";
+import { CLICKHOUSE_VERSION, clickhouseImage, versionFromReleaseTag } from "./pin";
+
+const snapshotText = readFileSync(SNAPSHOT_FILE, "utf-8");
+
+describe("the committed ClickHouse catalog snapshot", () => {
+  const catalog = parseCatalog(snapshotText);
+
+  test("is at the pin, read from the pinned image", () => {
+    expect(catalog.version).toBe(CLICKHOUSE_VERSION);
+    expect(catalog.image).toBe(clickhouseImage());
+  });
+
+  test("is written exactly as stringifyCatalog writes it, so a pin move is a line diff", () => {
+    expect(stringifyCatalog(catalog)).toBe(snapshotText);
+  });
+
+  test("keeps every section sorted by name", () => {
+    for (const section of CATALOG_SECTIONS) {
+      const names = (catalog[section] as Array<string | { name: string }>).map((r) => (typeof r === "string" ? r : r.name));
+      expect(names, section).toEqual([...names].sort((a, b) => (a < b ? -1 : a > b ? 1 : 0)));
+    }
+  });
+
+  test("holds no host-dependent default", () => {
+    expect(snapshotText).not.toMatch(/auto\(\d+\)/);
+  });
+});
+
+describe("catalog normalization", () => {
+  test("auto(<cores>) becomes auto", () => {
+    expect(normalizeDefault("auto(12)")).toBe("auto");
+    expect(normalizeDefault("8192")).toBe("8192");
+  });
+
+  test("a summary is the first prose line of the markdown", () => {
+    expect(summaryOf("## Heading\n\n```sql\nSELECT 1\n```\nThe first line.\nThe second.")).toBe("The first line.");
+    expect(summaryOf("\n\nThe first line.\nThe second.")).toBe("The first line.");
+    expect(summaryOf("x".repeat(300))).toHaveLength(200);
+  });
+
+  test("parseCatalog names the section a broken snapshot is missing", () => {
+    const broken = JSON.parse(snapshotText);
+    delete broken.codecs;
+    expect(() => parseCatalog(JSON.stringify(broken))).toThrow(/codecs/);
+  });
+});
+
+describe("the pin", () => {
+  test("a GitHub LTS tag reads as the bare version", () => {
+    expect(versionFromReleaseTag("v26.8.15.10-lts")).toBe("26.8.15.10");
+  });
+});