@aotter/mantle 0.2.0-alpha.6 → 0.2.0-alpha.7

package/docs/adr/0034-store-is-authored-as-sql.md

@@ -1,192 +0,0 @@
-# ADR-0034: Store is authored as SQL and compiled to IR by the CLI
-
-**Status:** Accepted for 0.2.0 (#1188), after the spike (#1203). Amends ADR-0030, ADR-0032 decisions 1, 2, 3, 4, 5, 10 and 13, and ADR-0033. Amended by [ADR-0035](0035-sql-dialects.md) and [ADR-0037](0037-postgresql-is-the-reference-dialect.md).
-
-**Date:** 2026-09-29
-
-**Related:** ADR-0030, ADR-0031, ADR-0032, ADR-0033, #1151, #1196
-
-## Context
-
-ADR-0032 decision 5 made the grammar the IR: an inline Procedure program and a View `select` are JSON write and read ops that #1196 implements. Business rules then asked for more of SQL: branches (`CASE`), column arithmetic, `CHECK` constraints, `RETURNING`, joins, aggregates and window functions. Each would be a node Mantle designs, documents and teaches, and for anything past a filter the result is a private query language (compare a join with an aggregate written as JSON). SQL is the query language agents know best, and it already has every one of these.
-
-Mantle runs on D1. Measured on production D1 and on local workerd, which enforce the same function allowlist:
-
-- SQLite is at least 3.45 (`jsonb`, `string_agg`, `concat`, `unixepoch('subsec')`), with window functions, recursive CTEs, `FULL`/`RIGHT JOIN`, `NULLS LAST`, `RETURNING`, `UPDATE … FROM`, `ON CONFLICT … DO UPDATE`, JSON, math functions, `STRICT` tables, virtual generated columns added by `ALTER`, and triggers that `RAISE(ABORT, …)`. Foreign keys are enforced by default.
-- D1 authorizes functions from an allowlist (`sqlite_version`, `json_pretty`, `power`, `ceiling`, `unistr` and others are refused), binds at most 100 parameters per statement, accepts statements up to 100 KB and has no interactive transaction: a program is one `batch`, applied in order, all or nothing.
-- D1 binds a JavaScript number as a floating-point value (`? / 2` with 5 is 2.5) and rounds integers past 2^53 without an error.
-
-Four parsers were measured. `libpg-query` (the PostgreSQL parser, MIT, WASM) parsed every case. `pgsql-ast-parser` misses `IS DISTINCT FROM` and `INTERSECT`; `node-sql-parser` misses `ON CONFLICT`, window functions, `INTERSECT`, `UPDATE FROM` and `FULL JOIN`; `sql-parser-cst` covers SQLite fully but is GPL-2.0-or-later, which Mantle's Apache-2.0 package cannot ship. `libpg-query` declares a 128 MiB minimum WASM memory, which is the whole isolate limit of a Cloudflare Worker, so it cannot run in one.
-
-## Decision
-
-### 1. Manifests carry SQL; the plan carries IR
-
-SQL is source and the IR is the compiled artifact, the way TypeScript is source and JavaScript is output.
-
-| Atom | v2 in #1196 | This ADR |
-|---|---|---|
-| View | `spec.select` (JSON) or `spec.sql` (native) | `spec.sql`: one `SELECT` |
-| Procedure | `handler: { store: [ops] }` | `handler: { sql: <statements> }`: one or more statements, applied as one batch |
-| Schema | — | `checks: [<boolean expression>]`, `sequences: [<name>]`, `search: [<field>]`, fields with `format: geo` |
-
-`handler: { ref }` is unchanged. The IR stays internal to the plan. It is public only through `ctx.store.select` and `ctx.store.write` in handler code, which keep ADR-0030's JSON shape; a manifest never contains IR.
-
-### 2. The dialect: SQLite semantics in the syntax PostgreSQL and SQLite share
-
-- **Semantics and functions are SQLite's, limited to what D1 authorizes.** The compiler carries the allowlist measured on D1, and CI checks it against local D1.
-- **Syntax is a chosen subset** of what PostgreSQL and SQLite share, parsed by `libpg-query`. It covers what applications commonly do (checked against MongoDB's everyday operations and the repository's examples) and stops where a feature would be hard to control. Anything left out can be added later without breaking a manifest; removing a feature would break one. SQLite-only spellings the parser rejects have shared equivalents: `IS DISTINCT FROM` for `IS NOT`, `ON CONFLICT` for `INSERT OR …`.
-
-  | Area | Supported in 0.2.0 | Rule |
-  |---|---|---|
-  | Expressions | columns, aliases, literals, arithmetic, `\|\|`, `CASE`, `COALESCE`, `NULLIF`, `CAST` | `SELECT *` and `RETURNING *` expand to the declared fields, never to scope or system columns. A `CAST` to a Mantle type is rewritten by the compiler. SQLite truncates where PostgreSQL rounds, so `CAST(x AS int)` of anything but an integer literal is refused (`SQL_TYPE`, in the validation the runtime shares) and asks for `round(x)` (a tie rounds away from zero, as PostgreSQL's `numeric` does; PostgreSQL's `float8` rounds half to even); `CAST(x AS bool)` becomes `x <> 0` for a number and PostgreSQL's spellings (`t`, `true`, `yes`, `on`, `1`) for text, since `'false' <> 0` is true in SQLite |
-  | Conditions | comparison, `AND`/`OR`/`NOT`, `BETWEEN`, `IS [NOT] NULL`, `IS DISTINCT FROM`, `IN (list \| subquery)`, `[NOT] EXISTS`, `LIKE … ESCAPE` | an input array in `IN` lowers to `json_each` and binds once. `LIKE` is case-insensitive in SQLite, unlike PostgreSQL; the diagnostics say so |
-  | Relations | one Schema, `INNER`/`LEFT JOIN … ON` (self-joins included), a subquery in `FROM`, `json_each(<input or column>)` | the one comma join allowed is `t, json_each(t.col)` (MongoDB's `$unwind`) |
-  | Subqueries | scalar and correlated | — |
-  | Aggregation | `count`, `sum`, `min`, `max`, `avg`, `count(DISTINCT)`, `json_group_array([DISTINCT])`, `json_group_object`, `GROUP BY`, `HAVING` | a selected column must be grouped or aggregated |
-  | Windows | `row_number()`, `rank()`, `sum`/`count … OVER (PARTITION BY … ORDER BY …)`, in Views only | no frame clause; for `row_number()` only, the compiler appends `id` to the window's `ORDER BY`. `rank()` and running aggregates keep PostgreSQL's peer semantics, which an appended key would break |
-  | Order and paging | `ORDER BY … [NULLS FIRST \| LAST]`, `LIMIT`, `DISTINCT`, cursors | the compiler appends `id` as the final sort key (the group key under `GROUP BY`); `LIMIT` needs an `ORDER BY`; a nullable sort key states its NULL position; `DISTINCT` with `ORDER BY` is refused, since the appended key would change what is distinct; ordering a subquery in `FROM` needs that subquery to output `id` |
-  | JSON | `->>`, `json_extract`, `json_set`, `json_insert`, `json_remove`, `json_array_length` | a JSON result carries its declared type; `->` is refused |
-  | Writes | `INSERT … VALUES` (one row), `INSERT … SELECT`, `UPDATE … SET <expr> WHERE`, `DELETE … WHERE`, `RETURNING <columns>`, `ON CONFLICT (…) DO NOTHING \| DO UPDATE … EXCLUDED` | see decision 8 for what a write may name; an `INSERT` on a scoped Schema names no `id`, the compiler generates it. `ON CONFLICT` is a set op, and on a scoped Schema its conflict target includes the scope field |
-  | Programs | several statements, one batch | statements pass values only through tables |
-  | Time | `now()`, `date_trunc('hour' \| 'day' \| 'week' \| 'month' \| 'year', ts)`, `extract(year \| month \| day \| dow \| hour FROM ts)`, `ts ± interval` in seconds, minutes or hours, `ts - ts` | `date_trunc` and `extract` compute in the site time zone. `ts - ts` is microseconds. Display formatting belongs to the client |
-  | Search and places | `search()`, `search_rank()`, `near()`, `distance()` | decision 9 |
-  | Mantle | `input.<name>`, `auth.uid()`, `auth.role()`, `nextval()`, `numeric(p, s)` columns | below |
-
-  **Deferred** (each is additive): `UNION ALL`, `UNION`, `INTERSECT`, `EXCEPT`; non-recursive CTEs (a subquery in `FROM` covers them); aggregate `FILTER` (`sum(CASE …)` covers it); `string_agg`; `$pull`-style array rebuilds; calendar `interval` units and `timezone()` with any zone but the site's; arithmetic rewriting for `numeric`; before hooks on set ops; a cursor over `distance()`.
-
-  **Refused in 0.2.0:** `RIGHT`/`FULL JOIN` (rewrite as `LEFT JOIN`); `CROSS JOIN` and comma joins other than `json_each`, where one forgotten condition is a Cartesian product; recursive CTEs, whose work has no bound; `LATERAL`, `NATURAL`, `USING`, `GROUPING SETS`; `UPDATE … FROM` and `DELETE … USING`, where SQLite picks an arbitrary row when several match (a correlated subquery in `SET` covers them); multi-row `VALUES` (`json_each(input.items)` covers it); window frames.
-- **References.** `input.<name>` reads a declared input property, as PostgreSQL's `NEW`, `OLD` and `EXCLUDED` read pseudo-relations. `auth.uid()` is the caller's application subject key and `auth.role()` its staff role (ADR-0032 decision 8). `now()` is the invocation time. Positional `$1` is refused. `input` and `auth` are reserved and cannot name a Schema or an alias. A Schema's `scope` names the same reference (`scope: { ownerId: "auth.uid()" }`), so ADR-0032's `$input.<path>`, `$ctx.user.id`, `$now` and `{ $literal }` are gone; a SQL literal is a literal.
-- **Identifiers resolve case-insensitively,** as SQLite does. The parser folds unquoted identifiers to lower case, so `createdAt` and `createdat` name the same column. SQLite already refuses two tables or two columns whose names differ only by case, and `mantle validate` refuses them first. A Schema whose name is not a plain identifier is quoted: `"support-requests"`. A Schema or field named with one of the 14 keywords that SQLite rejects unquoted and the printer leaves bare is refused: `add alter autoincrement commit delete drop escape index insert nothing raise set transaction update` (measured against all 147 PostgreSQL keywords; `match` and `glob` are fine). The list belongs to `pgsql-deparser` 18.3.8 and D1's SQLite, and the spike's 1d check fails when either changes it.
-- **Mantle's additions keep PostgreSQL names** where PostgreSQL has one: `nextval('<sequence>')`, `date_trunc`, `extract`. `nextval()` appears only in a one-row `VALUES`, at most once per sequence per statement.
-- **`numeric(p, s)` is a storage type, not an arithmetic rewrite.** Inside SQL a `numeric(12, 2)` column is its integer count of the smallest unit (cents), so `sum`, comparison and integer arithmetic are exact, as Stripe's amounts are. Input and output convert at the boundary. An author who multiplies by a fraction writes the rounding (`round(amount * input.rate)`). Rewriting decimal arithmetic in the compiler is deferred.
-- **`interval`.** `second`, `minute` and `hour` compile to a microsecond constant. `day`, `week`, `month` and `year` are calendar units in PostgreSQL (a day is 23 or 25 hours across daylight saving), so 0.2.0 refuses them, and the diagnostic tells the author to bind the boundary as an input. Refusing is better than computing a wrong time; calendar units are additive later.
-- **Always refused:** `OFFSET` (cursor pagination only); DDL; transaction control; data-modifying CTEs; `SELECT … FOR UPDATE`; functions outside the allowlist; any table that is not a declared Schema, `_mantle_*` included; table-valued functions other than `json_each`; `rowid`; SQLite's own clock (`'now'`, `CURRENT_TIMESTAMP`), which bypasses `now()`; `strftime`, `date` and `unixepoch` in source, which return NULL on microsecond integers; `printf` width arguments, `zeroblob` and non-constant `randomblob` sizes, which can build values of any size; `ILIKE`; `REGEXP`, which D1 lacks. There is no `native` escape: on D1 nothing sits below SQLite, so an escape could only bypass policy.
-- **No `expect` in the source.** A row op must affect exactly one row, or the batch fails with `CONFLICT`. A conditional insert (`INSERT … SELECT … WHERE`) is a set op, and writing no row is a normal result.
-
-### 3. Only the CLI parses SQL
-
-- `@aotter/mantle/spec` exports `compilePlan(sources)`, returning a plan or diagnostics. It is the only compiler. The `mantle` CLI calls it, and so do the plugin's Cloud helper scripts: they load it from the project's installed `@aotter/mantle` with `createRequire`, as they already load the project's `esbuild`, and refuse to run when its version is not the Core version Cloud pinned (ADR-0031). The plugin bundles no parser. `compilePlan`'s signature is the stable contract between plugin and Core.
-- No Worker parses SQL. Cloud's Control receives the plan, with the SQL source and its hash, and validates the IR in plain JavaScript: structure, Schema references and compatibility with the pinned Core. A tenant Worker executes the IR.
-- Coding agents compile: the Claude Code CLI, desktop and cloud sessions, Cowork, and Codex local and remote all have a working directory and Node. A chat assistant operates a service through the Cloud MCP and does not build one.
-- If a browser ever compiles (the Builder), it dynamic-imports the same `libpg-query`; a page that does not compile never loads it.
-- **The IR is the parser's own AST.** It is `libpg-query`'s parse tree for the subset, with source locations stripped and each relation tagged `table` or `cte`. Mantle designs no node types. The plan records the PostgreSQL grammar version (`180004`), and the runtime refuses any other, since the AST changes between PostgreSQL majors; `libpg-query`, `@pgsql/types` and the printer are pinned to one major.
-- **Validation is an exact allowlist** of node types and of the keys each may carry, shared by the CLI and the runtime. An unknown node or key is refused, never skipped: a printer that skipped keys would silently drop `OFFSET`, `WITH` or `FOR UPDATE`. Diagnostics are produced in the CLI before locations are stripped.
-- **The printer is `pgsql-deparser`** (MIT, pure TypeScript, no WASM), subclassed in four places: `ParamRef` (numbered binds `?n`), `TypeCast` (`CAST(… AS …)` instead of `::`), `A_Const` (plain SQL string literals instead of `E'…'`) and `A_Expr` (`x LIKE p ESCAPE e` parses to `like_escape(p, e)`, which SQLite lacks). The `ParamRef` override is a correctness fix: SQLite reads `$n` as a named parameter numbered by first appearance, so binds would shift silently. Four AST rewrites also precede printing: the `pg_catalog.` prefix and SQL-syntax calls, `btrim` to `trim`, `= ANY (subquery)` to `IN`, and type names. The rest of the subset prints as valid SQLite, and each allowlisted node type has a conformance case on local D1. Eight changes are under "about a dozen", so Kysely's SQLite compiler is not needed.
-- **Handler-built queries join the same path.** `ctx.store.select` and `ctx.store.write` keep ADR-0030's JSON shape; a converter turns it into the same AST, which then passes the same validation and policy.
-- Measured on a prototype over the six programs of the conformance cases: runtime validation 127 lines, policy 97, printer 7 on top of `pgsql-deparser` (43 KB gzipped), no cross-owner leak.
-
-### 4. Row ops, set ops and hooks (amends ADR-0032 decisions 2 and 3)
-
-- **Classification.** An `UPDATE` or `DELETE` whose top-level `WHERE` conjunction contains `id = <scalar>` is a row op, and so is an `INSERT … VALUES` of one row, except that one row of `VALUES` with `ON CONFLICT` is a set op (a `DO NOTHING` that hits the conflict must not fail with `CONFLICT`). Every other write is a set op. A row op with a `version = input.<name>` conjunct is locked (ADR-0022). Classification now decides only two things: `Procedure.target` inference, and whether writing no row is a `CONFLICT`.
-- **Before hooks take row ops only.** A set op on a Schema with a `before_*` Trigger for that operation is refused, as ADR-0032 decision 2 already says, and so is a set op on a Schema whose lifecycle is `publishing`. After hooks no longer refuse set ops.
-- **One after-hook call per statement and Trigger.** A hook receives `ctx.cause.rows`, a non-empty array (`[Row, ...Row[]]`) whose rows always carry `id` and `version`: for a hooked target the compiler adds hidden `_mantle_id` and `_mantle_version` columns and the executor strips them before the hook sees the rows. `ctx.cause.id` is stable per (statement, hook) across retries. Rows come in no particular order and an upsert may repeat an id, so a handler is idempotent per row. A statement that writes no row calls no hook. Calling once per row is rejected: 500 rows would be 500 subrequests, past Cloud's limit.
-- **After hooks** read the statement's `RETURNING` rows, which the compiler adds when the target has an after hook, and run after the commit, as ADR-0032 decision 3 says.
-
-> **Amendment (2026-10-01, site trial):** The hook's rows and the statement's result are separate. The compiler adds the whole readable row to `RETURNING` under hidden `_mantle_h_` columns whatever the author's own `RETURNING` lists; the runner strips them from the result and hands them to the hook, decoded as `ctx.store.select` returns an entry (declared names, `createdAt`/`updatedAt` as ISO date-times). Before, an author's `RETURNING id, status` was all the hook saw, so the same hook got a different row from Admin, from a Store write and from a Procedure, against decision 3's "the row and version at commit".
-- **Before hooks** read the one row first and run on it. The statement then carries the version the hook saw (`AND version = ?`), so a change between the hook and the commit is `CONFLICT`. This is today's OCC path. A before hook's `ctx.cause.rows` holds that one row. When the row is not visible (scope, TTL or published-only), the operation fails with `CONFLICT` and the hook is not called, so a hook cannot probe for another owner's row.
-- A snapshot guard that would let before hooks cover set ops (pre-read, then recompute inside the batch and abort on any difference) is deferred. Review found it the costliest mechanism here: it needs read and write set analysis per statement, and a `SET` that reads its own target defeats it.
-- `last_insert_rowid()` is refused everywhere.
-- **Counting** uses SQLite's `changes()` inside the batch, not D1's `meta.changes`, which includes rows that triggers wrote (5 for a one-row `UPDATE` on a searchable row), so it cannot be used. Count checks fail with `CONFLICT op=<k>`, so `conflict.opIndex` becomes exact (amends ADR-0032 decision 1).
-- ADR-0032 decision 3's before-hook rules stand: read-only, fail closed, committed data only, OCC between the hook and the commit.
-- `mantle-update` rewrites `ctx.event.entry` into a loop over `ctx.cause.rows`, never into `rows[0]`.
-
-### 5. Storage (amends ADR-0033)
-
-- **Types are encoded so D1 computes them exactly:**
-  - `timestamptz` (`format: date-time`) is an integer of microseconds since the epoch, exact until the year 2255;
-  - `date` (`format: date`) is an integer count of days;
-  - `numeric(p, s)` is an integer scaled by 10^s, with `p` at most 15 because D1 rounds past 2^53 (decision 2).
-
-  Mantle's native timestamps move from milliseconds to microseconds.
-- **Every parameter is bound through a `CAST` to its declared input type,** so integer arithmetic gives the same result on D1, which binds numbers as floats, and on every other driver.
-- **Schema tables Mantle creates are `STRICT`.** An existing table is not rebuilt. Each carries `_rid INTEGER PRIMARY KEY`, an alias of the rowid, and `id TEXT NOT NULL UNIQUE`: the FTS5 and R*Tree tables key on the rowid, and a rowid that no `INTEGER PRIMARY KEY` aliases may change on `VACUUM`. `_rid` is not addressable.
-- **`checks` and foreign keys are enforced by triggers** that storage convergence creates, because SQLite cannot add a `CHECK` or a foreign key to an existing table. The triggers only `RAISE`, except the ones that keep `_mantle_fts_*` and `_mantle_geo_*` in step (decision 9), which write only those system tables. Foreign keys are `RESTRICT` only: storage never creates `CASCADE` or `SET NULL`, whose writes would not run Mantle hooks. A `check` reads only the row's own columns and pure functions, never a subquery.
-- **Sequences and time zones use system tables.** `nextval()` increments `_mantle_sequences` with `UPDATE … RETURNING`, and a later statement in the same batch reads the value back. `date_trunc` and `extract` join `_mantle_tz`, the site time zone's UTC-offset transitions, so daylight-saving zones compute correctly. Transitions match `Intl` on every measured instant. Where local time repeats, each pass of the repeated hour truncates to its own hour start; where a zone skips half an hour (Lord Howe), truncating to the hour has no single answer. Neither was compared with PostgreSQL, so parity is not claimed for them. The site time zone is a site setting, and the runtime generates its transitions with `Intl.DateTimeFormat`, whose time zone data ships with workerd and Node, when the setting changes or on deploy; no time zone library is bundled.
-
-### 6. One executor, one test line (amends ADR-0032 decisions 4 and 13)
-
-- **One executor.** `SqliteStoreExecutor` is the only executor, over a `DatabaseDriver`: D1, with Bun and libSQL experimental. `MemoryStoreExecutor`, the in-memory test fake and the IndexedDB repositories are removed, and so is `ViewQueryExecutor`, since every View compiles to IR.
-- **`/indexeddb` is removed, with no browser driver in 0.2.0.** Nothing uses `@aotter/mantle-indexeddb` today, and the Builder's preview moves to Cloud. A browser-only application that needs one later gets an additive subpath: a SQLite-in-WASM driver under `SqliteStoreExecutor`, not a new executor.
-- **One test line.** `@aotter/mantle/testing` stays engine-free, `runStorageConformance({ create })`, and each driver runs it on its own engine.
-  - D1 runs it on local D1 inside workerd (`@cloudflare/vitest-pool-workers`), locally and in CI alike; 500 cases take about 3.5 s. Code that runs in a Worker is tested there, while spec and CLI tests stay in Node, where `libpg-query` runs.
-  - One case prepares every function on the compiler's allowlist against local D1, so CI fails when D1 changes the list.
-  - D1 test helpers, if published, live at `@aotter/mantle/cloudflare/testing`, with `wrangler` as an optional peer.
-  - `node:sqlite` is not used, and the `sqlite-d1.ts` fake is removed, because its `batch` runs statements without a transaction.
-  - Ceiling: workerd's Linux binary needs glibc 2.35 and has no musl build, so the storage tests do not run on Alpine.
-
-### 7. Cloud validates the plan (amends ADR-0032 decision 10)
-
-The platform-verified facts become the **Cloud-validated plan**, the pinned Core, storage matching the plan and the fingerprint handshake. The runtime's Store injects scope, TTL, published-only and OCC at execution whatever compiled the IR, and a plan can express nothing its SQL author could not, provided the runtime enforces decision 8 on every IR it compiles, handler-built IR included. So this loses no guarantee.
-
-That the IR matches its SQL source is **service-reported**: the plan carries the source's hash, and the compiler is the pinned Core. Host protocol 3 uploads the plan.
-
-`verifyPlan(plan, storage)` in `@aotter/mantle` is that validation: boot's own checks of the plan (version, fingerprint, dialect, guard and hook targets) and every View and inline Procedure through the storage's dialect, its `restrict` and the policy rewrite, with no database and no SQL parser. It returns the diagnostics, each with the program's path; handlers and schedule wiring stay the host's.
-
-What `verifyPlan` proves is that a plan reads and writes only through the policy rewrite and refuses what boot would refuse of the plan. It does not type-check the SQL: a statement the database refuses (a function's arity or argument type, an operand type) fails that request, or boot for a Schema check, with the database's error, and never runs past the policy. It does not bound the CPU a plan's request costs either: a JSON Schema `pattern` runs a backtracking JavaScript regex, and the pattern checks catch the shapes that backtrack catastrophically by accident, not every slow regex an author can write. A host that runs untrusted plans bounds CPU per request instead (Cloud dispatches each tenant request with a CPU limit), so a slow plan stalls only its own requests.
-
-### 8. Policy injection
-
-- **Every Schema reference is wrapped.** The compiler has one function that prints a Schema table's name, and it always prints `(SELECT <declared columns> FROM "t" WHERE <scope> AND <ttl> AND <published>) AS <alias>`. Joins, subqueries, set-op branches and window inputs therefore see only visible rows, and a `LEFT JOIN` stays a left join. SQLite flattens the wrapper, so the index the scope leads is still used; RIGHT JOIN would materialize, one more reason it is refused.
-- **`UPDATE` and `DELETE` targets** cannot be subqueries in SQLite, so the same predicate is ANDed into their `WHERE`. `ON CONFLICT DO UPDATE` gets it in the `DO UPDATE … WHERE`, TTL included, so a conflict cannot overwrite another owner's row or revive an expired one.
-- **A write may not name** the scope field or a system column (`id` on update, `version`, `status`, `authorId`, timestamps) in an `INSERT` column list, an `UPDATE … SET` or a `DO UPDATE SET`. The compiler fills them. On a scoped Schema the compiler also generates `id` on insert, because a caller-chosen id collides with another owner's row and reveals it; a retry-safe insert there is an upsert on a unique key that includes the scope field. The refusal applies per Schema, to that Schema's own scope field. Changing `status` belongs to the lifecycle rules. Moving a row to another owner is impossible by construction.
-- **Name resolution is checked at run time too.** The CLI tags each relation `table` or `cte`. The runtime refuses an IR whose `cte` reference is not defined in scope or carries a Schema's name, since SQLite would resolve it to the unwrapped table. The same check runs on IR built by handler code.
-- **Constraints do not leak across owners.** On a scoped Schema a unique constraint includes the scope field, so a collision can only be with the caller's own rows. A foreign key between two scoped Schemas requires the same owner (`parent.owner = NEW.owner`), and a foreign key from an unscoped Schema to a scoped one is refused. Otherwise an insert could pin another owner's row or reveal that it exists.
-- **Published-only does not follow foreign keys.** A public View that joins a Schema without `publishing` must also join that row to a `publishing` Schema by an explicit condition, or `mantle validate` refuses it.
-- **Binds are numbered** (`?1`, `?2`), so the caller and `now()` bind once however many references a statement wraps.
-- `runtime.store` is trusted: no scope, TTL still applies (ADR-0030).
-- **One conformance case guards the rule.** It lists every relation position the IR has (`from`, joins, subqueries, `json_each`, window input, `INSERT … SELECT`, update and delete targets, `DO UPDATE`, search, near) in a `Record` checked with `satisfies` against the IR's position union, seeds a second owner's rows, including an expired and an unpublished one, and asserts none appear or change. A new position without a probe fails typecheck. The parser's AST does not say which keys hold a relation, so the position union is a hand-kept list, bound three ways: each edge is checked against `@pgsql/types` keys, `ALL_EDGES_REACHED` fails when an edge has no position, and the policy pass fails closed on a relation reached through an edge it cannot name.
-
-### 9. Search and places
-
-- **Full-text search.** A Schema declares `search: [<field>, …]`. Storage keeps `_mantle_fts_<schema>`, an FTS5 external-content table with the `trigram` tokenizer, so Chinese substrings match, in step through triggers, and builds it with FTS5's `rebuild` when `search` is added. `search(t, input.q)` lowers to `t.rowid IN (SELECT rowid FROM _mantle_fts_<schema> WHERE _mantle_fts_<schema> = ?)`, so policy still applies to `t`; the printed form is `fts = q` because PostgreSQL's grammar has no `MATCH`, and FTS5 gives the same results. The policy wrapper also outputs `rowid AS _rid`. `search_rank(t)` orders by `bm25`, taking the query of the `search(t, q)` in the same `SELECT`. The query binds as a quoted phrase, never as FTS5 syntax. A query shorter than three characters, which trigram cannot match, lowers to `LIKE` over the same fields. Admin's `searchFields` uses `search()` when the Schema declares it.
-- **Places.** A field with `format: geo` holds `{ lat, lng }` as two `REAL` columns, and storage keeps `_mantle_geo_<schema>`, an R*Tree of the points, in step through triggers. `near(t.f, lat, lng, meters)` binds a bounding box the runtime computes from the radius, then filters by haversine distance. `distance(t.f, lat, lng)` is that distance in meters, for `SELECT` and `ORDER BY`. `near()` requires a literal radius, at most 50 km (coordinates may be inputs or literals, since the bounding box is bound at run time), and a query ordered by `distance()` a `LIMIT` of at most 100: nearby search returns the closest K, with no cursor. A box that crosses the antimeridian or a pole is refused. Polygon containment is not provided.
-- Measured on local D1: FTS5 with `trigram` (a three-character Chinese query matches, a two-character one does not), external content kept by a trigger, `bm25` and `snippet`, R*Tree, and `radians`, `sin`, `cos`, `asin`, `sqrt` and `atan2` on the allowlist. Production D1 passed the same probes on 2026-09-29 (FTS5 trigram, R*Tree, the math functions, `RAISE(IGNORE)` with `changes()`; `next/spike/production-probe.sql`). Local D1 needs workerd 1.20260903.1 or later to create an R*Tree.
-
-## Conformance cases
-
-1. The requisition program: a `CASE` value, `RETURNING`, a `WHERE id AND cond` precondition that fails with `CONFLICT`, and a conditional `INSERT … SELECT … WHERE` that writes zero or one row and calls an after hook only when it writes.
-2. Stock: `SET stock = stock - input.qty` with `checks: ["stock >= 0"]`, where oversell fails through the check trigger.
-3. A report View with a join, `GROUP BY`/`HAVING` and a cursor, where scope and TTL are injected into every joined Schema.
-4. Before hooks: a row op whose row changes between the hook and the commit fails with `CONFLICT`; a set op on a Schema with a before hook is refused.
-5. The dialect: an unknown AST key, `OFFSET`, a function outside the allowlist, `$1`, `RIGHT JOIN`, `UPDATE … FROM`, `CURRENT_TIMESTAMP`, an undeclared table, a `cte` reference carrying a Schema's name and `_mantle_*` are refused with a position in the source SQL.
-6. Types: `? / 2` with an integer input, `sum` over a `numeric(12, 2)` column, a microsecond timestamp compared with `now() - interval '36 hours'`, and `date_trunc('day', ts)` on a daylight-saving day in the site time zone give PostgreSQL's results on D1 (outside the repeated and skipped hours of decision 5); a non-literal `CAST(x AS int)` is refused and `round(x)` matches PostgreSQL; `interval '1 day'` is refused.
-7. Policy: the relation-position probe of decision 8, on local D1.
-8. Search and places: a trigram match and a two-character fallback, a phrase containing FTS5 operators matched literally, another owner's rows absent from `search()` and `near()`, and `near()` returning the closest K in order.
-
-## Consequences
-
-- #1196's JSON program and `select` grammar, and the parser and graph rules written for them, are replaced by SQL lowering; `StoreProgram`'s rules move onto the compiled statement. The spike decides whether #1196 is rewritten or superseded.
-- The 16 MiB-patched `libpg-query`, its workerd wrapper and the Control memory test proposed for parsing in Control are not needed.
-- The Builder's in-browser preview (`packages/builder/src/lib/memory-storage.ts` in aotter/mantle-home) is built on ports this release removes, so its replacement must be decided before aotter/mantle-home adopts 0.2.0; the direction is a signed-in Cloud preview on the existing candidate preview.
-- Admin's Developer Console draws a program from the IR: `CASE` as a branch, a `WHERE` guard as an edge, joins, checks, aggregates and `RETURNING` as nodes, anything else as SQL text.
-
-## Alternatives
-
-- **IR in the manifest** (#1196 as is). Its JSON Schema validates exactly and can constrain an agent's output, and nothing needs a parser. Rejected: past a filter it is a private language agents must learn, and every SQL feature becomes a node Mantle designs.
-- **Parse in Control with a `libpg-query` patched to 16 MiB.** Rejected: a patched binary to rebuild on every upstream release, WASM memory that never shrinks (one 1.7 MB statement grew the heap to 221 MiB), and WASM in a Worker.
-- **Another parser.** `pgsql-ast-parser` and `node-sql-parser` miss syntax this ADR needs; `sql-parser-cst` is GPL; the Rust `sqlparser` as WASM is 1 MB gzipped and misses the same SQLite-only syntax; native addons and Python's `sqlglot` complicate installing the CLI. In a CLI, WASM is the most portable form.
-- **Compile policies into SQL ahead of time and ship statements.** Rejected: `ctx.store.select` in handler code is built at run time, so the runtime compiles IR anyway, and a second path would bypass the one Store.
-- **A JavaScript executor or an IndexedDB-backed executor.** Rejected: once the IR is what D1 runs, either is a second SQLite.
-- **A second test line on sqlite-wasm made to imitate D1, and a `prepare` check of every compiled statement in the CLI.** Rejected: local D1 in workerd is fast enough for everyday tests, so the imitation would be one more profile to keep in step with D1. The conformance suite on local D1 already catches compiler output that D1 refuses, and without the check the CLI does not depend on sqlite-wasm.
-- **Support as much of the shared syntax as possible.** Rejected in review: every feature is a policy position, a guard case and a diagnostic to keep correct, and a removal breaks manifests while an addition does not. The subset starts from what applications do and grows by addition.
-- **Inject policy through CTEs that shadow table names, or through persistent views reading a caller row written first in the batch.** Rejected: an `UPDATE` target does not resolve to a CTE, so writes would need a second mechanism, and a view reading a context row turns every read into a write and fails open when the context row is missing.
-- **Calling hooks once per row, or rewriting a before-hooked statement to the pre-read ids.** Rejected: the first exceeds subrequest limits; the second reproduced two corruptions on local D1 (a phantom between statements, and an expression reading another table that changed).
-- **A Mantle-designed IR printed by a hand-written printer, or Kysely operation nodes.** Rejected: the parser's AST already is a complete, typed IR, and `pgsql-deparser` prints it. Converting to Kysely nodes took more code than the hand-written printer, and Kysely's node kinds are internal to a 0.x library, so they cannot be a stored plan format.
-
-## How to apply
-
-1. Spike, in `next/`:
-   - print every allowlisted node type with `pgsql-deparser` and run it on local D1, counting the overrides needed;
-   - run the conformance programs on local D1 in workerd;
-   - measure how helpful the diagnostics are to an agent;
-   - prove the `CAST` rule, the type encodings and `date_trunc` across daylight saving.
-2. Settle #1196 from the spike's result.
-3. The diagnostic codes are `SQL_SYNTAX`, `SQL_UNSUPPORTED`, `SQL_FUNCTION`, `SQL_RELATION`, `SQL_COLUMN`, `SQL_WRITE`, `SQL_SHAPE` and `SQL_TYPE` (fixed by the spike, no warning code), and are added to ADR-0032 decision 5's table.
-4. Everything here ships in 0.2.0. The additions land in the order sequences, `numeric` columns, then the site time zone: a sequence is one system table, `numeric` is a boundary codec, and the time zone needs transition data generated and kept current.
-
-## Implementation status
-
-Proposed.
-
-> **Amendment (spike, #1203, 2026-09-29):** The spike's findings are folded into decisions 2 to 5, 8 and 9 and How to apply 3. Changes: the four printer overrides and four AST rewrites; the 14 measured reserved names; `row_number()`-only window key; one-row upsert as a set op; hook rows always carry `id` and `version`, invisible before-hook rows fail with `CONFLICT`; time zone parity not claimed; a non-literal `CAST` to int is refused in favor of `round(x)` (replacing the earlier warning); relation positions as a hand-kept, type-bound list; the printed form of `search`; production D1 probed.

package/docs/adr/0035-sql-dialects.md

@@ -1,154 +0,0 @@
-# ADR-0035: SQL dialects — PostgreSQL syntax is Mantle SQL, a dialect runs it on one engine
-
-**Status:** Accepted for 0.2.0. Amends ADR-0032 decisions 6 and 12, ADR-0033, and ADR-0034 decisions 2, 3, 5, 6, 7, 8 and 9. Amended by its PostgreSQL dialect amendment (#1293), [ADR-0036](0036-host-and-dialect.md) and [ADR-0037](0037-postgresql-is-the-reference-dialect.md).
-
-**Date:** 2026-09-30
-
-**Related:** ADR-0032, ADR-0033, ADR-0034
-
-## Context
-
-ADR-0034 made manifests SQL and D1 the one engine: its decision 2 calls the dialect "SQLite semantics in the syntax PostgreSQL and SQLite share", and decision 6 gives the Store one executor. The syntax is already PostgreSQL's (`libpg-query` parses it), and conformance case 6 already holds D1 to PostgreSQL's results. What is missing is a boundary: the D1 subset, the SQLite rendering, policy injection and storage convergence are spread through Core, so no other engine can run a Mantle plan without a fork.
-
-A service that needs another engine should be able to add one. Mature SQL tools split the same way: SQLGlot and Apache Calcite parse to one tree and generate per dialect; Kysely's `Dialect` bundles a driver, a query compiler, an adapter and an introspector; SQLAlchemy ships a dialect compliance suite a third-party dialect must pass. Mantle adopts that shape. The open-source repository ships one dialect, D1, and the interface and test suite that let anyone write another; an enterprise PostgreSQL dialect is the first one written outside it.
-
-## Decision
-
-### 1. Mantle SQL is PostgreSQL syntax, pinned (amends ADR-0034 decision 2)
-
-- A View or Procedure is written in PostgreSQL syntax, and its canonical form is the parse tree of `libpg-query` 18, PostgreSQL 18's own parser. The parser version is part of the language: moving to another major is a deliberate Mantle version change, recorded in an amendment.
-- Its meaning is PostgreSQL's. A dialect either gives PostgreSQL's result for a construct or refuses it at compile time with a position in the source; it never approximates. (ADR-0034 conformance case 6 is this rule for D1.)
-- A manifest that one dialect accepts may be refused by another. The D1 dialect accepts the ADR-0034 subset; a PostgreSQL dialect can accept nearly all of PostgreSQL.
-
-### 2. Mantle's own functions live in the `mantle` schema (amends ADR-0034 decision 9)
-
-- `search()`, `search_rank()`, `near()` and `distance()` become `mantle.search()`, `mantle.search_rank()`, `mantle.near()` and `mantle.distance()`, as `auth.uid()` and `auth.role()` already are. An unqualified call is PostgreSQL's function of that name, never Mantle's. 0.2.0 is unreleased, so there is no old spelling to keep.
-- A function is Mantle's exactly when its schema is `mantle` or `auth`. Each dialect lowers every `mantle.*` function it supports and refuses the rest with a position; the D1 dialect lowers all four to FTS5 and R*Tree (ADR-0034 decision 9), and a dialect may refuse them and leave authors to the engine's own syntax.
-- `auth.uid()` and `auth.role()` are required of every dialect: policy is written with them.
-
-### 3. The dialect interface
-
-A dialect is an npm module with two entry points, so the parser stays out of the runtime bundle (ADR-0034 decision 3):
-
-- **Compile side** (`<dialect>/compile`, loaded only by the CLI):
-  - `accepts(tree, context)`: every refusal, each with its source position. Context is the linked manifest set: declared Schemas and their fields, inputs, and surfaces.
-  - `lower(tree, context)`: the dialect's statement payload for the plan, opaque to Core, with `mantle.*` calls lowered.
-  - `outputs(tree, context)`: a View's row type for the typed `mantle.ts`, by ADR-0034's rule (an output that reads a Schema field unchanged has its type; any other output is `unknown`).
-- **Runtime side** (`<dialect>`, loaded by the host entry):
-  - `executor`: a `StoreExecutor` that runs plan payloads and renders the Store's structured operations (`select`, `write`, `search`) to the engine.
-  - `policy`: `"inject"`, where Mantle's policy rewriter adds visibility and ownership to every relation position (ADR-0034 decision 8), or `"native"`, where the engine enforces them (for example PostgreSQL row-level security) and the dialect writes that enforcement during convergence.
-  - `converge(plan)`: ADR-0033's storage convergence in the engine's DDL, including the tables Mantle owns (the migration ledger, boot state, site and media).
-  - `codec`: the Store's wire values. They are the same on every dialect (ADR-0034's timestamps, `numeric` and booleans), whatever the engine stores.
-- There are no capability flags. What a dialect cannot run, its compile side refuses.
-- Core keeps the Store API, the runtime, the policy rewriter as a library any `"inject"` dialect uses, and the interface's types. It holds no engine-specific code.
-
-### 4. What every dialect gets before `accepts`
-
-The CLI front end is shared and dialect-free. It parses, links, and refuses what no dialect may run:
-- A View is one `SELECT` (with `WITH`). A Procedure is `SELECT`, `INSERT`, `UPDATE`, `DELETE` and `MERGE` statements, with `WITH` and `RETURNING`. DDL, `SET`/`RESET`, transaction control, `COPY`, `DO`, `CALL`, `LISTEN`/`NOTIFY` and `PREPARE` are refused.
-- Every relation anywhere in the tree (`FROM`, joins, `LATERAL`, and subqueries in the select list, `WHERE`, `HAVING` and `RETURNING`) is a declared Schema or a CTE of the same statement; Mantle's tables and the identity provider's are never named.
-- `input.<name>` is a declared input. The `mantle` and `auth` schemas hold only the functions of decision 2.
-- Functions that change session or server state, or reach outside the database, are refused whatever the dialect: `set_config` and `current_setting` of `mantle.*`, advisory locks, file and large-object access, `dblink`, backend signals and `pg_reload_conf`. A `"native"` dialect relies on this list, because a caller setting could otherwise be forged from SQL.
-
-### 5. Choosing a dialect (amends ADR-0032 decision 12)
-
-- `mantle.config.json` version 2 gains `dialect`, an npm module name. Absent, it is the built-in D1 dialect, so existing projects do not change. There is no CLI flag and no `host` field.
-- The plan records the dialect's name and version, and both are part of the fingerprint. Boot refuses a plan compiled for another dialect. Cloud Control (ADR-0034 decision 7) accepts only D1 plans.
-- A dialect and a host are different things. The dialect is the engine and is a public interface; the host is the platform entry (a Worker, `Bun.serve`, scheduling, media), which stays an application-owned preset. The open-source generator writes the Cloudflare preset only (ADR-0032 decision 6); another host's preset belongs with whoever ships it.
-
-### 6. The D1 dialect (amends ADR-0034 decisions 5, 6 and 8)
-
-- The built-in dialect, `@aotter/mantle/d1`. It is everything ADR-0034 describes for D1: the subset check, the IR and its validator, the SQLite rendering and codec, FTS5 and R*Tree for `mantle.*`, `"inject"` policy, and trigger-based convergence. These move out of Core into the dialect; behavior does not change.
-- `@aotter/mantle/cloudflare` keeps the D1 binding driver and `d1Storage`, now built on the dialect.
-
-### 7. `StoreSelect.search` (new)
-
-- `StoreSelect` gains `search?: string`: rows whose declared `searchableFields` contain the text, or whose `id` is the text, under the caller's visibility like any `select`. Ordering stays `orderBy`. Each dialect's executor implements it; D1 uses its FTS5 trigram table, so Chinese substrings of three characters or more match, and a shorter query falls back to a scan.
-- Admin's collection search box sends `search` instead of building `like` conditions itself.
-- The manifest key is `searchableFields`; ADR-0034 decision 9 wrote `search`, and the plan stores it as `search`.
-
-### 8. The dialect compliance suite
-
-- `@aotter/mantle/testing` becomes the dialect compliance suite: every ADR-0034 conformance case, the relation-position probe, the wire-value checks and the `StoreSelect.search` checks, each against a dialect's runtime side on a real engine, and the refusal cases against its compile side. A case whose feature a dialect refuses at compile time passes by that refusal.
-- A dialect is supported when it passes the suite. The D1 dialect runs it on local D1 in CI, as today.
-
-### 9. The identity provider's SQL
-
-- `createMantleAuth` stops writing its own SQLite. What it still writes in SQL (members, invitations, consents, linked accounts, the bootstrap owner, its schema ledger) moves to Better Auth's own Kysely instance, which speaks every engine Better Auth supports. Auth then depends on Better Auth's adapters, not on a Mantle dialect.
-
-## Consequences
-
-- Another engine is a module, not a fork, and it proves itself with the same suite D1 passes.
-- Core loses its SQLite code; the D1 dialect owns it. Mantle Cloud stays D1-only.
-- Manifests name Mantle's extensions explicitly (`mantle.search(...)`), and anything unqualified reads as PostgreSQL.
-- A plan is dialect-specific; moving a service to another engine is a recompile.
-- The interface is new public surface, versioned with the plan.
-
-## Alternatives
-
-- **ISO or ANSI SQL as the canonical language.** There is no authoritative parser for it, and PostgreSQL is the implementation closest to it. Naming the standard would still mean choosing one vendor's parser.
-- **Substrait.** A cross-engine relational-plan standard, but it describes plans after planning, it is not authored, and neither D1 nor most transactional engines consume it; every dialect would still render SQL from it.
-- **Mantle's own neutral tree.** A new language to specify and teach; ADR-0034 left the JSON grammar for SQL for this reason.
-- **A `host` option in the open-source CLI.** A host is a platform preset, not an engine; making it a CLI option ties platform files to the generator. Rejected in favor of the dialect interface. *Superseded by ADR-0036: presets are written once and owned by the application, so the generator owns no platform file after the first run.*
-- **Bare `search()` and `near()`.** Shorter, but they read as PostgreSQL functions and can collide with an engine's or an extension's own.
-
-## How to apply
-
-1. This ADR, then the moves with no behavior change: the shared front end out of the SQL compiler, the D1 subset check, rendering, policy lowering and convergence into `@aotter/mantle/d1`, and the interface types.
-2. `mantle.*` names, the dialect recorded in the plan, and `dialect` in the config.
-3. `StoreSelect.search`, and Admin's search box on it.
-4. `createMantleAuth`'s SQL onto Better Auth's Kysely instance.
-5. The compliance suite over the dialect interface, run on the D1 dialect.
-
-## Conformance cases
-
-1. The shared front end refuses, with a position: DDL, `SET`, transaction control, `COPY`, `DO`, `set_config('mantle.…', …)`, a Mantle or identity-provider table in `FROM` and in a scalar subquery, an undeclared `input`, and an unknown `mantle.*` or `auth.*` function.
-2. An unqualified `search(…)` is not Mantle's: the D1 dialect refuses it as an unsupported function, and `mantle.search(…)` behaves as ADR-0034 decision 9's `search` did.
-3. A plan compiled for another dialect is refused at boot; a config without `dialect` compiles for D1.
-4. `StoreSelect.search` on D1: a three-character Chinese substring matches, a two-character one falls back to a scan, another owner's rows are absent.
-5. The D1 dialect passes the compliance suite on local D1, and every public barrel except the moved internals exports what it did before.
-
-## Implementation status
-
-Proposed.
-
-> **Amendment (How to apply 1, 2026-09-30):** The move, and the names it changes. Behavior does not change.
-> - `@aotter/mantle` no longer exports `sqliteStorage`, `runMigrations`, `readStoreInstanceId` or `Migration`; `@aotter/mantle/d1` exports them unchanged. `@aotter/mantle/spec` no longer exports the D1 allowlist (`validateProgram`, `validateIr`, `KEYS_SRC`, `KEYS`, `BARE`, `ENUM`, `FUNCS`, `TRUNC_UNITS`, `EXTRACT_FIELDS`, `MAX_RADIUS_M`, `MAX_NEAR_K`, `SQLITE_ONLY_KEYWORDS`, `SYSTEM`). `@aotter/mantle/d1/compile` exports `validateProgram` as `accepts`, and the runtime reaches the same validator as the dialect's `check`, which throws the first refusal instead of returning a list (D1 stops at the first).
-> - The shared front end (`compileSql`) calls D1's `accepts` directly until `dialect` in the config selects one (How to apply 2).
-> - `PreparedMantleStorage` gains `dialect`. Core calls four things on it: `check` (the IR validator, on every program), `lowering` (what the policy rewriter leaves to the engine: input and literal casts, `search`, `near` and the other Mantle functions, kept calls, subquery links, new ids, and extra wrapper columns such as D1's `_rid`), `bind` (the binds the lowering adds) and `codec`. These types stay internal until How to apply 5 makes the interface public. `name`, `version` and `policy` arrive with the plan's dialect record in How to apply 2.
-> - Still to move: `auth` calls `runMigrations` and `readStoreInstanceId` (How to apply 4), `@aotter/mantle/testing` reaches D1 internals (How to apply 5), and Store's TTL sweep pages on D1's `_rid` column.
-
-> **Amendment (How to apply 2, 2026-09-30):** The names, the plan record and the config key.
-> - Mantle's functions are spelled `mantle.search`, `mantle.search_rank`, `mantle.near` and `mantle.distance` everywhere: manifests, the D1 allowlist and its lowering. An unqualified `search(…)` is refused as an unsupported function.
-> - The plan records `dialect: { name, version }`, inside the fingerprint. Its version is 6 for all of 0.2.0: the plan version changes only between released formats (0.1.x shipped 5), never within an unreleased one, because no plan of an unreleased shape exists to refuse. Development had moved it to 7 and 8; both are gone. The D1 dialect is `{ name: "@aotter/mantle/d1", version: "1" }`; its version changes when what a D1 plan means changes. `dialect` moves from `PreparedMantleStorage` to `MantleStorageAdapter`, and the runtime's `MantleDialect` gains `name` and `version`, so boot compares them with the plan before it converges anything and refuses a mismatch with `PLAN_FINGERPRINT_MISMATCH`, as it refuses another plan version.
-> - `@aotter/mantle/d1/compile` exports `name`, `version` and `accepts`. `@aotter/mantle/spec` exports `SqlDialect`, the compile side's type, and `compileSql`, `compileLinkedPlan` and `compilePlan` take one as their last argument, D1 when omitted.
-> - `mantle.config.json` version 2 takes an optional `dialect`, a package name (never a path). `mantle generate` resolves `<dialect>/compile` from the project with `require.resolve`, so the package's `exports` must give it a `default` (or `require`) condition, imports it and compiles with it; absent, or `@aotter/mantle/d1`, it is the built-in dialect. The config's other keys are unchanged.
-> - The shared front end refuses decision 4's list before `accepts`: statements other than `SELECT`, `INSERT`, `UPDATE`, `DELETE` and `MERGE`; a relation that is not a declared Schema or a CTE of the statement, or that names a schema; an undeclared `input.<name>`; a `mantle.*` or `auth.*` function that decision 2 does not define; and the session, server and outside-access functions (`set_config` and `current_setting` of a `mantle.*` setting or of a name that is not a literal, advisory locks, file and large-object access, `dblink`, backend signals, `pg_reload_conf`, `pg_notify`, sequences, and the functions that run SQL text: `query_to_xml` and its kin, `ts_stat`, `ts_rewrite`); `SELECT … INTO`. Function names are compared as PostgreSQL resolves them (a catalog prefix and `pg_catalog.` dropped); a relation that names a CTE in scope, by PostgreSQL's scoping, is tagged `cte`.
-> - Lowering stays on the runtime side: Store builds its JSON queries at run time (and `StoreSelect.search`, How to apply 3), so they need it there. The compile side's `lower` and `outputs` are not built; `viewOutputs` reads a View's row type for every dialect.
-
-
-> **Amendment (How to apply 4, 2026-09-30):** Decision 9 as built. Better Auth does not expose its Kysely instance, and reaching it would add a dependency (`@better-auth/kysely-adapter`) for a handful of statements, so `createMantleAuth` does not use Kysely:
-> - Its tables come from Better Auth's own `getMigrations(...).runMigrations()`; the `_mantle_migrations` ledger row for them is gone. A digest of Better Auth's table definitions in `_mantle_boot_state` (`auth-schema`) lets a prepared database skip the introspection, so a warm isolate reads one row. Better Auth's statements are not idempotent, so an isolate that races another retries (up to five times) until the other has finished. Mantle adds one index, `user_role_idx`, in portable SQL.
-> - Revoking a consent deletes the consent and revokes its tokens in one batch first, so no new code is issued for it without a prompt; then its pending authorization codes are found and deleted through Better Auth's adapter (`findMany` paged past the rows it keeps, `deleteMany`), which reads the JSON value in code instead of `json_extract`.
-> - What Better Auth's API has no call for (the bootstrap owner's guarded promotion, member paging, the invite guard, linked accounts, consents and the MCP grant check) stays one statement each over the `DatabaseDriver`, in portable SQL: camelCase names and `"user"` quoted, binds numbered, a write's count read from `RETURNING`, no engine functions. The two guarded writes stay single statements, which SQLite and D1 serialize, so their guards hold there; on an engine with concurrent writers (PostgreSQL under READ COMMITTED) that engine's driver must run them serializable, or two first sign-ups could both become owner.
-> - The session cache's namespace (`_mantle_boot_state`'s instance id) is read by auth itself. `@aotter/mantle/d1` no longer exports `runMigrations`, `readStoreInstanceId` or `Migration`; it exports `sqliteStorage` only. `auth` no longer reaches `d1`.
-
-> **Amendment (How to apply 5, 2026-09-30):** The compliance suite over the dialect interface.
-> - `runStorageConformance({ create, compile })`: `create` returns, per case, a fresh database as `{ storage, driver, cleanup }`, where `storage` is the dialect's `MantleStorageAdapter` (its runtime side: `prepare` converges the fixture Schemas and returns the executor, and `dialect` carries the check, lowering and codec) and `driver` is the same database for the fixture's own seeding and reads, in plain SQL over the storage layout of ADR-0033, with typed values (timestamps, numerics, JSON) through the dialect's codec. `prepare` receives a whole `RuntimePlan` for the fixture Schemas. `compile` is the dialect's compile side, D1 when omitted. A corpus item the compile side refuses as unsupported (`SQL_UNSUPPORTED`, `SQL_FUNCTION`, `SQL_TYPE`) passes by that refusal, and the check names it; any other refusal, and a refusal in any other case, fails.
-> - The generic cases are requisition, stock, report View, before hook, policy (the relation-position probe), search and places, the printer corpus (rows, not SQL text) and Store (including `StoreSelect.search`). What depends on how D1 spells or stores things (the printed SQL, SQLite's query plan, FTS5's trigram floor, D1's `meta.changes`, the storage encodings of the types case, and that D1 accepts the whole corpus) is D1's own test, `test/d1/dialect.test.ts`. `testing` no longer reaches `d1`.
-> - **Not yet dialect-neutral: computed outputs.** Store decodes a View's output column only when it reads a Schema field unchanged; a computed typed value (`now()`, `date_trunc`, a numeric literal, a `CAST` to bool) reaches the caller as the engine stores it. So the corpus rows that compute such values, and the whole types case (integer division, exact numerics, the interval boundary, `round`, `date_trunc` and `extract` across daylight saving), expect D1's encodings, and the types case runs only in D1's test. Typing each output column in the compile side and decoding it in Store is the prerequisite for a second dialect; with it, the types case's View checks return to the generic suite and only its storage reads stay in D1's test.
-
-> **Amendment (PostgreSQL dialect, #1293, 2026-10-02):** The second dialect ships in this repository, not outside it: `@aotter/mantle/postgres` and `@aotter/mantle/postgres/compile`. This amends the Context's "an enterprise PostgreSQL dialect is the first one written outside it"; decision 5's "Cloud Control accepts only D1 plans" stands for now. A consumer service behind Cloudflare Hyperdrive wants PostgreSQL from day one, and a dialect inside the repository is the one the compliance suite keeps honest.
-> - **What it accepts** (*replaced by ADR-0037: the PostgreSQL dialect runs the reference profile, and D1 the base subset*): D1's subset (decision 6) less SQLite's own functions (`typeof`, `hex`, `json_extract`, `json_set`, `json_insert`, `json_remove`), refused with a position. A PostgreSQL manifest therefore compiles for D1 too. `mantle.near`, `mantle.distance` and `mantle.search_rank` are lowered without an extension (a range and a haversine; an occurrence count instead of bm25).
-> - **Values:** columns have native types. The executor asks node-postgres for every value as text and decodes it by the column's type OID, so computed outputs are typed wire values, which is the prerequisite How to apply 5 named, for this dialect. The printer corpus therefore accepts, per item, D1's encoding or the typed wire values (`typedRows`).
-> - **Core:** the `now` and `cutoff` binds and the TTL sweep's bind go through `codec.encode("timestamptz", …)` (the identity on D1), and `PolicyLowering.system` lets a dialect type `now()`'s bind (PostgreSQL cannot infer `$1 - interval '1 hour'`). D1's output is unchanged.
-> - **Concurrency:** every write batch is one `SERIALIZABLE` transaction, retried on 40001/40P01, which is the "engine's driver must run them serializable" that How to apply 4 asked for, for auth's guarded writes too (`pgDatabaseDriver`, `pgPool` for Better Auth).
-> - **Driver:** structural node-postgres types; `connect` opens a client per operation, as Workers and Hyperdrive need. Statements are sent one at a time; a pipelining driver is the upgrade if round trips show.
-> - **Same results as D1 where Mantle decides them:**
->   - Text columns use `COLLATE "C"`.
->   - A default NULL order is stated as Core pages Views: first ascending, last descending.
->   - Every transaction pins `DateStyle`, `IntervalStyle`, `extra_float_digits` and `TimeZone`.
->   - Reads run in read-only transactions, so Hyperdrive's cache never answers them.
->
->   Elsewhere the meaning is PostgreSQL's (`LIKE` case, division by zero, `->>` returning text).
-> - **Not yet:** site settings and media (D1 only), a trigram or GiST index behind `searchableFields` and `format: geo`, and `mantle generate --check`'s storage dry run (D1 only).

package/docs/adr/0036-host-and-dialect.md

@@ -1,65 +0,0 @@
-# ADR-0036: Host and dialect are the two axes of a project
-
-**Status:** Accepted for 0.2.0. Amends ADR-0035 decision 5 and replaces its rejected alternative "a `host` option in the open-source CLI". Amended by [ADR-0038](0038-bun-native-host.md).
-
-**Date:** 2026-10-02
-
-**Related:** ADR-0032 decision 6, ADR-0035, #1296
-
-## Context
-
-ADR-0035 made the SQL engine a dialect module, but left the CLI with one preset: Cloudflare over D1. A service that wants
-PostgreSQL behind Hyperdrive wrote its composition by hand, and the CLI rejected a `host` key as a 0.1.x config. ADR-0035
-rejected a `host` option because it "ties platform files to the generator". Two facts change that: the preset is already
-written once and owned by the application (ADR-0032 decision 6), so the generator owns no platform file after the first
-run; and a second built-in dialect exists, so "which engine on which platform" is a real choice.
-
-## Decision
-
-### 1. Two axes
-
-- `host`: where the service runs. `cloudflare` (the default) or `none`. A Bun host (`bun`) is added by its own ADR amendment
-  with `@aotter/mantle/bun`.
-- `dialect`: the SQL engine. `sqlite` (alias `d1`, the default), `postgres`, or a dialect package (ADR-0035 decision 5).
-- The driver follows from the pair (D1 binding, Hyperdrive and `pg`, ...); it is not a third option.
-- The host never enters the plan or its fingerprint. The dialect does, as ADR-0035 decision 5 says. The SQLite dialect keeps
-  recording `@aotter/mantle/d1`, so the rename changes no existing plan.
-
-### 2. Chosen once
-
-- `mantle generate --host <host> --dialect <name>` writes both into `mantle.config.json` on the first run, and the preset
-  for the pair if one exists.
-- A rerun that asks for another host or dialect is refused (exit 2), as another identity is: `src/service.ts` is the
-  application's, and another engine is a data move. The config is changed deliberately.
-
-### 3. Presets per pair
-
-| | `cloudflare` | `none` |
-|---|---|---|
-| `sqlite` | the D1 preset | plan and types only |
-| `postgres` | the Hyperdrive preset: `postgresStorage`, `pgPool` and `pgDatabaseDriver` over a client per operation; a `hyperdrive` binding with a placeholder id and a `localConnectionString` | plan and types only |
-| a package | plan and types only | plan and types only |
-
-- `host: none` is the escape hatch for any platform the presets do not cover (ChatGPT Sites keeps using the Cloudflare
-  preset with `identity: custom`): the application calls `createMantle` with a storage adapter itself.
-- The Cloudflare cron gate applies to host `cloudflare` only.
-- A pair's packages are checked like a feature's (`pg` for `postgres` on `cloudflare`).
-- Generate warns, never writes, when `wrangler.jsonc` lacks the pair's binding, or still holds the Hyperdrive placeholder.
-
-## Consequences
-
-- PostgreSQL on Cloudflare is one flag, and its preset is tested end to end (typecheck, and a Worker booted against
-  PostgreSQL through Hyperdrive's local connection).
-- The 0.1.x detection is `version: 1`, no longer any `host` key.
-- `--check --database`'s storage dry run stays the SQLite dialect's.
-
-## Alternatives
-
-- **A third `driver` axis.** Every built-in pair has one sensible driver; a third choice is configuration without a use.
-- **Host in the plan.** A plan runs the same on any host; recording it would make a host move a recompile for nothing.
-- **Renaming the recorded dialect to `@aotter/mantle/sqlite`.** Every existing plan would stop booting for a name.
-
-## Implementation status
-
-Implemented in #1296: the axes, the aliases, the Hyperdrive preset, `host: none`. The Bun host (`host: bun`, both dialects)
-is ADR-0038, which amends decisions 1 and 3.

package/docs/adr/0037-postgresql-is-the-reference-dialect.md

@@ -1,125 +0,0 @@
-# ADR-0037: PostgreSQL is the reference dialect; D1 runs a subset
-
-**Status:** Accepted for 0.2.0. Amends ADR-0034 decisions 2 and 8 and ADR-0035 decisions 1, 4 and 6, and replaces the
-PostgreSQL dialect amendment's "What it accepts".
-
-**Date:** 2026-10-02
-
-**Related:** ADR-0014 (Cloud and tenants), ADR-0034, ADR-0035, ADR-0036, #1293
-
-## Context
-
-ADR-0035 decision 1 made Mantle SQL PostgreSQL's syntax and meaning, and said a PostgreSQL dialect "can accept nearly all of
-PostgreSQL". The PostgreSQL dialect shipped as D1's subset less SQLite's functions, so a PostgreSQL manifest also compiled for
-D1. That kept engines interchangeable, but it held services that chose PostgreSQL to SQLite's limits: no `WITH`, no window
-frames, no `FILTER`, no jsonb operators. A View that needed one rule in many places (a plan's visible window, for example)
-had to repeat it.
-
-Services that pick PostgreSQL pick it for good (Swolhalla runs on PostgreSQL from day one). The subset should be D1's
-limitation, not every dialect's.
-
-Two kinds of author write Mantle SQL, and the allowlist means something different to each:
-- **A self-hosted service** (enterprise, Swolhalla): the author also operates the database and can reach it directly. The
-  allowlist does not stop malice there; it stops mistakes, above all SQL (often written by an AI) that would read another
-  member's rows.
-- **Mantle Cloud:** tenants are third parties. Today each tenant has its own D1 and its own Workers for Platforms script, and
-  the tenant's `ref` handlers run in that script with the raw `DB` binding (aotter/mantle-home `apps/control/src/deploy/provider.ts`).
-  Isolation between tenants is the separate database, not the allowlist. A PostgreSQL cell shared by thousands of tenants
-  behind one Hyperdrive config (one connection string, one role) cannot isolate tenants that way.
-
-## Decision
-
-### 1. One reference allowlist, PostgreSQL's
-
-- Mantle SQL's allowlist lives in Core (`src/core/sql/allowlist.ts`): the walker and two profiles, **base** (what every
-  dialect runs, ADR-0034's subset) and **reference** (base plus decision 2). The PostgreSQL dialect accepts the reference
-  profile. The D1 dialect accepts base.
-- A construct D1 refuses but the reference accepts is refused with the reason "needs the PostgreSQL dialect", so an author
-  knows it is an engine limit, not a mistake.
-- The allowlist stays closed on every dialect. "PostgreSQL first" means a larger closed list, never "anything PostgreSQL
-  parses": PostgreSQL's catalog has thousands of functions, and one that runs SQL text (`query_to_xml`) or reads files
-  bypasses every policy.
-- Each construct added to the reference profile lands with: the walker's rule, the policy rewriter's handling (a new relation
-  position has a probe in the compliance suite, ADR-0034 decision 8), a compliance case on PostgreSQL, and a D1 refusal test.
-
-### 2. The reference profile, first wave
-
-| Construct | Rule |
-|---|---|
-| `WITH` and `WITH RECURSIVE` | Every CTE body is a `SELECT`. A data-modifying CTE (`WITH x AS (DELETE …)`) is refused. Each Schema read inside a body is wrapped like any other (relation position `cte`); a reference to the CTE is not. A reference is a CTE only by PostgreSQL's scope (a body sees its earlier siblings, every sibling under `RECURSIVE`); the allowlist and the rewriter both check that scope at runtime, because a `cte` tag on any other name would make PostgreSQL read the table, or the catalog view, of that name unwrapped. |
-| `UNION`, `UNION ALL`, `INTERSECT`, `EXCEPT` | Inside a CTE body or a subquery only, never a View's own top-level `SELECT`, because Core's paging and deterministic order act on that `SELECT`. Each branch is wrapped (position `setop`). |
-| `DISTINCT ON` | Inside a CTE body or a subquery only, for the same reason: Core's paging rewrites the top-level order, which decides which row `DISTINCT ON` keeps. |
-| `LATERAL` subqueries | `JOIN LATERAL (…) x ON …` or `, LATERAL (…) x`. The subquery is wrapped as any FROM subquery. |
-| Window frames | `ROWS` and `RANGE` with literal offsets; functions `avg`, `min`, `max`, `lag`, `lead`, `first_value`, `last_value` and `dense_rank` gain `OVER`. |
-| Aggregates | `FILTER (WHERE …)`, `ORDER BY` inside an aggregate, `string_agg`. |
-| jsonb | Operators `->`, `#>`, `#>>`, `@>`, `<@`, `?`, `?|`, `?&`; `jsonb_build_object`, `jsonb_build_array`, `jsonb_strip_nulls`, `jsonb_agg`, `jsonb_object_agg`, `to_jsonb`, `jsonb_typeof`; casts to `jsonb`. |
-| Text | `ILIKE`, `NOT ILIKE`, and the regular-expression operators `~`, `~*`, `!~`, `!~*`; `split_part`. |
-| Numbers | `greatest`, `least`, `floor`, `ceil`, `sqrt`, `power`. |
-| Time | `AT TIME ZONE`; `date_trunc` adds `minute` and `quarter`; `extract` adds `minute`, `quarter`, `week`, `isoyear`, `isodow`, `doy` and `epoch`. |
-| Casts | A cast of any expression to `int4`, `int8`, `numeric(p, s)`, `date` or `timestamptz` (D1 keeps its literal-only rule, which exists because SQLite truncates and stores time as integers). A bare text literal compared with a date-time, date or boolean column is PostgreSQL's cast, so the D1 refusal of it does not apply. |
-
-Still refused everywhere: data-modifying CTEs, set-returning functions in FROM other than `json_each`, `generate_series`,
-`FOR UPDATE`, `OFFSET`, `RIGHT` and `FULL` joins, and every function in ADR-0035 decision 4's list. Each needs its own
-amendment.
-
-### 3. A View may read another View
-
-- A View's or a Procedure's `FROM` may name an **internal** View (`surface: internal`) that declares no `input` and no `requires`, by its name
-  with `-` written `_` (`free-window` is `free_window`). A Schema of the same name keeps it: the name reads the Schema.
-- The compiler inlines the referenced View's compiled `SELECT` as a `FROM` subquery before the dialect sees the tree. Policy
-  and every check then apply to it as to any subquery. Nothing reaches the runtime that a hand-written subquery could not.
-- References are resolved in dependency order; a cycle is a compile error.
-- On every dialect. This is how a rule used by many Views is written once.
-
-### 4. A narrowing hook
-
-- `postgresStorage` and `sqliteStorage` take `restrict?: (plan, context) => readonly SqlDiagnostic[]`. Core runs it on every
-  program's IR after the dialect's own check, at runtime (ADR-0035: the runtime never trusts an IR). Since the SQL compile
-  cache, an accepted verdict is cached with the compiled program, so `restrict` runs again only for a program Core compiles anew. It can only refuse.
-- It is for an operator that runs other people's plans. A self-hosted service does not set it.
-
-### 5. Every PostgreSQL transaction has a statement timeout
-
-- The PostgreSQL driver pins `statement_timeout` with `SET LOCAL` in every transaction, as it pins `DateStyle` and `TimeZone`:
-  10 seconds unless `postgresStorage({ statementTimeoutMs })` says otherwise. A recursive CTE or a regular expression that runs
-  away ends there.
-- A database role used by Mantle is not a superuser and holds no file or server privilege (`pg_read_server_files`,
-  `pg_execute_server_program`). The handbook says so; Mantle does not check.
-
-### 6. Mantle Cloud's boundary
-
-- **D1 (today):** the tenant's database is the isolation unit, so the tenant's code may hold its binding, as it does.
-- **PostgreSQL (when Cloud adds it):** the cell's connection never enters a tenant script. The platform owns the Hyperdrive
-  binding and runs a Store service the tenant script calls (as it calls `MAIL` and `MEDIA`). The service validates every IR
-  with the dialect, then a Cloud `restrict` profile, then the policy rewriter, and runs each transaction as the tenant with
-  `SET LOCAL ROLE` and the tenant's `search_path`. The cell role is a member of every tenant role; a tenant role holds only its
-  schema. The allowlist is then a real boundary, and PostgreSQL's privileges are the second one.
-- The Cloud profile and the Store service are aotter/mantle-home's to build; Core provides decision 4.
-
-## Consequences
-
-- A PostgreSQL manifest may no longer compile for D1. Moving from PostgreSQL to D1 can need a rewrite; moving from D1 to
-  PostgreSQL still needs none.
-- The reference profile grows by amendment, one construct at a time, each with its probe.
-- Swolhalla's free-plan window is one internal View every other View reads.
-
-## Alternatives
-
-- **Accept everything PostgreSQL parses and deny a list of dangerous functions.** A denylist is never complete: an extension,
-  a new catalog function or a server upgrade adds one. Rejected.
-- **A Schema-level visibility predicate** (a second `scope`, injected by the rewriter). It solves the repeated window, but only
-  for row filters; View references solve it and every other shared rule.
-- **Native row-level security on PostgreSQL** (ADR-0035 decision 3's `policy: "native"`). The right second layer for Cloud
-  (decision 6); a larger change than this ADR needs.
-
-## Conformance cases
-
-1. Each decision 2 construct gives PostgreSQL's result on the PostgreSQL dialect, and D1 refuses it with "needs the
-   PostgreSQL dialect".
-2. The relation-position probe covers `cte` and `setop`, and a Schema read inside a recursive CTE, a `UNION` branch and a
-   `LATERAL` subquery never shows another owner's rows. An IR whose `cte` tag names a relation outside PostgreSQL's CTE
-   scope (the CTE's own body, a later sibling, a catalog view) is refused at runtime.
-3. A View's top-level `UNION` or `DISTINCT ON`, and a data-modifying CTE, are refused with a position.
-4. A View reads an internal View; the reference to a View with an input, a public View or a cycle is refused.
-5. A `restrict` that refuses a program stops it at runtime with its diagnostic.
-6. A statement past `statement_timeout` fails with `RESOURCE_UNAVAILABLE`, and nothing is written.