@syncular/typegen 0.2.1 → 0.3.0

package/README.md

@@ -4,7 +4,7 @@ SQL migrations + one manifest → neutral schema IR (JSON) → generated TS
 module (REVISE B5). This file is the authoritative contract for the three
 tool-level formats: the manifest, the IR, and the SQL subset. Wire-protocol
 semantics (column types, scope patterns, schema-version gating) live in
-[`../../SPEC.md`](../../SPEC.md) §2.4, §3.1, §1.5 — this document never
+[`../../docs/SPEC.md`](../../docs/SPEC.md) §2.4, §3.1, §1.5 — this document never
 overrides it.
 
 ```sh
@@ -67,7 +67,9 @@ schema guide and suggests `syncular init`.
 |---|---|---|
 | `manifestVersion` | yes | Must be `1`. Format growth happens by bumping this, not by tolerating unknown keys. |
 | `migrations` | no (default `./migrations`) | Directory of `NNNN_name/up.sql` migrations, relative to the manifest. |
-| `queries` | no (default `./queries`) | Directory of `.sql` named-query files (see §6). Only read when some output requests a queries file. |
+| `queries` | no (default `./queries`) | Directory of `.sql` / `.syql` named-query files (see §6). Only read when some output requests a queries file. |
+| `naming` | no (default `"camel"`) | §5-of-DESIGN-queries casing: `"camel"` maps snake_case SQL names to camelCase in generated row types/params (projections lower with `AS` aliases so runtime keys match); `"preserve"` keeps SQL-truth names. Collisions/keyword hazards are generate-time errors. |
+| `queryBackend` | no (default `"neutralize"`) | `.syql` conditional lowering: `"neutralize"` (one guarded statement), `"variants"` (enumerate per provided-combination whenever eligible), `"auto"` (enumerate at ≤ 2 optional groups). The per-query `variants` knob always forces. |
 | `output.ir` | no (default `./syncular.ir.json`) | IR output path, relative to the manifest. |
 | `output.module` | no (default `./syncular.generated.ts`) | Generated TS module path, relative to the manifest. |
 | `output.queries` | no | Opt-in TS named-queries output path (see §6). A sibling `.ts` file. |
@@ -228,7 +230,7 @@ duplicate or unknown-column index; trailing clauses (`STRICT`).
 
 **`DEFAULT` literals are accepted and ignored**: typegen extracts the
 schema *shape*; executing migrations (where defaults matter) is the
-host's job. Rejecting them would make real v1-style migrations unusable
+host's job. Rejecting them would make real-world migrations unusable
 as input; recording them is not needed by any emitter today.
 
 ## 4. Generated-module contract
@@ -318,10 +320,23 @@ null/nil (fail-soft at the decode boundary, not a crash).
 ## 6. Named queries (the `.sql` → typed-function tier)
 
 The **named-query** tier is syncular's cross-platform answer to sqlc /
-SQLDelight: you write a `.sql` file, and typegen transpiles it into a typed
+SQLDelight: you write a query file, and typegen transpiles it into a typed
 function on every platform — killing query↔type drift *by construction*. It is
-the type-safe **read** tier; Kysely stays the TS **dynamic** tier, and raw
-`query(sql, params)` stays the escape hatch.
+the type-safe **read** tier; raw `query(sql, params)` — guarded read-only —
+stays the escape hatch for queries built at runtime.
+
+Two frontends feed one pipeline (DESIGN-queries.md is the design of record):
+**`.sql`** — plain SQL + `:params`, the compatibility floor, documented in
+this section — and **`.syql`** — the DSL container (declared optional params
+with auto-guarded conjuncts, `from+to?` groups, `?: flag` guards with
+`if (…) { … }`, file-scoped `@fragments`, `orderBy`/`limit` knobs, and the
+opt-in `variants` enumeration backend), which lowers at generate time to the
+same checked plain SQL. Tooling: `syncular generate --print <name>` (the
+lowered SQL), `syncular fmt` (canonical `.syql` style), `syncular lsp`
+(diagnostics/hover/definition over stdio). Casing follows the manifest
+`naming` mode (§1): generated rows/params are camelCase and projections are
+`AS`-aliased so runtime keys match; `mutate` accepts camel and snake keys
+both.
 
 **File & naming convention.** Named queries live in a `queries/` directory
 next to `migrations/` (override with the top-level `"queries"` manifest key).
@@ -428,7 +443,7 @@ stripped and whitespace collapsed to a clean one-line string.
 **The tables dependency set** (for React's exact invalidation). Each query
 emits a `tables` set — the FROM/JOIN tables it reads, resolved against the IR
 and *validated by the same SQLite `prepare()`* (an unknown table would have
-thrown). This feeds `useSyncQuery`'s `{tables}` option so a named query
+thrown). This feeds `useRawSql`'s `{tables}` option so a named query
 re-runs exactly when a depended-on table invalidates. **Boundary**:
 `bun:sqlite` exposes no authorizer and no statement table-list, and EXPLAIN
 opcodes are fragile — so the set is derived by scanning every `FROM`/`JOIN`
@@ -451,12 +466,12 @@ the same `fromRow` mapping rules (0/1 booleans, `{"$bytes":"<hex>"}` bytes) as
 the schema structs. `--check` gates every queries file byte-exactly, like the
 schema files; each carries the DO-NOT-EDIT header + IR hash.
 
-**React helper.** `@syncular/react` exports `useNamedQuery(query, params?)`,
-which takes the TS `NamedQuery` descriptor and reuses `useSyncQuery`'s
+**React helper.** `@syncular/react` exports `useQuery(query, params?)`,
+which takes the TS `NamedQuery` descriptor and reuses `useRawSql`'s
 invalidation machinery with the descriptor's exact `tables` set:
 
 ```ts
 import { listTodosQuery } from './syncular.queries';
-const { rows } = useNamedQuery(listTodosQuery, { listId });
+const { rows } = useQuery(listTodosQuery, { listId });
 //      ^ ListTodosRow[] — typed by the query's own projection
 ```

package/dist/cli.js

@@ -15,23 +15,35 @@
  * The generate CONTRACT (inputs, outputs, byte-exact `--check`) is unchanged;
  * this file only adds `init`, `--watch`, and friendlier errors.
  */
-import { watch } from 'node:fs';
+import { existsSync, readFileSync, watch, writeFileSync } from 'node:fs';
 import { resolve } from 'node:path';
-import { checkOutputs, generate, writeOutputs } from './generate.js';
+import { formatSyql } from './fmt.js';
+import { checkOutputs, generate, loadQueries, writeOutputs } from './generate.js';
 import { InitError, initProject } from './init.js';
-import { MANIFEST_FILENAME } from './manifest.js';
+import { runLspStdio } from './lsp.js';
+import { MANIFEST_FILENAME, parseManifest } from './manifest.js';
 const DOCS_HINT = 'See the schema guide: https://github.com/bkniffler/syncular (guide-schema), ' +
     'or run `syncular init` to scaffold a starter manifest + migration.';
 const USAGE = `usage: syncular <command> [options]
 
 commands:
   generate   build the IR + typed module from syncular.json + migrations/
+  fmt        format .syql query files canonically (one style, no options)
+  lsp        run the .syql language server over stdio (editor tooling)
   init       scaffold a starter syncular.json + migrations/0001_initial
 
 generate options:
   --manifest-dir <dir>   directory holding syncular.json (default: .)
   --check                fail unless generated files are byte-exactly fresh
   --watch                regenerate on file change (Ctrl-C to stop)
+  --print <name>         print one named query's lowered, checked SQL
+                         (params, tables, knob variants) and exit
+
+fmt options:
+  [files…]               .syql files to format (default: the manifest's
+                         queries directory, recursively)
+  --manifest-dir <dir>   directory holding syncular.json (default: .)
+  --check                fail unless every file is already canonical
 
 init options:
   --manifest-dir <dir>   directory to scaffold into (default: .)`;
@@ -40,7 +52,12 @@ function fail(message) {
     process.exit(1);
 }
 function parseGenerateArgs(argv) {
-    const args = { manifestDir: '.', check: false, watch: false };
+    const args = {
+        manifestDir: '.',
+        check: false,
+        watch: false,
+        print: undefined,
+    };
     for (let i = 0; i < argv.length; i++) {
         const arg = argv[i];
         if (arg === '--check') {
@@ -56,6 +73,13 @@ function parseGenerateArgs(argv) {
             args.manifestDir = value;
             i += 1;
         }
+        else if (arg === '--print') {
+            const value = argv[i + 1];
+            if (value === undefined)
+                fail('--print requires a query name');
+            args.print = value;
+            i += 1;
+        }
         else {
             fail(`unknown argument ${JSON.stringify(arg)}\n${USAGE}`);
         }
@@ -63,8 +87,52 @@ function parseGenerateArgs(argv) {
     if (args.check && args.watch) {
         fail('--check and --watch cannot be combined');
     }
+    if (args.print !== undefined && (args.check || args.watch)) {
+        fail('--print cannot be combined with --check/--watch');
+    }
     return args;
 }
+/** `generate --print <name>`: "what does this actually run" — the lowered,
+ * checked SQL for one named query, plus its params/tables/knob variants. */
+function runGeneratePrint(args, name) {
+    let result;
+    try {
+        result = generate(args.manifestDir);
+    }
+    catch (error) {
+        fail(friendlyGenerateError(error));
+    }
+    const query = result.queries.find((q) => q.name === name);
+    if (query === undefined) {
+        const known = result.queries.map((q) => q.name).join(', ');
+        fail(`no named query ${JSON.stringify(name)} — this manifest defines: ${known.length > 0 ? known : '(none)'}`);
+    }
+    console.log(`-- ${query.name} (${query.file})`);
+    console.log(`-- tables: ${query.tables.join(', ')}`);
+    if (query.params.length > 0) {
+        console.log('-- params:');
+        for (const param of query.params) {
+            const marks = [
+                param.optional === true ? 'optional' : 'required',
+                ...(param.flag === true ? ['flag'] : []),
+                ...(param.group !== undefined ? [`group ${param.group}`] : []),
+            ].join(', ');
+            console.log(`--   :${param.name} ${param.type} (${marks})`);
+        }
+    }
+    console.log(query.sql);
+    if (query.orderBy !== undefined) {
+        console.log('-- orderBy variants (each checked against the schema):');
+        const base = query.positionalSqlBase ?? '';
+        for (const col of query.orderBy.allowed) {
+            const isDefault = col.name === query.orderBy.defaultColumn;
+            console.log(`--   ${col.langName}: ${base} order by ${col.name} asc|desc${query.positionalLimitTail ?? ''}${isDefault ? '   (default)' : ''}`);
+        }
+    }
+    if (query.limit !== undefined) {
+        console.log(`-- limit: bound value, default ${query.limit.default ?? query.limit.max}${query.limit.max !== undefined ? `, clamped to ${query.limit.max}` : ''}`);
+    }
+}
 function parseManifestDir(argv) {
     let manifestDir = '.';
     for (let i = 0; i < argv.length; i++) {
@@ -142,10 +210,85 @@ function runGenerateWatch(args) {
         timer = setTimeout(safeRun, 50);
     });
 }
+/** `syncular fmt`: canonicalize `.syql` files in place (or `--check`). */
+function runFmt(argv) {
+    let manifestDir = '.';
+    let check = false;
+    const files = [];
+    for (let i = 0; i < argv.length; i++) {
+        const arg = argv[i];
+        if (arg === '--check')
+            check = true;
+        else if (arg === '--manifest-dir') {
+            const value = argv[i + 1];
+            if (value === undefined)
+                fail('--manifest-dir requires a value');
+            manifestDir = value;
+            i += 1;
+        }
+        else if (arg.startsWith('--')) {
+            fail(`unknown argument ${JSON.stringify(arg)}\n${USAGE}`);
+        }
+        else {
+            files.push(arg);
+        }
+    }
+    let targets;
+    if (files.length > 0) {
+        targets = files.map((f) => resolve(f));
+    }
+    else {
+        const manifestPath = resolve(manifestDir, MANIFEST_FILENAME);
+        if (!existsSync(manifestPath)) {
+            fail(`no files given and no ${MANIFEST_FILENAME} in ${manifestDir} — pass .syql files or --manifest-dir`);
+        }
+        const manifest = parseManifest(JSON.parse(readFileSync(manifestPath, 'utf8')));
+        const queriesDir = resolve(manifestDir, manifest.queries);
+        targets = loadQueries(queriesDir)
+            .filter((q) => q.file.endsWith('.syql'))
+            .map((q) => resolve(queriesDir, q.file));
+    }
+    if (targets.length === 0) {
+        console.log('no .syql files to format');
+        return;
+    }
+    let drifted = 0;
+    for (const path of targets) {
+        if (!path.endsWith('.syql')) {
+            fail(`fmt formats .syql files only, got ${path}`);
+        }
+        const source = readFileSync(path, 'utf8');
+        let formatted;
+        try {
+            formatted = formatSyql(path, source);
+        }
+        catch (error) {
+            fail(error instanceof Error ? error.message : String(error));
+        }
+        if (formatted === source)
+            continue;
+        drifted += 1;
+        if (check) {
+            console.error(`${path}: not canonical — run \`syncular fmt\``);
+        }
+        else {
+            writeFileSync(path, formatted, 'utf8');
+            console.log(`formatted ${path}`);
+        }
+    }
+    if (check && drifted > 0)
+        process.exit(1);
+    if (drifted === 0)
+        console.log('all .syql files canonical');
+}
 export function runCli(argv) {
     const command = argv[0];
     if (command === 'generate') {
         const args = parseGenerateArgs(argv.slice(1));
+        if (args.print !== undefined) {
+            runGeneratePrint(args, args.print);
+            return;
+        }
         if (args.watch) {
             runGenerateWatch(args);
             return;
@@ -153,6 +296,14 @@ export function runCli(argv) {
         runGenerateOnce(args);
         return;
     }
+    if (command === 'fmt') {
+        runFmt(argv.slice(1));
+        return;
+    }
+    if (command === 'lsp') {
+        void runLspStdio();
+        return;
+    }
     if (command === 'init') {
         const dir = parseManifestDir(argv.slice(1));
         try {

package/dist/emit-dart.js

@@ -1,3 +1,4 @@
+import { snakeToCamel } from './naming.js';
 /** §2.4 column type → honest Dart type. `json`/`blob_ref` are the raw
  * canonical JSON string; `bytes`/`crdt` are opaque `List<int>`. */
 const DART_TYPE = {
@@ -24,9 +25,9 @@ function pascalCase(name) {
         .map((part) => part.charAt(0).toUpperCase() + part.slice(1))
         .join('');
 }
+/** Language-facing field name — the pinned §12 naming map. */
 function camelCase(name) {
-    const pascal = pascalCase(name);
-    return pascal.charAt(0).toLowerCase() + pascal.slice(1);
+    return snakeToCamel(name);
 }
 function quote(value) {
     return `'${value.replace(/\\/g, '\\\\').replace(/'/g, "\\'").replace(/\$/g, '\\$')}'`;

package/dist/emit-kotlin.js

@@ -1,3 +1,4 @@
+import { snakeToCamel } from './naming.js';
 /** §2.4 column type → honest Kotlin type. `json`/`blob_ref` are the raw
  * canonical JSON string; `bytes`/`crdt` are opaque `ByteArray`. */
 const KOTLIN_TYPE = {
@@ -24,9 +25,9 @@ function pascalCase(name) {
         .map((part) => part.charAt(0).toUpperCase() + part.slice(1))
         .join('');
 }
+/** Language-facing field name — the pinned §12 naming map. */
 function camelCase(name) {
-    const pascal = pascalCase(name);
-    return pascal.charAt(0).toLowerCase() + pascal.slice(1);
+    return snakeToCamel(name);
 }
 function quote(value) {
     return `"${value.replace(/\\/g, '\\\\').replace(/"/g, '\\"').replace(/\$/g, '\\$')}"`;

package/dist/emit-queries-dart.js

@@ -1,3 +1,4 @@
+import { snakeToCamel } from './naming.js';
 const DART_TYPE = {
     string: 'String',
     integer: 'int',
@@ -15,9 +16,9 @@ function pascalCase(name) {
         .map((p) => p.charAt(0).toUpperCase() + p.slice(1))
         .join('');
 }
+/** Language-facing field name — the pinned §12 naming map. */
 function camelCase(name) {
-    const pascal = pascalCase(name);
-    return pascal.charAt(0).toLowerCase() + pascal.slice(1);
+    return snakeToCamel(name);
 }
 function typeName(name) {
     return name.charAt(0).toUpperCase() + name.slice(1);
@@ -26,7 +27,7 @@ function quote(value) {
     return `'${value.replace(/\\/g, '\\\\').replace(/'/g, "\\'").replace(/\$/g, '\\$')}'`;
 }
 function rowAccessor(column) {
-    const key = `row[${quote(column.name)}]`;
+    const key = `row[${quote(column.langName)}]`;
     switch (column.type) {
         case 'string':
         case 'json':
@@ -53,6 +54,38 @@ function paramValue(type, name) {
             return name;
     }
 }
+/** Bind for a §4 OPTIONAL param: Dart `null` rides as JSON null (the §7
+ * neutralization guards make it a no-op). */
+function optionalParamValue(type, name) {
+    switch (type) {
+        case 'bytes':
+        case 'crdt':
+            return `${name} == null ? null : _queryBindBytes(${name})`;
+        default:
+            return name;
+    }
+}
+function isOptionalParam(query, p) {
+    return (p.optional === true ||
+        p.flag === true ||
+        (query.limit !== undefined && p.name === 'limit'));
+}
+/** Per-query orderBy allowlist enum (column = the checked SQL column). */
+function emitOrderByEnum(query) {
+    if (query.orderBy === undefined)
+        return [];
+    const lines = [];
+    lines.push(`/// §6 orderBy allowlist for ${query.name} — checked at generate time.`);
+    lines.push(`enum ${typeName(query.name)}OrderBy {`);
+    lines.push(query.orderBy.allowed
+        .map((col) => `  ${camelCase(col.langName)}(${quote(col.name)})`)
+        .join(',\n') + ';');
+    lines.push('');
+    lines.push(`  const ${typeName(query.name)}OrderBy(this.column);`);
+    lines.push('  final String column;');
+    lines.push('}');
+    return lines;
+}
 function emitClass(query) {
     const Row = `${typeName(query.name)}Row`;
     const lines = [];
@@ -60,25 +93,25 @@ function emitClass(query) {
     lines.push(`class ${Row} {`);
     for (const column of query.columns) {
         const opt = column.nullable ? '?' : '';
-        lines.push(`  final ${DART_TYPE[column.type]}${opt} ${camelCase(column.name)};`);
+        lines.push(`  final ${DART_TYPE[column.type]}${opt} ${camelCase(column.langName)};`);
     }
     lines.push('');
     const ctorParams = query.columns
-        .map((c) => `${c.nullable ? '' : 'required '}this.${camelCase(c.name)}`)
+        .map((c) => `${c.nullable ? '' : 'required '}this.${camelCase(c.langName)}`)
         .join(', ');
     lines.push(`  const ${Row}({${ctorParams}});`);
     lines.push('');
     lines.push(`  static ${Row}? fromRow(Map<String, Object?> row) {`);
     for (const column of query.columns) {
         if (!column.nullable) {
-            const name = camelCase(column.name);
+            const name = camelCase(column.langName);
             lines.push(`    final ${name} = ${rowAccessor(column)};`);
             lines.push(`    if (${name} == null) return null;`);
         }
     }
     const args = query.columns
         .map((c) => {
-        const name = camelCase(c.name);
+        const name = camelCase(c.langName);
         return c.nullable ? `${name}: ${rowAccessor(c)}` : `${name}: ${name}`;
     })
         .join(', ');
@@ -94,23 +127,53 @@ function emitRunner(query) {
     lines.push(`/// Tables the ${query.name} query reads (exact invalidation set).`);
     lines.push(`const List<String> syncular${Pascal}QueryTables = [${query.tables.map(quote).join(', ')}];`);
     lines.push('');
-    lines.push(`const String _${query.name}Sql = ${quote(query.positionalSql)};`);
+    if (query.orderBy !== undefined) {
+        lines.push(`const String _${query.name}SqlBase = ${quote(query.positionalSqlBase ?? '')};`);
+    }
+    else {
+        lines.push(`const String _${query.name}Sql = ${quote(query.positionalSql)};`);
+    }
     lines.push('');
     lines.push(`/// Run the ${query.name} named query (SELECT-only).`);
-    const args = query.params
-        .map((p) => `required ${DART_TYPE[p.type]} ${camelCase(p.name)}`)
-        .join(', ');
-    const argsClause = query.params.length > 0 ? `, {${args}}` : '';
+    const args = [];
+    for (const p of query.params) {
+        const name = camelCase(p.langName);
+        if (isOptionalParam(query, p)) {
+            args.push(`${DART_TYPE[p.type]}? ${name}`);
+        }
+        else {
+            args.push(`required ${DART_TYPE[p.type]} ${name}`);
+        }
+    }
+    if (query.orderBy !== undefined) {
+        const defaultCase = camelCase(query.orderBy.allowed.find((c) => c.name === query.orderBy?.defaultColumn)
+            ?.langName ?? query.orderBy.defaultColumn);
+        args.push(`${typeName(query.name)}OrderBy orderBy = ${typeName(query.name)}OrderBy.${defaultCase}`);
+        args.push(`SyncularQueryDir dir = SyncularQueryDir.${query.orderBy.defaultDir}`);
+    }
+    const argsClause = args.length > 0 ? `, {${args.join(', ')}}` : '';
     lines.push(`List<${Row}> syncular${Pascal}Query(SyncularClient client${argsClause}) {`);
+    if (query.orderBy !== undefined) {
+        const limitTail = query.positionalLimitTail !== undefined
+            ? ` ${quote(query.positionalLimitTail.trim())}`
+            : '';
+        lines.push(`  final sql = '$_${query.name}SqlBase order by \${orderBy.column} \${dir.name}'${limitTail === '' ? '' : `\n      ' ' ${limitTail.trim()}`};`);
+    }
+    const sqlRef = query.orderBy !== undefined ? 'sql' : `_${query.name}Sql`;
     if (query.params.length > 0) {
         const binds = query.params
-            .map((p) => paramValue(p.type, camelCase(p.name)))
+            .map((p) => {
+            const name = camelCase(p.langName);
+            return isOptionalParam(query, p)
+                ? optionalParamValue(p.type, name)
+                : paramValue(p.type, name);
+        })
             .join(', ');
         lines.push(`  final params = <Object?>[${binds}];`);
-        lines.push(`  return client.query(_${query.name}Sql, params: params).map(${Row}.fromRow).whereType<${Row}>().toList();`);
+        lines.push(`  return client.query(${sqlRef}, params: params).map(${Row}.fromRow).whereType<${Row}>().toList();`);
     }
     else {
-        lines.push(`  return client.query(_${query.name}Sql).map(${Row}.fromRow).whereType<${Row}>().toList();`);
+        lines.push(`  return client.query(${sqlRef}).map(${Row}.fromRow).whereType<${Row}>().toList();`);
     }
     lines.push('}');
     return lines;
@@ -150,8 +213,17 @@ export function emitQueriesDartModule(queries, hash, irVersion) {
         "  return {r'$bytes': hex};",
         '}',
     ].join('\n'));
+    if (queries.some((q) => q.orderBy !== undefined)) {
+        parts.push([
+            '/// §6 orderBy direction (shared by every orderBy-knob query).',
+            'enum SyncularQueryDir { asc, desc }',
+        ].join('\n'));
+    }
     for (const query of queries) {
         parts.push(emitClass(query).join('\n'));
+        const orderByEnum = emitOrderByEnum(query);
+        if (orderByEnum.length > 0)
+            parts.push(orderByEnum.join('\n'));
     }
     for (const query of queries) {
         parts.push(emitRunner(query).join('\n'));