@syncular/typegen 0.5.1 → 0.7.0

package/README.md

@@ -68,8 +68,8 @@ schema guide and suggests `syncular init`.
 | `manifestVersion` | yes | Must be `1`. Format growth happens by bumping this, not by tolerating unknown keys. |
 | `migrations` | no (default `./migrations`) | Directory of `NNNN_name/up.sql` migrations, relative to the manifest. |
 | `queries` | no (default `./queries`) | Directory of `.sql` / `.syql` named-query files (see §6). Only read when some output requests a queries file. |
-| `naming` | no (default `"camel"`) | §5-of-DESIGN-queries casing: `"camel"` maps snake_case SQL names to camelCase in generated row types/params (projections lower with `AS` aliases so runtime keys match); `"preserve"` keeps SQL-truth names. Collisions/keyword hazards are generate-time errors. |
-| `queryBackend` | no (default `"neutralize"`) | `.syql` conditional lowering: `"neutralize"` (one guarded statement), `"variants"` (enumerate per provided-combination whenever eligible), `"auto"` (enumerate at ≤ 2 optional groups). The per-query `variants` knob always forces. |
+| `naming` | no (default `"camel"`) | `"camel"` maps snake_case SQL names to camelCase in generated row types/params (projections lower with `AS` aliases so runtime keys match); `"preserve"` keeps SQL-truth names. Collisions/keyword hazards are generate-time errors. |
+| `queryBackend` | no (default `"auto"`) | Advanced revision-1 SYQL lowering override: `"neutralize"` emits guarded statements, `"variants"` enumerates activation states, and `"auto"` deterministically enumerates at ≤ 2 activation controls. This never changes source meaning or the public API. |
 | `output.ir` | no (default `./syncular.ir.json`) | IR output path, relative to the manifest. |
 | `output.module` | no (default `./syncular.generated.ts`) | Generated TS module path, relative to the manifest. |
 | `output.queryIr` | no | Deterministic analyzed QueryIR JSON. Its hash keys generated reactive query descriptors, so SQL-only changes invalidate caches. |
@@ -326,18 +326,54 @@ function on every platform — killing query↔type drift *by construction*. It
 the type-safe **read** tier; raw `query(sql, params)` — guarded read-only —
 stays the escape hatch for queries built at runtime.
 
-Two frontends feed one pipeline (DESIGN-queries.md is the design of record):
-**`.sql`** — plain SQL + `:params`, the compatibility floor, documented in
-this section — and **`.syql`** — the DSL container (declared optional params
-with auto-guarded conjuncts, `from+to?` groups, `?: flag` guards with
-`if (…) { … }`, file-scoped `@fragments`, `orderBy`/`limit` knobs, and the
-opt-in `variants` enumeration backend), which lowers at generate time to the
-same checked plain SQL. Tooling: `syncular generate --print <name>` (the
-lowered SQL), `syncular fmt` (canonical `.syql` style), `syncular lsp`
-(diagnostics/hover/definition over stdio). Casing follows the manifest
-`naming` mode (§1): generated rows/params are camelCase and projections are
-`AS`-aliased so runtime keys match; `mutate` accepts camel and snake keys
-both.
+Two frontends feed one QueryIR pipeline. **`.sql`** is plain SQL plus
+`:params`, the compatibility floor documented below. **`.syql`** is the
+revision-1 structured frontend: authoritative typed inputs, explicit
+`when(...)` conjuncts, atomic optional groups, imported hygienic predicates,
+inferred scope dependencies, explicit sync coverage, complete sort profiles,
+bounded limits, and inferred identity. Its complete normative definition is
+[`../../docs/SYQL.md`](../../docs/SYQL.md); the rationale and implementation
+record are [`../../docs/rfcs/0004-syql-language.md`](../../docs/rfcs/0004-syql-language.md).
+
+Tooling: `syncular generate --print <name>` prints every selected checked
+statement and bind, `syncular fmt` is the semantic-preserving canonical
+formatter, and `syncular lsp` provides project-aware diagnostics, hover,
+definitions, references, symbols, and formatting. Casing follows the manifest
+`naming` mode (§1): generated rows/inputs are camelCase and projections are
+`AS`-aliased so runtime keys match.
+
+```syql
+import { matchesTitle } from "./predicates.syql";
+
+sync query listTodos(
+  listId,
+  status?: string | null,
+  range?,
+  q?: string,
+  unassigned: bool = false,
+) {
+  select id, title, status, created_at
+  from todos
+  where todos.list_id = :listId
+    and when(status) status is :status
+    and when(range) created_at between :range
+    and when(unassigned) assignee_id is null
+    and when(q) matchesTitle(:q)
+  order by sortBy default newest {
+    newest: created_at desc, id desc;
+    oldest: created_at asc, id asc;
+  }
+  limit pageSize default 50 max 200;
+}
+```
+
+Optional nullable scalars use a generated presence wrapper so absent,
+present-null, and present-value remain distinct. A group is one optional host
+object whose members are all required. A `bool = false` flag activates on
+true. Sort is a generated enum/union of complete checked profiles; limit is validated as a
+positive bounded integer before execution. `integer` inputs are exact signed
+64-bit values in the SYQL API (`bigint` in TypeScript, `Int64`/`Long`/`int` on
+native targets).
 
 **File & naming convention.** Named queries live in a `queries/` directory
 next to `migrations/` (override with the top-level `"queries"` manifest key).
@@ -412,6 +448,27 @@ boundary):
 | **plain column ref** (`title`, `t.title`, `title AS x`) | resolved to the IR column (decltype confirms) — exact IR type, incl. `json`/`blob_ref`/`crdt` | the IR column's `NOT NULL` | **exact** |
 | **computed expression** (`count(*)`, `done + 1`, `:p AS l`) | decltype is null → documented fallback: aggregate/arithmetic → number, else string | always nullable (an expression's nullability is not knowable from decltype) | **fallback** |
 
+Every local synced table also has the protocol-owned `_sync_version` column.
+Named queries may project it with an explicit public alias when an application
+needs the observed server version for optimistic concurrency:
+
+```syql
+query getTodo(listId, todoId) {
+  select id, title, _sync_version as server_version
+  from todos
+  where todos.list_id = :listId and todos.id = :todoId;
+}
+```
+
+The generated `serverVersion` field is an exact, non-null `integer`. The
+physical name is intentionally not a public result name: project it with an
+alias such as `server_version`. `_sync_version` is query-only, is excluded from
+schema and mutation types, and is not added by `select *`; this prevents client
+code from writing engine-owned concurrency state. Pass a positive observed
+value as a mutation `baseVersion`. Locally-created or still-unacknowledged rows
+can carry a non-positive sentinel and should omit the base version until the
+server assigns one.
+
 `bun:sqlite` exposes `columnNames`, `declaredTypes` (once executed once — we
 run the statement against the empty DB), and `paramsCount`. It does **not**
 expose column origin (table/column), param **names**, or `NOT NULL` flags — so
@@ -455,30 +512,20 @@ identifier and keeping those that name an IR table. This captures subquery /
 prepare() still guarantees the SQL itself is correct.
 
 Inference is deliberately conservative around `OR`, joins, grouping, and
-computed identity. A `.syql` query can state the missing fact explicitly:
+computed identity. Ordinary scope predicates construct exact reactive facts;
+`sync query` additionally requests checked coverage:
 
 ```syql
-query compareLists(left, right)
-  depends tasks on project_id = left | right
-  window tasks by project_id = left | right
-  key by id
-{
+sync query compareLists(left, right) {
   select id, title from tasks
-  where project_id = :left or project_id = :right
+  where tasks.project_id in (:left, :right);
 }
 ```
 
-The declaration grammar is:
-
-- `depends <table> on <scope> = <param> | <param>`;
-- `window <table> by <scope> = <param> | <param>` with optional
-  `fixed <other_scope> = <param>, ...` for every other scope on that table;
-- `key by <result-column> | <result-column>`.
-
-These escape hatches are checked against the prepared query, schema, inferred
-param types, and result projection. They reject unread tables, unknown scopes
-or params, optional/incompatible coverage params, incomplete fixed scopes,
-duplicate declarations, and unprojected key fields.
+Coverage must resolve one table instance and bind every declared scope. Only
+required, non-null, exactly typed binds are allowed. Result identity is inferred
+from schema keys and the projection. Without constructive proof the compiler
+falls back to table-wide dependency, no coverage, and/or unkeyed reconciliation.
 
 **Emitted shape, per language** (one query, five outputs — abbreviated):
 

package/dist/cli.js

@@ -109,29 +109,52 @@ function runGeneratePrint(args, name) {
     }
     console.log(`-- ${query.name} (${query.file})`);
     console.log(`-- tables: ${query.tables.join(', ')}`);
+    if (query.syql !== undefined) {
+        console.log('-- revision: SYQL 1');
+        console.log(`-- backend: ${query.syql.plan.backend}`);
+        if (query.syql.inputs.length > 0) {
+            console.log('-- public inputs:');
+            for (const input of query.syql.inputs) {
+                if (input.kind === 'value') {
+                    console.log(`--   ${input.name}: ${input.type}${input.nullable ? ' | null' : ''} (${input.required ? 'required' : 'optional'})`);
+                }
+                else if (input.kind === 'group') {
+                    console.log(`--   ${input.name}: optional group (${input.members.map((member) => `${member.name}: ${member.type}${member.nullable ? ' | null' : ''}`).join(', ')})`);
+                }
+                else if (input.kind === 'sort') {
+                    console.log(`--   ${input.name}: sort [${input.profiles.map((profile) => profile.name).join(', ')}] (default ${input.defaultProfile})`);
+                }
+                else {
+                    console.log(`--   ${input.name}: limit 1..${input.maxSize} (default ${input.defaultSize})`);
+                }
+            }
+        }
+        if (query.syql.identity !== undefined) {
+            console.log(`-- identity: ${query.syql.identity.join(', ')}`);
+        }
+        console.log(`-- checked statements: ${query.syql.plan.statements.length} (activation controls: ${query.syql.plan.activationControls.join(', ') || 'none'})`);
+        for (const statement of query.syql.plan.statements) {
+            const selectors = [
+                ...(statement.activationMask === undefined
+                    ? []
+                    : [`mask=${statement.activationMask}`]),
+                ...(statement.sortProfile === undefined
+                    ? []
+                    : [`sort=${statement.sortProfile}`]),
+            ];
+            console.log(`-- [${selectors.join(', ') || 'default'}]`);
+            console.log(statement.sql);
+            console.log(`-- binds: ${statement.binds.map((bind) => `:${bind.name}`).join(', ') || '(none)'}`);
+        }
+        return;
+    }
     if (query.params.length > 0) {
         console.log('-- params:');
         for (const param of query.params) {
-            const marks = [
-                param.optional === true ? 'optional' : 'required',
-                ...(param.flag === true ? ['flag'] : []),
-                ...(param.group !== undefined ? [`group ${param.group}`] : []),
-            ].join(', ');
-            console.log(`--   :${param.name} ${param.type} (${marks})`);
+            console.log(`--   :${param.name} ${param.type} (required)`);
         }
     }
     console.log(query.sql);
-    if (query.orderBy !== undefined) {
-        console.log('-- orderBy variants (each checked against the schema):');
-        const base = query.positionalSqlBase ?? '';
-        for (const col of query.orderBy.allowed) {
-            const isDefault = col.name === query.orderBy.defaultColumn;
-            console.log(`--   ${col.langName}: ${base} order by ${col.name} asc|desc${query.positionalLimitTail ?? ''}${isDefault ? '   (default)' : ''}`);
-        }
-    }
-    if (query.limit !== undefined) {
-        console.log(`-- limit: bound value, default ${query.limit.default ?? query.limit.max}${query.limit.max !== undefined ? `, clamped to ${query.limit.max}` : ''}`);
-    }
 }
 function parseManifestDir(argv) {
     let manifestDir = '.';

package/dist/emit-queries-dart.js

@@ -65,25 +65,148 @@ function optionalParamValue(type, name) {
             return name;
     }
 }
-function isOptionalParam(query, p) {
-    return (p.optional === true ||
-        p.flag === true ||
-        (query.limit !== undefined && p.name === 'limit'));
-}
-/** Per-query orderBy allowlist enum (column = the checked SQL column). */
-function emitOrderByEnum(query) {
-    if (query.orderBy === undefined)
-        return [];
+function syqlInput(query, name) {
+    const input = query.syql?.inputs.find((candidate) => candidate.name === name);
+    if (input === undefined)
+        throw new Error(`unknown SYQL input ${name}`);
+    return input;
+}
+function syqlDartType(type, nullable) {
+    return `${DART_TYPE[type]}${nullable ? '?' : ''}`;
+}
+function syqlControlActive(query, name) {
+    const input = syqlInput(query, name);
+    const access = camelCase(input.langName);
+    if (input.kind === 'value' && input.default === false)
+        return access;
+    if (input.kind === 'value' && input.nullable)
+        return `${access}.isPresent`;
+    if (input.kind === 'value' || input.kind === 'group') {
+        return `${access} != null`;
+    }
+    throw new Error(`${name} is not an activation control`);
+}
+function syqlBindExpr(query, bind) {
+    if (bind.kind === 'condition-active') {
+        return bind.controls
+            .map((control) => syqlControlActive(query, control))
+            .join(' && ');
+    }
+    const input = syqlInput(query, bind.input);
+    const access = camelCase(input.langName);
+    if (bind.kind === 'limit')
+        return `effective${typeName(access)}`;
+    if (bind.kind === 'group-member') {
+        if (input.kind !== 'group')
+            throw new Error('group bind/input mismatch');
+        const member = input.members.find((candidate) => candidate.name === bind.member);
+        if (member === undefined)
+            throw new Error(`unknown group member ${bind.member}`);
+        const memberAccess = `${access}.${camelCase(member.langName)}`;
+        const value = member.nullable
+            ? optionalParamValue(member.type, memberAccess)
+            : paramValue(member.type, memberAccess);
+        return `${access} == null ? null : ${value}`;
+    }
+    if (input.kind !== 'value')
+        throw new Error('value bind/input mismatch');
+    if (input.default === false)
+        return paramValue(input.type, access);
+    if (input.required) {
+        return input.nullable
+            ? optionalParamValue(input.type, access)
+            : paramValue(input.type, access);
+    }
+    return input.nullable
+        ? `${access}.isPresent ? ${optionalParamValue(input.type, `${access}.value`)} : null`
+        : optionalParamValue(input.type, access);
+}
+function emitSyqlDartTypes(query) {
     const lines = [];
-    lines.push(`/// §6 orderBy allowlist for ${query.name} — checked at generate time.`);
-    lines.push(`enum ${typeName(query.name)}OrderBy {`);
-    lines.push(`${query.orderBy.allowed
-        .map((col) => `  ${camelCase(col.langName)}(${quote(col.name)})`)
-        .join(',\n')};`);
-    lines.push('');
-    lines.push(`  const ${typeName(query.name)}OrderBy(this.column);`);
-    lines.push('  final String column;');
-    lines.push('}');
+    for (const input of query.syql?.inputs ?? []) {
+        if (input.kind === 'group') {
+            const name = `${typeName(query.name)}${typeName(input.langName)}`;
+            lines.push(`class ${name} {`);
+            for (const member of input.members) {
+                lines.push(`  final ${syqlDartType(member.type, member.nullable)} ${camelCase(member.langName)};`);
+            }
+            const args = input.members
+                .map((member) => `required this.${camelCase(member.langName)}`)
+                .join(', ');
+            lines.push(`  const ${name}({${args}});`, '}', '');
+        }
+        else if (input.kind === 'sort') {
+            const name = `${typeName(query.name)}${typeName(input.langName)}`;
+            lines.push(`enum ${name} { ${input.profiles.map((profile) => camelCase(profile.langName)).join(', ')} }`, '');
+        }
+    }
+    if (lines[lines.length - 1] === '')
+        lines.pop();
+    return lines;
+}
+function emitSyqlDartRunner(query) {
+    const metadata = query.syql;
+    if (metadata === undefined)
+        throw new Error('missing SYQL metadata');
+    const Row = `${typeName(query.name)}Row`;
+    const Pascal = pascalCase(query.name);
+    const lines = [];
+    lines.push(`/// Tables the ${query.name} query reads (exact invalidation set).`, `const List<String> syncular${Pascal}QueryTables = [${query.tables.map(quote).join(', ')}];`, '', `/// Run the ${query.name} revision-1 SYQL query.`);
+    const args = [];
+    for (const input of metadata.inputs) {
+        const name = camelCase(input.langName);
+        if (input.kind === 'value') {
+            const type = syqlDartType(input.type, input.nullable);
+            if (input.required)
+                args.push(`required ${type} ${name}`);
+            else if (input.default === false)
+                args.push(`bool ${name} = false`);
+            else if (input.nullable) {
+                args.push(`SyqlQueryPresence<${type}> ${name} = const SyqlQueryPresence.absent()`);
+            }
+            else
+                args.push(`${type}? ${name}`);
+        }
+        else if (input.kind === 'group') {
+            args.push(`${typeName(query.name)}${typeName(input.langName)}? ${name}`);
+        }
+        else if (input.kind === 'sort') {
+            const defaultCase = input.profiles.find((profile) => profile.name === input.defaultProfile)
+                ?.langName ?? input.defaultProfile;
+            const type = `${typeName(query.name)}${typeName(input.langName)}`;
+            args.push(`${type} ${name} = ${type}.${camelCase(defaultCase)}`);
+        }
+        else {
+            args.push(`int? ${name}`);
+        }
+    }
+    const argsClause = args.length > 0 ? `, {${args.join(', ')}}` : '';
+    lines.push(`List<${Row}> syncular${Pascal}Query(SyncularClient client${argsClause}) {`);
+    const page = metadata.inputs.find((input) => input.kind === 'limit');
+    if (page?.kind === 'limit') {
+        const name = camelCase(page.langName);
+        lines.push(`  final effective${typeName(name)} = ${name} ?? ${page.defaultSize};`, `  if (effective${typeName(name)} < 1 || effective${typeName(name)} > ${page.maxSize}) {`, `    throw SyqlQueryInputException('SYQL_RUNTIME_INVALID_LIMIT', ${quote(`${query.name}: invalid limit`)});`, '  }');
+    }
+    if (metadata.plan.backend === 'variants') {
+        lines.push('  var activationMask = 0;');
+        metadata.plan.activationControls.forEach((control, index) => {
+            lines.push(`  if (${syqlControlActive(query, control)}) activationMask |= ${2 ** index};`);
+        });
+    }
+    const sort = metadata.inputs.find((input) => input.kind === 'sort');
+    const profileCount = sort?.kind === 'sort' ? sort.profiles.length : 1;
+    const sortIndex = sort?.kind === 'sort' ? `${camelCase(sort.langName)}.index` : '0';
+    const index = metadata.plan.backend === 'variants'
+        ? `activationMask * ${profileCount} + ${sortIndex}`
+        : sortIndex;
+    lines.push(`  final statementIndex = ${index};`, '  late final String sql;', '  late final List<Object?> params;', '  switch (statementIndex) {');
+    metadata.plan.statements.forEach((statement, statementIndex) => {
+        const binds = statement.binds
+            .map((bind) => syqlBindExpr(query, bind))
+            .join(', ');
+        lines.push(`    case ${statementIndex}: sql = ${quote(statement.positionalSql)}; params = <Object?>[${binds}]; break;`);
+    });
+    lines.push("    default: throw StateError('invalid generated SYQL statement index');", '  }', `  return client.query(sql, params: params).map(${Row}.fromRow).whereType<${Row}>().toList();`, '}');
     return lines;
 }
 function emitClass(query) {
@@ -121,52 +244,30 @@ function emitClass(query) {
     return lines;
 }
 function emitRunner(query) {
+    if (query.syql !== undefined)
+        return emitSyqlDartRunner(query);
     const Row = `${typeName(query.name)}Row`;
     const Pascal = pascalCase(query.name);
     const lines = [];
     lines.push(`/// Tables the ${query.name} query reads (exact invalidation set).`);
     lines.push(`const List<String> syncular${Pascal}QueryTables = [${query.tables.map(quote).join(', ')}];`);
     lines.push('');
-    if (query.orderBy !== undefined) {
-        lines.push(`const String _${query.name}SqlBase = ${quote(query.positionalSqlBase ?? '')};`);
-    }
-    else {
-        lines.push(`const String _${query.name}Sql = ${quote(query.positionalSql)};`);
-    }
+    lines.push(`const String _${query.name}Sql = ${quote(query.positionalSql)};`);
     lines.push('');
     lines.push(`/// Run the ${query.name} named query (SELECT-only).`);
     const args = [];
     for (const p of query.params) {
         const name = camelCase(p.langName);
-        if (isOptionalParam(query, p)) {
-            args.push(`${DART_TYPE[p.type]}? ${name}`);
-        }
-        else {
-            args.push(`required ${DART_TYPE[p.type]} ${name}`);
-        }
-    }
-    if (query.orderBy !== undefined) {
-        const defaultCase = camelCase(query.orderBy.allowed.find((c) => c.name === query.orderBy?.defaultColumn)
-            ?.langName ?? query.orderBy.defaultColumn);
-        args.push(`${typeName(query.name)}OrderBy orderBy = ${typeName(query.name)}OrderBy.${defaultCase}`);
-        args.push(`SyncularQueryDir dir = SyncularQueryDir.${query.orderBy.defaultDir}`);
+        args.push(`required ${DART_TYPE[p.type]} ${name}`);
     }
     const argsClause = args.length > 0 ? `, {${args.join(', ')}}` : '';
     lines.push(`List<${Row}> syncular${Pascal}Query(SyncularClient client${argsClause}) {`);
-    if (query.orderBy !== undefined) {
-        const limitTail = query.positionalLimitTail !== undefined
-            ? ` ${quote(query.positionalLimitTail.trim())}`
-            : '';
-        lines.push(`  final sql = '$_${query.name}SqlBase order by \${orderBy.column} \${dir.name}'${limitTail === '' ? '' : `\n      ' ' ${limitTail.trim()}`};`);
-    }
-    const sqlRef = query.orderBy !== undefined ? 'sql' : `_${query.name}Sql`;
+    const sqlRef = `_${query.name}Sql`;
     if (query.params.length > 0) {
         const binds = query.params
             .map((p) => {
             const name = camelCase(p.langName);
-            return isOptionalParam(query, p)
-                ? optionalParamValue(p.type, name)
-                : paramValue(p.type, name);
+            return paramValue(p.type, name);
         })
             .join(', ');
         lines.push(`  final params = <Object?>[${binds}];`);
@@ -188,6 +289,24 @@ export function emitQueriesDartModule(queries, hash, irVersion) {
         '',
         "import 'package:syncular/syncular.dart';",
     ].join('\n'));
+    if (queries.some((query) => query.syql !== undefined)) {
+        parts.push([
+            'class SyqlQueryPresence<T> {',
+            '  final bool isPresent;',
+            '  final T? value;',
+            '  const SyqlQueryPresence.absent() : isPresent = false, value = null;',
+            '  const SyqlQueryPresence.present(this.value) : isPresent = true;',
+            '}',
+            '',
+            'class SyqlQueryInputException implements Exception {',
+            '  final String code;',
+            '  final String message;',
+            '  const SyqlQueryInputException(this.code, this.message);',
+            '  @override',
+            "  String toString() => '$code: $message';",
+            '}',
+        ].join('\n'));
+    }
     parts.push([
         '/// Lift a SQLite boolean: a real bool, or 0/1 as a number.',
         'bool? _queryRowBool(Object? value) {',
@@ -213,17 +332,11 @@ export function emitQueriesDartModule(queries, hash, irVersion) {
         "  return {r'$bytes': hex};",
         '}',
     ].join('\n'));
-    if (queries.some((q) => q.orderBy !== undefined)) {
-        parts.push([
-            '/// §6 orderBy direction (shared by every orderBy-knob query).',
-            'enum SyncularQueryDir { asc, desc }',
-        ].join('\n'));
-    }
     for (const query of queries) {
+        const syqlTypes = emitSyqlDartTypes(query);
+        if (syqlTypes.length > 0)
+            parts.push(syqlTypes.join('\n'));
         parts.push(emitClass(query).join('\n'));
-        const orderByEnum = emitOrderByEnum(query);
-        if (orderByEnum.length > 0)
-            parts.push(orderByEnum.join('\n'));
     }
     for (const query of queries) {
         parts.push(emitRunner(query).join('\n'));