@syncular/typegen 0.15.27 → 0.15.29

package/README.md

@@ -557,7 +557,7 @@ boundary):
 
 | Result column | Type source | Nullability | Fidelity |
 |---|---|---|---|
-| **plain column ref** (`title`, `t.title`, `title AS x`) | resolved to the IR column (decltype confirms) — exact IR type, incl. `json`/`blob_ref`/`crdt` | the IR column's `NOT NULL` | **exact** |
+| **plain column ref** (`title`, `t.title`, `title AS x`) | resolved to the IR column (decltype confirms) — exact IR type, incl. `json`/`blob_ref`/`crdt` | the IR column's nullability, lifted to nullable whenever `LEFT`, `RIGHT`, or `FULL OUTER JOIN` can null-extend its table alias | **exact** |
 | **computed expression** (`count(*)`, `done + 1`, `:p AS l`) | decltype is null → documented fallback: aggregate/arithmetic → number, else string | always nullable (an expression's nullability is not knowable from decltype) | **fallback** |
 
 Every local synced table also has the protocol-owned `_sync_version` column.
@@ -572,7 +572,9 @@ query getTodo(listId, todoId) {
 }
 ```
 
-The generated `serverVersion` field is an exact, non-null `integer`. The
+The generated `serverVersion` field is an exact, non-null `integer` for a base
+or required join relation; it is nullable when projected from an optional side
+of an outer join. The
 physical name is intentionally not a public result name: project it with an
 alias such as `server_version`. `_sync_version` is query-only, is excluded from
 schema and mutation types, and is not added by `select *`; this prevents client
@@ -585,9 +587,17 @@ server assigns one.
 run the statement against the empty DB), and `paramsCount`. It does **not**
 expose column origin (table/column), param **names**, or `NOT NULL` flags — so
 typegen resolves plain refs against the IR itself (parse the SELECT list +
-FROM/JOIN, match `name`/`alias.name`) to attach exact nullability, and parses
-`:name` placeholders from the SQL text for param names. Want an exact type for
-a computed column? Alias it to a plain ref, or accept the honest fallback.
+FROM/JOIN, match `name`/`alias.name`) to attach schema nullability plus SQL
+outer-join null-extension, and parses `:name` placeholders from the SQL text
+for param names. Each join chain is evaluated left-to-right: `LEFT` lifts the
+new right relation unit, `RIGHT` lifts every prior relation in that chain, and
+`FULL` lifts both. A parenthesized relation group is one unit for its enclosing
+join while its nested join order remains intact. `INNER`, `CROSS`, and ordinary
+`JOIN` do not remove nullability already introduced earlier in the chain.
+Unqualified coalesced `USING`/`NATURAL` columns are not assigned a guessed
+physical origin and stay on the honest nullable fallback path. Want an exact
+type for a computed column? Alias it to a plain ref, or accept the honest
+fallback.
 
 **Parameters** use the `:name` convention. A param's type is **inferred** where
 it compares against a plain column ref — `WHERE list_id = :listId` (equality,
@@ -623,8 +633,8 @@ identifier and keeping those that name an IR table. This captures subquery /
 `WHERE EXISTS (…)` tables (they still appear under `FROM`/`JOIN`), and the
 prepare() still guarantees the SQL itself is correct.
 
-Inference is deliberately conservative around `OR`, joins, grouping, and
-computed identity. Ordinary scope predicates construct exact reactive facts;
+Inference is deliberately conservative around `OR`, grouping, ambiguous joins,
+and computed identity. Ordinary scope predicates construct exact reactive facts;
 `sync query` additionally requests checked coverage:
 
 ```syql
@@ -634,10 +644,21 @@ sync query compareLists(left, right) {
 }
 ```
 
-Coverage must resolve one table instance and bind every declared scope. Only
+Coverage must resolve every read schema table, with one instance per table, and
+bind every declared scope. A required scope bind can propagate across a
+qualified scope-column equality in an unconditional outer `WHERE` clause or a
+simple mandatory `ON` clause. The generated descriptor contains aggregate
+coverage for every table, and readiness requires all of it. Self-joins and
+`ON` clauses containing `OR`, `NOT`, or nested SQL cannot claim coverage. Only
 required, non-null, exactly typed binds are allowed. Result identity is inferred
-from schema keys and the projection. Without constructive proof the compiler
-falls back to table-wide dependency, no coverage, and/or unkeyed reconciliation.
+from schema keys and the projection. Without constructive proof an ordinary
+query falls back to table-wide dependency, no coverage, and/or unkeyed
+reconciliation; a `sync query` fails generation rather than claiming partial
+completeness.
+
+Use explicit `JOIN ... ON` relations. SQLite comma joins are rejected because
+the compiler will not emit reactive metadata unless every read relation is
+represented by the same proof model.
 
 **Emitted shape, per language** (abbreviated):
 

package/dist/emit-queries-rust.js

@@ -299,7 +299,9 @@ function emitSelect(query) {
         const name = field(limit.langName);
         lines.push(`        let effective_${name} = params.${name}.unwrap_or(${limit.defaultSize});`, `        if !(1..=${limit.maxSize}).contains(&effective_${name}) {`, '            return Err(QueryError::Input {', '                code: "SYQL_RUNTIME_INVALID_LIMIT",', '                query: ID,', `                message: ${quote(`${query.name}: limit must be an integer from 1 through ${limit.maxSize}`)}.to_owned(),`, '            });', '        }');
     }
-    if (metadata.plan.backend === 'variants') {
+    const usesActivationMask = metadata.plan.backend === 'variants' &&
+        metadata.plan.activationControls.length > 0;
+    if (usesActivationMask) {
         lines.push('        let mut activation_mask = 0usize;');
         metadata.plan.activationControls.forEach((control, index) => {
             lines.push(`        if ${syqlControlActive(query, control)} {`, `            activation_mask |= ${2 ** index};`, '        }');
@@ -315,7 +317,7 @@ function emitSelect(query) {
     }
     const sortIndex = sort?.kind === 'sort' ? 'sort_index' : '0usize';
     const profileCount = sort?.kind === 'sort' ? sort.profiles.length : 1;
-    const index = metadata.plan.backend === 'variants'
+    const index = usesActivationMask
         ? profileCount === 1
             ? 'activation_mask'
             : `activation_mask * ${profileCount} + ${sortIndex}`

package/dist/query.d.ts

@@ -221,6 +221,8 @@ export interface TableRef {
     readonly table: string;
     /** Alias (or the table name when un-aliased). */
     readonly alias: string;
+    /** True when an enclosing flat join chain can null-extend this relation. */
+    readonly nullable: boolean;
 }
 export declare function scanTableRefs(sql: string, ir: IrDocument): TableRef[];
 /**

package/dist/query.js

@@ -28,9 +28,10 @@
  *   origin column's exact declared type. We map it back through the SAME
  *   TYPE_MAP the migration parser uses → the exact IR column type. For
  *   NULLABILITY we resolve the ref against the IR ourselves (parse the SELECT
- *   list + FROM/JOIN, match `name`/`alias.name` to an IR table column) — an
- *   IR-exact non-nullable/nullable answer. This is the drift-kill: same type
- *   AND nullability as the schema, guaranteed by SQLite's own reference check.
+ *   list + FROM/JOIN, match `name`/`alias.name` to an IR table column), then
+ *   lift it when an outer join can synthesize NULL for that relation. This is
+ *   the drift-kill: schema nullability plus SQL join null-extension, with
+ *   SQLite still proving every reference.
  * - **Computed expression** (`count(*)`, `done + 1`, `:label AS l`): decltype
  *   is null. We fall back to a documented honest type from the raw column
  *   name/shape: aggregate/arith → nullable number, anything else → nullable
@@ -450,6 +451,7 @@ const RESERVED_ALIAS = new Set([
     'left',
     'right',
     'outer',
+    'natural',
     'join',
     'cross',
     'using',
@@ -457,23 +459,159 @@ const RESERVED_ALIAS = new Set([
     'having',
 ]);
 const RESERVED_ALIAS_PATTERN = [...RESERVED_ALIAS].join('|');
-const TABLE_REF_RE = new RegExp(`\\b(?:FROM|JOIN)\\s+(${IDENT})(?:\\s+(?:AS\\s+)?((?!(?:${RESERVED_ALIAS_PATTERN})\\b)${IDENT}))?`, 'gi');
-export function scanTableRefs(sql, ir) {
+const TABLE_REF_RE = new RegExp(`\\b(FROM|(?:NATURAL\\s+)?(?:(LEFT|RIGHT|FULL)(?:\\s+OUTER)?|INNER|CROSS)?\\s*JOIN)\\s+((?:\\(\\s*)*)(${IDENT})(?:\\s+(?:AS\\s+)?((?!(?:${RESERVED_ALIAS_PATTERN})\\b)${IDENT}))?`, 'gi');
+function parenthesisDepthAt(sql, index) {
+    let depth = 0;
+    for (let cursor = 0; cursor < index; cursor += 1) {
+        if (sql[cursor] === '(')
+            depth += 1;
+        else if (sql[cursor] === ')')
+            depth = Math.max(0, depth - 1);
+    }
+    return depth;
+}
+function matchingParenthesis(sql, open) {
+    let depth = 0;
+    for (let cursor = open; cursor < sql.length; cursor += 1) {
+        if (sql[cursor] === '(')
+            depth += 1;
+        else if (sql[cursor] === ')') {
+            depth -= 1;
+            if (depth === 0)
+                return cursor;
+        }
+    }
+    return sql.length;
+}
+function hasCommaJoinedSchemaTable(sql, ir) {
     const cleaned = stripCommentsAndStrings(sql);
     const known = new Set(ir.tables.flatMap((table) => [
-        table.name,
-        ...table.ftsIndexes.map((index) => index.name),
+        table.name.toLowerCase(),
+        ...table.ftsIndexes.map((index) => index.name.toLowerCase()),
+    ]));
+    const activeFromDepths = new Set();
+    const clauseEnders = new Set([
+        'WHERE',
+        'GROUP',
+        'HAVING',
+        'ORDER',
+        'LIMIT',
+        'WINDOW',
+        'UNION',
+        'EXCEPT',
+        'INTERSECT',
+        'RETURNING',
+    ]);
+    let depth = 0;
+    let cursor = 0;
+    const nextIdentifier = (start) => {
+        let index = start;
+        while (index < cleaned.length &&
+            (/\s/.test(cleaned[index]) || cleaned[index] === '(')) {
+            index += 1;
+        }
+        const match = /^[A-Za-z_][A-Za-z0-9_]*/.exec(cleaned.slice(index));
+        return match?.[0];
+    };
+    while (cursor < cleaned.length) {
+        const char = cleaned[cursor];
+        if (char === '(') {
+            const next = nextIdentifier(cursor + 1);
+            if (activeFromDepths.has(depth) &&
+                next !== undefined &&
+                known.has(next.toLowerCase())) {
+                activeFromDepths.add(depth + 1);
+            }
+            depth += 1;
+            cursor += 1;
+            continue;
+        }
+        if (char === ')') {
+            activeFromDepths.delete(depth);
+            depth = Math.max(0, depth - 1);
+            cursor += 1;
+            continue;
+        }
+        if (char === ',' && activeFromDepths.has(depth)) {
+            const next = nextIdentifier(cursor + 1);
+            if (next !== undefined && known.has(next.toLowerCase()))
+                return true;
+            cursor += 1;
+            continue;
+        }
+        if (/[A-Za-z_]/.test(char)) {
+            let end = cursor + 1;
+            while (end < cleaned.length &&
+                /[A-Za-z0-9_]/.test(cleaned[end])) {
+                end += 1;
+            }
+            const word = cleaned.slice(cursor, end).toUpperCase();
+            if (word === 'FROM')
+                activeFromDepths.add(depth);
+            else if (clauseEnders.has(word))
+                activeFromDepths.delete(depth);
+            cursor = end;
+            continue;
+        }
+        cursor += 1;
+    }
+    return false;
+}
+export function scanTableRefs(sql, ir) {
+    const cleaned = stripCommentsAndStrings(sql);
+    const known = new Map(ir.tables.flatMap((table) => [
+        [table.name.toLowerCase(), table.name],
+        ...table.ftsIndexes.map((index) => [index.name.toLowerCase(), index.name]),
     ]));
+    const nullableGroups = [
+        ...cleaned.matchAll(/\b(?:LEFT|FULL)(?:\s+OUTER)?\s+JOIN\s*(\()/gi),
+    ].map((match) => {
+        const open = (match.index ?? 0) + match[0].lastIndexOf('(');
+        return { open, close: matchingParenthesis(cleaned, open) };
+    });
     const refs = [];
+    const relationStartByDepth = new Map();
     for (const m of cleaned.matchAll(TABLE_REF_RE)) {
-        const table = m[1];
-        if (!known.has(table))
-            continue; // e.g. FROM (subquery) — table is `(`; skip
-        let alias = m[2];
+        const operator = m[1].toUpperCase();
+        const outerKind = m[2]?.toUpperCase();
+        const rawTable = m[4];
+        const operatorDepth = parenthesisDepthAt(cleaned, m.index ?? 0);
+        const relativeTableIndex = m[0]
+            .toLowerCase()
+            .indexOf(rawTable.toLowerCase(), m[1].length);
+        const tableIndex = (m.index ?? 0) + Math.max(m[1].length, relativeTableIndex);
+        const tableDepth = parenthesisDepthAt(cleaned, tableIndex);
+        if (operator === 'FROM') {
+            relationStartByDepth.set(operatorDepth, refs.length);
+            relationStartByDepth.set(tableDepth, refs.length);
+        }
+        else if (tableDepth > operatorDepth &&
+            !relationStartByDepth.has(tableDepth)) {
+            relationStartByDepth.set(tableDepth, refs.length);
+        }
+        if (outerKind === 'RIGHT' || outerKind === 'FULL') {
+            const relationStart = relationStartByDepth.get(operatorDepth) ?? refs.length;
+            for (let index = relationStart; index < refs.length; index += 1) {
+                const prior = refs[index];
+                if (prior !== undefined && !prior.nullable) {
+                    refs[index] = { ...prior, nullable: true };
+                }
+            }
+        }
+        const table = known.get(rawTable.toLowerCase());
+        if (table === undefined)
+            continue; // e.g. a derived SELECT/CTE name
+        let alias = m[5];
         if (alias !== undefined && RESERVED_ALIAS.has(alias.toLowerCase())) {
             alias = undefined;
         }
-        refs.push({ table, alias: alias ?? table });
+        refs.push({
+            table,
+            alias: alias ?? table,
+            nullable: outerKind === 'LEFT' ||
+                outerKind === 'FULL' ||
+                nullableGroups.some((group) => tableIndex > group.open && tableIndex < group.close),
+        });
     }
     return refs;
 }
@@ -532,43 +670,66 @@ function resolveSource(item, refs, ir) {
     const columnName = m[2];
     const byName = new Map(ir.tables.map((t) => [t.name, t]));
     if (qualifier !== undefined) {
-        const ref = refs.find((r) => r.alias === qualifier);
+        const ref = refs.find((candidate) => candidate.alias.toLowerCase() === qualifier.toLowerCase());
         if (ref === undefined)
             return null;
         const table = byName.get(ref.table);
-        const col = table?.columns.find((c) => c.name === columnName);
-        if (col !== undefined)
-            return { table: ref.table, column: col };
+        const col = table?.columns.find((candidate) => candidate.name.toLowerCase() === columnName.toLowerCase());
+        if (col !== undefined) {
+            return {
+                table: ref.table,
+                column: col,
+                nullableByJoin: ref.nullable,
+            };
+        }
         if (columnName === FTS_SOURCE_ID_COLUMN &&
             declaredFtsProjection(ir, ref.table)) {
-            return { table: ref.table, column: FTS_SOURCE_ID_IR_COLUMN };
+            return {
+                table: ref.table,
+                column: FTS_SOURCE_ID_IR_COLUMN,
+                nullableByJoin: ref.nullable,
+            };
         }
         return null;
     }
     // Unqualified: search every FROM/JOIN table (SQLite already resolved
-    // ambiguity; a single-table query is the common case).
+    // ambiguity; a single-table query is the common case). A USING/NATURAL
+    // coalesced name can have more than one physical origin, so keep it on the
+    // honest nullable fallback path instead of guessing one side.
+    const matches = [];
     for (const ref of refs) {
         const table = byName.get(ref.table);
-        const col = table?.columns.find((c) => c.name === columnName);
-        if (col !== undefined)
-            return { table: ref.table, column: col };
+        const col = table?.columns.find((candidate) => candidate.name.toLowerCase() === columnName.toLowerCase());
+        if (col !== undefined) {
+            matches.push({
+                table: ref.table,
+                column: col,
+                nullableByJoin: ref.nullable,
+            });
+        }
         if (columnName === FTS_SOURCE_ID_COLUMN &&
             declaredFtsProjection(ir, ref.table)) {
-            return { table: ref.table, column: FTS_SOURCE_ID_IR_COLUMN };
+            matches.push({
+                table: ref.table,
+                column: FTS_SOURCE_ID_IR_COLUMN,
+                nullableByJoin: ref.nullable,
+            });
         }
     }
-    return null;
+    return matches.length === 1 ? matches[0] : null;
 }
 /** Resolve the one supported query-only protocol column. SQLite still proves
  * qualifier/ambiguity against the synthesized local tables. */
-function isSyncVersionSource(item, refs) {
+function syncVersionSourceNullability(item, refs) {
     const match = PLAIN_REF_RE.exec(item.expr);
     if (match === null || match[2] !== SYNC_VERSION_COLUMN)
-        return false;
+        return null;
     const qualifier = match[1];
-    return qualifier === undefined
-        ? refs.length === 1
-        : refs.some((ref) => ref.alias === qualifier);
+    if (qualifier === undefined) {
+        return refs.length === 1 ? (refs[0]?.nullable ?? null) : null;
+    }
+    const matches = refs.filter((ref) => ref.alias.toLowerCase() === qualifier.toLowerCase());
+    return matches.length === 1 ? (matches[0]?.nullable ?? null) : null;
 }
 // -- param type inference -----------------------------------------------------
 //
@@ -883,6 +1044,9 @@ export function analyzeStatement(name, location, statementText, ir, db, naming =
     if (sourceSql.length === 0) {
         throw new TypegenError(file, 'query file is empty');
     }
+    if (hasCommaJoinedSchemaTable(sourceSql, ir)) {
+        throw new TypegenError(file, 'comma-separated table sources are unsupported because reactive proof requires every relation; use an explicit JOIN ... ON clause');
+    }
     // SELECT-only (the read tier). A `WITH` is allowed when its main statement
     // is a SELECT (SQLite also allows WITH … INSERT/UPDATE/DELETE — writes).
     const firstKeyword = /^\s*([A-Za-z]+)/.exec(stripCommentsAndStrings(statementText).trimStart())?.[1];
@@ -944,7 +1108,8 @@ export function analyzeStatement(name, location, statementText, ir, db, naming =
         const source = item !== undefined ? resolveSource(item, refs, ir) : null;
         const sqlName = sqlNames[index] ?? colName;
         const langName = langNames[index] ?? colName;
-        if (item !== undefined && isSyncVersionSource(item, refs)) {
+        const syncVersionNullable = item !== undefined ? syncVersionSourceNullability(item, refs) : null;
+        if (syncVersionNullable !== null) {
             if (sqlName === SYNC_VERSION_COLUMN) {
                 throw new TypegenError(file, `${SYNC_VERSION_COLUMN} is stripped from app-facing query rows unless explicitly aliased (for example \`${SYNC_VERSION_COLUMN} AS server_version\`)`);
             }
@@ -952,7 +1117,7 @@ export function analyzeStatement(name, location, statementText, ir, db, naming =
                 name: sqlName,
                 langName,
                 type: 'integer',
-                nullable: false,
+                nullable: syncVersionNullable,
                 fidelity: 'exact',
             };
         }
@@ -968,7 +1133,7 @@ export function analyzeStatement(name, location, statementText, ir, db, naming =
                 name: sqlName,
                 langName,
                 type: sourceType,
-                nullable: source.column.nullable,
+                nullable: source.column.nullable || source.nullableByJoin,
                 fidelity: 'exact',
                 origin: { table: source.table, column: source.column.name },
             };