inline-sql-toolkit 0.4.13 → 0.5.0

package/CHANGELOG.md

@@ -1,5 +1,38 @@
 # Changelog
 
+## 0.5.0 - 2026-10-01
+
+- Changed: `inlineSql.format.replaceOrdinals` now defaults to `false`, since it
+  rewrites the query rather than its layout. GROUP BY ordinals are replaced by
+  the copied expression (never an alias); ordinals at or after `*`, after
+  UNION / EXCEPT / INTERSECT, with SELECT modifiers, or naming volatile calls
+  are kept. The CLI gains `--ordinals`.
+- Fixed: keyword-only candidates need a SQL statement shape and a single-case
+  keyword, so prose such as "Update available" is no longer formatted.
+- Fixed: comment-, continuation- and `+ name +`-joined literals are treated as
+  concatenation (unsupported) instead of standalone SQL; single-quoted literals
+  keep their edge spaces.
+- Fixed: documents with an unterminated string, and f-strings that reuse their
+  own quote, are skipped.
+- Safety: every candidate whose SQL tokens (strings, identifiers, numbers,
+  symbols, comments) would change is skipped as `FORMATTER_FAILED`. DISTRIBUTE
+  splitting and f-string field markers no longer touch strings or comments.
+- Fixed: UTF-16 positions on lines with emoji; SQL marker matched in any case;
+  SQL-language notebook cells are no longer formatted as Python.
+- Fixed: notebook Format All re-checks cells, cancellation and trust before
+  one WorkspaceEdit; size limits report `RESOURCE_LIMIT_EXCEEDED`.
+- CLI: skipped candidates are reported on stderr (`-q` suppresses).
+- Performance: each candidate is formatted on its own text; Code Action scans
+  are cached per document version.
+
+## 0.4.14 - 2026-09-25
+
+- Changed: `inlineSql.format.commaPosition: "before"` emits the moved comma
+  with a space after it again (`id` / `    , name`). 0.4.13 removed the space
+  for a single release; the spaced form is the wanted style. The default
+  `after` output is unchanged, and `src/` and `test/` are byte-identical to
+  v0.4.12.
+
 ## 0.4.13 - 2026-09-25
 
 - Changed: `inlineSql.format.commaPosition: "before"` now emits the moved

package/README.md

@@ -1,5 +1,7 @@
 # Inline SQL Toolkit
 
+![Inline SQL Toolkit demo](https://raw.githubusercontent.com/hidenobunagai/inline-sql-toolkit/main/docs/demo.gif)
+
 Inline SQL Toolkit highlights SQL embedded in Python strings and formats selected
 SQL with the bundled `sql-formatter` layout engine. It supports ordinary `.py`
 files, marimo
@@ -106,13 +108,20 @@ later atomic precondition.
 - `inlineSql.format.useSpaceAroundOperators`: add spaces around operators
   (default `true`).
 - `inlineSql.format.replaceOrdinals`: replace `GROUP BY` / `ORDER BY` ordinal
-  numbers (`1`, `2`, ...) with the referenced column names, preferring
-  aliases (default `true`).
+  numbers (`1`, `2`, ...) with the referenced select-list column (default
+  `false`). This rewrites the query rather than its layout, so it is opt-in.
+  `GROUP BY` copies the column's expression, because PostgreSQL, MySQL, and
+  SQLite resolve a `GROUP BY` name to an input column before an alias;
+  `ORDER BY` uses the alias when it is unique. An ordinal is kept when its
+  position is unknown (at or after `*` / `t.*`), when it follows `UNION`,
+  `EXCEPT`, or `INTERSECT` (`ORDER BY`), when the replacement would name
+  another output column, or when the expression is an aggregate without an
+  alias or calls a volatile function such as `random()`.
 - `inlineSql.format.dialect`: SQL dialect used by the formatter (`sql`,
   `mysql`, `postgresql`, or `sqlite`; default `postgresql`).
 - `inlineSql.format.commaPosition`: `after` (default) keeps a wrapping comma at
   the end of the previous line; `before` moves it to the start of the next
-  line, with no space after the comma (`id` / `    ,name`).
+  line with a space after it (`id` / `    , name`).
 - `inlineSql.format.keepFunctionsInline`: re-join the formatter's line breaks
   inside `word(...)` groups so `SUM(...)`, `COUNT(CASE ... END)`, and nested
   calls stay on one line (default `false`). Newlines inside string literals and
@@ -129,7 +138,7 @@ disappears, disable semantic highlighting for the language server
 
 ## Command line (CLI)
 
-The `inline-sql-toolkit` command-line tool formats Python files or standard input using the exact same formatting engine and safety checks as the VS Code extension's **Format All** command. Unsafe candidates (such as invalid Python syntax, unsupported literals, or unparseable f-strings) are skipped rather than producing corrupt output.
+The `inline-sql-toolkit` command-line tool formats Python files or standard input using the exact same formatting engine and safety checks as the VS Code extension's **Format All** command. Unsafe candidates (such as unsupported literals, unparseable f-strings, or any candidate in a document with an unterminated string) are skipped rather than producing corrupt output.
 
 Run directly via `npx` or install with `bun add -d inline-sql-toolkit` (or `npm install -D inline-sql-toolkit`):
 
@@ -141,24 +150,28 @@ When run without file arguments, the CLI reads Python source from standard input
 
 ### CLI Options
 
-| Option                        | Description                                                       | Default                    |
-| ----------------------------- | ----------------------------------------------------------------- | -------------------------- |
-| `-w, --write`                 | Rewrite files in place (no write if unchanged)                    | off                        |
-| `--check`                     | Exit with code 1 if any file would change                         | off                        |
-| `--dialect <name>`            | SQL dialect: `sql`, `mysql`, `postgresql`, or `sqlite`            | `postgresql`               |
-| `--keyword-case <case>`       | Case for SQL keywords: `upper`, `lower`, or `preserve`            | `upper`                    |
-| `--indent-width <1-8>`        | SQL indentation width in spaces                                   | `2`                        |
-| `--wrap-after <20-500>`       | Preferred expression line width                                   | `88`                       |
-| `--no-space-around-operators` | Keep dense operators                                              | spaced                     |
-| `--no-ordinals`               | Do not replace `GROUP BY` / `ORDER BY` ordinals with column names | replace                    |
-| `--comma-position <pos>`      | Where a wrapping comma sits: `after` or `before`                  | `after`                    |
-| `--keep-functions-inline`     | Keep `SUM(...)` / `COUNT(CASE ... END)` on one line               | off                        |
-| `-c, --config <file>`         | Configuration JSON file                                           | Nearest `.inline-sql.json` |
-| `-h, --help`                  | Show usage help                                                   |                            |
-| `--version`                   | Show version number                                               |                            |
+| Option                        | Description                                                | Default                    |
+| ----------------------------- | ---------------------------------------------------------- | -------------------------- |
+| `-w, --write`                 | Rewrite files in place (no write if unchanged)             | off                        |
+| `--check`                     | Exit with code 1 if any file would change                  | off                        |
+| `--dialect <name>`            | SQL dialect: `sql`, `mysql`, `postgresql`, or `sqlite`     | `postgresql`               |
+| `--keyword-case <case>`       | Case for SQL keywords: `upper`, `lower`, or `preserve`     | `upper`                    |
+| `--indent-width <1-8>`        | SQL indentation width in spaces                            | `2`                        |
+| `--wrap-after <20-500>`       | Preferred expression line width                            | `88`                       |
+| `--no-space-around-operators` | Keep dense operators                                       | spaced                     |
+| `--ordinals`                  | Replace `GROUP BY` / `ORDER BY` ordinals with column names | off                        |
+| `--no-ordinals`               | Do not replace ordinals (overrides the configuration file) |                            |
+| `--comma-position <pos>`      | Where a wrapping comma sits: `after` or `before`           | `after`                    |
+| `--keep-functions-inline`     | Keep `SUM(...)` / `COUNT(CASE ... END)` on one line        | off                        |
+| `-c, --config <file>`         | Configuration JSON file                                    | Nearest `.inline-sql.json` |
+| `-q, --quiet`                 | Do not report skipped candidates on stderr                 | report                     |
+| `-h, --help`                  | Show usage help                                            |                            |
+| `--version`                   | Show version number                                        |                            |
 
 Exit codes: `0` on success, `1` when `--check` finds unformatted files, and `2` on usage, configuration, I/O, or formatting errors.
 
+A detected SQL candidate that cannot be formatted safely is left as it is and does not change the exit code. Unless `--quiet` is given, the CLI prints one line per affected file on stderr, such as `inline-sql-toolkit: app.py: skipped 2 SQL candidates (UNSUPPORTED_LITERAL x2)`; [SUPPORT.md](SUPPORT.md) lists the reason codes.
+
 ### Configuration file (`.inline-sql.json`)
 
 The CLI automatically searches for a `.inline-sql.json` file in the current working directory and its parent directories. The configuration structure matches VS Code settings:
@@ -170,7 +183,7 @@ The CLI automatically searches for a `.inline-sql.json` file in the current work
     "indentWidth": 2,
     "wrapAfter": 88,
     "useSpaceAroundOperators": true,
-    "replaceOrdinals": true,
+    "replaceOrdinals": false,
     "dialect": "postgresql",
     "commaPosition": "after",
     "keepFunctionsInline": false
@@ -194,7 +207,21 @@ is found when either condition holds:
    characters, the source starts with one of `SELECT`, `WITH`, `INSERT`,
    `UPDATE`, `DELETE`, `MERGE`, `CREATE`, `ALTER`, `DROP`, `TRUNCATE`, or
    `EXPLAIN`, followed by a word boundary. The two source characters `\n` are
-   not treated as whitespace.
+   not treated as whitespace before the keyword. The keyword must be written
+   entirely in upper or lower case (`SELECT` / `select`, never `Select`), and
+   it must open a recognizable statement so that prose and bare values are
+   left alone:
+   - `SELECT` is followed by a `FROM` clause or by a select-list expression
+     (`*`, a number, a string, a parameter, a function call such as `now()`,
+     `CASE`, ...), so `SELECT 1` matches but `Select an option` does not.
+   - `WITH name AS (`, `INSERT INTO`, `UPDATE table SET`, `DELETE FROM`,
+     `MERGE INTO`, `CREATE`/`ALTER`/`DROP` followed by an object kind such as
+     `TABLE`, `VIEW`, or `INDEX`, `TRUNCATE [TABLE] table`, and `EXPLAIN`
+     followed by a statement.
+
+   Inside this shape check, the escapes `\n`, `\r`, `\t`, and a
+   backslash-newline count as whitespace. Use the `--sql` marker for any
+   statement that does not fit these shapes.
 
 Standalone plain and raw strings, f-strings, and raw f-strings (`f`, `rf`, and
 `fr`, in either case) are supported with single, double, and triple delimiters
@@ -202,9 +229,22 @@ Standalone plain and raw strings, f-strings, and raw f-strings (`f`, `rf`, and
 also the candidate used by the syntax highlighting grammar.
 
 The following are intentionally skipped: bytes and byte strings (`b`/`rb`),
-implicit or explicit string concatenation, t-strings, invalid Python, dynamic
-or non-literal SQL, and SQL-language cells. A candidate that cannot be restored
-without changing Python source is reported as unsafe and is not edited.
+implicit or explicit string concatenation, t-strings, f-strings whose
+replacement field reuses the literal's own quote (`f"{row["id"]}"`), dynamic or
+non-literal SQL, and SQL-language cells. The extension does not run a Python
+parser: it scans string literals only. A document with an unterminated string
+literal is not formatted at all, because the scan can no longer tell code from
+string text; other syntax errors outside a literal do not stop formatting. Concatenation covers adjacent
+literals even across comments or `\` line continuations, and any literal joined
+with `+` or `+=`, whatever the other operand is. A candidate that cannot be
+restored without changing Python source is reported as unsafe and is not
+edited. Before any edit, the formatted SQL is lexed again and compared with
+the original: apart from whitespace and keyword case, every string, quoted
+identifier, number, operator, and comment must be unchanged and in the same
+order, or the candidate is skipped (`FORMATTER_FAILED`).
+
+A single-quoted literal stays on one line and keeps its leading and trailing
+spaces, because the string may be joined to other text at runtime.
 
 ## Trust, privacy, and offline behavior
 
@@ -221,7 +261,7 @@ over the network, passed to a shell/database, or executed. The bundled
   the first logical line has `-- sql`/`--sql`, or that a listed keyword is at
   the source-level start with a word boundary.
 - **Unsupported literal or unsafe f-string:** remove concatenation, bytes or
-  t-string syntax, and verify that Python parses the document. Complex f-string
+  t-string syntax, and close every string literal in the document. Complex f-string
   expressions are skipped when their source spans cannot be restored exactly.
 - **Formatting is unavailable:** use a trusted workspace and check the
   diagnostic reason shown by the extension (`WORKSPACE_UNTRUSTED`,

package/dist/cli.js

@@ -473,6 +473,7 @@ __export(cli_exports, {
   buildRawOptions: () => buildRawOptions,
   findConfigFile: () => findConfigFile,
   formatPythonSource: () => formatPythonSource,
+  formatPythonSourceWithReport: () => formatPythonSourceWithReport,
   getPackageVersion: () => getPackageVersion,
   loadConfigFile: () => loadConfigFile,
   runCli: () => runCli
@@ -492,7 +493,7 @@ function resolveFormatOptions(raw) {
   const indentWidth = raw.indentWidth === void 0 ? 2 : raw.indentWidth;
   const wrapAfter = raw.wrapAfter === void 0 ? 88 : raw.wrapAfter;
   const useSpaceAroundOperators = raw.useSpaceAroundOperators === void 0 ? true : raw.useSpaceAroundOperators;
-  const replaceOrdinals2 = raw.replaceOrdinals === void 0 ? true : raw.replaceOrdinals;
+  const replaceOrdinals2 = raw.replaceOrdinals === void 0 ? false : raw.replaceOrdinals;
   const dialect = raw.dialect === void 0 ? "postgresql" : raw.dialect;
   const commaPosition = raw.commaPosition === void 0 ? "after" : raw.commaPosition;
   const keepFunctionsInline = raw.keepFunctionsInline === void 0 ? false : raw.keepFunctionsInline;
@@ -610,7 +611,8 @@ var SourceMap = class _SourceMap {
   /** Convert a zero-based VS Code line and UTF-16 column to an offset. */
   offsetFromVscode(line, utf16Col) {
     const record = this.line(line);
-    return record.start + exactIndex(record.utf16AtCodepoint, utf16Col);
+    exactIndex(record.utf16AtCodepoint, utf16Col);
+    return record.start + utf16Col;
   }
   /** Convert a source offset to a strict zero-based VS Code position. */
   vscodeFromOffset(offset) {
@@ -625,11 +627,8 @@ var SourceMap = class _SourceMap {
     if (offset > record.contentEnd) {
       throw new PositionMappingError("offset is inside a line terminator");
     }
-    const codepointCol = offset - record.start;
-    const character = record.utf16AtCodepoint[codepointCol];
-    if (character === void 0) {
-      throw new PositionMappingError("column is not a code-point boundary");
-    }
+    const character = offset - record.start;
+    exactIndex(record.utf16AtCodepoint, character);
     return { line: lineIndex, character };
   }
   /** Convert a source span to a half-open VS Code range. */
@@ -640,20 +639,74 @@ var SourceMap = class _SourceMap {
 
 // src/python-analysis/detection.ts
 var MARKERS = /* @__PURE__ */ new Set(["-- sql", "--sql"]);
-var KEYWORDS = [
-  "select",
-  "with",
-  "insert",
-  "update",
-  "delete",
-  "merge",
-  "create",
-  "alter",
-  "drop",
-  "truncate",
-  "explain"
-];
+function isSqlMarker(line) {
+  return MARKERS.has(line.trim().toLowerCase());
+}
 var ASCII_WHITESPACE = /* @__PURE__ */ new Set([" ", "	", "\r", "\n"]);
+var IDENT = String.raw`(?:[\p{L}_][\p{L}\p{N}_$]*|"[^"]*"|\x60[^\x60]*\x60|\[[^\]]*\]|\{[^{}]*\}|%(?:\([^)]*\))?s)`;
+var TABLE_REF = String.raw`${IDENT}(?:\s*\.\s*${IDENT})*`;
+var OBJECT = String.raw`(?:table|view|index|schema|database|function|procedure|trigger|sequence|type|extension|domain|policy|aggregate|collation|tablespace|publication|subscription|event|server|rule|statistics)`;
+var CREATE_MODIFIER = String.raw`(?:global|local|temp|temporary|unlogged|unique|materialized|recursive|virtual|external|fulltext|spatial|secure|transient)`;
+var SELECT_LIST_START = new RegExp(
+  String.raw`^\s*(?:[*(\d'"\x60{:@$?%-]|[\p{L}_][\p{L}\p{N}_$.]*\s*\(|(?:case|cast|exists|not|null|true|false|current_\w+|localtime\w*)\b)`,
+  "iu"
+);
+var KEYWORD_SHAPES = [
+  [
+    "select",
+    (text) => {
+      const rest = text.slice("select".length);
+      return /\bfrom\b/i.test(rest) || SELECT_LIST_START.test(rest);
+    }
+  ],
+  [
+    "with",
+    shape(
+      String.raw`with\s+(?:recursive\s+)?${IDENT}\s*(?:\([^)]*\)\s*)?as\s*(?:(?:not\s+)?materialized\s*)?\(`
+    )
+  ],
+  [
+    "insert",
+    shape(
+      String.raw`insert\s+(?:or\s+\w+\s+)?(?:(?:low_priority|delayed|high_priority|ignore)\s+)*(?:into|overwrite)\b`
+    )
+  ],
+  [
+    "update",
+    shape(
+      String.raw`update\s+(?:(?:only|low_priority|ignore)\s+|or\s+\w+\s+)*${TABLE_REF}(?:\s+(?:as\s+)?${IDENT})?\s+set\b`
+    )
+  ],
+  [
+    "delete",
+    shape(
+      String.raw`delete\s+(?:(?:low_priority|quick|ignore)\s+)*(?:from\b|${TABLE_REF}(?:\s*,\s*${TABLE_REF})*\s+from\b)`
+    )
+  ],
+  ["merge", shape(String.raw`merge\s+into\b`)],
+  ["create", shape(String.raw`create\s+(?:or\s+replace\s+)?(?:${CREATE_MODIFIER}\s+)*${OBJECT}\b`)],
+  ["alter", shape(String.raw`alter\s+(?:materialized\s+)?${OBJECT}\b`)],
+  ["drop", shape(String.raw`drop\s+(?:${CREATE_MODIFIER}\s+)*${OBJECT}\b`)],
+  [
+    "truncate",
+    shape(
+      String.raw`truncate\s+(?:table\b|(?:only\s+)?${TABLE_REF}(?:\s*,\s*${TABLE_REF})*\s*(?:;|$|(?:restart|continue|cascade|restrict)\b))`
+    )
+  ],
+  [
+    "explain",
+    shape(
+      String.raw`explain\s+(?:(?:analyze|analyse|verbose|extended|query\s+plan)\s+|\([^)]*\)\s*)*(?:select|with|insert|update|delete|merge|values|table|execute|create|declare)\b`
+    )
+  ]
+];
+function shape(pattern) {
+  const regex = new RegExp(`^${pattern}`, "iu");
+  return (text) => regex.test(text);
+}
+function shapeText(text) {
+  return text.replace(/\\(?:[nrt]|\r\n|\r|\n)/g, " ");
+}
 function continuesIdentifier(character) {
   if (character === "") return false;
   if (character === "_") return true;
@@ -675,7 +728,7 @@ function detectSourceSlice(text, base) {
       cursor += line.length;
       continue;
     }
-    if (MARKERS.has(body.trim().toLowerCase())) {
+    if (isSqlMarker(body)) {
       const marker = new SourceSpan(base + cursor, base + cursor + body.length);
       return {
         matched: true,
@@ -690,11 +743,12 @@ function detectSourceSlice(text, base) {
   while (significant < text.length && ASCII_WHITESPACE.has(text[significant] ?? "")) {
     significant++;
   }
-  const folded = text.slice(significant).toLowerCase();
-  for (const keyword of KEYWORDS) {
-    if (!folded.startsWith(keyword)) continue;
+  const statement = text.slice(significant);
+  for (const [keyword, matchesShape] of KEYWORD_SHAPES) {
+    const written = statement.slice(0, keyword.length);
+    if (written !== keyword && written !== keyword.toUpperCase()) continue;
     const following = text[significant + keyword.length] ?? "";
-    if (!continuesIdentifier(following)) {
+    if (!continuesIdentifier(following) && matchesShape(shapeText(statement))) {
       return {
         matched: true,
         markerSpan: void 0,
@@ -752,8 +806,9 @@ function stringSurfaceAt(source, index) {
     contentSpan: new SourceSpan(contentStart, end - delimiter.length)
   };
 }
-function scanStringSurfaces(source) {
+function scanSource(source) {
   const surfaces = [];
+  let unterminated = false;
   let index = 0;
   while (index < source.length) {
     const char = source[index] ?? "";
@@ -773,6 +828,7 @@ function scanStringSurfaces(source) {
         index = surface.span.end;
         continue;
       }
+      if (char === '"' || char === "'") unterminated = true;
     }
     if (NAME.test(source.slice(index))) {
       const name = NAME.exec(source.slice(index))?.[0];
@@ -781,9 +837,9 @@ function scanStringSurfaces(source) {
     }
     index++;
   }
-  return surfaces;
+  return { surfaces, unterminated };
 }
-function scanFstringFieldSpans(source, contentSpan) {
+function scanFstringFields(source, contentSpan) {
   const fields = [];
   let fieldStart = -1;
   const closers = [];
@@ -853,7 +909,7 @@ function scanFstringFieldSpans(source, contentSpan) {
     }
     index++;
   }
-  return fields;
+  return { fields, unclosed: fieldStart !== -1 };
 }
 function fstringKind(prefix) {
   const normalized = prefix.toLowerCase();
@@ -866,25 +922,26 @@ function fstringKind(prefix) {
 function onlyWhitespace(text) {
   return /^[\s]*$/.test(text);
 }
+function codeOnly(text) {
+  return text.replace(/#.*$/gm, "").replace(/\\(?:\r\n|\r|\n)/g, " ");
+}
+var OPERATOR_ONLY = /^\s*[+\-*/%&|^<>]+\s*$/;
 function isConcatenated(source, surfaces, index) {
   const current = surfaces[index];
   if (current === void 0) return false;
-  const neighbors = [
-    index > 0 ? surfaces[index - 1] : void 0,
-    index + 1 < surfaces.length ? surfaces[index + 1] : void 0
-  ];
-  return neighbors.some((neighbor) => {
-    if (neighbor === void 0) return false;
-    const between = neighbor.span.end <= current.span.start ? source.slice(neighbor.span.end, current.span.start) : source.slice(current.span.end, neighbor.span.start);
-    if (onlyWhitespace(between)) return true;
-    const code = between.replace(/#.*$/gm, "");
-    if (onlyWhitespace(code)) return false;
-    return /^\s*[+\-*/%&|^<>]+\s*$/.test(code);
-  });
+  const previous = surfaces[index - 1];
+  const next = surfaces[index + 1];
+  const before = codeOnly(source.slice(previous?.span.end ?? 0, current.span.start));
+  const after = codeOnly(source.slice(current.span.end, next?.span.start ?? source.length));
+  if (previous !== void 0 && onlyWhitespace(before)) return true;
+  if (next !== void 0 && onlyWhitespace(after)) return true;
+  if (/\+=?\s*$/.test(before) || /^\s*\+/.test(after)) return true;
+  if (previous !== void 0 && OPERATOR_ONLY.test(before)) return true;
+  return next !== void 0 && OPERATOR_ONLY.test(after);
 }
 function analyzeDocument(source) {
   const sourceMap = SourceMap.fromText(source);
-  const surfaces = scanStringSurfaces(source);
+  const { surfaces, unterminated } = scanSource(source);
   const supported = [];
   const unsupported = [];
   surfaces.forEach((surface, index) => {
@@ -915,10 +972,12 @@ function analyzeDocument(source) {
     }
     if (surface.kind === "fstring") {
       const kind = fstringKind(surface.prefix);
-      if (kind === void 0) {
+      const fields = scanFstringFields(source, surface.contentSpan);
+      if (kind === void 0 || fields.unclosed) {
         unsupported.push({
           span: surface.span,
-          detectionContentSpan: void 0,
+          // Still SQL to the user: report it as skipped rather than ignore it.
+          detectionContentSpan: kind === void 0 ? void 0 : surface.contentSpan,
           reason: "UNSUPPORTED_LITERAL"
         });
         return;
@@ -929,7 +988,7 @@ function analyzeDocument(source) {
         prefix: surface.prefix,
         delimiter: surface.delimiter,
         kind,
-        fieldSpans: scanFstringFieldSpans(source, surface.contentSpan)
+        fieldSpans: fields.fields
       });
       return;
     }
@@ -944,6 +1003,7 @@ function analyzeDocument(source) {
   });
   return {
     sourceMap,
+    unterminated,
     supported: [...supported].sort((left, right) => left.span.start - right.span.start),
     unsupported: [...unsupported].sort((left, right) => left.span.start - right.span.start)
   };
@@ -1231,6 +1291,31 @@ var AGGREGATE_FUNCTIONS = /* @__PURE__ */ new Set([
   "sum",
   "xmlagg"
 ]);
+var VOLATILE_FUNCTIONS = /* @__PURE__ */ new Set([
+  "clock_timestamp",
+  "gen_random_uuid",
+  "newid",
+  "nextval",
+  "rand",
+  "random",
+  "setval",
+  "timeofday",
+  "uuid",
+  "uuid_generate_v1",
+  "uuid_generate_v4"
+]);
+var SELECT_MODIFIERS = /* @__PURE__ */ new Set([
+  "all",
+  "distinct",
+  "distinctrow",
+  "high_priority",
+  "sql_big_result",
+  "sql_buffer_result",
+  "sql_calc_found_rows",
+  "sql_no_cache",
+  "sql_small_result",
+  "straight_join"
+]);
 var ORDINAL_SUFFIXES = /* @__PURE__ */ new Set(["asc", "desc", "nulls", "first", "last"]);
 var RESERVED = /* @__PURE__ */ new Set([
   "as",
@@ -1298,9 +1383,9 @@ function isSelectStart(tokens, index) {
   if (index === 0) return true;
   const previous = tokens[index - 1];
   const text = previous?.text.toLowerCase() ?? "";
-  if (text === "all") {
+  if (text === "all" || text === "distinct") {
     const before = tokens[index - 2]?.text.toLowerCase() ?? "";
-    return before === "union" || before === "except";
+    return before === "union" || before === "except" || before === "intersect";
   }
   return text === "(" || text === ";" || text === ")" || text === "union" || text === "except" || text === "intersect" || text.startsWith("--") || text.startsWith("{");
 }
@@ -1310,12 +1395,51 @@ function isOperator(token) {
 function isCommentToken(token) {
   return token.text.startsWith("--") || token.text.startsWith("/*");
 }
+function isSetOperand(tokens, index) {
+  const operator2 = (offset) => ["union", "except", "intersect"].includes(tokens[index - offset]?.text.toLowerCase() ?? "");
+  const previous = tokens[index - 1]?.text.toLowerCase() ?? "";
+  return operator2(1) || (previous === "all" || previous === "distinct") && operator2(2);
+}
+function firstItemStart(tokens, start) {
+  let index = start;
+  while (index < tokens.length) {
+    const token = tokens[index];
+    if (token === void 0 || isCommentToken(token)) {
+      index += 1;
+      continue;
+    }
+    if (!SELECT_MODIFIERS.has(token.text.toLowerCase())) break;
+    index += 1;
+    const on = tokens[index];
+    const open = tokens[index + 1];
+    if (token.text.toLowerCase() === "distinct" && on !== void 0 && isKeyword(on, "on")) {
+      if (open?.text !== "(") break;
+      const close = tokens.findIndex(
+        (candidate, at) => at > index + 1 && candidate.text === ")" && candidate.depth === open.depth + 1
+      );
+      if (close === -1) return tokens.length;
+      index = close + 1;
+    }
+  }
+  return index;
+}
+function normalizeName(text) {
+  return text.replace(/^["`]|["`]$/g, "").toLowerCase();
+}
 function columnOf(tokens, start, end) {
   const visible = tokens.slice(start, end).filter((token) => !isCommentToken(token));
   const first = visible[0];
   const last2 = visible[visible.length - 1];
   if (first === void 0 || last2 === void 0) {
-    return { expressionStart: 0, expressionEnd: 0, alias: void 0, aggregate: false };
+    return {
+      expressionStart: 0,
+      expressionEnd: 0,
+      alias: void 0,
+      aggregate: false,
+      star: false,
+      outputName: void 0,
+      names: []
+    };
   }
   const secondLast = visible[visible.length - 2];
   const simpleColumn = visible.length % 2 === 1 && visible.every((token, i) => i % 2 === 0 ? isNameToken(token) : token.text === ".");
@@ -1329,34 +1453,70 @@ function columnOf(tokens, start, end) {
     expressionEnd = last2.start;
   }
   const aggregate = AGGREGATE_FUNCTIONS.has(first.text.toLowerCase()) || first.text === "*";
+  const star = last2.text === "*" && (visible.length === 1 || secondLast?.text === ".");
+  const outputName = alias !== void 0 ? normalizeName(alias) : simpleColumn ? normalizeName(last2.text) : void 0;
+  const names = visible.filter(
+    (token) => token.end <= expressionEnd && (isNameToken(token) || /^["`]/.test(token.text))
+  ).map((token) => normalizeName(token.text));
   return {
     expressionStart: first.start,
     expressionEnd,
     alias,
-    aggregate
+    aggregate,
+    star,
+    outputName,
+    names
   };
 }
-function flatten(text) {
-  return text.replace(/\s+/g, " ").trim();
+function ordinalReplacement(sql2, tokens, scope, clause, ordinal) {
+  const column = scope.columns[ordinal - 1];
+  if (column === void 0) return void 0;
+  if (scope.columns.slice(0, ordinal).some((candidate) => candidate.star)) return void 0;
+  if (clause === "order" && scope.setOperand) return void 0;
+  const others = scope.columns.filter((candidate) => candidate !== column);
+  const otherNames = new Set(others.map((candidate) => candidate.outputName));
+  if (clause === "order" && column.alias !== void 0) {
+    return otherNames.has(column.outputName) ? void 0 : column.alias;
+  }
+  if (column.aggregate || column.names.some((name) => VOLATILE_FUNCTIONS.has(name))) {
+    return void 0;
+  }
+  if (clause === "order" && column.names.some((name) => otherNames.has(name))) return void 0;
+  const expression = expressionText(sql2, tokens, column);
+  if (expression.length === 0 || expression.includes("{") || expression.includes("%")) {
+    return void 0;
+  }
+  return expression;
 }
 function expressionText(sql2, tokens, column) {
   let result = "";
   let cursor = column.expressionStart;
+  let spaced = false;
   for (const token of tokens) {
     if (token.start < column.expressionStart || token.end > column.expressionEnd) continue;
-    if (!isCommentToken(token)) continue;
-    result += sql2.slice(cursor, token.start);
+    const gap = sql2.slice(cursor, token.start);
+    if (gap.trim() !== "") return "";
+    spaced ||= gap !== "";
     cursor = token.end;
+    if (isCommentToken(token)) {
+      spaced = true;
+      continue;
+    }
+    if (spaced && result !== "" && !result.endsWith("(") && token.text !== ")" && token.text !== ",") {
+      result += " ";
+    }
+    result += token.text;
+    spaced = false;
   }
-  result += sql2.slice(cursor, column.expressionEnd);
-  return flatten(result);
+  return sql2.slice(cursor, column.expressionEnd).trim() === "" ? result : "";
 }
 function replaceOrdinals(sql2) {
   const tokens = tokenize(sql2);
   const scopes = [];
   const replacements = [];
-  const newScope = (depth, columnStart) => ({
+  const newScope = (depth, columnStart, setOperand) => ({
     depth,
+    setOperand,
     columns: [],
     phase: "select",
     columnStart,
@@ -1379,7 +1539,10 @@ function replaceOrdinals(sql2) {
   };
   const finishScope = (scope, endIndex) => {
     closeClause(scope, endIndex);
-    for (const clause of [scope.groupBy, scope.orderBy]) {
+    for (const [kind, clause] of [
+      ["group", scope.groupBy],
+      ["order", scope.orderBy]
+    ]) {
       if (clause === void 0) continue;
       for (const [start, end] of clause.itemRanges) {
         const itemTokens = tokens.slice(start, end);
@@ -1392,16 +1555,8 @@ function replaceOrdinals(sql2) {
         }
         if (ordinalToken === void 0 || !isNumberToken(ordinalToken)) continue;
         const ordinal = Number.parseInt(ordinalToken.text, 10);
-        const column = scope.columns[ordinal - 1];
-        if (column === void 0) continue;
-        let text;
-        if (column.alias !== void 0) {
-          text = column.alias;
-        } else if (!column.aggregate) {
-          const expression = expressionText(sql2, tokens, column);
-          if (!expression.includes("{") && !expression.includes("%")) text = expression;
-        }
-        if (text === void 0 || text.length === 0) continue;
+        const text = ordinalReplacement(sql2, tokens, scope, kind, ordinal);
+        if (text === void 0) continue;
         replacements.push({ start: ordinalToken.start, end: ordinalToken.end, text });
       }
     }
@@ -1421,7 +1576,9 @@ function replaceOrdinals(sql2) {
         finishScope(top, index);
         scopes.pop();
       }
-      scopes.push(newScope(token.depth, index + 1));
+      scopes.push(
+        newScope(token.depth, firstItemStart(tokens, index + 1), isSetOperand(tokens, index))
+      );
       continue;
     }
     const scope = scopes[scopes.length - 1];
@@ -1488,6 +1645,138 @@ function replaceOrdinals(sql2) {
   return result;
 }
 
+// src/python-analysis/sql-lexer.ts
+var SPACE = /\s+/y;
+var WORD = /[\p{L}_][\p{L}\p{N}_$]*/uy;
+var NUMBER2 = /(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][+-]?\d+)?/y;
+var DOLLAR_TAG = /\$(?:[A-Za-z_][A-Za-z0-9_]*)?\$/y;
+function quotedEnd(text, start, quote, backslash) {
+  let index = start + 1;
+  while (index < text.length) {
+    const char = text[index];
+    if (backslash && char === "\\") {
+      index += 2;
+      continue;
+    }
+    if (char === quote) {
+      if (text[index + 1] === quote) {
+        index += 2;
+        continue;
+      }
+      return index + 1;
+    }
+    index += 1;
+  }
+  return text.length;
+}
+function lexSql(text, dialect) {
+  const mysql2 = dialect === "mysql";
+  const tokens = [];
+  const push = (kind, start, end) => {
+    tokens.push({ kind, text: text.slice(start, end), start, end });
+  };
+  const sticky = (pattern, at) => {
+    pattern.lastIndex = at;
+    return pattern.exec(text) === null ? void 0 : pattern.lastIndex;
+  };
+  let index = 0;
+  while (index < text.length) {
+    const char = text[index] ?? "";
+    const next = text[index + 1] ?? "";
+    const spaceEnd = sticky(SPACE, index);
+    if (spaceEnd !== void 0) {
+      push("space", index, spaceEnd);
+      index = spaceEnd;
+      continue;
+    }
+    if (char === "-" && next === "-" || mysql2 && char === "#") {
+      const newline = text.slice(index).search(/[\r\n]/);
+      const end = newline === -1 ? text.length : index + newline;
+      push("line_comment", index, end);
+      index = end;
+      continue;
+    }
+    if (char === "/" && next === "*") {
+      const close = text.indexOf("*/", index + 2);
+      const end = close === -1 ? text.length : close + 2;
+      push("block_comment", index, end);
+      index = end;
+      continue;
+    }
+    if (char === "'") {
+      const previous = tokens[tokens.length - 1];
+      const escaped = mysql2 || previous?.kind === "word" && /^[eE]$/.test(previous.text) && previous.end === index;
+      const end = quotedEnd(text, index, "'", escaped);
+      push("string", index, end);
+      index = end;
+      continue;
+    }
+    if (char === '"' || char === "`") {
+      const end = quotedEnd(text, index, char, mysql2 && char === '"');
+      push(mysql2 && char === '"' ? "string" : "quoted", index, end);
+      index = end;
+      continue;
+    }
+    if (char === "$") {
+      const tagEnd = sticky(DOLLAR_TAG, index);
+      if (tagEnd !== void 0) {
+        const tag = text.slice(index, tagEnd);
+        const close = text.indexOf(tag, tagEnd);
+        const end = close === -1 ? text.length : close + tag.length;
+        push("dollar", index, end);
+        index = end;
+        continue;
+      }
+    }
+    const wordEnd = sticky(WORD, index);
+    if (wordEnd !== void 0) {
+      push("word", index, wordEnd);
+      index = wordEnd;
+      continue;
+    }
+    const numberEnd = sticky(NUMBER2, index);
+    if (numberEnd !== void 0) {
+      push("number", index, numberEnd);
+      index = numberEnd;
+      continue;
+    }
+    const codePoint = text.codePointAt(index) ?? 0;
+    const width = codePoint > 65535 ? 2 : 1;
+    push("symbol", index, index + width);
+    index += width;
+  }
+  return tokens;
+}
+function commentText(token) {
+  return token.text.replace(/\s+/g, " ").trim();
+}
+function sqlTokenDifference(before, after, dialect) {
+  const signature = (text) => {
+    const code = [];
+    const comments = [];
+    for (const token of lexSql(text, dialect)) {
+      if (token.kind === "space") continue;
+      if (token.kind === "line_comment" || token.kind === "block_comment") {
+        comments.push(commentText(token));
+      } else {
+        code.push(`${token.kind}:${token.kind === "word" ? token.text.toLowerCase() : token.text}`);
+      }
+    }
+    return { code, comments };
+  };
+  const left = signature(before);
+  const right = signature(after);
+  for (const part of ["code", "comments"]) {
+    const length = Math.max(left[part].length, right[part].length);
+    for (let index = 0; index < length; index += 1) {
+      if (left[part][index] !== right[part][index]) {
+        return `${part} token ${index} (${left[part].length} before, ${right[part].length} after)`;
+      }
+    }
+  }
+  return void 0;
+}
+
 // src/python-analysis/validation.ts
 var CandidateFailure = class extends Error {
   reason;
@@ -1513,7 +1802,7 @@ function applyBaseIndent(text, baseIndent, extraIndent, tripleQuoted) {
   const first = nonEmpty[0];
   if (first === void 0) return text;
   const firstTrimmed = first.line.trim();
-  const isMarker = firstTrimmed.startsWith("--sql") || firstTrimmed.startsWith("-- sql");
+  const isMarker = isSqlMarker(firstTrimmed);
   const keepsFirstLine = !tripleQuoted || isMarker;
   const shifted = keepsFirstLine ? nonEmpty.slice(1) : nonEmpty;
   const minIndent = shifted.length === 0 ? 0 : Math.min(...shifted.map(({ line }) => /^[ \t]*/.exec(line)?.[0].length ?? 0));
@@ -1526,12 +1815,9 @@ function normalizeFrame(content, literal, analysis, baseIndent) {
   if (literal.delimiter.length !== 3 || !content.includes("\n")) return content;
   const sourceContent = analysis.sourceMap.slice(literal.contentSpan);
   const firstNonEmptyLine = sourceContent.split("\n").find((line) => line.trim() !== "");
-  const startsWithMarker = firstNonEmptyLine?.trim().startsWith("--sql") || firstNonEmptyLine?.trim().startsWith("-- sql");
+  const startsWithMarker = firstNonEmptyLine !== void 0 && isSqlMarker(firstNonEmptyLine);
   const lines = content.split("\n");
-  const markerIndex = lines.findIndex((line) => {
-    const trimmed = line.trim();
-    return trimmed.startsWith("--sql") || trimmed.startsWith("-- sql");
-  });
+  const markerIndex = lines.findIndex(isSqlMarker);
   let normalized = content;
   if (startsWithMarker) {
     if (markerIndex > 0) {
@@ -1548,11 +1834,15 @@ ${normalized}`;
   }
   return `${normalized}${baseIndent}`;
 }
-function breakTrailingFieldMarkers(text) {
+function breakTrailingFieldMarkers(text, dialect) {
   const markerPattern = /(__INLINE_SQL_[0-9a-f]{32}_[A-Z_]+_[0-9]+__)\s*$/;
   const match = markerPattern.exec(text);
   if (match === null || match[1] === void 0) return text;
   const markerStart = match.index;
+  const token = lexSql(text, dialect).find(
+    (candidate) => candidate.start <= markerStart && markerStart < candidate.end
+  );
+  if (token?.kind !== "word") return text;
   const lineStart = text.lastIndexOf("\n", markerStart - 1) + 1;
   const before = text.slice(lineStart, markerStart);
   if (before.trim() === "") return text;
@@ -1581,12 +1871,25 @@ function moveCommasBeforeLineComments(text) {
   }
   return result.join("\n");
 }
-function breakTrailingDistributeLines(text) {
-  return text.replace(
-    /^([ \t]*)(.*?)\s+(DISTRIBUTE\b.*)$/gim,
-    (_, indent, before, rest) => `${indent}${before}
-${indent}${rest}`
-  );
+function breakTrailingDistributeLines(text, dialect) {
+  const tokens = lexSql(text, dialect);
+  const splits = [];
+  tokens.forEach((token, index) => {
+    if (token.kind !== "word" || token.text.toLowerCase() !== "distribute") return;
+    const gap = tokens[index + 1];
+    const clause = tokens[index + 2];
+    if (gap?.kind !== "space" || /[\r\n]/.test(gap.text) || clause?.kind !== "word") return;
+    const lineStart = text.lastIndexOf("\n", token.start - 1) + 1;
+    if (text.slice(lineStart, token.start).trim() !== "") splits.push(token.start);
+  });
+  let result = text;
+  for (const start of splits.reverse()) {
+    const lineStart = result.lastIndexOf("\n", start - 1) + 1;
+    const indent = /^[ \t]*/.exec(result.slice(lineStart))?.[0] ?? "";
+    result = `${result.slice(0, start).trimEnd()}
+${indent}${result.slice(start)}`;
+  }
+  return result;
 }
 function wrappingComma(line) {
   let comma = -1;
@@ -1616,7 +1919,7 @@ function moveCommasToLineStarts(text) {
     const indent = nextParts?.[1] ?? "";
     const content = nextParts?.[2] ?? "";
     lines[index] = `${line.slice(0, comma)}${line.slice(comma + 1)}`.trimEnd();
-    lines[index + 1] = `${indent},${content}`;
+    lines[index + 1] = `${indent}, ${content}`;
   }
   return lines.join("\n");
 }
@@ -1726,7 +2029,7 @@ function rejoinFunctionCalls(text) {
   }
   return out;
 }
-function formatOnce(analysis, literal, detection, options, nonce, sqlFormatter) {
+function formatOnce(analysis, literal, detection, options, nonce, sqlFormatter, baseIndent) {
   const plan = buildProtectionPlan(analysis.sourceMap, literal, detection, nonce);
   let formatted = sqlFormatter(plan.protectedSql, {
     tripleQuoted: literal.delimiter.length === 3,
@@ -1738,7 +2041,7 @@ function formatOnce(analysis, literal, detection, options, nonce, sqlFormatter)
       formatted = formatted.replace(fragment.marker + lineEnding, fragment.marker);
     }
   }
-  formatted = breakTrailingFieldMarkers(formatted);
+  formatted = breakTrailingFieldMarkers(formatted, options.dialect);
   formatted = moveCommasBeforeLineComments(formatted);
   if (options.keepFunctionsInline) {
     formatted = rejoinFunctionCalls(formatted);
@@ -1746,76 +2049,71 @@ function formatOnce(analysis, literal, detection, options, nonce, sqlFormatter)
   if (options.commaPosition === "before") {
     formatted = moveCommasToLineStarts(formatted);
   }
-  formatted = breakTrailingDistributeLines(formatted);
+  formatted = breakTrailingDistributeLines(formatted, options.dialect);
+  const difference = sqlTokenDifference(plan.protectedSql, formatted, options.dialect);
+  if (difference !== void 0) {
+    throw new CandidateFailure("FORMATTER_FAILED", `formatting changed the SQL: ${difference}`);
+  }
   const restored = restoreProtected(formatted, plan);
   const resolved = options.replaceOrdinals ? replaceOrdinals(restored) : restored;
-  const baseIndent = baseIndentOf(analysis, literal);
-  const indented = applyBaseIndent(
-    resolved,
-    baseIndent,
-    " ".repeat(options.indentWidth),
-    literal.delimiter.length === 3
-  );
+  if (literal.delimiter.length !== 3) {
+    return literalText(
+      literal,
+      singleLineContent(analysis.sourceMap.slice(literal.contentSpan), resolved)
+    );
+  }
+  const indented = applyBaseIndent(resolved, baseIndent, " ".repeat(options.indentWidth), true);
   return literalText(literal, normalizeFrame(indented, literal, analysis, baseIndent));
 }
-function replaceSource(source, span, replacement) {
-  return source.slice(0, span.start) + replacement + source.slice(span.end);
+function singleLineContent(sourceContent, formatted) {
+  const leading = /^[ \t]*/.exec(sourceContent)?.[0] ?? "";
+  const trailing = /[ \t]*$/.exec(sourceContent)?.[0] ?? "";
+  return `${leading}${formatted.replace(/\s*\n\s*/g, " ").trim()}${trailing}`;
 }
-function replacementLiteral(updated, original) {
-  const matches = updated.supported.filter((item) => item.span.start === original.span.start);
-  if (matches.length !== 1) {
-    throw new CandidateFailure("FORMATTER_FAILED", `reparse found ${matches.length} literals`);
+function analyzeLiteral(text, original) {
+  let analysis;
+  try {
+    analysis = analyzeDocument(text);
+  } catch (error) {
+    const message = error instanceof Error ? error.message.split("\n")[0] : String(error);
+    throw new CandidateFailure("FORMATTER_FAILED", `reparse threw: ${message}`);
   }
-  const result = matches[0];
-  if (result === void 0) throw new CandidateFailure("FORMATTER_FAILED", "reparse found none");
-  if (result.prefix !== original.prefix || result.delimiter !== original.delimiter || result.kind !== original.kind) {
+  const literal = analysis.supported[0];
+  if (literal === void 0 || analysis.supported.length !== 1 || analysis.unsupported.length !== 0 || literal.span.start !== 0 || literal.span.end !== text.length) {
     throw new CandidateFailure("UNSAFE_RAW_STRING");
   }
-  return result;
+  if (literal.prefix !== original.prefix || literal.delimiter !== original.delimiter || literal.kind !== original.kind) {
+    throw new CandidateFailure("UNSAFE_RAW_STRING");
+  }
+  return { analysis, literal };
 }
 function fieldTexts(analysis, literal) {
   return literal.fieldSpans.map((span) => analysis.sourceMap.slice(span));
 }
-function convergeToFixedPoint(source, analysis, literal, detection, options, nonce, sqlFormatter) {
-  let current = formatOnce(analysis, literal, detection, options, nonce, sqlFormatter);
+function convergeToFixedPoint(analysis, literal, detection, options, nonce, sqlFormatter, baseIndent) {
+  const fieldsBefore = fieldTexts(analysis, literal).join("\0");
+  let current = formatOnce(analysis, literal, detection, options, nonce, sqlFormatter, baseIndent);
   for (let attempt = 0; attempt < 3; attempt += 1) {
-    try {
-      const reparsed = analyzeDocument(current);
-      if (reparsed.supported.length !== 1 || reparsed.unsupported.length !== 0) {
-        throw new CandidateFailure("UNSAFE_RAW_STRING");
-      }
-    } catch (error) {
-      if (error instanceof CandidateFailure) throw error;
-      throw new CandidateFailure("UNSAFE_RAW_STRING");
-    }
-    const updatedSource = replaceSource(source, literal.span, current);
-    let updatedAnalysis;
-    try {
-      updatedAnalysis = analyzeDocument(updatedSource);
-    } catch (error) {
-      const message = error instanceof Error ? error.message.split("\n")[0] : String(error);
-      throw new CandidateFailure("FORMATTER_FAILED", `reparse threw: ${message}`);
-    }
-    const updatedLiteral = replacementLiteral(updatedAnalysis, literal);
-    const before = fieldTexts(analysis, literal).join("\0");
-    const after = fieldTexts(updatedAnalysis, updatedLiteral).join("\0");
-    if (before !== after) {
+    const updated = analyzeLiteral(current, literal);
+    const fieldsAfter = fieldTexts(updated.analysis, updated.literal).join("\0");
+    if (fieldsBefore !== fieldsAfter) {
       throw new CandidateFailure(
         "UNSAFE_FSTRING_RESTORE",
-        `field texts changed: before=[${before.split("\0").join(", ")}] after=[${after.split("\0").join(", ")}]`
+        `field texts changed: before=[${fieldsBefore.split("\0").join(", ")}] after=[${fieldsAfter.split("\0").join(", ")}]`
       );
     }
-    const updatedDetection = detectSql(updatedLiteral, updatedAnalysis.sourceMap);
+    const updatedDetection = detectSql(updated.literal, updated.analysis.sourceMap);
     if (!updatedDetection.matched) {
       throw new CandidateFailure("FORMATTER_FAILED", "updated candidate no longer matches --sql");
     }
     const next = formatOnce(
-      updatedAnalysis,
-      updatedLiteral,
+      updated.analysis,
+      updated.literal,
       updatedDetection,
       options,
       nonce,
-      sqlFormatter
+      sqlFormatter,
+      baseIndent
     );
     if (next === current) return current;
     current = next;
@@ -1837,14 +2135,15 @@ function formatCandidate(source, analysis, literal, detection, options, nonce, s
   }
   let first;
   try {
+    const local = analyzeLiteral(expected, literal);
     first = convergeToFixedPoint(
-      source,
-      analysis,
-      literal,
-      detection,
+      local.analysis,
+      local.literal,
+      detectSql(local.literal, local.analysis.sourceMap),
       options,
       nonce,
-      sqlFormatter
+      sqlFormatter,
+      baseIndentOf(analysis, literal)
     );
   } catch (error) {
     if (error instanceof CandidateFailure) {
@@ -1874,6 +2173,8 @@ function formatCandidate(source, analysis, literal, detection, options, nonce, s
 var MAX_DOCUMENT_BYTES = 5 * 1024 * 1024;
 var MAX_CANDIDATE_BYTES = 1024 * 1024;
 var MAX_CANDIDATES = 1e3;
+var ResourceLimitError = class extends Error {
+};
 function discover(analysis) {
   const literals = [
     ...analysis.supported,
@@ -1919,14 +2220,39 @@ function combinedSource(source, edits) {
   }
   return result;
 }
+function literalsOf(analysis) {
+  return [...analysis.supported, ...analysis.unsupported].sort(
+    (left, right) => left.span.start - right.span.start
+  );
+}
+function preservesDocumentShape(source, analysis, edits) {
+  const before = literalsOf(analysis);
+  const after = literalsOf(analyzeDocument(combinedSource(source, edits)));
+  if (before.length !== after.length) return false;
+  const editAt = new Map(edits.map((edit) => [edit.sourceSpan.start, edit]));
+  let delta = 0;
+  return before.every((original, index) => {
+    const updated = after[index];
+    if (updated === void 0 || updated.span.start !== original.span.start + delta) return false;
+    const edit = editAt.get(original.span.start);
+    if (edit !== void 0) {
+      delta += edit.replacementText.length - (original.span.end - original.span.start);
+    }
+    if (updated.span.end !== original.span.end + delta) return false;
+    if (!("contentSpan" in original) || !("contentSpan" in updated)) {
+      return !("contentSpan" in original) && !("contentSpan" in updated);
+    }
+    return updated.prefix === original.prefix && updated.delimiter === original.delimiter && updated.kind === original.kind;
+  });
+}
 function formatDocument(source, options, target, nonce, sqlFormatter, logger) {
   if (Buffer.byteLength(source, "utf8") > MAX_DOCUMENT_BYTES) {
-    throw new PositionMappingError("document exceeds the size limit");
+    throw new ResourceLimitError("document exceeds the size limit");
   }
   const analysis = analyzeDocument(source);
   const units = discover(analysis);
   if (units.length > MAX_CANDIDATES) {
-    throw new PositionMappingError("candidate count exceeds the limit");
+    throw new ResourceLimitError("candidate count exceeds the limit");
   }
   const selected = selectUnits(units, target, analysis.sourceMap);
   const edits = [];
@@ -1934,7 +2260,7 @@ function formatDocument(source, options, target, nonce, sqlFormatter, logger) {
   let changed = 0;
   let unchanged = 0;
   for (const unit of selected) {
-    if (!("contentSpan" in unit.literal)) {
+    if (analysis.unterminated || !("contentSpan" in unit.literal)) {
       skipReasons.push("UNSUPPORTED_LITERAL");
       continue;
     }
@@ -1961,8 +2287,17 @@ function formatDocument(source, options, target, nonce, sqlFormatter, logger) {
       unchanged++;
     }
   }
-  const combined = combinedSource(source, edits);
-  analyzeDocument(combined);
+  if (!preservesDocumentShape(source, analysis, edits)) {
+    const kept = edits.filter((edit) => preservesDocumentShape(source, analysis, [edit]));
+    const accepted = preservesDocumentShape(source, analysis, kept) ? kept : [];
+    for (const edit of edits) {
+      if (accepted.includes(edit)) continue;
+      logger?.("candidate skipped (FORMATTER_FAILED): the document no longer reads back the same");
+      skipReasons.push("FORMATTER_FAILED");
+      changed--;
+    }
+    edits.splice(0, edits.length, ...accepted);
+  }
   const summary = {
     discovered: units.length,
     selected: selected.length,
@@ -1984,12 +2319,6 @@ function formatDocument(source, options, target, nonce, sqlFormatter, logger) {
   };
 }
 
-// src/replacement.ts
-function collapseReplacement(literalText2, replacement) {
-  if (literalText2.includes("\n")) return replacement;
-  return replacement.replace(/\s*\n\s*/g, " ").trim();
-}
-
 // node_modules/sql-formatter/dist/esm/allDialects.js
 var allDialects_exports = {};
 __export(allDialects_exports, {
@@ -22606,7 +22935,7 @@ function formatProtectedSql(sql2, options) {
   return format(sql2, {
     language: options.dialect,
     tabWidth: options.indentWidth,
-    keywordCase: options.keywordCase === "preserve" ? "preserve" : options.keywordCase,
+    keywordCase: options.keywordCase,
     linesBetweenQueries: 1,
     expressionWidth: options.wrapAfter,
     denseOperators: !options.useSpaceAroundOperators
@@ -22623,10 +22952,12 @@ var USAGE = `Usage: inline-sql-toolkit [options] [files...]
       --indent-width <1-8>         (default: 2)
       --wrap-after <20-500>        (default: 88)
       --no-space-around-operators  keep dense operators (default: spaced)
-      --no-ordinals                do not replace GROUP BY / ORDER BY ordinals
+      --ordinals                   replace GROUP BY / ORDER BY ordinals (default: off)
+      --no-ordinals                do not replace ordinals (overrides the config file)
       --comma-position <pos>       after | before (default: after)
       --keep-functions-inline      keep SUM(...) / COUNT(CASE ...) on one line
   -c, --config <file>              config JSON (default: nearest .inline-sql.json)
+  -q, --quiet                      do not report skipped candidates on stderr
   -h, --help / --version`;
 function printError(message) {
   process.stderr.write(`${message}
@@ -22684,12 +23015,12 @@ function buildRawOptions(fileOptions, cliValues) {
     indentWidth: cliValues["indent-width"] !== void 0 ? Number(cliValues["indent-width"]) : fileOptions.indentWidth,
     wrapAfter: cliValues["wrap-after"] !== void 0 ? Number(cliValues["wrap-after"]) : fileOptions.wrapAfter,
     useSpaceAroundOperators: cliValues["no-space-around-operators"] ? false : fileOptions.useSpaceAroundOperators,
-    replaceOrdinals: cliValues["no-ordinals"] ? false : fileOptions.replaceOrdinals,
+    replaceOrdinals: cliValues.ordinals ? true : cliValues["no-ordinals"] ? false : fileOptions.replaceOrdinals,
     commaPosition: cliValues["comma-position"] ?? fileOptions.commaPosition,
     keepFunctionsInline: cliValues["keep-functions-inline"] ? true : fileOptions.keepFunctionsInline
   };
 }
-function formatPythonSource(text, options, logger) {
+function formatPythonSourceWithReport(text, options, logger) {
   const nonce = allocateNonce(text, () => (0, import_node_crypto.randomBytes)(16).toString("hex"));
   const result = formatDocument(
     text,
@@ -22699,14 +23030,17 @@ function formatPythonSource(text, options, logger) {
     (sql2, formatterOptions) => formatProtectedSql(sql2, formatterOptions.options),
     logger
   );
-  const edits = result.edits.map((edit) => ({
-    ...edit,
-    replacementText: collapseReplacement(
-      text.slice(edit.sourceSpan.start, edit.sourceSpan.end),
-      edit.replacementText
-    )
-  }));
-  return combinedSource(text, edits);
+  return { output: combinedSource(text, result.edits), skipReasons: result.skipReasons };
+}
+function formatPythonSource(text, options, logger) {
+  return formatPythonSourceWithReport(text, options, logger).output;
+}
+function skipNote(name, reasons) {
+  const counts = /* @__PURE__ */ new Map();
+  for (const reason of reasons) counts.set(reason, (counts.get(reason) ?? 0) + 1);
+  const detail = [...counts].map(([reason, count]) => `${reason} x${count}`).join(", ");
+  const noun = reasons.length === 1 ? "candidate" : "candidates";
+  return `inline-sql-toolkit: ${name}: skipped ${reasons.length} SQL ${noun} (${detail})`;
 }
 function runCli(argv) {
   let parsed;
@@ -22721,10 +23055,12 @@ function runCli(argv) {
         "indent-width": { type: "string" },
         "wrap-after": { type: "string" },
         "no-space-around-operators": { type: "boolean", default: false },
+        ordinals: { type: "boolean", default: false },
         "no-ordinals": { type: "boolean", default: false },
         "comma-position": { type: "string" },
         "keep-functions-inline": { type: "boolean", default: false },
         config: { type: "string", short: "c" },
+        quiet: { type: "boolean", short: "q", default: false },
         help: { type: "boolean", short: "h", default: false },
         version: { type: "boolean", default: false }
       },
@@ -22749,6 +23085,10 @@ function runCli(argv) {
     printError("inline-sql-toolkit: cannot use --write and --check together");
     return 2;
   }
+  if (values.ordinals && values["no-ordinals"]) {
+    printError("inline-sql-toolkit: cannot use --ordinals and --no-ordinals together");
+    return 2;
+  }
   let fileConfig = {};
   if (values.config) {
     try {
@@ -22789,7 +23129,11 @@ function runCli(argv) {
     }
     let output;
     try {
-      output = formatPythonSource(input, options);
+      const report = formatPythonSourceWithReport(input, options);
+      output = report.output;
+      if (!values.quiet && report.skipReasons.length > 0) {
+        printError(skipNote("<stdin>", report.skipReasons));
+      }
     } catch (err) {
       printError(`inline-sql-toolkit: ${err.message}`);
       return 2;
@@ -22815,7 +23159,11 @@ function runCli(argv) {
     }
     let output;
     try {
-      output = formatPythonSource(input, options);
+      const report = formatPythonSourceWithReport(input, options);
+      output = report.output;
+      if (!values.quiet && report.skipReasons.length > 0) {
+        printError(skipNote(file, report.skipReasons));
+      }
     } catch (err) {
       printError(`inline-sql-toolkit: ${file}: ${err.message}`);
       return 2;
@@ -22861,6 +23209,7 @@ if (typeof require !== "undefined" && typeof module !== "undefined" && require.m
   buildRawOptions,
   findConfigFile,
   formatPythonSource,
+  formatPythonSourceWithReport,
   getPackageVersion,
   loadConfigFile,
   runCli

package/package.json

@@ -11,7 +11,7 @@
   "bugs": {
     "url": "https://github.com/hidenobunagai/inline-sql-toolkit/issues"
   },
-  "version": "0.4.13",
+  "version": "0.5.0",
   "icon": "icon.png",
   "type": "module",
   "bin": {
@@ -89,7 +89,7 @@
         },
         "inlineSql.format.replaceOrdinals": {
           "type": "boolean",
-          "default": true,
+          "default": false,
           "scope": "resource",
           "description": "%configuration.format.replaceOrdinals.description%"
         },
@@ -135,69 +135,6 @@
           "meta.embedded.sql": "sql"
         }
       }
-    ],
-    "semanticTokenTypes": [
-      {
-        "id": "inlineSqlKeyword",
-        "superType": "keyword",
-        "description": "%semanticTokenTypes.inlineSqlKeyword.description%"
-      },
-      {
-        "id": "inlineSqlString",
-        "superType": "string",
-        "description": "%semanticTokenTypes.inlineSqlString.description%"
-      },
-      {
-        "id": "inlineSqlNumber",
-        "superType": "number",
-        "description": "%semanticTokenTypes.inlineSqlNumber.description%"
-      },
-      {
-        "id": "inlineSqlOperator",
-        "superType": "operator",
-        "description": "%semanticTokenTypes.inlineSqlOperator.description%"
-      },
-      {
-        "id": "inlineSqlComment",
-        "superType": "comment",
-        "description": "%semanticTokenTypes.inlineSqlComment.description%"
-      },
-      {
-        "id": "inlineSqlIdentifier",
-        "description": "%semanticTokenTypes.inlineSqlIdentifier.description%"
-      }
-    ],
-    "semanticTokenScopes": [
-      {
-        "scopes": {
-          "inlineSqlKeyword": [
-            "keyword.other.sql",
-            "keyword"
-          ],
-          "inlineSqlString": [
-            "string.quoted.sql",
-            "string.quoted.double.sql",
-            "string.quoted.single.sql",
-            "string"
-          ],
-          "inlineSqlNumber": [
-            "constant.numeric.sql",
-            "constant.numeric"
-          ],
-          "inlineSqlOperator": [
-            "keyword.operator.sql",
-            "keyword.operator"
-          ],
-          "inlineSqlComment": [
-            "comment.line.double-dash.sql",
-            "comment.block.sql",
-            "comment"
-          ],
-          "inlineSqlIdentifier": [
-            "source.sql"
-          ]
-        }
-      }
     ]
   },
   "packageManager": "bun@1.3.8",