sqlglot 30.21.0__tar.gz → 30.22.0__tar.gz

{sqlglot-30.21.0 → sqlglot-30.22.0}/CHANGELOG.md

@@ -1,6 +1,60 @@
 Changelog
 =========
 
+## [v30.21.0] - 2026-09-30
+### :boom: BREAKING CHANGES
+- due to [`2be1e0e`](https://github.com/tobymao/sqlglot/commit/2be1e0e2addc0ef01cfa3fd3cf2ef751cdbf411d) - annotate `bit_xor` for duckdb *(PR [#8452](https://github.com/tobymao/sqlglot/pull/8452) by [@opencode13241-eng](https://github.com/opencode13241-eng))*:
+
+  annotate `bit_xor` for duckdb (#8452)
+
+- due to [`65a0e22`](https://github.com/tobymao/sqlglot/commit/65a0e22447fcca9ea4a89f6167954fbb872c8e6d) - new projection pushdown algorithm *(PR [#8440](https://github.com/tobymao/sqlglot/pull/8440) by [@georgesittas](https://github.com/georgesittas))*:
+
+  new projection pushdown algorithm (#8440)
+
+- due to [`e3f71dc`](https://github.com/tobymao/sqlglot/commit/e3f71dc1a1186ada4b017a5f2498b8c2b0a8a25b) - decode backslash escape sequences in string literals *(PR [#8456](https://github.com/tobymao/sqlglot/pull/8456) by [@georgesittas](https://github.com/georgesittas))*:
+
+  decode backslash escape sequences in string literals (#8456)
+
+- due to [`b4d9e37`](https://github.com/tobymao/sqlglot/commit/b4d9e37d84a61eeb78e8d652f697e2e0d4d57869) - inherit Trino dialect, parser and generator defaults *(PR [#8464](https://github.com/tobymao/sqlglot/pull/8464) by [@georgesittas](https://github.com/georgesittas))*:
+
+  inherit Trino dialect, parser and generator defaults (#8464)
+
+- due to [`d84f7f3`](https://github.com/tobymao/sqlglot/commit/d84f7f3ee1ff50778c93970b086d59692714dcdc) - preserve BD literal semantics in Hive and Spark *(PR [#8455](https://github.com/tobymao/sqlglot/pull/8455) by [@georgesittas](https://github.com/georgesittas))*:
+
+  preserve BD literal semantics in Hive and Spark (#8455)
+
+- due to [`1122046`](https://github.com/tobymao/sqlglot/commit/11220460d0e78a3c83f14b006239f636e4ddde71) - convert ARRAY_INSERT position between 0- and 1-based dialects [CLAUDE] *(PR [#8460](https://github.com/tobymao/sqlglot/pull/8460) by [@anishmehta24](https://github.com/anishmehta24))*:
+
+  convert ARRAY_INSERT position between 0- and 1-based dialects [CLAUDE] (#8460)
+
+
+### :sparkles: New Features
+- [`2be1e0e`](https://github.com/tobymao/sqlglot/commit/2be1e0e2addc0ef01cfa3fd3cf2ef751cdbf411d) - **optimizer**: annotate `bit_xor` for duckdb *(PR [#8452](https://github.com/tobymao/sqlglot/pull/8452) by [@opencode13241-eng](https://github.com/opencode13241-eng))*
+- [`7e48d16`](https://github.com/tobymao/sqlglot/commit/7e48d16936216f9f93d7a52baef0e482e3fb14e7) - **optimizer**: add #schema fixture directive *(PR [#8435](https://github.com/tobymao/sqlglot/pull/8435) by [@fivetran-kwoodbeck](https://github.com/fivetran-kwoodbeck))*
+
+### :bug: Bug Fixes
+- [`1d40af1`](https://github.com/tobymao/sqlglot/commit/1d40af146d77dcf890ae18c1a59e649bf0862db3) - **optimizer**: unqualified semi/anti join column bound to table joined *(PR [#8430](https://github.com/tobymao/sqlglot/pull/8430) by [@fivetran-amrutabhimsenayachit](https://github.com/fivetran-amrutabhimsenayachit))*
+- [`d9b2b96`](https://github.com/tobymao/sqlglot/commit/d9b2b9660be8a6b77a0bb0475f6bf0509c6c6aa0) - **parser**: constraint parsing fixup *(PR [#8457](https://github.com/tobymao/sqlglot/pull/8457) by [@georgesittas](https://github.com/georgesittas))*
+- [`e3f71dc`](https://github.com/tobymao/sqlglot/commit/e3f71dc1a1186ada4b017a5f2498b8c2b0a8a25b) - **snowflake**: decode backslash escape sequences in string literals *(PR [#8456](https://github.com/tobymao/sqlglot/pull/8456) by [@georgesittas](https://github.com/georgesittas))*
+  - :arrow_lower_right: *fixes issue [#4344](https://github.com/tobymao/sqlglot/issues/4344) opened by [@dylanscott](https://github.com/dylanscott)*
+  - :arrow_lower_right: *fixes issue [#8441](https://github.com/tobymao/sqlglot/issues/8441) opened by [@yvonne-chou-1](https://github.com/yvonne-chou-1)*
+- [`b4d9e37`](https://github.com/tobymao/sqlglot/commit/b4d9e37d84a61eeb78e8d652f697e2e0d4d57869) - **athena**: inherit Trino dialect, parser and generator defaults *(PR [#8464](https://github.com/tobymao/sqlglot/pull/8464) by [@georgesittas](https://github.com/georgesittas))*
+  - :arrow_lower_right: *fixes issue [#8458](https://github.com/tobymao/sqlglot/issues/8458) opened by [@deepyaman](https://github.com/deepyaman)*
+- [`d84f7f3`](https://github.com/tobymao/sqlglot/commit/d84f7f3ee1ff50778c93970b086d59692714dcdc) - preserve BD literal semantics in Hive and Spark *(PR [#8455](https://github.com/tobymao/sqlglot/pull/8455) by [@georgesittas](https://github.com/georgesittas))*
+- [`1122046`](https://github.com/tobymao/sqlglot/commit/11220460d0e78a3c83f14b006239f636e4ddde71) - **spark**: convert ARRAY_INSERT position between 0- and 1-based dialects [CLAUDE] *(PR [#8460](https://github.com/tobymao/sqlglot/pull/8460) by [@anishmehta24](https://github.com/anishmehta24))*
+- [`18520e2`](https://github.com/tobymao/sqlglot/commit/18520e2b5bfa3f1d7ed1a261d1bd463132054db1) - **clickhouse**: generate varPop for VariancePop [CLAUDE] *(PR [#8469](https://github.com/tobymao/sqlglot/pull/8469) by [@ekut](https://github.com/ekut))*
+- [`2960bdd`](https://github.com/tobymao/sqlglot/commit/2960bdd7470f1c21518710ed468466b9bab4bb7b) - **optimizer**: preserve tautological aggregate filters [CODEX] *(PR [#8466](https://github.com/tobymao/sqlglot/pull/8466) by [@Boulea7](https://github.com/Boulea7))*
+- [`840b482`](https://github.com/tobymao/sqlglot/commit/840b4823e649e380e1dda30cf21907baebb6c58c) - **postgres**: keep WITH ORDINALITY alias columns on LATERAL UNNEST [GROK] *(PR [#8463](https://github.com/tobymao/sqlglot/pull/8463) by [@00200200](https://github.com/00200200))*
+  - :arrow_lower_right: *fixes issue [#8462](https://github.com/tobymao/sqlglot/issues/8462) opened by [@SoTrx](https://github.com/SoTrx)*
+- [`aa70202`](https://github.com/tobymao/sqlglot/commit/aa702020725f4d1591cfb2d308c72937a38e8fd4) - **optimizer**: qualify struct paths with conflicting aliases [CODEX] *(PR [#8467](https://github.com/tobymao/sqlglot/pull/8467) by [@tobymao](https://github.com/tobymao))*
+- [`b54d32f`](https://github.com/tobymao/sqlglot/commit/b54d32f036d4c1158bb889324f4ac6d2846d2701) - **optimizer**: stop struct lookup at nearest alias [CODEX] *(commit by [@tobymao](https://github.com/tobymao))*
+- [`5d16fa8`](https://github.com/tobymao/sqlglot/commit/5d16fa83bfb8f3582eaffa4b99c7d05a061d8a76) - **ci**: publish sqlglot independently of sqlglotc [CODEX] *(commit by [@tobymao](https://github.com/tobymao))*
+
+### :recycle: Refactors
+- [`65a0e22`](https://github.com/tobymao/sqlglot/commit/65a0e22447fcca9ea4a89f6167954fbb872c8e6d) - **optimizer**: new projection pushdown algorithm *(PR [#8440](https://github.com/tobymao/sqlglot/pull/8440) by [@georgesittas](https://github.com/georgesittas))*
+- [`ac940f0`](https://github.com/tobymao/sqlglot/commit/ac940f016a1d398137b4d666db0f191e6c4b6d1f) - **tokenizer**: generalize backslash escape handling, report unterminated escapes *(PR [#8470](https://github.com/tobymao/sqlglot/pull/8470) by [@georgesittas](https://github.com/georgesittas))*
+
+
 ## [v30.20.0] - 2026-09-27
 ### :boom: BREAKING CHANGES
 - due to [`1d0e3f3`](https://github.com/tobymao/sqlglot/commit/1d0e3f34864037c54cf6bdf45af594db62ad7319) - Fix mysql roundtrip for DATE_ADD and DATE_SUB *(PR [#8417](https://github.com/tobymao/sqlglot/pull/8417) by [@fivetran-amrutabhimsenayachit](https://github.com/fivetran-amrutabhimsenayachit))*:
@@ -15990,3 +16044,4 @@ pip install "sqlglot[c]"   # compiled — faster, but no subclassing
 [v30.18.0]: https://github.com/tobymao/sqlglot/compare/v30.17.0...v30.18.0
 [v30.19.0]: https://github.com/tobymao/sqlglot/compare/v30.18.0...v30.19.0
 [v30.20.0]: https://github.com/tobymao/sqlglot/compare/v30.19.0...v30.20.0
+[v30.21.0]: https://github.com/tobymao/sqlglot/compare/v30.20.0...v30.21.0

{sqlglot-30.21.0 → sqlglot-30.22.0}/Makefile

@@ -42,7 +42,7 @@ install-dev:
 		fi; \
 	fi
 
-# sqlglotc requires Python 3.10+ (sqlglot-mypy 1.20+ dropped 3.9). On 3.9
+# sqlglotc requires Python 3.10+ (mypy 2.4+ dropped 3.9). On 3.9
 # we skip the C build; tests fall back to pure-Python sqlglot.
 PY_GE_310 := $(shell python -c "import sys; print(int(sys.version_info >= (3, 10)))")
 

{sqlglot-30.21.0/sqlglot.egg-info → sqlglot-30.22.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sqlglot
-Version: 30.21.0
+Version: 30.22.0
 Summary: An easily customizable SQL parser and transpiler
 Author-email: Toby Mao <toby.mao@gmail.com>
 License-Expression: MIT
@@ -19,7 +19,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Provides-Extra: dev
 Requires-Dist: duckdb>=0.6; extra == "dev"
-Requires-Dist: sqlglot-mypy>=2.3.0.post2; python_version >= "3.10" and extra == "dev"
+Requires-Dist: mypy>=2.4.0; python_version >= "3.10" and extra == "dev"
 Requires-Dist: mypy; python_version < "3.10" and extra == "dev"
 Requires-Dist: setuptools_scm; extra == "dev"
 Requires-Dist: pandas; extra == "dev"
@@ -34,10 +34,10 @@ Requires-Dist: types-pytz; extra == "dev"
 Requires-Dist: typing_extensions; extra == "dev"
 Requires-Dist: pyperf; extra == "dev"
 Provides-Extra: c
-Requires-Dist: sqlglotc==30.21.0; python_version >= "3.10" and extra == "c"
+Requires-Dist: sqlglotc==30.22.0; python_version >= "3.10" and extra == "c"
 Provides-Extra: rs
 Requires-Dist: sqlglotrs==0.13.0; extra == "rs"
-Requires-Dist: sqlglotc==30.21.0; python_version >= "3.10" and extra == "rs"
+Requires-Dist: sqlglotc==30.22.0; python_version >= "3.10" and extra == "rs"
 Dynamic: license-file
 Dynamic: provides-extra
 
@@ -87,11 +87,13 @@ From PyPI:
 # Pure python version
 pip3 install sqlglot
 
-# C extensions compiled with mypyc
+# C extensions compiled with mypyc (Python 3.10+)
 # prebuilt wheel if available for your platform, otherwise builds from source
 pip3 install "sqlglot[c]"
 ```
 
+On Python 3.9, `sqlglot[c]` installs the pure Python version.
+
 Or with a local checkout:
 
 ```
@@ -581,12 +583,12 @@ make docs-serve
 ## Run Tests and Lint
 
 ```
-make style   # Only linter checks
+make style   # Format, lint, and type-check
 make unit    # Only unit tests (pure Python)
 make test    # Unit and integration tests (pure Python)
 make unitc   # Only unit tests (mypyc compiled)
 make testc   # Unit and integration tests (mypyc compiled)
-make check   # Full test suite & linter checks
+make check   # Full test suite, formatting, linting, and type checks
 make clean   # Remove compiled C artifacts (.so files, build dirs)
 ```
 
@@ -616,18 +618,28 @@ sqlglot, sqltree, sqlparse, and sqlfluff are python based whereas sqloxide and p
 |      many_numbers | 0.103898 (1.00) | 0.024483 (0.24) | 0.120119 (1.16) |              N/A |               N/A | 0.031667 (0.30) | 0.026880 (0.26) |
 
 ```
-make bench            # Run parsing benchmark
+make bench            # Run parsing, transpilation, and optimization benchmarks
+make bench-parse      # Run parsing benchmark
+make bench-transpile  # Run transpilation benchmark
 make bench-optimize   # Run optimization benchmark
 ```
 
 ## Optional Dependencies
 
-SQLGlot uses [dateutil](https://github.com/dateutil/dateutil) to simplify literal timedelta expressions. The optimizer will not simplify expressions like the following if the module cannot be found:
+SQLGlot uses [dateutil](https://github.com/dateutil/dateutil) to simplify constant date and interval arithmetic. With dateutil installed, the optimizer simplifies:
+
+```sql
+DATE '2021-01-01' + INTERVAL '1' MONTH
+```
+
+to:
 
 ```sql
-x + interval '1' month
+CAST('2021-02-01' AS DATE)
 ```
 
+Without dateutil, the optimizer leaves this arithmetic unevaluated.
+
 ## Supported Dialects
 
 | Dialect | Support Level |

{sqlglot-30.21.0 → sqlglot-30.22.0}/README.md

@@ -44,11 +44,13 @@ From PyPI:
 # Pure python version
 pip3 install sqlglot
 
-# C extensions compiled with mypyc
+# C extensions compiled with mypyc (Python 3.10+)
 # prebuilt wheel if available for your platform, otherwise builds from source
 pip3 install "sqlglot[c]"
 ```
 
+On Python 3.9, `sqlglot[c]` installs the pure Python version.
+
 Or with a local checkout:
 
 ```
@@ -538,12 +540,12 @@ make docs-serve
 ## Run Tests and Lint
 
 ```
-make style   # Only linter checks
+make style   # Format, lint, and type-check
 make unit    # Only unit tests (pure Python)
 make test    # Unit and integration tests (pure Python)
 make unitc   # Only unit tests (mypyc compiled)
 make testc   # Unit and integration tests (mypyc compiled)
-make check   # Full test suite & linter checks
+make check   # Full test suite, formatting, linting, and type checks
 make clean   # Remove compiled C artifacts (.so files, build dirs)
 ```
 
@@ -573,18 +575,28 @@ sqlglot, sqltree, sqlparse, and sqlfluff are python based whereas sqloxide and p
 |      many_numbers | 0.103898 (1.00) | 0.024483 (0.24) | 0.120119 (1.16) |              N/A |               N/A | 0.031667 (0.30) | 0.026880 (0.26) |
 
 ```
-make bench            # Run parsing benchmark
+make bench            # Run parsing, transpilation, and optimization benchmarks
+make bench-parse      # Run parsing benchmark
+make bench-transpile  # Run transpilation benchmark
 make bench-optimize   # Run optimization benchmark
 ```
 
 ## Optional Dependencies
 
-SQLGlot uses [dateutil](https://github.com/dateutil/dateutil) to simplify literal timedelta expressions. The optimizer will not simplify expressions like the following if the module cannot be found:
+SQLGlot uses [dateutil](https://github.com/dateutil/dateutil) to simplify constant date and interval arithmetic. With dateutil installed, the optimizer simplifies:
+
+```sql
+DATE '2021-01-01' + INTERVAL '1' MONTH
+```
+
+to:
 
 ```sql
-x + interval '1' month
+CAST('2021-02-01' AS DATE)
 ```
 
+Without dateutil, the optimizer leaves this arithmetic unevaluated.
+
 ## Supported Dialects
 
 | Dialect | Support Level |

{sqlglot-30.21.0 → sqlglot-30.22.0}/setup.py

@@ -7,9 +7,9 @@ setup(
     extras_require={
         "dev": [
             "duckdb>=0.6",
-            # sqlglot-mypy 1.20+ is the build dep for sqlglotc and only ships
+            # mypy 2.4+ is the build dep for sqlglotc and only ships
             # for py3.10+; on py3.9 just use upstream mypy for type checking.
-            "sqlglot-mypy >= 2.3.0.post2; python_version >= '3.10'",
+            "mypy >= 2.4.0; python_version >= '3.10'",
             "mypy; python_version < '3.10'",
             "setuptools_scm",
             "pandas",
@@ -25,7 +25,7 @@ setup(
             "pyperf",
         ],
         # Compiles from source on the user's machine. Requires Python 3.10+
-        # because the build dep (sqlglot-mypy 1.20+) dropped 3.9; on 3.9
+        # because the build dep (mypy 2.4+) dropped 3.9; on 3.9
         # `pip install sqlglot[c]` is a no-op and you just get pure-Python sqlglot.
         "c": [f"sqlglotc=={version}; python_version >= '3.10'"],
         # Deprecated: the Rust tokenizer has been replaced by sqlglotc.

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/__init__.py

@@ -81,6 +81,7 @@ def tokenize(sql: str, read: DialectType = None, dialect: DialectType = None) ->
     Args:
         sql: the SQL code string to tokenize.
         read: the SQL dialect to apply during tokenizing (eg. "spark", "hive", "presto", "mysql").
+            See `sqlglot.dialects.Dialects` for options.
         dialect: the SQL dialect (alias for read).
 
     Returns:
@@ -101,6 +102,7 @@ def parse(
     Args:
         sql: the SQL code string to parse.
         read: the SQL dialect to apply during parsing (eg. "spark", "hive", "presto", "mysql").
+            See `sqlglot.dialects.Dialects` for options.
         dialect: the SQL dialect (alias for read).
         **opts: other `sqlglot.parser.Parser` options.
 
@@ -144,6 +146,7 @@ def parse_one(
     Args:
         sql: the SQL code string to parse.
         read: the SQL dialect to apply during parsing (eg. "spark", "hive", "presto", "mysql").
+            See `sqlglot.dialects.Dialects` for options.
         dialect: the SQL dialect (alias for read)
         into: the SQLGlot Expr to parse into.
         **opts: other `sqlglot.parser.Parser` options.
@@ -180,7 +183,9 @@ def transpile(
     Args:
         sql: the SQL code string to transpile.
         read: the source dialect used to parse the input string (eg. "spark", "hive", "presto", "mysql").
+            See `sqlglot.dialects.Dialects` for options.
         write: the target dialect into which the input should be transformed (eg. "spark", "hive", "presto", "mysql").
+            See `sqlglot.dialects.Dialects` for options.
         identity: if set to `True` and if the target dialect is not specified the source dialect will be used as both:
             the source and the target dialect.
         error_level: the desired error level of the parser.

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '30.21.0'
-__version_tuple__ = version_tuple = (30, 21, 0)
+__version__ = version = '30.22.0'
+__version_tuple__ = version_tuple = (30, 22, 0)
 
-__commit_id__ = commit_id = 'g5d16fa83b'
+__commit_id__ = commit_id = 'gb8333a81f'

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/bigquery.py

@@ -29,7 +29,7 @@ class BigQuery(Dialect):
     HEX_LOWERCASE = True
     FORCE_EARLY_ALIAS_REF_EXPANSION = True
     EXPAND_ONLY_GROUP_ALIAS_REF = True
-    PRESERVE_ORIGINAL_NAMES = True
+    ORIGINAL_NAME_META_KEY = "bigquery_name"
     HEX_STRING_IS_INTEGER_TYPE = True
     BYTE_STRING_IS_BYTES_TYPE = True
     UUID_IS_STRING_TYPE = True
@@ -168,6 +168,14 @@ class BigQuery(Dialect):
         }
 
     class Tokenizer(tokens.Tokenizer):
+        NUMERIC_ESCAPES = {
+            "x": (16, 2, 2, 0xFF),
+            "X": (16, 2, 2, 0xFF),
+            "u": (16, 4, 4, 0xFFFF),
+            "U": (16, 8, 8, 0x10FFFF),
+            "0": (8, 3, 3, 0xFF),
+        }
+        DROP_UNKNOWN_ESCAPES = True
         QUOTES = ["'", '"', '"""', "'''"]
         COMMENTS = ["--", "#", ("/*", "*/")]
         IDENTIFIERS = ["`"]

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/clickhouse.py

@@ -19,7 +19,7 @@ class ClickHouse(Dialect):
     SUPPORTS_USER_DEFINED_TYPES = False
     LOG_BASE_FIRST: bool | None = None
     FORCE_EARLY_ALIAS_REF_EXPANSION = True
-    PRESERVE_ORIGINAL_NAMES = True
+    ORIGINAL_NAME_META_KEY = "clickhouse_name"
     NUMBERS_CAN_BE_UNDERSCORE_SEPARATED = True
     IDENTIFIERS_CAN_START_WITH_DIGIT = True
     HEX_STRING_IS_INTEGER_TYPE = True
@@ -31,6 +31,13 @@ class ClickHouse(Dialect):
 
     UNESCAPED_SEQUENCES = {
         "\\0": "\0",
+        "\\e": "\x1b",
+        "\\N": "",
+        "\\/": "/",
+        '\\"': '"',
+        "\\=": "=",
+        "\\`": "`",
+        **{"\\" + chr(i): chr(i) for i in range(1, 32)},
     }
 
     CREATABLE_KIND_MAPPING = {"DATABASE": "SCHEMA"}
@@ -68,7 +75,10 @@ class ClickHouse(Dialect):
         return column_aliases
 
     class Tokenizer(tokens.Tokenizer):
+        NUMERIC_ESCAPES = {"x": (16, 2, 2, 0xFF)}
+        NUMERIC_ESCAPES_ARE_BYTES = True
         COMMENTS = ["--", "#", "#!", ("/*", "*/")]
+        COMMENTS_TERMINATE_AT_NEWLINE_ONLY = True
         IDENTIFIERS = ['"', "`"]
         IDENTIFIER_ESCAPES = ["\\"]
         STRING_ESCAPES = ["'", "\\"]

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/dialect.py

@@ -140,6 +140,19 @@ class NormalizationStrategy(str, AutoName):
     """Always case-insensitive (uppercase), regardless of quotes."""
 
 
+class UsingColumnOrder(str, AutoName):
+    """Where star expansion places the columns of a `USING` or `NATURAL` join."""
+
+    USING_LIST = auto()
+    """First, in the order they're listed in USING (SQL standard, e.g. Postgres)."""
+
+    LEFT_TABLE = auto()
+    """First, in the order they appear in the left table (e.g. MySQL)."""
+
+    IN_PLACE = auto()
+    """Where the left table has them (e.g. DuckDB)."""
+
+
 # "Strict" dialects (e.g. modern Hive, Spark 3+) map their zero-padded MM/dd/HH/hh/mm/ss to these in
 # TIME_MAPPING so they roundtrip, since a lax %m/%d renders non-padded there for parse expressions
 # (see HiveGenerator.format_time).
@@ -343,7 +356,15 @@ class _Dialect(type):
             # The filter is necessary because of `\\a -> a` in Snowflake; we can't replace `a` with `\a`.
             v: k
             for k, v in klass.UNESCAPED_SEQUENCES.items()
-            if not v.isprintable() or v == "\\"
+            # Sequences that only drop the backslash (e.g. "\\\n" -> "\n") aren't needed for generation
+            if v == "\\" or (not v.isprintable() and k[1:] != v)
+        }
+
+        identifier_escapes = klass.tokenizer_class.IDENTIFIER_ESCAPES
+        klass.IDENTIFIER_ESCAPED_SEQUENCES = {
+            v: k
+            for k, v in klass.UNESCAPED_SEQUENCES.items()
+            if k[0] in identifier_escapes and v in identifier_escapes
         }
 
         # Prefer hex, then octal, then Unicode for numeric control-character escapes.
@@ -428,11 +449,11 @@ class Dialect(metaclass=_Dialect):
         False: Disables function name normalization.
     """
 
-    PRESERVE_ORIGINAL_NAMES: bool = False
+    ORIGINAL_NAME_META_KEY: str | None = None
     """
-    Whether the name of the function should be preserved inside the node's metadata,
-    can be useful for roundtripping deprecated vs new functions that share an AST node
-    e.g JSON_VALUE vs JSON_EXTRACT_SCALAR in BigQuery
+    Dialect-specific metadata key for preserving original function names, or None to disable.
+    Only generators using the same key reuse these names, allowing round trips of aliases
+    that share an AST node, e.g. JSON_VALUE vs JSON_EXTRACT_SCALAR in BigQuery.
     """
 
     LOG_BASE_FIRST: bool | None = True
@@ -490,6 +511,9 @@ class Dialect(metaclass=_Dialect):
     BYTE_STRINGS_SUPPORT_ESCAPED_SEQUENCES: bool = False
     """Whether byte string literals support escape sequences. Set by the metaclass based on the tokenizer's BYTE_STRING_ESCAPES."""
 
+    IDENTIFIER_ESCAPED_SEQUENCES: dict[str, str] = {}
+    """Mapping of an identifier escape char (`\\`) to its escaped version (`\\\\`). Set by the metaclass based on the tokenizer's IDENTIFIER_ESCAPES."""
+
     INVERSE_VECTOR_TYPE_ALIASES: dict[str, str] = {}
     """Mapping of vector type aliases back to their canonical names. Overridden by dialects like SingleStore."""
 
@@ -651,6 +675,19 @@ class Dialect(metaclass=_Dialect):
     they must be explicitly selected.
     """
 
+    USING_COLUMN_ORDER = UsingColumnOrder.USING_LIST
+    """
+    Where star expansion places the columns of a `USING` or `NATURAL` join.
+
+    Given `a(a_id, k1, k2)` and `b(k2, b_id, k1)`, `SELECT * FROM a JOIN b USING (k2, k1)` returns:
+    - `USING_LIST`: `k2, k1, a_id, b_id`
+    - `LEFT_TABLE`: `k1, k2, a_id, b_id`
+    - `IN_PLACE`: `a_id, k1, k2, b_id`
+
+    When join columns come first, this applies at every join: each USING join moves its columns
+    ahead of all the columns to its left.
+    """
+
     QUERY_RESULTS_ARE_STRUCTS = False
     """
     Whether query results are typed as structs in metadata for type inference.
@@ -774,6 +811,9 @@ class Dialect(metaclass=_Dialect):
     # Whether ADD is present for each column added by ALTER TABLE
     ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN = True
 
+    # Whether DROP is present for each column dropped by ALTER TABLE
+    ALTER_TABLE_DROP_REQUIRED_FOR_EACH_COLUMN = True
+
     # Whether the value/LHS of the TRY_CAST(<value> AS <type>) should strictly be a
     # STRING type (Snowflake's case) or can be of any type
     TRY_CAST_REQUIRES_STRING: bool | None = None
@@ -1305,7 +1345,13 @@ def inline_array_sql(self: Generator, expression: exp.Expr) -> str:
 
 def inline_array_unless_query(self: Generator, expression: exp.Expr) -> str:
     elem = seq_get(expression.expressions, 0)
-    if isinstance(elem, exp.Expr) and elem.find(exp.Query):
+    if (
+        len(expression.expressions) == 1
+        and isinstance(elem, exp.Query)
+        # value_constructor=True keeps scalar subqueries as array elements. In [(SELECT …)], the
+        # subquery supplies one scalar value; in ARRAY(SELECT …) (BigQuery), the query supplies rows.
+        and not expression.args.get("value_constructor")
+    ):
         return self.func("ARRAY", elem)
     return inline_array_sql(self, expression)
 
@@ -1358,6 +1404,71 @@ def no_map_from_entries_sql(self: Generator, expression: exp.MapFromEntries) ->
     return ""
 
 
+def no_filter_sql(self: Generator, expression: exp.Filter) -> str:
+    # Rewrite `AGG(x) FILTER (WHERE cond)` into an equivalent conditional aggregation, i.e. wrap
+    # the input values in an IF, which works because aggregate functions skip NULLs
+    agg = expression.this
+    # IGNORE NULLS can wrap the aggregate, but the condition still belongs to its arguments
+    if isinstance(agg, exp.IgnoreNulls):
+        agg = agg.this
+
+    agg_arg = seq_get(agg.expressions, 0) if isinstance(agg, exp.Anonymous) else agg.this
+    cond = expression.expression.this
+
+    if isinstance(agg, exp.WithinGroup):
+        # Ordered-set aggregates take their input from the ORDER BY key, so the
+        # condition has to wrap that instead of the aggregate's own argument
+        if isinstance(agg_arg, (exp.Mode, *exp.PERCENTILES)):
+            for ordered in agg.expression.expressions:
+                key = ordered.this
+                key.replace(exp.If(this=cond.copy(), true=key.copy(), false=exp.null()))
+
+            return self.sql(expression.this)
+
+        # Besides the ordered-set aggregates, these are the only functions that take WITHIN GROUP
+        # and the input values as arguments, so anything else can't be rewritten correctly
+        if isinstance(agg_arg, (exp.ArrayAgg, exp.GroupConcat)):
+            agg_arg = agg_arg.this
+        else:
+            self.unsupported("Unable to rewrite FILTER into the aggregate's arguments")
+            return self.sql(expression.this)
+
+    # Folding the condition into COUNT_IF's predicate avoids an IF that returns a predicate, which
+    # dialects without a boolean type (e.g. T-SQL) reject
+    if isinstance(agg, exp.CountIf) and not isinstance(agg_arg, exp.Distinct):
+        return self.sql(exp.CountIf(this=exp.and_(cond, agg_arg)))
+
+    # `COUNT(*/t.*) FILTER (WHERE cond)` counts qualifying rows, but a star can't be an IF
+    # argument: `IF(cond, *, NULL)` expands to multiple columns once the table has 2+ of
+    # them. Each dialect generates CountIf in its own way instead, e.g. COUNT_IF or COUNTIF.
+    if isinstance(agg, exp.Count) and (agg_arg is None or agg_arg.is_star):
+        return self.sql(exp.CountIf(this=cond))
+
+    # `DISTINCT` and `ORDER BY` are part of the aggregate's own argument list, so the
+    # condition has to wrap the values underneath them rather than the whole clause --
+    # `IF(cond, DISTINCT x, NULL)` is not a call any dialect accepts.
+    if isinstance(agg_arg, exp.Order):
+        agg_arg = agg_arg.this
+
+    if isinstance(agg_arg, exp.Distinct):
+        targets = agg_arg.expressions
+    else:
+        targets = [agg_arg]
+
+    for target in targets:
+        target.replace(exp.If(this=cond.copy(), true=target.copy(), false=exp.null()))
+
+    # In dialects where ARRAY_AGG keeps NULLs (e.g. BigQuery, which even fails on them), the
+    # rows that don't satisfy the condition have to be dropped explicitly
+    if isinstance(agg, exp.ArrayAgg) and self.dialect.ARRAY_AGG_INCLUDES_NULLS:
+        # IGNORE NULLS already drops them, so no extra FILTER (WHERE x IS NOT NULL) is needed
+        agg.set("nulls_excluded", None)
+        if agg is expression.this:
+            return self.sql(exp.IgnoreNulls(this=agg))
+
+    return self.sql(expression.this)
+
+
 def property_sql(self: Generator, expression: exp.Property) -> str:
     return f"{self.property_name(expression, string_key=True)}={self.sql(expression, 'value')}"
 
@@ -1648,11 +1759,7 @@ def build_formatted_time(
     """
 
     def _builder(args: BuilderArgs, dialect: Dialect) -> E:
-        target_dialect = (
-            t.cast(Dialect, Dialect[dialect_override])
-            if isinstance(dialect_override, str)
-            else dialect
-        )
+        target_dialect = Dialect[dialect_override] if isinstance(dialect_override, str) else dialect
 
         fmt = seq_get(args, 1)
         if not fmt:

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/doris.py

@@ -10,6 +10,10 @@ class Doris(MySQL):
     DATEINT_FORMAT = "'yyyyMMdd'"
     TIME_FORMAT = "'yyyy-MM-dd HH:mm:ss'"
 
+    class Tokenizer(MySQL.Tokenizer):
+        DASH_COMMENT_REQUIRES_BOUNDARY = False
+        COMMENTS_TERMINATE_AT_NEWLINE_ONLY = False
+
     Parser = DorisParser
 
     Generator = DorisGenerator

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/drill.py

@@ -9,7 +9,7 @@ from sqlglot.parsers.drill import DrillParser
 
 class Drill(Dialect):
     NORMALIZE_FUNCTIONS: bool | str = False
-    PRESERVE_ORIGINAL_NAMES = True
+    ORIGINAL_NAME_META_KEY = "drill_name"
     NULL_ORDERING = "nulls_are_last"
     DATE_FORMAT = "'yyyy-MM-dd'"
     DATEINT_FORMAT = "'yyyyMMdd'"
@@ -53,7 +53,7 @@ class Drill(Dialect):
 
     class Tokenizer(tokens.Tokenizer):
         IDENTIFIERS = ["`"]
-        STRING_ESCAPES = ["\\"]
+        STRING_ESCAPES = ["'"]
 
         KEYWORDS = tokens.Tokenizer.KEYWORDS.copy()
         KEYWORDS.pop("/*+")

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/duckdb.py

@@ -6,6 +6,7 @@ from sqlglot import exp, tokens
 from sqlglot.dialects.dialect import (
     Dialect,
     NormalizationStrategy,
+    UsingColumnOrder,
 )
 from sqlglot.generators.duckdb import DuckDBGenerator, WS_CONTROL_CHARS_TO_DUCK  # noqa: F401
 from sqlglot.parsers.duckdb import DuckDBParser
@@ -22,9 +23,11 @@ class DuckDB(Dialect):
     SUPPORTS_ORDER_BY_ALL = True
     SUPPORTS_LIMIT_ALL = True
     SUPPORTS_FIXED_SIZE_ARRAYS = True
+    TABLES_REFERENCEABLE_AS_COLUMNS = True
     STRICT_JSON_PATH_SYNTAX = False
     NUMBERS_CAN_BE_UNDERSCORE_SEPARATED = True
     UUID_IS_STRING_TYPE = False
+    USING_COLUMN_ORDER = UsingColumnOrder.IN_PLACE
 
     # https://duckdb.org/docs/sql/introduction.html#creating-a-new-table
     NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
@@ -66,7 +69,12 @@ class DuckDB(Dialect):
 
         return super().to_json_path(path)
 
+    UNESCAPED_SEQUENCES = {"\\a": "a", "\\v": "v"}
+
     class Tokenizer(tokens.Tokenizer):
+        NUMERIC_ESCAPES = {"x": (16, 1, 2, 0xFF), "0": (8, 1, 3, 0o777)}
+        NUMERIC_ESCAPES_ARE_BYTES = True
+        DROP_UNKNOWN_ESCAPES = True
         BYTE_STRINGS = [("e'", "'"), ("E'", "'")]
         BYTE_STRING_ESCAPES = ["'", "\\"]
         HEREDOC_STRINGS = ["$"]

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/hive.py

@@ -88,7 +88,20 @@ class Hive(Dialect):
             TokenType.DASH,
         }
 
+    UNESCAPED_SEQUENCES = {
+        "\\0": "\0",
+        "\\Z": "\x1a",
+        "\\%": "\\%",
+        "\\_": "\\_",
+        "\\a": "a",
+        "\\f": "f",
+        "\\v": "v",
+    }
+
     class Tokenizer(tokens.Tokenizer):
+        NUMERIC_ESCAPES = {"u": (16, 4, 4, 0xFFFF), "0": (8, 3, 3, 0x7F)}
+        DROP_UNKNOWN_ESCAPES = True
+        LONE_SURROGATE_REPLACEMENT = "?"
         QUOTES = ["'", '"']
         IDENTIFIERS = ["`"]
         STRING_ESCAPES = ["\\"]

{sqlglot-30.21.0 → sqlglot-30.22.0}/sqlglot/dialects/materialize.py

@@ -8,6 +8,11 @@ from sqlglot.parsers.materialize import MaterializeParser
 class Materialize(Postgres):
     NORMALIZE_NOT_NULL = True
 
+    UNESCAPED_SEQUENCES = {"\\a": "a", "\\v": "v"}
+
+    class Tokenizer(Postgres.Tokenizer):
+        NUMERIC_ESCAPES = {"u": (16, 4, 4, 0xFFFF), "U": (16, 8, 8, 0x10FFFF)}
+
     Parser = MaterializeParser
 
     Generator = MaterializeGenerator