sqlglot 30.0.1__tar.gz → 30.0.2__tar.gz

{sqlglot-30.0.1 → sqlglot-30.0.2}/CHANGELOG.md

@@ -1,7 +1,144 @@
 Changelog
 =========
 
+## [v30.0.1] - 2026-03-16
+### :bug: Bug Fixes
+- [`7bcad5c`](https://github.com/tobymao/sqlglot/commit/7bcad5c61673a8567e9c22d059ed4326e6698457) - auto pin sqlglotc version to sqlglot so there's no version mismatch closes [#7304](https://github.com/tobymao/sqlglot/pull/7304) *(commit by [@tobymao](https://github.com/tobymao))*
+
+
 ## [v30.0.0] - 2026-03-16
+
+SQLGlot v30 is a major release focused on performance and compilation. Some of the core components of the library are now fully compilable by [mypyc](https://mypyc.readthedocs.io/), delivering significant speedups when installed with the `[c]` extra. This required restructuring several internal modules, which introduces breaking changes for users who depend on internal APIs, subclass parsers, or import from internal paths.
+
+> **If you only use the public API** (`sqlglot.parse`, `sqlglot.parse_one`, `sqlglot.transpile`, `sqlglot.exp.*`, `sqlglot.optimizer.*`), most code will work without changes.
+---
+
+## Migration Guide
+
+### 1. Rust tokenizer removed — use `[c]` instead of `[rs]`
+
+The Rust-based tokenizer (`sqlglotrs`) has been removed (since v29) and replaced with a mypyc-compiled C extension (`sqlglotc`).
+
+```bash
+# Before
+pip install "sqlglot[rs]"
+
+# After
+pip install "sqlglot[c]"
+```
+
+The `[rs]` extra still installs but is now a deprecated no-op stub. The following APIs are removed:
+- `use_rs_tokenizer` parameter and attribute on `Tokenizer`
+- `RsTokenizer`, `RsTokenizerSettings`, `RsTokenTypeSettings` imports
+- `USE_RS_TOKENIZER` constant from `tokens.py`
+
+### 2. `expressions.py` split into a package
+
+The monolithic `sqlglot/expressions.py` has been split into `sqlglot/expressions/` with submodules:
+
+| Module | Contents |
+|---|---|
+| `core.py` | `Expr`, `Expression`, `Condition`, `Func`, `AggFunc`, `Column`, `Literal`, etc. |
+| `datatypes.py` | `DataType`, `DType`, `DataTypeParam`, `Interval` |
+| `query.py` | `Select`, `Query`, `SetOperation`, `UDTF`, `Subquery` |
+| `ddl.py` | `Create`, `Alter`, `Drop`, DDL statements |
+| `dml.py` | `Insert`, `Update`, `Delete`, `Merge` |
+| `properties.py` | All `*Property` classes, `PropertiesLocation` |
+| `constraints.py` | All `*ColumnConstraint` classes |
+| `math.py` | Arithmetic operators (`Add`, `Sub`, `Mul`, `Div`, etc.) |
+| `string.py` | String functions (`Concat`, `Length`, `Upper`, etc.) |
+| `temporal.py` | Date/time functions (`DateAdd`, `DateDiff`, etc.) |
+| `aggregate.py` | Aggregate functions (`Count`, `Sum`, `Avg`, etc.) |
+| `array.py` | Array functions (`ArrayAgg`, `Explode`, etc.) |
+| `json.py` | JSON functions (`JSONExtract`, etc.) |
+| `functions.py` | Other functions (`Coalesce`, `If`, `Case`, `Cast`, etc.) |
+| `builders.py` | Builder helpers (`select()`, `from_()`, `condition()`, etc.) |
+
+**Backwards-compatibility:** `from sqlglot.expressions import *` and `from sqlglot import expressions as exp` still work, because everything is re-exported from `expressions/__init__.py`. However, if you were importing from `sqlglot.expressions` by relying on it being a single file (e.g., inspecting `__file__`), that will break.
+
+### 3. `Parser.expression()` no longer accepts `**kwargs`
+
+This affects anyone subclassing `Parser` or calling `self.expression()` in custom parse methods.
+
+```python
+# Before
+self.expression(exp.Select, distinct=True, expressions=cols)
+
+# After
+self.expression(exp.Select(distinct=True, expressions=cols))
+```
+
+The expression instance is now constructed by the caller and passed directly. This eliminates `**kwargs` dict allocation overhead.
+
+### 4. Scope traversal: `bfs` parameter removed
+
+The `bfs` parameter has been removed from all scope traversal functions. Traversal is now always depth-first (DFS).
+
+```python
+# Before
+scope.walk(bfs=True)
+scope.find(exp.Column, bfs=False)
+walk_in_scope(expr, bfs=True)
+
+# After
+scope.walk()
+scope.find(exp.Column)
+walk_in_scope(expr)
+```
+
+**Behavioral change:** The old default was `bfs=True`. Now traversal is always DFS. Code that depended on BFS ordering from these functions will get results in a different order.
+
+Affected functions: `Scope.walk()`, `Scope.find()`, `Scope.find_all()`, `walk_in_scope()`, `find_in_scope()`, `find_all_in_scope()`.
+
+### 5. Dialect metaclass no longer mutates Parser token sets
+
+Previously, the `_Dialect` metaclass dynamically modified parser token sets (`ID_VAR_TOKENS`, `TABLE_ALIAS_TOKENS`, `NO_PAREN_FUNCTIONS`) at class creation time based on dialect flags like `SUPPORTS_SEMI_ANTI_JOIN`. These mutations are removed — each parser now declares its token sets statically.
+
+- `Dialect.SUPPORTS_SEMI_ANTI_JOIN` has been removed.
+- `SHOW_TRIE` / `SET_TRIE` are no longer auto-computed from `SHOW_PARSERS` / `SET_PARSERS`.
+
+### 6. Use `Expr` instead of `Expression` for generic `isinstance` checks
+
+Base classes like `Func`, `Condition`, `Binary`, and other traits now inherit from `Expr` directly, not from `Expression`. This means `isinstance(node, exp.Expression)` will **not** match these trait classes. If your code uses `isinstance` to check for "any AST node", switch to `exp.Expr`:
+
+```python
+# Before
+isinstance(node, exp.Expression)
+
+# After
+isinstance(node, exp.Expr)
+```
+
+### 7. Compiled classes cannot be subclassed (when using `[c]`)
+
+When `sqlglot[c]` is installed, many core classes are compiled via mypyc. **Compiled classes cannot be subclassed at runtime** — class definition succeeds, but instantiation raises `TypeError: interpreted classes cannot inherit from compiled`.
+
+**Affected classes (compiled):**
+
+| Class | Subclassable? |
+|---|---|
+| All parsers (`BigQueryParser`, `SnowflakeParser`, etc.) | No |
+| `Parser` (base) | No |
+| `Expression`, `Expr`, and all AST nodes (`Select`, `Column`, `Func`, etc.) | No |
+| `MappingSchema`, `AbstractMappingSchema` | No |
+| `Scope` | No |
+| Optimizer rules (`scope.py`, `qualify.py`, `qualify_columns.py`, etc.) | No |
+
+**Not compiled (still subclassable):**
+
+| Class | Subclassable? |
+|---|---|
+| `Generator` and all dialect generators | Yes |
+| `Tokenizer` and all dialect tokenizers | Yes |
+| `Dialect` and all dialect classes | Yes |
+
+If you need to subclass compiled classes (parsers, expressions, schema, etc.), install the pure Python version instead:
+
+```bash
+pip install sqlglot        # pure Python — full subclassing support
+pip install "sqlglot[c]"   # compiled — faster, but no subclassing
+```
+
 ### :boom: BREAKING CHANGES
 - due to [`8ee0646`](https://github.com/tobymao/sqlglot/commit/8ee0646baa6dfae7e96ca86e2c1af5d53fc04290) - Transpile numeric literals with underscores from ClickHouse to other dialects *(PR [#7132](https://github.com/tobymao/sqlglot/pull/7132) by [@fivetran-amrutabhimsenayachit](https://github.com/fivetran-amrutabhimsenayachit))*:
 
@@ -13663,3 +13800,4 @@ Changelog
 [v29.0.0]: https://github.com/tobymao/sqlglot/compare/v28.10.1...v29.0.0
 [v29.0.1]: https://github.com/tobymao/sqlglot/compare/v29.0.0...v29.0.1
 [v30.0.0]: https://github.com/tobymao/sqlglot/compare/v29.0.1...v30.0.0
+[v30.0.1]: https://github.com/tobymao/sqlglot/compare/v30.0.0...v30.0.1

{sqlglot-30.0.1 → sqlglot-30.0.2}/Makefile

@@ -1,4 +1,4 @@
-.PHONY: install install-dev install-devc install-devc-release install-pre-commit bench bench-parse bench-optimize test test-fast unit testc unitc style check docs docs-serve hidec showc clean resolve-integration-conflicts
+.PHONY: install install-dev install-devc install-devc-release install-pre-commit bench bench-parse bench-optimize test test-fast unit testc unitc style check docs docs-serve hidec showc clean resolve-integration-conflicts update-fixtures
 
 ifdef UV
     PIP := uv pip
@@ -7,7 +7,7 @@ else
 endif
 
 SO_BACKUP := /tmp/sqlglot_so_backup
-FIND_SO := find . -name "*.so" -not -path "*/build/*" -not -path "*/site-packages/*"
+FIND_SO := find sqlglot -name "*.so"
 
 hidec:
 	rm -rf $(SO_BACKUP) && $(FIND_SO) | tar cf $(SO_BACKUP) -T - 2>/dev/null && $(FIND_SO) -delete; true
@@ -41,7 +41,7 @@ install-devc:
 	cd sqlglotc && MYPYC_OPT=0 python setup.py build_ext --inplace
 
 install-devc-release: clean
-	cd sqlglotc && $(PIP) install -e .
+	cd sqlglotc && python setup.py build_ext --inplace
 
 install-pre-commit:
 	pre-commit install
@@ -87,3 +87,6 @@ docs-serve:
 
 resolve-integration-conflicts:
 	cd sqlglot-integration-tests && git pull --rebase --autostash
+
+update-fixtures:
+	python sqlglot-integration-tests/scripts/update_dbt_fixtures.py

{sqlglot-30.0.1 → sqlglot-30.0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sqlglot
-Version: 30.0.1
+Version: 30.0.2
 Summary: An easily customizable SQL parser and transpiler
 Author-email: Toby Mao <toby.mao@gmail.com>
 License-Expression: MIT
@@ -33,10 +33,10 @@ Requires-Dist: types-pytz; extra == "dev"
 Requires-Dist: typing_extensions; extra == "dev"
 Requires-Dist: pyperf; extra == "dev"
 Provides-Extra: c
-Requires-Dist: sqlglotc==30.0.1; extra == "c"
+Requires-Dist: sqlglotc==30.0.2; extra == "c"
 Provides-Extra: rs
 Requires-Dist: sqlglotrs==0.13.0; extra == "rs"
-Requires-Dist: sqlglotc==30.0.1; extra == "rs"
+Requires-Dist: sqlglotc==30.0.2; extra == "rs"
 Dynamic: license-file
 Dynamic: provides-extra
 
@@ -590,22 +590,22 @@ sqlglot, sqltree, sqlparse, and sqlfluff are python based whereas sqloxide and p
 
 |             Query |         sqlglot |      sqlglot[c] |         sqltree |         sqlparse |          sqlfluff |        sqloxide |    polyglot-sql |
 | ----------------- | --------------- | --------------- | --------------- | ---------------- | ----------------- | --------------- | --------------- |
-|              tpch | 0.002986 (1.00) | 0.000946 (0.32) | 0.002173 (0.73) |  0.014071 (4.71) |  0.240238 (80.45) | 0.000566 (0.19) | 0.000613 (0.21) |
-|             short | 0.000257 (1.00) | 0.000092 (0.36) | 0.000169 (0.66) |  0.000889 (3.45) | 0.034076 (132.35) | 0.000036 (0.14) | 0.000098 (0.38) |
-|   deep_arithmetic | 0.007875 (1.00) | 0.002343 (0.30) | 0.006065 (0.77) |              N/A | 1.361707 (172.90) | 0.003295 (0.42) | 0.002896 (0.37) |
-|          large_in | 0.441550 (1.00) | 0.126875 (0.29) | 0.482375 (1.09) |              N/A |               N/A | 0.122851 (0.28) | 0.135547 (0.31) |
-|            values | 0.520747 (1.00) | 0.140796 (0.27) | 0.566855 (1.09) |              N/A |               N/A | 0.118219 (0.23) | 0.128448 (0.25) |
-|        many_joins | 0.015098 (1.00) | 0.004023 (0.27) | 0.010876 (0.72) |  0.060144 (3.98) |  1.250214 (82.81) | 0.002953 (0.20) | 0.002926 (0.19) |
-|       many_unions | 0.056118 (1.00) | 0.012016 (0.21) | 0.039004 (0.70) |              N/A |  1.838802 (32.77) | 0.013354 (0.24) |             N/A |
-| nested_subqueries | 0.001643 (1.00) | 0.000285 (0.17) |             N/A |  0.003768 (2.29) |  0.090923 (55.35) | 0.000215 (0.13) | 0.000246 (0.15) |
-|      many_columns | 0.013389 (1.00) | 0.003844 (0.29) | 0.012286 (0.92) | 0.238084 (17.78) |  1.057274 (78.96) | 0.002520 (0.19) | 0.004034 (0.30) |
-|        large_case | 0.037965 (1.00) | 0.010664 (0.28) | 0.035426 (0.93) |              N/A | 4.165526 (109.72) | 0.008970 (0.24) | 0.010843 (0.29) |
-|     complex_where | 0.030109 (1.00) | 0.007886 (0.26) |             N/A |  0.133731 (4.44) |  2.497400 (82.95) | 0.005788 (0.19) | 0.008039 (0.27) |
-|         many_ctes | 0.021528 (1.00) | 0.004891 (0.23) | 0.013173 (0.61) |  0.125813 (5.84) |  0.683930 (31.77) | 0.004004 (0.19) | 0.003397 (0.16) |
-|      many_windows | 0.024819 (1.00) | 0.006944 (0.28) |             N/A |  0.207969 (8.38) |  1.409374 (56.79) | 0.003544 (0.14) | 0.004391 (0.18) |
-|  nested_functions | 0.001483 (1.00) | 0.000327 (0.22) | 0.000665 (0.45) |  0.005086 (3.43) |  0.092523 (62.37) | 0.000170 (0.11) | 0.000215 (0.14) |
-|     large_strings | 0.005686 (1.00) | 0.001612 (0.28) | 0.014111 (2.48) |  0.049465 (8.70) |  0.329009 (57.86) | 0.001463 (0.26) | 0.002308 (0.41) |
-|      many_numbers | 0.107281 (1.00) | 0.030352 (0.28) | 0.120840 (1.13) |              N/A |               N/A | 0.027484 (0.26) | 0.032682 (0.30) |
+|              tpch | 0.002646 (1.00) | 0.000776 (0.29) | 0.002171 (0.82) |  0.014029 (5.30) |  0.247869 (93.68) | 0.000567 (0.21) | 0.000565 (0.21) |
+|             short | 0.000221 (1.00) | 0.000077 (0.35) | 0.000169 (0.76) |  0.000891 (4.03) | 0.032397 (146.59) | 0.000036 (0.16) | 0.000100 (0.45) |
+|   deep_arithmetic | 0.007739 (1.00) | 0.002249 (0.29) | 0.006108 (0.79) |              N/A | 1.404935 (181.54) | 0.002756 (0.36) | 0.002376 (0.31) |
+|          large_in | 0.426694 (1.00) | 0.104540 (0.24) | 0.481306 (1.13) |              N/A |               N/A | 0.118200 (0.28) | 0.109950 (0.26) |
+|            values | 0.470626 (1.00) | 0.114497 (0.24) | 0.547718 (1.16) |              N/A |               N/A | 0.118737 (0.25) | 0.113932 (0.24) |
+|        many_joins | 0.012012 (1.00) | 0.002515 (0.21) | 0.010995 (0.92) |  0.057475 (4.78) | 1.354047 (112.72) | 0.002927 (0.24) | 0.002988 (0.25) |
+|       many_unions | 0.048105 (1.00) | 0.008505 (0.18) | 0.038477 (0.80) |              N/A |  1.894275 (39.38) | 0.012482 (0.26) | 0.014072 (0.29) |
+| nested_subqueries | 0.001377 (1.00) | 0.000238 (0.17) |             N/A |  0.003820 (2.77) |  0.092038 (66.84) | 0.000231 (0.17) | 0.000272 (0.20) |
+|      many_columns | 0.012463 (1.00) | 0.002794 (0.22) | 0.012433 (1.00) | 0.240845 (19.32) |  1.097327 (88.05) | 0.002403 (0.19) | 0.003731 (0.30) |
+|        large_case | 0.043128 (1.00) | 0.008559 (0.20) | 0.034192 (0.79) |              N/A | 4.794309 (111.16) | 0.009326 (0.22) | 0.009723 (0.23) |
+|     complex_where | 0.032286 (1.00) | 0.006645 (0.21) |             N/A |  0.136158 (4.22) | 2.623021 (81.24)  | 0.005642 (0.17) | 0.007409 (0.23) |
+|         many_ctes | 0.018625 (1.00) | 0.003961 (0.21) | 0.012569 (0.67) |  0.125719 (6.75) |  0.680498 (36.54) | 0.004813 (0.26) | 0.003537 (0.19) |
+|      many_windows | 0.021923 (1.00) | 0.005856 (0.27) |             N/A |  0.205469 (9.37) | 1.465070 (66.83)  | 0.003638 (0.17) | 0.004376 (0.20) |
+|  nested_functions | 0.000816 (1.00) | 0.000195 (0.24) | 0.000703 (0.86) |  0.005044 (6.18) | 0.093864 (115.03) | 0.000170 (0.21) | 0.000188 (0.23) |
+|     large_strings | 0.005391 (1.00) | 0.001474 (0.27) | 0.014024 (2.60) |  0.050929 (9.45) |  0.335191 (62.18) | 0.001559 (0.29) | 0.002095 (0.39) |
+|      many_numbers | 0.107804 (1.00) | 0.025544 (0.24) | 0.122202 (1.13) |              N/A |               N/A | 0.029262 (0.27) | 0.026409 (0.24) |
 
 ```
 make bench            # Run parsing benchmark

{sqlglot-30.0.1 → sqlglot-30.0.2}/README.md

@@ -548,22 +548,22 @@ sqlglot, sqltree, sqlparse, and sqlfluff are python based whereas sqloxide and p
 
 |             Query |         sqlglot |      sqlglot[c] |         sqltree |         sqlparse |          sqlfluff |        sqloxide |    polyglot-sql |
 | ----------------- | --------------- | --------------- | --------------- | ---------------- | ----------------- | --------------- | --------------- |
-|              tpch | 0.002986 (1.00) | 0.000946 (0.32) | 0.002173 (0.73) |  0.014071 (4.71) |  0.240238 (80.45) | 0.000566 (0.19) | 0.000613 (0.21) |
-|             short | 0.000257 (1.00) | 0.000092 (0.36) | 0.000169 (0.66) |  0.000889 (3.45) | 0.034076 (132.35) | 0.000036 (0.14) | 0.000098 (0.38) |
-|   deep_arithmetic | 0.007875 (1.00) | 0.002343 (0.30) | 0.006065 (0.77) |              N/A | 1.361707 (172.90) | 0.003295 (0.42) | 0.002896 (0.37) |
-|          large_in | 0.441550 (1.00) | 0.126875 (0.29) | 0.482375 (1.09) |              N/A |               N/A | 0.122851 (0.28) | 0.135547 (0.31) |
-|            values | 0.520747 (1.00) | 0.140796 (0.27) | 0.566855 (1.09) |              N/A |               N/A | 0.118219 (0.23) | 0.128448 (0.25) |
-|        many_joins | 0.015098 (1.00) | 0.004023 (0.27) | 0.010876 (0.72) |  0.060144 (3.98) |  1.250214 (82.81) | 0.002953 (0.20) | 0.002926 (0.19) |
-|       many_unions | 0.056118 (1.00) | 0.012016 (0.21) | 0.039004 (0.70) |              N/A |  1.838802 (32.77) | 0.013354 (0.24) |             N/A |
-| nested_subqueries | 0.001643 (1.00) | 0.000285 (0.17) |             N/A |  0.003768 (2.29) |  0.090923 (55.35) | 0.000215 (0.13) | 0.000246 (0.15) |
-|      many_columns | 0.013389 (1.00) | 0.003844 (0.29) | 0.012286 (0.92) | 0.238084 (17.78) |  1.057274 (78.96) | 0.002520 (0.19) | 0.004034 (0.30) |
-|        large_case | 0.037965 (1.00) | 0.010664 (0.28) | 0.035426 (0.93) |              N/A | 4.165526 (109.72) | 0.008970 (0.24) | 0.010843 (0.29) |
-|     complex_where | 0.030109 (1.00) | 0.007886 (0.26) |             N/A |  0.133731 (4.44) |  2.497400 (82.95) | 0.005788 (0.19) | 0.008039 (0.27) |
-|         many_ctes | 0.021528 (1.00) | 0.004891 (0.23) | 0.013173 (0.61) |  0.125813 (5.84) |  0.683930 (31.77) | 0.004004 (0.19) | 0.003397 (0.16) |
-|      many_windows | 0.024819 (1.00) | 0.006944 (0.28) |             N/A |  0.207969 (8.38) |  1.409374 (56.79) | 0.003544 (0.14) | 0.004391 (0.18) |
-|  nested_functions | 0.001483 (1.00) | 0.000327 (0.22) | 0.000665 (0.45) |  0.005086 (3.43) |  0.092523 (62.37) | 0.000170 (0.11) | 0.000215 (0.14) |
-|     large_strings | 0.005686 (1.00) | 0.001612 (0.28) | 0.014111 (2.48) |  0.049465 (8.70) |  0.329009 (57.86) | 0.001463 (0.26) | 0.002308 (0.41) |
-|      many_numbers | 0.107281 (1.00) | 0.030352 (0.28) | 0.120840 (1.13) |              N/A |               N/A | 0.027484 (0.26) | 0.032682 (0.30) |
+|              tpch | 0.002646 (1.00) | 0.000776 (0.29) | 0.002171 (0.82) |  0.014029 (5.30) |  0.247869 (93.68) | 0.000567 (0.21) | 0.000565 (0.21) |
+|             short | 0.000221 (1.00) | 0.000077 (0.35) | 0.000169 (0.76) |  0.000891 (4.03) | 0.032397 (146.59) | 0.000036 (0.16) | 0.000100 (0.45) |
+|   deep_arithmetic | 0.007739 (1.00) | 0.002249 (0.29) | 0.006108 (0.79) |              N/A | 1.404935 (181.54) | 0.002756 (0.36) | 0.002376 (0.31) |
+|          large_in | 0.426694 (1.00) | 0.104540 (0.24) | 0.481306 (1.13) |              N/A |               N/A | 0.118200 (0.28) | 0.109950 (0.26) |
+|            values | 0.470626 (1.00) | 0.114497 (0.24) | 0.547718 (1.16) |              N/A |               N/A | 0.118737 (0.25) | 0.113932 (0.24) |
+|        many_joins | 0.012012 (1.00) | 0.002515 (0.21) | 0.010995 (0.92) |  0.057475 (4.78) | 1.354047 (112.72) | 0.002927 (0.24) | 0.002988 (0.25) |
+|       many_unions | 0.048105 (1.00) | 0.008505 (0.18) | 0.038477 (0.80) |              N/A |  1.894275 (39.38) | 0.012482 (0.26) | 0.014072 (0.29) |
+| nested_subqueries | 0.001377 (1.00) | 0.000238 (0.17) |             N/A |  0.003820 (2.77) |  0.092038 (66.84) | 0.000231 (0.17) | 0.000272 (0.20) |
+|      many_columns | 0.012463 (1.00) | 0.002794 (0.22) | 0.012433 (1.00) | 0.240845 (19.32) |  1.097327 (88.05) | 0.002403 (0.19) | 0.003731 (0.30) |
+|        large_case | 0.043128 (1.00) | 0.008559 (0.20) | 0.034192 (0.79) |              N/A | 4.794309 (111.16) | 0.009326 (0.22) | 0.009723 (0.23) |
+|     complex_where | 0.032286 (1.00) | 0.006645 (0.21) |             N/A |  0.136158 (4.22) | 2.623021 (81.24)  | 0.005642 (0.17) | 0.007409 (0.23) |
+|         many_ctes | 0.018625 (1.00) | 0.003961 (0.21) | 0.012569 (0.67) |  0.125719 (6.75) |  0.680498 (36.54) | 0.004813 (0.26) | 0.003537 (0.19) |
+|      many_windows | 0.021923 (1.00) | 0.005856 (0.27) |             N/A |  0.205469 (9.37) | 1.465070 (66.83)  | 0.003638 (0.17) | 0.004376 (0.20) |
+|  nested_functions | 0.000816 (1.00) | 0.000195 (0.24) | 0.000703 (0.86) |  0.005044 (6.18) | 0.093864 (115.03) | 0.000170 (0.21) | 0.000188 (0.23) |
+|     large_strings | 0.005391 (1.00) | 0.001474 (0.27) | 0.014024 (2.60) |  0.050929 (9.45) |  0.335191 (62.18) | 0.001559 (0.29) | 0.002095 (0.39) |
+|      many_numbers | 0.107804 (1.00) | 0.025544 (0.24) | 0.122202 (1.13) |              N/A |               N/A | 0.029262 (0.27) | 0.026409 (0.24) |
 
 ```
 make bench            # Run parsing benchmark

{sqlglot-30.0.1 → sqlglot-30.0.2}/pyproject.toml

@@ -40,6 +40,9 @@ include = ["sqlglot", "sqlglot.*"]
 [tool.setuptools.package-data]
 "*" = ["py.typed"]
 
+[tool.uv.sources]
+sqlglotc = { path = "./sqlglotc" }
+
 [tool.ruff]
 line-length = 100
 

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/__init__.py

@@ -1,4 +1,4 @@
-# ruff: noqa: F401
+# ruff: noqa: F401, E402
 """
 .. include:: ../README.md
 
@@ -7,6 +7,23 @@
 
 from __future__ import annotations
 
+# bootstrap mypyc runtime: compiled .so modules do a top-level `import HASH__mypyc`,
+# but the runtime .so lives inside sqlglot/. Pre-load it into sys.modules.
+# this is only needed for editable builds
+import sys
+from pathlib import Path
+
+for path in Path(__file__).parent.glob("*__mypyc*.so"):
+    name = path.stem.split(".")[0]
+    if name not in sys.modules:
+        import importlib.util
+
+        spec = importlib.util.spec_from_file_location(name, path)
+        if spec and spec.loader:
+            mod = importlib.util.module_from_spec(spec)
+            sys.modules[name] = mod
+            spec.loader.exec_module(mod)
+
 import logging
 import typing as t
 

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/_version.py

@@ -28,7 +28,7 @@ version_tuple: VERSION_TUPLE
 commit_id: COMMIT_ID
 __commit_id__: COMMIT_ID
 
-__version__ = version = '30.0.1'
-__version_tuple__ = version_tuple = (30, 0, 1)
+__version__ = version = '30.0.2'
+__version_tuple__ = version_tuple = (30, 0, 2)
 
-__commit_id__ = commit_id = 'g7bcad5c61'
+__commit_id__ = commit_id = 'g64509a2a3'

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/dialects/databricks.py

@@ -43,6 +43,7 @@ class Databricks(Spark):
     class Tokenizer(Spark.Tokenizer):
         KEYWORDS = {
             **Spark.Tokenizer.KEYWORDS,
+            "STREAM": TokenType.STREAM,
             "VOID": TokenType.VOID,
         }
 

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/dialects/duckdb.py

@@ -2202,6 +2202,58 @@ class DuckDB(Dialect):
             """
         )
 
+        # Template for STRTOK function transpilation
+        #
+        # DuckDB itself doesn't have a strtok function. This handles the transpilation from Snowflake to DuckDB.
+        # We may need to adjust this if we want to support transpilation from other dialects
+        #
+        # CASE
+        #     -- Snowflake: empty delimiter + empty input string → NULL
+        #     WHEN delimiter = '' AND input_str = '' THEN NULL
+        #
+        #     -- Snowflake: empty delimiter + non-empty input string → treats whole input as 1 token → return input string if index is 1
+        #     WHEN delimiter = '' AND index = 1 THEN input_str
+        #
+        #     -- Snowflake: empty delimiter + non-empty input string → treats whole input as 1 token → return NULL if index is not 1
+        #     WHEN delimiter = '' THEN NULL
+        #
+        #     -- Snowflake: negative indices return NULL
+        #     WHEN index < 0 THEN NULL
+        #
+        #     -- Snowflake: return NULL if any argument is NULL
+        #     WHEN input_str IS NULL OR delimiter IS NULL OR index IS NULL THEN NULL
+        #
+        #
+        #     ELSE LIST_FILTER(
+        #         REGEXP_SPLIT_TO_ARRAY(
+        #             input_str,
+        #             CASE
+        #                 -- if delimiter is '', we don't want to surround it with '[' and ']' as '[]' is invalid for DuckDB
+        #                 WHEN delimiter = '' THEN ''
+        #
+        #                 -- handle problematic regex characters in delimiter with REGEXP_REPLACE
+        #                 -- turn delimiter into a regex char set, otherwise DuckDB will match in order, which we don't want
+        #                 ELSE '[' || REGEXP_REPLACE(delimiter, problematic_char_set, '\\\1', 'g') || ']'
+        #             END
+        #         ),
+        #
+        #         -- Snowflake: don't return empty strings
+        #         x -> NOT x = ''
+        #     )[index]
+        # END
+        STRTOK_TEMPLATE: exp.Expr = exp.maybe_parse(
+            """
+            CASE
+                WHEN :delimiter = '' AND :string = '' THEN NULL
+                WHEN :delimiter = '' AND :part_index = 1 THEN :string
+                WHEN :delimiter = '' THEN NULL
+                WHEN :part_index < 0 THEN NULL
+                WHEN :string IS NULL OR :delimiter IS NULL OR :part_index IS NULL THEN NULL
+                ELSE :base_func
+            END
+            """
+        )
+
         def _array_bag_sql(self, condition: exp.Expr, arr1: exp.Expr, arr2: exp.Expr) -> str:
             cond = exp.Paren(this=exp.replace_placeholders(condition, arr1=arr1, arr2=arr2))
             return self.sql(
@@ -3642,14 +3694,14 @@ class DuckDB(Dialect):
                 """
                 Output looks something like this:
 
-                CASE 
-                WHEN delimiter is '' THEN 
+                CASE
+                WHEN delimiter is '' THEN
                     (
-                        CASE 
+                        CASE
                         WHEN adjusted_part_index = 1 OR adjusted_part_index = -1 THEN input
                         ELSE '' END
-                    ) 
-                ELSE SPLIT_PART(input, delimiter, adjusted_part_index) 
+                    )
+                ELSE SPLIT_PART(input, delimiter, adjusted_part_index)
                 END
 
                 """
@@ -3667,13 +3719,46 @@ class DuckDB(Dialect):
             return self.sql(expression, "this")
 
         def arraytostring_sql(self, expression: exp.ArrayToString) -> str:
-            this = self.sql(expression, "this")
-            null_text = self.sql(expression, "null")
+            null = expression.args.get("null")
+
+            if expression.args.get("null_is_empty"):
+                x = exp.to_identifier("x")
+                list_transform = exp.Transform(
+                    this=expression.this.copy(),
+                    expression=exp.Lambda(
+                        this=exp.Coalesce(
+                            this=exp.cast(x, "TEXT"), expressions=[exp.Literal.string("")]
+                        ),
+                        expressions=[x],
+                    ),
+                )
+                array_to_string = exp.ArrayToString(
+                    this=list_transform, expression=expression.expression
+                )
+                if expression.args.get("null_delim_is_null"):
+                    return self.sql(
+                        exp.case()
+                        .when(expression.expression.copy().is_(exp.null()), exp.null())
+                        .else_(array_to_string)
+                    )
+                return self.sql(array_to_string)
 
-            if null_text:
-                this = f"LIST_TRANSFORM({this}, x -> COALESCE(x, {null_text}))"
+            if null:
+                x = exp.to_identifier("x")
+                return self.sql(
+                    exp.ArrayToString(
+                        this=exp.Transform(
+                            this=expression.this,
+                            expression=exp.Lambda(
+                                this=exp.Coalesce(this=x, expressions=[null]),
+                                expressions=[x],
+                            ),
+                        ),
+                        expression=expression.expression,
+                    )
+                )
 
-            return self.func("ARRAY_TO_STRING", this, expression.expression)
+            return self.func("ARRAY_TO_STRING", expression.this, expression.expression)
 
         def _regexp_extract_sql(self, expression: exp.RegexpExtract | exp.RegexpExtractAll) -> str:
             this = expression.this
@@ -4033,6 +4118,69 @@ class DuckDB(Dialect):
 
             return self.func(func, this, decimals, truncate)
 
+        def strtok_sql(self, expression: exp.Strtok) -> str:
+            string_arg = expression.this
+            delimiter_arg = expression.args.get("delimiter")
+            part_index_arg = expression.args.get("part_index")
+
+            if delimiter_arg and part_index_arg:
+                # Escape regex chars and build character class at runtime using REGEXP_REPLACE
+                escaped_delimiter = exp.Anonymous(
+                    this="REGEXP_REPLACE",
+                    expressions=[
+                        delimiter_arg,
+                        exp.Literal.string(
+                            r"([\[\]^.\-*+?(){}|$\\])"
+                        ),  # Escape problematic regex chars
+                        exp.Literal.string(
+                            r"\\\1"
+                        ),  # Replace with escaped version using $1 backreference
+                        exp.Literal.string("g"),  # Global flag
+                    ],
+                )
+                # CASE WHEN delimiter = '' THEN '' ELSE CONCAT('[', escaped_delimiter, ']') END
+                regex_pattern = (
+                    exp.case()
+                    .when(delimiter_arg.eq(exp.Literal.string("")), exp.Literal.string(""))
+                    .else_(
+                        exp.func(
+                            "CONCAT",
+                            exp.Literal.string("["),
+                            escaped_delimiter,
+                            exp.Literal.string("]"),
+                        )
+                    )
+                )
+
+                # STRTOK skips empty strings, so we need to filter them out
+                # LIST_FILTER(REGEXP_SPLIT_TO_ARRAY(string, pattern), x -> x != '')[index]
+                split_array = exp.func("REGEXP_SPLIT_TO_ARRAY", string_arg, regex_pattern)
+                x = exp.to_identifier("x")
+                is_empty = x.eq(exp.Literal.string(""))
+                filtered_array = exp.func(
+                    "LIST_FILTER",
+                    split_array,
+                    exp.Lambda(this=exp.not_(is_empty.copy()), expressions=[x.copy()]),
+                )
+                base_func = exp.Bracket(
+                    this=filtered_array,
+                    expressions=[part_index_arg],
+                    offset=1,
+                )
+
+                # Use template with the built regex pattern
+                result = exp.replace_placeholders(
+                    self.STRTOK_TEMPLATE.copy(),
+                    string=string_arg,
+                    delimiter=delimiter_arg,
+                    part_index=part_index_arg,
+                    base_func=base_func,
+                )
+
+                return self.sql(result)
+
+            return self.function_fallback_sql(expression)
+
         def approxquantile_sql(self, expression: exp.ApproxQuantile) -> str:
             result = self.func("APPROX_QUANTILE", expression.this, expression.args.get("quantile"))
 

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/dialects/snowflake.py

@@ -1201,6 +1201,9 @@ class Snowflake(Dialect):
 
             return expr_sql
 
+        def arraytostring_sql(self, expression: exp.ArrayToString) -> str:
+            return self.func("ARRAY_TO_STRING", expression.this, expression.expression)
+
         def array_sql(self, expression: exp.Array) -> str:
             expressions = expression.expressions
 

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/expressions/array.py

@@ -174,7 +174,13 @@ class ArraysZip(Expression, Func):
 
 
 class ArrayToString(Expression, Func):
-    arg_types = {"this": True, "expression": True, "null": False}
+    arg_types = {
+        "this": True,
+        "expression": True,
+        "null": False,
+        "null_is_empty": False,
+        "null_delim_is_null": False,
+    }
     _sql_names = ["ARRAY_TO_STRING", "ARRAY_JOIN"]
 
 

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/expressions/core.py

@@ -617,8 +617,8 @@ class Expression(Expr):
         Returns the alias of the expression, or an empty string if it's not aliased.
         """
         alias = self.args.get("alias")
-        if type(alias).__name__ == "TableAlias":
-            return alias.name  # type: ignore[union-attr]
+        if isinstance(alias, Expression):
+            return alias.name
         return self.text("alias")
 
     @property

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/expressions/string.py

@@ -177,6 +177,14 @@ class SplitPart(Expression, Func):
     }
 
 
+class Strtok(Expression, Func):
+    arg_types = {
+        "this": True,
+        "delimiter": False,
+        "part_index": False,
+    }
+
+
 class StartsWith(Expression, Func):
     _sql_names = ["STARTS_WITH", "STARTSWITH"]
     arg_types = {"this": True, "expression": True}

{sqlglot-30.0.1 → sqlglot-30.0.2}/sqlglot/generator.py

@@ -933,9 +933,10 @@ class Generator(metaclass=_Generator):
         comment = " " + comment if comment[0].strip() else comment
         comment = comment + " " if comment[-1].strip() else comment
 
-        if not self.dialect.tokenizer_class.NESTED_COMMENTS:
-            # Necessary workaround to avoid syntax errors due to nesting: /* ... */ ... */
-            comment = comment.replace("*/", "* /")
+        # Escape block comment markers to prevent premature closure or unintended nesting.
+        # This is necessary because single-line comments (--) are converted to block comments
+        # (/* */) on output, and any */ in the original text would close the comment early.
+        comment = comment.replace("*/", "* /").replace("/*", "/ *")
 
         return comment
 
@@ -4086,6 +4087,8 @@ class Generator(metaclass=_Generator):
 
             if isinstance(exprs, exp.Tuple):
                 exprs = exprs.expressions
+            else:
+                exprs = [exprs]
 
             connective = exp.or_ if isinstance(rhs, exp.Any) else exp.and_