sqlglot 27.10.0__tar.gz → 27.11.0__tar.gz

{sqlglot-27.10.0 → sqlglot-27.11.0}/CHANGELOG.md

@@ -1,6 +1,44 @@
 Changelog
 =========
 
+## [v27.10.0] - 2025-08-28
+### :boom: BREAKING CHANGES
+- due to [`de2fe15`](https://github.com/tobymao/sqlglot/commit/de2fe1503b5bb003431d1f0c7b9ae87932a6cc1c) - annotate type for bq CONTAINS_SUBSTR *(PR [#5705](https://github.com/tobymao/sqlglot/pull/5705) by [@geooo109](https://github.com/geooo109))*:
+
+  annotate type for bq CONTAINS_SUBSTR (#5705)
+
+- due to [`770888f`](https://github.com/tobymao/sqlglot/commit/770888f4e9a9061329e3c416f968f7dd9639fb81) - annotate type for bq NORMALIZE *(PR [#5711](https://github.com/tobymao/sqlglot/pull/5711) by [@geooo109](https://github.com/geooo109))*:
+
+  annotate type for bq NORMALIZE (#5711)
+
+- due to [`506033f`](https://github.com/tobymao/sqlglot/commit/506033f299f7a4c28f6efd8bf715be5dcf73e929) - parse and annotate type for bq NORMALIZE_AND_CASEFOLD *(PR [#5712](https://github.com/tobymao/sqlglot/pull/5712) by [@geooo109](https://github.com/geooo109))*:
+
+  parse and annotate type for bq NORMALIZE_AND_CASEFOLD (#5712)
+
+- due to [`848aea1`](https://github.com/tobymao/sqlglot/commit/848aea1dbaaeb580b633796dcca06c28314b9c3e) - parse and annotate type for bq OCTET_LENGTH *(PR [#5713](https://github.com/tobymao/sqlglot/pull/5713) by [@geooo109](https://github.com/geooo109))*:
+
+  parse and annotate type for bq OCTET_LENGTH (#5713)
+
+- due to [`727bf83`](https://github.com/tobymao/sqlglot/commit/727bf8378f232188d35834d980b035552999ea3b) - add support for REVOKE DDL *(PR [#5703](https://github.com/tobymao/sqlglot/pull/5703) by [@newtonapple](https://github.com/newtonapple))*:
+
+  add support for REVOKE DDL (#5703)
+
+
+### :sparkles: New Features
+- [`f6f8f56`](https://github.com/tobymao/sqlglot/commit/f6f8f56a59d550dfc7dfcab0c3b9a6885c7e758a) - **singlestore**: Fixed parsing/generation of exp.JSONFormat *(PR [#5706](https://github.com/tobymao/sqlglot/pull/5706) by [@AdalbertMemSQL](https://github.com/AdalbertMemSQL))*
+- [`de2fe15`](https://github.com/tobymao/sqlglot/commit/de2fe1503b5bb003431d1f0c7b9ae87932a6cc1c) - **optimizer**: annotate type for bq CONTAINS_SUBSTR *(PR [#5705](https://github.com/tobymao/sqlglot/pull/5705) by [@geooo109](https://github.com/geooo109))*
+- [`a78146e`](https://github.com/tobymao/sqlglot/commit/a78146e37bfc972050b4467c39769407061e9bc3) - **singlestore**: Fixed parsing/generation of exp.DateBin *(PR [#5709](https://github.com/tobymao/sqlglot/pull/5709) by [@AdalbertMemSQL](https://github.com/AdalbertMemSQL))*
+- [`ab0c985`](https://github.com/tobymao/sqlglot/commit/ab0c985424ae9d9340eafd15ecdc9b31bdd8837c) - **singlestore**: Marked exp.Reduce finish argument as unsupported *(PR [#5707](https://github.com/tobymao/sqlglot/pull/5707) by [@AdalbertMemSQL](https://github.com/AdalbertMemSQL))*
+- [`770888f`](https://github.com/tobymao/sqlglot/commit/770888f4e9a9061329e3c416f968f7dd9639fb81) - **optimizer**: annotate type for bq NORMALIZE *(PR [#5711](https://github.com/tobymao/sqlglot/pull/5711) by [@geooo109](https://github.com/geooo109))*
+- [`506033f`](https://github.com/tobymao/sqlglot/commit/506033f299f7a4c28f6efd8bf715be5dcf73e929) - **optimizer**: parse and annotate type for bq NORMALIZE_AND_CASEFOLD *(PR [#5712](https://github.com/tobymao/sqlglot/pull/5712) by [@geooo109](https://github.com/geooo109))*
+- [`848aea1`](https://github.com/tobymao/sqlglot/commit/848aea1dbaaeb580b633796dcca06c28314b9c3e) - **optimizer**: parse and annotate type for bq OCTET_LENGTH *(PR [#5713](https://github.com/tobymao/sqlglot/pull/5713) by [@geooo109](https://github.com/geooo109))*
+- [`727bf83`](https://github.com/tobymao/sqlglot/commit/727bf8378f232188d35834d980b035552999ea3b) - add support for REVOKE DDL *(PR [#5703](https://github.com/tobymao/sqlglot/pull/5703) by [@newtonapple](https://github.com/newtonapple))*
+
+### :bug: Bug Fixes
+- [`0427c7b`](https://github.com/tobymao/sqlglot/commit/0427c7b7aa9f8161324085a98c5f531fa35c8b0c) - **optimizer**: qualify columns for AggFunc with DISTINCT *(PR [#5708](https://github.com/tobymao/sqlglot/pull/5708) by [@geooo109](https://github.com/geooo109))*
+  - :arrow_lower_right: *fixes issue [#5698](https://github.com/tobymao/sqlglot/issues/5698) opened by [@georgesittas](https://github.com/georgesittas)*
+
+
 ## [v27.9.0] - 2025-08-27
 ### :boom: BREAKING CHANGES
 - due to [`7b180bd`](https://github.com/tobymao/sqlglot/commit/7b180bdc3da9e39946c22970bd2523f7d8beaf29) - raise if query modifier is specified multiple times *(PR [#5608](https://github.com/tobymao/sqlglot/pull/5608) by [@georgesittas](https://github.com/georgesittas))*:
@@ -6774,3 +6812,4 @@ Changelog
 [v27.7.0]: https://github.com/tobymao/sqlglot/compare/v27.6.0...v27.7.0
 [v27.8.0]: https://github.com/tobymao/sqlglot/compare/v27.7.0...v27.8.0
 [v27.9.0]: https://github.com/tobymao/sqlglot/compare/v27.8.0...v27.9.0
+[v27.10.0]: https://github.com/tobymao/sqlglot/compare/v27.9.0...v27.10.0

{sqlglot-27.10.0 → sqlglot-27.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sqlglot
-Version: 27.10.0
+Version: 27.11.0
 Summary: An easily customizable SQL parser and transpiler
 Author-email: Toby Mao <toby.mao@gmail.com>
 License-Expression: MIT
@@ -33,7 +33,7 @@ Requires-Dist: typing_extensions; extra == "dev"
 Requires-Dist: maturin<2.0,>=1.4; extra == "dev"
 Requires-Dist: pyperf; extra == "dev"
 Provides-Extra: rs
-Requires-Dist: sqlglotrs==0.6.1; extra == "rs"
+Requires-Dist: sqlglotrs==0.6.2; extra == "rs"
 Dynamic: license-file
 Dynamic: provides-extra
 

{sqlglot-27.10.0 → sqlglot-27.11.0}/sqlglot/_version.py

@@ -28,7 +28,7 @@ version_tuple: VERSION_TUPLE
 commit_id: COMMIT_ID
 __commit_id__: COMMIT_ID
 
-__version__ = version = '27.10.0'
-__version_tuple__ = version_tuple = (27, 10, 0)
+__version__ = version = '27.11.0'
+__version_tuple__ = version_tuple = (27, 11, 0)
 
-__commit_id__ = commit_id = 'g727bf8378'
+__commit_id__ = commit_id = 'g41521e31b'

{sqlglot-27.10.0 → sqlglot-27.11.0}/sqlglot/dialects/bigquery.py

@@ -4,6 +4,9 @@ import logging
 import re
 import typing as t
 
+
+from sqlglot.optimizer.annotate_types import TypeAnnotator
+
 from sqlglot import exp, generator, jsonpath, parser, tokens, transforms
 from sqlglot._typing import E
 from sqlglot.dialects.dialect import (
@@ -172,6 +175,18 @@ def _build_to_hex(args: t.List) -> exp.Hex | exp.MD5:
     return exp.MD5(this=arg.this) if isinstance(arg, exp.MD5Digest) else exp.LowerHex(this=arg)
 
 
+def _build_json_strip_nulls(args: t.List) -> exp.JSONStripNulls:
+    expression = exp.JSONStripNulls(this=seq_get(args, 0))
+
+    for arg in args[1:]:
+        if isinstance(arg, exp.Kwarg):
+            expression.set(arg.this.name.lower(), arg)
+        else:
+            expression.set("expression", arg)
+
+    return expression
+
+
 def _array_contains_sql(self: BigQuery.Generator, expression: exp.ArrayContains) -> str:
     return self.sql(
         exp.Exists(
@@ -295,6 +310,24 @@ def _annotate_math_functions(self: TypeAnnotator, expression: E) -> E:
     return expression
 
 
+def _annotate_perncentile_cont(
+    self: TypeAnnotator, expression: exp.PercentileCont
+) -> exp.PercentileCont:
+    """
+    +------------+-----------+------------+---------+
+    | INPUT      | NUMERIC   | BIGNUMERIC | FLOAT64 |
+    +------------+-----------+------------+---------+
+    | NUMERIC    | NUMERIC   | BIGNUMERIC | FLOAT64 |
+    | BIGNUMERIC | BIGNUMERIC| BIGNUMERIC | FLOAT64 |
+    | FLOAT64    | FLOAT64   | FLOAT64    | FLOAT64 |
+    +------------+-----------+------------+---------+
+    """
+    self._annotate_args(expression)
+
+    self._set_type(expression, self._maybe_coerce(expression.this.type, expression.expression.type))
+    return expression
+
+
 def _annotate_by_args_approx_top(self: TypeAnnotator, expression: exp.ApproxTopK) -> exp.ApproxTopK:
     self._annotate_args(expression)
 
@@ -453,6 +486,13 @@ class BigQuery(Dialect):
     # All set operations require either a DISTINCT or ALL specifier
     SET_OP_DISTINCT_BY_DEFAULT = dict.fromkeys((exp.Except, exp.Intersect, exp.Union), None)
 
+    # https://cloud.google.com/bigquery/docs/reference/standard-sql/navigation_functions#percentile_cont
+    COERCES_TO = {
+        **TypeAnnotator.COERCES_TO,
+        exp.DataType.Type.BIGDECIMAL: {exp.DataType.Type.DOUBLE},
+    }
+    COERCES_TO[exp.DataType.Type.DECIMAL] |= {exp.DataType.Type.BIGDECIMAL}
+
     # BigQuery maps Type.TIMESTAMP to DATETIME, so we need to amend the inferred types
     TYPE_TO_EXPRESSIONS = {
         **Dialect.TYPE_TO_EXPRESSIONS,
@@ -511,37 +551,68 @@ class BigQuery(Dialect):
         exp.Corr: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DOUBLE),
         exp.CovarPop: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DOUBLE),
         exp.CovarSamp: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DOUBLE),
+        exp.CumeDist: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DOUBLE),
         exp.DateFromUnixDate: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DATE),
         exp.DateTrunc: lambda self, e: self._annotate_by_args(e, "this"),
+        exp.DenseRank: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
         exp.FarmFingerprint: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
+        exp.FirstValue: lambda self, e: self._annotate_by_args(e, "this"),
         exp.Unhex: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BINARY),
         exp.Float64: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DOUBLE),
+        exp.Format: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.VARCHAR),
         exp.GenerateTimestampArray: lambda self, e: self._annotate_with_type(
             e, exp.DataType.build("ARRAY<TIMESTAMP>", dialect="bigquery")
         ),
         exp.Grouping: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
+        exp.IgnoreNulls: lambda self, e: self._annotate_by_args(e, "this"),
         exp.JSONArray: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.JSON),
+        exp.JSONArrayAppend: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.JSON),
+        exp.JSONArrayInsert: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.JSON),
         exp.JSONBool: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BOOLEAN),
         exp.JSONExtractScalar: lambda self, e: self._annotate_with_type(
             e, exp.DataType.Type.VARCHAR
         ),
-        exp.JSONValueArray: lambda self, e: self._annotate_with_type(
-            e, exp.DataType.build("ARRAY<VARCHAR>")
+        exp.JSONExtract: lambda self, e: self._annotate_by_args(e, "this"),
+        exp.JSONExtractArray: lambda self, e: self._annotate_by_args(e, "this", array=True),
+        exp.JSONFormat: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.VARCHAR),
+        exp.JSONKeysAtDepth: lambda self, e: self._annotate_with_type(
+            e, exp.DataType.build("ARRAY<VARCHAR>", dialect="bigquery")
         ),
+        exp.JSONObject: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.JSON),
+        exp.JSONRemove: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.JSON),
+        exp.JSONSet: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.JSON),
+        exp.JSONStripNulls: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.JSON),
         exp.JSONType: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.VARCHAR),
+        exp.JSONValueArray: lambda self, e: self._annotate_with_type(
+            e, exp.DataType.build("ARRAY<VARCHAR>", dialect="bigquery")
+        ),
         exp.Lag: lambda self, e: self._annotate_by_args(e, "this", "default"),
+        exp.Lead: lambda self, e: self._annotate_by_args(e, "this"),
         exp.LowerHex: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.VARCHAR),
+        exp.LaxBool: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BOOLEAN),
+        exp.LaxFloat64: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DOUBLE),
+        exp.LaxInt64: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
+        exp.LaxString: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.VARCHAR),
         exp.MD5Digest: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BINARY),
         exp.Normalize: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.VARCHAR),
+        exp.NthValue: lambda self, e: self._annotate_by_args(e, "this"),
+        exp.Ntile: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
         exp.ParseTime: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.TIME),
         exp.ParseDatetime: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DATETIME),
         exp.ParseBignumeric: lambda self, e: self._annotate_with_type(
             e, exp.DataType.Type.BIGDECIMAL
         ),
         exp.ParseNumeric: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DECIMAL),
+        exp.PercentileCont: lambda self, e: _annotate_perncentile_cont(self, e),
+        exp.PercentileDisc: lambda self, e: self._annotate_by_args(e, "this"),
+        exp.PercentRank: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.DOUBLE),
+        exp.Rank: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
         exp.RegexpExtractAll: lambda self, e: self._annotate_by_args(e, "this", array=True),
+        exp.RegexpInstr: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
         exp.Replace: lambda self, e: self._annotate_by_args(e, "this"),
+        exp.RespectNulls: lambda self, e: self._annotate_by_args(e, "this"),
         exp.Reverse: lambda self, e: self._annotate_by_args(e, "this"),
+        exp.RowNumber: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.BIGINT),
         exp.SafeConvertBytesToString: lambda self, e: self._annotate_with_type(
             e, exp.DataType.Type.VARCHAR
         ),
@@ -682,8 +753,11 @@ class BigQuery(Dialect):
             "GENERATE_ARRAY": exp.GenerateSeries.from_arg_list,
             "JSON_EXTRACT_SCALAR": _build_extract_json_with_default_path(exp.JSONExtractScalar),
             "JSON_EXTRACT_ARRAY": _build_extract_json_with_default_path(exp.JSONExtractArray),
+            "JSON_EXTRACT_STRING_ARRAY": _build_extract_json_with_default_path(exp.JSONValueArray),
+            "JSON_KEYS": exp.JSONKeysAtDepth.from_arg_list,
             "JSON_QUERY": parser.build_extract_json_with_path(exp.JSONExtract),
             "JSON_QUERY_ARRAY": _build_extract_json_with_default_path(exp.JSONExtractArray),
+            "JSON_STRIP_NULLS": _build_json_strip_nulls,
             "JSON_VALUE": _build_extract_json_with_default_path(exp.JSONExtractScalar),
             "JSON_VALUE_ARRAY": _build_extract_json_with_default_path(exp.JSONValueArray),
             "LENGTH": lambda args: exp.Length(this=seq_get(args, 0), binary=True),
@@ -798,9 +872,13 @@ class BigQuery(Dialect):
             "SAFE_ORDINAL": (1, True),
         }
 
-        def _parse_for_in(self) -> exp.ForIn:
+        def _parse_for_in(self) -> t.Union[exp.ForIn, exp.Command]:
+            index = self._index
             this = self._parse_range()
             self._match_text_seq("DO")
+            if self._match(TokenType.COMMAND):
+                self._retreat(index)
+                return self._parse_as_command(self._prev)
             return self.expression(exp.ForIn, this=this, expression=self._parse_statement())
 
         def _parse_table_part(self, schema: bool = False) -> t.Optional[exp.Expression]:
@@ -1197,6 +1275,8 @@ class BigQuery(Dialect):
             exp.JSONExtractArray: _json_extract_sql,
             exp.JSONExtractScalar: _json_extract_sql,
             exp.JSONFormat: rename_func("TO_JSON_STRING"),
+            exp.JSONKeysAtDepth: rename_func("JSON_KEYS"),
+            exp.JSONValueArray: rename_func("JSON_VALUE_ARRAY"),
             exp.Levenshtein: _levenshtein_sql,
             exp.Max: max_or_greatest,
             exp.MD5: lambda self, e: self.func("TO_HEX", self.func("MD5", e.this)),

{sqlglot-27.10.0 → sqlglot-27.11.0}/sqlglot/dialects/dremio.py

@@ -74,6 +74,27 @@ def build_date_delta_with_cast_interval(
     return _builder
 
 
+def datetype_handler(args: t.List[exp.Expression], dialect: DialectType) -> exp.Expression:
+    year, month, day = args
+
+    if all(isinstance(arg, exp.Literal) and arg.is_int for arg in (year, month, day)):
+        date_str = f"{int(year.this):04d}-{int(month.this):02d}-{int(day.this):02d}"
+        return exp.Date(this=exp.Literal.string(date_str))
+
+    return exp.Cast(
+        this=exp.Concat(
+            expressions=[
+                year,
+                exp.Literal.string("-"),
+                month,
+                exp.Literal.string("-"),
+                day,
+            ]
+        ),
+        to=exp.DataType.build("DATE"),
+    )
+
+
 class Dremio(Dialect):
     SUPPORTS_USER_DEFINED_TYPES = False
     CONCAT_COALESCE = True
@@ -145,12 +166,16 @@ class Dremio(Dialect):
 
         FUNCTIONS = {
             **parser.Parser.FUNCTIONS,
-            "TO_CHAR": to_char_is_numeric_handler,
-            "DATE_FORMAT": build_formatted_time(exp.TimeToStr, "dremio"),
-            "TO_DATE": build_formatted_time(exp.TsOrDsToDate, "dremio"),
+            "ARRAY_GENERATE_RANGE": exp.GenerateSeries.from_arg_list,
             "DATE_ADD": build_date_delta_with_cast_interval(exp.DateAdd),
+            "DATE_FORMAT": build_formatted_time(exp.TimeToStr, "dremio"),
             "DATE_SUB": build_date_delta_with_cast_interval(exp.DateSub),
-            "ARRAY_GENERATE_RANGE": exp.GenerateSeries.from_arg_list,
+            "REGEXP_MATCHES": exp.RegexpLike.from_arg_list,
+            "REPEATSTR": exp.Repeat.from_arg_list,
+            "TO_CHAR": to_char_is_numeric_handler,
+            "TO_DATE": build_formatted_time(exp.TsOrDsToDate, "dremio"),
+            "DATE_PART": exp.Extract.from_arg_list,
+            "DATETYPE": datetype_handler,
         }
 
         def _parse_current_date_utc(self) -> exp.Cast:

{sqlglot-27.10.0 → sqlglot-27.11.0}/sqlglot/dialects/singlestore.py

@@ -10,8 +10,11 @@ from sqlglot.dialects.dialect import (
     rename_func,
     bool_xor_sql,
     count_if_to_sum,
+    timestamptrunc_sql,
+    date_add_interval_sql,
+    timestampdiff_sql,
 )
-from sqlglot.dialects.mysql import MySQL
+from sqlglot.dialects.mysql import MySQL, _remove_ts_or_ds_to_date, date_add_sql
 from sqlglot.expressions import DataType
 from sqlglot.generator import unsupported_args
 from sqlglot.helper import seq_get
@@ -55,6 +58,7 @@ class SingleStore(MySQL):
             **MySQL.Tokenizer.KEYWORDS,
             "BSON": TokenType.JSONB,
             "GEOGRAPHYPOINT": TokenType.GEOGRAPHYPOINT,
+            "TIMESTAMP": TokenType.TIMESTAMP,
             ":>": TokenType.COLON_GT,
             "!:>": TokenType.NCOLON_GT,
             "::$": TokenType.DCOLONDOLLAR,
@@ -159,6 +163,11 @@ class SingleStore(MySQL):
                 this=seq_get(args, 0),
                 format=MySQL.format_time(exp.Literal.string("%W")),
             ),
+            "TIMESTAMPDIFF": lambda args: exp.TimestampDiff(
+                this=seq_get(args, 2),
+                expression=seq_get(args, 1),
+                unit=seq_get(args, 0),
+            ),
             "APPROX_COUNT_DISTINCT": exp.Hll.from_arg_list,
             "APPROX_PERCENTILE": lambda args, dialect: exp.ApproxQuantile(
                 this=seq_get(args, 0),
@@ -189,6 +198,7 @@ class SingleStore(MySQL):
         CAST_COLUMN_OPERATORS = {TokenType.COLON_GT, TokenType.NCOLON_GT}
 
         COLUMN_OPERATORS = {
+            **MySQL.Parser.COLUMN_OPERATORS,
             TokenType.COLON_GT: lambda self, this, to: self.expression(
                 exp.Cast,
                 this=this,
@@ -209,6 +219,11 @@ class SingleStore(MySQL):
                 exp.JSONExtractScalar, json_type="DOUBLE"
             )([this, exp.Literal.string(path.name)]),
         }
+        COLUMN_OPERATORS.pop(TokenType.ARROW)
+        COLUMN_OPERATORS.pop(TokenType.DARROW)
+        COLUMN_OPERATORS.pop(TokenType.HASH_ARROW)
+        COLUMN_OPERATORS.pop(TokenType.DHASH_ARROW)
+        COLUMN_OPERATORS.pop(TokenType.PLACEHOLDER)
 
     class Generator(MySQL.Generator):
         SUPPORTED_JSON_PATH_PARTS = {
@@ -277,6 +292,28 @@ class SingleStore(MySQL):
             exp.DateBin: unsupported_args("unit", "zone")(
                 lambda self, e: self.func("TIME_BUCKET", e.this, e.expression, e.args.get("origin"))
             ),
+            exp.TimeStrToDate: lambda self, e: self.sql(exp.cast(e.this, exp.DataType.Type.DATE)),
+            exp.FromTimeZone: lambda self, e: self.func(
+                "CONVERT_TZ", e.this, e.args.get("zone"), "'UTC'"
+            ),
+            exp.DiToDate: lambda self,
+            e: f"STR_TO_DATE({self.sql(e, 'this')}, {SingleStore.DATEINT_FORMAT})",
+            exp.DateToDi: lambda self,
+            e: f"(DATE_FORMAT({self.sql(e, 'this')}, {SingleStore.DATEINT_FORMAT}) :> INT)",
+            exp.TsOrDiToDi: lambda self,
+            e: f"(DATE_FORMAT({self.sql(e, 'this')}, {SingleStore.DATEINT_FORMAT}) :> INT)",
+            exp.Time: unsupported_args("zone")(lambda self, e: f"{self.sql(e, 'this')} :> TIME"),
+            exp.DatetimeAdd: _remove_ts_or_ds_to_date(date_add_sql("ADD")),
+            exp.DatetimeTrunc: unsupported_args("zone")(timestamptrunc_sql()),
+            exp.DatetimeSub: date_add_interval_sql("DATE", "SUB"),
+            exp.DatetimeDiff: timestampdiff_sql,
+            exp.DateTrunc: unsupported_args("zone")(timestamptrunc_sql()),
+            exp.DateDiff: unsupported_args("zone")(
+                lambda self, e: timestampdiff_sql(self, e)
+                if e.unit is not None
+                else self.func("DATEDIFF", e.this, e.expression)
+            ),
+            exp.TimestampTrunc: unsupported_args("zone")(timestamptrunc_sql()),
             exp.JSONExtract: unsupported_args(
                 "only_json_types",
                 "expressions",
@@ -353,6 +390,94 @@ class SingleStore(MySQL):
         }
         TRANSFORMS.pop(exp.JSONExtractScalar)
 
+        UNSUPPORTED_TYPES = {
+            exp.DataType.Type.ARRAY,
+            exp.DataType.Type.AGGREGATEFUNCTION,
+            exp.DataType.Type.SIMPLEAGGREGATEFUNCTION,
+            exp.DataType.Type.BIGSERIAL,
+            exp.DataType.Type.BPCHAR,
+            exp.DataType.Type.DATEMULTIRANGE,
+            exp.DataType.Type.DATERANGE,
+            exp.DataType.Type.DYNAMIC,
+            exp.DataType.Type.HLLSKETCH,
+            exp.DataType.Type.HSTORE,
+            exp.DataType.Type.IMAGE,
+            exp.DataType.Type.INET,
+            exp.DataType.Type.INT128,
+            exp.DataType.Type.INT256,
+            exp.DataType.Type.INT4MULTIRANGE,
+            exp.DataType.Type.INT4RANGE,
+            exp.DataType.Type.INT8MULTIRANGE,
+            exp.DataType.Type.INT8RANGE,
+            exp.DataType.Type.INTERVAL,
+            exp.DataType.Type.IPADDRESS,
+            exp.DataType.Type.IPPREFIX,
+            exp.DataType.Type.IPV4,
+            exp.DataType.Type.IPV6,
+            exp.DataType.Type.LIST,
+            exp.DataType.Type.MAP,
+            exp.DataType.Type.LOWCARDINALITY,
+            exp.DataType.Type.MONEY,
+            exp.DataType.Type.MULTILINESTRING,
+            exp.DataType.Type.NAME,
+            exp.DataType.Type.NESTED,
+            exp.DataType.Type.NOTHING,
+            exp.DataType.Type.NULL,
+            exp.DataType.Type.NUMMULTIRANGE,
+            exp.DataType.Type.NUMRANGE,
+            exp.DataType.Type.OBJECT,
+            exp.DataType.Type.RANGE,
+            exp.DataType.Type.ROWVERSION,
+            exp.DataType.Type.SERIAL,
+            exp.DataType.Type.SMALLSERIAL,
+            exp.DataType.Type.SMALLMONEY,
+            exp.DataType.Type.STRUCT,
+            exp.DataType.Type.SUPER,
+            exp.DataType.Type.TIMETZ,
+            exp.DataType.Type.TIMESTAMPNTZ,
+            exp.DataType.Type.TIMESTAMPLTZ,
+            exp.DataType.Type.TIMESTAMPTZ,
+            exp.DataType.Type.TIMESTAMP_NS,
+            exp.DataType.Type.TSMULTIRANGE,
+            exp.DataType.Type.TSRANGE,
+            exp.DataType.Type.TSTZMULTIRANGE,
+            exp.DataType.Type.TSTZRANGE,
+            exp.DataType.Type.UINT128,
+            exp.DataType.Type.UINT256,
+            exp.DataType.Type.UNION,
+            exp.DataType.Type.UNKNOWN,
+            exp.DataType.Type.USERDEFINED,
+            exp.DataType.Type.UUID,
+            exp.DataType.Type.VARIANT,
+            exp.DataType.Type.XML,
+            exp.DataType.Type.TDIGEST,
+        }
+
+        TYPE_MAPPING = {
+            **MySQL.Generator.TYPE_MAPPING,
+            exp.DataType.Type.BIGDECIMAL: "DECIMAL",
+            exp.DataType.Type.BIT: "BOOLEAN",
+            exp.DataType.Type.DATE32: "DATE",
+            exp.DataType.Type.DATETIME64: "DATETIME",
+            exp.DataType.Type.DECIMAL32: "DECIMAL",
+            exp.DataType.Type.DECIMAL64: "DECIMAL",
+            exp.DataType.Type.DECIMAL128: "DECIMAL",
+            exp.DataType.Type.DECIMAL256: "DECIMAL",
+            exp.DataType.Type.ENUM8: "ENUM",
+            exp.DataType.Type.ENUM16: "ENUM",
+            exp.DataType.Type.FIXEDSTRING: "TEXT",
+            exp.DataType.Type.GEOMETRY: "GEOGRAPHY",
+            exp.DataType.Type.POINT: "GEOGRAPHYPOINT",
+            exp.DataType.Type.RING: "GEOGRAPHY",
+            exp.DataType.Type.LINESTRING: "GEOGRAPHY",
+            exp.DataType.Type.POLYGON: "GEOGRAPHY",
+            exp.DataType.Type.MULTIPOLYGON: "GEOGRAPHY",
+            exp.DataType.Type.JSONB: "BSON",
+            exp.DataType.Type.TIMESTAMP: "TIMESTAMP",
+            exp.DataType.Type.TIMESTAMP_S: "TIMESTAMP",
+            exp.DataType.Type.TIMESTAMP_MS: "TIMESTAMP(6)",
+        }
+
         # https://docs.singlestore.com/cloud/reference/sql-reference/restricted-keywords/list-of-restricted-keywords/
         RESERVED_KEYWORDS = {
             "abs",
@@ -1452,3 +1577,32 @@ class SingleStore(MySQL):
                 expression.expression,
                 self.func("TO_JSON", expression.this),
             )
+
+        @unsupported_args("kind", "nested", "values")
+        def datatype_sql(self, expression: exp.DataType) -> str:
+            if expression.is_type(exp.DataType.Type.VARBINARY) and not expression.expressions:
+                # `VARBINARY` must always have a size - if it doesn't, we always generate `BLOB`
+                return "BLOB"
+            if expression.is_type(
+                exp.DataType.Type.DECIMAL32,
+                exp.DataType.Type.DECIMAL64,
+                exp.DataType.Type.DECIMAL128,
+                exp.DataType.Type.DECIMAL256,
+            ):
+                scale = self.expressions(expression, flat=True)
+
+                if expression.is_type(exp.DataType.Type.DECIMAL32):
+                    precision = "9"
+                elif expression.is_type(exp.DataType.Type.DECIMAL64):
+                    precision = "18"
+                elif expression.is_type(exp.DataType.Type.DECIMAL128):
+                    precision = "38"
+                else:
+                    # 65 is a maximum precision supported in SingleStore
+                    precision = "65"
+                if scale is not None:
+                    return f"DECIMAL({precision}, {scale[0]})"
+                else:
+                    return f"DECIMAL({precision})"
+
+            return super().datatype_sql(expression)

{sqlglot-27.10.0 → sqlglot-27.11.0}/sqlglot/dialects/snowflake.py

@@ -552,6 +552,7 @@ class Snowflake(Dialect):
 
         ID_VAR_TOKENS = {
             *parser.Parser.ID_VAR_TOKENS,
+            TokenType.EXCEPT,
             TokenType.MATCH_CONDITION,
         }
 

{sqlglot-27.10.0 → sqlglot-27.11.0}/sqlglot/dialects/spark2.py

@@ -187,6 +187,7 @@ class Spark2(Hive):
             "DAYOFYEAR": lambda args: exp.DayOfYear(this=exp.TsOrDsToDate(this=seq_get(args, 0))),
             "DOUBLE": _build_as_cast("double"),
             "FLOAT": _build_as_cast("float"),
+            "FORMAT_STRING": exp.Format.from_arg_list,
             "FROM_UTC_TIMESTAMP": lambda args, dialect: exp.AtTimeZone(
                 this=exp.cast(
                     seq_get(args, 0) or exp.Var(this=""),
@@ -292,6 +293,7 @@ class Spark2(Hive):
             # (DAY_OF_WEEK(datetime) % 7) + 1 is equivalent to DAYOFWEEK_ISO(datetime)
             exp.DayOfWeekIso: lambda self, e: f"(({self.func('DAYOFWEEK', e.this)} % 7) + 1)",
             exp.DayOfYear: rename_func("DAYOFYEAR"),
+            exp.Format: rename_func("FORMAT_STRING"),
             exp.From: transforms.preprocess([_unalias_pivot]),
             exp.FromTimeZone: lambda self, e: self.func(
                 "TO_UTC_TIMESTAMP", e.this, e.args.get("zone")

{sqlglot-27.10.0 → sqlglot-27.11.0}/sqlglot/dialects/sqlite.py

@@ -156,6 +156,7 @@ class SQLite(Dialect):
         EXCEPT_INTERSECT_SUPPORT_ALL_CLAUSE = False
         SUPPORTS_MEDIAN = False
         JSON_KEY_VALUE_PAIR_SEP = ","
+        PARSE_JSON_NAME: t.Optional[str] = None
 
         SUPPORTED_JSON_PATH_PARTS = {
             exp.JSONPathKey,