snowflake-data-validation 1.45.1__tar.gz → 1.45.3__tar.gz

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: snowflake-data-validation
-Version: 1.45.1
+Version: 1.45.3
 Summary: Snowflake Data Validation
 Project-URL: Bug Tracker, https://github.com/snowflake-eng/migrations-data-validation/issues
 Project-URL: Source code, https://github.com/snowflake-eng/migrations-data-validation/
@@ -79,7 +79,7 @@ Requires-Dist: psycopg[binary]>=3.2.0; extra == 'postgresql'
 Provides-Extra: redshift
 Requires-Dist: redshift-connector>=2.1.14; extra == 'redshift'
 Provides-Extra: scai
-Requires-Dist: snowflake-code-unit-registry>=0.9.2; extra == 'scai'
+Requires-Dist: snowflake-code-unit-registry>=0.9.5; extra == 'scai'
 Provides-Extra: sqlite
 Requires-Dist: sqlean-py>=3.45.0; extra == 'sqlite'
 Provides-Extra: sqlserver
@@ -156,12 +156,16 @@ Compares table structure between source and target:
 
 Some differences are reported as `WARNING` and do not fail L1:
 - Snowflake still holds every source value: a different column order, a Snowflake precision or scale that can hold the source's, and a `NOT NULL` source column whose Snowflake column is nullable (a Snowflake virtual column is always nullable).
-- Snowflake may truncate or reject source values, so review it: a source character length longer than Snowflake's.
+- Snowflake may truncate or reject source values, so review it: a source character length longer than Snowflake's, and a source number with no declared precision or scale (Oracle's bare `NUMBER`) on a Snowflake `NUMBER(p,s)`, such as `NUMBER(38,18)`. The warning names the precision and scale Snowflake fixes.
 
 A nullable source column on a `NOT NULL` Snowflake column still fails, since Snowflake can reject the source's NULLs.
 
+An integer column mapped to a `NUMBER` with scale 0 is the exception to the precision warning: SQL Server `INT` (precision 10) in `NUMBER(38,0)` holds every value, so its `NUMERIC_PRECISION` row is `SUCCESS`.
+
 A PostgreSQL user-defined `ENUM` column is validated as `ENUM` against a Snowflake `VARCHAR`. `information_schema` reports every enum as `USER-DEFINED`, so the metadata queries read `pg_type.typtype` to tell enums apart, and report the longest label as the column's length. L2 and L3 compare the label text.
 
+Snowflake database, schema and table names may contain a backslash or an apostrophe, for example `NAFMAIN\TUSDB102`. Snowflake reads backslash escapes inside a string literal, so the Snowflake metadata queries (table metadata, column metadata, case-sensitive columns and object-type lookups) write each name through `escape_snowflake_sql_literal` / `quote_snowflake_sql_literal` in `shared/queries/sql_literals.py`, which double backslashes as well as single quotes. Use these helpers, not `escape_sql_literal`, for any name embedded in a Snowflake string literal.
+
 ### Metrics Validation
 Compares statistical metrics for each column:
 - Row count
@@ -226,6 +230,25 @@ For configuration options and examples, see the [official documentation](https:/
 
 ## Changelog
 
+### v1.45.3
+
+**Bug fixes**
+
+- Fixed schema validation reporting SQL Server `INT` columns mapped to `NUMBER(38,0)` as precision differences.
+- Surfaced rounding of scale-less `NUMBER` values as a lossy mapping instead of a hidden match.
+- Fixed schema validation reporting every column as missing when the schema name contains a backslash (for example `DOMAIN\user` schemas).
+- Fixed Teradata `VARCHAR` distinct and empty-value metrics not matching when values have trailing spaces.
+- Fixed Iceberg timestamp comparisons to use `FF6` or `FF9` fractional precision based on the catalog format version.
+- Fixed metrics validation for signed-year `TIMESTAMP` values, including BC and post-9999 dates.
+
+### v1.45.2
+
+**Bug fixes**
+
+- Fixed hybrid row validation reporting false mismatches for SQL Server `DATETIMEOFFSET` columns landed as `VARCHAR` in Iceberg v3 compatible tables.
+- Fixed schema validation failing an Oracle unconstrained `NUMBER` against a Snowflake `NUMBER(38,18)`; it now reports a warning.
+- Fixed Snowflake target connections not resolving from `connections.toml`.
+
 ### v1.45.1
 
 **Improvements**

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/README.md

@@ -65,12 +65,16 @@ Compares table structure between source and target:
 
 Some differences are reported as `WARNING` and do not fail L1:
 - Snowflake still holds every source value: a different column order, a Snowflake precision or scale that can hold the source's, and a `NOT NULL` source column whose Snowflake column is nullable (a Snowflake virtual column is always nullable).
-- Snowflake may truncate or reject source values, so review it: a source character length longer than Snowflake's.
+- Snowflake may truncate or reject source values, so review it: a source character length longer than Snowflake's, and a source number with no declared precision or scale (Oracle's bare `NUMBER`) on a Snowflake `NUMBER(p,s)`, such as `NUMBER(38,18)`. The warning names the precision and scale Snowflake fixes.
 
 A nullable source column on a `NOT NULL` Snowflake column still fails, since Snowflake can reject the source's NULLs.
 
+An integer column mapped to a `NUMBER` with scale 0 is the exception to the precision warning: SQL Server `INT` (precision 10) in `NUMBER(38,0)` holds every value, so its `NUMERIC_PRECISION` row is `SUCCESS`.
+
 A PostgreSQL user-defined `ENUM` column is validated as `ENUM` against a Snowflake `VARCHAR`. `information_schema` reports every enum as `USER-DEFINED`, so the metadata queries read `pg_type.typtype` to tell enums apart, and report the longest label as the column's length. L2 and L3 compare the label text.
 
+Snowflake database, schema and table names may contain a backslash or an apostrophe, for example `NAFMAIN\TUSDB102`. Snowflake reads backslash escapes inside a string literal, so the Snowflake metadata queries (table metadata, column metadata, case-sensitive columns and object-type lookups) write each name through `escape_snowflake_sql_literal` / `quote_snowflake_sql_literal` in `shared/queries/sql_literals.py`, which double backslashes as well as single quotes. Use these helpers, not `escape_sql_literal`, for any name embedded in a Snowflake string literal.
+
 ### Metrics Validation
 Compares statistical metrics for each column:
 - Row count
@@ -135,6 +139,25 @@ For configuration options and examples, see the [official documentation](https:/
 
 ## Changelog
 
+### v1.45.3
+
+**Bug fixes**
+
+- Fixed schema validation reporting SQL Server `INT` columns mapped to `NUMBER(38,0)` as precision differences.
+- Surfaced rounding of scale-less `NUMBER` values as a lossy mapping instead of a hidden match.
+- Fixed schema validation reporting every column as missing when the schema name contains a backslash (for example `DOMAIN\user` schemas).
+- Fixed Teradata `VARCHAR` distinct and empty-value metrics not matching when values have trailing spaces.
+- Fixed Iceberg timestamp comparisons to use `FF6` or `FF9` fractional precision based on the catalog format version.
+- Fixed metrics validation for signed-year `TIMESTAMP` values, including BC and post-9999 dates.
+
+### v1.45.2
+
+**Bug fixes**
+
+- Fixed hybrid row validation reporting false mismatches for SQL Server `DATETIMEOFFSET` columns landed as `VARCHAR` in Iceberg v3 compatible tables.
+- Fixed schema validation failing an Oracle unconstrained `NUMBER` against a Snowflake `NUMBER(38,18)`; it now reports a warning.
+- Fixed Snowflake target connections not resolving from `connections.toml`.
+
 ### v1.45.1
 
 **Improvements**

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/pyproject.toml

@@ -4,6 +4,7 @@ requires = ["hatchling"]
 
 [tool.uv]
 constraint-dependencies = [
+  "certifi>=2026.10.23,<2026.11",
   "virtualenv>=21.7.13,<21.8",
 ]
 
@@ -105,7 +106,7 @@ db2 = [
 # musllinux build, and SDV is published to public PyPI. Inside DMO/DEA the
 # dependency is already declared, so SDV's logger tree is covered there.
 scai = [
-  "snowflake-code-unit-registry>=0.9.2",
+  "snowflake-code-unit-registry>=0.9.5",
 ]
 
 [project.urls]

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/bigquery/query/query_generator_bigquery.py

@@ -17,7 +17,9 @@ from __future__ import annotations
 
 import logging
 
-from shared.datatypes.bigquery import BigQueryType, serializes_exactly_through_to_json
+from typing import TYPE_CHECKING
+
+from shared.datatypes.bigquery import BigQueryType, field_wise_json_text, serializes_exactly_through_to_json
 from shared.datatypes.common import CommonType
 from shared.datatypes.snowflake import SnowflakeType
 from snowflake.snowflake_data_validation.query.decimal_text_normalization import (
@@ -51,6 +53,10 @@ from snowflake.snowflake_data_validation.validation.comparison_codec import (
 )
 
 
+if TYPE_CHECKING:  # pragma: no cover - used only for type hints.
+    from snowflake.snowflake_data_validation.utils.model.column_metadata import ColumnMetadata
+
+
 LOGGER = logging.getLogger(__name__)
 
 _WIDE_NUMERIC_BASE_TYPES = frozenset({BigQueryType.BIGNUMERIC, BigQueryType.BIGDECIMAL})
@@ -264,32 +270,63 @@ class QueryGeneratorBigQuery(QueryGeneratorBase):
                 raise ValueError(f"No normalization template found for data type {column.data_type} and no DEFAULT template available.")
         return normalization_template
 
+    def _field_wise_normalization(self, column: ColumnMetadata) -> str | None:
+        """Return the L3 text of an ARRAY / STRUCT mixing INT64 and decimals, else None.
+
+        Data migration writes such a column's JSON text field by field, so this hashes that
+        text, keys ordered by PARSE_JSON as for other nests. The text spells every declared
+        field name, and a name may hold a template placeholder, so it is written with the
+        column already in place and never rendered as a template.
+        """
+        column_ref = f"`{self._normalization_column_name(column)}`"
+        field_wise = field_wise_json_text(column.data_type, column_ref)
+        if field_wise is None:
+            return None
+        return f"NULLIF(TO_JSON_STRING(PARSE_JSON({field_wise})), 'null')"
+
+    def _column_normalization(
+        self,
+        column: ColumnMetadata,
+        table_context: TableContext,
+        column_normalization_map: dict[str, str] | None,
+    ) -> str:
+        """Return the column's rendered L3 normalization.
+
+        A user's normalization, a per-column rule or an override of the column's base type
+        (``STRUCT``, ``ARRAY``), wins over the built-in field-wise text.
+        """
+        field_wise = self._field_wise_normalization(column)
+        if field_wise is not None and not has_custom_column_normalization(
+            column.name,
+            self.normalize_bigquery_type(column.data_type),
+            column_normalization_map,
+            table_context.templates_loader_manager,
+        ):
+            return field_wise
+        normalization_template = self._resolve_datatype_normalization_template(
+            column,
+            table_context.templates_loader_manager.datatypes_normalization_templates,
+            table_context,
+        )
+        normalization_template = resolve_column_normalization_template(
+            column.name,
+            normalization_template,
+            column_normalization_map,
+        )
+        # Resolve ``{{ col_name }}`` and, for numeric types, the scale-aware
+        # ``{{ numeric_format_mask }}`` (BigQuery emits a printf ``%.Nf``).
+        return self._render_normalization_template(normalization_template, column)
+
     def _build_datatypes_normalization_renderer_templates(
         self,
         table_context: TableContext,
         column_codec_map: dict[str, ComparisonCodec] | None = None,
         column_normalization_map: dict[str, str] | None = None,
     ) -> dict[str, str]:
-        datatypes_normalization_templates = table_context.templates_loader_manager.datatypes_normalization_templates
         renderer_columns = self._row_md5_normalization_columns(table_context)
         renderer_templates: dict[str, str] = {}
         for column in renderer_columns:
-            normalization_template = self._resolve_datatype_normalization_template(
-                column,
-                datatypes_normalization_templates,
-                table_context,
-            )
-            normalization_template = resolve_column_normalization_template(
-                column.name,
-                normalization_template,
-                column_normalization_map,
-            )
-            # Resolve ``{{ col_name }}`` and, for numeric types, the scale-aware
-            # ``{{ numeric_format_mask }}`` (BigQuery emits a printf ``%.Nf``).
-            renderer_templates[column.name] = self._render_normalization_template(
-                normalization_template,
-                column,
-            )
+            renderer_templates[column.name] = self._column_normalization(column, table_context, column_normalization_map)
         renderer_templates = apply_text_codec_to_renderer_templates(
             self.platform,
             renderer_templates,

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/extractor/sql_queries_template_generator.py

@@ -26,7 +26,7 @@ import snowflake.snowflake_data_validation as sdv_pkg
 
 from shared.identifiers.managers import identifier_manager
 from shared.identifiers.sql_identifier import SqlIdentifier
-from shared.queries.sql_literals import escape_sql_literal
+from shared.queries.sql_literals import escape_snowflake_sql_literal, escape_sql_literal
 from shared.query_modifiers.strict import check_query_modifiers_strict
 from shared.query_modifiers.template_context import QueryModifiersTemplateContext
 from snowflake.snowflake_data_validation.utils.constants import (
@@ -132,6 +132,7 @@ class SQLQueriesTemplateGenerator:
         # Identifiers are written in Python (write_sql_name, column_sql_names)
         # and handed to the templates already written; only literals are escaped here.
         self.env.filters["escape_sql_literal"] = escape_sql_literal
+        self.env.filters["escape_snowflake_sql_literal"] = escape_snowflake_sql_literal
         # Single source of truth for the NULL sentinel used by L3 row-MD5 and
         # cell-validation normalization. Exposed to every template so the
         # COALESCE fallback can keep a real SQL NULL distinct from "".
@@ -351,6 +352,39 @@ class SQLQueriesTemplateGenerator:
         except Exception as e:
             raise Exception(f"Error generating template: {e}") from e
 
+    def get_iceberg_format_version(self, database_name: str, schema_name: str, table_name: str, platform: str) -> str:
+        """Generate the query that reads one target table's Iceberg format version.
+
+        Args:
+            database_name (str): Database that contains the table.
+            schema_name (str): Schema that contains the table.
+            table_name (str): Table whose ``ICEBERG_VERSION`` parameter is read.
+            platform (str): Platform identifier (``Platform.SNOWFLAKE.value``).
+
+        Returns:
+            str: The SQL query. The ``value`` column is ``ICEBERG_FORMAT_VERSION_V2``
+            or ``ICEBERG_FORMAT_VERSION_V3``.
+
+        Raises:
+            ValueError: If the template file is not found.
+            Exception: If an error occurs during template rendering.
+
+        """
+        template_file = f"{platform}_get_iceberg_format_version.sql.j2"
+        try:
+            return self._render(
+                template_file,
+                database_sql=write_sql_name(database_name, platform),
+                schema_sql=write_sql_name(schema_name, platform),
+                table_sql=write_sql_name(table_name, platform),
+            )
+        except jinja2.exceptions.TemplateNotFound:
+            raise ValueError(
+                f"Template not found for platform: {platform}. Please create {template_file} in the {self.template_dir} directory."
+            ) from None
+        except Exception as e:
+            raise Exception(f"Error generating template: {e}") from e
+
     def generate_compute_md5_query(
         self,
         platform: str,

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/oracle/query/query_generator_oracle.py

@@ -77,6 +77,7 @@ from snowflake.snowflake_data_validation.utils.constants import (
     ROW_HASH_COLUMN_DELIMITER,
     ROW_HASH_COLUMN_DELIMITER_KEY,
     Platform,
+    renders_full_precision,
 )
 from snowflake.snowflake_data_validation.utils.iso_dv_common import (
     get_paired_target_column_types,
@@ -156,6 +157,28 @@ def build_oracle_numeric_canonical_expr(col_ref: str, format_mask: str) -> str:
     )
 
 
+def build_oracle_full_precision_numeric_expr(col_ref: str, format_mask: str) -> str:
+    """Render a value that fits the mask's scale through the mask, any other at its stored precision.
+
+    For a bare NUMBER the target scale was never checked against the data. A value
+    the target could hold renders exactly like :func:`build_oracle_numeric_canonical_expr`,
+    the text Snowflake prints. A value with more fractional digits renders with
+    ``TM9`` (shortest exact text, scientific only past 64 characters, which no
+    NUMBER(38, s) value reaches), so the target's rounding shows as a cell diff.
+    """
+    scale = _oracle_scale_from_fm_mask(format_mask)
+    full_text = f"TRIM(REGEXP_REPLACE(TRIM(REGEXP_REPLACE(TRIM(TO_CHAR({col_ref}, 'TM9')), '^-\\.', '-0.')), '^\\.', '0.'))"
+    return (
+        f"CASE WHEN {col_ref} IS NULL THEN NULL "
+        f"WHEN {col_ref} = ROUND({col_ref}, {scale}) THEN {build_oracle_numeric_canonical_expr(col_ref, format_mask)} "
+        f"ELSE {full_text} END"
+    )
+
+
+def _renders_full_precision(column: object) -> bool:
+    return renders_full_precision(getattr(column, "properties", None))
+
+
 class QueryGeneratorOracle(QueryGeneratorBase):
     """Oracle-specific implementation of the query generator.
 
@@ -250,7 +273,10 @@ class QueryGeneratorOracle(QueryGeneratorBase):
         """Wrap numeric ``TO_CHAR`` output to restore the FM-suppressed leading zero."""
         if NUMERIC_FORMAT_MASK_PLACEHOLDER in template and "TO_CHAR" in template and self._is_numeric_normalization_type(column.data_type):
             mask = self._build_numeric_format_mask(self._resolve_numeric_scale(column))
-            return build_oracle_numeric_canonical_expr(self._write_sql_identifier(column.name), mask)
+            col_ref = self._write_sql_identifier(column.name)
+            if _renders_full_precision(column):
+                return build_oracle_full_precision_numeric_expr(col_ref, mask)
+            return build_oracle_numeric_canonical_expr(col_ref, mask)
         return super().render_normalization_template(template, column)
 
     def build_float_bit_shim(self, column, col_ref: str) -> str:

snowflake_data_validation-1.45.3/src/snowflake/snowflake_data_validation/query/normalization_render_context.py

@@ -0,0 +1,51 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, replace
+from typing import TYPE_CHECKING
+
+from snowflake.snowflake_data_validation.utils.constants import Platform
+from snowflake.snowflake_data_validation.utils.iceberg_dv_common import IcebergProfile
+
+
+if TYPE_CHECKING:
+    from snowflake.snowflake_data_validation.utils.model.table_context import TableContext
+
+
+@dataclass(frozen=True)
+class NormalizationRenderContext:
+    """Per-render inputs a normalization template needs beyond the column itself.
+
+    Resolve the per-table inputs once with ``from_table_context``, then specialize
+    per column with ``for_column``. A new render input becomes a field here instead
+    of another parameter on ``render_normalization_template``.
+    """
+
+    paired_source_platform: Platform | None = None
+    paired_source_type: str | None = None
+    iceberg_fraction_digits: int = IcebergProfile().fraction_digits
+    has_custom_normalization: bool = False
+
+    @classmethod
+    def from_table_context(cls, table_context: TableContext) -> NormalizationRenderContext:
+        """Resolve the per-table inputs; ``for_column`` adds the per-column paired type."""
+        return cls(
+            paired_source_platform=table_context.paired_source_platform,
+            iceberg_fraction_digits=IcebergProfile.from_table_context(table_context).fraction_digits,
+        )
+
+    def for_column(
+        self,
+        paired_source_type: str | None,
+        *,
+        has_custom_normalization: bool = False,
+    ) -> NormalizationRenderContext:
+        """Return a copy specialized for one column; the base context is left unchanged.
+
+        Sets the column's paired source type and whether its template is a user
+        override. ``has_custom_normalization`` resets to False unless passed.
+        """
+        return replace(
+            self,
+            paired_source_type=paired_source_type,
+            has_custom_normalization=has_custom_normalization,
+        )

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/query/query_generator_base.py

@@ -58,6 +58,7 @@ from snowflake.snowflake_data_validation.utils.constants import (
     DEFAULT_MAX_COLUMNS_PER_METRICS_BATCH,
     EXPLICIT_METRIC_NORMALIZATION_COLUMN_KEY,
     FLOAT_BIT_SHIM_PLACEHOLDER,
+    ICEBERG_FRACTION_DIGITS_PLACEHOLDER,
     MEASURE_COLLATION_PREFIX_PLACEHOLDER,
     MEASURE_COLLATION_SUFFIX_PLACEHOLDER,
     METRIC_COLUMN_KEY,
@@ -892,6 +893,7 @@ class QueryGeneratorBase(ABC):
         # set the attribute keep the default instead of a truthy Mock.
         include_count_distinct = getattr(table_context, "include_count_distinct", False) is True
         apply_metric_column_modifier = table_context.apply_metric_column_modifier
+        iceberg_fraction_digits = IcebergProfile.from_table_context(table_context).fraction_digits
 
         select_exprs: list[str] = []
         cells: list[tuple[str, str]] = []
@@ -953,6 +955,10 @@ class QueryGeneratorBase(ABC):
                         if NUMERIC_FORMAT_MASK_PLACEHOLDER in normalization:
                             mask = self._build_numeric_format_mask(self._resolve_numeric_scale(col))
                             normalization = normalization.replace(NUMERIC_FORMAT_MASK_PLACEHOLDER, mask)
+                        if ICEBERG_FRACTION_DIGITS_PLACEHOLDER in normalization:
+                            normalization = normalization.replace(
+                                ICEBERG_FRACTION_DIGITS_PLACEHOLDER, str(iceberg_fraction_digits)
+                            )
                         metric_expr = self._wrap_metric_with_normalization(normalization, metric_expr)
                 alias = f"c{measured_idx}_m{metric_idx}"
                 select_exprs.append(f"{metric_expr} AS {alias}")

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/redshift/extractor/templates/redshift_column_metrics_templates.yaml

@@ -619,6 +619,22 @@ TIMESTAMP WITHOUT TIME ZONE:
      metric_query: SUM(CASE WHEN "{{ col_name }}" IS NULL THEN 1 ELSE 0 END)
      metric_return_datatype: BIGINT
 
+# Iceberg VARCHAR target (SNOW-4353618): MIN/MAX of the instant, rendered as the
+# signed astronomical-year text the UNLOAD footer writes (``TO_CHAR 'YYYY'``
+# drops the era). Kept in sync with DMO by test_redshift_consistency.py.
+TIMESTAMP_AS_TEXT:
+  min:
+    metric_query: MIN("{{ col_name }}")
+    metric_return_datatype: VARCHAR
+    metric_normalization: '(CASE WHEN (CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END) < 0 THEN ''-'' ELSE '''' END || (CASE WHEN ABS((CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END)) < 1000 THEN LPAD(ABS((CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END))::VARCHAR, 4, ''0'') ELSE ABS((CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END))::VARCHAR END)) || TO_CHAR({{ metric_query }}, ''-MM-DD"T"HH24:MI:SS.US'') || ''000'''
+  max:
+    metric_query: MAX("{{ col_name }}")
+    metric_return_datatype: VARCHAR
+    metric_normalization: '(CASE WHEN (CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END) < 0 THEN ''-'' ELSE '''' END || (CASE WHEN ABS((CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END)) < 1000 THEN LPAD(ABS((CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END))::VARCHAR, 4, ''0'') ELSE ABS((CASE WHEN EXTRACT(YEAR FROM {{ metric_query }})::INT < 0 THEN EXTRACT(YEAR FROM {{ metric_query }})::INT + 1 ELSE EXTRACT(YEAR FROM {{ metric_query }})::INT END))::VARCHAR END)) || TO_CHAR({{ metric_query }}, ''-MM-DD"T"HH24:MI:SS.US'') || ''000'''
+  count_null:
+    metric_query: SUM(CASE WHEN "{{ col_name }}" IS NULL THEN 1 ELSE 0 END)
+    metric_return_datatype: BIGINT
+
 TIMESTAMPTZ:
   min: 
     metric_query: MIN("{{ col_name }}")

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/redshift/query/query_generator_redshift.py

@@ -62,6 +62,7 @@ from snowflake.snowflake_data_validation.utils.constants import (
     SchemaValidationFields,
 )
 from snowflake.snowflake_data_validation.utils.iceberg_dv_common import (
+    TIMESTAMP_AS_TEXT_KEY,
     IcebergProfile,
     resolve_iceberg_v2_compatible_source_normalization_key,
 )
@@ -71,6 +72,7 @@ from snowflake.snowflake_data_validation.utils.interval_common import (
 )
 from snowflake.snowflake_data_validation.utils.iso_dv_common import (
     get_paired_target_column_types,
+    is_snowflake_varchar_like_type,
 )
 from snowflake.snowflake_data_validation.utils.measure_collation import (
     COUNT_DISTINCT_METRIC,
@@ -83,6 +85,7 @@ from snowflake.snowflake_data_validation.utils.model.table_context import TableC
 
 LOGGER = logging.getLogger(__name__)
 _REDSHIFT_NORMALIZED_COLLATION_TOKENS = ("COLLATE(", ", 'case_sensitive')")
+_TIMESTAMP_BASE_TYPES = frozenset({CommonType.TIMESTAMP, CommonType.TIMESTAMP_WITHOUT_TIME_ZONE})
 
 # Redshift ``TO_CHAR`` formats numbers with 64-bit arithmetic. A numeric format
 # mask with N total digit placeholders (integer nines + fractional zeros) admits
@@ -752,6 +755,21 @@ class QueryGeneratorRedshift(QueryGeneratorBase):
             raise ValueError(f"No normalization template found for data type {data_type} and no DEFAULT template available.")
         return normalization_template
 
+    def _resolve_metrics_col_type(self, col, table_context: TableContext) -> str:
+        """Key an Iceberg VARCHAR TIMESTAMP to the signed-year metrics (SNOW-4353618).
+
+        ``TO_CHAR 'YYYY'`` drops the era, so MIN / MAX would not match the text
+        the UNLOAD staged. Native targets keep the shared TIMESTAMP template.
+        """
+        col_type = super()._resolve_metrics_col_type(col, table_context)
+        if not IcebergProfile.from_table_context(table_context).is_active:
+            return col_type
+        if col_type.upper() not in _TIMESTAMP_BASE_TYPES:
+            return col_type
+        if not is_snowflake_varchar_like_type(get_paired_target_column_types(table_context).get(col.name)):
+            return col_type
+        return TIMESTAMP_AS_TEXT_KEY
+
     def _resolve_normalization_templates(
         self,
         table_context: TableContext,

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/snowflake/connector/connector_snowflake.py

@@ -181,7 +181,13 @@ class ConnectorSnowflake(BaseConnector):
             raise ConnectionError(f"Failed to establish Snowflake connection: {e}") from e
 
     def _load_named_connection(self, connection_name: str) -> dict:
-        """Load connection parameters from config.toml or connections.toml file.
+        """Load connection parameters from connections.toml or config.toml.
+
+        Directories are searched in order (``~/.snowflake``, then the working
+        directory). Within one directory a ``connections.toml``, when it exists,
+        replaces ``config.toml``'s connections -- the rule
+        ``snowflake.connector`` and the Snow CLI apply -- so this opens the same
+        definition of a name as ``snow sql`` and the test-runner's resolver.
 
         Args:
             connection_name: Name of the connection in the config file
@@ -193,40 +199,39 @@ class ConnectorSnowflake(BaseConnector):
             typer.BadParameter: If connection not found or file doesn't exist
 
         """
-        # Check standard locations for config.toml (new) and connections.toml (legacy)
-        config_paths = [
-            Path.home() / ".snowflake" / "config.toml",
-            Path.home() / ".snowflake" / "connections.toml",
-            Path.cwd() / "config.toml",
-            Path.cwd() / "connections.toml",
-        ]
-
-        for config_path in config_paths:
-            if config_path.exists():
-                try:
-                    config = toml.load(config_path)
-                    # Check for nested structure (config.toml format): connections.name
-                    if "connections" in config and connection_name in config["connections"]:
-                        LOGGER.debug(
-                            "Loaded connection '%s' from %s (nested)",
-                            connection_name,
-                            config_path,
-                        )
-                        return config["connections"][connection_name]
-                    # Check for flat structure (legacy format): name
-                    elif connection_name in config:
-                        LOGGER.debug(
-                            "Loaded connection '%s' from %s (flat)",
-                            connection_name,
-                            config_path,
-                        )
-                        return config[connection_name]
-                except Exception as e:
-                    LOGGER.warning("Failed to load config from %s: %s", config_path, e)
+        config_dirs = [Path.home() / ".snowflake", Path.cwd()]
+        searched: list[Path] = []
+
+        for config_dir in config_dirs:
+            connections_path = config_dir / "connections.toml"
+            config_path = connections_path if connections_path.exists() else config_dir / "config.toml"
+            searched.append(config_path)
+            if not config_path.exists():
+                continue
+            try:
+                config = toml.load(config_path)
+                # Check for nested structure (config.toml format): connections.name
+                if "connections" in config and connection_name in config["connections"]:
+                    LOGGER.debug(
+                        "Loaded connection '%s' from %s (nested)",
+                        connection_name,
+                        config_path,
+                    )
+                    return config["connections"][connection_name]
+                # Check for flat structure (connections.toml format): name
+                elif connection_name in config:
+                    LOGGER.debug(
+                        "Loaded connection '%s' from %s (flat)",
+                        connection_name,
+                        config_path,
+                    )
+                    return config[connection_name]
+            except Exception as e:
+                LOGGER.warning("Failed to load config from %s: %s", config_path, e)
 
         raise typer.BadParameter(
-            f"Connection '{connection_name}' not found in config.toml or connections.toml. "
-            f"Searched locations: {', '.join(str(p) for p in config_paths)}"
+            f"Connection '{connection_name}' not found in connections.toml or config.toml. "
+            f"Searched locations: {', '.join(str(p) for p in searched)}"
         )
 
     def _get_default_connection_params(self) -> dict:

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/snowflake/extractor/templates/snowflake_column_metrics_templates.yaml

@@ -249,6 +249,22 @@ VARCHAR_TIMESTAMP:
     metric_query: SUM(CASE WHEN "{{ col_name }}" IS NULL THEN 1 ELSE 0 END)
     metric_return_datatype: NUMBER
 
+# Signed-year ISO text (Redshift Iceberg, SNOW-4353618): TRY_TO_TIMESTAMP_NTZ
+# returns NULL for negative and 5+ digit years, so parse from parts like the
+# Iceberg column comment's cast back (DMO iceberg_column_comment.py).
+VARCHAR_TIMESTAMP_SIGNED:
+  min:
+    metric_query: 'MIN(TIMESTAMP_NTZ_FROM_PARTS(IFF(TRY_TO_NUMBER(REGEXP_SUBSTR(TO_VARCHAR(TRY_TO_DATE(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''^(-?[0-9]+-[0-9]{2}-[0-9]{2})'', 1, 1, ''e'', 1)), ''YYYY-MM-DD''), ''^(-?[0-9]+)-'', 1, 1, ''e'', 1), 38, 0) = TRY_TO_NUMBER(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''^(-?[0-9]+)-'', 1, 1, ''e'', 1), 38, 0), TRY_TO_DATE(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''^(-?[0-9]+-[0-9]{2}-[0-9]{2})'', 1, 1, ''e'', 1)), NULL), TRY_TO_TIME(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''T([0-9]{2}:[0-9]{2}:[0-9]{2}(\\.[0-9]+)?)'', 1, 1, ''e'', 1))))'
+    metric_return_datatype: VARCHAR
+    metric_normalization: TO_CHAR({{ metric_query }}, 'YYYY-MM-DD"T"HH24:MI:SS.FF9')
+  max:
+    metric_query: 'MAX(TIMESTAMP_NTZ_FROM_PARTS(IFF(TRY_TO_NUMBER(REGEXP_SUBSTR(TO_VARCHAR(TRY_TO_DATE(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''^(-?[0-9]+-[0-9]{2}-[0-9]{2})'', 1, 1, ''e'', 1)), ''YYYY-MM-DD''), ''^(-?[0-9]+)-'', 1, 1, ''e'', 1), 38, 0) = TRY_TO_NUMBER(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''^(-?[0-9]+)-'', 1, 1, ''e'', 1), 38, 0), TRY_TO_DATE(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''^(-?[0-9]+-[0-9]{2}-[0-9]{2})'', 1, 1, ''e'', 1)), NULL), TRY_TO_TIME(REGEXP_SUBSTR(TRIM("{{ col_name }}"), ''T([0-9]{2}:[0-9]{2}:[0-9]{2}(\\.[0-9]+)?)'', 1, 1, ''e'', 1))))'
+    metric_return_datatype: VARCHAR
+    metric_normalization: TO_CHAR({{ metric_query }}, 'YYYY-MM-DD"T"HH24:MI:SS.FF9')
+  count_null:
+    metric_query: SUM(CASE WHEN "{{ col_name }}" IS NULL THEN 1 ELSE 0 END)
+    metric_return_datatype: NUMBER
+
 # Order by UTC time-of-day. A plain MIN of the parsed instant uses today's date
 # and can cross midnight, so the formatted value would keep the wrong offset.
 VARCHAR_TIME_TZ:

{snowflake_data_validation-1.45.1 → snowflake_data_validation-1.45.3}/src/snowflake/snowflake_data_validation/snowflake/extractor/templates/snowflake_datatypes_normalization_templates.yaml

@@ -11,20 +11,22 @@ NUMBER: TO_CHAR("{{ col_name }}", '{{ numeric_format_mask }}')
 # truncates the same way on every engine; a CAST to TIME(p) would round.
 TIME: LEFT(TO_CHAR("{{ col_name }}", 'HH24:MI:SS.FF9'), {{ time_text_length }})
 TIME WITH TIME ZONE: LEFT(TO_CHAR("{{ col_name }}", 'HH24:MI:SS.FF9'), {{ time_text_length }})
-TIMESTAMP_LTZ: TO_CHAR("{{ col_name }}", 'YYYY-MM-DD"T"HH24:MI:SS.FF9')
+# Typed Snowflake timestamps. The fractional-second digits are the Iceberg Parquet
+# format version (v2 micro FF6 / v3 nano FF9), substituted at render from
+# resolve_iceberg_fraction_digits; non-Iceberg paths default to FF9.
+TIMESTAMP_LTZ: TO_CHAR("{{ col_name }}", 'YYYY-MM-DD"T"HH24:MI:SS.FF{{ fraction_digits }}')
 # Paired Oracle TIMESTAMP WITH LOCAL TIME ZONE: compare the retained instant in UTC.
-TIMESTAMP_LTZ_UTC: TO_CHAR(CONVERT_TIMEZONE('UTC', "{{ col_name }}"), 'YYYY-MM-DD"T"HH24:MI:SS.FF9')
-TIMESTAMP_NTZ: TO_CHAR("{{ col_name }}", 'YYYY-MM-DD"T"HH24:MI:SS.FF9')
-TIMESTAMP_TZ: REGEXP_REPLACE(TO_CHAR("{{ col_name }}", 'YYYY-MM-DD"T"HH24:MI:SS.FF9TZH:TZM'), 'Z$', '+00:00')
+TIMESTAMP_LTZ_UTC: TO_CHAR(CONVERT_TIMEZONE('UTC', "{{ col_name }}"), 'YYYY-MM-DD"T"HH24:MI:SS.FF{{ fraction_digits }}')
+TIMESTAMP_NTZ: TO_CHAR("{{ col_name }}", 'YYYY-MM-DD"T"HH24:MI:SS.FF{{ fraction_digits }}')
+TIMESTAMP_TZ: REGEXP_REPLACE(TO_CHAR("{{ col_name }}", 'YYYY-MM-DD"T"HH24:MI:SS.FF{{ fraction_digits }}TZH:TZM'), 'Z$', '+00:00')
 # Paired PostgreSQL timestamptz: compare UTC instants (Oracle/Teradata use TIMESTAMP_TZ).
-TIMESTAMP_TZ_UTC: TO_CHAR(CONVERT_TIMEZONE('UTC', "{{ col_name }}"), 'YYYY-MM-DD"T"HH24:MI:SS.FF9') || '+00:00'
-VARCHAR_ISO_TZ: REGEXP_REPLACE(TO_CHAR(TRY_TO_TIMESTAMP_TZ(TRIM("{{ col_name }}")), 'YYYY-MM-DD"T"HH24:MI:SS.FF9TZH:TZM'), 'Z$', '+00:00')
+TIMESTAMP_TZ_UTC: TO_CHAR(CONVERT_TIMEZONE('UTC', "{{ col_name }}"), 'YYYY-MM-DD"T"HH24:MI:SS.FF{{ fraction_digits }}') || '+00:00'
+VARCHAR_ISO_TZ: REGEXP_REPLACE(TO_CHAR(TRY_TO_TIMESTAMP_TZ(TRIM("{{ col_name }}")), 'YYYY-MM-DD"T"HH24:MI:SS.FF{{ fraction_digits }}TZH:TZM'), 'Z$', '+00:00')
 # Canonical UTC-instant form for every Snowflake TIMESTAMP_LTZ mirror of a
 # UTC-instant TIMESTAMPTZ source (Redshift / PostgreSQL), reached from either
 # the native path (iceberg_spec and storage_mode both None, target
 # TIMESTAMP_LTZ / TIMESTAMP_LTZ(6) / TIMESTAMP_LTZ(9)) or the Iceberg v2
-# compatible overlay (TIMESTAMP_LTZ(6)). Source side emits the same
-# FF9-padded +00:00 string.
+# compatible overlay (TIMESTAMP_LTZ(6)). Always FF9-padded +00:00 on both sides.
 TIMESTAMP_LTZ_UTC_FF9_OFFSET: TO_CHAR(CONVERT_TIMEZONE('UTC', "{{ col_name }}"), 'YYYY-MM-DD"T"HH24:MI:SS.FF9') || '+00:00'
 # Complex values flattened to VARCHAR under Iceberg compatible DDL retain the
 # exact Redshift JSON_SERIALIZE text staged by UNLOAD. Compare that migration