sql-testing-library 0.21.0__tar.gz → 0.22.1__tar.gz

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/CHANGELOG.md

@@ -5,6 +5,23 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 0.22.1 (2026-03-02)
+
+### Fix
+
+- skip CTE alias check for qualified table names and make default_namespace optional (#143)
+- remove worker log suffix and handle leading SQL comments before WITH clause (#142)
+
+## 0.22.0 (2026-01-05)
+
+### Feat
+
+- add Snowflake support for deeply nested complex types via OBJECT
+
+### Fix
+
+- update Snowflake unit test to use JSON format
+
 ## 0.21.0 (2026-01-03)
 
 ### Feat

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sql-testing-library
-Version: 0.21.0
+Version: 0.22.1
 Summary: SQL Testing Framework for Python: Unit test SQL queries with mock data injection for BigQuery, Snowflake, Redshift, Athena, Trino, and DuckDB. Simplify data engineering ETL testing and analytics validation.
 License: MIT
 License-File: LICENSE
@@ -145,11 +145,11 @@ The library supports different data types across database engines. All checkmark
 | **Decimal Array** | `List[Decimal]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | **Optional Array** | `Optional[List[T]]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | **Map/Dict** | `Dict[K, V]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| **Struct/Record** | `dataclass` | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
-| **Nested Arrays** | `List[List[T]]` | ❌ | ✅ | ✅ | ✅ | 🚧 | ✅ |
-| **Arrays of Structs** | `List[dataclass]` | ✅ | ✅ | ✅ | ✅ | 🚧 | ✅ |
-| **3D Arrays** | `List[List[List[T]]]` | ❌ | ✅ | ✅ | ✅ | 🚧 | ✅ |
-| **Arrays of Arrays of Structs** | `List[List[dataclass]]` | ❌ | ✅ | ✅ | ✅ | 🚧 | ✅ |
+| **Struct/Record** | `dataclass` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Nested Arrays** | `List[List[T]]` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Arrays of Structs** | `List[dataclass]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **3D Arrays** | `List[List[List[T]]]` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Arrays of Arrays of Structs** | `List[List[dataclass]]` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
 
 ### Database-Specific Notes
 
@@ -157,7 +157,7 @@ The library supports different data types across database engines. All checkmark
 - **Athena**: 256KB query size limit; supports arrays and maps using `ARRAY[]` and `MAP(ARRAY[], ARRAY[])` syntax; supports struct types using `ROW` with named fields (dataclasses and Pydantic models); **full support for deeply nested types** including nested arrays, arrays of structs, and 3D arrays
 - **Redshift**: Arrays and maps implemented via SUPER type (JSON parsing); 16MB query size limit; **full support for deeply nested types** including struct types using SUPER + JSON_PARSE, nested arrays (2D, 3D+), arrays of structs, and arrays of arrays of structs
 - **Trino**: Memory catalog for testing; excellent decimal precision; supports arrays, maps, and struct types using `ROW` with named fields (dataclasses and Pydantic models); **full support for deeply nested types** including nested arrays, arrays of structs, and 3D arrays
-- **Snowflake**: Column names normalized to lowercase; 1MB query size limit; dict/map types implemented via VARIANT type (JSON parsing); struct types not yet supported (🚧 TODO); nested arrays not yet supported (🚧 TODO)
+- **Snowflake**: Column names normalized to lowercase; 1MB query size limit; dict/map types implemented via VARIANT type (JSON parsing); **full support for deeply nested types** including struct types using OBJECT + PARSE_JSON, nested arrays (2D, 3D+), arrays of structs, and arrays of arrays of structs
 - **DuckDB**: Fast embedded analytics database; excellent SQL standards compliance; supports arrays, maps, and struct types using `STRUCT` syntax with named fields (dataclasses and Pydantic models); **full support for deeply nested types** including nested arrays (2D, 3D+), arrays of structs, and arrays of arrays of structs
 
 ## Execution Modes Support
@@ -1434,25 +1434,23 @@ The library has a few known limitations that are planned to be addressed in futu
 
 ### Deeply Nested Complex Types Support
 
-**✅ Fully Supported (Athena, Trino, DuckDB & Redshift):**
+**✅ Fully Supported (All Major Adapters):**
 - Nested arrays (2D, 3D+): `List[List[int]]`, `List[List[List[int]]]`
 - Arrays of structs: `List[Address]` where Address is a dataclass
 - Arrays of arrays of structs: `List[List[OrderItem]]`
 - Maps with complex values: `Dict[str, str]`, `Dict[str, int]`
 - See `tests/integration/test_deeply_nested_types_integration.py` for comprehensive examples
-- **16 tests passing** across Athena, Trino, DuckDB, and Redshift (both CTE and physical tables modes)
+- **20 tests passing** across Athena, Trino, DuckDB, Redshift, and Snowflake (both CTE and physical tables modes)
 
 **Implementation Details:**
 - **Athena/Trino**: Use ROW types with named fields
 - **DuckDB**: Use STRUCT types with dictionary-style syntax
-- **Redshift**: Use SUPER type with JSON_PARSE for serialization, recursive JSON parsing for deserialization
-
-**🚧 TODO - Implementation Needed:**
-- **BigQuery**: Does not support nested arrays - this is a **database limitation**, not a library limitation. BigQuery's type system doesn't allow `ARRAY<ARRAY<T>>` constructs
-- **Snowflake**:
-  - Struct type support not implemented (see `Struct type not yet supported for dialect: snowflake`)
-  - Nested arrays not yet supported
-  - TODO: Implement struct/nested type support using VARIANT/OBJECT types
+- **Redshift**: Use SUPER type with JSON_PARSE, recursive JSON parsing
+- **Snowflake**: Use OBJECT type with PARSE_JSON, recursive JSON parsing
+- **Shared Helper**: `_parse_json_if_string()` in BaseTypeConverter for Redshift/Snowflake
+
+**🚧 Database Limitations:**
+- **BigQuery**: Does not support nested arrays - this is a **database limitation**, not a library limitation. BigQuery's type system doesn't allow `ARRAY<ARRAY<T>>` constructs. Struct types and arrays of structs work fine.
 
 ### Database-Specific Limitations
 - **BigQuery**: Does not support nested arrays (arrays of arrays). This is a BigQuery database limitation, not a library limitation.

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/README.md

@@ -85,11 +85,11 @@ The library supports different data types across database engines. All checkmark
 | **Decimal Array** | `List[Decimal]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | **Optional Array** | `Optional[List[T]]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
 | **Map/Dict** | `Dict[K, V]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| **Struct/Record** | `dataclass` | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
-| **Nested Arrays** | `List[List[T]]` | ❌ | ✅ | ✅ | ✅ | 🚧 | ✅ |
-| **Arrays of Structs** | `List[dataclass]` | ✅ | ✅ | ✅ | ✅ | 🚧 | ✅ |
-| **3D Arrays** | `List[List[List[T]]]` | ❌ | ✅ | ✅ | ✅ | 🚧 | ✅ |
-| **Arrays of Arrays of Structs** | `List[List[dataclass]]` | ❌ | ✅ | ✅ | ✅ | 🚧 | ✅ |
+| **Struct/Record** | `dataclass` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Nested Arrays** | `List[List[T]]` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Arrays of Structs** | `List[dataclass]` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **3D Arrays** | `List[List[List[T]]]` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Arrays of Arrays of Structs** | `List[List[dataclass]]` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
 
 ### Database-Specific Notes
 
@@ -97,7 +97,7 @@ The library supports different data types across database engines. All checkmark
 - **Athena**: 256KB query size limit; supports arrays and maps using `ARRAY[]` and `MAP(ARRAY[], ARRAY[])` syntax; supports struct types using `ROW` with named fields (dataclasses and Pydantic models); **full support for deeply nested types** including nested arrays, arrays of structs, and 3D arrays
 - **Redshift**: Arrays and maps implemented via SUPER type (JSON parsing); 16MB query size limit; **full support for deeply nested types** including struct types using SUPER + JSON_PARSE, nested arrays (2D, 3D+), arrays of structs, and arrays of arrays of structs
 - **Trino**: Memory catalog for testing; excellent decimal precision; supports arrays, maps, and struct types using `ROW` with named fields (dataclasses and Pydantic models); **full support for deeply nested types** including nested arrays, arrays of structs, and 3D arrays
-- **Snowflake**: Column names normalized to lowercase; 1MB query size limit; dict/map types implemented via VARIANT type (JSON parsing); struct types not yet supported (🚧 TODO); nested arrays not yet supported (🚧 TODO)
+- **Snowflake**: Column names normalized to lowercase; 1MB query size limit; dict/map types implemented via VARIANT type (JSON parsing); **full support for deeply nested types** including struct types using OBJECT + PARSE_JSON, nested arrays (2D, 3D+), arrays of structs, and arrays of arrays of structs
 - **DuckDB**: Fast embedded analytics database; excellent SQL standards compliance; supports arrays, maps, and struct types using `STRUCT` syntax with named fields (dataclasses and Pydantic models); **full support for deeply nested types** including nested arrays (2D, 3D+), arrays of structs, and arrays of arrays of structs
 
 ## Execution Modes Support
@@ -1374,25 +1374,23 @@ The library has a few known limitations that are planned to be addressed in futu
 
 ### Deeply Nested Complex Types Support
 
-**✅ Fully Supported (Athena, Trino, DuckDB & Redshift):**
+**✅ Fully Supported (All Major Adapters):**
 - Nested arrays (2D, 3D+): `List[List[int]]`, `List[List[List[int]]]`
 - Arrays of structs: `List[Address]` where Address is a dataclass
 - Arrays of arrays of structs: `List[List[OrderItem]]`
 - Maps with complex values: `Dict[str, str]`, `Dict[str, int]`
 - See `tests/integration/test_deeply_nested_types_integration.py` for comprehensive examples
-- **16 tests passing** across Athena, Trino, DuckDB, and Redshift (both CTE and physical tables modes)
+- **20 tests passing** across Athena, Trino, DuckDB, Redshift, and Snowflake (both CTE and physical tables modes)
 
 **Implementation Details:**
 - **Athena/Trino**: Use ROW types with named fields
 - **DuckDB**: Use STRUCT types with dictionary-style syntax
-- **Redshift**: Use SUPER type with JSON_PARSE for serialization, recursive JSON parsing for deserialization
-
-**🚧 TODO - Implementation Needed:**
-- **BigQuery**: Does not support nested arrays - this is a **database limitation**, not a library limitation. BigQuery's type system doesn't allow `ARRAY<ARRAY<T>>` constructs
-- **Snowflake**:
-  - Struct type support not implemented (see `Struct type not yet supported for dialect: snowflake`)
-  - Nested arrays not yet supported
-  - TODO: Implement struct/nested type support using VARIANT/OBJECT types
+- **Redshift**: Use SUPER type with JSON_PARSE, recursive JSON parsing
+- **Snowflake**: Use OBJECT type with PARSE_JSON, recursive JSON parsing
+- **Shared Helper**: `_parse_json_if_string()` in BaseTypeConverter for Redshift/Snowflake
+
+**🚧 Database Limitations:**
+- **BigQuery**: Does not support nested arrays - this is a **database limitation**, not a library limitation. BigQuery's type system doesn't allow `ARRAY<ARRAY<T>>` constructs. Struct types and arrays of structs work fine.
 
 ### Database-Specific Limitations
 - **BigQuery**: Does not support nested arrays (arrays of arrays). This is a BigQuery database limitation, not a library limitation.

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "sql-testing-library"
-version = "0.21.0"
+version = "0.22.1"
 description = "SQL Testing Framework for Python: Unit test SQL queries with mock data injection for BigQuery, Snowflake, Redshift, Athena, Trino, and DuckDB. Simplify data engineering ETL testing and analytics validation."
 authors = ["Gurmeet Saran <gurmeetx@gmail.com>", "Kushal Thakkar <kushal.thakkar@gmail.com>"]
 maintainers = ["Gurmeet Saran <gurmeetx@gmail.com>", "Kushal Thakkar <kushal.thakkar@gmail.com>"]

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/src/sql_testing_library/__init__.py

@@ -27,7 +27,7 @@ try:
 except ImportError:
     __all__ = []
 
-__version__ = "0.21.0"
+__version__ = "0.22.1"
 __all__.extend(
     [
         "SQLTestFramework",

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/src/sql_testing_library/_adapters/redshift.py

@@ -29,17 +29,6 @@ except ImportError:
 class RedshiftTypeConverter(BaseTypeConverter):
     """Redshift-specific type converter."""
 
-    def _parse_json_if_string(self, value: Any) -> Any:
-        """Helper to parse JSON strings from SUPER columns."""
-        if isinstance(value, str):
-            import json
-
-            try:
-                return json.loads(value)
-            except json.JSONDecodeError:
-                return value
-        return value
-
     def convert(self, value: Any, target_type: Type) -> Any:
         """Convert Redshift result value to target type."""
         # Handle None/NULL values first

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/src/sql_testing_library/_adapters/snowflake.py

@@ -50,21 +50,32 @@ class SnowflakeTypeConverter(BaseTypeConverter):
                 return None
             target_type = self.get_optional_inner_type(target_type)
 
+        # Handle struct types from Snowflake OBJECT/VARIANT columns
+        from .._types import is_struct_type
+
+        if is_struct_type(target_type):
+            # Parse JSON string if needed (using base class helper)
+            parsed_value = self._parse_json_if_string(value)
+            # Delegate to base converter (handles dict → struct)
+            return super().convert(parsed_value, target_type)
+
+        # Handle list/array types from Snowflake ARRAY/VARIANT columns
+        if hasattr(target_type, "__origin__") and target_type.__origin__ is list:
+            # Parse JSON string if needed (using base class helper)
+            parsed_value = self._parse_json_if_string(value)
+
+            if isinstance(parsed_value, list):
+                # Recursively convert each element with proper typing
+                element_type = get_args(target_type)[0] if get_args(target_type) else str
+                return [self.convert(elem, element_type) for elem in parsed_value]
+            else:
+                return []
+
         # Handle dict/map types from Snowflake VARIANT columns
         if hasattr(target_type, "__origin__") and target_type.__origin__ is dict:
-            # Snowflake returns VARIANT types as Python dicts already
-            if isinstance(value, dict):
-                return value
-            # If it's a string, parse it
-            elif isinstance(value, str):
-                import json
-
-                try:
-                    return json.loads(value)
-                except json.JSONDecodeError:
-                    return {}
-            else:
-                return {}
+            # Parse JSON string if needed (using base class helper)
+            parsed_value = self._parse_json_if_string(value)
+            return parsed_value if isinstance(parsed_value, dict) else {}
 
         # Snowflake returns proper Python types in most cases, so use base converter
         return super().convert(value, target_type)

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/src/sql_testing_library/_core.py

@@ -1,6 +1,7 @@
 """Core SQL testing framework."""
 
 import os
+import re
 from concurrent.futures import ThreadPoolExecutor, as_completed
 from dataclasses import dataclass
 from typing import (
@@ -35,6 +36,14 @@ from ._sql_logger import SQLLogger
 # Type for adapter types
 AdapterType = Literal["bigquery", "athena", "redshift", "trino", "snowflake", "duckdb"]
 
+
+# Matches an optional run of SQL comments (-- … and /* … */) followed by the WITH
+# keyword.  Group 1 captures everything before WITH so it can be preserved as a prefix.
+_WITH_AFTER_COMMENTS_RE = re.compile(
+    r"\A(\s*(?:(?:/\*.*?\*/|--[^\n]*)\s*)*)WITH\b",
+    re.DOTALL | re.IGNORECASE,
+)
+
 T = TypeVar("T")
 
 # Global storage for SQL execution data (used by pytest plugin)
@@ -84,12 +93,8 @@ class SQLTestCase(Generic[T]):
                 stacklevel=2,
             )
             self.default_namespace = self.execution_database
-        elif self.default_namespace is None and self.execution_database is None:
-            # Neither provided - this is an error
-            raise ValueError(
-                "Must provide either 'default_namespace' (preferred) or 'execution_database' "
-                "(deprecated) parameter"
-            )
+        # Both None is fine — queries using only fully-qualified table names
+        # don't need a default_namespace.
 
 
 class SQLTestFramework:
@@ -138,9 +143,8 @@ class SQLTestFramework:
             # Parse SQL to find table references
             referenced_tables = self._parse_sql_tables(test_case.query)
 
-            # Resolve unqualified table names
-            # default_namespace is guaranteed to be set by __post_init__
-            assert test_case.default_namespace is not None
+            # Resolve unqualified table names (default_namespace may be None if
+            # all table references in the query are already fully qualified)
             resolved_tables = self._resolve_table_names(
                 referenced_tables, test_case.default_namespace
             )
@@ -354,8 +358,11 @@ class SQLTestFramework:
             # Find all real tables (excluding the CTEs)
             tables = []
             for table in parsed.find_all(exp.Table):
-                # Skip tables that are actually CTE references
-                if str(table.name) in cte_aliases:
+                # Skip tables that are actually CTE references.
+                # Only unqualified names (no db/catalog) can be CTE references;
+                # a fully-qualified name like project.dataset.table is always a
+                # real table even when its unqualified name matches a CTE alias.
+                if not table.db and not table.catalog and str(table.name) in cte_aliases:
                     continue
 
                 # Get the fully qualified name including catalog/schema if present
@@ -374,7 +381,7 @@ class SQLTestFramework:
             raise SQLParseError(query, str(e))  # noqa:  B904
 
     def _resolve_table_names(
-        self, referenced_tables: List[str], default_namespace: str
+        self, referenced_tables: List[str], default_namespace: Optional[str]
     ) -> Dict[str, str]:
         """
         Resolve unqualified table names using default namespace context.
@@ -388,7 +395,13 @@ class SQLTestFramework:
                 # Already qualified
                 resolved[table_name] = table_name
             else:
-                # Add namespace prefix
+                if default_namespace is None:
+                    raise ValueError(
+                        f"Table '{table_name}' is not fully qualified and no "
+                        f"default_namespace was provided. Either use a fully qualified "
+                        f"table name (e.g. 'project.dataset.{table_name}') or set "
+                        f"default_namespace in your TestCase."
+                    )
                 qualified_name = f"{default_namespace}.{table_name}"
                 resolved[table_name] = qualified_name
 
@@ -463,12 +476,17 @@ class SQLTestFramework:
 
         # Combine CTEs with original query
         if ctes:
-            # Check if modified query already starts with WITH
             modified_query_stripped = modified_query.strip()
-            if modified_query_stripped.upper().startswith("WITH"):
-                # Query already has WITH clause, so append our CTEs with comma
+            m = _WITH_AFTER_COMMENTS_RE.match(modified_query_stripped)
+            if m:
+                # Query already has a WITH clause (possibly after leading comments).
+                # Preserve any leading comments and inject mock-table CTEs right
+                # before the existing WITH keyword.
+                prefix = m.group(1)
                 cte_block = ",\n".join(ctes)
-                final_query = f"WITH {cte_block},\n{modified_query_stripped[4:].strip()}"
+                final_query = (
+                    f"{prefix}WITH {cte_block},\n{modified_query_stripped[m.end() :].strip()}"
+                )
             else:
                 # Query doesn't have WITH clause, add it
                 cte_block = "WITH " + ",\n".join(ctes)

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/src/sql_testing_library/_sql_logger.py

@@ -96,14 +96,7 @@ class SQLLogger:
                 # Generate run ID with timestamp
                 timestamp = datetime.now().strftime("%Y%m%dT%H%M%S")
 
-                # Check if running in parallel mode
-                worker_id = self._get_worker_id()
-                if worker_id:
-                    # Include worker ID in run directory name for parallel execution
-                    SQLLogger._run_id = f"runid_{timestamp}_{worker_id}"
-                else:
-                    # Serial execution - use standard run ID
-                    SQLLogger._run_id = f"runid_{timestamp}"
+                SQLLogger._run_id = f"runid_{timestamp}"
 
                 SQLLogger._run_directory = self.log_dir / SQLLogger._run_id
                 SQLLogger._run_directory.mkdir(parents=True, exist_ok=True)

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/src/sql_testing_library/_sql_utils.py

@@ -676,6 +676,19 @@ def _format_struct_value(value: Any, struct_type: Type, dialect: str) -> str:
         json_str = json.dumps(json_obj, cls=DecimalEncoder)
         return f"JSON_PARSE('{json_str}')"
 
+    # For Snowflake (using OBJECT type with JSON)
+    elif dialect == "snowflake":
+        # Handle NULL struct values
+        if value is None:
+            return "NULL"
+
+        # Use helper function to convert struct to JSON-serializable dict
+        json_obj = _convert_to_json_serializable(value, struct_type)
+        json_str = json.dumps(json_obj, cls=DecimalEncoder)
+        # Escape single quotes for SQL
+        json_str = json_str.replace("'", "''")
+        return f"PARSE_JSON('{json_str}')"
+
     # For other databases, struct support would need to be implemented
     else:
         raise NotImplementedError(f"Struct type not yet supported for dialect: {dialect}")

{sql_testing_library-0.21.0 → sql_testing_library-0.22.1}/src/sql_testing_library/_types.py

@@ -224,6 +224,21 @@ class BaseTypeConverter:
         inner_type: Type = next(arg for arg in args if arg is not type(None))
         return inner_type
 
+    @staticmethod
+    def _parse_json_if_string(value: Any) -> Any:
+        """Helper to parse JSON strings from VARIANT/SUPER/OBJECT columns.
+
+        Used by adapters that store complex types as JSON (Redshift, Snowflake).
+        """
+        if isinstance(value, str):
+            import json
+
+            try:
+                return json.loads(value)
+            except json.JSONDecodeError:
+                return value
+        return value
+
     def convert(self, value: Any, target_type: Type) -> Any:
         """Convert value to target type."""
         # Handle None/NULL values