snowflake-sqlalchemy 1.7.0__tar.gz → 1.7.2__tar.gz

{snowflake_sqlalchemy-1.7.0 → snowflake_sqlalchemy-1.7.2}/DESCRIPTION.md

@@ -9,8 +9,19 @@ Source code is also available at:
 
 # Release Notes
 
-- v1.7.0(November 22, 2024)
-
+- v1.7.2(December 18, 2024)
+  - Fix quoting of `_` as column name
+  - Fix index columns was not being reflected
+  - Fix index reflection cache not working
+  - Add support for structured OBJECT datatype
+  - Add support for structured ARRAY datatype
+
+- v1.7.1(December 02, 2024)
+  - Add support for partition by to copy into <location>
+  - Fix BOOLEAN type not found in snowdialect
+  - Add support for autocommit Isolation Level
+
+- v1.7.0(November 21, 2024)
   - Add support for dynamic tables and required options
   - Add support for hybrid tables
   - Fixed SAWarning when registering functions with existing name in default namespace

{snowflake_sqlalchemy-1.7.0 → snowflake_sqlalchemy-1.7.2}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.3
+Metadata-Version: 2.4
 Name: snowflake-sqlalchemy
-Version: 1.7.0
+Version: 1.7.2
 Summary: Snowflake SQLAlchemy Dialect
 Project-URL: Changelog, https://github.com/snowflakedb/snowflake-sqlalchemy/blob/main/DESCRIPTION.md
 Project-URL: Documentation, https://docs.snowflake.com/en/user-guide/sqlalchemy.html
@@ -8,7 +8,8 @@ Project-URL: Homepage, https://www.snowflake.com/
 Project-URL: Issues, https://github.com/snowflakedb/snowflake-sqlalchemy/issues
 Project-URL: Source, https://github.com/snowflakedb/snowflake-sqlalchemy
 Author-email: "Snowflake Inc." <triage-snowpark-python-api-dl@snowflake.com>
-License: Apache-2.0
+License-Expression: Apache-2.0
+License-File: LICENSE.txt
 Keywords: Snowflake,analytics,cloud,database,db,warehouse
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Console
@@ -45,6 +46,7 @@ Requires-Dist: pytest-cov; extra == 'development'
 Requires-Dist: pytest-rerunfailures; extra == 'development'
 Requires-Dist: pytest-timeout; extra == 'development'
 Requires-Dist: pytz; extra == 'development'
+Requires-Dist: setuptools; extra == 'development'
 Requires-Dist: syrupy==4.6.1; extra == 'development'
 Provides-Extra: pandas
 Requires-Dist: snowflake-connector-python[pandas]; extra == 'pandas'
@@ -60,6 +62,11 @@ Description-Content-Type: text/markdown
 
 Snowflake SQLAlchemy runs on the top of the Snowflake Connector for Python as a [dialect](http://docs.sqlalchemy.org/en/latest/dialects/) to bridge a Snowflake database and SQLAlchemy applications.
 
+
+| :exclamation:        | For production-affecting or urgent issues related to the connector, please [create a case with Snowflake Support](https://community.snowflake.com/s/article/How-To-Submit-a-Support-Case-in-Snowflake-Lodge). |
+|---------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+
+
 ## Prerequisites
 
 ### Snowflake Connector for Python
@@ -256,7 +263,7 @@ finally:
 
 # Best
 try:
-    with engine.connext() as connection:
+    with engine.connect() as connection:
         connection.execute(text(<SQL>))
         # or
         connection.exec_driver_sql(<SQL>)
@@ -277,11 +284,43 @@ t = Table('mytable', metadata,
 
 ### Object Name Case Handling
 
-Snowflake stores all case-insensitive object names in uppercase text. In contrast, SQLAlchemy considers all lowercase object names to be case-insensitive. Snowflake SQLAlchemy converts the object name case during schema-level communication, i.e. during table and index reflection. If you use uppercase object names, SQLAlchemy assumes they are case-sensitive and encloses the names with quotes. This behavior will cause mismatches agaisnt data dictionary data received from Snowflake, so unless identifier names have been truly created as case sensitive using quotes, e.g., `"TestDb"`, all lowercase names should be used on the SQLAlchemy side.
+Snowflake stores all case-insensitive object names in uppercase text. In contrast, SQLAlchemy considers all lowercase object names to be case-insensitive. Snowflake SQLAlchemy converts the object name case during schema-level communication, i.e. during table and index reflection. If you use uppercase object names, SQLAlchemy assumes they are case-sensitive and encloses the names with quotes. This behavior will cause mismatches against data dictionary data received from Snowflake, so unless identifier names have been truly created as case sensitive using quotes, e.g., `"TestDb"`, all lowercase names should be used on the SQLAlchemy side.
 
 ### Index Support
 
-Snowflake does not utilize indexes, so neither does Snowflake SQLAlchemy.
+Indexes are supported only for Hybrid Tables in Snowflake SQLAlchemy. For more details on limitations and use cases, refer to the [Create Index documentation](https://docs.snowflake.com/en/sql-reference/constraints-indexes.html). You can create an index using the following methods:
+
+#### Single Column Index
+
+You can create a single column index by setting the `index=True` parameter on the column or by explicitly defining an `Index` object.
+
+```python
+hybrid_test_table_1 = HybridTable(
+  "table_name",
+  metadata,
+  Column("column1", Integer, primary_key=True),
+  Column("column2", String, index=True),
+  Index("index_1", "column1", "column2")
+)
+
+metadata.create_all(engine_testaccount)
+```
+
+#### Multi-Column Index
+
+For multi-column indexes, you define the `Index` object specifying the columns that should be indexed.
+
+```python
+hybrid_test_table_1 = HybridTable(
+  "table_name",
+  metadata,
+  Column("column1", Integer, primary_key=True),
+  Column("column2", String),
+  Index("index_1", "column1", "column2")
+)
+
+metadata.create_all(engine_testaccount)
+```
 
 ### Numpy Data Type Support
 
@@ -382,6 +421,79 @@ data_object  = json.loads(row[1])
 data_array   = json.loads(row[2])
 ```
 
+### Structured Data Types Support
+
+This module defines custom SQLAlchemy types for Snowflake structured data, specifically for **Iceberg tables**.
+The types —**MAP**, **OBJECT**, and **ARRAY**— allow you to store complex data structures in your SQLAlchemy models.
+For detailed information, refer to the Snowflake [Structured data types](https://docs.snowflake.com/en/sql-reference/data-types-structured) documentation.
+
+---
+
+#### MAP
+
+The `MAP` type represents a collection of key-value pairs, where each key and value can have different types.
+
+- **Key Type**: The type of the keys (e.g., `TEXT`, `NUMBER`).
+- **Value Type**: The type of the values (e.g., `TEXT`, `NUMBER`).
+- **Not Null**: Whether `NULL` values are allowed (default is `False`).
+
+*Example Usage*
+
+```python
+IcebergTable(
+    table_name,
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("map_col", MAP(NUMBER(10, 0), TEXT(16777216))),
+    external_volume="external_volume",
+    base_location="base_location",
+)
+```
+
+#### OBJECT
+
+The `OBJECT` type represents a semi-structured object with named fields. Each field can have a specific type, and you can also specify whether each field is nullable.
+
+- **Items Types**: A dictionary of field names and their types. The type can optionally include a nullable flag (`True` for not nullable, `False` for nullable, default is `False`).
+
+*Example Usage*
+
+```python
+IcebergTable(
+    table_name,
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column(
+        "object_col",
+        OBJECT(key1=(TEXT(16777216), False), key2=(NUMBER(10, 0), False)),
+        OBJECT(key1=TEXT(16777216), key2=NUMBER(10, 0)), # Without nullable flag
+    ),
+    external_volume="external_volume",
+    base_location="base_location",
+)
+```
+
+#### ARRAY
+
+The `ARRAY` type represents an ordered list of values, where each element has the same type. The type of the elements is defined when creating the array.
+
+- **Value Type**: The type of the elements in the array (e.g., `TEXT`, `NUMBER`).
+- **Not Null**: Whether `NULL` values are allowed (default is `False`).
+
+*Example Usage*
+
+```python
+IcebergTable(
+    table_name,
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("array_col", ARRAY(TEXT(16777216))),
+    external_volume="external_volume",
+    base_location="base_location",
+)
+```
+
+
 ### CLUSTER BY Support
 
 Snowflake SQLAchemy supports the `CLUSTER BY` parameter for tables. For information about the parameter, see :doc:`/sql-reference/sql/create-table`.
@@ -508,6 +620,117 @@ copy_into = CopyIntoStorage(from_=users,
 connection.execute(copy_into)
 ```
 
+### Iceberg Table with Snowflake Catalog support
+
+Snowflake SQLAlchemy supports Iceberg Tables with the Snowflake Catalog, along with various related parameters. For detailed information about Iceberg Tables, refer to the Snowflake [CREATE ICEBERG](https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-snowflake) documentation.
+
+To create an Iceberg Table using Snowflake SQLAlchemy, you can define the table using the SQLAlchemy Core syntax as follows:
+
+```python
+table = IcebergTable(
+    "myuser",
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("name", String),
+    external_volume=external_volume_name,
+    base_location="my_iceberg_table",
+    as_query="SELECT * FROM table"
+)
+```
+
+Alternatively, you can define the table using a declarative approach:
+
+```python
+class MyUser(Base):
+    __tablename__ = "myuser"
+
+    @classmethod
+    def __table_cls__(cls, name, metadata, *arg, **kw):
+        return IcebergTable(name, metadata, *arg, **kw)
+
+    __table_args__ = {
+        "external_volume": "my_external_volume",
+        "base_location": "my_iceberg_table",
+        "as_query": "SELECT * FROM table",
+    }
+
+    id = Column(Integer, primary_key=True)
+    name = Column(String)
+```
+
+### Hybrid Table support
+
+Snowflake SQLAlchemy supports Hybrid Tables with indexes. For detailed information, refer to the Snowflake [CREATE HYBRID TABLE](https://docs.snowflake.com/en/sql-reference/sql/create-hybrid-table) documentation.
+
+To create a Hybrid Table and add an index, you can use the SQLAlchemy Core syntax as follows:
+
+```python
+table = HybridTable(
+    "myuser",
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("name", String),
+    Index("idx_name", "name")
+)
+```
+
+Alternatively, you can define the table using the declarative approach:
+
+```python
+class MyUser(Base):
+    __tablename__ = "myuser"
+
+    @classmethod
+    def __table_cls__(cls, name, metadata, *arg, **kw):
+        return HybridTable(name, metadata, *arg, **kw)
+
+    __table_args__ = (
+        Index("idx_name", "name"),
+    )
+
+    id = Column(Integer, primary_key=True)
+    name = Column(String)
+```
+
+### Dynamic Tables support
+
+Snowflake SQLAlchemy supports Dynamic Tables. For detailed information, refer to the Snowflake [CREATE DYNAMIC TABLE](https://docs.snowflake.com/en/sql-reference/sql/create-dynamic-table) documentation.
+
+To create a Dynamic Table, you can use the SQLAlchemy Core syntax as follows:
+
+```python
+dynamic_test_table_1 = DynamicTable(
+    "dynamic_MyUser",
+    metadata,
+    Column("id", Integer),
+    Column("name", String),
+    target_lag=(1, TimeUnit.HOURS),  # Additionally, you can use SnowflakeKeyword.DOWNSTREAM
+    warehouse='test_wh',
+    refresh_mode=SnowflakeKeyword.FULL,
+    as_query="SELECT id, name from MyUser;"
+)
+```
+
+Alternatively, you can define a table without columns using the SQLAlchemy `select()` construct:
+
+```python
+dynamic_test_table_1 = DynamicTable(
+    "dynamic_MyUser",
+    metadata,
+    target_lag=(1, TimeUnit.HOURS),
+    warehouse='test_wh',
+    refresh_mode=SnowflakeKeyword.FULL,
+    as_query=select(MyUser.id, MyUser.name)
+)
+```
+
+### Notes
+
+- Defining a primary key in a Dynamic Table is not supported, meaning declarative tables don’t support Dynamic Tables.
+- When using the `as_query` parameter with a string, you must explicitly define the columns. However, if you use the SQLAlchemy `select()` construct, you don’t need to explicitly define the columns.
+- Direct data insertion into Dynamic Tables is not supported.
+
+
 ## Support
 
 Feel free to file an issue or submit a PR here for general cases. For official support, contact Snowflake support at:

{snowflake_sqlalchemy-1.7.0 → snowflake_sqlalchemy-1.7.2}/README.md

@@ -8,6 +8,11 @@
 
 Snowflake SQLAlchemy runs on the top of the Snowflake Connector for Python as a [dialect](http://docs.sqlalchemy.org/en/latest/dialects/) to bridge a Snowflake database and SQLAlchemy applications.
 
+
+| :exclamation:        | For production-affecting or urgent issues related to the connector, please [create a case with Snowflake Support](https://community.snowflake.com/s/article/How-To-Submit-a-Support-Case-in-Snowflake-Lodge). |
+|---------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+
+
 ## Prerequisites
 
 ### Snowflake Connector for Python
@@ -204,7 +209,7 @@ finally:
 
 # Best
 try:
-    with engine.connext() as connection:
+    with engine.connect() as connection:
         connection.execute(text(<SQL>))
         # or
         connection.exec_driver_sql(<SQL>)
@@ -225,11 +230,43 @@ t = Table('mytable', metadata,
 
 ### Object Name Case Handling
 
-Snowflake stores all case-insensitive object names in uppercase text. In contrast, SQLAlchemy considers all lowercase object names to be case-insensitive. Snowflake SQLAlchemy converts the object name case during schema-level communication, i.e. during table and index reflection. If you use uppercase object names, SQLAlchemy assumes they are case-sensitive and encloses the names with quotes. This behavior will cause mismatches agaisnt data dictionary data received from Snowflake, so unless identifier names have been truly created as case sensitive using quotes, e.g., `"TestDb"`, all lowercase names should be used on the SQLAlchemy side.
+Snowflake stores all case-insensitive object names in uppercase text. In contrast, SQLAlchemy considers all lowercase object names to be case-insensitive. Snowflake SQLAlchemy converts the object name case during schema-level communication, i.e. during table and index reflection. If you use uppercase object names, SQLAlchemy assumes they are case-sensitive and encloses the names with quotes. This behavior will cause mismatches against data dictionary data received from Snowflake, so unless identifier names have been truly created as case sensitive using quotes, e.g., `"TestDb"`, all lowercase names should be used on the SQLAlchemy side.
 
 ### Index Support
 
-Snowflake does not utilize indexes, so neither does Snowflake SQLAlchemy.
+Indexes are supported only for Hybrid Tables in Snowflake SQLAlchemy. For more details on limitations and use cases, refer to the [Create Index documentation](https://docs.snowflake.com/en/sql-reference/constraints-indexes.html). You can create an index using the following methods:
+
+#### Single Column Index
+
+You can create a single column index by setting the `index=True` parameter on the column or by explicitly defining an `Index` object.
+
+```python
+hybrid_test_table_1 = HybridTable(
+  "table_name",
+  metadata,
+  Column("column1", Integer, primary_key=True),
+  Column("column2", String, index=True),
+  Index("index_1", "column1", "column2")
+)
+
+metadata.create_all(engine_testaccount)
+```
+
+#### Multi-Column Index
+
+For multi-column indexes, you define the `Index` object specifying the columns that should be indexed.
+
+```python
+hybrid_test_table_1 = HybridTable(
+  "table_name",
+  metadata,
+  Column("column1", Integer, primary_key=True),
+  Column("column2", String),
+  Index("index_1", "column1", "column2")
+)
+
+metadata.create_all(engine_testaccount)
+```
 
 ### Numpy Data Type Support
 
@@ -330,6 +367,79 @@ data_object  = json.loads(row[1])
 data_array   = json.loads(row[2])
 ```
 
+### Structured Data Types Support
+
+This module defines custom SQLAlchemy types for Snowflake structured data, specifically for **Iceberg tables**.
+The types —**MAP**, **OBJECT**, and **ARRAY**— allow you to store complex data structures in your SQLAlchemy models.
+For detailed information, refer to the Snowflake [Structured data types](https://docs.snowflake.com/en/sql-reference/data-types-structured) documentation.
+
+---
+
+#### MAP
+
+The `MAP` type represents a collection of key-value pairs, where each key and value can have different types.
+
+- **Key Type**: The type of the keys (e.g., `TEXT`, `NUMBER`).
+- **Value Type**: The type of the values (e.g., `TEXT`, `NUMBER`).
+- **Not Null**: Whether `NULL` values are allowed (default is `False`).
+
+*Example Usage*
+
+```python
+IcebergTable(
+    table_name,
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("map_col", MAP(NUMBER(10, 0), TEXT(16777216))),
+    external_volume="external_volume",
+    base_location="base_location",
+)
+```
+
+#### OBJECT
+
+The `OBJECT` type represents a semi-structured object with named fields. Each field can have a specific type, and you can also specify whether each field is nullable.
+
+- **Items Types**: A dictionary of field names and their types. The type can optionally include a nullable flag (`True` for not nullable, `False` for nullable, default is `False`).
+
+*Example Usage*
+
+```python
+IcebergTable(
+    table_name,
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column(
+        "object_col",
+        OBJECT(key1=(TEXT(16777216), False), key2=(NUMBER(10, 0), False)),
+        OBJECT(key1=TEXT(16777216), key2=NUMBER(10, 0)), # Without nullable flag
+    ),
+    external_volume="external_volume",
+    base_location="base_location",
+)
+```
+
+#### ARRAY
+
+The `ARRAY` type represents an ordered list of values, where each element has the same type. The type of the elements is defined when creating the array.
+
+- **Value Type**: The type of the elements in the array (e.g., `TEXT`, `NUMBER`).
+- **Not Null**: Whether `NULL` values are allowed (default is `False`).
+
+*Example Usage*
+
+```python
+IcebergTable(
+    table_name,
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("array_col", ARRAY(TEXT(16777216))),
+    external_volume="external_volume",
+    base_location="base_location",
+)
+```
+
+
 ### CLUSTER BY Support
 
 Snowflake SQLAchemy supports the `CLUSTER BY` parameter for tables. For information about the parameter, see :doc:`/sql-reference/sql/create-table`.
@@ -456,6 +566,117 @@ copy_into = CopyIntoStorage(from_=users,
 connection.execute(copy_into)
 ```
 
+### Iceberg Table with Snowflake Catalog support
+
+Snowflake SQLAlchemy supports Iceberg Tables with the Snowflake Catalog, along with various related parameters. For detailed information about Iceberg Tables, refer to the Snowflake [CREATE ICEBERG](https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-snowflake) documentation.
+
+To create an Iceberg Table using Snowflake SQLAlchemy, you can define the table using the SQLAlchemy Core syntax as follows:
+
+```python
+table = IcebergTable(
+    "myuser",
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("name", String),
+    external_volume=external_volume_name,
+    base_location="my_iceberg_table",
+    as_query="SELECT * FROM table"
+)
+```
+
+Alternatively, you can define the table using a declarative approach:
+
+```python
+class MyUser(Base):
+    __tablename__ = "myuser"
+
+    @classmethod
+    def __table_cls__(cls, name, metadata, *arg, **kw):
+        return IcebergTable(name, metadata, *arg, **kw)
+
+    __table_args__ = {
+        "external_volume": "my_external_volume",
+        "base_location": "my_iceberg_table",
+        "as_query": "SELECT * FROM table",
+    }
+
+    id = Column(Integer, primary_key=True)
+    name = Column(String)
+```
+
+### Hybrid Table support
+
+Snowflake SQLAlchemy supports Hybrid Tables with indexes. For detailed information, refer to the Snowflake [CREATE HYBRID TABLE](https://docs.snowflake.com/en/sql-reference/sql/create-hybrid-table) documentation.
+
+To create a Hybrid Table and add an index, you can use the SQLAlchemy Core syntax as follows:
+
+```python
+table = HybridTable(
+    "myuser",
+    metadata,
+    Column("id", Integer, primary_key=True),
+    Column("name", String),
+    Index("idx_name", "name")
+)
+```
+
+Alternatively, you can define the table using the declarative approach:
+
+```python
+class MyUser(Base):
+    __tablename__ = "myuser"
+
+    @classmethod
+    def __table_cls__(cls, name, metadata, *arg, **kw):
+        return HybridTable(name, metadata, *arg, **kw)
+
+    __table_args__ = (
+        Index("idx_name", "name"),
+    )
+
+    id = Column(Integer, primary_key=True)
+    name = Column(String)
+```
+
+### Dynamic Tables support
+
+Snowflake SQLAlchemy supports Dynamic Tables. For detailed information, refer to the Snowflake [CREATE DYNAMIC TABLE](https://docs.snowflake.com/en/sql-reference/sql/create-dynamic-table) documentation.
+
+To create a Dynamic Table, you can use the SQLAlchemy Core syntax as follows:
+
+```python
+dynamic_test_table_1 = DynamicTable(
+    "dynamic_MyUser",
+    metadata,
+    Column("id", Integer),
+    Column("name", String),
+    target_lag=(1, TimeUnit.HOURS),  # Additionally, you can use SnowflakeKeyword.DOWNSTREAM
+    warehouse='test_wh',
+    refresh_mode=SnowflakeKeyword.FULL,
+    as_query="SELECT id, name from MyUser;"
+)
+```
+
+Alternatively, you can define a table without columns using the SQLAlchemy `select()` construct:
+
+```python
+dynamic_test_table_1 = DynamicTable(
+    "dynamic_MyUser",
+    metadata,
+    target_lag=(1, TimeUnit.HOURS),
+    warehouse='test_wh',
+    refresh_mode=SnowflakeKeyword.FULL,
+    as_query=select(MyUser.id, MyUser.name)
+)
+```
+
+### Notes
+
+- Defining a primary key in a Dynamic Table is not supported, meaning declarative tables don’t support Dynamic Tables.
+- When using the `as_query` parameter with a string, you must explicitly define the columns. However, if you use the SQLAlchemy `select()` construct, you don’t need to explicitly define the columns.
+- Direct data insertion into Dynamic Tables is not supported.
+
+
 ## Support
 
 Feel free to file an issue or submit a PR here for general cases. For official support, contact Snowflake support at:

{snowflake_sqlalchemy-1.7.0 → snowflake_sqlalchemy-1.7.2}/pyproject.toml

@@ -47,6 +47,7 @@ path = "src/snowflake/sqlalchemy/version.py"
 development = [
   "pre-commit",
   "pytest",
+  "setuptools",
   "pytest-cov",
   "pytest-timeout",
   "pytest-rerunfailures",
@@ -74,6 +75,8 @@ exclude = ["/.github"]
 packages = ["src/snowflake"]
 
 [tool.hatch.envs.default]
+path = ".venv"
+type = "virtual"
 extra-dependencies = ["SQLAlchemy>=1.4.19,<2.1.0"]
 features = ["development", "pandas"]
 python = "3.8"