apache-airflow-providers-snowflake 6.4.0__tar.gz → 6.5.0rc1__tar.gz

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-snowflake
-Version: 6.4.0
+Version: 6.5.0rc1
 Summary: Provider package apache-airflow-providers-snowflake for Apache Airflow
 Keywords: airflow-provider,snowflake,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
 Maintainer-email: Apache Software Foundation <dev@airflow.apache.org>
-Requires-Python: ~=3.9
+Requires-Python: ~=3.10
 Description-Content-Type: text/x-rst
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Console
@@ -15,23 +15,24 @@ Classifier: Intended Audience :: System Administrators
 Classifier: Framework :: Apache Airflow
 Classifier: Framework :: Apache Airflow :: Provider
 Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: System :: Monitoring
-Requires-Dist: apache-airflow>=2.10.0
-Requires-Dist: apache-airflow-providers-common-compat>=1.6.0
-Requires-Dist: apache-airflow-providers-common-sql>=1.21.0
-Requires-Dist: pandas>=2.1.2,<2.2
-Requires-Dist: pyarrow>=14.0.1
+Requires-Dist: apache-airflow>=2.10.0rc1
+Requires-Dist: apache-airflow-providers-common-compat>=1.6.0rc1
+Requires-Dist: apache-airflow-providers-common-sql>=1.21.0rc1
+Requires-Dist: pandas>=2.1.2; python_version <"3.13"
+Requires-Dist: pandas>=2.2.3; python_version >="3.13"
+Requires-Dist: pyarrow>=16.1.0
 Requires-Dist: snowflake-connector-python>=3.7.1
 Requires-Dist: snowflake-sqlalchemy>=1.4.0
 Requires-Dist: snowflake-snowpark-python>=1.17.0;python_version<'3.12'
-Requires-Dist: apache-airflow-providers-openlineage>=2.3.0 ; extra == "openlineage"
+Requires-Dist: snowflake-snowpark-python>=1.27.0;python_version>='3.12'
+Requires-Dist: apache-airflow-providers-openlineage>=2.3.0rc1 ; extra == "openlineage"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0/changelog.html
-Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0
+Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-snowflake/6.5.0/changelog.html
+Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-snowflake/6.5.0
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -63,7 +64,7 @@ Provides-Extra: openlineage
 
 Package ``apache-airflow-providers-snowflake``
 
-Release: ``6.4.0``
+Release: ``6.5.0``
 
 
 `Snowflake <https://www.snowflake.com/>`__
@@ -76,7 +77,7 @@ This is a provider package for ``snowflake`` provider. All classes for this prov
 are in ``airflow.providers.snowflake`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.5.0/>`_.
 
 Installation
 ------------
@@ -85,23 +86,25 @@ You can install this package on top of an existing Airflow 2 installation (see `
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-snowflake``
 
-The package supports the following python versions: 3.9,3.10,3.11,3.12
+The package supports the following python versions: 3.10,3.11,3.12
 
 Requirements
 ------------
 
-==========================================  =====================================
+==========================================  ======================================
 PIP package                                 Version required
-==========================================  =====================================
+==========================================  ======================================
 ``apache-airflow``                          ``>=2.10.0``
 ``apache-airflow-providers-common-compat``  ``>=1.6.0``
 ``apache-airflow-providers-common-sql``     ``>=1.21.0``
-``pandas``                                  ``>=2.1.2,<2.2``
-``pyarrow``                                 ``>=14.0.1``
+``pandas``                                  ``>=2.1.2; python_version < "3.13"``
+``pandas``                                  ``>=2.2.3; python_version >= "3.13"``
+``pyarrow``                                 ``>=16.1.0``
 ``snowflake-connector-python``              ``>=3.7.1``
 ``snowflake-sqlalchemy``                    ``>=1.4.0``
 ``snowflake-snowpark-python``               ``>=1.17.0; python_version < "3.12"``
-==========================================  =====================================
+``snowflake-snowpark-python``               ``>=1.27.0; python_version >= "3.12"``
+==========================================  ======================================
 
 Cross provider package dependencies
 -----------------------------------
@@ -125,5 +128,5 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.5.0/changelog.html>`_.
 

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-snowflake``
 
-Release: ``6.4.0``
+Release: ``6.5.0``
 
 
 `Snowflake <https://www.snowflake.com/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``snowflake`` provider. All classes for this prov
 are in ``airflow.providers.snowflake`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.5.0/>`_.
 
 Installation
 ------------
@@ -45,23 +45,25 @@ You can install this package on top of an existing Airflow 2 installation (see `
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-snowflake``
 
-The package supports the following python versions: 3.9,3.10,3.11,3.12
+The package supports the following python versions: 3.10,3.11,3.12
 
 Requirements
 ------------
 
-==========================================  =====================================
+==========================================  ======================================
 PIP package                                 Version required
-==========================================  =====================================
+==========================================  ======================================
 ``apache-airflow``                          ``>=2.10.0``
 ``apache-airflow-providers-common-compat``  ``>=1.6.0``
 ``apache-airflow-providers-common-sql``     ``>=1.21.0``
-``pandas``                                  ``>=2.1.2,<2.2``
-``pyarrow``                                 ``>=14.0.1``
+``pandas``                                  ``>=2.1.2; python_version < "3.13"``
+``pandas``                                  ``>=2.2.3; python_version >= "3.13"``
+``pyarrow``                                 ``>=16.1.0``
 ``snowflake-connector-python``              ``>=3.7.1``
 ``snowflake-sqlalchemy``                    ``>=1.4.0``
 ``snowflake-snowpark-python``               ``>=1.17.0; python_version < "3.12"``
-==========================================  =====================================
+``snowflake-snowpark-python``               ``>=1.27.0; python_version >= "3.12"``
+==========================================  ======================================
 
 Cross provider package dependencies
 -----------------------------------
@@ -85,4 +87,4 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.5.0/changelog.html>`_.

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/docs/changelog.rst

@@ -27,6 +27,35 @@
 Changelog
 ---------
 
+6.5.0
+.....
+
+Features
+~~~~~~~~
+
+* ``feat: Add explicit support for SnowflakeSqlApiHook to Openlineage helper (#52161)``
+* ``feat: Add new query related methods to SnowflakeSqlApiHook (#52157)``
+* ``feat: Add SnowflakeSqlApiHook Retry Logic (#51463)``
+
+
+Misc
+~~~~
+
+* ``Provider Migration: Update Snowflake provider for Airflow 3.0 compatibility (#52629)``
+* ``Disable UP038 ruff rule and revert mandatory 'X | Y' in insintance checks (#52644)``
+* ``Bump pyarrow to 16.1.0 minimum version for several providers (#52635)``
+* ``Replace models.BaseOperator to Task SDK one for Common Providers (#52443)``
+* ``Relax snowflake-snowpark-python for Python>=3.12 (#52356)``
+* ``Drop support for Python 3.9 (#52072)``
+* ``Replace 'models.BaseOperator' to Task SDK one for Standard Provider (#52292)``
+* ``Bump upper binding on pandas in all providers (#52060)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Fix StopIteration in snowflake sql tests (#52394)``
+   * ``Make sure all test version imports come from test_common (#52425)``
+   * ``Add tests to test whether snowflake sql API handles invalid JSON (#52118)``
+
 6.4.0
 .....
 

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/docs/index.rst

@@ -78,7 +78,7 @@ apache-airflow-providers-snowflake package
 `Snowflake <https://www.snowflake.com/>`__
 
 
-Release: 6.4.0
+Release: 6.5.0
 
 Provider package
 ----------------
@@ -98,18 +98,20 @@ Requirements
 
 The minimum Apache Airflow version supported by this provider distribution is ``2.10.0``.
 
-==========================================  =====================================
+==========================================  ======================================
 PIP package                                 Version required
-==========================================  =====================================
+==========================================  ======================================
 ``apache-airflow``                          ``>=2.10.0``
 ``apache-airflow-providers-common-compat``  ``>=1.6.0``
 ``apache-airflow-providers-common-sql``     ``>=1.21.0``
-``pandas``                                  ``>=2.1.2,<2.2``
-``pyarrow``                                 ``>=14.0.1``
+``pandas``                                  ``>=2.1.2; python_version < "3.13"``
+``pandas``                                  ``>=2.2.3; python_version >= "3.13"``
+``pyarrow``                                 ``>=16.1.0``
 ``snowflake-connector-python``              ``>=3.7.1``
 ``snowflake-sqlalchemy``                    ``>=1.4.0``
 ``snowflake-snowpark-python``               ``>=1.17.0; python_version < "3.12"``
-==========================================  =====================================
+``snowflake-snowpark-python``               ``>=1.27.0; python_version >= "3.12"``
+==========================================  ======================================
 
 Cross provider package dependencies
 -----------------------------------
@@ -138,5 +140,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-snowflake 6.4.0 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.4.0.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.4.0.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.4.0.tar.gz.sha512>`__)
-* `The apache-airflow-providers-snowflake 6.4.0 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.4.0-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.4.0-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.4.0-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-snowflake 6.5.0 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.5.0.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.5.0.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.5.0.tar.gz.sha512>`__)
+* `The apache-airflow-providers-snowflake 6.5.0 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.5.0-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.5.0-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_snowflake-6.5.0-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/provider.yaml

@@ -22,12 +22,13 @@ description: |
     `Snowflake <https://www.snowflake.com/>`__
 
 state: ready
-source-date-epoch: 1749896974
+source-date-epoch: 1751474253
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 6.5.0
   - 6.4.0
   - 6.3.1
   - 6.3.0

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-snowflake"
-version = "6.4.0"
+version = "6.5.0rc1"
 description = "Provider package apache-airflow-providers-snowflake for Apache Airflow"
 readme = "README.rst"
 authors = [
@@ -44,38 +44,35 @@ classifiers = [
     "Framework :: Apache Airflow",
     "Framework :: Apache Airflow :: Provider",
     "License :: OSI Approved :: Apache Software License",
-    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
     "Topic :: System :: Monitoring",
 ]
-requires-python = "~=3.9"
+requires-python = "~=3.10"
 
 # The dependencies should be modified in place in the generated file.
 # Any change in the dependencies is preserved when the file is regenerated
 # Make sure to run ``breeze static-checks --type update-providers-dependencies --all-files``
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
-    "apache-airflow>=2.10.0",
-    "apache-airflow-providers-common-compat>=1.6.0",
-    "apache-airflow-providers-common-sql>=1.21.0",
-    # In pandas 2.2 minimal version of the sqlalchemy is 2.0
-    # https://pandas.pydata.org/docs/whatsnew/v2.2.0.html#increased-minimum-versions-for-dependencies
-    # However Airflow not fully supports it yet: https://github.com/apache/airflow/issues/28723
-    # In addition FAB also limit sqlalchemy to < 2.0
-    "pandas>=2.1.2,<2.2",
-    "pyarrow>=14.0.1",
+    "apache-airflow>=2.10.0rc1",
+    "apache-airflow-providers-common-compat>=1.6.0rc1",
+    "apache-airflow-providers-common-sql>=1.21.0rc1",
+    'pandas>=2.1.2; python_version <"3.13"',
+    'pandas>=2.2.3; python_version >="3.13"',
+    "pyarrow>=16.1.0",
     "snowflake-connector-python>=3.7.1",
     "snowflake-sqlalchemy>=1.4.0",
     "snowflake-snowpark-python>=1.17.0;python_version<'3.12'",
+    "snowflake-snowpark-python>=1.27.0;python_version>='3.12'",
 ]
 
 # The optional dependencies should be modified in place in the generated file
 # Any change in the dependencies is preserved when the file is regenerated
 [project.optional-dependencies]
 "openlineage" = [
-    "apache-airflow-providers-openlineage>=2.3.0"
+    "apache-airflow-providers-openlineage>=2.3.0rc1"
 ]
 
 [dependency-groups]
@@ -116,8 +113,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0"
-"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.4.0/changelog.html"
+"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-snowflake/6.5.0"
+"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-snowflake/6.5.0/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/src/airflow/providers/snowflake/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "6.4.0"
+__version__ = "6.5.0"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.10.0"

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/src/airflow/providers/snowflake/decorators/snowpark.py

@@ -17,8 +17,8 @@
 
 from __future__ import annotations
 
-from collections.abc import Sequence
-from typing import TYPE_CHECKING, Callable
+from collections.abc import Callable, Sequence
+from typing import TYPE_CHECKING
 
 from airflow.providers.snowflake.version_compat import AIRFLOW_V_3_0_PLUS
 

{apache_airflow_providers_snowflake-6.4.0 → apache_airflow_providers_snowflake-6.5.0rc1}/src/airflow/providers/snowflake/hooks/snowflake.py

@@ -19,12 +19,12 @@ from __future__ import annotations
 
 import base64
 import os
-from collections.abc import Iterable, Mapping
+from collections.abc import Callable, Iterable, Mapping
 from contextlib import closing, contextmanager
 from functools import cached_property
 from io import StringIO
 from pathlib import Path
-from typing import TYPE_CHECKING, Any, Callable, TypeVar, overload
+from typing import TYPE_CHECKING, Any, TypeVar, overload
 from urllib.parse import urlparse
 
 import requests
@@ -617,10 +617,9 @@ class SnowflakeHook(DbApiHook):
 
     def get_openlineage_database_specific_lineage(self, task_instance) -> OperatorLineage | None:
         """
-        Generate OpenLineage metadata for a Snowflake task instance based on executed query IDs.
+        Emit separate OpenLineage events for each Snowflake query, based on executed query IDs.
 
-        If a single query ID is present, attach an `ExternalQueryRunFacet` to the lineage metadata.
-        If multiple query IDs are present, emits separate OpenLineage events for each query.
+        If a single query ID is present, also add an `ExternalQueryRunFacet` to the returned lineage metadata.
 
         Note that `get_openlineage_database_specific_lineage` is usually called after task's execution,
         so if multiple query IDs are present, both START and COMPLETE event for each query will be emitted
@@ -641,13 +640,22 @@ class SnowflakeHook(DbApiHook):
         )
 
         if not self.query_ids:
-            self.log.debug("openlineage: no snowflake query ids found.")
+            self.log.info("OpenLineage could not find snowflake query ids.")
             return None
 
         self.log.debug("openlineage: getting connection to get database info")
         connection = self.get_connection(self.get_conn_id())
         namespace = SQLParser.create_namespace(self.get_openlineage_database_info(connection))
 
+        self.log.info("Separate OpenLineage events will be emitted for each query_id.")
+        emit_openlineage_events_for_snowflake_queries(
+            task_instance=task_instance,
+            hook=self,
+            query_ids=self.query_ids,
+            query_for_extra_metadata=True,
+            query_source_namespace=namespace,
+        )
+
         if len(self.query_ids) == 1:
             self.log.debug("Attaching ExternalQueryRunFacet with single query_id to OpenLineage event.")
             return OperatorLineage(
@@ -658,20 +666,4 @@ class SnowflakeHook(DbApiHook):
                 }
             )
 
-        self.log.info("Multiple query_ids found. Separate OpenLineage event will be emitted for each query.")
-        try:
-            from airflow.providers.openlineage.utils.utils import should_use_external_connection
-
-            use_external_connection = should_use_external_connection(self)
-        except ImportError:
-            # OpenLineage provider release < 1.8.0 - we always use connection
-            use_external_connection = True
-
-        emit_openlineage_events_for_snowflake_queries(
-            query_ids=self.query_ids,
-            query_source_namespace=namespace,
-            task_instance=task_instance,
-            hook=self if use_external_connection else None,
-        )
-
         return None