apache-airflow-providers-common-sql 2.0.2__tar.gz → 2.0.3rc1__tar.gz

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-common-sql
-Version: 2.0.2
+Version: 2.0.3rc1
 Summary: Provider package apache-airflow-providers-common-sql for Apache Airflow
 Keywords: airflow-provider,common.sql,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -23,8 +23,8 @@ Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: System :: Monitoring
 License-File: LICENSE
 License-File: NOTICE
-Requires-Dist: apache-airflow>=2.11.0
-Requires-Dist: apache-airflow-providers-common-compat>=1.14.1
+Requires-Dist: apache-airflow>=2.11.0rc1
+Requires-Dist: apache-airflow-providers-common-compat>=1.14.1rc1
 Requires-Dist: sqlparse>=0.5.1
 Requires-Dist: more-itertools>=9.0.0
 Requires-Dist: methodtools>=0.4.7
@@ -39,8 +39,8 @@ Requires-Dist: polars>=1.26.0 ; extra == "polars"
 Requires-Dist: pyiceberg-core>=0.8.0 ; extra == "pyiceberg-core"
 Requires-Dist: sqlalchemy>=1.4.54 ; extra == "sqlalchemy"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2/changelog.html
-Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2
+Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-common-sql/2.0.3/changelog.html
+Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-common-sql/2.0.3
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -79,7 +79,7 @@ Provides-Extra: sqlalchemy
 
 Package ``apache-airflow-providers-common-sql``
 
-Release: ``2.0.2``
+Release: ``2.0.3``
 
 
 `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__
@@ -92,7 +92,7 @@ This is a provider package for ``common.sql`` provider. All classes for this pro
 are in ``airflow.providers.common.sql`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.3/>`_.
 
 Installation
 ------------
@@ -154,5 +154,5 @@ Extra               Dependencies
 ==================  =======================================================================================================================================================================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.3/changelog.html>`_.
 

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-common-sql``
 
-Release: ``2.0.2``
+Release: ``2.0.3``
 
 
 `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__
@@ -36,7 +36,7 @@ This is a provider package for ``common.sql`` provider. All classes for this pro
 are in ``airflow.providers.common.sql`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.3/>`_.
 
 Installation
 ------------
@@ -98,4 +98,4 @@ Extra               Dependencies
 ==================  =======================================================================================================================================================================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.3/changelog.html>`_.

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/docs/changelog.rst

@@ -25,6 +25,18 @@
 Changelog
 ---------
 
+2.0.3
+.....
+
+Misc
+~~~~
+
+* ``Make psycopg (v3) the default synchronous Postgres driver (#69526)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+
+
 2.0.2
 .....
 
@@ -780,7 +792,6 @@ Misc
   `Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>`_.
 
 
-
 Features
 ~~~~~~~~
 

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/docs/index.rst

@@ -79,7 +79,7 @@ apache-airflow-providers-common-sql package
 `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__
 
 
-Release: 2.0.2
+Release: 2.0.3
 
 Provider package
 ----------------
@@ -160,5 +160,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-common-sql 2.0.2 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.2.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.2.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.2.tar.gz.sha512>`__)
-* `The apache-airflow-providers-common-sql 2.0.2 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.2-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.2-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.2-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-common-sql 2.0.3 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.3.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.3.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.3.tar.gz.sha512>`__)
+* `The apache-airflow-providers-common-sql 2.0.3 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.3-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.3-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-2.0.3-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/provider.yaml

@@ -23,12 +23,13 @@ description: |
 
 state: ready
 lifecycle: production
-source-date-epoch: 1783356304
+source-date-epoch: 1784746595
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 2.0.3
   - 2.0.2
   - 2.0.1
   - 2.0.0

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-common-sql"
-version = "2.0.2"
+version = "2.0.3rc1"
 description = "Provider package apache-airflow-providers-common-sql for Apache Airflow"
 readme = "README.rst"
 license = "Apache-2.0"
@@ -59,8 +59,8 @@ requires-python = ">=3.10"
 # Make sure to run ``prek update-providers-dependencies --all-files``
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
-    "apache-airflow>=2.11.0",
-    "apache-airflow-providers-common-compat>=1.14.1",
+    "apache-airflow>=2.11.0rc1",
+    "apache-airflow-providers-common-compat>=1.14.1rc1",
     "sqlparse>=0.5.1",
     "more-itertools>=9.0.0",
     # The methodtools dependency is necessary since the introduction of dialects:
@@ -148,8 +148,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2"
-"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-common-sql/2.0.2/changelog.html"
+"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-common-sql/2.0.3"
+"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-common-sql/2.0.3/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/src/airflow/providers/common/sql/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "2.0.2"
+__version__ = "2.0.3"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.11.0"

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/src/airflow/providers/common/sql/hooks/sql.py

@@ -22,6 +22,7 @@ from collections.abc import Callable, Generator, Iterable, Mapping, MutableMappi
 from contextlib import closing, contextmanager, suppress
 from datetime import datetime
 from functools import cached_property
+from importlib.util import find_spec
 from typing import TYPE_CHECKING, Any, Literal, Protocol, TypeVar, cast, overload
 from urllib.parse import urlparse
 
@@ -66,6 +67,16 @@ if TYPE_CHECKING:
 
 T = TypeVar("T")
 SQL_PLACEHOLDERS = frozenset({"%s", "?"})
+
+
+def _is_sqlalchemy_2() -> bool:
+    """Whether the installed SQLAlchemy has the native psycopg (v3) dialect (added in 2.0)."""
+    from packaging.version import Version
+    from sqlalchemy import __version__ as sqlalchemy_version
+
+    return Version(sqlalchemy_version) >= Version("2.0.0")
+
+
 WARNING_MESSAGE = """Import of {} from the 'airflow.providers.common.sql.hooks' module is deprecated and will
 be removed in the future. Please import it from 'airflow.providers.common.sql.hooks.handlers'."""
 
@@ -334,7 +345,31 @@ class DbApiHook(BaseHook):
 
         self.log.debug("url: %s", url)
         self.log.debug("engine_kwargs: %s", engine_kwargs)
-        return create_engine(url=url, **engine_kwargs)
+        try:
+            return create_engine(url=url, **engine_kwargs)
+        except (ImportError, ModuleNotFoundError):
+            # SQLAlchemy resolves a bare "postgresql" scheme to the psycopg2 DB-API by default.
+            # Retry with a driver this hook can actually resolve, since psycopg2 may not be
+            # installed now that it's an optional extra of apache-airflow-providers-postgres.
+            parsed_url = make_url(url) if make_url is not None else None
+            if parsed_url is None or parsed_url.drivername != "postgresql":
+                raise
+            # psycopg (v3) may be importable where SQLAlchemy is pinned below 2.0, which has no
+            # native "postgresql+psycopg" dialect and would raise NoSuchModuleError — so only
+            # prefer it on SQLAlchemy 2.0+; otherwise fall back to psycopg2.
+            if find_spec("psycopg") is not None and _is_sqlalchemy_2():
+                self.log.info(
+                    "SQLAlchemy could not load a DB-API driver for the bare 'postgresql://' URL; "
+                    "retrying with psycopg (v3) ('postgresql+psycopg')."
+                )
+                return create_engine(url=parsed_url.set(drivername="postgresql+psycopg"), **engine_kwargs)
+            if find_spec("psycopg2") is not None:
+                self.log.info(
+                    "SQLAlchemy could not load a DB-API driver for the bare 'postgresql://' URL; "
+                    "retrying with 'postgresql+psycopg2'."
+                )
+                return create_engine(url=parsed_url.set(drivername="postgresql+psycopg2"), **engine_kwargs)
+            raise
 
     @cached_property
     def inspector(self) -> Inspector:

{apache_airflow_providers_common_sql-2.0.2 → apache_airflow_providers_common_sql-2.0.3rc1}/tests/unit/common/sql/hooks/test_sql.py

@@ -334,6 +334,98 @@ class TestDbApiHook:
             assert isinstance(df, expected_type)
 
 
+class TestDbApiHookGetSqlalchemyEngine:
+    """SQLAlchemy resolves a bare ``postgresql`` scheme to the psycopg2 DB-API by default, which
+    may not be installed now that it's an optional extra of apache-airflow-providers-postgres."""
+
+    @pytest.mark.db_test
+    @patch("airflow.providers.common.sql.hooks.sql._is_sqlalchemy_2", return_value=True)
+    @patch("airflow.providers.common.sql.hooks.sql.find_spec")
+    @patch("airflow.providers.common.sql.hooks.sql.create_engine")
+    def test_falls_back_to_psycopg_when_psycopg2_import_fails(
+        self, mock_create_engine, mock_find_spec, mock_is_sqlalchemy_2, caplog
+    ):
+        mock_find_spec.return_value = object()
+        mock_create_engine.side_effect = [ModuleNotFoundError("No module named 'psycopg2'"), MagicMock()]
+        dbapi_hook = mock_db_hook(DbApiHook, conn_params={"conn_type": "postgresql"})
+
+        dbapi_hook.get_sqlalchemy_engine()
+
+        assert mock_create_engine.call_count == 2
+        retried_url = mock_create_engine.call_args.kwargs["url"]
+        assert retried_url.drivername == "postgresql+psycopg"
+        assert (
+            "SQLAlchemy could not load a DB-API driver for the bare 'postgresql://' URL; "
+            "retrying with psycopg (v3) ('postgresql+psycopg')."
+        ) in caplog.text
+
+    @pytest.mark.db_test
+    @patch("airflow.providers.common.sql.hooks.sql._is_sqlalchemy_2", return_value=False)
+    @patch("airflow.providers.common.sql.hooks.sql.find_spec")
+    @patch("airflow.providers.common.sql.hooks.sql.create_engine")
+    def test_falls_back_to_psycopg2_when_sqlalchemy_below_2(
+        self, mock_create_engine, mock_find_spec, mock_is_sqlalchemy_2, caplog
+    ):
+        # SQLAlchemy 1.4 (shipped with Airflow 2.11) has no native "postgresql+psycopg" dialect,
+        # so even when psycopg (v3) is importable the retry must use psycopg2, not psycopg.
+        mock_find_spec.return_value = object()
+        mock_create_engine.side_effect = [ModuleNotFoundError("No module named 'psycopg2'"), MagicMock()]
+        dbapi_hook = mock_db_hook(DbApiHook, conn_params={"conn_type": "postgresql"})
+
+        dbapi_hook.get_sqlalchemy_engine()
+
+        assert mock_create_engine.call_count == 2
+        retried_url = mock_create_engine.call_args.kwargs["url"]
+        assert retried_url.drivername == "postgresql+psycopg2"
+        assert (
+            "SQLAlchemy could not load a DB-API driver for the bare 'postgresql://' URL; "
+            "retrying with 'postgresql+psycopg2'."
+        ) in caplog.text
+
+    @pytest.mark.db_test
+    @patch("airflow.providers.common.sql.hooks.sql.find_spec")
+    @patch("airflow.providers.common.sql.hooks.sql.create_engine")
+    def test_falls_back_to_psycopg2_when_psycopg_unavailable(
+        self, mock_create_engine, mock_find_spec, caplog
+    ):
+        mock_find_spec.side_effect = lambda name: None if name == "psycopg" else object()
+        mock_create_engine.side_effect = [ModuleNotFoundError("No module named 'psycopg2'"), MagicMock()]
+        dbapi_hook = mock_db_hook(DbApiHook, conn_params={"conn_type": "postgresql"})
+
+        dbapi_hook.get_sqlalchemy_engine()
+
+        assert mock_create_engine.call_count == 2
+        retried_url = mock_create_engine.call_args.kwargs["url"]
+        assert retried_url.drivername == "postgresql+psycopg2"
+        assert (
+            "SQLAlchemy could not load a DB-API driver for the bare 'postgresql://' URL; "
+            "retrying with 'postgresql+psycopg2'."
+        ) in caplog.text
+
+    @pytest.mark.db_test
+    @patch("airflow.providers.common.sql.hooks.sql.find_spec", return_value=None)
+    @patch("airflow.providers.common.sql.hooks.sql.create_engine")
+    def test_reraises_when_no_postgres_driver_is_available(self, mock_create_engine, mock_find_spec):
+        original_error = ModuleNotFoundError("No module named 'psycopg2'")
+        mock_create_engine.side_effect = original_error
+        dbapi_hook = mock_db_hook(DbApiHook, conn_params={"conn_type": "postgresql"})
+
+        with pytest.raises(ModuleNotFoundError, match="psycopg2"):
+            dbapi_hook.get_sqlalchemy_engine()
+        mock_create_engine.assert_called_once()
+
+    @pytest.mark.db_test
+    @patch("airflow.providers.common.sql.hooks.sql.create_engine")
+    def test_does_not_retry_for_non_postgres_schemes(self, mock_create_engine):
+        original_error = ModuleNotFoundError("No module named 'some_other_driver'")
+        mock_create_engine.side_effect = original_error
+        dbapi_hook = mock_db_hook(DbApiHook, conn_params={"conn_type": "mysql"})
+
+        with pytest.raises(ModuleNotFoundError, match="some_other_driver"):
+            dbapi_hook.get_sqlalchemy_engine()
+        mock_create_engine.assert_called_once()
+
+
 class TestDbApiHookGetTableSchema:
     @pytest.mark.db_test
     def test_get_table_schema(self):