apache-airflow-providers-postgres 7.0.2__tar.gz → 7.1.0__tar.gz

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: apache-airflow-providers-postgres
-Version: 7.0.2
+Version: 7.1.0
 Summary: Provider package apache-airflow-providers-postgres for Apache Airflow
 Keywords: airflow-provider,postgres,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -43,8 +43,8 @@ Requires-Dist: psycopg2-binary>=2.9.9 ; extra == "psycopg2" and ( python_version
 Requires-Dist: psycopg2-binary>=2.9.10 ; extra == "psycopg2" and ( python_version >= '3.13')
 Requires-Dist: sqlalchemy>=1.4.54 ; extra == "sqlalchemy"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2/changelog.html
-Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2
+Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0/changelog.html
+Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -86,7 +86,7 @@ Import-Namespace: airflow.providers
 
 Package ``apache-airflow-providers-postgres``
 
-Release: ``7.0.2``
+Release: ``7.1.0``
 
 
 `PostgreSQL <https://www.postgresql.org/>`__
@@ -99,7 +99,7 @@ This is a provider package for ``postgres`` provider. All classes for this provi
 are in ``airflow.providers.postgres`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0/>`_.
 
 Installation
 ------------
@@ -164,5 +164,5 @@ Extra                Dependencies
 ===================  ============================================================================================================================================================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0/changelog.html>`_.
 

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-postgres``
 
-Release: ``7.0.2``
+Release: ``7.1.0``
 
 
 `PostgreSQL <https://www.postgresql.org/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``postgres`` provider. All classes for this provi
 are in ``airflow.providers.postgres`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0/>`_.
 
 Installation
 ------------
@@ -101,4 +101,4 @@ Extra                Dependencies
 ===================  ============================================================================================================================================================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0/changelog.html>`_.

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/docs/changelog.rst

@@ -27,6 +27,25 @@
 Changelog
 ---------
 
+7.1.0
+.....
+
+Features
+~~~~~~~~
+
+* ``Allow keeping PostgresHook SQLAlchemy engines on psycopg2 (#72000)``
+
+Doc-only
+~~~~~~~~
+
+* ``Fix grammar in the dialects documentation (#73145)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Revert "[main] Upgrade important CI environment (#73308)" (#73621)``
+   * ``[main] Upgrade important CI environment (#73308)``
+   * ``Prepare providers release 2026-09-22``
+
 7.0.2
 .....
 

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/docs/connections/postgres.rst

@@ -108,6 +108,15 @@ Extra (optional)
       If not specified than hostname from **Connection Host** is used.
     * ``azure_conn_id`` - Azure Connection ID to be used for authentication via Azure Entra ID. Azure Oauth token
       is retrieved from the azure connection which is used as password for PostgreSQL connection. Scope for the Azure OAuth token can be set in the config option ``azure_oauth_scope`` under the section ``[postgres]``. Requires `apache-airflow-providers-microsoft-azure>=12.8.0`.
+    * ``sqlalchemy_scheme`` - The SQLAlchemy ``drivername`` used for the URLs the hook builds
+      (``get_uri``, ``get_sqlalchemy_engine``). Must be ``postgresql`` or ``postgresql+<driver>``.
+      Since provider 7.0.0 the hook selects psycopg (v3) whenever SQLAlchemy 2.x is installed;
+      set this to ``postgresql+psycopg2`` to keep SQLAlchemy engines created from this connection
+      on psycopg2. This matters if your Dags rely on psycopg2-specific behaviour — for example,
+      psycopg2 sends string parameters without a type so PostgreSQL implicitly coerces them, while
+      the psycopg (v3) SQLAlchemy dialect renders typed casts, so inserting string values into
+      e.g. ``uuid`` columns (as ``pandas.DataFrame.to_sql`` does) fails with
+      *"column is of type uuid but expression is of type character varying"*.
 
     Example "extras" field (Amazon RDS PostgreSQL or Amazon Aurora PostgreSQL):
 

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/docs/dialects.rst

@@ -25,7 +25,7 @@ and the :class:`~airflow.providers.jdbc.hooks.jdbc.JdbcHook` are generic hooks w
 database that has a driver for it, it needed an abstraction layer which allows us to run specialized queries
 depending of the database to which we connect and that's why dialects where introduced.
 
-The default :class:`~airflow.providers.common.sql.dialects.dialect.Dialect` class has following operations
+The default :class:`~airflow.providers.common.sql.dialects.dialect.Dialect` class has the following operations
 available which underneath use SQLAlchemy to execute, but can be overloaded with specialized implementations
 per database:
 

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/docs/index.rst

@@ -78,7 +78,7 @@ apache-airflow-providers-postgres package
 `PostgreSQL <https://www.postgresql.org/>`__
 
 
-Release: 7.0.2
+Release: 7.1.0
 
 Provider package
 ----------------
@@ -162,5 +162,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-postgres 7.0.2 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.0.2.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.0.2.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.0.2.tar.gz.sha512>`__)
-* `The apache-airflow-providers-postgres 7.0.2 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.0.2-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.0.2-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.0.2-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-postgres 7.1.0 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.1.0.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.1.0.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.1.0.tar.gz.sha512>`__)
+* `The apache-airflow-providers-postgres 7.1.0 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.1.0-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.1.0-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_postgres-7.1.0-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/provider.yaml

@@ -23,13 +23,13 @@ description: |
 
 state: ready
 lifecycle: production
-source-date-epoch: 1787011200
+source-date-epoch: 1790023613
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
-  - 7.0.2
+  - 7.1.0
   - 7.0.1
   - 7.0.0
   - 6.8.0

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-postgres"
-version = "7.0.2"
+version = "7.1.0"
 description = "Provider package apache-airflow-providers-postgres for Apache Airflow"
 readme = "README.rst"
 license = "Apache-2.0"
@@ -151,8 +151,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2"
-"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.0.2/changelog.html"
+"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0"
+"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-postgres/7.1.0/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/src/airflow/providers/postgres/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "7.0.2"
+__version__ = "7.1.0"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.11.0"

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/src/airflow/providers/postgres/hooks/postgres.py

@@ -135,6 +135,11 @@ class PostgresHook(DbApiHook):
     :param options: Optional. Specifies command-line options to send to the server
         at connection start. For example, setting this to ``-c search_path=myschema``
         sets the session's value of the ``search_path`` to ``myschema``.
+    :param sqlalchemy_scheme: Optional. The SQLAlchemy ``drivername`` used for the URLs the hook
+        builds (``get_uri``, ``get_sqlalchemy_engine``), e.g. ``postgresql+psycopg2``. Must be
+        ``postgresql`` or ``postgresql+<driver>``. Defaults to ``postgresql+psycopg`` when
+        psycopg (v3) serves SQLAlchemy 2.x and to ``postgresql`` otherwise. Can also be set via
+        the connection extra ``sqlalchemy_scheme``; this parameter takes precedence.
     :param enable_log_db_messages: Optional. If enabled logs database messages sent to the client
         during the session. To avoid a memory leak psycopg2 only saves the last 50 messages.
         For details, see: `PostgreSQL logging configuration parameters
@@ -164,18 +169,39 @@ class PostgresHook(DbApiHook):
     default_azure_oauth_scope = "https://ossrdbms-aad.database.windows.net/.default"
 
     def __init__(
-        self, *args, options: str | None = None, enable_log_db_messages: bool = False, **kwargs
+        self,
+        *args,
+        options: str | None = None,
+        enable_log_db_messages: bool = False,
+        sqlalchemy_scheme: str | None = None,
+        **kwargs,
     ) -> None:
         super().__init__(*args, **kwargs)
         self.conn: CompatConnection | None = None
         self.database: str | None = kwargs.pop("database", None)
         self.options = options
         self.enable_log_db_messages = enable_log_db_messages
+        self._sqlalchemy_scheme = sqlalchemy_scheme
 
     @staticmethod
     def __cast_nullable(value, dst_type: type) -> Any:
         return dst_type(value) if value is not None else None
 
+    @property
+    def sqlalchemy_scheme(self) -> str:
+        """SQLAlchemy ``drivername`` used for the URLs built by this hook."""
+        scheme = self._sqlalchemy_scheme or self.connection.extra_dejson.get("sqlalchemy_scheme")
+        if not scheme:
+            return "postgresql+psycopg" if USE_PSYCOPG3 else "postgresql"
+        if ":" in scheme or "/" in scheme:
+            raise ValueError("The parameter 'sqlalchemy_scheme' must not contain ':' or '/' characters!")
+        if scheme != "postgresql" and not scheme.startswith("postgresql+"):
+            raise ValueError(
+                f"The parameter 'sqlalchemy_scheme' must be 'postgresql' or 'postgresql+<driver>', "
+                f"got: {scheme!r}"
+            )
+        return scheme
+
     @property
     def sqlalchemy_url(self) -> URL:
         try:
@@ -192,7 +218,7 @@ class PostgresHook(DbApiHook):
         if conn.extra_dejson.get("iam", False):
             conn.login, conn.password, conn.port = self.get_iam_token(conn)
         return URL.create(
-            drivername="postgresql+psycopg" if USE_PSYCOPG3 else "postgresql",
+            drivername=self.sqlalchemy_scheme,
             username=self.__cast_nullable(conn.login, str),
             password=self.__cast_nullable(conn.password, str),
             host=self.__cast_nullable(conn.host, str),

{apache_airflow_providers_postgres-7.0.2 → apache_airflow_providers_postgres-7.1.0}/tests/unit/postgres/hooks/test_postgres.py

@@ -633,6 +633,93 @@ class TestPostgresHookConnPPG3:
         )
 
 
+class TestPostgresHookSqlalchemyScheme:
+    """Tests for overriding the SQLAlchemy ``drivername`` via the sqlalchemy_scheme extra/parameter."""
+
+    @staticmethod
+    def get_hook(extra: dict | None = None, **hook_kwargs) -> PostgresHook:
+        conn = Connection(
+            login="login-conn", password="password-conn", host="host", schema="database", extra=extra
+        )
+        return PostgresHook(connection=conn, **hook_kwargs)
+
+    @pytest.mark.parametrize("scheme", ["postgresql", "postgresql+psycopg2", "postgresql+psycopg"])
+    def test_sqlalchemy_scheme_from_extra(self, scheme):
+        hook = self.get_hook(extra=dict(sqlalchemy_scheme=scheme))
+        expected = f"{scheme}://login-conn:password-conn@host/database"
+        assert hook.sqlalchemy_url.render_as_string(hide_password=False) == expected
+
+    def test_sqlalchemy_scheme_parameter_takes_precedence_over_extra(self):
+        hook = self.get_hook(
+            extra=dict(sqlalchemy_scheme="postgresql"), sqlalchemy_scheme="postgresql+psycopg2"
+        )
+        expected = "postgresql+psycopg2://login-conn:password-conn@host/database"
+        assert hook.sqlalchemy_url.render_as_string(hide_password=False) == expected
+
+    def test_get_uri_with_sqlalchemy_scheme(self):
+        hook = self.get_hook(extra=dict(sqlalchemy_scheme="postgresql+psycopg2"))
+        assert hook.get_uri() == "postgresql+psycopg2://login-conn:password-conn@host/database"
+
+    @pytest.mark.parametrize("scheme", ["mysql", "mysql+pymysql", "postgres+psycopg2"])
+    def test_sqlalchemy_scheme_with_wrong_dialect(self, scheme):
+        hook = self.get_hook(extra=dict(sqlalchemy_scheme=scheme))
+        with pytest.raises(
+            ValueError, match="'sqlalchemy_scheme' must be 'postgresql' or 'postgresql\\+<driver>'"
+        ):
+            hook.sqlalchemy_url
+
+    @pytest.mark.parametrize("scheme", ["postgresql+psycopg2://malicious", "postgresql+psycopg2/malicious"])
+    def test_sqlalchemy_scheme_with_forbidden_characters(self, scheme):
+        hook = self.get_hook(extra=dict(sqlalchemy_scheme=scheme))
+        with pytest.raises(ValueError, match="must not contain ':' or '/' characters"):
+            hook.sqlalchemy_url
+
+
+@pytest.mark.backend("postgres")
+class TestPostgresHookPandasToSqlUuid:
+    """DataFrame.to_sql with string values into a uuid column: fails on psycopg3 because SQLAlchemy
+    renders typed bind casts, works when the connection opts back into psycopg2 via sqlalchemy_scheme."""
+
+    table = "test_pandas_to_sql_uuid_table"
+
+    PSYCOPG3_XFAIL_REASON = (
+        "The psycopg3 SQLAlchemy dialect renders typed bind casts, so string params fail with "
+        "'column is of type uuid but expression is of type character varying' instead of being "
+        "implicitly coerced as under psycopg2. Tracked upstream in "
+        "https://github.com/pandas-dev/pandas/issues/63511, "
+        "https://github.com/apache/arrow/pull/50325, "
+        "https://github.com/sqlalchemy/sqlalchemy/discussions/10839 and "
+        "https://github.com/sqlalchemy/sqlalchemy/issues/12060"
+    )
+
+    def teardown_method(self):
+        with PostgresHook().get_conn() as conn:
+            with conn.cursor() as cur:
+                cur.execute(f"DROP TABLE IF EXISTS {self.table}")
+
+    def insert_string_uuid_df(self, hook: PostgresHook) -> None:
+        engine = hook.get_sqlalchemy_engine()
+        with engine.begin() as conn:
+            conn.execute(sqlalchemy.text(f"CREATE TABLE {self.table} (id UUID PRIMARY KEY, item TEXT)"))
+        df = pd.DataFrame({"id": ["b29d6cda-04ca-4073-8ef8-4f70d35e41e2"], "item": ["laptop"]})
+        df.to_sql(self.table, engine, if_exists="append", index=False)
+
+    @pytest.mark.skipif(not USE_PSYCOPG3, reason="psycopg v3 or sqlalchemy v2 not available")
+    @pytest.mark.xfail(
+        raises=(sqlalchemy.exc.ProgrammingError, pd.errors.DatabaseError),
+        strict=True,
+        reason=PSYCOPG3_XFAIL_REASON,
+    )
+    def test_to_sql_string_uuid_fails_on_psycopg3(self):
+        self.insert_string_uuid_df(PostgresHook())
+
+    def test_to_sql_string_uuid_works_with_psycopg2_scheme(self):
+        hook = PostgresHook(sqlalchemy_scheme="postgresql+psycopg2")
+        self.insert_string_uuid_df(hook)
+        with hook.get_sqlalchemy_engine().connect() as conn:
+            assert conn.execute(sqlalchemy.text(f"SELECT COUNT(*) FROM {self.table}")).scalar() == 1
+
+
 @pytest.mark.backend("postgres")
 class TestPostgresHook:
     """Tests that are identical between psycopg2 and psycopg3."""