apache-airflow-providers-common-sql 1.27.5__tar.gz → 1.28.0__tar.gz

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-common-sql
-Version: 1.27.5
+Version: 1.28.0
 Summary: Provider package apache-airflow-providers-common-sql for Apache Airflow
 Keywords: airflow-provider,common.sql,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -29,8 +29,8 @@ Requires-Dist: pandas[sql-other]>=2.1.2 ; extra == "pandas" and ( python_version
 Requires-Dist: pandas>=2.2.3 ; extra == "pandas" and ( python_version >="3.13")
 Requires-Dist: polars>=1.26.0 ; extra == "polars"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5/changelog.html
-Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5
+Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0/changelog.html
+Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -64,9 +64,8 @@ Provides-Extra: polars
 
 Package ``apache-airflow-providers-common-sql``
 
-Release: ``1.27.5``
+Release: ``1.28.0``
 
-Release Date: ``|PypiReleaseDate|``
 
 `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__
 
@@ -78,12 +77,12 @@ This is a provider package for ``common.sql`` provider. All classes for this pro
 are in ``airflow.providers.common.sql`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0/>`_.
 
 Installation
 ------------
 
-You can install this package on top of an existing Airflow 2 installation (see ``Requirements`` below
+You can install this package on top of an existing Airflow installation (see ``Requirements`` below
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-common-sql``
 
@@ -121,5 +120,5 @@ Dependent package
 ==============================================================================================================  ===============
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0/changelog.html>`_.
 

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/README.rst

@@ -23,9 +23,8 @@
 
 Package ``apache-airflow-providers-common-sql``
 
-Release: ``1.27.5``
+Release: ``1.28.0``
 
-Release Date: ``|PypiReleaseDate|``
 
 `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__
 
@@ -37,12 +36,12 @@ This is a provider package for ``common.sql`` provider. All classes for this pro
 are in ``airflow.providers.common.sql`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0/>`_.
 
 Installation
 ------------
 
-You can install this package on top of an existing Airflow 2 installation (see ``Requirements`` below
+You can install this package on top of an existing Airflow installation (see ``Requirements`` below
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-common-sql``
 
@@ -80,4 +79,4 @@ Dependent package
 ==============================================================================================================  ===============
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0/changelog.html>`_.

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/docs/changelog.rst

@@ -25,6 +25,44 @@
 Changelog
 ---------
 
+1.28.0
+......
+
+
+Release Date: ``|PypiReleaseDate|``
+
+Features
+~~~~~~~~
+
+* ``Add SQLInsertRowsOperator in common.sql provider (#55031)``
+* ``Added paginated_sql_statement_clause parameter to GenericTransfer (#48866)``
+* ``Add rudimentary support for psycopg3 (#52976)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fixed resolving of dialect name when host of JdbcHook is an JDBC URL (#54446)``
+
+Misc
+~~~~
+
+* ``Remove unneeded isort skip comments (#54979)``
+
+Doc-only
+~~~~~~~~
+
+* ``Make term Dag consistent in providers docs (#55101)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Remove airflow.models.DAG (#54383)``
+   * ``Replace API server's direct Connection access workaround in BaseHook (#54083)``
+   * ``Switch pre-commit to prek (#54258)``
+   * ``make bundle_name not nullable (#47592)``
+
+.. Review and move the new changes to one of the sections above:
+   * ``Fix Airflow 2 reference in README/index of providers (#55240)``
+
 1.27.5
 ......
 

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/docs/index.rst

@@ -58,7 +58,7 @@
     :maxdepth: 1
     :caption: Resources
 
-    Example DAGs <https://github.com/apache/airflow/tree/providers-common-sql/|version|/providers/common/sql/tests/system/common/sql>
+    Example Dags <https://github.com/apache/airflow/tree/providers-common-sql/|version|/providers/common/sql/tests/system/common/sql>
     PyPI Repository <https://pypi.org/project/apache-airflow-providers-common-sql/>
     Installing from sources <installing-providers-from-sources>
 
@@ -79,9 +79,7 @@ apache-airflow-providers-common-sql package
 `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__
 
 
-Release: 1.27.5
-
-Release Date: ``|PypiReleaseDate|``
+Release: 1.28.0
 
 Provider package
 ----------------
@@ -92,7 +90,7 @@ All classes for this package are included in the ``airflow.providers.common.sql`
 Installation
 ------------
 
-You can install this package on top of an existing Airflow 2 installation via
+You can install this package on top of an existing Airflow installation via
 ``pip install apache-airflow-providers-common-sql``.
 For the minimum Airflow version supported, see ``Requirements`` below.
 
@@ -135,5 +133,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-common-sql 1.27.5 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.27.5.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.27.5.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.27.5.tar.gz.sha512>`__)
-* `The apache-airflow-providers-common-sql 1.27.5 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.27.5-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.27.5-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.27.5-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-common-sql 1.28.0 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.28.0.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.28.0.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.28.0.tar.gz.sha512>`__)
+* `The apache-airflow-providers-common-sql 1.28.0 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.28.0-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.28.0-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_common_sql-1.28.0-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/docs/operators.rst

@@ -174,6 +174,37 @@ The below example demonstrates how to instantiate the SQLThresholdCheckOperator
 
 If the value returned by the query, is within the thresholds, the task passes. Otherwise, it fails.
 
+.. _howto/operator:SQLInsertRowsOperator:
+
+Insert rows into Table
+~~~~~~~~~~~~~~~~~~~~~~
+
+Use the :class:`~airflow.providers.common.sql.operators.sql.SQLInsertRowsOperator` to insert rows into a database table
+directly from Python data structures or an XCom. Parameters of the operator are:
+
+- ``table_name`` - name of the table in which the rows will be inserted (templated).
+- ``conn_id`` - the Airflow connection ID used to connect to the database.
+- ``schema`` (optional) - the schema in which the table is defined.
+- ``database`` (optional) - name of the database which overrides the one defined in the connection.
+- ``columns`` (optional) - list of columns to use for the insert when passing a list of dictionaries.
+- ``ignored_columns`` (optional) - list of columns to ignore for the insert, if no columns are specified,
+  columns will be dynamically resolved from the metadata.
+- ``rows`` - rows to insert, a list of tuples.
+- ``rows_processor`` (optional) - a function applied to the rows before inserting them.
+- ``preoperator`` (optional) - SQL statement or list of statements to execute before inserting data (templated).
+- ``postoperator`` (optional) - SQL statement or list of statements to execute after inserting data (templated).
+- ``hook_params`` (optional) - dictionary of additional parameters passed to the underlying hook.
+- ``insert_args`` (optional) - dictionary of additional arguments passed to the hook's ``insert_rows`` method,
+  can include ``replace``, ``executemany``, ``fast_executemany``, ``autocommit``, and others supported by the hook.
+
+The example below shows how to instantiate the SQLInsertRowsOperator task.
+
+.. exampleinclude:: /../tests/system/common/sql/example_sql_insert_rows.py
+    :language: python
+    :dedent: 4
+    :start-after: [START howto_operator_sql_insert_rows]
+    :end-before: [END howto_operator_sql_insert_rows]
+
 .. _howto/operator:GenericTransfer:
 
 Generic Transfer

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/provider.yaml

@@ -22,12 +22,13 @@ description: |
     `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__
 
 state: ready
-source-date-epoch: 1754503073
+source-date-epoch: 1756876759
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 1.28.0
   - 1.27.5
   - 1.27.4
   - 1.27.3

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-common-sql"
-version = "1.27.5"
+version = "1.28.0"
 description = "Provider package apache-airflow-providers-common-sql for Apache Airflow"
 readme = "README.rst"
 authors = [
@@ -54,7 +54,7 @@ requires-python = ">=3.10"
 
 # The dependencies should be modified in place in the generated file.
 # Any change in the dependencies is preserved when the file is regenerated
-# Make sure to run ``breeze static-checks --type update-providers-dependencies --all-files``
+# Make sure to run ``prek update-providers-dependencies --all-files``
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
     "apache-airflow>=2.10.0",
@@ -121,8 +121,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5"
-"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.27.5/changelog.html"
+"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0"
+"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/README_API.md

@@ -30,7 +30,7 @@ The approach we take is similar to one that has been applied by Android OS team
 and it is based on storing the current version of API and flagging changes that are potentially breaking.
 This is done by comparing the previous API (store in stub files) and the upcoming API from the PR.
 The upcoming API is automatically extracted from `common.sql` Python files using `update-common-sql-api-stubs`
-pre-commit using mypy `stubgen` and stored as `.pyi` files in the `airflow.providers.common.sql` package.
+prek hook using mypy `stubgen` and stored as `.pyi` files in the `airflow.providers.common.sql` package.
 We also post-process the `.pyi` files to add some historically exposed methods that should be also
 considered as public API.
 
@@ -40,22 +40,22 @@ to review the changes and manually regenerate the stub files.
 The details of the workflow are as follows:
 
 1) The previous API is stored in the (committed to repository) stub files.
-2) Every time when common.sql Python files are modified the `update-common-sql-api-stubs` pre-commit
+2) Whenever common.sql Python files are modified the `update-common-sql-api-stubs` prek hook
   regenerates the stubs (including post-processing it) and looks for potentially breaking changes
    (removals or updates of the existing classes/methods).
-3) If the check reveals there are no changes to the API, nothing happens, pre-commit succeeds.
-4) If there are only additions, the pre-commit automatically updates the stub files,
-   asks the contributor to commit resulting updates and fails the pre-commit. This is very similar to
+3) If the check reveals there are no changes to the API, nothing happens, prek hook succeeds.
+4) If there are only additions, the prek hook automatically updates the stub files,
+   asks the contributor to commit resulting updates and fails the prek hook. This is very similar to
    other static checks that automatically modify/fix source code.
-5) If the pre-commit detects potentially breaking changes, the process is a bit more involved for the
-   contributor. The pre-commit flags such changes to the contributor by failing the pre-commit and
+5) If the prek hook detects potentially breaking changes, the process is a bit more involved for the
+   contributor. The prek hook flags such changes to the contributor by failing the prek hook and
    asks the contributor to review the change looking specifically for breaking compatibility with previous
    providers (and fix any backwards compatibility). Once this is completed, the contributor is asked to
-   manually and explicitly regenerate and commit the new version of the stubs by running the pre-commit
+   manually and explicitly regenerate and commit the new version of the stubs by running the prek hook
    with manually added environment variable:
 
 ```shell
-UPDATE_COMMON_SQL_API=1 pre-commit run update-common-sql-api-stubs
+UPDATE_COMMON_SQL_API=1 prek update-common-sql-api-stubs
 ```
 
 # Verifying other providers to use only public API of the `common.sql` provider

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "1.27.5"
+__version__ = "1.28.0"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.10.0"

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/dialects/dialect.pyi

@@ -17,7 +17,7 @@
 #
 # This is automatically generated stub for the `common.sql` provider
 #
-# This file is generated automatically by the `update-common-sql-api stubs` pre-commit
+# This file is generated automatically by the `update-common-sql-api stubs` prek hook
 # and the .pyi file represents part of the "public" API that the
 # `common.sql` provider exposes to other providers.
 #
@@ -28,8 +28,8 @@
 # You can read more in the README_API.md file
 #
 """
-Definition of the public interface for airflow.providers.common.sql.src.airflow.providers.common.sql.dialects.dialect
-isort:skip_file
+Definition of the public interface for
+airflow.providers.common.sql.src.airflow.providers.common.sql.dialects.dialect.
 """
 
 from collections.abc import Callable, Iterable, Mapping

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/get_provider_info.pyi

@@ -17,7 +17,7 @@
 #
 # This is automatically generated stub for the `common.sql` provider
 #
-# This file is generated automatically by the `update-common-sql-api stubs` pre-commit
+# This file is generated automatically by the `update-common-sql-api stubs` prek hook
 # and the .pyi file represents part of the "public" API that the
 # `common.sql` provider exposes to other providers.
 #
@@ -28,8 +28,8 @@
 # You can read more in the README_API.md file
 #
 """
-Definition of the public interface for airflow.providers.common.sql.src.airflow.providers.common.sql.get_provider_info
-isort:skip_file
+Definition of the public interface for
+airflow.providers.common.sql.src.airflow.providers.common.sql.get_provider_info.
 """
 
 def get_provider_info() -> None: ...

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/hooks/handlers.pyi

@@ -17,7 +17,7 @@
 #
 # This is automatically generated stub for the `common.sql` provider
 #
-# This file is generated automatically by the `update-common-sql-api stubs` pre-commit
+# This file is generated automatically by the `update-common-sql-api stubs` prek hook
 # and the .pyi file represents part of the "public" API that the
 # `common.sql` provider exposes to other providers.
 #
@@ -28,8 +28,8 @@
 # You can read more in the README_API.md file
 #
 """
-Definition of the public interface for airflow.providers.common.sql.src.airflow.providers.common.sql.hooks.handlers
-isort:skip_file
+Definition of the public interface for
+airflow.providers.common.sql.src.airflow.providers.common.sql.hooks.handlers.
 """
 
 from collections.abc import Iterable

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/hooks/sql.py

@@ -332,7 +332,7 @@ class DbApiHook(BaseHook):
     def dialect_name(self) -> str:
         try:
             return make_url(self.get_uri()).get_dialect().name
-        except (ArgumentError, NoSuchModuleError):
+        except (ArgumentError, NoSuchModuleError, ValueError):
             config = self.connection_extra
             sqlalchemy_scheme = config.get("sqlalchemy_scheme")
             if sqlalchemy_scheme:
@@ -810,6 +810,9 @@ class DbApiHook(BaseHook):
             self.log.info("Running statement: %s, parameters: %s", sql_statement, parameters)
 
         if parameters:
+            # If we're using psycopg3, we might need to handle parameters differently
+            if hasattr(cur, "__module__") and "psycopg" in cur.__module__ and isinstance(parameters, list):
+                parameters = tuple(parameters)
             cur.execute(sql_statement, parameters)
         else:
             cur.execute(sql_statement)

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/hooks/sql.pyi

@@ -17,7 +17,7 @@
 #
 # This is automatically generated stub for the `common.sql` provider
 #
-# This file is generated automatically by the `update-common-sql-api stubs` pre-commit
+# This file is generated automatically by the `update-common-sql-api stubs` prek hook
 # and the .pyi file represents part of the "public" API that the
 # `common.sql` provider exposes to other providers.
 #
@@ -28,8 +28,8 @@
 # You can read more in the README_API.md file
 #
 """
-Definition of the public interface for airflow.providers.common.sql.src.airflow.providers.common.sql.hooks.sql
-isort:skip_file
+Definition of the public interface for
+airflow.providers.common.sql.src.airflow.providers.common.sql.hooks.sql.
 """
 
 from collections.abc import Callable, Generator, Iterable, Mapping, MutableMapping, Sequence

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/operators/generic_transfer.py

@@ -56,6 +56,7 @@ class GenericTransfer(BaseOperator):
         executed prior to loading the data. (templated)
     :param insert_args: extra params for `insert_rows` method.
     :param page_size: number of records to be read in paginated mode (optional).
+    :param paginated_sql_statement_clause: SQL statement clause to be used for pagination (optional).
     """
 
     template_fields: Sequence[str] = (
@@ -65,6 +66,8 @@ class GenericTransfer(BaseOperator):
         "destination_table",
         "preoperator",
         "insert_args",
+        "page_size",
+        "paginated_sql_statement_clause",
     )
     template_ext: Sequence[str] = (
         ".sql",
@@ -85,6 +88,7 @@ class GenericTransfer(BaseOperator):
         preoperator: str | list[str] | None = None,
         insert_args: dict | None = None,
         page_size: int | None = None,
+        paginated_sql_statement_clause: str | None = None,
         **kwargs,
     ) -> None:
         super().__init__(**kwargs)
@@ -97,9 +101,7 @@ class GenericTransfer(BaseOperator):
         self.preoperator = preoperator
         self.insert_args = insert_args or {}
         self.page_size = page_size
-        self._paginated_sql_statement_format = kwargs.get(
-            "paginated_sql_statement_format", "{} LIMIT {} OFFSET {}"
-        )
+        self.paginated_sql_statement_clause = paginated_sql_statement_clause or "{} LIMIT {} OFFSET {}"
 
     @classmethod
     def get_hook(cls, conn_id: str, hook_params: dict | None = None) -> DbApiHook:
@@ -126,7 +128,7 @@ class GenericTransfer(BaseOperator):
 
     def get_paginated_sql(self, offset: int) -> str:
         """Format the paginated SQL statement using the current format."""
-        return self._paginated_sql_statement_format.format(self.sql, self.page_size, offset)
+        return self.paginated_sql_statement_clause.format(self.sql, self.page_size, offset)
 
     def render_template_fields(
         self,

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/operators/generic_transfer.pyi

@@ -17,7 +17,7 @@
 #
 # This is automatically generated stub for the `common.sql` provider
 #
-# This file is generated automatically by the `update-common-sql-api stubs` pre-commit
+# This file is generated automatically by the `update-common-sql-api stubs` prek hook
 # and the .pyi file represents part of the "public" API that the
 # `common.sql` provider exposes to other providers.
 #
@@ -27,10 +27,7 @@
 #
 # You can read more in the README_API.md file
 #
-"""
-Definition of the public interface for airflow.providers.common.sql.operators.generic_transfer
-isort:skip_file
-"""
+"""Definition of the public interface for airflow.providers.common.sql.operators.generic_transfer."""
 
 from collections.abc import Sequence
 from functools import cached_property as cached_property

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/operators/sql.py

@@ -23,7 +23,8 @@ from collections.abc import Callable, Iterable, Mapping, Sequence
 from functools import cached_property
 from typing import TYPE_CHECKING, Any, ClassVar, NoReturn, SupportsAbs
 
-from airflow.exceptions import AirflowException, AirflowFailException
+from airflow import XComArg
+from airflow.exceptions import AirflowException, AirflowFailException, AirflowSkipException
 from airflow.models import SkipMixin
 from airflow.providers.common.sql.hooks.handlers import fetch_all_handler, return_single_query_results
 from airflow.providers.common.sql.hooks.sql import DbApiHook
@@ -31,6 +32,8 @@ from airflow.providers.common.sql.version_compat import BaseHook, BaseOperator
 from airflow.utils.helpers import merge_dicts
 
 if TYPE_CHECKING:
+    import jinja2
+
     from airflow.providers.openlineage.extractors import OperatorLineage
     from airflow.utils.context import Context
 
@@ -1252,6 +1255,135 @@ class BranchSQLOperator(BaseSQLOperator, SkipMixin):
         self.skip_all_except(context["ti"], follow_branch)
 
 
+class SQLInsertRowsOperator(BaseSQLOperator):
+    """
+    Insert rows (e.g. a collection of tuples) into a database table directly from an XCom or Python data structure.
+
+    :param table: the name of the table in which the rows will be inserted (templated).
+    :param conn_id: the connection ID used to connect to the database
+    :param schema: (optional) the name of schema in which the table is defined
+    :param database: name of database (e.g. schema) which overwrite the defined one in connection
+    :param columns: (optional) specify a list of columns being used for the insert when passing a list of
+        dictionaries.
+    :param ignore_columns: (optional) specify a list of columns being ignored for the insert.  If no columns
+        where specified, the columns will be resolved dynamically from the metadata.
+    :param rows: the rows to insert into the table. Rows can be a list of tuples or a list of dictionaries.
+        When a list of dictionaries is provided, the column names are inferred from the dictionary keys and
+        will be matched with the column names, ignored columns will be filtered out.
+    :rows_processor: (optional) a function that will be applied to the rows before inserting them into the table.
+    :param preoperator: sql statement or list of statements to be executed prior to loading the data. (templated)
+    :param postoperator: sql statement or list of statements to be executed after loading the data. (templated)
+    :param insert_args: (optional) dictionary of additional arguments passed to the underlying hook's
+        `insert_rows` method. This allows you to configure options such as `replace`, `executemany`,
+        `fast_executemany`, and `autocommit`.
+
+    .. seealso::
+        For more information on how to use this operator, take a look at the guide:
+        :ref:`howto/operator:SQLInsertRowsOperator`
+    """
+
+    template_fields: Sequence[str] = (
+        "table_name",
+        "conn_id",
+        "schema",
+        "database",
+        "_columns",
+        "ignored_columns",
+        "preoperator",
+        "postoperator",
+        "insert_args",
+    )
+    template_ext: Sequence[str] = (".sql",)
+    template_fields_renderers = {"preoperator": "sql"}
+
+    def __init__(
+        self,
+        *,
+        table_name: str,
+        conn_id: str | None = None,
+        schema: str | None = None,
+        database: str | None = None,
+        columns: Iterable[str] | None = None,
+        ignored_columns: Iterable[str] | None = None,
+        rows: list[Any] | XComArg | None = None,
+        rows_processor: Callable[[Any, Context], Any] = lambda rows, **context: rows,
+        preoperator: str | list[str] | None = None,
+        postoperator: str | list[str] | None = None,
+        hook_params: dict | None = None,
+        insert_args: dict | None = None,
+        **kwargs,
+    ):
+        super().__init__(
+            conn_id=conn_id,
+            database=database,
+            hook_params=hook_params,
+            **kwargs,
+        )
+        self.table_name = table_name
+        self.schema = schema
+        self._columns: list | None = list(columns) if columns else None
+        self.ignored_columns = set(ignored_columns or {})
+        self.rows = rows or []
+        self._rows_processor = rows_processor
+        self.preoperator = preoperator
+        self.postoperator = postoperator
+        self.insert_args = insert_args or {}
+        self.do_xcom_push = False
+
+    def render_template_fields(
+        self,
+        context: Context,
+        jinja_env: jinja2.Environment | None = None,
+    ) -> None:
+        super().render_template_fields(context=context, jinja_env=jinja_env)
+
+        if isinstance(self.rows, XComArg):
+            self.rows = self.rows.resolve(context=context)
+
+    @property
+    def table_name_with_schema(self) -> str:
+        if self.schema is not None:
+            return f"{self.schema}.{self.table_name}"
+        return self.table_name
+
+    @cached_property
+    def columns(self):
+        if self._columns is None:
+            self._columns = self.get_db_hook().dialect.get_column_names(self.table_name_with_schema)
+        return self._columns
+
+    @property
+    def column_names(self) -> list[str]:
+        if self.ignored_columns:
+            return [column for column in self.columns if column not in self.ignored_columns]
+        return self.columns
+
+    def _process_rows(self, context: Context):
+        return self._rows_processor(context, self.rows)  # type: ignore
+
+    def execute(self, context: Context) -> Any:
+        if not self.rows:
+            raise AirflowSkipException(f"Skipping task {self.task_id} because no rows.")
+
+        self.log.debug("Table: %s", self.table_name_with_schema)
+        self.log.debug("Column names: %s", self.column_names)
+        if self.preoperator:
+            self.log.debug("Running preoperator")
+            self.log.debug(self.preoperator)
+            self.get_db_hook().run(self.preoperator)
+        rows = self._process_rows(context=context)
+        self.get_db_hook().insert_rows(
+            table=self.table_name_with_schema,
+            rows=rows,
+            target_fields=self.column_names,
+            **self.insert_args,
+        )
+        if self.postoperator:
+            self.log.debug("Running postoperator")
+            self.log.debug(self.postoperator)
+            self.get_db_hook().run(self.postoperator)
+
+
 def _initialize_partition_clause(clause: str | None) -> str | None:
     """Ensure the partition_clause contains only valid patterns."""
     if clause is None:

{apache_airflow_providers_common_sql-1.27.5 → apache_airflow_providers_common_sql-1.28.0}/src/airflow/providers/common/sql/sensors/sql.pyi

@@ -17,7 +17,7 @@
 #
 # This is automatically generated stub for the `common.sql` provider
 #
-# This file is generated automatically by the `update-common-sql-api stubs` pre-commit
+# This file is generated automatically by the `update-common-sql-api stubs` prek hook
 # and the .pyi file represents part of the "public" API that the
 # `common.sql` provider exposes to other providers.
 #
@@ -28,8 +28,8 @@
 # You can read more in the README_API.md file
 #
 """
-Definition of the public interface for airflow.providers.common.sql.src.airflow.providers.common.sql.sensors.sql
-isort:skip_file
+Definition of the public interface for
+airflow.providers.common.sql.src.airflow.providers.common.sql.sensors.sql.
 """
 
 from collections.abc import Callable, Mapping, Sequence