apache-airflow-providers-databricks 7.8.1rc1__tar.gz → 7.8.3rc1__tar.gz

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/NOTICE

@@ -1,5 +1,5 @@
 Apache Airflow
-Copyright 2016-2025 The Apache Software Foundation
+Copyright 2016-2026 The Apache Software Foundation
 
 This product includes software developed at
 The Apache Software Foundation (http://www.apache.org/).

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-databricks
-Version: 7.8.1rc1
+Version: 7.8.3rc1
 Summary: Provider package apache-airflow-providers-databricks for Apache Airflow
 Keywords: airflow-provider,databricks,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -23,11 +23,10 @@ Classifier: Topic :: System :: Monitoring
 License-File: LICENSE
 License-File: NOTICE
 Requires-Dist: apache-airflow>=2.11.0rc1
-Requires-Dist: apache-airflow-providers-common-compat>=1.10.1rc1
+Requires-Dist: apache-airflow-providers-common-compat>=1.12.0rc1
 Requires-Dist: apache-airflow-providers-common-sql>=1.27.0rc1
 Requires-Dist: requests>=2.32.0,<3
 Requires-Dist: databricks-sql-connector>=4.0.0
-Requires-Dist: databricks-sqlalchemy>=1.0.2
 Requires-Dist: aiohttp>=3.9.2, <4
 Requires-Dist: mergedeep>=1.3.4
 Requires-Dist: pandas>=2.1.2; python_version <"3.13"
@@ -38,10 +37,11 @@ Requires-Dist: azure-identity>=1.3.1 ; extra == "azure-identity"
 Requires-Dist: apache-airflow-providers-fab>=2.2.0rc1 ; extra == "fab" and ( python_version < '3.13')
 Requires-Dist: apache-airflow-providers-openlineage>=2.3.0rc1 ; extra == "openlineage"
 Requires-Dist: databricks-sdk==0.10.0 ; extra == "sdk"
+Requires-Dist: databricks-sqlalchemy>=1.0.2 ; extra == "sqlalchemy"
 Requires-Dist: apache-airflow-providers-standard ; extra == "standard"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.1/changelog.html
-Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.1
+Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.3/changelog.html
+Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.3
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -50,6 +50,7 @@ Provides-Extra: azure-identity
 Provides-Extra: fab
 Provides-Extra: openlineage
 Provides-Extra: sdk
+Provides-Extra: sqlalchemy
 Provides-Extra: standard
 
 
@@ -77,7 +78,7 @@ Provides-Extra: standard
 
 Package ``apache-airflow-providers-databricks``
 
-Release: ``7.8.1``
+Release: ``7.8.3``
 
 
 `Databricks <https://databricks.com/>`__
@@ -90,7 +91,7 @@ This is a provider package for ``databricks`` provider. All classes for this pro
 are in ``airflow.providers.databricks`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.1/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.3/>`_.
 
 Installation
 ------------
@@ -112,7 +113,6 @@ PIP package                                 Version required
 ``apache-airflow-providers-common-sql``     ``>=1.27.0``
 ``requests``                                ``>=2.32.0,<3``
 ``databricks-sql-connector``                ``>=4.0.0``
-``databricks-sqlalchemy``                   ``>=1.0.2``
 ``aiohttp``                                 ``>=3.9.2,<4``
 ``mergedeep``                               ``>=1.3.4``
 ``pandas``                                  ``>=2.1.2; python_version < "3.13"``
@@ -153,8 +153,9 @@ Extra               Dependencies
 ``fab``             ``apache-airflow-providers-fab>=2.2.0; python_version < '3.13'``
 ``standard``        ``apache-airflow-providers-standard``
 ``openlineage``     ``apache-airflow-providers-openlineage>=2.3.0``
+``sqlalchemy``      ``databricks-sqlalchemy>=1.0.2``
 ==================  ================================================================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.1/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.3/changelog.html>`_.
 

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-databricks``
 
-Release: ``7.8.1``
+Release: ``7.8.3``
 
 
 `Databricks <https://databricks.com/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``databricks`` provider. All classes for this pro
 are in ``airflow.providers.databricks`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.1/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.3/>`_.
 
 Installation
 ------------
@@ -58,7 +58,6 @@ PIP package                                 Version required
 ``apache-airflow-providers-common-sql``     ``>=1.27.0``
 ``requests``                                ``>=2.32.0,<3``
 ``databricks-sql-connector``                ``>=4.0.0``
-``databricks-sqlalchemy``                   ``>=1.0.2``
 ``aiohttp``                                 ``>=3.9.2,<4``
 ``mergedeep``                               ``>=1.3.4``
 ``pandas``                                  ``>=2.1.2; python_version < "3.13"``
@@ -99,7 +98,8 @@ Extra               Dependencies
 ``fab``             ``apache-airflow-providers-fab>=2.2.0; python_version < '3.13'``
 ``standard``        ``apache-airflow-providers-standard``
 ``openlineage``     ``apache-airflow-providers-openlineage>=2.3.0``
+``sqlalchemy``      ``databricks-sqlalchemy>=1.0.2``
 ==================  ================================================================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.1/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-databricks/7.8.3/changelog.html>`_.

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/docs/changelog.rst

@@ -26,6 +26,39 @@
 Changelog
 ---------
 
+7.8.3
+.....
+
+Misc
+~~~~
+
+* ``Consume ''AirflowOptionalProviderFeatureException'' from compat sdk in providers (#60335)``
+* ``Limit deltalake again to 1.3.1 due to missing ARM .whl files (#60376)``
+* ``New year means updated Copyright notices (#60344)``
+* ``Made sqlalchemy dependency optional for Databricks provider (#60110)``
+* ``Move over plugins_manager to a shared library (#59956)``
+* ``Limit deltalake again to 1.3.0 due to missing ARM .whl files (#60098)``
+* ``Source databricks provider to use airflow.sdk.configuration.conf (#60021)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Revert "Limit deltalake to not include 1.3.0 version (#59977)" (#60005)``
+   * ``Limit deltalake to not include 1.3.0 version (#59977)``
+
+7.8.2
+.....
+
+Misc
+~~~~
+
+* ``'issue-59189:' Updating Databricks provider to point to '2.2/jobs/...' endpoint (#59217)``
+* ``Remove top-level SDK reference in Core (#59817)``
+* ``Refactor/sqla2 providers(celery, kubernetes, databricks, mysql) to remove SQLA query usage (#59537)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``TaskInstance unused method cleanup (#59835)``
+
 7.8.1
 .....
 

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/docs/index.rst

@@ -78,7 +78,7 @@ apache-airflow-providers-databricks package
 `Databricks <https://databricks.com/>`__
 
 
-Release: 7.8.1
+Release: 7.8.3
 
 Provider package
 ----------------
@@ -102,11 +102,10 @@ The minimum Apache Airflow version supported by this provider distribution is ``
 PIP package                                 Version required
 ==========================================  ======================================
 ``apache-airflow``                          ``>=2.11.0``
-``apache-airflow-providers-common-compat``  ``>=1.10.1``
+``apache-airflow-providers-common-compat``  ``>=1.12.0``
 ``apache-airflow-providers-common-sql``     ``>=1.27.0``
 ``requests``                                ``>=2.32.0,<3``
 ``databricks-sql-connector``                ``>=4.0.0``
-``databricks-sqlalchemy``                   ``>=1.0.2``
 ``aiohttp``                                 ``>=3.9.2,<4``
 ``mergedeep``                               ``>=1.3.4``
 ``pandas``                                  ``>=2.1.2; python_version < "3.13"``
@@ -142,5 +141,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-databricks 7.8.1 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.1.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.1.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.1.tar.gz.sha512>`__)
-* `The apache-airflow-providers-databricks 7.8.1 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.1-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.1-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.1-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-databricks 7.8.3 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.3.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.3.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.3.tar.gz.sha512>`__)
+* `The apache-airflow-providers-databricks 7.8.3 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.3-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.3-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_databricks-7.8.3-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/docs/operators/jobs_create.rst

@@ -31,12 +31,12 @@ Using the Operator
 ------------------
 
 There are three ways to instantiate this operator. In the first way, you can take the JSON payload that you typically use
-to call the ``api/2.1/jobs/create`` endpoint and pass it directly to our ``DatabricksCreateJobsOperator`` through the
+to call the ``api/2.2/jobs/create`` endpoint and pass it directly to our ``DatabricksCreateJobsOperator`` through the
 ``json`` parameter.  With this approach you get full control over the underlying payload to Jobs REST API, including
 execution of Databricks jobs with multiple tasks, but it's harder to detect errors because of the lack of the type checking.
 
 The second way to accomplish the same thing is to use the named parameters of the ``DatabricksCreateJobsOperator`` directly. Note that there is exactly
-one named parameter for each top level parameter in the ``api/2.1/jobs/create`` endpoint.
+one named parameter for each top level parameter in the ``api/2.2/jobs/create`` endpoint.
 
 The third way is to use both the json parameter **AND** the named parameters. They will be merged
 together. If there are conflicts during the merge, the named parameters will take precedence and

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/docs/operators/run_now.rst

@@ -21,14 +21,14 @@ DatabricksRunNowOperator
 ========================
 
 Use the :class:`~airflow.providers.databricks.operators.DatabricksRunNowOperator` to trigger a run of an existing Databricks job
-via `api/2.1/jobs/run-now <https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunNow>`_ API endpoint.
+via `api/2.2/jobs/run-now <https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunNow>`_ API endpoint.
 
 
 Using the Operator
 ^^^^^^^^^^^^^^^^^^
 
 There are two ways to instantiate this operator. In the first way, you can take the JSON payload that you typically use
-to call the ``api/2.1/jobs/run-now`` endpoint and pass it directly to our ``DatabricksRunNowOperator`` through the ``json`` parameter.
+to call the ``api/2.2/jobs/run-now`` endpoint and pass it directly to our ``DatabricksRunNowOperator`` through the ``json`` parameter.
 
 Another way to accomplish the same thing is to use the named parameters of the ``DatabricksRunNowOperator`` directly.
 Note that there is exactly one named parameter for each top level parameter in the ``jobs/run-now`` endpoint.

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/docs/operators/submit_run.rst

@@ -24,14 +24,14 @@ DatabricksSubmitRunOperator
 ===========================
 
 Use the :class:`~airflow.providers.databricks.operators.DatabricksSubmitRunOperator` to submit
-a new Databricks job via Databricks `api/2.1/jobs/runs/submit <https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsSubmit>`_ API endpoint.
+a new Databricks job via Databricks `api/2.2/jobs/runs/submit <https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsSubmit>`_ API endpoint.
 
 
 Using the Operator
 ------------------
 
 There are three ways to instantiate this operator. In the first way, you can take the JSON payload that you typically use
-to call the ``api/2.1/jobs/runs/submit`` endpoint and pass it directly to our ``DatabricksSubmitRunOperator`` through the
+to call the ``api/2.2/jobs/runs/submit`` endpoint and pass it directly to our ``DatabricksSubmitRunOperator`` through the
 ``json`` parameter.  With this approach you get full control over the underlying payload to Jobs REST API, including
 execution of Databricks jobs with multiple tasks, but it's harder to detect errors because of the lack of the type checking.
 
@@ -91,7 +91,7 @@ Currently the named parameters that ``DatabricksSubmitRunOperator`` supports are
       task_id="notebook_run", new_cluster=new_cluster, notebook_task=notebook_task
   )
 
-Another way to do is use the param tasks to pass array of objects to instantiate this operator. Here the value of tasks param that is used to invoke ``api/2.1/jobs/runs/submit`` endpoint is passed through the ``tasks`` param in ``DatabricksSubmitRunOperator``. Instead of invoking single task, you can pass array of task and submit a one-time run.
+Another way to do is use the param tasks to pass array of objects to instantiate this operator. Here the value of tasks param that is used to invoke ``api/2.2/jobs/runs/submit`` endpoint is passed through the ``tasks`` param in ``DatabricksSubmitRunOperator``. Instead of invoking single task, you can pass array of task and submit a one-time run.
 
 .. code-block:: python
 

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/provider.yaml

@@ -22,12 +22,14 @@ description: |
     `Databricks <https://databricks.com/>`__
 
 state: ready
-source-date-epoch: 1765298937
+source-date-epoch: 1768334769
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 7.8.3
+  - 7.8.2
   - 7.8.1
   - 7.8.0
   - 7.7.5

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-databricks"
-version = "7.8.1rc1"
+version = "7.8.3rc1"
 description = "Provider package apache-airflow-providers-databricks for Apache Airflow"
 readme = "README.rst"
 license = "Apache-2.0"
@@ -59,11 +59,10 @@ requires-python = ">=3.10"
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
     "apache-airflow>=2.11.0rc1",
-    "apache-airflow-providers-common-compat>=1.10.1rc1",
+    "apache-airflow-providers-common-compat>=1.12.0rc1",
     "apache-airflow-providers-common-sql>=1.27.0rc1",
     "requests>=2.32.0,<3",
     "databricks-sql-connector>=4.0.0",
-    "databricks-sqlalchemy>=1.0.2",
     "aiohttp>=3.9.2, <4",
     "mergedeep>=1.3.4",
     'pandas>=2.1.2; python_version <"3.13"',
@@ -91,6 +90,9 @@ dependencies = [
 "openlineage" = [
     "apache-airflow-providers-openlineage>=2.3.0rc1"
 ]
+"sqlalchemy" = [
+    "databricks-sqlalchemy>=1.0.2",
+]
 
 [dependency-groups]
 dev = [
@@ -101,11 +103,13 @@ dev = [
     "apache-airflow-providers-common-sql",
     "apache-airflow-providers-openlineage",
     # Additional devel dependencies (do not remove this line and add extra development dependencies)
-    "deltalake>=1.1.3",
+    # Limit deltalake to avoid issue with missing linux ARM wheels: https://github.com/delta-io/delta-rs/issues/4041
+    "deltalake>=1.1.3,!=1.3.0,!=1.3.1",
     "apache-airflow-providers-fab>=2.2.0; python_version < '3.13'",
     "apache-airflow-providers-microsoft-azure",
     "apache-airflow-providers-common-sql[pandas,polars]",
     "apache-airflow-providers-fab",
+    "apache-airflow-providers-databricks[sqlalchemy]",
 ]
 
 # To build docs:
@@ -134,8 +138,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.1"
-"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.1/changelog.html"
+"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.3"
+"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-databricks/7.8.3/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/src/airflow/providers/databricks/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "7.8.1"
+__version__ = "7.8.3"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.11.0"

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/src/airflow/providers/databricks/hooks/databricks.py

@@ -20,9 +20,9 @@ Databricks hook.
 
 This hook enable the submitting and running of jobs to the Databricks platform. Internally the
 operators talk to the
-``api/2.1/jobs/run-now``
+``api/2.2/jobs/run-now``
 `endpoint <https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunNow>_`
-or the ``api/2.1/jobs/runs/submit``
+or the ``api/2.2/jobs/runs/submit``
 `endpoint <https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsSubmit>`_.
 """
 
@@ -37,37 +37,37 @@ from requests import exceptions as requests_exceptions
 from airflow.providers.common.compat.sdk import AirflowException
 from airflow.providers.databricks.hooks.databricks_base import BaseDatabricksHook
 
-GET_CLUSTER_ENDPOINT = ("GET", "2.0/clusters/get")
-RESTART_CLUSTER_ENDPOINT = ("POST", "2.0/clusters/restart")
-START_CLUSTER_ENDPOINT = ("POST", "2.0/clusters/start")
-TERMINATE_CLUSTER_ENDPOINT = ("POST", "2.0/clusters/delete")
+GET_CLUSTER_ENDPOINT = ("GET", "2.2/clusters/get")
+RESTART_CLUSTER_ENDPOINT = ("POST", "2.2/clusters/restart")
+START_CLUSTER_ENDPOINT = ("POST", "2.2/clusters/start")
+TERMINATE_CLUSTER_ENDPOINT = ("POST", "2.2/clusters/delete")
 
-CREATE_ENDPOINT = ("POST", "2.1/jobs/create")
-RESET_ENDPOINT = ("POST", "2.1/jobs/reset")
-UPDATE_ENDPOINT = ("POST", "2.1/jobs/update")
-RUN_NOW_ENDPOINT = ("POST", "2.1/jobs/run-now")
-SUBMIT_RUN_ENDPOINT = ("POST", "2.1/jobs/runs/submit")
-GET_RUN_ENDPOINT = ("GET", "2.1/jobs/runs/get")
-CANCEL_RUN_ENDPOINT = ("POST", "2.1/jobs/runs/cancel")
-DELETE_RUN_ENDPOINT = ("POST", "2.1/jobs/runs/delete")
-REPAIR_RUN_ENDPOINT = ("POST", "2.1/jobs/runs/repair")
-OUTPUT_RUNS_JOB_ENDPOINT = ("GET", "2.1/jobs/runs/get-output")
-CANCEL_ALL_RUNS_ENDPOINT = ("POST", "2.1/jobs/runs/cancel-all")
+CREATE_ENDPOINT = ("POST", "2.2/jobs/create")
+RESET_ENDPOINT = ("POST", "2.2/jobs/reset")
+UPDATE_ENDPOINT = ("POST", "2.2/jobs/update")
+RUN_NOW_ENDPOINT = ("POST", "2.2/jobs/run-now")
+SUBMIT_RUN_ENDPOINT = ("POST", "2.2/jobs/runs/submit")
+GET_RUN_ENDPOINT = ("GET", "2.2/jobs/runs/get")
+CANCEL_RUN_ENDPOINT = ("POST", "2.2/jobs/runs/cancel")
+DELETE_RUN_ENDPOINT = ("POST", "2.2/jobs/runs/delete")
+REPAIR_RUN_ENDPOINT = ("POST", "2.2/jobs/runs/repair")
+OUTPUT_RUNS_JOB_ENDPOINT = ("GET", "2.2/jobs/runs/get-output")
+CANCEL_ALL_RUNS_ENDPOINT = ("POST", "2.2/jobs/runs/cancel-all")
 
-INSTALL_LIBS_ENDPOINT = ("POST", "2.0/libraries/install")
-UNINSTALL_LIBS_ENDPOINT = ("POST", "2.0/libraries/uninstall")
-UPDATE_REPO_ENDPOINT = ("PATCH", "2.0/repos/")
-DELETE_REPO_ENDPOINT = ("DELETE", "2.0/repos/")
-CREATE_REPO_ENDPOINT = ("POST", "2.0/repos")
+INSTALL_LIBS_ENDPOINT = ("POST", "2.2/libraries/install")
+UNINSTALL_LIBS_ENDPOINT = ("POST", "2.2/libraries/uninstall")
+UPDATE_REPO_ENDPOINT = ("PATCH", "2.2/repos/")
+DELETE_REPO_ENDPOINT = ("DELETE", "2.2/repos/")
+CREATE_REPO_ENDPOINT = ("POST", "2.2/repos")
 
-LIST_JOBS_ENDPOINT = ("GET", "2.1/jobs/list")
-LIST_PIPELINES_ENDPOINT = ("GET", "2.0/pipelines")
-LIST_SQL_ENDPOINTS_ENDPOINT = ("GET", "2.0/sql/endpoints")
+LIST_JOBS_ENDPOINT = ("GET", "2.2/jobs/list")
+LIST_PIPELINES_ENDPOINT = ("GET", "2.2/pipelines")
+LIST_SQL_ENDPOINTS_ENDPOINT = ("GET", "2.2/sql/endpoints")
 
-WORKSPACE_GET_STATUS_ENDPOINT = ("GET", "2.0/workspace/get-status")
+WORKSPACE_GET_STATUS_ENDPOINT = ("GET", "2.2/workspace/get-status")
 
-SPARK_VERSIONS_ENDPOINT = ("GET", "2.0/clusters/spark-versions")
-SQL_STATEMENTS_ENDPOINT = "2.0/sql/statements"
+SPARK_VERSIONS_ENDPOINT = ("GET", "2.2/clusters/spark-versions")
+SQL_STATEMENTS_ENDPOINT = "2.2/sql/statements"
 
 
 class RunLifeCycleState(Enum):
@@ -293,7 +293,7 @@ class DatabricksHook(BaseDatabricksHook):
 
     def create_job(self, json: dict) -> int:
         """
-        Call the ``api/2.1/jobs/create`` endpoint.
+        Call the ``api/2.2/jobs/create`` endpoint.
 
         :param json: The data used in the body of the request to the ``create`` endpoint.
         :return: the job_id as an int
@@ -303,7 +303,7 @@ class DatabricksHook(BaseDatabricksHook):
 
     def reset_job(self, job_id: str, json: dict) -> None:
         """
-        Call the ``api/2.1/jobs/reset`` endpoint.
+        Call the ``api/2.2/jobs/reset`` endpoint.
 
         :param json: The data used in the new_settings of the request to the ``reset`` endpoint.
         """
@@ -321,7 +321,7 @@ class DatabricksHook(BaseDatabricksHook):
 
     def update_job(self, job_id: str, json: dict) -> None:
         """
-        Call the ``api/2.1/jobs/update`` endpoint.
+        Call the ``api/2.2/jobs/update`` endpoint.
 
         :param job_id: The id of the job to update.
         :param json: The data used in the new_settings of the request to the ``update`` endpoint.
@@ -330,7 +330,7 @@ class DatabricksHook(BaseDatabricksHook):
 
     def run_now(self, json: dict) -> int:
         """
-        Call the ``api/2.1/jobs/run-now`` endpoint.
+        Call the ``api/2.2/jobs/run-now`` endpoint.
 
         :param json: The data used in the body of the request to the ``run-now`` endpoint.
         :return: the run_id as an int
@@ -340,7 +340,7 @@ class DatabricksHook(BaseDatabricksHook):
 
     def submit_run(self, json: dict) -> int:
         """
-        Call the ``api/2.1/jobs/runs/submit`` endpoint.
+        Call the ``api/2.2/jobs/runs/submit`` endpoint.
 
         :param json: The data used in the body of the request to the ``submit`` endpoint.
         :return: the run_id as an int
@@ -385,9 +385,9 @@ class DatabricksHook(BaseDatabricksHook):
                 all_jobs += [j for j in jobs if j["settings"]["name"] == job_name]
             else:
                 all_jobs += jobs
-            has_more = response.get("has_more", False)
-            if has_more:
-                page_token = response.get("next_page_token", "")
+            # issue-59189: API v2.2 removes "has_more" field
+            page_token = response.get("next_page_token", "")
+            has_more = bool(page_token)
 
         return all_jobs
 
@@ -717,7 +717,7 @@ class DatabricksHook(BaseDatabricksHook):
         """
         Install libraries on the cluster.
 
-        Utility function to call the ``2.0/libraries/install`` endpoint.
+        Utility function to call the ``2.2/libraries/install`` endpoint.
 
         :param json: json dictionary containing cluster_id and an array of library
         """
@@ -727,7 +727,7 @@ class DatabricksHook(BaseDatabricksHook):
         """
         Uninstall libraries on the cluster.
 
-        Utility function to call the ``2.0/libraries/uninstall`` endpoint.
+        Utility function to call the ``2.2/libraries/uninstall`` endpoint.
 
         :param json: json dictionary containing cluster_id and an array of library
         """
@@ -790,7 +790,7 @@ class DatabricksHook(BaseDatabricksHook):
         :param json: payload
         :return: json containing permission specification
         """
-        return self._do_api_call(("PATCH", f"2.0/permissions/jobs/{job_id}"), json)
+        return self._do_api_call(("PATCH", f"2.2/permissions/jobs/{job_id}"), json)
 
     def post_sql_statement(self, json: dict[str, Any]) -> str:
         """

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/src/airflow/providers/databricks/hooks/databricks_base.py

@@ -49,8 +49,7 @@ from tenacity import (
 )
 
 from airflow import __version__
-from airflow.exceptions import AirflowOptionalProviderFeatureException
-from airflow.providers.common.compat.sdk import AirflowException
+from airflow.providers.common.compat.sdk import AirflowException, AirflowOptionalProviderFeatureException
 from airflow.providers_manager import ProvidersManager
 
 try:

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/src/airflow/providers/databricks/hooks/databricks_sql.py

@@ -32,9 +32,8 @@ from typing import (
 
 from databricks import sql
 from databricks.sql.types import Row
-from sqlalchemy.engine import URL
 
-from airflow.providers.common.compat.sdk import AirflowException
+from airflow.providers.common.compat.sdk import AirflowException, AirflowOptionalProviderFeatureException
 from airflow.providers.common.sql.hooks.handlers import return_single_query_results
 from airflow.providers.common.sql.hooks.sql import DbApiHook
 from airflow.providers.databricks.exceptions import DatabricksSqlExecutionError, DatabricksSqlExecutionTimeout
@@ -43,6 +42,7 @@ from airflow.providers.databricks.hooks.databricks_base import BaseDatabricksHoo
 
 if TYPE_CHECKING:
     from databricks.sql.client import Connection
+    from sqlalchemy.engine import URL
 
     from airflow.models.connection import Connection as AirflowConnection
     from airflow.providers.openlineage.extractors import OperatorLineage
@@ -179,6 +179,14 @@ class DatabricksSqlHook(BaseDatabricksHook, DbApiHook):
 
         :return: the extracted sqlalchemy.engine.URL object.
         """
+        try:
+            from sqlalchemy.engine import URL
+        except ImportError:
+            raise AirflowOptionalProviderFeatureException(
+                "sqlalchemy is required to generate the connection URL. "
+                "Install it with: pip install 'apache-airflow-providers-databricks[sqlalchemy]'"
+            )
+
         url_query = {
             "http_path": self._http_path,
             "catalog": self.catalog,

{apache_airflow_providers_databricks-7.8.1rc1 → apache_airflow_providers_databricks-7.8.3rc1}/src/airflow/providers/databricks/operators/databricks.py

@@ -26,8 +26,7 @@ from collections.abc import Sequence
 from functools import cached_property
 from typing import TYPE_CHECKING, Any
 
-from airflow.configuration import conf
-from airflow.providers.common.compat.sdk import AirflowException, BaseOperator, BaseOperatorLink, XCom
+from airflow.providers.common.compat.sdk import AirflowException, BaseOperator, BaseOperatorLink, XCom, conf
 from airflow.providers.databricks.hooks.databricks import (
     DatabricksHook,
     RunLifeCycleState,
@@ -260,7 +259,7 @@ class DatabricksCreateJobsOperator(BaseOperator):
         https://docs.databricks.com/api/workspace/jobs/reset
 
     :param json: A JSON object containing API parameters which will be passed
-        directly to the ``api/2.1/jobs/create`` endpoint. The other named parameters
+        directly to the ``api/2.2/jobs/create`` endpoint. The other named parameters
         (i.e. ``name``, ``tags``, ``tasks``, etc.) to this operator will
         be merged with this json dictionary if they are provided.
         If there are conflicts during the merge, the named parameters will
@@ -391,7 +390,7 @@ class DatabricksCreateJobsOperator(BaseOperator):
 
 class DatabricksSubmitRunOperator(BaseOperator):
     """
-    Submits a Spark job run to Databricks using the api/2.1/jobs/runs/submit API endpoint.
+    Submits a Spark job run to Databricks using the api/2.2/jobs/runs/submit API endpoint.
 
     See: https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsSubmit
 
@@ -406,7 +405,7 @@ class DatabricksSubmitRunOperator(BaseOperator):
         .. seealso::
             https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsSubmit
     :param json: A JSON object containing API parameters which will be passed
-        directly to the ``api/2.1/jobs/runs/submit`` endpoint. The other named parameters
+        directly to the ``api/2.2/jobs/runs/submit`` endpoint. The other named parameters
         (i.e. ``spark_jar_task``, ``notebook_task``..) to this operator will
         be merged with this json dictionary if they are provided.
         If there are conflicts during the merge, the named parameters will
@@ -644,14 +643,14 @@ class DatabricksSubmitRunOperator(BaseOperator):
 
 class DatabricksRunNowOperator(BaseOperator):
     """
-    Runs an existing Spark job run to Databricks using the api/2.1/jobs/run-now API endpoint.
+    Runs an existing Spark job run to Databricks using the api/2.2/jobs/run-now API endpoint.
 
     See: https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunNow
 
     There are two ways to instantiate this operator.
 
     In the first way, you can take the JSON payload that you typically use
-    to call the ``api/2.1/jobs/run-now`` endpoint and pass it directly
+    to call the ``api/2.2/jobs/run-now`` endpoint and pass it directly
     to our ``DatabricksRunNowOperator`` through the ``json`` parameter.
     For example ::
 
@@ -729,7 +728,7 @@ class DatabricksRunNowOperator(BaseOperator):
             https://docs.databricks.com/en/workflows/jobs/settings.html#add-parameters-for-all-job-tasks
 
     :param json: A JSON object containing API parameters which will be passed
-        directly to the ``api/2.1/jobs/run-now`` endpoint. The other named parameters
+        directly to the ``api/2.2/jobs/run-now`` endpoint. The other named parameters
         (i.e. ``notebook_params``, ``spark_submit_params``..) to this operator will
         be merged with this json dictionary if they are provided.
         If there are conflicts during the merge, the named parameters will