apache-airflow-providers-openlineage 2.7.1__tar.gz → 2.7.2rc1__tar.gz

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-openlineage
-Version: 2.7.1
+Version: 2.7.2rc1
 Summary: Provider package apache-airflow-providers-openlineage for Apache Airflow
 Keywords: airflow-provider,openlineage,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -20,15 +20,15 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: System :: Monitoring
-Requires-Dist: apache-airflow>=2.10.0
-Requires-Dist: apache-airflow-providers-common-sql>=1.20.0
-Requires-Dist: apache-airflow-providers-common-compat>=1.4.0
+Requires-Dist: apache-airflow>=2.10.0rc1
+Requires-Dist: apache-airflow-providers-common-sql>=1.20.0rc1
+Requires-Dist: apache-airflow-providers-common-compat>=1.4.0rc1
 Requires-Dist: attrs>=22.2
 Requires-Dist: openlineage-integration-common>=1.36.0
 Requires-Dist: openlineage-python>=1.36.0
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1/changelog.html
-Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1
+Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-openlineage/2.7.2/changelog.html
+Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-openlineage/2.7.2
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -59,7 +59,7 @@ Project-URL: YouTube, https://www.youtube.com/channel/UCSXwxpWZQ7XZ1WL3wqevChA/
 
 Package ``apache-airflow-providers-openlineage``
 
-Release: ``2.7.1``
+Release: ``2.7.2``
 
 
 `OpenLineage <https://openlineage.io/>`__
@@ -72,7 +72,7 @@ This is a provider package for ``openlineage`` provider. All classes for this pr
 are in ``airflow.providers.openlineage`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.2/>`_.
 
 Installation
 ------------
@@ -118,5 +118,5 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.2/changelog.html>`_.
 

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-openlineage``
 
-Release: ``2.7.1``
+Release: ``2.7.2``
 
 
 `OpenLineage <https://openlineage.io/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``openlineage`` provider. All classes for this pr
 are in ``airflow.providers.openlineage`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.2/>`_.
 
 Installation
 ------------
@@ -82,4 +82,4 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.2/changelog.html>`_.

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/docs/changelog.rst

@@ -26,6 +26,19 @@
 Changelog
 ---------
 
+2.7.2
+.....
+
+Misc
+~~~~
+
+* ``Adjust OpenLineage utils to be compatible with Airflow 3.1.0 (#56040)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Enable PT011 rule to prvoider tests (#56021)``
+   * ``Move DagBag to airflow/dag_processing (#55139)``
+
 2.7.1
 .....
 

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/docs/index.rst

@@ -81,7 +81,7 @@ apache-airflow-providers-openlineage package
 `OpenLineage <https://openlineage.io/>`__
 
 
-Release: 2.7.1
+Release: 2.7.2
 
 Provider package
 ----------------
@@ -138,5 +138,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-openlineage 2.7.1 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.1.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.1.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.1.tar.gz.sha512>`__)
-* `The apache-airflow-providers-openlineage 2.7.1 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.1-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.1-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.1-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-openlineage 2.7.2 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.2.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.2.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.2.tar.gz.sha512>`__)
+* `The apache-airflow-providers-openlineage 2.7.2 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.2-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.2-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_openlineage-2.7.2-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/provider.yaml

@@ -22,12 +22,13 @@ description: |
   `OpenLineage <https://openlineage.io/>`__
 
 state: ready
-source-date-epoch: 1757950855
+source-date-epoch: 1758787152
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 2.7.2
   - 2.7.1
   - 2.7.0
   - 2.6.1

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-openlineage"
-version = "2.7.1"
+version = "2.7.2rc1"
 description = "Provider package apache-airflow-providers-openlineage for Apache Airflow"
 readme = "README.rst"
 authors = [
@@ -57,9 +57,9 @@ requires-python = ">=3.10"
 # Make sure to run ``prek update-providers-dependencies --all-files``
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
-    "apache-airflow>=2.10.0",
-    "apache-airflow-providers-common-sql>=1.20.0",
-    "apache-airflow-providers-common-compat>=1.4.0",
+    "apache-airflow>=2.10.0rc1",
+    "apache-airflow-providers-common-sql>=1.20.0rc1",
+    "apache-airflow-providers-common-compat>=1.4.0rc1",
     "attrs>=22.2",
     "openlineage-integration-common>=1.36.0",
     "openlineage-python>=1.36.0",
@@ -105,8 +105,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1"
-"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-openlineage/2.7.1/changelog.html"
+"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-openlineage/2.7.2"
+"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-openlineage/2.7.2/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/src/airflow/providers/openlineage/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "2.7.1"
+__version__ = "2.7.2"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.10.0"

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/src/airflow/providers/openlineage/utils/utils.py

@@ -464,14 +464,25 @@ class DagInfo(InfoJsonEncodable):
         "fileloc",
         "owner",
         "owner_links",
-        "schedule_interval",  # For Airflow 2.
-        "timetable_summary",  # For Airflow 3.
+        "schedule_interval",  # For Airflow 2 only -> AF3 has timetable_summary
         "start_date",
         "tags",
     ]
-    casts = {"timetable": lambda dag: DagInfo.serialize_timetable(dag)}
+    casts = {
+        "timetable": lambda dag: DagInfo.serialize_timetable(dag),
+        "timetable_summary": lambda dag: DagInfo.timetable_summary(dag),
+    }
     renames = {"_dag_id": "dag_id"}
 
+    @classmethod
+    def timetable_summary(cls, dag: DAG) -> str | None:
+        """Extract summary from timetable if missing a ``timetable_summary`` property."""
+        if getattr(dag, "timetable_summary", None):
+            return dag.timetable_summary
+        if getattr(dag, "timetable", None):
+            return dag.timetable.summary
+        return None
+
     @classmethod
     def serialize_timetable(cls, dag: DAG) -> dict[str, Any]:
         # This is enough for Airflow 2.10+ and has all the information needed
@@ -797,6 +808,14 @@ def _emits_ol_events(task: AnyOperator) -> bool:
             not getattr(task, "on_success_callback", None),
             not task.outlets,
             not (task.inlets and get_base_airflow_version_tuple() >= (3, 0, 2)),  # Added in 3.0.2 #50773
+            not (
+                getattr(task, "has_on_execute_callback", None)  # Added in 3.1.0 #54569
+                and get_base_airflow_version_tuple() >= (3, 1, 0)
+            ),
+            not (
+                getattr(task, "has_on_success_callback", None)  # Added in 3.1.0 #54569
+                and get_base_airflow_version_tuple() >= (3, 1, 0)
+            ),
         )
     )
 

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/tests/unit/openlineage/plugins/test_execution.py

@@ -28,7 +28,7 @@ import pytest
 
 from airflow.jobs.job import Job
 from airflow.listeners.listener import get_listener_manager
-from airflow.models import DagBag, TaskInstance
+from airflow.models import TaskInstance
 from airflow.providers.google.cloud.openlineage.utils import get_from_nullable_chain
 from airflow.providers.openlineage.plugins.listener import OpenLineageListener
 from airflow.utils import timezone
@@ -37,7 +37,12 @@ from airflow.utils.types import DagRunType
 
 from tests_common.test_utils.config import conf_vars
 from tests_common.test_utils.db import clear_db_runs
-from tests_common.test_utils.version_compat import AIRFLOW_V_3_0_PLUS
+from tests_common.test_utils.version_compat import AIRFLOW_V_3_0_PLUS, AIRFLOW_V_3_1_PLUS
+
+if AIRFLOW_V_3_1_PLUS:
+    from airflow.dag_processing.dagbag import DagBag
+else:
+    from airflow.models.dagbag import DagBag  # type: ignore[attr-defined, no-redef]
 
 TEST_DAG_FOLDER = os.path.join(os.path.dirname(os.path.dirname(__file__)), "dags")
 DEFAULT_DATE = timezone.datetime(2016, 1, 1)

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/tests/unit/openlineage/test_conf.py

@@ -320,7 +320,7 @@ def test_transport_valid():
 @pytest.mark.parametrize("transport_value", ('["a", "b"]', "[]", '[{"a": "b"}]'))
 def test_transport_not_valid(transport_value):
     with conf_vars({(_CONFIG_SECTION, _CONFIG_OPTION_TRANSPORT): transport_value}):
-        with pytest.raises(ValueError):
+        with pytest.raises(ValueError, match="OpenLineage transport"):
             transport()
 
 

{apache_airflow_providers_openlineage-2.7.1 → apache_airflow_providers_openlineage-2.7.2rc1}/tests/unit/openlineage/utils/test_utils.py

@@ -182,14 +182,13 @@ def test_get_airflow_dag_run_facet():
         "fileloc": pathlib.Path(__file__).resolve().as_posix(),
         "owner": "airflow",
         "timetable": {},
+        "timetable_summary": "@once",
         "start_date": "2024-06-01T00:00:00+00:00",
         "tags": "['test']",
         "owner_links": {},
     }
     if hasattr(dag, "schedule_interval"):  # Airflow 2 compat.
         expected_dag_info["schedule_interval"] = "@once"
-    else:  # Airflow 3 and up.
-        expected_dag_info["timetable_summary"] = "@once"
     assert result == {
         "airflowDagRun": AirflowDagRunFacet(
             dag=expected_dag_info,
@@ -1055,7 +1054,21 @@ def test_get_user_provided_run_facets_with_exception(mock_custom_facet_funcs):
     assert result == {}
 
 
-@pytest.mark.skipif(AIRFLOW_V_3_0_PLUS, reason="Airflow 2.9+ tests")
+def test_daginfo_timetable_summary():
+    from airflow.timetables.simple import NullTimetable
+
+    dag = MagicMock()
+    # timetable is enough to get summary
+    dag.timetable = NullTimetable()
+    dag.timetable_summary = None
+    assert DagInfo(dag).timetable_summary == "None"
+
+    # but if summary is present, it's preferred
+    dag.timetable_summary = "explicit_summary"
+    assert DagInfo(dag).timetable_summary == "explicit_summary"
+
+
+@pytest.mark.skipif(AIRFLOW_V_3_0_PLUS, reason="Airflow 2 tests")
 class TestDagInfoAirflow2:
     def test_dag_info(self):
         with DAG(
@@ -1080,6 +1093,7 @@ class TestDagInfoAirflow2:
             "start_date": "2024-06-01T00:00:00+00:00",
             "tags": "['test']",
             "timetable": {},
+            "timetable_summary": "@once",
             "owner_links": {"some_owner": "https://airflow.apache.org"},
         }
 
@@ -1100,6 +1114,7 @@ class TestDagInfoAirflow2:
             "start_date": "2024-06-01T00:00:00+00:00",
             "tags": "[]",
             "timetable": {"expression": "*/4 3 * * *", "timezone": "UTC"},
+            "timetable_summary": "*/4 3 * * *",
             "owner_links": {},
         }
 
@@ -1124,6 +1139,7 @@ class TestDagInfoAirflow2:
             "fileloc": pathlib.Path(__file__).resolve().as_posix(),
             "owner": "",
             "schedule_interval": "My Team's Baseball Games",
+            "timetable_summary": "My Team's Baseball Games",
             "start_date": "2024-06-01T00:00:00+00:00",
             "tags": "[]",
             "owner_links": {},
@@ -1151,6 +1167,7 @@ class TestDagInfoAirflow2:
             "fileloc": pathlib.Path(__file__).resolve().as_posix(),
             "owner": "",
             "schedule_interval": "Dataset",
+            "timetable_summary": "Dataset",
             "start_date": "2024-06-01T00:00:00+00:00",
             "tags": "[]",
             "owner_links": {},
@@ -1176,6 +1193,7 @@ class TestDagInfoAirflow2:
             "fileloc": pathlib.Path(__file__).resolve().as_posix(),
             "owner": "",
             "schedule_interval": "Dataset",
+            "timetable_summary": "Dataset",
             "start_date": "2024-06-01T00:00:00+00:00",
             "tags": "[]",
             "owner_links": {},
@@ -1204,6 +1222,7 @@ class TestDagInfoAirflow2:
             "fileloc": pathlib.Path(__file__).resolve().as_posix(),
             "owner": "",
             "schedule_interval": "Dataset",
+            "timetable_summary": "Dataset",
             "start_date": "2024-06-01T00:00:00+00:00",
             "tags": "[]",
             "owner_links": {},
@@ -1250,6 +1269,7 @@ class TestDagInfoAirflow2:
             "fileloc": pathlib.Path(__file__).resolve().as_posix(),
             "owner": "",
             "schedule_interval": "Dataset or */4 3 * * *",
+            "timetable_summary": "Dataset or */4 3 * * *",
             "start_date": "2024-06-01T00:00:00+00:00",
             "tags": "[]",
             "owner_links": {},
@@ -1301,6 +1321,7 @@ class TestDagInfoAirflow210:
             "owner_links": {},
             "timetable": {"dataset_condition": {"__type": "dataset", "uri": "uri1", "extra": {"a": 1}}},
             "schedule_interval": "Dataset",
+            "timetable_summary": "Dataset",
         }