apache-airflow-providers-elasticsearch 6.2.2rc1__tar.gz → 6.3.0rc1__tar.gz

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-elasticsearch
-Version: 6.2.2rc1
+Version: 6.3.0rc1
 Summary: Provider package apache-airflow-providers-elasticsearch for Apache Airflow
 Keywords: airflow-provider,elasticsearch,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -20,12 +20,12 @@ Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: System :: Monitoring
-Requires-Dist: apache-airflow>=2.9.0rc0
-Requires-Dist: apache-airflow-providers-common-sql>=1.20.0rc0
+Requires-Dist: apache-airflow>=2.10.0rc1
+Requires-Dist: apache-airflow-providers-common-sql>=1.27.0rc1
 Requires-Dist: elasticsearch>=8.10,<9
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2/changelog.html
-Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2
+Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0/changelog.html
+Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -56,7 +56,7 @@ Project-URL: YouTube, https://www.youtube.com/channel/UCSXwxpWZQ7XZ1WL3wqevChA/
 
 Package ``apache-airflow-providers-elasticsearch``
 
-Release: ``6.2.2``
+Release: ``6.3.0``
 
 
 `Elasticsearch <https://www.elastic.co/elasticsearch>`__
@@ -69,7 +69,7 @@ This is a provider package for ``elasticsearch`` provider. All classes for this
 are in ``airflow.providers.elasticsearch`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0/>`_.
 
 Installation
 ------------
@@ -86,8 +86,8 @@ Requirements
 =======================================  ==================
 PIP package                              Version required
 =======================================  ==================
-``apache-airflow``                       ``>=2.9.0``
-``apache-airflow-providers-common-sql``  ``>=1.20.0``
+``apache-airflow``                       ``>=2.10.0``
+``apache-airflow-providers-common-sql``  ``>=1.27.0``
 ``elasticsearch``                        ``>=8.10,<9``
 =======================================  ==================
 
@@ -111,5 +111,5 @@ Dependent package
 ============================================================================================================  ==============
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0/changelog.html>`_.
 

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-elasticsearch``
 
-Release: ``6.2.2``
+Release: ``6.3.0``
 
 
 `Elasticsearch <https://www.elastic.co/elasticsearch>`__
@@ -36,7 +36,7 @@ This is a provider package for ``elasticsearch`` provider. All classes for this
 are in ``airflow.providers.elasticsearch`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0/>`_.
 
 Installation
 ------------
@@ -53,8 +53,8 @@ Requirements
 =======================================  ==================
 PIP package                              Version required
 =======================================  ==================
-``apache-airflow``                       ``>=2.9.0``
-``apache-airflow-providers-common-sql``  ``>=1.20.0``
+``apache-airflow``                       ``>=2.10.0``
+``apache-airflow-providers-common-sql``  ``>=1.27.0``
 ``elasticsearch``                        ``>=8.10,<9``
 =======================================  ==================
 
@@ -78,4 +78,4 @@ Dependent package
 ============================================================================================================  ==============
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0/changelog.html>`_.

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/docs/changelog.rst

@@ -27,6 +27,27 @@
 Changelog
 ---------
 
+6.3.0
+.....
+
+.. note::
+    This release of provider is only available for Airflow 2.10+ as explained in the
+    Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>_.
+
+Misc
+~~~~
+
+* ``Migrate 'ElasticsearchSQLHook' to use 'get_df' (#50454)``
+* ``Remove AIRFLOW_2_10_PLUS conditions (#49877)``
+* ``Bump min Airflow version in providers to 2.10 (#49843)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Update description of provider.yaml dependencies (#50231)``
+   * ``Avoid committing history for providers (#49907)``
+   * ``test: migrate 'get_pandas_df' to 'get_df' in 'provider' test (#49339)``
+   * ``capitalize the term airflow (#49450)``
+
 6.2.2
 .....
 

apache_airflow_providers_elasticsearch-6.3.0rc1/docs/commits.rst

@@ -0,0 +1,35 @@
+
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+ .. NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE OVERWRITTEN!
+
+ .. IF YOU WANT TO MODIFY THIS FILE, YOU SHOULD MODIFY THE TEMPLATE
+    `PROVIDER_COMMITS_TEMPLATE.rst.jinja2` IN the `dev/breeze/src/airflow_breeze/templates` DIRECTORY
+
+ .. THE REMAINDER OF THE FILE IS AUTOMATICALLY GENERATED. IT WILL BE OVERWRITTEN!
+
+Package apache-airflow-providers-elasticsearch
+------------------------------------------------------
+
+`Elasticsearch <https://www.elastic.co/elasticsearch>`__
+
+
+This is detailed commit list of changes for versions provider package: ``elasticsearch``.
+For high-level changelog, see :doc:`package information including changelog <index>`.
+
+.. airflow-providers-commits::

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/docs/index.rst

@@ -79,7 +79,7 @@ apache-airflow-providers-elasticsearch package
 `Elasticsearch <https://www.elastic.co/elasticsearch>`__
 
 
-Release: 6.2.2
+Release: 6.3.0
 
 Provider package
 ----------------
@@ -97,13 +97,13 @@ For the minimum Airflow version supported, see ``Requirements`` below.
 Requirements
 ------------
 
-The minimum Apache Airflow version supported by this provider distribution is ``2.9.0``.
+The minimum Apache Airflow version supported by this provider distribution is ``2.10.0``.
 
 =======================================  ==================
 PIP package                              Version required
 =======================================  ==================
-``apache-airflow``                       ``>=2.9.0``
-``apache-airflow-providers-common-sql``  ``>=1.20.0``
+``apache-airflow``                       ``>=2.10.0``
+``apache-airflow-providers-common-sql``  ``>=1.27.0``
 ``elasticsearch``                        ``>=8.10,<9``
 =======================================  ==================
 
@@ -132,5 +132,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-elasticsearch 6.2.2 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.2.2.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.2.2.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.2.2.tar.gz.sha512>`__)
-* `The apache-airflow-providers-elasticsearch 6.2.2 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.2.2-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.2.2-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.2.2-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-elasticsearch 6.3.0 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.3.0.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.3.0.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.3.0.tar.gz.sha512>`__)
+* `The apache-airflow-providers-elasticsearch 6.3.0 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.3.0-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.3.0-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_elasticsearch-6.3.0-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/docs/logging/index.rst

@@ -22,7 +22,7 @@ Writing logs to Elasticsearch
 
 Airflow can be configured to read task logs from Elasticsearch and optionally write logs to stdout in standard or json format. These logs can later be collected and forwarded to the Elasticsearch cluster using tools like fluentd, logstash or others.
 
-Airflow also supports writing log to Elasticsearch directly without requiring additional software like filebeat and logstash. To enable this feature, set ``write_to_es`` and ``json_format`` to ``True`` and ``write_stdout`` to ``False`` in ``airflow.cfg``. Please be aware that if you set both ``write_to_es`` and ``delete_local_logs`` in logging section to true, airflow will delete the local copy of task logs upon successfully writing task logs to ElasticSearch.
+Airflow also supports writing log to Elasticsearch directly without requiring additional software like filebeat and logstash. To enable this feature, set ``write_to_es`` and ``json_format`` to ``True`` and ``write_stdout`` to ``False`` in ``airflow.cfg``. Please be aware that if you set both ``write_to_es`` and ``delete_local_logs`` in logging section to true, Airflow will delete the local copy of task logs upon successfully writing task logs to ElasticSearch.
 
 You can choose to have all task logs from workers output to the highest parent level process, instead of the standard file locations. This allows for some additional flexibility in container environments like Kubernetes, where container stdout is already being logged to the host nodes. From there a log shipping tool can be used to forward them along to Elasticsearch. To use this feature, set the ``write_stdout`` option in ``airflow.cfg``.
 You can also choose to have the logs output in a JSON format, using the ``json_format`` option. Airflow uses the standard Python logging module and JSON fields are directly extracted from the LogRecord object. To use this feature, set the ``json_fields`` option in ``airflow.cfg``. Add the fields to the comma-delimited string that you want collected for the logs. These fields are from the LogRecord object in the ``logging`` module. `Documentation on different attributes can be found here <https://docs.python.org/3/library/logging.html#logrecord-objects/>`_.

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/provider.yaml

@@ -22,9 +22,13 @@ description: |
     `Elasticsearch <https://www.elastic.co/elasticsearch>`__
 
 state: ready
-source-date-epoch: 1744791912
-# note that those versions are maintained by release manager - do not update them manually
+source-date-epoch: 1747132714
+# Note that those versions are maintained by release manager - do not update them manually
+# with the exception of case where other provider in sources has >= new provider version.
+# In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
+# to be done in the same PR
 versions:
+  - 6.3.0
   - 6.2.2
   - 6.2.1
   - 6.2.0

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-elasticsearch"
-version = "6.2.2.rc1"
+version = "6.3.0rc1"
 description = "Provider package apache-airflow-providers-elasticsearch for Apache Airflow"
 readme = "README.rst"
 authors = [
@@ -57,8 +57,8 @@ requires-python = "~=3.9"
 # Make sure to run ``breeze static-checks --type update-providers-dependencies --all-files``
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
-    "apache-airflow>=2.9.0rc0",
-    "apache-airflow-providers-common-sql>=1.20.0rc0",
+    "apache-airflow>=2.10.0rc1",
+    "apache-airflow-providers-common-sql>=1.27.0rc1",
     "elasticsearch>=8.10,<9",
 ]
 
@@ -69,7 +69,7 @@ dev = [
     "apache-airflow-devel-common",
     "apache-airflow-providers-common-sql",
     # Additional devel dependencies (do not remove this line and add extra development dependencies)
-    "apache-airflow-providers-common-sql[pandas]",
+    "apache-airflow-providers-common-sql[pandas,polars]",
 ]
 
 # To build docs:
@@ -98,8 +98,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2"
-"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/6.2.2/changelog.html"
+"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0"
+"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-elasticsearch/6.3.0/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/src/airflow/providers/elasticsearch/__init__.py

@@ -29,11 +29,11 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "6.2.2"
+__version__ = "6.3.0"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
-    "2.9.0"
+    "2.10.0"
 ):
     raise RuntimeError(
-        f"The package `apache-airflow-providers-elasticsearch:{__version__}` needs Apache Airflow 2.9.0+"
+        f"The package `apache-airflow-providers-elasticsearch:{__version__}` needs Apache Airflow 2.10.0+"
     )

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/src/airflow/providers/elasticsearch/hooks/elasticsearch.py

@@ -219,6 +219,17 @@ class ElasticsearchSQLHook(DbApiHook):
 
         return uri
 
+    def _get_polars_df(
+        self,
+        sql,
+        parameters: list | tuple | Mapping[str, Any] | None = None,
+        **kwargs,
+    ):
+        # TODO: Custom ElasticsearchSQLCursor is incompatible with polars.read_database.
+        # To support: either adapt cursor to polars._executor interface or create custom polars reader.
+        # https://github.com/apache/airflow/pull/50454
+        raise NotImplementedError("Polars is not supported for Elasticsearch")
+
 
 class ElasticsearchPythonHook(BaseHook):
     """

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/src/airflow/providers/elasticsearch/version_compat.py

@@ -32,5 +32,4 @@ def get_base_airflow_version_tuple() -> tuple[int, int, int]:
     return airflow_version.major, airflow_version.minor, airflow_version.micro
 
 
-AIRFLOW_V_2_10_PLUS = get_base_airflow_version_tuple() >= (2, 10, 0)
 AIRFLOW_V_3_0_PLUS = get_base_airflow_version_tuple() >= (3, 0, 0)

{apache_airflow_providers_elasticsearch-6.2.2rc1 → apache_airflow_providers_elasticsearch-6.3.0rc1}/tests/unit/elasticsearch/hooks/test_elasticsearch.py

@@ -166,9 +166,9 @@ class TestElasticsearchSQLHook:
         self.spy_agency.assert_spy_called(self.cur.close)
         self.spy_agency.assert_spy_called(self.cur.execute)
 
-    def test_get_pandas_df(self):
+    def test_get_df_pandas(self):
         statement = "SELECT * FROM hollywood.actors"
-        df = self.db_hook.get_pandas_df(statement)
+        df = self.db_hook.get_df(statement, df_type="pandas")
 
         assert list(df.columns) == ["index", "name", "firstname", "age"]
         assert df.values.tolist() == ROWS
@@ -177,6 +177,10 @@ class TestElasticsearchSQLHook:
         self.spy_agency.assert_spy_called(self.cur.close)
         self.spy_agency.assert_spy_called(self.cur.execute)
 
+    def test_get_df_polars(self):
+        with pytest.raises(NotImplementedError):
+            self.db_hook.get_df("SQL", df_type="polars")
+
     def test_run(self):
         statement = "SELECT * FROM hollywood.actors"