apache-airflow-providers-sftp 5.3.0rc1__tar.gz → 5.3.1__tar.gz

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-sftp
-Version: 5.3.0rc1
+Version: 5.3.1
 Summary: Provider package apache-airflow-providers-sftp for Apache Airflow
 Keywords: airflow-provider,sftp,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -20,15 +20,15 @@ Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: System :: Monitoring
-Requires-Dist: apache-airflow>=2.10.0rc1
-Requires-Dist: apache-airflow-providers-ssh>=2.1.0rc1
+Requires-Dist: apache-airflow>=2.10.0
+Requires-Dist: apache-airflow-providers-ssh>=2.1.0
 Requires-Dist: paramiko>=2.9.0
 Requires-Dist: asyncssh>=2.12.0
 Requires-Dist: apache-airflow-providers-common-compat ; extra == "common-compat"
 Requires-Dist: apache-airflow-providers-openlineage ; extra == "openlineage"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.0/changelog.html
-Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.0
+Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html
+Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -61,7 +61,7 @@ Provides-Extra: openlineage
 
 Package ``apache-airflow-providers-sftp``
 
-Release: ``5.3.0``
+Release: ``5.3.1``
 
 
 `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
@@ -74,7 +74,7 @@ This is a provider package for ``sftp`` provider. All classes for this provider
 are in ``airflow.providers.sftp`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/>`_.
 
 Installation
 ------------
@@ -119,5 +119,5 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html>`_.
 

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-sftp``
 
-Release: ``5.3.0``
+Release: ``5.3.1``
 
 
 `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``sftp`` provider. All classes for this provider
 are in ``airflow.providers.sftp`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/>`_.
 
 Installation
 ------------
@@ -81,4 +81,4 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html>`_.

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/docs/changelog.rst

@@ -27,6 +27,17 @@
 Changelog
 ---------
 
+5.3.1
+.....
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Updating SFTPSensor to properly handle scenario where file is missing (#51167)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+
 5.3.0
 .....
 

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/docs/index.rst

@@ -70,7 +70,7 @@ apache-airflow-providers-sftp package
 `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
 
 
-Release: 5.3.0
+Release: 5.3.1
 
 Provider package
 ----------------
@@ -126,5 +126,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-sftp 5.3.0 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.0.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.0.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.0.tar.gz.sha512>`__)
-* `The apache-airflow-providers-sftp 5.3.0 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.0-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.0-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.0-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-sftp 5.3.1 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1.tar.gz.sha512>`__)
+* `The apache-airflow-providers-sftp 5.3.1 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/provider.yaml

@@ -22,12 +22,13 @@ description: |
     `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
 
 state: ready
-source-date-epoch: 1747133724
+source-date-epoch: 1749896917
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 5.3.1
   - 5.3.0
   - 5.2.1
   - 5.2.0

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-sftp"
-version = "5.3.0rc1"
+version = "5.3.1"
 description = "Provider package apache-airflow-providers-sftp for Apache Airflow"
 readme = "README.rst"
 authors = [
@@ -57,8 +57,8 @@ requires-python = "~=3.9"
 # Make sure to run ``breeze static-checks --type update-providers-dependencies --all-files``
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
-    "apache-airflow>=2.10.0rc1",
-    "apache-airflow-providers-ssh>=2.1.0rc1",
+    "apache-airflow>=2.10.0",
+    "apache-airflow-providers-ssh>=2.1.0",
     "paramiko>=2.9.0",
     "asyncssh>=2.12.0",
 ]
@@ -110,8 +110,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.0"
-"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.0/changelog.html"
+"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1"
+"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/src/airflow/providers/sftp/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "5.3.0"
+__version__ = "5.3.1"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.10.0"

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/src/airflow/providers/sftp/sensors/sftp.py

@@ -89,25 +89,25 @@ class SFTPSensor(BaseSensorOperator):
         if self.file_pattern:
             files_from_pattern = self.hook.get_files_by_pattern(self.path, self.file_pattern)
             if files_from_pattern:
-                actual_files_to_check = [
+                actual_files_present = [
                     os.path.join(self.path, file_from_pattern) for file_from_pattern in files_from_pattern
                 ]
             else:
                 return False
         else:
             try:
-                self.hook.isfile(self.path)
-                actual_files_to_check = [self.path]
-            except OSError as e:
-                if e.errno != SFTP_NO_SUCH_FILE:
-                    raise AirflowException from e
-                actual_files_to_check = []
+                # If a file is present, it is the single element added to the actual_files_present list to be
+                # processed. If the file is a directory, actual_file_present will be assigned an empty list,
+                # since SFTPHook.isfile(...) returns False
+                actual_files_present = [self.path] if self.hook.isfile(self.path) else []
+            except Exception as e:
+                raise AirflowException from e
 
         if self.newer_than:
-            for actual_file_to_check in actual_files_to_check:
+            for actual_file_present in actual_files_present:
                 try:
-                    mod_time = self.hook.get_mod_time(actual_file_to_check)
-                    self.log.info("Found File %s last modified: %s", actual_file_to_check, mod_time)
+                    mod_time = self.hook.get_mod_time(actual_file_present)
+                    self.log.info("Found File %s last modified: %s", actual_file_present, mod_time)
                 except OSError as e:
                     if e.errno != SFTP_NO_SUCH_FILE:
                         raise AirflowException from e
@@ -118,22 +118,22 @@ class SFTPSensor(BaseSensorOperator):
                 _mod_time = convert_to_utc(datetime.strptime(mod_time, "%Y%m%d%H%M%S"))
                 _newer_than = convert_to_utc(self.newer_than)
                 if _newer_than <= _mod_time:
-                    files_found.append(actual_file_to_check)
+                    files_found.append(actual_file_present)
                     self.log.info(
                         "File %s has modification time: '%s', which is newer than: '%s'",
-                        actual_file_to_check,
+                        actual_file_present,
                         str(_mod_time),
                         str(_newer_than),
                     )
                 else:
                     self.log.info(
                         "File %s has modification time: '%s', which is older than: '%s'",
-                        actual_file_to_check,
+                        actual_file_present,
                         str(_mod_time),
                         str(_newer_than),
                     )
         else:
-            files_found = actual_files_to_check
+            files_found = actual_files_present
 
         if not len(files_found):
             return False

{apache_airflow_providers_sftp-5.3.0rc1 → apache_airflow_providers_sftp-5.3.1}/tests/unit/sftp/sensors/test_sftp.py

@@ -22,7 +22,7 @@ from unittest import mock
 from unittest.mock import Mock, patch
 
 import pytest
-from paramiko.sftp import SFTP_FAILURE, SFTP_NO_SUCH_FILE
+from paramiko.sftp import SFTP_FAILURE
 from pendulum import datetime as pendulum_datetime, timezone
 
 from airflow.exceptions import AirflowException
@@ -44,9 +44,22 @@ class TestSFTPSensor:
         sftp_hook_mock.return_value.close_conn.assert_not_called()
         assert output
 
+    @patch("airflow.providers.sftp.sensors.sftp.SFTPHook")
+    def test_file_irregular(self, sftp_hook_mock):
+        # This mocks the behavior of SFTPHook.isfile when an OSError is raised in that method, resulting in
+        # False being returned
+        sftp_hook_mock.return_value.isfile.return_value = False
+        sftp_sensor = SFTPSensor(task_id="unit_test", path="/path/to/file/1970-01-01.txt")
+        context = {"ds": "1970-01-01"}
+        output = sftp_sensor.poke(context)
+        sftp_hook_mock.return_value.isfile.assert_called_once_with("/path/to/file/1970-01-01.txt")
+        sftp_hook_mock.return_value.close_conn.assert_not_called()
+        assert not output
+
     @patch("airflow.providers.sftp.sensors.sftp.SFTPHook")
     def test_file_absent(self, sftp_hook_mock):
-        sftp_hook_mock.return_value.isfile.side_effect = OSError(SFTP_NO_SUCH_FILE, "File missing")
+        # This is the same implementation above, however, it's simulating instead the absence of a file
+        sftp_hook_mock.return_value.isfile.return_value = False
         sftp_sensor = SFTPSensor(task_id="unit_test", path="/path/to/file/1970-01-01.txt")
         context = {"ds": "1970-01-01"}
         output = sftp_sensor.poke(context)