apache-airflow-providers-sftp 5.3.1rc1__tar.gz → 5.3.2__tar.gz

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-sftp
-Version: 5.3.1rc1
+Version: 5.3.2
 Summary: Provider package apache-airflow-providers-sftp for Apache Airflow
 Keywords: airflow-provider,sftp,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
 Maintainer-email: Apache Software Foundation <dev@airflow.apache.org>
-Requires-Python: ~=3.9
+Requires-Python: ~=3.10
 Description-Content-Type: text/x-rst
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Console
@@ -15,20 +15,19 @@ Classifier: Intended Audience :: System Administrators
 Classifier: Framework :: Apache Airflow
 Classifier: Framework :: Apache Airflow :: Provider
 Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: System :: Monitoring
-Requires-Dist: apache-airflow>=2.10.0rc1
-Requires-Dist: apache-airflow-providers-ssh>=2.1.0rc1
+Requires-Dist: apache-airflow>=2.10.0
+Requires-Dist: apache-airflow-providers-ssh>=2.1.0
 Requires-Dist: paramiko>=2.9.0
 Requires-Dist: asyncssh>=2.12.0
 Requires-Dist: apache-airflow-providers-common-compat ; extra == "common-compat"
 Requires-Dist: apache-airflow-providers-openlineage ; extra == "openlineage"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html
-Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.1
+Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2/changelog.html
+Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -61,7 +60,7 @@ Provides-Extra: openlineage
 
 Package ``apache-airflow-providers-sftp``
 
-Release: ``5.3.1``
+Release: ``5.3.2``
 
 
 `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
@@ -74,7 +73,7 @@ This is a provider package for ``sftp`` provider. All classes for this provider
 are in ``airflow.providers.sftp`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2/>`_.
 
 Installation
 ------------
@@ -83,7 +82,7 @@ You can install this package on top of an existing Airflow 2 installation (see `
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-sftp``
 
-The package supports the following python versions: 3.9,3.10,3.11,3.12
+The package supports the following python versions: 3.10,3.11,3.12
 
 Requirements
 ------------
@@ -119,5 +118,5 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2/changelog.html>`_.
 

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/README.rst

@@ -23,7 +23,7 @@
 
 Package ``apache-airflow-providers-sftp``
 
-Release: ``5.3.1``
+Release: ``5.3.2``
 
 
 `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``sftp`` provider. All classes for this provider
 are in ``airflow.providers.sftp`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2/>`_.
 
 Installation
 ------------
@@ -45,7 +45,7 @@ You can install this package on top of an existing Airflow 2 installation (see `
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-sftp``
 
-The package supports the following python versions: 3.9,3.10,3.11,3.12
+The package supports the following python versions: 3.10,3.11,3.12
 
 Requirements
 ------------
@@ -81,4 +81,4 @@ Dependent package
 ==================================================================================================================  =================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2/changelog.html>`_.

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/docs/changelog.rst

@@ -27,6 +27,26 @@
 Changelog
 ---------
 
+5.3.2
+.....
+
+Bug Fixes
+~~~~~~~~~
+
+* ``bugfix: removed cache for proxycommand in SFTPHook (#52641)``
+
+Misc
+~~~~
+
+* ``Move 'BaseHook' implementation to task SDK (#51873)``
+* ``Replace 'models.BaseOperator' to Task SDK one for SFTP (#52435)``
+* ``Drop support for Python 3.9 (#52072)``
+* ``Use BaseSensorOperator from task sdk in providers (#52296)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Separate out creation of default Connections for tests and non-tests (#52129)``
+
 5.3.1
 .....
 

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/docs/index.rst

@@ -70,7 +70,7 @@ apache-airflow-providers-sftp package
 `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
 
 
-Release: 5.3.1
+Release: 5.3.2
 
 Provider package
 ----------------
@@ -126,5 +126,5 @@ Downloading official packages
 You can download officially released packages and verify their checksums and signatures from the
 `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
 
-* `The apache-airflow-providers-sftp 5.3.1 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1.tar.gz.sha512>`__)
-* `The apache-airflow-providers-sftp 5.3.1 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.1-py3-none-any.whl.sha512>`__)
+* `The apache-airflow-providers-sftp 5.3.2 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.2.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.2.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.2.tar.gz.sha512>`__)
+* `The apache-airflow-providers-sftp 5.3.2 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.2-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.2-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_sftp-5.3.2-py3-none-any.whl.sha512>`__)

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/provider.yaml

@@ -22,12 +22,13 @@ description: |
     `SSH File Transfer Protocol (SFTP) <https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/>`__
 
 state: ready
-source-date-epoch: 1749896917
+source-date-epoch: 1751474190
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 5.3.2
   - 5.3.1
   - 5.3.0
   - 5.2.1

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-sftp"
-version = "5.3.1rc1"
+version = "5.3.2"
 description = "Provider package apache-airflow-providers-sftp for Apache Airflow"
 readme = "README.rst"
 authors = [
@@ -44,21 +44,20 @@ classifiers = [
     "Framework :: Apache Airflow",
     "Framework :: Apache Airflow :: Provider",
     "License :: OSI Approved :: Apache Software License",
-    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
     "Topic :: System :: Monitoring",
 ]
-requires-python = "~=3.9"
+requires-python = "~=3.10"
 
 # The dependencies should be modified in place in the generated file.
 # Any change in the dependencies is preserved when the file is regenerated
 # Make sure to run ``breeze static-checks --type update-providers-dependencies --all-files``
 # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
 dependencies = [
-    "apache-airflow>=2.10.0rc1",
-    "apache-airflow-providers-ssh>=2.1.0rc1",
+    "apache-airflow>=2.10.0",
+    "apache-airflow-providers-ssh>=2.1.0",
     "paramiko>=2.9.0",
     "asyncssh>=2.12.0",
 ]
@@ -110,8 +109,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.1"
-"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-sftp/5.3.1/changelog.html"
+"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2"
+"Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-sftp/5.3.2/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/src/airflow/providers/sftp/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "5.3.1"
+__version__ = "5.3.2"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.10.0"

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/src/airflow/providers/sftp/decorators/sensors/sftp.py

@@ -17,8 +17,7 @@
 
 from __future__ import annotations
 
-from collections.abc import Sequence
-from typing import Callable
+from collections.abc import Callable, Sequence
 
 from airflow.providers.sftp.version_compat import AIRFLOW_V_3_0_PLUS
 

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/src/airflow/providers/sftp/hooks/sftp.py

@@ -24,20 +24,24 @@ import datetime
 import os
 import stat
 import warnings
-from collections.abc import Generator, Sequence
+from collections.abc import Callable, Generator, Sequence
 from contextlib import contextmanager
 from fnmatch import fnmatch
 from io import BytesIO
 from pathlib import Path
-from typing import TYPE_CHECKING, Any, Callable
+from typing import TYPE_CHECKING, Any
 
 import asyncssh
 from asgiref.sync import sync_to_async
 
 from airflow.exceptions import AirflowException, AirflowProviderDeprecationWarning
-from airflow.hooks.base import BaseHook
 from airflow.providers.ssh.hooks.ssh import SSHHook
 
+try:
+    from airflow.sdk import BaseHook
+except ImportError:
+    from airflow.hooks.base import BaseHook  # type: ignore[attr-defined,no-redef]
+
 if TYPE_CHECKING:
     from paramiko import SSHClient
     from paramiko.sftp_attr import SFTPAttributes
@@ -148,6 +152,8 @@ class SFTPHook(SSHHook):
                 self._sftp_conn = None
                 self._ssh_conn.close()
                 self._ssh_conn = None
+                if hasattr(self, "host_proxy"):
+                    del self.host_proxy
 
     def get_conn_count(self) -> int:
         """Get the number of open connections."""
@@ -723,9 +729,9 @@ class SFTPHookAsync(BaseHook):
         """
         conn = await sync_to_async(self.get_connection)(self.sftp_conn_id)
         if conn.extra is not None:
-            self._parse_extras(conn)
+            self._parse_extras(conn)  # type: ignore[arg-type]
 
-        conn_config = {
+        conn_config: dict[str, Any] = {
             "host": conn.host,
             "port": conn.port,
             "username": conn.login,
@@ -737,10 +743,10 @@ class SFTPHookAsync(BaseHook):
             if self.known_hosts.lower() == "none":
                 conn_config.update(known_hosts=None)
             else:
-                conn_config.update(known_hosts=self.known_hosts)
+                conn_config.update(known_hosts=self.known_hosts)  # type: ignore
         if self.private_key:
             _private_key = asyncssh.import_private_key(self.private_key, self.passphrase)
-            conn_config.update(client_keys=[_private_key])
+            conn_config["client_keys"] = [_private_key]
         if self.passphrase:
             conn_config.update(passphrase=self.passphrase)
         ssh_client_conn = await asyncssh.connect(**conn_config)

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/src/airflow/providers/sftp/operators/sftp.py

@@ -28,8 +28,8 @@ from typing import Any
 import paramiko
 
 from airflow.exceptions import AirflowException
-from airflow.models import BaseOperator
 from airflow.providers.sftp.hooks.sftp import SFTPHook
+from airflow.providers.sftp.version_compat import BaseOperator
 
 
 class SFTPOperation:

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/src/airflow/providers/sftp/sensors/sftp.py

@@ -20,9 +20,9 @@
 from __future__ import annotations
 
 import os
-from collections.abc import Sequence
+from collections.abc import Callable, Sequence
 from datetime import datetime, timedelta
-from typing import TYPE_CHECKING, Any, Callable
+from typing import TYPE_CHECKING, Any
 
 from paramiko.sftp import SFTP_NO_SUCH_FILE
 
@@ -30,7 +30,7 @@ from airflow.configuration import conf
 from airflow.exceptions import AirflowException
 from airflow.providers.sftp.hooks.sftp import SFTPHook
 from airflow.providers.sftp.triggers.sftp import SFTPTrigger
-from airflow.sensors.base import BaseSensorOperator, PokeReturnValue
+from airflow.providers.sftp.version_compat import BaseSensorOperator, PokeReturnValue
 from airflow.utils.timezone import convert_to_utc, parse
 
 if TYPE_CHECKING:

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/src/airflow/providers/sftp/version_compat.py

@@ -33,3 +33,11 @@ def get_base_airflow_version_tuple() -> tuple[int, int, int]:
 
 
 AIRFLOW_V_3_0_PLUS = get_base_airflow_version_tuple() >= (3, 0, 0)
+
+if AIRFLOW_V_3_0_PLUS:
+    from airflow.sdk import BaseOperator, BaseSensorOperator, PokeReturnValue
+else:
+    from airflow.models import BaseOperator
+    from airflow.sensors.base import BaseSensorOperator, PokeReturnValue  # type: ignore[no-redef]
+
+__all__ = ["AIRFLOW_V_3_0_PLUS", "BaseOperator", "BaseSensorOperator", "PokeReturnValue"]

{apache_airflow_providers_sftp-5.3.1rc1 → apache_airflow_providers_sftp-5.3.2}/tests/unit/sftp/hooks/test_sftp.py

@@ -34,9 +34,6 @@ from paramiko.sftp_client import SFTPClient
 from airflow.exceptions import AirflowException
 from airflow.models import Connection
 from airflow.providers.sftp.hooks.sftp import SFTPHook, SFTPHookAsync
-from airflow.utils.session import provide_session
-
-pytestmark = pytest.mark.db_test
 
 
 def generate_host_key(pkey: paramiko.PKey):
@@ -62,13 +59,31 @@ TEST_KEY_FILE = "~/.ssh/id_rsa"
 
 
 class TestSFTPHook:
-    @provide_session
-    def update_connection(self, login, session=None):
-        connection = session.query(Connection).filter(Connection.conn_id == "sftp_default").first()
-        old_login = connection.login
-        connection.login = login
-        connection.extra = ""  # clear out extra so it doesn't look for a key file
-        session.commit()
+    def update_connection(self, login):
+        import os
+
+        # Get the current connection from environment variable to find the old login
+        old_connection = os.environ.get("AIRFLOW_CONN_SFTP_DEFAULT")
+        old_login = "airflow"  # default fallback
+
+        if old_connection:
+            try:
+                old_conn = Connection.from_json(old_connection)
+                old_login = old_conn.login
+            except Exception:
+                pass
+
+        # Set the connection as an environment variable
+        new_connection = Connection(
+            conn_id="sftp_default",
+            conn_type="sftp",
+            host="localhost",
+            login=login,
+            password="airflow",
+            extra="",  # clear out extra so it doesn't look for a key file
+        )
+        os.environ[f"AIRFLOW_CONN_{new_connection.conn_id.upper()}"] = new_connection.as_json()
+
         return old_login
 
     def _create_additional_test_file(self, file_name):
@@ -123,6 +138,48 @@ class TestSFTPHook:
         assert self.hook.get_conn_count() == 0
         assert self.hook.conn is None
 
+    @patch("paramiko.SSHClient")
+    @patch("paramiko.ProxyCommand")
+    @patch("airflow.providers.sftp.hooks.sftp.SFTPHook.get_connection")
+    def test_proxy_command_cache_invalidated_after_connection_closed(
+        self, mock_get_connection, mock_proxy_command, mock_ssh_client
+    ):
+        """
+        Assert that the ProxyCommand gets invalidated after the connection is closed
+        """
+
+        mock_connection = MagicMock()
+        mock_connection.login = "user"
+        mock_connection.password = None
+        mock_connection.host = "example.com"
+        mock_connection.port = 22
+        mock_connection.extra = None
+        mock_get_connection.return_value = mock_connection
+
+        mock_sftp_client = MagicMock(spec=SFTPClient)
+        mock_ssh_client.open_sftp.return_value = mock_sftp_client
+
+        mock_transport = MagicMock()
+        mock_ssh_client.return_value.get_transport.return_value = mock_transport
+        mock_proxy_command.return_value = MagicMock()
+
+        host_proxy_cmd = "ncat --proxy-auth proxy_user:**** --proxy proxy_host:port %h %p"
+        prev_proxy_command = None
+
+        hook = SFTPHook(
+            remote_host="example.com",
+            username="user",
+            host_proxy_cmd=host_proxy_cmd,
+        )
+
+        with hook.get_managed_conn() as _:
+            assert hasattr(self.hook, "host_proxy")
+            prev_proxy_command = hook.host_proxy
+
+        mock_proxy_command.return_value = MagicMock()
+
+        assert prev_proxy_command != hook.host_proxy
+
     @patch("airflow.providers.ssh.hooks.ssh.SSHHook.get_conn")
     def test_get_close_conn(self, mock_get_conn):
         mock_sftp_client = MagicMock(spec=SFTPClient)
@@ -549,7 +606,17 @@ class TestSFTPHook:
 
     @patch("paramiko.SSHClient")
     @patch("paramiko.ProxyCommand")
-    def test_sftp_hook_with_proxy_command(self, mock_proxy_command, mock_ssh_client):
+    @patch("airflow.providers.sftp.hooks.sftp.SFTPHook.get_connection")
+    def test_sftp_hook_with_proxy_command(self, mock_get_connection, mock_proxy_command, mock_ssh_client):
+        # Mock the connection to not have a password
+        mock_connection = MagicMock()
+        mock_connection.login = "user"
+        mock_connection.password = None
+        mock_connection.host = "example.com"
+        mock_connection.port = 22
+        mock_connection.extra = None
+        mock_get_connection.return_value = mock_connection
+
         mock_sftp_client = MagicMock(spec=SFTPClient)
         mock_ssh_client.open_sftp.return_value = mock_sftp_client