ssh2-sftp-client 7.2.0 → 7.2.3

package/README.md

@@ -61,11 +61,11 @@ an SFTP client for node.js, a wrapper around [SSH2](https://github.com/mscdex/ss
 
 Documentation on the methods and available options in the underlying modules can be found on the [SSH2](https://github.com/mscdex/ssh2) project pages.
 
-Current stable release is **v7.2.0**.
+Current stable release is **v7.2.3**.
 
-Code has been tested against Node versions 14.18.2, 16.13.1 and 17.2.0
+Code has been tested against Node versions 14.19.1, 16.14.2 and 17.8.0
 
-Node versions < 10.x are not supported.
+Node versions < 12.x are not supported. However, node v10.x should still work, although some tests will fail due to changes in file system functions used in test setup and tear down.
 
 # Installation<a id="sec-2"></a>
 
@@ -103,6 +103,8 @@ sftp.connect({
 
 -   Improved event handling. A listener for a global error event is now defined to catch errors which occur in-between method calls i.e. connection lost in-between calls to the library methods. A new mechanism has also been added for removal of listeners when no longer required.
 
+-   uploadDir/downloadDir change in 7.2.3. THe uploadDir() and downloadDir() methods previously used fastPut() and fastGet() to transfer data. Unfortunately, not all SFTP servers support the concurrent processing used by these methods. This meant these methods would fail on some platforms. For now, the fastPut() and fastGet() calls have been replaced with plain put() and get() calls. This will mean uploadDir()/downloadDir() will be slower. However, there is other larger changes in the works which should see a significant speed imp[rovement for these (and other methods). We may also add an option which will allow for selection of fastPut()/fastGet().
+
 # Documentation<a id="sec-5"></a>
 
 The connection options are the same as those offered by the underlying SSH2 module. For full details, please see [SSH2 client methods](https://github.com/mscdex/ssh2#user-content-client-methods)
@@ -392,7 +394,7 @@ Returns the attributes associated with the object pointed to by `path`.
 
 Retrieve a file from a remote SFTP server. The `dst` argument defines the destination and can be either a string, a stream object or undefined. If it is a string, it is interpreted as the path to a location on the local file system (path should include the file name). If it is a stream object, the remote data is passed to it via a call to pipe(). If `dst` is undefined, the method will put the data into a buffer and return that buffer when the Promise is resolved. If `dst` is defined, it is returned when the Promise is resolved.
 
-In general, if your going to pass in a string as the destination, you are better off using the `fastGet()` method.
+In general, if you're going to pass in a string as the destination, you are better off using the `fastGet()` method.
 
 -   **path:** String. Path to the remote file to download
 -   **dst:** String|Stream. Destination for the data. If a string, it should be a local file path.
@@ -1332,7 +1334,7 @@ Perhaps the best assistance is a minimal reproducible example of the issue. Once
 
 # Pull Requests<a id="sec-11"></a>
 
-Pull requests are always welcomed. However, please ensure your changes pass all tests and if your adding a new feature, that tests for that feature are included. Likewise, for new features or enhancements, please include any relevant documentation updates.
+Pull requests are always welcomed. However, please ensure your changes pass all tests and if you're adding a new feature, that tests for that feature are included. Likewise, for new features or enhancements, please include any relevant documentation updates.
 
 **Note**: The `README.md` file is generated from the `README.org` file. Therefore, any documentation updates or fixes need to be made to the `README.org` file. This file is *tangled* using `Emacs` org mode. If you don't use Emacs or org-mode, don't be too concerned. The org-mode syntax is straight-forward and similar to *markdown*. I will verify any updates to `README.org` and generate a new `README.md` when necessary. The main point to note is that any changes made directly to `README.md` will not persist and will be lost when a new version is generated, so don't modify that file.
 
@@ -1361,3 +1363,4 @@ Thanks to the following for their contributions -
 -   **Emma Milner:** Contributed fix for put() bug
 -   **Witni Davis:** Contributed PR to fix put() RCE when using 'finish' rather than 'close' to resolve promise
 -   **Maik Marschner:** Contributed fix for connect() not returning sftp object. Also included test to check for this regression in future.
+-   **cakemasher:** Contributed fix for removeTempListeners().

package/README.org

@@ -9,11 +9,13 @@ convenience abstraction as well as a Promise based API.
 Documentation on the methods and available options in the underlying modules can
 be found on the [[https://github.com/mscdex/ssh2][SSH2]] project pages.
 
-Current stable release is *v7.2.0*.
+Current stable release is *v7.2.3*.
 
-Code has been tested against Node versions 14.18.2, 16.13.1 and 17.2.0
+Code has been tested against Node versions 14.19.1, 16.14.2 and 17.8.0
 
-Node versions < 10.x are not supported.
+Node versions < 12.x are not supported. However, node v10.x should still work,
+although some tests will fail due to changes in file system functions used in
+test setup and tear down.
 
 * Installation
 
@@ -75,6 +77,16 @@ npm install ssh2-sftp-client
   in-between calls to the library methods. A new mechanism has also been added
   for removal of listeners when no longer required. 
 
+- uploadDir/downloadDir change in 7.2.3. THe uploadDir() and downloadDir()
+  methods previously used fastPut() and fastGet() to transfer data.
+  Unfortunately, not all SFTP servers support the concurrent processing used by
+  these methods. This meant these methods would fail on some platforms. For now,
+  the fastPut() and fastGet() calls have been replaced with plain put() and
+  get() calls. This will mean uploadDir()/downloadDir() will be slower. However,
+  there is other larger changes in the works which should see a significant
+  speed imp[rovement for these (and other methods). We may also add an option
+  which will allow for selection of fastPut()/fastGet().
+
 * Documentation
 
 The connection options are the same as those offered by the underlying SSH2
@@ -422,7 +434,7 @@ is passed to it via a call to pipe(). If ~dst~ is undefined, the method will put
 the data into a buffer and return that buffer when the Promise is resolved. If
 ~dst~ is defined, it is returned when the Promise is resolved.
 
-In general, if your going to pass in a string as the destination, you are
+In general, if you're going to pass in a string as the destination, you are
 better off using the ~fastGet()~ method.
 
 - path :: String. Path to the remote file to download
@@ -1732,10 +1744,10 @@ the issue. Things which will help
 Perhaps the best assistance is a minimal reproducible example of the issue. Once
 the issue can be readily reproduced, it can usually be fixed very quickly.
 
-* Pull Requests
+* Pull Requests 
 
 Pull requests are always welcomed. However, please ensure your changes pass all
-tests and if your adding a new feature, that tests for that feature are
+tests and if you're adding a new feature, that tests for that feature are
 included. Likewise, for new features or enhancements, please include any
 relevant documentation updates.
 
@@ -1784,3 +1796,5 @@ Thanks to the following for their contributions -
   'close' to resolve promise
 - Maik Marschner :: Contributed fix for connect() not returning sftp object.
   Also included test to check for this regression in future.
+- cakemasher :: Contributed fix for removeTempListeners().
+  

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ssh2-sftp-client",
-  "version": "7.2.0",
+  "version": "7.2.3",
   "description": "ssh2 sftp client for node",
   "main": "src/index.js",
   "repository": {
@@ -25,28 +25,28 @@
     }
   ],
   "license": "Apache-2.0",
-  "dependencies": {
-    "concat-stream": "^2.0.0",
-    "promise-retry": "^2.0.1",
-    "ssh2": "^1.5.0"
-  },
   "devDependencies": {
-    "chai": "^4.3.4",
+    "chai": "^4.3.6",
     "chai-as-promised": "^7.1.1",
     "chai-subset": "^1.6.0",
     "checksum": "^1.0.0",
-    "dotenv": "^10.0.0",
-    "eslint": "^8.3.0",
-    "eslint-config-prettier": "^8.3.0",
-    "eslint-plugin-mocha": "^9.0.0",
+    "dotenv": "^16.0.0",
+    "eslint": "^8.12.0",
+    "eslint-config-prettier": "^8.5.0",
+    "eslint-plugin-mocha": "^10.0.3",
     "eslint-plugin-node": "^11.1.0",
-    "eslint-plugin-promise": "^5.2.0",
-    "eslint-plugin-unicorn": "^39.0.0",
-    "mocha": "^9.1.2",
+    "eslint-plugin-promise": "^6.0.0",
+    "eslint-plugin-unicorn": "^41.0.1",
+    "mocha": "^9.2.2",
     "moment": "^2.29.1",
     "nyc": "^15.1.0",
-    "prettier": "^2.5.0",
+    "prettier": "^2.6.1",
     "through2": "^4.0.2",
-    "winston": "^3.3.3"
+    "winston": "^3.6.0"
+  },
+  "dependencies": {
+    "concat-stream": "^2.0.0",
+    "promise-retry": "^2.0.1",
+    "ssh2": "^1.8.0"
   }
 }

package/src/index.js

@@ -59,7 +59,7 @@ class SftpClient {
     this.client.on('error', (err) => {
       if (this.endCalled || this.errorHandled) {
         // error event expected or handled elsewhere
-        this.debugMsg('Global: Ignoring handled error');
+        this.debugMsg(`Global: Ignoring handled error: ${err.message}`);
       } else {
         this.debugMsg(`Global; Handling unexpected error; ${err.message}`);
         this.sftp = undefined;
@@ -120,28 +120,24 @@ class SftpClient {
    */
   getConnection(config) {
     let doReady;
-    return (
-      new Promise((resolve, reject) => {
-        addTempListeners(this, 'getConnection', reject);
-        this.debugMsg('getConnection: created promise');
-        doReady = () => {
-          this.debugMsg('getConnection: got connection - promise resolved');
-          resolve(true);
-        };
-        this.on('ready', doReady);
-        this.client.connect(config);
-      })
-        // .catch((err) => {
-        //   return Promise.reject(err);
-        // })
-        .finally(async () => {
-          this.debugMsg('getConnection: finally clause fired');
-          await sleep(500);
-          this.removeListener('ready', doReady);
-          removeTempListeners(this, 'getConnection');
-          this._resetEventFlags();
-        })
-    );
+    return new Promise((resolve, reject) => {
+      addTempListeners(this, 'getConnection', reject);
+      this.debugMsg('getConnection: created promise');
+      doReady = () => {
+        this.debugMsg(
+          'getConnection ready listener: got connection - promise resolved'
+        );
+        resolve(true);
+      };
+      this.on('ready', doReady);
+      this.client.connect(config);
+    }).finally(async () => {
+      this.debugMsg('getConnection: finally clause fired');
+      await sleep(500);
+      this.removeListener('ready', doReady);
+      removeTempListeners(this, 'getConnection');
+      this._resetEventFlags();
+    });
   }
 
   getSftpChannel() {
@@ -196,8 +192,17 @@ class SftpClient {
         (retry, attempt) => {
           this.debugMsg(`connect: Connect attempt ${attempt}`);
           return this.getConnection(config).catch((err) => {
-            this.debugMsg('getConnection retry catch');
-            retry(err);
+            this.debugMsg(
+              `getConnection retry catch: ${err.message} Code: ${err.code}`
+            );
+            switch (err.code) {
+              case 'ENOTFOUND':
+              case 'ECONNREFUSED':
+              case 'ERR_SOCKET_BAD_PORT':
+                throw err;
+              default:
+                retry(err);
+            }
           });
         },
         {
@@ -515,13 +520,29 @@ class SftpClient {
               )
             );
           });
-          rdr.once('end', () => {
-            if (typeof dst === 'string') {
-              resolve(dst);
-            } else {
-              resolve(wtr);
-            }
-          });
+          if (
+            Object.hasOwnProperty.call(options, 'pipeOptions') &&
+            Object.hasOwnProperty.call(options.pipeOptions, 'end') &&
+            !options.pipeOptions.end
+          ) {
+            rdr.once('end', () => {
+              this.debugMsg('get resolved on reader end event');
+              if (typeof dst === 'string') {
+                resolve(dst);
+              } else {
+                resolve(wtr);
+              }
+            });
+          } else {
+            wtr.once('finish', () => {
+              this.debugMsg('get resolved on writer finish event');
+              if (typeof dst === 'string') {
+                resolve(dst);
+              } else {
+                resolve(wtr);
+              }
+            });
+          }
         }
         rdr.pipe(wtr, options.pipeOptions ? options.pipeOptions : {});
       }
@@ -530,14 +551,16 @@ class SftpClient {
       this._resetEventFlags();
       if (
         rdr &&
-        options.readStreamOptions &&
+        Object.hasOwnProperty.call(options, 'readStreamOptions') &&
+        Object.hasOwnProperty.call(options.readStreamOptions, 'autoClose') &&
         options.readStreamOptions.autoClose === false
       ) {
         rdr.destroy();
       }
       if (
         wtr &&
-        options.writeStreamOptions &&
+        Object.hasOwnProperty.call(options, 'writeStreamOptions') &&
+        Object.hasOwnProperty.call(options.writeStreamOptions, 'autoClose') &&
         options.writeStreamOptions.autoClose === false &&
         typeof dst === 'string'
       ) {
@@ -747,7 +770,8 @@ class SftpClient {
       this._resetEventFlags();
       if (
         rdr &&
-        options.readStreamOptions &&
+        Object.hasOwnProperty.call(options, 'readStreamOptions') &&
+        Object.hasOwnProperty.call(options.readStreamOptions, 'autoClose') &&
         options.readStreamOptions.autoClose === false &&
         typeof localSrc === 'string'
       ) {
@@ -1110,7 +1134,7 @@ class SftpClient {
         } else if (e.isFile()) {
           let src = join(srcDir, e.name);
           let dst = dstDir + this.remotePathSep + e.name;
-          await this.fastPut(src, dst);
+          await this.put(src, dst);
           this.client.emit('upload', { source: src, destination: dst });
         } else {
           this.debugMsg(
@@ -1166,7 +1190,7 @@ class SftpClient {
         } else if (f.type === '-') {
           let src = srcDir + this.remotePathSep + f.name;
           let dst = join(dstDir, f.name);
-          await this.fastGet(src, dst);
+          await this.get(src, dst);
           this.client.emit('download', { source: src, destination: dst });
         } else {
           this.debugMsg(

package/src/utils.js

@@ -76,15 +76,15 @@ function addToTempListenerList(obj, name, evt, fn) {
 function errorListener(client, name, reject) {
   let fn = (err) => {
     if (client.endCalled || client.errorHandled) {
-      client.debugMsg(`${name}: Ignoring handled error: ${err.message}`);
+      client.debugMsg(`${name} Error: Ignoring handled error: ${err.message}`);
     } else {
-      client.debugMsg(`${name}: Handling error: ${err.message}`);
+      client.debugMsg(`${name} Error: Handling error: ${err.message}`);
       client.errorHandled = true;
       if (reject) {
-        client.debugMsg(`${name}: handled error with reject`);
+        client.debugMsg(`${name} Error: handled error with reject`);
         reject(fmtError(err, name, err.code));
       } else {
-        client.debugMsg(`${name}: handling error with throw`);
+        client.debugMsg(`${name} Error: handling error with throw`);
         throw fmtError(err, name, err.code);
       }
     }
@@ -96,16 +96,16 @@ function errorListener(client, name, reject) {
 function endListener(client, name, reject) {
   let fn = function () {
     if (client.endCalled || client.endHandled) {
-      client.debugMsg(`${name}: Ignoring expected end event`);
+      client.debugMsg(`${name} End: Ignoring expected end event`);
     } else {
-      client.debugMsg(`${name}: Handling end event`);
+      client.debugMsg(`${name} End: Handling end event`);
       client.sftp = undefined;
       client.endHandled = true;
       if (reject) {
-        client.debugMsg(`${name}: handling end event with reject'`);
+        client.debugMsg(`${name} End: handling end event with reject'`);
         reject(fmtError('Unexpected end event raised', name));
       } else {
-        client.debugMsg(`${name}: handling end event with throw`);
+        client.debugMsg(`${name} End: handling end event with throw`);
         throw fmtError('Unexpected end event raised', name);
       }
     }
@@ -117,16 +117,16 @@ function endListener(client, name, reject) {
 function closeListener(client, name, reject) {
   let fn = function () {
     if (client.endCalled || client.closeHandled) {
-      client.debugMsg(`${name}: ignoring expected close event`);
+      client.debugMsg(`${name} Close: ignoring expected close event`);
     } else {
-      client.debugMsg(`${name}: handling unexpected close event`);
+      client.debugMsg(`${name} Close: handling unexpected close event`);
       client.sftp = undefined;
       client.closeHandled = true;
       if (reject) {
-        client.debugMsg(`${name}: handling close event with reject`);
+        client.debugMsg(`${name} Close: handling close event with reject`);
         reject(fmtError('Unexpected close event raised', name));
       } else {
-        client.debugMsg(`${name}: handling close event with throw`);
+        client.debugMsg(`${name} Close: handling close event with throw`);
         throw fmtError('Unexpected close event raised', name);
       }
     }
@@ -148,7 +148,7 @@ function removeTempListeners(obj, name) {
     obj.tempListeners[name].forEach(([e, fn]) => {
       obj.client.removeListener(e, fn);
     });
-    obj.tempListeners = [];
+    obj.tempListeners[name] = [];
   }
 }