ssh2-sftp-client 7.2.1 → 8.0.0

package/README.org

@@ -9,12 +9,42 @@ convenience abstraction as well as a Promise based API.
 Documentation on the methods and available options in the underlying modules can
 be found on the [[https://github.com/mscdex/ssh2][SSH2]] project pages.
 
-Current stable release is *v7.2.1*.
-
-Code has been tested against Node versions 14.18.2, 16.13.1 and 17.2.0
-
-Node versions < 10.x are not supported.
-
+Current stable release is *v8.0.0*.
+
+Code has been tested against Node versions 14.19.1, 16.14.2 and 17.8.0
+
+Node versions < 12.x are not supported. However, node v10.x should still work,
+although some tests will fail due to changes in file system functions used in
+test setup and tear down.
+
+** Version 8.x.x Changes
+
+  - *Breaking Change*: The API for ~uploadDir()~ and ~downloadDir()~ has been
+    changed. These methods now expect a function as the optional 3rd argument.
+    Previously, the 3rd argument was a regular expression used to filter out
+    which files and directories should be included in the upload or download
+    action. The method now expects a predicate function which will return true
+    if the target is to be included in the upload or download and false if it is
+    to be excluded. The predicate function will be called with two arguments, a
+    full path to the target object and a boolean value which is true when the
+    target is a directory, false otherwise. If no filter predicate is supplied,
+    all files and directories under the initial target directory will be
+    transferred. At this time, asynchronous filter functions are not supported.
+
+  - Internal Change: The ~rmdir()~ method has been refactored to enable
+    asynchronous deletion of files and sub-directories. This has significantly
+    increased performance when deleting larger directory trees, especially trees
+    which are /broad/ with lots of files and directories at the same level. For
+    deep narrow trees, there is less performance benefit because sub-directories
+    must be removed before parents, which imposes synchronous processing. It is
+    likely that for extremely large directory trees, the additional resources
+    required to run large numbers of asynchronous tasks will become problematic.
+    In such situations, it may be necessary to manually break up the deletion
+    process into multiple ~rmdir()~ calls. However, this is considered a fairly
+    extreme use case which is rare (a use case which wold also have been
+    problematic with the old implementation as performance would have been very
+    poor).
+    
 * Installation
 
 #+begin_src shell
@@ -41,40 +71,6 @@ npm install ssh2-sftp-client
   });
 #+end_src
 
-* Version 7.x Changes
-
-- This version is based on version 1.x.x of ~ssh2~. This version of ~ssh2~ is a
-  complete re-write of the ~ssh2~ library. This re-write addresses issues
-  encountered when using node v14 as well as some design weaknesses in the
-  previous 0.8.x version.
-
-- *Breaking Change* Expanded option handling for ~get()~ and ~put()~ methods. A number of use
-  cases were identified where setting specific options on the read and write
-  streams and the pipe operation are necessary. For example, disabling
-  ~autoClose~ on streams or the ~end~ event in pipes. The ~options~ argument for
-  ~get()~ and ~put()~ calls now supports properties for ~readStreamOptions~,
-  ~writeStreamOptions~ and ~pipeOptions~. Note that options are only applied to
-  streams created by the ~get()~ and ~put()~ methods. Streams passed into these
-  methods are under the control of the client code and therefore cannot have
-  options supplied in arguments to those streams (you would apply such options
-  when you create the streams). Options are typically only necessary in special
-  use cases. Most of the time, no options are required. However, if you are
-  currently using options to either ~put()~ or ~get()~, you will need to update
-  your code to map these options to the new structure. 
-
-- *Breaking Change 7.1.0* A race condition was identified when using a put()
-  call with a writeStream option of ~autoClose: false~. In some situations, the
-  promise would be resolved before the final close of the write stream. This
-  could result in errors if you immediately attempt to access the uploaded
-  file. To avoid this situatioin, the promise is now resolved once a ~close~
-  event is emitted. This means that setting ~autoClose: false~ can no longer be
-  supported. The write stream for ~put()~ will autoClose once data writing has completed.
-   
-- Improved event handling. A listener for a global error event is now defined to
-  catch errors which occur in-between method calls i.e. connection lost
-  in-between calls to the library methods. A new mechanism has also been added
-  for removal of listeners when no longer required. 
-
 * Documentation
 
 The connection options are the same as those offered by the underlying SSH2
@@ -422,7 +418,7 @@ is passed to it via a call to pipe(). If ~dst~ is undefined, the method will put
 the data into a buffer and return that buffer when the Promise is resolved. If
 ~dst~ is defined, it is returned when the Promise is resolved.
 
-In general, if your going to pass in a string as the destination, you are
+In general, if you're going to pass in a string as the destination, you are
 better off using the ~fastGet()~ method.
 
 - path :: String. Path to the remote file to download
@@ -890,18 +886,29 @@ The upload process also emits 'upload' events. These events are fired for each
 successfully uploaded file. The ~upload~ event calls listeners with 1 argument,
 an object which has properties source and destination. The source property is
 the path of the file uploaded and the destination property is the path to where
-the file was uploaded to. The purpose of this event is to provide some way for
-client code to get feedback on the upload progress. You can add your own lisener
+the file was uploaded. The purpose of this event is to provide some way for
+client code to get feedback on the upload progress. You can add your own listener
 using the ~on()~ method.
 
-The optionsl /filter/ argument is a regular expression which can be used to
-select which files and directories to include in the upload.
+The optionsl /filter/ argument is a function which will be called for each item
+to be uploaded. The function will be called with two arguments. The first
+argument is the full path of the item to be uploaded and the second argument is
+a boolean, which will be true if the target path is for a directory. The filter
+function will be called for each item in the source path. If the function
+returns true, the item will be uploaded. If it returns false, it will be
+filtered and not uploaded. The filter function is called via the ~Array.filter~
+method. These array comprehension methods are known to be unsafe for
+asynchronous functions. Therefore, only synchronous filter functions are
+supported at this time.
 
 - srcDir :: A local file path specified as a string
 - dstDir :: A remote file path specified as a string
-- filter :: A regular expression used to filter which files and directories to
-  include in the upload
-
+- filter :: A filter predicate function which is called for each item in the
+  source path. The argument will receive two arguments. The first is the full
+  path to the item and the second is a boolean which will be true if the item is
+  a directory. If the function returns true, the item will be uploaded,
+  otherwise it will be filtered out and ignored.
+  
 **** Example
 
      #+begin_src javascript
@@ -917,36 +924,38 @@ select which files and directories to include in the upload.
        require('dotenv').config({path: dotenvPath});
 
        const config = {
-   host: process.env.SFTP_SERVER,
-   username: process.env.SFTP_USER,
-   password: process.env.SFTP_PASSWORD,
-   port: process.env.SFTP_PORT || 22
+         host: process.env.SFTP_SERVER,
+         username: process.env.SFTP_USER,
+         password: process.env.SFTP_PASSWORD,
+         port: process.env.SFTP_PORT || 22
        };
 
        async function main() {
-   const client = new SftpClient('upload-test');
-   const src = path.join(__dirname, '..', 'test', 'testData', 'upload-src');
-   const dst = '/home/tim/upload-test';
-
-   try {
-     await client.connect(config);
-     client.on('upload', info => {
-       console.log(`Listener: Uploaded ${info.source}`);
-     });
-     let rslt = await client.uploadDir(src, dst);
-     return rslt;
-   } finally {
-     client.end();
-   }
+         const client = new SftpClient('upload-test');
+         const src = path.join(__dirname, '..', 'test', 'testData', 'upload-src');
+         const dst = '/home/tim/upload-test';
+
+         try {
+           await client.connect(config);
+           client.on('upload', info => {
+             console.log(`Listener: Uploaded ${info.source}`);
+           });
+           let rslt = await client.uploadDir(src, dst);
+           return rslt;
+         } catch (err) {
+           console.error(err);
+         } finally {
+           client.end();
+         }
        }
 
        main()
-   .then(msg => {
-     console.log(msg);
-   })
-   .catch(err => {
-     console.log(`main error: ${err.message}`);
-   });
+         .then(msg => {
+           console.log(msg);
+         })
+         .catch(err => {
+           console.log(`main error: ${err.message}`);
+         });
 
      #+end_src
 
@@ -965,14 +974,21 @@ the remote file that has been downloaded and the destination is the local path
 to where the file was downloaded to. You can add a listener for this event using
 the ~on()~ method.
 
-The optional /filter/ argument is a regular expression which can be used to
-select which files and directories will be downloaded from the remote server.
+The optional /filter/ argument is a predicate function which will be called with
+two arguments for each potential item to be downloaded. The first argument is
+the full path of the item and the second argument is a boolean, which will be
+true if the item is a directory. If the function returns true, the item will be
+included in the download. If it returns false, it will be filtered and ignored.
+The filter function is called via the ~Array.filter~ method. These array
+comprehension methods are known to be unsafe for asynchronous functions.
+Therefore, only synchronous filter functions are supported at this time.
 
 - srcDir :: A remote file path specified as a string
 - dstDir :: A local file path specified as a string
-- filter :: A regular expression used to match the files and directories to be
-  downloaded
-
+- filter :: A predicate function called with two arguments, the full path to an
+  item and a boolean value which will be true if the item is a directory. The
+  function is called for each item in the download path.
+  
 **** Example
 
 #+begin_src javascript
@@ -1732,10 +1748,10 @@ the issue. Things which will help
 Perhaps the best assistance is a minimal reproducible example of the issue. Once
 the issue can be readily reproduced, it can usually be fixed very quickly.
 
-* Pull Requests
+* Pull Requests 
 
 Pull requests are always welcomed. However, please ensure your changes pass all
-tests and if your adding a new feature, that tests for that feature are
+tests and if you're adding a new feature, that tests for that feature are
 included. Likewise, for new features or enhancements, please include any
 relevant documentation updates.
 
@@ -1784,3 +1800,5 @@ Thanks to the following for their contributions -
   'close' to resolve promise
 - Maik Marschner :: Contributed fix for connect() not returning sftp object.
   Also included test to check for this regression in future.
+- cakemasher :: Contributed fix for removeTempListeners().
+  

package/package.json

@@ -1,17 +1,13 @@
 {
   "name": "ssh2-sftp-client",
-  "version": "7.2.1",
+  "version": "8.0.0",
   "description": "ssh2 sftp client for node",
   "main": "src/index.js",
   "repository": {
     "type": "git",
     "url": "https://github.com/theophilusx/ssh2-sftp-client"
   },
-  "keywords": [
-    "sftp",
-    "nodejs",
-    "promises"
-  ],
+  "keywords": ["sftp", "nodejs", "promises"],
   "scripts": {
     "test": "mocha",
     "coverage": "nyc npm run test",
@@ -29,28 +25,28 @@
     }
   ],
   "license": "Apache-2.0",
-  "dependencies": {
-    "concat-stream": "^2.0.0",
-    "promise-retry": "^2.0.1",
-    "ssh2": "^1.5.0"
-  },
   "devDependencies": {
-    "chai": "^4.3.4",
+    "chai": "^4.3.6",
     "chai-as-promised": "^7.1.1",
     "chai-subset": "^1.6.0",
     "checksum": "^1.0.0",
-    "dotenv": "^10.0.0",
-    "eslint": "^8.5.0",
-    "eslint-config-prettier": "^8.3.0",
+    "dotenv": "^16.0.0",
+    "eslint": "^8.12.0",
+    "eslint-config-prettier": "^8.5.0",
     "eslint-plugin-mocha": "^10.0.3",
     "eslint-plugin-node": "^11.1.0",
     "eslint-plugin-promise": "^6.0.0",
-    "eslint-plugin-unicorn": "^39.0.0",
-    "mocha": "^9.1.2",
+    "eslint-plugin-unicorn": "^42.0.0",
+    "mocha": "^9.2.2",
     "moment": "^2.29.1",
     "nyc": "^15.1.0",
-    "prettier": "^2.5.0",
+    "prettier": "^2.6.1",
     "through2": "^4.0.2",
-    "winston": "^3.3.3"
+    "winston": "^3.6.0"
+  },
+  "dependencies": {
+    "concat-stream": "^2.0.0",
+    "promise-retry": "^2.0.1",
+    "ssh2": "^1.9.0"
   }
 }