ssh2-sftp-client 8.1.0 → 9.0.0

package/README.org

@@ -9,7 +9,7 @@ convenience abstraction as well as a Promise based API.
 Documentation on the methods and available options in the underlying modules can
 be found on the [[https://github.com/mscdex/ssh2][SSH2]] project pages.
 
-Current stable release is *v8.1.0*.
+Current stable release is *v9.0.0*.
 
 Code has been tested against Node versions 14.19.1, 16.15.0 and 18.1.0
 
@@ -17,43 +17,52 @@ Node versions < 12.x are not supported. However, node v10.x should still work,
 although some tests will fail due to changes in file system functions used in
 test setup and tear down.
 
-** Version 8.x.x Changes
-
-  - *Breaking Change*: The API for ~uploadDir()~ and ~downloadDir()~ has been
-    changed. These methods now expect a function as the optional 3rd argument.
-    Previously, the 3rd argument was a regular expression used to filter out
-    which files and directories should be included in the upload or download
-    action. The method now expects a predicate function which will return true
-    if the target is to be included in the upload or download and false if it is
-    to be excluded. The predicate function will be called with two arguments, a
-    full path to the target object and a boolean value which is true when the
-    target is a directory, false otherwise. If no filter predicate is supplied,
-    all files and directories under the initial target directory will be
-    transferred. At this time, asynchronous filter functions are not supported.
-
-  - Internal Change: The ~rmdir()~ method has been refactored to enable
-    asynchronous deletion of files and sub-directories. This has significantly
-    increased performance when deleting larger directory trees, especially trees
-    which are /broad/ with lots of files and directories at the same level. For
-    deep narrow trees, there is less performance benefit because sub-directories
-    must be removed before parents, which imposes synchronous processing. It is
-    likely that for extremely large directory trees, the additional resources
-    required to run large numbers of asynchronous tasks will become problematic.
-    In such situations, it may be necessary to manually break up the deletion
-    process into multiple ~rmdir()~ calls. However, this is considered a fairly
-    extreme use case which is rare (a use case which wold also have been
-    problematic with the old implementation as performance would have been very
-    poor).
-
- - v8.1.0: The ~list()~ method now includes a ~longname~ property in the descriptor
-   object for each file in the listing. The ~longname~ value is a string which
-   resembles the ~ls -l~ listing.
-   
- - v8.1.0: The ~rmdir()~ method has been further modified to not recurse into
-   directories using async. It was found that on large directories, this could
-   result in too many separate async processes. Only deletion of files is now
-   handled with asynchronous calls. 
-    
+** Version 9.x Changes
+
+  - *Breaking Change*: This ~list()~ method no longer accepts a regular expression
+    for filtering the entries to be returned. You can now specify a filter
+    function instead. The function is called for each item in the list of items
+    to be returned, passing in the item object as its only argument.
+    Essentially, this is just a call to ~Array.filter()~, so the filter function
+    should behave in the same way i.e. return true for items to be retained and
+    false for those to be dropped.
+  - *Breaking Change*: The ability to set ~autoClose~ on read and write streams and
+    the ability to set ~end~ on ~pipe~ operations has been removed. These options
+    caused confusion for users and were too easy to get wrong, plus it made the
+    methods overly complicated. For those use-cases where you want to control
+    streams at a low level, two new methods have been added, ~createReadStream()~ and
+    ~createWriteStream()~. However, it should be noted that client code is 100%
+    responsible for managing streams obtained using these methods. Use at your
+    own risk!
+  - *Breaking Change*: The 3rd argument to ~uploadDir()~ and ~downloadDir()~ methods
+    has been change. Previously, the argument was a filter function used to
+    select which directories and files to be transferred. The 3rd argument is
+    now an options object with two supported properties, ~filter~ and ~useFastput~
+    (for ~uploadDir()~) or ~useFastget~ (for ~downloadDir()~). If ~useFastput~ is true,
+    the ~fastPut()~ method will be pused to upload files. If ~false~ or missing, the
+    slower, but better supported, ~put()~ method will be used. Likewise, the
+    ~useFastget~ options can be set to ~true~ to use the ~fastGet()~ method for
+    donwloading files, otherwise the slower, but more reliable, ~get()~ method
+    will be used.
+  - The ~uploadDir()~ and ~downloadDir()~ methods now use asynchrounous processes to
+    upload/download files. This should result in improved performance for these
+    two methods.
+  - New Methods: Two new methods, ~createWriteStream()~ and ~createReadStream()~
+    have been added. These methods will return a stream object connected to a
+    remote file on the ~sftp~ server. Client code is responsible for managing
+    these stream objects. This includes adding any necessary event listeners and
+    disposing of the objects once finished with them.
+  - Refactoring of Listeners: The library manages temporary listeners in order
+    to provide a way to catch events and processes them inside a ~Promise~
+    context. Previously, every method added its own set of temporary listeners.
+    However, this could result in multiple sets of listeners being added,
+    especially for methods which call other methods as part of their processing
+    e.g. ~rmdir(),~ ~uploadDir()~ and ~dowqnloadDir()~. To avoid this, /internal only/
+    versions of each method have been created. These internal methods use an
+    /underscore/ ~_~ prefix. Client code should not use these methods directly. 
+  - New method: Added ~rcopy()~ method to perform a remote copy of a file on the remote SFTP server. 
+  - Bumped ssh2 version to 1.11.0
+
 * Installation
 
 #+begin_src shell
@@ -259,16 +268,17 @@ for that package for an explanation of these values.
   });
 #+end_src
 
-*** list(path, pattern) ==> Array[object]
+*** list(path, filter) ==> Array[object]
 
 Retrieves a directory listing. This method returns a Promise, which once
 realised, returns an array of objects representing items in the remote
 directory.
 
 - path :: {String} Remote directory path
-- pattern :: (optional) {string|RegExp} A pattern used to filter the items included in the returned
-             array. Pattern can be a simple /glob/-style string or a regular
-             expression. Defaults to ~/.*/~.
+- filter :: (optional) {function} A function used to filter the items included
+  in the returned array. The function is called for each item with the item
+  object being passed in as the argument. The function is passed to
+  Array.filter() to perform the filtering.
 
 **** Example Use
 
@@ -885,7 +895,7 @@ exists. instead, use the ~exist()~ method.
 
 Returns what the server believes is the current remote working directory.
 
-*** uploadDir(srcDir, dstDir, filter) ==> string
+*** uploadDir(srcDir, dstDir, options) ==> string
 
 Upload the directory specified by ~srcDir~ to the remote directory specified by
 ~dstDir~. The ~dstDir~ will be created if necessary. Any sub directories within
@@ -900,7 +910,9 @@ the file was uploaded. The purpose of this event is to provide some way for
 client code to get feedback on the upload progress. You can add your own listener
 using the ~on()~ method.
 
-The optionsl /filter/ argument is a function which will be called for each item
+The 3rd argument is an options object with two supported properties, ~filter~ and ~useFastput~. 
+
+The ~filter~ option is a function which will be called for each item
 to be uploaded. The function will be called with two arguments. The first
 argument is the full path of the item to be uploaded and the second argument is
 a boolean, which will be true if the target path is for a directory. The filter
@@ -911,13 +923,22 @@ method. These array comprehension methods are known to be unsafe for
 asynchronous functions. Therefore, only synchronous filter functions are
 supported at this time.
 
+The ~useFastput~ option is a boolean option. If ~true~, the method will use the
+faster ~fastPut()~ method to upload files. Although this method is faster, it is
+not supported by all SFTP servers. Enabling this option when unsupported by the
+remote SFTP server will result in failures.
+
 - srcDir :: A local file path specified as a string
 - dstDir :: A remote file path specified as a string
-- filter :: A filter predicate function which is called for each item in the
+- options :: An options object which supports two properties, ~filter~ and
+  ~useFastput~. A filter predicate function which is called for each item in the
   source path. The argument will receive two arguments. The first is the full
   path to the item and the second is a boolean which will be true if the item is
   a directory. If the function returns true, the item will be uploaded,
-  otherwise it will be filtered out and ignored.
+  otherwise it will be filtered out and ignored. The ~useFastput~ option is a
+  boolean option. If ~true~, the method will use the faster, but less supported,
+  ~fastPut()~ method to transfer files. The default is to use the slightly slower,
+  but better supported, ~put()~ method.
   
 **** Example
 
@@ -969,7 +990,7 @@ supported at this time.
 
      #+end_src
 
-*** downloadDir(srcDir, dstDir, filter) ==> string
+*** downloadDir(srcDir, dstDir, options) ==> string
 
 Download the remote directory specified by ~srcDir~ to the local file system
 directory specified by ~dstDir~. The ~dstDir~ directory will be created if
@@ -984,20 +1005,30 @@ the remote file that has been downloaded and the destination is the local path
 to where the file was downloaded to. You can add a listener for this event using
 the ~on()~ method.
 
-The optional /filter/ argument is a predicate function which will be called with
-two arguments for each potential item to be downloaded. The first argument is
-the full path of the item and the second argument is a boolean, which will be
+The ~options~ argument is an options object with two supported properties, ~filter~
+and ~useFastget~. The ~filter~ argument is a predicate function which will be called
+with two arguments for each potential item to be downloaded. The first argument
+is the full path of the item and the second argument is a boolean, which will be
 true if the item is a directory. If the function returns true, the item will be
 included in the download. If it returns false, it will be filtered and ignored.
 The filter function is called via the ~Array.filter~ method. These array
 comprehension methods are known to be unsafe for asynchronous functions.
 Therefore, only synchronous filter functions are supported at this time.
 
+If the ~useFastget~ property is set to ~true~, the method will use ~fastGet()~ to
+transfer files. The ~fastGet~ method is faster, but not supported by all SFTP
+services.
+
 - srcDir :: A remote file path specified as a string
 - dstDir :: A local file path specified as a string
-- filter :: A predicate function called with two arguments, the full path to an
+- options :: An object with two supported properties, ~filter~ and ~useFastget~. The
+  filter property is a function accepting two arguments, the full path to an
   item and a boolean value which will be true if the item is a directory. The
-  function is called for each item in the download path.
+  function is called for each item in the download path and should return true
+  to include the item and false to exclude it in the download. The ~useFastget~
+  property is a boolean. If true, the ~fastGet()~ method will be used to transfer
+  files. If ~false~ (the default), the slower but better supported ~get()~ mehtod is
+  used. .
   
 **** Example
 
@@ -1047,6 +1078,52 @@ Therefore, only synchronous filter functions are supported at this time.
 
 #+end_src
 
+
+*** createReadStream(remotePath, options)) ==> stream object
+
+  Returns a read stream object which is attached to the remote file specified by
+  the ~remotePath~ argument. This is a low level method which just returns a read
+  stream object. Client code is fully responsible for managing and releasing the
+  resources associated with the stream once finished i.e. closing files,
+  removing listeners etc. 
+
+  - remotePath :: A remote file path specified as a string
+  - options :: An options object. Supported properties are 
+    - flags :: defaults to 'r'
+    - encoding :: defaults to null
+    - handle :: defaults to null
+    - mode :: 0o666
+    - autoClose :: defaults to true. If set to false, client code is responsible
+      for closing file descriptors when finished
+    - start :: Default 0. Position to start reading bytes from (inclusive)
+    - end :: Postion to stop reading bytes (inclusive). 
+
+*** createWriteStream(remotePath, options) ==> stream object
+
+   Returns a write stream object which is attached to the remote file specified
+   in the ~remotePath~ argument. This is a low legvel function which just returns
+   the stream object. Client code is fully responsible for managing that object,
+   including closing any file descriptiors and removing listeners etc.
+
+   - remotePath :: Path to the remote file specified as a string
+   - options :: An object containing stream options. Supported properties include 
+     - flags :: default 'w'
+     - encoding :: defulat null
+     - mode :: 0o666
+     - autoClose :: true 
+     - start :: Byte position to start writing from (inclusive). May require
+       changing flag to 'r+'.
+
+*** rcopy(srcPath, dstPath) ==> string
+
+   Perfrom a remote file copy. The file identified by the ~srcPath~ argument will
+   be copied to the file specified as the ~dstPath~ argument. The directory where
+   ~dstPath~ will be placed must exist, but the actual file must not i.e. no
+   overwrites allowed. 
+
+   - srcPath :: Path to remote file to be copied specified as a string
+   - dstPath :: Path to where the copy will be creaeted specified as a string
+
 *** end() ==> boolean
 
 Ends the current client session, releasing the client socket and associated

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ssh2-sftp-client",
-  "version": "8.1.0",
+  "version": "9.0.0",
   "description": "ssh2 sftp client for node",
   "main": "src/index.js",
   "repository": {
@@ -35,13 +35,13 @@
     "chai-subset": "^1.6.0",
     "checksum": "^1.0.0",
     "dotenv": "^16.0.0",
-    "eslint": "^8.12.0",
+    "eslint": "^8.17.0",
     "eslint-config-prettier": "^8.5.0",
     "eslint-plugin-mocha": "^10.0.3",
     "eslint-plugin-node": "^11.1.0",
     "eslint-plugin-promise": "^6.0.0",
     "eslint-plugin-unicorn": "^42.0.0",
-    "mocha": "^9.2.2",
+    "mocha": "^10.0.0",
     "moment": "^2.29.1",
     "nyc": "^15.1.0",
     "prettier": "^2.6.1",
@@ -51,6 +51,6 @@
   "dependencies": {
     "concat-stream": "^2.0.0",
     "promise-retry": "^2.0.1",
-    "ssh2": "^1.10.0"
+    "ssh2": "^1.11.0"
   }
 }

package/src/constants.js

@@ -5,6 +5,7 @@ const errorCode = {
   permission: 'EACCES',
   notexist: 'ENOENT',
   notdir: 'ENOTDIR',
+  badAuth: 'ERR_BAD_AUTH',
 };
 
 const targetType = {