ssh2-sftp-client 8.1.0 → 9.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/README.md +198 -131
  2. package/README.org +162 -56
  3. package/package.json +4 -4
  4. package/src/constants.js +1 -0
  5. package/src/index.js +825 -692
  6. package/src/utils.js +65 -115
package/README.md CHANGED
@@ -1,62 +1,66 @@
1
- - [Overview](#orgbeea605)
2
- - [Version 8.x.x Changes](#org2589748)
3
- - [Installation](#orgd7b0a7e)
4
- - [Basic Usage](#org2e1bee4)
5
- - [Documentation](#orgfac43d1)
6
- - [Specifying Paths](#org38509ac)
7
- - [Methods](#org26ecc4f)
8
- - [new SftpClient(name) ===> SFTP client object](#orga50cd31)
9
- - [connect(config) ===> SFTP object](#orgbc0c617)
10
- - [list(path, pattern) ==> Array[object]](#orgdfab34d)
11
- - [exists(path) ==> boolean](#orgd09b949)
12
- - [stat(path) ==> object](#org31bacc1)
13
- - [get(path, dst, options) ==> String|Stream|Buffer](#org12ed2ca)
14
- - [fastGet(remotePath, localPath, options) ===> string](#org7b0d9e8)
15
- - [put(src, remotePath, options) ==> string](#orgd38e78b)
16
- - [fastPut(localPath, remotePath, options) ==> string](#orge130096)
17
- - [append(input, remotePath, options) ==> string](#org4f2a1b1)
18
- - [mkdir(path, recursive) ==> string](#orga9acefe)
19
- - [rmdir(path, recursive) ==> string](#org9e51da5)
20
- - [delete(path, noErrorOK) ==> string](#org2054a87)
21
- - [rename(fromPath, toPath) ==> string](#org7497d9f)
22
- - [posixRename(fromPath, toPath) ==> string](#orgb179088)
23
- - [chmod(path, mode) ==> string](#org335a008)
24
- - [realPath(path) ===> string](#org384f34a)
25
- - [cwd() ==> string](#org8fa77d4)
26
- - [uploadDir(srcDir, dstDir, filter) ==> string](#orgb824235)
27
- - [downloadDir(srcDir, dstDir, filter) ==> string](#org46aa66f)
28
- - [end() ==> boolean](#orge3fcb81)
29
- - [Add and Remove Listeners](#org4607268)
30
- - [Platform Quirks & Warnings](#org66a7061)
31
- - [Server Capabilities](#org740553b)
32
- - [Promises, Events & Managing Exceptions](#orgf97d765)
33
- - [Adding Custom Handlers](#org6acf9c6)
34
- - [Windows Based Servers](#orgb2f13ee)
35
- - [Don't Re-use SftpClient Objects](#orgcd82e68)
36
- - [FAQ](#orge1aa1d7)
37
- - [Remote server drops connections with only an end event](#org9296404)
38
- - [How can I pass writable stream as dst for get method?](#org3e48901)
39
- - [How can I upload files without having to specify a password?](#org1474028)
40
- - [How can I connect through a Socks Proxy](#orgb9a0ec1)
41
- - [Timeout while waiting for handshake or handshake errors](#org62e1f3a)
42
- - [How can I limit upload/download speed](#org7f26ef8)
43
- - [Connection hangs or fails for larger files](#orgfb52441)
44
- - [Examples](#orgbe2b99e)
45
- - [Troubleshooting](#orgd2dcb05)
46
- - [Common Errors](#orgae04d0a)
47
- - [Not returning the promise in a `then()` block](#org2018983)
48
- - [Mixing Promise Chains and Async/Await](#org6a3654b)
49
- - [Try/catch and Error Handlers](#org36087fd)
50
- - [Server Differences](#orgd09c9c8)
51
- - [Avoid Concurrent Operations](#org67e5a24)
52
- - [Debugging Support](#org00c676b)
53
- - [Logging Issues](#org6f7f508)
54
- - [Pull Requests](#org2d2ad0d)
55
- - [Contributors](#orgba32989)
56
-
57
-
58
-
59
- <a id="orgbeea605"></a>
1
+ - [Overview](#org50de852)
2
+ - [Version 9.x Changes](#orgaeda8f1)
3
+ - [Installation](#orgf01877e)
4
+ - [Basic Usage](#org5950004)
5
+ - [Documentation](#org5057006)
6
+ - [Specifying Paths](#org4ef62d6)
7
+ - [Methods](#org197917e)
8
+ - [new SftpClient(name) ===> SFTP client object](#orge668bfa)
9
+ - [connect(config) ===> SFTP object](#orgcf80c45)
10
+ - [list(path, filter) ==> Array[object]](#org329ff56)
11
+ - [exists(path) ==> boolean](#orgc52d8a8)
12
+ - [stat(path) ==> object](#orgcfa8682)
13
+ - [get(path, dst, options) ==> String|Stream|Buffer](#orgbe46842)
14
+ - [fastGet(remotePath, localPath, options) ===> string](#org740cf5c)
15
+ - [put(src, remotePath, options) ==> string](#org429e96a)
16
+ - [fastPut(localPath, remotePath, options) ==> string](#orgcab50c6)
17
+ - [append(input, remotePath, options) ==> string](#orgc15d676)
18
+ - [mkdir(path, recursive) ==> string](#org6fab860)
19
+ - [rmdir(path, recursive) ==> string](#org520f6bc)
20
+ - [delete(path, noErrorOK) ==> string](#org1be8ca9)
21
+ - [rename(fromPath, toPath) ==> string](#org3d89131)
22
+ - [posixRename(fromPath, toPath) ==> string](#org58dfee5)
23
+ - [chmod(path, mode) ==> string](#orgb319965)
24
+ - [realPath(path) ===> string](#orgf665476)
25
+ - [cwd() ==> string](#org409acd1)
26
+ - [uploadDir(srcDir, dstDir, options) ==> string](#org668b085)
27
+ - [downloadDir(srcDir, dstDir, options) ==> string](#org395ac4e)
28
+ - [createReadStream(remotePath, options)) ==> stream object](#org8a73b83)
29
+ - [createWriteStream(remotePath, options) ==> stream object](#orgb71fb72)
30
+ - [rcopy(srcPath, dstPath) ==> string](#org936f530)
31
+ - [end() ==> boolean](#org62a8323)
32
+ - [Add and Remove Listeners](#orgd1ebc36)
33
+ - [Platform Quirks & Warnings](#orgdfff943)
34
+ - [Server Capabilities](#org886fe2e)
35
+ - [Issues with `fastPut()` and `fastGet()` Methods](#org1719a52)
36
+ - [Promises, Events & Managing Exceptions](#org42a51fc)
37
+ - [Adding Custom Handlers](#orgd50a35a)
38
+ - [Windows Based Servers](#org7b0987f)
39
+ - [Don't Re-use SftpClient Objects](#orgbe548c2)
40
+ - [FAQ](#orgfddb94b)
41
+ - [Remote server drops connections with only an end event](#orgbd2a31e)
42
+ - [How can I pass writable stream as dst for get method?](#org9d72a61)
43
+ - [How can I upload files without having to specify a password?](#org9fda086)
44
+ - [How can I connect through a Socks Proxy](#org2ccc5e8)
45
+ - [Timeout while waiting for handshake or handshake errors](#org5306b6f)
46
+ - [How can I limit upload/download speed](#org3a77529)
47
+ - [Connection hangs or fails for larger files](#org68eedc7)
48
+ - [Examples](#orgd436fa7)
49
+ - [Troubleshooting](#orge21d870)
50
+ - [Common Errors](#org287b86f)
51
+ - [Not returning the promise in a `then()` block](#org20a99bb)
52
+ - [Mixing Promise Chains and Async/Await](#org51482b4)
53
+ - [Try/catch and Error Handlers](#orgb9a556a)
54
+ - [Server Differences](#org0d148f0)
55
+ - [Avoid Concurrent Operations](#orgf115391)
56
+ - [Debugging Support](#orgc7c437f)
57
+ - [Logging Issues](#orgc79ef28)
58
+ - [Pull Requests](#org64bd2a2)
59
+ - [Contributors](#org709b0f9)
60
+
61
+
62
+
63
+ <a id="org50de852"></a>
60
64
 
61
65
  # Overview
62
66
 
@@ -64,27 +68,33 @@ an SFTP client for node.js, a wrapper around [SSH2](https://github.com/mscdex/ss
64
68
 
65
69
  Documentation on the methods and available options in the underlying modules can be found on the [SSH2](https://github.com/mscdex/ssh2) project pages.
66
70
 
67
- Current stable release is **v8.1.0**.
71
+ Current stable release is **v9.0.2**.
68
72
 
69
73
  Code has been tested against Node versions 14.19.1, 16.15.0 and 18.1.0
70
74
 
71
- Node versions < 12.x are not supported. However, node v10.x should still work, although some tests will fail due to changes in file system functions used in test setup and tear down.
75
+ Node versions < 14.x are not supported.
72
76
 
73
77
 
74
- <a id="org2589748"></a>
78
+ <a id="orgaeda8f1"></a>
75
79
 
76
- ## Version 8.x.x Changes
80
+ ## Version 9.x Changes
77
81
 
78
- - **Breaking Change**: The API for `uploadDir()` and `downloadDir()` has been changed. These methods now expect a function as the optional 3rd argument. Previously, the 3rd argument was a regular expression used to filter out which files and directories should be included in the upload or download action. The method now expects a predicate function which will return true if the target is to be included in the upload or download and false if it is to be excluded. The predicate function will be called with two arguments, a full path to the target object and a boolean value which is true when the target is a directory, false otherwise. If no filter predicate is supplied, all files and directories under the initial target directory will be transferred. At this time, asynchronous filter functions are not supported.
82
+ - Change the default end and close handlers not to throw error or reject promises. Previously, an end or close event would cause an error to be raised or a promise to be rejected if the event was deemed to be *unexpected*. However, classification of events as being unexpected was unreliable and didn't add much real value. Both these handlers will now invalidate the sftp connection object and log that the event fired and nothing else.
83
+ - Changed when event handled flags are reset. Now they are reset after a new set of temporary listeners are added.
84
+ - Don't throw an error when calling end() if there is no active sftp connection. It does no harm to call end() when there is no connection, so no need to raise an error.
85
+ - Use nullish coalescing when setting retry parameters instead of or'ing with defaults. Allows setting values to 0.
86
+ - **Breaking Change**: This version uses syntax not supported in node versions prior to v14. Therefore, node versions less than v14 will not work.
87
+ - **Breaking Change**: This `list()` method no longer accepts a regular expression for filtering the entries to be returned. You can now specify a filter function instead. The function is called for each item in the list of items to be returned, passing in the item object as its only argument. Essentially, this is just a call to `Array.filter()`, so the filter function should behave in the same way i.e. return true for items to be retained and false for those to be dropped.
88
+ - **Breaking Change**: The ability to set `autoClose` on read and write streams and the ability to set `end` on `pipe` operations has been removed. These options caused confusion for users and were too easy to get wrong, plus it made the methods overly complicated. For those use-cases where you want to control streams at a low level, two new methods have been added, `createReadStream()` and `createWriteStream()`. However, it should be noted that client code is 100% responsible for managing streams obtained using these methods. Use at your own risk!
89
+ - **Breaking Change**: The 3rd argument to `uploadDir()` and `downloadDir()` methods has been change. Previously, the argument was a filter function used to select which directories and files to be transferred. The 3rd argument is now an options object with two supported properties, `filter` and `useFastput` (for `uploadDir()`) or `useFastget` (for `downloadDir()`). If `useFastput` is true, the `fastPut()` method will be pused to upload files. If `false` or missing, the slower, but better supported, `put()` method will be used. Likewise, the `useFastget` options can be set to `true` to use the `fastGet()` method for donwloading files, otherwise the slower, but more reliable, `get()` method will be used.
90
+ - The `uploadDir()` and `downloadDir()` methods now use asynchrounous processes to upload/download files. This should result in improved performance for these two methods.
91
+ - New Methods: Two new methods, `createWriteStream()` and `createReadStream()` have been added. These methods will return a stream object connected to a remote file on the `sftp` server. Client code is responsible for managing these stream objects. This includes adding any necessary event listeners and disposing of the objects once finished with them.
92
+ - Refactoring of Listeners: The library manages temporary listeners in order to provide a way to catch events and processes them inside a `Promise` context. Previously, every method added its own set of temporary listeners. However, this could result in multiple sets of listeners being added, especially for methods which call other methods as part of their processing e.g. `rmdir(),` `uploadDir()` and `dowqnloadDir()`. To avoid this, *internal only* versions of each method have been created. These internal methods use an *underscore* `_` prefix. Client code should not use these methods directly.
93
+ - New method: Added `rcopy()` method to perform a remote copy of a file on the remote SFTP server.
94
+ - Bumped ssh2 version to 1.11.0
79
95
 
80
- - Internal Change: The `rmdir()` method has been refactored to enable asynchronous deletion of files and sub-directories. This has significantly increased performance when deleting larger directory trees, especially trees which are *broad* with lots of files and directories at the same level. For deep narrow trees, there is less performance benefit because sub-directories must be removed before parents, which imposes synchronous processing. It is likely that for extremely large directory trees, the additional resources required to run large numbers of asynchronous tasks will become problematic. In such situations, it may be necessary to manually break up the deletion process into multiple `rmdir()` calls. However, this is considered a fairly extreme use case which is rare (a use case which wold also have been problematic with the old implementation as performance would have been very poor).
81
96
 
82
- - v8.1.0: The `list()` method now includes a `longname` property in the descriptor object for each file in the listing. The `longname` value is a string which resembles the `ls -l` listing.
83
-
84
- - v8.1.0: The `rmdir()` method has been further modified to not recurse into directories using async. It was found that on large directories, this could result in too many separate async processes. Only deletion of files is now handled with asynchronous calls.
85
-
86
-
87
- <a id="orgd7b0a7e"></a>
97
+ <a id="orgf01877e"></a>
88
98
 
89
99
  # Installation
90
100
 
@@ -93,7 +103,7 @@ npm install ssh2-sftp-client
93
103
  ```
94
104
 
95
105
 
96
- <a id="org2e1bee4"></a>
106
+ <a id="org5950004"></a>
97
107
 
98
108
  # Basic Usage
99
109
 
@@ -116,7 +126,7 @@ sftp.connect({
116
126
  ```
117
127
 
118
128
 
119
- <a id="orgfac43d1"></a>
129
+ <a id="org5057006"></a>
120
130
 
121
131
  # Documentation
122
132
 
@@ -125,7 +135,7 @@ The connection options are the same as those offered by the underlying SSH2 modu
125
135
  All the methods will return a Promise, except for `on()` and `removeListener()`, which are typically only used in special use cases.
126
136
 
127
137
 
128
- <a id="org38509ac"></a>
138
+ <a id="org4ef62d6"></a>
129
139
 
130
140
  ## Specifying Paths
131
141
 
@@ -158,12 +168,12 @@ client.put('/home/fred/test.txt', '/remote/dir/test-copy.txt');
158
168
  This will copy the local file `test.txt` to the remote file `test-copy.txt` in the directory `/remote/dir`.
159
169
 
160
170
 
161
- <a id="org26ecc4f"></a>
171
+ <a id="org197917e"></a>
162
172
 
163
173
  ## Methods
164
174
 
165
175
 
166
- <a id="orga50cd31"></a>
176
+ <a id="orge668bfa"></a>
167
177
 
168
178
  ### new SftpClient(name) ===> SFTP client object
169
179
 
@@ -202,7 +212,7 @@ Constructor to create a new `ssh2-sftp-client` object. An optional `name` string
202
212
  ```
203
213
 
204
214
 
205
- <a id="orgbc0c617"></a>
215
+ <a id="orgcf80c45"></a>
206
216
 
207
217
  ### connect(config) ===> SFTP object
208
218
 
@@ -268,14 +278,14 @@ Connect to an sftp server. Full documentation for connection options is availabl
268
278
  ```
269
279
 
270
280
 
271
- <a id="orgdfab34d"></a>
281
+ <a id="org329ff56"></a>
272
282
 
273
- ### list(path, pattern) ==> Array[object]
283
+ ### list(path, filter) ==> Array[object]
274
284
 
275
285
  Retrieves a directory listing. This method returns a Promise, which once realised, returns an array of objects representing items in the remote directory.
276
286
 
277
287
  - **path:** {String} Remote directory path
278
- - **pattern:** (optional) {string|RegExp} A pattern used to filter the items included in the returned array. Pattern can be a simple *glob*-style string or a regular expression. Defaults to `/.*/`.
288
+ - **filter:** (optional) {function} A function used to filter the items included in the returned array. The function is called for each item with the item object being passed in as the argument. The function is passed to Array.filter() to perform the filtering.
279
289
 
280
290
  1. Example Use
281
291
 
@@ -339,7 +349,7 @@ Retrieves a directory listing. This method returns a Promise, which once realise
339
349
  The *glob*-style matching is very simple. In most cases, you are best off using a real regular expression which will allow you to do more powerful matching and anchor matches to the beginning/end of the string etc.
340
350
 
341
351
 
342
- <a id="orgd09b949"></a>
352
+ <a id="orgc52d8a8"></a>
343
353
 
344
354
  ### exists(path) ==> boolean
345
355
 
@@ -375,7 +385,7 @@ Tests to see if remote file or directory exists. Returns type of remote object i
375
385
  ```
376
386
 
377
387
 
378
- <a id="org31bacc1"></a>
388
+ <a id="orgcfa8682"></a>
379
389
 
380
390
  ### stat(path) ==> object
381
391
 
@@ -426,7 +436,7 @@ Returns the attributes associated with the object pointed to by `path`.
426
436
  ```
427
437
 
428
438
 
429
- <a id="org12ed2ca"></a>
439
+ <a id="orgbe46842"></a>
430
440
 
431
441
  ### get(path, dst, options) ==> String|Stream|Buffer
432
442
 
@@ -482,7 +492,7 @@ In general, if you're going to pass in a string as the destination, you are bett
482
492
  - **Tip:** See examples file in the Git repository for more examples. You can pass any writeable stream in as the destination. For example, if you pass in `zlib.createGunzip()` writeable stream, you can both download and decompress a gzip file 'on the fly'.
483
493
 
484
494
 
485
- <a id="org7b0d9e8"></a>
495
+ <a id="org740cf5c"></a>
486
496
 
487
497
  ### fastGet(remotePath, localPath, options) ===> string
488
498
 
@@ -525,7 +535,7 @@ Downloads a file at remotePath to localPath using parallel reads for faster thro
525
535
  ```
526
536
 
527
537
 
528
- <a id="orgd38e78b"></a>
538
+ <a id="org429e96a"></a>
529
539
 
530
540
  ### put(src, remotePath, options) ==> string
531
541
 
@@ -575,7 +585,7 @@ Upload data from local system to remote server. If the `src` argument is a strin
575
585
  - **Tip:** If the src argument is a path string, consider just using `fastPut()`.
576
586
 
577
587
 
578
- <a id="orge130096"></a>
588
+ <a id="orgcab50c6"></a>
579
589
 
580
590
  ### fastPut(localPath, remotePath, options) ==> string
581
591
 
@@ -619,7 +629,7 @@ Uploads the data in file at `localPath` to a new file on remote server at `remot
619
629
  ```
620
630
 
621
631
 
622
- <a id="org4f2a1b1"></a>
632
+ <a id="orgc15d676"></a>
623
633
 
624
634
  ### append(input, remotePath, options) ==> string
625
635
 
@@ -663,7 +673,7 @@ Append the `input` data to an existing remote file. There is no integrity checki
663
673
  ```
664
674
 
665
675
 
666
- <a id="orga9acefe"></a>
676
+ <a id="org6fab860"></a>
667
677
 
668
678
  ### mkdir(path, recursive) ==> string
669
679
 
@@ -691,7 +701,7 @@ Create a new directory. If the recursive flag is set to true, the method will cr
691
701
  ```
692
702
 
693
703
 
694
- <a id="org9e51da5"></a>
704
+ <a id="org520f6bc"></a>
695
705
 
696
706
  ### rmdir(path, recursive) ==> string
697
707
 
@@ -721,7 +731,7 @@ Remove a directory. If removing a directory and recursive flag is set to `true`,
721
731
  ```
722
732
 
723
733
 
724
- <a id="org2054a87"></a>
734
+ <a id="org1be8ca9"></a>
725
735
 
726
736
  ### delete(path, noErrorOK) ==> string
727
737
 
@@ -750,7 +760,7 @@ Delete a file on the remote server.
750
760
  ```
751
761
 
752
762
 
753
- <a id="org7497d9f"></a>
763
+ <a id="org3d89131"></a>
754
764
 
755
765
  ### rename(fromPath, toPath) ==> string
756
766
 
@@ -779,7 +789,7 @@ Rename a file or directory from `fromPath` to `toPath`. You must have the necess
779
789
  ```
780
790
 
781
791
 
782
- <a id="orgb179088"></a>
792
+ <a id="org58dfee5"></a>
783
793
 
784
794
  ### posixRename(fromPath, toPath) ==> string
785
795
 
@@ -806,7 +816,7 @@ client.connect(config)
806
816
  ```
807
817
 
808
818
 
809
- <a id="org335a008"></a>
819
+ <a id="orgb319965"></a>
810
820
 
811
821
  ### chmod(path, mode) ==> string
812
822
 
@@ -835,7 +845,7 @@ Change the mode (read, write or execute permissions) of a remote file or directo
835
845
  ```
836
846
 
837
847
 
838
- <a id="org384f34a"></a>
848
+ <a id="orgf665476"></a>
839
849
 
840
850
  ### realPath(path) ===> string
841
851
 
@@ -846,26 +856,30 @@ Converts a relative path to an absolute path on the remote server. This method i
846
856
  - **path:** A file path, either relative or absolute. Can handle '.' and '..', but does not expand '~'.
847
857
 
848
858
 
849
- <a id="org8fa77d4"></a>
859
+ <a id="org409acd1"></a>
850
860
 
851
861
  ### cwd() ==> string
852
862
 
853
863
  Returns what the server believes is the current remote working directory.
854
864
 
855
865
 
856
- <a id="orgb824235"></a>
866
+ <a id="org668b085"></a>
857
867
 
858
- ### uploadDir(srcDir, dstDir, filter) ==> string
868
+ ### uploadDir(srcDir, dstDir, options) ==> string
859
869
 
860
870
  Upload the directory specified by `srcDir` to the remote directory specified by `dstDir`. The `dstDir` will be created if necessary. Any sub directories within `srcDir` will also be uploaded. Any existing files in the remote path will be overwritten.
861
871
 
862
872
  The upload process also emits 'upload' events. These events are fired for each successfully uploaded file. The `upload` event calls listeners with 1 argument, an object which has properties source and destination. The source property is the path of the file uploaded and the destination property is the path to where the file was uploaded. The purpose of this event is to provide some way for client code to get feedback on the upload progress. You can add your own listener using the `on()` method.
863
873
 
864
- The optionsl *filter* argument is a function which will be called for each item to be uploaded. The function will be called with two arguments. The first argument is the full path of the item to be uploaded and the second argument is a boolean, which will be true if the target path is for a directory. The filter function will be called for each item in the source path. If the function returns true, the item will be uploaded. If it returns false, it will be filtered and not uploaded. The filter function is called via the `Array.filter` method. These array comprehension methods are known to be unsafe for asynchronous functions. Therefore, only synchronous filter functions are supported at this time.
874
+ The 3rd argument is an options object with two supported properties, `filter` and `useFastput`.
875
+
876
+ The `filter` option is a function which will be called for each item to be uploaded. The function will be called with two arguments. The first argument is the full path of the item to be uploaded and the second argument is a boolean, which will be true if the target path is for a directory. The filter function will be called for each item in the source path. If the function returns true, the item will be uploaded. If it returns false, it will be filtered and not uploaded. The filter function is called via the `Array.filter` method. These array comprehension methods are known to be unsafe for asynchronous functions. Therefore, only synchronous filter functions are supported at this time.
877
+
878
+ The `useFastput` option is a boolean option. If `true`, the method will use the faster `fastPut()` method to upload files. Although this method is faster, it is not supported by all SFTP servers. Enabling this option when unsupported by the remote SFTP server will result in failures.
865
879
 
866
880
  - **srcDir:** A local file path specified as a string
867
881
  - **dstDir:** A remote file path specified as a string
868
- - **filter:** A filter predicate function which is called for each item in the source path. The argument will receive two arguments. The first is the full path to the item and the second is a boolean which will be true if the item is a directory. If the function returns true, the item will be uploaded, otherwise it will be filtered out and ignored.
882
+ - **options:** An options object which supports two properties, `filter` and `useFastput`. A filter predicate function which is called for each item in the source path. The argument will receive two arguments. The first is the full path to the item and the second is a boolean which will be true if the item is a directory. If the function returns true, the item will be uploaded, otherwise it will be filtered out and ignored. The `useFastput` option is a boolean option. If `true`, the method will use the faster, but less supported, `fastPut()` method to transfer files. The default is to use the slightly slower, but better supported, `put()` method.
869
883
 
870
884
  1. Example
871
885
 
@@ -918,19 +932,21 @@ The optionsl *filter* argument is a function which will be called for each item
918
932
  ```
919
933
 
920
934
 
921
- <a id="org46aa66f"></a>
935
+ <a id="org395ac4e"></a>
922
936
 
923
- ### downloadDir(srcDir, dstDir, filter) ==> string
937
+ ### downloadDir(srcDir, dstDir, options) ==> string
924
938
 
925
939
  Download the remote directory specified by `srcDir` to the local file system directory specified by `dstDir`. The `dstDir` directory will be created if required. All sub directories within `srcDir` will also be copied. Any existing files in the local path will be overwritten. No files in the local path will be deleted.
926
940
 
927
941
  The method also emites `download` events to provide a way to monitor download progress. The download event listener is called with one argument, an object with two properties, source and destination. The source property is the path to the remote file that has been downloaded and the destination is the local path to where the file was downloaded to. You can add a listener for this event using the `on()` method.
928
942
 
929
- The optional *filter* argument is a predicate function which will be called with two arguments for each potential item to be downloaded. The first argument is the full path of the item and the second argument is a boolean, which will be true if the item is a directory. If the function returns true, the item will be included in the download. If it returns false, it will be filtered and ignored. The filter function is called via the `Array.filter` method. These array comprehension methods are known to be unsafe for asynchronous functions. Therefore, only synchronous filter functions are supported at this time.
943
+ The `options` argument is an options object with two supported properties, `filter` and `useFastget`. The `filter` argument is a predicate function which will be called with two arguments for each potential item to be downloaded. The first argument is the full path of the item and the second argument is a boolean, which will be true if the item is a directory. If the function returns true, the item will be included in the download. If it returns false, it will be filtered and ignored. The filter function is called via the `Array.filter` method. These array comprehension methods are known to be unsafe for asynchronous functions. Therefore, only synchronous filter functions are supported at this time.
944
+
945
+ If the `useFastget` property is set to `true`, the method will use `fastGet()` to transfer files. The `fastGet` method is faster, but not supported by all SFTP services.
930
946
 
931
947
  - **srcDir:** A remote file path specified as a string
932
948
  - **dstDir:** A local file path specified as a string
933
- - **filter:** A predicate function called with two arguments, the full path to an item and a boolean value which will be true if the item is a directory. The function is called for each item in the download path.
949
+ - **options:** An object with two supported properties, `filter` and `useFastget`. The filter property is a function accepting two arguments, the full path to an item and a boolean value which will be true if the item is a directory. The function is called for each item in the download path and should return true to include the item and false to exclude it in the download. The `useFastget` property is a boolean. If true, the `fastGet()` method will be used to transfer files. If `false` (the default), the slower but better supported `get()` mehtod is used. .
934
950
 
935
951
  1. Example
936
952
 
@@ -981,7 +997,49 @@ The optional *filter* argument is a predicate function which will be called with
981
997
  ```
982
998
 
983
999
 
984
- <a id="orge3fcb81"></a>
1000
+ <a id="org8a73b83"></a>
1001
+
1002
+ ### createReadStream(remotePath, options)) ==> stream object
1003
+
1004
+ Returns a read stream object which is attached to the remote file specified by the `remotePath` argument. This is a low level method which just returns a read stream object. Client code is fully responsible for managing and releasing the resources associated with the stream once finished i.e. closing files, removing listeners etc.
1005
+
1006
+ - **remotePath:** A remote file path specified as a string
1007
+ - **options:** An options object. Supported properties are
1008
+ - **flags:** defaults to 'r'
1009
+ - **encoding:** defaults to null
1010
+ - **handle:** defaults to null
1011
+ - **mode:** 0o666
1012
+ - **autoClose:** defaults to true. If set to false, client code is responsible for closing file descriptors when finished
1013
+ - **start:** Default 0. Position to start reading bytes from (inclusive)
1014
+ - **end:** Postion to stop reading bytes (inclusive).
1015
+
1016
+
1017
+ <a id="orgb71fb72"></a>
1018
+
1019
+ ### createWriteStream(remotePath, options) ==> stream object
1020
+
1021
+ Returns a write stream object which is attached to the remote file specified in the `remotePath` argument. This is a low legvel function which just returns the stream object. Client code is fully responsible for managing that object, including closing any file descriptiors and removing listeners etc.
1022
+
1023
+ - **remotePath:** Path to the remote file specified as a string
1024
+ - **options:** An object containing stream options. Supported properties include
1025
+ - **flags:** default 'w'
1026
+ - **encoding:** defulat null
1027
+ - **mode:** 0o666
1028
+ - **autoClose:** true
1029
+ - **start:** Byte position to start writing from (inclusive). May require changing flag to 'r+'.
1030
+
1031
+
1032
+ <a id="org936f530"></a>
1033
+
1034
+ ### rcopy(srcPath, dstPath) ==> string
1035
+
1036
+ Perfrom a remote file copy. The file identified by the `srcPath` argument will be copied to the file specified as the `dstPath` argument. The directory where `dstPath` will be placed must exist, but the actual file must not i.e. no overwrites allowed.
1037
+
1038
+ - **srcPath:** Path to remote file to be copied specified as a string
1039
+ - **dstPath:** Path to where the copy will be creaeted specified as a string
1040
+
1041
+
1042
+ <a id="org62a8323"></a>
985
1043
 
986
1044
  ### end() ==> boolean
987
1045
 
@@ -1005,7 +1063,7 @@ Ends the current client session, releasing the client socket and associated reso
1005
1063
  ```
1006
1064
 
1007
1065
 
1008
- <a id="org4607268"></a>
1066
+ <a id="orgd1ebc36"></a>
1009
1067
 
1010
1068
  ### Add and Remove Listeners
1011
1069
 
@@ -1026,12 +1084,12 @@ Although normally not required, you can add and remove custom listeners on the s
1026
1084
  Removes the specified listener from the event specified in eventType. Note that the `end()` method automatically removes all listeners from the client object.
1027
1085
 
1028
1086
 
1029
- <a id="org66a7061"></a>
1087
+ <a id="orgdfff943"></a>
1030
1088
 
1031
1089
  # Platform Quirks & Warnings
1032
1090
 
1033
1091
 
1034
- <a id="org740553b"></a>
1092
+ <a id="org886fe2e"></a>
1035
1093
 
1036
1094
  ## Server Capabilities
1037
1095
 
@@ -1040,7 +1098,16 @@ All SFTP servers and platforms are not equal. Some facilities provided by `ssh2-
1040
1098
  One way to determine whether an issue you are encountering is due to `ssh2-sftp-client` or due to the remote server or server platform is to use a simple CLI sftp program, such as openSSH's sftp command. If you observe the same behaviour using plain `sftp` on the command line, the issue is likely due to server or remote platform limitations. Note that you should not use a GUI sftp client, like `Filezilla` or `winSCP` as such GUI programs often attempt to hide these server and platform incompatibilities and will take additional steps to simulate missing functionality etc. You want to use a CLI program which does as little as possible.
1041
1099
 
1042
1100
 
1043
- <a id="orgf97d765"></a>
1101
+ <a id="org1719a52"></a>
1102
+
1103
+ ## Issues with `fastPut()` and `fastGet()` Methods
1104
+
1105
+ The `fastPut()` and `fastGet()` methods are known to be somewhat dependent on SFTP server capabilities. Some SFTP servers just do not work correctly with concurrent connections and some are known to have issues with negotiating packet sizes. These issues can sometimes be resolved by tweaking the options supplied to the methods, such as setting number of concurrent connections or a psecific packet size.
1106
+
1107
+ To see an example of the type of issues you can observe with `fastPut()` or `fastGet()`, have a look at [issue 407](https://github.com/theophilusx/ssh2-sftp-client/issues/407), which describes the experiences of one user. Bottom line, when it works, it tends to work well and be significantly faster than using just `get()` or `put()`. However, when developing code to run against different SFTP servers, especially where you are unable to test against each server, you are likely better off just using `get()` and `put()` or structuring your code so that users can select which method to use (this is what `ssh2-sftp-client` does - for example, see the `!downloadDir()` and `uploadDir()` methods.
1108
+
1109
+
1110
+ <a id="org42a51fc"></a>
1044
1111
 
1045
1112
  ## Promises, Events & Managing Exceptions
1046
1113
 
@@ -1059,14 +1126,14 @@ The other area where additional events are fired is during the end() call. To de
1059
1126
  In addition to the promise based event handlers, `ssh2-sftp-client` also implements global event handlers which will catch any `error`, `end` or `close` events. Essentially, these global handlers only reset the `sftp` property of the client object, effectively ensuring any subsequent calls are rejected and in the case of an error, send the error to the console.
1060
1127
 
1061
1128
 
1062
- <a id="org6acf9c6"></a>
1129
+ <a id="orgd50a35a"></a>
1063
1130
 
1064
1131
  ### Adding Custom Handlers
1065
1132
 
1066
1133
  While the above strategies appear to work for the majority of use cases, there are always going to be edge cases which require more flexible or powerful event handling. To support this, the `on()` and `removeListener()` methods are provided. Any event listener added using the `on()` method will be added at the beginning of the list of handlers for that event, ensuring it will be called before any global or promise local events. See the documentation for the `on()` method for details.
1067
1134
 
1068
1135
 
1069
- <a id="orgb2f13ee"></a>
1136
+ <a id="org7b0987f"></a>
1070
1137
 
1071
1138
  ## Windows Based Servers
1072
1139
 
@@ -1075,7 +1142,7 @@ It appears that when the sftp server is running on Windows, a *ECONNRESET* error
1075
1142
  The best way to avoid this issue is to not re-use client objects. Always generate a new sftp client object for each new connection.
1076
1143
 
1077
1144
 
1078
- <a id="orgcd82e68"></a>
1145
+ <a id="orgbe548c2"></a>
1079
1146
 
1080
1147
  ## Don't Re-use SftpClient Objects
1081
1148
 
@@ -1084,12 +1151,12 @@ Due to an issue with *ECONNRESET* error signals when connecting to Windows based
1084
1151
  To avoid this problem, don't re-use SftpClient objects. Generate a new SftpClient object for each connection. You can perform multiple actions with a single connection e.g. upload multiple files, download multiple files etc, but after you have called end(), you should not try to re-use the object with a further connect() call. Create a new object instead.
1085
1152
 
1086
1153
 
1087
- <a id="orge1aa1d7"></a>
1154
+ <a id="orgfddb94b"></a>
1088
1155
 
1089
1156
  # FAQ
1090
1157
 
1091
1158
 
1092
- <a id="org9296404"></a>
1159
+ <a id="orgbd2a31e"></a>
1093
1160
 
1094
1161
  ## Remote server drops connections with only an end event
1095
1162
 
@@ -1100,7 +1167,7 @@ Clients first make an unauthenticated connection to the SFTP server to begin neg
1100
1167
  One way to avoid this type of issue is to add a delay between connection attempts. It does not need to be a very long delay - just sufficient to permit the previous connection to be authenticated. In fact, the default setting for openSSH is `10:30:60`, so you really just need to have enough delay to ensure that the 1st connection has completed authentication before the 11th connection is attempted.
1101
1168
 
1102
1169
 
1103
- <a id="org3e48901"></a>
1170
+ <a id="org9d72a61"></a>
1104
1171
 
1105
1172
  ## How can I pass writable stream as dst for get method?
1106
1173
 
@@ -1159,7 +1226,7 @@ sftp
1159
1226
  ```
1160
1227
 
1161
1228
 
1162
- <a id="org1474028"></a>
1229
+ <a id="org9fda086"></a>
1163
1230
 
1164
1231
  ## How can I upload files without having to specify a password?
1165
1232
 
@@ -1194,7 +1261,7 @@ sftp.connect({
1194
1261
  ```
1195
1262
 
1196
1263
 
1197
- <a id="orgb9a0ec1"></a>
1264
+ <a id="org2ccc5e8"></a>
1198
1265
 
1199
1266
  ## How can I connect through a Socks Proxy
1200
1267
 
@@ -1230,7 +1297,7 @@ client.connect({
1230
1297
  ```
1231
1298
 
1232
1299
 
1233
- <a id="org62e1f3a"></a>
1300
+ <a id="org5306b6f"></a>
1234
1301
 
1235
1302
  ## Timeout while waiting for handshake or handshake errors
1236
1303
 
@@ -1239,7 +1306,7 @@ Some users have encountered the error 'Timeout while waiting for handshake' or '
1239
1306
  When encountering this type of problem, one worthwhile approach is to use openSSH's CLI sftp program with the `-v` switch to raise loggin levels. This will show you what algorithms the CLI is using. You can then use this information to match the names with the accepted algorithm names documented in the `ssh2` README to set the properties in the `algorithms` object.
1240
1307
 
1241
1308
 
1242
- <a id="org7f26ef8"></a>
1309
+ <a id="org3a77529"></a>
1243
1310
 
1244
1311
  ## How can I limit upload/download speed
1245
1312
 
@@ -1279,7 +1346,7 @@ try {
1279
1346
  ```
1280
1347
 
1281
1348
 
1282
- <a id="orgfb52441"></a>
1349
+ <a id="org68eedc7"></a>
1283
1350
 
1284
1351
  ## Connection hangs or fails for larger files
1285
1352
 
@@ -1290,14 +1357,14 @@ A symptom of this issue is that you are able to upload small files, but uploadin
1290
1357
  For more explanation, see [issue #342](https://github.com/theophilusx/ssh2-sftp-client/issues/342).
1291
1358
 
1292
1359
 
1293
- <a id="orgbe2b99e"></a>
1360
+ <a id="orgd436fa7"></a>
1294
1361
 
1295
1362
  # Examples
1296
1363
 
1297
1364
  I have started collecting example scripts in the example directory of the repository. These are mainly scripts I have put together in order to investigate issues or provide samples for users. They are not robust, lack adequate error handling and may contain errors. However, I think they are still useful for helping developers see how the module and API can be used.
1298
1365
 
1299
1366
 
1300
- <a id="orgd2dcb05"></a>
1367
+ <a id="orge21d870"></a>
1301
1368
 
1302
1369
  # Troubleshooting
1303
1370
 
@@ -1312,14 +1379,14 @@ Note also that in the repository there are two useful directories. The first is
1312
1379
  The second directory is the validation directory. I have some very simple scripts in this directory which perform basic tasks using only the `ssh2` modules (no `ssh2-sftp-client` module). These can be useful when trying to determine if the issue is with the underlying `ssh2` module or the `ssh2-sftp-client` wrapper module.
1313
1380
 
1314
1381
 
1315
- <a id="orgae04d0a"></a>
1382
+ <a id="org287b86f"></a>
1316
1383
 
1317
1384
  ## Common Errors
1318
1385
 
1319
1386
  There are some common errors people tend to make when using Promises or Asyc/Await. These are by far the most common problem found in issues logged against this module. Please check for some of these before logging your issue.
1320
1387
 
1321
1388
 
1322
- <a id="org2018983"></a>
1389
+ <a id="org20a99bb"></a>
1323
1390
 
1324
1391
  ### Not returning the promise in a `then()` block
1325
1392
 
@@ -1356,7 +1423,7 @@ Note the `return` statements. These ensure that the Promise returned by the clie
1356
1423
  A common symptom of this type of error is for file uploads or download to fail to complete or for data in those files to be truncated. What is happening is that the connection is being ended before the transfer has completed.
1357
1424
 
1358
1425
 
1359
- <a id="org6a3654b"></a>
1426
+ <a id="org51482b4"></a>
1360
1427
 
1361
1428
  ### Mixing Promise Chains and Async/Await
1362
1429
 
@@ -1416,7 +1483,7 @@ async function doSftp() {
1416
1483
  ```
1417
1484
 
1418
1485
 
1419
- <a id="org36087fd"></a>
1486
+ <a id="orgb9a556a"></a>
1420
1487
 
1421
1488
  ### Try/catch and Error Handlers
1422
1489
 
@@ -1427,14 +1494,14 @@ The basic problem is that the try/catch block will have completed execution befo
1427
1494
  Error events are essentially asynchronous code. You don't know when such events will fire. Therefore, you cannot use a try/catch block to catch such event errors. Even creating an error handler which then throws an exception won't help as the key problem is that your try/catch block has already executed. There are a number of alternative ways to deal with this situation. However, the key symptom is that you see occasional uncaught error exceptions that cause your script to exit abnormally despite having try/catch blocks in your script. What you need to do is look at your code and find where errors are raised asynchronously and use an event handler or some other mechanism to manage any errors raised.
1428
1495
 
1429
1496
 
1430
- <a id="orgd09c9c8"></a>
1497
+ <a id="org0d148f0"></a>
1431
1498
 
1432
1499
  ### Server Differences
1433
1500
 
1434
1501
  Not all SFTP servers are the same. Like most standards, the SFTP protocol has some level of interpretation and allows different levels of compliance. This means there can be differences in behaviour between different servers and code which works with one server will not work the same with another. For example, the value returned by *realpath* for non-existent objects can differ significantly. Some servers will throw an error for a particular operation while others will just return null, some servers support concurrent operations (such as used by fastGet/fastPut) while others will not and of course, the text of error messages can vary significantly. In particular, we have noticed significant differences across different platforms. It is therefore advisable to do comprehensive testing when the SFTP server is moved to a new platform. This includes moving from to a cloud based service even if the underlying platform remains the same. I have noticed that some cloud platforms can generate unexpected events, possibly related to additional functionality or features associated with the cloud implementation. For example, it appears SFTP servers running under Azure will generate an error event when the connection is closed even when the client has requested the connection be terminated. The same SFTP server running natively on Windows does not appear to exhibit such behaviour.
1435
1502
 
1436
1503
 
1437
- <a id="org67e5a24"></a>
1504
+ <a id="orgf115391"></a>
1438
1505
 
1439
1506
  ### Avoid Concurrent Operations
1440
1507
 
@@ -1443,7 +1510,7 @@ Technically, SFTP should be able to perform multiple operations concurrently. As
1443
1510
  If you are going to try and perform concurrent operations, you need to test extensively and ensure you are using data which is large enough that context switching does occur (i.e. the request is not completed in a single run). Some SFTP servers will handle concurrent operations better than others.
1444
1511
 
1445
1512
 
1446
- <a id="org00c676b"></a>
1513
+ <a id="orgc7c437f"></a>
1447
1514
 
1448
1515
  ## Debugging Support
1449
1516
 
@@ -1476,7 +1543,7 @@ If you just want to see debug messages from `ssh2-sftp-client` and exclude debug
1476
1543
  ```
1477
1544
 
1478
1545
 
1479
- <a id="org6f7f508"></a>
1546
+ <a id="orgc79ef28"></a>
1480
1547
 
1481
1548
  # Logging Issues
1482
1549
 
@@ -1493,7 +1560,7 @@ I am happy to try and help diagnose and fix any issues you encounter while using
1493
1560
  Perhaps the best assistance is a minimal reproducible example of the issue. Once the issue can be readily reproduced, it can usually be fixed very quickly.
1494
1561
 
1495
1562
 
1496
- <a id="org2d2ad0d"></a>
1563
+ <a id="org64bd2a2"></a>
1497
1564
 
1498
1565
  # Pull Requests
1499
1566
 
@@ -1508,7 +1575,7 @@ This module will adopt a standard semantic versioning policy. Please indicate in
1508
1575
  - **Bug Fix:** No change to functionality or features. Simple fix of an existing bug.
1509
1576
 
1510
1577
 
1511
- <a id="orgba32989"></a>
1578
+ <a id="org709b0f9"></a>
1512
1579
 
1513
1580
  # Contributors
1514
1581