ssh2-sftp-client 10.0.3 → 12.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +177 -142
- package/README.org +152 -50
- package/package.json +9 -10
- package/src/index.js +76 -147
- package/src/utils.js +83 -38
package/README.org
CHANGED
|
@@ -3,41 +3,145 @@
|
|
|
3
3
|
|
|
4
4
|
* Overview
|
|
5
5
|
|
|
6
|
-
an SFTP client for node.js
|
|
7
|
-
|
|
6
|
+
This package provides the class SftpClient, an SFTP client for node.js. It is a promise
|
|
7
|
+
based decorator class around the excellent [[https://github.com/mscdex/ssh2][SSH2]] package, which provides a pure node
|
|
8
|
+
Javascript event based ssh2 implementation.
|
|
8
9
|
|
|
9
10
|
Documentation on the methods and available options in the underlying modules can
|
|
10
|
-
be found on the [[https://github.com/mscdex/ssh2][SSH2]] project pages. As
|
|
11
|
+
be found on the [[https://github.com/mscdex/ssh2][SSH2]] project pages. As the ssh2-sftp-client package is just a wrapper around the
|
|
11
12
|
~ssh2~ module, you will find lots of useful information, tips and examples in the ~ssh2~
|
|
12
13
|
repository.
|
|
13
14
|
|
|
14
|
-
Current stable release is *
|
|
15
|
+
Current stable release is *v12.0.0.
|
|
15
16
|
|
|
16
|
-
Code has been tested against Node versions
|
|
17
|
-
|
|
18
|
-
performance improvement has been observed with versions >= 18. Version v16 is
|
|
19
|
-
significantly slower.
|
|
20
|
-
|
|
21
|
-
Node versions < 16.x are not supported.
|
|
17
|
+
Code has been tested against Node versions 20.18.3, 22.14.0 and 23.8.0. Node versions
|
|
18
|
+
prior to v20.x are not supported.
|
|
22
19
|
|
|
23
20
|
If you find this module useful and you would like to support the on-going maintenance and
|
|
24
21
|
support of users, please consider making a small [[https://square.link/u/gB2kSdkY?src=embed][donation]].
|
|
25
22
|
|
|
26
|
-
** Version
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
23
|
+
** Version 12.0.0 Changes
|
|
24
|
+
|
|
25
|
+
The big and breaking change in this version is the removal of the connection retry
|
|
26
|
+
code. This module no longer supports automatic connection retries when establishing the
|
|
27
|
+
initial connection. This functionality was removed for a number of reasons. The main
|
|
28
|
+
reasons are that it significantly complicated the connection handling code and error
|
|
29
|
+
handling in particular and it was seldom useful. In most cases, if the cvonnection failed
|
|
30
|
+
on thye first attempt, it would also fail in all subsequent attempts and often resulted in
|
|
31
|
+
just extending the time to failure as multiple retries were attempted.
|
|
32
|
+
|
|
33
|
+
The other reason this functionality was removed is because if you really want it, it is
|
|
34
|
+
fairly straight forward to create your own wrapper around the connect call which uses one
|
|
35
|
+
of the mnay promise retry packages available. Doing this is significantly easier when you
|
|
36
|
+
don't need to cater for all the variables available when making a connection i.e. strict
|
|
37
|
+
host verification, ssh agents, keys, proxies etc.
|
|
38
|
+
|
|
39
|
+
** Background
|
|
40
|
+
|
|
41
|
+
In basic terms =ssh2-sftp-client= is a simple wrapper around the =ssh2= package which provides
|
|
42
|
+
a promise base API for interacting with a remote SFTP server . The =ssh2= package provides
|
|
43
|
+
an event based API for interacting with the ~ssh~ protocol. The ~ssh2-sftp-client~ package
|
|
44
|
+
uses the ~sftp~ subsystem of this protocol to implement the basic operations typically
|
|
45
|
+
associated with an ~sftp~ client.
|
|
46
|
+
|
|
47
|
+
Wrapping an event based API with a promised based API comes with a number of
|
|
48
|
+
challenges. In particular, efficiently and reliably managing events within the context of
|
|
49
|
+
asynchrounous code execution. This package uses the following strategies;
|
|
50
|
+
|
|
51
|
+
- All direct interactions with the ~ssh2~ API are wrapped in promise objects. When the
|
|
52
|
+
API call succeeds, the assoiated promise will be sucessfully resolved. When an error occurs,
|
|
53
|
+
the promise is rejected.
|
|
54
|
+
|
|
55
|
+
- An error can either be due to a low level network error, such as a lost connection
|
|
56
|
+
to the remote server or due to an operational error, such as a file not
|
|
57
|
+
existing or not having the appropriate permissions for access.
|
|
58
|
+
|
|
59
|
+
- Each of the available ~SftpClient~ methods wrap the method call inside a promise. In
|
|
60
|
+
crwating eacvh promise, the class adds temporary event listenrs for the error, end
|
|
61
|
+
and close events and links those liseners to the method's promise via it's ~reject()~
|
|
62
|
+
method.
|
|
63
|
+
|
|
64
|
+
- If a promise is waiting to be fulfilled when either of the two types of errors
|
|
65
|
+
occurs, the error will be communicated back to client code via a rejected promise.
|
|
66
|
+
|
|
67
|
+
- When the ~ssh2~ emitter raises an event outside the context of any promise, that event
|
|
68
|
+
will be handled by global event handlers. By default, these event handlers will log
|
|
69
|
+
the event and will invalidate any existing socket connection objects, preventing any
|
|
70
|
+
further API calls until a new connection is extablished.
|
|
71
|
+
|
|
72
|
+
- The ~SftpClient~ class constructor supports an optoinal second argument which is an
|
|
73
|
+
objedct whi8ch can have any of three properties representing event callbaqck
|
|
74
|
+
funct5ions which will be executed for each of the possible events error, end and
|
|
75
|
+
close.
|
|
76
|
+
|
|
77
|
+
The need for both global listeners and temporary promise listeners is because network end,
|
|
78
|
+
close or error events can occur at any time, including in-betwseen API calls. During an
|
|
79
|
+
API call, a promise is active and can be used to communicate event information back to the calling
|
|
80
|
+
code via normal promise communication means i.e. async/await with try/catch or promise chain's then/catch
|
|
81
|
+
mechanism. However, outside API calls, no promise exists and there is no reliable
|
|
82
|
+
mechanism to return error and other event information back to calling code. You cannot
|
|
83
|
+
reliably use try/cdatch to catch errorsx thrown inside event listenrs as you lack control
|
|
84
|
+
over when the listener code runs. Your try/catch block can easily complete before the
|
|
85
|
+
error is raised as there is no equivalent /await/ type functionality in this situation.
|
|
86
|
+
|
|
87
|
+
As there is no simple default way to return error and other event information back to the
|
|
88
|
+
calling code, ~ssh2-sftp-client~ doesn't try to. Instead, the default action is
|
|
89
|
+
to just log the event information and invalidate any existing sftp connections. This
|
|
90
|
+
strategy is often sufficient for many use cases. For those cases where it isn't, client
|
|
91
|
+
code can pass in end, close and/or error listener functions when instantiating the
|
|
92
|
+
~SftpClient~ object. If provided, these listners will be executed whenever the default
|
|
93
|
+
global listeners are executed, which is whenever the ~ssh2~ event emitter raises an end,
|
|
94
|
+
close or error event which is not handled by one of the temporary promise linked event
|
|
95
|
+
listeners.
|
|
96
|
+
|
|
97
|
+
Version 11 of ~ssh2-sftp-client~ also changes the behaviour of the temporary promise linked
|
|
98
|
+
/end/ and /close/ listeners. Prior to versioln 11, these listeners did not reject
|
|
99
|
+
promises. They would only invalidate the underlying ~ssh~ connection object. Only the /error/
|
|
100
|
+
listener would actually reject the associated promise. This was done because you cannot
|
|
101
|
+
guarantee the order in which events are responded to.
|
|
102
|
+
|
|
103
|
+
In most cases, events will occur in the order /error/, /end/ and then /close/. The error event
|
|
104
|
+
would contain details about the cause of the error while the end and close events just
|
|
105
|
+
communicvate that these events have been raised. The normal flow would be
|
|
106
|
+
|
|
107
|
+
- Error event occurs including error cause description. Listener catches event,
|
|
108
|
+
creates an error object and calls the associated promises reject function to reject
|
|
109
|
+
the promise. The calling process receives a rejected promise object.
|
|
110
|
+
|
|
111
|
+
- End event occurs. The end listener catches the event and marks the connection object
|
|
112
|
+
as invalid as the socket connection has been ended. There is no need to call the
|
|
113
|
+
reject method of the associated promise as it has already been called by the error
|
|
114
|
+
listener and you can only call one promise resolution function.
|
|
115
|
+
|
|
116
|
+
- Close event occurs. This event means the socket connection has been closed. The
|
|
117
|
+
listener will ensure any connection information has been invalidated. Aga8in, ther
|
|
118
|
+
is no need to call the reject method of the associated promise as it has already
|
|
119
|
+
been called by the error listener.
|
|
120
|
+
|
|
121
|
+
In some cases, no ende event is raised and you only get an error event followed by a close
|
|
122
|
+
event. In versions of ~ssh2-sftp-client~ prior to version 11, neither the end or the close
|
|
123
|
+
listeners attempted to call the reject method of the associated promise. It was assumed
|
|
124
|
+
that all sftp servers would raise an error event whenever a connection was unexpectedly
|
|
125
|
+
ended or closed. Unfortunately, it turns out some sftp servers are not well behaved and
|
|
126
|
+
will terminate the connection without providing any error information or raising an error
|
|
127
|
+
event. When this occurred in versions prior to version 11, it could result in either an
|
|
128
|
+
API call hanging because its associated promise never gets rejected or resolved or the
|
|
129
|
+
call gets rejected with a timeout error aftrer a significant delay.
|
|
130
|
+
|
|
131
|
+
In order to handle the possible hanging issue in version 11, the temporary promise linked
|
|
132
|
+
end and close listeners have been updated to always call the promise's reject function if
|
|
133
|
+
they fire. While this works, it can cause a minor issue. As wse cannot gurantee the order
|
|
134
|
+
in which events are resonded to by listeners, it is possible that either the end or close
|
|
135
|
+
listener may be executed before the error listener. When this occurs, the promise is
|
|
136
|
+
rejected, but the only information wse have at that point is that the promise wsas reject
|
|
137
|
+
due to either an end or close event. We don't yet have any details regarding what error
|
|
138
|
+
has caused the unexpected end or close event. Furthermore, because only the first promise
|
|
139
|
+
resolution function call has any effect, calling reject within the error listener
|
|
140
|
+
(assuming an error event does eventually arrive) has no effect and does not communicate
|
|
141
|
+
error information back to the caller. This means that in some circumstances, especially
|
|
142
|
+
when working with some poorly behaved sftp servers, an sftp connection will be lost/closed
|
|
143
|
+
with no indication as to reason. This can make diagnosis and bug tracking frustrating.
|
|
37
144
|
|
|
38
|
-
- Various minor documentation fixes and some minor fixes for typos in option property
|
|
39
|
-
names.
|
|
40
|
-
|
|
41
145
|
* Installation
|
|
42
146
|
|
|
43
147
|
#+begin_src shell
|
|
@@ -66,12 +170,12 @@ support of users, please consider making a small [[https://square.link/u/gB2kSdk
|
|
|
66
170
|
|
|
67
171
|
* Documentation
|
|
68
172
|
|
|
69
|
-
The connection options are the same as those offered by the underlying SSH2
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
details relating to setting various key
|
|
74
|
-
part of the ssh2 protocol.
|
|
173
|
+
The connection options are the same as those offered by the underlying SSH2 module, with
|
|
174
|
+
just a couple of additional properties added to add a ~debug~ function and set the
|
|
175
|
+
~promiseLimit~ property. For full details on the other properties, please see
|
|
176
|
+
[[https://github.com/mscdex/ssh2#user-content-client-methods][SSH2 client methods]]. In
|
|
177
|
+
particular, see the ~ssh2~ documentation for details relating to setting various key
|
|
178
|
+
exchange and encryption/signing algorithms used as part of the ssh2 protocol.
|
|
75
179
|
|
|
76
180
|
All the methods will return a Promise, except for ~on(), ~removeListener()~, ~createReadStream~
|
|
77
181
|
and ~createWriteStream~, which are typically only used in special use cases.
|
|
@@ -144,7 +248,7 @@ refer your issues to the maintainers of those modules.
|
|
|
144
248
|
|
|
145
249
|
** Methods
|
|
146
250
|
|
|
147
|
-
*** new SftpClient(name) ===> SFTP client object
|
|
251
|
+
*** new SftpClient(name, callbacks) ===> SFTP client object
|
|
148
252
|
|
|
149
253
|
Constructor to create a new ~ssh2-sftp-client~ object. An optional ~name~ string
|
|
150
254
|
can be provided, which will be used in error messages to help identify which
|
|
@@ -152,8 +256,14 @@ client has thrown the error.
|
|
|
152
256
|
|
|
153
257
|
**** Constructor Arguments
|
|
154
258
|
|
|
155
|
-
- name :: string. An optional name string used in error messages
|
|
156
|
-
|
|
259
|
+
- name :: string. An optional name string used in error messages. Defaults to the string
|
|
260
|
+
'sftp'
|
|
261
|
+
- callbacks :: object. An object with the properties error, end and close. Associted with
|
|
262
|
+
each property is a function that will be executed whenever the associated global event
|
|
263
|
+
listener for error, end or close is executed. The error function should accept one
|
|
264
|
+
argument. The end and close functions have no arguments. Default functions just print a
|
|
265
|
+
message to ~console.log()~.
|
|
266
|
+
|
|
157
267
|
**** Example Use
|
|
158
268
|
|
|
159
269
|
#+begin_src javascript
|
|
@@ -189,23 +299,18 @@ available [[https://github.com/mscdex/ssh2#user-content-client-methods][here]]
|
|
|
189
299
|
|
|
190
300
|
**** Connection Options
|
|
191
301
|
|
|
192
|
-
This module is based on the excellent [[https://github.com/mscdex/ssh2#client][SSH2]]
|
|
193
|
-
client and server library and provides much more
|
|
194
|
-
connectivity. Many of the connect options provided by that
|
|
195
|
-
relevant for SFTP connections. It is recommended you keep the config
|
|
196
|
-
the minimum needed and stick to the options listed in the ~commonOpts~ below.
|
|
197
|
-
|
|
198
|
-
The ~retries~, ~retry_factor~ and ~retry_minTimeout~ options are not part of the
|
|
199
|
-
SSH2 module. These are part of the configuration for the [[https://www.npmjs.com/package/retry][retry]] package and what
|
|
200
|
-
is used to enable retrying of sftp connection attempts. See the documentation
|
|
201
|
-
for that package for an explanation of these values.
|
|
302
|
+
This module is based on the excellent [[https://github.com/mscdex/ssh2#client][SSH2]]
|
|
303
|
+
module. That module is a general SSH2 client and server library and provides much more
|
|
304
|
+
functionality than just SFTP connectivity. Many of the connect options provided by that
|
|
305
|
+
module are less relevant for SFTP connections. It is recommended you keep the config
|
|
306
|
+
options to the minimum needed and stick to the options listed in the ~commonOpts~ below.
|
|
202
307
|
|
|
203
|
-
The ~promiseLimit~ is
|
|
308
|
+
The ~promiseLimit~ is an option which is not part of the ~ssh2~ module and is specific to
|
|
204
309
|
~ssh2-sftp-client~. It is a property used to limit the maximum number of concurrent promises
|
|
205
310
|
possible when either downloading or uploading a directory tree using the ~downloadDir()~ or
|
|
206
|
-
~uploadDir()~ methods. The default setting for this property is 10. *NOTE*: bigger
|
|
311
|
+
~uploadDir()~ methods. The default setting for this property is 10. *NOTE*: bigger does not
|
|
207
312
|
mean better. Many factors can affect what is the ideal setting for ~promiseLimit~. If it is
|
|
208
|
-
too large, any benefits are lost while node spends time switching contexts and/or
|
|
313
|
+
too large, any benefits are lost while node spends time switching contexts and/or with
|
|
209
314
|
the overheads associated with creating and cleaning up promises. Lots of factors can
|
|
210
315
|
affect what the setting should be, including size of files, number of files, speed of
|
|
211
316
|
network, version of node, capabilities of remote sftp server etc. A setting of 10 seems to
|
|
@@ -230,9 +335,6 @@ work out what is the best maximum size before you begin to see a performance dro
|
|
|
230
335
|
strictVendor: true, // boolean - Performs a strict server vendor check
|
|
231
336
|
debug: myDebug,// function - Set this to a function that receives a single
|
|
232
337
|
// string argument to get detailed (local) debug information.
|
|
233
|
-
retries: 2, // integer. Number of times to retry connecting
|
|
234
|
-
retry_factor: 2, // integer. Time factor used to calculate time between retries
|
|
235
|
-
retry_minTimeout: 2000, // integer. Minimum timeout between attempts
|
|
236
338
|
promiseLimit: 10, // max concurrent promises for downloadDir/uploadDir
|
|
237
339
|
};
|
|
238
340
|
|
|
@@ -989,7 +1091,7 @@ remote SFTP server will result in failures.
|
|
|
989
1091
|
|
|
990
1092
|
*** downloadDir(srcDir, dstDir, options) ==> string
|
|
991
1093
|
|
|
992
|
-
Download the remote directory specified by ~srcDir~ to the local file system
|
|
1094
|
+
Download the content of the remote directory specified by ~srcDir~ to the local file system
|
|
993
1095
|
directory specified by ~dstDir~. The ~dstDir~ directory will be created if
|
|
994
1096
|
required. All sub directories within ~srcDir~ will also be copied. Any existing
|
|
995
1097
|
files in the local path will be overwritten. No files in the local path will be
|
package/package.json
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ssh2-sftp-client",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "12.0.0",
|
|
4
4
|
"description": "ssh2 sftp client for node",
|
|
5
5
|
"main": "src/index.js",
|
|
6
6
|
"repository": {
|
|
7
7
|
"type": "git",
|
|
8
|
-
"url": "https://github.com/theophilusx/ssh2-sftp-client"
|
|
8
|
+
"url": "git+https://github.com/theophilusx/ssh2-sftp-client.git"
|
|
9
9
|
},
|
|
10
10
|
"keywords": [
|
|
11
11
|
"sftp",
|
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
"lint": "eslint \"src/**/*.js\" \"test/**/*.js\""
|
|
19
19
|
},
|
|
20
20
|
"engines": {
|
|
21
|
-
"node": ">=
|
|
21
|
+
"node": ">=18.20.4"
|
|
22
22
|
},
|
|
23
23
|
"author": "Tim Cross",
|
|
24
24
|
"email": "theophilusx@gmail.com",
|
|
@@ -34,27 +34,26 @@
|
|
|
34
34
|
],
|
|
35
35
|
"license": "Apache-2.0",
|
|
36
36
|
"devDependencies": {
|
|
37
|
-
"chai": "^4.
|
|
38
|
-
"chai-as-promised": "^7.1.
|
|
37
|
+
"chai": "^4.5.0",
|
|
38
|
+
"chai-as-promised": "^7.1.2",
|
|
39
39
|
"chai-subset": "^1.6.0",
|
|
40
40
|
"checksum": "^1.0.0",
|
|
41
41
|
"dotenv": "^16.0.0",
|
|
42
|
-
"eslint": "^
|
|
42
|
+
"eslint": "^9.6.0",
|
|
43
43
|
"eslint-config-prettier": "^9.0.0",
|
|
44
44
|
"eslint-plugin-mocha": "^10.2.0",
|
|
45
45
|
"eslint-plugin-node": "^11.1.0",
|
|
46
46
|
"eslint-plugin-promise": "^6.0.0",
|
|
47
|
-
"eslint-plugin-unicorn": "^
|
|
47
|
+
"eslint-plugin-unicorn": "^54.0.0",
|
|
48
48
|
"mocha": "^10.0.0",
|
|
49
49
|
"moment": "^2.29.1",
|
|
50
|
-
"nyc": "^
|
|
50
|
+
"nyc": "^17.0.0",
|
|
51
51
|
"prettier": "^3.0.3",
|
|
52
52
|
"through2": "^4.0.2",
|
|
53
53
|
"winston": "^3.11.0"
|
|
54
54
|
},
|
|
55
55
|
"dependencies": {
|
|
56
56
|
"concat-stream": "^2.0.0",
|
|
57
|
-
"
|
|
58
|
-
"ssh2": "^1.15.0"
|
|
57
|
+
"ssh2": "^1.16.0"
|
|
59
58
|
}
|
|
60
59
|
}
|