tus-server 2.0.2 → 2.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/CHANGELOG.md +10 -0
- data/README.md +154 -107
- data/lib/tus/server.rb +34 -13
- data/lib/tus/storage/filesystem.rb +14 -2
- data/lib/tus/storage/gridfs.rb +2 -0
- data/lib/tus/storage/s3.rb +27 -35
- data/tus-server.gemspec +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 16cf0c007be7d383ca0bf2d4aefa6d6c477ac7d32b1905a8ad6c0413da233e57
|
4
|
+
data.tar.gz: 258d7d591fc1416468ea21ea42d29ed53f2a64eeade9e5293bdbecc3e5f6a599
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d642a6f120fd220af27bfc7cf8904459acafa6191934a4922913b10b296e0308cd3093223b39317ffb901318e83da2bcd3fa056cf7d2a36a2f7131ce18b8d8ca
|
7
|
+
data.tar.gz: c0bf56b77ee95fa1aab167f29fd38feabb1919f7e7d85b1c7705c5adbcba97f9612b5633880af9b603b433a3778518b7875bd5bfafff4f446c476d621c0b9c99
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,13 @@
|
|
1
|
+
## 2.1.0 (2018-05-15)
|
2
|
+
|
3
|
+
* Add `:download_url` server option for redirecting to a download URL (@janko-m)
|
4
|
+
|
5
|
+
* Allow application servers to serve files stored on disk via the `Rack::Sendfile` middleware (@janko-m)
|
6
|
+
|
7
|
+
* Reject `Upload-Metadata` which contains key-value pairs separated by spaces (@janko-m)
|
8
|
+
|
9
|
+
* Don't overwite info file if it already exists in `Tus::Storage::FileSystem` (@janko-m)
|
10
|
+
|
1
11
|
## 2.0.2 (2017-12-24)
|
2
12
|
|
3
13
|
* Handle `name` and `type` metadata for Uppy compatibility (@janko-m)
|
data/README.md
CHANGED
@@ -12,62 +12,81 @@ A Ruby server for the [tus resumable upload protocol]. It implements the core
|
|
12
12
|
## Installation
|
13
13
|
|
14
14
|
```rb
|
15
|
-
|
15
|
+
# Gemfile
|
16
|
+
gem "tus-server", "~> 2.0"
|
16
17
|
```
|
17
18
|
|
18
19
|
## Usage
|
19
20
|
|
20
|
-
|
21
|
-
|
22
|
-
part of your main app.
|
21
|
+
The gem provides a `Tus::Server` Rack app, which you can mount inside your
|
22
|
+
main application. If you're using Rails, you can mount it in `config/routes.rb`:
|
23
23
|
|
24
24
|
```rb
|
25
|
-
# config.
|
25
|
+
# config/routes.rb (Rails)
|
26
|
+
Rails.application.routes.draw do
|
27
|
+
# ...
|
28
|
+
mount Tus::Server => "/files"
|
29
|
+
end
|
30
|
+
```
|
31
|
+
|
32
|
+
Otherwise you can run it in `config.ru`:
|
33
|
+
|
34
|
+
```rb
|
35
|
+
# config.ru (Rack)
|
26
36
|
require "tus/server"
|
27
37
|
|
28
38
|
map "/files" do
|
29
39
|
run Tus::Server
|
30
40
|
end
|
31
|
-
|
32
|
-
run YourApp
|
33
41
|
```
|
34
42
|
|
35
|
-
While this is the most flexible option, it's not optimal in terms of
|
36
|
-
performance; see the [Goliath](#goliath) section for an alternative approach.
|
37
|
-
|
38
43
|
Now you can tell your tus client library (e.g. [tus-js-client]) to use this
|
39
44
|
endpoint:
|
40
45
|
|
41
46
|
```js
|
42
47
|
// using tus-js-client
|
43
48
|
new tus.Upload(file, {
|
44
|
-
endpoint: "
|
45
|
-
chunkSize: 5*1024*1024, // required unless using Goliath
|
49
|
+
endpoint: "/files",
|
50
|
+
chunkSize: 5*1024*1024, // required unless using Goliath
|
46
51
|
// ...
|
47
52
|
})
|
48
53
|
```
|
49
54
|
|
50
|
-
|
51
|
-
|
52
|
-
|
55
|
+
By default uploaded files will be stored in the `data/` directory. After the
|
56
|
+
upload is complete, you'll probably want to attach the uploaded file to a
|
57
|
+
database record. [Shrine] is one file attachment library that integrates nicely
|
58
|
+
with tus-ruby-server, see [shrine-tus-demo] for an example integration.
|
53
59
|
|
54
60
|
### Goliath
|
55
61
|
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
|
62
|
+
Running the tus server alongside your main app using classic web servers like
|
63
|
+
Puma or Unicorn is probably fine for most cases, however, it does come with a
|
64
|
+
few gotchas. First, since these web servers don't accept partial requests
|
65
|
+
(request where the request body hasn't been fully received), the tus client
|
66
|
+
must be configured to split the upload into multiple requests. Second, since
|
67
|
+
web workers are tied for the duration of the request, serving uploaded files
|
68
|
+
through the tus server app could significantly impact request throughput, so
|
69
|
+
you need to be careful to avoid that.
|
70
|
+
|
71
|
+
There is an alternative. [Goliath] is an asychronous web server built on top of
|
72
|
+
[EventMachine], which supports streaming requests and streaming responses.
|
73
|
+
|
74
|
+
* Asynchronous streaming requests allows the tus server to begin saving
|
75
|
+
uploaded data while it's still being received. If the request is interrupted,
|
76
|
+
the tus server will attempt to save as much of the data that was received so
|
77
|
+
far. This means it's not necessary for the tus client to split the upload
|
78
|
+
into multiple smaller requests.
|
64
79
|
|
65
|
-
|
66
|
-
|
80
|
+
* Asynchronous streaming responses allows the tus server to stream large files
|
81
|
+
with very small impact to the request throughput.
|
82
|
+
|
83
|
+
Since Goliath is web server, to run tus server on it we'll have to run it as a
|
84
|
+
standalone web app. It's recommended that you use [goliath-rack_proxy] for
|
85
|
+
running your tus server app:
|
67
86
|
|
68
87
|
```rb
|
69
88
|
# Gemfile
|
70
|
-
gem "tus-server", "~>
|
89
|
+
gem "tus-server", "~> 2.0"
|
71
90
|
gem "goliath-rack_proxy", "~> 1.0"
|
72
91
|
```
|
73
92
|
```rb
|
@@ -92,66 +111,34 @@ path you can use `Rack::Builder`:
|
|
92
111
|
```rb
|
93
112
|
class GoliathTusServer < Goliath::RackProxy
|
94
113
|
rack_app Rack::Builder.new {
|
95
|
-
map
|
96
|
-
run Tus::Server
|
97
|
-
end
|
114
|
+
map("/files") { run Tus::Server }
|
98
115
|
}
|
99
116
|
rewindable_input false # set to true if you're using checksums
|
100
117
|
end
|
101
118
|
```
|
102
119
|
|
103
|
-
|
104
|
-
|
105
|
-
Like Goliath, Unicorn also support streaming uploads, and tus-ruby-server knows
|
106
|
-
how to automatically recover from `Unicorn::ClientShutdown` exceptions during
|
107
|
-
upload, storing data that it has received up until that point. Just note that
|
108
|
-
in order to achieve streaming uploads, Nginx should be configured **not** to
|
109
|
-
buffer incoming requests, and to disable worker timeout to enable long running
|
110
|
-
upload requests:
|
120
|
+
In this case you'll have to configure the tus client to point to the standalone
|
121
|
+
Goliath app:
|
111
122
|
|
112
|
-
```
|
113
|
-
|
114
|
-
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
for resumable upload to be possible the client needs to send data in multiple
|
119
|
-
upload requests (which can then be retried individually).
|
120
|
-
|
121
|
-
Unless you're using the "checksum" tus feature, you might want to consider
|
122
|
-
disabling rewindability of request body, to prevent Unicorn from additionally
|
123
|
-
caching received data onto the disk (since that's not necessary unless request
|
124
|
-
body needs to be rewinded).
|
125
|
-
|
126
|
-
```rb
|
127
|
-
# config/unicorn.rb
|
128
|
-
# ...
|
129
|
-
|
130
|
-
rewindable_input false
|
123
|
+
```js
|
124
|
+
// using tus-js-client
|
125
|
+
new tus.Upload(file, {
|
126
|
+
endpoint: "http://localhost:9000/files",
|
127
|
+
// ...
|
128
|
+
})
|
131
129
|
```
|
132
130
|
|
133
|
-
### Other web servers
|
134
|
-
|
135
|
-
It's perfectly feasible to run tus-ruby-server on web servers other than
|
136
|
-
Goliath or Unicorn (even necessary if you want to run it inside another app).
|
137
|
-
Just keep in mind that most other web servers don't support request streaming,
|
138
|
-
which means that tus-ruby-server will be able to start processing upload
|
139
|
-
requess only once the whole request body has been received. Additionally,
|
140
|
-
incomplete upload requests won't be forwarded to tus-ruby-server, so in order
|
141
|
-
for resumable upload to be possible the client needs to send data in multiple
|
142
|
-
upload requests (which can then be retried individually).
|
143
|
-
|
144
131
|
## Storage
|
145
132
|
|
146
133
|
### Filesystem
|
147
134
|
|
148
|
-
By default `Tus::Server` stores uploaded files
|
149
|
-
|
135
|
+
By default `Tus::Server` stores uploaded files in the `data/` directory. You
|
136
|
+
can configure a different directory:
|
150
137
|
|
151
138
|
```rb
|
152
139
|
require "tus/storage/filesystem"
|
153
140
|
|
154
|
-
Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/
|
141
|
+
Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/tus")
|
155
142
|
```
|
156
143
|
|
157
144
|
If the configured directory doesn't exist, it will automatically be created.
|
@@ -171,59 +158,56 @@ store files on the filesystem as they won't persist.
|
|
171
158
|
All these are reasons why you might store uploaded data on a different storage,
|
172
159
|
and luckily tus-ruby-server ships with two more storages.
|
173
160
|
|
174
|
-
|
175
|
-
|
176
|
-
MongoDB has a specification for storing and retrieving large files, called
|
177
|
-
"[GridFS]". Tus-ruby-server ships with `Tus::Storage::Gridfs` that you can
|
178
|
-
use, which uses the [Mongo] gem.
|
179
|
-
|
180
|
-
```rb
|
181
|
-
gem "mongo", ">= 2.2.2", "< 3"
|
182
|
-
```
|
161
|
+
#### Serving files
|
183
162
|
|
184
|
-
|
185
|
-
|
163
|
+
If your retrieving uploaded files through the download endpoint, by default the
|
164
|
+
files will be served through the Ruby application. However, that's very
|
165
|
+
inefficient, as web workers are tied when serving download requests and cannot
|
166
|
+
serve additional requests for that duration.
|
186
167
|
|
187
|
-
|
188
|
-
|
189
|
-
|
168
|
+
Therefore, it's highly recommended to delegate serving uploaded files to your
|
169
|
+
frontend server (Nginx, Apache). This can be achieved with the
|
170
|
+
`Rack::Sendfile` middleware, see its [documentation][Rack::Sendfile] to learn
|
171
|
+
more about how to use it with popular frontend servers.
|
190
172
|
|
191
|
-
|
173
|
+
If you're using Rails, you can enable the `Rack::Sendfile` middleware by
|
174
|
+
setting the `config.action_dispatch.x_sendfile_header` value accordingly:
|
192
175
|
|
193
176
|
```rb
|
194
|
-
|
177
|
+
config.action_dispatch.x_sendfile_header = "X-Sendfile" # Apache and lighttpd
|
178
|
+
# or
|
179
|
+
config.action_dispatch.x_sendfile_header = "X-Accel-Redirect" # Nginx
|
195
180
|
```
|
196
181
|
|
197
|
-
|
198
|
-
|
182
|
+
Otherwise you can add the `Rack::Sendfile` middleware to the stack in
|
183
|
+
`config.ru`:
|
199
184
|
|
200
185
|
```rb
|
201
|
-
|
186
|
+
use Rack::Sendfile, "X-Sendfile" # Apache and lighttpd
|
187
|
+
# or
|
188
|
+
use Rack::Sendfile, "X-Accel-Redirect" # Nginx
|
202
189
|
```
|
203
190
|
|
204
|
-
Note that if you're using the [concatenation] tus feature with Gridfs, all
|
205
|
-
partial uploads except the last one are required to fill in their Gridfs
|
206
|
-
chunks, meaning the length of each partial upload needs to be a multiple of the
|
207
|
-
`:chunk_size` number.
|
208
|
-
|
209
191
|
### Amazon S3
|
210
192
|
|
211
|
-
|
212
|
-
|
213
|
-
|
193
|
+
You can switch to `Tus::Storage::S3` to uploads files to AWS S3 using the
|
194
|
+
multipart API. For this you'll also need to add the [aws-sdk-s3] gem to your
|
195
|
+
Gemfile.
|
214
196
|
|
215
197
|
```rb
|
198
|
+
# Gemfile
|
216
199
|
gem "aws-sdk-s3", "~> 1.2"
|
217
200
|
```
|
218
201
|
|
219
202
|
```rb
|
220
203
|
require "tus/storage/s3"
|
221
204
|
|
205
|
+
# You can omit AWS credentials if you're authenticating in other ways
|
222
206
|
Tus::Server.opts[:storage] = Tus::Storage::S3.new(
|
207
|
+
bucket: "my-app", # required
|
223
208
|
access_key_id: "abc",
|
224
209
|
secret_access_key: "xyz",
|
225
210
|
region: "eu-west-1",
|
226
|
-
bucket: "my-app",
|
227
211
|
)
|
228
212
|
```
|
229
213
|
|
@@ -231,6 +215,27 @@ One thing to note is that S3's multipart API requires each chunk except the
|
|
231
215
|
last to be **5MB or larger**, so that is the minimum chunk size that you can
|
232
216
|
specify on your tus client if you want to use the S3 storage.
|
233
217
|
|
218
|
+
If you'll be retrieving uploaded files through the tus server app, it's
|
219
|
+
recommended to set `Tus::Server.opts[:download_url]` to `true`. This will avoid
|
220
|
+
tus server downloading and serving the file from S3, and instead have the
|
221
|
+
download endpoint redirect to the direct S3 object URL.
|
222
|
+
|
223
|
+
```rb
|
224
|
+
Tus::Server.opts[:download_url] = true
|
225
|
+
```
|
226
|
+
|
227
|
+
You can customize how the S3 object URL is being generated by passing a block
|
228
|
+
to `:download_url`, which will then be evaluated in the context of the
|
229
|
+
`Tus::Server` instance (which allows accessing the `request` object). See
|
230
|
+
[`Aws::S3::Object#get`] for the list of options that
|
231
|
+
`Tus::Storage::S3#file_url` accepts.
|
232
|
+
|
233
|
+
```rb
|
234
|
+
Tus::Server.opts[:download_url] = -> (uid, info, **options) do
|
235
|
+
storage.file_url(uid, info, response_expires: 10, **options) # link expires after 10 seconds
|
236
|
+
end
|
237
|
+
```
|
238
|
+
|
234
239
|
If you want to files to be stored in a certain subdirectory, you can specify
|
235
240
|
a `:prefix` in the storage configuration.
|
236
241
|
|
@@ -252,6 +257,42 @@ can for example change the `:endpoint` to use S3's accelerate host:
|
|
252
257
|
Tus::Storage::S3.new(endpoint: "https://s3-accelerate.amazonaws.com", **options)
|
253
258
|
```
|
254
259
|
|
260
|
+
### MongoDB GridFS
|
261
|
+
|
262
|
+
MongoDB has a specification for storing and retrieving large files, called
|
263
|
+
"[GridFS]". Tus-ruby-server ships with `Tus::Storage::Gridfs` that you can
|
264
|
+
use, which uses the [Mongo] gem.
|
265
|
+
|
266
|
+
```rb
|
267
|
+
# Gemfile
|
268
|
+
gem "mongo", "~> 2.3"
|
269
|
+
```
|
270
|
+
|
271
|
+
```rb
|
272
|
+
require "tus/storage/gridfs"
|
273
|
+
|
274
|
+
client = Mongo::Client.new("mongodb://127.0.0.1:27017/mydb")
|
275
|
+
Tus::Server.opts[:storage] = Tus::Storage::Gridfs.new(client: client)
|
276
|
+
```
|
277
|
+
|
278
|
+
You can change the database prefix (defaults to `fs`):
|
279
|
+
|
280
|
+
```rb
|
281
|
+
Tus::Storage::Gridfs.new(client: client, prefix: "fs_temp")
|
282
|
+
```
|
283
|
+
|
284
|
+
By default MongoDB Gridfs stores files in chunks of 256KB, but you can change
|
285
|
+
that with the `:chunk_size` option:
|
286
|
+
|
287
|
+
```rb
|
288
|
+
Tus::Storage::Gridfs.new(client: client, chunk_size: 1*1024*1024) # 1 MB
|
289
|
+
```
|
290
|
+
|
291
|
+
Note that if you're using the [concatenation] tus feature with Gridfs, all
|
292
|
+
partial uploads except the last one are required to fill in their Gridfs
|
293
|
+
chunks, meaning the length of each partial upload needs to be a multiple of the
|
294
|
+
`:chunk_size` number.
|
295
|
+
|
255
296
|
### Other storages
|
256
297
|
|
257
298
|
If none of these storages suit you, you can write your own, you just need to
|
@@ -264,6 +305,7 @@ def patch_file(uid, io, info = {}) ... end
|
|
264
305
|
def update_info(uid, info) ... end
|
265
306
|
def read_info(uid) ... end
|
266
307
|
def get_file(uid, info = {}, range: nil) ... end
|
308
|
+
def file_url(uid, info = {}, **options) ... end # optional
|
267
309
|
def delete_file(uid, info = {}) ... end
|
268
310
|
def expire_files(expiration_date) ... end
|
269
311
|
```
|
@@ -302,14 +344,20 @@ tus_storage.expire_files(expiration_time)
|
|
302
344
|
## Download
|
303
345
|
|
304
346
|
In addition to implementing the tus protocol, tus-ruby-server also comes with a
|
305
|
-
GET endpoint for downloading the uploaded file, which
|
306
|
-
storage
|
347
|
+
GET endpoint for downloading the uploaded file, which by default streams the
|
348
|
+
file from the storage. It supports [Range requests], so you can use the tus
|
349
|
+
file URL as `src` in `<video>` and `<audio>` HTML tags.
|
350
|
+
|
351
|
+
It's highly recommended not to serve files through the app, but offload it to
|
352
|
+
your frontend server if using disk storage, or if using S3 storage have the
|
353
|
+
download endpoint redirect to the S3 object URL. See the documentation for the
|
354
|
+
individual storage for instructions how to set this up.
|
307
355
|
|
308
356
|
The endpoint will automatically use the following `Upload-Metadata` values if
|
309
357
|
they're available:
|
310
358
|
|
311
|
-
* `type` -- used
|
312
|
-
* `name` -- used
|
359
|
+
* `type` -- used to set `Content-Type` response header
|
360
|
+
* `name` -- used to set `Content-Disposition` response header
|
313
361
|
|
314
362
|
The `Content-Disposition` header will be set to "inline" by default, but you
|
315
363
|
can change it to "attachment" if you want the browser to always force download:
|
@@ -318,9 +366,6 @@ can change it to "attachment" if you want the browser to always force download:
|
|
318
366
|
Tus::Server.opts[:disposition] = "attachment"
|
319
367
|
```
|
320
368
|
|
321
|
-
The download endpoint supports [Range requests], so you can use the tus
|
322
|
-
file URL as `src` in `<video>` and `<audio>` HTML tags.
|
323
|
-
|
324
369
|
## Checksum
|
325
370
|
|
326
371
|
The following checksum algorithms are supported for the `checksum` extension:
|
@@ -358,8 +403,8 @@ The tus-ruby-server was inspired by [rubytus].
|
|
358
403
|
[termination]: http://tus.io/protocols/resumable-upload.html#termination
|
359
404
|
[GridFS]: https://docs.mongodb.org/v3.0/core/gridfs/
|
360
405
|
[Mongo]: https://github.com/mongodb/mongo-ruby-driver
|
361
|
-
[shrine-tus-demo]: https://github.com/
|
362
|
-
[Shrine]: https://github.com/
|
406
|
+
[shrine-tus-demo]: https://github.com/shrinerb/shrine-tus-demo
|
407
|
+
[Shrine]: https://github.com/shrinerb/shrine
|
363
408
|
[trailing headers]: https://tools.ietf.org/html/rfc7230#section-4.1.2
|
364
409
|
[rubytus]: https://github.com/picocandy/rubytus
|
365
410
|
[aws-sdk-s3]: https://github.com/aws/aws-sdk-ruby/tree/master/gems/aws-sdk-s3
|
@@ -369,3 +414,5 @@ The tus-ruby-server was inspired by [rubytus].
|
|
369
414
|
[Goliath]: https://github.com/postrank-labs/goliath
|
370
415
|
[EventMachine]: https://github.com/eventmachine/eventmachine
|
371
416
|
[goliath-rack_proxy]: https://github.com/janko-m/goliath-rack_proxy
|
417
|
+
[Rack::Sendfile]: https://www.rubydoc.info/github/rack/rack/master/Rack/Sendfile
|
418
|
+
[`Aws::S3::Object#get`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#get-instance_method
|
data/lib/tus/server.rb
CHANGED
@@ -26,6 +26,7 @@ module Tus
|
|
26
26
|
opts[:max_size] = nil
|
27
27
|
opts[:expiration_time] = 7*24*60*60
|
28
28
|
opts[:disposition] = "inline"
|
29
|
+
opts[:download_url] = nil
|
29
30
|
|
30
31
|
plugin :all_verbs
|
31
32
|
plugin :default_headers, {"Content-Type" => ""}
|
@@ -112,6 +113,7 @@ module Tus
|
|
112
113
|
error!(404, "Upload Not Found")
|
113
114
|
end
|
114
115
|
|
116
|
+
# HEAD /{uid}
|
115
117
|
r.head do
|
116
118
|
response.headers.update(info.headers)
|
117
119
|
response.headers["Cache-Control"] = "no-store"
|
@@ -157,22 +159,33 @@ module Tus
|
|
157
159
|
# GET /{uid}
|
158
160
|
r.get do
|
159
161
|
validate_upload_finished!(info)
|
160
|
-
range = handle_range_request!(info.length)
|
161
|
-
|
162
|
-
response.headers["Content-Length"] = (range.end - range.begin + 1).to_s
|
163
162
|
|
164
163
|
metadata = info.metadata
|
165
164
|
name = metadata["name"] || metadata["filename"]
|
166
165
|
type = metadata["type"] || metadata["content_type"]
|
167
166
|
|
168
|
-
|
169
|
-
|
170
|
-
|
171
|
-
|
167
|
+
content_disposition = opts[:disposition]
|
168
|
+
content_disposition += "; filename=\"#{name}\"" if name
|
169
|
+
content_type = type || "application/octet-stream"
|
170
|
+
|
171
|
+
if download_url
|
172
|
+
redirect_url = instance_exec(uid, info.to_h,
|
173
|
+
content_type: content_type,
|
174
|
+
content_disposition: content_disposition,
|
175
|
+
&download_url)
|
176
|
+
|
177
|
+
r.redirect redirect_url
|
178
|
+
else
|
179
|
+
range = handle_range_request!(info.length)
|
180
|
+
|
181
|
+
response.headers["Content-Length"] = range.size.to_s
|
182
|
+
response.headers["Content-Disposition"] = content_disposition
|
183
|
+
response.headers["Content-Type"] = content_type
|
172
184
|
|
173
|
-
|
185
|
+
body = storage.get_file(uid, info.to_h, range: range)
|
174
186
|
|
175
|
-
|
187
|
+
r.halt response.finish_with_body(body)
|
188
|
+
end
|
176
189
|
end
|
177
190
|
|
178
191
|
# DELETE /{uid}
|
@@ -248,7 +261,7 @@ module Tus
|
|
248
261
|
upload_metadata = request.headers["Upload-Metadata"]
|
249
262
|
|
250
263
|
upload_metadata.split(",").each do |string|
|
251
|
-
key, value = string.split(" ")
|
264
|
+
key, value = string.split(" ", 2)
|
252
265
|
|
253
266
|
error!(400, "Invalid Upload-Metadata header") if key.nil?
|
254
267
|
error!(400, "Invalid Upload-Metadata header") if key.ord > 127
|
@@ -309,15 +322,15 @@ module Tus
|
|
309
322
|
# Handles partial responses requested in the "Range" header. Implementation
|
310
323
|
# is mostly copied from Rack::File.
|
311
324
|
def handle_range_request!(length)
|
312
|
-
# we support ranged requests
|
313
|
-
response.headers["Accept-Ranges"] = "bytes"
|
314
|
-
|
315
325
|
if Rack.release >= "2.0"
|
316
326
|
ranges = Rack::Utils.get_byte_ranges(request.headers["Range"], length)
|
317
327
|
else
|
318
328
|
ranges = Rack::Utils.byte_ranges(request.env, length)
|
319
329
|
end
|
320
330
|
|
331
|
+
# we support ranged requests
|
332
|
+
response.headers["Accept-Ranges"] = "bytes"
|
333
|
+
|
321
334
|
if ranges.nil? || ranges.length > 1
|
322
335
|
# no ranges, or multiple ranges (which we don't support)
|
323
336
|
response.status = 200
|
@@ -369,6 +382,14 @@ module Tus
|
|
369
382
|
request.halt
|
370
383
|
end
|
371
384
|
|
385
|
+
def download_url
|
386
|
+
if opts[:download_url] == true
|
387
|
+
storage.method(:file_url)
|
388
|
+
else
|
389
|
+
opts[:download_url]
|
390
|
+
end
|
391
|
+
end
|
392
|
+
|
372
393
|
def storage
|
373
394
|
opts[:storage] || Tus::Storage::Filesystem.new("data")
|
374
395
|
end
|
@@ -27,7 +27,7 @@ module Tus
|
|
27
27
|
file_path(uid).binwrite("")
|
28
28
|
file_path(uid).chmod(@permissions)
|
29
29
|
|
30
|
-
info_path(uid).binwrite("{}")
|
30
|
+
info_path(uid).binwrite("{}") unless info_path(uid).exist?
|
31
31
|
info_path(uid).chmod(@permissions)
|
32
32
|
end
|
33
33
|
|
@@ -99,7 +99,7 @@ module Tus
|
|
99
99
|
end
|
100
100
|
end
|
101
101
|
|
102
|
-
|
102
|
+
Response.new(chunks: chunks, close: file.method(:close), path: file_path(uid).to_s)
|
103
103
|
end
|
104
104
|
|
105
105
|
# Deletes data and info files for the specified upload.
|
@@ -136,6 +136,18 @@ module Tus
|
|
136
136
|
directory.mkpath
|
137
137
|
directory.chmod(@directory_permissions)
|
138
138
|
end
|
139
|
+
|
140
|
+
class Response < Tus::Response
|
141
|
+
def initialize(path:, **options)
|
142
|
+
super(**options)
|
143
|
+
@path = path
|
144
|
+
end
|
145
|
+
|
146
|
+
# Rack::Sendfile middleware needs response body to respond to #to_path
|
147
|
+
def to_path
|
148
|
+
@path
|
149
|
+
end
|
150
|
+
end
|
139
151
|
end
|
140
152
|
end
|
141
153
|
end
|
data/lib/tus/storage/gridfs.rb
CHANGED
data/lib/tus/storage/s3.rb
CHANGED
@@ -1,9 +1,8 @@
|
|
1
1
|
# frozen-string-literal: true
|
2
2
|
|
3
|
+
gem "aws-sdk-s3", "~> 1.2"
|
4
|
+
|
3
5
|
require "aws-sdk-s3"
|
4
|
-
if Gem::Version.new(Aws::S3::GEM_VERSION) < Gem::Version.new("1.2.0")
|
5
|
-
raise "Tus::Storage::S3 requires aws-sdk-s3 version 1.2.0 or above"
|
6
|
-
end
|
7
6
|
|
8
7
|
require "tus/info"
|
9
8
|
require "tus/response"
|
@@ -11,8 +10,6 @@ require "tus/errors"
|
|
11
10
|
|
12
11
|
require "json"
|
13
12
|
require "cgi"
|
14
|
-
require "fiber"
|
15
|
-
require "stringio"
|
16
13
|
|
17
14
|
module Tus
|
18
15
|
module Storage
|
@@ -106,39 +103,34 @@ module Tus
|
|
106
103
|
bytes_uploaded = 0
|
107
104
|
|
108
105
|
jobs = []
|
109
|
-
chunk =
|
106
|
+
chunk = input.read(MIN_PART_SIZE)
|
110
107
|
|
111
|
-
|
112
|
-
next_chunk =
|
108
|
+
while chunk
|
109
|
+
next_chunk = input.read(MIN_PART_SIZE)
|
113
110
|
|
114
111
|
# merge next chunk into previous if it's smaller than minimum chunk size
|
115
|
-
if next_chunk.
|
116
|
-
chunk
|
117
|
-
next_chunk.
|
112
|
+
if next_chunk && next_chunk.bytesize < MIN_PART_SIZE
|
113
|
+
chunk << next_chunk
|
114
|
+
next_chunk.clear
|
118
115
|
next_chunk = nil
|
119
116
|
end
|
120
117
|
|
121
118
|
# abort if chunk is smaller than 5MB and is not the last chunk
|
122
|
-
if chunk.
|
119
|
+
if chunk.bytesize < MIN_PART_SIZE
|
123
120
|
break if (tus_info.length && tus_info.offset) &&
|
124
|
-
chunk.
|
121
|
+
chunk.bytesize + tus_info.offset < tus_info.length
|
125
122
|
end
|
126
123
|
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
|
132
|
-
|
133
|
-
begin
|
134
|
-
jobs.each do |thread, body|
|
135
|
-
info["multipart_parts"] << thread.value
|
136
|
-
bytes_uploaded += body.size
|
137
|
-
body.close
|
124
|
+
begin
|
125
|
+
info["multipart_parts"] << upload_part(chunk, uid, upload_id, part_offset += 1)
|
126
|
+
bytes_uploaded += chunk.bytesize
|
127
|
+
rescue Seahorse::Client::NetworkingError => exception
|
128
|
+
warn "ERROR: #{exception.inspect} occurred during upload"
|
129
|
+
break # ignore networking errors and return what client has uploaded so far
|
138
130
|
end
|
139
|
-
|
140
|
-
|
141
|
-
|
131
|
+
|
132
|
+
chunk.clear
|
133
|
+
chunk = next_chunk
|
142
134
|
end
|
143
135
|
|
144
136
|
bytes_uploaded
|
@@ -177,14 +169,20 @@ module Tus
|
|
177
169
|
# upload can be retrieved in a streaming fashion. Accepts an optional
|
178
170
|
# range parameter for selecting a subset of bytes to retrieve.
|
179
171
|
def get_file(uid, info = {}, range: nil)
|
180
|
-
tus_info = Tus::Info.new(info)
|
181
|
-
|
182
172
|
range = "bytes=#{range.begin}-#{range.end}" if range
|
183
173
|
chunks = object(uid).enum_for(:get, range: range)
|
184
174
|
|
185
175
|
Tus::Response.new(chunks: chunks)
|
186
176
|
end
|
187
177
|
|
178
|
+
# Returns a signed expiring URL to the S3 object.
|
179
|
+
def file_url(uid, info = {}, content_type: nil, content_disposition: nil, **options)
|
180
|
+
options[:response_content_type] ||= content_type
|
181
|
+
options[:response_content_disposition] ||= content_disposition
|
182
|
+
|
183
|
+
object(uid).presigned_url(:get, **options)
|
184
|
+
end
|
185
|
+
|
188
186
|
# Deletes resources for the specified upload. If multipart upload is
|
189
187
|
# still in progress, aborts the multipart upload, otherwise deletes the
|
190
188
|
# object.
|
@@ -222,12 +220,6 @@ module Tus
|
|
222
220
|
|
223
221
|
private
|
224
222
|
|
225
|
-
# Spawns a thread which uploads given body as a new multipart part with
|
226
|
-
# the specified part number to the specified multipart upload.
|
227
|
-
def upload_part_thread(body, key, upload_id, part_number)
|
228
|
-
Thread.new { upload_part(body, key, upload_id, part_number) }
|
229
|
-
end
|
230
|
-
|
231
223
|
# Uploads given body as a new multipart part with the specified part
|
232
224
|
# number to the specified multipart upload. Returns part number and ETag
|
233
225
|
# that will be required later for completing the multipart upload.
|
data/tus-server.gemspec
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: tus-server
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0
|
4
|
+
version: 2.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Janko Marohnić
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2018-05-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: roda
|
@@ -156,7 +156,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
156
156
|
version: '0'
|
157
157
|
requirements: []
|
158
158
|
rubyforge_project:
|
159
|
-
rubygems_version: 2.6
|
159
|
+
rubygems_version: 2.7.6
|
160
160
|
signing_key:
|
161
161
|
specification_version: 4
|
162
162
|
summary: Ruby server implementation of tus.io, the open protocol for resumable file
|