tus-server 2.0.2 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/CHANGELOG.md +10 -0
- data/README.md +154 -107
- data/lib/tus/server.rb +34 -13
- data/lib/tus/storage/filesystem.rb +14 -2
- data/lib/tus/storage/gridfs.rb +2 -0
- data/lib/tus/storage/s3.rb +27 -35
- data/tus-server.gemspec +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 16cf0c007be7d383ca0bf2d4aefa6d6c477ac7d32b1905a8ad6c0413da233e57
|
4
|
+
data.tar.gz: 258d7d591fc1416468ea21ea42d29ed53f2a64eeade9e5293bdbecc3e5f6a599
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d642a6f120fd220af27bfc7cf8904459acafa6191934a4922913b10b296e0308cd3093223b39317ffb901318e83da2bcd3fa056cf7d2a36a2f7131ce18b8d8ca
|
7
|
+
data.tar.gz: c0bf56b77ee95fa1aab167f29fd38feabb1919f7e7d85b1c7705c5adbcba97f9612b5633880af9b603b433a3778518b7875bd5bfafff4f446c476d621c0b9c99
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,13 @@
|
|
1
|
+
## 2.1.0 (2018-05-15)
|
2
|
+
|
3
|
+
* Add `:download_url` server option for redirecting to a download URL (@janko-m)
|
4
|
+
|
5
|
+
* Allow application servers to serve files stored on disk via the `Rack::Sendfile` middleware (@janko-m)
|
6
|
+
|
7
|
+
* Reject `Upload-Metadata` which contains key-value pairs separated by spaces (@janko-m)
|
8
|
+
|
9
|
+
* Don't overwite info file if it already exists in `Tus::Storage::FileSystem` (@janko-m)
|
10
|
+
|
1
11
|
## 2.0.2 (2017-12-24)
|
2
12
|
|
3
13
|
* Handle `name` and `type` metadata for Uppy compatibility (@janko-m)
|
data/README.md
CHANGED
@@ -12,62 +12,81 @@ A Ruby server for the [tus resumable upload protocol]. It implements the core
|
|
12
12
|
## Installation
|
13
13
|
|
14
14
|
```rb
|
15
|
-
|
15
|
+
# Gemfile
|
16
|
+
gem "tus-server", "~> 2.0"
|
16
17
|
```
|
17
18
|
|
18
19
|
## Usage
|
19
20
|
|
20
|
-
|
21
|
-
|
22
|
-
part of your main app.
|
21
|
+
The gem provides a `Tus::Server` Rack app, which you can mount inside your
|
22
|
+
main application. If you're using Rails, you can mount it in `config/routes.rb`:
|
23
23
|
|
24
24
|
```rb
|
25
|
-
# config.
|
25
|
+
# config/routes.rb (Rails)
|
26
|
+
Rails.application.routes.draw do
|
27
|
+
# ...
|
28
|
+
mount Tus::Server => "/files"
|
29
|
+
end
|
30
|
+
```
|
31
|
+
|
32
|
+
Otherwise you can run it in `config.ru`:
|
33
|
+
|
34
|
+
```rb
|
35
|
+
# config.ru (Rack)
|
26
36
|
require "tus/server"
|
27
37
|
|
28
38
|
map "/files" do
|
29
39
|
run Tus::Server
|
30
40
|
end
|
31
|
-
|
32
|
-
run YourApp
|
33
41
|
```
|
34
42
|
|
35
|
-
While this is the most flexible option, it's not optimal in terms of
|
36
|
-
performance; see the [Goliath](#goliath) section for an alternative approach.
|
37
|
-
|
38
43
|
Now you can tell your tus client library (e.g. [tus-js-client]) to use this
|
39
44
|
endpoint:
|
40
45
|
|
41
46
|
```js
|
42
47
|
// using tus-js-client
|
43
48
|
new tus.Upload(file, {
|
44
|
-
endpoint: "
|
45
|
-
chunkSize: 5*1024*1024, // required unless using Goliath
|
49
|
+
endpoint: "/files",
|
50
|
+
chunkSize: 5*1024*1024, // required unless using Goliath
|
46
51
|
// ...
|
47
52
|
})
|
48
53
|
```
|
49
54
|
|
50
|
-
|
51
|
-
|
52
|
-
|
55
|
+
By default uploaded files will be stored in the `data/` directory. After the
|
56
|
+
upload is complete, you'll probably want to attach the uploaded file to a
|
57
|
+
database record. [Shrine] is one file attachment library that integrates nicely
|
58
|
+
with tus-ruby-server, see [shrine-tus-demo] for an example integration.
|
53
59
|
|
54
60
|
### Goliath
|
55
61
|
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
|
62
|
+
Running the tus server alongside your main app using classic web servers like
|
63
|
+
Puma or Unicorn is probably fine for most cases, however, it does come with a
|
64
|
+
few gotchas. First, since these web servers don't accept partial requests
|
65
|
+
(request where the request body hasn't been fully received), the tus client
|
66
|
+
must be configured to split the upload into multiple requests. Second, since
|
67
|
+
web workers are tied for the duration of the request, serving uploaded files
|
68
|
+
through the tus server app could significantly impact request throughput, so
|
69
|
+
you need to be careful to avoid that.
|
70
|
+
|
71
|
+
There is an alternative. [Goliath] is an asychronous web server built on top of
|
72
|
+
[EventMachine], which supports streaming requests and streaming responses.
|
73
|
+
|
74
|
+
* Asynchronous streaming requests allows the tus server to begin saving
|
75
|
+
uploaded data while it's still being received. If the request is interrupted,
|
76
|
+
the tus server will attempt to save as much of the data that was received so
|
77
|
+
far. This means it's not necessary for the tus client to split the upload
|
78
|
+
into multiple smaller requests.
|
64
79
|
|
65
|
-
|
66
|
-
|
80
|
+
* Asynchronous streaming responses allows the tus server to stream large files
|
81
|
+
with very small impact to the request throughput.
|
82
|
+
|
83
|
+
Since Goliath is web server, to run tus server on it we'll have to run it as a
|
84
|
+
standalone web app. It's recommended that you use [goliath-rack_proxy] for
|
85
|
+
running your tus server app:
|
67
86
|
|
68
87
|
```rb
|
69
88
|
# Gemfile
|
70
|
-
gem "tus-server", "~>
|
89
|
+
gem "tus-server", "~> 2.0"
|
71
90
|
gem "goliath-rack_proxy", "~> 1.0"
|
72
91
|
```
|
73
92
|
```rb
|
@@ -92,66 +111,34 @@ path you can use `Rack::Builder`:
|
|
92
111
|
```rb
|
93
112
|
class GoliathTusServer < Goliath::RackProxy
|
94
113
|
rack_app Rack::Builder.new {
|
95
|
-
map
|
96
|
-
run Tus::Server
|
97
|
-
end
|
114
|
+
map("/files") { run Tus::Server }
|
98
115
|
}
|
99
116
|
rewindable_input false # set to true if you're using checksums
|
100
117
|
end
|
101
118
|
```
|
102
119
|
|
103
|
-
|
104
|
-
|
105
|
-
Like Goliath, Unicorn also support streaming uploads, and tus-ruby-server knows
|
106
|
-
how to automatically recover from `Unicorn::ClientShutdown` exceptions during
|
107
|
-
upload, storing data that it has received up until that point. Just note that
|
108
|
-
in order to achieve streaming uploads, Nginx should be configured **not** to
|
109
|
-
buffer incoming requests, and to disable worker timeout to enable long running
|
110
|
-
upload requests:
|
120
|
+
In this case you'll have to configure the tus client to point to the standalone
|
121
|
+
Goliath app:
|
111
122
|
|
112
|
-
```
|
113
|
-
|
114
|
-
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
for resumable upload to be possible the client needs to send data in multiple
|
119
|
-
upload requests (which can then be retried individually).
|
120
|
-
|
121
|
-
Unless you're using the "checksum" tus feature, you might want to consider
|
122
|
-
disabling rewindability of request body, to prevent Unicorn from additionally
|
123
|
-
caching received data onto the disk (since that's not necessary unless request
|
124
|
-
body needs to be rewinded).
|
125
|
-
|
126
|
-
```rb
|
127
|
-
# config/unicorn.rb
|
128
|
-
# ...
|
129
|
-
|
130
|
-
rewindable_input false
|
123
|
+
```js
|
124
|
+
// using tus-js-client
|
125
|
+
new tus.Upload(file, {
|
126
|
+
endpoint: "http://localhost:9000/files",
|
127
|
+
// ...
|
128
|
+
})
|
131
129
|
```
|
132
130
|
|
133
|
-
### Other web servers
|
134
|
-
|
135
|
-
It's perfectly feasible to run tus-ruby-server on web servers other than
|
136
|
-
Goliath or Unicorn (even necessary if you want to run it inside another app).
|
137
|
-
Just keep in mind that most other web servers don't support request streaming,
|
138
|
-
which means that tus-ruby-server will be able to start processing upload
|
139
|
-
requess only once the whole request body has been received. Additionally,
|
140
|
-
incomplete upload requests won't be forwarded to tus-ruby-server, so in order
|
141
|
-
for resumable upload to be possible the client needs to send data in multiple
|
142
|
-
upload requests (which can then be retried individually).
|
143
|
-
|
144
131
|
## Storage
|
145
132
|
|
146
133
|
### Filesystem
|
147
134
|
|
148
|
-
By default `Tus::Server` stores uploaded files
|
149
|
-
|
135
|
+
By default `Tus::Server` stores uploaded files in the `data/` directory. You
|
136
|
+
can configure a different directory:
|
150
137
|
|
151
138
|
```rb
|
152
139
|
require "tus/storage/filesystem"
|
153
140
|
|
154
|
-
Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/
|
141
|
+
Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/tus")
|
155
142
|
```
|
156
143
|
|
157
144
|
If the configured directory doesn't exist, it will automatically be created.
|
@@ -171,59 +158,56 @@ store files on the filesystem as they won't persist.
|
|
171
158
|
All these are reasons why you might store uploaded data on a different storage,
|
172
159
|
and luckily tus-ruby-server ships with two more storages.
|
173
160
|
|
174
|
-
|
175
|
-
|
176
|
-
MongoDB has a specification for storing and retrieving large files, called
|
177
|
-
"[GridFS]". Tus-ruby-server ships with `Tus::Storage::Gridfs` that you can
|
178
|
-
use, which uses the [Mongo] gem.
|
179
|
-
|
180
|
-
```rb
|
181
|
-
gem "mongo", ">= 2.2.2", "< 3"
|
182
|
-
```
|
161
|
+
#### Serving files
|
183
162
|
|
184
|
-
|
185
|
-
|
163
|
+
If your retrieving uploaded files through the download endpoint, by default the
|
164
|
+
files will be served through the Ruby application. However, that's very
|
165
|
+
inefficient, as web workers are tied when serving download requests and cannot
|
166
|
+
serve additional requests for that duration.
|
186
167
|
|
187
|
-
|
188
|
-
|
189
|
-
|
168
|
+
Therefore, it's highly recommended to delegate serving uploaded files to your
|
169
|
+
frontend server (Nginx, Apache). This can be achieved with the
|
170
|
+
`Rack::Sendfile` middleware, see its [documentation][Rack::Sendfile] to learn
|
171
|
+
more about how to use it with popular frontend servers.
|
190
172
|
|
191
|
-
|
173
|
+
If you're using Rails, you can enable the `Rack::Sendfile` middleware by
|
174
|
+
setting the `config.action_dispatch.x_sendfile_header` value accordingly:
|
192
175
|
|
193
176
|
```rb
|
194
|
-
|
177
|
+
config.action_dispatch.x_sendfile_header = "X-Sendfile" # Apache and lighttpd
|
178
|
+
# or
|
179
|
+
config.action_dispatch.x_sendfile_header = "X-Accel-Redirect" # Nginx
|
195
180
|
```
|
196
181
|
|
197
|
-
|
198
|
-
|
182
|
+
Otherwise you can add the `Rack::Sendfile` middleware to the stack in
|
183
|
+
`config.ru`:
|
199
184
|
|
200
185
|
```rb
|
201
|
-
|
186
|
+
use Rack::Sendfile, "X-Sendfile" # Apache and lighttpd
|
187
|
+
# or
|
188
|
+
use Rack::Sendfile, "X-Accel-Redirect" # Nginx
|
202
189
|
```
|
203
190
|
|
204
|
-
Note that if you're using the [concatenation] tus feature with Gridfs, all
|
205
|
-
partial uploads except the last one are required to fill in their Gridfs
|
206
|
-
chunks, meaning the length of each partial upload needs to be a multiple of the
|
207
|
-
`:chunk_size` number.
|
208
|
-
|
209
191
|
### Amazon S3
|
210
192
|
|
211
|
-
|
212
|
-
|
213
|
-
|
193
|
+
You can switch to `Tus::Storage::S3` to uploads files to AWS S3 using the
|
194
|
+
multipart API. For this you'll also need to add the [aws-sdk-s3] gem to your
|
195
|
+
Gemfile.
|
214
196
|
|
215
197
|
```rb
|
198
|
+
# Gemfile
|
216
199
|
gem "aws-sdk-s3", "~> 1.2"
|
217
200
|
```
|
218
201
|
|
219
202
|
```rb
|
220
203
|
require "tus/storage/s3"
|
221
204
|
|
205
|
+
# You can omit AWS credentials if you're authenticating in other ways
|
222
206
|
Tus::Server.opts[:storage] = Tus::Storage::S3.new(
|
207
|
+
bucket: "my-app", # required
|
223
208
|
access_key_id: "abc",
|
224
209
|
secret_access_key: "xyz",
|
225
210
|
region: "eu-west-1",
|
226
|
-
bucket: "my-app",
|
227
211
|
)
|
228
212
|
```
|
229
213
|
|
@@ -231,6 +215,27 @@ One thing to note is that S3's multipart API requires each chunk except the
|
|
231
215
|
last to be **5MB or larger**, so that is the minimum chunk size that you can
|
232
216
|
specify on your tus client if you want to use the S3 storage.
|
233
217
|
|
218
|
+
If you'll be retrieving uploaded files through the tus server app, it's
|
219
|
+
recommended to set `Tus::Server.opts[:download_url]` to `true`. This will avoid
|
220
|
+
tus server downloading and serving the file from S3, and instead have the
|
221
|
+
download endpoint redirect to the direct S3 object URL.
|
222
|
+
|
223
|
+
```rb
|
224
|
+
Tus::Server.opts[:download_url] = true
|
225
|
+
```
|
226
|
+
|
227
|
+
You can customize how the S3 object URL is being generated by passing a block
|
228
|
+
to `:download_url`, which will then be evaluated in the context of the
|
229
|
+
`Tus::Server` instance (which allows accessing the `request` object). See
|
230
|
+
[`Aws::S3::Object#get`] for the list of options that
|
231
|
+
`Tus::Storage::S3#file_url` accepts.
|
232
|
+
|
233
|
+
```rb
|
234
|
+
Tus::Server.opts[:download_url] = -> (uid, info, **options) do
|
235
|
+
storage.file_url(uid, info, response_expires: 10, **options) # link expires after 10 seconds
|
236
|
+
end
|
237
|
+
```
|
238
|
+
|
234
239
|
If you want to files to be stored in a certain subdirectory, you can specify
|
235
240
|
a `:prefix` in the storage configuration.
|
236
241
|
|
@@ -252,6 +257,42 @@ can for example change the `:endpoint` to use S3's accelerate host:
|
|
252
257
|
Tus::Storage::S3.new(endpoint: "https://s3-accelerate.amazonaws.com", **options)
|
253
258
|
```
|
254
259
|
|
260
|
+
### MongoDB GridFS
|
261
|
+
|
262
|
+
MongoDB has a specification for storing and retrieving large files, called
|
263
|
+
"[GridFS]". Tus-ruby-server ships with `Tus::Storage::Gridfs` that you can
|
264
|
+
use, which uses the [Mongo] gem.
|
265
|
+
|
266
|
+
```rb
|
267
|
+
# Gemfile
|
268
|
+
gem "mongo", "~> 2.3"
|
269
|
+
```
|
270
|
+
|
271
|
+
```rb
|
272
|
+
require "tus/storage/gridfs"
|
273
|
+
|
274
|
+
client = Mongo::Client.new("mongodb://127.0.0.1:27017/mydb")
|
275
|
+
Tus::Server.opts[:storage] = Tus::Storage::Gridfs.new(client: client)
|
276
|
+
```
|
277
|
+
|
278
|
+
You can change the database prefix (defaults to `fs`):
|
279
|
+
|
280
|
+
```rb
|
281
|
+
Tus::Storage::Gridfs.new(client: client, prefix: "fs_temp")
|
282
|
+
```
|
283
|
+
|
284
|
+
By default MongoDB Gridfs stores files in chunks of 256KB, but you can change
|
285
|
+
that with the `:chunk_size` option:
|
286
|
+
|
287
|
+
```rb
|
288
|
+
Tus::Storage::Gridfs.new(client: client, chunk_size: 1*1024*1024) # 1 MB
|
289
|
+
```
|
290
|
+
|
291
|
+
Note that if you're using the [concatenation] tus feature with Gridfs, all
|
292
|
+
partial uploads except the last one are required to fill in their Gridfs
|
293
|
+
chunks, meaning the length of each partial upload needs to be a multiple of the
|
294
|
+
`:chunk_size` number.
|
295
|
+
|
255
296
|
### Other storages
|
256
297
|
|
257
298
|
If none of these storages suit you, you can write your own, you just need to
|
@@ -264,6 +305,7 @@ def patch_file(uid, io, info = {}) ... end
|
|
264
305
|
def update_info(uid, info) ... end
|
265
306
|
def read_info(uid) ... end
|
266
307
|
def get_file(uid, info = {}, range: nil) ... end
|
308
|
+
def file_url(uid, info = {}, **options) ... end # optional
|
267
309
|
def delete_file(uid, info = {}) ... end
|
268
310
|
def expire_files(expiration_date) ... end
|
269
311
|
```
|
@@ -302,14 +344,20 @@ tus_storage.expire_files(expiration_time)
|
|
302
344
|
## Download
|
303
345
|
|
304
346
|
In addition to implementing the tus protocol, tus-ruby-server also comes with a
|
305
|
-
GET endpoint for downloading the uploaded file, which
|
306
|
-
storage
|
347
|
+
GET endpoint for downloading the uploaded file, which by default streams the
|
348
|
+
file from the storage. It supports [Range requests], so you can use the tus
|
349
|
+
file URL as `src` in `<video>` and `<audio>` HTML tags.
|
350
|
+
|
351
|
+
It's highly recommended not to serve files through the app, but offload it to
|
352
|
+
your frontend server if using disk storage, or if using S3 storage have the
|
353
|
+
download endpoint redirect to the S3 object URL. See the documentation for the
|
354
|
+
individual storage for instructions how to set this up.
|
307
355
|
|
308
356
|
The endpoint will automatically use the following `Upload-Metadata` values if
|
309
357
|
they're available:
|
310
358
|
|
311
|
-
* `type` -- used
|
312
|
-
* `name` -- used
|
359
|
+
* `type` -- used to set `Content-Type` response header
|
360
|
+
* `name` -- used to set `Content-Disposition` response header
|
313
361
|
|
314
362
|
The `Content-Disposition` header will be set to "inline" by default, but you
|
315
363
|
can change it to "attachment" if you want the browser to always force download:
|
@@ -318,9 +366,6 @@ can change it to "attachment" if you want the browser to always force download:
|
|
318
366
|
Tus::Server.opts[:disposition] = "attachment"
|
319
367
|
```
|
320
368
|
|
321
|
-
The download endpoint supports [Range requests], so you can use the tus
|
322
|
-
file URL as `src` in `<video>` and `<audio>` HTML tags.
|
323
|
-
|
324
369
|
## Checksum
|
325
370
|
|
326
371
|
The following checksum algorithms are supported for the `checksum` extension:
|
@@ -358,8 +403,8 @@ The tus-ruby-server was inspired by [rubytus].
|
|
358
403
|
[termination]: http://tus.io/protocols/resumable-upload.html#termination
|
359
404
|
[GridFS]: https://docs.mongodb.org/v3.0/core/gridfs/
|
360
405
|
[Mongo]: https://github.com/mongodb/mongo-ruby-driver
|
361
|
-
[shrine-tus-demo]: https://github.com/
|
362
|
-
[Shrine]: https://github.com/
|
406
|
+
[shrine-tus-demo]: https://github.com/shrinerb/shrine-tus-demo
|
407
|
+
[Shrine]: https://github.com/shrinerb/shrine
|
363
408
|
[trailing headers]: https://tools.ietf.org/html/rfc7230#section-4.1.2
|
364
409
|
[rubytus]: https://github.com/picocandy/rubytus
|
365
410
|
[aws-sdk-s3]: https://github.com/aws/aws-sdk-ruby/tree/master/gems/aws-sdk-s3
|
@@ -369,3 +414,5 @@ The tus-ruby-server was inspired by [rubytus].
|
|
369
414
|
[Goliath]: https://github.com/postrank-labs/goliath
|
370
415
|
[EventMachine]: https://github.com/eventmachine/eventmachine
|
371
416
|
[goliath-rack_proxy]: https://github.com/janko-m/goliath-rack_proxy
|
417
|
+
[Rack::Sendfile]: https://www.rubydoc.info/github/rack/rack/master/Rack/Sendfile
|
418
|
+
[`Aws::S3::Object#get`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#get-instance_method
|
data/lib/tus/server.rb
CHANGED
@@ -26,6 +26,7 @@ module Tus
|
|
26
26
|
opts[:max_size] = nil
|
27
27
|
opts[:expiration_time] = 7*24*60*60
|
28
28
|
opts[:disposition] = "inline"
|
29
|
+
opts[:download_url] = nil
|
29
30
|
|
30
31
|
plugin :all_verbs
|
31
32
|
plugin :default_headers, {"Content-Type" => ""}
|
@@ -112,6 +113,7 @@ module Tus
|
|
112
113
|
error!(404, "Upload Not Found")
|
113
114
|
end
|
114
115
|
|
116
|
+
# HEAD /{uid}
|
115
117
|
r.head do
|
116
118
|
response.headers.update(info.headers)
|
117
119
|
response.headers["Cache-Control"] = "no-store"
|
@@ -157,22 +159,33 @@ module Tus
|
|
157
159
|
# GET /{uid}
|
158
160
|
r.get do
|
159
161
|
validate_upload_finished!(info)
|
160
|
-
range = handle_range_request!(info.length)
|
161
|
-
|
162
|
-
response.headers["Content-Length"] = (range.end - range.begin + 1).to_s
|
163
162
|
|
164
163
|
metadata = info.metadata
|
165
164
|
name = metadata["name"] || metadata["filename"]
|
166
165
|
type = metadata["type"] || metadata["content_type"]
|
167
166
|
|
168
|
-
|
169
|
-
|
170
|
-
|
171
|
-
|
167
|
+
content_disposition = opts[:disposition]
|
168
|
+
content_disposition += "; filename=\"#{name}\"" if name
|
169
|
+
content_type = type || "application/octet-stream"
|
170
|
+
|
171
|
+
if download_url
|
172
|
+
redirect_url = instance_exec(uid, info.to_h,
|
173
|
+
content_type: content_type,
|
174
|
+
content_disposition: content_disposition,
|
175
|
+
&download_url)
|
176
|
+
|
177
|
+
r.redirect redirect_url
|
178
|
+
else
|
179
|
+
range = handle_range_request!(info.length)
|
180
|
+
|
181
|
+
response.headers["Content-Length"] = range.size.to_s
|
182
|
+
response.headers["Content-Disposition"] = content_disposition
|
183
|
+
response.headers["Content-Type"] = content_type
|
172
184
|
|
173
|
-
|
185
|
+
body = storage.get_file(uid, info.to_h, range: range)
|
174
186
|
|
175
|
-
|
187
|
+
r.halt response.finish_with_body(body)
|
188
|
+
end
|
176
189
|
end
|
177
190
|
|
178
191
|
# DELETE /{uid}
|
@@ -248,7 +261,7 @@ module Tus
|
|
248
261
|
upload_metadata = request.headers["Upload-Metadata"]
|
249
262
|
|
250
263
|
upload_metadata.split(",").each do |string|
|
251
|
-
key, value = string.split(" ")
|
264
|
+
key, value = string.split(" ", 2)
|
252
265
|
|
253
266
|
error!(400, "Invalid Upload-Metadata header") if key.nil?
|
254
267
|
error!(400, "Invalid Upload-Metadata header") if key.ord > 127
|
@@ -309,15 +322,15 @@ module Tus
|
|
309
322
|
# Handles partial responses requested in the "Range" header. Implementation
|
310
323
|
# is mostly copied from Rack::File.
|
311
324
|
def handle_range_request!(length)
|
312
|
-
# we support ranged requests
|
313
|
-
response.headers["Accept-Ranges"] = "bytes"
|
314
|
-
|
315
325
|
if Rack.release >= "2.0"
|
316
326
|
ranges = Rack::Utils.get_byte_ranges(request.headers["Range"], length)
|
317
327
|
else
|
318
328
|
ranges = Rack::Utils.byte_ranges(request.env, length)
|
319
329
|
end
|
320
330
|
|
331
|
+
# we support ranged requests
|
332
|
+
response.headers["Accept-Ranges"] = "bytes"
|
333
|
+
|
321
334
|
if ranges.nil? || ranges.length > 1
|
322
335
|
# no ranges, or multiple ranges (which we don't support)
|
323
336
|
response.status = 200
|
@@ -369,6 +382,14 @@ module Tus
|
|
369
382
|
request.halt
|
370
383
|
end
|
371
384
|
|
385
|
+
def download_url
|
386
|
+
if opts[:download_url] == true
|
387
|
+
storage.method(:file_url)
|
388
|
+
else
|
389
|
+
opts[:download_url]
|
390
|
+
end
|
391
|
+
end
|
392
|
+
|
372
393
|
def storage
|
373
394
|
opts[:storage] || Tus::Storage::Filesystem.new("data")
|
374
395
|
end
|
@@ -27,7 +27,7 @@ module Tus
|
|
27
27
|
file_path(uid).binwrite("")
|
28
28
|
file_path(uid).chmod(@permissions)
|
29
29
|
|
30
|
-
info_path(uid).binwrite("{}")
|
30
|
+
info_path(uid).binwrite("{}") unless info_path(uid).exist?
|
31
31
|
info_path(uid).chmod(@permissions)
|
32
32
|
end
|
33
33
|
|
@@ -99,7 +99,7 @@ module Tus
|
|
99
99
|
end
|
100
100
|
end
|
101
101
|
|
102
|
-
|
102
|
+
Response.new(chunks: chunks, close: file.method(:close), path: file_path(uid).to_s)
|
103
103
|
end
|
104
104
|
|
105
105
|
# Deletes data and info files for the specified upload.
|
@@ -136,6 +136,18 @@ module Tus
|
|
136
136
|
directory.mkpath
|
137
137
|
directory.chmod(@directory_permissions)
|
138
138
|
end
|
139
|
+
|
140
|
+
class Response < Tus::Response
|
141
|
+
def initialize(path:, **options)
|
142
|
+
super(**options)
|
143
|
+
@path = path
|
144
|
+
end
|
145
|
+
|
146
|
+
# Rack::Sendfile middleware needs response body to respond to #to_path
|
147
|
+
def to_path
|
148
|
+
@path
|
149
|
+
end
|
150
|
+
end
|
139
151
|
end
|
140
152
|
end
|
141
153
|
end
|
data/lib/tus/storage/gridfs.rb
CHANGED
data/lib/tus/storage/s3.rb
CHANGED
@@ -1,9 +1,8 @@
|
|
1
1
|
# frozen-string-literal: true
|
2
2
|
|
3
|
+
gem "aws-sdk-s3", "~> 1.2"
|
4
|
+
|
3
5
|
require "aws-sdk-s3"
|
4
|
-
if Gem::Version.new(Aws::S3::GEM_VERSION) < Gem::Version.new("1.2.0")
|
5
|
-
raise "Tus::Storage::S3 requires aws-sdk-s3 version 1.2.0 or above"
|
6
|
-
end
|
7
6
|
|
8
7
|
require "tus/info"
|
9
8
|
require "tus/response"
|
@@ -11,8 +10,6 @@ require "tus/errors"
|
|
11
10
|
|
12
11
|
require "json"
|
13
12
|
require "cgi"
|
14
|
-
require "fiber"
|
15
|
-
require "stringio"
|
16
13
|
|
17
14
|
module Tus
|
18
15
|
module Storage
|
@@ -106,39 +103,34 @@ module Tus
|
|
106
103
|
bytes_uploaded = 0
|
107
104
|
|
108
105
|
jobs = []
|
109
|
-
chunk =
|
106
|
+
chunk = input.read(MIN_PART_SIZE)
|
110
107
|
|
111
|
-
|
112
|
-
next_chunk =
|
108
|
+
while chunk
|
109
|
+
next_chunk = input.read(MIN_PART_SIZE)
|
113
110
|
|
114
111
|
# merge next chunk into previous if it's smaller than minimum chunk size
|
115
|
-
if next_chunk.
|
116
|
-
chunk
|
117
|
-
next_chunk.
|
112
|
+
if next_chunk && next_chunk.bytesize < MIN_PART_SIZE
|
113
|
+
chunk << next_chunk
|
114
|
+
next_chunk.clear
|
118
115
|
next_chunk = nil
|
119
116
|
end
|
120
117
|
|
121
118
|
# abort if chunk is smaller than 5MB and is not the last chunk
|
122
|
-
if chunk.
|
119
|
+
if chunk.bytesize < MIN_PART_SIZE
|
123
120
|
break if (tus_info.length && tus_info.offset) &&
|
124
|
-
chunk.
|
121
|
+
chunk.bytesize + tus_info.offset < tus_info.length
|
125
122
|
end
|
126
123
|
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
|
132
|
-
|
133
|
-
begin
|
134
|
-
jobs.each do |thread, body|
|
135
|
-
info["multipart_parts"] << thread.value
|
136
|
-
bytes_uploaded += body.size
|
137
|
-
body.close
|
124
|
+
begin
|
125
|
+
info["multipart_parts"] << upload_part(chunk, uid, upload_id, part_offset += 1)
|
126
|
+
bytes_uploaded += chunk.bytesize
|
127
|
+
rescue Seahorse::Client::NetworkingError => exception
|
128
|
+
warn "ERROR: #{exception.inspect} occurred during upload"
|
129
|
+
break # ignore networking errors and return what client has uploaded so far
|
138
130
|
end
|
139
|
-
|
140
|
-
|
141
|
-
|
131
|
+
|
132
|
+
chunk.clear
|
133
|
+
chunk = next_chunk
|
142
134
|
end
|
143
135
|
|
144
136
|
bytes_uploaded
|
@@ -177,14 +169,20 @@ module Tus
|
|
177
169
|
# upload can be retrieved in a streaming fashion. Accepts an optional
|
178
170
|
# range parameter for selecting a subset of bytes to retrieve.
|
179
171
|
def get_file(uid, info = {}, range: nil)
|
180
|
-
tus_info = Tus::Info.new(info)
|
181
|
-
|
182
172
|
range = "bytes=#{range.begin}-#{range.end}" if range
|
183
173
|
chunks = object(uid).enum_for(:get, range: range)
|
184
174
|
|
185
175
|
Tus::Response.new(chunks: chunks)
|
186
176
|
end
|
187
177
|
|
178
|
+
# Returns a signed expiring URL to the S3 object.
|
179
|
+
def file_url(uid, info = {}, content_type: nil, content_disposition: nil, **options)
|
180
|
+
options[:response_content_type] ||= content_type
|
181
|
+
options[:response_content_disposition] ||= content_disposition
|
182
|
+
|
183
|
+
object(uid).presigned_url(:get, **options)
|
184
|
+
end
|
185
|
+
|
188
186
|
# Deletes resources for the specified upload. If multipart upload is
|
189
187
|
# still in progress, aborts the multipart upload, otherwise deletes the
|
190
188
|
# object.
|
@@ -222,12 +220,6 @@ module Tus
|
|
222
220
|
|
223
221
|
private
|
224
222
|
|
225
|
-
# Spawns a thread which uploads given body as a new multipart part with
|
226
|
-
# the specified part number to the specified multipart upload.
|
227
|
-
def upload_part_thread(body, key, upload_id, part_number)
|
228
|
-
Thread.new { upload_part(body, key, upload_id, part_number) }
|
229
|
-
end
|
230
|
-
|
231
223
|
# Uploads given body as a new multipart part with the specified part
|
232
224
|
# number to the specified multipart upload. Returns part number and ETag
|
233
225
|
# that will be required later for completing the multipart upload.
|
data/tus-server.gemspec
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: tus-server
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0
|
4
|
+
version: 2.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Janko Marohnić
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2018-05-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: roda
|
@@ -156,7 +156,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
156
156
|
version: '0'
|
157
157
|
requirements: []
|
158
158
|
rubyforge_project:
|
159
|
-
rubygems_version: 2.6
|
159
|
+
rubygems_version: 2.7.6
|
160
160
|
signing_key:
|
161
161
|
specification_version: 4
|
162
162
|
summary: Ruby server implementation of tus.io, the open protocol for resumable file
|