tus-server 2.0.2 → 2.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: f187175b5569bccd517340434f973d7e96f132ba
4
- data.tar.gz: 0ff7f0123216ce0dbdb2489b6adcebf3926c5c45
2
+ SHA256:
3
+ metadata.gz: 16cf0c007be7d383ca0bf2d4aefa6d6c477ac7d32b1905a8ad6c0413da233e57
4
+ data.tar.gz: 258d7d591fc1416468ea21ea42d29ed53f2a64eeade9e5293bdbecc3e5f6a599
5
5
  SHA512:
6
- metadata.gz: 2dd9f5915eb84b1a1ac82bffa71b5d928dcba9663e2af9cc296643fd22bd378fa09d211df0e3099c339bab7e4da64503d36c3b6b4e6041bf990edd5991691591
7
- data.tar.gz: f7fba129bb83e2e2ff8d526bd31581192952f7b86690809c4f1c011df7d31d20b618a088831c7d0e777e86c5c4ce031b652146c3bf54c7a13091a1524e32af20
6
+ metadata.gz: d642a6f120fd220af27bfc7cf8904459acafa6191934a4922913b10b296e0308cd3093223b39317ffb901318e83da2bcd3fa056cf7d2a36a2f7131ce18b8d8ca
7
+ data.tar.gz: c0bf56b77ee95fa1aab167f29fd38feabb1919f7e7d85b1c7705c5adbcba97f9612b5633880af9b603b433a3778518b7875bd5bfafff4f446c476d621c0b9c99
data/CHANGELOG.md CHANGED
@@ -1,3 +1,13 @@
1
+ ## 2.1.0 (2018-05-15)
2
+
3
+ * Add `:download_url` server option for redirecting to a download URL (@janko-m)
4
+
5
+ * Allow application servers to serve files stored on disk via the `Rack::Sendfile` middleware (@janko-m)
6
+
7
+ * Reject `Upload-Metadata` which contains key-value pairs separated by spaces (@janko-m)
8
+
9
+ * Don't overwite info file if it already exists in `Tus::Storage::FileSystem` (@janko-m)
10
+
1
11
  ## 2.0.2 (2017-12-24)
2
12
 
3
13
  * Handle `name` and `type` metadata for Uppy compatibility (@janko-m)
data/README.md CHANGED
@@ -12,62 +12,81 @@ A Ruby server for the [tus resumable upload protocol]. It implements the core
12
12
  ## Installation
13
13
 
14
14
  ```rb
15
- gem "tus-server"
15
+ # Gemfile
16
+ gem "tus-server", "~> 2.0"
16
17
  ```
17
18
 
18
19
  ## Usage
19
20
 
20
- Tus-ruby-server provides a `Tus::Server` Roda app, which you can run in your
21
- `config.ru`. That way you can run `Tus::Server` both as a standalone app or as
22
- part of your main app.
21
+ The gem provides a `Tus::Server` Rack app, which you can mount inside your
22
+ main application. If you're using Rails, you can mount it in `config/routes.rb`:
23
23
 
24
24
  ```rb
25
- # config.ru
25
+ # config/routes.rb (Rails)
26
+ Rails.application.routes.draw do
27
+ # ...
28
+ mount Tus::Server => "/files"
29
+ end
30
+ ```
31
+
32
+ Otherwise you can run it in `config.ru`:
33
+
34
+ ```rb
35
+ # config.ru (Rack)
26
36
  require "tus/server"
27
37
 
28
38
  map "/files" do
29
39
  run Tus::Server
30
40
  end
31
-
32
- run YourApp
33
41
  ```
34
42
 
35
- While this is the most flexible option, it's not optimal in terms of
36
- performance; see the [Goliath](#goliath) section for an alternative approach.
37
-
38
43
  Now you can tell your tus client library (e.g. [tus-js-client]) to use this
39
44
  endpoint:
40
45
 
41
46
  ```js
42
47
  // using tus-js-client
43
48
  new tus.Upload(file, {
44
- endpoint: "http://localhost:9292/files",
45
- chunkSize: 5*1024*1024, // required unless using Goliath or Unicorn
49
+ endpoint: "/files",
50
+ chunkSize: 5*1024*1024, // required unless using Goliath
46
51
  // ...
47
52
  })
48
53
  ```
49
54
 
50
- After the upload is complete, you'll probably want to attach the uploaded file
51
- to a database record. [Shrine] is one file attachment library that integrates
52
- nicely with tus-ruby-server, see [shrine-tus-demo] for an example integration.
55
+ By default uploaded files will be stored in the `data/` directory. After the
56
+ upload is complete, you'll probably want to attach the uploaded file to a
57
+ database record. [Shrine] is one file attachment library that integrates nicely
58
+ with tus-ruby-server, see [shrine-tus-demo] for an example integration.
53
59
 
54
60
  ### Goliath
55
61
 
56
- Among all the existing Ruby web servers, [Goliath] is probably the ideal one to
57
- run tus-ruby-server on. It's built on top of [EventMachine], making it
58
- asynchronous both in reading the request body and writing to the response body.
59
- Goliath also allows tus-ruby-server to handle interrupted requests, by saving
60
- data that has been uploaded until the interruption. This means that with
61
- Goliath it's **not** mandatory for client to chunk the upload into multiple
62
- requests in order to achieve resumable upload (which would be the case for most
63
- other web servers).
62
+ Running the tus server alongside your main app using classic web servers like
63
+ Puma or Unicorn is probably fine for most cases, however, it does come with a
64
+ few gotchas. First, since these web servers don't accept partial requests
65
+ (request where the request body hasn't been fully received), the tus client
66
+ must be configured to split the upload into multiple requests. Second, since
67
+ web workers are tied for the duration of the request, serving uploaded files
68
+ through the tus server app could significantly impact request throughput, so
69
+ you need to be careful to avoid that.
70
+
71
+ There is an alternative. [Goliath] is an asychronous web server built on top of
72
+ [EventMachine], which supports streaming requests and streaming responses.
73
+
74
+ * Asynchronous streaming requests allows the tus server to begin saving
75
+ uploaded data while it's still being received. If the request is interrupted,
76
+ the tus server will attempt to save as much of the data that was received so
77
+ far. This means it's not necessary for the tus client to split the upload
78
+ into multiple smaller requests.
64
79
 
65
- It's recommended that you use [goliath-rack_proxy] for running your tus server
66
- app:
80
+ * Asynchronous streaming responses allows the tus server to stream large files
81
+ with very small impact to the request throughput.
82
+
83
+ Since Goliath is web server, to run tus server on it we'll have to run it as a
84
+ standalone web app. It's recommended that you use [goliath-rack_proxy] for
85
+ running your tus server app:
67
86
 
68
87
  ```rb
69
88
  # Gemfile
70
- gem "tus-server", "~> 1.0"
89
+ gem "tus-server", "~> 2.0"
71
90
  gem "goliath-rack_proxy", "~> 1.0"
72
91
  ```
73
92
  ```rb
@@ -92,66 +111,34 @@ path you can use `Rack::Builder`:
92
111
  ```rb
93
112
  class GoliathTusServer < Goliath::RackProxy
94
113
  rack_app Rack::Builder.new {
95
- map "/files" do
96
- run Tus::Server
97
- end
114
+ map("/files") { run Tus::Server }
98
115
  }
99
116
  rewindable_input false # set to true if you're using checksums
100
117
  end
101
118
  ```
102
119
 
103
- ### Unicorn
104
-
105
- Like Goliath, Unicorn also support streaming uploads, and tus-ruby-server knows
106
- how to automatically recover from `Unicorn::ClientShutdown` exceptions during
107
- upload, storing data that it has received up until that point. Just note that
108
- in order to achieve streaming uploads, Nginx should be configured **not** to
109
- buffer incoming requests, and to disable worker timeout to enable long running
110
- upload requests:
120
+ In this case you'll have to configure the tus client to point to the standalone
121
+ Goliath app:
111
122
 
112
- ```rb
113
- timeout 60*60*24*30 # set worker timeout to 30 days
114
- ```
115
-
116
- But it's also fine to have Nginx buffer requests, just note that in this case
117
- Nginx won't forward incomplete upload requests to tus-ruby-server, so in order
118
- for resumable upload to be possible the client needs to send data in multiple
119
- upload requests (which can then be retried individually).
120
-
121
- Unless you're using the "checksum" tus feature, you might want to consider
122
- disabling rewindability of request body, to prevent Unicorn from additionally
123
- caching received data onto the disk (since that's not necessary unless request
124
- body needs to be rewinded).
125
-
126
- ```rb
127
- # config/unicorn.rb
128
- # ...
129
-
130
- rewindable_input false
123
+ ```js
124
+ // using tus-js-client
125
+ new tus.Upload(file, {
126
+ endpoint: "http://localhost:9000/files",
127
+ // ...
128
+ })
131
129
  ```
132
130
 
133
- ### Other web servers
134
-
135
- It's perfectly feasible to run tus-ruby-server on web servers other than
136
- Goliath or Unicorn (even necessary if you want to run it inside another app).
137
- Just keep in mind that most other web servers don't support request streaming,
138
- which means that tus-ruby-server will be able to start processing upload
139
- requess only once the whole request body has been received. Additionally,
140
- incomplete upload requests won't be forwarded to tus-ruby-server, so in order
141
- for resumable upload to be possible the client needs to send data in multiple
142
- upload requests (which can then be retried individually).
143
-
144
131
  ## Storage
145
132
 
146
133
  ### Filesystem
147
134
 
148
- By default `Tus::Server` stores uploaded files to disk, in the `data/`
149
- directory. You can configure a different directory:
135
+ By default `Tus::Server` stores uploaded files in the `data/` directory. You
136
+ can configure a different directory:
150
137
 
151
138
  ```rb
152
139
  require "tus/storage/filesystem"
153
140
 
154
- Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/cache")
141
+ Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/tus")
155
142
  ```
156
143
 
157
144
  If the configured directory doesn't exist, it will automatically be created.
@@ -171,59 +158,56 @@ store files on the filesystem as they won't persist.
171
158
  All these are reasons why you might store uploaded data on a different storage,
172
159
  and luckily tus-ruby-server ships with two more storages.
173
160
 
174
- ### MongoDB GridFS
175
-
176
- MongoDB has a specification for storing and retrieving large files, called
177
- "[GridFS]". Tus-ruby-server ships with `Tus::Storage::Gridfs` that you can
178
- use, which uses the [Mongo] gem.
179
-
180
- ```rb
181
- gem "mongo", ">= 2.2.2", "< 3"
182
- ```
161
+ #### Serving files
183
162
 
184
- ```rb
185
- require "tus/storage/gridfs"
163
+ If your retrieving uploaded files through the download endpoint, by default the
164
+ files will be served through the Ruby application. However, that's very
165
+ inefficient, as web workers are tied when serving download requests and cannot
166
+ serve additional requests for that duration.
186
167
 
187
- client = Mongo::Client.new("mongodb://127.0.0.1:27017/mydb")
188
- Tus::Server.opts[:storage] = Tus::Storage::Gridfs.new(client: client)
189
- ```
168
+ Therefore, it's highly recommended to delegate serving uploaded files to your
169
+ frontend server (Nginx, Apache). This can be achieved with the
170
+ `Rack::Sendfile` middleware, see its [documentation][Rack::Sendfile] to learn
171
+ more about how to use it with popular frontend servers.
190
172
 
191
- You can change the database prefix (defaults to `fs`):
173
+ If you're using Rails, you can enable the `Rack::Sendfile` middleware by
174
+ setting the `config.action_dispatch.x_sendfile_header` value accordingly:
192
175
 
193
176
  ```rb
194
- Tus::Storage::Gridfs.new(client: client, prefix: "fs_temp")
177
+ config.action_dispatch.x_sendfile_header = "X-Sendfile" # Apache and lighttpd
178
+ # or
179
+ config.action_dispatch.x_sendfile_header = "X-Accel-Redirect" # Nginx
195
180
  ```
196
181
 
197
- By default MongoDB Gridfs stores files in chunks of 256KB, but you can change
198
- that with the `:chunk_size` option:
182
+ Otherwise you can add the `Rack::Sendfile` middleware to the stack in
183
+ `config.ru`:
199
184
 
200
185
  ```rb
201
- Tus::Storage::Gridfs.new(client: client, chunk_size: 1*1024*1024) # 1 MB
186
+ use Rack::Sendfile, "X-Sendfile" # Apache and lighttpd
187
+ # or
188
+ use Rack::Sendfile, "X-Accel-Redirect" # Nginx
202
189
  ```
203
190
 
204
- Note that if you're using the [concatenation] tus feature with Gridfs, all
205
- partial uploads except the last one are required to fill in their Gridfs
206
- chunks, meaning the length of each partial upload needs to be a multiple of the
207
- `:chunk_size` number.
208
-
209
191
  ### Amazon S3
210
192
 
211
- Amazon S3 is probably one of the most popular services for storing files, and
212
- tus-ruby-server ships with `Tus::Storage::S3` which utilizes S3's multipart API
213
- to upload files, and depends on the [aws-sdk-s3] gem.
193
+ You can switch to `Tus::Storage::S3` to uploads files to AWS S3 using the
194
+ multipart API. For this you'll also need to add the [aws-sdk-s3] gem to your
195
+ Gemfile.
214
196
 
215
197
  ```rb
198
+ # Gemfile
216
199
  gem "aws-sdk-s3", "~> 1.2"
217
200
  ```
218
201
 
219
202
  ```rb
220
203
  require "tus/storage/s3"
221
204
 
205
+ # You can omit AWS credentials if you're authenticating in other ways
222
206
  Tus::Server.opts[:storage] = Tus::Storage::S3.new(
207
+ bucket: "my-app", # required
223
208
  access_key_id: "abc",
224
209
  secret_access_key: "xyz",
225
210
  region: "eu-west-1",
226
- bucket: "my-app",
227
211
  )
228
212
  ```
229
213
 
@@ -231,6 +215,27 @@ One thing to note is that S3's multipart API requires each chunk except the
231
215
  last to be **5MB or larger**, so that is the minimum chunk size that you can
232
216
  specify on your tus client if you want to use the S3 storage.
233
217
 
218
+ If you'll be retrieving uploaded files through the tus server app, it's
219
+ recommended to set `Tus::Server.opts[:download_url]` to `true`. This will avoid
220
+ tus server downloading and serving the file from S3, and instead have the
221
+ download endpoint redirect to the direct S3 object URL.
222
+
223
+ ```rb
224
+ Tus::Server.opts[:download_url] = true
225
+ ```
226
+
227
+ You can customize how the S3 object URL is being generated by passing a block
228
+ to `:download_url`, which will then be evaluated in the context of the
229
+ `Tus::Server` instance (which allows accessing the `request` object). See
230
+ [`Aws::S3::Object#get`] for the list of options that
231
+ `Tus::Storage::S3#file_url` accepts.
232
+
233
+ ```rb
234
+ Tus::Server.opts[:download_url] = -> (uid, info, **options) do
235
+ storage.file_url(uid, info, response_expires: 10, **options) # link expires after 10 seconds
236
+ end
237
+ ```
238
+
234
239
  If you want to files to be stored in a certain subdirectory, you can specify
235
240
  a `:prefix` in the storage configuration.
236
241
 
@@ -252,6 +257,42 @@ can for example change the `:endpoint` to use S3's accelerate host:
252
257
  Tus::Storage::S3.new(endpoint: "https://s3-accelerate.amazonaws.com", **options)
253
258
  ```
254
259
 
260
+ ### MongoDB GridFS
261
+
262
+ MongoDB has a specification for storing and retrieving large files, called
263
+ "[GridFS]". Tus-ruby-server ships with `Tus::Storage::Gridfs` that you can
264
+ use, which uses the [Mongo] gem.
265
+
266
+ ```rb
267
+ # Gemfile
268
+ gem "mongo", "~> 2.3"
269
+ ```
270
+
271
+ ```rb
272
+ require "tus/storage/gridfs"
273
+
274
+ client = Mongo::Client.new("mongodb://127.0.0.1:27017/mydb")
275
+ Tus::Server.opts[:storage] = Tus::Storage::Gridfs.new(client: client)
276
+ ```
277
+
278
+ You can change the database prefix (defaults to `fs`):
279
+
280
+ ```rb
281
+ Tus::Storage::Gridfs.new(client: client, prefix: "fs_temp")
282
+ ```
283
+
284
+ By default MongoDB Gridfs stores files in chunks of 256KB, but you can change
285
+ that with the `:chunk_size` option:
286
+
287
+ ```rb
288
+ Tus::Storage::Gridfs.new(client: client, chunk_size: 1*1024*1024) # 1 MB
289
+ ```
290
+
291
+ Note that if you're using the [concatenation] tus feature with Gridfs, all
292
+ partial uploads except the last one are required to fill in their Gridfs
293
+ chunks, meaning the length of each partial upload needs to be a multiple of the
294
+ `:chunk_size` number.
295
+
255
296
  ### Other storages
256
297
 
257
298
  If none of these storages suit you, you can write your own, you just need to
@@ -264,6 +305,7 @@ def patch_file(uid, io, info = {}) ... end
264
305
  def update_info(uid, info) ... end
265
306
  def read_info(uid) ... end
266
307
  def get_file(uid, info = {}, range: nil) ... end
308
+ def file_url(uid, info = {}, **options) ... end # optional
267
309
  def delete_file(uid, info = {}) ... end
268
310
  def expire_files(expiration_date) ... end
269
311
  ```
@@ -302,14 +344,20 @@ tus_storage.expire_files(expiration_time)
302
344
  ## Download
303
345
 
304
346
  In addition to implementing the tus protocol, tus-ruby-server also comes with a
305
- GET endpoint for downloading the uploaded file, which streams the file from the
306
- storage into the response body.
347
+ GET endpoint for downloading the uploaded file, which by default streams the
348
+ file from the storage. It supports [Range requests], so you can use the tus
349
+ file URL as `src` in `<video>` and `<audio>` HTML tags.
350
+
351
+ It's highly recommended not to serve files through the app, but offload it to
352
+ your frontend server if using disk storage, or if using S3 storage have the
353
+ download endpoint redirect to the S3 object URL. See the documentation for the
354
+ individual storage for instructions how to set this up.
307
355
 
308
356
  The endpoint will automatically use the following `Upload-Metadata` values if
309
357
  they're available:
310
358
 
311
- * `type` -- used in the `Content-Type` response header
312
- * `name` -- used in the `Content-Disposition` response header
359
+ * `type` -- used to set `Content-Type` response header
360
+ * `name` -- used to set `Content-Disposition` response header
313
361
 
314
362
  The `Content-Disposition` header will be set to "inline" by default, but you
315
363
  can change it to "attachment" if you want the browser to always force download:
@@ -318,9 +366,6 @@ can change it to "attachment" if you want the browser to always force download:
318
366
  Tus::Server.opts[:disposition] = "attachment"
319
367
  ```
320
368
 
321
- The download endpoint supports [Range requests], so you can use the tus
322
- file URL as `src` in `<video>` and `<audio>` HTML tags.
323
-
324
369
  ## Checksum
325
370
 
326
371
  The following checksum algorithms are supported for the `checksum` extension:
@@ -358,8 +403,8 @@ The tus-ruby-server was inspired by [rubytus].
358
403
  [termination]: http://tus.io/protocols/resumable-upload.html#termination
359
404
  [GridFS]: https://docs.mongodb.org/v3.0/core/gridfs/
360
405
  [Mongo]: https://github.com/mongodb/mongo-ruby-driver
361
- [shrine-tus-demo]: https://github.com/janko-m/shrine-tus-demo
362
- [Shrine]: https://github.com/janko-m/shrine
406
+ [shrine-tus-demo]: https://github.com/shrinerb/shrine-tus-demo
407
+ [Shrine]: https://github.com/shrinerb/shrine
363
408
  [trailing headers]: https://tools.ietf.org/html/rfc7230#section-4.1.2
364
409
  [rubytus]: https://github.com/picocandy/rubytus
365
410
  [aws-sdk-s3]: https://github.com/aws/aws-sdk-ruby/tree/master/gems/aws-sdk-s3
@@ -369,3 +414,5 @@ The tus-ruby-server was inspired by [rubytus].
369
414
  [Goliath]: https://github.com/postrank-labs/goliath
370
415
  [EventMachine]: https://github.com/eventmachine/eventmachine
371
416
  [goliath-rack_proxy]: https://github.com/janko-m/goliath-rack_proxy
417
+ [Rack::Sendfile]: https://www.rubydoc.info/github/rack/rack/master/Rack/Sendfile
418
+ [`Aws::S3::Object#get`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#get-instance_method
data/lib/tus/server.rb CHANGED
@@ -26,6 +26,7 @@ module Tus
26
26
  opts[:max_size] = nil
27
27
  opts[:expiration_time] = 7*24*60*60
28
28
  opts[:disposition] = "inline"
29
+ opts[:download_url] = nil
29
30
 
30
31
  plugin :all_verbs
31
32
  plugin :default_headers, {"Content-Type" => ""}
@@ -112,6 +113,7 @@ module Tus
112
113
  error!(404, "Upload Not Found")
113
114
  end
114
115
 
116
+ # HEAD /{uid}
115
117
  r.head do
116
118
  response.headers.update(info.headers)
117
119
  response.headers["Cache-Control"] = "no-store"
@@ -157,22 +159,33 @@ module Tus
157
159
  # GET /{uid}
158
160
  r.get do
159
161
  validate_upload_finished!(info)
160
- range = handle_range_request!(info.length)
161
-
162
- response.headers["Content-Length"] = (range.end - range.begin + 1).to_s
163
162
 
164
163
  metadata = info.metadata
165
164
  name = metadata["name"] || metadata["filename"]
166
165
  type = metadata["type"] || metadata["content_type"]
167
166
 
168
- response.headers["Content-Disposition"] = opts[:disposition]
169
- response.headers["Content-Disposition"] += "; filename=\"#{name}\"" if name
170
- response.headers["Content-Type"] = type || "application/octet-stream"
171
- response.headers
167
+ content_disposition = opts[:disposition]
168
+ content_disposition += "; filename=\"#{name}\"" if name
169
+ content_type = type || "application/octet-stream"
170
+
171
+ if download_url
172
+ redirect_url = instance_exec(uid, info.to_h,
173
+ content_type: content_type,
174
+ content_disposition: content_disposition,
175
+ &download_url)
176
+
177
+ r.redirect redirect_url
178
+ else
179
+ range = handle_range_request!(info.length)
180
+
181
+ response.headers["Content-Length"] = range.size.to_s
182
+ response.headers["Content-Disposition"] = content_disposition
183
+ response.headers["Content-Type"] = content_type
172
184
 
173
- body = storage.get_file(uid, info.to_h, range: range)
185
+ body = storage.get_file(uid, info.to_h, range: range)
174
186
 
175
- request.halt response.finish_with_body(body)
187
+ r.halt response.finish_with_body(body)
188
+ end
176
189
  end
177
190
 
178
191
  # DELETE /{uid}
@@ -248,7 +261,7 @@ module Tus
248
261
  upload_metadata = request.headers["Upload-Metadata"]
249
262
 
250
263
  upload_metadata.split(",").each do |string|
251
- key, value = string.split(" ")
264
+ key, value = string.split(" ", 2)
252
265
 
253
266
  error!(400, "Invalid Upload-Metadata header") if key.nil?
254
267
  error!(400, "Invalid Upload-Metadata header") if key.ord > 127
@@ -309,15 +322,15 @@ module Tus
309
322
  # Handles partial responses requested in the "Range" header. Implementation
310
323
  # is mostly copied from Rack::File.
311
324
  def handle_range_request!(length)
312
- # we support ranged requests
313
- response.headers["Accept-Ranges"] = "bytes"
314
-
315
325
  if Rack.release >= "2.0"
316
326
  ranges = Rack::Utils.get_byte_ranges(request.headers["Range"], length)
317
327
  else
318
328
  ranges = Rack::Utils.byte_ranges(request.env, length)
319
329
  end
320
330
 
331
+ # we support ranged requests
332
+ response.headers["Accept-Ranges"] = "bytes"
333
+
321
334
  if ranges.nil? || ranges.length > 1
322
335
  # no ranges, or multiple ranges (which we don't support)
323
336
  response.status = 200
@@ -369,6 +382,14 @@ module Tus
369
382
  request.halt
370
383
  end
371
384
 
385
+ def download_url
386
+ if opts[:download_url] == true
387
+ storage.method(:file_url)
388
+ else
389
+ opts[:download_url]
390
+ end
391
+ end
392
+
372
393
  def storage
373
394
  opts[:storage] || Tus::Storage::Filesystem.new("data")
374
395
  end
@@ -27,7 +27,7 @@ module Tus
27
27
  file_path(uid).binwrite("")
28
28
  file_path(uid).chmod(@permissions)
29
29
 
30
- info_path(uid).binwrite("{}")
30
+ info_path(uid).binwrite("{}") unless info_path(uid).exist?
31
31
  info_path(uid).chmod(@permissions)
32
32
  end
33
33
 
@@ -99,7 +99,7 @@ module Tus
99
99
  end
100
100
  end
101
101
 
102
- Tus::Response.new(chunks: chunks, close: file.method(:close))
102
+ Response.new(chunks: chunks, close: file.method(:close), path: file_path(uid).to_s)
103
103
  end
104
104
 
105
105
  # Deletes data and info files for the specified upload.
@@ -136,6 +136,18 @@ module Tus
136
136
  directory.mkpath
137
137
  directory.chmod(@directory_permissions)
138
138
  end
139
+
140
+ class Response < Tus::Response
141
+ def initialize(path:, **options)
142
+ super(**options)
143
+ @path = path
144
+ end
145
+
146
+ # Rack::Sendfile middleware needs response body to respond to #to_path
147
+ def to_path
148
+ @path
149
+ end
150
+ end
139
151
  end
140
152
  end
141
153
  end
@@ -1,5 +1,7 @@
1
1
  # frozen-string-literal: true
2
2
 
3
+ gem "mongo", "~> 2.3"
4
+
3
5
  require "mongo"
4
6
 
5
7
  require "tus/info"
@@ -1,9 +1,8 @@
1
1
  # frozen-string-literal: true
2
2
 
3
+ gem "aws-sdk-s3", "~> 1.2"
4
+
3
5
  require "aws-sdk-s3"
4
- if Gem::Version.new(Aws::S3::GEM_VERSION) < Gem::Version.new("1.2.0")
5
- raise "Tus::Storage::S3 requires aws-sdk-s3 version 1.2.0 or above"
6
- end
7
6
 
8
7
  require "tus/info"
9
8
  require "tus/response"
@@ -11,8 +10,6 @@ require "tus/errors"
11
10
 
12
11
  require "json"
13
12
  require "cgi"
14
- require "fiber"
15
- require "stringio"
16
13
 
17
14
  module Tus
18
15
  module Storage
@@ -106,39 +103,34 @@ module Tus
106
103
  bytes_uploaded = 0
107
104
 
108
105
  jobs = []
109
- chunk = StringIO.new(input.read(MIN_PART_SIZE).to_s)
106
+ chunk = input.read(MIN_PART_SIZE)
110
107
 
111
- loop do
112
- next_chunk = StringIO.new(input.read(MIN_PART_SIZE).to_s)
108
+ while chunk
109
+ next_chunk = input.read(MIN_PART_SIZE)
113
110
 
114
111
  # merge next chunk into previous if it's smaller than minimum chunk size
115
- if next_chunk.size < MIN_PART_SIZE
116
- chunk = StringIO.new(chunk.string + next_chunk.string)
117
- next_chunk.close
112
+ if next_chunk && next_chunk.bytesize < MIN_PART_SIZE
113
+ chunk << next_chunk
114
+ next_chunk.clear
118
115
  next_chunk = nil
119
116
  end
120
117
 
121
118
  # abort if chunk is smaller than 5MB and is not the last chunk
122
- if chunk.size < MIN_PART_SIZE
119
+ if chunk.bytesize < MIN_PART_SIZE
123
120
  break if (tus_info.length && tus_info.offset) &&
124
- chunk.size + tus_info.offset < tus_info.length
121
+ chunk.bytesize + tus_info.offset < tus_info.length
125
122
  end
126
123
 
127
- thread = upload_part_thread(chunk, uid, upload_id, part_offset += 1)
128
- jobs << [thread, chunk]
129
-
130
- chunk = next_chunk or break
131
- end
132
-
133
- begin
134
- jobs.each do |thread, body|
135
- info["multipart_parts"] << thread.value
136
- bytes_uploaded += body.size
137
- body.close
124
+ begin
125
+ info["multipart_parts"] << upload_part(chunk, uid, upload_id, part_offset += 1)
126
+ bytes_uploaded += chunk.bytesize
127
+ rescue Seahorse::Client::NetworkingError => exception
128
+ warn "ERROR: #{exception.inspect} occurred during upload"
129
+ break # ignore networking errors and return what client has uploaded so far
138
130
  end
139
- rescue Seahorse::Client::NetworkingError => exception
140
- warn "ERROR: #{exception.inspect} occurred during upload"
141
- # ignore networking errors and return what client has uploaded so far
131
+
132
+ chunk.clear
133
+ chunk = next_chunk
142
134
  end
143
135
 
144
136
  bytes_uploaded
@@ -177,14 +169,20 @@ module Tus
177
169
  # upload can be retrieved in a streaming fashion. Accepts an optional
178
170
  # range parameter for selecting a subset of bytes to retrieve.
179
171
  def get_file(uid, info = {}, range: nil)
180
- tus_info = Tus::Info.new(info)
181
-
182
172
  range = "bytes=#{range.begin}-#{range.end}" if range
183
173
  chunks = object(uid).enum_for(:get, range: range)
184
174
 
185
175
  Tus::Response.new(chunks: chunks)
186
176
  end
187
177
 
178
+ # Returns a signed expiring URL to the S3 object.
179
+ def file_url(uid, info = {}, content_type: nil, content_disposition: nil, **options)
180
+ options[:response_content_type] ||= content_type
181
+ options[:response_content_disposition] ||= content_disposition
182
+
183
+ object(uid).presigned_url(:get, **options)
184
+ end
185
+
188
186
  # Deletes resources for the specified upload. If multipart upload is
189
187
  # still in progress, aborts the multipart upload, otherwise deletes the
190
188
  # object.
@@ -222,12 +220,6 @@ module Tus
222
220
 
223
221
  private
224
222
 
225
- # Spawns a thread which uploads given body as a new multipart part with
226
- # the specified part number to the specified multipart upload.
227
- def upload_part_thread(body, key, upload_id, part_number)
228
- Thread.new { upload_part(body, key, upload_id, part_number) }
229
- end
230
-
231
223
  # Uploads given body as a new multipart part with the specified part
232
224
  # number to the specified multipart upload. Returns part number and ETag
233
225
  # that will be required later for completing the multipart upload.
data/tus-server.gemspec CHANGED
@@ -1,6 +1,6 @@
1
1
  Gem::Specification.new do |gem|
2
2
  gem.name = "tus-server"
3
- gem.version = "2.0.2"
3
+ gem.version = "2.1.0"
4
4
 
5
5
  gem.required_ruby_version = ">= 2.2"
6
6
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: tus-server
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.2
4
+ version: 2.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Janko Marohnić
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-12-24 00:00:00.000000000 Z
11
+ date: 2018-05-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: roda
@@ -156,7 +156,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
156
156
  version: '0'
157
157
  requirements: []
158
158
  rubyforge_project:
159
- rubygems_version: 2.6.11
159
+ rubygems_version: 2.7.6
160
160
  signing_key:
161
161
  specification_version: 4
162
162
  summary: Ruby server implementation of tus.io, the open protocol for resumable file