tus-server 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 9dfd9cb38d7ef46fe3ac684a7ee3569f66e97c84
-  data.tar.gz: 19b75e3d0c222d47c46432f1fd198ab9e27f56bb
+  metadata.gz: 0016bea27534678199b2a3631ab710a9bb4bfd71
+  data.tar.gz: d2026f565ba010d7e6e64e9ab89bd06a1afc2be0
 SHA512:
-  metadata.gz: 48c902ba0c5ff439869a93b3b4fa851fe17644c7a41978b25041f4955b5c0697e9a725ece84a5ee1d294acd7c631ad04b9d4348f327a64c1ca0002549b1db509
-  data.tar.gz: 482543723d79aeb1df06a9fb429d8f39584e4891905e795c32df660f860413ffcac4408892114a85c67e4b1c867e1d6b9462157d6f495c9399d3d641e921f6b8
+  metadata.gz: 0c11c877cad9b293df94ec7859bfea15b40f9401c99ea593a1662e9735ba6c4d216cf986b5f761e8c81d5bee6081ac49686dc3550682f6b695c08c39a7306a9b
+  data.tar.gz: 19671d54820222ebb0855df11da3d673975e8e50afa3fe1f3ab30659b23cdaadc45e71100923c132562f348856949d3e216816800278e96d729ce5ff8f88455c

data/README.md

@@ -42,7 +42,7 @@ endpoint:
 // using tus-js-client
 new tus.Upload(file, {
   endpoint: "http://localhost:9292/files",
-  chunkSize: 5*1024*1024, // required unless using Goliath
+  chunkSize: 5*1024*1024, // required unless using Goliath or Unicorn
   // ...
 })
 ```
@@ -53,39 +53,62 @@ nicely with tus-ruby-server, see [shrine-tus-demo] for an example integration.
 
 ### Goliath
 
-[Goliath] is the ideal web server to run tus-ruby-server on, because by
-utilizing [EventMachine] it's asnychronous both in reading the request body and
-writing to the response body, so it's not affected by slow clients. Goliath
-also allows tus-ruby-server to handle interrupted requests, by saving data that
-has been uploaded until the interruption. This means that with Goliath it's
-**not** mandatory for client to chunk the upload into multiple requests in
-order to achieve resumable uploads, which would be the case for most other web
-servers.
+Among all the existing Ruby web servers, [Goliath] is probably the ideal one to
+run tus-ruby-server on. It's built on top of [EventMachine], making it
+asynchronous both in reading the request body and writing to the response body.
+Goliath also allows tus-ruby-server to handle interrupted requests, by saving
+data that has been uploaded until the interruption. This means that with
+Goliath it's **not** mandatory for client to chunk the upload into multiple
+requests in order to achieve resumable upload (which would be the case for most
+other web servers).
 
-Tus-ruby-server ships with Goliath integration, you just need to require it in
-a Ruby file and run that file, and that will automatically start up Goliath.
+It's recommended that you use [goliath-rack_proxy] for running your tus server
+app:
 
 ```rb
 # Gemfile
 gem "tus-server", "~> 1.0"
-gem "goliath"
-gem "async-rack", ">= 0.5.1"
+gem "goliath-rack_proxy"
 ```
 ```rb
 # tus.rb
-require "tus/server/goliath"
+require "tus/server"
+require "goliath/rack_proxy"
 
 # any additional Tus::Server configuration you want to put in here
+
+class GoliathTusServer < Goliath::RackProxy
+  rack_app Tus::Server
+  rewindable_input false # set to true if you're using checksums
+end
 ```
 ```sh
 $ ruby tus.rb --stdout # enable logging
 ```
 
-Any options provided after the Ruby file will be passed in to the Goliath
-server, see [this wiki][goliath server options] for all available options that
-Goliath supports. As shown above, running tus-ruby-server on Goliath means you
-have to run it separately from your main app (unless your main app is also on
-Goliath).
+### Unicorn
+
+Like Goliath, Unicorn also support streaming uploads, and tus-ruby-server knows
+how to automatically recover from `Unicorn::ClientShutdown` exceptions during
+upload, storing data that it has received up until that point. Just note that
+in order to achieve streaming uploads, Nginx should be configured **not** to
+buffer incoming requests.
+
+But it's also fine to have Nginx buffer requests, just note that in this case
+Nginx won't forward incomplete upload requests to tus-ruby-server, so in order
+for resumable upload to be possible the client needs to send data in multiple
+upload requests (which can then be retried individually).
+
+### Other web servers
+
+It's perfectly feasible to run tus-ruby-server on web servers other than
+Goliath or Unicorn (even necessary if you want to run it inside another app).
+Just keep in mind that most other web servers don't support request streaming,
+which means that tus-ruby-server will be able to start processing upload
+requess only once the whole request body has been received. Additionally,
+incomplete upload requests won't be forwarded to tus-ruby-server, so in order
+for resumable upload to be possible the client needs to send data in multiple
+upload requests (which can then be retried individually).
 
 ## Storage
 
@@ -311,4 +334,4 @@ The tus-ruby-server was inspired by [rubytus].
 [Range requests]: https://tools.ietf.org/html/rfc7233
 [Goliath]: https://github.com/postrank-labs/goliath
 [EventMachine]: https://github.com/eventmachine/eventmachine
-[goliath server options]: https://github.com/postrank-labs/goliath/wiki/Server
+[goliath-rack_proxy]: https://github.com/janko-m/goliath-rack_proxy

data/lib/tus/checksum.rb

@@ -1,3 +1,5 @@
+# frozen-string-literal: true
+
 module Tus
   class Checksum
     CHUNK_SIZE = 16*1024
@@ -48,7 +50,7 @@ module Tus
       require "zlib"
       require "base64"
       crc = Zlib.crc32("")
-      while (data = io.read(CHUNK_SIZE, buffer ||= ""))
+      while (data = io.read(CHUNK_SIZE, buffer ||= String.new))
         crc = Zlib.crc32(data, crc)
       end
       Base64.strict_encode64(crc.to_s)
@@ -57,7 +59,7 @@ module Tus
     def digest(name, io)
       require "digest"
       digest = Digest.const_get(name).new
-      while (data = io.read(CHUNK_SIZE, buffer ||= ""))
+      while (data = io.read(CHUNK_SIZE, buffer ||= String.new))
         digest.update(data)
       end
       digest.base64digest

data/lib/tus/errors.rb

@@ -1,3 +1,5 @@
+# frozen-string-literal: true
+
 module Tus
   Error           = Class.new(StandardError)
   NotFound        = Class.new(Error)

data/lib/tus/info.rb

@@ -1,3 +1,4 @@
+# frozen-string-literal: true
 require "base64"
 require "time"
 

data/lib/tus/input.rb

@@ -1,3 +1,4 @@
+# frozen-string-literal: true
 require "tus/errors"
 
 module Tus

data/lib/tus/server.rb

@@ -1,3 +1,4 @@
+# frozen-string-literal: true
 require "roda"
 
 require "tus/storage/filesystem"

data/lib/tus/server/goliath.rb

@@ -1,6 +1,9 @@
+# frozen-string-literal: true
 require "tus/server"
 require "goliath"
 
+warn "Tus::Server::Goliath has been deprecated in favor of goliath-rack_proxy -- https://github.com/janko-m/goliath-rack_proxy"
+
 class Tus::Server::Goliath < Goliath::API
   # Called as soon as request headers are parsed.
   def on_headers(env, headers)

data/lib/tus/storage/filesystem.rb

@@ -1,3 +1,4 @@
+# frozen-string-literal: true
 require "tus/errors"
 
 require "pathname"

data/lib/tus/storage/gridfs.rb

@@ -1,3 +1,4 @@
+# frozen-string-literal: true
 require "mongo"
 
 require "tus/info"

data/lib/tus/storage/s3.rb

@@ -1,3 +1,4 @@
+# frozen-string-literal: true
 require "aws-sdk"
 
 require "tus/info"
@@ -102,10 +103,15 @@ module Tus
           chunk = next_chunk or break
         end
 
-        jobs.each do |thread, body|
-          info["multipart_parts"] << thread.value
-          bytes_uploaded += body.size
-          body.close
+        begin
+          jobs.each do |thread, body|
+            info["multipart_parts"] << thread.value
+            bytes_uploaded += body.size
+            body.close
+          end
+        rescue Seahorse::Client::NetworkingError => exception
+          warn "ERROR: #{exception.inspect} occurred during upload"
+          # ignore networking errors and return what client has uploaded so far
         end
 
         bytes_uploaded
@@ -214,6 +220,7 @@ module Tus
       def copy_parts(objects, multipart_upload)
         parts = compute_parts(objects, multipart_upload)
         queue = parts.inject(Queue.new) { |queue, part| queue << part }
+        queue.close
 
         threads = @thread_count.times.map { copy_part_thread(queue) }
 
@@ -236,8 +243,7 @@ module Tus
         Thread.new do
           begin
             results = []
-            loop do
-              part = queue.deq(true) rescue break
+            while part = queue.pop
               results << copy_part(part)
             end
             results

data/tus-server.gemspec

@@ -1,6 +1,6 @@
 Gem::Specification.new do |gem|
   gem.name         = "tus-server"
-  gem.version      = "1.0.0"
+  gem.version      = "1.1.0"
 
   gem.required_ruby_version = ">= 2.1"
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: tus-server
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Janko Marohnić
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-07-17 00:00:00.000000000 Z
+date: 2017-07-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: roda