tus-server 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +164 -0
- data/lib/tus/checksum.rb +48 -0
- data/lib/tus/expirator.rb +58 -0
- data/lib/tus/info.rb +70 -0
- data/lib/tus/server.rb +310 -0
- data/lib/tus/storage/filesystem.rb +78 -0
- data/lib/tus/storage/gridfs.rb +73 -0
- data/lib/tus-server.rb +1 -0
- data/tus-server.gemspec +24 -0
- metadata +138 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: ac1db8e0078cd0824caf54dc3c1993957ca90902
|
4
|
+
data.tar.gz: 032f12c4aaa676e7f55388f8af96d7a46cad42a7
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 45aa99b75d86eb9197fa3edcb8fd4a140004e9e48bb955327e5e77dd7ac3fa3b407a5782f4bcf12fb2a7277704f4ea4cfd6a2972a35d3d69b5e3c778708dcb0f
|
7
|
+
data.tar.gz: b27487dc797127e1c51010e65e1ad44e22206bfcc8ff5dbecc4136eedf4c2197c6ae13b1d32bada612d614844f61e9780349748050559bb47c2ea386306bf31d
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2015-2016 Janko Marohnić
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,164 @@
|
|
1
|
+
# tus-ruby-server
|
2
|
+
|
3
|
+
A Ruby server for the [tus resumable upload protocol]. It implements the core
|
4
|
+
1.0 protocol, along with the following extensions:
|
5
|
+
|
6
|
+
* [`creation`][creation] (and `creation-defer-length`)
|
7
|
+
* [`concatenation`][concatenation] (and `concatenation-unfinished`)
|
8
|
+
* [`checksum`][checksum]
|
9
|
+
* [`expiration`][expiration]
|
10
|
+
* [`termination`][termination]
|
11
|
+
|
12
|
+
## Installation
|
13
|
+
|
14
|
+
```rb
|
15
|
+
gem "tus-server"
|
16
|
+
```
|
17
|
+
|
18
|
+
## Usage
|
19
|
+
|
20
|
+
You can run the `Tus::Server` in your `config.ru`:
|
21
|
+
|
22
|
+
```rb
|
23
|
+
# config.ru
|
24
|
+
require "tus/server"
|
25
|
+
|
26
|
+
map "/files" do
|
27
|
+
run Tus::Server
|
28
|
+
end
|
29
|
+
|
30
|
+
run YourApp
|
31
|
+
```
|
32
|
+
|
33
|
+
Now you can tell your tus client library (e.g. [tus-js-client]) to use this
|
34
|
+
endpoint:
|
35
|
+
|
36
|
+
```js
|
37
|
+
// using tus-js-client
|
38
|
+
new tus.Upload(file, {
|
39
|
+
endpoint: "http://localhost:9292/files",
|
40
|
+
chunkSize: 15*1024*1024, # 15 MB
|
41
|
+
// ...
|
42
|
+
})
|
43
|
+
```
|
44
|
+
|
45
|
+
After upload is complete, you'll probably want to attach the uploaded files to
|
46
|
+
database records. [Shrine] is one file attachment library that supports this,
|
47
|
+
see [shrine-tus-demo] on how you can integrate the two.
|
48
|
+
|
49
|
+
### Metadata
|
50
|
+
|
51
|
+
As per tus protocol, you can assign custom metadata when creating a file using
|
52
|
+
the `Upload-Metadata` header. When retrieving the file via a GET request,
|
53
|
+
tus-server will use
|
54
|
+
|
55
|
+
* `content_type` -- for setting the `Content-Type` header
|
56
|
+
* `filename` -- for setting the `Content-Disposition` header
|
57
|
+
|
58
|
+
Both of these are optional, and will be used if available.
|
59
|
+
|
60
|
+
### Storage
|
61
|
+
|
62
|
+
By default `Tus::Server` saves partial and complete files on the filesystem,
|
63
|
+
inside the `data/` directory. You can easily change the directory:
|
64
|
+
|
65
|
+
```rb
|
66
|
+
require "tus/server"
|
67
|
+
|
68
|
+
Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/cache")
|
69
|
+
```
|
70
|
+
|
71
|
+
The downside of storing files on the filesystem is that it isn't distributed,
|
72
|
+
so for resumable uploads to work you have to host the tus application on a
|
73
|
+
single server.
|
74
|
+
|
75
|
+
However, tus-server also ships with MongoDB [GridFS] storage, which among other
|
76
|
+
things is convenient for a multi-server setup. It requires the [Mongo] gem:
|
77
|
+
|
78
|
+
```rb
|
79
|
+
gem "mongo"
|
80
|
+
```
|
81
|
+
|
82
|
+
```rb
|
83
|
+
require "tus/server"
|
84
|
+
require "tus/storage/gridfs"
|
85
|
+
|
86
|
+
client = Mongo::Client.new("mongodb://127.0.0.1:27017/mydb")
|
87
|
+
Tus::Server.opts[:storage] = Tus::Storage::Gridfs.new(client: client)
|
88
|
+
```
|
89
|
+
|
90
|
+
You can also write your own storage, you just need to implement the same
|
91
|
+
public interface that `Tus::Storage::Filesystem` and `Tus::Storage::Gridfs` do.
|
92
|
+
|
93
|
+
### Maximum size
|
94
|
+
|
95
|
+
By default the maximum size for an uploaded file is 1GB, but you can change
|
96
|
+
that:
|
97
|
+
|
98
|
+
```rb
|
99
|
+
require "tus/server"
|
100
|
+
|
101
|
+
Tus::Server.opts[:max_size] = 5 * 1024*1024*1024 # 5GB
|
102
|
+
# or
|
103
|
+
Tus::Server.opts[:max_size] = nil # no limit
|
104
|
+
```
|
105
|
+
|
106
|
+
### Expiration
|
107
|
+
|
108
|
+
By default both partially and fully uploaded files will get deleted after one
|
109
|
+
week, and the interval of checking for expired files is 1 hour. You can change
|
110
|
+
both of these:
|
111
|
+
|
112
|
+
```rb
|
113
|
+
require "tus/server"
|
114
|
+
|
115
|
+
Tus::Server.opts[:expiration_time] = 14*24*60*60 # 2 weeks
|
116
|
+
Tus::Server.opts[:expiration_interval] = 24*60*60 # 1 day
|
117
|
+
```
|
118
|
+
|
119
|
+
### Checksum
|
120
|
+
|
121
|
+
The following checksum algorithms are supported for the `checksum` extension:
|
122
|
+
|
123
|
+
* SHA1
|
124
|
+
* SHA256
|
125
|
+
* SHA384
|
126
|
+
* SHA512
|
127
|
+
* MD5
|
128
|
+
* CRC32
|
129
|
+
|
130
|
+
## Limitations
|
131
|
+
|
132
|
+
Since tus-server is built using a Rack-based web framework (Roda), if a PATCH
|
133
|
+
request gets interrupted, none of the received data will be stored. It's
|
134
|
+
recommended to configure your client tus library not to use a single PATCH
|
135
|
+
request for large files, but rather to split it into multiple chunks. You can
|
136
|
+
do that for [tus-js-client] by specifying a maximum chunk size:
|
137
|
+
|
138
|
+
```js
|
139
|
+
new tus.Upload(file, {
|
140
|
+
endpoint: "http://localhost:9292/files",
|
141
|
+
chunkSize: 15*1024*1024, # 15 MB
|
142
|
+
// ...
|
143
|
+
})
|
144
|
+
```
|
145
|
+
|
146
|
+
Tus-server also currently doesn't support the `checksum-trailer` extension,
|
147
|
+
which would allow sending the checksum header *after* the data has been sent,
|
148
|
+
using [trailing headers].
|
149
|
+
|
150
|
+
## License
|
151
|
+
|
152
|
+
[MIT](/LICENSE.txt)
|
153
|
+
|
154
|
+
[tus resumable upload protocol]: http://tus.io/
|
155
|
+
[tus-js-client]: https://github.com/tus/tus-js-client
|
156
|
+
[creation]: http://tus.io/protocols/resumable-upload.html#creation
|
157
|
+
[concatenation]: http://tus.io/protocols/resumable-upload.html#concatenation
|
158
|
+
[checksum]: http://tus.io/protocols/resumable-upload.html#checksum
|
159
|
+
[expiration]: http://tus.io/protocols/resumable-upload.html#expiration
|
160
|
+
[termination]: http://tus.io/protocols/resumable-upload.html#termination
|
161
|
+
[GridFS]: https://docs.mongodb.org/v3.0/core/gridfs/
|
162
|
+
[shrine-tus-demo]: https://github.com/janko-m/shrine-tus-demo
|
163
|
+
[Shrine]: https://github.com/janko-m/shrine
|
164
|
+
[trailing headers]: https://tools.ietf.org/html/rfc7230#section-4.1.2
|
data/lib/tus/checksum.rb
ADDED
@@ -0,0 +1,48 @@
|
|
1
|
+
require "base64"
|
2
|
+
require "digest"
|
3
|
+
require "zlib"
|
4
|
+
|
5
|
+
module Tus
|
6
|
+
class Checksum
|
7
|
+
attr_reader :algorithm
|
8
|
+
|
9
|
+
def initialize(algorithm)
|
10
|
+
@algorithm = algorithm
|
11
|
+
end
|
12
|
+
|
13
|
+
def match?(checksum, content)
|
14
|
+
checksum = Base64.decode64(checksum)
|
15
|
+
generate(content) == checksum
|
16
|
+
end
|
17
|
+
|
18
|
+
def generate(content)
|
19
|
+
send("generate_#{algorithm}", content)
|
20
|
+
end
|
21
|
+
|
22
|
+
private
|
23
|
+
|
24
|
+
def generate_sha1(content)
|
25
|
+
Digest::SHA1.hexdigest(content)
|
26
|
+
end
|
27
|
+
|
28
|
+
def generate_sha256(content)
|
29
|
+
Digest::SHA256.hexdigest(content)
|
30
|
+
end
|
31
|
+
|
32
|
+
def generate_sha384(content)
|
33
|
+
Digest::SHA384.hexdigest(content)
|
34
|
+
end
|
35
|
+
|
36
|
+
def generate_sha512(content)
|
37
|
+
Digest::SHA512.hexdigest(content)
|
38
|
+
end
|
39
|
+
|
40
|
+
def generate_md5(content)
|
41
|
+
Digest::MD5.hexdigest(content)
|
42
|
+
end
|
43
|
+
|
44
|
+
def generate_crc32(content)
|
45
|
+
Zlib.crc32(content).to_s
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -0,0 +1,58 @@
|
|
1
|
+
require "tus/info"
|
2
|
+
require "time"
|
3
|
+
|
4
|
+
module Tus
|
5
|
+
class Expirator
|
6
|
+
attr_reader :storage, :interval
|
7
|
+
|
8
|
+
def initialize(storage, interval: 60)
|
9
|
+
@storage = storage
|
10
|
+
@interval = interval
|
11
|
+
end
|
12
|
+
|
13
|
+
def expire_files!
|
14
|
+
return unless expiration_due?
|
15
|
+
update_last_expiration
|
16
|
+
|
17
|
+
Thread.new do
|
18
|
+
thread = Thread.current
|
19
|
+
thread.abort_on_exception = false
|
20
|
+
thread.report_on_exception = true if thread.respond_to?(:report_on_exception) # Ruby 2.4
|
21
|
+
|
22
|
+
_expire_files!
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
def expiration_due?
|
27
|
+
Time.now - interval > last_expiration
|
28
|
+
end
|
29
|
+
|
30
|
+
private
|
31
|
+
|
32
|
+
def _expire_files!
|
33
|
+
storage.list_files.each do |uid|
|
34
|
+
next if uid == "expirator"
|
35
|
+
begin
|
36
|
+
info = Info.new(storage.read_info(uid))
|
37
|
+
storage.delete_file(uid) if Time.now > info.expires
|
38
|
+
rescue
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
def last_expiration
|
44
|
+
info = storage.read_info("expirator")
|
45
|
+
Time.parse(info["Last-Expiration"])
|
46
|
+
rescue
|
47
|
+
Time.new(0)
|
48
|
+
end
|
49
|
+
|
50
|
+
def update_last_expiration
|
51
|
+
if storage.file_exists?("expirator")
|
52
|
+
storage.update_info("expirator", {"Last-Expiration" => Time.now.httpdate})
|
53
|
+
else
|
54
|
+
storage.create_file("expirator", {"Last-Expiration" => Time.now.httpdate})
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
58
|
+
end
|
data/lib/tus/info.rb
ADDED
@@ -0,0 +1,70 @@
|
|
1
|
+
require "base64"
|
2
|
+
require "time"
|
3
|
+
|
4
|
+
module Tus
|
5
|
+
class Info
|
6
|
+
def initialize(hash)
|
7
|
+
@hash = hash
|
8
|
+
end
|
9
|
+
|
10
|
+
def [](key)
|
11
|
+
@hash[key]
|
12
|
+
end
|
13
|
+
|
14
|
+
def []=(key, value)
|
15
|
+
@hash[key] = value
|
16
|
+
end
|
17
|
+
|
18
|
+
def to_h
|
19
|
+
@hash.reject { |key, value| value.nil? }
|
20
|
+
end
|
21
|
+
|
22
|
+
def length
|
23
|
+
Integer(@hash["Upload-Length"])
|
24
|
+
end
|
25
|
+
|
26
|
+
def offset
|
27
|
+
Integer(@hash["Upload-Offset"])
|
28
|
+
end
|
29
|
+
|
30
|
+
def metadata
|
31
|
+
parse_metadata(@hash["Upload-Metadata"])
|
32
|
+
end
|
33
|
+
|
34
|
+
def expires
|
35
|
+
Time.parse(@hash["Upload-Expires"])
|
36
|
+
end
|
37
|
+
|
38
|
+
def final_upload?
|
39
|
+
@hash["Upload-Concat"].to_s.start_with?("final")
|
40
|
+
end
|
41
|
+
|
42
|
+
def defer_length?
|
43
|
+
@hash["Upload-Defer-Length"] == "1"
|
44
|
+
end
|
45
|
+
|
46
|
+
def partial_uploads
|
47
|
+
urls = @hash["Upload-Concat"].split(";").last.split(" ")
|
48
|
+
urls.map { |url| url.split("/").last }
|
49
|
+
end
|
50
|
+
|
51
|
+
def remaining_length
|
52
|
+
length - offset
|
53
|
+
end
|
54
|
+
|
55
|
+
private
|
56
|
+
|
57
|
+
def parse_metadata(string)
|
58
|
+
return {} if string == nil || string == ""
|
59
|
+
|
60
|
+
pairs = string.split(",").map { |s| s.split(" ") }
|
61
|
+
|
62
|
+
hash = Hash[pairs]
|
63
|
+
hash.each do |key, value|
|
64
|
+
hash[key] = Base64.decode64(value)
|
65
|
+
end
|
66
|
+
|
67
|
+
hash
|
68
|
+
end
|
69
|
+
end
|
70
|
+
end
|
data/lib/tus/server.rb
ADDED
@@ -0,0 +1,310 @@
|
|
1
|
+
require "roda"
|
2
|
+
|
3
|
+
require "tus/storage/filesystem"
|
4
|
+
require "tus/info"
|
5
|
+
require "tus/expirator"
|
6
|
+
require "tus/checksum"
|
7
|
+
|
8
|
+
require "securerandom"
|
9
|
+
|
10
|
+
module Tus
|
11
|
+
class Server < Roda
|
12
|
+
SUPPORTED_VERSIONS = ["1.0.0"]
|
13
|
+
SUPPORTED_EXTENSIONS = [
|
14
|
+
"creation", "creation-defer-length",
|
15
|
+
"termination",
|
16
|
+
"expiration",
|
17
|
+
"concatenation", "concatenation-unfinished",
|
18
|
+
"checksum",
|
19
|
+
]
|
20
|
+
SUPPORTED_CHECKSUM_ALGORITHMS = %w[sha1 sha256 sha384 sha512 md5 crc32]
|
21
|
+
RESUMABLE_CONTENT_TYPE = "application/offset+octet-stream"
|
22
|
+
|
23
|
+
opts[:max_size] = 1024*1024*1024
|
24
|
+
opts[:expiration_time] = 7*24*60*60
|
25
|
+
opts[:expiration_interval] = 60*60
|
26
|
+
|
27
|
+
plugin :all_verbs
|
28
|
+
plugin :delete_empty_headers
|
29
|
+
plugin :request_headers
|
30
|
+
plugin :default_headers, "Content-Type" => ""
|
31
|
+
plugin :not_allowed
|
32
|
+
|
33
|
+
route do |r|
|
34
|
+
expire_files
|
35
|
+
|
36
|
+
if request.headers["X-HTTP-Method-Override"]
|
37
|
+
request.env["REQUEST_METHOD"] = request.headers["X-HTTP-Method-Override"]
|
38
|
+
end
|
39
|
+
|
40
|
+
response.headers.update(
|
41
|
+
"Tus-Resumable" => SUPPORTED_VERSIONS.first,
|
42
|
+
)
|
43
|
+
|
44
|
+
handle_cors!
|
45
|
+
validate_tus_resumable! unless request.options? || request.get?
|
46
|
+
|
47
|
+
r.is ['', true] do
|
48
|
+
r.options do
|
49
|
+
response.headers.update(
|
50
|
+
"Tus-Version" => SUPPORTED_VERSIONS.join(","),
|
51
|
+
"Tus-Extension" => SUPPORTED_EXTENSIONS.join(","),
|
52
|
+
"Tus-Max-Size" => max_size.to_s,
|
53
|
+
"Tus-Checksum-Algorithm" => SUPPORTED_CHECKSUM_ALGORITHMS.join(","),
|
54
|
+
)
|
55
|
+
|
56
|
+
no_content!
|
57
|
+
end
|
58
|
+
|
59
|
+
r.post do
|
60
|
+
validate_upload_concat! if request.headers["Upload-Concat"]
|
61
|
+
validate_upload_length! unless request.headers["Upload-Concat"].to_s.start_with?("final") || request.headers["Upload-Defer-Length"] == "1"
|
62
|
+
validate_upload_metadata! if request.headers["Upload-Metadata"]
|
63
|
+
|
64
|
+
uid = SecureRandom.hex
|
65
|
+
info = Info.new(
|
66
|
+
"Upload-Length" => request.headers["Upload-Length"],
|
67
|
+
"Upload-Offset" => "0",
|
68
|
+
"Upload-Defer-Length" => request.headers["Upload-Defer-Length"],
|
69
|
+
"Upload-Metadata" => request.headers["Upload-Metadata"],
|
70
|
+
"Upload-Concat" => request.headers["Upload-Concat"],
|
71
|
+
"Upload-Expires" => (Time.now + expiration_time).httpdate,
|
72
|
+
)
|
73
|
+
|
74
|
+
storage.create_file(uid, info.to_h)
|
75
|
+
|
76
|
+
if info.final_upload?
|
77
|
+
length = info.partial_uploads.inject(0) do |length, partial_uid|
|
78
|
+
content = storage.read_file(partial_uid)
|
79
|
+
storage.patch_file(uid, content)
|
80
|
+
storage.delete_file(partial_uid)
|
81
|
+
length += content.length
|
82
|
+
end
|
83
|
+
|
84
|
+
info["Upload-Length"] = length.to_s
|
85
|
+
info["Upload-Offset"] = length.to_s
|
86
|
+
|
87
|
+
storage.update_info(uid, info.to_h)
|
88
|
+
end
|
89
|
+
|
90
|
+
response.headers.update(info.to_h)
|
91
|
+
|
92
|
+
file_url = "#{request.url.chomp("/")}/#{uid}"
|
93
|
+
created!(file_url)
|
94
|
+
end
|
95
|
+
end
|
96
|
+
|
97
|
+
r.is ":uid" do |uid|
|
98
|
+
not_found! unless storage.file_exists?(uid)
|
99
|
+
|
100
|
+
r.options do
|
101
|
+
response.headers.update(
|
102
|
+
"Tus-Version" => SUPPORTED_VERSIONS.join(","),
|
103
|
+
"Tus-Extension" => SUPPORTED_EXTENSIONS.join(","),
|
104
|
+
"Tus-Max-Size" => max_size.to_s,
|
105
|
+
"Tus-Checksum-Algorithm" => SUPPORTED_CHECKSUM_ALGORITHMS.join(","),
|
106
|
+
)
|
107
|
+
|
108
|
+
no_content!
|
109
|
+
end
|
110
|
+
|
111
|
+
r.get do
|
112
|
+
path = storage.download_file(uid)
|
113
|
+
info = Info.new(storage.read_info(uid))
|
114
|
+
|
115
|
+
file = Rack::File.new(File.dirname(path))
|
116
|
+
|
117
|
+
result = file.serving(request, path)
|
118
|
+
|
119
|
+
response.status = result[0]
|
120
|
+
response.headers.update(result[1])
|
121
|
+
|
122
|
+
metadata = info.metadata
|
123
|
+
response.headers["Content-Disposition"] = "attachment; filename=\"#{metadata["filename"]}\"" if metadata["filename"]
|
124
|
+
response.headers["Content-Type"] = metadata["content_type"] if metadata["content_type"]
|
125
|
+
|
126
|
+
request.halt response.finish_with_body(result[2])
|
127
|
+
end
|
128
|
+
|
129
|
+
r.head do
|
130
|
+
info = storage.read_info(uid)
|
131
|
+
|
132
|
+
response.headers.update(info.to_h)
|
133
|
+
response.headers["Cache-Control"] = "no-store"
|
134
|
+
|
135
|
+
no_content!
|
136
|
+
end
|
137
|
+
|
138
|
+
r.patch do
|
139
|
+
validate_content_type!
|
140
|
+
|
141
|
+
content = request.body.read
|
142
|
+
info = Info.new(storage.read_info(uid))
|
143
|
+
|
144
|
+
if info.defer_length?
|
145
|
+
validate_upload_length!
|
146
|
+
info["Upload-Length"] = request.headers["Upload-Length"]
|
147
|
+
info["Upload-Defer-Length"] = nil
|
148
|
+
end
|
149
|
+
|
150
|
+
validate_upload_checksum!(content) if request.headers["Upload-Checksum"]
|
151
|
+
validate_upload_offset!(info.offset)
|
152
|
+
validate_content_length!(content, info.remaining_length)
|
153
|
+
|
154
|
+
storage.patch_file(uid, content)
|
155
|
+
|
156
|
+
info["Upload-Offset"] = (info.offset + content.length).to_s
|
157
|
+
storage.update_info(uid, info.to_h)
|
158
|
+
|
159
|
+
response.headers.update(info.to_h)
|
160
|
+
|
161
|
+
no_content!
|
162
|
+
end
|
163
|
+
|
164
|
+
r.delete do
|
165
|
+
storage.delete_file(uid)
|
166
|
+
|
167
|
+
no_content!
|
168
|
+
end
|
169
|
+
end
|
170
|
+
end
|
171
|
+
|
172
|
+
def expire_files
|
173
|
+
expirator = Expirator.new(storage, interval: expiration_interval)
|
174
|
+
expirator.expire_files!
|
175
|
+
end
|
176
|
+
|
177
|
+
def validate_content_type!
|
178
|
+
content_type = request.headers["Content-Type"]
|
179
|
+
error!(415, "Invalid Content-Type header") if content_type != RESUMABLE_CONTENT_TYPE
|
180
|
+
end
|
181
|
+
|
182
|
+
def validate_tus_resumable!
|
183
|
+
client_version = request.headers["Tus-Resumable"]
|
184
|
+
|
185
|
+
unless SUPPORTED_VERSIONS.include?(client_version)
|
186
|
+
response.headers["Tus-Version"] = SUPPORTED_VERSIONS.join(",")
|
187
|
+
error!(412, "Unsupported version")
|
188
|
+
end
|
189
|
+
end
|
190
|
+
|
191
|
+
def validate_upload_length!
|
192
|
+
upload_length = request.headers["Upload-Length"]
|
193
|
+
|
194
|
+
error!(400, "Missing Upload-Length header") if upload_length.to_s == ""
|
195
|
+
error!(400, "Invalid Upload-Length header") if upload_length =~ /\D/
|
196
|
+
error!(400, "Invalid Upload-Length header") if upload_length.to_i < 0
|
197
|
+
|
198
|
+
if max_size && upload_length.to_i > max_size
|
199
|
+
error!(413, "Upload-Length header too large")
|
200
|
+
end
|
201
|
+
end
|
202
|
+
|
203
|
+
def validate_upload_offset!(current_offset)
|
204
|
+
upload_offset = request.headers["Upload-Offset"]
|
205
|
+
|
206
|
+
error!(400, "Missing Upload-Offset header") if upload_offset.to_s == ""
|
207
|
+
error!(400, "Invalid Upload-Offset header") if upload_offset =~ /\D/
|
208
|
+
error!(400, "Invalid Upload-Offset header") if upload_offset.to_i < 0
|
209
|
+
|
210
|
+
if upload_offset.to_i != current_offset
|
211
|
+
error!(409, "Upload-Offset header doesn't match current offset")
|
212
|
+
end
|
213
|
+
end
|
214
|
+
|
215
|
+
def validate_content_length!(content, remaining_length)
|
216
|
+
error!(403, "Cannot modify completed upload") if remaining_length == 0
|
217
|
+
error!(413, "Size of this chunk surpasses Upload-Length") if content.length > remaining_length
|
218
|
+
end
|
219
|
+
|
220
|
+
def validate_upload_metadata!
|
221
|
+
upload_metadata = request.headers["Upload-Metadata"]
|
222
|
+
|
223
|
+
upload_metadata.split(",").each do |string|
|
224
|
+
key, value = string.split(" ")
|
225
|
+
|
226
|
+
error!(400, "Invalid Upload-Metadata header") if key.nil? || value.nil?
|
227
|
+
error!(400, "Invalid Upload-Metadata header") if key.ord > 127
|
228
|
+
error!(400, "Invalid Upload-Metadata header") if key =~ /,| /
|
229
|
+
|
230
|
+
error!(400, "Invalid Upload-Metadata header") if value =~ /[^a-zA-Z0-9+\/=]/
|
231
|
+
end
|
232
|
+
end
|
233
|
+
|
234
|
+
def validate_upload_concat!
|
235
|
+
upload_concat = request.headers["Upload-Concat"]
|
236
|
+
|
237
|
+
error!(400, "Invalid Upload-Concat header") if upload_concat !~ /^(partial|final)/
|
238
|
+
|
239
|
+
if upload_concat.start_with?("final")
|
240
|
+
string = upload_concat.split(";").last
|
241
|
+
string.split(" ").each do |url|
|
242
|
+
error!(400, "Invalid Upload-Concat header") if url !~ %r{^#{request.script_name}/\w+$}
|
243
|
+
end
|
244
|
+
end
|
245
|
+
end
|
246
|
+
|
247
|
+
def validate_upload_checksum!(content)
|
248
|
+
algorithm, checksum = request.headers["Upload-Checksum"].split(" ")
|
249
|
+
|
250
|
+
error!(400, "Invalid Upload-Checksum header") if algorithm.nil? || checksum.nil?
|
251
|
+
error!(400, "Invalid Upload-Checksum header") unless SUPPORTED_CHECKSUM_ALGORITHMS.include?(algorithm)
|
252
|
+
|
253
|
+
unless Checksum.new(algorithm).match?(checksum, content)
|
254
|
+
error!(460, "Checksum from Upload-Checksum header doesn't match generated")
|
255
|
+
end
|
256
|
+
end
|
257
|
+
|
258
|
+
def handle_cors!
|
259
|
+
origin = request.headers["Origin"]
|
260
|
+
|
261
|
+
return if origin.to_s == ""
|
262
|
+
|
263
|
+
response.headers["Access-Control-Allow-Origin"] = origin
|
264
|
+
|
265
|
+
if request.options?
|
266
|
+
response.headers["Access-Control-Allow-Methods"] = "POST, GET, HEAD, PATCH, DELETE, OPTIONS"
|
267
|
+
response.headers["Access-Control-Allow-Headers"] = "Origin, X-Requested-With, Content-Type, Upload-Length, Upload-Offset, Tus-Resumable, Upload-Metadata"
|
268
|
+
response.headers["Access-Control-Max-Age"] = "86400"
|
269
|
+
else
|
270
|
+
response.headers["Access-Control-Expose-Headers"] = "Upload-Offset, Location, Upload-Length, Tus-Version, Tus-Resumable, Tus-Max-Size, Tus-Extension, Upload-Metadata"
|
271
|
+
end
|
272
|
+
end
|
273
|
+
|
274
|
+
def no_content!
|
275
|
+
response.status = 204
|
276
|
+
response.headers["Content-Length"] = ""
|
277
|
+
end
|
278
|
+
|
279
|
+
def created!(location)
|
280
|
+
response.status = 201
|
281
|
+
response.headers["Location"] = location
|
282
|
+
end
|
283
|
+
|
284
|
+
def not_found!(message = "Upload not found")
|
285
|
+
error!(404, message)
|
286
|
+
end
|
287
|
+
|
288
|
+
def error!(status, message)
|
289
|
+
response.status = status
|
290
|
+
response.write(message) unless request.head?
|
291
|
+
request.halt
|
292
|
+
end
|
293
|
+
|
294
|
+
def storage
|
295
|
+
opts[:storage] || Tus::Storage::Filesystem.new("data")
|
296
|
+
end
|
297
|
+
|
298
|
+
def max_size
|
299
|
+
opts[:max_size]
|
300
|
+
end
|
301
|
+
|
302
|
+
def expiration_time
|
303
|
+
opts[:expiration_time]
|
304
|
+
end
|
305
|
+
|
306
|
+
def expiration_interval
|
307
|
+
opts[:expiration_interval]
|
308
|
+
end
|
309
|
+
end
|
310
|
+
end
|
@@ -0,0 +1,78 @@
|
|
1
|
+
require "pathname"
|
2
|
+
require "json"
|
3
|
+
|
4
|
+
module Tus
|
5
|
+
module Storage
|
6
|
+
class Filesystem
|
7
|
+
attr_reader :directory
|
8
|
+
|
9
|
+
def initialize(directory)
|
10
|
+
@directory = Pathname(directory)
|
11
|
+
|
12
|
+
create_directory! unless @directory.exist?
|
13
|
+
end
|
14
|
+
|
15
|
+
def create_file(uid, info = {})
|
16
|
+
write(file_path(uid), "")
|
17
|
+
write(info_path(uid), info.to_json)
|
18
|
+
end
|
19
|
+
|
20
|
+
def file_exists?(uid)
|
21
|
+
file_path(uid).exist? && info_path(uid).exist?
|
22
|
+
end
|
23
|
+
|
24
|
+
def read_file(uid)
|
25
|
+
file_path(uid).binread
|
26
|
+
end
|
27
|
+
|
28
|
+
def patch_file(uid, content)
|
29
|
+
write(file_path(uid), content, mode: "ab")
|
30
|
+
end
|
31
|
+
|
32
|
+
def download_file(uid)
|
33
|
+
file_path(uid).to_s
|
34
|
+
end
|
35
|
+
|
36
|
+
def delete_file(uid)
|
37
|
+
file_path(uid).delete
|
38
|
+
info_path(uid).delete
|
39
|
+
end
|
40
|
+
|
41
|
+
def read_info(uid)
|
42
|
+
data = info_path(uid).binread
|
43
|
+
JSON.parse(data)
|
44
|
+
end
|
45
|
+
|
46
|
+
def update_info(uid, info)
|
47
|
+
write(info_path(uid), info.to_json)
|
48
|
+
end
|
49
|
+
|
50
|
+
def list_files
|
51
|
+
paths = Dir[directory.join("*.file")]
|
52
|
+
paths.map { |path| File.basename(path, ".file") }
|
53
|
+
end
|
54
|
+
|
55
|
+
private
|
56
|
+
|
57
|
+
def write(pathname, content, mode: "wb")
|
58
|
+
pathname.open(mode) do |file|
|
59
|
+
file.sync = true
|
60
|
+
file.write(content)
|
61
|
+
end
|
62
|
+
end
|
63
|
+
|
64
|
+
def file_path(uid)
|
65
|
+
directory.join("#{uid}.file")
|
66
|
+
end
|
67
|
+
|
68
|
+
def info_path(uid)
|
69
|
+
directory.join("#{uid}.info")
|
70
|
+
end
|
71
|
+
|
72
|
+
def create_directory!
|
73
|
+
directory.mkpath
|
74
|
+
directory.chmod(0755)
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
@@ -0,0 +1,73 @@
|
|
1
|
+
require "mongo"
|
2
|
+
require "stringio"
|
3
|
+
require "tempfile"
|
4
|
+
|
5
|
+
module Tus
|
6
|
+
module Storage
|
7
|
+
class Gridfs
|
8
|
+
attr_reader :client, :prefix, :bucket
|
9
|
+
|
10
|
+
def initialize(client:, prefix: "fs")
|
11
|
+
@client = client
|
12
|
+
@prefix = prefix
|
13
|
+
@bucket = @client.database.fs(bucket_name: @prefix)
|
14
|
+
@bucket.send(:ensure_indexes!)
|
15
|
+
end
|
16
|
+
|
17
|
+
def create_file(uid, metadata = {})
|
18
|
+
file = Mongo::Grid::File.new("", filename: uid, metadata: metadata)
|
19
|
+
bucket.insert_one(file)
|
20
|
+
end
|
21
|
+
|
22
|
+
def file_exists?(uid)
|
23
|
+
!!bucket.files_collection.find(filename: uid).first
|
24
|
+
end
|
25
|
+
|
26
|
+
def read_file(uid)
|
27
|
+
file = bucket.find_one(filename: uid)
|
28
|
+
file.data
|
29
|
+
end
|
30
|
+
|
31
|
+
def patch_file(uid, content)
|
32
|
+
file_info = bucket.files_collection.find(filename: uid).first
|
33
|
+
file_info["md5"] = Digest::MD5.new # hack around not able to update digest
|
34
|
+
file_info = Mongo::Grid::File::Info.new(file_info)
|
35
|
+
offset = bucket.chunks_collection.find(files_id: file_info.id).count
|
36
|
+
chunks = Mongo::Grid::File::Chunk.split(content, file_info, offset)
|
37
|
+
bucket.chunks_collection.insert_many(chunks)
|
38
|
+
end
|
39
|
+
|
40
|
+
def download_file(uid)
|
41
|
+
tempfile = Tempfile.new("tus", binmode: true)
|
42
|
+
tempfile.sync = true
|
43
|
+
bucket.download_to_stream_by_name(uid, tempfile)
|
44
|
+
tempfile.path
|
45
|
+
end
|
46
|
+
|
47
|
+
def delete_file(uid)
|
48
|
+
file_info = bucket.files_collection.find(filename: uid).first
|
49
|
+
bucket.delete(file_info.fetch("_id"))
|
50
|
+
end
|
51
|
+
|
52
|
+
def read_info(uid)
|
53
|
+
info = bucket.files_collection.find(filename: uid).first
|
54
|
+
info.fetch("metadata")
|
55
|
+
end
|
56
|
+
|
57
|
+
def update_info(uid, info)
|
58
|
+
bucket.files_collection.find(filename: uid).update_one("$set" => {metadata: info})
|
59
|
+
end
|
60
|
+
|
61
|
+
def list_files
|
62
|
+
infos = bucket.files_collection.find.to_a
|
63
|
+
infos.map { |info| info.fetch("filename") }
|
64
|
+
end
|
65
|
+
|
66
|
+
private
|
67
|
+
|
68
|
+
def bson_id(uid)
|
69
|
+
BSON::ObjectId(uid)
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
73
|
+
end
|
data/lib/tus-server.rb
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require "tus/server"
|
data/tus-server.gemspec
ADDED
@@ -0,0 +1,24 @@
|
|
1
|
+
Gem::Specification.new do |gem|
|
2
|
+
gem.name = "tus-server"
|
3
|
+
gem.version = "0.1.0"
|
4
|
+
|
5
|
+
gem.required_ruby_version = ">= 2.1"
|
6
|
+
|
7
|
+
gem.summary = "Ruby server implementation of the Open Protocol for Resumable File Uploads."
|
8
|
+
|
9
|
+
gem.homepage = "https://github.com/janko-m/tus-ruby-server"
|
10
|
+
gem.authors = ["Janko Marohnić"]
|
11
|
+
gem.email = ["janko.marohnic@gmail.com"]
|
12
|
+
gem.license = "MIT"
|
13
|
+
|
14
|
+
gem.files = Dir["README.md", "LICENSE.txt", "lib/**/*.rb", "*.gemspec"]
|
15
|
+
gem.require_path = "lib"
|
16
|
+
|
17
|
+
gem.add_dependency "roda", "~> 2.17"
|
18
|
+
gem.add_dependency "rack", "~> 2.0"
|
19
|
+
|
20
|
+
gem.add_development_dependency "rake", "~> 11.1"
|
21
|
+
gem.add_development_dependency "minitest", "~> 5.8"
|
22
|
+
gem.add_development_dependency "rack-test_app"
|
23
|
+
gem.add_development_dependency "mongo"
|
24
|
+
end
|
metadata
ADDED
@@ -0,0 +1,138 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: tus-server
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Janko Marohnić
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2016-08-22 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: roda
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '2.17'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '2.17'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rack
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '2.0'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '2.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rake
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '11.1'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '11.1'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: minitest
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '5.8'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '5.8'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: rack-test_app
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - ">="
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - ">="
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '0'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: mongo
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - ">="
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '0'
|
90
|
+
type: :development
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - ">="
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: '0'
|
97
|
+
description:
|
98
|
+
email:
|
99
|
+
- janko.marohnic@gmail.com
|
100
|
+
executables: []
|
101
|
+
extensions: []
|
102
|
+
extra_rdoc_files: []
|
103
|
+
files:
|
104
|
+
- LICENSE.txt
|
105
|
+
- README.md
|
106
|
+
- lib/tus-server.rb
|
107
|
+
- lib/tus/checksum.rb
|
108
|
+
- lib/tus/expirator.rb
|
109
|
+
- lib/tus/info.rb
|
110
|
+
- lib/tus/server.rb
|
111
|
+
- lib/tus/storage/filesystem.rb
|
112
|
+
- lib/tus/storage/gridfs.rb
|
113
|
+
- tus-server.gemspec
|
114
|
+
homepage: https://github.com/janko-m/tus-ruby-server
|
115
|
+
licenses:
|
116
|
+
- MIT
|
117
|
+
metadata: {}
|
118
|
+
post_install_message:
|
119
|
+
rdoc_options: []
|
120
|
+
require_paths:
|
121
|
+
- lib
|
122
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
123
|
+
requirements:
|
124
|
+
- - ">="
|
125
|
+
- !ruby/object:Gem::Version
|
126
|
+
version: '2.1'
|
127
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
128
|
+
requirements:
|
129
|
+
- - ">="
|
130
|
+
- !ruby/object:Gem::Version
|
131
|
+
version: '0'
|
132
|
+
requirements: []
|
133
|
+
rubyforge_project:
|
134
|
+
rubygems_version: 2.5.1
|
135
|
+
signing_key:
|
136
|
+
specification_version: 4
|
137
|
+
summary: Ruby server implementation of the Open Protocol for Resumable File Uploads.
|
138
|
+
test_files: []
|