tus-server 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +164 -0
- data/lib/tus/checksum.rb +48 -0
- data/lib/tus/expirator.rb +58 -0
- data/lib/tus/info.rb +70 -0
- data/lib/tus/server.rb +310 -0
- data/lib/tus/storage/filesystem.rb +78 -0
- data/lib/tus/storage/gridfs.rb +73 -0
- data/lib/tus-server.rb +1 -0
- data/tus-server.gemspec +24 -0
- metadata +138 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: ac1db8e0078cd0824caf54dc3c1993957ca90902
|
4
|
+
data.tar.gz: 032f12c4aaa676e7f55388f8af96d7a46cad42a7
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 45aa99b75d86eb9197fa3edcb8fd4a140004e9e48bb955327e5e77dd7ac3fa3b407a5782f4bcf12fb2a7277704f4ea4cfd6a2972a35d3d69b5e3c778708dcb0f
|
7
|
+
data.tar.gz: b27487dc797127e1c51010e65e1ad44e22206bfcc8ff5dbecc4136eedf4c2197c6ae13b1d32bada612d614844f61e9780349748050559bb47c2ea386306bf31d
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2015-2016 Janko Marohnić
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,164 @@
|
|
1
|
+
# tus-ruby-server
|
2
|
+
|
3
|
+
A Ruby server for the [tus resumable upload protocol]. It implements the core
|
4
|
+
1.0 protocol, along with the following extensions:
|
5
|
+
|
6
|
+
* [`creation`][creation] (and `creation-defer-length`)
|
7
|
+
* [`concatenation`][concatenation] (and `concatenation-unfinished`)
|
8
|
+
* [`checksum`][checksum]
|
9
|
+
* [`expiration`][expiration]
|
10
|
+
* [`termination`][termination]
|
11
|
+
|
12
|
+
## Installation
|
13
|
+
|
14
|
+
```rb
|
15
|
+
gem "tus-server"
|
16
|
+
```
|
17
|
+
|
18
|
+
## Usage
|
19
|
+
|
20
|
+
You can run the `Tus::Server` in your `config.ru`:
|
21
|
+
|
22
|
+
```rb
|
23
|
+
# config.ru
|
24
|
+
require "tus/server"
|
25
|
+
|
26
|
+
map "/files" do
|
27
|
+
run Tus::Server
|
28
|
+
end
|
29
|
+
|
30
|
+
run YourApp
|
31
|
+
```
|
32
|
+
|
33
|
+
Now you can tell your tus client library (e.g. [tus-js-client]) to use this
|
34
|
+
endpoint:
|
35
|
+
|
36
|
+
```js
|
37
|
+
// using tus-js-client
|
38
|
+
new tus.Upload(file, {
|
39
|
+
endpoint: "http://localhost:9292/files",
|
40
|
+
chunkSize: 15*1024*1024, # 15 MB
|
41
|
+
// ...
|
42
|
+
})
|
43
|
+
```
|
44
|
+
|
45
|
+
After upload is complete, you'll probably want to attach the uploaded files to
|
46
|
+
database records. [Shrine] is one file attachment library that supports this,
|
47
|
+
see [shrine-tus-demo] on how you can integrate the two.
|
48
|
+
|
49
|
+
### Metadata
|
50
|
+
|
51
|
+
As per tus protocol, you can assign custom metadata when creating a file using
|
52
|
+
the `Upload-Metadata` header. When retrieving the file via a GET request,
|
53
|
+
tus-server will use
|
54
|
+
|
55
|
+
* `content_type` -- for setting the `Content-Type` header
|
56
|
+
* `filename` -- for setting the `Content-Disposition` header
|
57
|
+
|
58
|
+
Both of these are optional, and will be used if available.
|
59
|
+
|
60
|
+
### Storage
|
61
|
+
|
62
|
+
By default `Tus::Server` saves partial and complete files on the filesystem,
|
63
|
+
inside the `data/` directory. You can easily change the directory:
|
64
|
+
|
65
|
+
```rb
|
66
|
+
require "tus/server"
|
67
|
+
|
68
|
+
Tus::Server.opts[:storage] = Tus::Storage::Filesystem.new("public/cache")
|
69
|
+
```
|
70
|
+
|
71
|
+
The downside of storing files on the filesystem is that it isn't distributed,
|
72
|
+
so for resumable uploads to work you have to host the tus application on a
|
73
|
+
single server.
|
74
|
+
|
75
|
+
However, tus-server also ships with MongoDB [GridFS] storage, which among other
|
76
|
+
things is convenient for a multi-server setup. It requires the [Mongo] gem:
|
77
|
+
|
78
|
+
```rb
|
79
|
+
gem "mongo"
|
80
|
+
```
|
81
|
+
|
82
|
+
```rb
|
83
|
+
require "tus/server"
|
84
|
+
require "tus/storage/gridfs"
|
85
|
+
|
86
|
+
client = Mongo::Client.new("mongodb://127.0.0.1:27017/mydb")
|
87
|
+
Tus::Server.opts[:storage] = Tus::Storage::Gridfs.new(client: client)
|
88
|
+
```
|
89
|
+
|
90
|
+
You can also write your own storage, you just need to implement the same
|
91
|
+
public interface that `Tus::Storage::Filesystem` and `Tus::Storage::Gridfs` do.
|
92
|
+
|
93
|
+
### Maximum size
|
94
|
+
|
95
|
+
By default the maximum size for an uploaded file is 1GB, but you can change
|
96
|
+
that:
|
97
|
+
|
98
|
+
```rb
|
99
|
+
require "tus/server"
|
100
|
+
|
101
|
+
Tus::Server.opts[:max_size] = 5 * 1024*1024*1024 # 5GB
|
102
|
+
# or
|
103
|
+
Tus::Server.opts[:max_size] = nil # no limit
|
104
|
+
```
|
105
|
+
|
106
|
+
### Expiration
|
107
|
+
|
108
|
+
By default both partially and fully uploaded files will get deleted after one
|
109
|
+
week, and the interval of checking for expired files is 1 hour. You can change
|
110
|
+
both of these:
|
111
|
+
|
112
|
+
```rb
|
113
|
+
require "tus/server"
|
114
|
+
|
115
|
+
Tus::Server.opts[:expiration_time] = 14*24*60*60 # 2 weeks
|
116
|
+
Tus::Server.opts[:expiration_interval] = 24*60*60 # 1 day
|
117
|
+
```
|
118
|
+
|
119
|
+
### Checksum
|
120
|
+
|
121
|
+
The following checksum algorithms are supported for the `checksum` extension:
|
122
|
+
|
123
|
+
* SHA1
|
124
|
+
* SHA256
|
125
|
+
* SHA384
|
126
|
+
* SHA512
|
127
|
+
* MD5
|
128
|
+
* CRC32
|
129
|
+
|
130
|
+
## Limitations
|
131
|
+
|
132
|
+
Since tus-server is built using a Rack-based web framework (Roda), if a PATCH
|
133
|
+
request gets interrupted, none of the received data will be stored. It's
|
134
|
+
recommended to configure your client tus library not to use a single PATCH
|
135
|
+
request for large files, but rather to split it into multiple chunks. You can
|
136
|
+
do that for [tus-js-client] by specifying a maximum chunk size:
|
137
|
+
|
138
|
+
```js
|
139
|
+
new tus.Upload(file, {
|
140
|
+
endpoint: "http://localhost:9292/files",
|
141
|
+
chunkSize: 15*1024*1024, # 15 MB
|
142
|
+
// ...
|
143
|
+
})
|
144
|
+
```
|
145
|
+
|
146
|
+
Tus-server also currently doesn't support the `checksum-trailer` extension,
|
147
|
+
which would allow sending the checksum header *after* the data has been sent,
|
148
|
+
using [trailing headers].
|
149
|
+
|
150
|
+
## License
|
151
|
+
|
152
|
+
[MIT](/LICENSE.txt)
|
153
|
+
|
154
|
+
[tus resumable upload protocol]: http://tus.io/
|
155
|
+
[tus-js-client]: https://github.com/tus/tus-js-client
|
156
|
+
[creation]: http://tus.io/protocols/resumable-upload.html#creation
|
157
|
+
[concatenation]: http://tus.io/protocols/resumable-upload.html#concatenation
|
158
|
+
[checksum]: http://tus.io/protocols/resumable-upload.html#checksum
|
159
|
+
[expiration]: http://tus.io/protocols/resumable-upload.html#expiration
|
160
|
+
[termination]: http://tus.io/protocols/resumable-upload.html#termination
|
161
|
+
[GridFS]: https://docs.mongodb.org/v3.0/core/gridfs/
|
162
|
+
[shrine-tus-demo]: https://github.com/janko-m/shrine-tus-demo
|
163
|
+
[Shrine]: https://github.com/janko-m/shrine
|
164
|
+
[trailing headers]: https://tools.ietf.org/html/rfc7230#section-4.1.2
|
data/lib/tus/checksum.rb
ADDED
@@ -0,0 +1,48 @@
|
|
1
|
+
require "base64"
|
2
|
+
require "digest"
|
3
|
+
require "zlib"
|
4
|
+
|
5
|
+
module Tus
|
6
|
+
class Checksum
|
7
|
+
attr_reader :algorithm
|
8
|
+
|
9
|
+
def initialize(algorithm)
|
10
|
+
@algorithm = algorithm
|
11
|
+
end
|
12
|
+
|
13
|
+
def match?(checksum, content)
|
14
|
+
checksum = Base64.decode64(checksum)
|
15
|
+
generate(content) == checksum
|
16
|
+
end
|
17
|
+
|
18
|
+
def generate(content)
|
19
|
+
send("generate_#{algorithm}", content)
|
20
|
+
end
|
21
|
+
|
22
|
+
private
|
23
|
+
|
24
|
+
def generate_sha1(content)
|
25
|
+
Digest::SHA1.hexdigest(content)
|
26
|
+
end
|
27
|
+
|
28
|
+
def generate_sha256(content)
|
29
|
+
Digest::SHA256.hexdigest(content)
|
30
|
+
end
|
31
|
+
|
32
|
+
def generate_sha384(content)
|
33
|
+
Digest::SHA384.hexdigest(content)
|
34
|
+
end
|
35
|
+
|
36
|
+
def generate_sha512(content)
|
37
|
+
Digest::SHA512.hexdigest(content)
|
38
|
+
end
|
39
|
+
|
40
|
+
def generate_md5(content)
|
41
|
+
Digest::MD5.hexdigest(content)
|
42
|
+
end
|
43
|
+
|
44
|
+
def generate_crc32(content)
|
45
|
+
Zlib.crc32(content).to_s
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -0,0 +1,58 @@
|
|
1
|
+
require "tus/info"
|
2
|
+
require "time"
|
3
|
+
|
4
|
+
module Tus
|
5
|
+
class Expirator
|
6
|
+
attr_reader :storage, :interval
|
7
|
+
|
8
|
+
def initialize(storage, interval: 60)
|
9
|
+
@storage = storage
|
10
|
+
@interval = interval
|
11
|
+
end
|
12
|
+
|
13
|
+
def expire_files!
|
14
|
+
return unless expiration_due?
|
15
|
+
update_last_expiration
|
16
|
+
|
17
|
+
Thread.new do
|
18
|
+
thread = Thread.current
|
19
|
+
thread.abort_on_exception = false
|
20
|
+
thread.report_on_exception = true if thread.respond_to?(:report_on_exception) # Ruby 2.4
|
21
|
+
|
22
|
+
_expire_files!
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
def expiration_due?
|
27
|
+
Time.now - interval > last_expiration
|
28
|
+
end
|
29
|
+
|
30
|
+
private
|
31
|
+
|
32
|
+
def _expire_files!
|
33
|
+
storage.list_files.each do |uid|
|
34
|
+
next if uid == "expirator"
|
35
|
+
begin
|
36
|
+
info = Info.new(storage.read_info(uid))
|
37
|
+
storage.delete_file(uid) if Time.now > info.expires
|
38
|
+
rescue
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
def last_expiration
|
44
|
+
info = storage.read_info("expirator")
|
45
|
+
Time.parse(info["Last-Expiration"])
|
46
|
+
rescue
|
47
|
+
Time.new(0)
|
48
|
+
end
|
49
|
+
|
50
|
+
def update_last_expiration
|
51
|
+
if storage.file_exists?("expirator")
|
52
|
+
storage.update_info("expirator", {"Last-Expiration" => Time.now.httpdate})
|
53
|
+
else
|
54
|
+
storage.create_file("expirator", {"Last-Expiration" => Time.now.httpdate})
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
58
|
+
end
|
data/lib/tus/info.rb
ADDED
@@ -0,0 +1,70 @@
|
|
1
|
+
require "base64"
|
2
|
+
require "time"
|
3
|
+
|
4
|
+
module Tus
|
5
|
+
class Info
|
6
|
+
def initialize(hash)
|
7
|
+
@hash = hash
|
8
|
+
end
|
9
|
+
|
10
|
+
def [](key)
|
11
|
+
@hash[key]
|
12
|
+
end
|
13
|
+
|
14
|
+
def []=(key, value)
|
15
|
+
@hash[key] = value
|
16
|
+
end
|
17
|
+
|
18
|
+
def to_h
|
19
|
+
@hash.reject { |key, value| value.nil? }
|
20
|
+
end
|
21
|
+
|
22
|
+
def length
|
23
|
+
Integer(@hash["Upload-Length"])
|
24
|
+
end
|
25
|
+
|
26
|
+
def offset
|
27
|
+
Integer(@hash["Upload-Offset"])
|
28
|
+
end
|
29
|
+
|
30
|
+
def metadata
|
31
|
+
parse_metadata(@hash["Upload-Metadata"])
|
32
|
+
end
|
33
|
+
|
34
|
+
def expires
|
35
|
+
Time.parse(@hash["Upload-Expires"])
|
36
|
+
end
|
37
|
+
|
38
|
+
def final_upload?
|
39
|
+
@hash["Upload-Concat"].to_s.start_with?("final")
|
40
|
+
end
|
41
|
+
|
42
|
+
def defer_length?
|
43
|
+
@hash["Upload-Defer-Length"] == "1"
|
44
|
+
end
|
45
|
+
|
46
|
+
def partial_uploads
|
47
|
+
urls = @hash["Upload-Concat"].split(";").last.split(" ")
|
48
|
+
urls.map { |url| url.split("/").last }
|
49
|
+
end
|
50
|
+
|
51
|
+
def remaining_length
|
52
|
+
length - offset
|
53
|
+
end
|
54
|
+
|
55
|
+
private
|
56
|
+
|
57
|
+
def parse_metadata(string)
|
58
|
+
return {} if string == nil || string == ""
|
59
|
+
|
60
|
+
pairs = string.split(",").map { |s| s.split(" ") }
|
61
|
+
|
62
|
+
hash = Hash[pairs]
|
63
|
+
hash.each do |key, value|
|
64
|
+
hash[key] = Base64.decode64(value)
|
65
|
+
end
|
66
|
+
|
67
|
+
hash
|
68
|
+
end
|
69
|
+
end
|
70
|
+
end
|
data/lib/tus/server.rb
ADDED
@@ -0,0 +1,310 @@
|
|
1
|
+
require "roda"
|
2
|
+
|
3
|
+
require "tus/storage/filesystem"
|
4
|
+
require "tus/info"
|
5
|
+
require "tus/expirator"
|
6
|
+
require "tus/checksum"
|
7
|
+
|
8
|
+
require "securerandom"
|
9
|
+
|
10
|
+
module Tus
|
11
|
+
class Server < Roda
|
12
|
+
SUPPORTED_VERSIONS = ["1.0.0"]
|
13
|
+
SUPPORTED_EXTENSIONS = [
|
14
|
+
"creation", "creation-defer-length",
|
15
|
+
"termination",
|
16
|
+
"expiration",
|
17
|
+
"concatenation", "concatenation-unfinished",
|
18
|
+
"checksum",
|
19
|
+
]
|
20
|
+
SUPPORTED_CHECKSUM_ALGORITHMS = %w[sha1 sha256 sha384 sha512 md5 crc32]
|
21
|
+
RESUMABLE_CONTENT_TYPE = "application/offset+octet-stream"
|
22
|
+
|
23
|
+
opts[:max_size] = 1024*1024*1024
|
24
|
+
opts[:expiration_time] = 7*24*60*60
|
25
|
+
opts[:expiration_interval] = 60*60
|
26
|
+
|
27
|
+
plugin :all_verbs
|
28
|
+
plugin :delete_empty_headers
|
29
|
+
plugin :request_headers
|
30
|
+
plugin :default_headers, "Content-Type" => ""
|
31
|
+
plugin :not_allowed
|
32
|
+
|
33
|
+
route do |r|
|
34
|
+
expire_files
|
35
|
+
|
36
|
+
if request.headers["X-HTTP-Method-Override"]
|
37
|
+
request.env["REQUEST_METHOD"] = request.headers["X-HTTP-Method-Override"]
|
38
|
+
end
|
39
|
+
|
40
|
+
response.headers.update(
|
41
|
+
"Tus-Resumable" => SUPPORTED_VERSIONS.first,
|
42
|
+
)
|
43
|
+
|
44
|
+
handle_cors!
|
45
|
+
validate_tus_resumable! unless request.options? || request.get?
|
46
|
+
|
47
|
+
r.is ['', true] do
|
48
|
+
r.options do
|
49
|
+
response.headers.update(
|
50
|
+
"Tus-Version" => SUPPORTED_VERSIONS.join(","),
|
51
|
+
"Tus-Extension" => SUPPORTED_EXTENSIONS.join(","),
|
52
|
+
"Tus-Max-Size" => max_size.to_s,
|
53
|
+
"Tus-Checksum-Algorithm" => SUPPORTED_CHECKSUM_ALGORITHMS.join(","),
|
54
|
+
)
|
55
|
+
|
56
|
+
no_content!
|
57
|
+
end
|
58
|
+
|
59
|
+
r.post do
|
60
|
+
validate_upload_concat! if request.headers["Upload-Concat"]
|
61
|
+
validate_upload_length! unless request.headers["Upload-Concat"].to_s.start_with?("final") || request.headers["Upload-Defer-Length"] == "1"
|
62
|
+
validate_upload_metadata! if request.headers["Upload-Metadata"]
|
63
|
+
|
64
|
+
uid = SecureRandom.hex
|
65
|
+
info = Info.new(
|
66
|
+
"Upload-Length" => request.headers["Upload-Length"],
|
67
|
+
"Upload-Offset" => "0",
|
68
|
+
"Upload-Defer-Length" => request.headers["Upload-Defer-Length"],
|
69
|
+
"Upload-Metadata" => request.headers["Upload-Metadata"],
|
70
|
+
"Upload-Concat" => request.headers["Upload-Concat"],
|
71
|
+
"Upload-Expires" => (Time.now + expiration_time).httpdate,
|
72
|
+
)
|
73
|
+
|
74
|
+
storage.create_file(uid, info.to_h)
|
75
|
+
|
76
|
+
if info.final_upload?
|
77
|
+
length = info.partial_uploads.inject(0) do |length, partial_uid|
|
78
|
+
content = storage.read_file(partial_uid)
|
79
|
+
storage.patch_file(uid, content)
|
80
|
+
storage.delete_file(partial_uid)
|
81
|
+
length += content.length
|
82
|
+
end
|
83
|
+
|
84
|
+
info["Upload-Length"] = length.to_s
|
85
|
+
info["Upload-Offset"] = length.to_s
|
86
|
+
|
87
|
+
storage.update_info(uid, info.to_h)
|
88
|
+
end
|
89
|
+
|
90
|
+
response.headers.update(info.to_h)
|
91
|
+
|
92
|
+
file_url = "#{request.url.chomp("/")}/#{uid}"
|
93
|
+
created!(file_url)
|
94
|
+
end
|
95
|
+
end
|
96
|
+
|
97
|
+
r.is ":uid" do |uid|
|
98
|
+
not_found! unless storage.file_exists?(uid)
|
99
|
+
|
100
|
+
r.options do
|
101
|
+
response.headers.update(
|
102
|
+
"Tus-Version" => SUPPORTED_VERSIONS.join(","),
|
103
|
+
"Tus-Extension" => SUPPORTED_EXTENSIONS.join(","),
|
104
|
+
"Tus-Max-Size" => max_size.to_s,
|
105
|
+
"Tus-Checksum-Algorithm" => SUPPORTED_CHECKSUM_ALGORITHMS.join(","),
|
106
|
+
)
|
107
|
+
|
108
|
+
no_content!
|
109
|
+
end
|
110
|
+
|
111
|
+
r.get do
|
112
|
+
path = storage.download_file(uid)
|
113
|
+
info = Info.new(storage.read_info(uid))
|
114
|
+
|
115
|
+
file = Rack::File.new(File.dirname(path))
|
116
|
+
|
117
|
+
result = file.serving(request, path)
|
118
|
+
|
119
|
+
response.status = result[0]
|
120
|
+
response.headers.update(result[1])
|
121
|
+
|
122
|
+
metadata = info.metadata
|
123
|
+
response.headers["Content-Disposition"] = "attachment; filename=\"#{metadata["filename"]}\"" if metadata["filename"]
|
124
|
+
response.headers["Content-Type"] = metadata["content_type"] if metadata["content_type"]
|
125
|
+
|
126
|
+
request.halt response.finish_with_body(result[2])
|
127
|
+
end
|
128
|
+
|
129
|
+
r.head do
|
130
|
+
info = storage.read_info(uid)
|
131
|
+
|
132
|
+
response.headers.update(info.to_h)
|
133
|
+
response.headers["Cache-Control"] = "no-store"
|
134
|
+
|
135
|
+
no_content!
|
136
|
+
end
|
137
|
+
|
138
|
+
r.patch do
|
139
|
+
validate_content_type!
|
140
|
+
|
141
|
+
content = request.body.read
|
142
|
+
info = Info.new(storage.read_info(uid))
|
143
|
+
|
144
|
+
if info.defer_length?
|
145
|
+
validate_upload_length!
|
146
|
+
info["Upload-Length"] = request.headers["Upload-Length"]
|
147
|
+
info["Upload-Defer-Length"] = nil
|
148
|
+
end
|
149
|
+
|
150
|
+
validate_upload_checksum!(content) if request.headers["Upload-Checksum"]
|
151
|
+
validate_upload_offset!(info.offset)
|
152
|
+
validate_content_length!(content, info.remaining_length)
|
153
|
+
|
154
|
+
storage.patch_file(uid, content)
|
155
|
+
|
156
|
+
info["Upload-Offset"] = (info.offset + content.length).to_s
|
157
|
+
storage.update_info(uid, info.to_h)
|
158
|
+
|
159
|
+
response.headers.update(info.to_h)
|
160
|
+
|
161
|
+
no_content!
|
162
|
+
end
|
163
|
+
|
164
|
+
r.delete do
|
165
|
+
storage.delete_file(uid)
|
166
|
+
|
167
|
+
no_content!
|
168
|
+
end
|
169
|
+
end
|
170
|
+
end
|
171
|
+
|
172
|
+
def expire_files
|
173
|
+
expirator = Expirator.new(storage, interval: expiration_interval)
|
174
|
+
expirator.expire_files!
|
175
|
+
end
|
176
|
+
|
177
|
+
def validate_content_type!
|
178
|
+
content_type = request.headers["Content-Type"]
|
179
|
+
error!(415, "Invalid Content-Type header") if content_type != RESUMABLE_CONTENT_TYPE
|
180
|
+
end
|
181
|
+
|
182
|
+
def validate_tus_resumable!
|
183
|
+
client_version = request.headers["Tus-Resumable"]
|
184
|
+
|
185
|
+
unless SUPPORTED_VERSIONS.include?(client_version)
|
186
|
+
response.headers["Tus-Version"] = SUPPORTED_VERSIONS.join(",")
|
187
|
+
error!(412, "Unsupported version")
|
188
|
+
end
|
189
|
+
end
|
190
|
+
|
191
|
+
def validate_upload_length!
|
192
|
+
upload_length = request.headers["Upload-Length"]
|
193
|
+
|
194
|
+
error!(400, "Missing Upload-Length header") if upload_length.to_s == ""
|
195
|
+
error!(400, "Invalid Upload-Length header") if upload_length =~ /\D/
|
196
|
+
error!(400, "Invalid Upload-Length header") if upload_length.to_i < 0
|
197
|
+
|
198
|
+
if max_size && upload_length.to_i > max_size
|
199
|
+
error!(413, "Upload-Length header too large")
|
200
|
+
end
|
201
|
+
end
|
202
|
+
|
203
|
+
def validate_upload_offset!(current_offset)
|
204
|
+
upload_offset = request.headers["Upload-Offset"]
|
205
|
+
|
206
|
+
error!(400, "Missing Upload-Offset header") if upload_offset.to_s == ""
|
207
|
+
error!(400, "Invalid Upload-Offset header") if upload_offset =~ /\D/
|
208
|
+
error!(400, "Invalid Upload-Offset header") if upload_offset.to_i < 0
|
209
|
+
|
210
|
+
if upload_offset.to_i != current_offset
|
211
|
+
error!(409, "Upload-Offset header doesn't match current offset")
|
212
|
+
end
|
213
|
+
end
|
214
|
+
|
215
|
+
def validate_content_length!(content, remaining_length)
|
216
|
+
error!(403, "Cannot modify completed upload") if remaining_length == 0
|
217
|
+
error!(413, "Size of this chunk surpasses Upload-Length") if content.length > remaining_length
|
218
|
+
end
|
219
|
+
|
220
|
+
def validate_upload_metadata!
|
221
|
+
upload_metadata = request.headers["Upload-Metadata"]
|
222
|
+
|
223
|
+
upload_metadata.split(",").each do |string|
|
224
|
+
key, value = string.split(" ")
|
225
|
+
|
226
|
+
error!(400, "Invalid Upload-Metadata header") if key.nil? || value.nil?
|
227
|
+
error!(400, "Invalid Upload-Metadata header") if key.ord > 127
|
228
|
+
error!(400, "Invalid Upload-Metadata header") if key =~ /,| /
|
229
|
+
|
230
|
+
error!(400, "Invalid Upload-Metadata header") if value =~ /[^a-zA-Z0-9+\/=]/
|
231
|
+
end
|
232
|
+
end
|
233
|
+
|
234
|
+
def validate_upload_concat!
|
235
|
+
upload_concat = request.headers["Upload-Concat"]
|
236
|
+
|
237
|
+
error!(400, "Invalid Upload-Concat header") if upload_concat !~ /^(partial|final)/
|
238
|
+
|
239
|
+
if upload_concat.start_with?("final")
|
240
|
+
string = upload_concat.split(";").last
|
241
|
+
string.split(" ").each do |url|
|
242
|
+
error!(400, "Invalid Upload-Concat header") if url !~ %r{^#{request.script_name}/\w+$}
|
243
|
+
end
|
244
|
+
end
|
245
|
+
end
|
246
|
+
|
247
|
+
def validate_upload_checksum!(content)
|
248
|
+
algorithm, checksum = request.headers["Upload-Checksum"].split(" ")
|
249
|
+
|
250
|
+
error!(400, "Invalid Upload-Checksum header") if algorithm.nil? || checksum.nil?
|
251
|
+
error!(400, "Invalid Upload-Checksum header") unless SUPPORTED_CHECKSUM_ALGORITHMS.include?(algorithm)
|
252
|
+
|
253
|
+
unless Checksum.new(algorithm).match?(checksum, content)
|
254
|
+
error!(460, "Checksum from Upload-Checksum header doesn't match generated")
|
255
|
+
end
|
256
|
+
end
|
257
|
+
|
258
|
+
def handle_cors!
|
259
|
+
origin = request.headers["Origin"]
|
260
|
+
|
261
|
+
return if origin.to_s == ""
|
262
|
+
|
263
|
+
response.headers["Access-Control-Allow-Origin"] = origin
|
264
|
+
|
265
|
+
if request.options?
|
266
|
+
response.headers["Access-Control-Allow-Methods"] = "POST, GET, HEAD, PATCH, DELETE, OPTIONS"
|
267
|
+
response.headers["Access-Control-Allow-Headers"] = "Origin, X-Requested-With, Content-Type, Upload-Length, Upload-Offset, Tus-Resumable, Upload-Metadata"
|
268
|
+
response.headers["Access-Control-Max-Age"] = "86400"
|
269
|
+
else
|
270
|
+
response.headers["Access-Control-Expose-Headers"] = "Upload-Offset, Location, Upload-Length, Tus-Version, Tus-Resumable, Tus-Max-Size, Tus-Extension, Upload-Metadata"
|
271
|
+
end
|
272
|
+
end
|
273
|
+
|
274
|
+
def no_content!
|
275
|
+
response.status = 204
|
276
|
+
response.headers["Content-Length"] = ""
|
277
|
+
end
|
278
|
+
|
279
|
+
def created!(location)
|
280
|
+
response.status = 201
|
281
|
+
response.headers["Location"] = location
|
282
|
+
end
|
283
|
+
|
284
|
+
def not_found!(message = "Upload not found")
|
285
|
+
error!(404, message)
|
286
|
+
end
|
287
|
+
|
288
|
+
def error!(status, message)
|
289
|
+
response.status = status
|
290
|
+
response.write(message) unless request.head?
|
291
|
+
request.halt
|
292
|
+
end
|
293
|
+
|
294
|
+
def storage
|
295
|
+
opts[:storage] || Tus::Storage::Filesystem.new("data")
|
296
|
+
end
|
297
|
+
|
298
|
+
def max_size
|
299
|
+
opts[:max_size]
|
300
|
+
end
|
301
|
+
|
302
|
+
def expiration_time
|
303
|
+
opts[:expiration_time]
|
304
|
+
end
|
305
|
+
|
306
|
+
def expiration_interval
|
307
|
+
opts[:expiration_interval]
|
308
|
+
end
|
309
|
+
end
|
310
|
+
end
|
@@ -0,0 +1,78 @@
|
|
1
|
+
require "pathname"
|
2
|
+
require "json"
|
3
|
+
|
4
|
+
module Tus
|
5
|
+
module Storage
|
6
|
+
class Filesystem
|
7
|
+
attr_reader :directory
|
8
|
+
|
9
|
+
def initialize(directory)
|
10
|
+
@directory = Pathname(directory)
|
11
|
+
|
12
|
+
create_directory! unless @directory.exist?
|
13
|
+
end
|
14
|
+
|
15
|
+
def create_file(uid, info = {})
|
16
|
+
write(file_path(uid), "")
|
17
|
+
write(info_path(uid), info.to_json)
|
18
|
+
end
|
19
|
+
|
20
|
+
def file_exists?(uid)
|
21
|
+
file_path(uid).exist? && info_path(uid).exist?
|
22
|
+
end
|
23
|
+
|
24
|
+
def read_file(uid)
|
25
|
+
file_path(uid).binread
|
26
|
+
end
|
27
|
+
|
28
|
+
def patch_file(uid, content)
|
29
|
+
write(file_path(uid), content, mode: "ab")
|
30
|
+
end
|
31
|
+
|
32
|
+
def download_file(uid)
|
33
|
+
file_path(uid).to_s
|
34
|
+
end
|
35
|
+
|
36
|
+
def delete_file(uid)
|
37
|
+
file_path(uid).delete
|
38
|
+
info_path(uid).delete
|
39
|
+
end
|
40
|
+
|
41
|
+
def read_info(uid)
|
42
|
+
data = info_path(uid).binread
|
43
|
+
JSON.parse(data)
|
44
|
+
end
|
45
|
+
|
46
|
+
def update_info(uid, info)
|
47
|
+
write(info_path(uid), info.to_json)
|
48
|
+
end
|
49
|
+
|
50
|
+
def list_files
|
51
|
+
paths = Dir[directory.join("*.file")]
|
52
|
+
paths.map { |path| File.basename(path, ".file") }
|
53
|
+
end
|
54
|
+
|
55
|
+
private
|
56
|
+
|
57
|
+
def write(pathname, content, mode: "wb")
|
58
|
+
pathname.open(mode) do |file|
|
59
|
+
file.sync = true
|
60
|
+
file.write(content)
|
61
|
+
end
|
62
|
+
end
|
63
|
+
|
64
|
+
def file_path(uid)
|
65
|
+
directory.join("#{uid}.file")
|
66
|
+
end
|
67
|
+
|
68
|
+
def info_path(uid)
|
69
|
+
directory.join("#{uid}.info")
|
70
|
+
end
|
71
|
+
|
72
|
+
def create_directory!
|
73
|
+
directory.mkpath
|
74
|
+
directory.chmod(0755)
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
@@ -0,0 +1,73 @@
|
|
1
|
+
require "mongo"
|
2
|
+
require "stringio"
|
3
|
+
require "tempfile"
|
4
|
+
|
5
|
+
module Tus
|
6
|
+
module Storage
|
7
|
+
class Gridfs
|
8
|
+
attr_reader :client, :prefix, :bucket
|
9
|
+
|
10
|
+
def initialize(client:, prefix: "fs")
|
11
|
+
@client = client
|
12
|
+
@prefix = prefix
|
13
|
+
@bucket = @client.database.fs(bucket_name: @prefix)
|
14
|
+
@bucket.send(:ensure_indexes!)
|
15
|
+
end
|
16
|
+
|
17
|
+
def create_file(uid, metadata = {})
|
18
|
+
file = Mongo::Grid::File.new("", filename: uid, metadata: metadata)
|
19
|
+
bucket.insert_one(file)
|
20
|
+
end
|
21
|
+
|
22
|
+
def file_exists?(uid)
|
23
|
+
!!bucket.files_collection.find(filename: uid).first
|
24
|
+
end
|
25
|
+
|
26
|
+
def read_file(uid)
|
27
|
+
file = bucket.find_one(filename: uid)
|
28
|
+
file.data
|
29
|
+
end
|
30
|
+
|
31
|
+
def patch_file(uid, content)
|
32
|
+
file_info = bucket.files_collection.find(filename: uid).first
|
33
|
+
file_info["md5"] = Digest::MD5.new # hack around not able to update digest
|
34
|
+
file_info = Mongo::Grid::File::Info.new(file_info)
|
35
|
+
offset = bucket.chunks_collection.find(files_id: file_info.id).count
|
36
|
+
chunks = Mongo::Grid::File::Chunk.split(content, file_info, offset)
|
37
|
+
bucket.chunks_collection.insert_many(chunks)
|
38
|
+
end
|
39
|
+
|
40
|
+
def download_file(uid)
|
41
|
+
tempfile = Tempfile.new("tus", binmode: true)
|
42
|
+
tempfile.sync = true
|
43
|
+
bucket.download_to_stream_by_name(uid, tempfile)
|
44
|
+
tempfile.path
|
45
|
+
end
|
46
|
+
|
47
|
+
def delete_file(uid)
|
48
|
+
file_info = bucket.files_collection.find(filename: uid).first
|
49
|
+
bucket.delete(file_info.fetch("_id"))
|
50
|
+
end
|
51
|
+
|
52
|
+
def read_info(uid)
|
53
|
+
info = bucket.files_collection.find(filename: uid).first
|
54
|
+
info.fetch("metadata")
|
55
|
+
end
|
56
|
+
|
57
|
+
def update_info(uid, info)
|
58
|
+
bucket.files_collection.find(filename: uid).update_one("$set" => {metadata: info})
|
59
|
+
end
|
60
|
+
|
61
|
+
def list_files
|
62
|
+
infos = bucket.files_collection.find.to_a
|
63
|
+
infos.map { |info| info.fetch("filename") }
|
64
|
+
end
|
65
|
+
|
66
|
+
private
|
67
|
+
|
68
|
+
def bson_id(uid)
|
69
|
+
BSON::ObjectId(uid)
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
73
|
+
end
|
data/lib/tus-server.rb
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require "tus/server"
|
data/tus-server.gemspec
ADDED
@@ -0,0 +1,24 @@
|
|
1
|
+
Gem::Specification.new do |gem|
|
2
|
+
gem.name = "tus-server"
|
3
|
+
gem.version = "0.1.0"
|
4
|
+
|
5
|
+
gem.required_ruby_version = ">= 2.1"
|
6
|
+
|
7
|
+
gem.summary = "Ruby server implementation of the Open Protocol for Resumable File Uploads."
|
8
|
+
|
9
|
+
gem.homepage = "https://github.com/janko-m/tus-ruby-server"
|
10
|
+
gem.authors = ["Janko Marohnić"]
|
11
|
+
gem.email = ["janko.marohnic@gmail.com"]
|
12
|
+
gem.license = "MIT"
|
13
|
+
|
14
|
+
gem.files = Dir["README.md", "LICENSE.txt", "lib/**/*.rb", "*.gemspec"]
|
15
|
+
gem.require_path = "lib"
|
16
|
+
|
17
|
+
gem.add_dependency "roda", "~> 2.17"
|
18
|
+
gem.add_dependency "rack", "~> 2.0"
|
19
|
+
|
20
|
+
gem.add_development_dependency "rake", "~> 11.1"
|
21
|
+
gem.add_development_dependency "minitest", "~> 5.8"
|
22
|
+
gem.add_development_dependency "rack-test_app"
|
23
|
+
gem.add_development_dependency "mongo"
|
24
|
+
end
|
metadata
ADDED
@@ -0,0 +1,138 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: tus-server
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Janko Marohnić
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2016-08-22 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: roda
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '2.17'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '2.17'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rack
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '2.0'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '2.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rake
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '11.1'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '11.1'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: minitest
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '5.8'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '5.8'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: rack-test_app
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - ">="
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - ">="
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '0'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: mongo
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - ">="
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '0'
|
90
|
+
type: :development
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - ">="
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: '0'
|
97
|
+
description:
|
98
|
+
email:
|
99
|
+
- janko.marohnic@gmail.com
|
100
|
+
executables: []
|
101
|
+
extensions: []
|
102
|
+
extra_rdoc_files: []
|
103
|
+
files:
|
104
|
+
- LICENSE.txt
|
105
|
+
- README.md
|
106
|
+
- lib/tus-server.rb
|
107
|
+
- lib/tus/checksum.rb
|
108
|
+
- lib/tus/expirator.rb
|
109
|
+
- lib/tus/info.rb
|
110
|
+
- lib/tus/server.rb
|
111
|
+
- lib/tus/storage/filesystem.rb
|
112
|
+
- lib/tus/storage/gridfs.rb
|
113
|
+
- tus-server.gemspec
|
114
|
+
homepage: https://github.com/janko-m/tus-ruby-server
|
115
|
+
licenses:
|
116
|
+
- MIT
|
117
|
+
metadata: {}
|
118
|
+
post_install_message:
|
119
|
+
rdoc_options: []
|
120
|
+
require_paths:
|
121
|
+
- lib
|
122
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
123
|
+
requirements:
|
124
|
+
- - ">="
|
125
|
+
- !ruby/object:Gem::Version
|
126
|
+
version: '2.1'
|
127
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
128
|
+
requirements:
|
129
|
+
- - ">="
|
130
|
+
- !ruby/object:Gem::Version
|
131
|
+
version: '0'
|
132
|
+
requirements: []
|
133
|
+
rubyforge_project:
|
134
|
+
rubygems_version: 2.5.1
|
135
|
+
signing_key:
|
136
|
+
specification_version: 4
|
137
|
+
summary: Ruby server implementation of the Open Protocol for Resumable File Uploads.
|
138
|
+
test_files: []
|