bucket_client 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitignore +19 -0
- data/.gitlab-ci.yml +70 -0
- data/.idea/bucket_client.iml +105 -0
- data/.idea/encodings.xml +4 -0
- data/.idea/misc.xml +7 -0
- data/.idea/modules.xml +8 -0
- data/.idea/runConfigurations/Integration_Test.xml +37 -0
- data/.idea/runConfigurations/Unit_Test.xml +37 -0
- data/.rspec +3 -0
- data/CODE_OF_CONDUCT.md +74 -0
- data/Gemfile +6 -0
- data/Gemfile.lock +114 -0
- data/LICENSE.txt +21 -0
- data/README.md +870 -0
- data/Rakefile +6 -0
- data/bin/console +14 -0
- data/bin/setup +8 -0
- data/bucket_client.gemspec +46 -0
- data/integration/aws_blob_spec.rb +134 -0
- data/integration/aws_bucket_spec.rb +145 -0
- data/integration/azure_blob_spec.rb +132 -0
- data/integration/azure_bucket_spec.rb +132 -0
- data/integration/dev_blob_spec.rb +131 -0
- data/integration/dev_bucket_spec.rb +140 -0
- data/integration/do_blob_spec.rb +134 -0
- data/integration/do_bucket_spec.rb +144 -0
- data/integration/gcp_blob_spec.rb +132 -0
- data/integration/gcp_bucket_spec.rb +132 -0
- data/integration/img.jpg +0 -0
- data/lib/bucket_client.rb +66 -0
- data/lib/bucket_client/aws/aws_bucket.rb +85 -0
- data/lib/bucket_client/aws/aws_client.rb +195 -0
- data/lib/bucket_client/aws/aws_http_client.rb +32 -0
- data/lib/bucket_client/aws/aws_policy_factory.rb +26 -0
- data/lib/bucket_client/aws4_request_signer.rb +133 -0
- data/lib/bucket_client/azure/azure_bucket.rb +83 -0
- data/lib/bucket_client/azure/azure_client.rb +197 -0
- data/lib/bucket_client/bucket.rb +388 -0
- data/lib/bucket_client/bucket_operation_exception.rb +8 -0
- data/lib/bucket_client/client.rb +408 -0
- data/lib/bucket_client/dev/local_bucket.rb +84 -0
- data/lib/bucket_client/dev/local_client.rb +148 -0
- data/lib/bucket_client/digital_ocean/digital_ocean_acl_factory.rb +39 -0
- data/lib/bucket_client/digital_ocean/digital_ocean_bucket.rb +81 -0
- data/lib/bucket_client/digital_ocean/digital_ocean_client.rb +275 -0
- data/lib/bucket_client/digital_ocean/digital_ocean_http_client.rb +31 -0
- data/lib/bucket_client/gcp/gcp_bucket.rb +79 -0
- data/lib/bucket_client/gcp/gcp_client.rb +171 -0
- data/lib/bucket_client/operation_result.rb +33 -0
- data/lib/bucket_client/version.rb +3 -0
- metadata +246 -0
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2019 kirinnee
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,870 @@
|
|
1
|
+
# BucketClient
|
2
|
+
|
3
|
+
Bucket Client is a ruby gem that allows programmers to interact with popular Blob Storage cloud
|
4
|
+
services. This intends to act as a layer of abstraction, much like ORM is to databases.
|
5
|
+
|
6
|
+
With this, you may easily change the blob storage provider or even defer them.
|
7
|
+
|
8
|
+
The supported cloud storage include:
|
9
|
+
- Google Cloud Platform Cloud Storage
|
10
|
+
- Amazon Web Service S3 Bucket
|
11
|
+
- Digital Ocean Spaces
|
12
|
+
- Azure Blob Storage (Microsoft)
|
13
|
+
|
14
|
+
## Installation
|
15
|
+
|
16
|
+
Add this line to your application's Gemfile:
|
17
|
+
|
18
|
+
```ruby
|
19
|
+
gem 'bucket_client'
|
20
|
+
```
|
21
|
+
|
22
|
+
And then execute:
|
23
|
+
|
24
|
+
$ bundle
|
25
|
+
|
26
|
+
Or install it yourself as:
|
27
|
+
|
28
|
+
$ gem install bucket_client
|
29
|
+
|
30
|
+
## Usage
|
31
|
+
|
32
|
+
### Creation of Client
|
33
|
+
|
34
|
+
To begin using BucketClient, you have to create a Client object based on the cloud service you want to use.
|
35
|
+
|
36
|
+
It is advised to use the LocalClient for development and test, as it uses the local disk to simulate how
|
37
|
+
online blob storage will work, except for `set_get_cors` and `set_read_policy` functions. This ensure fast
|
38
|
+
tests and development without polluting the actual bucket.
|
39
|
+
|
40
|
+
The creation of client use the `generate` class-level (or rather, module-level) method of the `BucketClient` module.
|
41
|
+
|
42
|
+
#### Local Client
|
43
|
+
Local Client uses a local path to simulate storage of blob. The `path` variable is the relative path from
|
44
|
+
terminal that it uses to simulate storage. Please be careful as it may delete files within that folder.
|
45
|
+
|
46
|
+
To create a local client:
|
47
|
+
```ruby
|
48
|
+
require "bucket_client"
|
49
|
+
|
50
|
+
client = BucketClient::generate type: :local, path: "./public/sample-bucket"
|
51
|
+
client # => the client used to perform CRUD on bucket and blobs
|
52
|
+
```
|
53
|
+
|
54
|
+
#### Amazon Web Service S3 Bucket
|
55
|
+
AWS Client requires 3 values, the `access id`, `access key`, which is the secret, and the `region`.
|
56
|
+
|
57
|
+
To create a AWS S3 client:
|
58
|
+
```ruby
|
59
|
+
require "bucket_client"
|
60
|
+
|
61
|
+
client = BucketClient::generate type: :aws, id: ENV["AWS_ID"], secret: ENV["AWS_SECRET"], region: ENV["AWS_REGION"]
|
62
|
+
client # => the client used to perform CRUD on bucket and blobs
|
63
|
+
```
|
64
|
+
|
65
|
+
#### Digital Ocean Spaces
|
66
|
+
Digital Ocean spaces requires 3 values, the `access id`, `access key`, which is the secret, and the `region`.
|
67
|
+
|
68
|
+
To create a Digital Ocean client:
|
69
|
+
```ruby
|
70
|
+
require "bucket_client"
|
71
|
+
|
72
|
+
client = BucketClient::generate type: :spaces, id: ENV["AWS_ID"], secret: ENV["AWS_SECRET"], region: ENV["AWS_REGION"]
|
73
|
+
client #=> the client used to perform CRUD on bucket and blobs
|
74
|
+
```
|
75
|
+
|
76
|
+
#### Azure Blob Storage
|
77
|
+
Azure Blob Storage require 2 values, the `account name` and the `key`, which is the secret.
|
78
|
+
|
79
|
+
To create a Azure client:
|
80
|
+
```ruby
|
81
|
+
require "bucket_client"
|
82
|
+
|
83
|
+
client = BucketClient::generate type: :azure, id: ENV["AZURE_ACC_NAME"], secret: ENV["AZURE_KEY"]
|
84
|
+
client #=> the client used to perform CRUD on bucket and blobs
|
85
|
+
```
|
86
|
+
|
87
|
+
#### Google Cloud Platform Storage Service
|
88
|
+
GCP Cloud Storage require 2 value, the `project_id` and the secret
|
89
|
+
The secret can be passed in via 2 methods,
|
90
|
+
- serialized Hash object of the secret
|
91
|
+
- path to the secret json file on disk
|
92
|
+
|
93
|
+
To create a GCP client using the Hash object (assume the JSON value is stored as environment variable):
|
94
|
+
```ruby
|
95
|
+
require "json"
|
96
|
+
require "bucket_client"
|
97
|
+
|
98
|
+
secret = JSON.parse(ENV["GOOGLE_KEY"])
|
99
|
+
client = BucketClient::generate type: :gcp, id: ENV["GOOGLE_ID"], secret: secret
|
100
|
+
client #=> the client used to perform CRUD on bucket and blobs
|
101
|
+
```
|
102
|
+
|
103
|
+
To create a GCP client using the path to json secret:
|
104
|
+
```ruby
|
105
|
+
require "bucket_client"
|
106
|
+
client = BucketClient::generate type: :gcp, id: ENV["GOOGLE_ID"], secret: "path/to/secret.json"
|
107
|
+
client #=> the client used to perform CRUD on bucket and blobs
|
108
|
+
```
|
109
|
+
|
110
|
+
---
|
111
|
+
|
112
|
+
### Operation Result
|
113
|
+
|
114
|
+
OperationResult is the object you obtain from the normal operations. It contains details of the operation where you
|
115
|
+
can check:
|
116
|
+
|
117
|
+
|Property| Description|
|
118
|
+
| --- | --- |
|
119
|
+
| success | whether the operation was successful. Boolean value|
|
120
|
+
| code | the status code of the operation |
|
121
|
+
| message | the message of the operation. Error messages can be checked here |
|
122
|
+
| value | the usable value of the operation. May be url or binary |
|
123
|
+
|
124
|
+
If you rather immediately obtain the value, and raise error when it is unsuccessful, you may use the ! version of
|
125
|
+
the method.
|
126
|
+
|
127
|
+
```ruby
|
128
|
+
# Using OperationResult Object
|
129
|
+
result = bucket.create_blob binary, "path/to/bucket/file.bin"
|
130
|
+
if result.success
|
131
|
+
p result.code #=> prints the status code
|
132
|
+
p result.message #=> prints the success message, if any
|
133
|
+
p result.value # => prints the URI obtain from "create_blob" method, when successful
|
134
|
+
else
|
135
|
+
p result.code #=> check what HTTP error code is obtained
|
136
|
+
p result.message #=> check error message
|
137
|
+
end
|
138
|
+
|
139
|
+
# Or use ! method to immediate capture the method
|
140
|
+
begin
|
141
|
+
result = bucket.create_blob! binary, "path/to/bucket/file.bin"
|
142
|
+
p result #=> prints the URI obtained from "create_blob" method, when successful
|
143
|
+
rescue StandardError => e
|
144
|
+
p e.message #=> prints the error message. This will include the status code
|
145
|
+
end
|
146
|
+
|
147
|
+
```
|
148
|
+
|
149
|
+
---
|
150
|
+
|
151
|
+
### Using Client object for Bucket CRUD
|
152
|
+
|
153
|
+
The client object obtain via the `generate` method can be used to perform Bucket CRUD actions. It works across
|
154
|
+
all platforms.
|
155
|
+
|
156
|
+
bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
|
157
|
+
|
158
|
+
#### exist_bucket `key:string` => `boolean`
|
159
|
+
Checks whether the bucket of a certain key exist. Raises exception when the HTTP request underneath fails
|
160
|
+
|
161
|
+
```ruby
|
162
|
+
# client from above
|
163
|
+
|
164
|
+
exist = client.exist_bucket "sample-bucket"
|
165
|
+
exist #=> true if exist, false if it does not exist
|
166
|
+
```
|
167
|
+
#### create_bucket `key:string` => `OperationResult<Bucket>`
|
168
|
+
Creates a bucket using the provided key. Fails if bucket already exist.
|
169
|
+
|
170
|
+
`value` of `OperationResult` if successful is the `Bucket` object that has been created.
|
171
|
+
|
172
|
+
```ruby
|
173
|
+
# client from above
|
174
|
+
result = client.create_bucket "sample-bucket"
|
175
|
+
if result.success
|
176
|
+
bucket = result.value
|
177
|
+
bucket #=> obtains the bucket
|
178
|
+
else
|
179
|
+
p result.message #=> prints the error message
|
180
|
+
p result.code #=> prints the status code
|
181
|
+
end
|
182
|
+
```
|
183
|
+
|
184
|
+
#### create_bucket! `key:string` => `Bucket`
|
185
|
+
Creates a bucket using the provided key. Fails if bucket already exist.
|
186
|
+
|
187
|
+
Raises exception if fail, returns `Bucket` object that has been created if it succeeds
|
188
|
+
|
189
|
+
```ruby
|
190
|
+
# client from above
|
191
|
+
result = client.create_bucket "sample-bucket"
|
192
|
+
result #=> obtains bucket
|
193
|
+
```
|
194
|
+
#### delete_bucket `key:string` => `OperationResult<nil>`
|
195
|
+
Deletes the bucket provided in the key. Fails if bucket does not exist.
|
196
|
+
To prevent that behaviour, use `delete_bucket_if_exist` to not fail even if
|
197
|
+
bucket does not exist.
|
198
|
+
|
199
|
+
Does not return anything on success. `value` will always return `nil`
|
200
|
+
|
201
|
+
```ruby
|
202
|
+
# client from above
|
203
|
+
result = client.delete_bucket "sample-bucket"
|
204
|
+
result.success #=> whether the bucket has been successfully deleted
|
205
|
+
result.message #=> Error message or success message
|
206
|
+
result.code #=> status code of the operation
|
207
|
+
```
|
208
|
+
|
209
|
+
#### delete_bucket! `key:string` => `nil`
|
210
|
+
Deletes the bucket provided in the key. Fails if bucket does not exist.
|
211
|
+
To prevent that behaviour, use `delete_bucket_if_exist!` to not fail even if
|
212
|
+
bucket does not exist.
|
213
|
+
|
214
|
+
Raises exception if fail. Returns `nil`
|
215
|
+
|
216
|
+
```ruby
|
217
|
+
# client from above
|
218
|
+
client.delete_bucket! "sample-bucket" #=> nil
|
219
|
+
```
|
220
|
+
|
221
|
+
#### delete_bucket_if_exist `key:string` => `OperationResult<nil>`
|
222
|
+
Deletes the bucket provided in the key. Will succeed even if bucket does not exist.
|
223
|
+
|
224
|
+
Does not return anything on success. `value` will always return `nil`
|
225
|
+
|
226
|
+
```ruby
|
227
|
+
# client from above
|
228
|
+
result = client.delete_bucket_if_exist "sample-bucket"
|
229
|
+
result.success #=> whether the bucket has been successfully deleted
|
230
|
+
result.message #=> Error message or success message
|
231
|
+
result.code #=> status code of the operation
|
232
|
+
```
|
233
|
+
|
234
|
+
#### delete_bucket_if_exist! `key:string` => `nil`
|
235
|
+
Deletes the bucket provided in the key. Will succeed even if bucket does not exist.
|
236
|
+
|
237
|
+
Raises exception if the operation fails
|
238
|
+
|
239
|
+
```ruby
|
240
|
+
# client from above
|
241
|
+
client.delete_bucket_if_exist! "sample-bucket" #=> nil
|
242
|
+
```
|
243
|
+
|
244
|
+
|
245
|
+
#### put_bucket `key:string` => `OperationResult<Bucket>`
|
246
|
+
Creates the bucket provided in key if it does not exist.
|
247
|
+
|
248
|
+
This method will succeed and return the bucket even if the bucket exist, unlike `create_bucket`
|
249
|
+
|
250
|
+
`value` of the `OperationResult` is the `Bucket` object if the operation succeeds
|
251
|
+
|
252
|
+
```ruby
|
253
|
+
# client from above
|
254
|
+
result = client.put_bucket "sample-bucket"
|
255
|
+
if result.success
|
256
|
+
bucket = result.value
|
257
|
+
bucket #=> obtains the bucket
|
258
|
+
else
|
259
|
+
p result.message #=> prints the error message
|
260
|
+
p result.code #=> prints the status code
|
261
|
+
end
|
262
|
+
```
|
263
|
+
|
264
|
+
#### put_bucket! `key:string` => `Bucket`
|
265
|
+
Creates the bucket provided in key if it does not exist.
|
266
|
+
|
267
|
+
This method will succeed and return the bucket even if the bucket exist, unlike `create_bucket`
|
268
|
+
|
269
|
+
Returns the `Bucket` object
|
270
|
+
|
271
|
+
Raises exception if the operation fails
|
272
|
+
|
273
|
+
```ruby
|
274
|
+
# client from above
|
275
|
+
bucket = client.put_bucket! "sample-bucket"
|
276
|
+
bucket #=> obtains the bucket that has been creted
|
277
|
+
```
|
278
|
+
|
279
|
+
|
280
|
+
#### set_read_policy `key:string`, `access:symbol` => `OperationResult<nil>`
|
281
|
+
Sets the read policy of the bucket. This does not work for `LocalBucket` as `LocalBucket` does not have
|
282
|
+
concept of "access".
|
283
|
+
|
284
|
+
Only two values are accepted: `:public` and `:private`.
|
285
|
+
`:public` allows everyone with access to the link to read the blobs within the bucket
|
286
|
+
`:private` only allows people with authorization (with secret key) to read the blob within the bucket
|
287
|
+
|
288
|
+
```ruby
|
289
|
+
# client from above
|
290
|
+
result = client.set_read_policy "sample-bucket", :public
|
291
|
+
result.success #=> whether the bucket has been made public
|
292
|
+
result.message #=> Error message or success message
|
293
|
+
result.code #=> status code of the operation
|
294
|
+
```
|
295
|
+
|
296
|
+
#### set_read_policy! `key:string`, `access:symbol` => `nil`
|
297
|
+
Sets the read policy of the bucket. This does not work for `LocalBucket` as `LocalBucket` does not have
|
298
|
+
concept of "access".
|
299
|
+
|
300
|
+
Raises exception if the operation fails.
|
301
|
+
|
302
|
+
Only two values are accepted: `:public` and `:private`.
|
303
|
+
`:public` allows everyone with access to the link to read the blobs within the bucket
|
304
|
+
`:private` only allows people with authorization (with secret key) to read the blob within the bucket
|
305
|
+
|
306
|
+
```ruby
|
307
|
+
# client from above
|
308
|
+
client.set_read_policy! "sample-bucket", :public #=> nil
|
309
|
+
```
|
310
|
+
|
311
|
+
#### set_get_cors `key:string`, `cors:array<string>` => `OperationResult<nil>`
|
312
|
+
Sets the GET CORS of the bucket. This is limits the Cross Origin Resource Sharing to the domains
|
313
|
+
within the `cors` array you input. To allow all origin, please use `["*"]` as cors value.
|
314
|
+
|
315
|
+
This does not work for `LocalBucket` as it does not have concept of `cors`.
|
316
|
+
This is one-level higher for `AzureBucket`, where it modifies the whole accounts' `CORS`, not just the bucket.
|
317
|
+
|
318
|
+
```ruby
|
319
|
+
# client from above
|
320
|
+
result = client.set_get_cors "sample-bucket", ["*"]
|
321
|
+
result.success #=> whether it has succeeded allowing all origin to read
|
322
|
+
result.message #=> Error message or success message
|
323
|
+
result.code #=> status code of the operation
|
324
|
+
```
|
325
|
+
|
326
|
+
#### set_get_cors! `key:string`, `cors:array<string>` => `nil`
|
327
|
+
Sets the GET CORS of the bucket. This is limits the Cross Origin Resource Sharing to the domains
|
328
|
+
within the `cors` array you input. To allow all origin, please use `["*"]` as cors value.
|
329
|
+
|
330
|
+
This does not work for `LocalBucket` as it does not have concept of `cors`.
|
331
|
+
This is one-level higher for `AzureBucket`, where it modifies the whole accounts' `CORS`, not just the bucket.
|
332
|
+
|
333
|
+
Raises exception if the operation fails
|
334
|
+
|
335
|
+
```ruby
|
336
|
+
#client from above
|
337
|
+
client.set_get_cors! "sample-bucket", ["*"] #=> nil
|
338
|
+
```
|
339
|
+
|
340
|
+
#### get_bucket `key:string` => `Bucket`
|
341
|
+
Obtains the `Bucket` instance with the key.
|
342
|
+
|
343
|
+
The bucket instance can be used to perform CRUD for blobs within the bucket.
|
344
|
+
|
345
|
+
This method will raise exception if the bucket does not exist. To improve speed as you are sure that the bucket
|
346
|
+
already exist, please use the bang version, `get_bucket!`, where it will not do a look up.
|
347
|
+
|
348
|
+
```ruby
|
349
|
+
#client from above
|
350
|
+
bucket = get_bucket "sample-bucket"
|
351
|
+
bucket #=> bucket instance obtained.
|
352
|
+
```
|
353
|
+
|
354
|
+
#### get_bucket! `key:string` => `Bucket`
|
355
|
+
Obtains the `Bucket` instance with the key.
|
356
|
+
|
357
|
+
The bucket instance can be used to perform CRUD for blobs within the
|
358
|
+
bucket.
|
359
|
+
|
360
|
+
This method will not do a look up, so you instance's blob CRUD operation may fail if you did not verify the existence
|
361
|
+
of this bucket. This performs faster than the non-bang version as it does not spend operation to check existence of
|
362
|
+
the bucket, making the assumption that it exist.
|
363
|
+
|
364
|
+
```ruby
|
365
|
+
#client from above
|
366
|
+
bucket = get_bucket! "sample-bucket"
|
367
|
+
bucket #=> bucket instance obtained.
|
368
|
+
```
|
369
|
+
|
370
|
+
---
|
371
|
+
|
372
|
+
### Using Client object for Blob CRUD
|
373
|
+
The client object can perform Blob CRUD if it has access to the full URI or URL of the blob.
|
374
|
+
|
375
|
+
Bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
|
376
|
+
|
377
|
+
#### get_blob `uri:string` => `OperationResult<array<byte>>`
|
378
|
+
Obtains the binary of the blob via the URI of the blob.
|
379
|
+
|
380
|
+
`value` of the `OperationResult` is the byte array of the binary if the operation succeeds
|
381
|
+
|
382
|
+
```ruby
|
383
|
+
# client from above
|
384
|
+
result = client.get_blob "https://host.com/bucket/blob.bin"
|
385
|
+
if result.success #=> whether the obtaining of the blob succeeded
|
386
|
+
binary = result.value #=> obtain the binary value
|
387
|
+
IO.binwrite "blob.bin", binary #=> writes it to disk
|
388
|
+
else
|
389
|
+
p result.message #=> Error message or success message
|
390
|
+
p result.code #=> status code of the operation
|
391
|
+
end
|
392
|
+
```
|
393
|
+
|
394
|
+
#### get_blob! `uri:string` => `<array<byte>>`
|
395
|
+
Obtains the binary of the blob via the URI of the blob
|
396
|
+
|
397
|
+
Raises exception if it fails
|
398
|
+
```ruby
|
399
|
+
binary = client.get_blob! "https://host.com/bucket/blob.bin"
|
400
|
+
IO.binwrite "blob.bin", binary
|
401
|
+
```
|
402
|
+
|
403
|
+
#### exist_blob `uri:string` => `boolean`
|
404
|
+
Checks whether the blob exist
|
405
|
+
|
406
|
+
```ruby
|
407
|
+
exist = client.exist_blob "https://host.com/bucket/blob.bin"
|
408
|
+
exist #=> true if blob exist, false if it doesn't
|
409
|
+
```
|
410
|
+
|
411
|
+
#### update_blob `payload:array<byte>`, `uri:string` => `Operation<string>`
|
412
|
+
Updates a blob with new payload in byte array
|
413
|
+
|
414
|
+
`value` of the `OperationResult` will return URI of the blob if success
|
415
|
+
|
416
|
+
Fails if blob with the URI doesn't exist
|
417
|
+
```ruby
|
418
|
+
img = IO.binread "pic.png"
|
419
|
+
uri = "https://host.com/folder/pic.png"
|
420
|
+
result = client.update_blob img, uri
|
421
|
+
result.success #=> Whether the operation succeeded
|
422
|
+
result.code #=> Status Code of the operation
|
423
|
+
result.message #=> Error message if it failed
|
424
|
+
result.value #=> Uri of update blob
|
425
|
+
```
|
426
|
+
|
427
|
+
#### update_blob! `payload:array<byte>`, `uri:string` => `string`
|
428
|
+
Updates a blob with new payload in byte array
|
429
|
+
Fails if blob doesnt exist
|
430
|
+
|
431
|
+
Raises exception if operation fails
|
432
|
+
|
433
|
+
```ruby
|
434
|
+
img = IO.binread "pic.png"
|
435
|
+
uri = "https://host.com/folder/pic.png"
|
436
|
+
result = client.update_blob! img, uri
|
437
|
+
result #=> URI of update blob
|
438
|
+
```
|
439
|
+
|
440
|
+
#### put_blob `payload:array<byte>`, `uri:string` => `OperationResult<string>`
|
441
|
+
Creates the blob with the payload if it does not exist,
|
442
|
+
updates the blob with the new payload if it exist
|
443
|
+
|
444
|
+
`value` of the `OperationResult` will return URI of the blob if success
|
445
|
+
|
446
|
+
```ruby
|
447
|
+
img = IO.binread "pic.png"
|
448
|
+
uri = "https://host.com/folder/pic.png"
|
449
|
+
result = client.put_blob img, uri
|
450
|
+
result.success #=> Whether the operation succeeded
|
451
|
+
result.code #=> Status Code of the operation
|
452
|
+
result.message #=> Error message if it failed
|
453
|
+
result.value #=> Uri of blob
|
454
|
+
```
|
455
|
+
|
456
|
+
#### put_blob! `payload:array<byte>`, `uri:string` => `string`
|
457
|
+
Creates the blob with the payload if it does not exist,
|
458
|
+
updates the blob with the new payload if it exist
|
459
|
+
|
460
|
+
Raises exception if the operation fails
|
461
|
+
|
462
|
+
```ruby
|
463
|
+
img = IO.binread "pic.png"
|
464
|
+
uri = "https://host.com/folder/pic.png"
|
465
|
+
result = client.put_blob! img, uri
|
466
|
+
result #=> returns URI of updated blob
|
467
|
+
```
|
468
|
+
|
469
|
+
#### delete_blob `uri:string` => `OperationResult<nil>`
|
470
|
+
Deletes the blob in the provided URI
|
471
|
+
|
472
|
+
Fails if the blob does not exist. Use delete_blob_if_exist if you do not want this behaviour
|
473
|
+
|
474
|
+
`value` of `OperationResult` is always `nil`
|
475
|
+
|
476
|
+
```ruby
|
477
|
+
uri = "https://host.com/folder/pic.png"
|
478
|
+
result = client.delete_blob uri
|
479
|
+
result.success #=> Whether the operation succeeded
|
480
|
+
result.code #=> Status Code of the operation
|
481
|
+
result.message #=> Error message if it failed
|
482
|
+
result.value #=> nil
|
483
|
+
```
|
484
|
+
|
485
|
+
#### delete_blob! `uri:string` => `nil`
|
486
|
+
Deletes the blob in the provided URI
|
487
|
+
Fails if the blob does not exist. Use delete_blob_if_exist if you
|
488
|
+
do not want this behaviour
|
489
|
+
|
490
|
+
Raises exception if the operation fails
|
491
|
+
|
492
|
+
```ruby
|
493
|
+
uri = "https://host.com/folder/pic.png"
|
494
|
+
client.delete_blob! uri
|
495
|
+
```
|
496
|
+
|
497
|
+
#### delete_blob_if_exist `uri:string` => `OperationResult<nil>`
|
498
|
+
Deletes the blob if it exist, else does nothing
|
499
|
+
|
500
|
+
`value` of `OperationResult` is always `nil`
|
501
|
+
|
502
|
+
```ruby
|
503
|
+
uri = "https://host.com/folder/pic.png"
|
504
|
+
result = client.delete_blob uri
|
505
|
+
result.success #=> Whether the operation succeeded
|
506
|
+
result.code #=> Status Code of the operation
|
507
|
+
result.message #=> Error message if it failed
|
508
|
+
result.value #=> nil
|
509
|
+
```
|
510
|
+
|
511
|
+
#### delete_blob_if_exist! `uri:string` => `nil`
|
512
|
+
Deletes the blob if it exist, else does nothing
|
513
|
+
|
514
|
+
Raises exception if the operation fails
|
515
|
+
```ruby
|
516
|
+
uri = "https://host.com/folder/pic.png"
|
517
|
+
client.delete_blob! uri
|
518
|
+
```
|
519
|
+
---
|
520
|
+
|
521
|
+
### Using Bucket object to perform blob CRUD with blob keys
|
522
|
+
The bucket instance is able to perform CRUD operations on blobs it owns.
|
523
|
+
|
524
|
+
Bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
|
525
|
+
|
526
|
+
In this section, we assume we obtain a `Bucket` instance from the `Client` instance using the `get_bucket` method.
|
527
|
+
|
528
|
+
```ruby
|
529
|
+
bucket = client.get_bucket! "first-bucket"
|
530
|
+
bucket #=> bucket instance used to illustrate the examples below
|
531
|
+
```
|
532
|
+
|
533
|
+
#### get_blob `key:string` => `OperationResult<array<byte>>`
|
534
|
+
Get blob as byte array
|
535
|
+
|
536
|
+
`value` of the `OperationResult` is the blob's byte array, if the operation succeeds
|
537
|
+
|
538
|
+
```ruby
|
539
|
+
result = bucket.get_blob "image.png"
|
540
|
+
result.success #=> Whether the operation succeeded
|
541
|
+
result.code #=> Status Code of the operation
|
542
|
+
result.message #=> Error message if it failed
|
543
|
+
result.value #=> the byte array of the blob
|
544
|
+
```
|
545
|
+
|
546
|
+
#### get_blob! `key:string` => `array<byte>`
|
547
|
+
Get blob as byte array
|
548
|
+
|
549
|
+
Raises exception if the operation fails.
|
550
|
+
```ruby
|
551
|
+
img = bucket.get_blob! "image.png"
|
552
|
+
IO.binwrite "image.png", img
|
553
|
+
```
|
554
|
+
|
555
|
+
#### exist_blob `key:string` => `boolean`
|
556
|
+
Checks if the blob with the given key exist.
|
557
|
+
|
558
|
+
```ruby
|
559
|
+
exist = bucket.exist_blob "image.png"
|
560
|
+
exist #=> true if exist, false if it does not exist
|
561
|
+
```
|
562
|
+
|
563
|
+
#### create_blob `payload:byte<array>`,`key:string` => `Operation<string>`
|
564
|
+
Create blob with payload. Fails if blob already exist.
|
565
|
+
|
566
|
+
`value` of `OperationResult` will return URI of the created blob if operations succeeded
|
567
|
+
|
568
|
+
```ruby
|
569
|
+
img = IO.binread "image.png"
|
570
|
+
result = bucket.create_blob img, "image.png"
|
571
|
+
result.success #=> Whether the operation succeeded
|
572
|
+
result.code #=> Status Code of the operation
|
573
|
+
result.message #=> Error message if it failed
|
574
|
+
result.value #=> URI of the blob
|
575
|
+
```
|
576
|
+
|
577
|
+
#### create_blob! `payload:array<byte>`,`key:string` => `string`
|
578
|
+
Create blob with payload. Fails if blob already exist.
|
579
|
+
|
580
|
+
Raises exception if operation fails
|
581
|
+
|
582
|
+
```ruby
|
583
|
+
img = IO.binread "image.png"
|
584
|
+
uri = bucket.create_blob! img, "image.png"
|
585
|
+
uri #=> URI of the created blob
|
586
|
+
```
|
587
|
+
|
588
|
+
#### update_blob `payload:array<byte>`, `key:string` => `OperationResult<string>`
|
589
|
+
Updates the blob with new payload. Fails if blob does not exist
|
590
|
+
|
591
|
+
`value` of `OperationResult` will return URI of the created blob if operations succeeded
|
592
|
+
|
593
|
+
```ruby
|
594
|
+
img = IO.binread "image.png"
|
595
|
+
result = bucket.update_blob img, "image.png"
|
596
|
+
result.success #=> whether the operation succeeded
|
597
|
+
result.code #=> Status Code of the operation
|
598
|
+
result.message #=> Error message if it failed
|
599
|
+
result.value #=> URI of the blob
|
600
|
+
```
|
601
|
+
|
602
|
+
#### update_blob! `payload:array<byte>`, `key:string` => `string`
|
603
|
+
Updates the blob with new payload. Fails if blob does not exist
|
604
|
+
|
605
|
+
Raises exception if the operation fails
|
606
|
+
|
607
|
+
```ruby
|
608
|
+
img = IO.binread "image.png"
|
609
|
+
result = bucket.update_blob!(img, "image.png")
|
610
|
+
result #=> URI of updated blob
|
611
|
+
```
|
612
|
+
|
613
|
+
#### put_blob `payload:array<byte>`, `key:string` => `OperationResult<string>`
|
614
|
+
Creates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
|
615
|
+
|
616
|
+
`value` of `OperationResult` will return URI of the created blob if operations succeeded
|
617
|
+
|
618
|
+
```ruby
|
619
|
+
img = IO.binread "image.png"
|
620
|
+
result = bucket.put_blob(img, "image.png")
|
621
|
+
result.success #=> whether the operation succeeded
|
622
|
+
result.code #=> Status Code of the operation
|
623
|
+
result.message #=> Error message if it failed
|
624
|
+
result.value #=> URI of the blob
|
625
|
+
```
|
626
|
+
|
627
|
+
#### put_blob! `payload:array<byte>`, `key:string` => `string`
|
628
|
+
Creates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
|
629
|
+
|
630
|
+
Raises exception if operation fails
|
631
|
+
|
632
|
+
```ruby
|
633
|
+
img = IO.binread "image.png"
|
634
|
+
uri = bucket.put_blob! img, "image.png"
|
635
|
+
uri #=> uri of the blob
|
636
|
+
```
|
637
|
+
|
638
|
+
#### delete_blob `key:string` => `Operation<nil>`
|
639
|
+
Deletes a blob. Fails if the blob does not exist. To prevent this behaviour, use
|
640
|
+
delete_blob_if_exist method
|
641
|
+
|
642
|
+
`value` of `OperationResult` will always return `nil`
|
643
|
+
|
644
|
+
```ruby
|
645
|
+
result = bucket.delete_blob "image.png"
|
646
|
+
result.success #=> whether the operation succeeded
|
647
|
+
result.code #=> Status Code of the operation
|
648
|
+
result.message #=> Error message if it failed
|
649
|
+
result.value #=> nil
|
650
|
+
```
|
651
|
+
|
652
|
+
#### delete_blob! `key:string` => `nil`
|
653
|
+
Deletes a blob. Fails if the blob does not exist. To prevent this behaviour, use
|
654
|
+
delete_blob_if_exist method
|
655
|
+
|
656
|
+
Raises exception if the operation fails
|
657
|
+
|
658
|
+
```ruby
|
659
|
+
bucket.delete_blob! "image.png"
|
660
|
+
```
|
661
|
+
|
662
|
+
#### delete_blob_if_exist `key:string` => `Operation<nil>`
|
663
|
+
Deletes a blob if it exist.
|
664
|
+
|
665
|
+
`value` of `OperationResult` will always return `nil`
|
666
|
+
|
667
|
+
```ruby
|
668
|
+
result = bucket.delete_blob_if_exist "image.png"
|
669
|
+
result.success #=> whether the operation succeeded
|
670
|
+
result.code #=> Status Code of the operation
|
671
|
+
result.message #=> Error message if it failed
|
672
|
+
result.value #=> nil
|
673
|
+
```
|
674
|
+
|
675
|
+
#### delete_blob_if_exist! `key:string` => `nil`
|
676
|
+
Deletes a blob if it exist.
|
677
|
+
|
678
|
+
Raises exception if the operation fails
|
679
|
+
|
680
|
+
```ruby
|
681
|
+
bucket.delete_blob_if_exist! "image.png"
|
682
|
+
```
|
683
|
+
|
684
|
+
---
|
685
|
+
|
686
|
+
### Using Bucket object to perform blob CRUD with full URI
|
687
|
+
The bucket instance is able to perform CRUD operations on blobs it owns, via the full URI of the blob.
|
688
|
+
|
689
|
+
Bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
|
690
|
+
|
691
|
+
In this section, we assume we obtain a `Bucket` instance from the `Client` instance using the `get_bucket!` method.
|
692
|
+
|
693
|
+
```ruby
|
694
|
+
bucket = client.get_bucket! "first-bucket"
|
695
|
+
bucket #=> bucket instance used to illustrate the examples below
|
696
|
+
```
|
697
|
+
|
698
|
+
#### get_blob_with_uri `uri:string` => `OperationResult<array<byte>>`
|
699
|
+
Get blob in target URI as byte array
|
700
|
+
|
701
|
+
`value` of the `OperationResult` is the blob's byte array, if the operation succeeds
|
702
|
+
|
703
|
+
```ruby
|
704
|
+
result = bucket.get_blob_with_uri "https://domain.com/bucket/binary.ext"
|
705
|
+
result.success #=> Whether the operation succeeded
|
706
|
+
result.code #=> Status Code of the operation
|
707
|
+
result.message #=> Error message if it failed
|
708
|
+
result.value #=> the byte array of the blob
|
709
|
+
```
|
710
|
+
|
711
|
+
#### get_blob_with_uri! `key:string` => `array<byte>`
|
712
|
+
Get blob in target URI as byte array
|
713
|
+
|
714
|
+
Raises exception if the operation fails.
|
715
|
+
```ruby
|
716
|
+
img = bucket.get_blob_with_uri! "https://domain.com/bucket/binary.ext"
|
717
|
+
IO.binwrite "image.png", img
|
718
|
+
```
|
719
|
+
|
720
|
+
#### exist_blob_uri `uri:string` => `boolean`
|
721
|
+
Checks if the blob with the given URI exist.
|
722
|
+
|
723
|
+
```ruby
|
724
|
+
exist = bucket.exist_blob_with_uri "https://domain.com/bucket/binary.ext"
|
725
|
+
exist #=> true if exist, false if it does not exist
|
726
|
+
```
|
727
|
+
|
728
|
+
#### update_blob_with_uri `payload:array<byte>`, `uri:string` => `OperationResult<string>`
|
729
|
+
Updates the blob with new payload to the uri. Fails if blob does not exist
|
730
|
+
|
731
|
+
`value` of `OperationResult` will return URI of the created blob if operations succeeded
|
732
|
+
|
733
|
+
```ruby
|
734
|
+
img = IO.binread "image.png"
|
735
|
+
result = bucket.update_blob_with_uri img, "https://domain.com/bucket/binary.ext"
|
736
|
+
result.success #=> whether the operation succeeded
|
737
|
+
result.code #=> Status Code of the operation
|
738
|
+
result.message #=> Error message if it failed
|
739
|
+
result.value #=> URI of the blob
|
740
|
+
```
|
741
|
+
|
742
|
+
#### update_blob_with_uri! `payload:array<byte>`, `uri:string` => `string`
|
743
|
+
Updates the blob with new payload. Fails if blob does not exist
|
744
|
+
|
745
|
+
Raises exception if the operation fails
|
746
|
+
|
747
|
+
```ruby
|
748
|
+
img = IO.binread "image.png"
|
749
|
+
result = bucket.update_blob_with_uri! img, "https://domain.com/bucket/binary.ext"
|
750
|
+
result #=> URI of updated blob
|
751
|
+
```
|
752
|
+
|
753
|
+
#### put_blob_with_uri `payload:array<byte>`, `uri:string` => `OperationResult<string>`
|
754
|
+
Creates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
|
755
|
+
|
756
|
+
`value` of `OperationResult` will return URI of the created blob if operations succeeded
|
757
|
+
|
758
|
+
```ruby
|
759
|
+
img = IO.binread "image.png"
|
760
|
+
result = bucket.put_blob_with_uri img, "https://domain.com/bucket/binary.ext"
|
761
|
+
result.success #=> whether the operation succeeded
|
762
|
+
result.code #=> Status Code of the operation
|
763
|
+
result.message #=> Error message if it failed
|
764
|
+
result.value #=> URI of the blob
|
765
|
+
```
|
766
|
+
|
767
|
+
#### put_blob_with_uri! `payload:array<byte>`, `uri:string` => `string`
|
768
|
+
Creates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
|
769
|
+
|
770
|
+
Raises exception if operation fails
|
771
|
+
|
772
|
+
```ruby
|
773
|
+
img = IO.binread "image.png"
|
774
|
+
uri = bucket.put_blob_with_uri! img, "https://domain.com/bucket/binary.ext"
|
775
|
+
uri #=> uri of the blob
|
776
|
+
```
|
777
|
+
|
778
|
+
#### delete_blob_with_uri `uri:string` => `Operation<nil>`
|
779
|
+
Deletes a blob in the uri. Fails if the blob does not exist. To prevent this behaviour, use
|
780
|
+
delete_blob_if_exist_with_uri method
|
781
|
+
|
782
|
+
`value` of `OperationResult` will always return `nil`
|
783
|
+
|
784
|
+
```ruby
|
785
|
+
result = bucket.delete_blob_with_uri "https://domain.com/bucket/binary.ext"
|
786
|
+
result.success #=> whether the operation succeeded
|
787
|
+
result.code #=> Status Code of the operation
|
788
|
+
result.message #=> Error message if it failed
|
789
|
+
result.value #=> nil
|
790
|
+
```
|
791
|
+
|
792
|
+
#### delete_blob_with_uri! `uri:string` => `nil`
|
793
|
+
Deletes a blob. Fails if the blob does not exist. To prevent this behaviour, use
|
794
|
+
delete_blob_if_exist_with_uri! method
|
795
|
+
|
796
|
+
Raises exception if the operation fails
|
797
|
+
|
798
|
+
```ruby
|
799
|
+
bucket.delete_blob_with_uri! "https://domain.com/bucket/binary.ext"
|
800
|
+
```
|
801
|
+
|
802
|
+
#### delete_blob_if_exist_with_uri `uri:string` => `Operation<nil>`
|
803
|
+
Deletes a blob in the uri if it exist.
|
804
|
+
|
805
|
+
`value` of `OperationResult` will always return `nil`
|
806
|
+
|
807
|
+
```ruby
|
808
|
+
result = bucket.delete_blob_if_exist_with_uri "https://domain.com/bucket/binary.ext"
|
809
|
+
result.success #=> whether the operation succeeded
|
810
|
+
result.code #=> Status Code of the operation
|
811
|
+
result.message #=> Error message if it failed
|
812
|
+
result.value #=> nil
|
813
|
+
```
|
814
|
+
|
815
|
+
#### delete_blob_if_exist_with_uri! `uri:string` => `nil`
|
816
|
+
Deletes a blob in the uri if it exist.
|
817
|
+
|
818
|
+
Raises exception if the operation fails
|
819
|
+
|
820
|
+
```ruby
|
821
|
+
bucket.delete_blob_if_exist_with_uri! "https://domain.com/bucket/binary.ext"
|
822
|
+
```
|
823
|
+
---
|
824
|
+
|
825
|
+
## Development
|
826
|
+
|
827
|
+
After checking out the repo, run `bin/setup` to install dependencies.
|
828
|
+
|
829
|
+
Then, run `bundle exec rspec` to run the unit tests, or if you are on RubyMime, the run configuration of
|
830
|
+
Unit Test is configured. You can run the test by selecting the `Unit Test` configuration.
|
831
|
+
|
832
|
+
To run the integration test with the cloud, please provide your cloud credentials in a `.env` file. The
|
833
|
+
credentials required are:
|
834
|
+
|
835
|
+
| Env Variable | Description |
|
836
|
+
| --- | --- |
|
837
|
+
| AWS_ID| The AWS access key ID |
|
838
|
+
| AWS_SECRET | The AWS secret |
|
839
|
+
| AWS_REGION | The 3region of the AWS bucket|
|
840
|
+
| DO_ID | Digital Ocean Spaces access key ID |
|
841
|
+
| DO_SECRET | Digital Ocean Spaces secret |
|
842
|
+
| DO_REGION | Digital Ocean Spaces region |
|
843
|
+
| AZURE_ACC_ID | The Azure Blob Storage account name |
|
844
|
+
| AZURE_KEY| The Azure Blob Storage secret |
|
845
|
+
| GOOGLE_ID | The project ID for the Blob Storage account |
|
846
|
+
| GOOGLE_KEY | The content of the secret JSON file in 1 line |
|
847
|
+
|
848
|
+
After setting up the .env file, you can now run the command `bundle exec rspec ./integration`.
|
849
|
+
|
850
|
+
Alternatively, the gitlab ci has prepared a set of blob storage account online to run integration test. You can
|
851
|
+
activate the test by pushing a commit to your branch if setting up the environment it too complicated.
|
852
|
+
|
853
|
+
To install this gem onto your local machine, run `bundle exec rake install`.
|
854
|
+
## Contributing
|
855
|
+
|
856
|
+
Bug reports and pull requests are welcome on GitLab at
|
857
|
+
https://gitlab.com/ruby-gem/bucket_client.
|
858
|
+
This project is intended to be a safe, welcoming space for collaboration,
|
859
|
+
and contributors are expected to adhere to the
|
860
|
+
[Contributor Covenant](http://contributor-covenant.org) code of conduct.
|
861
|
+
|
862
|
+
## License
|
863
|
+
|
864
|
+
The gem is available as open source under the terms of the [MIT License](LICENSE.txt).
|
865
|
+
|
866
|
+
## Code of Conduct
|
867
|
+
|
868
|
+
Everyone interacting in the BucketClient project’s codebases, issue trackers,
|
869
|
+
chat rooms and mailing lists is expected to follow the
|
870
|
+
[code of conduct](CODE_OF_CONDUCT.md).
|