azure-blob 0.4.1 → 0.4.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 20f2d1c40dbc979782d017b025e5369f21e8ec9c1de02e50ef030648b8fd557b
4
- data.tar.gz: d818c311fb5a55db068d1b6d7e8d112d0fc44aa5f862144ec789ce3e3cd97a99
3
+ metadata.gz: 7eab2543de0e2e4663211095fd459507761bf270038308464723270506f63ac5
4
+ data.tar.gz: 1c0b7016bf21df9c1134be50eb2555f14f9f4b76e9a8664e70e003cee5b04208
5
5
  SHA512:
6
- metadata.gz: 9f58536a8295aa300c0be73a4654e2a888a51db9c7acc0bb793297c3878c39947aef7bf25c4231ab30e2ee764bd65a3f5c5863d0a2b5b555bf9c7bd976abeb13
7
- data.tar.gz: ae0f1e93eaff5c7f196fa78b6cf230f6be2a3c05cb4ba9635eb19828539cf381c8c23415453678ea19e790d860b65180260e9175ad1c825cb904047b09893047
6
+ metadata.gz: 6f03e61001c0c41ed31661116d5f7ed5d5f95f91e8ded26d40aa72d446119e835c281d852904e00b02f68fd536d0451e4e71401410ab41ec68f7ff97fb030a88
7
+ data.tar.gz: b0b1beb93ad7b3950bddb41dfb6e04354e08b46e29f5f3db5d474ddd4964cd09d65776a1f57625d103d1e56e9a0d235579372c74a867fa2080beabcfc757395f
data/CHANGELOG.md CHANGED
@@ -1,5 +1,12 @@
1
1
  ## [Unreleased]
2
2
 
3
- ## [0.1.0] - 2024-04-05
3
+ ## [0.4.2] 2024-06-06
4
4
 
5
- - Initial release
5
+ Documentation
6
+
7
+ ## [0.4.1] 2024-05-27
8
+
9
+ First working release.
10
+
11
+ - Re-implemented the required parts of the azure-storage-blob API to make Active Storage work.
12
+ - Extracted the AzureStorage adapter from Rails.
data/README.md CHANGED
@@ -2,9 +2,45 @@
2
2
 
3
3
  This gem was built to replace azure-storage-blob (deprecated) in Active Storage, but was written to be Rails agnostic.
4
4
 
5
+ ## Active Storage
6
+
7
+ ## Migration
8
+ To migrate from azure-storage-blob to azure-blob:
9
+
10
+ 1. Replace `azure-storage-blob` in your Gemfile with `azure-blob`
11
+ 2. Run `bundle install`
12
+ 3. Change the `AzureStorage` service to `AzureBlob` in your Active Storage config (`config/storage.yml`)
13
+ 4. Restart or deploy the app.
14
+
15
+
16
+ ## Standalone
17
+
18
+ Instantiate a client with your account name, an access key and the container name:
19
+
20
+ ```ruby
21
+ client = AzureBlob::Client.new(
22
+ account_name: @account_name,
23
+ access_key: @access_key,
24
+ container: @container,
25
+ )
26
+
27
+ path = "some/new/file"
28
+
29
+ # Upload
30
+ client.create_block_blob(path, "Hello world!")
31
+
32
+ # Download
33
+ client.get_blob(path) #=> "Hello world!"
34
+
35
+ # Delete
36
+ client.delete_blob(path)
37
+ ```
38
+
39
+ For the full list of methods: https://www.rubydoc.info/gems/azure-blob/AzureBlob/Client
40
+
5
41
  ## Contributing
6
42
 
7
- ### dev environment
43
+ ### Dev environment
8
44
 
9
45
  Ensure your version of Ruby fit the minimum version in `azure-blob.gemspec`
10
46
 
@@ -19,24 +55,14 @@ and setup those Env variables:
19
55
  A dev environment setup is also supplied through Nix with [devenv](https://devenv.sh/).
20
56
 
21
57
  To use the Nix environment:
22
- 1- install [devenv](https://devenv.sh/)
23
- 2- Copy `devenv.local.nix.example` to `devenv.local.nix`
24
- 3- Insert your azure credentials into `devenv.local.nix`
25
- 4- Start the shell with `devenv shell` or with [direnv](https://direnv.net/).
58
+ 1. install [devenv](https://devenv.sh/)
59
+ 2. Copy `devenv.local.nix.example` to `devenv.local.nix`
60
+ 3. Insert your azure credentials into `devenv.local.nix`
61
+ 4. Start the shell with `devenv shell` or with [direnv](https://direnv.net/).
26
62
 
27
63
  ### Tests
28
64
 
29
- `bin/rake test`.
30
-
31
- # Active Storage
32
-
33
- ## Migration
34
- To migrate from azure-storage-blob to azure-blob:
35
-
36
- 1- Replace `azure-storage-blob` in your Gemfile with `azure-blob`
37
- 2- Run `bundle install`
38
- 3- change the `AzureStorage` service to `AzureBlob` in your Active Storage config (`config/storage.yml`)
39
- 4- Restart or deploy the app.
65
+ `bin/rake test`
40
66
 
41
67
  ## License
42
68
 
data/Rakefile CHANGED
@@ -2,7 +2,21 @@
2
2
 
3
3
  require "bundler/gem_tasks"
4
4
  require "minitest/test_task"
5
+ require 'azure_blob'
5
6
 
6
7
  Minitest::TestTask.create
7
8
 
8
9
  task default: %i[test]
10
+
11
+ task :flush_test_container do |t|
12
+ AzureBlob::Client.new(
13
+ account_name: ENV["AZURE_ACCOUNT_NAME"],
14
+ access_key: ENV["AZURE_ACCESS_KEY"],
15
+ container: ENV["AZURE_PRIVATE_CONTAINER"],
16
+ ).delete_prefix ''
17
+ AzureBlob::Client.new(
18
+ account_name: ENV["AZURE_ACCOUNT_NAME"],
19
+ access_key: ENV["AZURE_ACCESS_KEY"],
20
+ container: ENV["AZURE_PUBLIC_CONTAINER"],
21
+ ).delete_prefix ''
22
+ end
data/azure-blob.gemspec CHANGED
@@ -9,13 +9,13 @@ Gem::Specification.new do |spec|
9
9
  spec.email = [ "joe@dupuis.io" ]
10
10
 
11
11
  spec.summary = "Azure blob client"
12
- spec.homepage = "https://github.com/JoeDupuis/azure-blob"
12
+ spec.homepage = "https://github.com/testdouble/azure-blob"
13
13
  spec.license = "MIT"
14
14
  spec.required_ruby_version = ">= 3.1"
15
15
 
16
16
  spec.metadata["homepage_uri"] = spec.homepage
17
17
  spec.metadata["source_code_uri"] = spec.homepage
18
- spec.metadata["changelog_uri"] = "https://github.com/JoeDupuis/azure-blob/blob/main/CHANGELOG.md"
18
+ spec.metadata["changelog_uri"] = "https://github.com/testdouble/azure-blob/blob/main/CHANGELOG.md"
19
19
 
20
20
  spec.add_dependency "rexml"
21
21
 
@@ -25,13 +25,13 @@
25
25
  require "active_support/core_ext/numeric/bytes"
26
26
  require "active_storage/service"
27
27
 
28
- require 'azure_blob'
28
+ require "azure_blob"
29
29
 
30
30
  module ActiveStorage
31
- # = Active Storage \Azure Storage \Service
31
+ # = Active Storage \Azure Blob \Service
32
32
  #
33
33
  # Wraps the Microsoft Azure Storage Blob Service as an Active Storage service.
34
- # See ActiveStorage::Service for the generic API documentation that applies to all services.
34
+ # See {ActiveStorage::Service}[https://api.rubyonrails.org/classes/ActiveStorage/Service.html] for more details.
35
35
  class Service::AzureBlobService < Service
36
36
  attr_reader :client, :container, :signer
37
37
 
@@ -1,7 +1,13 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module AzureBlob
4
+ # AzureBlob::Blob holds the metada for a given Blob.
4
5
  class Blob
6
+ # You should not instanciate this object directly,
7
+ # but obtain one when calling relevant methods of AzureBlob::Client.
8
+ #
9
+ # Expects a Net::HTTPResponse object from a
10
+ # HEAD or GET request to a blob uri.
5
11
  def initialize(response)
6
12
  @response = response
7
13
  end
@@ -26,6 +32,7 @@ module AzureBlob
26
32
  response.code == "200"
27
33
  end
28
34
 
35
+ # Returns the custom Azure metada tagged on the blob.
29
36
  def metadata
30
37
  @metadata || response
31
38
  .to_hash
@@ -3,10 +3,28 @@
3
3
  require "rexml"
4
4
 
5
5
  module AzureBlob
6
+ # Enumerator class to lazily iterate over a list of Blob keys.
6
7
  class BlobList
7
8
  include REXML
8
9
  include Enumerable
9
10
 
11
+ # You should not instanciate this object directly,
12
+ # but obtain one when calling relevant methods of AzureBlob::Client.
13
+ #
14
+ # Expects a callable object that takes an Azure API page marker as an
15
+ # argument and returns the raw body response of a call to the list blob endpoint.
16
+ #
17
+ # Example:
18
+ #
19
+ # fetcher = ->(marker) do
20
+ # uri.query = URI.encode_www_form(
21
+ # marker: marker,
22
+ # ...
23
+ # )
24
+ # response = Http.new(uri, signer:).get
25
+ # end
26
+ # AzureBlob::BlobList.new(fetcher)
27
+ #
10
28
  def initialize(fetcher)
11
29
  @fetcher = fetcher
12
30
  end
@@ -3,7 +3,9 @@
3
3
  require "rexml"
4
4
 
5
5
  module AzureBlob
6
- class BlockList
6
+ class BlockList # :nodoc:
7
+ # Internal
8
+ # BlockList builds the XML list of blocks to commit to a blob
7
9
  include REXML
8
10
  def initialize(blocks)
9
11
  @blocks = blocks
@@ -1,5 +1,5 @@
1
1
  module AzureBlob
2
- class CanonicalizedHeaders
2
+ class CanonicalizedHeaders # :nodoc:
3
3
  STANDARD_HEADERS = [
4
4
  :"x-ms-version",
5
5
  ]
@@ -1,7 +1,7 @@
1
1
  require "cgi"
2
2
 
3
3
  module AzureBlob
4
- class CanonicalizedResource
4
+ class CanonicalizedResource # :nodoc:
5
5
  def initialize(uri, account_name, service_name: nil, url_safe: true)
6
6
  # This next line is needed because CanonicalizedResource
7
7
  # need to be escaped for auhthorization headers, but not SAS tokens
@@ -9,6 +9,8 @@ require "time"
9
9
  require "base64"
10
10
 
11
11
  module AzureBlob
12
+ # AzureBlob Client class. You interact with the Azure Blob api
13
+ # through an instance of this class.
12
14
  class Client
13
15
  def initialize(account_name:, access_key:, container:)
14
16
  @account_name = account_name
@@ -16,6 +18,27 @@ module AzureBlob
16
18
  @signer = Signer.new(account_name:, access_key:)
17
19
  end
18
20
 
21
+ # Create a blob of type block. Will automatically split the the blob in multiple block and send the blob in pieces (blocks) if the blob is too big.
22
+ #
23
+ # When the blob is small enough this method will send the blob through {Put Blob}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-blob]
24
+ #
25
+ # If the blob is too big, the blob is split in blocks sent through a series of {Put Block}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-block] requests
26
+ # followed by a {Put Block List}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-block-list] to commit the block list.
27
+ #
28
+ # Takes a key (path), the content (String or IO object), and options.
29
+ #
30
+ # Options:
31
+ #
32
+ # [+:content_type+]
33
+ # Will be saved on the blob in Azure.
34
+ # [+:content_disposition+]
35
+ # Will be saved on the blob in Azure.
36
+ # [+:content_md5+]
37
+ # Will ensure integrity of the upload. The checksum must be a base64 digest. Can be produced with +OpenSSL::Digest::MD5.base64digest+.
38
+ # The checksum is only checked on a single upload! To verify checksum when uploading multiple blocks, call directly put_blob_block with
39
+ # a checksum for each block, then commit the blocks with commit_blob_blocks.
40
+ # [+:block_size+]
41
+ # Block size in bytes, can be used to force the method to split the upload in smaller chunk. Defaults to +AzureBlob::DEFAULT_BLOCK_SIZE+ and cannot be bigger than +AzureBlob::MAX_UPLOAD_SIZE+
19
42
  def create_block_blob(key, content, options = {})
20
43
  if content.size > (options[:block_size] || DEFAULT_BLOCK_SIZE)
21
44
  put_blob_multiple(key, content, **options)
@@ -24,6 +47,18 @@ module AzureBlob
24
47
  end
25
48
  end
26
49
 
50
+ # Returns the full or partial content of the blob
51
+ #
52
+ # Calls to the {Get Blob}[https://learn.microsoft.com/en-us/rest/api/storageservices/get-blob] endpoint.
53
+ #
54
+ # Takes a key (path) and options.
55
+ #
56
+ # Options:
57
+ #
58
+ # [+:start+]
59
+ # Starting point in bytes
60
+ # [+:end+]
61
+ # Ending point in bytes
27
62
  def get_blob(key, options = {})
28
63
  uri = generate_uri("#{container}/#{key}")
29
64
 
@@ -34,6 +69,15 @@ module AzureBlob
34
69
  Http.new(uri, headers, signer:).get
35
70
  end
36
71
 
72
+ # Delete a blob
73
+ #
74
+ # Calls to {Delete Blob}[https://learn.microsoft.com/en-us/rest/api/storageservices/delete-blob]
75
+ #
76
+ # Takes a key (path) and options.
77
+ #
78
+ # Options:
79
+ # [+:delete_snapshots+]
80
+ # Sets the value of the x-ms-delete-snapshots header. Default to +include+
37
81
  def delete_blob(key, options = {})
38
82
  uri = generate_uri("#{container}/#{key}")
39
83
 
@@ -44,11 +88,28 @@ module AzureBlob
44
88
  Http.new(uri, headers, signer:).delete
45
89
  end
46
90
 
91
+ # Delete all blobs prefixed by the given prefix.
92
+ #
93
+ # Calls to {List blobs}[https://learn.microsoft.com/en-us/rest/api/storageservices/list-blobs]
94
+ # followed to a series of calls to {Delete Blob}[https://learn.microsoft.com/en-us/rest/api/storageservices/delete-blob]
95
+ #
96
+ # Takes a prefix and options
97
+ #
98
+ # Look delete_blob for the list of options.
47
99
  def delete_prefix(prefix, options = {})
48
100
  results = list_blobs(prefix:)
49
101
  results.each { |key| delete_blob(key) }
50
102
  end
51
103
 
104
+ # Returns a BlobList containing a list of keys (paths)
105
+ #
106
+ # Calls to {List blobs}[https://learn.microsoft.com/en-us/rest/api/storageservices/list-blobs]
107
+ #
108
+ # Options:
109
+ # [+:prefix+]
110
+ # Prefix of the blobs to be listed. Defaults to listing everything in the container.
111
+ # [:+max_results+]
112
+ # Maximum number of results to return per page.
52
113
  def list_blobs(options = {})
53
114
  uri = generate_uri(container)
54
115
  query = {
@@ -69,6 +130,11 @@ module AzureBlob
69
130
  BlobList.new(fetcher)
70
131
  end
71
132
 
133
+ # Returns a Blob object without the content.
134
+ #
135
+ # Calls to {Get Blob Properties}[https://learn.microsoft.com/en-us/rest/api/storageservices/get-blob-properties]
136
+ #
137
+ # This can be used to see if the blob exist or obtain metada such as content type, disposition, checksum or Azure custom metadata.
72
138
  def get_blob_properties(key, options = {})
73
139
  uri = generate_uri("#{container}/#{key}")
74
140
 
@@ -77,16 +143,37 @@ module AzureBlob
77
143
  Blob.new(response)
78
144
  end
79
145
 
146
+ # Return a URI object to a resource in the container. Takes a path.
147
+ #
148
+ # Example: +generate_uri("#{container}/#{key}")+
80
149
  def generate_uri(path)
81
150
  URI.parse(URI::DEFAULT_PARSER.escape(File.join(host, path)))
82
151
  end
83
152
 
153
+ # Returns an SAS signed URI
154
+ #
155
+ # Takes a
156
+ # - key (path)
157
+ # - A permission string (+"r"+, +"rw"+)
158
+ # - expiry as a UTC iso8601 time string
159
+ # - options
84
160
  def signed_uri(key, permissions:, expiry:, **options)
85
161
  uri = generate_uri("#{container}/#{key}")
86
162
  uri.query = signer.sas_token(uri, permissions:, expiry:, **options)
87
163
  uri
88
164
  end
89
165
 
166
+ # Creates a Blob of type append.
167
+ #
168
+ # Calls to {Put Blob}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-blob]
169
+ #
170
+ # You are expected to append blocks to the blob with append_blob_block after creating the blob.
171
+ # Options:
172
+ #
173
+ # [+:content_type+]
174
+ # Will be saved on the blob in Azure.
175
+ # [+:content_disposition+]
176
+ # Will be saved on the blob in Azure.
90
177
  def create_append_blob(key, options = {})
91
178
  uri = generate_uri("#{container}/#{key}")
92
179
 
@@ -101,6 +188,15 @@ module AzureBlob
101
188
  Http.new(uri, headers, metadata: options[:metadata], signer:).put(nil)
102
189
  end
103
190
 
191
+ # Append a block to an Append Blob
192
+ #
193
+ # Calls to {Append Block}[https://learn.microsoft.com/en-us/rest/api/storageservices/append-block]
194
+ #
195
+ # Options:
196
+ #
197
+ # [+:content_md5+]
198
+ # Will ensure integrity of the upload. The checksum must be a base64 digest. Can be produced with +OpenSSL::Digest::MD5.base64digest+.
199
+ # The checksum must be the checksum of the block not the blob.
104
200
  def append_blob_block(key, content, options = {})
105
201
  uri = generate_uri("#{container}/#{key}")
106
202
  uri.query = URI.encode_www_form(comp: "appendblock")
@@ -114,6 +210,16 @@ module AzureBlob
114
210
  Http.new(uri, headers, signer:).put(content)
115
211
  end
116
212
 
213
+ # Uploads a block to a blob.
214
+ #
215
+ # Calls to {Put Block}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-block]
216
+ #
217
+ # Returns the id of the block. Required to commit the list of blocks to a blob.
218
+ #
219
+ # Options:
220
+ #
221
+ # [+:content_md5+]
222
+ # Must be the checksum for the block not the blob. The checksum must be a base64 digest. Can be produced with +OpenSSL::Digest::MD5.base64digest+.
117
223
  def put_blob_block(key, index, content, options = {})
118
224
  block_id = generate_block_id(index)
119
225
  uri = generate_uri("#{container}/#{key}")
@@ -130,6 +236,17 @@ module AzureBlob
130
236
  block_id
131
237
  end
132
238
 
239
+ # Commits the list of blocks to a blob.
240
+ #
241
+ # Calls to {Put Block List}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-block-list]
242
+ #
243
+ # Takes a key (path) and an array of block ids
244
+ #
245
+ # Options:
246
+ #
247
+ # [+:content_md5+]
248
+ # This is the checksum for the whole blob. The checksum is saved on the blob, but it is not validated!
249
+ # Add a checksum for each block if you want Azure to validate integrity.
133
250
  def commit_blob_blocks(key, block_ids, options = {})
134
251
  block_list = BlockList.new(block_ids)
135
252
  content = block_list.to_s
@@ -139,7 +256,7 @@ module AzureBlob
139
256
  headers = {
140
257
  "Content-Length": content.size,
141
258
  "Content-Type": options[:content_type],
142
- "Content-MD5": options[:content_md5],
259
+ "x-ms-blob-content-md5": options[:content_md5],
143
260
  "x-ms-blob-content-disposition": options[:content_disposition],
144
261
  }
145
262
 
@@ -171,7 +288,7 @@ module AzureBlob
171
288
  "x-ms-blob-type": "BlockBlob",
172
289
  "Content-Length": content.size,
173
290
  "Content-Type": options[:content_type],
174
- "Content-MD5": options[:content_md5],
291
+ "x-ms-blob-content-md5": options[:content_md5],
175
292
  "x-ms-blob-content-disposition": options[:content_disposition],
176
293
  }
177
294
 
@@ -6,7 +6,7 @@ require "net/http"
6
6
  require "rexml"
7
7
 
8
8
  module AzureBlob
9
- class Http
9
+ class Http # :nodoc:
10
10
  class Error < AzureBlob::Error; end
11
11
  class FileNotFoundError < Error; end
12
12
  class ForbidenError < Error; end
@@ -1,5 +1,5 @@
1
1
  module AzureBlob
2
- class Metadata
2
+ class Metadata # :nodoc:
3
3
  def initialize(metadata = nil)
4
4
  @metadata = metadata || {}
5
5
  @headers = @metadata.map do |key, value|
@@ -6,7 +6,7 @@ require_relative "canonicalized_headers"
6
6
  require_relative "canonicalized_resource"
7
7
 
8
8
  module AzureBlob
9
- class Signer
9
+ class Signer # :nodoc:
10
10
  def initialize(account_name:, access_key:)
11
11
  @account_name = account_name
12
12
  @access_key = Base64.decode64(access_key)
@@ -83,9 +83,9 @@ module AzureBlob
83
83
  headers
84
84
  end
85
85
 
86
- module SAS
86
+ module SAS # :nodoc:
87
87
  Version = "2024-05-04"
88
- module Fields
88
+ module Fields # :nodoc:
89
89
  Permissions = :sp
90
90
  Version = :sv
91
91
  Expiry = :se
@@ -94,7 +94,7 @@ module AzureBlob
94
94
  Disposition = :rscd
95
95
  Type = :rsct
96
96
  end
97
- module Resources
97
+ module Resources # :nodoc:
98
98
  Blob = :b
99
99
  end
100
100
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module AzureBlob
4
- VERSION = "0.4.1"
4
+ VERSION = "0.4.2"
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: azure-blob
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.1
4
+ version: 0.4.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Joé Dupuis
@@ -57,13 +57,13 @@ files:
57
57
  - lib/azure_blob/metadata.rb
58
58
  - lib/azure_blob/signer.rb
59
59
  - lib/azure_blob/version.rb
60
- homepage: https://github.com/JoeDupuis/azure-blob
60
+ homepage: https://github.com/testdouble/azure-blob
61
61
  licenses:
62
62
  - MIT
63
63
  metadata:
64
- homepage_uri: https://github.com/JoeDupuis/azure-blob
65
- source_code_uri: https://github.com/JoeDupuis/azure-blob
66
- changelog_uri: https://github.com/JoeDupuis/azure-blob/blob/main/CHANGELOG.md
64
+ homepage_uri: https://github.com/testdouble/azure-blob
65
+ source_code_uri: https://github.com/testdouble/azure-blob
66
+ changelog_uri: https://github.com/testdouble/azure-blob/blob/main/CHANGELOG.md
67
67
  post_install_message:
68
68
  rdoc_options: []
69
69
  require_paths: