azure-blob 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +22 -2
- data/README.md +98 -0
- data/lib/azure_blob/blob.rb +9 -0
- data/lib/azure_blob/client.rb +109 -19
- data/lib/azure_blob/entra_id_signer.rb +21 -3
- data/lib/azure_blob/http.rb +30 -25
- data/lib/azure_blob/user_delegation_key.rb +25 -7
- data/lib/azure_blob/version.rb +1 -1
- metadata +4 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 803bd81be45a0ba597d53d7967864d4bc79606e239fc29e85b2c4f31b21e3b34
|
|
4
|
+
data.tar.gz: 99f58e80bac3ce0c670ef8ccc86b7673a40c22711ebb77bbf34196ce84c55fa5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a4aa69a0ee49c7a5c0d11122ec7519c9d947e04739d34a27f741ca573c9c3867acd9bb05a34c31cec8e560a703e86ff1ab5427eeefa48c05d9a4d1e6decadd56
|
|
7
|
+
data.tar.gz: 65159fa66b8649f0dba35b8a3d5d7ad3d76f32b65f03128fe100c6df680d03288333b28ae50712e76bcdb19a872a85fe19fdaff23923d8dbc909f603980ee6ed
|
data/CHANGELOG.md
CHANGED
|
@@ -1,10 +1,30 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
-
## [0.
|
|
3
|
+
## [0.10.0] 2026-11-22
|
|
4
|
+
|
|
5
|
+
- Add persistent connections: HTTP connections are kept open between requests when the `net-http-persistent` gem is loaded (#53)
|
|
6
|
+
- Add optional `connection_pool` to `AzureBlob::Client.new` and the ActiveStorage config to send requests through a custom connection pool
|
|
7
|
+
|
|
8
|
+
## [0.9.0] 2026-10-11
|
|
9
|
+
|
|
10
|
+
- Drop support for Ruby 3.1 and 3.2. Ruby 3.3 or newer is now required
|
|
11
|
+
- Drop support for Rails 7.x. The Active Storage service now supports Rails 8.0 or newer
|
|
12
|
+
- Fix `copy_blob` failing on append blobs and on blobs larger than 256 MiB
|
|
13
|
+
- Fix `compose` failing when its source is a composed blob (#48)
|
|
14
|
+
- Add `put_blob_block_from_url`
|
|
15
|
+
- Add `AzureBlob::Blob#blob_type` and `AzureBlob::Blob#block_blob?`
|
|
16
|
+
- Fix the content type being dropped when a blob is uploaded in multiple blocks
|
|
17
|
+
- Add optional `delegation_key_expiration` to `AzureBlob::Client.new` and the ActiveStorage config to set the minimum lifetime of the Entra ID user delegation key in seconds (default 7 hours, maximum 7 days)
|
|
18
|
+
- Fix Entra ID signed URLs that stopped working before their expiry because the user delegation key expired first
|
|
19
|
+
- Raise `ArgumentError` when signing an Entra ID URL that expires more than 7 days from now
|
|
20
|
+
- Include the HTTP status and Azure error code in `AzureBlob::Http::Error` messages (e.g. `500 InternalError`) and add `AzureBlob::Http::Error#code`
|
|
21
|
+
- Fix error responses without an XML body raising `NoMethodError` instead of `AzureBlob::Http::Error`, which skipped the managed identity token retries
|
|
22
|
+
|
|
23
|
+
## [0.8.0] 2026-01-18
|
|
4
24
|
|
|
5
25
|
- Add cgi gem dependency for Ruby 4.0 compatibility
|
|
6
26
|
|
|
7
|
-
## [0.7.0]
|
|
27
|
+
## [0.7.0] 2026-01-05
|
|
8
28
|
|
|
9
29
|
- Add optional `timeout` parameter to all API methods
|
|
10
30
|
- Add `AzureBlob::Http::TimeoutError` for handling Azure `OperationTimedOut` errors
|
data/README.md
CHANGED
|
@@ -4,6 +4,8 @@ Azure Blob client and Active Storage adapter to replace the now abandoned azure-
|
|
|
4
4
|
|
|
5
5
|
An Active Storage is supplied, but the gem is Rails agnostic and can be used in any Ruby project.
|
|
6
6
|
|
|
7
|
+
Requires Ruby 3.3 or newer. The Active Storage service supports Rails 8.0 or newer.
|
|
8
|
+
|
|
7
9
|
## Active Storage
|
|
8
10
|
|
|
9
11
|
### Migration
|
|
@@ -45,6 +47,35 @@ prod:
|
|
|
45
47
|
principal_id: 71b34410-4c50-451d-b456-95ead1b18cce
|
|
46
48
|
```
|
|
47
49
|
|
|
50
|
+
#### User delegation key expiration
|
|
51
|
+
|
|
52
|
+
When authenticating through Entra ID, signed URLs are signed with a user delegation key
|
|
53
|
+
that AzureBlob requests and renews as needed. `delegation_key_expiration` (in seconds)
|
|
54
|
+
sets how long each key lasts: 7 hours by default, up to Azure's maximum of 7 days. Set it
|
|
55
|
+
above the longest URL lifetime your app issues.
|
|
56
|
+
|
|
57
|
+
```ruby
|
|
58
|
+
client = AzureBlob::Client.new(
|
|
59
|
+
account_name: "account_name",
|
|
60
|
+
container: "container_name",
|
|
61
|
+
principal_id: "71b34410-4c50-451d-b456-95ead1b18cce",
|
|
62
|
+
delegation_key_expiration: 86400, # 1 day
|
|
63
|
+
)
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
ActiveStorage config example:
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
prod:
|
|
70
|
+
service: AzureBlob
|
|
71
|
+
container: container_name
|
|
72
|
+
storage_account_name: account_name
|
|
73
|
+
principal_id: 71b34410-4c50-451d-b456-95ead1b18cce
|
|
74
|
+
delegation_key_expiration: 86400 # 1 day
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
The option is ignored when authenticating with a shared key.
|
|
78
|
+
|
|
48
79
|
#### AKS with Workload Identity
|
|
49
80
|
|
|
50
81
|
ActiveStorage config example:
|
|
@@ -121,6 +152,73 @@ To enable lazy loading and ignore missing credentials, set the `lazy` option:
|
|
|
121
152
|
|
|
122
153
|
or add `lazy: true` to your `config/storage.yml` for Active Storage.
|
|
123
154
|
|
|
155
|
+
### Persistent connections
|
|
156
|
+
|
|
157
|
+
By default, every request opens a new connection to Azure and closes it afterward. Persistent connections stay open and are reused, which saves a TCP and TLS handshake on every request.
|
|
158
|
+
|
|
159
|
+
This relies on the [net-http-persistent](https://github.com/drbrain/net-http-persistent) gem. If you want to activate persistent connections, add it to your Gemfile:
|
|
160
|
+
|
|
161
|
+
```ruby
|
|
162
|
+
gem "net-http-persistent"
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Rails requires it automatically. To use it outside of Rails, set the `persistent_connections` option to `true` on the client:
|
|
166
|
+
|
|
167
|
+
```ruby
|
|
168
|
+
client = AzureBlob::Client.new(
|
|
169
|
+
account_name: "account_name",
|
|
170
|
+
access_key: "SECRET_KEY",
|
|
171
|
+
container: "container_name",
|
|
172
|
+
persistent_connections: true,
|
|
173
|
+
)
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
If your app uses net-http-persistent but you do not want azure-blob to use it, set `persistent_connections: false` on the client or in your `config/storage.yml`.
|
|
177
|
+
|
|
178
|
+
#### Custom connection pool
|
|
179
|
+
|
|
180
|
+
To tune the connections (pool size, timeouts, etc.), build your own pool and pass it as `connection_pool`:
|
|
181
|
+
|
|
182
|
+
```ruby
|
|
183
|
+
connection_pool = Net::HTTP::Persistent.new(name: "azure-blob", pool_size: 10)
|
|
184
|
+
connection_pool.idle_timeout = 30
|
|
185
|
+
|
|
186
|
+
client = AzureBlob::Client.new(
|
|
187
|
+
account_name: "account_name",
|
|
188
|
+
access_key: "SECRET_KEY",
|
|
189
|
+
container: "container_name",
|
|
190
|
+
connection_pool: connection_pool,
|
|
191
|
+
)
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
`connection_pool` also takes a class, or the name of a class, and calls `new` on it. This is how to set a pool from `config/storage.yml`:
|
|
195
|
+
|
|
196
|
+
```ruby
|
|
197
|
+
class BlobConnectionPool < Net::HTTP::Persistent
|
|
198
|
+
def initialize
|
|
199
|
+
super(name: "azure-blob", pool_size: 10)
|
|
200
|
+
self.idle_timeout = 30
|
|
201
|
+
end
|
|
202
|
+
end
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
```yaml
|
|
206
|
+
microsoft:
|
|
207
|
+
service: AzureBlob
|
|
208
|
+
storage_account_name: account_name
|
|
209
|
+
storage_access_key: SECRET_KEY
|
|
210
|
+
container: container_name
|
|
211
|
+
connection_pool: BlobConnectionPool
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
A pool does not have to be a `Net::HTTP::Persistent`. It can be any object that:
|
|
215
|
+
|
|
216
|
+
- responds to `request(uri, request)`, where `uri` is a `URI` and `request` a `Net::HTTPRequest`
|
|
217
|
+
- returns the `Net::HTTPResponse`
|
|
218
|
+
- is safe to call from several threads at once
|
|
219
|
+
|
|
220
|
+
A `connection_pool` is used whatever `persistent_connections` is set to.
|
|
221
|
+
|
|
124
222
|
|
|
125
223
|
## Contributing
|
|
126
224
|
|
data/lib/azure_blob/blob.rb
CHANGED
|
@@ -28,6 +28,15 @@ module AzureBlob
|
|
|
28
28
|
response.content_length
|
|
29
29
|
end
|
|
30
30
|
|
|
31
|
+
# Returns the blob type: +"BlockBlob"+, +"AppendBlob"+ or +"PageBlob"+.
|
|
32
|
+
def blob_type
|
|
33
|
+
response["x-ms-blob-type"]
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def block_blob?
|
|
37
|
+
blob_type == "BlockBlob"
|
|
38
|
+
end
|
|
39
|
+
|
|
31
40
|
def present?
|
|
32
41
|
response.code == "200"
|
|
33
42
|
end
|
data/lib/azure_blob/client.rb
CHANGED
|
@@ -24,6 +24,8 @@ module AzureBlob
|
|
|
24
24
|
@access_key = access_key
|
|
25
25
|
@principal_id = principal_id
|
|
26
26
|
@use_managed_identities = options[:use_managed_identities]
|
|
27
|
+
@delegation_key_expiration = options[:delegation_key_expiration]
|
|
28
|
+
@connection_pool = build_connection_pool(options[:connection_pool], persistent_connections: options[:persistent_connections])
|
|
27
29
|
signer unless options[:lazy]
|
|
28
30
|
end
|
|
29
31
|
|
|
@@ -76,12 +78,18 @@ module AzureBlob
|
|
|
76
78
|
"x-ms-range": options[:start] && "bytes=#{options[:start]}-#{options[:end]}",
|
|
77
79
|
}.merge(additional_headers(options))
|
|
78
80
|
|
|
79
|
-
Http.new(uri, headers, signer:).get
|
|
81
|
+
Http.new(uri, headers, signer:, connection_pool:).get
|
|
80
82
|
end
|
|
81
83
|
|
|
82
84
|
# Copy a blob between containers or within the same container
|
|
83
85
|
#
|
|
84
|
-
# Calls to {Copy Blob From URL}[https://learn.microsoft.com/en-us/rest/api/storageservices/copy-blob-from-url]
|
|
86
|
+
# Calls to {Copy Blob From URL}[https://learn.microsoft.com/en-us/rest/api/storageservices/copy-blob-from-url],
|
|
87
|
+
# which copies a block blob of at most 256 MiB in a single request. When Azure refuses the source
|
|
88
|
+
# (an append blob, or a bigger block blob) the blob is copied block by block with
|
|
89
|
+
# {Put Block From URL}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-block-from-url] instead.
|
|
90
|
+
#
|
|
91
|
+
# Either way the copy is a block blob that keeps the content type, content disposition and checksum
|
|
92
|
+
# of the source, and its metadata unless +:metadata+ is given.
|
|
85
93
|
#
|
|
86
94
|
# Parameters:
|
|
87
95
|
# - key: destination blob path
|
|
@@ -89,9 +97,11 @@ module AzureBlob
|
|
|
89
97
|
# - options: additional options
|
|
90
98
|
# - source_client: AzureBlob::Client instance for the source container (optional)
|
|
91
99
|
# If not provided, copies from within the same container
|
|
100
|
+
# - metadata: metadata of the copy (optional). Defaults to the metadata of the source
|
|
101
|
+
# - block_size: block size of a block by block copy. Defaults to +AzureBlob::DEFAULT_BLOCK_SIZE+
|
|
92
102
|
#
|
|
93
103
|
def copy_blob(key, source_key, options = {})
|
|
94
|
-
source_client = options
|
|
104
|
+
source_client = options[:source_client] || self
|
|
95
105
|
uri = generate_uri("#{container}/#{key}")
|
|
96
106
|
uri.query = URI.encode_www_form(timeout: options[:timeout]) if options[:timeout]
|
|
97
107
|
|
|
@@ -102,7 +112,10 @@ module AzureBlob
|
|
|
102
112
|
"x-ms-requires-sync": "true",
|
|
103
113
|
}.merge(additional_headers(options))
|
|
104
114
|
|
|
105
|
-
Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put
|
|
115
|
+
Http.new(uri, headers, signer:, connection_pool:, **options.slice(:metadata, :tags)).put
|
|
116
|
+
rescue Http::Error => error
|
|
117
|
+
raise unless error.status == 409
|
|
118
|
+
copy_blob_blocks(key, source_key, source_client, options)
|
|
106
119
|
end
|
|
107
120
|
|
|
108
121
|
# Delete a blob
|
|
@@ -122,7 +135,7 @@ module AzureBlob
|
|
|
122
135
|
"x-ms-delete-snapshots": options[:delete_snapshots] || "include",
|
|
123
136
|
}.merge(additional_headers(options))
|
|
124
137
|
|
|
125
|
-
Http.new(uri, headers, signer:).delete
|
|
138
|
+
Http.new(uri, headers, signer:, connection_pool:).delete
|
|
126
139
|
end
|
|
127
140
|
|
|
128
141
|
# Delete all blobs prefixed by the given prefix.
|
|
@@ -162,7 +175,7 @@ module AzureBlob
|
|
|
162
175
|
query[:marker] = marker
|
|
163
176
|
query.reject! { |key, value| value.to_s.empty? }
|
|
164
177
|
uri.query = URI.encode_www_form(**query)
|
|
165
|
-
response = Http.new(uri, additional_headers(options), signer:).get
|
|
178
|
+
response = Http.new(uri, additional_headers(options), signer:, connection_pool:).get
|
|
166
179
|
end
|
|
167
180
|
|
|
168
181
|
BlobList.new(fetcher)
|
|
@@ -178,7 +191,7 @@ module AzureBlob
|
|
|
178
191
|
uri = generate_uri("#{container}/#{key}")
|
|
179
192
|
uri.query = URI.encode_www_form(timeout: options[:timeout]) if options[:timeout]
|
|
180
193
|
|
|
181
|
-
response = Http.new(uri, additional_headers(options), signer:).head
|
|
194
|
+
response = Http.new(uri, additional_headers(options), signer:, connection_pool:).head
|
|
182
195
|
|
|
183
196
|
Blob.new(response)
|
|
184
197
|
end
|
|
@@ -204,7 +217,7 @@ module AzureBlob
|
|
|
204
217
|
query = { comp: "tags" }
|
|
205
218
|
query[:timeout] = options[:timeout] if options[:timeout]
|
|
206
219
|
uri.query = URI.encode_www_form(**query)
|
|
207
|
-
response = Http.new(uri, additional_headers(options), signer:).get
|
|
220
|
+
response = Http.new(uri, additional_headers(options), signer:, connection_pool:).get
|
|
208
221
|
|
|
209
222
|
Tags.from_response(response).to_h
|
|
210
223
|
end
|
|
@@ -219,7 +232,7 @@ module AzureBlob
|
|
|
219
232
|
query = { restype: "container" }
|
|
220
233
|
query[:timeout] = options[:timeout] if options[:timeout]
|
|
221
234
|
uri.query = URI.encode_www_form(**query)
|
|
222
|
-
response = Http.new(uri, additional_headers(options), signer:, raise_on_error: false).head
|
|
235
|
+
response = Http.new(uri, additional_headers(options), signer:, connection_pool:, raise_on_error: false).head
|
|
223
236
|
|
|
224
237
|
Container.new(response)
|
|
225
238
|
end
|
|
@@ -244,7 +257,7 @@ module AzureBlob
|
|
|
244
257
|
query = { restype: "container" }
|
|
245
258
|
query[:timeout] = options[:timeout] if options[:timeout]
|
|
246
259
|
uri.query = URI.encode_www_form(**query)
|
|
247
|
-
response = Http.new(uri, headers, signer:).put
|
|
260
|
+
response = Http.new(uri, headers, signer:, connection_pool:).put
|
|
248
261
|
end
|
|
249
262
|
|
|
250
263
|
# Delete the container
|
|
@@ -255,7 +268,7 @@ module AzureBlob
|
|
|
255
268
|
query = { restype: "container" }
|
|
256
269
|
query[:timeout] = options[:timeout] if options[:timeout]
|
|
257
270
|
uri.query = URI.encode_www_form(**query)
|
|
258
|
-
response = Http.new(uri, additional_headers(options), signer:).delete
|
|
271
|
+
response = Http.new(uri, additional_headers(options), signer:, connection_pool:).delete
|
|
259
272
|
end
|
|
260
273
|
|
|
261
274
|
# Return a URI object to a resource in the container. Takes a path.
|
|
@@ -306,7 +319,7 @@ module AzureBlob
|
|
|
306
319
|
"x-ms-blob-content-disposition": options[:content_disposition],
|
|
307
320
|
}.merge(additional_headers(options))
|
|
308
321
|
|
|
309
|
-
Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put(nil)
|
|
322
|
+
Http.new(uri, headers, signer:, connection_pool:, **options.slice(:metadata, :tags)).put(nil)
|
|
310
323
|
end
|
|
311
324
|
|
|
312
325
|
# Append a block to an Append Blob
|
|
@@ -330,7 +343,7 @@ module AzureBlob
|
|
|
330
343
|
"Content-MD5": options[:content_md5],
|
|
331
344
|
}.merge(additional_headers(options))
|
|
332
345
|
|
|
333
|
-
Http.new(uri, headers, signer:).put(content)
|
|
346
|
+
Http.new(uri, headers, signer:, connection_pool:).put(content)
|
|
334
347
|
end
|
|
335
348
|
|
|
336
349
|
# Uploads a block to a blob.
|
|
@@ -356,7 +369,36 @@ module AzureBlob
|
|
|
356
369
|
"Content-MD5": options[:content_md5],
|
|
357
370
|
}.merge(additional_headers(options))
|
|
358
371
|
|
|
359
|
-
Http.new(uri, headers, signer:).put(content)
|
|
372
|
+
Http.new(uri, headers, signer:, connection_pool:).put(content)
|
|
373
|
+
|
|
374
|
+
block_id
|
|
375
|
+
end
|
|
376
|
+
|
|
377
|
+
# Copies a block from a URL into a blob.
|
|
378
|
+
#
|
|
379
|
+
# Calls to {Put Block From URL}[https://learn.microsoft.com/en-us/rest/api/storageservices/put-block-from-url]
|
|
380
|
+
#
|
|
381
|
+
# Returns the id of the block. Required to commit the list of blocks to a blob.
|
|
382
|
+
#
|
|
383
|
+
# Options:
|
|
384
|
+
#
|
|
385
|
+
# [+:source_range+]
|
|
386
|
+
# Range of bytes to copy from the source, for example +0..(block_size - 1)+. Defaults to the whole source.
|
|
387
|
+
def put_blob_block_from_url(key, index, source_uri, options = {})
|
|
388
|
+
block_id = generate_block_id(index)
|
|
389
|
+
uri = generate_uri("#{container}/#{key}")
|
|
390
|
+
query = { comp: "block", blockid: block_id }
|
|
391
|
+
query[:timeout] = options[:timeout] if options[:timeout]
|
|
392
|
+
uri.query = URI.encode_www_form(**query)
|
|
393
|
+
|
|
394
|
+
source_range = options[:source_range]
|
|
395
|
+
headers = {
|
|
396
|
+
"Content-Length": 0,
|
|
397
|
+
"x-ms-copy-source": source_uri.to_s,
|
|
398
|
+
"x-ms-source-range": source_range && "bytes=#{source_range.min}-#{source_range.max}",
|
|
399
|
+
}.merge(additional_headers(options))
|
|
400
|
+
|
|
401
|
+
Http.new(uri, headers, signer:, connection_pool:).put
|
|
360
402
|
|
|
361
403
|
block_id
|
|
362
404
|
end
|
|
@@ -382,12 +424,12 @@ module AzureBlob
|
|
|
382
424
|
|
|
383
425
|
headers = {
|
|
384
426
|
"Content-Length": content_size(content),
|
|
385
|
-
"
|
|
427
|
+
"x-ms-blob-content-type": options[:content_type],
|
|
386
428
|
"x-ms-blob-content-md5": options[:content_md5],
|
|
387
429
|
"x-ms-blob-content-disposition": options[:content_disposition],
|
|
388
430
|
}.merge(additional_headers(options))
|
|
389
431
|
|
|
390
|
-
Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put(content)
|
|
432
|
+
Http.new(uri, headers, signer:, connection_pool:, **options.slice(:metadata, :tags)).put(content)
|
|
391
433
|
end
|
|
392
434
|
|
|
393
435
|
private
|
|
@@ -425,7 +467,35 @@ module AzureBlob
|
|
|
425
467
|
"x-ms-blob-content-disposition": options[:content_disposition],
|
|
426
468
|
}.merge(additional_headers(options))
|
|
427
469
|
|
|
428
|
-
Http.new(uri, headers, signer:, **options.slice(:metadata, :tags)).put(content.read)
|
|
470
|
+
Http.new(uri, headers, signer:, connection_pool:, **options.slice(:metadata, :tags)).put(content.read)
|
|
471
|
+
end
|
|
472
|
+
|
|
473
|
+
# One Put Block From URL request per block, then one Put Block List to commit them.
|
|
474
|
+
# Works with any source blob type and size, and carries the properties and metadata over itself.
|
|
475
|
+
def copy_blob_blocks(key, source_key, source_client, options)
|
|
476
|
+
source = source_client.get_blob_properties(source_key, options.slice(:timeout))
|
|
477
|
+
block_size = options[:block_size] || DEFAULT_BLOCK_SIZE
|
|
478
|
+
# Every block reads the source through this URI, so it has to outlive the whole copy.
|
|
479
|
+
source_uri = source_client.signed_uri(source_key, permissions: "r", expiry: Time.at(Time.now.to_i + 3600).utc.iso8601)
|
|
480
|
+
|
|
481
|
+
block_ids = (source.size.to_f / block_size).ceil.times.map do |index|
|
|
482
|
+
first = index * block_size
|
|
483
|
+
last = [ first + block_size, source.size ].min - 1
|
|
484
|
+
put_blob_block_from_url(key, index, source_uri, source_range: first..last, **options.slice(:timeout, :headers))
|
|
485
|
+
end
|
|
486
|
+
|
|
487
|
+
metadata = options[:metadata]
|
|
488
|
+
metadata = source.metadata if metadata.nil? || metadata.empty?
|
|
489
|
+
|
|
490
|
+
commit_blob_blocks(
|
|
491
|
+
key,
|
|
492
|
+
block_ids,
|
|
493
|
+
content_type: source.content_type,
|
|
494
|
+
content_md5: source.checksum,
|
|
495
|
+
content_disposition: source.content_disposition,
|
|
496
|
+
metadata: metadata,
|
|
497
|
+
**options.slice(:tags, :timeout, :headers),
|
|
498
|
+
)
|
|
429
499
|
end
|
|
430
500
|
|
|
431
501
|
def content_size(content)
|
|
@@ -440,6 +510,26 @@ module AzureBlob
|
|
|
440
510
|
@host ||= "https://#{account_name}.blob.#{CLOUD_REGIONS_SUFFIX[cloud_regions]}"
|
|
441
511
|
end
|
|
442
512
|
|
|
513
|
+
# Uses connection_pool when it is set: an object, a class or the name of a class.
|
|
514
|
+
# Otherwise uses persistent connections when net-http-persistent is available.
|
|
515
|
+
# Falls back to one connection per request when it is not, or when persistent_connections is false.
|
|
516
|
+
def build_connection_pool(connection_pool, persistent_connections:)
|
|
517
|
+
connection_pool = Object.const_get(connection_pool) if connection_pool.is_a?(String)
|
|
518
|
+
connection_pool = connection_pool.new if connection_pool.is_a?(Class)
|
|
519
|
+
connection_pool || persistent_connection_pool(persistent_connections)
|
|
520
|
+
end
|
|
521
|
+
|
|
522
|
+
# net-http-persistent is available when the application has loaded it.
|
|
523
|
+
# persistent_connections: true loads it, and raises when the gem is missing.
|
|
524
|
+
def persistent_connection_pool(persistent_connections)
|
|
525
|
+
return if persistent_connections == false
|
|
526
|
+
|
|
527
|
+
require "net/http/persistent" if persistent_connections
|
|
528
|
+
Net::HTTP::Persistent.new(name: "azure-blob", proxy: :ENV) if defined?(Net::HTTP::Persistent)
|
|
529
|
+
rescue LoadError
|
|
530
|
+
raise AzureBlob::Error, "`persistent_connections` requires the net-http-persistent gem. Add `gem \"net-http-persistent\"` to your Gemfile."
|
|
531
|
+
end
|
|
532
|
+
|
|
443
533
|
def signer
|
|
444
534
|
@signer ||=
|
|
445
535
|
begin
|
|
@@ -453,11 +543,11 @@ module AzureBlob
|
|
|
453
543
|
end
|
|
454
544
|
|
|
455
545
|
using_managed_identities ?
|
|
456
|
-
AzureBlob::EntraIdSigner.new(account_name:, host:, principal_id:) :
|
|
546
|
+
AzureBlob::EntraIdSigner.new(account_name:, host:, principal_id:, delegation_key_expiration:) :
|
|
457
547
|
AzureBlob::SharedKeySigner.new(account_name:, access_key:, host:)
|
|
458
548
|
end
|
|
459
549
|
end
|
|
460
550
|
|
|
461
|
-
attr_reader :account_name, :container, :http, :cloud_regions, :access_key, :principal_id, :use_managed_identities
|
|
551
|
+
attr_reader :account_name, :container, :http, :connection_pool, :cloud_regions, :access_key, :principal_id, :use_managed_identities, :delegation_key_expiration
|
|
462
552
|
end
|
|
463
553
|
end
|
|
@@ -14,7 +14,8 @@ module AzureBlob
|
|
|
14
14
|
attr_reader :account_name
|
|
15
15
|
attr_reader :host
|
|
16
16
|
|
|
17
|
-
def initialize(account_name:, host:, principal_id: nil)
|
|
17
|
+
def initialize(account_name:, host:, principal_id: nil, delegation_key_expiration: nil)
|
|
18
|
+
@delegation_key_expiration = parse_delegation_key_expiration(delegation_key_expiration)
|
|
18
19
|
@token = AzureBlob::IdentityToken.new(principal_id:)
|
|
19
20
|
@account_name = account_name
|
|
20
21
|
@host = host
|
|
@@ -25,7 +26,7 @@ module AzureBlob
|
|
|
25
26
|
end
|
|
26
27
|
|
|
27
28
|
def sas_token(uri, options = {})
|
|
28
|
-
delegation_key.refresh
|
|
29
|
+
delegation_key.refresh(valid_until: options[:expiry])
|
|
29
30
|
to_sign = [
|
|
30
31
|
options[:permissions],
|
|
31
32
|
options[:start],
|
|
@@ -82,8 +83,25 @@ module AzureBlob
|
|
|
82
83
|
|
|
83
84
|
private
|
|
84
85
|
|
|
86
|
+
attr_reader :delegation_key_expiration
|
|
87
|
+
|
|
88
|
+
def parse_delegation_key_expiration(value)
|
|
89
|
+
return if value.nil?
|
|
90
|
+
|
|
91
|
+
seconds = Integer(value, exception: false)
|
|
92
|
+
unless seconds&.positive?
|
|
93
|
+
raise ArgumentError, "delegation_key_expiration must be a positive number of seconds, got #{value.inspect}"
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
if seconds > UserDelegationKey::MAX_EXPIRATION
|
|
97
|
+
raise ArgumentError, "delegation_key_expiration cannot be greater than #{UserDelegationKey::MAX_EXPIRATION} seconds (7 days), got #{value.inspect}"
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
seconds
|
|
101
|
+
end
|
|
102
|
+
|
|
85
103
|
def delegation_key
|
|
86
|
-
@delegation_key ||= UserDelegationKey.new(account_name:, signer: self)
|
|
104
|
+
@delegation_key ||= UserDelegationKey.new(account_name:, signer: self, expiration: delegation_key_expiration || UserDelegationKey::EXPIRATION)
|
|
87
105
|
end
|
|
88
106
|
|
|
89
107
|
def sign(body, key:)
|
data/lib/azure_blob/http.rb
CHANGED
|
@@ -8,10 +8,12 @@ require "rexml"
|
|
|
8
8
|
module AzureBlob
|
|
9
9
|
class Http # :nodoc:
|
|
10
10
|
class Error < AzureBlob::Error
|
|
11
|
-
attr_reader :body, :status
|
|
12
|
-
def initialize(body: nil, status: nil)
|
|
11
|
+
attr_reader :body, :status, :code
|
|
12
|
+
def initialize(body: nil, status: nil, code: nil)
|
|
13
13
|
@body = body
|
|
14
14
|
@status = status
|
|
15
|
+
@code = code
|
|
16
|
+
super(status && [ status, code ].compact.join(" "))
|
|
15
17
|
end
|
|
16
18
|
|
|
17
19
|
def inspect
|
|
@@ -25,64 +27,52 @@ module AzureBlob
|
|
|
25
27
|
|
|
26
28
|
include REXML
|
|
27
29
|
|
|
28
|
-
def initialize(uri, headers = {}, signer: nil, metadata: {}, tags: {}, debug: false, raise_on_error: true)
|
|
30
|
+
def initialize(uri, headers = {}, signer: nil, connection_pool: nil, metadata: {}, tags: {}, debug: false, raise_on_error: true)
|
|
29
31
|
@raise_on_error = raise_on_error
|
|
30
32
|
@date = Time.now.httpdate
|
|
31
33
|
@uri = uri
|
|
32
34
|
@signer = signer
|
|
35
|
+
@connection_pool = connection_pool
|
|
36
|
+
@debug = debug
|
|
33
37
|
@headers = headers.merge(
|
|
34
38
|
Metadata.new(metadata).headers,
|
|
35
39
|
Tags.new(tags).headers,
|
|
36
40
|
)
|
|
37
41
|
|
|
38
42
|
sanitize_headers
|
|
39
|
-
|
|
40
|
-
@http = Net::HTTP.new(uri.hostname, uri.port)
|
|
41
|
-
@http.use_ssl = uri.port == 443
|
|
42
|
-
@http.set_debug_output($stdout) if debug
|
|
43
43
|
end
|
|
44
44
|
|
|
45
45
|
def get
|
|
46
46
|
sign_request("GET") if signer
|
|
47
|
-
@response =
|
|
48
|
-
http.get(uri, headers)
|
|
49
|
-
end
|
|
47
|
+
@response = perform(Net::HTTP::Get.new(uri, headers))
|
|
50
48
|
raise_error unless success?
|
|
51
49
|
response.body
|
|
52
50
|
end
|
|
53
51
|
|
|
54
52
|
def put(content = "")
|
|
55
53
|
sign_request("PUT") if signer
|
|
56
|
-
@response =
|
|
57
|
-
http.put(uri, content, headers)
|
|
58
|
-
end
|
|
54
|
+
@response = perform(Net::HTTP::Put.new(uri, headers), content)
|
|
59
55
|
raise_error unless success?
|
|
60
56
|
true
|
|
61
57
|
end
|
|
62
58
|
|
|
63
59
|
def post(content = "")
|
|
64
60
|
sign_request("POST") if signer
|
|
65
|
-
@response =
|
|
66
|
-
http.post(uri, content, headers)
|
|
67
|
-
end
|
|
61
|
+
@response = perform(Net::HTTP::Post.new(uri, headers), content)
|
|
68
62
|
raise_error unless success?
|
|
69
63
|
response.body
|
|
70
64
|
end
|
|
71
65
|
|
|
72
66
|
def head
|
|
73
67
|
sign_request("HEAD") if signer
|
|
74
|
-
@response =
|
|
75
|
-
http.head(uri, headers)
|
|
76
|
-
end
|
|
68
|
+
@response = perform(Net::HTTP::Head.new(uri, headers))
|
|
77
69
|
raise_error unless success?
|
|
78
70
|
response
|
|
79
71
|
end
|
|
80
72
|
|
|
81
73
|
def delete
|
|
82
74
|
sign_request("DELETE") if signer
|
|
83
|
-
@response =
|
|
84
|
-
http.delete(uri, headers)
|
|
85
|
-
end
|
|
75
|
+
@response = perform(Net::HTTP::Delete.new(uri, headers))
|
|
86
76
|
raise_error unless success?
|
|
87
77
|
response.body
|
|
88
78
|
end
|
|
@@ -93,6 +83,19 @@ module AzureBlob
|
|
|
93
83
|
|
|
94
84
|
private
|
|
95
85
|
|
|
86
|
+
# Sends the request through the connection pool when the client has one, which keeps the
|
|
87
|
+
# connection open for the next request. Without one, the request gets a connection of its own.
|
|
88
|
+
def perform(request, body = nil)
|
|
89
|
+
request.body = body if body
|
|
90
|
+
|
|
91
|
+
return connection_pool.request(uri, request) if connection_pool
|
|
92
|
+
|
|
93
|
+
http = Net::HTTP.new(uri.hostname, uri.port)
|
|
94
|
+
http.use_ssl = uri.port == 443
|
|
95
|
+
http.set_debug_output($stdout) if debug
|
|
96
|
+
http.start { http.request(request) }
|
|
97
|
+
end
|
|
98
|
+
|
|
96
99
|
ERROR_MAPPINGS = {
|
|
97
100
|
Net::HTTPNotFound => FileNotFoundError,
|
|
98
101
|
Net::HTTPForbidden => ForbiddenError,
|
|
@@ -118,7 +121,7 @@ module AzureBlob
|
|
|
118
121
|
|
|
119
122
|
def raise_error
|
|
120
123
|
return unless raise_on_error
|
|
121
|
-
raise error_from_response.new(body: @response.body, status: @response.code&.to_i)
|
|
124
|
+
raise error_from_response.new(body: @response.body, status: @response.code&.to_i, code: azure_error_code)
|
|
122
125
|
end
|
|
123
126
|
|
|
124
127
|
def status
|
|
@@ -126,13 +129,15 @@ module AzureBlob
|
|
|
126
129
|
end
|
|
127
130
|
|
|
128
131
|
def azure_error_code
|
|
129
|
-
Document.new(response.body).get_elements("//Error/Code").first
|
|
132
|
+
response["x-ms-error-code"] || Document.new(response.body.to_s).get_elements("//Error/Code").first&.text
|
|
133
|
+
rescue ParseException
|
|
134
|
+
nil
|
|
130
135
|
end
|
|
131
136
|
|
|
132
137
|
def error_from_response
|
|
133
138
|
ERROR_MAPPINGS[status] || ERROR_CODE_MAPPINGS[azure_error_code] || Error
|
|
134
139
|
end
|
|
135
140
|
|
|
136
|
-
attr_accessor :host, :
|
|
141
|
+
attr_accessor :host, :connection_pool, :debug, :signer, :response, :headers, :uri, :date, :raise_on_error
|
|
137
142
|
end
|
|
138
143
|
end
|
|
@@ -1,15 +1,20 @@
|
|
|
1
|
+
require "time"
|
|
2
|
+
|
|
1
3
|
require_relative "http"
|
|
2
4
|
|
|
3
5
|
module AzureBlob
|
|
4
6
|
class UserDelegationKey # :nodoc:
|
|
5
7
|
EXPIRATION = 25200 # 7 hours
|
|
8
|
+
MAX_EXPIRATION = 604800 # 7 days
|
|
6
9
|
EXPIRATION_BUFFER = 3600 # 1 hours
|
|
7
|
-
|
|
10
|
+
|
|
11
|
+
def initialize(account_name:, signer:, expiration: EXPIRATION)
|
|
8
12
|
@uri = URI.parse(
|
|
9
13
|
"#{signer.host}/?restype=service&comp=userdelegationkey"
|
|
10
14
|
)
|
|
11
15
|
|
|
12
16
|
@signer = signer
|
|
17
|
+
@expiration_duration = expiration
|
|
13
18
|
|
|
14
19
|
refresh
|
|
15
20
|
end
|
|
@@ -19,14 +24,22 @@ module AzureBlob
|
|
|
19
24
|
user_delegation_key
|
|
20
25
|
end
|
|
21
26
|
|
|
22
|
-
|
|
23
|
-
|
|
27
|
+
# Requests a new key when this one is about to expire, or when it expires
|
|
28
|
+
# before +valid_until+ (the expiry of the SAS about to be signed with it).
|
|
29
|
+
def refresh(valid_until: nil)
|
|
30
|
+
valid_until = Time.parse(valid_until) if valid_until.is_a?(String)
|
|
24
31
|
now = Time.now.utc
|
|
25
32
|
|
|
33
|
+
if valid_until && valid_until > now + MAX_EXPIRATION
|
|
34
|
+
raise ArgumentError, "signed URL expiry #{valid_until.getutc.iso8601} is more than #{MAX_EXPIRATION} seconds (7 days) away, " \
|
|
35
|
+
"Azure cannot sign a user delegation SAS that lasts longer than 7 days"
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
return unless expired? || (valid_until && valid_until > expiration)
|
|
26
39
|
|
|
27
40
|
start = now.iso8601
|
|
28
|
-
|
|
29
|
-
expiry =
|
|
41
|
+
new_expiration = [ now + expiration_duration, valid_until ].compact.max
|
|
42
|
+
expiry = new_expiration.getutc.iso8601
|
|
30
43
|
|
|
31
44
|
content = <<-XML.gsub!(/[[:space:]]+/, " ").strip!
|
|
32
45
|
<?xml version="1.0" encoding="utf-8"?>
|
|
@@ -47,6 +60,7 @@ module AzureBlob
|
|
|
47
60
|
@signed_service = doc.get_elements("/UserDelegationKey/SignedService").first.get_text.to_s
|
|
48
61
|
@signed_version = doc.get_elements("/UserDelegationKey/SignedVersion").first.get_text.to_s
|
|
49
62
|
@user_delegation_key = Base64.decode64(doc.get_elements("/UserDelegationKey/Value").first.get_text.to_s)
|
|
63
|
+
@expiration = new_expiration
|
|
50
64
|
end
|
|
51
65
|
|
|
52
66
|
attr_reader :signed_oid,
|
|
@@ -60,9 +74,13 @@ module AzureBlob
|
|
|
60
74
|
private
|
|
61
75
|
|
|
62
76
|
def expired?
|
|
63
|
-
expiration.nil? || Time.now >= (expiration -
|
|
77
|
+
expiration.nil? || Time.now >= (expiration - refresh_buffer)
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
def refresh_buffer
|
|
81
|
+
[ EXPIRATION_BUFFER, expiration_duration / 2 ].min
|
|
64
82
|
end
|
|
65
83
|
|
|
66
|
-
attr_reader :uri, :user_delegation_key, :signer, :expiration
|
|
84
|
+
attr_reader :uri, :user_delegation_key, :signer, :expiration, :expiration_duration
|
|
67
85
|
end
|
|
68
86
|
end
|
data/lib/azure_blob/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: azure-blob
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.10.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Joé Dupuis
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2026-11-22 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: cgi
|
|
@@ -86,14 +86,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
86
86
|
requirements:
|
|
87
87
|
- - ">="
|
|
88
88
|
- !ruby/object:Gem::Version
|
|
89
|
-
version: '3.
|
|
89
|
+
version: '3.3'
|
|
90
90
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
91
91
|
requirements:
|
|
92
92
|
- - ">="
|
|
93
93
|
- !ruby/object:Gem::Version
|
|
94
94
|
version: '0'
|
|
95
95
|
requirements: []
|
|
96
|
-
rubygems_version: 3.
|
|
96
|
+
rubygems_version: 3.5.22
|
|
97
97
|
signing_key:
|
|
98
98
|
specification_version: 4
|
|
99
99
|
summary: Azure Blob client and Active Storage adapter
|