studio-engine 0.94.0 → 0.95.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +60 -0
- data/README.md +58 -0
- data/lib/studio/image_cache.rb +370 -31
- data/lib/studio/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1714a5594192d5463c4c2597efd50d5821b4f608dc6ad6b38f576133e7abdd36
|
|
4
|
+
data.tar.gz: 83da688d462f1a56dda9a7a62029860ae9803496a29262faa53c9cf8a8b27717
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8654bb4ff77ee2b2e6af6a3ab5b89b27328538b9c9a962788bcc8565504ce97662ae18d66bb5498619cc622f7be8c213f878eb361a0301d24042ae96ea44e69f
|
|
7
|
+
data.tar.gz: 73c9d8465a52f47fe74570f13fdc2e042ed25addb982ea4f7620f599e45f7e7f260c71bb5581b2afb6eb76642c9bf12926aac57cf95de539ffefd69f4d066fc3
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,66 @@ The format is [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This pro
|
|
|
4
4
|
|
|
5
5
|
## Unreleased
|
|
6
6
|
|
|
7
|
+
## 0.95.0 — 2026-10-07
|
|
8
|
+
|
|
9
|
+
### Security
|
|
10
|
+
|
|
11
|
+
- **`Studio::ImageCache.validate_source_url!` judges the address, not the
|
|
12
|
+
spelling.** It read the URL's text only, so an internal address written any
|
|
13
|
+
way but a plain dotted quad went through. Now refused:
|
|
14
|
+
- IPv4 inside IPv6: mapped (`[::ffff:127.0.0.1]`), compatible
|
|
15
|
+
(`[::127.0.0.1]`), NAT64 (`[64:ff9b::7f00:1]`) and 6to4 (`[2002:7f00:1::]`).
|
|
16
|
+
- Numeric IPv4 as `inet_aton` reads it: short (`127.1`), integer
|
|
17
|
+
(`2130706433`), hex (`0x7f.1`) and octal (`017700000001`). Any numeric host
|
|
18
|
+
that is not a plain dotted quad is refused, public or not, because two
|
|
19
|
+
parsers can disagree about it.
|
|
20
|
+
- `localhost.` and every other name with a trailing dot, `*.localhost`, and a
|
|
21
|
+
host that is not a hostname at all (`%31%32%37.0.0.1`).
|
|
22
|
+
- More ranges: `0.0.0.0/8`, carrier-grade NAT `100.64.0.0/10`, multicast,
|
|
23
|
+
reserved `240.0.0.0/4`, the documentation and benchmarking blocks, and
|
|
24
|
+
IPv6 site-local, multicast and documentation.
|
|
25
|
+
- **A name that resolves to a non-public address.** Names are resolved (the
|
|
26
|
+
hosts file, then every A and AAAA record, 2 seconds an attempt) and one
|
|
27
|
+
non-public address refuses the URL. A name that does not resolve raises
|
|
28
|
+
`Studio::ImageCache::UnresolvedSourceHost`, a subclass of
|
|
29
|
+
`InvalidSourceURL`.
|
|
30
|
+
- **`Studio::ImageCache.fetch_remote` vets every redirect and connects to the
|
|
31
|
+
address it vetted.** It was `URI.open(redirect: true)`: a public URL that
|
|
32
|
+
answered `302` to `http://127.0.0.1/` was followed unchecked, and the name
|
|
33
|
+
was resolved a second time by the socket. It now follows at most five
|
|
34
|
+
redirects itself, checks each one, and pins each connection to a vetted
|
|
35
|
+
address, so a name cannot point elsewhere between the check and the fetch.
|
|
36
|
+
|
|
37
|
+
### Added
|
|
38
|
+
|
|
39
|
+
- `Studio::ImageCache.vet_source_url!(url)` answers the parsed URI, the judged
|
|
40
|
+
host and the vetted `addresses`, and `Studio::ImageCache.pinned_http(uri,
|
|
41
|
+
address)` builds a `Net::HTTP` that connects to one of them. A caller that
|
|
42
|
+
fetches for itself uses the pair to close the gap `validate_source_url!`
|
|
43
|
+
alone leaves. `Studio::ImageCache.fetch_response(url)` is `fetch_remote`
|
|
44
|
+
with the status and headers. `Studio::ImageCache.public_address?(ip)` is the
|
|
45
|
+
range check on its own. README, *Remote image URLs*.
|
|
46
|
+
- `Studio::ImageCache.resolver=` sets the callable names are resolved with, and
|
|
47
|
+
`validate_source_url!`, `vet_source_url!`, `fetch_remote` and
|
|
48
|
+
`fetch_response` take `resolver:`.
|
|
49
|
+
|
|
50
|
+
### Changed
|
|
51
|
+
|
|
52
|
+
- `validate_source_url!` keeps its return value (the parsed URI) and takes one
|
|
53
|
+
new optional keyword. What changes for a caller:
|
|
54
|
+
- **It makes DNS queries** outside a Rails test environment: one lookup per
|
|
55
|
+
call, and a URL whose name cannot be resolved is refused where it used to
|
|
56
|
+
pass.
|
|
57
|
+
- **Under `Rails.env.test?` names are not resolved by default**, so a suite
|
|
58
|
+
that passes `https://cdn.example.com/…` through the guard gets the answers
|
|
59
|
+
it got before and touches no network. Addresses and refused names are
|
|
60
|
+
judged the same in every environment. A test of resolution passes
|
|
61
|
+
`resolver:`.
|
|
62
|
+
- `fetch_remote` ignores `http_proxy`/`https_proxy`, refuses an
|
|
63
|
+
`https` to `http` redirect with `InvalidSourceURL` (open-uri raised a bare
|
|
64
|
+
`RuntimeError`), and still raises `OpenURI::HTTPError` carrying
|
|
65
|
+
`io.status` for a non-2xx answer or a redirect loop.
|
|
66
|
+
|
|
7
67
|
## 0.94.0 — 2026-10-07
|
|
8
68
|
|
|
9
69
|
### Added
|
data/README.md
CHANGED
|
@@ -1086,6 +1086,64 @@ destroyed before the object was trashed, so the task prints the
|
|
|
1086
1086
|
and the record must be re-attached by hand. `filename` is known only when the
|
|
1087
1087
|
upload carried a Content-Disposition; supply it otherwise.
|
|
1088
1088
|
|
|
1089
|
+
## Remote image URLs
|
|
1090
|
+
|
|
1091
|
+
`Studio::ImageCache.validate_source_url!(url)` is the check to run before the
|
|
1092
|
+
server fetches a URL someone else supplied. It returns the parsed URI or raises
|
|
1093
|
+
`Studio::ImageCache::InvalidSourceURL`.
|
|
1094
|
+
|
|
1095
|
+
It refuses anything but `http`/`https`, and any host that is not public:
|
|
1096
|
+
|
|
1097
|
+
- **An address, however it is written.** Loopback, private (`10/8`,
|
|
1098
|
+
`172.16/12`, `192.168/16`), link-local (`169.254/16`, `fe80::/10`),
|
|
1099
|
+
unique-local (`fc00::/7`), carrier-grade NAT (`100.64/10`), unspecified,
|
|
1100
|
+
multicast and reserved ranges. IPv4 inside IPv6 (`[::ffff:127.0.0.1]`) is
|
|
1101
|
+
unwrapped. Numeric IPv4 is decoded as `inet_aton` decodes it (`127.1`,
|
|
1102
|
+
`2130706433`, `0x7f.1`, `017700000001`), and a numeric host that is not a
|
|
1103
|
+
plain dotted quad is refused whatever it decodes to.
|
|
1104
|
+
- **An internal name.** `localhost`, `*.localhost`, `*.local`, `*.internal`,
|
|
1105
|
+
`*.lan`, with or without a trailing dot, in any case.
|
|
1106
|
+
- **A name that resolves to a non-public address.** The hosts file is read,
|
|
1107
|
+
then every A and AAAA record; one non-public address refuses the URL. A name
|
|
1108
|
+
that cannot be resolved raises `UnresolvedSourceHost` (a subclass).
|
|
1109
|
+
|
|
1110
|
+
### The check and the fetch are two moments
|
|
1111
|
+
|
|
1112
|
+
A name can resolve differently a moment after it was checked. What that means
|
|
1113
|
+
depends on who fetches:
|
|
1114
|
+
|
|
1115
|
+
| Who fetches | What to call | The gap |
|
|
1116
|
+
|-------------|--------------|---------|
|
|
1117
|
+
| The engine | `Studio::ImageCache.cache!`, `fetch_remote` or `fetch_response` | None. Each hop, redirects included, is vetted and the connection goes to the vetted address |
|
|
1118
|
+
| Your own HTTP client | `vet_source_url!`, then `pinned_http` | None, when you connect through `pinned_http` and vet each redirect yourself |
|
|
1119
|
+
| Your own client, by name | `validate_source_url!`, then a fetch of the URL | Open. The socket resolves the name again |
|
|
1120
|
+
| A third party you hand the URL to | `validate_source_url!` | Open, and theirs to close. The check says where the name pointed for us |
|
|
1121
|
+
|
|
1122
|
+
```ruby
|
|
1123
|
+
vetted = Studio::ImageCache.vet_source_url!(url) # raises InvalidSourceURL
|
|
1124
|
+
http = Studio::ImageCache.pinned_http(vetted.uri, vetted.addresses.first)
|
|
1125
|
+
http.start { |h| h.request(Net::HTTP::Get.new(vetted.uri.request_uri)) { |response| … } }
|
|
1126
|
+
```
|
|
1127
|
+
|
|
1128
|
+
`pinned_http` keeps the name for the `Host` header and the TLS certificate, sets
|
|
1129
|
+
the timeouts, and uses no proxy. It does not follow redirects: vet the
|
|
1130
|
+
`Location` the same way before you request it.
|
|
1131
|
+
|
|
1132
|
+
### Tests, and choosing the resolver
|
|
1133
|
+
|
|
1134
|
+
Under `Rails.env.test?` the default is **no resolution**: names are judged on
|
|
1135
|
+
their text, so a suite that passes `https://cdn.example.com/a.png` through the
|
|
1136
|
+
guard needs no network and gets a stable answer. Addresses and internal names
|
|
1137
|
+
are judged the same in every environment. To test what a name resolves to,
|
|
1138
|
+
pass a resolver, a callable from a host to its addresses:
|
|
1139
|
+
|
|
1140
|
+
```ruby
|
|
1141
|
+
Studio::ImageCache.validate_source_url!(url, resolver: ->(host) { ["10.0.0.5"] }) # raises
|
|
1142
|
+
Studio::ImageCache.resolver = ->(host) { my_lookup(host) } # app-wide
|
|
1143
|
+
```
|
|
1144
|
+
|
|
1145
|
+
`resolver: nil` on a call checks the text only.
|
|
1146
|
+
|
|
1089
1147
|
## Overriding Views
|
|
1090
1148
|
|
|
1091
1149
|
This is a non-isolated engine -- app views at the same path automatically override engine views. For example, placing `app/views/sessions/new.html.erb` in the consuming app replaces the engine's login page.
|
data/lib/studio/image_cache.rb
CHANGED
|
@@ -1,3 +1,6 @@
|
|
|
1
|
+
require "ipaddr"
|
|
2
|
+
require "socket"
|
|
3
|
+
|
|
1
4
|
module Studio
|
|
2
5
|
module ImageCache
|
|
3
6
|
EXT_BY_TYPE = {
|
|
@@ -16,6 +19,10 @@ module Studio
|
|
|
16
19
|
MAX_REMOTE_BYTES = 50 * 1024 * 1024
|
|
17
20
|
|
|
18
21
|
class InvalidSourceURL < ArgumentError; end
|
|
22
|
+
# The host is a well-formed name that could not be looked up, or has no
|
|
23
|
+
# address. Still a refusal, but possibly a passing one: DNS was down, or the
|
|
24
|
+
# name is gone. A caller that retries distinguishes it by class.
|
|
25
|
+
class UnresolvedSourceHost < InvalidSourceURL; end
|
|
19
26
|
class UnsupportedContentType < ArgumentError; end
|
|
20
27
|
class SourceTooLarge < StandardError; end
|
|
21
28
|
|
|
@@ -31,12 +38,10 @@ module Studio
|
|
|
31
38
|
# file). source_url is recorded on each ImageCache row regardless — for
|
|
32
39
|
# source_path callers, pass the original URL too if you want it tracked.
|
|
33
40
|
#
|
|
34
|
-
# source_url is validated against SSRF
|
|
35
|
-
# host must
|
|
36
|
-
#
|
|
37
|
-
#
|
|
38
|
-
# protection there requires resolving DNS once then passing the
|
|
39
|
-
# resolved IP to the HTTP client.
|
|
41
|
+
# source_url is validated against SSRF (see THE SSRF GUARD below): the
|
|
42
|
+
# scheme must be http/https and the host must be public, however it is
|
|
43
|
+
# written and wherever its name resolves. The fetch then connects to the
|
|
44
|
+
# address that was vetted, and vets every redirect the same way.
|
|
40
45
|
#
|
|
41
46
|
# Idempotent: variants already present in ImageCache are skipped. If
|
|
42
47
|
# nothing is missing, the source is never read.
|
|
@@ -103,50 +108,384 @@ module Studio
|
|
|
103
108
|
existing
|
|
104
109
|
end
|
|
105
110
|
|
|
106
|
-
#
|
|
107
|
-
#
|
|
108
|
-
|
|
111
|
+
# ── THE SSRF GUARD ────────────────────────────────────────────────────────
|
|
112
|
+
#
|
|
113
|
+
# Three questions, asked in order, about the host of a remote source_url:
|
|
114
|
+
#
|
|
115
|
+
# 1. IS IT AN ADDRESS, HOWEVER IT IS WRITTEN? A bracketed IPv6 literal is
|
|
116
|
+
# parsed and any IPv4 address inside it (mapped, compatible, NAT64,
|
|
117
|
+
# 6to4) is unwrapped. A numeric host is decoded the way inet_aton
|
|
118
|
+
# decodes it, which is what the OS hands a socket: `127.1`,
|
|
119
|
+
# `2130706433`, `0x7f.1` and `017700000001` are all 127.0.0.1.
|
|
120
|
+
# 2. IS IT A NAME WE REFUSE ON SIGHT? `localhost`, `*.localhost`,
|
|
121
|
+
# `*.local`, `*.internal`, `*.lan`, after the trailing dot is stripped
|
|
122
|
+
# and the case folded.
|
|
123
|
+
# 3. WHERE DOES THE NAME POINT? Every A and AAAA record is read, and ONE
|
|
124
|
+
# non-public address refuses the URL. A name that does not resolve is
|
|
125
|
+
# refused too.
|
|
126
|
+
#
|
|
127
|
+
# WHAT IT DOES NOT PROVE. The answer is true at the moment of the check. A
|
|
128
|
+
# name can resolve differently a moment later, so a caller that checks here
|
|
129
|
+
# and then fetches BY NAME has a gap between the two. `fetch_remote` below
|
|
130
|
+
# has no such gap: it connects to the address it vetted. A caller with its
|
|
131
|
+
# own HTTP client closes it the same way, with `vet_source_url!` and
|
|
132
|
+
# `pinned_http`. A caller that hands the URL to a third party's fetcher
|
|
133
|
+
# cannot close it at all, and should say so.
|
|
134
|
+
|
|
135
|
+
# Every range a fetch must not reach, by family. Anything outside is public.
|
|
136
|
+
NON_PUBLIC_RANGES = {
|
|
137
|
+
"0.0.0.0/8" => "unspecified",
|
|
138
|
+
"10.0.0.0/8" => "private",
|
|
139
|
+
"100.64.0.0/10" => "carrier-grade NAT",
|
|
140
|
+
"127.0.0.0/8" => "loopback",
|
|
141
|
+
"169.254.0.0/16" => "link-local",
|
|
142
|
+
"172.16.0.0/12" => "private",
|
|
143
|
+
"192.0.0.0/24" => "IETF protocol assignment",
|
|
144
|
+
"192.0.2.0/24" => "documentation",
|
|
145
|
+
"192.168.0.0/16" => "private",
|
|
146
|
+
"198.18.0.0/15" => "benchmarking",
|
|
147
|
+
"198.51.100.0/24" => "documentation",
|
|
148
|
+
"203.0.113.0/24" => "documentation",
|
|
149
|
+
"224.0.0.0/4" => "multicast",
|
|
150
|
+
"240.0.0.0/4" => "reserved",
|
|
151
|
+
"::/128" => "unspecified",
|
|
152
|
+
"::1/128" => "loopback",
|
|
153
|
+
"64:ff9b:1::/48" => "local NAT64",
|
|
154
|
+
"100::/64" => "discard",
|
|
155
|
+
"2001:db8::/32" => "documentation",
|
|
156
|
+
"fc00::/7" => "unique-local",
|
|
157
|
+
"fe80::/10" => "link-local",
|
|
158
|
+
"fec0::/10" => "site-local",
|
|
159
|
+
"ff00::/8" => "multicast"
|
|
160
|
+
}.map { |cidr, label| [IPAddr.new(cidr), label].freeze }.freeze
|
|
161
|
+
|
|
162
|
+
# IPv6 prefixes that carry an IPv4 address, and where in the 128 bits it sits
|
|
163
|
+
# (the right shift that brings it to the low 32).
|
|
164
|
+
EMBEDDED_IPV4 = {
|
|
165
|
+
"::ffff:0:0/96" => 0, # IPv4-mapped
|
|
166
|
+
"::/96" => 0, # IPv4-compatible (deprecated, still parsed)
|
|
167
|
+
"64:ff9b::/96" => 0, # NAT64
|
|
168
|
+
"2002::/16" => 80 # 6to4
|
|
169
|
+
}.map { |cidr, shift| [IPAddr.new(cidr), shift].freeze }.freeze
|
|
170
|
+
|
|
171
|
+
INTERNAL_HOSTNAMES = %w[localhost].freeze
|
|
172
|
+
INTERNAL_SUFFIXES = %w[.localhost .local .internal .lan].freeze
|
|
173
|
+
|
|
174
|
+
HOSTNAME_LABEL = /\A[a-z0-9_](?:[a-z0-9_-]*[a-z0-9_])?\z/
|
|
175
|
+
NUMERIC_LABEL = /\A(?:0x[0-9a-f]*|[0-9]+)\z/
|
|
176
|
+
|
|
177
|
+
# Seconds per DNS attempt, and for the whole lookup.
|
|
178
|
+
RESOLVE_TIMEOUTS = [2, 2].freeze
|
|
179
|
+
RESOLVE_DEADLINE = 6
|
|
180
|
+
|
|
181
|
+
MAX_REDIRECTS = 5
|
|
182
|
+
OPEN_TIMEOUT = 10
|
|
183
|
+
READ_TIMEOUT = 30
|
|
184
|
+
|
|
185
|
+
# What a check answers: the parsed URI, the host as it was judged (folded,
|
|
186
|
+
# no trailing dot), and the public addresses it was vetted against, IPv4
|
|
187
|
+
# first. `addresses` is empty only when no resolver was used.
|
|
188
|
+
VettedSource = Struct.new(:uri, :host, :addresses, keyword_init: true)
|
|
189
|
+
|
|
190
|
+
# One HTTP exchange, not followed.
|
|
191
|
+
Hop = Struct.new(:status, :reason, :location, :body, :headers, keyword_init: true)
|
|
192
|
+
|
|
193
|
+
# The resolver every name goes through: the hosts file, then DNS, the order
|
|
194
|
+
# the OS uses. Answers address strings; raises when DNS does.
|
|
195
|
+
SYSTEM_RESOLVER = lambda do |host|
|
|
196
|
+
require "timeout"
|
|
197
|
+
Timeout.timeout(RESOLVE_DEADLINE) { Studio::ImageCache.system_addresses(host) }
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
class << self
|
|
201
|
+
# A callable `host -> [address strings]` every check resolves names with.
|
|
202
|
+
# Set one to answer from somewhere other than the system resolver; a test
|
|
203
|
+
# suite sets one to decide what a name resolves to. nil restores the
|
|
204
|
+
# default.
|
|
205
|
+
attr_writer :resolver
|
|
206
|
+
|
|
207
|
+
def resolver
|
|
208
|
+
@resolver || default_resolver
|
|
209
|
+
end
|
|
210
|
+
end
|
|
211
|
+
|
|
212
|
+
# The system resolver, except under a Rails test environment, where the
|
|
213
|
+
# default is NO resolution: names are judged on their text alone, as they
|
|
214
|
+
# were before resolution existed. A consumer suite passes made-up hosts
|
|
215
|
+
# (`cdn.example.com`) through this guard and must neither reach DNS nor
|
|
216
|
+
# change its answers with the network. Literal addresses and refused names
|
|
217
|
+
# are judged identically in every environment.
|
|
218
|
+
def self.default_resolver(rails = (defined?(::Rails) ? ::Rails : nil))
|
|
219
|
+
test_env = rails.respond_to?(:env) && rails.env.respond_to?(:test?) && rails.env.test?
|
|
220
|
+
test_env ? nil : SYSTEM_RESOLVER
|
|
221
|
+
end
|
|
222
|
+
|
|
223
|
+
# SSRF guard for a remote source_url. Raises InvalidSourceURL on anything
|
|
224
|
+
# that reaches, or might reach, an internal service; returns the parsed URI.
|
|
225
|
+
#
|
|
226
|
+
# `resolver:` is the callable names are resolved with (default:
|
|
227
|
+
# `Studio::ImageCache.resolver`). Passing nil checks the text only.
|
|
228
|
+
def self.validate_source_url!(url, resolver: self.resolver)
|
|
229
|
+
vet_source_url!(url, resolver: resolver).uri
|
|
230
|
+
end
|
|
231
|
+
|
|
232
|
+
# The same check, answering a VettedSource: the URI, the judged host, and
|
|
233
|
+
# the addresses it was vetted against. A caller that makes the request
|
|
234
|
+
# itself connects to one of `addresses` (see `pinned_http`) so the name
|
|
235
|
+
# cannot resolve differently between this check and the connection.
|
|
236
|
+
def self.vet_source_url!(url, resolver: self.resolver)
|
|
109
237
|
require "uri"
|
|
110
238
|
uri = URI.parse(url)
|
|
111
239
|
unless %w[http https].include?(uri.scheme)
|
|
112
240
|
raise InvalidSourceURL, "URL scheme must be http or https, got #{uri.scheme.inspect}"
|
|
113
241
|
end
|
|
114
242
|
|
|
115
|
-
|
|
243
|
+
raw = uri.host.to_s
|
|
244
|
+
raise InvalidSourceURL, "URL missing host: #{url.inspect}" if raw.empty?
|
|
245
|
+
|
|
246
|
+
if raw.start_with?("[")
|
|
247
|
+
ip = parse_address(raw.delete_prefix("[").delete_suffix("]")) ||
|
|
248
|
+
raise(InvalidSourceURL, "Malformed IP host #{raw.inspect}")
|
|
249
|
+
refuse_non_public!(ip, "URL host #{raw}")
|
|
250
|
+
return VettedSource.new(uri: uri, host: ip.to_s, addresses: [ip.to_s])
|
|
251
|
+
end
|
|
252
|
+
|
|
253
|
+
host = raw.downcase.delete_suffix(".")
|
|
116
254
|
raise InvalidSourceURL, "URL missing host: #{url.inspect}" if host.empty?
|
|
117
255
|
|
|
118
|
-
|
|
119
|
-
|
|
256
|
+
if (ip = decode_numeric_ipv4(host))
|
|
257
|
+
refuse_non_public!(ip, "URL host #{raw}")
|
|
258
|
+
unless host == ip.to_s
|
|
259
|
+
raise InvalidSourceURL, "URL host #{raw} is a non-canonical IPv4 address (#{ip}); write it as a dotted quad"
|
|
260
|
+
end
|
|
261
|
+
return VettedSource.new(uri: uri, host: host, addresses: [ip.to_s])
|
|
262
|
+
end
|
|
263
|
+
|
|
264
|
+
unless hostname?(host)
|
|
265
|
+
raise InvalidSourceURL, "URL host is not a hostname: #{raw.inspect}"
|
|
266
|
+
end
|
|
267
|
+
if INTERNAL_HOSTNAMES.include?(host) || host.end_with?(*INTERNAL_SUFFIXES)
|
|
120
268
|
raise InvalidSourceURL, "URL points to internal hostname: #{host.inspect}"
|
|
121
269
|
end
|
|
122
270
|
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
271
|
+
addresses = resolver ? resolve_public!(host, resolver) : []
|
|
272
|
+
VettedSource.new(uri: uri, host: host, addresses: addresses)
|
|
273
|
+
end
|
|
274
|
+
|
|
275
|
+
# True when `address` (a String or IPAddr) is one a fetch may reach.
|
|
276
|
+
def self.public_address?(address)
|
|
277
|
+
ip = address.is_a?(IPAddr) ? address : parse_address(address.to_s)
|
|
278
|
+
!ip.nil? && non_public_reason(ip).nil?
|
|
279
|
+
end
|
|
280
|
+
|
|
281
|
+
# The IPv4 address a numeric host means to inet_aton, or nil when the host
|
|
282
|
+
# is a name. One to four parts; each decimal, `0x` hex, or leading-zero
|
|
283
|
+
# octal; the last part fills whatever bytes the earlier ones left. A host
|
|
284
|
+
# whose last label is a number is never a name (no TLD is numeric), so one
|
|
285
|
+
# that does not decode is refused rather than handed to a resolver.
|
|
286
|
+
def self.decode_numeric_ipv4(host)
|
|
287
|
+
host = host.to_s.downcase
|
|
288
|
+
parts = host.split(".", -1)
|
|
289
|
+
return nil unless parts.last.to_s.match?(NUMERIC_LABEL)
|
|
290
|
+
|
|
291
|
+
malformed = InvalidSourceURL.new("Malformed numeric host #{host.inspect}")
|
|
292
|
+
raise malformed if parts.size > 4
|
|
293
|
+
|
|
294
|
+
values = parts.map do |part|
|
|
295
|
+
raise malformed unless part.match?(NUMERIC_LABEL)
|
|
296
|
+
|
|
297
|
+
if part.start_with?("0x") then part.delete_prefix("0x").to_i(16)
|
|
298
|
+
elsif part.start_with?("0") && part.size > 1
|
|
299
|
+
raise malformed unless part.match?(/\A[0-7]+\z/)
|
|
300
|
+
|
|
301
|
+
part.to_i(8)
|
|
302
|
+
else part.to_i(10)
|
|
134
303
|
end
|
|
135
|
-
|
|
136
|
-
|
|
304
|
+
end
|
|
305
|
+
|
|
306
|
+
*leading, last = values
|
|
307
|
+
raise malformed if leading.any? { |value| value > 0xff }
|
|
308
|
+
raise malformed if last >= 256**(4 - leading.size)
|
|
309
|
+
|
|
310
|
+
IPAddr.new(leading.each_with_index.sum { |value, index| value << (8 * (3 - index)) } + last, Socket::AF_INET)
|
|
311
|
+
end
|
|
312
|
+
|
|
313
|
+
# Every address the system resolver gives for `host`: the hosts file first
|
|
314
|
+
# (a hit there is the whole answer, as it is for the OS), then every A and
|
|
315
|
+
# AAAA record. `hosts:` and `dns:` exist so a test can stand in for both.
|
|
316
|
+
def self.system_addresses(host, hosts: nil, dns: nil)
|
|
317
|
+
require "resolv"
|
|
318
|
+
local = (hosts || Resolv::Hosts.new).getaddresses(host).map(&:to_s)
|
|
319
|
+
return local unless local.empty?
|
|
320
|
+
|
|
321
|
+
lookup = lambda do |client|
|
|
322
|
+
client.timeouts = RESOLVE_TIMEOUTS
|
|
323
|
+
[Resolv::DNS::Resource::IN::A, Resolv::DNS::Resource::IN::AAAA]
|
|
324
|
+
.flat_map { |type| client.getresources(host, type) }
|
|
325
|
+
.map { |record| record.address.to_s }
|
|
326
|
+
end
|
|
327
|
+
dns ? lookup.call(dns) : Resolv::DNS.open(&lookup)
|
|
328
|
+
end
|
|
329
|
+
|
|
330
|
+
# Fetches a remote source_url. Every hop is vetted, redirects included, and
|
|
331
|
+
# every connection is made to an address that hop was vetted against, so
|
|
332
|
+
# there is no window in which the name can point somewhere else.
|
|
333
|
+
#
|
|
334
|
+
# A non-2xx answer raises OpenURI::HTTPError with the status on `io.status`,
|
|
335
|
+
# as the open-uri fetch this replaced did.
|
|
336
|
+
def self.fetch_remote(source_url, resolver: self.resolver)
|
|
337
|
+
fetch_response(source_url, resolver: resolver).body
|
|
338
|
+
end
|
|
339
|
+
|
|
340
|
+
# The same fetch, answering the final Hop: `body`, `status`, and `headers`
|
|
341
|
+
# (lower-case names, so `headers["content-type"]`).
|
|
342
|
+
def self.fetch_response(source_url, resolver: self.resolver)
|
|
343
|
+
url = source_url.to_s
|
|
344
|
+
previous = nil
|
|
345
|
+
hop = nil
|
|
346
|
+
|
|
347
|
+
(MAX_REDIRECTS + 1).times do
|
|
348
|
+
vetted = vet_source_url!(url, resolver: resolver)
|
|
349
|
+
if previous&.scheme == "https" && vetted.uri.scheme == "http"
|
|
350
|
+
raise InvalidSourceURL, "redirection forbidden: #{previous} -> #{vetted.uri}"
|
|
137
351
|
end
|
|
352
|
+
|
|
353
|
+
hop = request_hop(vetted)
|
|
354
|
+
return hop if (200..299).cover?(hop.status)
|
|
355
|
+
raise http_error(hop, vetted.uri) unless [301, 302, 303, 307, 308].include?(hop.status) && !hop.location.to_s.empty?
|
|
356
|
+
|
|
357
|
+
previous = vetted.uri
|
|
358
|
+
url = URI.join(vetted.uri, hop.location).to_s
|
|
138
359
|
end
|
|
139
360
|
|
|
140
|
-
|
|
361
|
+
raise http_error(hop, previous, "too many redirects (more than #{MAX_REDIRECTS})")
|
|
141
362
|
end
|
|
142
363
|
|
|
143
|
-
|
|
364
|
+
# One GET, to the first vetted address that accepts a connection. An
|
|
365
|
+
# address that cannot be reached (an AAAA record on a host with no IPv6
|
|
366
|
+
# route) falls through to the next; the last one's error is raised.
|
|
367
|
+
def self.request_hop(vetted)
|
|
368
|
+
require "net/http"
|
|
369
|
+
candidates = vetted.addresses.empty? ? [nil] : vetted.addresses
|
|
370
|
+
unreachable = [Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Errno::ENETUNREACH, Errno::EADDRNOTAVAIL,
|
|
371
|
+
Net::OpenTimeout, SocketError]
|
|
372
|
+
|
|
373
|
+
candidates.each_with_index do |address, index|
|
|
374
|
+
return get_from(vetted.uri, address)
|
|
375
|
+
rescue *unreachable
|
|
376
|
+
raise if index == candidates.size - 1
|
|
377
|
+
end
|
|
378
|
+
end
|
|
379
|
+
|
|
380
|
+
# A Net::HTTP for `uri` that connects to `address` and nowhere else. The
|
|
381
|
+
# name still goes in the Host header and is what TLS verifies the
|
|
382
|
+
# certificate against. No proxy is used, even when the environment names
|
|
383
|
+
# one: a proxy would resolve the name again, somewhere this check cannot
|
|
384
|
+
# see. `address` nil connects by name (only when nothing was resolved).
|
|
385
|
+
def self.pinned_http(uri, address)
|
|
386
|
+
require "net/http"
|
|
387
|
+
http = Net::HTTP.new(uri.hostname, uri.port, nil)
|
|
388
|
+
http.ipaddr = address if address
|
|
389
|
+
if uri.scheme == "https"
|
|
390
|
+
http.use_ssl = true
|
|
391
|
+
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
|
|
392
|
+
end
|
|
393
|
+
http.open_timeout = OPEN_TIMEOUT
|
|
394
|
+
http.read_timeout = READ_TIMEOUT
|
|
395
|
+
http
|
|
396
|
+
end
|
|
397
|
+
|
|
398
|
+
def self.get_from(uri, address)
|
|
399
|
+
pinned_http(uri, address).start do |http|
|
|
400
|
+
http.request(Net::HTTP::Get.new(uri.request_uri)) do |response|
|
|
401
|
+
body = +"".b
|
|
402
|
+
response.read_body do |chunk|
|
|
403
|
+
body << chunk
|
|
404
|
+
if body.bytesize > MAX_REMOTE_BYTES
|
|
405
|
+
raise SourceTooLarge, "remote payload exceeds cap #{MAX_REMOTE_BYTES} bytes"
|
|
406
|
+
end
|
|
407
|
+
end
|
|
408
|
+
return Hop.new(status: response.code.to_i, reason: response.message.to_s, location: response["location"],
|
|
409
|
+
body: body, headers: response.to_hash.transform_values { |values| values.join(", ") })
|
|
410
|
+
end
|
|
411
|
+
end
|
|
412
|
+
end
|
|
413
|
+
private_class_method :get_from
|
|
414
|
+
|
|
415
|
+
def self.http_error(hop, uri, message = nil)
|
|
144
416
|
require "open-uri"
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
417
|
+
require "stringio"
|
|
418
|
+
io = StringIO.new(hop.body.to_s)
|
|
419
|
+
OpenURI::Meta.init(io)
|
|
420
|
+
io.status = [hop.status.to_s, hop.reason.to_s]
|
|
421
|
+
io.base_uri = uri
|
|
422
|
+
hop.headers.each { |name, value| io.meta_add_field(name, value) }
|
|
423
|
+
OpenURI::HTTPError.new(message || "#{hop.status} #{hop.reason}", io)
|
|
424
|
+
end
|
|
425
|
+
private_class_method :http_error
|
|
426
|
+
|
|
427
|
+
def self.parse_address(text)
|
|
428
|
+
return nil if text.include?("%") # a zone id names an interface on THIS machine
|
|
429
|
+
|
|
430
|
+
IPAddr.new(text)
|
|
431
|
+
rescue IPAddr::Error
|
|
432
|
+
nil
|
|
433
|
+
end
|
|
434
|
+
private_class_method :parse_address
|
|
435
|
+
|
|
436
|
+
def self.hostname?(host)
|
|
437
|
+
host.size <= 253 && host.split(".", -1).all? { |label| label.size <= 63 && label.match?(HOSTNAME_LABEL) }
|
|
438
|
+
end
|
|
439
|
+
private_class_method :hostname?
|
|
440
|
+
|
|
441
|
+
# The IPv4 address an IPv6 address carries, or the address itself.
|
|
442
|
+
def self.unwrap(ip)
|
|
443
|
+
return ip unless ip.ipv6?
|
|
444
|
+
|
|
445
|
+
_, shift = EMBEDDED_IPV4.find { |range, _| range.include?(ip) }
|
|
446
|
+
return ip if shift.nil? || ip.to_i <= 1 # :: and ::1 are IPv6's own, not IPv4 in a wrapper
|
|
447
|
+
|
|
448
|
+
IPAddr.new((ip.to_i >> shift) & 0xffffffff, Socket::AF_INET)
|
|
449
|
+
end
|
|
450
|
+
private_class_method :unwrap
|
|
451
|
+
|
|
452
|
+
# Why this address is not public, or nil when it is.
|
|
453
|
+
def self.non_public_reason(ip)
|
|
454
|
+
[ip, unwrap(ip)].uniq.each do |candidate|
|
|
455
|
+
_, label = NON_PUBLIC_RANGES.find { |range, _| range.family == candidate.family && range.include?(candidate) }
|
|
456
|
+
return label if label
|
|
457
|
+
end
|
|
458
|
+
nil
|
|
459
|
+
end
|
|
460
|
+
private_class_method :non_public_reason
|
|
461
|
+
|
|
462
|
+
def self.refuse_non_public!(ip, subject)
|
|
463
|
+
reason = non_public_reason(ip)
|
|
464
|
+
return if reason.nil?
|
|
465
|
+
|
|
466
|
+
inner = unwrap(ip)
|
|
467
|
+
named = inner == ip ? ip.to_s : "#{inner} (written as #{ip})"
|
|
468
|
+
raise InvalidSourceURL, "#{subject} points to a non-public address: #{named} [#{reason}]"
|
|
469
|
+
end
|
|
470
|
+
private_class_method :refuse_non_public!
|
|
471
|
+
|
|
472
|
+
def self.resolve_public!(host, resolver)
|
|
473
|
+
answers =
|
|
474
|
+
begin
|
|
475
|
+
Array(resolver.call(host)).map(&:to_s)
|
|
476
|
+
rescue StandardError => e
|
|
477
|
+
raise UnresolvedSourceHost, "URL host #{host.inspect} could not be resolved: #{e.class}: #{e.message}"
|
|
478
|
+
end
|
|
479
|
+
raise UnresolvedSourceHost, "URL host #{host.inspect} did not resolve to any address" if answers.empty?
|
|
480
|
+
|
|
481
|
+
ips = answers.map do |answer|
|
|
482
|
+
ip = parse_address(answer) ||
|
|
483
|
+
raise(InvalidSourceURL, "URL host #{host.inspect} resolved to something that is not an address: #{answer.inspect}")
|
|
484
|
+
refuse_non_public!(ip, "URL host #{host.inspect} resolves to #{answers.size} address(es), and one")
|
|
485
|
+
ip.ipv4_mapped? ? ip.native : ip
|
|
148
486
|
end
|
|
149
|
-
|
|
487
|
+
ips.partition(&:ipv4?).flatten.map(&:to_s).uniq
|
|
150
488
|
end
|
|
489
|
+
private_class_method :resolve_public!
|
|
151
490
|
end
|
|
152
491
|
end
|
data/lib/studio/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: studio-engine
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.95.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Alex McRitchie
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-10-
|
|
11
|
+
date: 2026-10-08 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: rails
|