radioactive 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +77 -0
- data/CLAUDE.md +9 -6
- data/README.md +72 -3
- data/lib/radioactive/address_check.rb +3 -1
- data/lib/radioactive/errors.rb +14 -7
- data/lib/radioactive/fetcher.rb +108 -38
- data/lib/radioactive/version.rb +1 -1
- data/lib/radioactive.rb +1 -0
- data/lib/tasks/lint/all.rake +2 -2
- data/lib/tasks/lint/rubyfmt.rake +13 -0
- data/lib/tasks/security/all.rake +2 -0
- data/lib/tasks/security/gitleaks.rake +6 -0
- data/sig/radioactive.rbs +4 -2
- data.tar.gz.sig +0 -0
- metadata +7 -5
- metadata.gz.sig +0 -0
- data/lib/tasks/lint/rubocop.rake +0 -15
- /data/lib/tasks/{security.rake → security/bundler_audit.rake} +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c9f817e25ec6dd75ec6f603f24df5a6c0f01eef4c1f47e5581da6a1f732fb9ad
|
|
4
|
+
data.tar.gz: 66a603a6d7ba82219ff172045a3c2577debfa4cb86d2c0d7df685687ac89a772
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 561781780d9e484a5984d7605b122f1be24deeeaceba4fae230191ef33d69458e9f451e5b160ddb8a7f52d334cae814a9f66c8abd701d19146356298252064c0
|
|
7
|
+
data.tar.gz: 23d1368172643c0736e75ff3a4db3b83af0075fd577b606a79ccd2bb28bc82d137237343d47309a077a0772e187a8086f54253dc7ff407b670a3481543e11efd
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,83 @@ project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.1.3] - 2026-10-14
|
|
10
|
+
|
|
11
|
+
No runtime behavior changes.
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
|
|
15
|
+
- Minimum supported Ruby raised from 3.2 to 3.3 (`required_ruby_version
|
|
16
|
+
>= 3.3.0`). Ruby 3.2 reached end-of-life in March 2026.
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
|
|
20
|
+
- CI never ran on pushes: the workflow's `push` trigger pointed at a
|
|
21
|
+
placeholder branch (`.invalid`). It now runs on pushes to `main` as well as
|
|
22
|
+
on pull requests.
|
|
23
|
+
- CI tested a single Ruby version. The `test` job now runs on every Ruby the
|
|
24
|
+
gemspec supports (3.3, 3.4, 4.0), installing without the `development` and
|
|
25
|
+
`tools` groups, whose gems require Ruby >= 4.0.
|
|
26
|
+
- The packaged gem's file exclusion list still referenced the removed
|
|
27
|
+
`.rubocop.yml`. It now excludes the development-only `.rubyfmtignore`,
|
|
28
|
+
`.gitleaks.toml`, and `.pre-commit-config.yaml`, so they don't ship in the
|
|
29
|
+
`.gem`.
|
|
30
|
+
- `rake types` (Steep) failed with two nil-safety errors in `Fetcher`:
|
|
31
|
+
`parse_url` re-read `uri.host` after its nil check, and the connect-fallback
|
|
32
|
+
error in `perform_request` had no declared type. Both are now typed
|
|
33
|
+
explicitly. Behavior is unchanged.
|
|
34
|
+
|
|
35
|
+
### Security
|
|
36
|
+
|
|
37
|
+
- Development dependencies with published advisories updated:
|
|
38
|
+
`concurrent-ruby` 1.3.6 → 1.3.8 and `json` 2.19.5 → 3.1.0 (both pulled in
|
|
39
|
+
by Steep). Neither is a runtime dependency of the gem.
|
|
40
|
+
- New `rake security` task (bundler-audit + gitleaks), run by prek
|
|
41
|
+
pre-commit and pre-push hooks and by the CI `scan_ruby` and `secrets` jobs.
|
|
42
|
+
|
|
43
|
+
### Development
|
|
44
|
+
|
|
45
|
+
- Formatting moved from Standard/RuboCop to `rubyfmt`; the codebase was
|
|
46
|
+
reformatted accordingly. `rake lint` and `rake lint:all:autocorrect` now run
|
|
47
|
+
rubyfmt.
|
|
48
|
+
- CI split into separate `test`, `lint`, `types`, `scan_ruby`, and `secrets`
|
|
49
|
+
jobs.
|
|
50
|
+
|
|
51
|
+
## [0.1.2] - 2026-05-06
|
|
52
|
+
|
|
53
|
+
### Fixed
|
|
54
|
+
|
|
55
|
+
- Dual-stack hosts whose resolver returns AAAA before A on a network without
|
|
56
|
+
IPv6 reachability no longer surface `ResponseError: transport error:
|
|
57
|
+
Errno::EHOSTUNREACH`. `Fetcher#perform_request` now iterates over the full
|
|
58
|
+
pre-validated candidate list returned by `pin_addresses`, advancing to the
|
|
59
|
+
next address on connect-phase transport errors (`Errno::EHOSTUNREACH`,
|
|
60
|
+
`Errno::ENETUNREACH`, `Errno::ECONNREFUSED`, `Net::OpenTimeout`). Errors
|
|
61
|
+
raised after a connection is established (TLS handshake failure, read
|
|
62
|
+
timeout, post-connect `ECONNRESET`, non-2xx response) do **not** trigger
|
|
63
|
+
fallback — silently retrying against a different IP would mask real
|
|
64
|
+
problems.
|
|
65
|
+
|
|
66
|
+
### Security
|
|
67
|
+
|
|
68
|
+
- Strict dual-A SSRF rejection is preserved end-to-end. Validation runs
|
|
69
|
+
across the entire resolved address list before any socket opens, so the
|
|
70
|
+
fallback added above cannot weaken the "if any resolved address is in a
|
|
71
|
+
forbidden range, refuse the request" rule. A new HTTP-level test
|
|
72
|
+
exercises the dual-stack-with-one-forbidden-address case to lock this in.
|
|
73
|
+
- Caller-supplied header validation now provably runs before any socket
|
|
74
|
+
attempt across the candidate list (regression-tested via the existing
|
|
75
|
+
CRLF / NUL header tests, which previously passed only because the single
|
|
76
|
+
pinned IP refused the connection first).
|
|
77
|
+
|
|
78
|
+
### Documentation
|
|
79
|
+
|
|
80
|
+
- `docs/REQUIREMENTS.md` §175 ("DNS resolution and IP pinning") rewritten:
|
|
81
|
+
step 4 now describes a candidate list rather than "the first remaining
|
|
82
|
+
address," and a new step 6 specifies the connect-phase fallback
|
|
83
|
+
semantics, what does *not* trigger fallback, and the exhaustion-error
|
|
84
|
+
shape (`TimeoutError` for `Net::OpenTimeout`, otherwise `ResponseError`).
|
|
85
|
+
|
|
9
86
|
## [0.1.1] - 2026-05-04
|
|
10
87
|
|
|
11
88
|
Tooling and documentation only. No runtime behavior changes.
|
data/CLAUDE.md
CHANGED
|
@@ -12,8 +12,6 @@ Radioactive is a hardened HTTP fetcher for URLs supplied by untrusted users —
|
|
|
12
12
|
|
|
13
13
|
The scaffold from `bundle gem radioactive` is still mostly untouched: `lib/radioactive.rb` defines only the module and a base `Error` class, `test/test_radioactive.rb` contains the generator's failing `assert false`, and the gemspec metadata (`summary`, `description`, `homepage`, `allowed_push_host`, `source_code_uri`, `changelog_uri`) is still TODO placeholders. Implementation work is greenfield — match the API surface in `docs/REQUIREMENTS.md` rather than inventing one.
|
|
14
14
|
|
|
15
|
-
Note: the parent directory `/Users/pablo/code/posiczko/CLAUDE.md` documents a different project (Ougai). It does not apply here — Radioactive uses Minitest (not RSpec), Standard (not RuboCop directly), and has no Oj/JrJackson dependency.
|
|
16
|
-
|
|
17
15
|
## Architectural constraints (from the spec)
|
|
18
16
|
|
|
19
17
|
These are non-negotiable design choices in `docs/REQUIREMENTS.md`. Don't propose changes that violate them without flagging the spec change first:
|
|
@@ -35,11 +33,16 @@ These are non-negotiable design choices in `docs/REQUIREMENTS.md`. Don't propose
|
|
|
35
33
|
|
|
36
34
|
- `bin/setup` — install dependencies (wraps `bundle install`).
|
|
37
35
|
- `bin/console` — IRB session with the gem preloaded.
|
|
38
|
-
- `bundle exec rake` — default task: runs `test` then `
|
|
36
|
+
- `bundle exec rake` — default task: runs `test`, `lint`, then `types`. All must pass.
|
|
39
37
|
- `bundle exec rake test` — Minitest suite (driven by `Minitest::TestTask`, picks up `test/**/test_*.rb`).
|
|
40
38
|
- `bundle exec rake test TESTOPTS="--name=/pattern/"` — run a single test or matching subset.
|
|
41
39
|
- `ruby -Ilib -Itest test/test_radioactive.rb` — run one test file directly without rake.
|
|
42
|
-
- `bundle exec rake
|
|
40
|
+
- `bundle exec rake lint` / `bundle exec rake lint:all:autocorrect` — run all linters / autofix.
|
|
41
|
+
- `bundle exec rake lint:rubyfmt` / `lint:rubyfmt:autocorrect` — `rubyfmt` (Homebrew binary, not a gem) check / reformat; skips paths in `.rubyfmtignore`.
|
|
42
|
+
- `bundle exec rake security` — bundler-audit + gitleaks; also runs as a pre-push hook (`prek install --hook-type pre-push`).
|
|
43
|
+
- `bundle exec rake security:audit` — `bundler-audit` update + scan; also runs as a pre-commit hook on every commit.
|
|
44
|
+
- `bundle exec rake security:gitleaks` — secret scan of the working tree (respects `.gitignore`). gitleaks also runs on staged changes (pre-commit) and on PR commits (CI `secrets` job); config and allowlists in `.gitleaks.toml`.
|
|
45
|
+
- Pre-commit hooks (`.pre-commit-config.yaml`, run by `prek`) cover gitleaks, security audit, and rubyfmt. Install with `prek install`.
|
|
43
46
|
- `bundle exec rake build` / `install` / `release` — gem packaging tasks (release is maintainer-only and currently blocked by the `allowed_push_host` TODO in the gemspec).
|
|
44
47
|
|
|
45
48
|
## Layout and conventions
|
|
@@ -48,5 +51,5 @@ These are non-negotiable design choices in `docs/REQUIREMENTS.md`. Don't propose
|
|
|
48
51
|
- Long-form design docs live in `docs/`. `docs/REQUIREMENTS.md` is the spec; treat it as a contract and update it deliberately (in the same commit as behavior changes) rather than letting code and spec drift.
|
|
49
52
|
- RBS signatures live in `sig/radioactive.rbs` — keep them in sync when adding public API.
|
|
50
53
|
- Versioning: bump `Radioactive::VERSION` in `lib/radioactive/version.rb` for releases.
|
|
51
|
-
- Ruby version: gemspec requires `>= 3.2.0
|
|
52
|
-
- CI (`.github/workflows/main.yml`)
|
|
54
|
+
- Ruby version: gemspec requires `>= 3.2.0`. Match that when writing code (no 3.3-only syntax unless you also bump it).
|
|
55
|
+
- CI (`.github/workflows/main.yml`) runs on pushes to `main` and on pull requests. Jobs: `test` (matrix over Ruby 3.2–4.0, installed without the `development`/`tools` groups because some of those gems need Ruby >= 4.0, so tests are loaded directly rather than via rake), `lint` (rubyfmt binary, pinned to the Homebrew version), `types` (`rake types`), `scan_ruby` (bundler-audit), and `secrets` (gitleaks over the pushed/PR commits). When you add a Ruby version to the gemspec range or bump rubyfmt locally, update the workflow to match.
|
data/README.md
CHANGED
|
@@ -420,6 +420,13 @@ bundle exec ruby -Ilib -Itest test/test_fetcher.rb # run one test file directly
|
|
|
420
420
|
|
|
421
421
|
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
422
422
|
|
|
423
|
+
The formatter, secret scanner, and hook runner are standalone binaries rather than gems, so install them separately and then install the git hooks (see [Git hooks](#git-hooks)):
|
|
424
|
+
|
|
425
|
+
```sh
|
|
426
|
+
brew install rubyfmt gitleaks prek
|
|
427
|
+
prek install && prek install --hook-type pre-push
|
|
428
|
+
```
|
|
429
|
+
|
|
423
430
|
To install this gem onto your local machine, run `bundle exec rake gem:install`. See [Releasing](#releasing) below for publishing a new version.
|
|
424
431
|
|
|
425
432
|
## Releasing
|
|
@@ -429,7 +436,7 @@ A release cuts a new version, tags it in git, and pushes the `.gem` to [rubygems
|
|
|
429
436
|
### Pre-flight
|
|
430
437
|
|
|
431
438
|
1. `bundle exec rake` exits 0. Tests, lint, and type checks all green.
|
|
432
|
-
2. `lib/radioactive/version.rb` bumped to the new version following [SemVer](https://semver.org).
|
|
439
|
+
2. `lib/radioactive/version.rb` bumped to the new version following [SemVer](https://semver.org), followed by `bundle install` so the `radioactive (X.Y.Z)` lines in `Gemfile.lock` match. Commit both files together: if they disagree, the pre-commit `security:audit` hook fails because `bundle exec` rewrites the lockfile.
|
|
433
440
|
3. `CHANGELOG.md` has a populated section for the new version with a date, and `[Unreleased]` is empty (or its contents have been moved into the new section).
|
|
434
441
|
4. Working tree is clean (`rake gem:release` enforces this).
|
|
435
442
|
5. You have an active rubygems session: `gem signin` if `gem whoami` returns nothing.
|
|
@@ -455,6 +462,18 @@ If anything fails midway (e.g. push rejected, MFA timeout), the tag may already
|
|
|
455
462
|
- Bump `lib/radioactive/version.rb` to the next anticipated version with a `.dev` or `.alpha` suffix if you want subsequent local builds to be distinguishable from the release.
|
|
456
463
|
- Push that commit; future changes accumulate under `[Unreleased]` until the next release.
|
|
457
464
|
|
|
465
|
+
### Release Cheatsheet
|
|
466
|
+
|
|
467
|
+
1. Edit CHANGELOG.md — add the [0.1.2] - 2026-05-06 section
|
|
468
|
+
2. Edit lib/radioactive/version.rb — "0.1.1" → "0.1.2", then `bundle install` to sync Gemfile.lock
|
|
469
|
+
3. Run bundle exec rake — confirm tests + lint + types still green
|
|
470
|
+
4. Run git status — sanity check what's about to be committed
|
|
471
|
+
5. Run git add -A && git commit — single commit covering fix + version + changelog
|
|
472
|
+
6. Run git push origin main — gets the commit on the remote before the tag
|
|
473
|
+
7. Run bundle exec rake gem:release — guard_clean → build → tag → push → publish (MFA)
|
|
474
|
+
8. Edit lib/radioactive/version.rb — bump to 0.1.3.dev (post-release commit), then `bundle install`
|
|
475
|
+
9. Run git add ... && git commit && git push
|
|
476
|
+
|
|
458
477
|
### Stronger publishing setup (optional)
|
|
459
478
|
|
|
460
479
|
For a security-focused gem, consider [RubyGems Trusted Publishing](https://guides.rubygems.org/trusted-publishing/) instead of pushing from a developer machine: a tagged commit triggers a GitHub Actions workflow that authenticates to rubygems via OIDC and publishes without any long-lived API key on disk. Removes the "stolen laptop = compromised gem" risk and complements the MFA requirement.
|
|
@@ -516,15 +535,65 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/[USERN
|
|
|
516
535
|
|
|
517
536
|
### Before opening a PR
|
|
518
537
|
|
|
519
|
-
1. **Run the full check locally.** `bundle exec rake` runs tests,
|
|
538
|
+
1. **Run the full check locally.** `bundle exec rake` runs tests, rubyfmt, and the type check (`rbs validate` + `steep`). All three must pass.
|
|
520
539
|
2. **Update the spec if you're changing behavior.** The contract lives at [docs/REQUIREMENTS.md](docs/REQUIREMENTS.md). Code and spec drift is a bug - fix them in the same commit.
|
|
521
540
|
3. **Update the signatures if you're changing the public API.** Any change to a public method on `Radioactive`, `Fetcher`, `Result`, or any error class needs a matching update in `sig/radioactive.rbs`. The Type checking section above covers what to do; `bundle exec rake types` will tell you when something's off.
|
|
522
541
|
4. **Add a test.** Especially for security-sensitive paths: every defense in the threat model should have a test that proves the failure path actually closes. See `test/test_fetcher.rb` and `test/test_fetcher_http.rb` for patterns.
|
|
523
542
|
|
|
543
|
+
### Security checks
|
|
544
|
+
|
|
545
|
+
```sh
|
|
546
|
+
bundle exec rake security # run both checks below
|
|
547
|
+
bundle exec rake security:audit # bundler-audit: update advisory DB, scan Gemfile.lock
|
|
548
|
+
bundle exec rake security:gitleaks # gitleaks: scan the working tree for secrets
|
|
549
|
+
```
|
|
550
|
+
|
|
551
|
+
Secrets are scanned with [gitleaks](https://github.com/gitleaks/gitleaks)
|
|
552
|
+
(`brew install gitleaks`), configured in `.gitleaks.toml`.
|
|
553
|
+
|
|
554
|
+
### Git hooks
|
|
555
|
+
|
|
556
|
+
Hooks are defined in `.pre-commit-config.yaml` and run by
|
|
557
|
+
[prek](https://github.com/j178/prek) (`brew install prek`). Install them once
|
|
558
|
+
per clone:
|
|
559
|
+
|
|
560
|
+
```sh
|
|
561
|
+
prek install # pre-commit
|
|
562
|
+
prek install --hook-type pre-push # pre-push
|
|
563
|
+
```
|
|
564
|
+
|
|
565
|
+
* **pre-commit:** gitleaks on staged changes, `security:audit` on every commit,
|
|
566
|
+
rubyfmt on staged `.rb` files.
|
|
567
|
+
* **pre-push:** `rake security` (bundler-audit + gitleaks).
|
|
568
|
+
|
|
569
|
+
CI (`.github/workflows/main.yml`) runs bundler-audit and gitleaks too, so
|
|
570
|
+
skipping a hook with `--no-verify` only delays the failure.
|
|
571
|
+
|
|
572
|
+
### CI
|
|
573
|
+
|
|
574
|
+
`.github/workflows/main.yml` runs on pushes to `main` and on pull requests:
|
|
575
|
+
|
|
576
|
+
| Job | What it does |
|
|
577
|
+
|-------------|--------------------------------------------------------------------------------------------------|
|
|
578
|
+
| `test` | Test suite on Ruby 3.2, 3.3, 3.4, and 4.0 (every version the gemspec supports). |
|
|
579
|
+
| `lint` | `rubyfmt --check .`, using the release binary pinned in `RUBYFMT_VERSION`. |
|
|
580
|
+
| `types` | `rake types` (`rbs validate` + `steep check`) on Ruby 4.0. |
|
|
581
|
+
| `scan_ruby` | `rake security:audit` against a freshly updated advisory database. |
|
|
582
|
+
| `secrets` | gitleaks over the commits the push or PR adds. |
|
|
583
|
+
|
|
584
|
+
The `test` job installs gems without the `development` and `tools` groups, because some of those gems require Ruby >= 4.0, and loads the test files directly instead of through rake. When you widen the gemspec's Ruby range or upgrade rubyfmt locally, update the matrix or `RUBYFMT_VERSION` to match.
|
|
585
|
+
|
|
524
586
|
### Security issues
|
|
525
587
|
|
|
526
588
|
Please report security-sensitive issues privately rather than via a public GitHub issue. (TODO: contact email or GitHub Security Advisory link.)
|
|
527
589
|
|
|
528
590
|
### Style
|
|
529
591
|
|
|
530
|
-
The project uses [`
|
|
592
|
+
The project uses [`rubyfmt`](https://github.com/fables-tales/rubyfmt) for formatting. It's a standalone binary, not a gem: `brew install rubyfmt`. Paths in `.rubyfmtignore` are skipped.
|
|
593
|
+
|
|
594
|
+
```sh
|
|
595
|
+
bundle exec rake lint # run all linters (currently rubyfmt)
|
|
596
|
+
bundle exec rake lint:all:autocorrect # apply all autocorrects
|
|
597
|
+
bundle exec rake lint:rubyfmt # check formatting
|
|
598
|
+
bundle exec rake lint:rubyfmt:autocorrect # reformat in place
|
|
599
|
+
```
|
data/lib/radioactive/errors.rb
CHANGED
|
@@ -1,19 +1,26 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
module Radioactive
|
|
4
|
-
class Error < StandardError
|
|
4
|
+
class Error < StandardError
|
|
5
|
+
end
|
|
5
6
|
|
|
6
|
-
class SchemeError < Error
|
|
7
|
+
class SchemeError < Error
|
|
8
|
+
end
|
|
7
9
|
|
|
8
|
-
class AddressError < Error
|
|
10
|
+
class AddressError < Error
|
|
11
|
+
end
|
|
9
12
|
|
|
10
|
-
class TimeoutError < Error
|
|
13
|
+
class TimeoutError < Error
|
|
14
|
+
end
|
|
11
15
|
|
|
12
|
-
class SizeError < Error
|
|
16
|
+
class SizeError < Error
|
|
17
|
+
end
|
|
13
18
|
|
|
14
|
-
class RedirectError < Error
|
|
19
|
+
class RedirectError < Error
|
|
20
|
+
end
|
|
15
21
|
|
|
16
|
-
class EncodingError < Error
|
|
22
|
+
class EncodingError < Error
|
|
23
|
+
end
|
|
17
24
|
|
|
18
25
|
class ResponseError < Error
|
|
19
26
|
attr_reader :status, :headers, :body
|
data/lib/radioactive/fetcher.rb
CHANGED
|
@@ -15,6 +15,18 @@ module Radioactive
|
|
|
15
15
|
CHUNK_SIZE = 16 * 1024
|
|
16
16
|
DEFAULT_USER_AGENT = "Radioactive/#{Radioactive::VERSION}"
|
|
17
17
|
|
|
18
|
+
# Connect-phase transport failures that justify trying the next pinned
|
|
19
|
+
# candidate. Anything else (TLS handshake failure, read timeout,
|
|
20
|
+
# post-connect ECONNRESET) is terminal — the server engaged with us, so
|
|
21
|
+
# silently retrying against a different IP would mask real problems and
|
|
22
|
+
# risks duplicating side-effecting requests in a future non-GET world.
|
|
23
|
+
CONNECT_FALLBACK_ERRORS = [
|
|
24
|
+
Errno::EHOSTUNREACH,
|
|
25
|
+
Errno::ENETUNREACH,
|
|
26
|
+
Errno::ECONNREFUSED,
|
|
27
|
+
Net::OpenTimeout
|
|
28
|
+
].freeze
|
|
29
|
+
|
|
18
30
|
# Single-label hosts that are entirely digits or 0x-prefix hex are not
|
|
19
31
|
# valid RFC 1123 hostnames; they're SSRF-bypass attempts that some libc
|
|
20
32
|
# getaddrinfo implementations historically resolved as IPs.
|
|
@@ -84,6 +96,7 @@ module Radioactive
|
|
|
84
96
|
if opts.key?(:max_redirects) && opts[:max_redirects].negative?
|
|
85
97
|
raise ArgumentError, "max_redirects must be >= 0"
|
|
86
98
|
end
|
|
99
|
+
|
|
87
100
|
if opts.key?(:max_size) && opts[:max_size] <= 0
|
|
88
101
|
raise ArgumentError, "max_size must be > 0"
|
|
89
102
|
end
|
|
@@ -110,8 +123,8 @@ module Radioactive
|
|
|
110
123
|
loop do
|
|
111
124
|
check_deadline(deadline, clock)
|
|
112
125
|
|
|
113
|
-
|
|
114
|
-
kind, status, headers, body = perform_request(current,
|
|
126
|
+
ips = pin_addresses(current, resolver, opts)
|
|
127
|
+
kind, status, headers, body = perform_request(current, ips, opts, deadline, clock, &chunk_block)
|
|
115
128
|
|
|
116
129
|
case kind
|
|
117
130
|
when :redirect
|
|
@@ -122,9 +135,13 @@ module Radioactive
|
|
|
122
135
|
current = resolve_redirect(current, headers["location"], opts)
|
|
123
136
|
when :final
|
|
124
137
|
unless (200..299).cover?(status)
|
|
125
|
-
raise
|
|
126
|
-
|
|
127
|
-
|
|
138
|
+
raise(
|
|
139
|
+
ResponseError.new(
|
|
140
|
+
"non-success status: #{status}",
|
|
141
|
+
status: status,
|
|
142
|
+
headers: headers,
|
|
143
|
+
body: body
|
|
144
|
+
)
|
|
128
145
|
)
|
|
129
146
|
end
|
|
130
147
|
|
|
@@ -141,15 +158,17 @@ module Radioactive
|
|
|
141
158
|
|
|
142
159
|
def parse_url(url, opts)
|
|
143
160
|
uri = url.is_a?(URI) ? url.dup : URI.parse(url.to_s)
|
|
144
|
-
|
|
161
|
+
host = uri.host
|
|
162
|
+
raise SchemeError, "URL has no host" if host.nil? || host.empty?
|
|
145
163
|
unless opts[:schemes].include?(uri.scheme)
|
|
146
164
|
raise SchemeError, "scheme not allowed: #{uri.scheme.inspect}"
|
|
147
165
|
end
|
|
166
|
+
|
|
148
167
|
if (uri.userinfo || uri.user) && !opts[:allow_credentials]
|
|
149
168
|
raise SchemeError, "embedded credentials not allowed"
|
|
150
169
|
end
|
|
151
170
|
|
|
152
|
-
uri.host = canonicalize_host(
|
|
171
|
+
uri.host = canonicalize_host(host)
|
|
153
172
|
uri.fragment = nil
|
|
154
173
|
uri
|
|
155
174
|
rescue URI::InvalidURIError => e
|
|
@@ -170,8 +189,13 @@ module Radioactive
|
|
|
170
189
|
host
|
|
171
190
|
end
|
|
172
191
|
|
|
173
|
-
|
|
174
|
-
|
|
192
|
+
# Resolve the host once and validate every address against the forbidden
|
|
193
|
+
# ranges before returning. Returns the full list in resolver order so the
|
|
194
|
+
# connect loop can fall back across dual-stack records when one path is
|
|
195
|
+
# down — the strict "any forbidden → reject" rule still defeats dual-A
|
|
196
|
+
# SSRF because validation runs upfront, before any socket opens.
|
|
197
|
+
def pin_addresses(uri, resolver, opts)
|
|
198
|
+
host = uri.host or raise(AddressError, "URL has no host")
|
|
175
199
|
addresses = AddressCheck.resolve(host, resolver)
|
|
176
200
|
raise AddressError, "no addresses for #{host}" if addresses.empty?
|
|
177
201
|
|
|
@@ -183,7 +207,7 @@ module Radioactive
|
|
|
183
207
|
end
|
|
184
208
|
end
|
|
185
209
|
|
|
186
|
-
addresses
|
|
210
|
+
addresses
|
|
187
211
|
end
|
|
188
212
|
|
|
189
213
|
def resolve_redirect(current, location, opts)
|
|
@@ -209,44 +233,84 @@ module Radioactive
|
|
|
209
233
|
[value, remaining].min
|
|
210
234
|
end
|
|
211
235
|
|
|
212
|
-
def perform_request(uri,
|
|
213
|
-
|
|
236
|
+
def perform_request(uri, ips, opts, deadline, clock, &chunk_block)
|
|
237
|
+
# Build the request up front so caller-supplied input (e.g. CRLF in
|
|
238
|
+
# headers) is rejected before any socket opens — independent of which
|
|
239
|
+
# candidate we'd reach.
|
|
214
240
|
req = build_request(uri, opts)
|
|
241
|
+
# @type var last_connect_error: Exception?
|
|
242
|
+
last_connect_error = nil
|
|
215
243
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
244
|
+
ips.each do |ip|
|
|
245
|
+
http = build_http(uri, ip, opts, deadline, clock)
|
|
246
|
+
|
|
247
|
+
begin
|
|
248
|
+
http.start
|
|
249
|
+
rescue *CONNECT_FALLBACK_ERRORS => e
|
|
250
|
+
last_connect_error = e
|
|
251
|
+
next
|
|
252
|
+
rescue OpenSSL::SSL::SSLError,
|
|
253
|
+
SocketError,
|
|
254
|
+
Errno::ECONNRESET,
|
|
255
|
+
IOError => e
|
|
256
|
+
raise ResponseError, "transport error: #{e.class}: #{e.message}"
|
|
257
|
+
end
|
|
258
|
+
|
|
259
|
+
begin
|
|
260
|
+
return execute_request(http, req, opts, deadline, clock, &chunk_block)
|
|
261
|
+
rescue Net::OpenTimeout, Net::ReadTimeout => e
|
|
262
|
+
raise TimeoutError, e.message
|
|
263
|
+
rescue OpenSSL::SSL::SSLError,
|
|
264
|
+
SocketError,
|
|
265
|
+
Errno::ECONNREFUSED,
|
|
266
|
+
Errno::EHOSTUNREACH,
|
|
267
|
+
Errno::ENETUNREACH,
|
|
268
|
+
Errno::ECONNRESET,
|
|
269
|
+
IOError => e
|
|
270
|
+
raise ResponseError, "transport error: #{e.class}: #{e.message}"
|
|
271
|
+
ensure
|
|
272
|
+
begin
|
|
273
|
+
http.finish if http.started?
|
|
274
|
+
rescue IOError
|
|
275
|
+
# already closed
|
|
235
276
|
end
|
|
236
277
|
end
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
278
|
+
end
|
|
279
|
+
|
|
280
|
+
# All candidates failed at connect. Surface the last error in the same
|
|
281
|
+
# shape a single-address failure would have taken. pin_addresses never
|
|
282
|
+
# returns an empty list, so the nil branch is unreachable.
|
|
283
|
+
raise ResponseError, "no candidate addresses" if last_connect_error.nil?
|
|
284
|
+
raise TimeoutError, last_connect_error.message if last_connect_error.is_a?(Net::OpenTimeout)
|
|
285
|
+
raise ResponseError, "transport error: #{last_connect_error.class}: #{last_connect_error.message}"
|
|
286
|
+
end
|
|
287
|
+
|
|
288
|
+
def execute_request(http, req, opts, deadline, clock, &chunk_block)
|
|
289
|
+
result = nil
|
|
290
|
+
|
|
291
|
+
http.request(req) do |res|
|
|
292
|
+
code = res.code.to_i
|
|
293
|
+
headers = headers_hash(res)
|
|
294
|
+
|
|
295
|
+
if REDIRECT_STATUSES.include?(code) && headers["location"]
|
|
296
|
+
result = [:redirect, code, headers, nil]
|
|
297
|
+
elsif (200..299).cover?(code)
|
|
298
|
+
# 2xx: stream chunks straight to caller; no buffering here.
|
|
299
|
+
read_body!(res, headers, opts, deadline, clock, &chunk_block)
|
|
300
|
+
result = [:final, code, headers, nil]
|
|
301
|
+
else
|
|
302
|
+
# Non-2xx: buffer body so ResponseError can carry partial data.
|
|
303
|
+
error_body = String.new(capacity: CHUNK_SIZE)
|
|
304
|
+
read_body!(res, headers, opts, deadline, clock) { |chunk| error_body << chunk }
|
|
305
|
+
result = [:final, code, headers, error_body]
|
|
306
|
+
end
|
|
243
307
|
end
|
|
244
308
|
|
|
245
309
|
result || raise(ResponseError, "request produced no response")
|
|
246
310
|
end
|
|
247
311
|
|
|
248
312
|
def build_http(uri, ip, opts, deadline, clock)
|
|
249
|
-
host = uri.host or raise
|
|
313
|
+
host = uri.host or raise(SchemeError, "URL has no host")
|
|
250
314
|
port = uri.port || ((uri.scheme == "https") ? 443 : 80)
|
|
251
315
|
http = Net::HTTP.new(host, port)
|
|
252
316
|
http.ipaddr = ip.to_s
|
|
@@ -256,12 +320,15 @@ module Radioactive
|
|
|
256
320
|
if (open_t = clamp_timeout(opts[:open_timeout], deadline, clock))
|
|
257
321
|
http.open_timeout = open_t
|
|
258
322
|
end
|
|
323
|
+
|
|
259
324
|
if (read_t = clamp_timeout(opts[:read_timeout], deadline, clock))
|
|
260
325
|
http.read_timeout = read_t
|
|
261
326
|
end
|
|
327
|
+
|
|
262
328
|
if http.respond_to?(:write_timeout=) && (write_t = clamp_timeout(opts[:open_timeout], deadline, clock))
|
|
263
329
|
http.write_timeout = write_t
|
|
264
330
|
end
|
|
331
|
+
|
|
265
332
|
http
|
|
266
333
|
end
|
|
267
334
|
|
|
@@ -282,6 +349,7 @@ module Radioactive
|
|
|
282
349
|
|
|
283
350
|
req[name] = value
|
|
284
351
|
end
|
|
352
|
+
|
|
285
353
|
req
|
|
286
354
|
end
|
|
287
355
|
|
|
@@ -334,6 +402,7 @@ module Radioactive
|
|
|
334
402
|
|
|
335
403
|
yield decoded unless decoded.empty?
|
|
336
404
|
end
|
|
405
|
+
|
|
337
406
|
tail = inflater.finish
|
|
338
407
|
unless tail.empty?
|
|
339
408
|
total += tail.bytesize
|
|
@@ -341,6 +410,7 @@ module Radioactive
|
|
|
341
410
|
|
|
342
411
|
yield tail
|
|
343
412
|
end
|
|
413
|
+
|
|
344
414
|
rescue Zlib::Error => e
|
|
345
415
|
raise EncodingError, "gzip decode failed: #{e.message}"
|
|
346
416
|
ensure
|
data/lib/radioactive/version.rb
CHANGED
data/lib/radioactive.rb
CHANGED
data/lib/tasks/lint/all.rake
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
namespace :lint do
|
|
2
2
|
desc "Run all linter checks"
|
|
3
|
-
task all: %i[
|
|
3
|
+
task all: %i[rubyfmt] do
|
|
4
4
|
puts "All lints completed successfully!"
|
|
5
5
|
end
|
|
6
6
|
|
|
7
7
|
namespace :all do
|
|
8
8
|
desc "Run all linter autocorrects"
|
|
9
|
-
task autocorrect: %i[
|
|
9
|
+
task autocorrect: %i[rubyfmt:autocorrect]
|
|
10
10
|
end
|
|
11
11
|
end
|
data/sig/radioactive.rbs
CHANGED
|
@@ -136,6 +136,7 @@ module Radioactive
|
|
|
136
136
|
RESERVED_HEADERS: Array[String]
|
|
137
137
|
CHUNK_SIZE: Integer
|
|
138
138
|
DEFAULT_USER_AGENT: String
|
|
139
|
+
CONNECT_FALLBACK_ERRORS: Array[Class]
|
|
139
140
|
NUMERIC_ONLY_HOST: Regexp
|
|
140
141
|
HEADER_INVALID_CHAR: Regexp
|
|
141
142
|
DEFAULTS: Hash[Symbol, untyped]
|
|
@@ -219,11 +220,12 @@ module Radioactive
|
|
|
219
220
|
def run_streaming: (uri_like url, Hash[Symbol, untyped] call_opts) { (String) -> void } -> Hash[Symbol, untyped]
|
|
220
221
|
def parse_url: (uri_like url, Hash[Symbol, untyped] opts) -> URI::Generic
|
|
221
222
|
def canonicalize_host: (String host) -> String
|
|
222
|
-
def
|
|
223
|
+
def pin_addresses: (URI::Generic uri, AddressCheck::_Resolver resolver, Hash[Symbol, untyped] opts) -> Array[IPAddr]
|
|
223
224
|
def resolve_redirect: (URI::Generic current, String location, Hash[Symbol, untyped] opts) -> URI::Generic
|
|
224
225
|
def check_deadline: (Float? deadline, _Clock clock) -> void
|
|
225
226
|
def clamp_timeout: ((Float | Integer)? value, Float? deadline, _Clock clock) -> (Float | Integer)?
|
|
226
|
-
def perform_request: (URI::Generic uri, IPAddr
|
|
227
|
+
def perform_request: (URI::Generic uri, Array[IPAddr] ips, Hash[Symbol, untyped] opts, Float? deadline, _Clock clock) { (String) -> void } -> Array[untyped]
|
|
228
|
+
def execute_request: (Net::HTTP http, Net::HTTPRequest req, Hash[Symbol, untyped] opts, Float? deadline, _Clock clock) { (String) -> void } -> Array[untyped]
|
|
227
229
|
def build_http: (URI::Generic uri, IPAddr ip, Hash[Symbol, untyped] opts, Float? deadline, _Clock clock) -> Net::HTTP
|
|
228
230
|
def build_request: (URI::Generic uri, Hash[Symbol, untyped] opts) -> Net::HTTPRequest
|
|
229
231
|
def headers_hash: (Net::HTTPResponse res) -> header_hash
|
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: radioactive
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Pawel Osiczko
|
|
@@ -75,8 +75,10 @@ files:
|
|
|
75
75
|
- lib/radioactive/version.rb
|
|
76
76
|
- lib/tasks/gem.rake
|
|
77
77
|
- lib/tasks/lint/all.rake
|
|
78
|
-
- lib/tasks/lint/
|
|
79
|
-
- lib/tasks/security.rake
|
|
78
|
+
- lib/tasks/lint/rubyfmt.rake
|
|
79
|
+
- lib/tasks/security/all.rake
|
|
80
|
+
- lib/tasks/security/bundler_audit.rake
|
|
81
|
+
- lib/tasks/security/gitleaks.rake
|
|
80
82
|
- lib/tasks/types.rake
|
|
81
83
|
- sig/radioactive.rbs
|
|
82
84
|
- sig/zeitwerk.rbs
|
|
@@ -98,14 +100,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
98
100
|
requirements:
|
|
99
101
|
- - ">="
|
|
100
102
|
- !ruby/object:Gem::Version
|
|
101
|
-
version: 3.
|
|
103
|
+
version: 3.3.0
|
|
102
104
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
103
105
|
requirements:
|
|
104
106
|
- - ">="
|
|
105
107
|
- !ruby/object:Gem::Version
|
|
106
108
|
version: '0'
|
|
107
109
|
requirements: []
|
|
108
|
-
rubygems_version: 4.0.
|
|
110
|
+
rubygems_version: 4.0.23
|
|
109
111
|
specification_version: 4
|
|
110
112
|
summary: Hardened HTTP fetcher for Ruby. Safe to point at URLs supplied by untrusted
|
|
111
113
|
users.
|
metadata.gz.sig
CHANGED
|
Binary file
|
data/lib/tasks/lint/rubocop.rake
DELETED
|
@@ -1,15 +0,0 @@
|
|
|
1
|
-
require "rubocop/rake_task"
|
|
2
|
-
|
|
3
|
-
namespace :lint do
|
|
4
|
-
desc "Run rubocop linter check"
|
|
5
|
-
task :rubocop do
|
|
6
|
-
exec "bundle exec rubocop"
|
|
7
|
-
end
|
|
8
|
-
|
|
9
|
-
namespace :rubocop do
|
|
10
|
-
desc "Run rubocop autocorrect"
|
|
11
|
-
task :autocorrect do
|
|
12
|
-
exec "bundle exec rubocop --autocorrect-all"
|
|
13
|
-
end
|
|
14
|
-
end
|
|
15
|
-
end
|
|
File without changes
|