radioactive 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +85 -0
- data/CLAUDE.md +52 -0
- data/README.md +530 -0
- data/Rakefile +11 -0
- data/Steepfile +24 -0
- data/lib/radioactive/address_check.rb +59 -0
- data/lib/radioactive/errors.rb +28 -0
- data/lib/radioactive/fetcher.rb +355 -0
- data/lib/radioactive/monotonic_clock.rb +9 -0
- data/lib/radioactive/result.rb +5 -0
- data/lib/radioactive/version.rb +5 -0
- data/lib/radioactive.rb +19 -0
- data/lib/tasks/gem.rake +5 -0
- data/lib/tasks/lint/all.rake +11 -0
- data/lib/tasks/lint/rubocop.rake +15 -0
- data/lib/tasks/security.rake +11 -0
- data/lib/tasks/types.rake +16 -0
- data/sig/radioactive.rbs +234 -0
- data/sig/zeitwerk.rbs +13 -0
- data.tar.gz.sig +0 -0
- metadata +112 -0
- metadata.gz.sig +0 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: db78ab47dc98a9125d0dde9cb3ec17e95b83cafc3fae5922fa3f48cbcb85f0f2
|
|
4
|
+
data.tar.gz: 8da7cf85d58c94bab1a13466ef7a236b24a2da1d6e021835bd5bf959b690d38c
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 594fcc7d8f2f3d670154b11d9d0194df8f9fe2da2e8d4eadf222516cb76b6b1d82ac70214e873ccdaaf9947f95f0ca6012aba1079ee8942e25ad50845b2035be
|
|
7
|
+
data.tar.gz: 35ee0b007e1e41807b9376d80c1e98d6b933af4388b8e4dae0aef6d5dc451d6ec0f012284619d9390cb4f055db82a5efce4199f73220df2ba8c26680a5006331
|
checksums.yaml.gz.sig
ADDED
|
Binary file
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file. The format
|
|
4
|
+
is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this
|
|
5
|
+
project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
## [Unreleased]
|
|
8
|
+
|
|
9
|
+
## [0.1.1] - 2026-05-04
|
|
10
|
+
|
|
11
|
+
Tooling and documentation only. No runtime behavior changes.
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
|
|
15
|
+
- Gem build / install / release rake tasks moved from the top-level namespace
|
|
16
|
+
to `gem:`, matching the project's `lint:` / `types:` / `security:` pattern.
|
|
17
|
+
Use `rake gem:build`, `rake gem:install`, `rake gem:release[remote]`.
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
|
|
21
|
+
- RBS signatures for three Fetcher constants/methods added in 0.1.0 were
|
|
22
|
+
missing from `sig/radioactive.rbs` (`NUMERIC_ONLY_HOST`, `HEADER_INVALID_CHAR`,
|
|
23
|
+
`Fetcher#canonicalize_host`). Caught by Steep on first run of `rake types:check`
|
|
24
|
+
after the security hardening landed.
|
|
25
|
+
|
|
26
|
+
### Documentation
|
|
27
|
+
|
|
28
|
+
- New Releasing section in the README covering pre-flight checks, the
|
|
29
|
+
`rake gem:release` flow, and a pointer to RubyGems Trusted Publishing
|
|
30
|
+
for stronger publishing setups.
|
|
31
|
+
|
|
32
|
+
## [0.1.0] - 2026-05-04
|
|
33
|
+
|
|
34
|
+
Initial release.
|
|
35
|
+
|
|
36
|
+
### Added
|
|
37
|
+
|
|
38
|
+
- `Radioactive.fetch(url, **opts)` returning a frozen
|
|
39
|
+
`Result(url, final_url, status, headers, body, hops)`.
|
|
40
|
+
- `Radioactive.open(url, **opts)` returning a `StringIO`; with a block, streams
|
|
41
|
+
the body chunk-by-chunk to a `Tempfile` so peak memory stays at ~16 KB
|
|
42
|
+
regardless of `max_size`.
|
|
43
|
+
- `Radioactive::Fetcher` class for reusable per-instance configuration.
|
|
44
|
+
- 14 configuration options: `schemes`, `max_size`, `open_timeout`,
|
|
45
|
+
`read_timeout`, `total_timeout`, `max_redirects`, `accept_encoding`,
|
|
46
|
+
`user_agent`, `private_ranges`, `allow_private`, `allow_credentials`,
|
|
47
|
+
`headers`, `resolver`, `clock`.
|
|
48
|
+
- Distinct `Radioactive::Error` subclasses for every failure mode:
|
|
49
|
+
`SchemeError`, `AddressError`, `TimeoutError`, `SizeError`, `RedirectError`,
|
|
50
|
+
`EncodingError`, `ResponseError` (the last carrying `#status`, `#headers`,
|
|
51
|
+
`#body` for partial response data).
|
|
52
|
+
- RBS signatures in `sig/radioactive.rbs`, validated by `rake types:validate`
|
|
53
|
+
and type-checked against the implementation by `rake types:check` (Steep).
|
|
54
|
+
|
|
55
|
+
### Security
|
|
56
|
+
|
|
57
|
+
Defenses on by default with zero configuration:
|
|
58
|
+
|
|
59
|
+
- **SSRF address blocklist.** 25 CIDR ranges blocked: RFC1918, loopback,
|
|
60
|
+
link-local (incl. cloud metadata at `169.254.169.254`), CGNAT, IPv6 ULA,
|
|
61
|
+
IPv6 link-local, multicast, TEST-NET, Teredo, and reserved ranges.
|
|
62
|
+
- **DNS rebinding defense.** Hostname is resolved once; the resolved IP is
|
|
63
|
+
pinned via `Net::HTTP#ipaddr=`; SNI and certificate verification still use
|
|
64
|
+
the original hostname.
|
|
65
|
+
- **Strict dual-A rejection.** If *any* resolved address falls in a forbidden
|
|
66
|
+
range, the request is refused (defeats split-horizon SSRF tricks).
|
|
67
|
+
- **Redirect re-validation.** Every redirect target is re-run through the full
|
|
68
|
+
pipeline (scheme, credentials, DNS, address check) before the next request.
|
|
69
|
+
- **Scheme allowlist.** Default `%w[http https]`; `file://`, `gopher://`,
|
|
70
|
+
`javascript:`, etc. are rejected.
|
|
71
|
+
- **Embedded-credential rejection.** URLs with `userinfo` are refused unless
|
|
72
|
+
`allow_credentials: true` is set explicitly.
|
|
73
|
+
- **Non-canonical IP rejection.** `http://2130706433/` (decimal) and
|
|
74
|
+
`http://0x7f000001/` (hex) hosts are rejected outright rather than relying
|
|
75
|
+
on resolver behavior.
|
|
76
|
+
- **Header CRLF/NUL injection rejection.** Caller-supplied header names and
|
|
77
|
+
values containing `\r`, `\n`, or `\0` are refused before the socket opens.
|
|
78
|
+
- **Slowloris and stall defenses.** `read_timeout` per chunk; `total_timeout`
|
|
79
|
+
applied as a wall-clock deadline across all redirects, with per-operation
|
|
80
|
+
timeouts clamped to remaining budget.
|
|
81
|
+
- **Response and decompression bombs.** `max_size` enforced per chunk on the
|
|
82
|
+
raw socket; `Content-Length` exceeding `max_size` rejected before any body
|
|
83
|
+
is read; default `Accept-Encoding: identity` rejects compressed responses;
|
|
84
|
+
opt-in `gzip` decompression bounded by **decoded** byte count.
|
|
85
|
+
- **TLS verification.** `OpenSSL::SSL::VERIFY_PEER`, no opt-out.
|
data/CLAUDE.md
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
+
|
|
5
|
+
## What this gem is
|
|
6
|
+
|
|
7
|
+
Radioactive is a hardened HTTP fetcher for URLs supplied by untrusted users — a near drop-in alternative to `URI.open` for link previews, image proxying, webhook delivery, and metadata extraction. It defends against the SSRF and DoS classes that `URI.open` leaves open (cloud-metadata exfiltration, RFC1918 access, DNS rebinding, slowloris, response/decompression bombs, redirect chains into private addresses, disallowed schemes).
|
|
8
|
+
|
|
9
|
+
**The spec is `docs/REQUIREMENTS.md` and it is authoritative.** Read it before designing or changing behavior; threat model, public API, defaults, error hierarchy, and explicit non-goals all live there. This file only summarizes things that constrain *how* you write code.
|
|
10
|
+
|
|
11
|
+
## Status
|
|
12
|
+
|
|
13
|
+
The scaffold from `bundle gem radioactive` is still mostly untouched: `lib/radioactive.rb` defines only the module and a base `Error` class, `test/test_radioactive.rb` contains the generator's failing `assert false`, and the gemspec metadata (`summary`, `description`, `homepage`, `allowed_push_host`, `source_code_uri`, `changelog_uri`) is still TODO placeholders. Implementation work is greenfield — match the API surface in `docs/REQUIREMENTS.md` rather than inventing one.
|
|
14
|
+
|
|
15
|
+
Note: the parent directory `/Users/pablo/code/posiczko/CLAUDE.md` documents a different project (Ougai). It does not apply here — Radioactive uses Minitest (not RSpec), Standard (not RuboCop directly), and has no Oj/JrJackson dependency.
|
|
16
|
+
|
|
17
|
+
## Architectural constraints (from the spec)
|
|
18
|
+
|
|
19
|
+
These are non-negotiable design choices in `docs/REQUIREMENTS.md`. Don't propose changes that violate them without flagging the spec change first:
|
|
20
|
+
|
|
21
|
+
- **No global state, no `Radioactive.configure`.** Configuration is per-call or per-`Fetcher` instance. This is explicit in the spec to keep tests sane and avoid cross-tenant leaks.
|
|
22
|
+
- **`Net::HTTP` only.** No Faraday, HTTParty, `http.rb`, HTTP/2, HTTP/3, or gRPC. Use `Net::HTTP.start` directly — not `URI.open` — so timeouts and chunked reads are first-class.
|
|
23
|
+
- **GET only in v1.** No POST/PUT/DELETE. No WebSocket/SSE.
|
|
24
|
+
- **No outbound proxy support in v1**, no connection pooling, no HTTP caching, no retries, no circuit breakers, no MIME sniffing.
|
|
25
|
+
- **TLS:** system trust store is authoritative. `verify_mode = VERIFY_PEER` and there is no option to disable it. No TLS pinning, CT, or mTLS.
|
|
26
|
+
- **DNS pinning preserves SNI.** Resolve once via the injected `resolver`, validate every resolved address against `private_ranges`, then connect via `http.ipaddr = ip` while leaving `http.address` as the hostname so SNI and cert verification still work. Re-resolve and re-validate on every redirect (defeats DNS rebinding TOCTOU).
|
|
27
|
+
- **Strict address check:** if *any* resolved address is in a forbidden range, reject — defeats dual-A-record SSRF.
|
|
28
|
+
- **Size enforcement is on the raw socket reader**, not after decompression. Default `Accept-Encoding: identity`; compression is opt-in. Per-chunk size check (16 KB chunks); also reject when `Content-Length` exceeds `max_size` *before* reading the body.
|
|
29
|
+
- **All failures raise distinct subclasses of `Radioactive::Error`** so callers can rescue the base class and degrade gracefully. The hierarchy (`SchemeError`, `AddressError`, `TimeoutError`, `SizeError`, `RedirectError`, `EncodingError`, `ResponseError`) is fixed by the spec — don't invent new top-level error classes.
|
|
30
|
+
- **`Result` is a frozen `Data.define(:url, :final_url, :status, :headers, :body, :hops)`.** Don't change the shape; downstream code pattern-matches on it.
|
|
31
|
+
- **Inject `resolver` and `clock`** rather than calling `Resolv` / `Process.clock_gettime` directly inside fetch logic — the spec calls these out as test seams.
|
|
32
|
+
- **Sockets close on every exit path**, success or raise. No leaking connections.
|
|
33
|
+
|
|
34
|
+
## Commands
|
|
35
|
+
|
|
36
|
+
- `bin/setup` — install dependencies (wraps `bundle install`).
|
|
37
|
+
- `bin/console` — IRB session with the gem preloaded.
|
|
38
|
+
- `bundle exec rake` — default task: runs `test` then `standard` (lint). Both must pass.
|
|
39
|
+
- `bundle exec rake test` — Minitest suite (driven by `Minitest::TestTask`, picks up `test/**/test_*.rb`).
|
|
40
|
+
- `bundle exec rake test TESTOPTS="--name=/pattern/"` — run a single test or matching subset.
|
|
41
|
+
- `ruby -Ilib -Itest test/test_radioactive.rb` — run one test file directly without rake.
|
|
42
|
+
- `bundle exec rake standard` / `bundle exec rake standard:fix` — lint / autofix.
|
|
43
|
+
- `bundle exec rake build` / `install` / `release` — gem packaging tasks (release is maintainer-only and currently blocked by the `allowed_push_host` TODO in the gemspec).
|
|
44
|
+
|
|
45
|
+
## Layout and conventions
|
|
46
|
+
|
|
47
|
+
- Code lives under `lib/radioactive/`; the entry point `lib/radioactive.rb` requires `radioactive/version` and defines the `Radioactive` module. New files should be required from there.
|
|
48
|
+
- Long-form design docs live in `docs/`. `docs/REQUIREMENTS.md` is the spec; treat it as a contract and update it deliberately (in the same commit as behavior changes) rather than letting code and spec drift.
|
|
49
|
+
- RBS signatures live in `sig/radioactive.rbs` — keep them in sync when adding public API.
|
|
50
|
+
- Versioning: bump `Radioactive::VERSION` in `lib/radioactive/version.rb` for releases.
|
|
51
|
+
- Ruby version: gemspec requires `>= 3.2.0`; `.standard.yml` pins the Standard target to `3.2`. Match that when writing code (no 3.3-only syntax unless you also bump these).
|
|
52
|
+
- CI (`.github/workflows/main.yml`) only runs on pull requests — its `push` trigger is wired to a placeholder branch (`.invalid`), so pushes to `main` will not run CI until that's fixed. The matrix currently lists Ruby `'4.0.3'`, which does not exist; expect CI to fail until the version is corrected.
|