otto 2.9.0 → 2.10.0

data/docs/guides/configuration_freezing.md

@@ -0,0 +1,146 @@
+# Configuration freezing
+
+Otto supports multi-step boot configuration. In normal operation, it freezes the
+configured application immediately before processing the first request. When
+`RSpec` is defined, automatic request-time freezing is skipped so test suites can
+assemble applications incrementally.
+
+Configuration must be complete before traffic reaches the application. For a
+more explicit boot boundary, call `freeze_configuration!` after setup and before
+starting the Rack server.
+
+## Configure before serving requests
+
+```ruby
+otto = Otto.new('routes.txt')
+
+otto.add_auth_strategy(
+  'session',
+  Otto::Security::Authentication::Strategies::SessionStrategy.new(
+    session_key: 'user_id'
+  )
+)
+otto.add_auth_strategy(
+  'api_key',
+  Otto::Security::Authentication::Strategies::APIKeyStrategy.new(
+    api_keys: ENV.fetch('API_KEYS').split(',')
+  )
+)
+otto.enable_csrf_protection!
+otto.use MyApp::Middleware
+otto.register_error_handler(MyApp::NotFound, status: 404, log_level: :info)
+
+# Optional: make the end of boot explicit rather than waiting for a request.
+otto.freeze_configuration!
+```
+
+Calling a supported configuration method after freezing raises `FrozenError`:
+
+```ruby
+otto.add_auth_strategy('other', MyApp::OtherStrategy.new)
+# FrozenError: Cannot modify frozen configuration
+```
+
+`freeze_configuration!` is idempotent and returns the Otto instance.
+
+## What is frozen
+
+`freeze_configuration!` freezes these owned structures:
+
+- security and locale configuration objects;
+- the middleware stack;
+- authentication configuration and instance options;
+- dynamic and literal route tables;
+- route definitions and reverse-route indexes.
+
+Hashes and arrays inside those structures are recursively frozen. Configuration
+objects that implement `deep_freeze!` can prepare memoized values before they
+are frozen.
+
+The static-file route structure is an intentional exception. Its outer hash is
+frozen, but the `routes_static[:GET]` `Concurrent::Map` remains writable because
+Otto caches newly discovered static paths during requests.
+
+## Scope and current limitations
+
+Freezing is a guard around Otto's supported configuration APIs, not a complete
+object-capability boundary. In particular, the current implementation exposes
+some state that is not included in `freeze_configuration!`:
+
+- `error_handlers` remains a mutable Hash, although
+  `register_error_handler` rejects calls after freezing;
+- the `not_found` and `server_error` fallback response writers remain available;
+- the inner static-file cache remains mutable by design.
+
+Application code should not mutate those objects directly after boot. Do not
+state or depend on a guarantee that every object reachable from an Otto instance
+is deeply immutable.
+
+## Expected failures after freezing
+
+These supported mutation paths reject changes after the freeze boundary:
+
+```ruby
+# Security configuration
+otto.security_config.disable_csrf_protection!
+otto.add_trusted_proxy('192.0.2.10')
+otto.add_rate_limit_rule('uploads', limit: 5, period: 60)
+
+# Middleware and authentication
+otto.use MyApp::OtherMiddleware
+otto.add_auth_strategy('other', MyApp::OtherStrategy.new)
+
+# Helpers and expected-error handling
+otto.register_request_helpers(MyApp::RequestHelpers)
+otto.register_response_helpers(MyApp::ResponseHelpers)
+otto.register_error_handler(MyApp::RateLimited, status: 429)
+```
+
+Direct mutation of frozen nested structures also raises `FrozenError`:
+
+```ruby
+otto.security_config.rate_limiting_config[:custom_rules] = {}
+otto.auth_config[:auth_strategies] = {}
+otto.routes[:GET] = []
+```
+
+## Testing the boundary
+
+Otto deliberately skips automatic first-request freezing when `RSpec` is
+defined. A freeze-focused spec must freeze explicitly:
+
+```ruby
+RSpec.describe 'configuration freezing' do
+  it 'rejects configuration changes after the boot boundary' do
+    otto = Otto.new
+    otto.freeze_configuration!
+
+    expect(otto.frozen_configuration?).to be(true)
+    expect {
+      otto.add_auth_strategy('other', MyApp::OtherStrategy.new)
+    }.to raise_error(FrozenError, /Cannot modify frozen configuration/)
+  end
+end
+```
+
+Use a fresh Otto instance when a later example needs mutable configuration.
+`Otto.unfreeze_for_testing` is an internal test helper that only resets the
+`@configuration_frozen` flag; it cannot unfreeze Ruby objects that were already
+frozen by `freeze_configuration!`.
+
+## Thread behavior
+
+For the automatic production path, `Otto#call` guards the first freeze with a
+mutex and checks the frozen flag again inside the critical section. Concurrent
+first requests therefore do not run the freeze operation concurrently.
+
+## Related source and tests
+
+- [`Otto#call`](../../lib/otto.rb) — automatic first-request boundary and the
+  RSpec exception.
+- [`Otto::Core::Configuration`](../../lib/otto/core/configuration.rb) — exact
+  freeze scope and mutation guard.
+- [`Otto::Core::Freezable`](../../lib/otto/core/freezable.rb) — recursive
+  freezing behavior.
+- [Configuration-freezing specs](../../spec/otto/configuration_freezing_spec.rb).
+- [Static-file freezing specs](../../spec/otto/static_file_freezing_spec.rb).

data/docs/guides/enrichment.md

@@ -0,0 +1,161 @@
+# ASN and anonymizer enrichment
+
+Two opt-in, country-adjacent signals layered on the same privacy pipeline as
+[geo-country](geo-country.md):
+
+- **ASN** — the network operator an address belongs to, as `req.asn` /
+  `env['otto.privacy.asn']`: `'AS15169'`.
+- **Anonymizer** — whether the address is a known anonymizing egress, as
+  `req.anonymizer` / `env['otto.privacy.anonymizer']`: `'tor'`, `'proxy'`,
+  `'vpn'`, `'residential_proxy'`, `'hosting'`, `'anonymous'`, `'none'`, `'**'`.
+
+Both are **off by default** (unlike geo) and **database-only**: no CDN
+publishes a client-ASN or anonymizer header with meaningful deployment, so
+there is no header tier and none of geo's header-trust machinery applies —
+including the `geo_header`/`trusted_proxy_depth` boot conflict.
+
+For requests that pass through the privacy fingerprint pipeline, each signal
+uses these values:
+
+| Value | Meaning |
+| --- | --- |
+| `nil` | no enrichment value was produced: the signal is off, IP privacy is disabled, no client IP resolved, or the client is exempt from fingerprinting |
+| `'**'` | the signal is on, but no database answered or the lookup failed |
+| a label | the configured reader returned a usable answer |
+
+By default, private and localhost clients are exempt from fingerprinting. Their
+enrichment environment keys are absent even when the signals are enabled. Use
+the `:anonymous` privacy profile only if those clients should also be masked and
+enriched.
+
+## Configuration
+
+Add the optional reader gem when configuring an MMDB path:
+
+```ruby
+# Gemfile
+gem 'maxmind-db', '~> 1.2'
+```
+
+Then install an ASN database and enable the signal before the first request:
+
+```sh
+bundle install
+mkdir -p data
+curl -fsSL -o data/origin-asn.mmdb \
+  https://github.com/sapics/ip-location-db/releases/download/latest/origin-asn.mmdb
+```
+
+```ruby
+otto.configure_ip_privacy(
+  asn: true,
+  asn_db_path: 'data/origin-asn.mmdb'
+)
+```
+
+Anonymizer classification requires a separate compatible database that Otto
+does not provide:
+
+```ruby
+otto.configure_ip_privacy(
+  anonymizer: true,
+  anonymizer_db_path: 'data/your-anonymous-ip.mmdb'
+)
+```
+
+Do not use that placeholder path until the file exists. Both signals also accept
+a bring-your-own reader (`asn_db_reader:` / `anonymizer_db_reader:`): any object
+responding to `#get(ip)`. A reader supplied in the same call wins over a path.
+
+The `maxmind-db` gem is loaded only when an enabled signal has a `*_db_path`.
+Otto supports version 1.2.0 or newer in the 1.x series. Missing or incompatible
+versions raise `Otto::OptionalDependencyError` during configuration. An
+unreadable path also raises during configuration when its signal is enabled; a
+path attached to a disabled signal is stored but not opened.
+
+## ASN: which address is looked up, and why that's safe
+
+The ASN lookup always uses the **masked** IP, same as geo. With the default
+IPv4 `/24` masking, accuracy depends on the chosen database assigning the same
+ASN across that `/24`; Otto accepts arbitrary MMDBs and injected readers, so it
+cannot guarantee that equivalence. IPv6 is coarser: at `octet_precision: 1`
+Otto zeroes the last 80 bits to produce a `/48`, which is wider than many IPv6
+announcements. Treat masked ASN results as best-effort and verify the behavior
+of the data source used by policy code.
+
+### Data file
+
+> **Dataset distinction:** sapics/ip-location-db publishes country and ASN data
+> as separate file types. Current country files such as `user-country.mmdb` do
+> not contain ASN records. Use an ASN file such as `origin-asn.mmdb` for this
+> signal.
+
+The recommended ASN file is
+[`origin-asn`](https://github.com/sapics/ip-location-db/tree/main/origin-asn/)
+from sapics/ip-location-db. It is **PDDL v1.0 (public domain)**, rebuilt daily,
+and downloaded by the setup command above.
+
+Compatible records must provide an integer `autonomous_system_number` either at
+the top level or inside an `asn` map; Otto also accepts a bare integer `asn`
+field. Reserved ASNs (0 per RFC 7607 and the AS_TRANS placeholder 23456 per RFC
+6793) resolve to `'**'` rather than being reported as operators.
+
+## Anonymizer: the one unmasked lookup
+
+Anonymizer classification reads the **unmasked** address. This is deliberate
+and documented in `AnonymizerResolver` itself: anonymizer databases list
+individual egress nodes at or near /32, so the /24 equivalence that justifies
+masked geo and ASN lookups does not hold. A masked lookup would flag a whole
+/24 because one host in it runs a Tor exit, and miss the exit node itself —
+wrong in both directions.
+
+Otto passes the full address to the configured
+`anonymizer_db_reader#get(ip)` and retains only the returned label. Because the
+reader may be any application-provided object, Otto cannot guarantee that the
+reader does not log, persist, or transmit the address. Use a trusted local
+reader and review its network, logging, and retention behavior. Otto itself does
+not place the raw address in the enrichment result or downstream environment.
+
+### Reading the labels
+
+When several database flags are set at once (a Tor exit hosted at a cloud
+provider), the **most specific** label wins: `tor` > `proxy` > `vpn` >
+`residential_proxy` > `hosting` > `anonymous`.
+
+Two labels deserve care:
+
+- **`'none'`** means the reader was consulted and returned no record or no
+  recognized true flag. For an anonymous-IP MMDB, this normally means the
+  address is not listed, but it is *not* a positive assertion that the visitor
+  is residential. The result is only as reliable and current as the reader's
+  data.
+- **`'**'`** means no database answered at all. Do not collapse it into
+  `'none'`: `'none'` is evidence, `'**'` is the absence of evidence. A
+  "block anonymizers" rule that treats `'**'` as `'none'` fails open when
+  the database file goes missing.
+
+### Data file
+
+This signal is bring-your-own-database. `AnonymizerResolver` expects top-level
+fields matching the MaxMind GeoIP2 Anonymous-IP schema:
+`is_tor_exit_node`, `is_public_proxy`, `is_anonymous_vpn`,
+`is_residential_proxy`, `is_hosting_provider`, and `is_anonymous`. Verify the
+record shape before choosing a commercial or self-built MMDB.
+
+The [Tor bulk exit list](https://check.torproject.org/torbulkexitlist) can be an
+input to a self-built database, but Otto does not include an MMDB compiler or an
+update job. A generated record must set `is_tor_exit_node` at the top level, and
+the deployment must refresh and verify the file on its own schedule.
+
+## Acceptance behavior summary
+
+| Scenario | Result |
+| --- | --- |
+| Signal not enabled | `nil` when read; a processed public request may carry an env key whose value is `nil` |
+| IP privacy disabled or no client IP resolves | `nil`; no enrichment values are written |
+| Enabled, no database configured | `'**'` for a non-exempt request |
+| Database read raises | `'**'` (a lookup must never crash a request) |
+| ASN lookup | masked IP only (re-masked defensively in the resolver) |
+| Anonymizer lookup | the configured reader receives the unmasked IP; Otto exposes only its label |
+| Private/localhost client under the default profile | no enrichment keys in env |
+| Bad `*_db_path` for an enabled signal | raises during configuration, not per-request |

data/docs/guides/forwarded-authority.md

@@ -0,0 +1,249 @@
+# Forwarded host authority
+
+Reverse proxies use forwarding headers to report the original request's host,
+scheme, and port. Rack reads these values before it falls back to the request's
+`Host` header and direct connection details. Without a trust boundary, a client
+can send `X-Forwarded-Host` or an RFC 7239 `Forwarded` header and choose the host
+that the application believes it serves.
+
+This affects any value derived from `request.host`, `request.scheme`,
+`request.ssl?`, or `request.port`, including redirect targets, generated links,
+WebAuthn `rp_id`, OmniAuth `redirect_uri`, mailer base URLs, secure-cookie
+decisions, and mounted Rack applications that build absolute URLs.
+
+Otto uses the proxy trust configured for client IP resolution to decide whether
+Rack may also use forwarded host, scheme, and port values.
+
+> [!WARNING]
+> Omitting proxy trust does not reject forwarded authority. Otto leaves the
+> headers intact for compatibility, and Rack applies its own process-global
+> policy. Choose an explicit trust posture before serving requests whenever
+> application behavior depends on these request values.
+
+## Choose a trust posture
+
+| Deployment | Configuration | Result |
+| --- | --- | --- |
+| The application is directly exposed and should trust no proxy | `trusted_proxies: :none` | Otto strips forwarded host, scheme, and port carriers from every request. |
+| Proxy addresses can be enumerated | `trusted_proxies: [...]` | Otto keeps the carriers only when `REMOTE_ADDR` matches a configured proxy. |
+| Proxy addresses cannot be enumerated, but the hop count is fixed | `trusted_proxy_depth: N` | Otto trusts the carriers on every request. The application origin must accept traffic only from the proxy tier. |
+| Another layer owns the trust decision | Leave proxy trust unconfigured | Otto leaves the carriers unchanged and makes no trust assertion. |
+
+> [!WARNING]
+> These settings control forwarded authority only. They do not validate the
+> ordinary `Host` header. After forwarded carriers are stripped, Rack falls back
+> to `Host`, which a direct client can still choose. If redirects, generated
+> links, WebAuthn, OAuth, or cookie policy require a canonical host, enforce a
+> host allowlist in the front server or application.
+
+For a directly exposed application:
+
+```ruby
+otto = Otto.new('routes', trusted_proxies: :none)
+```
+
+For an application behind proxies whose addresses are known:
+
+```ruby
+otto = Otto.new(
+  'routes',
+  trusted_proxies: ['10.0.0.0/8', '192.0.2.0/24']
+)
+```
+
+Use depth mode only when the proxy addresses cannot be listed and the number of
+proxy hops is fixed:
+
+```ruby
+otto = Otto.new('routes', trusted_proxy_depth: 1)
+```
+
+Depth mode treats every connecting peer as trusted. Before enabling it, prevent
+direct access to the application origin with private networking, firewall or
+security-group rules, or an equivalent control. Otherwise, a client can submit
+forwarded host, scheme, port, and IP values directly.
+
+Configure these options before the first request, when Otto freezes its
+configuration.
+
+## How Otto handles each trust state
+
+The decision is made by `IPPrivacyMiddleware` from the connecting peer
+(`REMOTE_ADDR`) before any masking, and recorded in
+`env['otto.via_trusted_proxy']`.
+
+| Trust state | `otto.via_trusted_proxy` | Forwarded host, scheme, and port carriers |
+| --- | --- | --- |
+| `REMOTE_ADDR` matches a configured trusted-proxy CIDR | `true` | Kept. When privacy masking applies, `Forwarded` keeps its `proto=`, `host=`, and `by=` fields while its `for=` value is replaced with the masked IP. |
+| Depth mode is enabled | `true` for every peer | Kept, subject to the same privacy masking. |
+| Proxy trust is configured, but the peer does not match a configured CIDR | `false` | Deleted. |
+| `trusted_proxies: :none` is configured | `false` for every peer | Deleted. |
+| Proxy trust is not configured | absent | Left unchanged. Otto makes no trust assertion, so Rack may apply its own policy. |
+
+The deleted keys are `HTTP_FORWARDED`, `HTTP_X_FORWARDED_HOST`,
+`HTTP_X_FORWARDED_PROTO`, `HTTP_X_FORWARDED_SCHEME`, `HTTP_X_FORWARDED_SSL`,
+and `HTTP_X_FORWARDED_PORT`. `X-Forwarded-For` is not deleted on this path;
+Otto's own client IP resolution already ignores it from an untrusted peer, and a
+masking privacy profile rewrites it separately. With IP privacy disabled the
+header reaches the application intact, and `Rack::Request#ip` returns its value
+whenever `REMOTE_ADDR` is private or loopback. Read `env['otto.client_ip']`
+rather than `Rack::Request#ip`, and configure any mounted gem that reads
+`request.ip` (for example `Rack::Attack`) accordingly.
+
+The absent `otto.via_trusted_proxy` key is intentional. It means the operator
+made no proxy-trust assertion, so downstream consumers may apply their own
+heuristics. This preserves compatibility for applications that already run
+behind a proxy without configuring Otto's trust controls.
+
+## Direct exposure: trust no proxy
+
+Use `trusted_proxies: :none` when clients connect directly to the application
+and no reverse proxy should influence request authority. This explicit assertion
+is different from omitting `trusted_proxies`. The String spelling `'none'` (any
+case) is accepted too, for YAML- or environment-driven configuration. The
+sentinel is only valid as the whole option: a list that contains it, such as
+`trusted_proxies: ['none']`, is rejected at configuration time rather than
+installed as a proxy entry.
+
+You can also make the assertion after construction, but before the first
+request:
+
+```ruby
+otto.trust_no_proxies!
+```
+
+The same operation is available through the security configurator and the
+underlying security configuration:
+
+```ruby
+otto.security.trust_no_proxies!
+otto.security_config.trust_no_proxies!
+```
+
+Under this assertion:
+
+- `env['otto.via_trusted_proxy']` is `false` for every peer;
+- client IP resolution ignores forwarded chains and uses `REMOTE_ADDR`;
+- forwarded host, scheme, and port carriers are stripped;
+- `Rack::Request#host` resolves from the `Host` header; and
+- trusted geo headers remain disabled because they require enumerated
+  trusted-proxy CIDRs.
+
+Loopback is not special-cased. A reverse proxy running on `127.0.0.1` in front
+of the application is an untrusted peer under this assertion and its forwarded
+headers are stripped. Use `add_trusted_proxy('127.0.0.1')` for that deployment
+instead. The separate `env['otto.peer_loopback']` signal is derived from the raw
+peer and is unaffected, as is `env['otto.peer_relayed']`, which records whether
+any relay marker header was present before the carriers were stripped so
+`Otto::CaddyTLS::LocalhostGuard` still refuses a relayed request.
+
+The assertion is mutually exclusive with an actual trust grant. Combining it
+with trusted-proxy CIDRs or a depth of 1 or more raises at configuration time:
+
+```text
+Cannot combine trusted_proxies: :none (trust no proxy) with trusted_proxies
+CIDRs or trusted_proxy_depth >= 1. Assert :none OR grant trust, not both.
+```
+
+## Why the whole Forwarded header is removed
+
+For an untrusted peer Otto deletes `Forwarded` entirely rather than editing out
+its `host=` field. Editing would require Otto to parse RFC 7239 itself, and a
+parser that disagrees with Rack's on quoting can let a `host=` survive the edit.
+A value such as `for=a"b;host=evil` is enough to produce that disagreement.
+Deletion has no such failure mode. On this path Otto reads nothing from the
+header itself, so nothing is lost.
+
+## Choose the forwarding family for depth mode
+
+`Rack::Request.forwarded_priority` is a process-wide setting that selects the
+main forwarding family Rack reads: `X-Forwarded-*`, RFC 7239 `Forwarded`, or
+both. Otto pins it to the family selected by `trusted_proxy_header` so client-IP
+resolution and Rack's main forwarded host, port, and scheme parsing agree.
+
+This is not a complete sanitizer for requests from a trusted peer. Rack checks
+some compatibility carriers independently, notably `X-Forwarded-SSL`. Otto
+keeps forwarded carriers from trusted peers because it cannot distinguish
+values created by the proxy from values the proxy passed through. Configure the
+trusted proxy to remove client-supplied forwarding headers before setting its
+own authoritative values.
+
+The pin governs host, port, and the `X-Forwarded-Proto` / `proto=` scheme
+lookup. It does not govern `X-Forwarded-SSL`: Rack (3.2.x) honors
+`X-Forwarded-SSL: on` before it consults `forwarded_priority`, in every family.
+Otto covers this by deleting `X-Forwarded-SSL` together with the other
+authority carriers for any untrusted peer, so the header only reaches Rack from
+a trusted proxy or an unconfigured deployment.
+
+```ruby
+otto = Otto.new(
+  'routes',
+  trusted_proxy_depth: 1,
+  trusted_proxy_header: 'Forwarded'  # or 'X-Forwarded-For' (default), or 'Both'
+)
+```
+
+Because the setting is process-wide, two Otto applications mounted in one
+process that both resolve proxied requests must agree. The later one raises:
+
+```text
+Cannot use forwarding family %s (trusted_proxy_header) because another Otto
+application in this process already uses %s. Rack's forwarded host, port,
+scheme, and IP policy is process-global, so every Otto application in one
+process that resolves proxied requests must use the same forwarding family.
+```
+
+The two placeholders are the requested family and the already committed one.
+
+An application that configures no proxy trust, and one that asserts
+`trusted_proxies: :none`, read no forwarded chain and therefore stake no claim
+on the family. Neither can block a later explicit choice, unless it also names
+`trusted_proxy_header` explicitly. Otto only reads the header in depth mode,
+but setting it is always a claim: it pins Rack's `forwarded_priority` and
+registers the family for the process, even under `trusted_proxies: :none`.
+
+`trusted_proxy_header` accepts `X-Forwarded-For` (the default), `Forwarded`, or
+`Both`. When configuring proxy trust, `Forwarded` and `Both` require depth mode.
+CIDR filter mode resolves client IPs from the `X-Forwarded-For` family only
+(`X-Forwarded-For`, then `X-Real-IP`, then `X-Client-IP`) and never from RFC
+7239 `Forwarded`, so a non-default family would make Rack read a header that
+Otto ignores:
+
+```text
+Cannot configure trusted_proxy_header 'Forwarded' or 'Both' together with
+trusted_proxies (CIDR filter mode): CIDR-walk resolves client IPs from the
+X-Forwarded-For family only (X-Forwarded-For, X-Real-IP, X-Client-IP), never
+RFC 7239 Forwarded. Use trusted_proxy_depth (count mode) to read the RFC 7239
+Forwarded header.
+```
+
+Use `trusted_proxy_depth` when the deployment requires RFC 7239 `Forwarded`.
+Remember that depth mode also requires origin lockdown because it trusts every
+connecting peer.
+
+## Place the middleware before other request consumers
+
+Otto performs this filtering in `IPPrivacyMiddleware`, which it installs first
+in its own stack. Downstream middleware, mounted applications, and handlers see
+the filtered environment.
+
+If middleware outside the Otto application reads the request first, mount
+`IPPrivacyMiddleware` ahead of it in the common Rack stack. Pass the
+application's security configuration, as shown in
+[Privacy-preserving request data](privacy.md#middleware-placement). Without that
+configuration, the outer middleware instance applies defaults and makes no
+trust decision, so forwarded authority reaches earlier middleware unchanged.
+
+The inner instance still enforces the application's own posture when an outer
+pass has already resolved the client IP:
+
+- `trusted_proxies: :none` always applies. The inner instance records
+  `otto.via_trusted_proxy` as `false` and strips the authority carriers, whatever
+  the outer instance did.
+- `trusted_proxy_depth` records `true` unless a configured outer pass already
+  recorded a verdict.
+- `trusted_proxies: [...]` keeps the verdict of a configured outer pass. When no
+  outer pass recorded one, the connecting peer can no longer be matched, because
+  the outer pass rewrote `REMOTE_ADDR`. Otto then treats the peer as untrusted,
+  strips the carriers, and logs a warning naming the fix: pass the application's
+  security configuration to the outer instance.