otto 2.6.0 → 2.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/ci.yml +1 -1
  3. data/.github/workflows/claude-code-review.yml +1 -1
  4. data/.github/workflows/claude.yml +1 -1
  5. data/.github/workflows/code-smells.yml +2 -2
  6. data/.github/workflows/release-gem.yml +1 -1
  7. data/.github/workflows/ruby-lint.yml +1 -1
  8. data/.github/workflows/yardoc.yml +1 -1
  9. data/.pre-commit-config.yaml +22 -5
  10. data/CHANGELOG.rst +254 -0
  11. data/Gemfile +2 -1
  12. data/Gemfile.lock +13 -11
  13. data/README.md +13 -3
  14. data/docs/.gitignore +1 -0
  15. data/docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md +1105 -0
  16. data/docs/1108-STREAMING_SUPPORT_SUMMARY.md +376 -0
  17. data/docs/geo-country.md +180 -0
  18. data/docs/migrating/v2.3.0.md +55 -22
  19. data/docs/reverse-proxy-network-services.md +19 -6
  20. data/examples/simple_geo_resolver.rb +38 -5
  21. data/lib/otto/caddy_tls/localhost_guard.rb +43 -25
  22. data/lib/otto/core/middleware_stack.rb +72 -25
  23. data/lib/otto/env_keys.rb +58 -12
  24. data/lib/otto/logging_helpers.rb +50 -1
  25. data/lib/otto/mcp/rate_limiting.rb +5 -2
  26. data/lib/otto/privacy/config.rb +245 -3
  27. data/lib/otto/privacy/core.rb +104 -14
  28. data/lib/otto/privacy/geo_resolver.rb +228 -128
  29. data/lib/otto/privacy/ip_privacy.rb +24 -0
  30. data/lib/otto/privacy/redacted_fingerprint.rb +54 -2
  31. data/lib/otto/privacy.rb +3 -1
  32. data/lib/otto/request.rb +25 -9
  33. data/lib/otto/security/authentication/auth_failure.rb +36 -2
  34. data/lib/otto/security/authentication/auth_strategy.rb +12 -2
  35. data/lib/otto/security/authentication/authorization_failure.rb +7 -0
  36. data/lib/otto/security/authentication/route_auth_wrapper.rb +138 -31
  37. data/lib/otto/security/config.rb +61 -1
  38. data/lib/otto/security/core.rb +4 -1
  39. data/lib/otto/security/csp/report_middleware.rb +3 -1
  40. data/lib/otto/security/middleware/ip_privacy_middleware.rb +228 -18
  41. data/lib/otto/security/rate_limiter.rb +7 -1
  42. data/lib/otto/utils.rb +100 -0
  43. data/lib/otto/version.rb +1 -1
  44. data/lib/otto.rb +11 -3
  45. metadata +5 -2
@@ -0,0 +1,376 @@
1
+ # Otto Streaming Support: Executive Summary
2
+
3
+ **Investigation Date**: 2025-11-08
4
+ **Question**: Should Otto support Server-Sent Events (SSE) and WebSockets?
5
+ **Answer**: **No** - Use separate services or long-polling instead
6
+
7
+ > **Note**: The Ruby snippets below illustrate the recommended architecture. They
8
+ > are written against the current Otto Logic-class contract
9
+ > (`initialize(context, params, locale)` + `process`), but have not been executed
10
+ > end-to-end. Runnable examples are tracked separately.
11
+
12
+ ---
13
+
14
+ ## Quick Recommendation
15
+
16
+ | Use Case | Solution | Complexity | Otto Compatible? |
17
+ |----------|----------|------------|------------------|
18
+ | **Low-frequency updates (<1/min)** | Long-polling | ⭐ Simple | ✅ Yes |
19
+ | **Medium-frequency updates (1-10/sec)** | Separate SSE service | ⭐⭐ Moderate | ✅ Via integration |
20
+ | **High-frequency updates (>10/sec)** | Separate WebSocket service | ⭐⭐⭐ Complex | ✅ Via integration |
21
+ | **Bidirectional communication** | Separate WebSocket service | ⭐⭐⭐ Complex | ✅ Via integration |
22
+
23
+ ---
24
+
25
+ ## Key Findings
26
+
27
+ ### 1. **Otto's Architecture is Fundamentally Incompatible with Streaming**
28
+
29
+ Otto is designed as a **stateless, synchronous, request/response** framework:
30
+
31
+ ```
32
+ Request → Middleware → Route → Handler → Response → Close Connection
33
+ ```
34
+
35
+ SSE/WebSocket require **stateful, long-lived connections**:
36
+
37
+ ```
38
+ Request → Upgrade → Keep Open → Stream Data (minutes/hours) → Close
39
+ ```
40
+
41
+ **Incompatibilities**:
42
+ - ❌ Response handlers expect complete responses (not streaming enumerators)
43
+ - ❌ Middleware stack can't unwind during long-lived connections
44
+ - ❌ Configuration freezing prevents runtime streaming adjustments
45
+ - ❌ Requires async servers (Falcon, Iodine) - breaks server-agnostic design
46
+ - ❌ Stateful routing complicates horizontal scaling
47
+
48
+ ### 2. **Industry Best Practice: Separate Services**
49
+
50
+ Modern frameworks separate real-time communication from REST APIs:
51
+
52
+ **Rails (ActionCable)**:
53
+ ```
54
+ Rails App (Puma) → Redis Pub/Sub ← ActionCable (Separate Process)
55
+ ```
56
+
57
+ **Node.js (Express + Socket.IO)**:
58
+ ```
59
+ Express (HTTP Routes) + Socket.IO (Separate Layer)
60
+ ```
61
+
62
+ **Benefits**:
63
+ - ✅ Independent scaling (scale WebSocket separately from API)
64
+ - ✅ Technology choice (use best tool for each job)
65
+ - ✅ Fault isolation (WebSocket crash doesn't affect API)
66
+ - ✅ Clear architectural boundaries
67
+
68
+ ### 3. **Long-Polling Works Perfectly with Otto**
69
+
70
+ For low-to-medium frequency updates, long-polling is **simple and effective**:
71
+
72
+ ```ruby
73
+ # Otto route (works with any Rack server)
74
+ GET /api/notifications/poll NotificationPollLogic response=json auth=session
75
+
76
+ class NotificationPollLogic
77
+ attr_reader :context, :params, :locale
78
+
79
+ def initialize(context, params, locale)
80
+ @context = context
81
+ @params = params
82
+ @locale = locale
83
+ end
84
+
85
+ def process
86
+ # Logic-class params are string-keyed (LogicClassHandler does not apply
87
+ # Otto::Static.indifferent_params).
88
+ timeout = params['timeout'].to_i.clamp(1, 30)
89
+ last_id = params['last_id'].to_i
90
+ start_time = Time.now
91
+
92
+ # Identity comes from the StrategyResult, never from request params.
93
+ user_id = context.user_id
94
+
95
+ loop do
96
+ notifications = fetch_new(user_id, last_id)
97
+ return { notifications: notifications } if notifications.any?
98
+
99
+ break if Time.now - start_time > timeout
100
+
101
+ sleep 0.5
102
+ end
103
+
104
+ { notifications: [] }
105
+ end
106
+ end
107
+ ```
108
+
109
+ **Benefits**:
110
+ - ✅ HTTP-based (cacheable, proxy-friendly, standard tooling)
111
+ - ✅ Works with Otto's synchronous model
112
+ - ✅ Compatible with any Rack server (Puma, Unicorn, Passenger)
113
+ - ✅ Simple debugging (standard HTTP requests/responses)
114
+ - ✅ No external dependencies
115
+
116
+ ---
117
+
118
+ ## Recommended Solutions
119
+
120
+ ### Option 1: Long-Polling (SIMPLEST)
121
+
122
+ **When to use**:
123
+ - Updates less than 1 per minute
124
+ - Moderate concurrency (<10,000 clients)
125
+ - Simple deployment preferred
126
+
127
+ **Example**: see the `NotificationPollLogic` sketch above.
128
+
129
+ **Complexity**: ⭐ Simple
130
+ **Otto Integration**: ✅ Native support (no changes needed)
131
+
132
+ ---
133
+
134
+ ### Option 2: Separate SSE Service (RECOMMENDED FOR REAL-TIME)
135
+
136
+ **When to use**:
137
+ - Updates 1-10 per second
138
+ - High concurrency (>10,000 clients)
139
+ - Near-instant updates required (<100ms latency)
140
+
141
+ **Architecture**:
142
+ ```
143
+ ┌─────────────────┐
144
+ │ Otto API │ ← Stateless HTTP (authentication, business logic)
145
+ │ (Puma) │
146
+ └─────────────────┘
147
+
148
+ ┌─────────┐
149
+ │ Redis │ ← Message queue (pub/sub)
150
+ │ Pub/Sub │
151
+ └─────────┘
152
+
153
+ ┌─────────────────┐
154
+ │ SSE Service │ ← Stateful streaming (Falcon/Iodine)
155
+ │ (Falcon) │
156
+ └─────────────────┘
157
+ ```
158
+
159
+ **Example**: see "Otto + Falcon SSE Integration" in
160
+ `docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md`.
161
+
162
+ **Complexity**: ⭐⭐ Moderate
163
+ **Otto Integration**: ✅ Via Redis pub/sub
164
+
165
+ ---
166
+
167
+ ### Option 3: Third-Party Service (COMMERCIAL)
168
+
169
+ **When to use**:
170
+ - Don't want to manage WebSocket infrastructure
171
+ - Need global CDN distribution
172
+ - Require guaranteed SLA
173
+
174
+ **Options**:
175
+ - **Mercure**: Open-source SSE hub (self-hosted or managed)
176
+ - **Ably**: Commercial real-time messaging platform
177
+ - **Pusher**: Commercial WebSocket/SSE service
178
+
179
+ **Complexity**: ⭐⭐ Moderate (integration)
180
+ **Otto Integration**: ✅ Via HTTP API
181
+
182
+ ---
183
+
184
+ ## Documentation Created
185
+
186
+ ### **docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md** (15,000+ words)
187
+ Comprehensive technical analysis covering:
188
+ - Otto's current architecture (detailed lifecycle analysis)
189
+ - Technical requirements for SSE/WebSocket (Rack 3, hijacking, etc.)
190
+ - Industry patterns (Rails, Sinatra, Roda, Go, Node.js)
191
+ - Compatibility analysis (why it doesn't fit)
192
+ - Best practices and anti-patterns
193
+ - Detailed recommendations with code examples
194
+
195
+ ### Runnable examples: not yet published
196
+
197
+ Draft `examples/otto_falcon_sse_integration.rb` and
198
+ `examples/long_polling_example.rb` were written alongside this analysis but
199
+ targeted an Otto API surface that does not exist (`Otto::RequestContext`,
200
+ `enable_sessions!`, `raise_concern`, a `session` helper, `Rack::Handler::Puma`
201
+ under the Rack 3 pin) and used symbol-keyed params that Logic routes never
202
+ populate. They were withheld rather than shipped as copy-paste material; the
203
+ architecture they demonstrated is preserved in the snippets here and in the
204
+ analysis document.
205
+
206
+ ---
207
+
208
+ ## Key Insights
209
+
210
+ ### The Real Question
211
+
212
+ **Not**: "Can Otto support SSE/WebSocket?"
213
+ (Technically possible with massive refactoring)
214
+
215
+ **But**: "Should Otto support SSE/WebSocket?"
216
+ (Architecturally inadvisable)
217
+
218
+ ### Answer: **No**
219
+
220
+ Otto should remain focused on its core strengths:
221
+ - ✅ **Stateless** HTTP APIs
222
+ - ✅ **Security-first** design (CSRF, rate limiting, validation)
223
+ - ✅ **Privacy by default** (IP masking, geo-location)
224
+ - ✅ **Server-agnostic** (works with any Rack server)
225
+ - ✅ **Simple** and predictable architecture
226
+
227
+ Adding SSE/WebSocket would:
228
+ - ❌ Compromise architectural integrity
229
+ - ❌ Force specific async servers (Falcon, Iodine)
230
+ - ❌ Complicate security guarantees (middleware assumptions broken)
231
+ - ❌ Add significant complexity for niche use case
232
+ - ❌ Go against industry best practices (separation of concerns)
233
+
234
+ ---
235
+
236
+ ## What Otto SHOULD Do
237
+
238
+ ### 1. ✅ Document Integration Patterns
239
+
240
+ Add official guide: "Integrating Otto with Real-Time Services"
241
+ - Long-polling patterns (built-in support)
242
+ - Separate SSE service pattern (Otto + Falcon + Redis)
243
+ - Third-party service integration (Mercure, Ably, Pusher)
244
+
245
+ ### 2. ✅ Provide Example Code
246
+
247
+ Add to `examples/` directory, written against the real Logic-class contract and
248
+ verified to boot:
249
+ - `long_polling_example.rb`
250
+ - `otto_falcon_sse_integration.rb`
251
+ - `otto_mercure_integration.rb`
252
+
253
+ ### 3. ⚠️ Consider Plugin System (If Community Demands)
254
+
255
+ **Only if there's strong demand**, create experimental plugin:
256
+ - Clearly marked "experimental" and "unsupported"
257
+ - Requires Falcon/Iodine (documented)
258
+ - Security implications documented
259
+ - No core changes required
260
+
261
+ ### 4. ❌ Do NOT Add to Core
262
+
263
+ Preserve Otto's architectural integrity by:
264
+ - Keeping core stateless and synchronous
265
+ - Maintaining server-agnostic design
266
+ - Focusing on security and simplicity
267
+ - Following industry best practices (separation of concerns)
268
+
269
+ ---
270
+
271
+ ## Migration Guide for Existing Users
272
+
273
+ If you currently need real-time updates:
274
+
275
+ ### Step 1: Assess Your Use Case
276
+
277
+ **Low-frequency updates (<1/min)**:
278
+ → Use long-polling (Otto native support)
279
+
280
+ **Medium-frequency updates (1-10/sec)**:
281
+ → Use separate SSE service (Otto + Falcon + Redis)
282
+
283
+ **High-frequency or bidirectional**:
284
+ → Use separate WebSocket service or commercial solution
285
+
286
+ ### Step 2: Implementation Path
287
+
288
+ #### For Long-Polling:
289
+ 1. Create Otto route with long-polling logic
290
+ 2. Use `sleep` loop with timeout
291
+ 3. Client polls with timeout parameter
292
+ 4. No external dependencies needed
293
+
294
+ See the `NotificationPollLogic` sketch above.
295
+
296
+ #### For Separate SSE Service:
297
+ 1. Otto API handles authentication and publishes to Redis
298
+ 2. Separate Falcon app subscribes to Redis and streams SSE
299
+ 3. Client connects to SSE service with JWT token from Otto
300
+ 4. Scale Otto and SSE services independently
301
+
302
+ See "Otto + Falcon SSE Integration" in
303
+ `docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md`.
304
+
305
+ ### Step 3: Deployment
306
+
307
+ **Long-Polling**:
308
+ - Deploy with existing Otto setup (Puma, Unicorn, Passenger)
309
+ - Increase thread pool size for long-polling routes
310
+ - Monitor connection pool (ensure enough threads)
311
+
312
+ **Separate SSE Service**:
313
+ - Deploy Otto API with Puma (standard)
314
+ - Deploy Falcon SSE service separately (dedicated servers)
315
+ - Use Redis for pub/sub (cluster-ready)
316
+ - Configure Nginx with sticky sessions for SSE
317
+ - Scale services independently based on load
318
+
319
+ ---
320
+
321
+ ## Performance Guidance
322
+
323
+ ### Long-Polling Capacity
324
+
325
+ **Example**: Puma with 5 workers × 32 threads = 160 concurrent requests
326
+
327
+ If long-polling uses 30s timeout:
328
+ - 160 concurrent connections
329
+ - ~320 clients with 50% utilization
330
+ - Up to 10,000 clients with proper thread tuning
331
+
332
+ **Good for**: Dashboard metrics, low-volume notifications
333
+
334
+ ### SSE/WebSocket Capacity
335
+
336
+ **Example**: Falcon with 4 workers (async)
337
+
338
+ Each worker handles thousands of concurrent connections via fibers:
339
+ - 10,000+ concurrent SSE connections per server
340
+ - Horizontal scaling via Redis pub/sub
341
+ - Near-instant message delivery
342
+
343
+ **Good for**: Chat, multiplayer, high-frequency updates
344
+
345
+ ---
346
+
347
+ ## Conclusion
348
+
349
+ **Otto should NOT integrate SSE/WebSocket support** because:
350
+
351
+ 1. **Architectural mismatch**: Stateless vs stateful paradigms
352
+ 2. **Industry consensus**: Separate services is best practice
353
+ 3. **Complexity cost**: Massive refactoring for niche use case
354
+ 4. **Better alternatives**: Long-polling (simple) or separate services (powerful)
355
+
356
+ **Instead, Otto should**:
357
+
358
+ 1. ✅ Document long-polling patterns (works today)
359
+ 2. ✅ Provide integration examples (Otto + Falcon + Redis)
360
+ 3. ✅ Recommend third-party solutions (Mercure, Ably, Pusher)
361
+ 4. ✅ Stay focused on stateless HTTP APIs
362
+
363
+ **This preserves Otto's core strengths** while enabling users who need real-time functionality to integrate appropriate solutions.
364
+
365
+ ---
366
+
367
+ ## Further Reading
368
+
369
+ - **Technical Analysis**: `docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md`
370
+ - **Rack 3 Streaming**: https://github.com/rack/rack/issues/1600
371
+ - **ActionCable Architecture**: https://guides.rubyonrails.org/action_cable_overview.html
372
+ - **SSE vs WebSocket**: https://ably.com/blog/websockets-vs-sse
373
+
374
+ ---
375
+
376
+ **End of Summary**
@@ -0,0 +1,180 @@
1
+ # Geo-country resolution
2
+
3
+ Otto resolves a country-level ISO 3166-1 alpha-2 code for each request and
4
+ exposes it as `req.geo_country` / `env['otto.privacy.geo_country']`. Resolution
5
+ is country-only by design — that is the privacy posture; there is no city or
6
+ region lookup.
7
+
8
+ ## Resolution order
9
+
10
+ `Otto::Privacy::GeoResolver.resolve` returns the first hit from:
11
+
12
+ 1. **Application-configured header** (`geo_header:`) — e.g. `X-Client-Country`.
13
+ 2. **Known provider headers** — Cloudflare (`CF-IPCountry`), AWS CloudFront,
14
+ Fastly, Akamai Edgescape, Azure Front Door, **Vercel**
15
+ (`X-Vercel-IP-Country`), and a few semi-standard names
16
+ (`X-Geo-Country`, `X-Country-Code`, `Country-Code`).
17
+ 3. **Custom resolver** (`GeoResolver.custom_resolver`) — your own callable.
18
+ Unlike the other geo settings, this is **class-level** (see
19
+ [Configuration](#configuration)).
20
+ 4. **Local MMDB database** (`geo_db_path:` / `geo_db_reader:`) — a MaxMind-DB
21
+ country database.
22
+ 5. **`'**'`** — the unknown sentinel, when nothing else matches.
23
+
24
+ Steps 1 and 2 are **only consulted when geo headers can be trusted** (see
25
+ [Header trust](#header-trust-and-spoofing) below).
26
+
27
+ Resolution is **honest**: Otto does not guess from a hardcoded IP-range table.
28
+ When no header, custom resolver, or database resolves a country, the result is
29
+ `'**'`.
30
+
31
+ ### Privacy: masked IP and masked env
32
+
33
+ The database lookup in step 4 runs on the request's **masked** IP
34
+ (e.g. `203.0.113.0`), never the real address. `check_geo_database` masks the IP
35
+ internally with the config's `octet_precision` before the lookup, so even a
36
+ direct `GeoResolver.resolve` caller passing a real IP does not expose it to the
37
+ database. Country-level MMDB networks are almost always ≥ /24, so the default
38
+ /24-masked value (`octet_precision: 1`) resolves to the same country.
39
+
40
+ In the middleware path Otto additionally hands `resolve` a **masked env view**:
41
+ `REMOTE_ADDR`, `X-Forwarded-For`, `X-Real-IP`, `X-Client-IP`, and the RFC 7239
42
+ `Forwarded` header are masked. So a `custom_resolver` cannot read the raw client
43
+ IP out of `env` either — use the `ip` argument (already masked), not `env`.
44
+
45
+ > **`octet_precision: 2`** masks two octets (a /16). That is coarser than most
46
+ > country networks, so it can reduce database hit rate for the small share of
47
+ > countries whose ranges are finer than /16 — those requests fall through to
48
+ > `'**'`. Header and custom-resolver sources are unaffected (they ignore the
49
+ > IP). Keep the default precision if you rely on the MMDB fallback.
50
+
51
+ ## Configuration
52
+
53
+ All geo configuration is **boot-time only** (set once during single-threaded
54
+ initialization, before serving requests), matching `custom_resolver`'s
55
+ contract. `geo_header`, `geo_db_path`, and `geo_db_reader` are stored on the
56
+ instance's `Otto::Privacy::Config`, so separate Otto instances hold independent
57
+ geo configuration.
58
+
59
+ > **`custom_resolver` is the exception — it is class-level, not per-instance.**
60
+ > `GeoResolver.custom_resolver=` sets a singleton on the `GeoResolver` class, so
61
+ > it is **shared across every Otto instance in the process** (last write wins).
62
+ > If you run multiple Otto instances that need different resolver strategies,
63
+ > the custom resolver cannot distinguish them — branch inside a single resolver
64
+ > on `env`, or use per-instance `geo_db_reader` instead.
65
+
66
+ ```ruby
67
+ otto.configure_ip_privacy(
68
+ geo: true, # default; false disables geo entirely
69
+ geo_header: 'X-Client-Country', # trusted app header (optional)
70
+ geo_db_path: 'data/geo-whois-asn-country.mmdb', # local MMDB fallback (optional)
71
+ # geo_db_reader: MaxMind::DB.new(path), # or bring your own reader (optional)
72
+ )
73
+ ```
74
+
75
+ - **`geo: false`** short-circuits everything: no header reads, and any loaded
76
+ database is unloaded from memory (`req.geo_country` becomes `nil`).
77
+ - **`geo_header:`** accepts either the HTTP header name (`X-Client-Country`) or
78
+ the Rack CGI env key (`HTTP_X_CLIENT_COUNTRY`), in any case, and is
79
+ canonicalized to the env-key form. Pass `''` to clear.
80
+ - **`geo_db_path:`** is loaded once at boot in `MODE_MEMORY`. An unreadable
81
+ path, a corrupt/non-MMDB file, or a missing `maxmind-db` gem raises
82
+ `ArgumentError` **at configuration time**, not per-request. Pass `''` to
83
+ unload.
84
+ - **`geo_db_reader:`** injects any object responding to `#get(ip)` (a
85
+ preconfigured `MaxMind::DB` reader or a test double), keeping the reader and
86
+ data-source choice independent of Otto. It **overrides** `geo_db_path` when
87
+ both are given in the same call; supplying `geo_db_path` alone in a later call
88
+ clears a prior reader override.
89
+
90
+ Each keyword follows a `nil` = "leave unchanged" contract; pass `''` to a header
91
+ or path to clear it. Any geo-affecting change triggers the boot-time database
92
+ (re)load, so a bad `geo_db_path` fails at the `configure_ip_privacy` call.
93
+
94
+ ## The database: gem and datafile
95
+
96
+ The reader and the data file are independent — the MMDB format is the interop
97
+ point.
98
+
99
+ ### Reader gem (`maxmind-db`)
100
+
101
+ The [`maxmind-db`](https://rubygems.org/gems/maxmind-db) gem (official MaxMind
102
+ reader, Apache-2.0, pure Ruby, zero runtime deps) is an **optional**
103
+ dependency. Otto only `require`s it when a database is configured. Add it to
104
+ your app when you use the database fallback:
105
+
106
+ ```ruby
107
+ # Gemfile
108
+ gem 'maxmind-db', '~> 1.4'
109
+ ```
110
+
111
+ ### Data file (`geo-whois-asn-country`)
112
+
113
+ The recommended data file is
114
+ [`geo-whois-asn-country`](https://github.com/sapics/ip-location-db) from
115
+ sapics/ip-location-db: **PDDL v1.0 (public domain, no attribution required)**,
116
+ rebuilt daily, shipped as MMDB. Otto vendors no database — country data goes
117
+ stale, and a public-domain file you refresh on your own schedule keeps
118
+ licensing and freshness in your control.
119
+
120
+ Download it (IPv4+IPv6) into a path of your choosing:
121
+
122
+ ```bash
123
+ mkdir -p data
124
+ curl -fsSL -o data/geo-whois-asn-country.mmdb \
125
+ https://github.com/sapics/ip-location-db/releases/download/latest/geo-whois-asn-country.mmdb
126
+ ```
127
+
128
+ Refresh it on your own schedule (e.g. a daily cron job running the same curl).
129
+ Any MMDB country database works — GeoLite2-Country, DB-IP Country Lite,
130
+ iplocate, etc. — since `GeoResolver` tolerates the record shapes country
131
+ databases actually use: nested `country.iso_code` (GeoLite2-Country style), a
132
+ flat `country_code` string, and a bare-string `country`.
133
+
134
+ > **Note on GeoLite2:** its EULA requires a MaxMind account/license key and
135
+ > obliges consumers to refresh within 30 days of each release. A PDDL dataset
136
+ > avoids both obligations.
137
+
138
+ ## Header trust and spoofing
139
+
140
+ Every geo header is trivially client-spoofable unless the request actually
141
+ arrived through the CDN that sets it. Otto trusts geo headers — both the
142
+ configured `geo_header` and the provider headers — **only** for a request that
143
+ demonstrably arrived via a configured **CIDR trusted proxy**
144
+ (`env['otto.via_trusted_proxy']` with `trusted_proxies` configured). A spoofed
145
+ header on a direct connection is ignored, and resolution falls through to the
146
+ custom resolver / database.
147
+
148
+ Origins Otto cannot verify are **not** trusted:
149
+
150
+ - **No trusted-proxy configuration.** A direct internet client could otherwise
151
+ pick its own country by sending `CF-IPCountry` / `X-Client-Country`, so with
152
+ no `trusted_proxies` configured, header steps are skipped and resolution falls
153
+ to the resolver / database (`'**'` if neither is set).
154
+ - **Count-based `trusted_proxy_depth` mode.** The header-setting hop cannot be
155
+ verified as a geo-CDN, so depth mode does not enable header trust. This
156
+ conflict fails loud: configuring a `geo_header` together with a
157
+ `trusted_proxy_depth` raises `ArgumentError` at configuration time (in
158
+ either order) instead of silently ignoring the header per-request.
159
+ Database-backed geo remains fully supported under depth. The built-in
160
+ provider headers stay legal (there is nothing to configure, so nothing
161
+ can raise) but are equally inert — header trust requires CIDR-verified
162
+ proxies — so only an explicitly configured `geo_header` is rejected, and
163
+ depth deployments that want geo should set `geo_db_path`.
164
+
165
+ **Migration:** to keep header-based geo, configure `trusted_proxies` (CIDR
166
+ matchers) so Otto can verify the proxy origin. Depth-mode and header-only
167
+ deployments should set `geo_db_path` for a local database instead; otherwise
168
+ resolution returns `'**'`.
169
+
170
+ ## Acceptance behavior summary
171
+
172
+ | Scenario | Result |
173
+ | --- | --- |
174
+ | Configured `geo_header` present and trusted | wins over provider headers |
175
+ | Request not via a verified CIDR trusted proxy | geo headers skipped |
176
+ | No `trusted_proxies` configured | geo headers skipped (not trusted) |
177
+ | Database lookup | uses the masked IP only |
178
+ | `geo: false` | `nil`, no database in memory |
179
+ | Bad `geo_db_path` | raises at boot, not per-request |
180
+ | Nothing matches | `'**'` |
@@ -55,8 +55,16 @@ return `false` behind a TLS-terminating trusted proxy.
55
55
 
56
56
  The middleware now records the peer-trust decision once (before masking) in a
57
57
  leak-free boolean `env['otto.via_trusted_proxy']`, and `secure?` reads it.
58
- When the middleware has not run (standalone request use), `secure?` falls back
59
- to its previous behavior. No app changes are required.
58
+ Since the first release after 2.7.0 the key is **tri-state**: it is written
59
+ only when proxy trust is actually configured (CIDR matchers or a depth), so a
60
+ present key is authoritative in both directions and an *absent* key means "no
61
+ proxy trust configured". When the key is absent (standalone request use, or an
62
+ unconfigured deployment), `secure?` falls back to the same decision the
63
+ middleware would have implied: a CIDR check of the connecting peer — or, since
64
+ the depth-mode peer-trust fix
65
+ ([#226](https://github.com/delano/otto/issues/226), first release after
66
+ 2.7.0), an unconditional grant when `trusted_proxy_depth` is configured. No
67
+ app changes are required.
60
68
 
61
69
  ### 3. Privacy helpers now return values (previously `nil`)
62
70
 
@@ -100,7 +108,7 @@ use `Otto::Request` standalone without the Otto middleware stack.
100
108
  | Key | Type | Meaning |
101
109
  |-----|------|---------|
102
110
  | `otto.client_ip` | String | Canonical client IP, resolved once. Masked when privacy is enabled; resolved real IP when disabled or exempt. Read by `Request#ip` / `#client_ipaddress`. |
103
- | `otto.via_trusted_proxy` | Boolean | Whether the request arrived via a trusted proxy, decided before masking. Read by `Request#secure?`. |
111
+ | `otto.via_trusted_proxy` | Boolean (tri-state: may be absent) | Peer trust decided before masking, written **only when proxy trust is configured** (first release after 2.7.0): `true` on a CIDR match — or unconditionally when depth mode is configured ([#226](https://github.com/delano/otto/issues/226)); `false` means trust is configured and the peer failed it (authoritative deny). Absent = no proxy trust configured — the only case for consumer-side fallback heuristics. Read by `Request#secure?`. |
104
112
 
105
113
  The privacy data keys remain `otto.privacy.{fingerprint,masked_ip,hashed_ip,geo_country}`.
106
114
 
@@ -187,15 +195,26 @@ them). Which *multi-hop* header depth counts from — `X-Forwarded-For` (default
187
195
  the RFC 7239 `Forwarded` header, or `Both` — is configurable as of 2.3.1; see
188
196
  *Selecting the forwarded header* below.
189
197
 
190
- **`secure?` is independent of depth.** Depth mode resolves the client **IP**
191
- only; it does **not** grant proxy trust for `X-Forwarded-Proto` / `X-Scheme`.
192
- `env['otto.via_trusted_proxy']`which `Otto::Request#secure?` consults to honor
193
- a forwarded proto is derived solely from the trusted-proxy *identity* check
194
- (does `REMOTE_ADDR` match a configured `trusted_proxies` CIDR?), never from hop
195
- depth. Because depth mode and `trusted_proxies` are mutually exclusive, that
196
- check is `false` under depth, so `secure?` does not honor a forwarded proto and
197
- reflects only a direct TLS connection (`HTTPS=on` / port 443). This mirrors the
198
- downstream (OneTimeSecret) behavior: proto-trust is never derived from depth.
198
+ **Depth grants peer trust ([#226](https://github.com/delano/otto/issues/226),
199
+ first release after 2.7.0).** Configuring a depth asserts that the connecting
200
+ peer *is* your proxy tier that is what the setting means — so depth mode
201
+ records `env['otto.via_trusted_proxy'] = true` on every request, and
202
+ `Otto::Request#secure?` (which consults that flag) honors a forwarded
203
+ `X-Forwarded-Proto` / `X-Scheme` in depth mode. Releases up to and including
204
+ 2.7.0 behaved differently: the flag was derived solely from the
205
+ `trusted_proxies` CIDR identity check, which is always empty under depth
206
+ (the modes are mutually exclusive), so it was recorded `false` and `secure?`
207
+ reflected only a direct TLS connection (`HTTPS=on` / port 443). Because the
208
+ grant is unconditional, the *origin lockdown* prerequisite below now covers
209
+ proto trust exactly as it covers IP resolution. Geo headers are unaffected:
210
+ they remain gated on enumerated `trusted_proxies` matchers and are still
211
+ **not** trusted in depth mode (a hop trusted by count cannot be verified as a
212
+ geo-setting CDN). As of the first release after 2.7.0 this carve-out fails
213
+ loud instead of silently: configuring an ip-privacy `geo_header` together
214
+ with a `trusted_proxy_depth` raises `ArgumentError` at configuration time
215
+ (in either order, with a freeze-time backstop) — use filter mode for
216
+ header-based geo, or a geo database (`geo_db_path`) under depth. Database-
217
+ backed geo with depth remains fully supported.
199
218
 
200
219
  ### Selecting the forwarded header (added in 2.3.1)
201
220
 
@@ -249,9 +268,10 @@ Otto.new(routes, trusted_proxy_depth: 1, trusted_proxy_header: 'Forwarded')
249
268
  This is the inherent trade-off versus CIDR-walk: depth relies on a fixed network
250
269
  **topology** instead of enumerable proxy **addresses**. If a client can reach
251
270
  your app directly (origin not locked down), it can pad `X-Forwarded-For` so that
252
- a forged value lands at `chain[-(N+1)]`, spoofing the resolved client IP. (Proto
253
- trust is unaffected depth never feeds `secure?` — but the resolved IP is only
254
- as trustworthy as the lockdown.)
271
+ a forged value lands at `chain[-(N+1)]`, spoofing the resolved client IP. (Since
272
+ [#226](https://github.com/delano/otto/issues/226) this applies to proto trust
273
+ too: depth grants `otto.via_trusted_proxy`, so a directly-reachable origin
274
+ could spoof the scheme via `X-Forwarded-Proto` as well as the client IP.)
255
275
 
256
276
  Before enabling depth, ensure the origin only accepts connections from the proxy
257
277
  tier (private networking, security groups, an authenticating header the proxy
@@ -262,13 +282,26 @@ injects, etc.). If you can enumerate your proxies instead, prefer CIDR-walk.
262
282
  If you are collapsing OneTimeSecret's `ClientIpHelpers` / `ConfigureTrustedProxy`
263
283
  depth logic onto this resolver, note two intentional differences:
264
284
 
265
- - **Off-by-one (Otto counts the peer).** Otto's chain is `X-Forwarded-For`
266
- **plus** `REMOTE_ADDR`, so it is one hop longer than OTS's XFF-only chain. To
267
- resolve the same client, Otto's depth must be **one higher** than the
268
- operator's OTS `depth:`. When the YAML→Otto translator is built, map
269
- **`trusted_proxy_depth = ots_depth + 1`** so existing `depth:` values keep
270
- their meaning. Keep a parity regression test on the OTS side to lock this
271
- mapping.
285
+ - **Map depth values directly — do NOT add one.** Otto's chain is
286
+ `X-Forwarded-For` **plus** `REMOTE_ADDR`, and the client is selected at
287
+ `chain[-(N+1)]` the appended peer is already accounted for by the index
288
+ arithmetic. `trusted_proxy_depth = N` therefore means "N proxy hops,
289
+ counting the connecting peer as hop 1", which is exactly what the
290
+ operator-facing OTS `depth: N` documents ("1 = standard single reverse
291
+ proxy"). Map **`trusted_proxy_depth = ots_depth`**, and keep a parity
292
+ regression test on the OTS side — including a padded-chain case asserting a
293
+ forged leftmost `X-Forwarded-For` entry is never selected.
294
+
295
+ > **Correction.** Earlier revisions of this guide recommended
296
+ > `trusted_proxy_depth = ots_depth + 1` to "keep existing `depth:` values'
297
+ > meaning". That reproduced an internal off-by-one of the deleted OTS
298
+ > walker (whose indexed path selected `XFF[-(depth+1)]`, one position left
299
+ > of its own documented contract), not the operator-facing meaning. Under
300
+ > the `+1` remap every honest documented-topology request hit the
301
+ > short-chain fallback and resolved the **proxy** address as the client,
302
+ > and a single forged leftmost `X-Forwarded-For` entry re-lengthened the
303
+ > chain so the forged value was selected. The direct mapping resolves the
304
+ > true client on honest chains and is padding-resistant.
272
305
 
273
306
  - **Stricter short-chain behavior (kept on purpose).** When the chain is shorter
274
307
  than `N + 1`, Otto returns `REMOTE_ADDR` (the peer), whereas OTS returned the