otto 2.6.0 → 2.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/ci.yml +1 -1
  3. data/.github/workflows/claude-code-review.yml +1 -1
  4. data/.github/workflows/claude.yml +1 -1
  5. data/.github/workflows/code-smells.yml +2 -2
  6. data/.github/workflows/release-gem.yml +1 -1
  7. data/.github/workflows/ruby-lint.yml +1 -1
  8. data/.github/workflows/yardoc.yml +1 -1
  9. data/.pre-commit-config.yaml +22 -5
  10. data/CHANGELOG.rst +218 -0
  11. data/Gemfile +2 -1
  12. data/Gemfile.lock +12 -10
  13. data/README.md +13 -3
  14. data/docs/.gitignore +1 -0
  15. data/docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md +1105 -0
  16. data/docs/1108-STREAMING_SUPPORT_SUMMARY.md +376 -0
  17. data/docs/geo-country.md +172 -0
  18. data/docs/reverse-proxy-network-services.md +19 -6
  19. data/examples/simple_geo_resolver.rb +38 -5
  20. data/lib/otto/caddy_tls/localhost_guard.rb +43 -25
  21. data/lib/otto/core/middleware_stack.rb +72 -25
  22. data/lib/otto/env_keys.rb +32 -0
  23. data/lib/otto/logging_helpers.rb +50 -1
  24. data/lib/otto/mcp/rate_limiting.rb +5 -2
  25. data/lib/otto/privacy/config.rb +245 -3
  26. data/lib/otto/privacy/core.rb +93 -14
  27. data/lib/otto/privacy/geo_resolver.rb +228 -128
  28. data/lib/otto/privacy/ip_privacy.rb +24 -0
  29. data/lib/otto/privacy/redacted_fingerprint.rb +54 -2
  30. data/lib/otto/privacy.rb +3 -1
  31. data/lib/otto/request.rb +8 -1
  32. data/lib/otto/security/authentication/auth_failure.rb +36 -2
  33. data/lib/otto/security/authentication/auth_strategy.rb +12 -2
  34. data/lib/otto/security/authentication/authorization_failure.rb +7 -0
  35. data/lib/otto/security/authentication/route_auth_wrapper.rb +138 -31
  36. data/lib/otto/security/config.rb +23 -1
  37. data/lib/otto/security/core.rb +4 -1
  38. data/lib/otto/security/csp/report_middleware.rb +3 -1
  39. data/lib/otto/security/middleware/ip_privacy_middleware.rb +201 -12
  40. data/lib/otto/security/rate_limiter.rb +7 -1
  41. data/lib/otto/utils.rb +100 -0
  42. data/lib/otto/version.rb +1 -1
  43. data/lib/otto.rb +11 -3
  44. metadata +6 -6
@@ -0,0 +1,376 @@
1
+ # Otto Streaming Support: Executive Summary
2
+
3
+ **Investigation Date**: 2025-11-08
4
+ **Question**: Should Otto support Server-Sent Events (SSE) and WebSockets?
5
+ **Answer**: **No** - Use separate services or long-polling instead
6
+
7
+ > **Note**: The Ruby snippets below illustrate the recommended architecture. They
8
+ > are written against the current Otto Logic-class contract
9
+ > (`initialize(context, params, locale)` + `process`), but have not been executed
10
+ > end-to-end. Runnable examples are tracked separately.
11
+
12
+ ---
13
+
14
+ ## Quick Recommendation
15
+
16
+ | Use Case | Solution | Complexity | Otto Compatible? |
17
+ |----------|----------|------------|------------------|
18
+ | **Low-frequency updates (<1/min)** | Long-polling | ⭐ Simple | ✅ Yes |
19
+ | **Medium-frequency updates (1-10/sec)** | Separate SSE service | ⭐⭐ Moderate | ✅ Via integration |
20
+ | **High-frequency updates (>10/sec)** | Separate WebSocket service | ⭐⭐⭐ Complex | ✅ Via integration |
21
+ | **Bidirectional communication** | Separate WebSocket service | ⭐⭐⭐ Complex | ✅ Via integration |
22
+
23
+ ---
24
+
25
+ ## Key Findings
26
+
27
+ ### 1. **Otto's Architecture is Fundamentally Incompatible with Streaming**
28
+
29
+ Otto is designed as a **stateless, synchronous, request/response** framework:
30
+
31
+ ```
32
+ Request → Middleware → Route → Handler → Response → Close Connection
33
+ ```
34
+
35
+ SSE/WebSocket require **stateful, long-lived connections**:
36
+
37
+ ```
38
+ Request → Upgrade → Keep Open → Stream Data (minutes/hours) → Close
39
+ ```
40
+
41
+ **Incompatibilities**:
42
+ - ❌ Response handlers expect complete responses (not streaming enumerators)
43
+ - ❌ Middleware stack can't unwind during long-lived connections
44
+ - ❌ Configuration freezing prevents runtime streaming adjustments
45
+ - ❌ Requires async servers (Falcon, Iodine) - breaks server-agnostic design
46
+ - ❌ Stateful routing complicates horizontal scaling
47
+
48
+ ### 2. **Industry Best Practice: Separate Services**
49
+
50
+ Modern frameworks separate real-time communication from REST APIs:
51
+
52
+ **Rails (ActionCable)**:
53
+ ```
54
+ Rails App (Puma) → Redis Pub/Sub ← ActionCable (Separate Process)
55
+ ```
56
+
57
+ **Node.js (Express + Socket.IO)**:
58
+ ```
59
+ Express (HTTP Routes) + Socket.IO (Separate Layer)
60
+ ```
61
+
62
+ **Benefits**:
63
+ - ✅ Independent scaling (scale WebSocket separately from API)
64
+ - ✅ Technology choice (use best tool for each job)
65
+ - ✅ Fault isolation (WebSocket crash doesn't affect API)
66
+ - ✅ Clear architectural boundaries
67
+
68
+ ### 3. **Long-Polling Works Perfectly with Otto**
69
+
70
+ For low-to-medium frequency updates, long-polling is **simple and effective**:
71
+
72
+ ```ruby
73
+ # Otto route (works with any Rack server)
74
+ GET /api/notifications/poll NotificationPollLogic response=json auth=session
75
+
76
+ class NotificationPollLogic
77
+ attr_reader :context, :params, :locale
78
+
79
+ def initialize(context, params, locale)
80
+ @context = context
81
+ @params = params
82
+ @locale = locale
83
+ end
84
+
85
+ def process
86
+ # Logic-class params are string-keyed (LogicClassHandler does not apply
87
+ # Otto::Static.indifferent_params).
88
+ timeout = params['timeout'].to_i.clamp(1, 30)
89
+ last_id = params['last_id'].to_i
90
+ start_time = Time.now
91
+
92
+ # Identity comes from the StrategyResult, never from request params.
93
+ user_id = context.user_id
94
+
95
+ loop do
96
+ notifications = fetch_new(user_id, last_id)
97
+ return { notifications: notifications } if notifications.any?
98
+
99
+ break if Time.now - start_time > timeout
100
+
101
+ sleep 0.5
102
+ end
103
+
104
+ { notifications: [] }
105
+ end
106
+ end
107
+ ```
108
+
109
+ **Benefits**:
110
+ - ✅ HTTP-based (cacheable, proxy-friendly, standard tooling)
111
+ - ✅ Works with Otto's synchronous model
112
+ - ✅ Compatible with any Rack server (Puma, Unicorn, Passenger)
113
+ - ✅ Simple debugging (standard HTTP requests/responses)
114
+ - ✅ No external dependencies
115
+
116
+ ---
117
+
118
+ ## Recommended Solutions
119
+
120
+ ### Option 1: Long-Polling (SIMPLEST)
121
+
122
+ **When to use**:
123
+ - Updates less than 1 per minute
124
+ - Moderate concurrency (<10,000 clients)
125
+ - Simple deployment preferred
126
+
127
+ **Example**: see the `NotificationPollLogic` sketch above.
128
+
129
+ **Complexity**: ⭐ Simple
130
+ **Otto Integration**: ✅ Native support (no changes needed)
131
+
132
+ ---
133
+
134
+ ### Option 2: Separate SSE Service (RECOMMENDED FOR REAL-TIME)
135
+
136
+ **When to use**:
137
+ - Updates 1-10 per second
138
+ - High concurrency (>10,000 clients)
139
+ - Near-instant updates required (<100ms latency)
140
+
141
+ **Architecture**:
142
+ ```
143
+ ┌─────────────────┐
144
+ │ Otto API │ ← Stateless HTTP (authentication, business logic)
145
+ │ (Puma) │
146
+ └─────────────────┘
147
+
148
+ ┌─────────┐
149
+ │ Redis │ ← Message queue (pub/sub)
150
+ │ Pub/Sub │
151
+ └─────────┘
152
+
153
+ ┌─────────────────┐
154
+ │ SSE Service │ ← Stateful streaming (Falcon/Iodine)
155
+ │ (Falcon) │
156
+ └─────────────────┘
157
+ ```
158
+
159
+ **Example**: see "Otto + Falcon SSE Integration" in
160
+ `docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md`.
161
+
162
+ **Complexity**: ⭐⭐ Moderate
163
+ **Otto Integration**: ✅ Via Redis pub/sub
164
+
165
+ ---
166
+
167
+ ### Option 3: Third-Party Service (COMMERCIAL)
168
+
169
+ **When to use**:
170
+ - Don't want to manage WebSocket infrastructure
171
+ - Need global CDN distribution
172
+ - Require guaranteed SLA
173
+
174
+ **Options**:
175
+ - **Mercure**: Open-source SSE hub (self-hosted or managed)
176
+ - **Ably**: Commercial real-time messaging platform
177
+ - **Pusher**: Commercial WebSocket/SSE service
178
+
179
+ **Complexity**: ⭐⭐ Moderate (integration)
180
+ **Otto Integration**: ✅ Via HTTP API
181
+
182
+ ---
183
+
184
+ ## Documentation Created
185
+
186
+ ### **docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md** (15,000+ words)
187
+ Comprehensive technical analysis covering:
188
+ - Otto's current architecture (detailed lifecycle analysis)
189
+ - Technical requirements for SSE/WebSocket (Rack 3, hijacking, etc.)
190
+ - Industry patterns (Rails, Sinatra, Roda, Go, Node.js)
191
+ - Compatibility analysis (why it doesn't fit)
192
+ - Best practices and anti-patterns
193
+ - Detailed recommendations with code examples
194
+
195
+ ### Runnable examples: not yet published
196
+
197
+ Draft `examples/otto_falcon_sse_integration.rb` and
198
+ `examples/long_polling_example.rb` were written alongside this analysis but
199
+ targeted an Otto API surface that does not exist (`Otto::RequestContext`,
200
+ `enable_sessions!`, `raise_concern`, a `session` helper, `Rack::Handler::Puma`
201
+ under the Rack 3 pin) and used symbol-keyed params that Logic routes never
202
+ populate. They were withheld rather than shipped as copy-paste material; the
203
+ architecture they demonstrated is preserved in the snippets here and in the
204
+ analysis document.
205
+
206
+ ---
207
+
208
+ ## Key Insights
209
+
210
+ ### The Real Question
211
+
212
+ **Not**: "Can Otto support SSE/WebSocket?"
213
+ (Technically possible with massive refactoring)
214
+
215
+ **But**: "Should Otto support SSE/WebSocket?"
216
+ (Architecturally inadvisable)
217
+
218
+ ### Answer: **No**
219
+
220
+ Otto should remain focused on its core strengths:
221
+ - ✅ **Stateless** HTTP APIs
222
+ - ✅ **Security-first** design (CSRF, rate limiting, validation)
223
+ - ✅ **Privacy by default** (IP masking, geo-location)
224
+ - ✅ **Server-agnostic** (works with any Rack server)
225
+ - ✅ **Simple** and predictable architecture
226
+
227
+ Adding SSE/WebSocket would:
228
+ - ❌ Compromise architectural integrity
229
+ - ❌ Force specific async servers (Falcon, Iodine)
230
+ - ❌ Complicate security guarantees (middleware assumptions broken)
231
+ - ❌ Add significant complexity for niche use case
232
+ - ❌ Go against industry best practices (separation of concerns)
233
+
234
+ ---
235
+
236
+ ## What Otto SHOULD Do
237
+
238
+ ### 1. ✅ Document Integration Patterns
239
+
240
+ Add official guide: "Integrating Otto with Real-Time Services"
241
+ - Long-polling patterns (built-in support)
242
+ - Separate SSE service pattern (Otto + Falcon + Redis)
243
+ - Third-party service integration (Mercure, Ably, Pusher)
244
+
245
+ ### 2. ✅ Provide Example Code
246
+
247
+ Add to `examples/` directory, written against the real Logic-class contract and
248
+ verified to boot:
249
+ - `long_polling_example.rb`
250
+ - `otto_falcon_sse_integration.rb`
251
+ - `otto_mercure_integration.rb`
252
+
253
+ ### 3. ⚠️ Consider Plugin System (If Community Demands)
254
+
255
+ **Only if there's strong demand**, create experimental plugin:
256
+ - Clearly marked "experimental" and "unsupported"
257
+ - Requires Falcon/Iodine (documented)
258
+ - Security implications documented
259
+ - No core changes required
260
+
261
+ ### 4. ❌ Do NOT Add to Core
262
+
263
+ Preserve Otto's architectural integrity by:
264
+ - Keeping core stateless and synchronous
265
+ - Maintaining server-agnostic design
266
+ - Focusing on security and simplicity
267
+ - Following industry best practices (separation of concerns)
268
+
269
+ ---
270
+
271
+ ## Migration Guide for Existing Users
272
+
273
+ If you currently need real-time updates:
274
+
275
+ ### Step 1: Assess Your Use Case
276
+
277
+ **Low-frequency updates (<1/min)**:
278
+ → Use long-polling (Otto native support)
279
+
280
+ **Medium-frequency updates (1-10/sec)**:
281
+ → Use separate SSE service (Otto + Falcon + Redis)
282
+
283
+ **High-frequency or bidirectional**:
284
+ → Use separate WebSocket service or commercial solution
285
+
286
+ ### Step 2: Implementation Path
287
+
288
+ #### For Long-Polling:
289
+ 1. Create Otto route with long-polling logic
290
+ 2. Use `sleep` loop with timeout
291
+ 3. Client polls with timeout parameter
292
+ 4. No external dependencies needed
293
+
294
+ See the `NotificationPollLogic` sketch above.
295
+
296
+ #### For Separate SSE Service:
297
+ 1. Otto API handles authentication and publishes to Redis
298
+ 2. Separate Falcon app subscribes to Redis and streams SSE
299
+ 3. Client connects to SSE service with JWT token from Otto
300
+ 4. Scale Otto and SSE services independently
301
+
302
+ See "Otto + Falcon SSE Integration" in
303
+ `docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md`.
304
+
305
+ ### Step 3: Deployment
306
+
307
+ **Long-Polling**:
308
+ - Deploy with existing Otto setup (Puma, Unicorn, Passenger)
309
+ - Increase thread pool size for long-polling routes
310
+ - Monitor connection pool (ensure enough threads)
311
+
312
+ **Separate SSE Service**:
313
+ - Deploy Otto API with Puma (standard)
314
+ - Deploy Falcon SSE service separately (dedicated servers)
315
+ - Use Redis for pub/sub (cluster-ready)
316
+ - Configure Nginx with sticky sessions for SSE
317
+ - Scale services independently based on load
318
+
319
+ ---
320
+
321
+ ## Performance Guidance
322
+
323
+ ### Long-Polling Capacity
324
+
325
+ **Example**: Puma with 5 workers × 32 threads = 160 concurrent requests
326
+
327
+ If long-polling uses 30s timeout:
328
+ - 160 concurrent connections
329
+ - ~320 clients with 50% utilization
330
+ - Up to 10,000 clients with proper thread tuning
331
+
332
+ **Good for**: Dashboard metrics, low-volume notifications
333
+
334
+ ### SSE/WebSocket Capacity
335
+
336
+ **Example**: Falcon with 4 workers (async)
337
+
338
+ Each worker handles thousands of concurrent connections via fibers:
339
+ - 10,000+ concurrent SSE connections per server
340
+ - Horizontal scaling via Redis pub/sub
341
+ - Near-instant message delivery
342
+
343
+ **Good for**: Chat, multiplayer, high-frequency updates
344
+
345
+ ---
346
+
347
+ ## Conclusion
348
+
349
+ **Otto should NOT integrate SSE/WebSocket support** because:
350
+
351
+ 1. **Architectural mismatch**: Stateless vs stateful paradigms
352
+ 2. **Industry consensus**: Separate services is best practice
353
+ 3. **Complexity cost**: Massive refactoring for niche use case
354
+ 4. **Better alternatives**: Long-polling (simple) or separate services (powerful)
355
+
356
+ **Instead, Otto should**:
357
+
358
+ 1. ✅ Document long-polling patterns (works today)
359
+ 2. ✅ Provide integration examples (Otto + Falcon + Redis)
360
+ 3. ✅ Recommend third-party solutions (Mercure, Ably, Pusher)
361
+ 4. ✅ Stay focused on stateless HTTP APIs
362
+
363
+ **This preserves Otto's core strengths** while enabling users who need real-time functionality to integrate appropriate solutions.
364
+
365
+ ---
366
+
367
+ ## Further Reading
368
+
369
+ - **Technical Analysis**: `docs/1108-STREAMING_ARCHITECTURE_ANALYSIS.md`
370
+ - **Rack 3 Streaming**: https://github.com/rack/rack/issues/1600
371
+ - **ActionCable Architecture**: https://guides.rubyonrails.org/action_cable_overview.html
372
+ - **SSE vs WebSocket**: https://ably.com/blog/websockets-vs-sse
373
+
374
+ ---
375
+
376
+ **End of Summary**
@@ -0,0 +1,172 @@
1
+ # Geo-country resolution
2
+
3
+ Otto resolves a country-level ISO 3166-1 alpha-2 code for each request and
4
+ exposes it as `req.geo_country` / `env['otto.privacy.geo_country']`. Resolution
5
+ is country-only by design — that is the privacy posture; there is no city or
6
+ region lookup.
7
+
8
+ ## Resolution order
9
+
10
+ `Otto::Privacy::GeoResolver.resolve` returns the first hit from:
11
+
12
+ 1. **Application-configured header** (`geo_header:`) — e.g. `X-Client-Country`.
13
+ 2. **Known provider headers** — Cloudflare (`CF-IPCountry`), AWS CloudFront,
14
+ Fastly, Akamai Edgescape, Azure Front Door, **Vercel**
15
+ (`X-Vercel-IP-Country`), and a few semi-standard names
16
+ (`X-Geo-Country`, `X-Country-Code`, `Country-Code`).
17
+ 3. **Custom resolver** (`GeoResolver.custom_resolver`) — your own callable.
18
+ Unlike the other geo settings, this is **class-level** (see
19
+ [Configuration](#configuration)).
20
+ 4. **Local MMDB database** (`geo_db_path:` / `geo_db_reader:`) — a MaxMind-DB
21
+ country database.
22
+ 5. **`'**'`** — the unknown sentinel, when nothing else matches.
23
+
24
+ Steps 1 and 2 are **only consulted when geo headers can be trusted** (see
25
+ [Header trust](#header-trust-and-spoofing) below).
26
+
27
+ Resolution is **honest**: Otto does not guess from a hardcoded IP-range table.
28
+ When no header, custom resolver, or database resolves a country, the result is
29
+ `'**'`.
30
+
31
+ ### Privacy: masked IP and masked env
32
+
33
+ The database lookup in step 4 runs on the request's **masked** IP
34
+ (e.g. `203.0.113.0`), never the real address. `check_geo_database` masks the IP
35
+ internally with the config's `octet_precision` before the lookup, so even a
36
+ direct `GeoResolver.resolve` caller passing a real IP does not expose it to the
37
+ database. Country-level MMDB networks are almost always ≥ /24, so the default
38
+ /24-masked value (`octet_precision: 1`) resolves to the same country.
39
+
40
+ In the middleware path Otto additionally hands `resolve` a **masked env view**:
41
+ `REMOTE_ADDR`, `X-Forwarded-For`, `X-Real-IP`, `X-Client-IP`, and the RFC 7239
42
+ `Forwarded` header are masked. So a `custom_resolver` cannot read the raw client
43
+ IP out of `env` either — use the `ip` argument (already masked), not `env`.
44
+
45
+ > **`octet_precision: 2`** masks two octets (a /16). That is coarser than most
46
+ > country networks, so it can reduce database hit rate for the small share of
47
+ > countries whose ranges are finer than /16 — those requests fall through to
48
+ > `'**'`. Header and custom-resolver sources are unaffected (they ignore the
49
+ > IP). Keep the default precision if you rely on the MMDB fallback.
50
+
51
+ ## Configuration
52
+
53
+ All geo configuration is **boot-time only** (set once during single-threaded
54
+ initialization, before serving requests), matching `custom_resolver`'s
55
+ contract. `geo_header`, `geo_db_path`, and `geo_db_reader` are stored on the
56
+ instance's `Otto::Privacy::Config`, so separate Otto instances hold independent
57
+ geo configuration.
58
+
59
+ > **`custom_resolver` is the exception — it is class-level, not per-instance.**
60
+ > `GeoResolver.custom_resolver=` sets a singleton on the `GeoResolver` class, so
61
+ > it is **shared across every Otto instance in the process** (last write wins).
62
+ > If you run multiple Otto instances that need different resolver strategies,
63
+ > the custom resolver cannot distinguish them — branch inside a single resolver
64
+ > on `env`, or use per-instance `geo_db_reader` instead.
65
+
66
+ ```ruby
67
+ otto.configure_ip_privacy(
68
+ geo: true, # default; false disables geo entirely
69
+ geo_header: 'X-Client-Country', # trusted app header (optional)
70
+ geo_db_path: 'data/geo-whois-asn-country.mmdb', # local MMDB fallback (optional)
71
+ # geo_db_reader: MaxMind::DB.new(path), # or bring your own reader (optional)
72
+ )
73
+ ```
74
+
75
+ - **`geo: false`** short-circuits everything: no header reads, and any loaded
76
+ database is unloaded from memory (`req.geo_country` becomes `nil`).
77
+ - **`geo_header:`** accepts either the HTTP header name (`X-Client-Country`) or
78
+ the Rack CGI env key (`HTTP_X_CLIENT_COUNTRY`), in any case, and is
79
+ canonicalized to the env-key form. Pass `''` to clear.
80
+ - **`geo_db_path:`** is loaded once at boot in `MODE_MEMORY`. An unreadable
81
+ path, a corrupt/non-MMDB file, or a missing `maxmind-db` gem raises
82
+ `ArgumentError` **at configuration time**, not per-request. Pass `''` to
83
+ unload.
84
+ - **`geo_db_reader:`** injects any object responding to `#get(ip)` (a
85
+ preconfigured `MaxMind::DB` reader or a test double), keeping the reader and
86
+ data-source choice independent of Otto. It **overrides** `geo_db_path` when
87
+ both are given in the same call; supplying `geo_db_path` alone in a later call
88
+ clears a prior reader override.
89
+
90
+ Each keyword follows a `nil` = "leave unchanged" contract; pass `''` to a header
91
+ or path to clear it. Any geo-affecting change triggers the boot-time database
92
+ (re)load, so a bad `geo_db_path` fails at the `configure_ip_privacy` call.
93
+
94
+ ## The database: gem and datafile
95
+
96
+ The reader and the data file are independent — the MMDB format is the interop
97
+ point.
98
+
99
+ ### Reader gem (`maxmind-db`)
100
+
101
+ The [`maxmind-db`](https://rubygems.org/gems/maxmind-db) gem (official MaxMind
102
+ reader, Apache-2.0, pure Ruby, zero runtime deps) is an **optional**
103
+ dependency. Otto only `require`s it when a database is configured. Add it to
104
+ your app when you use the database fallback:
105
+
106
+ ```ruby
107
+ # Gemfile
108
+ gem 'maxmind-db', '~> 1.4'
109
+ ```
110
+
111
+ ### Data file (`geo-whois-asn-country`)
112
+
113
+ The recommended data file is
114
+ [`geo-whois-asn-country`](https://github.com/sapics/ip-location-db) from
115
+ sapics/ip-location-db: **PDDL v1.0 (public domain, no attribution required)**,
116
+ rebuilt daily, shipped as MMDB. Otto vendors no database — country data goes
117
+ stale, and a public-domain file you refresh on your own schedule keeps
118
+ licensing and freshness in your control.
119
+
120
+ Download it (IPv4+IPv6) into a path of your choosing:
121
+
122
+ ```bash
123
+ mkdir -p data
124
+ curl -fsSL -o data/geo-whois-asn-country.mmdb \
125
+ https://github.com/sapics/ip-location-db/releases/download/latest/geo-whois-asn-country.mmdb
126
+ ```
127
+
128
+ Refresh it on your own schedule (e.g. a daily cron job running the same curl).
129
+ Any MMDB country database works — GeoLite2-Country, DB-IP Country Lite,
130
+ iplocate, etc. — since `GeoResolver` tolerates the record shapes country
131
+ databases actually use: nested `country.iso_code` (GeoLite2-Country style), a
132
+ flat `country_code` string, and a bare-string `country`.
133
+
134
+ > **Note on GeoLite2:** its EULA requires a MaxMind account/license key and
135
+ > obliges consumers to refresh within 30 days of each release. A PDDL dataset
136
+ > avoids both obligations.
137
+
138
+ ## Header trust and spoofing
139
+
140
+ Every geo header is trivially client-spoofable unless the request actually
141
+ arrived through the CDN that sets it. Otto trusts geo headers — both the
142
+ configured `geo_header` and the provider headers — **only** for a request that
143
+ demonstrably arrived via a configured **CIDR trusted proxy**
144
+ (`env['otto.via_trusted_proxy']` with `trusted_proxies` configured). A spoofed
145
+ header on a direct connection is ignored, and resolution falls through to the
146
+ custom resolver / database.
147
+
148
+ Origins Otto cannot verify are **not** trusted:
149
+
150
+ - **No trusted-proxy configuration.** A direct internet client could otherwise
151
+ pick its own country by sending `CF-IPCountry` / `X-Client-Country`, so with
152
+ no `trusted_proxies` configured, header steps are skipped and resolution falls
153
+ to the resolver / database (`'**'` if neither is set).
154
+ - **Count-based `trusted_proxy_depth` mode.** The header-setting hop cannot be
155
+ verified as a geo-CDN, so depth mode does not enable header trust.
156
+
157
+ **Migration:** to keep header-based geo, configure `trusted_proxies` (CIDR
158
+ matchers) so Otto can verify the proxy origin. Depth-mode and header-only
159
+ deployments should set `geo_db_path` for a local database instead; otherwise
160
+ resolution returns `'**'`.
161
+
162
+ ## Acceptance behavior summary
163
+
164
+ | Scenario | Result |
165
+ | --- | --- |
166
+ | Configured `geo_header` present and trusted | wins over provider headers |
167
+ | Request not via a verified CIDR trusted proxy | geo headers skipped |
168
+ | No `trusted_proxies` configured | geo headers skipped (not trusted) |
169
+ | Database lookup | uses the masked IP only |
170
+ | `geo: false` | `nil`, no database in memory |
171
+ | Bad `geo_db_path` | raises at boot, not per-request |
172
+ | Nothing matches | `'**'` |
@@ -177,12 +177,25 @@ deny). Everything fails closed.
177
177
  This is the load-bearing decision, and it corrects the obvious-but-wrong first
178
178
  instinct (which every initial design in the panel made).
179
179
 
180
- `Otto::CaddyTLS::LocalhostGuard` reads the **original `env['REMOTE_ADDR']`** the
181
- TCP socket peer and runs **before** `IPPrivacyMiddleware` rewrites `REMOTE_ADDR`
182
- from forwarded headers. Because the guard is installed via `Otto#use` (appended,
183
- therefore *outermost* in Otto's `reduce`-built stack) and `IPPrivacyMiddleware` is
184
- pinned *innermost*, the guard provably inspects the true socket peer regardless of
185
- when `enable_caddy_tls!` is called.
180
+ `Otto::CaddyTLS::LocalhostGuard` authenticates the **original TCP socket peer**,
181
+ never the address left in `REMOTE_ADDR` after `IPPrivacyMiddleware` rewrites it
182
+ from forwarded headers.
183
+
184
+ Otto originally guaranteed that by ordering: the guard is installed via `Otto#use`
185
+ (appended, therefore *outermost* in Otto's `reduce`-built stack) and
186
+ `IPPrivacyMiddleware` was pinned *innermost*, so the guard ran first and read a
187
+ pristine `REMOTE_ADDR`. Issue #219 inverted that: masking innermost meant every
188
+ *other* middleware saw raw IPs, so `IPPrivacyMiddleware` is now pinned **outermost**
189
+ (the `:entrypoint` tier) and runs *ahead* of the guard.
190
+
191
+ The guarantee is preserved by a record rather than by order.
192
+ `IPPrivacyMiddleware` evaluates the untouched peer before masking and stores the
193
+ verdict as `env['otto.peer_loopback']` — a boolean, never an address, so it leaks
194
+ nothing. The guard reads that record when present and evaluates `REMOTE_ADDR`
195
+ itself when it is not (guard mounted outside Otto, or no privacy middleware in the
196
+ stack). Both paths share `Otto::Utils.loopback_address?`, so they cannot drift, and
197
+ the decision is made on the raw peer either way — regardless of when
198
+ `enable_caddy_tls!` is called.
186
199
 
187
200
  Reading Otto's resolved `otto.client_ip` (or the rewritten `REMOTE_ADDR`) would be
188
201
  **exploitable**: `Otto::Utils.resolve_client_ip` honors `X-Forwarded-For` when the
@@ -3,15 +3,46 @@
3
3
 
4
4
  # Otto GeoResolver Extension Guide
5
5
  #
6
- # This guide shows two approaches to extend Otto's IP geolocation:
7
- # 1. Configuration-based (simple, inline)
8
- # 2. Subclass-based (full control)
6
+ # Otto resolves a country code in this order (first hit wins):
7
+ # 1. App-configured trusted header (configure_ip_privacy(geo_header:))
8
+ # 2. Built-in CDN/provider headers (Cloudflare, AWS, Vercel, ...)
9
+ # 3. Custom resolver hook (GeoResolver.custom_resolver = ...)
10
+ # 4. Local MMDB database (configure_ip_privacy(geo_db_path:/geo_db_reader:))
11
+ # 5. '**' (unknown) Otto does not guess from a hardcoded table
12
+ #
13
+ # This guide shows the extension points:
14
+ # A. Built-in configuration (trusted header + local database) — no code
15
+ # B. Custom resolver hook (inline or a callable object)
16
+ # C. Subclass-based (full control)
9
17
 
10
18
  require 'bundler/setup'
11
19
  require 'otto'
12
20
 
13
21
  # =============================================================================
14
- # Quick Start: Configuration-based Extension
22
+ # A. Built-in configuration: trusted header + local country database
23
+ # =============================================================================
24
+ #
25
+ # No custom code needed — just configure the Otto instance. The database is
26
+ # looked up on the already-MASKED IP, and a bad geo_db_path fails at boot.
27
+ #
28
+ # otto = Otto.new('routes.txt')
29
+ # otto.configure_ip_privacy(
30
+ # geo_header: 'X-Client-Country', # trusted header checked before CDN headers
31
+ # geo_db_path: 'data/country.mmdb' # offline fallback (needs the maxmind-db gem)
32
+ # )
33
+ #
34
+ # Prefer to bring your own reader (any object responding to #get)? Inject it —
35
+ # this keeps the reader/data-source choice independent of Otto:
36
+ #
37
+ # reader = MaxMind::DB.new('data/country.mmdb', mode: MaxMind::DB::MODE_MEMORY)
38
+ # otto.configure_ip_privacy(geo_db_reader: reader)
39
+ #
40
+ # Security note: geo headers are only trusted for requests that arrive via a
41
+ # configured trusted proxy (add_trusted_proxy), since they are client-spoofable
42
+ # otherwise. configure_ip_privacy(geo: false) disables geo entirely.
43
+
44
+ # =============================================================================
45
+ # B. Quick Start: Custom resolver hook
15
46
  # =============================================================================
16
47
 
17
48
  puts 'Simple Custom Geo Resolution'
@@ -31,7 +62,9 @@ Otto::Privacy::GeoResolver.custom_resolver = custom_resolver
31
62
 
32
63
  # Step 3: Test it
33
64
  puts "1.2.3.4 -> #{Otto::Privacy::GeoResolver.resolve('1.2.3.4', {})}"
34
- puts "8.8.8.8 -> #{Otto::Privacy::GeoResolver.resolve('8.8.8.8', {})} (fallback)"
65
+ # Resolver returns nil for 8.8.8.8, and there is no header or database, so the
66
+ # honest answer is '**' (unknown) — Otto does not guess.
67
+ puts "8.8.8.8 -> #{Otto::Privacy::GeoResolver.resolve('8.8.8.8', {})} (unknown)"
35
68
 
36
69
  # Reset for next example
37
70
  Otto::Privacy::GeoResolver.custom_resolver = nil