otto 2.9.0 → 2.10.0

data/docs/guides/privacy.md

@@ -0,0 +1,193 @@
+# Privacy-preserving request data
+
+Otto's default request posture is to reduce the precision of public client
+information before application code, logging, authentication, rate limiting,
+and other middleware see it. The privacy pipeline is a Rack concern, not only a
+router feature: if an outer Rack middleware logs the request before Otto runs,
+place the privacy middleware in the common stack first.
+
+## Default behavior
+
+With the default `:masked` profile:
+
+- public IP addresses are masked by the configured octet precision (one octet
+  by default: `203.0.113.9` becomes `203.0.113.0`);
+- requests from private and loopback addresses are exempt from the privacy
+  fingerprint by default, so their IP, user agent, and referer remain unchanged;
+- for requests that are masked, user-agent version details are anonymized and
+  referer query parameters are removed;
+- original public values are not retained in the Rack environment; and
+- country resolution is country-level only and returns `**` when enabled but no
+  configured source answers.
+
+Downstream code should read `req.ip`, `req.masked_ip`, and the documented privacy
+environment keys rather than re-resolving an address from forwarded headers.
+
+These profiles are technical data-minimization controls, not a compliance
+certification. Whether a deployment meets GDPR, CCPA, or another legal regime
+also depends on its purposes, notices, retention, access controls, vendors, and
+jurisdiction.
+
+## Profiles
+
+Choose a named profile when the deployment posture should be obvious in review:
+
+```ruby
+otto = Otto.new('routes')
+otto.configure_ip_privacy(profile: :masked)     # default
+# or:
+otto.configure_ip_privacy(profile: :anonymous)  # mask private/localhost too
+# or:
+otto.configure_ip_privacy(profile: :audit)      # disable IP privacy
+```
+
+| Profile | Public IPs | Private/localhost IPs | Use when |
+| --- | --- | --- | --- |
+| `:masked` | Masked | Exempt by default | General privacy-by-default deployments and local development. |
+| `:anonymous` | Masked | Masked | Internal addresses must also be treated as identifying data. |
+| `:audit` | Not masked | Not masked | The operator has deliberately accepted raw-IP retention and controls logs and downstream systems. |
+
+The `:audit` profile transfers retention responsibility to the operator. It is
+not a way to obtain precise matching while keeping privacy enabled; use
+`env['otto.ip_match']` for that narrower need.
+
+Configuration is boot-time only. The first request freezes configuration in
+normal operation, so set profiles, trusted proxies, database readers, and other
+privacy settings before serving traffic.
+
+## Privacy-safe request values
+
+Inside a handler:
+
+```ruby
+class Analytics
+  def self.record(req, res)
+    event = {
+      ip: req.ip,
+      country: req.geo_country,
+      asn: req.asn,
+      anonymizer: req.anonymizer,
+      user_agent: req.user_agent,
+    }
+
+    AuditLog.write(event)
+    res.status = 204
+    res.body = []
+  end
+end
+```
+
+Common values are also available in the Rack environment:
+
+| Value | Environment key | Contract |
+| --- | --- | --- |
+| Canonical client IP | `otto.client_ip` | Masked IP when masking applies; resolved full IP under `:audit` or for an exempt private/loopback request. |
+| Precise CIDR verdict | `otto.ip_match` | Callable that checks the resolved full IP against CIDRs and returns only `true` or `false`. |
+| Masked IP | `otto.privacy.masked_ip` | Set when the request runs through the privacy fingerprint; absent for exempt or `:audit` requests. |
+| Rotating IP hash | `otto.privacy.hashed_ip` | Correlation value computed with Otto's rotating key; absent for exempt or `:audit` requests. |
+| Stable correlation hash | `otto.privacy.correlation_hash` | HMAC value when `correlation_secret:` is configured; otherwise `nil`. |
+| Country | `otto.privacy.geo_country` | ISO 3166-1 alpha-2 code, `**` when enabled but unresolved, or `nil` when geo/privacy is disabled. |
+| ASN | `otto.privacy.asn` | `nil` when off, `**` when enabled but unresolved, or a value such as `AS15169`. |
+| Anonymizer | `otto.privacy.anonymizer` | `nil` when off, `**` when no database answers, or a classification label. |
+
+`req.hashed_ip` is designed for short-lived correlation using a rotating key.
+For long-lived correlation, explicitly configure a stable secret and protect
+that secret as sensitive configuration. Changing the secret changes every
+correlation hash.
+
+## Country, ASN, and anonymizer data
+
+These signals have different trust and precision models:
+
+- [Geo-country resolution](geo-country.md) documents trusted provider headers,
+  CIDR trusted-proxy requirements, masked MMDB lookup, and the unknown sentinel.
+- [ASN and anonymizer enrichment](enrichment.md) documents the opt-in database
+  contracts. ASN uses a masked address; anonymizer classification deliberately
+  uses the unmasked address internally and emits only a label.
+
+Enable optional signals explicitly:
+
+```ruby
+otto.configure_ip_privacy(
+  geo: true,
+  geo_db_path: 'data/country.mmdb',
+  asn: true,
+  asn_db_path: 'data/origin-asn.mmdb',
+  anonymizer: true,
+  anonymizer_db_path: 'data/anonymizer.mmdb'
+)
+```
+
+The `maxmind-db` gem is optional and is required only when a database path is
+configured. A reader object responding to `#get(ip)` can be injected instead.
+Database paths are opened at configuration time; invalid paths fail during boot
+rather than on an arbitrary request.
+
+## Trusted proxy and matching boundaries
+
+Configure trusted proxies before relying on forwarded client information:
+
+```ruby
+otto = Otto.new(
+  'routes',
+  trusted_proxies: ['10.0.0.0/8', '192.0.2.0/24']
+)
+```
+
+CIDR-based trust lets Otto verify the proxy peer. Count-based
+`trusted_proxy_depth` is a separate mode and does not make geo headers
+trustworthy. A configured `geo_header` combined with depth mode is rejected at
+configuration time; use a local database in depth-mode deployments instead.
+
+The same trust decision also gates the forwarded host, scheme, and port headers
+that `Rack::Request#host` reads. See
+[Forwarded host authority](forwarded-authority.md), including the explicit
+`trusted_proxies: :none` assertion for directly exposed applications.
+
+For precise access control without exposing the address to application code,
+call the automatically installed `env['otto.ip_match']` capability. It matches
+the resolved full client IP against application CIDRs and returns only
+`true`/`false`; it fails closed when no client IP resolves. Do not log or persist
+the closure.
+
+```ruby
+allowed = req.env.fetch('otto.ip_match').call(['192.0.2.0/24', '2001:db8::/32'])
+```
+
+## Middleware placement
+
+When building a larger Rack stack, put privacy before components that log or
+inspect the request:
+
+```ruby
+builder.use Otto::Security::Middleware::IPPrivacyMiddleware, otto.security_config
+builder.use Rack::CommonLogger
+builder.use Sentry::Rack::CaptureExceptions
+# Then mount the Otto application.
+```
+
+Otto also installs its privacy middleware internally. The outer common-stack
+placement is needed when middleware outside the Otto app would otherwise see the
+raw peer first.
+
+Pass `otto.security_config` as shown. The outer instance resolves `otto.client_ip`
+first and the inner one then short-circuits, so an outer instance constructed
+without the configuration would silently apply defaults instead of the
+application's profile, precision, correlation secret, and enrichment settings.
+Proxy trust is the exception: the inner instance re-applies its own trust
+posture even after an outer pass, and a CIDR configuration that can no longer
+see the connecting peer fails closed. See
+[Forwarded host authority](forwarded-authority.md#place-the-middleware-before-other-request-consumers).
+
+## Operational rules
+
+- Treat `nil`, `'**'`, `'none'`, and a real label as different states. In
+  particular, anonymizer `'none'` means the database was consulted and did not
+  list the address; `'**'` means no database answered.
+- Do not use provider geo headers without a verifiable CIDR trusted proxy. A
+  client can otherwise submit its own country.
+- Do not confuse the masked client IP with the raw peer used to authenticate a
+  local network service. Integrations such as Caddy TLS have their own trust
+  boundary.
+- Test logs and monitoring outside Otto's stack. Privacy is only effective if
+  those components receive the masked environment first.

data/docs/guides/routing.md

@@ -0,0 +1,181 @@
+# Routing applications with Otto
+
+Otto keeps the application boundary small: a plain-text route file maps an HTTP
+verb and path to a Ruby handler. Use this guide to choose a handler style and
+response contract. The exact route grammar is in the [route syntax reference](../reference/route-syntax.md).
+
+## Choose a handler style
+
+| Use this when | Route target | Invocation |
+| --- | --- | --- |
+| You need a class method with direct Rack access | `App.index` | `App.index(req, res)` |
+| You need an object with direct Rack access | `App#show` | `App.new(req, res).show` |
+| You want a constrained, testable application operation | `App::Operation` | `App::Operation.new(strategy_result, params, locale)` |
+| You need a small pre-registered endpoint function | `&name` | `call(req, res, captured_path_params)` |
+
+## Controller-style handlers
+
+Routes can call a class method or instantiate a class for an instance method:
+
+```text
+GET /                         App.index
+GET /products/:id             App#show
+```
+
+```ruby
+class App
+  def initialize(req, res)
+    @req = req
+    @res = res
+  end
+
+  def show
+    @res.body = "Product: #{@req.params[:id]}"
+  end
+
+  def self.index(req, res)
+    res['content-type'] = 'text/plain'
+    res.body = 'Hello Otto'
+  end
+end
+```
+
+Use this style when the handler needs cookies, request headers, the Rack
+request object, or direct response helpers.
+
+## Logic classes
+
+Use a bare class target for an operation with an explicit input context:
+
+```text
+GET  /products/:id  Products::Show auth=session response=json
+POST /products      Products::Create auth=session response=json
+```
+
+```ruby
+class Products::Show
+  def initialize(strategy_result, params, locale)
+    @context = strategy_result
+    @params = params
+    @locale = locale
+  end
+
+  def raise_concerns
+    @product = Product.find(@params[:id])
+    unless @product.public? || @product.owner_id == @context.user_id
+      raise Otto::Security::AuthorizationError, 'Product access denied'
+    end
+  end
+
+  def process
+    { id: @product.id, name: @product.name, locale: @locale }
+  end
+end
+```
+
+Otto runs `raise_concerns` before `process` when those methods exist. Put
+resource loading and resource-level authorization in `raise_concerns`; route
+authentication and broad role checks belong in the route definition.
+
+Logic classes do not receive the Rack environment. This keeps their inputs
+explicit and prevents application operations from depending on ambient request
+state. Choose a controller-style handler when direct request access is part of
+the operation.
+
+## Registered lambda handlers
+
+Lambda routes are useful for small endpoints that do not need a Ruby constant or
+handler class. Register the callable at boot:
+
+```ruby
+otto = Otto.new('routes', lambda_handlers: {
+  health_check: lambda do |_req, res, _extra_params|
+    res['content-type'] = 'text/plain'
+    res.body = 'ok'
+  end,
+})
+```
+
+```text
+GET /health &health_check
+```
+
+The registry is normalized and frozen during configuration. A lambda must
+accept three positional arguments: request, response, and captured path
+parameters. Query and form parameters remain available through `req.params`.
+The route name is an exact registry key; it is not evaluated as Ruby code.
+
+## Response selection
+
+Use `response=` when the handler returns a value that should pass through Otto's
+response handling:
+
+```text
+GET  /api/products  Products::Index response=json
+GET  /dashboard     Dashboard#show response=view
+POST /login         Sessions#create response=redirect
+GET  /data          Data#show response=auto
+```
+
+`response=default` is the default. Keep response selection in the route file
+so the HTTP contract is visible beside the endpoint.
+
+| Response type | Handler contract |
+| --- | --- |
+| `default` | Mutate `res` directly. The handler's return value is ignored. |
+| `json` | Return a Hash for direct JSON serialization. `nil` becomes `{ "success": true }`; another value is wrapped as `data`. A Logic class may instead provide `response_data`. |
+| `view` | Return a value rendered with `to_s`, or provide `view.render` on a Logic object. |
+| `redirect` | Return a path String, or provide `redirect_path` on a Logic object. The fallback path is `/`. |
+| `auto` | A Hash becomes JSON, a path-like String becomes a redirect, and a Logic object with `view` uses the view handler; other results use default behavior. |
+
+An unknown response name currently falls back to `default`. Treat response
+names as a fixed set; a typo otherwise changes the route to direct-response
+behavior.
+
+## Route parameters
+
+Named path segments are available in request parameters:
+
+```text
+GET /products/:id  Products::Show
+```
+
+A handler can read `req.params[:id]` or a Logic class can read `params[:id]`.
+Request query and body parameters are merged according to the handler's request
+contract. JSON bodies are parsed for Logic-class parameters when the content
+type is JSON and the body is a JSON object. A valid non-object JSON body is
+ignored. Malformed JSON is logged and the Logic class still runs with its other
+parameters; perform application validation when malformed JSON must return a
+client error.
+
+## Security options in routes
+
+Authentication, roles, and CSRF exemptions are explicit route options:
+
+```text
+GET  /profile  Profile#show auth=session
+GET  /admin    Admin::Dashboard auth=session role=admin
+POST /hook     Hooks#receive csrf=exempt
+```
+
+Malformed `auth`, `role`, and `csrf` tokens fail route parsing rather than
+silently weakening the route. Do not use `csrf=exempt` as a general API switch;
+choose an independent request-authentication and replay-protection model for
+webhooks or other non-browser endpoints.
+
+## Configuration timing
+
+Construct and configure the Otto instance before the first request:
+
+```ruby
+otto = Otto.new('routes')
+otto.add_auth_strategy(
+  'session',
+  Otto::Security::Authentication::Strategies::SessionStrategy.new
+)
+otto.register_request_helpers(MyApp::RequestHelpers)
+# Add middleware and other boot-time options here.
+```
+
+In normal operation, the first request freezes configuration. Runtime route or
+security changes are not part of the application contract.

data/docs/guides/structured_logging.md

@@ -0,0 +1,281 @@
+# Structured logging
+
+Otto provides explicit logging helpers for request context, microsecond timing,
+and backtrace path reduction. The helpers do not sanitize arbitrary metadata;
+callers remain responsible for excluding secrets and personal data.
+
+## Logging helpers
+
+Otto provides several helper methods for consistent logging:
+
+```ruby
+# Request context extraction
+Otto::LoggingHelpers.request_context(env)
+# May include: { method:, path:, ip:, country:, user_agent: }
+
+# Timed operation logging
+Otto::LoggingHelpers.log_timed_operation(level, message, env, **metadata) { block }
+```
+
+## Logging patterns
+
+For request-scoped structured logging, use `Otto.structured_log` with
+`LoggingHelpers.request_context(env).merge()`:
+
+`request_context(env)` creates a new context hash, and `.merge` creates another
+hash rather than mutating shared state. Treat the request `env` and values inside
+it as request-owned data; the helper does not deep-copy mutable strings.
+
+```ruby
+# Route logging
+Otto.structured_log(:debug, "Route matched",
+  Otto::LoggingHelpers.request_context(env).merge(
+    type: 'literal',
+    handler: route.route_definition.definition,
+    auth_strategy: route.route_definition.auth_requirement || 'none'
+  )
+)
+
+# Authentication logging
+Otto.structured_log(:info, "Auth strategy result",
+  Otto::LoggingHelpers.request_context(env).merge(
+    strategy: strategy.class.name.split('::').last.downcase.gsub('strategy', ''),
+    success: true,
+    user_id: result.user_id,
+    duration: duration_μs
+  )
+)
+```
+
+For operations that need timing, use `log_timed_operation` which wraps `structured_log` with automatic timing:
+
+```ruby
+# Template compilation with timing
+result = Otto::LoggingHelpers.log_timed_operation(:info, "Template compiled", env,
+  template_type: 'handlebars',
+  cached: false
+) do
+  compile_template(template_path)
+end
+
+# Database operation with timing
+Otto::LoggingHelpers.log_timed_operation(:debug, "User lookup", env,
+  user_id: user_id,
+  cache_hit: false
+) do
+  User.find(user_id)
+end
+```
+
+## Timing conventions
+
+Otto uses **microseconds** for all timing measurements via `Otto::Utils.now_in_μs`:
+
+```ruby
+# Manual timing
+start_time = Otto::Utils.now_in_μs
+result = perform_operation()
+duration = Otto::Utils.now_in_μs - start_time
+
+Otto.structured_log(:info, "Operation completed",
+  Otto::LoggingHelpers.request_context(env).merge(
+    operation: 'user_creation',
+    duration: duration  # Always in microseconds
+  )
+)
+
+# Automatic timing with error handling
+Otto::LoggingHelpers.log_timed_operation(:info, "Database query", env,
+  table: 'users',
+  query_type: 'SELECT'
+) do
+  database.execute(query)
+end
+```
+
+## Request fields
+
+Request-scoped events should normally merge `request_context(env)`, which adds
+available values for:
+
+- **method** - HTTP method (`GET`, `POST`, etc.)
+- **path** - request path
+- **ip** - canonical client IP under the configured privacy profile
+- **country** - resolved country when available
+- **user_agent** - current request user agent, truncated to 100 characters
+
+Unavailable values are omitted. Non-request events, such as configuration or
+startup logs, should include only relevant event-specific fields. Timed events
+also include **duration** in microseconds.
+
+Additional fields such as `user_id`, `handler`, `error`, or `error_class` may be
+useful, but Otto does not redact them. Do not log credentials, session tokens,
+raw request parameters, or exception messages that may contain secrets.
+
+## Error handling in timed operations
+
+`log_timed_operation` automatically handles exceptions:
+
+```ruby
+# Successful operation
+Otto::LoggingHelpers.log_timed_operation(:info, "Template compiled", env, template: 'user') do
+  compile_template('user')
+end
+# Logs: Template compiled: method=GET path=/users template=user duration=15230
+
+# Failed operation (automatic error logging + re-raise)
+Otto::LoggingHelpers.log_timed_operation(:info, "Template compiled", env, template: 'user') do
+  raise StandardError, "Template not found"
+end
+# Logs: Template compiled failed: method=GET path=/users template=user duration=1520 error="Template not found" error_class=StandardError
+# Then re-raises the original exception
+```
+
+## Privacy behavior
+
+`request_context` prefers `env['otto.client_ip']`, the canonical value produced
+by `IPPrivacyMiddleware`:
+
+- Under the normal privacy profiles, public addresses are masked.
+- Under the `:audit` profile or after `disable_ip_privacy!`, the canonical public
+  address is raw and `request_context` logs it unchanged.
+- Private and localhost addresses remain raw under the default `:masked` profile;
+  the `:anonymous` profile masks them too.
+- Outside Otto's middleware, when `otto.client_ip` is absent,
+  `privacy_safe_ip` masks a public `REMOTE_ADDR` using the default precision and
+  returns `[redacted]` for an unparseable address.
+- `request_context` truncates the current `HTTP_USER_AGENT` to 100 characters.
+  It is anonymized only if the privacy middleware has already anonymized it.
+
+The helper does not make arbitrary metadata privacy-safe. Review every field
+added by the caller.
+
+## Backtrace path reduction
+
+Otto reduces filesystem details in recognized Ruby backtrace lines. This is
+defense in depth, not a confidentiality boundary: custom or unrecognized lines
+are returned unchanged.
+
+**Risks of raw backtraces:**
+- Expose absolute paths revealing usernames (`/Users/alice/`, `/home/admin/`)
+- Reveal project structure and internal organization
+- Show gem installation paths and Ruby versions
+- Leak system architecture details
+
+**Automatic path reduction in `Otto.structured_log`:**
+
+`Otto.structured_log` applies the backtrace sanitizer only when the metadata is
+a hash whose exact `:backtrace` key contains an array. Other fields and arrays
+are passed unchanged.
+
+**Path-reduction rules:**
+
+```ruby
+# Project files → relative paths only
+"/Users/alice/myapp/app/controllers/users_controller.rb:42:in `create'"
+# ↓ SANITIZED TO:
+"app/controllers/users_controller.rb:42:in `create'"
+
+# Bundler gems → [GEM] tag with gem name only
+"/Users/alice/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/bundler/gems/otto-34f285412a44/lib/otto/route.rb:142"
+# ↓ SANITIZED TO:
+"[GEM] otto/lib/otto/route.rb:142"
+
+# Regular gems → [GEM] tag, version stripped
+"/opt/ruby/gems/3.4.0/gems/rack-3.2.4/lib/rack/builder.rb:310"
+# ↓ SANITIZED TO:
+"[GEM] rack/lib/rack/builder.rb:310"
+
+# Ruby stdlib → [RUBY] tag with filename only
+"/Users/alice/.rbenv/versions/3.4.7/lib/ruby/3.4.0/logger.rb:310"
+# ↓ SANITIZED TO:
+"[RUBY] logger.rb:310"
+
+# Unknown/external → filename only
+"/some/unknown/path/file.rb:50"
+# ↓ SANITIZED TO:
+"[EXTERNAL] file.rb:50"
+```
+
+**Usage:**
+
+Path reduction happens automatically when using `log_backtrace`. This helper
+logs at `:error` and limits the backtrace to its first 20 lines:
+
+```ruby
+# In error handlers (Otto does this automatically)
+Otto::LoggingHelpers.log_backtrace(error,
+  Otto::LoggingHelpers.request_context(env).merge(
+    error_id: error_id,
+    handler: 'UserController#create'
+  )
+)
+
+# Manual usage if needed
+sanitized = Otto::LoggingHelpers.sanitize_backtrace(error.backtrace)
+```
+
+## Anti-patterns
+
+**❌ Don't create event classes:**
+```ruby
+# NO - Adds unnecessary abstraction
+event = RouteMatchEvent.new(type: :literal, method: http_verb, path: path)
+Otto.structured_log(event.level, event.message, event.to_h)
+```
+
+**❌ Don't create helper wrappers:**
+```ruby
+# NO - Hides what's being logged
+Otto::Logging.log_route_match(type: :literal, method: http_verb, path: path, env: env)
+```
+
+**❌ Don't mix timing units:**
+```ruby
+# NO - Inconsistent units
+duration_ms = (Otto::Utils.now_in_μs - start_time) / 1000  # Converting to milliseconds
+Otto.structured_log(:info, "Operation done", { duration_ms: duration_ms })
+```
+
+**✅ Do use explicit inline logging:**
+```ruby
+# YES - Clear, simple, explicit
+Otto.structured_log(:debug, "Route matched",
+  Otto::LoggingHelpers.request_context(env).merge(
+    type: 'literal',
+    handler: 'App#index'
+  )
+)
+
+# YES - Consistent microsecond timing
+Otto.structured_log(:info, "Operation completed",
+  Otto::LoggingHelpers.request_context(env).merge(
+    operation: 'user_lookup',
+    duration: Otto::Utils.now_in_μs - start_time
+  )
+)
+```
+
+## Output behavior
+
+`Otto.logger` defaults to Ruby's `Logger`. With that logger, metadata is rendered
+inside a formatted string:
+
+```text
+I, [2025-01-21T14:39:39.462833 #82244] INFO -- : [Otto] Template compiled -- {method: "GET", path: "/users", ip: "192.0.2.0", template_type: "handlebars", cached: false, duration: 68}
+```
+
+For a logger whose level method has a fixed arity greater than one, Otto calls
+that method as `logger.info(message, metadata)`. Other logger APIs use the
+formatted-string fallback. Verify the adapter for a third-party structured
+logger before relying on separately indexed fields; Otto does not ship or test
+a SemanticLogger adapter.
+
+## Rationale
+
+- **Simplicity**: Direct logging calls are easier to understand than abstraction layers
+- **Explicitness**: You can see exactly what's being logged at the call site
+- **Flexibility**: Easy to add one-off fields without modifying event classes
+- **Performance**: Disabled debug events are not sent to the logger; guard expensive metadata construction with `if Otto.debug` because method arguments are evaluated first
+- **Consistency**: All timing in microseconds, automatic error handling for timed operations
+- **Maintainability**: One helper file vs multiple event classes/helpers