otto 2.0.0.pre3 → 2.0.0.pre7

data/examples/mcp_demo/README.md

@@ -1,42 +1,82 @@
 # Otto MCP Demo
 
-This example demonstrates Otto's Model-Controller-Protocol (MCP) feature. MCP provides a standardized JSON-RPC 2.0 endpoint (`/_mcp`) that allows you to expose application resources and tools securely.
+This example demonstrates Otto's Model-Controller-Protocol (MCP) feature. MCP provides a standardized JSON-RPC 2.0 endpoint (`/_mcp`) for programmatic access to your application resources and tools.
 
-This is useful for building CLIs, admin interfaces, or allowing other services to interact with your application programmatically.
+MCP is useful for building CLIs, admin interfaces, integrating with AI systems, or allowing other services to interact with your application.
+
+## What You'll Learn
+
+- How to set up an MCP endpoint for programmatic access
+- Exposing application resources via JSON-RPC 2.0
+- Securing MCP endpoints with bearer token authentication
+- Distinguishing between read-only resources and executable tools
+- Organizing methods for both web and MCP interfaces
+- How MCP integrates with your existing Otto application
 
 ## Features Demonstrated
 
-*   **MCP Endpoint:** A single `POST /_mcp` endpoint for all API interactions.
-*   **Authentication:** Requests to the MCP endpoint are protected by bearer token authentication.
-*   **Rate Limiting:** The endpoint has its own rate limiting to prevent abuse.
-*   **Resources:** Read-only data exposed via `MCP` routes (e.g., listing users).
-*   **Tools:** Actions or operations exposed via `TOOL` routes (e.g., creating a user).
+- **MCP Endpoint**: Single `POST /_mcp` endpoint for all interactions
+- **Authentication**: Bearer token authentication for secure access
+- **Rate Limiting**: Built-in rate limiting to prevent abuse
+- **Resources**: Read-only data exposed via `MCP` routes
+- **Tools**: Executable actions exposed via `TOOL` routes
+- **Web Interface**: Separate web routes coexist with MCP routes
+- **JSON-RPC 2.0**: Standard protocol for all MCP interactions
 
 ## How to Run
 
-1.  Make sure you have `bundler` and `thin` installed:
-    ```sh
-    gem install bundler thin
-    ```
+### Using rackup (recommended)
+
+```sh
+cd examples/mcp_demo
+rackup config.ru
+```
+
+### Using thin
+
+```sh
+cd examples/mcp_demo
+thin -R config.ru -p 9292 start
+```
+
+The server will start on `http://localhost:9292`.
+
+- **Web interface**: Navigate to `http://localhost:9292` in your browser
+- **MCP endpoint**: Send JSON-RPC 2.0 requests to `http://localhost:9292/_mcp`
 
-2.  Install the dependencies from the root of the project:
-    ```sh
-    bundle install
-    ```
+## Authentication
 
-3.  Start the server from this directory (`examples/mcp_demo`):
-    ```sh
-    thin -R config.ru -p 9292 start
-    ```
-    *Note: This demo uses port 9292 as is conventional for Rack apps.*
+All MCP requests require bearer token authentication via the `Authorization` header.
 
-4.  Open your browser and navigate to `http://localhost:9292` to see the welcome page.
+Valid tokens:
+- `demo-token-123` - Standard user
+- `another-token-456` - Alternative user
 
 ## Interacting with the MCP Endpoint
 
-All interactions happen via `POST` requests to `http://localhost:9292/_mcp`. You must provide an `Authorization: Bearer <token>` header and a `Content-Type: application/json` header.
+All MCP interactions use the `POST /_mcp` endpoint. Each request is a JSON-RPC 2.0 request with:
 
-Valid tokens are `demo-token-123` and `another-token-456`.
+- `jsonrpc`: Always `"2.0"`
+- `method`: The RPC method name (derived from route path)
+- `id`: Request ID (for matching responses)
+- `params`: Optional parameters as an object
+
+Required headers:
+- `Authorization: Bearer <token>`
+- `Content-Type: application/json`
+
+Example:
+```sh
+curl -X POST http://localhost:9292/_mcp \
+  -H 'Authorization: Bearer demo-token-123' \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "jsonrpc": "2.0",
+    "method": "initialize",
+    "id": 1,
+    "params": {}
+  }'
+```
 
 ### MCP: Initialize
 
@@ -79,9 +119,129 @@ curl -X POST http://localhost:9292/_mcp \
         }'
 ```
 
+## Expected Output
+
+### Successful Initialize Request
+```json
+{
+  "jsonrpc": "2.0",
+  "result": {
+    "resources": [
+      {
+        "uri": "users",
+        "name": "User List",
+        "description": "List all users"
+      }
+    ],
+    "tools": [
+      {
+        "name": "create_user",
+        "description": "Create a new user",
+        "inputSchema": {
+          "type": "object",
+          "properties": {
+            "name": { "type": "string" },
+            "email": { "type": "string" }
+          }
+        }
+      }
+    ]
+  },
+  "id": 1
+}
+```
+
+### Successful Resource Request
+```json
+{
+  "jsonrpc": "2.0",
+  "result": {
+    "users": [
+      { "id": 1, "name": "Alice", "email": "alice@example.com" },
+      { "id": 2, "name": "Bob", "email": "bob@example.com" }
+    ]
+  },
+  "id": 2
+}
+```
+
+### Successful Tool Execution
+```json
+{
+  "jsonrpc": "2.0",
+  "result": {
+    "user": {
+      "id": 3,
+      "name": "Charlie",
+      "email": "charlie@example.com"
+    }
+  },
+  "id": 3
+}
+```
+
+### Authentication Failure
+```json
+{
+  "jsonrpc": "2.0",
+  "error": {
+    "code": -32003,
+    "message": "Unauthorized"
+  },
+  "id": 1
+}
+```
+
 ## File Structure
 
-*   `README.md`: This file.
-*   `app.rb`: Contains the application logic, including the `DemoApp` for web pages and the `UserAPI` for MCP handlers.
-*   `config.ru`: The Rack configuration file. It loads the Otto framework, enables MCP, and runs the application.
-*   `routes`: Defines the standard web routes as well as the `MCP` and `TOOL` routes for the MCP endpoint.
+- `README.md`: This file
+- `app.rb`: Application logic
+  - `DemoApp`: Web interface with HTML pages
+  - `UserAPI`: MCP handlers for resources and tools
+- `config.ru`: Rack configuration (loads Otto, enables MCP)
+- `routes`: Route definitions for web and MCP routes
+
+## Route Types
+
+### Web Routes
+Regular HTTP routes for the web interface:
+```
+GET  /            DemoApp#welcome
+GET  /users       DemoApp#list_users
+```
+
+### MCP Resource Routes
+Read-only resources exposed via MCP:
+```
+MCP  /users       UserAPI#mcp_list_users
+```
+- Called via: `POST /_mcp` with method `users/list`
+
+### MCP Tool Routes
+Executable operations exposed via MCP:
+```
+TOOL /create_user UserAPI#mcp_create_user
+```
+- Called via: `POST /_mcp` with method `create_user`
+
+## Understanding MCP Method Names
+
+MCP route paths are converted to method names:
+
+| Route Type | Path | Method Name | Handler |
+|-----------|------|-------------|---------|
+| MCP | `/users` | `users/list` | `mcp_list_users` |
+| MCP | `/users/:id` | `users/get` | `mcp_get_user` |
+| TOOL | `/create_user` | `create_user` | `mcp_create_user` |
+| TOOL | `/users/:id/update` | `users/update` | `mcp_update_user` |
+
+## Next Steps
+
+- Build a CLI that communicates with the MCP endpoint
+- Integrate with AI systems that support MCP
+- Combine with [Authentication](../authentication_strategies/) for role-based MCP access
+- Explore [Advanced Routes](../advanced_routes/) for more routing patterns
+
+## Further Reading
+
+- [CLAUDE.md](../../CLAUDE.md#mcp) - Detailed MCP documentation (if available)

data/examples/security_features/README.md

@@ -1,46 +1,265 @@
 # Otto Security Features Example
 
-This example application demonstrates the built-in security features of the Otto framework. It is configured to be secure by default and provides a showcase of best practices.
+This example demonstrates Otto's built-in security features, showing best practices for CSRF protection, input validation, file upload handling, and security headers.
+
+## What You'll Learn
+
+- Enabling and using CSRF protection
+- Input validation for preventing injection attacks
+- XSS prevention through output escaping
+- Secure file upload handling with filename sanitization
+- Adding security headers (CSP, HSTS, etc.)
+- Request limiting to prevent denial-of-service
+- Trusted proxy configuration for reverse proxies
+- Privacy features (IP masking, user agent anonymization)
 
 ## Security Features Demonstrated
 
-*   **CSRF Protection:** All POST forms are protected with a CSRF token to prevent cross-site request forgery attacks.
-*   **Input Validation:** All user-submitted data is validated on the server-side for length and content, preventing common injection attacks.
-*   **XSS Prevention:** All output is properly escaped to prevent cross-site scripting (XSS). You can test this by submitting `<script>alert('XSS')</script>` in any form.
-*   **Secure File Uploads:** File uploads are validated, and filenames are sanitized to prevent directory traversal and other file-based attacks.
-*   **Security Headers:** The application sends important security headers like `Content-Security-Policy`, `Strict-Transport-Security`, and `X-Frame-Options`.
-*   **Request Limiting:** The application is configured to limit the maximum request size, parameter depth, and number of parameter keys to prevent denial-of-service attacks.
-*   **Trusted Proxies:** The configuration includes a list of trusted proxy servers, ensuring that `X-Forwarded-*` headers are handled correctly and securely.
+### CSRF Protection
+All POST/PUT/DELETE requests include CSRF tokens in forms:
+```ruby
+<form method="post">
+  <input type="hidden" name="_csrf_token" value="<%= @req.csrf_token %>">
+  <input type="text" name="message">
+</form>
+```
+
+### Input Validation
+Server-side validation of user-submitted data:
+- Length limits (max 1000 chars for messages)
+- Character restrictions (no HTML tags)
+- Required field validation
+- Type validation
+
+### XSS Prevention
+All output is properly escaped:
+```ruby
+@res.body = "<h1>#{ERB::Util.html_escape(user_input)}</h1>"
+```
+
+### Secure File Uploads
+File uploads are validated and sanitized:
+- File type checking (whitelist approach)
+- Size limits (prevent large uploads)
+- Filename sanitization (remove path traversal)
+- Safe storage location
+
+### Security Headers
+Automatic security headers are sent with responses:
+- `Content-Security-Policy` - Prevents inline scripts
+- `Strict-Transport-Security` - Enforces HTTPS
+- `X-Frame-Options` - Prevents clickjacking
+- `X-Content-Type-Options` - Prevents MIME sniffing
+
+### Request Limiting
+Configure limits to prevent DOS attacks:
+- Maximum request size
+- Maximum parameter keys
+- Maximum parameter depth
+
+### Trusted Proxies
+Configure reverse proxy IPs for X-Forwarded-For headers:
+```ruby
+app.add_trusted_proxy('10.0.0.0/8')
+app.add_trusted_proxy(/^192\.168\./)
+```
+
+### Privacy by Default
+Automatic privacy features:
+- Public IP masking (203.0.113.50 → 203.0.113.0)
+- User agent anonymization (versions stripped)
+- Country-level geo-location only
+- Private/localhost IPs NOT masked by default (configurable via `configure_ip_privacy()`)
 
 ## How to Run
 
-1.  Make sure you have `bundler` and `thin` installed:
-    ```sh
-    gem install bundler thin
-    ```
+### Using rackup (recommended)
+
+```sh
+cd examples/security_features
+rackup config.ru -p 10770
+```
+
+### Using thin
+
+```sh
+cd examples/security_features
+thin -e dev -R config.ru -p 10770 start
+```
+
+Open your browser and navigate to `http://localhost:10770`.
+
+## Testing Security Features
+
+### XSS Prevention
+
+Try entering `<script>alert("XSS")</script>` in form fields:
+- The script tag is rendered as text, not executed
+- You'll see it displayed as literal HTML tags
+- Browser's developer tools show escaped HTML
+
+### Input Validation
+
+Test validation rules:
+- Submit a message > 1000 characters (fails)
+- Submit special characters like `<>` (fails)
+- Submit valid text (succeeds)
+
+### CSRF Protection
+
+Examine form submissions:
+- All POST forms include a `_csrf_token` hidden field
+- Each request has a unique token
+- Removing the token causes 403 Forbidden
+- Browser's developer tools show token in form data
+
+### File Uploads
+
+Test file upload security:
+- Try uploading an executable file (rejected)
+- Try uploading a legitimate image (accepted)
+- Check saved filename (sanitized, safe)
+- Verify file permissions and location
+
+### Security Headers
+
+Check response headers:
+- Open browser's Network tab in developer tools
+- Click any response to view headers
+- Look for security headers in response
+- Visit `/headers` endpoint to see all headers
+
+## Expected Output
 
-2.  Install the dependencies from the root of the project:
-    ```sh
-    bundle install
-    ```
+### Successful Form Submission
+```
+POST /feedback HTTP/1.1
+Content-Type: application/x-www-form-urlencoded
 
-3.  Start the server from this directory (`examples/security_features`):
-    ```sh
-    thin -e dev -R config.ru -p 10770 start
-    ```
+_csrf_token=abc123...
+message=Hello+world
 
-4.  Open your browser and navigate to `http://localhost:10770`.
+HTTP/1.1 302 Found
+Location: http://localhost:10770/
+Content-Security-Policy: default-src 'self'
+Strict-Transport-Security: max-age=31536000
+```
 
-## What to Test
+### Failed CSRF Validation
+```
+HTTP/1.1 403 Forbidden
+Content-Type: text/plain
 
-*   **XSS Protection:** Try entering `<script>alert("XSS")</script>` into any of the form fields. You will see that the input is safely displayed as text instead of being executed as a script.
-*   **Input Validation:** Try submitting a very long message in the feedback form to see the length validation in action.
-*   **File Uploads:** Try uploading different types of files. The application will show you how it sanitizes the filename.
-*   **Security Headers:** Open your browser's developer tools and inspect the network requests. You will see the security headers in the response. You can also visit the `/headers` path to see a JSON representation of the request headers your browser is sending.
+CSRF token validation failed
+```
+
+### Failed Input Validation
+```
+HTTP/1.1 400 Bad Request
+Content-Type: text/plain
+
+Message is too long (max 1000 characters)
+```
+
+### Security Headers Response
+```
+Content-Security-Policy: default-src 'self'
+Strict-Transport-Security: max-age=31536000
+X-Frame-Options: SAMEORIGIN
+X-Content-Type-Options: nosniff
+Referrer-Policy: strict-origin-when-cross-origin
+```
 
 ## File Structure
 
-*   `README.md`: This file.
-*   `app.rb`: The main application logic, demonstrating how to handle forms, file uploads, and user input in a secure way.
-*   `config.ru`: The Rack configuration file. This is where the security features are enabled and configured for the Otto application.
-*   `routes`: Defines the URL routes and maps them to methods in the `SecureApp` class.
+- `README.md`: This file
+- `app.rb`: Main application logic with security implementations
+  - Form validation and escaping
+  - File upload handling
+  - Header configuration
+- `config.ru`: Rack configuration with security features enabled
+- `routes`: URL routes mapped to SecureApp class methods
+
+## Key Configuration
+
+In `config.ru`:
+```ruby
+app = Otto.new("./routes")
+
+# Enable security features
+app.enable_csrf_protection!
+
+# Configure request limits
+app.security_config.request_size_limit = 1.megabyte
+app.security_config.max_parameter_keys = 100
+app.security_config.max_parameter_depth = 5
+
+# Add trusted proxies if behind reverse proxy
+app.add_trusted_proxy('10.0.0.0/8')
+
+# Security headers
+app.add_security_header('X-Custom-Header', 'value')
+```
+
+## Common Attack Scenarios
+
+### XSS Attack
+```javascript
+<img src=x onerror="alert('XSS')">
+```
+**Result**: Safely displayed as text, not executed
+
+### SQL Injection
+```sql
+'; DROP TABLE users; --
+```
+**Result**: Stored as literal text, invalid SQL
+
+### Path Traversal
+```
+../../../../../../etc/passwd
+```
+**Result**: Filename sanitized to just `etc-passwd`
+
+### Large Request
+```
+POST with 10MB body
+```
+**Result**: Rejected with 413 Payload Too Large
+
+## Best Practices Demonstrated
+
+1. **Defense in Depth**: Multiple layers of security
+2. **Input Validation**: Whitelist approach (allow only safe input)
+3. **Output Escaping**: Escape all user-controlled output
+4. **CSRF Tokens**: Unique tokens for each request
+5. **Security Headers**: Prevent common attack vectors
+6. **File Upload Safety**: Validate type and sanitize names
+7. **Request Limiting**: Prevent denial-of-service
+
+## Testing with curl
+
+```sh
+# Test CSRF protection (will fail without token)
+curl -X POST http://localhost:10770/feedback \
+  -d "message=test"
+
+# Test with valid CSRF token (get token from form first)
+curl -X POST http://localhost:10770/feedback \
+  -d "_csrf_token=<token>" \
+  -d "message=test"
+
+# Test input validation
+curl -X POST http://localhost:10770/feedback \
+  -d "message=$(python -c 'print(\"x\" * 2000)')"
+```
+
+## Next Steps
+
+- Review the application code to see implementation details
+- Explore other examples for different features
+
+## Further Reading
+
+- [CLAUDE.md](../../CLAUDE.md#security-features) - Security configuration reference
+- [IP Privacy](../../CLAUDE.md#ip-privacy-privacy-by-default) - Privacy configuration

data/examples/simple_geo_resolver.rb

@@ -0,0 +1,107 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Otto GeoResolver Extension Guide
+#
+# This guide shows two approaches to extend Otto's IP geolocation:
+# 1. Configuration-based (simple, inline)
+# 2. Subclass-based (full control)
+
+require 'bundler/setup'
+require 'otto'
+
+# =============================================================================
+# Quick Start: Configuration-based Extension
+# =============================================================================
+
+puts 'Simple Custom Geo Resolution'
+puts '-' * 40
+
+# Step 1: Define your resolver function
+custom_resolver = lambda do |ip, _env|
+  case ip
+  when '1.2.3.4' then 'US'
+  when '5.6.7.8' then 'GB'
+  else nil # nil = use Otto's built-in resolver
+  end
+end
+
+# Step 2: Set it globally
+Otto::Privacy::GeoResolver.custom_resolver = custom_resolver
+
+# Step 3: Test it
+puts "1.2.3.4 -> #{Otto::Privacy::GeoResolver.resolve('1.2.3.4', {})}"
+puts "8.8.8.8 -> #{Otto::Privacy::GeoResolver.resolve('8.8.8.8', {})} (fallback)"
+
+# Reset for next example
+Otto::Privacy::GeoResolver.custom_resolver = nil
+
+# =============================================================================
+# Real-World Example: API Integration with Caching
+# =============================================================================
+
+puts "\nAPI Integration with Caching"
+puts '-' * 40
+
+class CachedGeoAPI
+  def initialize(api_key)
+    @api_key = api_key
+    @cache = {}
+  end
+
+  def call(ip, _env)
+    # Return cached result if available
+    return @cache[ip] if @cache.key?(ip)
+
+    # Simulate API call (replace with real HTTP request)
+    country = mock_api_call(ip)
+
+    # Cache the result
+    @cache[ip] = country
+    country
+  rescue StandardError => e
+    puts "API failed: #{e.message}"
+    nil # Fallback to Otto's resolver
+  end
+
+  private
+
+  def mock_api_call(ip)
+    # Replace this with: HTTP.get("https://api.example.com/geo?ip=#{ip}")
+    case ip
+    when /^1\./ then 'US'
+    when /^2\./ then 'GB'
+    end
+  end
+end
+
+# Use the cached API resolver
+api_resolver = CachedGeoAPI.new('your_api_key')
+Otto::Privacy::GeoResolver.custom_resolver = api_resolver
+
+puts "1.2.3.4 -> #{Otto::Privacy::GeoResolver.resolve('1.2.3.4', {})}"
+puts "1.2.3.4 -> #{Otto::Privacy::GeoResolver.resolve('1.2.3.4', {})} (cached)"
+
+Otto::Privacy::GeoResolver.custom_resolver = nil
+
+# =============================================================================
+# Performance Tips
+# =============================================================================
+
+puts "\nPerformance Tips"
+puts '-' * 40
+
+puts '• Cache API results to avoid repeated calls'
+puts "• Return nil from custom resolver to use Otto's fast fallback"
+puts '• Use CloudFlare headers when available (fastest)'
+puts '• Consider async/background geo updates for heavy traffic'
+
+puts "\nProduction Pattern: Valkey/Redis Bloom Filters"
+puts '-' * 40
+puts 'For high-traffic applications, consider Bloom/Cuckoo filters:'
+puts '• Store RIR IP prefixes in Valkey/Redis Bloom filter per country'
+puts '• Memory: ~1MB for entire IPv4 table at 1% false positive rate'
+puts '• Lookup: O(1) microsecond-level performance via BF.EXISTS'
+puts '• Zero external dependencies, rebuild nightly from public RIR files'
+puts '• Perfect for CDN header fallback when requests bypass edge'
+puts ''