itsi-server 0.1.1 → 0.2.2

data/lib/itsi/server/config/middleware/rate_limit.md

@@ -0,0 +1,126 @@
+---
+title: Rate Limiter
+url: /middleware/rate_limit
+---
+
+The **Rate Limiter** middleware enforces a fixed‑window rate limit on incoming requests. You configure a maximum number of requests allowed in a given time span, along with how to identify the client and where to store counters (in‑memory or Redis).
+
+By default it limits per IP address using an in‑memory store, but you can:
+
+- Change the window size (`requests` per `seconds`)
+- Limit based on other attributes (header or query parameter)
+- Swap to a Redis backend for cross‑process rate limiting
+- Customize the error response when the limit is exceeded
+
+## Configuration
+
+```ruby
+rate_limit \
+  requests: 100,                   # max 100 requests
+  seconds: 60,                     # per 60‑second window
+  key: "address",                  # limit by client IP
+  store_config: "in_memory",       # use local in‑memory store
+  error_response: "too_many_requests"
+```
+
+### `requests` / `seconds`
+
+- **`requests`**: Number of allowed requests in each window (positive integer).
+- **`seconds`**: Length of each fixed window in seconds (positive integer).
+
+### `key`
+
+How to identify the client:
+
+- **`"address"`** (default): use the client’s socket IP.
+- **Header or query parameter**:
+  ```ruby
+  key: { parameter: { header: { name: "X-Api-Key-Id" } } }
+  ```
+  or
+  ```ruby
+  key: { parameter: { query:  { name: "user_id" } } }
+  ```
+
+### `store_config`
+
+Where to keep counters:
+
+- **`"in_memory"`** (default): per‑process, reset when server restarts.
+- **Redis** (shared across workers):
+  ```ruby
+  store_config: { redis: { connection_url: "redis://localhost:6379/1" } }
+  ```
+
+### `error_response`
+
+Customizes the response when the limit is reached (default is built‑in `too_many_requests`):
+
+```ruby
+error_response: {
+  code: 429,
+  plaintext: { inline: "Rate limit exceeded" },
+  default:   "plaintext"
+}
+```
+
+## Rate Limit Response Headers
+
+When a request **exceeds** the allowed rate, the middleware returns your configured error response **plus** these headers:
+
+| Header                    | Meaning                                                                                  |
+|---------------------------|------------------------------------------------------------------------------------------|
+| **X-RateLimit-Limit**       | The maximum number of requests allowed per window (`requests` value).                    |
+| **X-RateLimit-Remaining**   | How many requests remain in the current window (will be `0` once the limit is hit).      |
+| **X-RateLimit-Reset**       | Seconds until the current window resets (time until your counter zeroes out).            |
+| **Retry-After**             | Same value as `X-RateLimit-Reset` — suggests when clients should retry their request.    |
+
+> **Example**
+> With `requests: 5, seconds: 60`, after 5 calls the 6th returns:
+> ```
+> 429 Too Many Requests
+> X-RateLimit-Limit:     5
+> X-RateLimit-Remaining: 0
+> X-RateLimit-Reset:     42
+> Retry-After:           42
+> ```
+
+## How It Works
+
+1. **On each request**
+   - Compute the client key (IP, header, or query).
+   - Increment a counter for the current time window.
+
+2. **Fixed‑window logic**
+   - If the counter ≤ `requests`, allow through.
+   - Otherwise, immediately return the configured `error_response` plus the **X-RateLimit** headers.
+
+3. **Store options**
+   - **In‑memory**: simple hash, fast but not shared across processes.
+   - **Redis**: atomic `INCR` + `EXPIRE` commands, shared across all workers.
+
+Place `rate_limit` anywhere in your routing DSL to apply it to all downstream handlers in that scope.
+
+
+## Trusted Proxies
+
+By default, a rate limiter middleware uses the IP address from the underlying socket (remote_addr). However, if your server is behind a reverse proxy, all requests will appear to come from the proxy’s IP address. This can break IP-based rules or cause rate-limiting to group all users together.
+
+To address this, you can declare trusted proxies and instruct the server to extract the original client IP from forwarded headers only if the request came from one of these proxies.
+
+
+### Configuring trusted_proxies
+
+To trust one or more upstream proxies, provide a trusted_proxies map in the middleware configuration.
+E.g.
+```ruby {filename=Itsi.rb}
+rate_limit \
+  requests: 100,                   # max 100 requests
+  seconds: 60,                     # per 60‑second window
+  key: "address",                  # limit by client IP
+  store_config: "in_memory",       # use local in‑memory store
+  error_response: "too_many_requests",
+  trusted_proxies: {
+    "192.168.1.1" => { header: { name: "X-Forwarded-For" } }
+  }
+```

data/lib/itsi/server/config/middleware/rate_limit.rb

@@ -0,0 +1,34 @@
+module Itsi
+  class Server
+    module Config
+      class RateLimit < Middleware
+        require_relative "rate_limit_store"
+        require_relative "token_source"
+
+        insert_text <<~SNIPPET
+        rate_limit \\
+          requests: ${1|1,5,100|},
+          seconds: ${2|1,5,100|},
+          key: ${3|"address",{parameter:{header:{name:"X-Forwarded-For"}}}|},
+          store_config: ${4|"in_memory",{redis:{connection_url: "redis://localhost:6379"}}|},
+          error_response: ${5|"too_many_requests", { code: 429\\, default_format: "html"\\, html: { inline: "<h1>Unauthorized</h1>" } }|}
+        SNIPPET
+
+        detail "Automatically limits the number of requests a client can make within a given time period."
+
+
+        schema do
+          {
+            requests: Required() & Type(Integer) & Range(1..2**32),
+            seconds: Required() & Type(Integer) & Range(1..2**32),
+            key: (Required() & Or(Enum(["address"]), Type(RateLimitKey))).default("address"),
+            store_config: (Required() & Or(Enum(["in_memory"]), Type(RateLimitStore))).default("in_memory"),
+            error_response: Type(ErrorResponseDef).default("too_many_requests"),
+            trusted_proxies: (Hash(Type(String), Type(TokenSource)) & Required()).default({})
+          }
+        end
+
+      end
+    end
+  end
+end

data/lib/itsi/server/config/middleware/rate_limit_store.rb

@@ -0,0 +1,25 @@
+module Itsi
+  class Server
+    module Config
+
+      RateLimitKey = TypedStruct.new do
+        {
+          parameter: Or(
+            Hash(Enum(["header"]), Hash(Enum(["name"]), Type(String))) & Required(),
+            Hash(Enum(["query"]), Type(String)) & Required()
+          )
+        }
+      end
+
+      RateLimitStore = TypedStruct.new do
+        {
+          redis: Type(TypedStruct.new do
+            {
+              connection_url: Type(String)& Required()
+            }
+          end) & Required()
+        }
+      end
+    end
+  end
+end

data/lib/itsi/server/config/middleware/redirect.md

@@ -0,0 +1,55 @@
+---
+title: Redirect
+url: /middleware/redirect
+---
+
+The Redirect middleware enables automatic redirection of incoming requests to a different URL. It computes the target URL using a string rewrite rule and responds with an HTTP redirect status code based on the configuration. Once the middleware processes a request, it immediately returns a redirection response without further processing.
+
+## Redirect configuration
+
+```ruby
+redirect \
+  to: "https://example.com/new-path", \
+  type: "permanent"
+```
+
+## Redirect Applied to a sub-location
+
+```ruby
+location "/old-path" do
+  redirect \
+    to: "https://example.com/new-path", \
+    type: "temporary"
+  # No further route processing occurs for /old-path.
+end
+```
+## Redirect HTTP to HTTPS
+
+```ruby
+location protocols: [:http] do
+  redirect \
+    to: "https://{host}{path_and_query}", \
+    type: "moved_permanently"
+end
+```
+> A shorthand for the above exists, call `redirect_http_to_https!`. Note that this is
+an alias for the above, and as such is subject to ordinary [location](/middleware/location) resolution rules.
+To make sure this rule takes precedence, place it above other locations in the `Itsi.rb` file.
+
+
+
+## Configuration Options
+
+- **to**:
+  A [string rewrite rule](/middleware/string_rewrites) (or literal string) specifying the target URL for the redirection. This value can incorporate dynamic portions of the incoming request.
+
+- **type**:
+  Specifies the type of redirection. Allowed values include:
+  - `"permanent"`: Responds with a 308 Permanent Redirect.
+  - `"temporary"`: Responds with a 307 Temporary Redirect.
+  - `"moved_permanently"`: Responds with a 301 Moved Permanently.
+  - `"found"`: Responds with a 302 Found.
+
+## How It Works
+
+When a request is received, the Redirect middleware immediately intercepts it and builds a redirect response. The target URL is computed via the provided rewrite rule (`to`), and the response is sent with the appropriate HTTP status code as determined by the `type` option. No further handling is performed on the request.

data/lib/itsi/server/config/middleware/redirect.rb

@@ -0,0 +1,25 @@
+module Itsi
+  class Server
+    module Config
+      class Redirect < Middleware
+
+        insert_text <<~SNIPPET
+        redirect \\
+          to: "${1:https://example.com/new-path}",
+          type: ${2|"permanent","temporary","found","moved_permanently"|}
+        SNIPPET
+
+        detail "Automatically redirects incoming requests to a new URL based on a string rewrite rule."
+
+        Redirect = TypedStruct.new do
+          {
+            to: (Required() & Type(String)),
+            type: Enum(["permanent", "temporary", "found", "moved_permanently"]).default("moved_permanently")
+          }
+        end
+
+        schema Redirect
+      end
+    end
+  end
+end

data/lib/itsi/server/config/middleware/request_headers.md

@@ -0,0 +1,34 @@
+---
+title: Request Headers
+url: /middleware/request_headers
+---
+
+The **Request Headers** middleware lets you add, override or remove HTTP headers **before** your application handler sees the request. You can inject static values or use the full power of [String Rewrite](/middleware/string_rewrite) to generate header values based on path, query, method, etc.
+
+## Configuration
+```ruby {filename=Itsi.rb}
+request_headers \
+  additions: {
+    "X-Correlation-ID" => ["{request_id_full}"],
+    "X-Env"            => ["production"]
+  },
+  removals: ["X-Forwarded-For"]
+```
+
+- **additions** (`Hash(String,Array(StringRewrite))`)
+  A map of header names to an array of StringRewrite templates. Each template is rendered **after** removals.
+- **removals** `Array(String)`
+  A list of header names to delete outright. If you want to override an existing header, include it in **both** removals and additions.
+
+## Examples
+```ruby {filename=Itsi.rb}
+# Add a per‑request UUID and drop any X‑Forwarded‑For header
+request_headers \
+  additions: { "X-Correlation-ID" => ["{request_id}"] },
+  removals:  ["X-Forwarded-For"]
+
+# Inject the full request path and query
+request_headers \
+  additions: { "X-Full-URL" => ["{path_and_query}"] },
+  removals:  []
+```

data/lib/itsi/server/config/middleware/request_headers.rb

@@ -0,0 +1,24 @@
+module Itsi
+  class Server
+    module Config
+      class RequestHeaders < Middleware
+
+        insert_text <<~SNIPPET
+        request_headers \\
+          additions: { ${1:"X-Correlation-ID" => ["{request_id_full\\}"]} },
+          removals:  [${2:"X-Forwarded-For"}]
+        SNIPPET
+
+        detail "Add, override or remove request headers before reaching your handler."
+
+        schema do
+          {
+            additions: Hash(Type(String), Array(Type(String))).default({}),
+            removals: Array(Type(String)).default([]),
+            combine: Bool().default(true)
+          }
+        end
+      end
+    end
+  end
+end

data/lib/itsi/server/config/middleware/response_headers.md

@@ -0,0 +1,33 @@
+---
+title: Response Headers
+url: /middleware/response_headers
+---
+
+The **Response Headers** middleware lets you add, override or remove HTTP headers **after** your application handler has produced a response. You can attach static values or dynamically compute them via [String Rewrite](/middleware/string_rewrite).
+
+## Configuration
+```ruby {filename=Itsi.rb}
+response_headers \
+  additions: {
+    "X-Processed-By"        => ["Itsi"],
+    "X-Response-Time-Millis" => ["{response_time}"]
+  },
+  removals: ["Server"]
+```
+- **additions** `Hash(String,Array(StringRewrite))`
+  A map of header names to StringRewrite templates. Templates are evaluated against the final response and context in **after**.
+- **removals** `Array(String)`
+  A list of header names to delete before adding new ones.
+
+## Examples
+```ruby {filename=Itsi.rb}
+# Stamp every response with our server name
+response_headers \
+  additions: { "X-Powered-By" => ["Itsi"] },
+  removals:  []
+
+# Remove the Server header and add timing info
+response_headers \
+  additions: { "X-Response-Time" => ["{response_time}"] },
+  removals:  ["Server"]
+```

data/lib/itsi/server/config/middleware/response_headers.rb

@@ -0,0 +1,25 @@
+module Itsi
+  class Server
+    module Config
+      class ResponseHeaders < Middleware
+
+        insert_text <<~SNIPPET
+          response_headers \\
+            additions: { ${1:"X-Powered-By" => ["Itsi"]} },
+            removals:  [${2:"Server"}]
+          SNIPPET
+
+
+        detail "Add, override or remove response headers before sending to the client."
+
+        schema do
+          {
+            additions: Hash(Type(String), Array(Type(String))).default({}),
+            removals: Array(Type(String)).default([]),
+            combine: Bool().default(true)
+          }
+        end
+      end
+    end
+  end
+end

data/lib/itsi/server/config/middleware/run.md

@@ -0,0 +1,60 @@
+---
+title: Run
+url: /middleware/run
+---
+
+You can use the `run` middleware to run an inline rack application.
+The alternative way to mount a rack application is to use a [`rackup_file`](/middleware/rackup_file) middleware.
+
+You can mount several rack applications  to run simultaneously within a single Itsi application, using different location blocks.
+Depending on *where* you mount the app, the application will receive different values for `PATH_INFO`, `SCRIPT_NAME`.
+
+## Configuration
+
+### Simple inline Rack app.
+```ruby {filename=Itsi.rb}
+run ->(env){ [200, { 'content-type' => 'text/plain' }, ['OK']] }
+```
+
+### Rack app using Rack::Builder
+```ruby {filename=Itsi.rb}
+run(Rack::Builder.app do
+  use Rack::CommonLogger
+  run ->(env) { [200, { 'content-type' => 'text/plain' }, ['OK']] }
+end)
+```
+
+
+### Rack app mounted at a subpath
+```ruby {filename=Itsi.rb}
+require 'rack'
+location "/subpath/*" do
+  run(Rack::Builder.app do
+    use Rack::CommonLogger
+    run ->(env) { [200, { 'content-type' => 'text/plain' }, [[env['SCRIPT_NAME'], env['PATH_INFO']].join(":") ]  ] }
+  end)
+end
+
+run(Rack::Builder.app do
+  use Rack::CommonLogger
+  run ->(env) { [200, { 'content-type' => 'text/plain' }, [[env['SCRIPT_NAME'], env['PATH_INFO']].join(":") ]  ] }
+end)
+
+```
+
+```bash
+$ curl http://0.0.0.0:3000/subpath/child_path
+/subpath:/child_path
+
+$ curl http://0.0.0.0:3000/root/child_path
+:/root/child_path
+```
+
+### Options
+* `nonblocking` (default false). Determines whether requests sent to this Rack application should be run on non-blocking threads. Only applies if running in hybrid (non-blocking and blocking thread pool) mode. Otherwise this is a no-op and will run in whatever mode is set globally.
+* `sendfile` (default true). Determines whether Itsi should respect the `X-Sendfile` header set by the Rack application and use the `sendfile` function to efficiently send files. (Despite the name, this does not use the OS-level `sendfile` system call). Note. Itsi enforces the restriction that the referenced file must be within a child directory of the application root.
+
+e.g.
+```ruby {filename=Itsi.rb}
+run ->(env){ [200, { 'content-type' => 'text/plain' }, ['OK']] }, nonblocking: true, sendfile: false
+```

data/lib/itsi/server/config/middleware/run.rb

@@ -0,0 +1,43 @@
+module Itsi
+  class Server
+    module Config
+      class Run < Middleware
+
+        insert_text <<~SNIPPET
+        run \\
+          ${1:->(env){[200, {"Content-Type" => "text/plain"\\}, ["Hello, World!"]]}},
+          nonblocking: ${2|true,false|},
+          sendfile: ${3|true,false|}
+        SNIPPET
+
+        detail "Define an inline Rack Application"
+
+        schema do
+          {
+            nonblocking: Bool().default(false),
+            sendfile: Bool().default(true)
+          }
+        end
+
+        def initialize(location, app, params={})
+          super(location, params)
+          raise "App must be a Rack application" unless app.respond_to?(:call)
+          @app = app
+        end
+
+        def build!
+          app_args = {
+            preloader: -> { Itsi::Server::RackInterface.for(@app) },
+            sendfile: @params[:sendfile],
+            nonblocking: @params[:nonblocking],
+            base_path: "^(?<base_path>#{location.paths_from_parent.gsub(/\.\*\)$/, ')')}).*$"
+          }
+          location.middleware[:app] = app_args
+          location.location("*") do
+            @middleware[:app] = app_args
+          end
+        end
+      end
+    end
+  end
+end

data/lib/itsi/server/config/middleware/static_assets.md

@@ -0,0 +1,73 @@
+---
+title: Static Assets
+url: /middleware/static_assets
+---
+
+The Static Assets middleware serves files from a specified root directory. It is capable of optimized delivery of static content such as HTML, CSS, JavaScript, images, as well as large assets, such as video files using streaming bodies and range requests.
+It can auto-index directories for simple directory listings.
+
+## Key Features
+- **Auto Indexing**: Optionally generate directory listings when an index file is missing.
+- **HTML Fallback**: When enabled, the middleware attempts to serve a file with a `.html` extension if the requested file is not found.
+- **In-Memory Caching**: Files under a certain size (and up to a configurable count) are cached in memory for performance.
+- **Custom Headers**: User-supplied headers (e.g., for caching) can be added to responses.
+- **Relative Path Processing**: When enabled, the middleware rewrites request paths relative to a configured base path.
+- **Partial Content**: Supports Range requests for serving partial file content.
+
+## Example
+
+```ruby {filename=Itsi.rb}
+static_assets root_dir: "./"
+```
+
+## Configuration Options
+
+
+| Option                         | Description                                                                                                                                                                                                                           |
+|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **`root_dir`** (Default ".")                | The relative or absolute disk path where static files reside.                                                                                                                                                                                   |
+| **`not_found_behavior`** (Default `{error: "not_found"}`)       | Determines what to do when a file isn’t found. See [Not Found Behavior](#not-found-behavior) below. |
+| **`auto_index`** (Default `false`)            | If <code>true</code>, directory listings are automatically generated when no index file is found. Defaults to <code>false</code>.                                                                                                    |
+| **`try_html_extension`** (Default `true`)      | If <code>true</code>, when the requested file isn’t found, the middleware attempts to serve the same path with a <code>.html</code> extension. Defaults to <code>true</code>.                                                 |
+| **`max_file_size_in_memory`** (Default `1048576`)  | Maximum file size (in bytes) for caching files in memory. Files larger than this will be served from disk. Defaults to <code>1048576</code> (1 MB).                                                                                |
+| **`max_files_in_memory`** (Default `100`)      | Maximum number of files to cache in memory. Defaults to <code>100</code>.                                                                                                                                                             |
+| **`file_check_interval`** (Default `1`)      | Max time (in seconds) a file stays in cache before the file-system is checked for modifications. Defaults to <code>1</code>.                                                                                                                                                |
+| **`headers`** (Default `nil`)                 | A hash of additional headers to include in responses (e.g. <code>{"Cache-Control" => "max-age=3600"}</code>).                                                                                                                         |
+| **`allowed_extensions`** (Default `[]`)       | An array of permitted file extensions (e.g. <code>["html", "css", "js", "png", "jpg"]</code>). If a requested file’s extension isn’t in this list, it won’t be served. An empty list (default) means all extensions are allowed.                                                           |
+| **`relative_path`** (Default `true`)           | If <code>true</code>, the effective file lookup path is computed relative to the base path at which the middleware is mounted (See [location](/middleware/location) to understand mounting options). Defaults to <code>true</code>.                                                                                               |
+| **`serve_hidden_files`** (Default `false`)       | If <code>false</code>, files whose names start with a dot (".") are not served. Defaults to <code>false</code>.                                                                                                                        |
+
+
+## Not Found Behavior
+The static assets middleware supports many configuration options when a file is not found.
+* **`fallthrough`**: The request is *not* handled by this middleware and is instead served by the next middleware in the chain. The option is given as a string. E.g.
+```ruby
+error_response: "fallthrough"
+```
+* **`index`**: Unsatisfiable requests instead result in serving an index file. (Useful for SPAs with client-side routing).
+```ruby
+error_response: {index: "./path/to/index.html" }
+```
+* **`redirect`**: Unsatisfiable requests are redirected. Options for type are the same as defined for the [redirect](/middleware/redirect) middleware.
+```ruby
+error_response: {redirect: { to: "http://example.com/redirect_path", type: "permanent" }}
+```
+* **`error`**: Provide an overridden error response. See [`error_response`](/middleware/error_response) for details.
+e.g.
+```ruby
+error_response: {error: "not_found"}
+```
+
+```ruby
+error_response: \
+  {
+    error:
+    {
+      code: 404,
+      plaintext: {inline: "File not found"},
+      json: {inline: "{\"message\": \"File not found\"}"},
+      html: {file: "./path/to/not_found.html"},
+      default: "plaintext"
+    }
+  }
+```

data/lib/itsi/server/config/middleware/static_assets.rb

@@ -0,0 +1,87 @@
+module Itsi
+  class Server
+    module Config
+      class StaticAssets < Middleware
+
+        insert_text <<~SNIPPET
+          static_assets \\
+            root_dir: "${1|./,/var/www,|}",
+            not_found_behavior: ${2|"fallthrough",{index: "index.html"},{error: "not_found"}|},
+            auto_index: ${3|true,false|},
+            try_html_extension: ${4|true,false|},
+            max_file_size_in_memory: ${5|1048576,2097152|},
+            max_files_in_memory: ${6|100,500,1000|},
+            file_check_interval: ${7|1,60,120|},
+            headers: { ${8|,"Cache-Control" => "max-age=3600",|} },
+            allowed_extensions: ${9|%w[html css js png jpg],[]|},
+            relative_path: ${10|true,false|},
+            serve_hidden_files: ${11|true,false|}
+        SNIPPET
+
+        detail "Serves static files from a designated directory with options for auto indexing, in-memory caching, and custom header support. Supports relative path rewriting and file range requests."
+
+        ErrorResponse = TypedStruct.new do
+          {
+            error: Type(ErrorResponseDef) & Required()
+          }
+        end
+
+        IndexResponse = TypedStruct.new do
+          {
+            index: Type(String) & Required()
+          }
+        end
+
+        RedirectResponse = TypedStruct.new do
+          {
+            redirect: Type(Redirect::Redirect) & Required()
+          }
+        end
+
+        schema do
+          {
+            root_dir: (Type(String) & Required()).default("./"),
+            not_found_behavior: Or(
+              Enum(["fallthrough", "index", "redirect", "internal_server_error"]),
+              Type(IndexResponse),
+              Type(RedirectResponse),
+              Type(ErrorResponse)
+            ).default({error: "not_found"}),
+            allowed_extensions: (Array(Type(String)) & Required()).default([]),
+            auto_index: Bool().default(false),
+            try_html_extension: Bool().default(true),
+            max_file_size_in_memory: Type(Integer).default(1048576),
+            max_files_in_memory: Type(Integer).default(100),
+            file_check_interval: Type(Integer).default(1),
+            headers: Hash(Type(String), Type(String)).default({}),
+            relative_path: Bool().default(true),
+            serve_hidden_files: Bool().default(false)
+          }
+        end
+
+        def build!
+          root_dir = @params[:root_dir] || "."
+
+          if !File.exist?(root_dir)
+            warn "Warning: static_assets root_dir '#{root_dir}' does not exist!"
+          elsif !File.directory?(root_dir)
+            warn "Warning: static_assets root_dir '#{root_dir}' is not a directory!"
+          end
+
+          @params[:relative_path] = true unless @params.key?(:relative_path)
+          @params[:allowed_extensions] ||= []
+
+          if @params[:try_html_extension] && @params[:allowed_extensions].include?("html")
+            @params[:allowed_extensions] << ""
+          end
+
+          @params[:base_path] = "^(?<base_path>#{location.paths_from_parent}).*$"
+          params = @params
+          location.location("*", extensions: @params[:allowed_extensions]) do
+            @middleware[:static_assets] = params
+          end
+        end
+      end
+    end
+  end
+end