ruvoy 0.2.1-aarch64-linux → 0.4.0-aarch64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a585700c8fa8895e0202a89938ba64268f56cd01936857684f276b3904ec408a
-  data.tar.gz: aac2b3e316c6ed0ef47c0c205ce232cc6164e0102cbfd4a8ca0761e885faa72a
+  metadata.gz: 569b932c2661eaffd11fde7921044f4eadeb6e79dc4c951ed293c664d7b778bc
+  data.tar.gz: bcbe0a8668f8d396d9b64779077c23d914a7de5678bd56edc3e197bcf9012f0d
 SHA512:
-  metadata.gz: 1cd84c20274deae0cb6105f37d5872c590de9a888ade90715e84be42a72f3a4c4e270dd5a3a412621845fa195c7cd15fedf6237a7fd0c7153c3bbd1e10b85b42
-  data.tar.gz: a3feba17ee4f6253718e95d1902d5edad010899228b4783e97c3e344ee736360edb9ecddef811bfeecfb534a32d7b1e74b4cfb8050ad8d50209d67ba6362484d
+  metadata.gz: faabfaa8fdfe5dcc4512613a340514ad68856535b78e419b9451a5afaa0fecd6b76e4e6f0b1e0e9712620c3f7e6c255eb5e068ffbc79b597f7ef2e8181534ad5
+  data.tar.gz: 9f3695b1f964180948a46529a3168942856e545486762f90b55361c64d1fb42e07172c613833a61cf64aa53150531988bb8ea67e8a3c2bc7c66a8c5d559eb75e

data/README.md

@@ -53,17 +53,20 @@ Usage: ruvoy [options] [config.ru]
         --print-config               Print the generated Envoy configuration and exit
 ```
 
-To run it under an Envoy you configure yourself, `--print-config` prints the
-listener Ruvoy would have used; the module is `ruvoy_fiber` and its filter
-config is the path to the rackup, or a JSON object naming it (see
+To run it under an Envoy you configure yourself, `--print-config` prints what
+Ruvoy would have used: the `ruvoy` bootstrap extension, which names the rackup
+(see [Lifecycle](#lifecycle)), and a listener whose rack filter joins it. The
+filter's own config is an optional JSON object (see
 [Envoy context and upstream calls](#envoy-context-and-upstream-calls)).
 
 ## Status
 
 The data path is implemented and covered by suites that run against a real
 Envoy in CI. Configuration reload has been driven through a running Envoy, and
-a one-hour soak of 1.8 million requests left the heap, file descriptors and
-threads flat. Interfaces may change.
+a three-hour soak of 5.7 million requests, 22,800 WebSocket sessions and 359
+listener updates left live memory, file descriptors, threads and p99 latency
+flat; the resident set still rose about 3% an hour, held by the allocator and
+Ruby's heap rather than by anything live. Interfaces may change.
 
 ## Features
 
@@ -141,6 +144,10 @@ kept, in memory up to 128 KiB and in a temporary file beyond that, so
 Rack versions expect. Raw socket hijacking
 (`rack.hijack`) is not supported; the connection always belongs to Envoy.
 
+WebSocket and other upgrades use Rack 3's `rack.protocol` with a streaming
+body instead, over HTTP/1.1 and HTTP/2, once Envoy allows them
+(`upgrade_configs`, plus `allow_connect` for HTTP/2).
+
 A response body that fails after its headers are already on the wire resets
 the stream, so the client sees an error rather than a short body that looks
 complete.
@@ -209,7 +216,6 @@ JSON object:
 
 ```json
 {
-  "rackup": "/srv/app/config.ru",
   "context": { "metadata_namespaces": ["acme.tenant"] },
   "upstream": { "clusters": ["users"], "timeout_ms": 15000, "max_response_bytes": 1048576 }
 }
@@ -246,7 +252,8 @@ What Envoy's module interface can report sets some limits:
 
 ### `ruvoy.upstream`
 
-Calls a cluster Envoy was configured with and returns a Rack-style triple:
+Calls a cluster Envoy was configured with and returns the status, the headers
+and the whole body as a String:
 
 ```ruby
 status, headers, body = env["ruvoy.upstream"].call(
@@ -276,32 +283,85 @@ one request overlap. A repeated response header arrives as an Array.
 - `call` buffers the response whole, so `max_response_bytes` bounds what one
   call can hold in memory.
 
-`stream` takes the same arguments and returns once the response head is in.
-The body yields the response as it arrives and can be returned as the Rack
-response, so a proxy is one line:
+`stream` takes the same arguments but returns once the head is in, with a
+body that streams and can be returned as the Rack response, so a proxy is one
+line:
 
 ```ruby
 run ->(env) { env["ruvoy.upstream"].stream("api", env["REQUEST_METHOD"], env["PATH_INFO"], body: env["rack.input"]) }
 ```
 
-- The request body may be a String or anything that responds to `read`, such
-  as `rack.input`. One that ends within its first 64 KiB goes with its length,
-  an empty one as none, and a longer one chunked while the response comes down.
-- `idle_timeout:` bounds the wait for the head and between chunks, `timeout_ms`
-  by default, and raises `Ruvoy::UpstreamTimeout`. `timeout:` bounds the whole
-  exchange and is unset by default: Envoy answers 504 before the head, and
-  resets the stream after it.
-- Envoy applies no flow control to a stream a module starts, so at most
-  `upstream.stream_buffer_bytes` (default 1 MiB) of the response waits for a
-  slow reader. Past that the stream is reset and the reader gets
-  `Ruvoy::UpstreamOverflow`. The request body is held to the same budget until
-  Envoy takes it; what Envoy buffers after that is not bounded by Ruvoy.
-- Closing the body before its end, or finishing the request, resets the
-  stream. `body.trailers` holds the upstream's trailers once the body has ended.
+Envoy applies no flow control to it, so a reader that falls more than
+`upstream.stream_buffer_bytes` (1 MiB by default) behind has the stream reset.
+With `protocol: "websocket"`, the response upgrades the client too and relays
+both ways.
+
+Calls and streams carry the request's trace context on: `traceparent`,
+`tracestate`, `x-request-id` and the B3 headers, unless the call sets them.
 
 [`examples/extensions`](examples/extensions) runs both against a stand-in
 service, including two calls from one request overlapping.
 
+## Lifecycle
+
+The `ruvoy` bootstrap extension owns the Ruby runtime: it names the rackup and
+the terms the runtime serves it on, and every listener's rack filter joins it.
+A rack filter configured without it is refused.
+
+```yaml
+bootstrap_extensions:
+  - name: ruvoy
+    typed_config:
+      "@type": type.googleapis.com/envoy.extensions.bootstrap.dynamic_modules.v3.DynamicModuleBootstrapExtension
+      dynamic_module_config: { name: ruvoy_fiber, do_not_close: true }
+      extension_name: ruvoy
+      extension_config:
+        "@type": type.googleapis.com/google.protobuf.StringValue
+        value: '{"rackup": "/srv/app/config.ru"}'
+```
+
+| Setting | Default | |
+|---|---|---|
+| `rackup` | | The application. Required. |
+| `max_inflight_requests` | 1024 | Requests in Ruby at once; one more is refused with 503. |
+| `shutdown_timeout_ms` | 10000 | How long shutdown waits for fibers still running. |
+| `diagnostics` | false | Adds the `x-ruvoy-*` diagnostic headers to responses. |
+
+- Envoy becomes ready only once the application has loaded and the hooks
+  registered with `Ruvoy::Lifecycle.before_ready { }` have run. An application
+  that fails to load stops Envoy.
+- `Ruvoy::Lifecycle.on_drain { }` runs when Envoy begins draining, while
+  requests are still served: close persistent connections, stop background
+  work.
+- Envoy's shutdown stops the runtime: fibers still running are cancelled and
+  given `shutdown_timeout_ms` to finish.
+- The admin interface reaches Ruby: `/ruvoy` (overview), `/ruvoy/gc` (`GET`
+  for `GC.stat`, `POST` to collect, `?compact=1` to compact too),
+  `/ruvoy/fibers` (live fibers with backtraces) and `/ruvoy/config`. Ruby
+  answers from its own thread; what it does not answer within two seconds gets
+  504.
+- `Ruvoy::Lifecycle.settings` holds the settings above.
+- Envoy's allocator keeps freed memory unless its `memory_allocator_manager`
+  sets `bytes_to_release`; without it the resident set only grows. The
+  generated configuration returns 1 MiB a second, and a hand-written one
+  should set it too.
+
+## Statistics
+
+Ruvoy reports through Envoy's statistics, so the numbers reach whatever sink
+Envoy has, the admin interface's `/stats/prometheus` among them.
+
+- `requests_total`, `rejected_total{reason}`, `responses_total{outcome}` and
+  `duration_ms`: requests, and how those that reached the application ended.
+- `inflight_requests`, `reactor_idle_ms` and `resident_bytes`: the bootstrap
+  extension reads them once a second on the main thread, so the request path
+  never publishes anything and an idle proxy reports an idle runtime.
+- `upstream_calls_total{cluster, outcome}`: calls by how the application saw
+  them end, `completed`, `refused`, `timed_out`, `overflow`, `too_large`,
+  `reset` or `abandoned`. `upstream_duration_ms{cluster}` times a `call` to its
+  whole response and a `stream` to its head. Status codes and Envoy's own
+  timeouts are in the cluster's statistics.
+
 ## Scaling
 
 Each Ruvoy process runs exactly one CRuby VM, so Ruby work in one process is

data/lib/ruvoy/envoy_config.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require "json"
 require "yaml"
 
 module Ruvoy
@@ -12,11 +13,17 @@ module Ruvoy
     STREAM_IDLE_TIMEOUT = "15s"
     REQUEST_TIMEOUT = "30s"
     CONNECTION_BUFFER_LIMIT_BYTES = 4_194_304
+    EXTENSION_NAME = "ruvoy"
+    STRING_VALUE = "type.googleapis.com/google.protobuf.StringValue"
 
     module_function
 
     def build(rackup:, address:, port:, admin_port: nil)
-      config = { "static_resources" => { "listeners" => [ listener(rackup, address, port) ] } }
+      config = {
+        "bootstrap_extensions" => [ bootstrap_extension(rackup: rackup) ],
+        "memory_allocator_manager" => memory_allocator_manager,
+        "static_resources" => { "listeners" => [ listener(address, port) ] }
+      }
       config["admin"] = admin(address, admin_port) if admin_port
       config
     end
@@ -25,16 +32,42 @@ module Ruvoy
       YAML.dump(build(**options))
     end
 
-    def listener(rackup, address, port)
+    # The extension that starts the Ruby runtime and names the application.
+    # Every listener's rack filter joins the runtime it started, so there is
+    # exactly one per Envoy.
+    def bootstrap_extension(rackup:, **settings)
+      {
+        "name" => EXTENSION_NAME,
+        "typed_config" => {
+          "@type" => "type.googleapis.com/envoy.extensions.bootstrap.dynamic_modules.v3." \
+                     "DynamicModuleBootstrapExtension",
+          "dynamic_module_config" => module_config,
+          "extension_name" => EXTENSION_NAME,
+          "extension_config" => {
+            "@type" => STRING_VALUE,
+            "value" => JSON.generate({ "rackup" => rackup }.merge(settings.transform_keys(&:to_s)))
+          }
+        }
+      }
+    end
+
+    # The same as a top-level YAML section, to head a configuration written
+    # by hand.
+    def bootstrap_yaml(rackup:, **settings)
+      YAML.dump("bootstrap_extensions" => [ bootstrap_extension(rackup: rackup, **settings) ])
+          .delete_prefix("---\n")
+    end
+
+    def listener(address, port)
       {
         "name" => "ruvoy",
         "per_connection_buffer_limit_bytes" => CONNECTION_BUFFER_LIMIT_BYTES,
         "address" => { "socket_address" => { "address" => address, "port_value" => port } },
-        "filter_chains" => [ { "filters" => [ connection_manager(rackup) ] } ]
+        "filter_chains" => [ { "filters" => [ connection_manager ] } ]
       }
     end
 
-    def connection_manager(rackup)
+    def connection_manager
       {
         "name" => "envoy.filters.network.http_connection_manager",
         "typed_config" => {
@@ -48,29 +81,41 @@ module Ruvoy
             "name" => "local_route",
             "virtual_hosts" => [ { "name" => "local_service", "domains" => [ "*" ] } ]
           },
-          "http_filters" => [ rack_filter(rackup) ]
+          "http_filters" => [ rack_filter ]
         }
       }
     end
 
     # Terminal: the Rack application answers the request, so no router follows.
-    def rack_filter(rackup)
+    # The filter has nothing of its own to configure here; the application is
+    # the bootstrap extension's.
+    def rack_filter
       {
         "name" => "envoy.extensions.filters.http.dynamic_modules",
         "typed_config" => {
           "@type" => "type.googleapis.com/envoy.extensions.filters.http." \
                      "dynamic_modules.v3.DynamicModuleFilter",
-          "dynamic_module_config" => { "name" => "ruvoy_fiber", "do_not_close" => true },
+          "dynamic_module_config" => module_config,
           "filter_name" => "fiber_rack",
           "terminal_filter" => true,
-          "filter_config" => {
-            "@type" => "type.googleapis.com/google.protobuf.StringValue",
-            "value" => rackup
-          }
+          "filter_config" => { "@type" => STRING_VALUE, "value" => "{}" }
         }
       }
     end
 
+    # Envoy's allocator keeps what Ruby and Envoy free unless told to return
+    # it, so the resident set would only ever grow. Its own background thread
+    # returns a megabyte a second, more than a busy process frees.
+    def memory_allocator_manager
+      { "bytes_to_release" => 1_048_576, "memory_release_interval" => "1s" }
+    end
+
+    # A fresh hash each time: one object in two places would come out of the
+    # YAML dumper as an anchor and an alias.
+    def module_config
+      { "name" => "ruvoy_fiber", "do_not_close" => true }
+    end
+
     def admin(address, port)
       {
         "address" => { "socket_address" => { "address" => address, "port_value" => port } }

data/lib/ruvoy/version.rb

@@ -3,5 +3,5 @@
 module Ruvoy
   # Kept in step with the Cargo workspace version by test/e2e/cli_test.rb: the
   # gem and the module it carries are the same release.
-  VERSION = "0.2.1"
+  VERSION = "0.4.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ruvoy
 version: !ruby/object:Gem::Version
-  version: 0.2.1
+  version: 0.4.0
 platform: aarch64-linux
 authors:
 - Zixin Zhou