ruvoy 0.3.0-x86_64-linux → 0.4.0-x86_64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 267edfc5bd783c89f4cdf67a901208abc8bdc864731e317f54844a90edccdc07
-  data.tar.gz: 3f2606403c1b214ac244e22bd4eb9bb93fb6a015fbc06cdc57412b7cc9822323
+  metadata.gz: 43b67f3cd76185d5e52d15ad82f8be514dfb50b6bfa3bcdfc4eec646f27674e4
+  data.tar.gz: 4389a11aaecd521e79cb814a0c8a3946fe94641a65fa83838fbd75595b4698f8
 SHA512:
-  metadata.gz: 04dff6a6abdaaf19a26d522a66d72c334ae59dc2e7b60bd9b149f0196df7718d90a5e41a76e43a743c64353620da088772ced2d13ce00bd32aaf3fa133194405
-  data.tar.gz: 4da1c6f133dd80441c1966eb396b1fd6b898d84a094dc8340257cb9b5e758648f5e55130e365a6296aff601426d65c776fde20dad770f784bffad023fd4541b4
+  metadata.gz: d7c5244ba2652458809d847392bd94b576f9f470b65fdaf83c0d4d0fa42bd15b23938f5c37b3923931dab0403867382428a8db439ef2bf64b304ac8d6c6af531
+  data.tar.gz: c0dd2782811b58957818c178959492dbda33af21474449e17e73a26b6e8b0c9568b8ba7f9180c61fe76b62bc91f55d87871133486a9286d40f72aa2b1db42306

data/README.md

@@ -53,17 +53,20 @@ Usage: ruvoy [options] [config.ru]
         --print-config               Print the generated Envoy configuration and exit
 ```
 
-To run it under an Envoy you configure yourself, `--print-config` prints the
-listener Ruvoy would have used; the module is `ruvoy_fiber` and its filter
-config is the path to the rackup, or a JSON object naming it (see
+To run it under an Envoy you configure yourself, `--print-config` prints what
+Ruvoy would have used: the `ruvoy` bootstrap extension, which names the rackup
+(see [Lifecycle](#lifecycle)), and a listener whose rack filter joins it. The
+filter's own config is an optional JSON object (see
 [Envoy context and upstream calls](#envoy-context-and-upstream-calls)).
 
 ## Status
 
 The data path is implemented and covered by suites that run against a real
 Envoy in CI. Configuration reload has been driven through a running Envoy, and
-a one-hour soak of 1.8 million requests left the heap, file descriptors and
-threads flat. Interfaces may change.
+a three-hour soak of 5.7 million requests, 22,800 WebSocket sessions and 359
+listener updates left live memory, file descriptors, threads and p99 latency
+flat; the resident set still rose about 3% an hour, held by the allocator and
+Ruby's heap rather than by anything live. Interfaces may change.
 
 ## Features
 
@@ -213,7 +216,6 @@ JSON object:
 
 ```json
 {
-  "rackup": "/srv/app/config.ru",
   "context": { "metadata_namespaces": ["acme.tenant"] },
   "upstream": { "clusters": ["users"], "timeout_ms": 15000, "max_response_bytes": 1048576 }
 }
@@ -250,7 +252,8 @@ What Envoy's module interface can report sets some limits:
 
 ### `ruvoy.upstream`
 
-Calls a cluster Envoy was configured with and returns a Rack-style triple:
+Calls a cluster Envoy was configured with and returns the status, the headers
+and the whole body as a String:
 
 ```ruby
 status, headers, body = env["ruvoy.upstream"].call(
@@ -293,9 +296,72 @@ Envoy applies no flow control to it, so a reader that falls more than
 With `protocol: "websocket"`, the response upgrades the client too and relays
 both ways.
 
+Calls and streams carry the request's trace context on: `traceparent`,
+`tracestate`, `x-request-id` and the B3 headers, unless the call sets them.
+
 [`examples/extensions`](examples/extensions) runs both against a stand-in
 service, including two calls from one request overlapping.
 
+## Lifecycle
+
+The `ruvoy` bootstrap extension owns the Ruby runtime: it names the rackup and
+the terms the runtime serves it on, and every listener's rack filter joins it.
+A rack filter configured without it is refused.
+
+```yaml
+bootstrap_extensions:
+  - name: ruvoy
+    typed_config:
+      "@type": type.googleapis.com/envoy.extensions.bootstrap.dynamic_modules.v3.DynamicModuleBootstrapExtension
+      dynamic_module_config: { name: ruvoy_fiber, do_not_close: true }
+      extension_name: ruvoy
+      extension_config:
+        "@type": type.googleapis.com/google.protobuf.StringValue
+        value: '{"rackup": "/srv/app/config.ru"}'
+```
+
+| Setting | Default | |
+|---|---|---|
+| `rackup` | | The application. Required. |
+| `max_inflight_requests` | 1024 | Requests in Ruby at once; one more is refused with 503. |
+| `shutdown_timeout_ms` | 10000 | How long shutdown waits for fibers still running. |
+| `diagnostics` | false | Adds the `x-ruvoy-*` diagnostic headers to responses. |
+
+- Envoy becomes ready only once the application has loaded and the hooks
+  registered with `Ruvoy::Lifecycle.before_ready { }` have run. An application
+  that fails to load stops Envoy.
+- `Ruvoy::Lifecycle.on_drain { }` runs when Envoy begins draining, while
+  requests are still served: close persistent connections, stop background
+  work.
+- Envoy's shutdown stops the runtime: fibers still running are cancelled and
+  given `shutdown_timeout_ms` to finish.
+- The admin interface reaches Ruby: `/ruvoy` (overview), `/ruvoy/gc` (`GET`
+  for `GC.stat`, `POST` to collect, `?compact=1` to compact too),
+  `/ruvoy/fibers` (live fibers with backtraces) and `/ruvoy/config`. Ruby
+  answers from its own thread; what it does not answer within two seconds gets
+  504.
+- `Ruvoy::Lifecycle.settings` holds the settings above.
+- Envoy's allocator keeps freed memory unless its `memory_allocator_manager`
+  sets `bytes_to_release`; without it the resident set only grows. The
+  generated configuration returns 1 MiB a second, and a hand-written one
+  should set it too.
+
+## Statistics
+
+Ruvoy reports through Envoy's statistics, so the numbers reach whatever sink
+Envoy has, the admin interface's `/stats/prometheus` among them.
+
+- `requests_total`, `rejected_total{reason}`, `responses_total{outcome}` and
+  `duration_ms`: requests, and how those that reached the application ended.
+- `inflight_requests`, `reactor_idle_ms` and `resident_bytes`: the bootstrap
+  extension reads them once a second on the main thread, so the request path
+  never publishes anything and an idle proxy reports an idle runtime.
+- `upstream_calls_total{cluster, outcome}`: calls by how the application saw
+  them end, `completed`, `refused`, `timed_out`, `overflow`, `too_large`,
+  `reset` or `abandoned`. `upstream_duration_ms{cluster}` times a `call` to its
+  whole response and a `stream` to its head. Status codes and Envoy's own
+  timeouts are in the cluster's statistics.
+
 ## Scaling
 
 Each Ruvoy process runs exactly one CRuby VM, so Ruby work in one process is

data/lib/ruvoy/envoy_config.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require "json"
 require "yaml"
 
 module Ruvoy
@@ -12,11 +13,17 @@ module Ruvoy
     STREAM_IDLE_TIMEOUT = "15s"
     REQUEST_TIMEOUT = "30s"
     CONNECTION_BUFFER_LIMIT_BYTES = 4_194_304
+    EXTENSION_NAME = "ruvoy"
+    STRING_VALUE = "type.googleapis.com/google.protobuf.StringValue"
 
     module_function
 
     def build(rackup:, address:, port:, admin_port: nil)
-      config = { "static_resources" => { "listeners" => [ listener(rackup, address, port) ] } }
+      config = {
+        "bootstrap_extensions" => [ bootstrap_extension(rackup: rackup) ],
+        "memory_allocator_manager" => memory_allocator_manager,
+        "static_resources" => { "listeners" => [ listener(address, port) ] }
+      }
       config["admin"] = admin(address, admin_port) if admin_port
       config
     end
@@ -25,16 +32,42 @@ module Ruvoy
       YAML.dump(build(**options))
     end
 
-    def listener(rackup, address, port)
+    # The extension that starts the Ruby runtime and names the application.
+    # Every listener's rack filter joins the runtime it started, so there is
+    # exactly one per Envoy.
+    def bootstrap_extension(rackup:, **settings)
+      {
+        "name" => EXTENSION_NAME,
+        "typed_config" => {
+          "@type" => "type.googleapis.com/envoy.extensions.bootstrap.dynamic_modules.v3." \
+                     "DynamicModuleBootstrapExtension",
+          "dynamic_module_config" => module_config,
+          "extension_name" => EXTENSION_NAME,
+          "extension_config" => {
+            "@type" => STRING_VALUE,
+            "value" => JSON.generate({ "rackup" => rackup }.merge(settings.transform_keys(&:to_s)))
+          }
+        }
+      }
+    end
+
+    # The same as a top-level YAML section, to head a configuration written
+    # by hand.
+    def bootstrap_yaml(rackup:, **settings)
+      YAML.dump("bootstrap_extensions" => [ bootstrap_extension(rackup: rackup, **settings) ])
+          .delete_prefix("---\n")
+    end
+
+    def listener(address, port)
       {
         "name" => "ruvoy",
         "per_connection_buffer_limit_bytes" => CONNECTION_BUFFER_LIMIT_BYTES,
         "address" => { "socket_address" => { "address" => address, "port_value" => port } },
-        "filter_chains" => [ { "filters" => [ connection_manager(rackup) ] } ]
+        "filter_chains" => [ { "filters" => [ connection_manager ] } ]
       }
     end
 
-    def connection_manager(rackup)
+    def connection_manager
       {
         "name" => "envoy.filters.network.http_connection_manager",
         "typed_config" => {
@@ -48,29 +81,41 @@ module Ruvoy
             "name" => "local_route",
             "virtual_hosts" => [ { "name" => "local_service", "domains" => [ "*" ] } ]
           },
-          "http_filters" => [ rack_filter(rackup) ]
+          "http_filters" => [ rack_filter ]
         }
       }
     end
 
     # Terminal: the Rack application answers the request, so no router follows.
-    def rack_filter(rackup)
+    # The filter has nothing of its own to configure here; the application is
+    # the bootstrap extension's.
+    def rack_filter
       {
         "name" => "envoy.extensions.filters.http.dynamic_modules",
         "typed_config" => {
           "@type" => "type.googleapis.com/envoy.extensions.filters.http." \
                      "dynamic_modules.v3.DynamicModuleFilter",
-          "dynamic_module_config" => { "name" => "ruvoy_fiber", "do_not_close" => true },
+          "dynamic_module_config" => module_config,
           "filter_name" => "fiber_rack",
           "terminal_filter" => true,
-          "filter_config" => {
-            "@type" => "type.googleapis.com/google.protobuf.StringValue",
-            "value" => rackup
-          }
+          "filter_config" => { "@type" => STRING_VALUE, "value" => "{}" }
         }
       }
     end
 
+    # Envoy's allocator keeps what Ruby and Envoy free unless told to return
+    # it, so the resident set would only ever grow. Its own background thread
+    # returns a megabyte a second, more than a busy process frees.
+    def memory_allocator_manager
+      { "bytes_to_release" => 1_048_576, "memory_release_interval" => "1s" }
+    end
+
+    # A fresh hash each time: one object in two places would come out of the
+    # YAML dumper as an anchor and an alias.
+    def module_config
+      { "name" => "ruvoy_fiber", "do_not_close" => true }
+    end
+
     def admin(address, port)
       {
         "address" => { "socket_address" => { "address" => address, "port_value" => port } }

data/lib/ruvoy/version.rb

@@ -3,5 +3,5 @@
 module Ruvoy
   # Kept in step with the Cargo workspace version by test/e2e/cli_test.rb: the
   # gem and the module it carries are the same release.
-  VERSION = "0.3.0"
+  VERSION = "0.4.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ruvoy
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: x86_64-linux
 authors:
 - Zixin Zhou