quonfig 1.1.0 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d12e38af54000d7cfeb34f7c3f50739527e63448668abf9d6fd4b436b27b277f
4
- data.tar.gz: 1421bc6f53246ee1260a5c832ef342faa5ca70b6854acba6f12d9b51a464e99d
3
+ metadata.gz: d9da26b9a9f898ec76bf8520043abc42db011adf5e298cffa16ee8764c2c0ed6
4
+ data.tar.gz: '0189ef6ee1e93cec985f9f5b1f1b81f833527bf3c22a1af7886778d8f2dcb50e'
5
5
  SHA512:
6
- metadata.gz: 73d25bb37c6c30e1d1b756172e0ffd4885bf901c9b600eec3c060f06bead7b45cf74e09544908d925a4345fcb5f9b6efe6d8331d7f8544079ffabf7b9d5b1228
7
- data.tar.gz: 04df7b6f24efb0f9a3c30c47e32c686f2407ac54c1776eb7a2313fe1332f8339eac4693fc49edb49d76e00b7521680701bedfbfae5839b4d9158b3ace473efbc
6
+ metadata.gz: e363cf1a35bbc428f35d1d4cebea4223dc84c4a1ddd39c3a7dcbc22bafb58a757b5fde92f357dd1a9b8c0e0cc90db247addd88c873e19887cd1d0eae0bc5332b
7
+ data.tar.gz: e81f8441d30449624e3c64b506e113d386ce8a55656943e4523cb2d919d97aad5d8afd68df6462df1324cb2368dd8d9694e808146d622dd790ccf722e630e442
data/CHANGELOG.md CHANGED
@@ -1,5 +1,17 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.2.0 - 2026-07-08
4
+
5
+ - **Feat (options): warn when an explicit `api_urls` disables automatic failover (qfg-41nh.26).** The default (and every `QUONFIG_DOMAIN`-derived) `api_urls` list carries a primary AND a secondary leg, and the HTTP config-fetch hedges/fails over between them (the secondary runs on separate infrastructure). An explicit `api_urls:` replaces that list wholesale, so a single-entry override silently dropped the secondary and disabled automatic failover. The client now logs one WARN at init when the caller explicitly set `api_urls` and the resolved list has fewer than two legs, pointing at the fix (pass both a primary and a secondary URL). Behavior is otherwise unchanged; no new dependencies. A new README "Failover & `QUONFIG_DOMAIN`" section documents the URL derivation and the failover model.
6
+ - **Feat (telemetry): emit failover-behavior signals on the existing telemetry wire (qfg-41nh.18).** The SDK now folds three additive failover counters into the periodic telemetry flush so the failover dashboards can be built — previously no SDK emitted any failover signal. Each flush window that saw failover activity ships one `failover` event alongside the existing `summaries` / `contextShapes` / `exampleContexts` events: `hedgeFired` (config-fetch cycles where the parallel primary+secondary hedge fired its secondary leg), `guardRejected` (installs dropped by the reject-older ordering guard, counted on BOTH the HTTP config-fetch path and the SSE message path), and `resolvedFromPrimary` / `resolvedFromSecondary` (which upstream leg served each successful HTTP install; SSE/datadir installs are not counted). `resolvedFromLkg` is reserved and always `0`. The event is emitted only when at least one counter is non-zero in the window, so a steady-state client with no config churn emits nothing. Recording rides the existing telemetry opt-out (nothing is emitted unless telemetry is enabled) and adds only a per-config-refresh mutex increment — no new dependencies, no per-evaluation overhead. Wire shape is additive and matches sdk-go exactly (an older api-telemetry strips the unknown field).
7
+
8
+ ## 1.1.1 - 2026-07-03
9
+
10
+ - **Fix (SSE): the stream leg is pinned to the primary stream URL — SSE never fails over (qfg-41nh.6).** The reconnect loop previously rotated through `sse_api_urls` on every reconnect, so with the default two derived stream URLs a primary blip silently parked the live stream on `stream.secondary` and bound config freshness to the mirror. The stream now always dials `sse_api_urls[0]` and retries it forever with backoff (matching sdk-go); failover remains HTTP-poll only, and the HTTP config-fetch hedge still uses both `api_urls` legs. The `sse_failed_over_to_secondary?` diagnostic is now latched at the dial site, and the failover chaos harness hands the client both stream legs so the f05 probe actually exercises the pin.
11
+ - **Fix (delivery): per-leg config-fetch aborts are now wall-clock (qfg-41nh.6).** The per-leg timeout was enforced with Faraday/Net::HTTP phase timeouts, which are per-read — a slow-drip upstream (one byte per interval) could hold a "bounded" leg open indefinitely and wedge the init fetch, a fallback poll tick, or the hedge drain. `timeout_ms` (plus a small headroom) is now also enforced as a wall-clock ceiling over the whole request, surfacing a `Faraday::TimeoutError` through the existing failure paths. Hedged legs additionally always settle their drain slot, so `fetch!` is guaranteed to return within the per-leg abort budget.
12
+ - **Fix (client): liveness signals no longer lie during outages (qfg-41nh.6).** The fallback poll worker stamped `last_successful_refresh` and fired `on_update` on every tick regardless of fetch outcome, so a total outage with `on_init_failure: :return` reported `ready? == true` over an empty store. The worker now stamps freshness only on successful fetches (`:updated` / `:not_modified`; a 304 or guard-rejected-but-successful response counts) and fires `on_update` only when an envelope is actually installed, matching sdk-go.
13
+ - **Fix (client): the fallback poller fetches immediately on engage and logs the engage edge (qfg-41nh.6).** The poll worker previously slept a full interval before its first fetch, so the first post-engage data arrived ~180s after SSE loss (default 60s interval) vs ~120s in sdk-go; it now fetches first, then sleeps. Engaging Layer 2 also logs at WARN (previously silent — only the stop path logged, at debug), matching sdk-go's operator signal.
14
+
3
15
  ## 1.1.0 - 2026-07-01
4
16
 
5
17
  - **Feat (delivery): the HTTP config-fetch is now a parallel-failover hedge (qfg-7h5d.1.14).** On every init/refresh config fetch the SDK fires the **primary** `api_urls` leg first; if it answers within `config_fetch_hedge_delay_ms` it **wins and the secondary is never contacted** (cold standby — a healthy system adds zero secondary load). If the primary is slow past the hedge delay **or** errors fast, the SDK **also** fires the secondary **in parallel** without cancelling the primary. Whatever arrives is installed through the existing reject-older guard, so watermark-max falls out for free: the higher generation wins, a late older payload never regresses an established client, and a late newer payload heals forward. Readiness latches on the first successful install; a late-but-newer leg heals forward afterward. SSE is untouched — only the HTTP config-fetch path hedges. Both legs failing preserves the existing init-failure semantics (`on_init_failure`).
data/README.md CHANGED
@@ -258,6 +258,42 @@ Quonfig::Client.new(
258
258
  | `data_dir_auto_reload_debounce_ms` | `Integer` (ms) | `200` | Debounce window for the auto-reload watcher — events arriving inside the window are coalesced into a single re-read. Ignored when `data_dir_auto_reload` is `false`. |
259
259
  | `logger` | Logger-like object | `nil` | Optional host-app logger (e.g. `Rails.logger`). Must respond to `debug`/`info`/`warn`/`error`. When set, all SDK warnings/errors flow through this logger instead of the default stderr / SemanticLogger backend. |
260
260
 
261
+ ## Failover & `QUONFIG_DOMAIN`
262
+
263
+ By default the SDK derives every hostname from `QUONFIG_DOMAIN` (default
264
+ `quonfig.com`):
265
+
266
+ | Role | URL |
267
+ |--------------------------|-----------------------------------------|
268
+ | Config fetch (primary) | `https://primary.quonfig.com` |
269
+ | SSE stream (primary) | `https://stream.primary.quonfig.com` |
270
+ | Config fetch (secondary) | `https://secondary.quonfig.com` |
271
+ | SSE stream (secondary) | `https://stream.secondary.quonfig.com` |
272
+ | Telemetry | `https://telemetry.quonfig.com` |
273
+
274
+ Set `QUONFIG_DOMAIN` to move all of them together (e.g.
275
+ `QUONFIG_DOMAIN=quonfig-staging.com`). **Automatic failover and hedging between
276
+ the primary and the secondary are on by default** — the secondary runs on
277
+ separate infrastructure, and the HTTP config-fetch fails over to it if the
278
+ primary is unreachable and hedges to it if the primary is slow.
279
+
280
+ `api_urls:` replaces the derived list wholesale. To keep automatic failover
281
+ with custom URLs, **pass both a primary and a secondary URL**:
282
+
283
+ ```ruby
284
+ Quonfig::Client.new(
285
+ sdk_key: 'your-sdk-key',
286
+ api_urls: [
287
+ 'https://primary.your-proxy.example',
288
+ 'https://secondary.your-proxy.example'
289
+ ]
290
+ )
291
+ ```
292
+
293
+ A single URL disables failover, and the SDK logs a warning at init. See
294
+ https://docs.quonfig.com/docs/explanations/architecture/resiliency for the full
295
+ model.
296
+
261
297
  ## Typed getters
262
298
 
263
299
  Each typed getter takes a config key and an optional context hash. If the key
@@ -66,6 +66,16 @@ module Quonfig
66
66
  @poll_supervisor = nil
67
67
  @stopped = false
68
68
  @telemetry_reporter = nil
69
+ # Shared failover-telemetry aggregator (qfg-41nh.18). Created once and
70
+ # shared between the ConfigLoader (which records hedge-fired /
71
+ # guard-rejected / resolved-from at the failover call sites) and the
72
+ # TelemetryReporter (which drains it on each flush). Created before the
73
+ # ConfigLoader so the initial HTTP fetch — often the ONLY HTTP fetch when
74
+ # SSE is healthy — is captured. Recording is a per-config-refresh mutex +
75
+ # increment (negligible); nothing is EMITTED unless telemetry is enabled
76
+ # (the reporter, which owns the drain, only starts then). Survives fork:
77
+ # the pre-fork stop drains it, the child's rebuilt reporter drains onward.
78
+ @failover_aggregator = Quonfig::Telemetry::FailoverAggregator.new
69
79
  @state_mutex = Mutex.new
70
80
  @last_successful_refresh = nil
71
81
  @sse_state = :idle
@@ -689,6 +699,7 @@ module Quonfig
689
699
  context_shape_aggregator: shape_aggregator,
690
700
  example_contexts_aggregator: example_aggregator,
691
701
  evaluation_summaries_aggregator: summaries_aggregator,
702
+ failover_aggregator: @failover_aggregator,
692
703
  sync_interval: @options.collect_sync_interval
693
704
  )
694
705
 
@@ -813,8 +824,9 @@ module Quonfig
813
824
 
814
825
  warn_if_pin_ignored_in_delivery_mode
815
826
  warn_if_hedge_abort_exceeds_init_timeout
827
+ warn_if_explicit_api_urls_disables_failover
816
828
 
817
- @config_loader = Quonfig::ConfigLoader.new(@store, @options)
829
+ @config_loader = Quonfig::ConfigLoader.new(@store, @options, failover_aggregator: @failover_aggregator)
818
830
 
819
831
  perform_initial_fetch
820
832
 
@@ -914,6 +926,27 @@ module Quonfig
914
926
  )
915
927
  end
916
928
 
929
+ # qfg-41nh.26: the SDK's default (and every QUONFIG_DOMAIN-derived) api_urls
930
+ # list carries a primary AND a secondary leg, and the HTTP config-fetch
931
+ # hedges/fails over between them (secondary runs on separate infra). An
932
+ # explicit `api_urls:` replaces that list wholesale, so a single-entry
933
+ # override silently drops the secondary and disables automatic failover.
934
+ # Warn once at init in delivery mode so the customer isn't surprised.
935
+ # Mirrors sdk-go's construction-time warning (quonfig.go). Does not change
936
+ # behavior. Fired only when the caller explicitly set api_urls AND the
937
+ # resolved list has fewer than two legs — never on the default/derived
938
+ # two-URL list.
939
+ def warn_if_explicit_api_urls_disables_failover
940
+ return unless @options.respond_to?(:api_urls_explicit)
941
+ return unless @options.api_urls_explicit
942
+ return unless Array(@options.config_api_urls).length < 2
943
+
944
+ LOG.warn(
945
+ '[quonfig] explicit api_urls disables automatic failover to the ' \
946
+ 'secondary; pass both primary and secondary URLs to keep it'
947
+ )
948
+ end
949
+
917
950
  def handle_init_failure(err)
918
951
  if @options.on_init_failure == Quonfig::Options::ON_INITIALIZATION_FAILURE::RETURN
919
952
  LOG.warn "[quonfig] Initialization did not complete cleanly; continuing with empty store: #{err.message}"
@@ -970,18 +1003,35 @@ module Quonfig
970
1003
 
971
1004
  poll_seconds = poll_ms / 1000.0
972
1005
  stopped_ref = -> { @stopped }
1006
+ # Fetch-first (qfg-41nh.6): the poller fetches IMMEDIATELY on engage and
1007
+ # then sleeps the interval — matching sdk-go's fallback_poller.go
1008
+ # engage() (immediate Fetch, then ticker). A sleep-first worker made the
1009
+ # first post-engage data arrive a full extra interval late (~180s after
1010
+ # SSE loss with the default 60s interval, vs ~120s in go).
973
1011
  worker = lambda do |notify_delivered|
974
1012
  loop do
975
1013
  break if stopped_ref.call
976
1014
 
977
- sleep poll_seconds
1015
+ result = @config_loader.fetch!
1016
+ # Liveness honesty (qfg-41nh.6): stamp freshness only when the fetch
1017
+ # actually SUCCEEDED. :updated and :not_modified (a 304, or a
1018
+ # guard-rejected-but-successful response) are successes — the
1019
+ # channel is healthy; :failed must NOT advance
1020
+ # last_successful_refresh (pre-fix every tick stamped + notified
1021
+ # regardless of outcome, so ready?/freshness reported healthy over
1022
+ # an empty store through a total outage). on_update fires only on a
1023
+ # real install, matching sdk-go (OnConfigUpdate fires in
1024
+ # installEnvelope only).
1025
+ if result != :failed
1026
+ sync_evaluator_env_id!
1027
+ record_refresh!
1028
+ notify_delivered.call
1029
+ end
1030
+ notify_on_update_callback if result == :updated
1031
+
978
1032
  break if stopped_ref.call
979
1033
 
980
- @config_loader.fetch!
981
- sync_evaluator_env_id!
982
- record_refresh!
983
- notify_delivered.call
984
- notify_on_update_callback
1034
+ sleep poll_seconds
985
1035
  end
986
1036
  end
987
1037
 
@@ -990,6 +1040,10 @@ module Quonfig
990
1040
  )
991
1041
  @state_mutex.synchronize { @poll_supervisor = supervisor }
992
1042
  supervisor.start
1043
+ # Operator signal (qfg-41nh.6): engage was previously unlogged (only the
1044
+ # stop path logged, at debug). Matches sdk-go's WARN on OnEngage — the
1045
+ # poller engaging means the SDK is in a degraded update mode.
1046
+ LOG.warn "[quonfig] Layer 2 fallback poller engaged (interval=#{poll_ms}ms)"
993
1047
  end
994
1048
 
995
1049
  # Invoke the customer-supplied on_update callback under a rescue. A raise
@@ -37,7 +37,7 @@ module Quonfig
37
37
  #
38
38
  # Backward compat: callers that pass a single +base_client+ (mock client
39
39
  # used by tests that expects `.options`) are still supported.
40
- def initialize(store_or_base_client, options = nil, logger: nil)
40
+ def initialize(store_or_base_client, options = nil, logger: nil, failover_aggregator: nil)
41
41
  if options.nil? && store_or_base_client.respond_to?(:options)
42
42
  # Legacy shape: ConfigLoader.new(base_client)
43
43
  @options = store_or_base_client.options
@@ -47,6 +47,11 @@ module Quonfig
47
47
  @options = options
48
48
  end
49
49
 
50
+ # Optional failover-telemetry sink (qfg-41nh.18). When telemetry is
51
+ # enabled the Client passes the shared FailoverAggregator; nil otherwise,
52
+ # so every record call below is a no-op under a telemetry opt-out.
53
+ @failover_aggregator = failover_aggregator
54
+
50
55
  @api_config = Concurrent::Map.new
51
56
  # qfg-7h5d.1.14: per-leg ETag is load-bearing for the parallel hedge. The
52
57
  # hedge runs the primary and secondary legs concurrently; a SINGLE shared
@@ -179,16 +184,26 @@ module Quonfig
179
184
  # mutually exclusive — exactly one of suppress/fire wins.
180
185
  gate = Mutex.new
181
186
  secondary_fired = false
182
-
187
+ # Distinct from secondary_fired (which is ALSO set true by suppress on a
188
+ # fast primary win): this is set only when the secondary leg was ACTUALLY
189
+ # spawned, so it is the exact "the hedge fired" signal (qfg-41nh.18).
190
+ secondary_spawned = false
191
+
192
+ # Each leg is wall-clock bounded by abort_ms (HttpConnection enforces a
193
+ # whole-request deadline, qfg-41nh.6), so a leg ALWAYS settles — a
194
+ # slow-drip upstream can no longer hold a leg open past the abort and
195
+ # wedge the drain below. The push lives in an ensure so even a
196
+ # non-StandardError escape still settles this leg's drain slot.
183
197
  run_leg = lambda do |index|
184
198
  Thread.new do
185
- result = begin
186
- fetch_from(urls[index], index, timeout_ms: abort_ms)
199
+ result = :failed
200
+ begin
201
+ result = fetch_from(urls[index], index, timeout_ms: abort_ms)
187
202
  rescue StandardError => e
188
203
  @logger.debug "Hedge leg #{index} failed: #{e.class}: #{e.message}"
189
- :failed
204
+ ensure
205
+ results.push([:done, index, result])
190
206
  end
191
- results.push([:done, index, result])
192
207
  end
193
208
  end
194
209
 
@@ -197,6 +212,7 @@ module Quonfig
197
212
  next false if secondary_fired
198
213
 
199
214
  secondary_fired = true
215
+ secondary_spawned = true
200
216
  end
201
217
  run_leg.call(1) if spawn
202
218
  end
@@ -250,6 +266,14 @@ module Quonfig
250
266
  # Stop the timer if it is still sleeping (already-fired is harmless).
251
267
  hedge_timer.kill if hedge_timer.alive?
252
268
 
269
+ # Failover observability (qfg-41nh.18): if the secondary leg was actually
270
+ # spawned this cycle, the hedge fired (the primary was slow past the hedge
271
+ # delay or errored fast). Recorded once per cycle regardless of which leg's
272
+ # payload won the reject-older guard. Read under the gate — the flag is
273
+ # written under it by fire_secondary, which may run on the timer thread.
274
+ hedged = gate.synchronize { secondary_spawned }
275
+ @failover_aggregator&.record_hedge_fired if hedged
276
+
253
277
  return :updated if installed
254
278
  return :not_modified if saw_not_modified
255
279
 
@@ -363,6 +387,10 @@ module Quonfig
363
387
  # on stale config would be worse (mirrors sdk-node).
364
388
  unless @held_generation.nil? || incoming_gen <= 0 || incoming_gen > @held_generation
365
389
  @logger.debug "Reject-older guard: dropping incoming generation #{incoming_gen} <= held #{@held_generation} (source=#{source})"
390
+ # Failover observability (qfg-41nh.18): count the guard rejection. This
391
+ # single guard covers BOTH the HTTP config-fetch path and the SSE
392
+ # message path (apply_envelope) — every network install funnels here.
393
+ @failover_aggregator&.record_guard_rejected
366
394
  return :not_modified
367
395
  end
368
396
 
@@ -382,6 +410,11 @@ module Quonfig
382
410
  @held_generation = incoming_gen
383
411
  @install_count += 1
384
412
  @resolved_from_index = source_index unless source_index.nil?
413
+ # Failover observability (qfg-41nh.18): record which leg served this
414
+ # successful HTTP install (source_index 0 = primary, > 0 = secondary).
415
+ # SSE / datadir installs pass source_index nil and are ignored by the
416
+ # aggregator, so they are never counted.
417
+ @failover_aggregator&.record_resolved_from(source_index)
385
418
 
386
419
  # Replace the live store atomically.
387
420
  return if @store.nil?
@@ -2,6 +2,7 @@
2
2
 
3
3
  require 'base64'
4
4
  require 'json'
5
+ require 'timeout'
5
6
 
6
7
  module Quonfig
7
8
  class HttpConnection
@@ -13,13 +14,26 @@ module Quonfig
13
14
  'X-Quonfig-SDK-Version' => SDK_VERSION
14
15
  }.freeze
15
16
 
17
+ # qfg-41nh.6 (WS2.4): headroom added to +timeout_ms+ to form the WALL-CLOCK
18
+ # ceiling for the whole request. Faraday's open/read timeouts are per-phase
19
+ # and — on the Net::HTTP adapter — per-READ: every byte that arrives resets
20
+ # the read deadline, so a slow-drip upstream (one byte per interval) holds a
21
+ # "bounded" request open indefinitely and wedges the caller (init fetch,
22
+ # fallback poll tick, hedge drain). The wall clock is the true per-leg abort
23
+ # (sdk-go's per-leg context deadline is wall-clock). The headroom keeps
24
+ # Faraday's own, more specific phase timeouts winning the simple-stall race;
25
+ # the wall clock only fires on drip-feed pathologies that per-read deadlines
26
+ # structurally cannot catch.
27
+ WALL_CLOCK_HEADROOM_S = 0.25
28
+
16
29
  # +timeout_ms+ (qfg-7h5d.1.9): per-request bound applied to BOTH the connect
17
- # (open) and read phases of every request made through this connection. nil
30
+ # (open) and read phases of every request made through this connection, AND
31
+ # (qfg-41nh.6) enforced as a wall-clock ceiling over the whole request. nil
18
32
  # leaves Faraday's defaults (no timeout) in place — preserving the prior
19
33
  # behavior for callers that don't pass one. The config-fetch path passes
20
- # Options#config_fetch_timeout_ms so a hung upstream (accepts the TCP
21
- # connection but never responds) aborts fast instead of blocking the caller's
22
- # whole init budget.
34
+ # Options#config_fetch_timeout_ms (sequential) or the hedge abort (hedged
35
+ # legs) so a hung OR drip-feeding upstream aborts fast instead of blocking
36
+ # the caller's whole init budget.
23
37
  def initialize(uri, sdk_key, timeout_ms: nil)
24
38
  @uri = uri
25
39
  @sdk_key = sdk_key
@@ -29,11 +43,11 @@ module Quonfig
29
43
  attr_reader :uri
30
44
 
31
45
  def get(path, headers = {})
32
- connection(headers).get(path)
46
+ with_wall_clock_deadline { connection(headers).get(path) }
33
47
  end
34
48
 
35
49
  def post(path, body)
36
- connection.post(path, body.to_json)
50
+ with_wall_clock_deadline { connection.post(path, body.to_json) }
37
51
  end
38
52
 
39
53
  def connection(headers = {})
@@ -54,6 +68,23 @@ module Quonfig
54
68
 
55
69
  private
56
70
 
71
+ # Enforce +timeout_ms+ (+ headroom) as a wall-clock deadline over the whole
72
+ # request. Timeout.timeout is the codebase's accepted backstop pattern (the
73
+ # init path wraps fetch! the same way in Client#perform_initial_fetch); a
74
+ # deadline check inside the read loop isn't practical here because Faraday
75
+ # 1.x's Net::HTTP adapter exposes no per-chunk streaming hook. Raises
76
+ # Faraday::TimeoutError so callers' existing timeout rescue paths apply
77
+ # unchanged. No-op when no timeout_ms was configured.
78
+ def with_wall_clock_deadline(&block)
79
+ return block.call unless @timeout_ms
80
+
81
+ deadline_s = (@timeout_ms / 1000.0) + WALL_CLOCK_HEADROOM_S
82
+ Timeout.timeout(deadline_s, Faraday::TimeoutError,
83
+ "wall-clock per-request deadline #{deadline_s}s exceeded for #{@uri}") do
84
+ block.call
85
+ end
86
+ end
87
+
57
88
  def auth_header
58
89
  "Basic #{Base64.strict_encode64("1:#{@sdk_key}")}"
59
90
  end
@@ -8,7 +8,7 @@ module Quonfig
8
8
  attr_reader :sdk_key, :environment, :api_urls, :sse_api_urls, :telemetry_destination, :config_api_urls,
9
9
  :on_no_default, :init_timeout_ms, :on_init_failure, :collect_sync_interval, :datadir, :enable_sse, :fallback_poll_enabled, :fallback_poll_interval_ms, :global_context, :logger_key, :logger, :enable_quonfig_user_context,
10
10
  :data_dir_auto_reload, :data_dir_auto_reload_debounce_ms, :config_fetch_timeout_ms,
11
- :config_fetch_hedge_delay_ms, :config_fetch_hedge_abort_ms
11
+ :config_fetch_hedge_delay_ms, :config_fetch_hedge_abort_ms, :api_urls_explicit
12
12
  attr_accessor :is_fork
13
13
 
14
14
  # Default fallback poll interval, in milliseconds. The SDK polls api-delivery
@@ -308,6 +308,16 @@ module Quonfig
308
308
  # 3. Hardcoded DEFAULT_DOMAIN ('quonfig.com')
309
309
  domain = Quonfig::Options.domain
310
310
 
311
+ # Whether the caller explicitly supplied api_urls (vs. the SDK deriving
312
+ # both legs from QUONFIG_DOMAIN / DEFAULT_DOMAIN). The default and every
313
+ # domain-derived list carries a primary AND a secondary leg, so the SDK
314
+ # hedges/fails over automatically; an explicit single-entry override
315
+ # silently drops the secondary. Client warns once at init when this is
316
+ # true and the resolved list has < 2 legs (see
317
+ # Client#warn_if_explicit_api_urls_disables_failover). Mirrors sdk-go's
318
+ # apiURLsExplicit flag (qfg-41nh.26).
319
+ @api_urls_explicit = !api_urls.nil?
320
+
311
321
  @api_urls = Array(api_urls || Quonfig::Options.derive_api_urls(domain))
312
322
  .map { |url| remove_trailing_slash(url) }
313
323
 
@@ -87,14 +87,16 @@ module Quonfig
87
87
  @conn_mutex = Mutex.new
88
88
  @active_http = nil
89
89
 
90
- @source_index = -1
91
90
  @last_event_id = nil
92
- # qfg-7h5d.1.9: latches true if the SSE reconnect rotation ever selects a
93
- # non-primary (index > 0) sse_api_urls leg. The failover epic asserts SSE
94
- # does NOT fail over to the secondary (f05) — it stays on its own endpoint
95
- # and degrades via the single-upstream SSE↔HTTP fallback. With a single SSE
96
- # URL configured this can never flip; the flag makes the design choice
97
- # observable (and would surface a regression if SSE were given two legs).
91
+ # qfg-7h5d.1.9 / qfg-41nh.6: latches true if a connection attempt ever
92
+ # targets a non-primary sse_api_urls leg. The failover epic asserts SSE
93
+ # does NOT fail over to the secondary (f05) — the stream is PINNED to
94
+ # sse_api_urls[0] and degrades via the single-upstream SSE↔HTTP fallback,
95
+ # so with a correct #current_url this never flips even when a secondary
96
+ # stream URL is configured (options derive one from api_urls by default).
97
+ # The latch is measured at the dial site in #stream_once, so a regression
98
+ # that reintroduces stream-URL rotation is observable to the failover
99
+ # chaos probe (f05) and to operators.
98
100
  @failed_over_to_secondary = false
99
101
  end
100
102
 
@@ -269,7 +271,13 @@ module Quonfig
269
271
  # gotcha and solve it the same way: per-chunk reset, async close on
270
272
  # expiry (chaos scenario 02 — sse_silent_stall).
271
273
  def stream_once(&block)
272
- url = "#{current_url}/api/v2/sse/config"
274
+ base = current_url
275
+ # f05 latch: record if this connection attempt targets a non-primary
276
+ # stream leg. With the pin in #current_url this is structurally false;
277
+ # measuring at the dial site (rather than inside URL selection) keeps a
278
+ # rotation regression observable to the chaos probe.
279
+ @failed_over_to_secondary = true if base != @prefab_options.sse_api_urls.first
280
+ url = "#{base}/api/v2/sse/config"
273
281
  cursor = current_cursor
274
282
  @logger.debug "SSE Streaming Connect to #{url} start_at #{cursor.inspect}"
275
283
 
@@ -405,14 +413,15 @@ module Quonfig
405
413
  end
406
414
  end
407
415
 
408
- # Rotate through configured SSE URLs. The same rotation rule the
409
- # previous implementation used, preserved so multi-region failover
410
- # behavior is unchanged.
416
+ # The stream leg is PINNED to the primary stream URL (sse_api_urls[0])
417
+ # and retried forever with backoff SSE never fails over (f05 invariant,
418
+ # qfg-41nh.6). Failover is HTTP-poll only: the config-fetch hedge keeps
419
+ # BOTH api_urls legs, but the live stream never repoints to
420
+ # stream.secondary (which is sized for poll rps, not SSE herds, and would
421
+ # silently bind freshness to the mirror). Matches sdk-go's sseClient — a
422
+ # single pinned URL with reconnect backoff (sse_client.go runLoop).
411
423
  def current_url
412
- urls = @prefab_options.sse_api_urls
413
- @source_index = (@source_index + 1) % urls.size
414
- @failed_over_to_secondary = true if @source_index.positive?
415
- urls[@source_index]
424
+ @prefab_options.sse_api_urls.first
416
425
  end
417
426
 
418
427
  # Internal: HTTP-status sentinel error for non-200 SSE responses. Surfaces
@@ -0,0 +1,113 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Quonfig
4
+ module Telemetry
5
+ # Accumulates failover-behavior counters over a flush window: how many times
6
+ # the config-fetch hedge fired its secondary leg, how many installs the
7
+ # reject-older ordering guard dropped, and which upstream leg served each
8
+ # successful HTTP install. Every counter is additive and carries NO user
9
+ # data.
10
+ #
11
+ # The aggregator is independently thread-safe (its own mutex) and is written
12
+ # directly from the failover call sites in ConfigLoader rather than through
13
+ # any queue — the call rate is per-config-refresh, not per-evaluation, so a
14
+ # plain mutex has negligible overhead.
15
+ #
16
+ # Wire shape matches sdk-go's FailoverEvent (the canonical reference) and
17
+ # api-telemetry's failover schema EXACTLY — camelCase keys, unix-millis
18
+ # window. #drain_event returns +nil+ when every counter is zero, so a healthy
19
+ # steady-state client emits no failover event at all.
20
+ #
21
+ # { "failover": {
22
+ # "start": <int ms>, "end": <int ms>,
23
+ # "hedgeFired": <int>, "guardRejected": <int>,
24
+ # "resolvedFromPrimary": <int>, "resolvedFromSecondary": <int>,
25
+ # "resolvedFromLkg": <int>
26
+ # } }
27
+ class FailoverAggregator
28
+ def initialize
29
+ @mutex = Mutex.new
30
+ @start_at_ms = nil
31
+ @hedge_fired = 0
32
+ @guard_rejected = 0
33
+ @resolved_from_primary = 0
34
+ @resolved_from_secondary = 0
35
+ # Reserved for a future last-known-good delivery path; backends emit 0.
36
+ @resolved_from_lkg = 0
37
+ end
38
+
39
+ # Count one config-fetch cycle whose hedge fired the secondary leg (the
40
+ # primary was slow past the hedge delay or errored fast).
41
+ def record_hedge_fired
42
+ @mutex.synchronize do
43
+ @start_at_ms ||= Quonfig::TimeHelpers.now_in_ms
44
+ @hedge_fired += 1
45
+ end
46
+ end
47
+
48
+ # Count one install dropped by the reject-older ordering guard (an
49
+ # equal-or-older snapshot on any install path — HTTP config-fetch or SSE).
50
+ def record_guard_rejected
51
+ @mutex.synchronize do
52
+ @start_at_ms ||= Quonfig::TimeHelpers.now_in_ms
53
+ @guard_rejected += 1
54
+ end
55
+ end
56
+
57
+ # Count one successful HTTP install by the leg that served it: source_index
58
+ # 0 is the primary, any index > 0 is a failover/secondary leg. A nil or
59
+ # negative index (SSE / datadir install with no HTTP leg) is ignored.
60
+ def record_resolved_from(source_index)
61
+ return if source_index.nil? || source_index.negative?
62
+
63
+ @mutex.synchronize do
64
+ @start_at_ms ||= Quonfig::TimeHelpers.now_in_ms
65
+ if source_index.zero?
66
+ @resolved_from_primary += 1
67
+ else
68
+ @resolved_from_secondary += 1
69
+ end
70
+ end
71
+ end
72
+
73
+ # Return the window's counters as a telemetry event hash and reset state.
74
+ # Returns +nil+ when no failover activity occurred (every counter zero).
75
+ def drain_event
76
+ @mutex.synchronize do
77
+ if @hedge_fired.zero? && @guard_rejected.zero? &&
78
+ @resolved_from_primary.zero? && @resolved_from_secondary.zero? &&
79
+ @resolved_from_lkg.zero?
80
+ return nil
81
+ end
82
+
83
+ event = {
84
+ 'failover' => {
85
+ 'start' => @start_at_ms || Quonfig::TimeHelpers.now_in_ms,
86
+ 'end' => Quonfig::TimeHelpers.now_in_ms,
87
+ 'hedgeFired' => @hedge_fired,
88
+ 'guardRejected' => @guard_rejected,
89
+ 'resolvedFromPrimary' => @resolved_from_primary,
90
+ 'resolvedFromSecondary' => @resolved_from_secondary,
91
+ 'resolvedFromLkg' => @resolved_from_lkg
92
+ }
93
+ }
94
+
95
+ reset
96
+ event
97
+ end
98
+ end
99
+
100
+ private
101
+
102
+ # Caller holds @mutex.
103
+ def reset
104
+ @start_at_ms = nil
105
+ @hedge_fired = 0
106
+ @guard_rejected = 0
107
+ @resolved_from_primary = 0
108
+ @resolved_from_secondary = 0
109
+ @resolved_from_lkg = 0
110
+ end
111
+ end
112
+ end
113
+ end
@@ -13,7 +13,8 @@ module Quonfig
13
13
  # "events": [
14
14
  # { "summaries": { "start": ..., "end": ..., "summaries": [...] } },
15
15
  # { "contextShapes": { "shapes": [...] } },
16
- # { "exampleContexts": { "examples": [...] } }
16
+ # { "exampleContexts": { "examples": [...] } },
17
+ # { "failover": { "start": ..., "end": ..., "hedgeFired": ..., ... } }
17
18
  # ]
18
19
  # }
19
20
  #
@@ -30,6 +31,7 @@ module Quonfig
30
31
  context_shape_aggregator: nil,
31
32
  example_contexts_aggregator: nil,
32
33
  evaluation_summaries_aggregator: nil,
34
+ failover_aggregator: nil,
33
35
  sync_interval: nil,
34
36
  http_connection: nil)
35
37
  @options = options
@@ -39,6 +41,11 @@ module Quonfig
39
41
  @context_shape_aggregator = context_shape_aggregator
40
42
  @example_contexts_aggregator = example_contexts_aggregator
41
43
  @evaluation_summaries_aggregator = evaluation_summaries_aggregator
44
+ # Failover counters carry no user data and are the operational signal for
45
+ # the secondary-delivery hardening (qfg-41nh.18), so they ride the
46
+ # existing telemetry stream. The ConfigLoader records directly into this
47
+ # aggregator at the failover call sites; the reporter only drains it.
48
+ @failover_aggregator = failover_aggregator
42
49
  @http_connection = http_connection
43
50
  @sync_interval = calculate_sync_interval(sync_interval)
44
51
  @stopped = Concurrent::AtomicBoolean.new(false)
@@ -125,6 +132,9 @@ module Quonfig
125
132
  if (example_event = @example_contexts_aggregator&.drain_event)
126
133
  events << example_event
127
134
  end
135
+ if (failover_event = @failover_aggregator&.drain_event)
136
+ events << failover_event
137
+ end
128
138
 
129
139
  return if events.empty?
130
140
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Quonfig
4
- VERSION = '1.1.0'
4
+ VERSION = '1.2.0'
5
5
  end
data/lib/quonfig.rb CHANGED
@@ -61,6 +61,7 @@ require 'quonfig/telemetry/context_shape'
61
61
  require 'quonfig/telemetry/context_shape_aggregator'
62
62
  require 'quonfig/telemetry/example_contexts_aggregator'
63
63
  require 'quonfig/telemetry/evaluation_summaries_aggregator'
64
+ require 'quonfig/telemetry/failover_aggregator'
64
65
  require 'quonfig/telemetry/telemetry_reporter'
65
66
  require 'quonfig/client'
66
67
  require 'quonfig/bound_client'
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: quonfig
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Jeff Dwyer
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-07-01 00:00:00.000000000 Z
11
+ date: 2026-07-08 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activesupport
@@ -130,6 +130,7 @@ files:
130
130
  - lib/quonfig/telemetry/context_shape_aggregator.rb
131
131
  - lib/quonfig/telemetry/evaluation_summaries_aggregator.rb
132
132
  - lib/quonfig/telemetry/example_contexts_aggregator.rb
133
+ - lib/quonfig/telemetry/failover_aggregator.rb
133
134
  - lib/quonfig/telemetry/telemetry_reporter.rb
134
135
  - lib/quonfig/time_helpers.rb
135
136
  - lib/quonfig/types.rb