quonfig 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +26 -0
- data/README.md +126 -48
- data/lib/quonfig/client.rb +547 -65
- data/lib/quonfig/config_loader.rb +6 -0
- data/lib/quonfig/sse_config_client.rb +9 -0
- data/lib/quonfig/telemetry/telemetry_reporter.rb +51 -0
- data/lib/quonfig/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 5e8807ad54199b5ee24245cdf140a72aed0644e712308d867457d729df1bb122
|
|
4
|
+
data.tar.gz: 20b1d12fb441394f4d31198854b55dd90f78c925115982a32548415993c77958
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cda8d4a8d332710a58f3be0e1e068e767858c9806f199bc23a5a2f010e7540d4fd9a159b20ac23f9d34f1a7f60b5aac10fcffdeda030356ace8536779048126a
|
|
7
|
+
data.tar.gz: 33dcc1b5201298630936e743f5e44b188393b810ee94287623dce7598b98ab68ee0a2eb8bddb70f92da2fb3419f6a0b7f2a3081e2510b7f2d144087765b541b6
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,31 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.4.0 - 2026-09-10
|
|
4
|
+
|
|
5
|
+
> **If you fork worker processes and keep evaluating in the parent, upgrade.** On 1.0.0–1.3.0 the parent process went permanently dark after any `fork(2)` — it kept serving whatever config snapshot it held at the moment of the fork, forever, while `connection_state` reported `:connected`.
|
|
6
|
+
|
|
7
|
+
- **Fix (fork): the `Process._fork` hook is now child-only — the parent is never touched (qfg-lv4n.1).** Since 0.0.16 the hook tore the SSE worker, fallback poller, telemetry reporter, and datadir watcher down **in the parent** before the fork syscall, and restarted them only in the child. Any topology where the parent keeps evaluating after a fork — a Sidekiq process using the `parallel` gem, a `fork { ... }` inside a job, a rake task that forks — lost live updates permanently. A customer's Sidekiq process served a 13-day-old snapshot through four rollout changes this way. The hook now does **nothing** on the parent side: the parent keeps its stream, its poller, its telemetry reporter, and its live config straight through any number of forks. This restores the design every other SDK in this space uses (Reforge's `Reforge.fork`, LaunchDarkly's `postfork`, dd-trace-rb / redis-client / connection_pool, which all branch on the child stage of `_fork` only). No customer wiring is required, and no public API was removed.
|
|
8
|
+
- **Change (fork): a forked child re-initializes on its FIRST use of the client, and never inherits the parent's config snapshot.** The hook does no I/O at all: it drops what the child inherited — including the parent's store — and arms a flag. The first `get` / `defined?` / `keys` / details lookup in the child then does exactly what `Client.new` does: its own blocking config fetch under the configured `init_timeout_ms` and `on_init_failure`, then its own SSE stream (or fallback poller) and its own telemetry reporter. Three consequences worth knowing: the first call in a forked child pays one fetch; a child that never uses the client costs nothing (no fetch, no socket, no thread, and `stop` returns immediately); and because the child's store starts empty, its first envelope is *installed* rather than dropped by the reject-older guard as same-generation. This is full parity with Reforge, where a forked process simply builds a new client. `connection_state` does not trigger the re-initialization — a diagnostic must never open a socket — and answers `:initializing` while one is pending.
|
|
9
|
+
- **Fix (fork): concurrent first use in a child waits for the one rebuild (qfg-lv4n.1).** The re-initialization used an unlocked fast path and cleared its pending flag *before* doing the work, so only the thread that won the mutex actually blocked: every other thread read the cleared flag, skipped the mutex, and evaluated against the brand-new **empty** store. Sixteen threads hitting a freshly forked client against a 400ms config endpoint returned `{nil=>15, "v1"=>1}`. The flag now stays set for the whole rebuild, so every other caller blocks on it and then sees the fetched config — one fetch, one stream dial, one reporter per child, however many threads race the first request. A same-thread guard keeps a customer logger (a `semantic_logger_filter` / `stdlib_formatter` that evaluates a config) from deadlocking on the non-reentrant lock if it fires from inside the rebuild.
|
|
10
|
+
- **Fix (fork): a hard timeout mid-rebuild no longer leaves the child dark forever (qfg-lv4n.1).** `Timeout::ExitException` (Ruby 3.3's `timeout` 0.4.1), rack-timeout's `RequestTimeoutException`, and `Thread#kill` are all `Exception`, not `StandardError`, so they crossed the rebuild's rescue untouched — and with the pending flag already cleared, every later call in that child returned `nil` over an empty store with no stream, no poller and no reporter, permanently. The flag is now cleared at the point the child actually has a live path to config, so anything that escapes before that leaves the rebuild armed and the next call retries it.
|
|
11
|
+
- **Fix (fork): `stop` racing an in-flight rebuild no longer orphans an SSE worker (qfg-lv4n.1).** `stop` cleared the pending flag and tore down without taking the rebuild lock, so a rebuild already past that point went on to build a stream nothing held a reference to and nothing could close. `stop` now raises its stopped flag **before** queueing for the rebuild lock — an in-flight rebuild sees it and starts neither an update channel nor a telemetry reporter — and the teardown itself runs under that lock, so it can never interleave with construction.
|
|
12
|
+
- **Fix (fork): a datadir child whose rebuild fails no longer dials SSE (qfg-lv4n.1).** The failure path started the update channel regardless of mode, so a child of a purely **offline** (datadir) client opened a stream to `stream.primary.quonfig.com` on its first lookup and then logged ``Error applying SSE envelope: undefined method `apply_envelope' for nil`` for every envelope that arrived — with nothing re-armed, repairing the workspace on disk never helped. A datadir client's healing path is the filesystem: the failure path now starts the datadir watcher when `data_dir_auto_reload` is on, and otherwise re-arms the rebuild so the next use retries the load.
|
|
13
|
+
- **Fix (fork): `after_fork_in_child` called in the PARENT is now a no-op (qfg-lv4n.1).** Releases 1.0.0–1.3.0 documented calling `Quonfig.instance.after_fork_in_child` in the parent after `fork` returned as the workaround for the parent going dark. On 1.4.0 the parent's components are alive, so each such call orphaned a live SSE worker and its stream, zeroed the store, and stopped the **owner's** telemetry reporter — three calls took a process from 2 worker threads and 1 live stream to 8 and 4, and the orphans outlived `stop`. The hook now early-returns (one debug line) in the process that **owns** the client, decided by comparing the current pid against the one stamped when the client was built. **If you added that call in the parent as a 1.3.0 workaround, remove it; it does nothing there now.**
|
|
14
|
+
- **Fix (fork): a child forked from inside an `on_update` callback rebuilds like any other child (qfg-lv4n.1).** Parent-vs-child detection asked whether the inherited SSE worker thread was alive. `on_update` runs *on* that worker thread, so a customer who forks from the callback forks on it — making it the child's one surviving thread, and making the inherited `@worker.alive?` answer **true in a real child**. Such a child was classified as the parent: the hook ignored it, it served the parent's snapshot for the rest of its life, reported `:connected`, and resumed the parent's SSE loop on the shared file descriptor. Ownership is now a pid stamp taken at construction (and re-taken when a child rebuilds), so a pid mismatch is proof of a fork child whatever the inherited `Thread` objects claim. The same stamp closes the other end of the hole: a parent-side `after_fork_in_child` on a client with no threads at all (datadir + `data_dir_auto_reload: false` + no SDK key) sailed straight past the old liveness guard and wiped the live store; it is now correctly a no-op.
|
|
15
|
+
- **Fix (fork): a retried rebuild no longer dials a second stream (qfg-lv4n.1).** The rebuild disarms its pending flag only *after* the update channel is up, so a non-`StandardError` landing in that window (rack-timeout, `Timeout::ExitException`, `Thread#kill`) left the flag armed **with a live stream**. The retry re-ran network init, opened a second SSE stream, and overwrote `@sse_client` — orphaning the first worker where `stop` could never reach it, and leaving the child holding two streams against the delivery service. `start_update_channel` is now idempotent: it returns immediately if an SSE worker or poll supervisor is already alive.
|
|
16
|
+
- **Fix (fork): `on_init_failure: :raise` behaves the same in a forked child as in a fresh client (qfg-lv4n.1).** A failed re-initialization was always swallowed and logged, so the documented "exactly like a newly constructed client" was false for the one option whose entire job is raise-vs-return. Under `:raise` the first use in a child now raises the init error out of the lookup — Reforge does the same, raising the init error out of `get` itself — and later lookups keep raising, without re-fetching, until the update channel lands an envelope. The update channel is still started on the way out so the child can heal. **Note that `:raise` is the default**, so with default options a forked child whose first lookup lands in a total delivery outage (primary and secondary both unreachable) raises where 1.3.0 silently served the parent's snapshot; set `on_init_failure: :return` if you would rather a child serve defaults through an outage. `:return` is unchanged: one line logged, defaults served.
|
|
17
|
+
- **Fix (fork): `Quonfig.fork` / `Client#fork` in a child the hook already prepared returns the same client (qfg-4t5o).** The 1.0–1.3 README taught `on_worker_boot { Quonfig.fork }`, and on Ruby 3.1+ that call ran *after* the `Process._fork` hook had already prepared the client in the worker. Before first use it discarded the prepared client and built a second one with an eager fetch; after first use it left the worker holding two live SSE streams and two telemetry reporters, the first pair orphaned where `stop` could never reach them. `Client#fork` now returns `self` when the hook has already prepared it in the current process, so a leftover call from older docs is harmless. Outside a hook-prepared child (the owning process, Ruby 3.0, a client stopped before the fork) it still builds a fresh client, which is the Ruby 3.0 manual-wiring path; the old client is never stopped.
|
|
18
|
+
- **Fix (fork): the public `store` / `resolver` / `evaluator` / `config_loader` readers route through the post-fork re-initialization (qfg-lv4n.1).** They bypassed it entirely, so in a forked child that had not been used yet `client.store.get(key)` answered `nil` and `client.resolver.get(key, {})` raised `MissingDefaultError` against the empty store. They stay public (semver) and now re-initialize before handing the component back; reading one in a forked child can therefore block on the child's own fetch.
|
|
19
|
+
- **Fix (fork): the child drops inherited references instead of closing them.** `fork(2)` duplicates file descriptors, so the child's copy of the SSE socket points at the connection the **parent** is still streaming on — closing it would write a TLS `close_notify` onto that shared connection and kill the parent's stream. The child now nils `@sse_client`, `@poll_supervisor`, `@telemetry_reporter`, `@datadir_watcher`, and the pending fallback-engage timer without calling `close` / `stop` / `join` on any of them (joining an inherited thread blocks forever, since the thread does not exist in the child), then builds everything fresh.
|
|
20
|
+
- **Fix (fork): the child's telemetry starts empty.** The forked child gets brand-new context-shape, example-context, evaluation-summary, and failover aggregators rather than inheriting the parent's half-full ones (the reporter is not *started* until the child's first use). The parent flushes the data it collected; the child reports only its own, so a fork no longer double-counts a window of evaluations.
|
|
21
|
+
- **Fix (fork/telemetry): the inherited `at_exit` drain no longer speaks for the parent.** `TelemetryReporter#start` registers a process-wide `Kernel.at_exit` closure over the reporter; `fork(2)` copies it, and dropping the client's reference in the child does not unregister it — so any child that exited the normal way (block-form `fork` + `exit`, which is what the `parallel` gem does) POSTed a full copy of the **parent's** un-flushed window under the parent's `instanceHash`, and the parent then POSTed it again. The reporter now records an owner pid on `start`, and `sync`, `stop`, and the `at_exit` drain are no-ops (one debug line) in any other process; the child additionally discards the inherited aggregators. Measured on a three-child `Parallel.map`: 404 evaluations reported for 101 performed, now 101. Nothing is closed, stopped, or joined — the thread does not exist in the child and the HTTP connection's fd is shared with the parent.
|
|
22
|
+
- **Fix (fork): one client failing to rebuild no longer takes the rest of the registry down with it.** The child-side fan-out was covered by a single hook-wide rescue, so the first `after_fork_in_child` to raise (thread exhaustion, a customer logger that raises) aborted the loop and every client behind it in the registry stayed dark. Each instance now has its own rescue and the fan-out continues, logging the failure at error.
|
|
23
|
+
- **Fix (fork): a forked datadir child gets its own telemetry reporter (qfg-vquv).** The datadir branch of `after_fork_in_child` returned before the aggregator/reporter rebuild, so a datadir + SDK-key child — an emitting combination since 1.3.0 — recorded nothing of its own for the rest of its life. It now rebuilds aggregators and reporter under exactly the SDK-key gating a fresh client applies.
|
|
24
|
+
- **Fix (diagnostics): `connection_state` derives from liveness, not from a stored flag.** It previously answered `:connected` off `@sse_state` alone, which the teardown path never reset — so a client with no SSE worker and no poller alive still reported healthy. That is why the incident above stayed invisible for 13 days. A network client that is supposed to hold an SSE stream and has no live worker now reports `:disconnected`. The documented value set (`:initializing`, `:connected`, `:disconnected`, `:falling_back`) is unchanged, and `:connected` still covers datadir mode and post-fetch clients with SSE disabled.
|
|
25
|
+
- **Deprecated: `Client#before_fork_in_parent`.** Still public and still works, but the fork hook no longer calls it. There is no longer any reason to tear a parent down before forking; call `stop` if you want a client dead. Slated for removal in 2.0.0.
|
|
26
|
+
- **Docs:** "one telemetry POST at exit" for a per-job forking worker is now qualified — the at-exit drain only happens when the child exits *normally*. `Parallel` children do; **Resque children call `exit!` by default**, which skips every `at_exit` handler, so there is no drain and no telemetry POST unless `RUN_AT_EXIT_HOOKS=1` is set.
|
|
27
|
+
- **Docs:** the README's fork section previously claimed the SDK covered "Sidekiq's parent-forks-workers model" (Sidekiq OSS does not fork — it runs jobs on threads) and documented "does not auto-restart the parent" as intentional. Both are corrected. The Puma snippet no longer suggests `before_fork { Quonfig.instance.stop }`, and the cases that actually fork inside Sidekiq (the `parallel` gem, an explicit `fork { }`, Enterprise swarm) are named as covered. The Ruby 3.0 `parallel` snippet no longer calls `Quonfig.fork` once per row (that builds a client per item; it is now a pid-memoized rebuild once per child process, with a note that 3.0 is EOL), and the Puma/Unicorn worker-boot snippets show `SemanticLogger.reopen` alone on 3.1+ — calling `Quonfig.fork` there after the hook has already rebuilt leaves the worker with two live SSE streams and two reporters.
|
|
28
|
+
|
|
3
29
|
## 1.3.0 - 2026-08-18
|
|
4
30
|
|
|
5
31
|
> **Heads up — datadir users will start emitting telemetry.** Datadir mode with a valid SDK key now emits usage telemetry as documented; it previously sent nothing. If you run the SDK with a `datadir:` (or `QUONFIG_DIR`) **and** an SDK key, upgrading to 1.3.0 means evaluation summaries and context telemetry begin flowing to the telemetry service from that deployment — data you were not sending on 1.2.0. Disable with the standard telemetry options if unwanted (`collect_evaluation_summaries: false` and `context_upload_mode: :none`). A datadir client with **no** SDK key still sends nothing at all.
|
data/README.md
CHANGED
|
@@ -210,16 +210,16 @@ Default is `false`; datadir mode is silent until you opt in.
|
|
|
210
210
|
|
|
211
211
|
The auto-reload watcher uses a background thread, which — like any Ruby
|
|
212
212
|
thread — does not survive `fork(2)`. **You do not need to wire this up
|
|
213
|
-
manually on Ruby 3.1+.**
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
Resque, Spring, and manual
|
|
213
|
+
manually on Ruby 3.1+.** After a fork, the child re-loads the workspace from
|
|
214
|
+
disk and registers a fresh watcher on its first use of the client (see [Rails
|
|
215
|
+
integration](#rails-integration) below); a child that never uses the client
|
|
216
|
+
starts no watcher at all. The parent's watcher is left alone and keeps
|
|
217
|
+
working. This covers Puma clustered mode, Unicorn, Resque, Spring, and manual
|
|
218
|
+
`fork { ... }` calls — including a `fork` inside a Sidekiq job.
|
|
218
219
|
|
|
219
|
-
On Ruby 3.0 (no `Process._fork`), follow the manual `
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
watcher, in the child.
|
|
220
|
+
On Ruby 3.0 (no `Process._fork`), follow the manual `on_worker_boot` pattern
|
|
221
|
+
in the [Rails integration](#rails-integration) section — `Quonfig.fork`
|
|
222
|
+
rebuilds the full client, including the datadir watcher, in the child.
|
|
223
223
|
|
|
224
224
|
### Tuning the debounce window
|
|
225
225
|
|
|
@@ -414,16 +414,41 @@ If both are supplied, the explicit `logger_name:` wins.
|
|
|
414
414
|
|
|
415
415
|
## Rails integration
|
|
416
416
|
|
|
417
|
-
The SDK runs a background SSE thread (and optional polling thread)
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
the child silently stops receiving live updates.
|
|
417
|
+
The SDK runs a background SSE thread (and optional polling thread). Ruby
|
|
418
|
+
threads do not survive `fork(2)`, so a child process inherits references to
|
|
419
|
+
threads that no longer exist and silently stops receiving live updates.
|
|
421
420
|
|
|
422
421
|
**On Ruby 3.1+ the SDK installs a `Process._fork` hook at load time** that
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
422
|
+
handles this for you. It covers any `Process.fork` / `Kernel#fork` path —
|
|
423
|
+
Puma's clustered mode, Unicorn, Spring, Resque, a `fork { ... }` inside a
|
|
424
|
+
Sidekiq job, and the `parallel` gem. **No customer wiring is required.**
|
|
425
|
+
|
|
426
|
+
**The hook is child-only. A fork never touches the process that forked.**
|
|
427
|
+
The parent keeps its SSE stream, its poller, its telemetry reporter, and its
|
|
428
|
+
live config straight through any number of forks — so a long-lived process
|
|
429
|
+
that forks workers *and keeps evaluating* (a Sidekiq process using the
|
|
430
|
+
`parallel` gem, a rake task that calls `fork`) stays current.
|
|
431
|
+
In the child, the SDK drops the inherited references without touching the
|
|
432
|
+
objects — it never closes the inherited socket, because `fork(2)` duplicates
|
|
433
|
+
the file descriptor and closing the child's copy of a TLS connection would
|
|
434
|
+
tear down the stream the **parent** is still using.
|
|
435
|
+
|
|
436
|
+
> **Upgrading from 1.3.0 or earlier:** if you added a manual
|
|
437
|
+
> `Quonfig.instance.after_fork_in_child` call **in the parent** as a
|
|
438
|
+
> workaround for the parent going dark, remove it. As of 1.4.0 that call is a
|
|
439
|
+
> no-op in the process that owns the client — the SDK decides that by
|
|
440
|
+
> comparing the current pid against the one it stamped when the client was
|
|
441
|
+
> built, so it is exact whether or not the parent has any threads running. It
|
|
442
|
+
> will not hurt you, but it is no longer doing anything, and the parent needs
|
|
443
|
+
> no call.
|
|
444
|
+
|
|
445
|
+
**After a fork, the child re-initializes on its first use of the client,
|
|
446
|
+
exactly like a newly constructed client — including its `on_init_failure`
|
|
447
|
+
policy: it fetches its own config and starts its own threads. It does not
|
|
448
|
+
evaluate from the parent's snapshot.** The hook itself does no I/O — it drops
|
|
449
|
+
what the child inherited and arms the re-initialization. So the first call in
|
|
450
|
+
a forked child pays one fetch, and a child that never uses the client costs
|
|
451
|
+
nothing: no fetch, no stream, no thread, no telemetry.
|
|
427
452
|
|
|
428
453
|
Caveats:
|
|
429
454
|
|
|
@@ -431,13 +456,44 @@ Caveats:
|
|
|
431
456
|
- `system("fork-and-exec ...")` and `Process.spawn` are not covered (they do
|
|
432
457
|
not go through `Process._fork`), but those execute a new program, so the
|
|
433
458
|
in-process SSE state is moot.
|
|
434
|
-
- The
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
the
|
|
459
|
+
- The first lookup in a forked child **blocks** on that child's own config
|
|
460
|
+
fetch, under the same `init_timeout_ms` and `on_init_failure` options a
|
|
461
|
+
fresh client uses. **The default is `on_init_failure: :raise`**, so if the
|
|
462
|
+
child's fetch fails against every `api_urls` leg (primary and secondary
|
|
463
|
+
both unreachable) the failure **raises out of that first lookup**, exactly
|
|
464
|
+
as `Client.new` would at boot, and later lookups keep raising — without
|
|
465
|
+
re-fetching — until the update channel lands an envelope, at which point
|
|
466
|
+
the client serves config normally again. On 1.3.0 and earlier a child in
|
|
467
|
+
that situation silently served the parent's snapshot instead. If you would
|
|
468
|
+
rather a forked child serve defaults through an outage, set
|
|
469
|
+
`on_init_failure: :return`: a failed fetch then logs one line and the child
|
|
470
|
+
serves defaults until its stream or poller lands the first envelope.
|
|
471
|
+
- **Other threads wait.** Every thread that reaches the client while that
|
|
472
|
+
first fetch is in flight blocks on it and then sees the fetched config. One
|
|
473
|
+
fetch, one stream dial, and one telemetry reporter per child, however many
|
|
474
|
+
threads race the first request.
|
|
475
|
+
- **`connection_state` never triggers the re-initialization** — a diagnostic
|
|
476
|
+
must not open a socket. A child that has not used the client yet answers
|
|
477
|
+
`:initializing`, which is exactly what it is; it flips to `:connected` on
|
|
478
|
+
first use.
|
|
479
|
+
- The child's telemetry aggregators start empty. The parent flushes the data
|
|
480
|
+
it collected before the fork; the child reports only its own.
|
|
481
|
+
- **Per-job forking pays per job.** A Resque-style worker that forks a child
|
|
482
|
+
per job (or `Parallel.map` with one row per process) pays, in each child
|
|
483
|
+
that touches the client, one config fetch, one SSE dial, and — **when the
|
|
484
|
+
child exits normally** — one telemetry POST at exit. That is the price of
|
|
485
|
+
the child holding its own current config and its own telemetry window, and
|
|
486
|
+
it is deliberate — the delivery service counts each of those connections as
|
|
487
|
+
a real client. A child that never uses the client pays none of it.
|
|
488
|
+
|
|
489
|
+
The at-exit drain depends on the child running `at_exit` handlers at all.
|
|
490
|
+
`Parallel` children do. **Resque children call `exit!` by default**, which
|
|
491
|
+
skips every `at_exit` handler — so there is no drain and no telemetry POST
|
|
492
|
+
unless you set `RUN_AT_EXIT_HOOKS=1`. Nothing else about the child changes;
|
|
493
|
+
it just never flushes the evaluations it collected.
|
|
494
|
+
- In datadir mode a child whose workspace fails to load never dials the
|
|
495
|
+
network: it logs the failure and, if `data_dir_auto_reload` is on, watches
|
|
496
|
+
for a repaired workspace; otherwise the next use retries the load.
|
|
441
497
|
|
|
442
498
|
### Puma (clustered mode)
|
|
443
499
|
|
|
@@ -450,52 +506,72 @@ handle the rest:
|
|
|
450
506
|
Quonfig.init(Quonfig::Options.new(sdk_key: ENV.fetch('QUONFIG_BACKEND_SDK_KEY')))
|
|
451
507
|
```
|
|
452
508
|
|
|
453
|
-
If you
|
|
509
|
+
If you use SemanticLogger you still need to reopen it in each worker — but
|
|
510
|
+
`Quonfig.fork` is not needed in that block on 3.1+. The SDK has already
|
|
511
|
+
handled the fork by the time `on_worker_boot` runs, and since 1.4.0 a
|
|
512
|
+
`Quonfig.fork` call in a child the hook has already prepared simply returns
|
|
513
|
+
the same client (so a leftover call from older docs is harmless):
|
|
454
514
|
|
|
455
515
|
```ruby
|
|
456
|
-
# config/puma.rb (Ruby 3.
|
|
457
|
-
|
|
458
|
-
|
|
516
|
+
# config/puma.rb (Ruby 3.1+)
|
|
517
|
+
on_worker_boot do
|
|
518
|
+
SemanticLogger.reopen
|
|
459
519
|
end
|
|
520
|
+
```
|
|
460
521
|
|
|
522
|
+
If you're on Ruby 3.0 (no `Process._fork`), wire the worker boot hook
|
|
523
|
+
manually:
|
|
524
|
+
|
|
525
|
+
```ruby
|
|
526
|
+
# config/puma.rb (Ruby 3.0 only)
|
|
461
527
|
on_worker_boot do
|
|
462
|
-
Quonfig.fork
|
|
528
|
+
Quonfig.fork # rebuild a fresh client per worker
|
|
529
|
+
SemanticLogger.reopen # if you use SemanticLogger
|
|
463
530
|
end
|
|
464
531
|
```
|
|
465
532
|
|
|
533
|
+
Do **not** add a `before_fork { Quonfig.instance.stop }` — the master's
|
|
534
|
+
client does not need to be torn down for the workers to be healthy, and
|
|
535
|
+
stopping it means the master stops receiving config.
|
|
536
|
+
|
|
466
537
|
### Sidekiq
|
|
467
538
|
|
|
468
|
-
|
|
469
|
-
`
|
|
539
|
+
Sidekiq OSS does not fork: it runs jobs on threads inside one process, so
|
|
540
|
+
`Quonfig.init` in your initializer is all you need on any Ruby version.
|
|
470
541
|
|
|
471
|
-
|
|
542
|
+
Some jobs *do* fork — the `parallel` gem, an explicit `fork { ... }`, or
|
|
543
|
+
Sidekiq Enterprise's swarm mode. On Ruby 3.1+ those are covered
|
|
544
|
+
automatically, with nothing to call, and (since 1.4.0) the Sidekiq process
|
|
545
|
+
itself keeps streaming config the whole time.
|
|
472
546
|
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
Quonfig.
|
|
547
|
+
Ruby 3.0 is end-of-life and has no `Process._fork` hook. The `parallel` gem
|
|
548
|
+
has no per-worker boot hook to wire a rebuild into either — `Parallel.each`
|
|
549
|
+
just runs your block in each child, once per row — so calling `Quonfig.fork`
|
|
550
|
+
at the top of the block builds a **new client per row**, each with its own
|
|
551
|
+
SSE stream and telemetry reporter. Upgrade to 3.1+ if you can. If you must
|
|
552
|
+
stay on 3.0, rebuild once per child process by memoizing on the pid:
|
|
476
553
|
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
554
|
+
```ruby
|
|
555
|
+
# Ruby 3.0 only — one rebuild per child process, not one per row.
|
|
556
|
+
Parallel.each(batch, in_processes: 4) do |row|
|
|
557
|
+
Quonfig.fork if $quonfig_pid != Process.pid
|
|
558
|
+
$quonfig_pid = Process.pid
|
|
559
|
+
# ...
|
|
481
560
|
end
|
|
482
561
|
```
|
|
483
562
|
|
|
484
|
-
For Sidekiq web/CLI processes that don't fork (default `concurrency: 1`),
|
|
485
|
-
`Quonfig.init` in the initializer is sufficient on any Ruby version.
|
|
486
|
-
|
|
487
563
|
### Spring / Bootsnap preloaders
|
|
488
564
|
|
|
489
|
-
Spring forks the preloader for each command.
|
|
490
|
-
|
|
491
|
-
|
|
565
|
+
Spring forks the preloader for each command. On Ruby 3.1+ the automatic hook
|
|
566
|
+
already rebuilds the client in each spawned command, and the preloader itself
|
|
567
|
+
keeps streaming. On Ruby 3.0, either:
|
|
492
568
|
|
|
493
569
|
1. **Recommended:** initialize lazily — wrap `Quonfig.init` so it only runs
|
|
494
570
|
the first time `Quonfig.instance` is called from a non-preloader process.
|
|
495
571
|
2. **Or:** call `Quonfig.fork` from a `Spring.after_fork` hook.
|
|
496
572
|
|
|
497
573
|
```ruby
|
|
498
|
-
# config/spring.rb
|
|
574
|
+
# config/spring.rb (Ruby 3.0 only)
|
|
499
575
|
Spring.after_fork do
|
|
500
576
|
Quonfig.fork if defined?(Quonfig) && Quonfig.instance_variable_get(:@singleton)
|
|
501
577
|
end
|
|
@@ -528,8 +604,10 @@ envelope is intentional: a reader concurrent with envelope application may
|
|
|
528
604
|
observe the new value for some keys and the old value for others, then
|
|
529
605
|
converge once the envelope finishes applying.
|
|
530
606
|
|
|
531
|
-
|
|
532
|
-
|
|
607
|
+
Forking is handled for you on Ruby 3.1+: the child rebuilds automatically and
|
|
608
|
+
the parent is left running (see [Rails integration](#rails-integration)). On
|
|
609
|
+
Ruby 3.0, `Quonfig.fork` is the way to "carry" a client into a child — do not
|
|
610
|
+
reuse the parent's client object in a child process without it.
|
|
533
611
|
|
|
534
612
|
## Diagnostic health signals
|
|
535
613
|
|
data/lib/quonfig/client.rb
CHANGED
|
@@ -21,14 +21,16 @@ module Quonfig
|
|
|
21
21
|
LOG = Quonfig::InternalLogger.new(self)
|
|
22
22
|
|
|
23
23
|
# qfg-ryov: instance registry for the Process._fork hook. Every live
|
|
24
|
-
# Client is tracked here so the hook can fan out
|
|
25
|
-
#
|
|
26
|
-
#
|
|
27
|
-
#
|
|
28
|
-
#
|
|
29
|
-
#
|
|
30
|
-
#
|
|
31
|
-
#
|
|
24
|
+
# Client is tracked here so the hook can fan out after_fork_in_child
|
|
25
|
+
# across all of them without the customer needing to name a specific
|
|
26
|
+
# instance. ObjectSpace::WeakMap means a Client that goes out of scope is
|
|
27
|
+
# GC'd without leaking through this registry. Stopped Clients stay in the
|
|
28
|
+
# registry until GC; after_fork_in_child early-returns on +@stopped+ so a
|
|
29
|
+
# stopped instance is effectively a no-op. (We don't use WeakMap#delete
|
|
30
|
+
# because it was added in Ruby 3.3 and the matrix still includes 3.2.)
|
|
31
|
+
#
|
|
32
|
+
# The registry is read in the CHILD only (qfg-lv4n.1) — the hook does
|
|
33
|
+
# nothing on the parent side, so no lock is taken across the syscall.
|
|
32
34
|
@instances = ObjectSpace::WeakMap.new
|
|
33
35
|
@instances_mutex = Mutex.new
|
|
34
36
|
|
|
@@ -43,8 +45,7 @@ module Quonfig
|
|
|
43
45
|
end
|
|
44
46
|
end
|
|
45
47
|
|
|
46
|
-
attr_reader :options, :
|
|
47
|
-
:config_loader, :telemetry_reporter
|
|
48
|
+
attr_reader :options, :instance_hash, :telemetry_reporter
|
|
48
49
|
|
|
49
50
|
def initialize(options = nil, store: nil, **option_kwargs)
|
|
50
51
|
@options =
|
|
@@ -82,6 +83,25 @@ module Quonfig
|
|
|
82
83
|
@sse_ever_connected = false
|
|
83
84
|
@fallback_engage_timer = nil
|
|
84
85
|
@sse_terminal_failure = false
|
|
86
|
+
# The process that owns this client's threads, sockets, and store.
|
|
87
|
+
# Re-stamped when a child takes ownership in #rebuild_in_child!. A pid
|
|
88
|
+
# mismatch is PROOF that we are looking at a fork(2) child, which is
|
|
89
|
+
# what makes #owned_by_this_process? exact. See that method for why
|
|
90
|
+
# thread liveness could not answer the question.
|
|
91
|
+
@owner_pid = Process.pid
|
|
92
|
+
# Post-fork lazy re-initialization (qfg-lv4n.1). Set by
|
|
93
|
+
# +after_fork_in_child+; cleared by the first use of the client in the
|
|
94
|
+
# child. See #ensure_initialized_after_fork.
|
|
95
|
+
@fork_rebuild_pending = false
|
|
96
|
+
@fork_rebuild_mutex = Mutex.new
|
|
97
|
+
# The thread currently running the rebuild, so a re-entrant read (a
|
|
98
|
+
# SemanticLoggerFilter or stdlib formatter that calls +get+ from inside
|
|
99
|
+
# the rebuild's own logging) does not deadlock on the non-reentrant
|
|
100
|
+
# Mutex above.
|
|
101
|
+
@fork_rebuild_owner = nil
|
|
102
|
+
# Sticky init error under +on_init_failure: :raise+ (see
|
|
103
|
+
# #raise_sticky_fork_init_error).
|
|
104
|
+
@fork_rebuild_error = nil
|
|
85
105
|
|
|
86
106
|
# If the caller injected a store, we're in test/bootstrap mode; skip I/O.
|
|
87
107
|
return if store
|
|
@@ -103,6 +123,7 @@ module Quonfig
|
|
|
103
123
|
# ---- Lookup --------------------------------------------------------
|
|
104
124
|
|
|
105
125
|
def get(key, default = NO_DEFAULT_PROVIDED, jit_context = NO_DEFAULT_PROVIDED)
|
|
126
|
+
ensure_initialized_after_fork
|
|
106
127
|
ctx = build_context(jit_context)
|
|
107
128
|
record_context_for_telemetry(ctx)
|
|
108
129
|
result =
|
|
@@ -186,10 +207,12 @@ module Quonfig
|
|
|
186
207
|
end
|
|
187
208
|
|
|
188
209
|
def defined?(key)
|
|
210
|
+
ensure_initialized_after_fork
|
|
189
211
|
!@store.get(key).nil?
|
|
190
212
|
end
|
|
191
213
|
|
|
192
214
|
def keys
|
|
215
|
+
ensure_initialized_after_fork
|
|
193
216
|
@store.keys
|
|
194
217
|
end
|
|
195
218
|
|
|
@@ -312,51 +335,117 @@ module Quonfig
|
|
|
312
335
|
end
|
|
313
336
|
|
|
314
337
|
def stop
|
|
338
|
+
# Order matters (qfg-lv4n.1 D5). The flag goes up BEFORE we queue for
|
|
339
|
+
# the rebuild lock, so a post-fork rebuild already in flight sees it and
|
|
340
|
+
# skips starting an update channel and a telemetry reporter at all —
|
|
341
|
+
# otherwise it builds an SSE worker after we have finished tearing down
|
|
342
|
+
# and nothing is left holding a reference to close it.
|
|
315
343
|
@stopped = true
|
|
316
|
-
|
|
344
|
+
# A child that never used the client must be able to stop it without
|
|
345
|
+
# paying for a re-initialization it never asked for.
|
|
346
|
+
@fork_rebuild_pending = false
|
|
347
|
+
@fork_rebuild_error = nil
|
|
348
|
+
|
|
349
|
+
# ...and the teardown itself is serialized against the rebuild, so it
|
|
350
|
+
# can never interleave with component construction. Re-entrancy: if the
|
|
351
|
+
# rebuild is what called `stop` (a customer on_update/logger hook), this
|
|
352
|
+
# thread already holds the lock.
|
|
353
|
+
if @fork_rebuild_owner == Thread.current
|
|
354
|
+
tear_down_threaded_components!
|
|
355
|
+
else
|
|
356
|
+
@fork_rebuild_mutex.synchronize { tear_down_threaded_components! }
|
|
357
|
+
end
|
|
317
358
|
end
|
|
318
359
|
|
|
319
|
-
#
|
|
320
|
-
#
|
|
321
|
-
#
|
|
322
|
-
#
|
|
360
|
+
# @deprecated Since 1.4.0 the +Process._fork+ hook NO LONGER CALLS THIS.
|
|
361
|
+
# A fork must not disturb the process that forked: the parent keeps its
|
|
362
|
+
# SSE stream, its poller, and its telemetry reporter, and keeps serving
|
|
363
|
+
# live config (qfg-lv4n.1). This method is retained for semver and for
|
|
364
|
+
# the Ruby 3.0 manual-wiring path, where a customer who genuinely wants
|
|
365
|
+
# the parent torn down before a fork can still call it. Prefer +stop+
|
|
366
|
+
# if you want the client dead.
|
|
323
367
|
#
|
|
324
|
-
#
|
|
325
|
-
#
|
|
326
|
-
#
|
|
327
|
-
# fork is the only safe shape.
|
|
368
|
+
# Closes the SSE worker, polling supervisor, telemetry reporter, datadir
|
|
369
|
+
# watcher, and any fallback-engage timer. Idempotent. Does NOT set
|
|
370
|
+
# +@stopped+, so +after_fork_in_child+ can still rebuild.
|
|
328
371
|
def before_fork_in_parent
|
|
329
372
|
return if @stopped
|
|
330
373
|
|
|
331
374
|
tear_down_threaded_components!
|
|
332
375
|
end
|
|
333
376
|
|
|
334
|
-
#
|
|
335
|
-
#
|
|
336
|
-
#
|
|
337
|
-
#
|
|
377
|
+
# Post-fork hook, run IN THE CHILD ONLY (see Quonfig::ForkSafety).
|
|
378
|
+
#
|
|
379
|
+
# Ruby threads do not survive fork(2), so everything threaded the child
|
|
380
|
+
# inherited is a dead reference. The child drops those references and
|
|
381
|
+
# rebuilds from scratch — matching Reforge's +Reforge.fork+, which simply
|
|
382
|
+
# constructs a brand-new client and lets the inherited one be collected.
|
|
383
|
+
#
|
|
384
|
+
# Two things we deliberately do NOT do to the inherited objects:
|
|
385
|
+
#
|
|
386
|
+
# * **Never close the inherited SSE socket.** fork(2) duplicates the fd,
|
|
387
|
+
# so the child's copy points at the connection the PARENT is still
|
|
388
|
+
# streaming on. Closing a TLS socket writes a +close_notify+ alert onto
|
|
389
|
+
# that shared connection and kills the parent's stream. Dropping the
|
|
390
|
+
# reference leaves the parent's fd untouched.
|
|
391
|
+
# * **Never join an inherited thread.** The thread does not exist in the
|
|
392
|
+
# child, so a +join+/+stop+ that waits on it blocks forever (see
|
|
393
|
+
# LaunchDarkly ruby-server-sdk PR #430: "close blocks forever, because
|
|
394
|
+
# EventProcessor#stop waits for a dispatcher thread that does not
|
|
395
|
+
# exist").
|
|
396
|
+
#
|
|
397
|
+
# No-op if the client was already stopped — the customer asked for it to
|
|
398
|
+
# be dead, and a fork must not resurrect it.
|
|
399
|
+
#
|
|
400
|
+
# Also a no-op in the process that OWNS the client, i.e. the PARENT —
|
|
401
|
+
# decided by a pid stamp, not by whether anything looks alive. Releases
|
|
402
|
+
# 1.0-1.3 documented calling this in the parent as the workaround for the
|
|
403
|
+
# parent going dark after a fork; on 1.4.0+ such a call would orphan the
|
|
404
|
+
# parent's live components and zero its store, so it is ignored (one
|
|
405
|
+
# debug line). See #owned_by_this_process?.
|
|
406
|
+
#
|
|
407
|
+
# The hook does NO I/O: no fetch, no socket, no thread. It throws away
|
|
408
|
+
# everything the child inherited — including the parent's config snapshot
|
|
409
|
+
# — and arms a flag. The child re-initializes on its FIRST use of the
|
|
410
|
+
# client (see #ensure_initialized_after_fork), exactly like a newly
|
|
411
|
+
# constructed client would. A child that never uses the client, which is
|
|
412
|
+
# most of them in a `Parallel.map` batch, costs nothing at all.
|
|
338
413
|
def after_fork_in_child
|
|
339
414
|
return if @stopped
|
|
415
|
+
return if owned_by_this_process?
|
|
340
416
|
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
417
|
+
# The inherited Mutexes may be held by threads that no longer exist.
|
|
418
|
+
# Only this thread exists in a fresh child, so swapping them is safe.
|
|
419
|
+
@state_mutex = Mutex.new
|
|
420
|
+
@fork_rebuild_mutex = Mutex.new
|
|
421
|
+
@fork_rebuild_owner = nil
|
|
422
|
+
@fork_rebuild_error = nil
|
|
423
|
+
drop_inherited_threaded_components!
|
|
345
424
|
|
|
346
|
-
|
|
425
|
+
# SSE state machine carries flags that describe the PARENT's session
|
|
426
|
+
# (it had connected, it had errored, ...). None of them apply here.
|
|
427
|
+
@sse_state = :idle
|
|
428
|
+
@sse_ever_connected = false
|
|
429
|
+
@sse_terminal_failure = false
|
|
430
|
+
@sse_error_callback = nil
|
|
347
431
|
|
|
348
|
-
#
|
|
349
|
-
|
|
350
|
-
@state_mutex.synchronize do
|
|
351
|
-
@sse_state = :idle
|
|
352
|
-
@sse_ever_connected = false
|
|
353
|
-
@sse_terminal_failure = false
|
|
354
|
-
end
|
|
432
|
+
# A client that never finished network init has nothing to rebuild.
|
|
433
|
+
return if @config_loader.nil? && !@options.datadir
|
|
355
434
|
|
|
356
|
-
|
|
357
|
-
|
|
435
|
+
# A brand-new, EMPTY store. The child must not evaluate from whatever
|
|
436
|
+
# snapshot the parent happened to hold at the instant of the fork: it
|
|
437
|
+
# fetches (or loads) its own on first use.
|
|
438
|
+
reset_store_in_child!
|
|
358
439
|
|
|
359
|
-
|
|
440
|
+
# Fresh aggregators. The parent flushes its own copy; a child that
|
|
441
|
+
# flushed inherited data would double-report it. The reporter is BUILT
|
|
442
|
+
# here (so the child's config loader points at the child's failover
|
|
443
|
+
# aggregator) but NOT started — starting it is I/O, and that waits for
|
|
444
|
+
# first use.
|
|
445
|
+
rebuild_aggregators_in_child!
|
|
446
|
+
|
|
447
|
+
@forked_in_pid = Process.pid
|
|
448
|
+
@fork_rebuild_pending = true
|
|
360
449
|
end
|
|
361
450
|
|
|
362
451
|
# quonfig_sdk_worker_restart_total counter (Tier 1 supervisor contract).
|
|
@@ -409,7 +498,17 @@ module Quonfig
|
|
|
409
498
|
def connection_state
|
|
410
499
|
@state_mutex.synchronize do
|
|
411
500
|
next :disconnected if @stopped
|
|
501
|
+
# Forked, not yet used: nothing has been fetched and nothing is
|
|
502
|
+
# running. Saying so is the honest answer, and a diagnostic must not
|
|
503
|
+
# be what triggers a blocking fetch.
|
|
504
|
+
next :initializing if @fork_rebuild_pending
|
|
412
505
|
next :falling_back if @poll_supervisor&.alive?
|
|
506
|
+
# Liveness beats the stored flag (qfg-lv4n.1). A client whose SSE
|
|
507
|
+
# session was torn down keeps a stale @sse_state; answering
|
|
508
|
+
# :connected off that flag is how a dark client reported healthy for
|
|
509
|
+
# 13 days. If this client is supposed to have a live SSE worker and
|
|
510
|
+
# does not, it is disconnected — whatever the flag says.
|
|
511
|
+
next :disconnected if sse_channel_expected? && !sse_worker_alive?
|
|
413
512
|
next :connected if @sse_state == :connected
|
|
414
513
|
next :disconnected if @sse_state == :error
|
|
415
514
|
|
|
@@ -461,7 +560,65 @@ module Quonfig
|
|
|
461
560
|
sse.failed_over_to_secondary?
|
|
462
561
|
end
|
|
463
562
|
|
|
563
|
+
# ---- Component readers ---------------------------------------------
|
|
564
|
+
#
|
|
565
|
+
# Public since 1.0 and kept public for semver. Each one routes through
|
|
566
|
+
# the post-fork chokepoint: in a forked child that has not been used yet
|
|
567
|
+
# the raw components read an EMPTY store (+store.get+ answered nil,
|
|
568
|
+
# +resolver.get+ raised MissingDefaultError), so handing them back
|
|
569
|
+
# without re-initializing is handing back a component that lies
|
|
570
|
+
# (qfg-lv4n.1 D6).
|
|
571
|
+
|
|
572
|
+
# @return [Quonfig::ConfigStore] the store backing this client.
|
|
573
|
+
# @note In a forked child, reading this triggers the lazy post-fork
|
|
574
|
+
# re-initialization (see #after_fork_in_child) — it can block on the
|
|
575
|
+
# child's own config fetch.
|
|
576
|
+
def store
|
|
577
|
+
ensure_initialized_after_fork
|
|
578
|
+
@store
|
|
579
|
+
end
|
|
580
|
+
|
|
581
|
+
# @return [Quonfig::Resolver]
|
|
582
|
+
# @note (see #store)
|
|
583
|
+
def resolver
|
|
584
|
+
ensure_initialized_after_fork
|
|
585
|
+
@resolver
|
|
586
|
+
end
|
|
587
|
+
|
|
588
|
+
# @return [Quonfig::Evaluator]
|
|
589
|
+
# @note (see #store)
|
|
590
|
+
def evaluator
|
|
591
|
+
ensure_initialized_after_fork
|
|
592
|
+
@evaluator
|
|
593
|
+
end
|
|
594
|
+
|
|
595
|
+
# @return [Quonfig::ConfigLoader, nil] nil in datadir mode.
|
|
596
|
+
# @note (see #store)
|
|
597
|
+
def config_loader
|
|
598
|
+
ensure_initialized_after_fork
|
|
599
|
+
@config_loader
|
|
600
|
+
end
|
|
601
|
+
|
|
602
|
+
# A client to use in a forked child.
|
|
603
|
+
#
|
|
604
|
+
# On Ruby 3.1+ the +Process._fork+ hook has already prepared THIS client
|
|
605
|
+
# in the child by the time any user code runs there (inherited threads
|
|
606
|
+
# and store dropped, re-initialization armed for first use). A call here
|
|
607
|
+
# in that child — the +on_worker_boot { Quonfig.fork }+ line the 1.0–1.3
|
|
608
|
+
# README taught — therefore returns +self+: the hook already did what the
|
|
609
|
+
# caller is asking for. Building a second client instead would discard
|
|
610
|
+
# the prepared one and pay an eager second fetch, or, after first use,
|
|
611
|
+
# leave the worker holding two live SSE streams and two reporters with
|
|
612
|
+
# the first pair orphaned where +stop+ can never reach it (qfg-4t5o).
|
|
613
|
+
#
|
|
614
|
+
# Everywhere else — the owning process, Ruby 3.0 where there is no hook,
|
|
615
|
+
# a client that was +stop+ped before the fork — this builds a fresh
|
|
616
|
+
# client, which is the Ruby 3.0 manual-wiring path. The old client is
|
|
617
|
+
# never stopped: on 3.0 it is the inherited one, and stopping it would
|
|
618
|
+
# close the inherited socket and tear down the PARENT's stream.
|
|
464
619
|
def fork
|
|
620
|
+
return self if @forked_in_pid == Process.pid
|
|
621
|
+
|
|
465
622
|
self.class.new(@options.for_fork)
|
|
466
623
|
end
|
|
467
624
|
|
|
@@ -471,10 +628,73 @@ module Quonfig
|
|
|
471
628
|
|
|
472
629
|
private
|
|
473
630
|
|
|
474
|
-
#
|
|
475
|
-
#
|
|
476
|
-
#
|
|
631
|
+
# True when THIS process is the one that owns the client — i.e. we are
|
|
632
|
+
# the parent, not a fork(2) child.
|
|
633
|
+
#
|
|
634
|
+
# The answer is a pid comparison against the stamp taken when the client
|
|
635
|
+
# was constructed (and re-taken when a child rebuilds it). A pid mismatch
|
|
636
|
+
# is PROOF of a fork child; a match is proof that nobody forked.
|
|
637
|
+
#
|
|
638
|
+
# It deliberately does NOT ask whether any component looks alive, which
|
|
639
|
+
# is what 1.4.0 shipped and what got this wrong at both ends:
|
|
640
|
+
#
|
|
641
|
+
# * **False negative in a real child.** +on_update+ runs on the SSE worker
|
|
642
|
+
# thread, so a customer who forks from that callback forks ON it — and
|
|
643
|
+
# the inherited +@worker.alive?+ is therefore true in the child. The
|
|
644
|
+
# child was classified as the parent, ignored the hook, served the
|
|
645
|
+
# parent's snapshot forever and reported +:connected+ (qfg-lv4n.1 E1).
|
|
646
|
+
# * **False positive in a real parent.** A datadir client with
|
|
647
|
+
# auto-reload off and no SDK key has no threads and no reporter at all,
|
|
648
|
+
# so the guard let a parent-side call through and it wiped the live
|
|
649
|
+
# store (qfg-lv4n.1 E4/D4).
|
|
650
|
+
#
|
|
651
|
+
# The parent case is what makes a stray +after_fork_in_child+ call a
|
|
652
|
+
# no-op. Releases 1.0-1.3 documented exactly that call as the workaround
|
|
653
|
+
# for the parent-keeps-evaluating topology, and that code is still out
|
|
654
|
+
# there: on 1.4.0+ it would orphan the live SSE worker and its stream,
|
|
655
|
+
# zero the store, and stop the owner's telemetry reporter.
|
|
656
|
+
def owned_by_this_process?
|
|
657
|
+
return false unless @owner_pid == Process.pid
|
|
658
|
+
|
|
659
|
+
LOG.debug '[quonfig] after_fork_in_child called in the process that OWNS the client ' \
|
|
660
|
+
"(pid=#{Process.pid}); ignoring. Since 1.4.0 a fork never touches the " \
|
|
661
|
+
'process that forked, and the child-side rebuild is automatic on Ruby 3.1+.'
|
|
662
|
+
true
|
|
663
|
+
end
|
|
664
|
+
|
|
665
|
+
# True when this client is a network-mode client that asked for SSE, i.e.
|
|
666
|
+
# one that is SUPPOSED to be holding a live stream. Datadir clients and
|
|
667
|
+
# store-injected (test/bootstrap) clients never are, so their
|
|
668
|
+
# +connection_state+ keeps deriving from envelope installs alone.
|
|
669
|
+
def sse_channel_expected?
|
|
670
|
+
return false if @options.datadir
|
|
671
|
+
return false unless @options.enable_sse
|
|
672
|
+
|
|
673
|
+
!@config_loader.nil?
|
|
674
|
+
end
|
|
675
|
+
|
|
676
|
+
# Is there an SSE worker thread actually running right now? Note this
|
|
677
|
+
# stays true across a reconnect: the worker owns the retry loop, so a
|
|
678
|
+
# blip does not read as "no channel".
|
|
679
|
+
def sse_worker_alive?
|
|
680
|
+
sse = @sse_client
|
|
681
|
+
return false if sse.nil?
|
|
682
|
+
return true unless sse.respond_to?(:alive?)
|
|
683
|
+
|
|
684
|
+
sse.alive?
|
|
685
|
+
end
|
|
686
|
+
|
|
687
|
+
# Close every threaded component and drop its reference. Used by +stop+
|
|
688
|
+
# (where @stopped is also flipped) and by the deprecated manual
|
|
689
|
+
# +before_fork_in_parent+ (where @stopped is left alone). NOT reachable
|
|
690
|
+
# from the fork hook any more — a fork never touches the process that
|
|
691
|
+
# forked (qfg-lv4n.1).
|
|
477
692
|
def tear_down_threaded_components!
|
|
693
|
+
# The SSE state machine describes a session that no longer exists.
|
|
694
|
+
# Leaving @sse_state == :connected behind is how `connection_state`
|
|
695
|
+
# came to answer :connected for a client with nothing alive.
|
|
696
|
+
@state_mutex.synchronize { @sse_state = :idle }
|
|
697
|
+
|
|
478
698
|
begin
|
|
479
699
|
@sse_client&.close
|
|
480
700
|
rescue StandardError => e
|
|
@@ -506,11 +726,233 @@ module Quonfig
|
|
|
506
726
|
@datadir_watcher = nil
|
|
507
727
|
end
|
|
508
728
|
|
|
509
|
-
#
|
|
510
|
-
#
|
|
511
|
-
|
|
729
|
+
# Drop every inherited threaded component WITHOUT closing, stopping, or
|
|
730
|
+
# joining it. See the comment on +after_fork_in_child+ for why touching
|
|
731
|
+
# these objects in the child is actively harmful (shared socket fds,
|
|
732
|
+
# threads that do not exist). Reforge, LaunchDarkly, dd-trace-rb,
|
|
733
|
+
# redis-client and connection_pool all do exactly this.
|
|
734
|
+
def drop_inherited_threaded_components!
|
|
735
|
+
inherited_reporter = @telemetry_reporter
|
|
736
|
+
|
|
737
|
+
@sse_client = nil
|
|
738
|
+
@poll_supervisor = nil
|
|
512
739
|
@telemetry_reporter = nil
|
|
513
|
-
|
|
740
|
+
@datadir_watcher = nil
|
|
741
|
+
@fallback_engage_timer = nil
|
|
742
|
+
|
|
743
|
+
# Dropping our reference is not enough for the reporter: its
|
|
744
|
+
# `Kernel.at_exit { final_drain_on_exit }` closure is process-wide, it
|
|
745
|
+
# was copied by fork(2), and it still holds a full copy of the PARENT's
|
|
746
|
+
# un-flushed telemetry window. The reporter's own owner-pid guard is
|
|
747
|
+
# what makes that closure inert (see TelemetryReporter#start); this
|
|
748
|
+
# additionally makes the copied window unreachable. Neither stops,
|
|
749
|
+
# closes, nor joins anything.
|
|
750
|
+
begin
|
|
751
|
+
inherited_reporter&.discard_inherited!
|
|
752
|
+
rescue StandardError => e
|
|
753
|
+
LOG.debug "Error discarding inherited telemetry reporter: #{e.message}"
|
|
754
|
+
end
|
|
755
|
+
end
|
|
756
|
+
|
|
757
|
+
# Lazy post-fork re-initialization. Called from every read entry point
|
|
758
|
+
# (+get+, +evaluate_details+, +defined?+, +keys+) — the flag read is a
|
|
759
|
+
# plain boolean, so the steady-state cost is one comparison per lookup.
|
|
760
|
+
#
|
|
761
|
+
# The first caller in the child does what +Client.new+ does: its own
|
|
762
|
+
# config fetch under the configured init timeout and +on_init_failure+
|
|
763
|
+
# policy, then its own SSE stream (or fallback poller) and its own
|
|
764
|
+
# telemetry reporter. It BLOCKS, so that first lookup already reflects
|
|
765
|
+
# the child's own current config.
|
|
766
|
+
#
|
|
767
|
+
# +connection_state+ deliberately does NOT trigger this: a diagnostic
|
|
768
|
+
# must never open a socket. It reports +:initializing+ while a rebuild is
|
|
769
|
+
# pending, which is exactly what the client is.
|
|
770
|
+
def ensure_initialized_after_fork
|
|
771
|
+
# Hot path: two ivar reads and no lock. Both are falsy for every client
|
|
772
|
+
# that has never been through a fork.
|
|
773
|
+
return unless @fork_rebuild_pending || @fork_rebuild_error
|
|
774
|
+
# Re-entrancy guard: a customer logger (SemanticLoggerFilter, stdlib
|
|
775
|
+
# formatter) that evaluates a config from inside the rebuild would
|
|
776
|
+
# otherwise deadlock on the non-reentrant Mutex. Such a call sees the
|
|
777
|
+
# half-built client, which is the same thing Client.new gives a logger
|
|
778
|
+
# that fires during construction.
|
|
779
|
+
return if @fork_rebuild_owner == Thread.current
|
|
780
|
+
|
|
781
|
+
run_pending_child_rebuild if @fork_rebuild_pending
|
|
782
|
+
raise_sticky_fork_init_error if @fork_rebuild_error
|
|
783
|
+
end
|
|
784
|
+
|
|
785
|
+
# Run the rebuild under the lock, or block until whoever is running it is
|
|
786
|
+
# done. The flag stays TRUE for the whole rebuild, which is what makes
|
|
787
|
+
# every other first-use caller take the mutex and WAIT rather than sail
|
|
788
|
+
# past on the unlocked fast path and evaluate against the empty store.
|
|
789
|
+
def run_pending_child_rebuild
|
|
790
|
+
@fork_rebuild_mutex.synchronize do
|
|
791
|
+
# Lost the race: the winner already rebuilt (or `stop` disarmed us).
|
|
792
|
+
return unless @fork_rebuild_pending
|
|
793
|
+
return if @stopped
|
|
794
|
+
|
|
795
|
+
@fork_rebuild_owner = Thread.current
|
|
796
|
+
begin
|
|
797
|
+
rebuild_in_child!
|
|
798
|
+
rescue StandardError => e
|
|
799
|
+
# Handled: the child gets whatever healing path its mode allows, so
|
|
800
|
+
# the next lookup must not re-run the blocking fetch. (The datadir
|
|
801
|
+
# recovery path may deliberately re-arm — see
|
|
802
|
+
# #recover_datadir_child_after_failed_rebuild.)
|
|
803
|
+
@fork_rebuild_pending = false
|
|
804
|
+
handle_child_rebuild_failure(e)
|
|
805
|
+
ensure
|
|
806
|
+
@fork_rebuild_owner = nil
|
|
807
|
+
# #rebuild_in_child! disarms the flag itself the moment the child
|
|
808
|
+
# has a live path to config. Anything that escapes before that —
|
|
809
|
+
# including a non-StandardError such as rack-timeout's
|
|
810
|
+
# RequestTimeoutException, Ruby 3.3's Timeout::ExitException, or a
|
|
811
|
+
# Thread#kill, none of which the rescue above can see — leaves the
|
|
812
|
+
# flag armed so the NEXT call retries instead of leaving the child
|
|
813
|
+
# dark forever (qfg-lv4n.1 D2).
|
|
814
|
+
@fork_rebuild_pending = false if @stopped
|
|
815
|
+
end
|
|
816
|
+
end
|
|
817
|
+
end
|
|
818
|
+
|
|
819
|
+
# Under +on_init_failure: :raise+ a failed rebuild raises out of the call
|
|
820
|
+
# that triggered it, exactly as +Client.new+ would — and keeps raising on
|
|
821
|
+
# subsequent calls (without re-fetching) until the update channel heals
|
|
822
|
+
# the store. Under +:return+ nothing is stored here and this never fires.
|
|
823
|
+
def raise_sticky_fork_init_error
|
|
824
|
+
err = @fork_rebuild_error
|
|
825
|
+
return if err.nil?
|
|
826
|
+
|
|
827
|
+
if ready?
|
|
828
|
+
# The SSE stream (or the poller) installed an envelope: the client is
|
|
829
|
+
# serving real config again, so the init failure is history.
|
|
830
|
+
@fork_rebuild_error = nil
|
|
831
|
+
return
|
|
832
|
+
end
|
|
833
|
+
|
|
834
|
+
raise err
|
|
835
|
+
end
|
|
836
|
+
|
|
837
|
+
# A rebuild that raised. Log it, give the child whatever healing path its
|
|
838
|
+
# mode has, and honor +on_init_failure+.
|
|
839
|
+
def handle_child_rebuild_failure(err)
|
|
840
|
+
LOG.error "[quonfig] post-fork re-initialization failed: #{err.class}: #{err.message}"
|
|
841
|
+
|
|
842
|
+
if @options.datadir
|
|
843
|
+
recover_datadir_child_after_failed_rebuild
|
|
844
|
+
else
|
|
845
|
+
begin
|
|
846
|
+
start_update_channel if @sse_client.nil? && @poll_supervisor.nil?
|
|
847
|
+
rescue StandardError => e
|
|
848
|
+
LOG.error "[quonfig] post-fork update channel failed to start: #{e.class}: #{e.message}"
|
|
849
|
+
end
|
|
850
|
+
end
|
|
851
|
+
|
|
852
|
+
return unless @options.on_init_failure == Quonfig::Options::ON_INITIALIZATION_FAILURE::RAISE
|
|
853
|
+
|
|
854
|
+
# Parity with a fresh Client.new, which raises under :raise. Stored so
|
|
855
|
+
# later calls keep raising rather than re-running the fetch on every
|
|
856
|
+
# lookup.
|
|
857
|
+
@fork_rebuild_error = err
|
|
858
|
+
raise err
|
|
859
|
+
end
|
|
860
|
+
|
|
861
|
+
# A datadir client is configured OFFLINE: it has no config loader, so
|
|
862
|
+
# opening an SSE stream here dials the network on a customer who asked
|
|
863
|
+
# for none and then blows up on every envelope that arrives
|
|
864
|
+
# ("undefined method `apply_envelope' for nil"). The healing path for a
|
|
865
|
+
# datadir child is the filesystem: start the watcher if auto-reload is on
|
|
866
|
+
# so a repaired workspace is picked up, and otherwise re-arm the rebuild
|
|
867
|
+
# so the next use retries the load (qfg-lv4n.1 D3).
|
|
868
|
+
def recover_datadir_child_after_failed_rebuild
|
|
869
|
+
begin
|
|
870
|
+
start_datadir_watcher if @options.data_dir_auto_reload && @datadir_watcher.nil?
|
|
871
|
+
rescue StandardError => e
|
|
872
|
+
LOG.error "[quonfig] post-fork datadir watcher failed to start: #{e.class}: #{e.message}"
|
|
873
|
+
end
|
|
874
|
+
|
|
875
|
+
return unless @datadir_watcher.nil?
|
|
876
|
+
|
|
877
|
+
@fork_rebuild_pending = true
|
|
878
|
+
end
|
|
879
|
+
|
|
880
|
+
# The child's own re-initialization, run on first use. Mirrors what
|
|
881
|
+
# +Client.new+ does for this client's mode, and logs one info line so a
|
|
882
|
+
# customer grepping their logs can see the SDK noticed the fork.
|
|
883
|
+
def rebuild_in_child!
|
|
884
|
+
# This process is taking ownership of the client. Re-stamping here is
|
|
885
|
+
# what keeps #owned_by_this_process? exact for everything that follows:
|
|
886
|
+
# a manual +after_fork_in_child+ in THIS child is now correctly a
|
|
887
|
+
# no-op, and a grandchild forked from here is still detected by pid.
|
|
888
|
+
@owner_pid = Process.pid
|
|
889
|
+
components = []
|
|
890
|
+
|
|
891
|
+
if @options.datadir
|
|
892
|
+
load_datadir_into_store
|
|
893
|
+
components << 'datadir'
|
|
894
|
+
start_datadir_watcher if @options.data_dir_auto_reload
|
|
895
|
+
components << 'datadir-watcher' if @datadir_watcher
|
|
896
|
+
else
|
|
897
|
+
initialize_network_mode
|
|
898
|
+
components << 'config' if ready?
|
|
899
|
+
components << 'sse' if @sse_client
|
|
900
|
+
components << 'polling' if @poll_supervisor
|
|
901
|
+
end
|
|
902
|
+
|
|
903
|
+
# The child now has a live path to config (a stream/poller, or a loaded
|
|
904
|
+
# datadir). Disarm HERE, not in the caller: everything above is
|
|
905
|
+
# retryable and must stay armed if it is interrupted, and everything
|
|
906
|
+
# below must never re-run the fetch or dial a second stream.
|
|
907
|
+
@fork_rebuild_pending = false
|
|
908
|
+
|
|
909
|
+
unless @stopped
|
|
910
|
+
@telemetry_reporter&.start
|
|
911
|
+
components << 'telemetry' if @telemetry_reporter
|
|
912
|
+
end
|
|
913
|
+
|
|
914
|
+
log_child_rebuild(components)
|
|
915
|
+
end
|
|
916
|
+
|
|
917
|
+
# A brand-new store (plus the evaluator, resolver, and config loader that
|
|
918
|
+
# read it) so the child starts from nothing and installs its own envelope.
|
|
919
|
+
# Two things this buys beyond "no stale config": the child's first
|
|
920
|
+
# envelope is ACCEPTED rather than dropped by the reject-older guard as
|
|
921
|
+
# same-generation, and a fork that lands mid-install can no longer hand
|
|
922
|
+
# the child a half-written store.
|
|
923
|
+
def reset_store_in_child!
|
|
924
|
+
@store = Quonfig::ConfigStore.new
|
|
925
|
+
@evaluator = Quonfig::Evaluator.new(@store, env_id: @options.environment)
|
|
926
|
+
@resolver = Quonfig::Resolver.new(@store, @evaluator)
|
|
927
|
+
@last_successful_refresh = nil
|
|
928
|
+
return if @options.datadir
|
|
929
|
+
|
|
930
|
+
@config_loader = Quonfig::ConfigLoader.new(@store, @options, failover_aggregator: @failover_aggregator)
|
|
931
|
+
end
|
|
932
|
+
|
|
933
|
+
# Replace every aggregator with a fresh, empty one so the child never
|
|
934
|
+
# re-reports data the parent collected (and is still going to flush from
|
|
935
|
+
# its own copy). Mirrors what a brand-new Client.new would allocate.
|
|
936
|
+
def rebuild_aggregators_in_child!
|
|
937
|
+
@failover_aggregator = Quonfig::Telemetry::FailoverAggregator.new
|
|
938
|
+
# The ConfigLoader records hedge/guard/resolved-from at its failover
|
|
939
|
+
# call sites, so it has to point at the child's aggregator too — the
|
|
940
|
+
# inherited one is now the parent's private object.
|
|
941
|
+
@config_loader.failover_aggregator = @failover_aggregator if @config_loader.respond_to?(:failover_aggregator=)
|
|
942
|
+
|
|
943
|
+
# initialize_telemetry allocates fresh context/example/summaries
|
|
944
|
+
# aggregators and a fresh reporter. It is NOT started here: starting the
|
|
945
|
+
# reporter is I/O and a thread, and both wait for the child's first use.
|
|
946
|
+
@telemetry_reporter = nil
|
|
947
|
+
initialize_telemetry(start: false)
|
|
948
|
+
end
|
|
949
|
+
|
|
950
|
+
# One line, at info, so a customer can see in their logs that the SDK
|
|
951
|
+
# noticed the fork and rebuilt. Deliberately not a warning: forking is
|
|
952
|
+
# normal and expected.
|
|
953
|
+
def log_child_rebuild(components)
|
|
954
|
+
list = components.empty? ? 'none' : components.join(',')
|
|
955
|
+
LOG.info "[quonfig] re-initialized after fork pid=#{Process.pid} components=#{list}"
|
|
514
956
|
end
|
|
515
957
|
|
|
516
958
|
# Stamp +last_successful_refresh+ at install time. Called by every code
|
|
@@ -668,7 +1110,7 @@ module Quonfig
|
|
|
668
1110
|
# Construct and start the telemetry reporter if the options permit it.
|
|
669
1111
|
# The reporter runs on a background thread and periodically POSTs
|
|
670
1112
|
# context-shape and example-context batches to +telemetry_destination+.
|
|
671
|
-
def initialize_telemetry
|
|
1113
|
+
def initialize_telemetry(start: true)
|
|
672
1114
|
shape_aggregator = nil
|
|
673
1115
|
example_aggregator = nil
|
|
674
1116
|
summaries_aggregator = nil
|
|
@@ -704,6 +1146,7 @@ module Quonfig
|
|
|
704
1146
|
)
|
|
705
1147
|
|
|
706
1148
|
return unless @telemetry_reporter.enabled?
|
|
1149
|
+
return unless start
|
|
707
1150
|
|
|
708
1151
|
@telemetry_reporter.start
|
|
709
1152
|
rescue StandardError => e
|
|
@@ -826,15 +1269,32 @@ module Quonfig
|
|
|
826
1269
|
warn_if_hedge_abort_exceeds_init_timeout
|
|
827
1270
|
warn_if_explicit_api_urls_disables_failover
|
|
828
1271
|
|
|
829
|
-
|
|
1272
|
+
# ||=: after a fork the child already built its loader over its fresh
|
|
1273
|
+
# store (see #reset_store_in_child!).
|
|
1274
|
+
@config_loader ||= Quonfig::ConfigLoader.new(@store, @options, failover_aggregator: @failover_aggregator)
|
|
830
1275
|
|
|
831
1276
|
perform_initial_fetch
|
|
1277
|
+
start_update_channel
|
|
1278
|
+
end
|
|
832
1279
|
|
|
833
|
-
|
|
1280
|
+
# SSE if enabled and it comes up; otherwise the HTTP polling fallback.
|
|
1281
|
+
# Polling is a fallback: if SSE is off or failed to start, poll. This
|
|
1282
|
+
# avoids double-work when SSE is healthy but still refreshes the store in
|
|
1283
|
+
# environments that block SSE (corporate proxies, Lambda, etc.).
|
|
1284
|
+
def start_update_channel
|
|
1285
|
+
# A `stop` that raced the post-fork rebuild must win: never dial a
|
|
1286
|
+
# stream for a client the customer has already killed (qfg-lv4n.1).
|
|
1287
|
+
return if @stopped
|
|
1288
|
+
# Idempotent. `rebuild_in_child!` disarms @fork_rebuild_pending only
|
|
1289
|
+
# AFTER initialize_network_mode has already started the channel, so a
|
|
1290
|
+
# non-StandardError landing in that window (rack-timeout,
|
|
1291
|
+
# Timeout::ExitException, Thread#kill) leaves the flag armed WITH a
|
|
1292
|
+
# live stream. The retry re-runs initialize_network_mode; without this
|
|
1293
|
+
# guard it dialled a SECOND stream and overwrote @sse_client, orphaning
|
|
1294
|
+
# the first worker where `stop` could never reach it (qfg-lv4n.1 E2).
|
|
1295
|
+
return if sse_worker_alive? || @poll_supervisor&.alive?
|
|
834
1296
|
|
|
835
|
-
|
|
836
|
-
# avoids double-work when SSE is healthy but still refreshes the store
|
|
837
|
-
# in environments that block SSE (corporate proxies, Lambda, etc.).
|
|
1297
|
+
sse_started = @options.enable_sse && start_sse
|
|
838
1298
|
start_polling if @options.enable_polling && !sse_started
|
|
839
1299
|
end
|
|
840
1300
|
|
|
@@ -959,6 +1419,7 @@ module Quonfig
|
|
|
959
1419
|
# Returns true if SSE started successfully, false otherwise. A false here
|
|
960
1420
|
# signals the caller to fall back to polling.
|
|
961
1421
|
def start_sse
|
|
1422
|
+
return false if @stopped
|
|
962
1423
|
return false if @options.sse_api_urls.nil? || @options.sse_api_urls.empty?
|
|
963
1424
|
|
|
964
1425
|
@sse_client = Quonfig::SSEConfigClient.new(
|
|
@@ -1143,6 +1604,7 @@ module Quonfig
|
|
|
1143
1604
|
# caller's context, after coercing/checking +expected_type+. Never
|
|
1144
1605
|
# raises; all exceptions become ERROR details.
|
|
1145
1606
|
def evaluate_details(key, expected_type, context)
|
|
1607
|
+
ensure_initialized_after_fork
|
|
1146
1608
|
jit = context == NO_DEFAULT_PROVIDED ? nil : context
|
|
1147
1609
|
ctx = build_context(jit)
|
|
1148
1610
|
record_context_for_telemetry(ctx)
|
|
@@ -1282,32 +1744,52 @@ module Quonfig
|
|
|
1282
1744
|
end
|
|
1283
1745
|
end
|
|
1284
1746
|
|
|
1285
|
-
# qfg-ryov: hook into Process._fork so customers using Puma's
|
|
1286
|
-
# mode (or any preload/fork-worker server
|
|
1287
|
-
# +before_fork+/+on_worker_boot+ manually.
|
|
1288
|
-
# +Kernel#fork+/+Process.fork+ call through
|
|
1289
|
-
# prepend covers them all.
|
|
1747
|
+
# qfg-ryov / qfg-lv4n.1: hook into Process._fork so customers using Puma's
|
|
1748
|
+
# clustered mode (or any preload/fork-worker server, or a gem that forks
|
|
1749
|
+
# inside a job) don't have to wire +before_fork+/+on_worker_boot+ manually.
|
|
1750
|
+
# Ruby 3.1+ routes every +Kernel#fork+/+Process.fork+ call through
|
|
1751
|
+
# +Process._fork+, so a single prepend covers them all.
|
|
1290
1752
|
#
|
|
1291
1753
|
# Process._fork's contract:
|
|
1292
1754
|
# - Called in the parent process before the fork syscall.
|
|
1293
1755
|
# - Returns 0 in the child, child's pid in the parent.
|
|
1294
1756
|
# - +super+ performs the actual fork.
|
|
1295
1757
|
#
|
|
1296
|
-
# The
|
|
1297
|
-
# the syscall
|
|
1298
|
-
#
|
|
1299
|
-
#
|
|
1300
|
-
#
|
|
1758
|
+
# **The hook is child-only.** Nothing happens in the parent — not before
|
|
1759
|
+
# the syscall, not after it. A fork is somebody else's business; the
|
|
1760
|
+
# process that forked keeps its SSE stream, its poller, its telemetry
|
|
1761
|
+
# reporter, and its live config. This is Reforge's model (+Reforge.fork+
|
|
1762
|
+
# builds a new client in the child and never touches the old one) and
|
|
1763
|
+
# matches dd-trace-rb, redis-client and connection_pool, which all branch
|
|
1764
|
+
# on the child stage of +_fork+ only.
|
|
1765
|
+
#
|
|
1766
|
+
# It replaces the qfg-ryov shape, which tore the parent down before the
|
|
1767
|
+
# syscall on the theory that the child must not inherit a live socket fd.
|
|
1768
|
+
# That was wrong twice over: the child never touches the inherited fd (it
|
|
1769
|
+
# drops the reference — see Client#after_fork_in_child), and a Sidekiq
|
|
1770
|
+
# parent that forks a worker and keeps evaluating went dark for 13 days in
|
|
1771
|
+
# production.
|
|
1301
1772
|
module ForkSafety
|
|
1302
1773
|
def _fork
|
|
1303
|
-
Quonfig::Client.each_instance(&:before_fork_in_parent)
|
|
1304
1774
|
pid = super
|
|
1305
|
-
|
|
1775
|
+
if pid.zero?
|
|
1776
|
+
# Per-instance, not per-fan-out: a process can hold more than one
|
|
1777
|
+
# Client (a second workspace, a test harness, a gem that builds its
|
|
1778
|
+
# own). One of them failing to rebuild — thread exhaustion, a
|
|
1779
|
+
# customer logger that raises — must not cost every client behind it
|
|
1780
|
+
# in the registry its rebuild and leave the child silently dark.
|
|
1781
|
+
Quonfig::Client.each_instance do |client|
|
|
1782
|
+
client.after_fork_in_child
|
|
1783
|
+
rescue StandardError => e
|
|
1784
|
+
Quonfig::Client::LOG.error 'Quonfig fork rebuild failed for one client ' \
|
|
1785
|
+
"(continuing with the rest): #{e.class}: #{e.message}"
|
|
1786
|
+
end
|
|
1787
|
+
end
|
|
1306
1788
|
pid
|
|
1307
1789
|
rescue StandardError => e
|
|
1308
1790
|
# Fork-hook failures must never break the customer's fork. Worst case
|
|
1309
|
-
# the child
|
|
1310
|
-
#
|
|
1791
|
+
# the child holds dropped references and no live threads — bad, but
|
|
1792
|
+
# recoverable. Crashing the fork itself is not.
|
|
1311
1793
|
Quonfig::Client::LOG.error "Quonfig fork hook error: #{e.class}: #{e.message}"
|
|
1312
1794
|
raise if pid.nil? # super never returned — propagate fork failures
|
|
1313
1795
|
|
|
@@ -31,6 +31,12 @@ module Quonfig
|
|
|
31
31
|
# SSE does not change it).
|
|
32
32
|
attr_reader :held_generation, :install_count
|
|
33
33
|
|
|
34
|
+
# Repoint the failover-telemetry sink. The one caller is
|
|
35
|
+
# Client#after_fork_in_child: a forked child gets fresh aggregators (the
|
|
36
|
+
# parent still owns and flushes its own), and the loader has to record
|
|
37
|
+
# into the child's copy rather than the inherited one (qfg-lv4n.1).
|
|
38
|
+
attr_writer :failover_aggregator
|
|
39
|
+
|
|
34
40
|
# +store+: the Quonfig::ConfigStore to populate on successful fetch.
|
|
35
41
|
# +options+: a Quonfig::Options instance (supplies sdk_key + config_api_urls).
|
|
36
42
|
# +logger+: optional logger override (defaults to module LOG).
|
|
@@ -129,6 +129,15 @@ module Quonfig
|
|
|
129
129
|
@worker = Thread.new { run_loop(&on_envelope) }
|
|
130
130
|
end
|
|
131
131
|
|
|
132
|
+
# True while the worker thread that owns the stream (and its reconnect
|
|
133
|
+
# loop) is running. Stays true across a reconnect — the worker owns the
|
|
134
|
+
# retry, so a blip is not "no channel". Used by Client#connection_state
|
|
135
|
+
# so that diagnostic derives from liveness rather than a stored flag
|
|
136
|
+
# (qfg-lv4n.1).
|
|
137
|
+
def alive?
|
|
138
|
+
@worker&.alive? || false
|
|
139
|
+
end
|
|
140
|
+
|
|
132
141
|
# Shut down. Interrupts the in-flight stream by closing the underlying
|
|
133
142
|
# socket from this thread — the worker thread observes the resulting
|
|
134
143
|
# IOError, sees @stopped == true, and exits cleanly.
|
|
@@ -51,6 +51,9 @@ module Quonfig
|
|
|
51
51
|
@stopped = Concurrent::AtomicBoolean.new(false)
|
|
52
52
|
@thread = nil
|
|
53
53
|
@at_exit_registered = false
|
|
54
|
+
# Set on #start. Everything that can EMIT is gated on it so a forked
|
|
55
|
+
# child never speaks for the process that created this reporter.
|
|
56
|
+
@owner_pid = nil
|
|
54
57
|
end
|
|
55
58
|
|
|
56
59
|
def enabled?
|
|
@@ -80,6 +83,11 @@ module Quonfig
|
|
|
80
83
|
return if @thread&.alive?
|
|
81
84
|
return unless enabled?
|
|
82
85
|
|
|
86
|
+
# Claim ownership for THIS process. fork(2) copies the reporter, its
|
|
87
|
+
# aggregators, and the process-wide at_exit closure registered below;
|
|
88
|
+
# the pid recorded here is what lets the copy know it is not the
|
|
89
|
+
# owner and must stay silent (qfg-lv4n.1, dd-trace-rb's pattern).
|
|
90
|
+
@owner_pid = Process.pid
|
|
83
91
|
@stopped.make_false
|
|
84
92
|
register_at_exit_handler
|
|
85
93
|
@thread = Thread.new do
|
|
@@ -106,6 +114,8 @@ module Quonfig
|
|
|
106
114
|
end
|
|
107
115
|
|
|
108
116
|
def stop
|
|
117
|
+
return if foreign_process?('stop')
|
|
118
|
+
|
|
109
119
|
@stopped.make_true
|
|
110
120
|
thread = @thread
|
|
111
121
|
@thread = nil
|
|
@@ -121,7 +131,13 @@ module Quonfig
|
|
|
121
131
|
|
|
122
132
|
# Drain all aggregators and POST the batch. Public so tests can
|
|
123
133
|
# trigger a sync without waiting for the background loop.
|
|
134
|
+
#
|
|
135
|
+
# Silent in any process other than the one that started the reporter:
|
|
136
|
+
# after a fork the child holds a full copy of the PARENT's un-flushed
|
|
137
|
+
# window, and the parent is still going to flush it itself.
|
|
124
138
|
def sync
|
|
139
|
+
return if foreign_process?('sync')
|
|
140
|
+
|
|
125
141
|
events = []
|
|
126
142
|
if (summaries_event = @evaluation_summaries_aggregator&.drain_event)
|
|
127
143
|
events << summaries_event
|
|
@@ -151,8 +167,41 @@ module Quonfig
|
|
|
151
167
|
@at_exit_registered
|
|
152
168
|
end
|
|
153
169
|
|
|
170
|
+
# Pid of the process that started this reporter, or nil if it was never
|
|
171
|
+
# started. Visible for tests / diagnostics.
|
|
172
|
+
attr_reader :owner_pid
|
|
173
|
+
|
|
174
|
+
# Called on the INHERITED reporter in a forked child, from
|
|
175
|
+
# +Quonfig::Client#after_fork_in_child+, once the child has dropped its
|
|
176
|
+
# reference to it. Makes the copied window unreachable so nothing can
|
|
177
|
+
# ever emit it — belt to the +@owner_pid+ braces.
|
|
178
|
+
#
|
|
179
|
+
# Deliberately does NOT stop, close, or join anything: the thread does
|
|
180
|
+
# not exist in the child, and the HTTP connection's fd is shared with
|
|
181
|
+
# the parent.
|
|
182
|
+
def discard_inherited!
|
|
183
|
+
@stopped.make_true
|
|
184
|
+
@thread = nil
|
|
185
|
+
@context_shape_aggregator = nil
|
|
186
|
+
@example_contexts_aggregator = nil
|
|
187
|
+
@evaluation_summaries_aggregator = nil
|
|
188
|
+
@failover_aggregator = nil
|
|
189
|
+
end
|
|
190
|
+
|
|
154
191
|
private
|
|
155
192
|
|
|
193
|
+
# True when this reporter belongs to a different process — i.e. we are
|
|
194
|
+
# a fork(2) copy. Never true before #start (nothing has been claimed,
|
|
195
|
+
# and nothing was registered at_exit either).
|
|
196
|
+
def foreign_process?(operation)
|
|
197
|
+
return false if @owner_pid.nil?
|
|
198
|
+
return false if @owner_pid == Process.pid
|
|
199
|
+
|
|
200
|
+
LOG.debug "[quonfig] Telemetry #{operation} skipped in forked child " \
|
|
201
|
+
"pid=#{Process.pid} owner_pid=#{@owner_pid}"
|
|
202
|
+
true
|
|
203
|
+
end
|
|
204
|
+
|
|
156
205
|
# Rails / Passenger / Puma workers often terminate via SIGTERM without
|
|
157
206
|
# a chance to call Client#stop. Register a Kernel.at_exit hook on
|
|
158
207
|
# first start so the in-flight batch still gets flushed.
|
|
@@ -174,6 +223,8 @@ module Quonfig
|
|
|
174
223
|
# no-op. Bounded so a stuck reporter thread or dead telemetry
|
|
175
224
|
# endpoint can't hang process exit.
|
|
176
225
|
def final_drain_on_exit
|
|
226
|
+
return if foreign_process?('at_exit drain')
|
|
227
|
+
|
|
177
228
|
@stopped.make_true
|
|
178
229
|
thread = @thread
|
|
179
230
|
@thread = nil
|
data/lib/quonfig/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: quonfig
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Jeff Dwyer
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-09-11 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activesupport
|