rails_pod_kit 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +362 -0
- data/VERSION +1 -0
- data/lib/rails_pod_kit/config.rb +118 -0
- data/lib/rails_pod_kit/global_exporter.rb +77 -0
- data/lib/rails_pod_kit/health.rb +101 -0
- data/lib/rails_pod_kit/puma.rb +72 -0
- data/lib/rails_pod_kit/railtie.rb +22 -0
- data/lib/rails_pod_kit/sidekiq.rb +99 -0
- data/lib/rails_pod_kit/version.rb +5 -0
- data/lib/rails_pod_kit.rb +48 -0
- metadata +172 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: fb47aab3b005492f13eaaa6ede0a89a5891cfbc8d034ebc0f1284b590efbb59a
|
|
4
|
+
data.tar.gz: 3f0f50b521bb1f96655edcbb296ea00156f90b631239491c49fad381f351989e
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 33733cd886176b0341314e9ba6641cb0368f6010e3d24e0b11851d8a18d32119dd183eb8f17a885d8be5f4c5e36d7b123205d5051bf79b90e8f0cc95b434d0ef
|
|
7
|
+
data.tar.gz: db116a08eb1b3fb3c0d4017a5be0082568e48907032717d35dbb10933f741811b0009c64749937747aa44e45db0dbb73a1647d288f1dca986c6ec8007cc4531e
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Fabio Napoleoni
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,362 @@
|
|
|
1
|
+
# rails_pod_kit
|
|
2
|
+
|
|
3
|
+
The operational endpoints a Rails pod needs to be a good Kubernetes citizen,
|
|
4
|
+
packaged behind a single, opinionated entry point:
|
|
5
|
+
|
|
6
|
+
- **Prometheus metrics** for Puma and Sidekiq, served **in-process** on a
|
|
7
|
+
single `/metrics` endpoint (default port **9394**) in both the web (Puma) and
|
|
8
|
+
worker (Sidekiq) processes — no sidecar, no separate collector process. A
|
|
9
|
+
metrics agent (e.g. the Datadog Agent via OpenMetrics autodiscovery) scrapes
|
|
10
|
+
the pod directly.
|
|
11
|
+
- **Health checks** on `/healthz` (database, cache, Redis, optionally Sidekiq),
|
|
12
|
+
wired for Kubernetes startup/liveness/readiness probes — a thin, opinionated
|
|
13
|
+
wrapper around [health-monitor-rails](https://github.com/lbeder/health-monitor-rails).
|
|
14
|
+
|
|
15
|
+
The metrics side is a thin wrapper around the
|
|
16
|
+
[yabeda](https://github.com/yabeda-rb) ecosystem:
|
|
17
|
+
|
|
18
|
+
| gem | what it gives us |
|
|
19
|
+
|-----|------------------|
|
|
20
|
+
| `yabeda` | in-process metrics registry |
|
|
21
|
+
| `yabeda-puma-plugin` | Puma thread-pool / worker stats + the `:yabeda` / `:yabeda_prometheus` Puma plugins |
|
|
22
|
+
| `yabeda-sidekiq` | Sidekiq per-process job metrics + global/Redis-wide queue metrics |
|
|
23
|
+
| `yabeda-prometheus-mmap` | Prometheus text exporter, multiprocess-safe via `prometheus-client-mmap` |
|
|
24
|
+
| `webrick` | HTTP server for the Sidekiq exporter |
|
|
25
|
+
|
|
26
|
+
> Scope: **runtime/worker metrics only** (Puma backlog/threads, Sidekiq queues
|
|
27
|
+
> and jobs). HTTP request-level metrics are intentionally out of scope — that is
|
|
28
|
+
> covered by APM.
|
|
29
|
+
|
|
30
|
+
The gem is deliberately **connection-agnostic**: it never reads `REDIS_URL` and
|
|
31
|
+
makes no TLS decisions. Wherever a Redis connection is needed (health checks,
|
|
32
|
+
the dedicated global exporter) the host injects its own options, so connection
|
|
33
|
+
config keeps a single, host-owned source of truth.
|
|
34
|
+
|
|
35
|
+
## Wiring
|
|
36
|
+
|
|
37
|
+
The host app wires the gem in three one-liners. Every metrics hook is a
|
|
38
|
+
complete no-op when the exporter is disabled or in the `test` environment, so
|
|
39
|
+
the suite never binds the port. The integrations wire themselves at
|
|
40
|
+
`Bundler.require` and the gem's Railtie mounts the health endpoint — no
|
|
41
|
+
`require:` gymnastics in the Gemfile, no routes change.
|
|
42
|
+
|
|
43
|
+
**1. `config/initializers/rails_pod_kit.rb`** — configure once:
|
|
44
|
+
|
|
45
|
+
```ruby
|
|
46
|
+
RailsPodKit.configure do |c|
|
|
47
|
+
c.enabled = !Rails.env.test?
|
|
48
|
+
c.port = Integer(ENV.fetch('PROMETHEUS_EXPORTER_PORT', 9394))
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
RailsPodKit::Health.install!(
|
|
52
|
+
redis: { url: ENV['REDIS_URL'] }, # the host's own Redis options
|
|
53
|
+
sidekiq: { queue_size: 200, latency: 10.minutes } # omit on hosts without Sidekiq
|
|
54
|
+
)
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
**2. `config/puma.rb`** — activate the Puma plugins:
|
|
58
|
+
|
|
59
|
+
```ruby
|
|
60
|
+
require 'rails_pod_kit/puma'
|
|
61
|
+
RailsPodKit::Puma.activate(self)
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
**3. `config/initializers/sidekiq.rb`** (inside `Sidekiq.configure_server`):
|
|
65
|
+
|
|
66
|
+
```ruby
|
|
67
|
+
RailsPodKit::Sidekiq.install!(config)
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Configuration
|
|
71
|
+
|
|
72
|
+
`RailsPodKit::Config` is an [anyway_config](https://github.com/palkan/anyway_config)
|
|
73
|
+
config: besides the `RailsPodKit.configure` block (which runs last and wins),
|
|
74
|
+
every setting can come from a `RAILS_POD_KIT_*` env var (e.g.
|
|
75
|
+
`RAILS_POD_KIT_ENABLED=false`, `RAILS_POD_KIT_PORT=9500`,
|
|
76
|
+
`RAILS_POD_KIT_SIDEKIQ_GLOBAL_METRICS=off`) or an optional
|
|
77
|
+
`config/rails_pod_kit.yml` — handy for the Rails-free exporter pod, which runs
|
|
78
|
+
no initializers.
|
|
79
|
+
|
|
80
|
+
| setting | default | meaning |
|
|
81
|
+
|---------|---------|---------|
|
|
82
|
+
| `enabled` | on, except in `test` | master switch; `false` ⇒ no exporter, no port bound |
|
|
83
|
+
| `port` | `9394` (env `PROMETHEUS_EXPORTER_PORT`) | exporter bind port for Puma **and** Sidekiq |
|
|
84
|
+
| `sidekiq_global_metrics` | `:web` | who exports the Redis-wide queue metrics: `:web` = only the always-on web process (no per-worker duplication); `:all` = every worker; `:off` = nobody |
|
|
85
|
+
| `puma_control_url` | `tcp://127.0.0.1:9293` (env `PUMA_CONTROL_URL`) | localhost-only Puma control app the stats reader queries |
|
|
86
|
+
| `retries_segmented_by_queue` | `false` | when `true`, the global retry gauge (`sidekiq_jobs_retry_count`) carries a `queue` tag instead of being a single scalar. Opt-in: per-queue retries increase series cardinality. |
|
|
87
|
+
| `silence_exporter_access_log` | `true` | suppress the exporter's per-scrape `GET /metrics` access log on **both** transports (Puma plugin + WEBrick). The endpoint is pod-internal and scraped on a fixed interval, so the line is pure noise. Set `false` only to debug the exporter. |
|
|
88
|
+
|
|
89
|
+
## Health checks (`RailsPodKit::Health`)
|
|
90
|
+
|
|
91
|
+
`Health.install!` configures health-monitor-rails with the kit's defaults:
|
|
92
|
+
endpoint at `/healthz`, checking **database** (health_monitor's default),
|
|
93
|
+
**cache**, **Redis** (connection injected by the host) and — when a `sidekiq:`
|
|
94
|
+
thresholds hash is given — **Sidekiq**. The gem's Railtie mounts
|
|
95
|
+
`HealthMonitor::Engine` at `/` automatically; pass `mount: false` to keep
|
|
96
|
+
route ownership (custom mount point, constraints) and mount it yourself in
|
|
97
|
+
`config/routes.rb`. Any further tuning goes through the optional block, which
|
|
98
|
+
receives the `HealthMonitor` configuration.
|
|
99
|
+
|
|
100
|
+
```ruby
|
|
101
|
+
RailsPodKit::Health.install!(
|
|
102
|
+
redis: { url: ENV['REDIS_URL'] }, # or a ready Redis/ConnectionPool object
|
|
103
|
+
path: :healthz, # default
|
|
104
|
+
sidekiq: { queue_size: 200, latency: 10.minutes }
|
|
105
|
+
) do |config|
|
|
106
|
+
config.error_callback = ->(e) { ... } # host-specific extras
|
|
107
|
+
end
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Probe wiring on Kubernetes:
|
|
111
|
+
|
|
112
|
+
- **startup probe** → `/healthz` — the full check, so the pod doesn't go ready
|
|
113
|
+
until its dependencies are actually up;
|
|
114
|
+
- **liveness/readiness probes** → `/healthz?providers[]=none` — short-circuits
|
|
115
|
+
the dependency checks once the pod is live, so a transient Redis hiccup
|
|
116
|
+
doesn't restart the app.
|
|
117
|
+
|
|
118
|
+
The same `providers[]=none` trick works for a manual basic check:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
curl '/healthz?providers[]=none' -H 'Accept: application/json'
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
The per-probe controller logging is silenced by default (kubelet probes hit the
|
|
125
|
+
endpoint every few seconds); pass `silence_controller_log: false` to keep it.
|
|
126
|
+
|
|
127
|
+
## Metrics exposed on `/metrics`
|
|
128
|
+
|
|
129
|
+
- **Puma:** `puma_workers`, `puma_running`, `puma_pool_capacity`,
|
|
130
|
+
`puma_max_threads`, `puma_backlog`, `puma_busy_threads`,
|
|
131
|
+
`puma_requests_count`, …
|
|
132
|
+
- **Sidekiq (per-process):** `sidekiq_jobs_executed_total`,
|
|
133
|
+
`sidekiq_jobs_success_total`, `sidekiq_jobs_failed_total`,
|
|
134
|
+
`sidekiq_job_runtime_seconds`, `sidekiq_job_latency_seconds`,
|
|
135
|
+
`sidekiq_running_job_runtime_seconds`, …
|
|
136
|
+
- **Sidekiq (global/Redis-wide):** `sidekiq_jobs_waiting_count`,
|
|
137
|
+
`sidekiq_queue_latency`, `sidekiq_active_processes`,
|
|
138
|
+
`sidekiq_active_workers_count`, `sidekiq_jobs_retry_count`,
|
|
139
|
+
`sidekiq_jobs_dead_count`, `sidekiq_jobs_scheduled_count`.
|
|
140
|
+
|
|
141
|
+
Series are intentionally **untagged**: a scraping agent adds
|
|
142
|
+
`service`/`env`/`version` and `kube_*` tags at scrape time, so the gem doesn't
|
|
143
|
+
duplicate them.
|
|
144
|
+
|
|
145
|
+
## Datadog naming — the canonical metric set
|
|
146
|
+
|
|
147
|
+
The gem emits the **untagged native Prometheus names** above (`puma_running`,
|
|
148
|
+
`sidekiq_queue_latency`, …). To get a clean, **org-shared** metric set every
|
|
149
|
+
adopter must use the **same Datadog OpenMetrics check config**, so all series
|
|
150
|
+
differ only by `service` / `env` (+ the `kube_*`/`host`/`pid` tags the Agent adds
|
|
151
|
+
at scrape time) and dashboards/monitors are reusable across projects.
|
|
152
|
+
|
|
153
|
+
The check `namespace` prepends `puma.` / `sidekiq.`; left alone that would
|
|
154
|
+
**double-prefix** the native names (`puma.puma_running`). `raw_metric_prefix`
|
|
155
|
+
strips the redundant native segment *before* the namespace is applied, so series
|
|
156
|
+
land under the canonical `puma.<name>` / `sidekiq.<name>`:
|
|
157
|
+
|
|
158
|
+
```yaml
|
|
159
|
+
# Puma (web pod) — :9394/metrics
|
|
160
|
+
instances:
|
|
161
|
+
- openmetrics_endpoint: "http://%%host%%:9394/metrics"
|
|
162
|
+
namespace: "puma"
|
|
163
|
+
raw_metric_prefix: "puma_" # puma_running -> puma.running
|
|
164
|
+
metrics: [".*"]
|
|
165
|
+
tag_by_endpoint: false
|
|
166
|
+
|
|
167
|
+
# Sidekiq (worker pod + dedicated global exporter) — :9394/metrics
|
|
168
|
+
instances:
|
|
169
|
+
- openmetrics_endpoint: "http://%%host%%:9394/metrics"
|
|
170
|
+
namespace: "sidekiq"
|
|
171
|
+
raw_metric_prefix: "sidekiq_" # sidekiq_queue_latency -> sidekiq.queue_latency
|
|
172
|
+
metrics: [".*"]
|
|
173
|
+
tag_by_endpoint: false
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
On Kubernetes this is typically wired as pod-annotation autodiscovery. The
|
|
177
|
+
`openmetrics_endpoint` above uses the Datadog Agent's `%%host%%` autodiscovery
|
|
178
|
+
template (resolves to the pod IP) — a **non-k8s adopter** (a plain `conf.yaml`
|
|
179
|
+
check) swaps `%%host%%:9394` for the real `host:port`; everything else
|
|
180
|
+
(`namespace`, `raw_metric_prefix`, `metrics`) is identical.
|
|
181
|
+
|
|
182
|
+
### Invariant — the endpoint must stay prefix-pure
|
|
183
|
+
|
|
184
|
+
The Datadog check uses `metrics: [".*"]`, which ingests **every** series on the
|
|
185
|
+
endpoint. `raw_metric_prefix` only *strips* the prefix when present — **it does
|
|
186
|
+
not filter**. So the naming scheme above holds only because the `/metrics`
|
|
187
|
+
endpoint exposes **solely** yabeda-registered series, all sharing the
|
|
188
|
+
`puma_` / `sidekiq_` group prefix:
|
|
189
|
+
|
|
190
|
+
- the gem registers only the `:puma` and `:sidekiq` yabeda groups (no process,
|
|
191
|
+
GC, or Ruby-runtime collectors);
|
|
192
|
+
- the exposition serves Yabeda's registry only — the Prometheus client's HTTP
|
|
193
|
+
request collector (`http_*`) is **not** mounted on the exporter.
|
|
194
|
+
|
|
195
|
+
If a series without the `puma_` / `sidekiq_` prefix ever appeared on the
|
|
196
|
+
endpoint, `metrics: [".*"]` would ingest it **un-stripped** under the namespace
|
|
197
|
+
(e.g. `puma.http_requests_total`). **Do not** add cross-cutting metrics (process,
|
|
198
|
+
runtime, HTTP request) to this exporter, and do not mount the Prometheus Rack
|
|
199
|
+
collector on it. If you ever need such metrics, expose them on a separate
|
|
200
|
+
endpoint with its own check rather than polluting this one. Conversely, any new
|
|
201
|
+
metric you *do* add to the `:puma` / `:sidekiq` groups is picked up automatically
|
|
202
|
+
by `[".*"]` — no check change needed.
|
|
203
|
+
|
|
204
|
+
> ⚠️ Renaming the namespace prefix is a **breaking metric rename** — existing
|
|
205
|
+
> dashboards/monitors built on the old `puma.puma_*` / `sidekiq.sidekiq_*` series
|
|
206
|
+
> must be migrated.
|
|
207
|
+
|
|
208
|
+
### Canonical catalog
|
|
209
|
+
|
|
210
|
+
Functional (non-technical) tags only; `service`/`env`/`kube_*`/`pid`/… are added
|
|
211
|
+
by the Agent. `index` = Puma worker index (`0` in single-mode); `worker` = the
|
|
212
|
+
Sidekiq job class.
|
|
213
|
+
|
|
214
|
+
**Puma** (web pod, `namespace: puma`):
|
|
215
|
+
|
|
216
|
+
| canonical Datadog metric | functional tags |
|
|
217
|
+
|---|---|
|
|
218
|
+
| `puma.running` | `index` |
|
|
219
|
+
| `puma.backlog` | `index` |
|
|
220
|
+
| `puma.busy_threads` | `index` |
|
|
221
|
+
| `puma.pool_capacity` | `index` |
|
|
222
|
+
| `puma.max_threads` | `index` |
|
|
223
|
+
| `puma.requests_count` | `index` |
|
|
224
|
+
|
|
225
|
+
**Sidekiq — global / Redis-wide** (dedicated global-exporter pod, `namespace: sidekiq`):
|
|
226
|
+
|
|
227
|
+
| canonical Datadog metric | functional tags |
|
|
228
|
+
|---|---|
|
|
229
|
+
| `sidekiq.jobs_waiting_count` | `queue` |
|
|
230
|
+
| `sidekiq.queue_latency` | `queue` |
|
|
231
|
+
| `sidekiq.jobs_scheduled_count` | — |
|
|
232
|
+
| `sidekiq.jobs_retry_count` | — (`queue` if `retries_segmented_by_queue`) |
|
|
233
|
+
| `sidekiq.jobs_dead_count` | — |
|
|
234
|
+
| `sidekiq.active_processes` | — |
|
|
235
|
+
| `sidekiq.active_workers_count` | — |
|
|
236
|
+
|
|
237
|
+
**Sidekiq — per-process / job** (worker pod, `namespace: sidekiq`; emitted on job activity):
|
|
238
|
+
|
|
239
|
+
| canonical Datadog metric | type | functional tags |
|
|
240
|
+
|---|---|---|
|
|
241
|
+
| `sidekiq.jobs_executed_total` | counter | `queue`, `worker` |
|
|
242
|
+
| `sidekiq.jobs_success_total` | counter | `queue`, `worker` |
|
|
243
|
+
| `sidekiq.jobs_failed_total` | counter | `queue`, `worker` |
|
|
244
|
+
| `sidekiq.jobs_enqueued_total` | counter | `queue`, `worker` |
|
|
245
|
+
| `sidekiq.jobs_rerouted_total` | counter | `from_queue`, `to_queue`, `worker` |
|
|
246
|
+
| `sidekiq.running_job_runtime` | gauge | `queue`, `worker` |
|
|
247
|
+
| `sidekiq.job_runtime` | histogram | `queue`, `worker` |
|
|
248
|
+
| `sidekiq.job_latency` | histogram | `queue`, `worker` |
|
|
249
|
+
|
|
250
|
+
### Where Sidekiq global metrics come from
|
|
251
|
+
|
|
252
|
+
The global (Redis-wide) queue gauges — `sidekiq_jobs_waiting_count`,
|
|
253
|
+
`sidekiq_queue_latency`, … — are read from Redis and are identical regardless of
|
|
254
|
+
which process reads them. The default **`:web`** policy collects them only in the
|
|
255
|
+
always-on **web** process (which already has the Sidekiq client / Redis access),
|
|
256
|
+
so there's a **single source and no per-worker duplication**: workers export only
|
|
257
|
+
their own per-process job metrics. Workers can scale (even to zero) without
|
|
258
|
+
losing the queue metrics.
|
|
259
|
+
|
|
260
|
+
Alternatives via `c.sidekiq_global_metrics`:
|
|
261
|
+
|
|
262
|
+
- **`:all`** — every worker pod exports the global gauges (the legacy behaviour).
|
|
263
|
+
Datadog then has one series **per pod**, so aggregate in dashboards/monitors
|
|
264
|
+
with `max:sidekiq.queue_latency{*} by {queue}` (Datadog space aggregation —
|
|
265
|
+
never `sum`, which would multiply by pod count).
|
|
266
|
+
- **`:off`** — neither web nor workers collect them. Use this with a **dedicated
|
|
267
|
+
exporter** (below) so the gauges still come from somewhere — exactly once.
|
|
268
|
+
|
|
269
|
+
> Under `:web`, if the web Deployment runs more than one replica the gauges are
|
|
270
|
+
> duplicated across those (few, stable) web pods — apply the same
|
|
271
|
+
> `max:… by {queue}` aggregation, or use the dedicated exporter for one series.
|
|
272
|
+
|
|
273
|
+
### Dedicated global-metrics exporter (recommended for multi-replica / KEDA)
|
|
274
|
+
|
|
275
|
+
When the web and worker pods are autoscaled (e.g. KEDA, possibly to zero), the
|
|
276
|
+
cleanest single source for the Redis-wide queue gauges is a **dedicated 1-replica
|
|
277
|
+
Deployment** that does nothing but read them from Redis and expose them. It is
|
|
278
|
+
decoupled from web/worker scaling and yields exactly **one series** per gauge.
|
|
279
|
+
|
|
280
|
+
Set `c.sidekiq_global_metrics = :off` (so web/workers don't collect them), then
|
|
281
|
+
run a 1-replica Deployment with a thin, **host-owned** entrypoint — the gem
|
|
282
|
+
deliberately ships no executable, so the Redis connection config stays with the
|
|
283
|
+
host. E.g. `bin/pod-exporter`:
|
|
284
|
+
|
|
285
|
+
```ruby
|
|
286
|
+
#!/usr/bin/env ruby
|
|
287
|
+
# bundler/setup only wires the $LOAD_PATH from the lockfile — nothing is
|
|
288
|
+
# required beyond the lines below, so the process stays tiny.
|
|
289
|
+
require 'bundler/setup'
|
|
290
|
+
require 'rails_pod_kit/global_exporter'
|
|
291
|
+
|
|
292
|
+
RailsPodKit::GlobalExporter.run!(redis: { url: ENV['REDIS_URL'] })
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
```
|
|
296
|
+
command: ["bin/pod-exporter"]
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
The entrypoint is deliberately **Rails-free**: `run!` only wires the Sidekiq
|
|
300
|
+
client's Redis connection from the injected options plus the yabeda stack,
|
|
301
|
+
declares the cluster gauges, starts the exporter and blocks until SIGTERM.
|
|
302
|
+
Booting the full host app just to read a handful of Redis counters would cost
|
|
303
|
+
~300Mi RSS for nothing — this process sits at ~60Mi.
|
|
304
|
+
|
|
305
|
+
## Caveats
|
|
306
|
+
|
|
307
|
+
- **Puma only.** The Puma plugins only activate under Puma; under any other
|
|
308
|
+
app server the in-process `/metrics` endpoint is **not** exposed.
|
|
309
|
+
- **Puma control app.** `yabeda-puma-plugin` reads Puma's thread-pool stats
|
|
310
|
+
through Puma's control app, so `Puma.activate` activates one on a
|
|
311
|
+
localhost-only socket (`no_token: true`, never network-exposed).
|
|
312
|
+
- **Rack version.** Under **Rack 3+** the mmap exporter's WEBrick handler also
|
|
313
|
+
needs the `rackup` gem. Under Rack 2.x `webrick` alone is enough, but on
|
|
314
|
+
Ruby ≥ 3.5 make sure `ostruct` is in the bundle (Rack 2.2 requires it
|
|
315
|
+
without declaring it, and it's no longer a default gem).
|
|
316
|
+
- **Two kinds of entry point.** The main file (`require 'rails_pod_kit'`,
|
|
317
|
+
what Bundler.require loads in a Rails app) pulls in every integration
|
|
318
|
+
unconditionally — including the health-monitor-rails engine and its
|
|
319
|
+
railties foundation — regardless of Gemfile declaration order. The
|
|
320
|
+
sub-entry points (`rails_pod_kit/puma`, `rails_pod_kit/global_exporter`)
|
|
321
|
+
require only the config core, so the Rails-free exporter stays
|
|
322
|
+
railties-free and a `puma -C config/puma.rb` boot (which evaluates
|
|
323
|
+
`config/puma.rb` before Rails) still gets the integrations once
|
|
324
|
+
`Bundler.require` runs.
|
|
325
|
+
|
|
326
|
+
## Local verification
|
|
327
|
+
|
|
328
|
+
Boot each process with the exporter enabled and scrape `:9394/metrics`:
|
|
329
|
+
|
|
330
|
+
```bash
|
|
331
|
+
# Web (Puma): puma_* series
|
|
332
|
+
bundle exec rails server
|
|
333
|
+
curl -s localhost:9394/metrics | grep '^puma_'
|
|
334
|
+
|
|
335
|
+
# Worker (Sidekiq): sidekiq_* series (run a job first so per-process counters appear)
|
|
336
|
+
bundle exec sidekiq
|
|
337
|
+
curl -s localhost:9394/metrics | grep '^sidekiq_'
|
|
338
|
+
|
|
339
|
+
# Health endpoint
|
|
340
|
+
curl -s localhost:3000/healthz -H 'Accept: application/json'
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
## Tests
|
|
344
|
+
|
|
345
|
+
The gem's specs run in isolation (they do not load a host Rails app):
|
|
346
|
+
|
|
347
|
+
```bash
|
|
348
|
+
bundle install
|
|
349
|
+
bundle exec rspec
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
The suite runs against a matrix of Rails/Rack versions via
|
|
353
|
+
[Appraisal](https://github.com/thoughtbot/appraisal):
|
|
354
|
+
|
|
355
|
+
```bash
|
|
356
|
+
bundle exec appraisal install
|
|
357
|
+
bundle exec appraisal rails-8.1 rspec
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
## License
|
|
361
|
+
|
|
362
|
+
Released under the [MIT License](LICENSE.txt).
|
data/VERSION
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
0.0.1
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'anyway_config'
|
|
4
|
+
|
|
5
|
+
require 'rails_pod_kit/version'
|
|
6
|
+
|
|
7
|
+
# Configuration core, shared by every entry point. The sub-entry points
|
|
8
|
+
# (rails_pod_kit/puma, rails_pod_kit/global_exporter, …) require this file —
|
|
9
|
+
# not the gem's main file — so they can run in Rails-free processes without
|
|
10
|
+
# pulling in railties.
|
|
11
|
+
module RailsPodKit
|
|
12
|
+
class Error < StandardError; end
|
|
13
|
+
|
|
14
|
+
DEFAULT_PORT = 9394
|
|
15
|
+
|
|
16
|
+
# Backed by anyway_config: every attribute can also come from RAILS_POD_KIT_*
|
|
17
|
+
# env vars or an optional config/rails_pod_kit.yml in the host, with the
|
|
18
|
+
# `RailsPodKit.configure` block (running last, from an initializer) winning
|
|
19
|
+
# over both. A single instance is memoized on the module.
|
|
20
|
+
class Config < Anyway::Config
|
|
21
|
+
config_name :rails_pod_kit
|
|
22
|
+
env_prefix 'RAILS_POD_KIT'
|
|
23
|
+
|
|
24
|
+
# enabled: master switch — when false the gem is a complete no-op: no
|
|
25
|
+
# exporter server is started and no port is bound. Defaults to off in the
|
|
26
|
+
# `test` environment so specs and CI never try to bind port 9394.
|
|
27
|
+
#
|
|
28
|
+
# port: TCP port the in-process exporter binds for both Puma and Sidekiq.
|
|
29
|
+
# Defaults to PROMETHEUS_EXPORTER_PORT (the yabeda-native env var) or 9394.
|
|
30
|
+
#
|
|
31
|
+
# sidekiq_global_metrics: policy for Sidekiq global (Redis-wide) queue
|
|
32
|
+
# metrics — queue latency, jobs waiting, etc. These are read from Redis
|
|
33
|
+
# and are identical across processes, so exporting them from every worker
|
|
34
|
+
# just duplicates series downstream.
|
|
35
|
+
# :web (default) -> collect them only in the always-on web (Puma)
|
|
36
|
+
# process; workers export only their per-process job
|
|
37
|
+
# metrics. One source, no per-worker duplication.
|
|
38
|
+
# :all -> every worker pod exports them (legacy). Aggregate
|
|
39
|
+
# with `max ... by {queue}` downstream, never `sum`.
|
|
40
|
+
# :off -> nobody collects them (per-process metrics
|
|
41
|
+
# unaffected). Pair with the dedicated global exporter
|
|
42
|
+
# (GlobalExporter) so the gauges still come from
|
|
43
|
+
# somewhere — exactly once.
|
|
44
|
+
#
|
|
45
|
+
# puma_control_url: Puma control-app URL. yabeda-puma-plugin reads the Puma
|
|
46
|
+
# thread-pool stats through the control app, so `Puma.activate` makes sure
|
|
47
|
+
# one is running. Defaults to PUMA_CONTROL_URL or a localhost-only TCP
|
|
48
|
+
# socket (not network-exposed in the pod).
|
|
49
|
+
#
|
|
50
|
+
# retries_segmented_by_queue: when true, the Sidekiq global retry gauge
|
|
51
|
+
# (`sidekiq_jobs_retry_count`) is broken down with a `queue` tag instead
|
|
52
|
+
# of a single global scalar. Maps to yabeda-sidekiq's setting of the same
|
|
53
|
+
# name. Off by default: per-queue retries multiply series cardinality, and
|
|
54
|
+
# the global scalar is enough for a failing-jobs anomaly monitor.
|
|
55
|
+
#
|
|
56
|
+
# silence_exporter_access_log: when true (default), the in-process /metrics
|
|
57
|
+
# exporter emits NO access log line per scrape. The endpoint is
|
|
58
|
+
# pod-internal and scraped on a fixed interval by the metrics agent, so a
|
|
59
|
+
# `GET /metrics` line per scrape per pod is pure noise. Silenced at both
|
|
60
|
+
# transports — the Puma plugin's exporter (`prometheus_silence_logger`)
|
|
61
|
+
# and the WEBrick exporter used by Sidekiq / the dedicated global exporter
|
|
62
|
+
# (`Rack::CommonLogger`). Flip to false only to debug the exporter itself.
|
|
63
|
+
attr_config :enabled,
|
|
64
|
+
:port,
|
|
65
|
+
:puma_control_url,
|
|
66
|
+
sidekiq_global_metrics: :web,
|
|
67
|
+
retries_segmented_by_queue: false,
|
|
68
|
+
silence_exporter_access_log: true
|
|
69
|
+
|
|
70
|
+
coerce_types port: :integer,
|
|
71
|
+
enabled: :boolean,
|
|
72
|
+
retries_segmented_by_queue: :boolean,
|
|
73
|
+
silence_exporter_access_log: :boolean
|
|
74
|
+
|
|
75
|
+
def initialize(overrides = nil)
|
|
76
|
+
super
|
|
77
|
+
|
|
78
|
+
# Env/YAML sources deliver the policy as a string; the API is symbols.
|
|
79
|
+
self.sidekiq_global_metrics = sidekiq_global_metrics.to_sym if sidekiq_global_metrics
|
|
80
|
+
# Dynamic defaults anyway_config can't express statically.
|
|
81
|
+
self.enabled = self.class.default_enabled? if enabled.nil?
|
|
82
|
+
self.port ||= Integer(ENV.fetch('PROMETHEUS_EXPORTER_PORT', DEFAULT_PORT))
|
|
83
|
+
self.puma_control_url ||= ENV.fetch('PUMA_CONTROL_URL', 'tcp://127.0.0.1:9293')
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
def enabled?
|
|
87
|
+
!!enabled
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
# True unless we're clearly in a test environment. Kept independent of
|
|
91
|
+
# Rails so the gem's own specs (which don't load the host app) get a safe
|
|
92
|
+
# default and never bind a socket.
|
|
93
|
+
def self.default_enabled?
|
|
94
|
+
env = ENV['RAILS_ENV'] || ENV['RACK_ENV']
|
|
95
|
+
env != 'test'
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
class << self
|
|
100
|
+
def config
|
|
101
|
+
@config ||= Config.new
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
def configure
|
|
105
|
+
yield config if block_given?
|
|
106
|
+
config
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
# Test/reset hook — drops the memoized config so a fresh one is built.
|
|
110
|
+
def reset_config!
|
|
111
|
+
@config = nil
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
def enabled?
|
|
115
|
+
config.enabled?
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
end
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'rails_pod_kit/config'
|
|
4
|
+
|
|
5
|
+
module RailsPodKit
|
|
6
|
+
# Standalone, always-on exporter for the Sidekiq global (Redis-wide) queue
|
|
7
|
+
# metrics (queue latency, jobs waiting, scheduled/retry/dead counts, …).
|
|
8
|
+
#
|
|
9
|
+
# Runs as its own 1-replica Deployment so those series come from a single
|
|
10
|
+
# source, fully decoupled from web/worker autoscaling: KEDA can scale the web
|
|
11
|
+
# and worker pods (even to zero) without the queue metrics disappearing or
|
|
12
|
+
# being duplicated per pod. It is NOT a Sidekiq server and serves no web
|
|
13
|
+
# traffic — it just reads the cluster stats from Redis and exposes them on the
|
|
14
|
+
# exporter port.
|
|
15
|
+
#
|
|
16
|
+
# Deliberately Rails-free. The whole job is reading a handful of counters from
|
|
17
|
+
# Redis, so booting the host app (every gem, every initializer) just to
|
|
18
|
+
# inherit the connection config would cost hundreds of Mi RSS for nothing.
|
|
19
|
+
# Instead the host injects its Redis options into `run!`, which configures the
|
|
20
|
+
# Sidekiq client itself — the process sits at ~60Mi.
|
|
21
|
+
#
|
|
22
|
+
# Entry point: a thin, host-owned executable (the gem ships none, so the host
|
|
23
|
+
# keeps full ownership of its connection config), e.g. `bin/pod-exporter`:
|
|
24
|
+
#
|
|
25
|
+
# #!/usr/bin/env ruby
|
|
26
|
+
# require 'bundler/setup'
|
|
27
|
+
# require 'rails_pod_kit/global_exporter'
|
|
28
|
+
# RailsPodKit::GlobalExporter.run!(redis: { url: ENV['REDIS_URL'] })
|
|
29
|
+
module GlobalExporter
|
|
30
|
+
module_function
|
|
31
|
+
|
|
32
|
+
# Force-enable cluster collection so yabeda declares the global gauges when
|
|
33
|
+
# it configures. Force-on regardless of the host's `sidekiq_global_metrics`
|
|
34
|
+
# policy — collecting them is this process's whole job; the web/worker pods
|
|
35
|
+
# stay :off so they don't duplicate it.
|
|
36
|
+
def install!
|
|
37
|
+
require 'rails_pod_kit/sidekiq'
|
|
38
|
+
require 'yabeda/prometheus/mmap'
|
|
39
|
+
RailsPodKit::Sidekiq.enable_global_collection!
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
# Boots the exporter and blocks. Self-sufficient: no Rails environment is
|
|
43
|
+
# required — it wires the Sidekiq client's Redis connection from the
|
|
44
|
+
# host-injected options, declares and configures the gauges, starts the
|
|
45
|
+
# metrics server, then sleeps.
|
|
46
|
+
#
|
|
47
|
+
# `redis:` takes the same options hash the host passes to its own
|
|
48
|
+
# `Sidekiq.configure_*` blocks (`url:`, `ssl_params:`, …), so the connection
|
|
49
|
+
# config stays a host decision with a single source of truth.
|
|
50
|
+
def run!(redis:)
|
|
51
|
+
unless RailsPodKit.enabled?
|
|
52
|
+
warn '[rails_pod_kit] disabled — global exporter not started'
|
|
53
|
+
return
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
configure_redis!(redis)
|
|
57
|
+
install!
|
|
58
|
+
RailsPodKit::Sidekiq.start_metrics_server!
|
|
59
|
+
|
|
60
|
+
require 'yabeda'
|
|
61
|
+
Yabeda.configure! unless Yabeda.already_configured?
|
|
62
|
+
|
|
63
|
+
# The exporter serves from a background thread; block the main thread so
|
|
64
|
+
# the process stays up until the kubelet sends SIGTERM.
|
|
65
|
+
sleep
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# Configure the Sidekiq client's Redis connection so this Rails-free process
|
|
69
|
+
# can read the cluster stats.
|
|
70
|
+
def configure_redis!(redis_options)
|
|
71
|
+
require 'sidekiq'
|
|
72
|
+
::Sidekiq.configure_client do |config|
|
|
73
|
+
config.redis = redis_options
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
end
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'rails_pod_kit/config'
|
|
4
|
+
|
|
5
|
+
# health-monitor-rails subclasses ::Rails::Engine and its provider registry
|
|
6
|
+
# leans on ActiveSupport core extensions without requiring either itself;
|
|
7
|
+
# `rails` (railties' entry file, not the full framework) sets both up in the
|
|
8
|
+
# right order.
|
|
9
|
+
require 'rails'
|
|
10
|
+
require 'active_support/core_ext/string'
|
|
11
|
+
require 'rails/engine'
|
|
12
|
+
require 'health-monitor-rails'
|
|
13
|
+
|
|
14
|
+
module RailsPodKit
|
|
15
|
+
# Opinionated health-monitor-rails configuration for a Rails pod's probe
|
|
16
|
+
# endpoint. One call from an initializer:
|
|
17
|
+
#
|
|
18
|
+
# RailsPodKit::Health.install!(
|
|
19
|
+
# redis: { url: ENV['REDIS_URL'] },
|
|
20
|
+
# sidekiq: { queue_size: 200, latency: 10.minutes }
|
|
21
|
+
# )
|
|
22
|
+
#
|
|
23
|
+
# gives the app a /healthz endpoint checking database (health_monitor's
|
|
24
|
+
# default), cache, Redis and (optionally) Sidekiq — suitable as a k8s startup
|
|
25
|
+
# probe. For liveness/readiness, probe `/healthz?providers[]=none` to
|
|
26
|
+
# short-circuit the dependency checks once the pod is live, so a transient
|
|
27
|
+
# Redis hiccup doesn't restart the app. Same trick for a manual basic check:
|
|
28
|
+
#
|
|
29
|
+
# curl '/healthz?providers[]=none' -H 'Accept: application/json'
|
|
30
|
+
#
|
|
31
|
+
# The Redis connection is injected by the host — the gem never reads
|
|
32
|
+
# REDIS_URL or makes TLS decisions.
|
|
33
|
+
module Health
|
|
34
|
+
module_function
|
|
35
|
+
|
|
36
|
+
# Configures HealthMonitor with the kit's defaults.
|
|
37
|
+
#
|
|
38
|
+
# redis: Redis connection options hash (same shape the host passes to
|
|
39
|
+
# Sidekiq) or a ready connection object (Redis / ConnectionPool).
|
|
40
|
+
# path: mount-relative endpoint path (default :healthz).
|
|
41
|
+
# sidekiq: optional thresholds hash; when given the Sidekiq provider is
|
|
42
|
+
# added with the provided `queue_size:` / `latency:` overrides
|
|
43
|
+
# (health_monitor defaults apply for missing keys). Omit entirely
|
|
44
|
+
# on hosts without Sidekiq.
|
|
45
|
+
# silence_controller_log: drop the per-probe INFO request logging (default
|
|
46
|
+
# true) — kubelet probes hit the endpoint every few seconds and
|
|
47
|
+
# would drown real request logs.
|
|
48
|
+
# mount: when true (default) the gem's Railtie mounts
|
|
49
|
+
# HealthMonitor::Engine at '/' automatically; pass false to keep
|
|
50
|
+
# route ownership and mount it yourself in config/routes.rb.
|
|
51
|
+
#
|
|
52
|
+
# Any further host-specific tuning (extra providers, error callback, …) can
|
|
53
|
+
# be done in the block, which receives the HealthMonitor configuration.
|
|
54
|
+
def install!(redis:, path: :healthz, sidekiq: nil, silence_controller_log: true, mount: true)
|
|
55
|
+
@auto_mount = mount
|
|
56
|
+
|
|
57
|
+
HealthMonitor.configure do |config|
|
|
58
|
+
config.path = path
|
|
59
|
+
config.cache
|
|
60
|
+
config.redis.configure do |redis_config|
|
|
61
|
+
redis_config.connection = build_connection(redis)
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
if sidekiq
|
|
65
|
+
config.sidekiq.configure do |sidekiq_config|
|
|
66
|
+
sidekiq_config.queue_size = sidekiq[:queue_size] if sidekiq[:queue_size]
|
|
67
|
+
sidekiq_config.latency = sidekiq[:latency] if sidekiq[:latency]
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
yield config if block_given?
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
silence_controller_log! if silence_controller_log
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# Read by the Railtie's deferred routes block: true only once install! ran
|
|
78
|
+
# (initializers run before the route set is drawn) and mount wasn't opted
|
|
79
|
+
# out.
|
|
80
|
+
def auto_mount?
|
|
81
|
+
!!@auto_mount
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def build_connection(redis)
|
|
85
|
+
return redis unless redis.is_a?(Hash)
|
|
86
|
+
|
|
87
|
+
require 'redis'
|
|
88
|
+
::Redis.new(**redis)
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# Quiet the engine controller so kubelet probes don't emit an INFO line per
|
|
92
|
+
# hit. Deferred to to_prepare because the engine controller is autoloaded.
|
|
93
|
+
def silence_controller_log!
|
|
94
|
+
return unless defined?(Rails.application) && Rails.application
|
|
95
|
+
|
|
96
|
+
Rails.application.reloader.to_prepare do
|
|
97
|
+
HealthMonitor::HealthController.logger.level = :warn
|
|
98
|
+
end
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
end
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'rails_pod_kit/config'
|
|
4
|
+
|
|
5
|
+
module RailsPodKit
|
|
6
|
+
# Puma integration. Encapsulates the Puma plugins the yabeda stack needs so
|
|
7
|
+
# that `config/puma.rb` stays a one-liner:
|
|
8
|
+
#
|
|
9
|
+
# require 'rails_pod_kit/puma'
|
|
10
|
+
# RailsPodKit::Puma.activate(self)
|
|
11
|
+
#
|
|
12
|
+
# - control app yabeda-puma-plugin reads the Puma thread-pool
|
|
13
|
+
# stats through Puma's control app, so we activate
|
|
14
|
+
# one (localhost-only, no token) if the app hasn't.
|
|
15
|
+
# - `plugin :yabeda` (yabeda-puma-plugin) registers a Yabeda collector
|
|
16
|
+
# for Puma thread-pool / worker stats.
|
|
17
|
+
# - `plugin :yabeda_prometheus` (yabeda-puma-plugin) starts the in-process
|
|
18
|
+
# HTTP server that serves /metrics, fork-safe under
|
|
19
|
+
# `preload_app!` + clustered workers.
|
|
20
|
+
#
|
|
21
|
+
# No-op when disabled or in the test env so the suite never binds the port.
|
|
22
|
+
module Puma
|
|
23
|
+
module_function
|
|
24
|
+
|
|
25
|
+
def activate(puma_config)
|
|
26
|
+
return unless RailsPodKit.enabled?
|
|
27
|
+
|
|
28
|
+
ENV['PROMETHEUS_EXPORTER_PORT'] ||= RailsPodKit.config.port.to_s
|
|
29
|
+
|
|
30
|
+
# The Puma plugin only pulls in the exporter class; we must also load the
|
|
31
|
+
# full mmap module so `Yabeda::Prometheus::Mmap.registry` exists and the
|
|
32
|
+
# mmap adapter is registered (it's a transitive dep, so Bundler.require in
|
|
33
|
+
# the host app doesn't auto-require it).
|
|
34
|
+
require 'yabeda/prometheus/mmap'
|
|
35
|
+
|
|
36
|
+
# The control app exposes Puma's /stats over a localhost-only socket that
|
|
37
|
+
# the yabeda collector queries. `no_token: true` is safe because the
|
|
38
|
+
# socket is never exposed outside the pod.
|
|
39
|
+
puma_config.activate_control_app(RailsPodKit.config.puma_control_url, no_token: true)
|
|
40
|
+
|
|
41
|
+
puma_config.plugin :yabeda
|
|
42
|
+
puma_config.plugin :yabeda_prometheus
|
|
43
|
+
|
|
44
|
+
# Silence the exporter's per-scrape access log. The `prometheus_silence_logger`
|
|
45
|
+
# DSL method is defined by the :yabeda_prometheus plugin, so this must run
|
|
46
|
+
# after the plugin is loaded above. See Config#silence_exporter_access_log.
|
|
47
|
+
puma_config.prometheus_silence_logger(true) if RailsPodKit.config.silence_exporter_access_log
|
|
48
|
+
|
|
49
|
+
# `config/puma.rb` is evaluated before Rails is loaded, so requiring this
|
|
50
|
+
# gem here loads yabeda *before* `defined?(Rails)`, and yabeda's Railtie
|
|
51
|
+
# (which would call `Yabeda.configure!` at after_initialize) never
|
|
52
|
+
# registers. So we drive configuration ourselves from the exporter-boot
|
|
53
|
+
# hook, which runs after Rails has booted and after the Puma plugin has
|
|
54
|
+
# registered its collector.
|
|
55
|
+
puma_config.on_prometheus_exporter_boot do
|
|
56
|
+
# Under the :web policy the always-on web process is the single source of
|
|
57
|
+
# Sidekiq global (Redis-wide) queue metrics, so workers don't duplicate
|
|
58
|
+
# them. Done here (not in activate) because requiring yabeda-sidekiq —
|
|
59
|
+
# which pulls in Sidekiq — before Rails boots corrupts Sidekiq's
|
|
60
|
+
# ActiveJob adapter load order. By exporter-boot, Rails has loaded
|
|
61
|
+
# Sidekiq properly and the web's Sidekiq client (Redis) is configured.
|
|
62
|
+
if RailsPodKit.config.sidekiq_global_metrics == :web
|
|
63
|
+
require 'rails_pod_kit/sidekiq'
|
|
64
|
+
RailsPodKit::Sidekiq.enable_global_collection!
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
require 'yabeda'
|
|
68
|
+
Yabeda.configure! unless Yabeda.already_configured?
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
end
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'rails_pod_kit/config'
|
|
4
|
+
|
|
5
|
+
require 'rails'
|
|
6
|
+
require 'rails/railtie'
|
|
7
|
+
|
|
8
|
+
module RailsPodKit
|
|
9
|
+
# Mounts the health endpoint automatically so the host doesn't have to touch
|
|
10
|
+
# config/routes.rb. The append block is evaluated when the route set is
|
|
11
|
+
# drawn (and on every reload) — after the initializers have run — so the
|
|
12
|
+
# guard sees whether Health.install! was called and with which `mount:`
|
|
13
|
+
# option. Hosts that want route ownership (custom mount point, constraints)
|
|
14
|
+
# pass `mount: false` to install! and mount HealthMonitor::Engine themselves.
|
|
15
|
+
class Railtie < ::Rails::Railtie
|
|
16
|
+
initializer 'rails_pod_kit.mount_health_endpoint' do |app|
|
|
17
|
+
app.routes.append do
|
|
18
|
+
mount HealthMonitor::Engine, at: '/' if RailsPodKit::Health.auto_mount?
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'rails_pod_kit/config'
|
|
4
|
+
|
|
5
|
+
module RailsPodKit
|
|
6
|
+
# Sidekiq integration. Called from inside `Sidekiq.configure_server`:
|
|
7
|
+
#
|
|
8
|
+
# RailsPodKit::Sidekiq.install!(config)
|
|
9
|
+
#
|
|
10
|
+
# It:
|
|
11
|
+
# - requires yabeda-sidekiq (registers the server middleware + metrics),
|
|
12
|
+
# - applies the global-metrics policy (see Config#sidekiq_global_metrics),
|
|
13
|
+
# - starts the in-process Prometheus exporter (WEBrick) bound to the
|
|
14
|
+
# configured port so the worker pod serves /metrics itself.
|
|
15
|
+
#
|
|
16
|
+
# No-op when disabled or in the test env so specs never bind the port.
|
|
17
|
+
module Sidekiq
|
|
18
|
+
module_function
|
|
19
|
+
|
|
20
|
+
def install!(sidekiq_config = nil)
|
|
21
|
+
return unless RailsPodKit.enabled?
|
|
22
|
+
|
|
23
|
+
require 'yabeda/sidekiq'
|
|
24
|
+
require 'yabeda/prometheus/mmap'
|
|
25
|
+
|
|
26
|
+
# Must run before Yabeda.configure! so the cluster-metric gauges are gated
|
|
27
|
+
# correctly (they are only declared when the policy leaves them enabled).
|
|
28
|
+
apply_global_metrics_policy!
|
|
29
|
+
|
|
30
|
+
# We can't rely on yabeda's Railtie to call Yabeda.configure!: depending on
|
|
31
|
+
# boot order it may never register. Drive it from Sidekiq's :startup
|
|
32
|
+
# lifecycle event, which fires after every initializer (so the sidekiq
|
|
33
|
+
# collector is registered) but before any job runs.
|
|
34
|
+
install_configure_hook(sidekiq_config)
|
|
35
|
+
|
|
36
|
+
start_metrics_server!
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
# Registers the Yabeda.configure! driver on Sidekiq's :startup event. The
|
|
40
|
+
# real app always passes a Sidekiq config that responds to :on; when it
|
|
41
|
+
# doesn't (e.g. unit specs that pass nil) we skip, so the global Yabeda
|
|
42
|
+
# singleton isn't configured as a test side effect.
|
|
43
|
+
def install_configure_hook(sidekiq_config)
|
|
44
|
+
return unless sidekiq_config.respond_to?(:on)
|
|
45
|
+
|
|
46
|
+
sidekiq_config.on(:startup) do
|
|
47
|
+
require 'yabeda'
|
|
48
|
+
Yabeda.configure! unless Yabeda.already_configured?
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# Decide whether this Sidekiq worker collects the global (Redis-wide) queue
|
|
53
|
+
# metrics. Only the :all policy keeps every worker collecting them; under
|
|
54
|
+
# :web they come from the web process instead, and under :off nobody does.
|
|
55
|
+
# Either way per-process job metrics are unaffected.
|
|
56
|
+
def apply_global_metrics_policy!
|
|
57
|
+
collect = RailsPodKit.config.sidekiq_global_metrics == :all
|
|
58
|
+
Yabeda::Sidekiq.config.collect_cluster_metrics = collect
|
|
59
|
+
apply_retries_segmentation!
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
# Mirror the gem's `retries_segmented_by_queue` flag onto yabeda-sidekiq. Must
|
|
63
|
+
# run before Yabeda.configure! so the retry gauge is declared with (or without)
|
|
64
|
+
# the `queue` tag. Off by default — see RailsPodKit::Config.
|
|
65
|
+
def apply_retries_segmentation!
|
|
66
|
+
Yabeda::Sidekiq.config.retries_segmented_by_queue =
|
|
67
|
+
RailsPodKit.config.retries_segmented_by_queue
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
# Enables global queue-metric collection in a process that is NOT a Sidekiq
|
|
71
|
+
# server — i.e. the web (Puma) process under the :web policy. The web has the
|
|
72
|
+
# Sidekiq client configured (Redis access), so yabeda-sidekiq can read the
|
|
73
|
+
# cluster stats there. We force `collect_cluster_metrics` on and keep
|
|
74
|
+
# `declare_process_metrics` off (the web runs no jobs). Must run before
|
|
75
|
+
# Yabeda.configure! so the gauges are declared.
|
|
76
|
+
def enable_global_collection!
|
|
77
|
+
require 'yabeda/sidekiq'
|
|
78
|
+
|
|
79
|
+
Yabeda::Sidekiq.config.collect_cluster_metrics = true
|
|
80
|
+
Yabeda::Sidekiq.config.declare_process_metrics = false
|
|
81
|
+
apply_retries_segmentation!
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
# Starts the background WEBrick exporter. `start_metrics_server!` reads the
|
|
85
|
+
# bind port from PROMETHEUS_EXPORTER_PORT, so we set it from config first.
|
|
86
|
+
# Guarded so a re-entrant Sidekiq boot doesn't try to double-bind the port.
|
|
87
|
+
def start_metrics_server!
|
|
88
|
+
return if @server_started
|
|
89
|
+
|
|
90
|
+
ENV['PROMETHEUS_EXPORTER_PORT'] ||= RailsPodKit.config.port.to_s
|
|
91
|
+
# Drop the WEBrick exporter's per-scrape access log (Rack::CommonLogger,
|
|
92
|
+
# which the mmap exporter mounts unless this is exactly 'false'). See
|
|
93
|
+
# Config#silence_exporter_access_log.
|
|
94
|
+
ENV['PROMETHEUS_EXPORTER_LOG_REQUESTS'] = 'false' if RailsPodKit.config.silence_exporter_access_log
|
|
95
|
+
Yabeda::Prometheus::Exporter.start_metrics_server!
|
|
96
|
+
@server_started = true
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
end
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'rails_pod_kit/version'
|
|
4
|
+
require 'rails_pod_kit/config'
|
|
5
|
+
|
|
6
|
+
# RailsPodKit packages the operational endpoints a Rails pod needs to be a good
|
|
7
|
+
# Kubernetes citizen behind a single, opinionated entry point:
|
|
8
|
+
#
|
|
9
|
+
# - Prometheus metrics: Puma and Sidekiq runtime metrics in Prometheus text
|
|
10
|
+
# format on an in-process /metrics endpoint (default port 9394) — no sidecar,
|
|
11
|
+
# no separate collector process. A thin wrapper around the yabeda ecosystem.
|
|
12
|
+
# - Health checks: an opinionated health-monitor-rails configuration serving
|
|
13
|
+
# liveness/readiness/startup probes on /healthz (see RailsPodKit::Health),
|
|
14
|
+
# auto-mounted by the gem's Railtie.
|
|
15
|
+
#
|
|
16
|
+
# Metric series are intentionally left untagged here: a scraping agent (e.g.
|
|
17
|
+
# the Datadog Agent) adds service/env/version and kube_* tags at scrape time,
|
|
18
|
+
# so the gem doesn't duplicate them.
|
|
19
|
+
#
|
|
20
|
+
# Wiring (see README) is three one-liners:
|
|
21
|
+
# - config/puma.rb -> RailsPodKit::Puma.activate(self)
|
|
22
|
+
# - config/initializers/sidekiq.rb -> RailsPodKit::Sidekiq.install!(config)
|
|
23
|
+
# - config/initializers/rails_pod_kit.rb -> RailsPodKit.configure { ... }
|
|
24
|
+
# RailsPodKit::Health.install!(redis: ...)
|
|
25
|
+
# plus, only when running the dedicated global exporter, a host-owned
|
|
26
|
+
# entrypoint (e.g. bin/pod-exporter) calling GlobalExporter.run!(redis: ...).
|
|
27
|
+
#
|
|
28
|
+
# The gem is deliberately connection-agnostic: it never reads REDIS_URL or
|
|
29
|
+
# makes TLS decisions. The host injects its Redis options where needed
|
|
30
|
+
# (GlobalExporter.run!, Health.install!).
|
|
31
|
+
|
|
32
|
+
# This is the Rails-app entry point: it loads every integration
|
|
33
|
+
# unconditionally (each file requires what it needs, so nothing here depends
|
|
34
|
+
# on Gemfile declaration order or on which constants happen to be defined
|
|
35
|
+
# yet). The Sidekiq integration is inert until install! is called and pulls in
|
|
36
|
+
# yabeda-sidekiq only then; health + railtie load the health-monitor-rails
|
|
37
|
+
# engine and railties.
|
|
38
|
+
#
|
|
39
|
+
# The sub-entry points deliberately require only rails_pod_kit/config instead
|
|
40
|
+
# of this file, with two effects:
|
|
41
|
+
# - a Rails-free process (bin/pod-exporter requiring
|
|
42
|
+
# rails_pod_kit/global_exporter) never pulls in railties or Sidekiq;
|
|
43
|
+
# - under `puma -C config/puma.rb` — where Puma evaluates config/puma.rb
|
|
44
|
+
# (and thus rails_pod_kit/puma) before Rails exists — this file is still
|
|
45
|
+
# fresh for Bundler.require, so the integrations load once Rails is up.
|
|
46
|
+
require 'rails_pod_kit/sidekiq'
|
|
47
|
+
require 'rails_pod_kit/health'
|
|
48
|
+
require 'rails_pod_kit/railtie'
|
metadata
ADDED
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: rails_pod_kit
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.0.1
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Fabio Napoleoni
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: yabeda
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - "~>"
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '0.16'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - "~>"
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '0.16'
|
|
26
|
+
- !ruby/object:Gem::Dependency
|
|
27
|
+
name: yabeda-sidekiq
|
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
|
29
|
+
requirements:
|
|
30
|
+
- - "~>"
|
|
31
|
+
- !ruby/object:Gem::Version
|
|
32
|
+
version: '0.12'
|
|
33
|
+
type: :runtime
|
|
34
|
+
prerelease: false
|
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
36
|
+
requirements:
|
|
37
|
+
- - "~>"
|
|
38
|
+
- !ruby/object:Gem::Version
|
|
39
|
+
version: '0.12'
|
|
40
|
+
- !ruby/object:Gem::Dependency
|
|
41
|
+
name: yabeda-puma-plugin
|
|
42
|
+
requirement: !ruby/object:Gem::Requirement
|
|
43
|
+
requirements:
|
|
44
|
+
- - "~>"
|
|
45
|
+
- !ruby/object:Gem::Version
|
|
46
|
+
version: '0.9'
|
|
47
|
+
type: :runtime
|
|
48
|
+
prerelease: false
|
|
49
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
50
|
+
requirements:
|
|
51
|
+
- - "~>"
|
|
52
|
+
- !ruby/object:Gem::Version
|
|
53
|
+
version: '0.9'
|
|
54
|
+
- !ruby/object:Gem::Dependency
|
|
55
|
+
name: yabeda-prometheus-mmap
|
|
56
|
+
requirement: !ruby/object:Gem::Requirement
|
|
57
|
+
requirements:
|
|
58
|
+
- - "~>"
|
|
59
|
+
- !ruby/object:Gem::Version
|
|
60
|
+
version: '0.4'
|
|
61
|
+
type: :runtime
|
|
62
|
+
prerelease: false
|
|
63
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
64
|
+
requirements:
|
|
65
|
+
- - "~>"
|
|
66
|
+
- !ruby/object:Gem::Version
|
|
67
|
+
version: '0.4'
|
|
68
|
+
- !ruby/object:Gem::Dependency
|
|
69
|
+
name: webrick
|
|
70
|
+
requirement: !ruby/object:Gem::Requirement
|
|
71
|
+
requirements:
|
|
72
|
+
- - "~>"
|
|
73
|
+
- !ruby/object:Gem::Version
|
|
74
|
+
version: '1.8'
|
|
75
|
+
type: :runtime
|
|
76
|
+
prerelease: false
|
|
77
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
78
|
+
requirements:
|
|
79
|
+
- - "~>"
|
|
80
|
+
- !ruby/object:Gem::Version
|
|
81
|
+
version: '1.8'
|
|
82
|
+
- !ruby/object:Gem::Dependency
|
|
83
|
+
name: anyway_config
|
|
84
|
+
requirement: !ruby/object:Gem::Requirement
|
|
85
|
+
requirements:
|
|
86
|
+
- - "~>"
|
|
87
|
+
- !ruby/object:Gem::Version
|
|
88
|
+
version: '2.0'
|
|
89
|
+
type: :runtime
|
|
90
|
+
prerelease: false
|
|
91
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
92
|
+
requirements:
|
|
93
|
+
- - "~>"
|
|
94
|
+
- !ruby/object:Gem::Version
|
|
95
|
+
version: '2.0'
|
|
96
|
+
- !ruby/object:Gem::Dependency
|
|
97
|
+
name: health-monitor-rails
|
|
98
|
+
requirement: !ruby/object:Gem::Requirement
|
|
99
|
+
requirements:
|
|
100
|
+
- - "~>"
|
|
101
|
+
- !ruby/object:Gem::Version
|
|
102
|
+
version: '12.8'
|
|
103
|
+
type: :runtime
|
|
104
|
+
prerelease: false
|
|
105
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
106
|
+
requirements:
|
|
107
|
+
- - "~>"
|
|
108
|
+
- !ruby/object:Gem::Version
|
|
109
|
+
version: '12.8'
|
|
110
|
+
- !ruby/object:Gem::Dependency
|
|
111
|
+
name: redis
|
|
112
|
+
requirement: !ruby/object:Gem::Requirement
|
|
113
|
+
requirements:
|
|
114
|
+
- - "~>"
|
|
115
|
+
- !ruby/object:Gem::Version
|
|
116
|
+
version: '5.0'
|
|
117
|
+
type: :runtime
|
|
118
|
+
prerelease: false
|
|
119
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
120
|
+
requirements:
|
|
121
|
+
- - "~>"
|
|
122
|
+
- !ruby/object:Gem::Version
|
|
123
|
+
version: '5.0'
|
|
124
|
+
description: |
|
|
125
|
+
Packages the yabeda ecosystem and health-monitor-rails into a single,
|
|
126
|
+
opinionated kit for running Rails applications on Kubernetes: Puma and
|
|
127
|
+
Sidekiq runtime metrics in Prometheus text format on an in-process /metrics
|
|
128
|
+
endpoint (default port 9394), plus a /healthz endpoint wired for
|
|
129
|
+
liveness/readiness/startup probes. No sidecar, no separate collector.
|
|
130
|
+
email:
|
|
131
|
+
- f.napoleoni@gmail.com
|
|
132
|
+
executables: []
|
|
133
|
+
extensions: []
|
|
134
|
+
extra_rdoc_files: []
|
|
135
|
+
files:
|
|
136
|
+
- LICENSE.txt
|
|
137
|
+
- README.md
|
|
138
|
+
- VERSION
|
|
139
|
+
- lib/rails_pod_kit.rb
|
|
140
|
+
- lib/rails_pod_kit/config.rb
|
|
141
|
+
- lib/rails_pod_kit/global_exporter.rb
|
|
142
|
+
- lib/rails_pod_kit/health.rb
|
|
143
|
+
- lib/rails_pod_kit/puma.rb
|
|
144
|
+
- lib/rails_pod_kit/railtie.rb
|
|
145
|
+
- lib/rails_pod_kit/sidekiq.rb
|
|
146
|
+
- lib/rails_pod_kit/version.rb
|
|
147
|
+
homepage: https://github.com/fabn/rails_pod_kit
|
|
148
|
+
licenses:
|
|
149
|
+
- MIT
|
|
150
|
+
metadata:
|
|
151
|
+
source_code_uri: https://github.com/fabn/rails_pod_kit
|
|
152
|
+
changelog_uri: https://github.com/fabn/rails_pod_kit/releases
|
|
153
|
+
rubygems_mfa_required: 'true'
|
|
154
|
+
rdoc_options: []
|
|
155
|
+
require_paths:
|
|
156
|
+
- lib
|
|
157
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
158
|
+
requirements:
|
|
159
|
+
- - ">="
|
|
160
|
+
- !ruby/object:Gem::Version
|
|
161
|
+
version: 3.3.0
|
|
162
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
163
|
+
requirements:
|
|
164
|
+
- - ">="
|
|
165
|
+
- !ruby/object:Gem::Version
|
|
166
|
+
version: '0'
|
|
167
|
+
requirements: []
|
|
168
|
+
rubygems_version: 3.6.9
|
|
169
|
+
specification_version: 4
|
|
170
|
+
summary: 'Operational endpoints for Rails pods: Prometheus metrics (Puma + Sidekiq)
|
|
171
|
+
and health checks'
|
|
172
|
+
test_files: []
|