dead_bro 0.2.29 → 0.2.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -0
- data/README.md +12 -173
- data/lib/dead_bro/configuration.rb +8 -3
- data/lib/dead_bro/job_sql_tracking_middleware.rb +2 -0
- data/lib/dead_bro/job_subscriber.rb +7 -0
- data/lib/dead_bro/sql_subscriber.rb +15 -0
- data/lib/dead_bro/sql_tracking_middleware.rb +3 -0
- data/lib/dead_bro/subscriber.rb +37 -0
- data/lib/dead_bro/version.rb +1 -1
- data/lib/dead_bro/watch_tracker.rb +189 -0
- data/lib/dead_bro.rb +25 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b6c0f13b23ff342d0ea4365872e2f380a949a8aaf3fd69d98606336854d14c9c
|
|
4
|
+
data.tar.gz: a9b8a2fae51c26e7cd9b2f1d676c6742a5a5b4829f0c182417af7eabdf2581c6
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e70b0d0b091dfacf1db61ea287a47bb1944153968bc3f225c943baf6a195c099dfd88199e59a942230284f3cfc94dcbf207d942e03630ac14850f818c1dc0928
|
|
7
|
+
data.tar.gz: 1e5f8605da3bf8c418d4c16025be5a24b7f89af1ca47d7bb6c767f790205c498e3023543b739b1568156547c21b8cfaf2bf327c573d8177aad88049e395352f2
|
data/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,11 @@
|
|
|
3
3
|
### Added
|
|
4
4
|
- Monitor thread now sends a synchronous heartbeat on startup before the first collection tick. This ensures remote settings — including `monitor_enabled` — are applied from the very first reporting cycle, so Sidekiq workers and other non-web processes that have not yet sent any metrics still receive the correct configuration immediately on boot rather than waiting up to 60 seconds for the first scheduled tick.
|
|
5
5
|
|
|
6
|
+
## [0.2.30] - 2026-06-19
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
- **`DeadBro.watch(label, **tags) { ... }`**: opt-in scoped timing for arbitrary code during instrumented web requests and background jobs. Records elapsed time, `start_offset_ms`, nesting depth, SQL count/duration attributable to the block, optional tags, and exception metadata (then re-raises). Spans are sent as `watch_events` in the normal APM payload for the Request Trace waterfall. Disabled by default; enable locally via `DeadBro.configure { |c| c.watch_enabled = true }` or remotely via the `watch_enabled` setting. Guardrails: max depth 10, max 50 spans per request, label length 200 chars, max 5 tags per span with tag values truncated to 100 chars. **Nested SQL:** each span diffs the request-wide SQL counter at enter/exit; parent spans include SQL from nested child spans (inclusive totals — use inner spans for per-block SQL, do not sum parent + child counts).
|
|
10
|
+
|
|
6
11
|
## [0.2.25] - 2026-06-14
|
|
7
12
|
|
|
8
13
|
### Added
|
data/README.md
CHANGED
|
@@ -65,115 +65,9 @@ Use the DeadBro UI to turn features on or off, set sample rates, define controll
|
|
|
65
65
|
|
|
66
66
|
You can still set `config.enabled` in Ruby if you need to force the integration off in a given environment before any remote settings arrive; otherwise the dashboard can control `enabled` like other remote settings.
|
|
67
67
|
|
|
68
|
-
##
|
|
68
|
+
## Request Sampling, Exclusions & Whitelisting
|
|
69
69
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
## Request Sampling
|
|
73
|
-
|
|
74
|
-
DeadBro supports configurable request sampling to reduce the volume of metrics sent to your APM endpoint, which is useful for high-traffic applications. Prefer setting this in the DeadBro app; use Ruby if you need a local override.
|
|
75
|
-
|
|
76
|
-
### Configuration
|
|
77
|
-
|
|
78
|
-
Set the sample rate as a percentage (1-100):
|
|
79
|
-
|
|
80
|
-
```ruby
|
|
81
|
-
# Track 50% of requests
|
|
82
|
-
DeadBro.configure do |config|
|
|
83
|
-
config.sample_rate = 50
|
|
84
|
-
end
|
|
85
|
-
|
|
86
|
-
# Track 10% of requests (useful for high-traffic apps)
|
|
87
|
-
DeadBro.configure do |config|
|
|
88
|
-
config.sample_rate = 10
|
|
89
|
-
end
|
|
90
|
-
|
|
91
|
-
# Track all requests (default)
|
|
92
|
-
DeadBro.configure do |config|
|
|
93
|
-
config.sample_rate = 100
|
|
94
|
-
end
|
|
95
|
-
```
|
|
96
|
-
|
|
97
|
-
### How It Works
|
|
98
|
-
|
|
99
|
-
- **Random Sampling**: Each request has a random chance of being tracked based on the sample rate
|
|
100
|
-
- **Consistent Per-Request**: The sampling decision is made once per request and applies to all metrics for that request
|
|
101
|
-
- **Debug Logging**: Skipped requests do not count towards the montly limit
|
|
102
|
-
- **Error Tracking**: Errors are still tracked regardless of sampling
|
|
103
|
-
|
|
104
|
-
### Use Cases
|
|
105
|
-
|
|
106
|
-
- **High-Traffic Applications**: Reduce APM data volume and costs
|
|
107
|
-
- **Development/Staging**: Sample fewer requests to reduce noise
|
|
108
|
-
- **Performance Testing**: Track a subset of requests during load testing
|
|
109
|
-
- **Cost Optimization**: Balance monitoring coverage with data costs
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
## Excluding Controllers and Jobs
|
|
113
|
-
|
|
114
|
-
You can exclude specific controllers and jobs from APM tracking (dashboard first; Ruby optional).
|
|
115
|
-
|
|
116
|
-
### Configuration
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
```ruby
|
|
120
|
-
DeadBro.configure do |config|
|
|
121
|
-
# Controller-only or controller#action patterns in one list (wildcards supported)
|
|
122
|
-
config.excluded_controllers = [
|
|
123
|
-
"HealthChecksController",
|
|
124
|
-
"Admin::*",
|
|
125
|
-
"UsersController#show",
|
|
126
|
-
"Admin::ReportsController#index",
|
|
127
|
-
"Admin::*#*"
|
|
128
|
-
]
|
|
129
|
-
|
|
130
|
-
config.excluded_jobs = [
|
|
131
|
-
"ActiveStorage::AnalyzeJob",
|
|
132
|
-
"Admin::*"
|
|
133
|
-
]
|
|
134
|
-
end
|
|
135
|
-
```
|
|
136
|
-
|
|
137
|
-
Notes:
|
|
138
|
-
- Wildcards `*` are supported (e.g., `Admin::*`, `Admin::*#*`).
|
|
139
|
-
- Matching uses full names like `UsersController`, `Admin::ReportsController#index`, `MyJob`.
|
|
140
|
-
|
|
141
|
-
## Exclusive Tracking (Whitelist Mode)
|
|
142
|
-
|
|
143
|
-
You can configure DeadBro to **only** track specific controllers, actions, or jobs. Prefer the dashboard; use Ruby for overrides.
|
|
144
|
-
|
|
145
|
-
### Configuration
|
|
146
|
-
|
|
147
|
-
```ruby
|
|
148
|
-
DeadBro.configure do |config|
|
|
149
|
-
# Only track these controllers/actions (patterns can include #action or wildcards)
|
|
150
|
-
config.exclusive_controllers = [
|
|
151
|
-
"UsersController#show",
|
|
152
|
-
"UsersController#index",
|
|
153
|
-
"Admin::ReportsController#*",
|
|
154
|
-
"Api::*#*"
|
|
155
|
-
]
|
|
156
|
-
|
|
157
|
-
config.exclusive_jobs = [
|
|
158
|
-
"PaymentProcessingJob",
|
|
159
|
-
"EmailDeliveryJob",
|
|
160
|
-
"Admin::*"
|
|
161
|
-
]
|
|
162
|
-
end
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
### How It Works
|
|
166
|
-
|
|
167
|
-
- **If `exclusive_controllers` or `exclusive_jobs` is empty/not defined**: All controllers/actions/jobs are tracked (default behavior)
|
|
168
|
-
- **If `exclusive_controllers` or `exclusive_jobs` is defined with values**: Only matching controllers/actions/jobs are tracked
|
|
169
|
-
- **Exclusion takes precedence**: If something matches both `excluded_*` and `exclusive_*`, it is excluded (exclusion is checked first)
|
|
170
|
-
|
|
171
|
-
### Use Cases
|
|
172
|
-
|
|
173
|
-
- **Focus on Critical Paths**: Monitor only your most important endpoints
|
|
174
|
-
- **Cost Optimization**: Track only specific high-value operations
|
|
175
|
-
- **Debugging**: Temporarily focus on specific controllers/jobs during investigation
|
|
176
|
-
- **Compliance**: Track only operations that require monitoring for compliance reasons
|
|
70
|
+
To control data volume, DeadBro supports request sampling (track a percentage of requests), excluding specific controllers/jobs from tracking, and whitelisting (tracking *only* specific controllers/jobs). All of this is configured in the DeadBro dashboard — sample rate, `excluded_controllers`/`excluded_jobs`, and `exclusive_controllers`/`exclusive_jobs` patterns (wildcards like `Admin::*` and `Admin::*#*` supported). Exclusion always takes precedence over whitelisting when a pattern matches both.
|
|
177
71
|
|
|
178
72
|
## SQL Query Tracking
|
|
179
73
|
|
|
@@ -198,43 +92,19 @@ DeadBro can automatically capture the query plan (`EXPLAIN`) of slow SELECT quer
|
|
|
198
92
|
- **Value scrubbing**: quoted string literals echoed in plan text (e.g. `Filter: email = 'a@b.com'`) are replaced with `?` before the plan leaves your app.
|
|
199
93
|
- **Bounded overhead**: plans are captured on background threads (max 3 concurrent per request); at request end DeadBro waits at most 0.5s for stragglers, then drops them.
|
|
200
94
|
|
|
201
|
-
### How It Works
|
|
202
|
-
|
|
203
|
-
- **Automatic Detection**: when a SELECT exceeds `slow_query_threshold_ms`, DeadBro captures its plan in the background
|
|
204
|
-
- **Database Support**: PostgreSQL, MySQL, SQLite (`EXPLAIN QUERY PLAN`), and any adapter with a standard `EXPLAIN`
|
|
205
|
-
|
|
206
95
|
### Configuration
|
|
207
96
|
|
|
208
|
-
- **`explain_analyze_enabled`** (default: `false`)
|
|
209
|
-
- **`slow_query_threshold_ms`** (default: `500`) - queries taking longer than this threshold will have their execution plan captured
|
|
97
|
+
- **`explain_analyze_enabled`** (default: `false`) — this is the one setting that must be turned on in Ruby; the dashboard toggle can only turn capture *off*, never on, as an extra safety rail:
|
|
210
98
|
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
config.explain_analyze_enabled = true
|
|
217
|
-
config.slow_query_threshold_ms = 500
|
|
218
|
-
|
|
219
|
-
# Or use a higher threshold for production
|
|
220
|
-
# config.slow_query_threshold_ms = 1000 # Only explain queries > 1 second
|
|
221
|
-
end
|
|
222
|
-
```
|
|
99
|
+
```ruby
|
|
100
|
+
DeadBro.configure do |config|
|
|
101
|
+
config.explain_analyze_enabled = true
|
|
102
|
+
end
|
|
103
|
+
```
|
|
223
104
|
|
|
224
|
-
|
|
105
|
+
- Everything else — the `slow_query_threshold_ms` cutoff and enabling/disabling the feature — is managed from the dashboard.
|
|
225
106
|
|
|
226
|
-
When a slow query is detected, the `explain_plan` field in the SQL query data
|
|
227
|
-
- **PostgreSQL / MySQL**: `EXPLAIN` output — the planner's chosen strategy, estimated costs and row counts
|
|
228
|
-
- **SQLite**: `EXPLAIN QUERY PLAN` output
|
|
229
|
-
- **Other databases**: standard `EXPLAIN` output
|
|
230
|
-
|
|
231
|
-
This execution plan helps you:
|
|
232
|
-
- Identify missing indexes
|
|
233
|
-
- Understand query execution order
|
|
234
|
-
- Spot full table scans
|
|
235
|
-
- Optimize JOIN operations
|
|
236
|
-
|
|
237
|
-
Because the statement is not executed, plans show the planner's *estimates* rather than actual runtimes — usually exactly what you need to spot a missing index or an unexpected sequential scan.
|
|
107
|
+
When a slow query is detected, the `explain_plan` field in the SQL query data contains the database's `EXPLAIN` (or `EXPLAIN QUERY PLAN` on SQLite) output — the planner's chosen strategy, estimated costs and row counts — useful for spotting missing indexes or unexpected sequential scans. Because the statement is never executed, plans show estimates rather than actual runtimes.
|
|
238
108
|
|
|
239
109
|
## View Rendering Tracking
|
|
240
110
|
|
|
@@ -259,20 +129,7 @@ By default, DeadBro uses **lightweight memory tracking** that has minimal perfor
|
|
|
259
129
|
- **GC Efficiency Analysis**: Monitor garbage collection effectiveness
|
|
260
130
|
- **Zero Allocation Tracking**: No object allocation tracking by default (can be enabled)
|
|
261
131
|
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
Usually managed in the dashboard; Ruby example:
|
|
265
|
-
|
|
266
|
-
```ruby
|
|
267
|
-
DeadBro.configure do |config|
|
|
268
|
-
config.memory_tracking_enabled = true # Enable lightweight memory tracking (default: true)
|
|
269
|
-
config.allocation_tracking_enabled = false # Enable detailed allocation tracking (default: false)
|
|
270
|
-
config.allocation_sample_rate = 100 # % of requests that pay for allocation tracking when enabled (1-100, default: 100)
|
|
271
|
-
|
|
272
|
-
# Sampling configuration
|
|
273
|
-
config.sample_rate = 100 # Percentage of requests to track (1-100, default: 100)
|
|
274
|
-
end
|
|
275
|
-
```
|
|
132
|
+
`memory_tracking_enabled`, `allocation_tracking_enabled`, and `allocation_sample_rate` are all managed from the dashboard.
|
|
276
133
|
|
|
277
134
|
**Performance Impact:**
|
|
278
135
|
- **Lightweight mode** (`memory_tracking_enabled`, ~0.1ms overhead per request): RSS before/after, GC pressure, the retained-vs-transient signals (`heap_live_slots_growth`, `malloc_increase_bytes`), and per-phase allocation attribution (`allocation_phases` — which of `sql`/`view`/`elasticsearch` allocated the request's objects).
|
|
@@ -310,25 +167,7 @@ Everything is **best effort** and designed to be **safe and low overhead**:
|
|
|
310
167
|
|
|
311
168
|
### Configuration
|
|
312
169
|
|
|
313
|
-
Enable or disable
|
|
314
|
-
|
|
315
|
-
```ruby
|
|
316
|
-
DeadBro.configure do |config|
|
|
317
|
-
# Enable the periodic job queue monitor (disabled by default)
|
|
318
|
-
config.job_queue_monitoring_enabled = true
|
|
319
|
-
|
|
320
|
-
# Enable best-effort collectors (all default to false)
|
|
321
|
-
config.enable_db_stats = true # ActiveRecord pool + ping latency
|
|
322
|
-
config.enable_process_stats = true # pid, hostname, RSS, GC, threads, fds
|
|
323
|
-
config.enable_system_stats = true # CPU%, memory, disk, network
|
|
324
|
-
|
|
325
|
-
# Filesystem paths to report disk usage for (default: ["/"])
|
|
326
|
-
config.disk_paths = ["/", "/var"]
|
|
327
|
-
|
|
328
|
-
# Network interfaces to ignore when computing rx/tx stats
|
|
329
|
-
config.interfaces_ignore = %w[lo docker0]
|
|
330
|
-
end
|
|
331
|
-
```
|
|
170
|
+
Enable or disable the job queue monitor and the individual collectors (`enable_db_stats`, `enable_process_stats`, `enable_system_stats`), plus `disk_paths` and `interfaces_ignore`, from the DeadBro dashboard.
|
|
332
171
|
|
|
333
172
|
### Example Payload Shape
|
|
334
173
|
|
|
@@ -17,7 +17,7 @@ module DeadBro
|
|
|
17
17
|
|
|
18
18
|
# Remote-managed settings (overwritten by backend JSON `settings` on successful API responses)
|
|
19
19
|
attr_accessor :memory_tracking_enabled, :allocation_tracking_enabled, :allocation_sample_rate,
|
|
20
|
-
:sample_rate, :slow_query_threshold_ms,
|
|
20
|
+
:sample_rate, :slow_query_threshold_ms, :watch_enabled,
|
|
21
21
|
:monitor_enabled, :enable_db_stats, :enable_process_stats, :enable_system_stats,
|
|
22
22
|
:max_sql_queries_to_send, :max_logs_to_send
|
|
23
23
|
|
|
@@ -67,7 +67,7 @@ module DeadBro
|
|
|
67
67
|
REMOTE_SETTING_KEYS = %w[
|
|
68
68
|
enabled sample_rate memory_tracking_enabled allocation_tracking_enabled allocation_sample_rate
|
|
69
69
|
explain_analyze_enabled slow_query_threshold_ms max_sql_queries_to_send max_logs_to_send
|
|
70
|
-
excluded_controllers excluded_jobs exclusive_controllers exclusive_jobs
|
|
70
|
+
watch_enabled excluded_controllers excluded_jobs exclusive_controllers exclusive_jobs
|
|
71
71
|
monitor_enabled enable_db_stats enable_process_stats enable_system_stats
|
|
72
72
|
sample_rates_by_type
|
|
73
73
|
].freeze
|
|
@@ -102,6 +102,7 @@ module DeadBro
|
|
|
102
102
|
@slow_query_threshold_ms = 500
|
|
103
103
|
@max_sql_queries_to_send = 500
|
|
104
104
|
@max_logs_to_send = 100
|
|
105
|
+
@watch_enabled = false
|
|
105
106
|
self.excluded_controllers = []
|
|
106
107
|
self.excluded_jobs = []
|
|
107
108
|
self.exclusive_controllers = []
|
|
@@ -178,7 +179,7 @@ module DeadBro
|
|
|
178
179
|
# must never be able to switch it on — only off. The local opt-in
|
|
179
180
|
# (explain_analyze_enabled) stays untouched; see #explain_analyze_active?
|
|
180
181
|
@remote_explain_analyze_enabled = !!value
|
|
181
|
-
when "enabled", "memory_tracking_enabled", "allocation_tracking_enabled",
|
|
182
|
+
when "enabled", "memory_tracking_enabled", "allocation_tracking_enabled", "watch_enabled",
|
|
182
183
|
"monitor_enabled", "enable_db_stats", "enable_process_stats", "enable_system_stats"
|
|
183
184
|
send(:"#{k}=", !!value)
|
|
184
185
|
when "excluded_controllers", "excluded_jobs", "exclusive_controllers", "exclusive_jobs"
|
|
@@ -208,6 +209,10 @@ module DeadBro
|
|
|
208
209
|
Time.now.utc < t
|
|
209
210
|
end
|
|
210
211
|
|
|
212
|
+
def watch_enabled?
|
|
213
|
+
!!@watch_enabled
|
|
214
|
+
end
|
|
215
|
+
|
|
211
216
|
def resolve_deploy_id
|
|
212
217
|
explicit = @explicit_deploy_revision&.to_s&.strip
|
|
213
218
|
return explicit unless explicit.nil? || explicit.empty?
|
|
@@ -40,6 +40,8 @@ module DeadBro
|
|
|
40
40
|
if defined?(DeadBro::DbConnectionSubscriber)
|
|
41
41
|
DeadBro::DbConnectionSubscriber.start_request_tracking
|
|
42
42
|
end
|
|
43
|
+
|
|
44
|
+
DeadBro::WatchTracker.start_request_tracking if defined?(DeadBro::WatchTracker)
|
|
43
45
|
end
|
|
44
46
|
rescue
|
|
45
47
|
# Never raise from instrumentation install
|
|
@@ -61,6 +61,7 @@ module DeadBro
|
|
|
61
61
|
DeadBro::SqlSubscriber.start_request_tracking
|
|
62
62
|
start_job_dependency_tracking
|
|
63
63
|
DeadBro::DbConnectionSubscriber.start_request_tracking if defined?(DeadBro::DbConnectionSubscriber)
|
|
64
|
+
DeadBro::WatchTracker.start_request_tracking if defined?(DeadBro::WatchTracker)
|
|
64
65
|
if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
|
|
65
66
|
DeadBro::MemoryTrackingSubscriber.start_request_tracking
|
|
66
67
|
else
|
|
@@ -74,6 +75,7 @@ module DeadBro
|
|
|
74
75
|
db_connection_stats = defined?(DeadBro::DbConnectionSubscriber) ? DeadBro::DbConnectionSubscriber.stop_request_tracking : {}
|
|
75
76
|
gc_pressure = defined?(DeadBro::GcTracker) ? DeadBro::GcTracker.stop_request_tracking : {}
|
|
76
77
|
ar_instantiation_count = defined?(DeadBro::ArObjectTracker) ? DeadBro::ArObjectTracker.stop_request_tracking : nil
|
|
78
|
+
watch_events = defined?(DeadBro::WatchTracker) ? DeadBro::WatchTracker.stop_request_tracking : []
|
|
77
79
|
|
|
78
80
|
# Stop memory tracking and get collected memory data
|
|
79
81
|
if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
|
|
@@ -124,6 +126,7 @@ module DeadBro
|
|
|
124
126
|
gc_stats: gc_stats,
|
|
125
127
|
memory_events: memory_events,
|
|
126
128
|
memory_performance: memory_performance,
|
|
129
|
+
watch_events: watch_events,
|
|
127
130
|
logs: DeadBro.logger.logs
|
|
128
131
|
}.merge(dependency_events)
|
|
129
132
|
|
|
@@ -162,6 +165,7 @@ module DeadBro
|
|
|
162
165
|
DeadBro::SqlSubscriber.start_request_tracking
|
|
163
166
|
start_job_dependency_tracking
|
|
164
167
|
DeadBro::DbConnectionSubscriber.start_request_tracking if defined?(DeadBro::DbConnectionSubscriber)
|
|
168
|
+
DeadBro::WatchTracker.start_request_tracking if defined?(DeadBro::WatchTracker)
|
|
165
169
|
if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
|
|
166
170
|
DeadBro::MemoryTrackingSubscriber.start_request_tracking
|
|
167
171
|
else
|
|
@@ -175,6 +179,7 @@ module DeadBro
|
|
|
175
179
|
db_connection_stats = defined?(DeadBro::DbConnectionSubscriber) ? DeadBro::DbConnectionSubscriber.stop_request_tracking : {}
|
|
176
180
|
gc_pressure = defined?(DeadBro::GcTracker) ? DeadBro::GcTracker.stop_request_tracking : {}
|
|
177
181
|
ar_instantiation_count = defined?(DeadBro::ArObjectTracker) ? DeadBro::ArObjectTracker.stop_request_tracking : nil
|
|
182
|
+
watch_events = defined?(DeadBro::WatchTracker) ? DeadBro::WatchTracker.stop_request_tracking : []
|
|
178
183
|
|
|
179
184
|
# Stop memory tracking and get collected memory data
|
|
180
185
|
if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
|
|
@@ -228,6 +233,7 @@ module DeadBro
|
|
|
228
233
|
gc_stats: gc_stats,
|
|
229
234
|
memory_events: memory_events,
|
|
230
235
|
memory_performance: memory_performance,
|
|
236
|
+
watch_events: watch_events,
|
|
231
237
|
logs: DeadBro.logger.logs
|
|
232
238
|
}.merge(dependency_events)
|
|
233
239
|
|
|
@@ -255,6 +261,7 @@ module DeadBro
|
|
|
255
261
|
if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
|
|
256
262
|
DeadBro::MemoryTrackingSubscriber.stop_request_tracking
|
|
257
263
|
end
|
|
264
|
+
DeadBro::WatchTracker.stop_request_tracking if defined?(DeadBro::WatchTracker)
|
|
258
265
|
rescue
|
|
259
266
|
# Best effort
|
|
260
267
|
end
|
|
@@ -50,6 +50,21 @@ module DeadBro
|
|
|
50
50
|
stack.last
|
|
51
51
|
end
|
|
52
52
|
|
|
53
|
+
# Sum of SQL counts and durations recorded so far in the active tracking
|
|
54
|
+
# context. Used by WatchTracker to attribute SQL to a DeadBro.watch block.
|
|
55
|
+
def self.current_sql_metrics
|
|
56
|
+
agg_stack = Thread.current[THREAD_LOCAL_AGGREGATES_KEY]
|
|
57
|
+
aggregates_h = (agg_stack.is_a?(Array) && agg_stack.any?) ? agg_stack.last : {}
|
|
58
|
+
return {count: 0, duration_ms: 0.0} unless aggregates_h.is_a?(Hash) && aggregates_h.any?
|
|
59
|
+
|
|
60
|
+
{
|
|
61
|
+
count: aggregates_h.values.sum { |a| (a[:count] || 0).to_i },
|
|
62
|
+
duration_ms: aggregates_h.values.sum { |a| (a[:total_duration_ms] || 0.0).to_f }.round(2)
|
|
63
|
+
}
|
|
64
|
+
rescue StandardError
|
|
65
|
+
{count: 0, duration_ms: 0.0}
|
|
66
|
+
end
|
|
67
|
+
|
|
53
68
|
# Check if we should continue tracking based on count and time limits
|
|
54
69
|
def self.should_continue_tracking?(current_queries_array, max_count)
|
|
55
70
|
return false unless current_queries_array.is_a?(Array)
|
|
@@ -82,6 +82,9 @@ module DeadBro
|
|
|
82
82
|
# Start CPU time tracking for this request (thread-local clock)
|
|
83
83
|
DeadBro::CpuTracker.start_request_tracking if defined?(DeadBro::CpuTracker)
|
|
84
84
|
|
|
85
|
+
# Start user-defined watch block tracking for this request
|
|
86
|
+
DeadBro::WatchTracker.start_request_tracking if defined?(DeadBro::WatchTracker)
|
|
87
|
+
|
|
85
88
|
# Start outgoing HTTP accumulation for this request
|
|
86
89
|
Thread.current[:dead_bro_http_events] = []
|
|
87
90
|
|
data/lib/dead_bro/subscriber.rb
CHANGED
|
@@ -80,6 +80,8 @@ module DeadBro
|
|
|
80
80
|
view_events = DeadBro::ViewRenderingSubscriber.stop_request_tracking
|
|
81
81
|
view_performance = DeadBro::ViewRenderingSubscriber.analyze_view_performance(view_events)
|
|
82
82
|
|
|
83
|
+
watch_events = defined?(DeadBro::WatchTracker) ? DeadBro::WatchTracker.stop_request_tracking : []
|
|
84
|
+
|
|
83
85
|
# Per-phase allocation attribution (under memory tracking) — which phase
|
|
84
86
|
# allocated the request's objects (sql / view / elasticsearch).
|
|
85
87
|
allocation_phases = if DeadBro.configuration.memory_tracking_enabled && defined?(DeadBro::MemoryPhaseTracker)
|
|
@@ -162,6 +164,7 @@ module DeadBro
|
|
|
162
164
|
duration_ms: duration_ms,
|
|
163
165
|
rails_env: DeadBro.env,
|
|
164
166
|
host: DeadBro.safe_hostname,
|
|
167
|
+
request_host: safe_request_host(data),
|
|
165
168
|
process_kind: DeadBro.process_kind,
|
|
166
169
|
params: safe_params(data),
|
|
167
170
|
user_agent: safe_user_agent(data),
|
|
@@ -195,6 +198,7 @@ module DeadBro
|
|
|
195
198
|
view_runtime_ms: data[:view_runtime],
|
|
196
199
|
db_runtime_ms: data[:db_runtime],
|
|
197
200
|
host: DeadBro.safe_hostname,
|
|
201
|
+
request_host: safe_request_host(data),
|
|
198
202
|
rails_env: DeadBro.env,
|
|
199
203
|
process_kind: DeadBro.process_kind,
|
|
200
204
|
params: safe_params(data),
|
|
@@ -222,6 +226,7 @@ module DeadBro
|
|
|
222
226
|
gc_pressure: gc_pressure,
|
|
223
227
|
ar_instantiation_count: ar_instantiation_count,
|
|
224
228
|
cpu_time_ms: cpu_time_ms,
|
|
229
|
+
watch_events: watch_events,
|
|
225
230
|
logs: DeadBro.logger.logs
|
|
226
231
|
}
|
|
227
232
|
# force: true — the sampling decision (global or per-request-type) was
|
|
@@ -254,6 +259,7 @@ module DeadBro
|
|
|
254
259
|
DeadBro::GcTracker.stop_request_tracking if defined?(DeadBro::GcTracker)
|
|
255
260
|
DeadBro::ArObjectTracker.stop_request_tracking if defined?(DeadBro::ArObjectTracker)
|
|
256
261
|
DeadBro::CpuTracker.stop_request_tracking if defined?(DeadBro::CpuTracker)
|
|
262
|
+
DeadBro::WatchTracker.stop_request_tracking if defined?(DeadBro::WatchTracker)
|
|
257
263
|
rescue
|
|
258
264
|
# Best effort — draining must never raise from the notifications callback.
|
|
259
265
|
end
|
|
@@ -326,6 +332,37 @@ module DeadBro
|
|
|
326
332
|
end
|
|
327
333
|
end
|
|
328
334
|
|
|
335
|
+
# The domain the request was served on (request.host) — distinct from `host`,
|
|
336
|
+
# which is the machine's OS hostname. Lets one app that answers on several
|
|
337
|
+
# domains be sliced by domain in the dashboard. Port and userinfo are dropped;
|
|
338
|
+
# value is lowercased so "Example.com" and "example.com" aren't two rows.
|
|
339
|
+
def self.safe_request_host(data)
|
|
340
|
+
raw =
|
|
341
|
+
if data[:request] && data[:request].respond_to?(:host)
|
|
342
|
+
data[:request].host
|
|
343
|
+
elsif data[:headers]
|
|
344
|
+
headers = data[:headers]
|
|
345
|
+
if headers.respond_to?(:[])
|
|
346
|
+
headers["HTTP_HOST"] || headers["Host"] || headers["host"]
|
|
347
|
+
elsif headers.respond_to?(:env)
|
|
348
|
+
headers.env && headers.env["HTTP_HOST"]
|
|
349
|
+
end
|
|
350
|
+
elsif data[:env].is_a?(Hash)
|
|
351
|
+
data[:env]["HTTP_HOST"]
|
|
352
|
+
end
|
|
353
|
+
|
|
354
|
+
host = sanitize_string(raw)
|
|
355
|
+
return "" if host.empty?
|
|
356
|
+
|
|
357
|
+
# Strip any userinfo@ and a trailing :port so only the hostname remains.
|
|
358
|
+
# Only a numeric trailing port is removed, so IPv6 literals ([::1]) survive.
|
|
359
|
+
host = host.split("@").last.to_s
|
|
360
|
+
host = host.sub(/:\d+\z/, "")
|
|
361
|
+
host.downcase[0, 255]
|
|
362
|
+
rescue
|
|
363
|
+
""
|
|
364
|
+
end
|
|
365
|
+
|
|
329
366
|
def self.safe_user_agent(data)
|
|
330
367
|
begin
|
|
331
368
|
# Prefer request object if available
|
data/lib/dead_bro/version.rb
CHANGED
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module DeadBro
|
|
4
|
+
# Records user-defined timed blocks (DeadBro.watch) during an instrumented
|
|
5
|
+
# request or background job. Spans include wall duration, nesting depth, and
|
|
6
|
+
# SQL attributable to the block (delta from SqlSubscriber aggregates).
|
|
7
|
+
#
|
|
8
|
+
# SQL attribution: each span diffs the global SQL counter at block entry vs
|
|
9
|
+
# exit. Nested blocks each get their own diff; a parent span's sql_count /
|
|
10
|
+
# sql_duration_ms therefore includes queries run inside nested child spans
|
|
11
|
+
# (inclusive totals, not exclusive). Compare inner spans when you need
|
|
12
|
+
# per-block SQL; do not sum parent and child counts.
|
|
13
|
+
#
|
|
14
|
+
# Unlike DeadBro.analyze, watch spans are sent to the DeadBro backend as part
|
|
15
|
+
# of the normal request/job payload when watch_enabled is on.
|
|
16
|
+
module WatchTracker
|
|
17
|
+
THREAD_LOCAL_EVENTS_KEY = :dead_bro_watch_events
|
|
18
|
+
THREAD_LOCAL_ACTIVE_STACK_KEY = :dead_bro_watch_active_stack
|
|
19
|
+
|
|
20
|
+
# Deepest depth index still recorded — depths 0..MAX_DEPTH are kept, anything
|
|
21
|
+
# nested below that yields untracked.
|
|
22
|
+
MAX_DEPTH = 10
|
|
23
|
+
MAX_SPANS_PER_REQUEST = 50
|
|
24
|
+
MAX_LABEL_LENGTH = 200
|
|
25
|
+
MAX_TAGS = 5
|
|
26
|
+
MAX_TAG_VALUE_LENGTH = 100
|
|
27
|
+
|
|
28
|
+
module_function
|
|
29
|
+
|
|
30
|
+
def enabled?
|
|
31
|
+
DeadBro.configuration.watch_enabled?
|
|
32
|
+
rescue StandardError
|
|
33
|
+
false
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def tracking_active?
|
|
37
|
+
Thread.current[THREAD_LOCAL_EVENTS_KEY].is_a?(Array)
|
|
38
|
+
rescue StandardError
|
|
39
|
+
false
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def start_request_tracking
|
|
43
|
+
Thread.current[THREAD_LOCAL_EVENTS_KEY] = []
|
|
44
|
+
Thread.current[THREAD_LOCAL_ACTIVE_STACK_KEY] = []
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def stop_request_tracking
|
|
48
|
+
events = Thread.current[THREAD_LOCAL_EVENTS_KEY]
|
|
49
|
+
Thread.current[THREAD_LOCAL_EVENTS_KEY] = nil
|
|
50
|
+
Thread.current[THREAD_LOCAL_ACTIVE_STACK_KEY] = nil
|
|
51
|
+
# compact drops reserved slots whose block never completed — can't happen
|
|
52
|
+
# for a lexical block, but keeps a nil out of the payload if it ever does.
|
|
53
|
+
events.is_a?(Array) ? events.compact : []
|
|
54
|
+
rescue StandardError
|
|
55
|
+
[]
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
def watch(label = nil, **tags)
|
|
59
|
+
return yield unless block_given?
|
|
60
|
+
return yield unless enabled?
|
|
61
|
+
return yield unless tracking_active?
|
|
62
|
+
|
|
63
|
+
events = Thread.current[THREAD_LOCAL_EVENTS_KEY]
|
|
64
|
+
active_stack = Thread.current[THREAD_LOCAL_ACTIVE_STACK_KEY]
|
|
65
|
+
return yield unless events.is_a?(Array) && active_stack.is_a?(Array)
|
|
66
|
+
|
|
67
|
+
depth = active_stack.length
|
|
68
|
+
return yield if depth > MAX_DEPTH
|
|
69
|
+
return yield if events.length >= MAX_SPANS_PER_REQUEST
|
|
70
|
+
|
|
71
|
+
sanitized_label = sanitize_label(label)
|
|
72
|
+
sanitized_tags = sanitize_tags(tags)
|
|
73
|
+
start_offset_ms = compute_start_offset_ms
|
|
74
|
+
sql_before = sql_metrics_snapshot
|
|
75
|
+
block_start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
76
|
+
|
|
77
|
+
frame = {
|
|
78
|
+
label: sanitized_label,
|
|
79
|
+
depth: depth,
|
|
80
|
+
tags: sanitized_tags,
|
|
81
|
+
start_offset_ms: start_offset_ms,
|
|
82
|
+
sql_before: sql_before,
|
|
83
|
+
block_start: block_start,
|
|
84
|
+
error: false,
|
|
85
|
+
exception_class: nil
|
|
86
|
+
}
|
|
87
|
+
# Reserve the slot on entry so spans come out in start order. Blocks only
|
|
88
|
+
# complete inner-first, so appending on completion would emit a nested span
|
|
89
|
+
# ahead of its own parent; the placeholder keeps parents before children.
|
|
90
|
+
# It also makes MAX_SPANS_PER_REQUEST count blocks still in flight.
|
|
91
|
+
frame[:slot] = events.length
|
|
92
|
+
events << nil
|
|
93
|
+
active_stack << frame
|
|
94
|
+
|
|
95
|
+
begin
|
|
96
|
+
yield
|
|
97
|
+
rescue StandardError => e
|
|
98
|
+
frame[:error] = true
|
|
99
|
+
# Anonymous classes (Class.new(StandardError)) have a nil name — leave the
|
|
100
|
+
# field unset rather than shipping an empty string the dashboard would
|
|
101
|
+
# render as a blank exception.
|
|
102
|
+
class_name = e.class.name
|
|
103
|
+
frame[:exception_class] = class_name[0, 200] if class_name
|
|
104
|
+
raise
|
|
105
|
+
ensure
|
|
106
|
+
active_stack.pop if active_stack.last.equal?(frame)
|
|
107
|
+
append_completed_span(events, frame)
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
def sanitize_label(label)
|
|
112
|
+
text = label.to_s.strip
|
|
113
|
+
text = "block" if text.empty?
|
|
114
|
+
(text.length > MAX_LABEL_LENGTH) ? text[0, MAX_LABEL_LENGTH] + "..." : text
|
|
115
|
+
rescue StandardError
|
|
116
|
+
"block"
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
def sanitize_tags(tags)
|
|
120
|
+
return {} unless tags.is_a?(Hash) && tags.any?
|
|
121
|
+
|
|
122
|
+
tags.first(MAX_TAGS).each_with_object({}) do |(key, value), out|
|
|
123
|
+
k = key.to_s.strip[0, 50]
|
|
124
|
+
next if k.empty?
|
|
125
|
+
|
|
126
|
+
v = value.to_s.strip
|
|
127
|
+
v = v[0, MAX_TAG_VALUE_LENGTH] + "..." if v.length > MAX_TAG_VALUE_LENGTH
|
|
128
|
+
out[k] = v
|
|
129
|
+
end
|
|
130
|
+
rescue StandardError
|
|
131
|
+
{}
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
def compute_start_offset_ms
|
|
135
|
+
tracking_start = Thread.current[DeadBro::TRACKING_START_TIME_KEY]
|
|
136
|
+
return 0.0 unless tracking_start
|
|
137
|
+
|
|
138
|
+
((Time.now - tracking_start) * 1000.0).round(2)
|
|
139
|
+
rescue StandardError
|
|
140
|
+
0.0
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
def sql_metrics_snapshot
|
|
144
|
+
return {count: 0, duration_ms: 0.0} unless defined?(DeadBro::SqlSubscriber)
|
|
145
|
+
|
|
146
|
+
DeadBro::SqlSubscriber.current_sql_metrics
|
|
147
|
+
rescue StandardError
|
|
148
|
+
{count: 0, duration_ms: 0.0}
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
def append_completed_span(events, frame)
|
|
152
|
+
return unless events.is_a?(Array)
|
|
153
|
+
|
|
154
|
+
slot = frame[:slot]
|
|
155
|
+
return unless slot.is_a?(Integer) && slot < events.length
|
|
156
|
+
|
|
157
|
+
elapsed_ms = begin
|
|
158
|
+
((Process.clock_gettime(Process::CLOCK_MONOTONIC) - frame[:block_start]) * 1000.0).round(2)
|
|
159
|
+
rescue StandardError
|
|
160
|
+
0.0
|
|
161
|
+
end
|
|
162
|
+
|
|
163
|
+
sql_after = sql_metrics_snapshot
|
|
164
|
+
sql_before = frame[:sql_before] || {count: 0, duration_ms: 0.0}
|
|
165
|
+
sql_count = [(sql_after[:count] || 0).to_i - (sql_before[:count] || 0).to_i, 0].max
|
|
166
|
+
sql_duration_ms = [
|
|
167
|
+
(sql_after[:duration_ms] || 0.0).to_f - (sql_before[:duration_ms] || 0.0).to_f,
|
|
168
|
+
0.0
|
|
169
|
+
].max.round(2)
|
|
170
|
+
|
|
171
|
+
span = {
|
|
172
|
+
label: frame[:label],
|
|
173
|
+
duration_ms: elapsed_ms,
|
|
174
|
+
start_offset_ms: frame[:start_offset_ms],
|
|
175
|
+
depth: frame[:depth],
|
|
176
|
+
sql_count: sql_count,
|
|
177
|
+
sql_duration_ms: sql_duration_ms,
|
|
178
|
+
error: frame[:error] == true
|
|
179
|
+
}
|
|
180
|
+
span[:exception_class] = frame[:exception_class] if frame[:exception_class]
|
|
181
|
+
tags = frame[:tags]
|
|
182
|
+
span[:tags] = tags if tags.is_a?(Hash) && tags.any?
|
|
183
|
+
|
|
184
|
+
events[slot] = span
|
|
185
|
+
rescue StandardError
|
|
186
|
+
# Never raise from instrumentation
|
|
187
|
+
end
|
|
188
|
+
end
|
|
189
|
+
end
|
data/lib/dead_bro.rb
CHANGED
|
@@ -29,6 +29,7 @@ module DeadBro
|
|
|
29
29
|
autoload :Monitor, "dead_bro/monitor"
|
|
30
30
|
autoload :MemoryDetails, "dead_bro/memory_details"
|
|
31
31
|
autoload :Logger, "dead_bro/logger"
|
|
32
|
+
autoload :WatchTracker, "dead_bro/watch_tracker"
|
|
32
33
|
begin
|
|
33
34
|
require "dead_bro/railtie"
|
|
34
35
|
rescue LoadError
|
|
@@ -458,4 +459,28 @@ module DeadBro
|
|
|
458
459
|
raise error if error
|
|
459
460
|
analysis_result
|
|
460
461
|
end
|
|
462
|
+
|
|
463
|
+
# Time a named block during an instrumented request or background job.
|
|
464
|
+
#
|
|
465
|
+
# When watch_enabled is on and tracking is active (web request or job),
|
|
466
|
+
# records elapsed time, optional tags, nested SQL count/duration, and errors
|
|
467
|
+
# as watch_events in the APM payload for the Request Trace waterfall.
|
|
468
|
+
#
|
|
469
|
+
# Nested watch blocks: each span's sql_count/sql_duration_ms is the SQL that
|
|
470
|
+
# ran while that block was active. Parent spans include SQL from nested child
|
|
471
|
+
# spans (inclusive). Inner spans show their own slice; do not add parent and
|
|
472
|
+
# child SQL counts together.
|
|
473
|
+
#
|
|
474
|
+
# When disabled or outside an instrumented context, yields with no overhead
|
|
475
|
+
# beyond a single config check.
|
|
476
|
+
#
|
|
477
|
+
# Usage:
|
|
478
|
+
# DeadBro.watch("load users") { User.where(active: true).load }
|
|
479
|
+
# DeadBro.watch("sync billing", customer_id: user.id) { sync!(user) }
|
|
480
|
+
#
|
|
481
|
+
# Compare to DeadBro.analyze — analyze is local-only console profiling;
|
|
482
|
+
# watch spans are sent to the DeadBro dashboard when enabled.
|
|
483
|
+
def self.watch(label = nil, **tags, &block)
|
|
484
|
+
WatchTracker.watch(label, **tags, &block)
|
|
485
|
+
end
|
|
461
486
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: dead_bro
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.2.
|
|
4
|
+
version: 0.2.31
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Emanuel Comsa
|
|
@@ -58,6 +58,7 @@ files:
|
|
|
58
58
|
- lib/dead_bro/subscriber.rb
|
|
59
59
|
- lib/dead_bro/version.rb
|
|
60
60
|
- lib/dead_bro/view_rendering_subscriber.rb
|
|
61
|
+
- lib/dead_bro/watch_tracker.rb
|
|
61
62
|
- lib/generators/dead_bro/install/install_generator.rb
|
|
62
63
|
- lib/generators/dead_bro/install/templates/dead_bro.rb
|
|
63
64
|
homepage: https://www.deadbro.com
|