dead_bro 0.2.26 → 0.2.28
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/FEATURES.md +333 -0
- data/README.md +21 -17
- data/lib/dead_bro/configuration.rb +23 -2
- data/lib/dead_bro/job_subscriber.rb +2 -1
- data/lib/dead_bro/sql_subscriber.rb +51 -31
- data/lib/dead_bro/subscriber.rb +3 -1
- data/lib/dead_bro/version.rb +1 -1
- data/lib/dead_bro.rb +1 -0
- metadata +3 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6b96d1dee673157c1b4c46ad86f7e0d4a4feb51774038de260c5464353ff292e
|
|
4
|
+
data.tar.gz: f922d71e3c1f132642bf5756aff381a5d372538672c3e0affd010cb31f30b397
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 20f9d81a03bb468cc4ded983ad77b1ad9191dde3ba5c5320b7dd1e11bb5295b5d90289f597df036770cb0d6740ec10f426cf75ba5c941a1922cce8785ee21fe8
|
|
7
|
+
data.tar.gz: c5325f9e3c8126236fb4386b9214d9637a4086176ae6e2675d4344958bc09d16bbb5718f6461f8bb8eaf75da4ef7e295d3ae012b15a5bb0ab1fbdea8ef0b8f48
|
data/FEATURES.md
ADDED
|
@@ -0,0 +1,333 @@
|
|
|
1
|
+
# ApmBro Feature List
|
|
2
|
+
|
|
3
|
+
A comprehensive feature list for comparing ApmBro with other APM (Application Performance Monitoring) tools.
|
|
4
|
+
|
|
5
|
+
## Core Architecture
|
|
6
|
+
|
|
7
|
+
- **Rails Integration**: Automatic subscription to Rails events via ActiveSupport::Notifications
|
|
8
|
+
- **Zero-Configuration Setup**: Works out of the box with minimal configuration
|
|
9
|
+
- **Asynchronous Metrics Posting**: Non-blocking HTTP requests using background threads
|
|
10
|
+
- **Thread-Local Storage**: Per-request metric collection using thread-local variables
|
|
11
|
+
- **Circuit Breaker Pattern**: Built-in circuit breaker to prevent cascading failures when APM endpoint is down
|
|
12
|
+
- **Deploy Tracking**: Automatic deploy ID resolution from multiple sources (Rails settings, ENV vars, Heroku, Git)
|
|
13
|
+
|
|
14
|
+
## Request Tracking
|
|
15
|
+
|
|
16
|
+
### Controller Action Monitoring
|
|
17
|
+
- **Automatic Tracking**: Tracks all controller actions automatically
|
|
18
|
+
- **Request Duration**: Measures total request processing time
|
|
19
|
+
- **HTTP Method & Path**: Captures HTTP method and request path
|
|
20
|
+
- **Status Codes**: Tracks HTTP response status codes
|
|
21
|
+
- **View Runtime**: Separate tracking of view rendering time
|
|
22
|
+
- **Database Runtime**: Separate tracking of database query time
|
|
23
|
+
- **Request Parameters**: Captures request parameters (with sensitive data filtering)
|
|
24
|
+
- **User Agent**: Tracks user agent strings
|
|
25
|
+
- **User ID Extraction**: Extracts authenticated user ID (supports Warden)
|
|
26
|
+
- **Environment Context**: Tracks Rails environment (development, staging, production)
|
|
27
|
+
|
|
28
|
+
### Request Sampling
|
|
29
|
+
- **Configurable Sample Rate**: Percentage-based sampling (1-100%)
|
|
30
|
+
- **Random Sampling**: Each request has random chance of being tracked
|
|
31
|
+
- **Consistent Per-Request**: Sampling decision applies to all metrics for a request
|
|
32
|
+
- **Error Override**: Errors are always tracked regardless of sampling
|
|
33
|
+
- **Cost Optimization**: Reduces data volume and costs for high-traffic applications
|
|
34
|
+
|
|
35
|
+
### Exclusion Rules
|
|
36
|
+
- **Controller Exclusion**: Exclude entire controllers from tracking
|
|
37
|
+
- **Action Exclusion**: Exclude specific controller#action combinations
|
|
38
|
+
- **Wildcard Support**: Pattern matching with `*` wildcards (e.g., `Admin::*`, `Admin::*#*`)
|
|
39
|
+
- **Job Exclusion**: Exclude specific background jobs from tracking
|
|
40
|
+
- **Flexible Configuration**: Configure via initializer, Rails settings, or environment variables
|
|
41
|
+
|
|
42
|
+
## SQL Query Tracking
|
|
43
|
+
|
|
44
|
+
### Query Details
|
|
45
|
+
- **Full SQL Tracking**: Captures all SQL queries executed during requests and jobs
|
|
46
|
+
- **Query Sanitization**: Automatically sanitizes SQL to remove sensitive data
|
|
47
|
+
- **Query Name**: Tracks query names (e.g., "User Load", "User Update")
|
|
48
|
+
- **Duration Measurement**: Precise query execution time in milliseconds
|
|
49
|
+
- **Cache Detection**: Identifies cached queries
|
|
50
|
+
- **Connection ID**: Tracks database connection ID
|
|
51
|
+
- **Call Stack Traces**: Full backtrace showing where queries were executed
|
|
52
|
+
- **Object Allocations**: Optional tracking of object allocations per query
|
|
53
|
+
|
|
54
|
+
### Query Performance Analysis
|
|
55
|
+
- **Slow Query Detection**: Configurable threshold for identifying slow queries
|
|
56
|
+
- **EXPLAIN ANALYZE**: Automatic execution plan capture for slow queries
|
|
57
|
+
- **Background Execution**: EXPLAIN ANALYZE runs in separate thread (non-blocking)
|
|
58
|
+
- **Multi-Database Support**: Works with PostgreSQL, MySQL, SQLite, and others
|
|
59
|
+
- **Smart Filtering**: Automatically skips transaction queries (BEGIN, COMMIT, ROLLBACK)
|
|
60
|
+
- **Execution Plan Details**:
|
|
61
|
+
- PostgreSQL: Full EXPLAIN ANALYZE with buffer usage statistics
|
|
62
|
+
- MySQL: EXPLAIN ANALYZE with actual execution times
|
|
63
|
+
- SQLite: EXPLAIN QUERY PLAN output
|
|
64
|
+
- **Query Optimization Insights**: Helps identify missing indexes, full table scans, JOIN issues
|
|
65
|
+
|
|
66
|
+
## View Rendering Tracking
|
|
67
|
+
|
|
68
|
+
### View Performance
|
|
69
|
+
- **Template Rendering**: Tracks main template rendering
|
|
70
|
+
- **Partial Rendering**: Tracks partial template rendering with cache key information
|
|
71
|
+
- **Collection Rendering**: Tracks collection rendering (partials in loops)
|
|
72
|
+
- **Rendering Duration**: Precise timing for each view component
|
|
73
|
+
- **Virtual Path Tracking**: Tracks view virtual paths
|
|
74
|
+
- **Layout Information**: Captures layout usage
|
|
75
|
+
|
|
76
|
+
### View Analysis
|
|
77
|
+
- **Slow View Detection**: Identifies the slowest rendering views
|
|
78
|
+
- **Frequency Analysis**: Tracks most frequently rendered views
|
|
79
|
+
- **Cache Hit Rate**: Calculates cache hit rates for partials
|
|
80
|
+
- **Collection Cache Analysis**: Tracks cache hit rates for collection rendering
|
|
81
|
+
- **Performance Metrics**:
|
|
82
|
+
- Total views rendered per request
|
|
83
|
+
- Total view rendering duration
|
|
84
|
+
- Average view rendering duration
|
|
85
|
+
- Breakdown by view type (template, partial, collection)
|
|
86
|
+
|
|
87
|
+
## Memory Tracking & Leak Detection
|
|
88
|
+
|
|
89
|
+
### Lightweight Memory Tracking (Default)
|
|
90
|
+
- **Memory Usage Monitoring**: Tracks memory consumption per request using GC stats
|
|
91
|
+
- **Memory Growth Tracking**: Measures memory growth during request processing
|
|
92
|
+
- **GC Statistics**: Tracks garbage collection count and heap pages
|
|
93
|
+
- **Minimal Performance Impact**: ~0.1ms overhead per request
|
|
94
|
+
- **Memory Before/After**: Captures memory state at request start and end
|
|
95
|
+
|
|
96
|
+
### Detailed Allocation Tracking (Optional)
|
|
97
|
+
- **Object Allocation Tracking**: Detailed tracking of object allocations (disabled by default)
|
|
98
|
+
- **Allocation Sampling**: Configurable sampling rate for allocations
|
|
99
|
+
- **Large Object Detection**: Identifies objects larger than 1MB threshold
|
|
100
|
+
- **Memory Snapshots**: Periodic memory snapshots during request processing
|
|
101
|
+
- **Object Count Tracking**: Tracks object counts before and after requests
|
|
102
|
+
- **Performance Impact**: ~2-5ms overhead per request (only when enabled)
|
|
103
|
+
|
|
104
|
+
### Memory Leak Detection
|
|
105
|
+
- **Pattern Detection**: Detects growing memory patterns over time
|
|
106
|
+
- **GC Efficiency Analysis**: Monitors garbage collection effectiveness
|
|
107
|
+
- **Heap Page Tracking**: Tracks heap page growth
|
|
108
|
+
- **Request Correlation**: Correlates memory growth with specific controllers/actions
|
|
109
|
+
|
|
110
|
+
## Background Job Tracking
|
|
111
|
+
|
|
112
|
+
### Job Execution Monitoring
|
|
113
|
+
- **ActiveJob Integration**: Automatic tracking when ActiveJob is available
|
|
114
|
+
- **Job Class Tracking**: Tracks job class names
|
|
115
|
+
- **Job ID**: Captures unique job identifiers
|
|
116
|
+
- **Queue Name**: Tracks which queue processed the job
|
|
117
|
+
- **Job Arguments**: Captures job arguments (with sensitive data filtering)
|
|
118
|
+
- **Duration Measurement**: Precise job execution time in milliseconds
|
|
119
|
+
- **Status Tracking**: Tracks job status (completed or failed)
|
|
120
|
+
|
|
121
|
+
### Job Error Tracking
|
|
122
|
+
- **Exception Capture**: Captures exceptions from failed jobs
|
|
123
|
+
- **Exception Class**: Tracks exception class names
|
|
124
|
+
- **Exception Messages**: Captures exception messages (truncated to 1000 chars)
|
|
125
|
+
- **Backtraces**: Full exception backtraces (first 50 lines)
|
|
126
|
+
- **SQL Query Context**: Includes SQL queries executed during failed jobs
|
|
127
|
+
- **Memory Context**: Includes memory usage during job execution
|
|
128
|
+
|
|
129
|
+
### Job SQL Tracking
|
|
130
|
+
- **SQL Query Tracking**: Tracks all SQL queries executed during job processing
|
|
131
|
+
- **Query Details**: Same detailed SQL tracking as request tracking
|
|
132
|
+
- **Query Context**: Full context of database operations in background jobs
|
|
133
|
+
|
|
134
|
+
## Cache Tracking
|
|
135
|
+
|
|
136
|
+
### Cache Operations
|
|
137
|
+
- **Read Operations**: Tracks cache read operations
|
|
138
|
+
- **Write Operations**: Tracks cache write operations
|
|
139
|
+
- **Delete Operations**: Tracks cache delete operations
|
|
140
|
+
- **Existence Checks**: Tracks cache existence checks
|
|
141
|
+
- **Fetch Operations**: Tracks cache fetch with hit/miss detection
|
|
142
|
+
- **Multi-Read Operations**: Tracks cache read_multi operations
|
|
143
|
+
- **Multi-Write Operations**: Tracks cache write_multi operations
|
|
144
|
+
- **Cache Generation**: Tracks cache generation events
|
|
145
|
+
|
|
146
|
+
### Cache Analysis
|
|
147
|
+
- **Cache Hit Detection**: Identifies cache hits vs misses
|
|
148
|
+
- **Cache Key Tracking**: Tracks cache keys (truncated to 200 chars)
|
|
149
|
+
- **Store Information**: Identifies which cache store was used
|
|
150
|
+
- **Namespace Tracking**: Tracks cache namespaces
|
|
151
|
+
- **Duration Measurement**: Precise timing for each cache operation
|
|
152
|
+
- **Hit Rate Calculation**: Calculates cache hit rates per request
|
|
153
|
+
|
|
154
|
+
## Redis Tracking
|
|
155
|
+
|
|
156
|
+
### Redis Command Tracking
|
|
157
|
+
- **Command Monitoring**: Tracks all Redis commands executed
|
|
158
|
+
- **Command Name**: Captures Redis command names (GET, SET, etc.)
|
|
159
|
+
- **Key Tracking**: Tracks Redis keys (truncated to 200 chars)
|
|
160
|
+
- **Argument Count**: Tracks number of arguments per command
|
|
161
|
+
- **Database Selection**: Tracks which Redis database is used
|
|
162
|
+
- **Duration Measurement**: Precise timing for each Redis command
|
|
163
|
+
- **Error Tracking**: Captures Redis command errors
|
|
164
|
+
|
|
165
|
+
### Advanced Redis Features
|
|
166
|
+
- **Pipeline Support**: Tracks Redis pipeline operations with command counts
|
|
167
|
+
- **Multi/Transaction Support**: Tracks Redis MULTI/EXEC transactions
|
|
168
|
+
- **ActiveSupport Integration**: Subscribes to ActiveSupport::Notifications for Redis events
|
|
169
|
+
- **Client Instrumentation**: Direct instrumentation of Redis::Client for comprehensive coverage
|
|
170
|
+
|
|
171
|
+
## Error Tracking
|
|
172
|
+
|
|
173
|
+
### Exception Handling
|
|
174
|
+
- **Automatic Exception Capture**: Captures exceptions from controller actions
|
|
175
|
+
- **Exception Class**: Tracks exception class names
|
|
176
|
+
- **Exception Messages**: Captures exception messages (truncated to 1000 chars)
|
|
177
|
+
- **Full Backtraces**: Captures complete exception backtraces (first 50 lines)
|
|
178
|
+
- **Request Context**: Includes full request context with exceptions
|
|
179
|
+
- **Error Flagging**: Errors are marked and always tracked (even with sampling)
|
|
180
|
+
|
|
181
|
+
### Error Context
|
|
182
|
+
- **Controller/Action**: Identifies where the error occurred
|
|
183
|
+
- **Request Parameters**: Includes request parameters at time of error
|
|
184
|
+
- **User Information**: Includes user ID if available
|
|
185
|
+
- **SQL Queries**: Includes SQL queries executed before error
|
|
186
|
+
- **Memory State**: Includes memory usage at time of error
|
|
187
|
+
- **Log Messages**: Includes application logs captured during request
|
|
188
|
+
|
|
189
|
+
## HTTP Instrumentation
|
|
190
|
+
|
|
191
|
+
### Outgoing HTTP Tracking
|
|
192
|
+
- **HTTP Request Tracking**: Tracks outgoing HTTP requests (via middleware)
|
|
193
|
+
- **Request Context**: Captures HTTP request details
|
|
194
|
+
- **Response Context**: Captures HTTP response details
|
|
195
|
+
- **Duration Measurement**: Tracks HTTP request duration
|
|
196
|
+
|
|
197
|
+
## Configuration & Flexibility
|
|
198
|
+
|
|
199
|
+
### Configuration Options
|
|
200
|
+
- **API Key Management**: Multiple sources (config, Rails credentials, ENV)
|
|
201
|
+
- **Endpoint Configuration**: Configurable endpoint URL
|
|
202
|
+
- **Timeout Settings**: Configurable open and read timeouts
|
|
203
|
+
- **Enable/Disable Toggle**: Can be enabled/disabled via configuration
|
|
204
|
+
- **Environment Detection**: Automatic Rails environment detection
|
|
205
|
+
|
|
206
|
+
### Circuit Breaker Configuration
|
|
207
|
+
- **Failure Threshold**: Configurable failure threshold (default: 3)
|
|
208
|
+
- **Recovery Timeout**: Configurable recovery timeout (default: 60 seconds)
|
|
209
|
+
- **Retry Timeout**: Configurable retry timeout (default: 300 seconds)
|
|
210
|
+
- **Enable/Disable**: Can enable/disable circuit breaker
|
|
211
|
+
|
|
212
|
+
### Memory Tracking Configuration
|
|
213
|
+
- **Memory Tracking Toggle**: Enable/disable memory tracking
|
|
214
|
+
- **Allocation Tracking Toggle**: Enable/disable detailed allocation tracking
|
|
215
|
+
- **Sampling Configuration**: Configurable request sampling rate
|
|
216
|
+
|
|
217
|
+
### Query Analysis Configuration
|
|
218
|
+
- **Slow Query Threshold**: Configurable threshold in milliseconds (default: 500ms)
|
|
219
|
+
- **EXPLAIN ANALYZE Toggle**: Enable/disable automatic EXPLAIN ANALYZE
|
|
220
|
+
|
|
221
|
+
## Data Safety & Privacy
|
|
222
|
+
|
|
223
|
+
### Data Sanitization
|
|
224
|
+
- **SQL Sanitization**: Automatically sanitizes SQL queries
|
|
225
|
+
- **Parameter Filtering**: Filters sensitive parameters (password, token, secret, key)
|
|
226
|
+
- **Argument Truncation**: Limits and truncates job arguments
|
|
227
|
+
- **Key Truncation**: Truncates cache and Redis keys to 200 characters
|
|
228
|
+
- **Value Truncation**: Recursively truncates nested values to prevent huge payloads
|
|
229
|
+
- **String Limits**: Limits string values (e.g., user agent to 200 chars, messages to 1000 chars)
|
|
230
|
+
|
|
231
|
+
### Data Limits
|
|
232
|
+
- **Array Limits**: Limits array sizes (e.g., first 10 job arguments, first 5 array elements)
|
|
233
|
+
- **Hash Limits**: Limits hash key counts (e.g., first 20 hash keys, first 30 params)
|
|
234
|
+
- **Backtrace Limits**: Limits backtraces to first 50 lines
|
|
235
|
+
- **Allocation Limits**: Limits allocations tracked per request (max 1000)
|
|
236
|
+
|
|
237
|
+
## Performance & Reliability
|
|
238
|
+
|
|
239
|
+
### Performance Optimizations
|
|
240
|
+
- **Asynchronous Posting**: Non-blocking HTTP requests
|
|
241
|
+
- **Lightweight Default Mode**: Minimal overhead in default configuration
|
|
242
|
+
- **Sampling Support**: Reduces data volume for high-traffic applications
|
|
243
|
+
- **Thread-Local Storage**: Efficient per-request data collection
|
|
244
|
+
- **Background EXPLAIN**: EXPLAIN ANALYZE runs in background thread
|
|
245
|
+
|
|
246
|
+
### Reliability Features
|
|
247
|
+
- **Circuit Breaker**: Prevents cascading failures
|
|
248
|
+
- **Error Handling**: Comprehensive error handling to prevent instrumentation failures
|
|
249
|
+
- **Graceful Degradation**: Continues working even if some features fail
|
|
250
|
+
- **Timeout Protection**: Configurable timeouts prevent hanging requests
|
|
251
|
+
|
|
252
|
+
## Integration & Compatibility
|
|
253
|
+
|
|
254
|
+
### Framework Support
|
|
255
|
+
- **Rails Integration**: Full Rails integration via Railtie
|
|
256
|
+
- **ActiveSupport Notifications**: Uses ActiveSupport::Notifications for event subscription
|
|
257
|
+
- **ActiveRecord Integration**: Tracks ActiveRecord SQL queries
|
|
258
|
+
- **ActiveJob Integration**: Tracks ActiveJob background jobs
|
|
259
|
+
- **ActionView Integration**: Tracks ActionView rendering
|
|
260
|
+
|
|
261
|
+
### Database Support
|
|
262
|
+
- **PostgreSQL**: Full support with EXPLAIN ANALYZE
|
|
263
|
+
- **MySQL**: Full support with EXPLAIN ANALYZE
|
|
264
|
+
- **SQLite**: Full support with EXPLAIN QUERY PLAN
|
|
265
|
+
- **Other Databases**: Basic support with standard EXPLAIN
|
|
266
|
+
|
|
267
|
+
### Cache Store Support
|
|
268
|
+
- **All Cache Stores**: Works with any Rails cache store
|
|
269
|
+
- **Multi-Store Support**: Tracks cache operations across different stores
|
|
270
|
+
|
|
271
|
+
### Redis Support
|
|
272
|
+
- **Redis Gem**: Works with redis gem
|
|
273
|
+
- **Client Instrumentation**: Direct instrumentation of Redis::Client
|
|
274
|
+
- **Pipeline Support**: Tracks Redis pipelines
|
|
275
|
+
- **Transaction Support**: Tracks Redis MULTI/EXEC transactions
|
|
276
|
+
|
|
277
|
+
## Logging & Debugging
|
|
278
|
+
|
|
279
|
+
### Application Logging
|
|
280
|
+
- **Log Capture**: Captures application logs during request processing
|
|
281
|
+
- **Log Context**: Includes logs in metric payloads
|
|
282
|
+
- **Debug Logging**: Optional debug logging for skipped requests
|
|
283
|
+
|
|
284
|
+
## Deployment & Environment
|
|
285
|
+
|
|
286
|
+
### Deploy Tracking
|
|
287
|
+
- **Deploy ID Resolution**: Multiple sources for deploy identification (`Configuration#deploy_id=` wins when set, then ENV in `Configuration::DEPLOY_REVISION_ENV_KEYS` order—including `DEAD_BRO_DEPLOY_ID`, git/CI vars, `DD_VERSION`, etc.), otherwise a **per-process UUID** (fine for single dyno/process; unusable alone for fleets like ECS replicas)
|
|
288
|
+
- **Revision Tracking**: Includes deploy/revision ID in all metric payloads
|
|
289
|
+
|
|
290
|
+
### Environment Support
|
|
291
|
+
- **Rails Environment**: Automatic Rails environment detection
|
|
292
|
+
- **Rack Environment**: Fallback to RACK_ENV or RAILS_ENV
|
|
293
|
+
- **Environment Context**: Includes environment in all metric payloads
|
|
294
|
+
|
|
295
|
+
## Data Collection & Transmission
|
|
296
|
+
|
|
297
|
+
### Metric Payload Structure
|
|
298
|
+
- **Structured Data**: Well-structured JSON payloads
|
|
299
|
+
- **Event Names**: Descriptive event names for different metric types
|
|
300
|
+
- **Timestamp Tracking**: ISO8601 timestamps for all metrics
|
|
301
|
+
- **Metadata**: Rich metadata including environment, host, deploy ID
|
|
302
|
+
|
|
303
|
+
### HTTP Client
|
|
304
|
+
- **HTTPS Support**: Secure HTTPS communication
|
|
305
|
+
- **Bearer Token Auth**: API key authentication via Bearer tokens
|
|
306
|
+
- **JSON Encoding**: JSON-encoded payloads
|
|
307
|
+
- **Custom Headers**: Proper Content-Type and Authorization headers
|
|
308
|
+
|
|
309
|
+
## Comparison-Ready Features
|
|
310
|
+
|
|
311
|
+
### Unique Differentiators
|
|
312
|
+
1. **Automatic EXPLAIN ANALYZE**: Background execution plan capture for slow queries
|
|
313
|
+
2. **Lightweight Memory Tracking**: Low-overhead memory monitoring by default
|
|
314
|
+
3. **Comprehensive Cache Tracking**: Detailed cache operation tracking
|
|
315
|
+
4. **Redis Instrumentation**: Full Redis command tracking including pipelines
|
|
316
|
+
5. **View Rendering Analysis**: Detailed view performance analysis with cache hit rates
|
|
317
|
+
6. **Flexible Exclusion Rules**: Wildcard support for controller/job exclusion
|
|
318
|
+
7. **Request Sampling**: Configurable percentage-based sampling
|
|
319
|
+
8. **Circuit Breaker**: Built-in resilience for APM endpoint failures
|
|
320
|
+
9. **Multi-Source Configuration**: Flexible configuration from multiple sources
|
|
321
|
+
10. **Deploy Tracking**: Automatic deploy ID resolution from multiple sources
|
|
322
|
+
|
|
323
|
+
### Standard APM Features
|
|
324
|
+
- Request/response tracking
|
|
325
|
+
- SQL query tracking
|
|
326
|
+
- Error tracking
|
|
327
|
+
- Background job tracking
|
|
328
|
+
- Memory tracking
|
|
329
|
+
- Performance metrics
|
|
330
|
+
- Exception handling
|
|
331
|
+
- User context
|
|
332
|
+
- Environment tracking
|
|
333
|
+
|
data/README.md
CHANGED
|
@@ -184,31 +184,35 @@ DeadBro automatically tracks SQL queries executed during each request and job. E
|
|
|
184
184
|
- `cached` - Whether the query was cached
|
|
185
185
|
- `connection_id` - Database connection ID
|
|
186
186
|
- `trace` - Call stack showing where the query was executed
|
|
187
|
-
- `explain_plan` - Query execution plan (when EXPLAIN
|
|
187
|
+
- `explain_plan` - Query execution plan (when EXPLAIN capture is enabled, see below)
|
|
188
188
|
|
|
189
|
-
## Automatic EXPLAIN
|
|
189
|
+
## Automatic EXPLAIN for Slow Queries
|
|
190
190
|
|
|
191
|
-
DeadBro can automatically
|
|
191
|
+
DeadBro can automatically capture the query plan (`EXPLAIN`) of slow SELECT queries to help you understand query performance and identify optimization opportunities.
|
|
192
|
+
|
|
193
|
+
### Safety guarantees
|
|
194
|
+
|
|
195
|
+
- **Plan-only, never executed**: DeadBro runs plain `EXPLAIN` (never `EXPLAIN ANALYZE`), which asks the database for its plan without executing the statement. Your queries are never re-run and your data is never touched.
|
|
196
|
+
- **SELECT-only allowlist**: only `SELECT` statements (and read-only `WITH ... SELECT` CTEs) are ever explained. INSERT/UPDATE/DELETE, DDL, and multi-statement strings are always skipped.
|
|
197
|
+
- **Local opt-in required**: the feature is off unless you enable it in your own `DeadBro.configure` block. The DeadBro backend can remotely *disable* it (e.g. plan gating), but can never switch it on.
|
|
198
|
+
- **Value scrubbing**: quoted string literals echoed in plan text (e.g. `Filter: email = 'a@b.com'`) are replaced with `?` before the plan leaves your app.
|
|
199
|
+
- **Bounded overhead**: plans are captured on background threads (max 3 concurrent per request); at request end DeadBro waits at most 0.5s for stragglers, then drops them.
|
|
192
200
|
|
|
193
201
|
### How It Works
|
|
194
202
|
|
|
195
|
-
- **Automatic Detection**:
|
|
196
|
-
- **
|
|
197
|
-
- **Database Support**: Works with PostgreSQL, MySQL, SQLite, and other databases
|
|
198
|
-
- **Smart Filtering**: Automatically skips transaction queries (BEGIN, COMMIT, ROLLBACK) and other queries that don't benefit from EXPLAIN
|
|
203
|
+
- **Automatic Detection**: when a SELECT exceeds `slow_query_threshold_ms`, DeadBro captures its plan in the background
|
|
204
|
+
- **Database Support**: PostgreSQL, MySQL, SQLite (`EXPLAIN QUERY PLAN`), and any adapter with a standard `EXPLAIN`
|
|
199
205
|
|
|
200
206
|
### Configuration
|
|
201
207
|
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
- **`explain_analyze_enabled`** (default: `false`) - Set to `true` to enable automatic EXPLAIN ANALYZE
|
|
205
|
-
- **`slow_query_threshold_ms`** (default: `500`) - Queries taking longer than this threshold will have their execution plan captured
|
|
208
|
+
- **`explain_analyze_enabled`** (default: `false`) - local opt-in for automatic EXPLAIN plan capture. Must be set in Ruby; the DeadBro UI toggle can only turn capture off, not on.
|
|
209
|
+
- **`slow_query_threshold_ms`** (default: `500`) - queries taking longer than this threshold will have their execution plan captured
|
|
206
210
|
|
|
207
211
|
### Example Configuration
|
|
208
212
|
|
|
209
213
|
```ruby
|
|
210
214
|
DeadBro.configure do |config|
|
|
211
|
-
#
|
|
215
|
+
# Capture EXPLAIN plans for SELECTs slower than 500ms
|
|
212
216
|
config.explain_analyze_enabled = true
|
|
213
217
|
config.slow_query_threshold_ms = 500
|
|
214
218
|
|
|
@@ -220,17 +224,17 @@ end
|
|
|
220
224
|
### What You Get
|
|
221
225
|
|
|
222
226
|
When a slow query is detected, the `explain_plan` field in the SQL query data will contain:
|
|
223
|
-
- **PostgreSQL**:
|
|
224
|
-
- **
|
|
225
|
-
- **
|
|
226
|
-
- **Other databases**: Standard EXPLAIN output
|
|
227
|
+
- **PostgreSQL / MySQL**: `EXPLAIN` output — the planner's chosen strategy, estimated costs and row counts
|
|
228
|
+
- **SQLite**: `EXPLAIN QUERY PLAN` output
|
|
229
|
+
- **Other databases**: standard `EXPLAIN` output
|
|
227
230
|
|
|
228
231
|
This execution plan helps you:
|
|
229
232
|
- Identify missing indexes
|
|
230
233
|
- Understand query execution order
|
|
231
234
|
- Spot full table scans
|
|
232
235
|
- Optimize JOIN operations
|
|
233
|
-
|
|
236
|
+
|
|
237
|
+
Because the statement is not executed, plans show the planner's *estimates* rather than actual runtimes — usually exactly what you need to spot a missing index or an unexpected sequential scan.
|
|
234
238
|
|
|
235
239
|
## View Rendering Tracking
|
|
236
240
|
|
|
@@ -9,9 +9,15 @@ module DeadBro
|
|
|
9
9
|
:circuit_breaker_enabled, :circuit_breaker_failure_threshold, :circuit_breaker_recovery_timeout,
|
|
10
10
|
:circuit_breaker_retry_timeout, :disk_paths, :interfaces_ignore
|
|
11
11
|
|
|
12
|
+
# Local-only opt-in for EXPLAIN plan capture on slow queries. The gem runs
|
|
13
|
+
# DB statements (plan-only EXPLAIN) when this is on, so it must be enabled
|
|
14
|
+
# in the app's own config — remote settings can only turn it OFF, never on
|
|
15
|
+
# (see apply_remote_settings). Effective state: #explain_analyze_active?
|
|
16
|
+
attr_accessor :explain_analyze_enabled
|
|
17
|
+
|
|
12
18
|
# Remote-managed settings (overwritten by backend JSON `settings` on successful API responses)
|
|
13
19
|
attr_accessor :memory_tracking_enabled, :allocation_tracking_enabled, :allocation_sample_rate,
|
|
14
|
-
:sample_rate, :slow_query_threshold_ms,
|
|
20
|
+
:sample_rate, :slow_query_threshold_ms,
|
|
15
21
|
:monitor_enabled, :enable_db_stats, :enable_process_stats, :enable_system_stats,
|
|
16
22
|
:max_sql_queries_to_send, :max_logs_to_send
|
|
17
23
|
|
|
@@ -89,6 +95,10 @@ module DeadBro
|
|
|
89
95
|
# ~2-5ms overhead can be capped without turning the feature fully off.
|
|
90
96
|
@allocation_sample_rate = 100
|
|
91
97
|
@explain_analyze_enabled = false
|
|
98
|
+
# Remote kill switch for EXPLAIN capture. Defaults to true so a local
|
|
99
|
+
# opt-in works against older backends that never send the key; any
|
|
100
|
+
# response that sends explain_analyze_enabled: false turns capture off.
|
|
101
|
+
@remote_explain_analyze_enabled = true
|
|
92
102
|
@slow_query_threshold_ms = 500
|
|
93
103
|
@max_sql_queries_to_send = 500
|
|
94
104
|
@max_logs_to_send = 100
|
|
@@ -163,7 +173,12 @@ module DeadBro
|
|
|
163
173
|
case k
|
|
164
174
|
when "sample_rate", "allocation_sample_rate", "slow_query_threshold_ms", "max_sql_queries_to_send", "max_logs_to_send"
|
|
165
175
|
send(:"#{k}=", value.to_i)
|
|
166
|
-
when "
|
|
176
|
+
when "explain_analyze_enabled"
|
|
177
|
+
# EXPLAIN runs statements against the customer DB, so the backend
|
|
178
|
+
# must never be able to switch it on — only off. The local opt-in
|
|
179
|
+
# (explain_analyze_enabled) stays untouched; see #explain_analyze_active?
|
|
180
|
+
@remote_explain_analyze_enabled = !!value
|
|
181
|
+
when "enabled", "memory_tracking_enabled", "allocation_tracking_enabled",
|
|
167
182
|
"monitor_enabled", "enable_db_stats", "enable_process_stats", "enable_system_stats"
|
|
168
183
|
send(:"#{k}=", !!value)
|
|
169
184
|
when "excluded_controllers", "excluded_jobs", "exclusive_controllers", "exclusive_jobs"
|
|
@@ -175,6 +190,12 @@ module DeadBro
|
|
|
175
190
|
end
|
|
176
191
|
end
|
|
177
192
|
|
|
193
|
+
# EXPLAIN capture requires BOTH the local opt-in and the remote flag —
|
|
194
|
+
# locally off means off no matter what the backend sends.
|
|
195
|
+
def explain_analyze_active?
|
|
196
|
+
!!(@explain_analyze_enabled && @remote_explain_analyze_enabled)
|
|
197
|
+
end
|
|
198
|
+
|
|
178
199
|
def heartbeat_due?
|
|
179
200
|
return false if api_key.nil?
|
|
180
201
|
last_heartbeat_attempt_at.nil? || (Time.now.utc - last_heartbeat_attempt_at) >= HEARTBEAT_INTERVAL
|
|
@@ -237,7 +237,8 @@ module DeadBro
|
|
|
237
237
|
# Release job-side thread-local tracking state when we've decided not to
|
|
238
238
|
# build a payload (excluded job / sampled out). Matches Subscriber.drain_request_tracking.
|
|
239
239
|
def self.drain_job_tracking
|
|
240
|
-
|
|
240
|
+
# wait_for_explains: false — result is discarded, don't block on pending plans.
|
|
241
|
+
DeadBro::SqlSubscriber.stop_request_tracking(wait_for_explains: false) if defined?(DeadBro::SqlSubscriber)
|
|
241
242
|
DeadBro::DbConnectionSubscriber.stop_request_tracking if defined?(DeadBro::DbConnectionSubscriber)
|
|
242
243
|
DeadBro::GcTracker.stop_request_tracking if defined?(DeadBro::GcTracker)
|
|
243
244
|
DeadBro::ArObjectTracker.stop_request_tracking if defined?(DeadBro::ArObjectTracker)
|
|
@@ -146,7 +146,7 @@ module DeadBro
|
|
|
146
146
|
if should_explain_query?(duration_ms, original_sql)
|
|
147
147
|
query_info[:explain_plan] = nil
|
|
148
148
|
binds = data[:type_casted_binds] || data[:binds]
|
|
149
|
-
|
|
149
|
+
start_explain_background(original_sql, data[:connection_id], query_info, binds)
|
|
150
150
|
end
|
|
151
151
|
|
|
152
152
|
if aggregates_h && normalized_key
|
|
@@ -195,8 +195,10 @@ module DeadBro
|
|
|
195
195
|
(Thread.current[THREAD_LOCAL_AGGREGATES_KEY] ||= []) << {}
|
|
196
196
|
end
|
|
197
197
|
|
|
198
|
-
|
|
199
|
-
|
|
198
|
+
# wait_for_explains: false on drain paths (excluded / sampled-out requests)
|
|
199
|
+
# where the result is discarded — no reason to block on pending plans.
|
|
200
|
+
def self.stop_request_tracking(wait_for_explains: true)
|
|
201
|
+
wait_for_pending_explains(EXPLAIN_WAIT_TIMEOUT_SECONDS) if wait_for_explains
|
|
200
202
|
|
|
201
203
|
stack = Thread.current[THREAD_LOCAL_KEY]
|
|
202
204
|
raw_queries = (stack.is_a?(Array) && stack.any?) ? stack.pop : []
|
|
@@ -227,12 +229,14 @@ module DeadBro
|
|
|
227
229
|
aggregates_h.values.sort_by { |a| -a[:total_duration_ms] }
|
|
228
230
|
end
|
|
229
231
|
|
|
230
|
-
# Upper bound on pending EXPLAIN threads per request
|
|
231
|
-
#
|
|
232
|
-
|
|
232
|
+
# Upper bound on pending EXPLAIN threads per request. Each thread checks
|
|
233
|
+
# out an AR pool connection, so this must stay well below common pool
|
|
234
|
+
# sizes (Rails default is 5) to avoid starving the app under a storm.
|
|
235
|
+
MAX_PENDING_EXPLAINS = 3
|
|
233
236
|
# Overall wall-clock we're willing to block the request thread for pending
|
|
234
|
-
# EXPLAINs.
|
|
235
|
-
|
|
237
|
+
# EXPLAINs. Plan-only EXPLAIN is planning work (typically <10ms); if a plan
|
|
238
|
+
# isn't ready by then, drop it rather than stall the request.
|
|
239
|
+
EXPLAIN_WAIT_TIMEOUT_SECONDS = 0.5
|
|
236
240
|
|
|
237
241
|
def self.wait_for_pending_explains(timeout_seconds)
|
|
238
242
|
pending = Thread.current[THREAD_LOCAL_EXPLAIN_PENDING_KEY]
|
|
@@ -246,7 +250,7 @@ module DeadBro
|
|
|
246
250
|
begin
|
|
247
251
|
thread.join(remaining_time)
|
|
248
252
|
rescue => e
|
|
249
|
-
DeadBro.logger.debug("Error waiting for EXPLAIN
|
|
253
|
+
DeadBro.logger.debug("Error waiting for EXPLAIN plan: #{e.message}")
|
|
250
254
|
end
|
|
251
255
|
end
|
|
252
256
|
end
|
|
@@ -276,25 +280,31 @@ module DeadBro
|
|
|
276
280
|
sql
|
|
277
281
|
end
|
|
278
282
|
|
|
283
|
+
# Read-only statements we are willing to EXPLAIN. Everything else —
|
|
284
|
+
# INSERT/UPDATE/DELETE, DDL, transaction control, EXPLAIN itself — is
|
|
285
|
+
# rejected by not matching (allowlist, not denylist).
|
|
286
|
+
EXPLAIN_ALLOWED_SQL_RE = /\A(SELECT|WITH)\b/i.freeze
|
|
287
|
+
# A WITH statement can hide data-modifying CTEs (WITH x AS (DELETE ...)).
|
|
288
|
+
# Plan-only EXPLAIN wouldn't run them, but keep them out entirely.
|
|
289
|
+
EXPLAIN_DML_KEYWORD_RE = /\b(INSERT|UPDATE|DELETE|MERGE)\b/i.freeze
|
|
290
|
+
|
|
279
291
|
def self.should_explain_query?(duration_ms, sql)
|
|
280
|
-
return false unless DeadBro.configuration.
|
|
292
|
+
return false unless DeadBro.configuration.explain_analyze_active?
|
|
281
293
|
return false if duration_ms < DeadBro.configuration.slow_query_threshold_ms
|
|
282
294
|
return false unless sql.is_a?(String)
|
|
283
|
-
return false if sql.strip.empty?
|
|
284
295
|
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
return false if
|
|
288
|
-
|
|
289
|
-
return false if
|
|
290
|
-
return false
|
|
291
|
-
return false if
|
|
292
|
-
return false if sql_upper.start_with?("RELEASE")
|
|
296
|
+
stripped = sql.strip
|
|
297
|
+
stripped = stripped.chomp(";").rstrip
|
|
298
|
+
return false if stripped.empty?
|
|
299
|
+
# No multi-statement strings — EXPLAIN must cover exactly one statement.
|
|
300
|
+
return false if stripped.include?(";")
|
|
301
|
+
return false unless stripped.match?(EXPLAIN_ALLOWED_SQL_RE)
|
|
302
|
+
return false if stripped.match?(/\AWITH\b/i) && stripped.match?(EXPLAIN_DML_KEYWORD_RE)
|
|
293
303
|
|
|
294
304
|
true
|
|
295
305
|
end
|
|
296
306
|
|
|
297
|
-
def self.
|
|
307
|
+
def self.start_explain_background(sql, connection_id, query_info, binds = nil)
|
|
298
308
|
return unless defined?(ActiveRecord)
|
|
299
309
|
return unless ActiveRecord::Base.respond_to?(:connection)
|
|
300
310
|
|
|
@@ -327,14 +337,14 @@ module DeadBro
|
|
|
327
337
|
connection.execute(explain_sql)
|
|
328
338
|
end
|
|
329
339
|
|
|
330
|
-
explain_plan = format_explain_result(result, connection)
|
|
340
|
+
explain_plan = scrub_explain_plan(format_explain_result(result, connection))
|
|
331
341
|
query_info[:explain_plan] = if explain_plan && !explain_plan.to_s.strip.empty?
|
|
332
342
|
explain_plan
|
|
333
343
|
end
|
|
334
344
|
end
|
|
335
345
|
rescue => e
|
|
336
346
|
# Silently fail — don't let EXPLAIN break the application.
|
|
337
|
-
append_log_to_thread(main_thread, :debug, "Failed to capture EXPLAIN
|
|
347
|
+
append_log_to_thread(main_thread, :debug, "Failed to capture EXPLAIN plan: #{e.message}")
|
|
338
348
|
query_info[:explain_plan] = nil
|
|
339
349
|
end
|
|
340
350
|
end
|
|
@@ -342,7 +352,22 @@ module DeadBro
|
|
|
342
352
|
pending << explain_thread
|
|
343
353
|
rescue => e
|
|
344
354
|
# Use DeadBro.logger here since we're still in the main thread
|
|
345
|
-
DeadBro.logger.debug("Failed to start EXPLAIN
|
|
355
|
+
DeadBro.logger.debug("Failed to start EXPLAIN thread: #{e.message}")
|
|
356
|
+
end
|
|
357
|
+
|
|
358
|
+
# Quoted string literal in plan text, e.g. Filter: ((email)::text = 'a@b.com'::text).
|
|
359
|
+
# '' is SQL escaping for a single quote inside a literal.
|
|
360
|
+
EXPLAIN_PLAN_QUOTED_LITERAL_RE = /'(?:[^']|'')*'/.freeze
|
|
361
|
+
|
|
362
|
+
# Plan text echoes the interpolated bind values in Filter / Index Cond
|
|
363
|
+
# lines, bypassing sanitize_sql. Blank out quoted literals before the plan
|
|
364
|
+
# is shipped. (Bare numbers are left alone — indistinguishable from the
|
|
365
|
+
# plan's own cost/row estimates.)
|
|
366
|
+
def self.scrub_explain_plan(plan)
|
|
367
|
+
return plan unless plan.is_a?(String)
|
|
368
|
+
plan.gsub(EXPLAIN_PLAN_QUOTED_LITERAL_RE, "?")
|
|
369
|
+
rescue
|
|
370
|
+
plan
|
|
346
371
|
end
|
|
347
372
|
|
|
348
373
|
# Append a log entry directly to a specific thread's log storage
|
|
@@ -389,18 +414,13 @@ module DeadBro
|
|
|
389
414
|
def self.build_explain_query(sql, connection)
|
|
390
415
|
adapter_name = connection.adapter_name.downcase
|
|
391
416
|
|
|
417
|
+
# Plan-only EXPLAIN everywhere. EXPLAIN ANALYZE *executes* the statement
|
|
418
|
+
# on PostgreSQL and MySQL 8 — a monitoring agent must never re-run
|
|
419
|
+
# customer queries, so ANALYZE is deliberately not used for any adapter.
|
|
392
420
|
case adapter_name
|
|
393
|
-
when "postgresql", "postgis"
|
|
394
|
-
# PostgreSQL supports ANALYZE and BUFFERS
|
|
395
|
-
"EXPLAIN (ANALYZE, BUFFERS) #{sql}"
|
|
396
|
-
when "mysql", "mysql2", "trilogy"
|
|
397
|
-
# MySQL uses different syntax - ANALYZE is a separate keyword
|
|
398
|
-
"EXPLAIN ANALYZE #{sql}"
|
|
399
421
|
when "sqlite3"
|
|
400
|
-
# SQLite supports EXPLAIN QUERY PLAN
|
|
401
422
|
"EXPLAIN QUERY PLAN #{sql}"
|
|
402
423
|
else
|
|
403
|
-
# Generic fallback - just EXPLAIN
|
|
404
424
|
"EXPLAIN #{sql}"
|
|
405
425
|
end
|
|
406
426
|
end
|
data/lib/dead_bro/subscriber.rb
CHANGED
|
@@ -235,7 +235,9 @@ module DeadBro
|
|
|
235
235
|
# a payload (disabled / excluded / sampled out). Without this, a subsequent
|
|
236
236
|
# request reusing the same Puma thread would see stale queries/events.
|
|
237
237
|
def self.drain_request_tracking
|
|
238
|
-
|
|
238
|
+
# wait_for_explains: false — the result is discarded, so don't block this
|
|
239
|
+
# thread waiting on pending EXPLAIN plans.
|
|
240
|
+
DeadBro::SqlSubscriber.stop_request_tracking(wait_for_explains: false) if defined?(DeadBro::SqlSubscriber)
|
|
239
241
|
DeadBro::CacheSubscriber.stop_request_tracking if defined?(DeadBro::CacheSubscriber)
|
|
240
242
|
DeadBro::RedisSubscriber.stop_request_tracking if defined?(DeadBro::RedisSubscriber)
|
|
241
243
|
DeadBro::ElasticsearchSubscriber.stop_request_tracking if defined?(DeadBro::ElasticsearchSubscriber)
|
data/lib/dead_bro/version.rb
CHANGED
data/lib/dead_bro.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: dead_bro
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.2.
|
|
4
|
+
version: 0.2.28
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Emanuel Comsa
|
|
@@ -18,6 +18,7 @@ extensions: []
|
|
|
18
18
|
extra_rdoc_files: []
|
|
19
19
|
files:
|
|
20
20
|
- CHANGELOG.md
|
|
21
|
+
- FEATURES.md
|
|
21
22
|
- README.md
|
|
22
23
|
- lib/dead_bro.rb
|
|
23
24
|
- lib/dead_bro/allocation_source_sampler.rb
|
|
@@ -79,7 +80,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
79
80
|
- !ruby/object:Gem::Version
|
|
80
81
|
version: '0'
|
|
81
82
|
requirements: []
|
|
82
|
-
rubygems_version: 4.0.
|
|
83
|
+
rubygems_version: 4.0.9
|
|
83
84
|
specification_version: 4
|
|
84
85
|
summary: Minimal APM for Rails apps.
|
|
85
86
|
test_files: []
|