forge_ops_tracker 0.2.1 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +49 -0
- data/README.md +191 -10
- data/lib/forge_ops_tracker/client.rb +40 -10
- data/lib/forge_ops_tracker/configuration.rb +89 -3
- data/lib/forge_ops_tracker/delivery_queue.rb +1 -1
- data/lib/forge_ops_tracker/error_subscriber.rb +6 -1
- data/lib/forge_ops_tracker/event_builder.rb +64 -8
- data/lib/forge_ops_tracker/infrastructure_metric_buffer.rb +69 -0
- data/lib/forge_ops_tracker/integrations/net_http.rb +32 -0
- data/lib/forge_ops_tracker/integrations/puma.rb +55 -0
- data/lib/forge_ops_tracker/integrations/sidekiq.rb +69 -0
- data/lib/forge_ops_tracker/integrations/solid_queue.rb +37 -0
- data/lib/forge_ops_tracker/metric_buffer.rb +80 -0
- data/lib/forge_ops_tracker/middleware/session_tracking.rb +38 -0
- data/lib/forge_ops_tracker/middleware/user_context.rb +52 -0
- data/lib/forge_ops_tracker/performance_flusher.rb +101 -0
- data/lib/forge_ops_tracker/performance_instrumentation.rb +60 -0
- data/lib/forge_ops_tracker/periodic_poller.rb +50 -0
- data/lib/forge_ops_tracker/pii_scrubber.rb +2 -2
- data/lib/forge_ops_tracker/railtie.rb +123 -0
- data/lib/forge_ops_tracker/session_flusher.rb +85 -0
- data/lib/forge_ops_tracker/version.rb +1 -1
- data/lib/forge_ops_tracker.rb +52 -0
- metadata +58 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 72692319cfdf068129d7d005773e21a3b6b78cb9f7816cf0e7c8e6409c9704f1
|
|
4
|
+
data.tar.gz: 713f244a7ab71a005e66b7f00d1dff3e1e3ffc44e304d5a36f79c911e508c74b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 57f5372fc1aa6ac8f8830804003c00d68d0633d4f330dc8f6c678faa1c1dbc3f09ad1da0f29c180c63a90dc41f2d0f6acb008e4a726d59ba6b806298af3d6d8f
|
|
7
|
+
data.tar.gz: 2b21f12200f99202c5f26cc739104021d5eaf1cfe16a01b075fe4bc3418ada28cf7befb91470fb1ebd8bb18865cd52e61a705decc5f511c2c497a79c74c36016
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,54 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.7.0
|
|
4
|
+
|
|
5
|
+
- Automatically associates a reported error with the current signed-in user, for any Warden-based
|
|
6
|
+
auth setup (Devise included; Devise just mounts Warden automatically, this isn't Devise-
|
|
7
|
+
specific). On by default, same as every other automatic instrumentation this gem does; opt out
|
|
8
|
+
with `config.track_current_user = false`. `ForgeOpsTracker.set_user(id:, email:, username:)`
|
|
9
|
+
manually sets or overrides it, for a custom auth setup or outside a request entirely (a
|
|
10
|
+
background job, a console session).
|
|
11
|
+
|
|
12
|
+
## 0.6.0
|
|
13
|
+
|
|
14
|
+
- Performance monitoring now also covers database queries (`sql.active_record`, bucketed by
|
|
15
|
+
Rails' own query name, never raw SQL text), background jobs (`perform.active_job`, any backend,
|
|
16
|
+
Solid Queue included, plus raw Sidekiq workers via a server middleware, deduplicated against
|
|
17
|
+
jobs dispatched through Active Job), and outbound `Net::HTTP` calls, alongside the existing
|
|
18
|
+
controller/action timing. Each shows up as its own `kind` on the same `performance` dashboard
|
|
19
|
+
dataset. Also reports Sidekiq's, Solid Queue's, and Puma's own operational gauges (queue depth,
|
|
20
|
+
worker/dispatcher counts, Puma's thread pool) as infrastructure metrics on a periodic timer
|
|
21
|
+
(`config.gauge_poll_interval`, default 60 seconds), whenever the corresponding library is
|
|
22
|
+
already loaded. All on the same `track_performance` flag as before; no new flags to opt into.
|
|
23
|
+
|
|
24
|
+
## 0.5.0
|
|
25
|
+
|
|
26
|
+
- Every reported event now carries `sdk_name` ("ruby"), so a project's language on the ForgeOps
|
|
27
|
+
dashboard auto-detects from whichever SDK is actually reporting into it, rather than staying
|
|
28
|
+
stuck on whatever it was created with. Never overrides a language picked by hand in project
|
|
29
|
+
settings.
|
|
30
|
+
|
|
31
|
+
## 0.4.0
|
|
32
|
+
|
|
33
|
+
- Session tracking (release health): every request is now counted as a session, crash-free unless
|
|
34
|
+
an unhandled exception actually escapes it, so the ForgeOps dashboard can show a crash-free rate
|
|
35
|
+
per release rather than only ever hearing about the requests that broke. Counted in-process and
|
|
36
|
+
flushed as a small periodic aggregate (`config.session_flush_interval`, default 60 seconds) on
|
|
37
|
+
its own background thread, not one network call per request. On by default, the same as error
|
|
38
|
+
tracking itself (`config.track_sessions = false` opts out); requires a ForgeOps plan that
|
|
39
|
+
includes release health, checked server-side on every flush.
|
|
40
|
+
|
|
41
|
+
## 0.3.0
|
|
42
|
+
|
|
43
|
+
- Source context capture: each in-app backtrace frame can now carry the 5 lines of source on
|
|
44
|
+
either side of the culprit line, read straight off disk at raise-time, so an issue's detail page
|
|
45
|
+
can show the actual code that broke, not just a file:line:method reference. On by default
|
|
46
|
+
(`config.capture_source_context = false` opts out), but the real, durable protection is a
|
|
47
|
+
per-project setting on the ForgeOps server, not this flag: turning it off there stops the server
|
|
48
|
+
from ever storing captured source lines for that project again, regardless of what any individual
|
|
49
|
+
app's own local setting is still set to. Never applies to a frame outside your app's own code,
|
|
50
|
+
and fails silently for any file that can't be read.
|
|
51
|
+
|
|
3
52
|
## 0.2.0
|
|
4
53
|
|
|
5
54
|
- Client-side PII scrubbing: payloads are scrubbed (email addresses, formatted SSNs/credit
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# ForgeOpsTracker
|
|
2
2
|
|
|
3
|
-
Rails exception reporting client for a
|
|
3
|
+
Rails exception reporting client for a [ForgeOps](../../) instance.
|
|
4
4
|
|
|
5
5
|
## Installation
|
|
6
6
|
|
|
@@ -22,17 +22,17 @@ end
|
|
|
22
22
|
```
|
|
23
23
|
|
|
24
24
|
Delivery happens on a small background thread with a bounded queue and short HTTP timeouts. Every
|
|
25
|
-
failure mode
|
|
25
|
+
failure mode (network errors, timeouts, a full queue, a malformed DSN) is caught and dropped
|
|
26
26
|
rather than raised, so a broken or unreachable tracker can never take down the host app.
|
|
27
27
|
|
|
28
28
|
## What gets reported automatically, and what doesn't
|
|
29
29
|
|
|
30
30
|
**Unhandled exceptions need no further wiring at all.** The gem's Railtie subscribes to
|
|
31
31
|
`Rails.error` automatically, and Rails itself reports anything that crashes a request or job
|
|
32
|
-
through that same channel
|
|
32
|
+
through that same channel: install the gem, set a DSN, and those show up in ForgeOps with zero
|
|
33
33
|
other code changes.
|
|
34
34
|
|
|
35
|
-
**Handled exceptions
|
|
35
|
+
**Handled exceptions (code that catches its own error to keep running) are a different story.**
|
|
36
36
|
A plain `rescue` the gem never hears about, no matter what:
|
|
37
37
|
|
|
38
38
|
```ruby
|
|
@@ -40,19 +40,19 @@ begin
|
|
|
40
40
|
charge_card(order)
|
|
41
41
|
rescue Stripe::CardError => e
|
|
42
42
|
logger.warn("card declined: #{e.message}")
|
|
43
|
-
# ForgeOps never sees this
|
|
43
|
+
# ForgeOps never sees this: nothing here goes through Rails.error at all.
|
|
44
44
|
end
|
|
45
45
|
```
|
|
46
46
|
|
|
47
47
|
To report it *and* keep swallowing it, swap the bare `rescue` for Rails' own built-in
|
|
48
|
-
`Rails.error.handle
|
|
48
|
+
`Rails.error.handle`: this isn't a ForgeOps-specific API, it's Rails' own error-reporting
|
|
49
49
|
convention (Rails 7+), which the gem just happens to already be subscribed to:
|
|
50
50
|
|
|
51
51
|
```ruby
|
|
52
52
|
Rails.error.handle(fallback: -> { nil }) do
|
|
53
53
|
charge_card(order)
|
|
54
54
|
end
|
|
55
|
-
# Reported to every Rails.error subscriber, including this gem, then swallowed
|
|
55
|
+
# Reported to every Rails.error subscriber, including this gem, then swallowed:
|
|
56
56
|
# execution continues past the block either way.
|
|
57
57
|
```
|
|
58
58
|
|
|
@@ -67,13 +67,41 @@ Bottom line: if an exception would otherwise crash something, you're already cov
|
|
|
67
67
|
code already catches and handles it, route that specific `rescue` through `Rails.error.handle`/
|
|
68
68
|
`.record` instead of a bare one wherever you want ForgeOps to know about it.
|
|
69
69
|
|
|
70
|
+
## Identifying users
|
|
71
|
+
|
|
72
|
+
If you're using Devise, or any other Warden-based authentication, the currently signed-in user is
|
|
73
|
+
already associated with every error automatically, on by default, no setup needed:
|
|
74
|
+
|
|
75
|
+
```ruby
|
|
76
|
+
ForgeOpsTracker.configure do |config|
|
|
77
|
+
config.track_current_user = false # opt out entirely
|
|
78
|
+
end
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
For a custom auth setup this can't detect, or to override its guess, call
|
|
82
|
+
`ForgeOpsTracker.set_user` yourself:
|
|
83
|
+
|
|
84
|
+
```ruby
|
|
85
|
+
before_action do
|
|
86
|
+
ForgeOpsTracker.set_user(id: current_user&.id, email: current_user&.email)
|
|
87
|
+
end
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Also usable outside a request entirely, a background job or a console session, for the rest of
|
|
91
|
+
whichever thread calls it. `id`/`email`/`username` are all independently optional; call it with
|
|
92
|
+
none of them (or with a signed-out `current_user`, as the `&.` above already handles) to clear
|
|
93
|
+
whatever was set. Shows up on an issue's own detail page, and as its own `affected_users_count`
|
|
94
|
+
alongside the regular event count.
|
|
95
|
+
|
|
70
96
|
## PII scrubbing
|
|
71
97
|
|
|
72
98
|
By default, the message, backtrace, and any context/tags you attach are scanned for likely
|
|
73
|
-
personal data
|
|
74
|
-
anything under a suspiciously-named key (`password`, `api_key`, `ssn`, and similar)
|
|
99
|
+
personal data: email addresses, formatted SSNs/credit cards, known API key/token formats, and
|
|
100
|
+
anything under a suspiciously-named key (`password`, `api_key`, `ssn`, and similar); redacted
|
|
75
101
|
before the payload ever leaves this process. ForgeOps itself scrubs again on arrival regardless, so
|
|
76
|
-
this is a second, earlier layer, not the only one.
|
|
102
|
+
this is a second, earlier layer, not the only one. The user identified via automatic Warden
|
|
103
|
+
detection or `ForgeOpsTracker.set_user` above is a deliberate exception: it's never scrubbed,
|
|
104
|
+
since redacting it would defeat the whole point of identifying users in the first place.
|
|
77
105
|
|
|
78
106
|
To disable it (e.g. if your app already scrubs its own error context, or you have your own reasons
|
|
79
107
|
to want the raw payload):
|
|
@@ -83,3 +111,156 @@ ForgeOpsTracker.configure do |config|
|
|
|
83
111
|
config.scrub_pii = false
|
|
84
112
|
end
|
|
85
113
|
```
|
|
114
|
+
|
|
115
|
+
## Source context
|
|
116
|
+
|
|
117
|
+
By default, each in-app backtrace frame (never a vendored gem) is captured along with the 5 lines
|
|
118
|
+
of source on either side of the culprit line, read straight off disk at raise-time, so an issue's
|
|
119
|
+
detail page can show the actual code that broke, not just a `file:line:method` reference. This
|
|
120
|
+
never applies to a frame outside `app_root`, and it fails silently (no context, not an error) for
|
|
121
|
+
any file that can't be read for whatever reason.
|
|
122
|
+
|
|
123
|
+
This is a real, deliberate exception to "off by default is safer": literal source code is being
|
|
124
|
+
transmitted, not just a reference to it, and the real protection here is not this flag. Every
|
|
125
|
+
project on ForgeOps has its own setting (on by default, off durably and immediately once an org
|
|
126
|
+
owner turns it off, regardless of what any individual app's own `config.capture_source_context` is
|
|
127
|
+
still set to) that governs whether the server will ever actually store what an SDK sends; see the
|
|
128
|
+
in-app help docs. Use this flag if you'd rather this gem never even attempt the disk read in the
|
|
129
|
+
first place:
|
|
130
|
+
|
|
131
|
+
```ruby
|
|
132
|
+
ForgeOpsTracker.configure do |config|
|
|
133
|
+
config.capture_source_context = false
|
|
134
|
+
end
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
## Session tracking (release health)
|
|
138
|
+
|
|
139
|
+
By default, every request is counted as a session: crash-free unless an unhandled exception
|
|
140
|
+
actually escapes it, giving ForgeOps a crash-free rate per release to show alongside the errors
|
|
141
|
+
themselves, not just the errors on their own. Counted in-process and flushed as a small periodic
|
|
142
|
+
aggregate on a background thread (never one network call per request), the same delivery
|
|
143
|
+
philosophy as everything else in this gem: a broken or unreachable tracker never affects the host
|
|
144
|
+
app either way.
|
|
145
|
+
|
|
146
|
+
```ruby
|
|
147
|
+
ForgeOpsTracker.configure do |config|
|
|
148
|
+
config.track_sessions = false # opt out entirely
|
|
149
|
+
config.session_flush_interval = 30 # seconds; default 60
|
|
150
|
+
end
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Requires a ForgeOps plan that includes release health; on a plan that doesn't, the periodic
|
|
154
|
+
flushes are simply rejected server-side and dropped, exactly like any other delivery failure.
|
|
155
|
+
|
|
156
|
+
## Performance monitoring
|
|
157
|
+
|
|
158
|
+
By default, every request's controller/action duration is timed (via Rails' own
|
|
159
|
+
`process_action.action_controller` instrumentation, no extra middleware needed) so a dashboard
|
|
160
|
+
widget on ForgeOps can show which parts of your app are actually slow, not just which ones raise.
|
|
161
|
+
Bucketed by transaction ("PostsController#show") and flushed as a small periodic aggregate per
|
|
162
|
+
transaction on a background thread, the same delivery philosophy as session tracking above: a
|
|
163
|
+
broken or unreachable tracker never affects the host app either way.
|
|
164
|
+
|
|
165
|
+
```ruby
|
|
166
|
+
ForgeOpsTracker.configure do |config|
|
|
167
|
+
config.track_performance = false # opt out entirely
|
|
168
|
+
config.performance_flush_interval = 30 # seconds; default 60
|
|
169
|
+
end
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Requires a ForgeOps plan that includes performance monitoring; on a plan that doesn't, the
|
|
173
|
+
periodic flushes are simply rejected server-side and dropped, exactly like any other delivery
|
|
174
|
+
failure.
|
|
175
|
+
|
|
176
|
+
### Database queries, background jobs, and outbound HTTP calls
|
|
177
|
+
|
|
178
|
+
The same automatic instrumentation, on the same `track_performance` flag, also covers:
|
|
179
|
+
|
|
180
|
+
- **Database queries**, via `sql.active_record`: bucketed by Rails' own auto-generated query name
|
|
181
|
+
("User Load", "Order Create"), not the raw SQL text. Internal schema-introspection queries
|
|
182
|
+
("SCHEMA") and cached reads (never a real round trip to the database) are skipped.
|
|
183
|
+
- **Background jobs**, via `perform.active_job`: bucketed by job class. Covers any Active Job
|
|
184
|
+
backend, Solid Queue included, with no backend-specific code needed.
|
|
185
|
+
- **Raw Sidekiq workers** (a `Sidekiq::Worker`/`Sidekiq::Job` not dispatched through Active Job):
|
|
186
|
+
a server middleware, registered automatically when Sidekiq is already loaded. A job dispatched
|
|
187
|
+
through Active Job and run on Sidekiq is still only counted once, by the Active Job hook above,
|
|
188
|
+
not twice.
|
|
189
|
+
- **Outbound `Net::HTTP` calls**, bucketed by `"<method> <host>"`, never the full URL (a path or
|
|
190
|
+
query string could carry an id or a token). Most other Ruby HTTP client libraries (Faraday's own
|
|
191
|
+
`net_http` adapter, HTTParty, RestClient) ultimately call through `Net::HTTP`, so this covers
|
|
192
|
+
those too as a side effect.
|
|
193
|
+
|
|
194
|
+
Every one of these shows up as its own `kind` ("controller", "job", "query", "http") on the same
|
|
195
|
+
`performance` dashboard dataset, so "slowest jobs" and "slowest queries" are just a filtered
|
|
196
|
+
version of the same widget builder "slowest transactions" already uses.
|
|
197
|
+
|
|
198
|
+
### Sidekiq, Solid Queue, and Puma gauges
|
|
199
|
+
|
|
200
|
+
Also on by default whenever the corresponding library is already loaded: Sidekiq's own aggregate
|
|
201
|
+
stats (`Sidekiq::Stats`: processed/failed/scheduled/retry/dead counts, plus a queue-depth reading
|
|
202
|
+
per queue), Solid Queue's own state (active workers, active dispatchers, failed/scheduled/blocked
|
|
203
|
+
counts, a queue-depth reading per queue), and Puma's own thread pool (backlog, running threads,
|
|
204
|
+
pool capacity). Reported as ordinary infrastructure metrics (`sidekiq.queue_depth.default`,
|
|
205
|
+
`puma.backlog`, and so on) via the same `capture_infrastructure_metric` call your own scripts use,
|
|
206
|
+
on a periodic timer:
|
|
207
|
+
|
|
208
|
+
```ruby
|
|
209
|
+
ForgeOpsTracker.configure do |config|
|
|
210
|
+
config.gauge_poll_interval = 30 # seconds; default 60
|
|
211
|
+
end
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
No separate opt-out: these are gated by the same `track_performance` flag as everything else in
|
|
215
|
+
this section, not a flag of their own.
|
|
216
|
+
|
|
217
|
+
## Custom metrics
|
|
218
|
+
|
|
219
|
+
Unlike session/performance tracking above, there's no automatic instrumentation here at all: a
|
|
220
|
+
signup or a payment isn't something this gem could ever detect on its own, so this is an explicit
|
|
221
|
+
call your own code makes. `value` defaults to `1.0` so a bare counter-style call needs no
|
|
222
|
+
argument; pass one for a metric with a real amount. Buffered and flushed as a batch on a
|
|
223
|
+
background thread, same delivery philosophy as everything else here, so this is safe to call from
|
|
224
|
+
inside a request (right after a signup completes, say) without adding network latency there.
|
|
225
|
+
|
|
226
|
+
```ruby
|
|
227
|
+
ForgeOpsTracker.capture_metric("signups")
|
|
228
|
+
ForgeOpsTracker.capture_metric("revenue", value: 49.00)
|
|
229
|
+
|
|
230
|
+
ForgeOpsTracker.configure do |config|
|
|
231
|
+
config.metric_flush_interval = 30 # seconds; default 60
|
|
232
|
+
end
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Requires a ForgeOps plan that includes custom metrics; on a plan that doesn't, the periodic
|
|
236
|
+
flushes are simply rejected server-side and dropped, exactly like any other delivery failure.
|
|
237
|
+
|
|
238
|
+
## Infrastructure monitoring
|
|
239
|
+
|
|
240
|
+
A deliberately different shape from everything above: there's no automatic instrumentation and no
|
|
241
|
+
ForgeOps-built agent. Run your own short-lived script on a cron entry or a systemd timer, reading
|
|
242
|
+
your own host's own stats; `hostname` defaults to the box the script is actually running on.
|
|
243
|
+
Buffered and flushed on exit, so a handful of capture calls in one short-lived process still cost
|
|
244
|
+
one network request, not several.
|
|
245
|
+
|
|
246
|
+
```ruby
|
|
247
|
+
# A cron entry or systemd timer runs this periodically, not your web app itself.
|
|
248
|
+
require "forge_ops_tracker"
|
|
249
|
+
ForgeOpsTracker.configure { |config| config.dsn = ENV["FORGE_OPS_DSN"] }
|
|
250
|
+
|
|
251
|
+
load_average = File.read("/proc/loadavg").split.first.to_f
|
|
252
|
+
ForgeOpsTracker.capture_infrastructure_metric("load_average", value: load_average)
|
|
253
|
+
|
|
254
|
+
meminfo = File.read("/proc/meminfo").lines.to_h { |line| line.split(":").map(&:strip) }
|
|
255
|
+
total_kb, available_kb = meminfo["MemTotal"].to_i, meminfo["MemAvailable"].to_i
|
|
256
|
+
ForgeOpsTracker.capture_infrastructure_metric("memory_used_percent",
|
|
257
|
+
value: 100.0 * (total_kb - available_kb) / total_kb)
|
|
258
|
+
|
|
259
|
+
disk_used_percent = `df --output=pcent / | tail -1`.strip.delete("%").to_f
|
|
260
|
+
ForgeOpsTracker.capture_infrastructure_metric("disk_used_percent", value: disk_used_percent)
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
The example above is Linux-specific (`/proc/loadavg`, `/proc/meminfo`); on another OS, read that
|
|
264
|
+
platform's own equivalents instead. Requires a ForgeOps plan that includes infrastructure
|
|
265
|
+
monitoring; on a plan that doesn't, the periodic flushes are simply rejected server-side and
|
|
266
|
+
dropped, exactly like any other delivery failure.
|
|
@@ -3,8 +3,8 @@ require "json"
|
|
|
3
3
|
require "uri"
|
|
4
4
|
|
|
5
5
|
module ForgeOpsTracker
|
|
6
|
-
# Delivers one payload over HTTP. Every failure mode
|
|
7
|
-
# timeout, TLS, a non-2xx response
|
|
6
|
+
# Delivers one payload over HTTP. Every failure mode; DNS, connection,
|
|
7
|
+
# timeout, TLS, a non-2xx response; is caught here and turned into a
|
|
8
8
|
# `false` return rather than a raised exception, since a broken or
|
|
9
9
|
# unreachable tracker must never be able to break the host app.
|
|
10
10
|
class Client
|
|
@@ -13,19 +13,49 @@ module ForgeOpsTracker
|
|
|
13
13
|
end
|
|
14
14
|
|
|
15
15
|
def deliver(payload)
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
16
|
+
post(configuration.ingestion_uri, payload)
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# Same delivery contract as deliver above (never raises, returns a bare
|
|
20
|
+
# true/false), just a different endpoint under the same DSN's host and
|
|
21
|
+
# api_key; see Configuration#session_checkins_uri.
|
|
22
|
+
def deliver_session_checkin(payload)
|
|
23
|
+
post(configuration.session_checkins_uri, payload)
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
# Same delivery contract again; see Configuration#performance_samples_uri. payload here is a
|
|
27
|
+
# batch (one entry per distinct transaction a flush interval saw), not a single aggregate the
|
|
28
|
+
# way deliver_session_checkin's own payload is, so this posts { samples: [...] } rather than
|
|
29
|
+
# the array bare, matching what Api::V1::PerformanceSamplesController expects.
|
|
30
|
+
def deliver_performance_samples(samples)
|
|
31
|
+
post(configuration.performance_samples_uri, { samples: samples })
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# Same delivery contract again; see Configuration#custom_metrics_uri. payload is a batch of
|
|
35
|
+
# individual capture_metric calls (see MetricBuffer), matching what
|
|
36
|
+
# Api::V1::CustomMetricsController expects.
|
|
37
|
+
def deliver_metrics(entries)
|
|
38
|
+
post(configuration.custom_metrics_uri, { metrics: entries })
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# Same again; see Configuration#infrastructure_metrics_uri and InfrastructureMetricBuffer.
|
|
42
|
+
def deliver_infrastructure_metrics(entries)
|
|
43
|
+
post(configuration.infrastructure_metrics_uri, { metrics: entries })
|
|
24
44
|
end
|
|
25
45
|
|
|
26
46
|
private
|
|
27
47
|
attr_reader :configuration
|
|
28
48
|
|
|
49
|
+
def post(uri, payload)
|
|
50
|
+
return false unless uri
|
|
51
|
+
|
|
52
|
+
response = http_for(uri).request(build_request(uri, payload))
|
|
53
|
+
response.is_a?(Net::HTTPSuccess)
|
|
54
|
+
rescue StandardError => e
|
|
55
|
+
log { "delivery failed: #{e.class}: #{e.message}" }
|
|
56
|
+
false
|
|
57
|
+
end
|
|
58
|
+
|
|
29
59
|
def http_for(uri)
|
|
30
60
|
http = Net::HTTP.new(uri.host, uri.port)
|
|
31
61
|
http.use_ssl = uri.scheme == "https"
|
|
@@ -3,10 +3,16 @@ require "socket"
|
|
|
3
3
|
|
|
4
4
|
module ForgeOpsTracker
|
|
5
5
|
class Configuration
|
|
6
|
-
# A single
|
|
6
|
+
# A single DSN string carries both the ingestion URL and
|
|
7
7
|
# the project's api_key: "https://<api_key>@host/api/v1/events".
|
|
8
8
|
attr_accessor :dsn, :environment, :release, :server_name, :app_root, :logger
|
|
9
9
|
attr_accessor :enabled_environments, :queue_size, :open_timeout, :read_timeout, :scrub_pii
|
|
10
|
+
attr_accessor :capture_source_context
|
|
11
|
+
attr_accessor :track_sessions, :session_flush_interval
|
|
12
|
+
attr_accessor :track_performance, :performance_flush_interval
|
|
13
|
+
attr_accessor :track_current_user
|
|
14
|
+
attr_accessor :metric_flush_interval, :infrastructure_metric_flush_interval
|
|
15
|
+
attr_accessor :gauge_poll_interval
|
|
10
16
|
|
|
11
17
|
def initialize
|
|
12
18
|
@dsn = ENV["FORGE_OPS_DSN"]
|
|
@@ -18,13 +24,53 @@ module ForgeOpsTracker
|
|
|
18
24
|
@open_timeout = 2
|
|
19
25
|
@read_timeout = 2
|
|
20
26
|
@logger = nil
|
|
21
|
-
# See PiiScrubber
|
|
27
|
+
# See PiiScrubber; redacts likely-sensitive content (emails,
|
|
22
28
|
# credit cards, known API key formats, anything under a
|
|
23
29
|
# suspiciously-named key) before a payload ever leaves this
|
|
24
30
|
# process. ForgeOps itself scrubs again on arrival regardless, so
|
|
25
|
-
# this is a second, earlier layer, not the only one
|
|
31
|
+
# this is a second, earlier layer, not the only one; but "on" is
|
|
26
32
|
# the only sane default.
|
|
27
33
|
@scrub_pii = true
|
|
34
|
+
# Whether EventBuilder reads a few lines of source off disk around
|
|
35
|
+
# each in-app frame's culprit line (see EventBuilder#attach_source_context).
|
|
36
|
+
# Defaults on so a snippet shows up without extra setup, but this flag
|
|
37
|
+
# alone isn't the real protection against sending proprietary source
|
|
38
|
+
# code somewhere it shouldn't go: ForgeOps' own per-project setting is
|
|
39
|
+
# the durable, server-enforced off switch, since it applies regardless
|
|
40
|
+
# of what this flag happens to be set to on any given deployment, and
|
|
41
|
+
# can't drift back on the way a local config value could. Turn this
|
|
42
|
+
# off here too if this host app never wants that disk read attempted
|
|
43
|
+
# in the first place.
|
|
44
|
+
@capture_source_context = true
|
|
45
|
+
# Auto-instruments every request the moment the gem loads, the same "on unless you turn it
|
|
46
|
+
# off" default error tracking itself already has; nothing else in this gem is opt-in. See
|
|
47
|
+
# ForgeOpsTracker::Middleware::SessionTracking for what this actually wraps.
|
|
48
|
+
@track_sessions = true
|
|
49
|
+
@session_flush_interval = 60
|
|
50
|
+
# Auto-instruments every request the same "on unless you turn it off" default as
|
|
51
|
+
# track_sessions above; see Railtie's own comment for exactly what this subscribes to.
|
|
52
|
+
@track_performance = true
|
|
53
|
+
@performance_flush_interval = 60
|
|
54
|
+
# Auto-detects the current user via Warden (env["warden"].user; Devise mounts Warden
|
|
55
|
+
# automatically, so this covers Devise apps too, but works for any Warden-based auth, not
|
|
56
|
+
# just Devise specifically) the same "on unless you turn it off" default every other
|
|
57
|
+
# automatic instrumentation flag above already has. See
|
|
58
|
+
# ForgeOpsTracker::Middleware::UserContext for how, and ForgeOpsTracker.set_user for the
|
|
59
|
+
# manual override/fallback when there's no Warden at all or its guess isn't right.
|
|
60
|
+
@track_current_user = true
|
|
61
|
+
# No track_metrics/track_infrastructure boolean the way track_sessions/track_performance
|
|
62
|
+
# each have one: those gate automatic instrumentation that's on unless you turn it off;
|
|
63
|
+
# capture_metric/capture_infrastructure_metric are explicit calls the customer's own code
|
|
64
|
+
# chooses to make at all, so there's no "automatic behavior" for a flag to disable. Only
|
|
65
|
+
# the flush interval needs a knob.
|
|
66
|
+
@metric_flush_interval = 60
|
|
67
|
+
@infrastructure_metric_flush_interval = 60
|
|
68
|
+
# How often the Sidekiq/Solid Queue/Puma stats reporters poll and report their own gauge
|
|
69
|
+
# readings (queue depth, worker counts, Puma backlog/threads/pool capacity). Gated by
|
|
70
|
+
# track_performance, same as controller/job/query/http timing: one flag for "automatically
|
|
71
|
+
# instrument this app's own operational data," reused rather than adding a separate boolean
|
|
72
|
+
# per new automatic instrumentation source this gem grows.
|
|
73
|
+
@gauge_poll_interval = 60
|
|
28
74
|
end
|
|
29
75
|
|
|
30
76
|
def api_key
|
|
@@ -42,6 +88,46 @@ module ForgeOpsTracker
|
|
|
42
88
|
uri
|
|
43
89
|
end
|
|
44
90
|
|
|
91
|
+
# Same host and api_key as ingestion_uri, a sibling path under the same DSN rather than a
|
|
92
|
+
# second DSN to configure; the DSN's own path is always /api/v1/events (what ForgeOps issues
|
|
93
|
+
# to every project), so the session-checkins endpoint is derived by substitution.
|
|
94
|
+
def session_checkins_uri
|
|
95
|
+
uri = ingestion_uri
|
|
96
|
+
return nil unless uri
|
|
97
|
+
|
|
98
|
+
uri = uri.dup
|
|
99
|
+
uri.path = uri.path.sub(%r{/events\z}, "/session_checkins")
|
|
100
|
+
uri
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
# Same substitution as session_checkins_uri above, its own sibling path under the same DSN.
|
|
104
|
+
def performance_samples_uri
|
|
105
|
+
uri = ingestion_uri
|
|
106
|
+
return nil unless uri
|
|
107
|
+
|
|
108
|
+
uri = uri.dup
|
|
109
|
+
uri.path = uri.path.sub(%r{/events\z}, "/performance_samples")
|
|
110
|
+
uri
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
def custom_metrics_uri
|
|
114
|
+
uri = ingestion_uri
|
|
115
|
+
return nil unless uri
|
|
116
|
+
|
|
117
|
+
uri = uri.dup
|
|
118
|
+
uri.path = uri.path.sub(%r{/events\z}, "/custom_metrics")
|
|
119
|
+
uri
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
def infrastructure_metrics_uri
|
|
123
|
+
uri = ingestion_uri
|
|
124
|
+
return nil unless uri
|
|
125
|
+
|
|
126
|
+
uri = uri.dup
|
|
127
|
+
uri.path = uri.path.sub(%r{/events\z}, "/infrastructure_metrics")
|
|
128
|
+
uri
|
|
129
|
+
end
|
|
130
|
+
|
|
45
131
|
def enabled?
|
|
46
132
|
!blank?(dsn) && !blank?(api_key) && enabled_environments.map(&:to_s).include?(environment.to_s)
|
|
47
133
|
end
|
|
@@ -17,7 +17,7 @@ module ForgeOpsTracker
|
|
|
17
17
|
end
|
|
18
18
|
|
|
19
19
|
# Enqueues a payload, dropping it silently (never blocking the caller)
|
|
20
|
-
# if the queue is already full
|
|
20
|
+
# if the queue is already full; a burst of exceptions must never apply
|
|
21
21
|
# backpressure to the host app.
|
|
22
22
|
def push(payload)
|
|
23
23
|
ensure_worker_started
|
|
@@ -14,7 +14,12 @@ module ForgeOpsTracker
|
|
|
14
14
|
def report(error, handled: true, severity: nil, context: {}, source: nil)
|
|
15
15
|
return unless configuration.enabled?
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
# Read here, not inside EventBuilder: Rails.error.subscribe's own #report interface never
|
|
18
|
+
# hands this class the Rack env, so ForgeOpsTracker::Middleware::UserContext (Warden
|
|
19
|
+
# auto-detection) and ForgeOpsTracker.set_user (the manual override/fallback) both
|
|
20
|
+
# communicate with this exact same thread-local instead, the only channel available here.
|
|
21
|
+
user = Thread.current[:forge_ops_tracker_current_user]
|
|
22
|
+
delivery_queue.push(event_builder.build(error, context: context, user: user))
|
|
18
23
|
nil
|
|
19
24
|
rescue StandardError => e
|
|
20
25
|
configuration.logger&.debug { "[ForgeOpsTracker] report failed: #{e.class}: #{e.message}" }
|
|
@@ -2,18 +2,32 @@ require "time"
|
|
|
2
2
|
|
|
3
3
|
module ForgeOpsTracker
|
|
4
4
|
# Turns a raised exception into the JSON-able payload shape the ingestion
|
|
5
|
-
# API expects. Backtrace parsing is a simple regex, not a full parser
|
|
5
|
+
# API expects. Backtrace parsing is a simple regex, not a full parser:
|
|
6
6
|
# good enough for standard MRI backtrace lines across Ruby versions
|
|
7
7
|
# (both the older `in \`method'` and newer `in 'method'` quoting styles).
|
|
8
8
|
class EventBuilder
|
|
9
9
|
LINE_PATTERN = /\A(?<file>.+?):(?<line>\d+)(?::in\s+(?<method>.+))?\z/
|
|
10
10
|
MAX_FRAMES = 500
|
|
11
11
|
|
|
12
|
+
# Identifies this gem to the server's auto language-detection on the project the event lands
|
|
13
|
+
# in (see Project#note_sdk_platform server-side); "ruby", matching this repo's own sdks/
|
|
14
|
+
# directory naming for every other language's client.
|
|
15
|
+
SDK_NAME = "ruby".freeze
|
|
16
|
+
|
|
17
|
+
# How many lines of source to grab on either side of the culprit line
|
|
18
|
+
# (see #attach_source_context), and the longest a single captured line
|
|
19
|
+
# is allowed to be before getting truncated; guards against a single
|
|
20
|
+
# pathological minified/generated line ballooning the payload. ForgeOps
|
|
21
|
+
# itself re-truncates on arrival too, the same "don't just trust the
|
|
22
|
+
# SDK" posture MAX_FRAMES already gets on the server side.
|
|
23
|
+
CONTEXT_LINES = 5
|
|
24
|
+
MAX_CONTEXT_LINE_LENGTH = 500
|
|
25
|
+
|
|
12
26
|
def initialize(configuration)
|
|
13
27
|
@configuration = configuration
|
|
14
28
|
end
|
|
15
29
|
|
|
16
|
-
def build(error, context: {})
|
|
30
|
+
def build(error, context: {}, user: nil)
|
|
17
31
|
payload = {
|
|
18
32
|
exception_class: error.class.name,
|
|
19
33
|
message: error.message.to_s,
|
|
@@ -23,18 +37,21 @@ module ForgeOpsTracker
|
|
|
23
37
|
release: configuration.release,
|
|
24
38
|
server_name: configuration.server_name,
|
|
25
39
|
context: context || {},
|
|
26
|
-
tags: {}
|
|
40
|
+
tags: {},
|
|
41
|
+
sdk_name: SDK_NAME
|
|
27
42
|
}
|
|
43
|
+
payload[:user] = user if user && !user.empty?
|
|
28
44
|
scrub(payload)
|
|
29
45
|
end
|
|
30
46
|
|
|
31
47
|
private
|
|
32
48
|
attr_reader :configuration
|
|
33
49
|
|
|
34
|
-
# exception_class/occurred_at/environment/release/server_name are
|
|
35
|
-
#
|
|
36
|
-
#
|
|
37
|
-
#
|
|
50
|
+
# exception_class/occurred_at/environment/release/server_name/sdk_name/user are left alone;
|
|
51
|
+
# structured fields this gem or the host app sets deliberately, not free text an exception
|
|
52
|
+
# or its context could accidentally spill sensitive data into. user specifically is a
|
|
53
|
+
# deliberate exemption, not an oversight: the server's own PiiScrubber-equivalent would
|
|
54
|
+
# otherwise redact the exact email address this field exists to carry.
|
|
38
55
|
def scrub(payload)
|
|
39
56
|
return payload unless configuration.scrub_pii
|
|
40
57
|
|
|
@@ -55,12 +72,13 @@ module ForgeOpsTracker
|
|
|
55
72
|
return nil unless match
|
|
56
73
|
|
|
57
74
|
file = match[:file]
|
|
58
|
-
{
|
|
75
|
+
frame = {
|
|
59
76
|
file: file,
|
|
60
77
|
line: match[:line].to_i,
|
|
61
78
|
method: match[:method]&.gsub(/\A[`'"]|['"]\z/, ""),
|
|
62
79
|
in_app: in_app?(file)
|
|
63
80
|
}
|
|
81
|
+
attach_source_context(frame)
|
|
64
82
|
end
|
|
65
83
|
|
|
66
84
|
def in_app?(file)
|
|
@@ -69,5 +87,43 @@ module ForgeOpsTracker
|
|
|
69
87
|
|
|
70
88
|
file.start_with?(root.to_s) && !file.include?("/gems/") && !file.include?("/bundle/")
|
|
71
89
|
end
|
|
90
|
+
|
|
91
|
+
# Reads a few lines of source straight off disk around the culprit
|
|
92
|
+
# line, at raise-time, in the same running process the exception came
|
|
93
|
+
# from. Gated on two things: the frame has to be in-app (never a
|
|
94
|
+
# vendored dependency; there'd be nothing meaningful to show, and
|
|
95
|
+
# it's not the host app's own code to begin with), and
|
|
96
|
+
# configuration.capture_source_context has to be on (see
|
|
97
|
+
# Configuration for why it defaults to true and why ForgeOps' own
|
|
98
|
+
# per-project setting, not this flag, is the durable, protected way
|
|
99
|
+
# to turn it off). Best-effort: any file that can't be read (deleted,
|
|
100
|
+
# permission denied, a path that only ever existed inside a build
|
|
101
|
+
# step and isn't present in this deployment) just means this one
|
|
102
|
+
# frame gets no source context, not a raised error of its own.
|
|
103
|
+
def attach_source_context(frame)
|
|
104
|
+
return frame unless configuration.capture_source_context && frame[:in_app]
|
|
105
|
+
|
|
106
|
+
lines = File.readlines(frame[:file])
|
|
107
|
+
index = frame[:line] - 1
|
|
108
|
+
return frame unless index.between?(0, lines.length - 1)
|
|
109
|
+
|
|
110
|
+
from = [ index - CONTEXT_LINES, 0 ].max
|
|
111
|
+
to = [ index + CONTEXT_LINES, lines.length - 1 ].min
|
|
112
|
+
|
|
113
|
+
frame.merge(
|
|
114
|
+
context_line: truncate_line(lines[index]),
|
|
115
|
+
pre_context: lines[from...index].map { |source_line| truncate_line(source_line) },
|
|
116
|
+
post_context: lines[(index + 1)..to].map { |source_line| truncate_line(source_line) }
|
|
117
|
+
)
|
|
118
|
+
rescue SystemCallError, IOError
|
|
119
|
+
frame
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
def truncate_line(line)
|
|
123
|
+
line = line.chomp
|
|
124
|
+
return line if line.length <= MAX_CONTEXT_LINE_LENGTH
|
|
125
|
+
|
|
126
|
+
"#{line[0, MAX_CONTEXT_LINE_LENGTH]}..."
|
|
127
|
+
end
|
|
72
128
|
end
|
|
73
129
|
end
|