railwatch 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,10 +1,12 @@
1
1
  # Configuration
2
2
 
3
- Everything below lives on `Railwatch::Configuration` (`lib/railwatch/configuration.rb`),
4
- set via `Railwatch.configure { |c| ... }` in `config/initializers/railwatch.rb`
5
- (created by `bin/rails generate railwatch:install`). Every setting has a
6
- `RAILWATCH_*` env var default; explicit values set in the initializer always
7
- win over the env var.
3
+ Everything below lives on `Railwatch::Configuration`, in
4
+ `lib/railwatch/configuration.rb`. Set it via
5
+ `Railwatch.configure { |c| ... }` in `config/initializers/railwatch.rb`.
6
+ That file is created by
7
+ `bin/rails generate railwatch:install`. Most settings have a `RAILWATCH_*`
8
+ env var default; the tables below show which. Explicit values set in the
9
+ initializer always win over the env var.
8
10
 
9
11
  ## Core
10
12
 
@@ -22,32 +24,33 @@ win over the env var.
22
24
  | `beacon_rate_limit` | `RAILWATCH_BEACON_RATE_LIMIT` | `120` | Beacon POSTs accepted per client IP per minute before `POST /railwatch/beacon` answers 429. The beacon is unauthenticated and keeps every browser error it is sent, so this is what stops a script from spending the app's event quota. Counted in the app's cache store; `0` turns it off. |
23
25
 
24
26
  `Railwatch.enabled?` delegates to `config.enabled?`, which is `@enabled &&
25
- token.present?` there is no separate "is configured" check elsewhere.
27
+ token.present?`. There is no separate "is configured" check elsewhere.
26
28
 
27
29
  Deploy detection stops at the first value found: `RAILWATCH_DEPLOY`,
28
30
  `KAMAL_VERSION`, `GIT_REV`, `GIT_SHA`, `SOURCE_VERSION`,
29
31
  `HEROKU_SLUG_COMMIT`, `RENDER_GIT_COMMIT`, the tag from `FLY_IMAGE_REF`,
30
32
  `VERCEL_GIT_COMMIT_SHA`, `CI_COMMIT_SHA`, `GITHUB_SHA`, a Capistrano
31
- `REVISION` file, then `.git/HEAD` (including loose and packed refs). Git is
33
+ `REVISION` file, then `.git/HEAD`, including loose and packed refs. Git is
32
34
  never run as a subprocess. An initializer assignment to `config.deploy`
33
35
  always wins.
34
36
 
35
37
  The request middleware also recognizes a reporter's own `POST /ingest` when
36
38
  the configured ingest endpoint runs in the instrumented application. It
37
- bypasses that request only when the
38
- method, bearer token, configured ingest path, and public scheme/host/port all
39
- match; an unrelated application route named `/ingest` remains observable.
40
- Rack's normalized forwarded origin is used so this works behind a trusted
41
- TLS-terminating proxy with `Forwarded` or `X-Forwarded-*` headers.
39
+ bypasses that request only when the method, bearer token, configured
40
+ ingest path, and public scheme/host/port all match. An unrelated
41
+ application route named `/ingest` remains observable. Rack's normalized
42
+ forwarded origin is used, so this works behind a trusted TLS-terminating
43
+ proxy with `Forwarded` or `X-Forwarded-*` headers.
42
44
 
43
45
  ## Sampling
44
46
 
45
- `sample` is a hash of rate per execution kind, each `0.0`–`1.0`, decided
46
- once per execution (`Railwatch::Sampler.decide`, `lib/railwatch/sampler.rb`)
47
- not per record. A sampled-in execution ships every child record it
48
- buffered; a sampled-out one ships nothing except an unhandled exception
49
- (governed by its own `exceptions` rate, decided once and memoized per
50
- execution — see `docs/records.md`'s `exception` section).
47
+ `sample` is a hash of rate per execution kind, each `0.0`–`1.0`. The rate
48
+ is decided once per execution, not per record, by
49
+ `Railwatch::Sampler.decide` in `lib/railwatch/sampler.rb`. A sampled-in
50
+ execution ships every child record it buffered. A sampled-out one ships
51
+ nothing except an unhandled exception. That exception is governed by its
52
+ own `exceptions` rate, decided once and memoized per execution. See
53
+ `docs/records.md`'s `exception` section.
51
54
 
52
55
  | Key | Env var | Default |
53
56
  |---|---|---|
@@ -58,12 +61,13 @@ execution — see `docs/records.md`'s `exception` section).
58
61
  | `channels` | `RAILWATCH_CHANNEL_SAMPLE_RATE` | `1.0` |
59
62
  | `exceptions` | `RAILWATCH_EXCEPTION_SAMPLE_RATE` | `1.0` |
60
63
 
61
- Set as a whole hash: `c.sample = { requests: 0.1, jobs: 1.0 }` keys you
62
- omit keep their default (`config.sample_rate` falls back to `1.0` for an
63
- unset kind).
64
+ Set as a whole hash: `c.sample = { requests: 0.1, jobs: 1.0 }`. Keys you
65
+ omit keep their default, since `config.sample_rate` falls back to `1.0`
66
+ for an unset kind.
64
67
 
65
- **Per-route overrides**, from `ControllerHelpers`
66
- (`lib/railwatch/controller_helpers.rb`), included into every controller:
68
+ **Per-route overrides** come from `ControllerHelpers`, in
69
+ `lib/railwatch/controller_helpers.rb`, which is included into every
70
+ controller:
67
71
 
68
72
  ```ruby
69
73
  class ReportsController < ApplicationController
@@ -72,30 +76,30 @@ class ReportsController < ApplicationController
72
76
  end
73
77
  ```
74
78
 
75
- Both accept the same options as `before_action` (`only:`, `except:`, ...).
76
- Programmatically: `Railwatch.sample(rate)` re-rolls the current execution's
77
- sampling decision; `Railwatch.dont_sample` forces it off; `Railwatch.sampling?`
78
- reads the current decision.
79
+ Both accept the same options as `before_action`: `only:`, `except:`, and
80
+ so on. Programmatically, `Railwatch.sample(rate)` re-rolls the current
81
+ execution's sampling decision. `Railwatch.dont_sample` forces it off.
82
+ `Railwatch.sampling?` reads the current decision.
79
83
 
80
84
  ### Tail-based sampling
81
85
 
82
86
  Head sampling decides at the *start* of an execution, before anything is
83
- known about it cheap, but it throws away exactly the slow requests you
84
- wanted to see. Tail sampling keeps buffering a head-sampled-out
85
- execution's child records and decides at the *end*, once the duration and
86
- outcome are known.
87
+ known about it. That is cheap, but it throws away exactly the slow
88
+ requests you wanted to see. Tail sampling keeps buffering a
89
+ head-sampled-out execution's child records and decides at the *end*, once
90
+ the duration and outcome are known.
87
91
 
88
92
  | Attribute | Env var | Default | Meaning |
89
93
  |---|---|---|---|
90
94
  | `tail_sample_slow_ms` | `RAILWATCH_TAIL_SAMPLE_SLOW_MS` | nil (off) | Keep a head-sampled-out execution that ran at least this many milliseconds. |
91
95
 
92
- With it set (or after `Railwatch.keep!`), a head-sampled-out execution
93
- ships its whole tree when it ran at least `tail_sample_slow_ms`, when
94
- `Railwatch.keep!` was called, or when it raised an unhandled exception
95
- (subject to the `exceptions` rate); otherwise the buffered records are
96
- discarded at the end and nothing ships. Such a tree's parent record
97
- carries `tail_sampled: true`, so a tail-kept execution is
98
- distinguishable from a head-sampled one.
96
+ With it set, or after `Railwatch.keep!`, a head-sampled-out execution
97
+ ships its whole tree in three cases: it ran at least
98
+ `tail_sample_slow_ms`, `Railwatch.keep!` was called, or it raised an
99
+ unhandled exception. The exception case is subject to the `exceptions`
100
+ rate. Otherwise the buffered records are discarded at the end and nothing
101
+ ships. Such a tree's parent record carries `tail_sampled: true`, so a
102
+ tail-kept execution is distinguishable from a head-sampled one.
99
103
 
100
104
  ```ruby
101
105
  c.sample = { requests: 0.05 } # keep 5% of requests...
@@ -103,22 +107,23 @@ c.tail_sample_slow_ms = 500 # ...plus every request slower than 500ms
103
107
  Railwatch.keep! # keep this one, whatever the head decision was
104
108
  ```
105
109
 
106
- **The trade-off is memory**: with tail sampling on, every sampled-out
107
- execution buffers its child records (queries, logs, cache events, ...)
108
- for its lifetime instead of discarding them as they happen, capped at
109
- `Execution::MAX_RECORDS` (10,000) per execution. With it off the
110
- default `Execution#recording?` is false for a sampled-out execution and
111
- nothing is built or buffered at all, which is the cheapest path and
112
- exactly the behaviour Railwatch had before. `Railwatch.keep!` can only keep
113
- records made *after* the call unless tail sampling was already on: what
114
- was never buffered can't be resurrected.
110
+ **The trade-off is memory.** With tail sampling on, every sampled-out
111
+ execution buffers its child records for its lifetime instead of
112
+ discarding them as they happen. Those are queries, logs, cache events,
113
+ and so on. The buffer is capped at `Execution::MAX_RECORDS` per
114
+ execution, which is 10,000. With it off, the default,
115
+ `Execution#recording?` is false for a sampled-out execution and nothing
116
+ is built or buffered at all. That is the cheapest path and exactly the
117
+ behaviour Railwatch had before. `Railwatch.keep!` can only keep records
118
+ made *after* the call unless tail sampling was already on. What was never
119
+ buffered can't be resurrected.
115
120
 
116
121
  ### Failure context
117
122
 
118
123
  Tail sampling buys diagnosability for sampled-out executions with the
119
124
  memory to buffer *every* one of them. Failure context is the same trade
120
- on a much shorter leash: keep a bounded ring of a head-sampled-out
121
- execution's most recent child records, and ship it only if that
125
+ on a much shorter leash. It keeps a bounded ring of a head-sampled-out
126
+ execution's most recent child records, and ships it only if that
122
127
  execution reports an unhandled exception.
123
128
 
124
129
  | Attribute | Env var | Default | Meaning |
@@ -132,33 +137,34 @@ c.failure_context = 200 # ...and the last 200 records of any that fails
132
137
 
133
138
  With this set, a head-sampled-out request, job attempt, scheduled task,
134
139
  or command buffers its child records in a ring of that many. If it
135
- reports an unhandled exception the same policy that decides whether
136
- the exception itself ships, i.e. subject to the `exceptions` rate — the
137
- ring is promoted, and the parent, the exception, and the retained
138
- children all ship together, with `tail_sampled: true` on the parent. If
139
- it completes normally the ring is discarded at the end and nothing ships,
140
- exactly as before.
141
-
142
- Nothing else promotes a ring. `exceptions: 0`, an exception in
143
- `ignored_exceptions`, an exception `Railwatch.report`s as handled or that a
144
- controller's `rescue_from` swallowed, one reported inside
145
- `Railwatch.ignore` / between `Railwatch.pause` and `Railwatch.resume`, and an
146
- interactive `bin/rails runner`'s error all leave the sampled-out
147
- execution shipping exactly what it shipped before the ring existed
148
- (nothing, or the lone parent record that gives an unhandled exception
149
- somewhere to hang). `Railwatch.sample(1.0)` and `Railwatch.keep!` still work
150
- from inside the execution, and now ship the ring's contents with it
151
- rather than only what followed the call.
140
+ reports an unhandled exception, the ring is promoted. The parent, the
141
+ exception, and the retained children then all ship together, with
142
+ `tail_sampled: true` on the parent. Whether it "reports" one follows the
143
+ same policy that decides whether the exception itself ships, i.e. it is
144
+ subject to the `exceptions` rate. If it completes normally the ring is
145
+ discarded at the end and nothing ships, exactly as before.
146
+
147
+ Nothing else promotes a ring. The following all leave the sampled-out
148
+ execution shipping exactly what it shipped before the ring existed:
149
+ `exceptions: 0`, an exception in `ignored_exceptions`, an exception
150
+ `Railwatch.report`s as handled or that a controller's `rescue_from`
151
+ swallowed, one reported inside `Railwatch.ignore` or between
152
+ `Railwatch.pause` and `Railwatch.resume`, and an interactive
153
+ `bin/rails runner`'s error. That is nothing, or the lone parent record
154
+ that gives an unhandled exception somewhere to hang.
155
+ `Railwatch.sample(1.0)` and `Railwatch.keep!` still work from inside the
156
+ execution. They now ship the ring's contents with it rather than only
157
+ what followed the call.
152
158
 
153
159
  **The cost** is that a sampled-out execution builds and buffers child
154
- records again the ring bounds how many are *kept*, not how many are
155
- built so this is a fraction of what tail sampling costs, but it is not
160
+ records again. The ring bounds how many are *kept*, not how many are
161
+ built. So this is a fraction of what tail sampling costs, but it is not
156
162
  free, which is why it is off by default. There is no separate byte
157
- limit: every record type is already truncated where it is built (SQL at
158
- 16 KB, exception messages at 4 KB, attributes at 200 bytes), so
159
- `failure_context` records is also the memory bound, and overflow
160
- increments the same dropped-record counter tail sampling uses, reported
161
- with the batch rather than swallowed.
163
+ limit. Every record type is already truncated where it is built: SQL at
164
+ 16 KB, exception messages at 4 KB, attributes at 200 bytes. So
165
+ `failure_context` records is also the memory bound. Overflow increments
166
+ the same dropped-record counter tail sampling uses, reported with the
167
+ batch rather than swallowed.
162
168
 
163
169
  `failure_context` and `tail_sample_slow_ms` are independent. With both
164
170
  set, tail sampling's larger buffer wins for the whole execution: it keeps
@@ -167,9 +173,9 @@ well as on failure.
167
173
 
168
174
  ### Profiling
169
175
 
170
- Sampling and tail sampling say *which* executions ship; profiling says
171
- which of them also ship a stack profile where the time inside a slow
172
- request or job actually went (`docs/records.md`'s `profile` record).
176
+ Sampling and tail sampling say *which* executions ship. Profiling says
177
+ which of them also ship a stack profile: where the time inside a slow
178
+ request or job actually went. See `docs/records.md`'s `profile` record.
173
179
 
174
180
  The backend is an optional dependency the app installs itself, because
175
181
  neither belongs in every Gemfile:
@@ -193,16 +199,16 @@ The two triggers are different bargains:
193
199
 
194
200
  - **`profile_sample`** decides at the *start*, like head sampling. A
195
201
  profiler runs for that fraction of executions and every profile it takes
196
- is shipped. Cheap and predictable 1% of requests pay for a profiler,
202
+ is shipped. Cheap and predictable: 1% of requests pay for a profiler,
197
203
  99% pay for one `Random.rand`.
198
- - **`profile_slow_ms`** can't know an execution is slow until it is over,
199
- so it profiles *every* tail-buffering execution from its first line and
204
+ - **`profile_slow_ms`** can't know an execution is slow until it is over.
205
+ So it profiles *every* tail-buffering execution from its first line and
200
206
  throws away the ones that turn out to be fast. That means it only works
201
- together with `tail_sample_slow_ms` (nothing tail-buffers without it),
202
- and **the CPU cost is paid on every execution, not just the slow ones**
203
- the profiler's sampling thread runs throughout, and the stack table it
204
- builds is held for the execution's lifetime. Raise
205
- `profile_interval_us` if that shows up in your latency; a 5000µs
207
+ together with `tail_sample_slow_ms`, since nothing tail-buffers without
208
+ it. It also means **the CPU cost is paid on every execution, not just
209
+ the slow ones**. The profiler's sampling thread runs throughout, and the
210
+ stack table it builds is held for the execution's lifetime. Raise
211
+ `profile_interval_us` if that shows up in your latency. A 5000µs
206
212
  interval still resolves a 500ms request perfectly well.
207
213
 
208
214
  ```ruby
@@ -212,10 +218,10 @@ c.profile_slow_ms = 500 # ...and profile it
212
218
  c.profile_sample = 0.01 # plus a profile of 1% of everything else
213
219
  ```
214
220
 
215
- Both backends are process-global, so there is one profiler per process:
216
- an execution that starts while another is being profiled simply isn't
221
+ Both backends are process-global, so there is one profiler per process.
222
+ An execution that starts while another is being profiled simply isn't
217
223
  profiled. In the Rails `test` env profiling is skipped entirely unless
218
- `profile_sample` is explicitly non-zero, so a suite that inherits the
224
+ `profile_sample` is explicitly non-zero. So a suite that inherits the
219
225
  app's `RAILWATCH_*` environment doesn't start a real profiler on every
220
226
  example.
221
227
 
@@ -230,27 +236,28 @@ other Railwatch-instrumented services shows up as one trace.
230
236
  | `trace_propagation_hosts` | `RAILWATCH_TRACE_PROPAGATION_HOSTS` (comma-separated) | nil (every host) | Allow list of hostnames. An entry starting with `.` matches as a suffix (`.services.example.com` matches `api.services.example.com`); anything else must match the host exactly. |
231
237
 
232
238
  Outgoing: `traceparent: 00-<trace_id>-<execution_id[0,16]>-<flags>`, with
233
- flags `01` when the execution is sampled and `00` when it isn't — a
239
+ flags `01` when the execution is sampled and `00` when it isn't. A
234
240
  sampled-out execution still propagates, it just says so. A `traceparent`
235
241
  the app set itself is never overwritten.
236
242
 
237
243
  Inbound: the Rack middleware parses `HTTP_TRACEPARENT` and adopts its
238
244
  trace id and parent id for this execution. A header the W3C spec calls
239
- invalid is ignored and the execution starts its own trace: wrong lengths or
240
- non-hex characters, the forbidden version `ff`, an all-zero trace id, an
241
- all-zero parent id, and anything trailing the flags on version `00`. A
242
- future version may append fields after the flags, which are accepted and
243
- never interpreted as long as they are dash-delimited, so a newer upstream
244
- still links to this service instead of losing the trace. If the upstream
245
- flags say the trace is sampled, the downstream execution is kept
246
- (`Railwatch.keep!`, above) whatever its own head decision was otherwise
247
- the trace would have a hole exactly where this service should be.
245
+ invalid is ignored and the execution starts its own trace. Invalid means
246
+ wrong lengths or non-hex characters, the forbidden version `ff`, an
247
+ all-zero trace id, an all-zero parent id, or anything trailing the flags
248
+ on version `00`. A future version may append fields after the flags.
249
+ Those are accepted and never interpreted as long as they are
250
+ dash-delimited, so a newer upstream still links to this service instead
251
+ of losing the trace. If the upstream flags say the trace is sampled, the
252
+ downstream execution is kept whatever its own head decision was, as with
253
+ `Railwatch.keep!` above. Otherwise the trace would have a hole exactly
254
+ where this service should be.
248
255
 
249
256
  ## Ignoring whole record types
250
257
 
251
- `ignore` drops a record type before it's ever built cheaper than
258
+ `ignore` drops a record type before it's ever built. That is cheaper than
252
259
  filtering after the fact, and the only way to stop the highest-volume
253
- types (`query`, `cache_event`, `log`) at the source.
260
+ types at the source: `query`, `cache_event`, `log`.
254
261
 
255
262
  | Value | Env var |
256
263
  |---|---|
@@ -271,12 +278,12 @@ types (`query`, `cache_event`, `log`) at the source.
271
278
  c.ignore = [:cache_events, :transactions]
272
279
  ```
273
280
 
274
- Setting an unknown type raises `ArgumentError` immediately (this is
275
- validated at assignment, not silently dropped). Note `query` and
276
- `n_plus_one` records both key off `:queries`; `notification` off
277
- `:notifications`; see `Railwatch::PLURALS` in `lib/railwatch.rb` for the full
278
- singular-to-plural mapping used everywhere ignore/redact/reject hooks key
279
- by plural.
281
+ Setting an unknown type raises `ArgumentError` immediately. This is
282
+ validated at assignment, not silently dropped. Note `query` and
283
+ `n_plus_one` records both key off `:queries`, and `notification` off
284
+ `:notifications`. See `Railwatch::PLURALS` in `lib/railwatch.rb` for the
285
+ full singular-to-plural mapping used everywhere ignore/redact/reject
286
+ hooks key by plural.
280
287
 
281
288
  ## Redaction
282
289
 
@@ -289,22 +296,23 @@ already hides:
289
296
  | `redact_params` | `RAILWATCH_REDACT_PARAMS` (comma-separated) | `password,password_confirmation,authenticity_token,_token` |
290
297
 
291
298
  `redact_params` is merged with `Rails.application.config.filter_parameters`
292
- at first use (`Railwatch::Redactor#param_filter`), so anything the app
299
+ at first use, in `Railwatch::Redactor#param_filter`. So anything the app
293
300
  already scrubs from its own logs is scrubbed here too, with no extra
294
301
  config. Request params are only captured at all when
295
302
  `capture_request_payload` is on, and even then only for a request that
296
- raised an exception (see `request` in `docs/records.md`).
303
+ raised an exception. See `request` in `docs/records.md`.
297
304
 
298
- Header names containing a credential-shaped segment (`api-key`, `access-key`,
299
- `private-key`, `auth`, `bearer`, `credential`, `hmac`, `jwt`, `token`,
300
- `secret`, or `signature`) are always masked as a safe default, even when they
301
- arrive as concatenated Rack aliases such as `X-AuthToken`, `X-ApiToken`,
302
- `X-AccessToken`, `X-ClientToken`, `X-SessionToken`, `X-RefreshToken`,
303
- `X-SecretKey`, `X-HmacSignature`, or `X-CSRFToken`. Add application-specific
304
- aliases to `redact_headers`; ordinary diagnostic headers remain available.
305
+ Header names containing a credential-shaped segment are always masked as
306
+ a safe default. The segments are `api-key`, `access-key`, `private-key`,
307
+ `auth`, `bearer`, `credential`, `hmac`, `jwt`, `token`, `secret`, and
308
+ `signature`. This holds even when they arrive as concatenated Rack
309
+ aliases such as `X-AuthToken`, `X-ApiToken`, `X-AccessToken`,
310
+ `X-ClientToken`, `X-SessionToken`, `X-RefreshToken`, `X-SecretKey`,
311
+ `X-HmacSignature`, or `X-CSRFToken`. Add application-specific aliases to
312
+ `redact_headers`. Ordinary diagnostic headers remain available.
305
313
 
306
314
  **Per-field redaction blocks** run after a record is built, before it's
307
- buffered the block receives and can mutate the record hash in place:
315
+ buffered. The block receives and can mutate the record hash in place:
308
316
 
309
317
  ```ruby
310
318
  Railwatch.redact_queries { |q| q[:sql] = q[:sql].gsub(/email = '[^']+'/, "email = '?'") }
@@ -317,12 +325,12 @@ Railwatch.redact_outgoing_requests { |o| ... }
317
325
  Railwatch.redact_logs { |l| ... }
318
326
  ```
319
327
 
320
- A redactor that raises drops the record entirely (logged via
321
- `Railwatch.debug`, never raised into app code).
328
+ A redactor that raises drops the record entirely. The error is logged via
329
+ `Railwatch.debug`, never raised into app code.
322
330
 
323
331
  ## Rejection
324
332
 
325
- Drop a record entirely based on its content for the record types that
333
+ Drop a record entirely based on its content, for the record types that
326
334
  don't have a matching `redact_*`:
327
335
 
328
336
  ```ruby
@@ -337,39 +345,41 @@ Railwatch.reject_logs { |l| ... }
337
345
  ```
338
346
 
339
347
  `Railwatch.reject_cache_keys(prefixes)` is a shortcut that appends to
340
- `config.ignored_cache_key_prefixes`, matched by `Configuration.match_cache_key?`:
341
- a `Regexp` matches as-is; a `String` starting with `^` (or containing
342
- another regex metacharacter) is compiled as one; a `String` ending in `*`
343
- matches as a prefix; anything else must match the key exactly.
348
+ `config.ignored_cache_key_prefixes`. Entries are matched by
349
+ `Configuration.match_cache_key?`. A `Regexp` matches as-is. A `String`
350
+ starting with `^`, or containing another regex metacharacter, is compiled
351
+ as one. A `String` ending in `*` matches as a prefix. Anything else must
352
+ match the key exactly.
344
353
 
345
354
  ```ruby
346
355
  Railwatch.reject_cache_keys %w[session: rack::attack* ^feature_flag_\d+$]
347
356
  ```
348
357
 
349
- A rejector block returning truthy drops the record before it's buffered;
350
- a raising rejector is treated as "don't reject" (fails open, logged via
351
- `Railwatch.debug`).
358
+ A rejector block returning truthy drops the record before it's buffered.
359
+ A raising rejector is treated as "don't reject": it fails open and is
360
+ logged via `Railwatch.debug`.
352
361
 
353
362
  ## before_ingest
354
363
 
355
- Runs once per batch, right before it's POSTed the last chance to
356
- inspect or drop records as a group (redact/reject hooks above run
357
- per-record, earlier, at record-build time):
364
+ Runs once per batch, right before it's POSTed. This is the last chance to
365
+ inspect or drop records as a group. The redact/reject hooks above run
366
+ per-record, earlier, at record-build time:
358
367
 
359
368
  ```ruby
360
369
  Railwatch.before_ingest { |batch| batch.size < 10_000 } # return false to drop the whole batch
361
370
  Railwatch.before_ingest { |batch| batch.reject { |r| r[:t] == "log" } } # return an Array to replace it
362
371
  ```
363
372
 
364
- Multiple hooks chain; any hook returning `false` drops the batch and
365
- skips remaining hooks (`Railwatch.run_before_ingest`, `lib/railwatch.rb`).
373
+ Multiple hooks chain. Any hook returning `false` drops the batch and
374
+ skips remaining hooks. See `Railwatch.run_before_ingest` in
375
+ `lib/railwatch.rb`.
366
376
 
367
377
  ## Buffering, flushing, transport
368
378
 
369
- One background thread per process (`Railwatch::Reporter`,
370
- `lib/railwatch/reporter.rb`), re-armed after fork so each Puma cluster
371
- worker / Solid Queue forked worker gets its own. Never touches the app
372
- database.
379
+ One background thread per process: `Railwatch::Reporter`, in
380
+ `lib/railwatch/reporter.rb`. It is re-armed after fork so each Puma
381
+ cluster worker / Solid Queue forked worker gets its own. Never touches
382
+ the app database.
373
383
 
374
384
  | Attribute | Env var | Default | Meaning |
375
385
  |---|---|---|---|
@@ -385,50 +395,52 @@ database.
385
395
  | `timeout` | `RAILWATCH_TIMEOUT` | `3.0` (seconds) | Read/write timeout for the ingest POST. |
386
396
  | `shutdown_timeout` | `RAILWATCH_SHUTDOWN_TIMEOUT` | `2.0` (seconds) | Deadline for the reporter thread to deliver retained records during `at_exit`. A deployment drain timeout must be longer than this. |
387
397
 
388
- Delivery (`Railwatch::Transport::Http`, `lib/railwatch/transport/http.rb`):
389
- gzip NDJSON POST to `{ingest_url}/ingest`, one retry on a raised error or
390
- a 5xx within each delivery attempt. If that still fails or ingest returns
391
- 402, 408, or 429 the immutable batch and its prior drop count are retained
392
- for retry. Every newly formed batch gets an `X-Railwatch-Batch-Id` UUID which is
393
- reused for the immediate HTTP retry and every later reporter retry; the
394
- platform can therefore return the first committed result without inserting
395
- the payload twice. Records written while a request is in flight collect in a
396
- separate bounded buffer, so they never change the retained request's identity.
397
- At most one retained batch plus one live buffer are held in memory. The
398
- reporter retries with jittered exponential backoff (one second up to 60
399
- seconds); it does not busy-loop. A 401 marks the transport
400
- permanently unauthorized (no further HTTP attempts for the process's
401
- lifetime); it and other permanent client rejections are reported through
398
+ Delivery is `Railwatch::Transport::Http`, in
399
+ `lib/railwatch/transport/http.rb`: a gzip NDJSON POST to
400
+ `{ingest_url}/ingest`, with one retry on a raised error or a 5xx within
401
+ each delivery attempt. If that still fails, or ingest returns 402, 408,
402
+ or 429, the immutable batch and its prior drop count are retained for
403
+ retry. Every newly formed batch gets an `X-Railwatch-Batch-Id` UUID. It is
404
+ reused for the immediate HTTP retry and every later reporter retry, so
405
+ the platform can return the first committed result without inserting the
406
+ payload twice. Records written while a request is in flight collect in a
407
+ separate bounded buffer, so they never change the retained request's
408
+ identity. At most one retained batch plus one live buffer are held in
409
+ memory. The reporter retries with jittered exponential backoff, from one
410
+ second up to 60 seconds. It does not busy-loop. A 401 marks the transport
411
+ permanently unauthorized: no further HTTP attempts for the process's
412
+ lifetime. It and other permanent client rejections are reported through
402
413
  `on_unrecoverable`. Delivery never raises into app code.
403
414
 
404
- HTTPS connections explicitly use OpenSSL `VERIFY_PEER`, and redirects are not
405
- followed. Plain HTTP is refused unless the host is loopback or
406
- `RAILWATCH_ALLOW_HTTP=true`; `railwatch:doctor` reports the policy and boot logs a
407
- warning when an insecure URL is refused.
408
-
409
- On every reporter flush tick, adaptive backpressure doubles a process-local
410
- sample divisor while either buffer ceiling is at least 80% full or the retry
411
- ladder is active, up to 8x. Clear ticks halve it back toward 1x. Eight is
412
- enough to create room after three pressured ticks and recovers in three clear
413
- ticks; a 16x ceiling would preserve less telemetry and take longer to recover.
414
- The sampler reads the reporter's Float without locking the request path; the
415
- reporter is its only writer, an ivar assignment is atomic, and one stale read
416
- only affects one probabilistic decision. Set `backpressure` false to keep the
415
+ HTTPS connections explicitly use OpenSSL `VERIFY_PEER`, and redirects are
416
+ not followed. Plain HTTP is refused unless the host is loopback or
417
+ `RAILWATCH_ALLOW_HTTP=true`. `railwatch:doctor` reports the policy, and
418
+ boot logs a warning when an insecure URL is refused.
419
+
420
+ On every reporter flush tick, adaptive backpressure doubles a
421
+ process-local sample divisor, up to 8x. It does so while either buffer
422
+ ceiling is at least 80% full or the retry ladder is active. Clear ticks
423
+ halve it back toward 1x. Eight is enough to create room after three
424
+ pressured ticks and recovers in three clear ticks. A 16x ceiling would
425
+ preserve less telemetry and take longer to recover. The sampler reads the
426
+ reporter's Float without locking the request path. The reporter is its
427
+ only writer, an ivar assignment is atomic, and one stale read only
428
+ affects one probabilistic decision. Set `backpressure` false to keep the
417
429
  factor at 1. The current value is sent as
418
430
  `X-Railwatch-Backpressure-Factor` whenever it is greater than 1.
419
431
 
420
- `Railwatch.flush` forces an immediate flush (also called by the `command`
421
- patches after a rake task/runner invocation finishes, so short-lived
422
- processes don't lose their last batch to the flush interval). An unhandled
423
- exception (`Railwatch.record_now` → `Reporter#write_now`) enqueues the record
424
- and asks for an urgent flush; it never performs network I/O or a timeout
425
- cycle on the application thread. Urgent means within a quarter of a second
426
- (`Reporter::URGENT_FLUSH_DELAY`), not instantly: during an exception storm
427
- every request would otherwise wake the reporter for a handful of records,
428
- and a burst that produced 4,000 records went out as 400 POSTs of ten. A
429
- lone exception still ships inside that window; a storm coalesces into full
430
- batches, and a buffer that crosses `flush_threshold` flushes at once
431
- regardless.
432
+ `Railwatch.flush` forces an immediate flush. The `command` patches also
433
+ call it after a rake task/runner invocation finishes, so short-lived
434
+ processes don't lose their last batch to the flush interval. An unhandled
435
+ exception goes through `Railwatch.record_now` → `Reporter#write_now`,
436
+ which enqueues the record and asks for an urgent flush. It never performs
437
+ network I/O or a timeout cycle on the application thread. Urgent means
438
+ within a quarter of a second, `Reporter::URGENT_FLUSH_DELAY`, not
439
+ instantly. During an exception storm every request would otherwise wake
440
+ the reporter for a handful of records, and a burst that produced 4,000
441
+ records went out as 400 POSTs of ten. A lone exception still ships inside
442
+ that window. A storm coalesces into full batches, and a buffer that
443
+ crosses `flush_threshold` flushes at once regardless.
432
444
 
433
445
  During shutdown the reporter immediately attempts any retained batch and
434
446
  keeps retrying within `shutdown_timeout`. If the deadline expires, the batch
@@ -453,32 +465,33 @@ process exit after that deadline cannot preserve records for the next boot.
453
465
  |---|---|---|---|
454
466
  | `health_interval` | `RAILWATCH_HEALTH_INTERVAL` | `15.0` | Seconds between `health` records (Puma thread pool, Active Record pool, Solid Queue backlog — see `health` in `docs/records.md`). One background thread per web/worker process; never runs in a console, a rake task, or the `test` env. |
455
467
 
456
- Railwatch re-arms the reporter, sampler, and profiler after `fork` (one
457
- `ActiveSupport::ForkTracker` callback, Rails' own `Process._fork` hook), so
458
- clustered Puma workers and forked Solid Queue workers each get a fresh
459
- buffer, transport policy state, process record, health thread, and profiler
460
- slot. The child never flushes records or drop accounting inherited from its
461
- parent, and no `on_worker_boot` configuration is needed.
468
+ Railwatch re-arms the reporter, sampler, and profiler after `fork`. It
469
+ uses one `ActiveSupport::ForkTracker` callback, Rails' own `Process._fork`
470
+ hook. So clustered Puma workers and forked Solid Queue workers each get a
471
+ fresh buffer, transport policy state, process record, health thread, and
472
+ profiler slot. The child never flushes records or drop accounting
473
+ inherited from its parent, and no `on_worker_boot` configuration is
474
+ needed.
462
475
 
463
476
  The `process` record is written from `config.after_initialize`, after the
464
477
  app's own initializers, so `boot_seconds` covers them. A Puma master that
465
- preloads the app runs those initializers too, so it writes its own
478
+ preloads the app runs those initializers too. So it writes its own
466
479
  `process` record and starts its own reporter, health, and session threads
467
- before forking; Puma prints "Detected N Thread(s) started in app boot"
480
+ before forking. Puma prints "Detected N Thread(s) started in app boot"
468
481
  for them. That is advisory: the threads it is warning about are exactly
469
482
  the ones the fork callback replaces in every worker. The Rake and
470
483
  `bin/rails runner` patches are installed from the engine's `rake_tasks`
471
- and `runner` hooks, which only a rake or runner process fires, so a web or
472
- worker boot does not require rake or railties' runner command.
484
+ and `runner` hooks, which only a rake or runner process fires. So a web
485
+ or worker boot does not require rake or railties' runner command.
473
486
 
474
- A numeric `RAILWATCH_*` value that is not a number (`RAILWATCH_BUFFER_SIZE=12px`)
475
- falls back to the default documented in the tables above rather than being
476
- coerced to `0`.
487
+ A numeric `RAILWATCH_*` value that is not a number, such as
488
+ `RAILWATCH_BUFFER_SIZE=12px`, falls back to the default documented in the
489
+ tables above rather than being coerced to `0`.
477
490
 
478
491
  ### Release health
479
492
 
480
493
  `session` records count sessions per deploy, which is what the platform's
481
- crash-free session and crash-free user rates are computed from — the
494
+ crash-free session and crash-free user rates are computed from. The
482
495
  `deploy` on every record *is* the release.
483
496
 
484
497
  | Attribute | Env var | Default | Meaning |
@@ -489,16 +502,17 @@ crash-free session and crash-free user rates are computed from — the
489
502
 
490
503
  There are two sources, and they meet on the same id:
491
504
 
492
- - **The browser client** (`app/frontend/lib/railwatch.ts`, installed by
493
- `railwatch:install`) mints one id per tab in `sessionStorage` and mirrors it
494
- into a `railwatch_session` cookie. It rides along on the beacon flushes the
495
- client already sends for visits, so this costs no extra requests. This is
496
- the primary source for a web app, and it is what makes session duration
497
- mean "how long the tab was open".
498
- - **The request middleware** aggregates, in memory, every request that either
499
- resolves a user or carries that cookie (or an `X-Railwatch-Session` header) —
500
- the only source for an API-only app, and the only one that can see an
501
- unhandled exception, which is what makes a session `crashed`.
505
+ - **The browser client** is `app/frontend/lib/railwatch.ts`, installed by
506
+ `railwatch:install`. It mints one id per tab in `sessionStorage` and
507
+ mirrors it into a `railwatch_session` cookie. It rides along on the
508
+ beacon flushes the client already sends for visits, so this costs no
509
+ extra requests. This is the primary source for a web app, and it is
510
+ what makes session duration mean "how long the tab was open".
511
+ - **The request middleware** aggregates, in memory, every request that
512
+ either resolves a user or carries that cookie or an
513
+ `X-Railwatch-Session` header. It is the only source for an API-only
514
+ app. It is also the only one that can see an unhandled exception, which
515
+ is what makes a session `crashed`.
502
516
 
503
517
  When a browser session's requests carry the cookie both sources produce
504
518
  records under the same id and the platform dedupes them.
@@ -514,15 +528,15 @@ dominated by Rails' own housekeeping:
514
528
  | `capture_default_vendor_cache_keys` | `RAILWATCH_CAPTURE_DEFAULT_VENDOR_CACHE_KEYS` | `false` | `Configuration::DEFAULT_VENDOR_CACHE_KEYS`: `rack::attack`, `flipper`, `solid_cable`, `active_storage`, `migration_`, `schema_cache` prefixes. |
515
529
  | `capture_framework_events` | `RAILWATCH_CAPTURE_FRAMEWORK_EVENTS` | `false` | Rails 8.1 structured `Rails.event` events under `action_controller.*`, `active_record.*`, etc. — already redundant with the `request`/`job_attempt` records, so off by default. |
516
530
 
517
- `ignored_cache_key_prefixes` (code only, no env var use
518
- `Railwatch.reject_cache_keys` above) is separate from these vendor
519
- defaults and always applies.
531
+ `ignored_cache_key_prefixes` is separate from these vendor defaults and
532
+ always applies. It is code only, with no env var; use
533
+ `Railwatch.reject_cache_keys` above.
520
534
 
521
535
  ## Interactive sessions: console and runner
522
536
 
523
537
  An engineer poking at production from a shell is not the application failing.
524
538
  Sentry never hooked `bin/rails console` at all, and Railwatch keeps that
525
- behaviour while making sure a deployed script still reports.
539
+ behaviour, while making sure a deployed script still reports.
526
540
 
527
541
  | Attribute | Env var | Default | Meaning |
528
542
  |---|---|---|---|
@@ -539,10 +553,10 @@ the only thing that separates a typo from a cron job:
539
553
  | `rails runner /tmp/probe.rb` | interactive | same (a `.rb` file under `interactive_runner_paths`) |
540
554
  | `rails runner script/nightly.rb` | deployed | `command` record and the exception, as before |
541
555
 
542
- An interactive run is still recorded: the `command` record ships with its
543
- `exit_code`, duration, and `exception_preview`, so you can see that someone
544
- ran something and that it died it just doesn't open an issue. Rake tasks
545
- and Solid Queue jobs are never interactive.
556
+ An interactive run is still recorded. The `command` record ships with its
557
+ `exit_code`, duration, and `exception_preview`, so you can see that
558
+ someone ran something and that it died. It just doesn't open an issue.
559
+ Rake tasks and Solid Queue jobs are never interactive.
546
560
 
547
561
  ## Exception source and request payload
548
562
 
@@ -558,7 +572,7 @@ and Solid Queue jobs are never interactive.
558
572
  | `capture_rescued_exceptions` | `RAILWATCH_CAPTURE_RESCUED_EXCEPTIONS` | `true` | Capture exceptions a controller swallows with `rescue_from` (Rails' `rescue_from_callback.action_controller` notification) as `handled: true`, `severity: :warning`, `source: "action_controller.rescue_from"`. Sentry calls this `report_rescued_exceptions`. |
559
573
 
560
574
  `DEFAULT_IGNORED_EXCEPTIONS` is the Rails-relevant subset of Sentry's own
561
- `excluded_exceptions` defaults routine 4xx plumbing rather than
575
+ `excluded_exceptions` defaults, routine 4xx plumbing rather than
562
576
  application bugs:
563
577
 
564
578
  `ActionController::BadRequest`, `ActionController::InvalidAuthenticityToken`,
@@ -571,9 +585,9 @@ application bugs:
571
585
  `Rack::QueryParser::ParameterTypeError`.
572
586
 
573
587
  Note that Rails never reports an exception that has a `rescue_response`
574
- (`ActiveRecord::RecordNotFound` → 404) to `Rails.error` in the first
575
- place, so several of these are belt-and-braces for the paths that *do*
576
- reach Railwatch jobs, `Railwatch.report`, and `rescue_from`.
588
+ to `Rails.error` in the first place. `ActiveRecord::RecordNotFound` → 404
589
+ is one such. So several of these are belt-and-braces for the paths that
590
+ *do* reach Railwatch: jobs, `Railwatch.report`, and `rescue_from`.
577
591
 
578
592
  ## Logging
579
593
 
@@ -582,9 +596,9 @@ reach Railwatch — jobs, `Railwatch.report`, and `rescue_from`.
582
596
  | `log_level` | `RAILWATCH_LOG_LEVEL` | `:info` |
583
597
 
584
598
  Only `Rails.logger` lines at or above this level become `log` records.
585
- Rails' own per-request/job noise (`"Started GET"`, `"Processing by"`,
586
- `"Rendered"`, etc.) is filtered regardless of level, since the
587
- `request`/`job_attempt` records already carry that information.
599
+ Rails' own per-request/job noise is filtered regardless of level, since
600
+ the `request`/`job_attempt` records already carry that information. That
601
+ noise is `"Started GET"`, `"Processing by"`, `"Rendered"`, etc.
588
602
  Message text is otherwise shipped as written and is not parsed for embedded
589
603
  secrets. Keep secrets out of logs, use `Railwatch.redact_logs` for an
590
604
  application-specific scrub, or disable log records with
@@ -596,44 +610,46 @@ application-specific scrub, or disable log records with
596
610
  c.user { |user| { id: user.id, name: user.name, email: user.email } }
597
611
  ```
598
612
 
599
- Default (no block set): reads `Current.user` (authentication-zero /
600
- Rails 8 auth generator convention) if defined, else Warden's `env["warden"].user`
601
- (Devise). The resolved id is memoized per user per process-hour so a
602
- `user` record ships once, not once per request (`Railwatch::Subscribers::Users`,
603
- `docs/records.md`'s `user` section).
613
+ The default, with no block set, reads `Current.user` if defined. That is
614
+ the authentication-zero / Rails 8 auth generator convention. Otherwise it
615
+ reads Warden's `env["warden"].user`, which is Devise. The resolved id is
616
+ memoized per user per process-hour so a `user` record ships once, not
617
+ once per request. See `Railwatch::Subscribers::Users` and
618
+ `docs/records.md`'s `user` section.
604
619
 
605
620
  Ids are tenant-scoped: with a tenant bound, `1` is recorded as `acme:1`. It
606
621
  does not matter whether the tenant binds before or after the user is
607
- resolved an app that resolves the user in one `before_action` and the
608
- tenant in the next still gets `acme:1`, on the records already buffered as
609
- well as the ones after. Return an already-scoped value from the block (an
610
- external id, or `"#{org.slug}:#{user.id}"`) and it is left alone.
611
-
612
- A request resolves its user at the end, but a job enqueued mid-action needs
613
- one immediately, so `JobTracing#serialize` resolves the enqueuing
614
- execution's user and tenant and puts those two identifier strings into the
615
- Active Job payload (`railwatch_user`/`railwatch_tenant`). The worker restores
616
- them before the attempt records anything, so a `job_attempt` and every
617
- child record under it are attributed to the person whose request enqueued
618
- the job rather than to a worker process that has no signed-in user — and a
619
- job that enqueues a job passes the same identity on. Nothing but the two
620
- strings crosses the queue; no model is serialized or hydrated. Payloads
621
- carry the keys only when there is something to carry, and a payload without
622
- them falls back to local resolution, so a queue drained across a deploy
623
- keeps working. See `docs/records.md`'s `job_attempt` section for retries,
624
- scheduled jobs, and the cardinality note.
622
+ resolved. An app that resolves the user in one `before_action` and the
623
+ tenant in the next still gets `acme:1`, on the records already buffered
624
+ as well as the ones after. Return an already-scoped value from the block
625
+ and it is left alone. That might be an external id, or
626
+ `"#{org.slug}:#{user.id}"`.
627
+
628
+ A request resolves its user at the end, but a job enqueued mid-action
629
+ needs one immediately. So `JobTracing#serialize` resolves the enqueuing
630
+ execution's user and tenant and puts those two identifier strings into
631
+ the Active Job payload, as `railwatch_user`/`railwatch_tenant`. The worker
632
+ restores them before the attempt records anything. So a `job_attempt` and
633
+ every child record under it are attributed to the person whose request
634
+ enqueued the job, rather than to a worker process that has no signed-in
635
+ user. A job that enqueues a job passes the same identity on. Nothing but
636
+ the two strings crosses the queue; no model is serialized or hydrated.
637
+ Payloads carry the keys only when there is something to carry. A payload
638
+ without them falls back to local resolution, so a queue drained across a
639
+ deploy keeps working. See `docs/records.md`'s `job_attempt` section for
640
+ retries, scheduled jobs, and the cardinality note.
625
641
 
626
642
  ```ruby
627
643
  c.beacon_user { |request| Session.find_by(id: request.cookie_jar.signed[:session_token])&.user }
628
644
  ```
629
645
 
630
- Who is behind a browser beacon (visits, browser sessions, JavaScript
631
- errors). The beacon is handled by the gem's engine controller, outside your
632
- `ApplicationController`, so an app that authenticates in a `before_action`
633
- -- a signed session cookie looked up per request -- has not run it when the
634
- beacon arrives, and `Current.user` is nil there. Give Railwatch the same
635
- lookup; it hands the result to the `user` block above. Not needed when
636
- `Current.user` is set in middleware or by Warden.
646
+ Who is behind a browser beacon: visits, browser sessions, JavaScript
647
+ errors. The beacon is handled by the gem's engine controller, outside
648
+ your `ApplicationController`. So an app that authenticates in a
649
+ `before_action`, such as a signed session cookie looked up per request,
650
+ has not run it when the beacon arrives, and `Current.user` is nil there.
651
+ Give Railwatch the same lookup; it hands the result to the `user` block
652
+ above. Not needed when `Current.user` is set in middleware or by Warden.
637
653
 
638
654
  ## Tenant / context
639
655
 
@@ -641,46 +657,48 @@ lookup; it hands the result to the `user` block above. Not needed when
641
657
  Railwatch.context(tenant: org.slug, plan: org.plan)
642
658
  ```
643
659
 
644
- Writes through to `ActiveSupport::ExecutionContext`, `Rails.error.set_context`,
645
- and `Rails.event.set_context` in one call (`Railwatch::Context.set`,
646
- `lib/railwatch/context.rb`) so context set for Railwatch also shows up
647
- anywhere else Rails' own context stores are read. Serialized onto every
648
- record's `context` field, through the same `ActiveSupport::ParameterFilter`
649
- that redacts request params (`c.redact_params` plus Rails'
650
- `config.filter_parameters`) so a token or password put in context is
651
- `[FILTERED]` on the wire, not written verbatim onto every record made while
652
- it was set. A context over 64KB is rebuilt smaller rather than cut: whole
653
- values are kept while they fit, an oversized string value ends with
654
- `[TRUNCATED]`, anything that still does not fit is dropped, and the result
655
- carries `"_railwatch_truncated": true`. It is always parseable JSON — the
656
- previous behaviour sliced the encoded string at 64KB, which produced a
657
- fragment the platform could not read at all. `tenant` specifically is
658
- auto-detected with no explicit `Railwatch.context` call needed when the app
659
- uses `activerecord-tenanted` (`ActiveRecord::Base.current_tenant`) or
660
- `TenantRecord` (`TenantRecord.current_tenant`) `Context.current_tenant`
661
- checks both. The tenant is re-read while it is still nil, so a tenant bound
662
- *inside* the execution (activerecord-tenanted's `TenantSelector` middleware
663
- sits under Railwatch's, as do `around_action`s and a job's `with_tenant`
664
- block) still lands on the request/job record and every child made after
665
- the bind. Records made before the bind (a `before_action` that loads the
666
- user, say) keep `tenant: nil`.
660
+ Writes through to `ActiveSupport::ExecutionContext`,
661
+ `Rails.error.set_context`, and `Rails.event.set_context` in one call, via
662
+ `Railwatch::Context.set` in `lib/railwatch/context.rb`. So context set for
663
+ Railwatch also shows up anywhere else Rails' own context stores are read.
664
+ It is serialized onto every record's `context` field, through the same
665
+ `ActiveSupport::ParameterFilter` that redacts request params. That filter
666
+ is `c.redact_params` plus Rails' `config.filter_parameters`. So a token or
667
+ password put in context is `[FILTERED]` on the wire, not written verbatim
668
+ onto every record made while it was set. A context over 64KB is rebuilt
669
+ smaller rather than cut. Whole values are kept while they fit, an
670
+ oversized string value ends with `[TRUNCATED]`, anything that still does
671
+ not fit is dropped, and the result carries `"_railwatch_truncated": true`.
672
+ It is always parseable JSON. The previous behaviour sliced the encoded
673
+ string at 64KB, which produced a fragment the platform could not read at
674
+ all. `tenant` specifically is auto-detected with no explicit
675
+ `Railwatch.context` call needed when the app uses `activerecord-tenanted`,
676
+ via `ActiveRecord::Base.current_tenant`, or `TenantRecord`, via
677
+ `TenantRecord.current_tenant`. `Context.current_tenant` checks both. The
678
+ tenant is re-read while it is still nil. So a tenant bound *inside* the
679
+ execution still lands on the request/job record and every child made
680
+ after the bind. activerecord-tenanted's `TenantSelector` middleware sits
681
+ under Railwatch's, as do `around_action`s and a job's `with_tenant` block.
682
+ Records made before the bind keep `tenant: nil`. A `before_action` that
683
+ loads the user, say, is one such.
667
684
 
668
685
  ## Inertia: beacon and SSR
669
686
 
670
- `beacon_enabled` (`RAILWATCH_BEACON`, default `true`) gates
671
- `POST /railwatch/beacon`, mounted by the install generator
672
- (`mount Railwatch::Engine, at: "/railwatch"`) see `visit` and `exception` in
673
- `docs/records.md` for the full field lists and client batching behavior.
674
- The same beacon carries visit timing, Core Web Vitals, browser sessions,
675
- and every JavaScript error the page throws; turning `beacon_enabled` off
676
- turns off all four. The endpoint takes no credential, so it is throttled
677
- per client IP (`beacon_rate_limit`, default 120 a minute, `0` to disable);
678
- a client past the limit gets a 429 with `Retry-After` and nothing from that
679
- POST is recorded. Client setup: call `startRailwatch()` (generated at
680
- `app/frontend/lib/railwatch.ts`) from your Inertia entrypoint.
687
+ `beacon_enabled`, env var `RAILWATCH_BEACON`, default `true`, gates
688
+ `POST /railwatch/beacon`. The install generator mounts that route with
689
+ `mount Railwatch::Engine, at: "/railwatch"`. See `visit` and `exception`
690
+ in `docs/records.md` for the full field lists and client batching
691
+ behavior. The same beacon carries visit timing, Core Web Vitals, browser
692
+ sessions, and every JavaScript error the page throws. Turning
693
+ `beacon_enabled` off turns off all four. The endpoint takes no
694
+ credential, so it is throttled per client IP by `beacon_rate_limit`,
695
+ default 120 a minute, `0` to disable. A client past the limit gets a 429
696
+ with `Retry-After` and nothing from that POST is recorded. Client setup:
697
+ call `startRailwatch()` from your Inertia entrypoint. It is generated at
698
+ `app/frontend/lib/railwatch.ts`.
681
699
 
682
700
  `startRailwatch` takes three optional settings, none of which has a
683
- server-side equivalent they are decisions about the browser the code is
701
+ server-side equivalent. They are decisions about the browser the code is
684
702
  running in:
685
703
 
686
704
  ```ts
@@ -703,8 +721,8 @@ startRailwatch({
703
721
  ```
704
722
 
705
723
  The same file exports two more things. `railwatchRootOptions()` returns
706
- React 19's `onCaughtError`/`onUncaughtError` root options
707
- `createRoot(el, railwatchRootOptions())` which is what reports an error a
724
+ React 19's `onCaughtError`/`onUncaughtError` root options, used as
725
+ `createRoot(el, railwatchRootOptions())`. That is what reports an error a
708
726
  boundary caught, since React only sends those to `console.error` outside
709
727
  a development build. `reportError(error, context?)` reports an error the
710
728
  app caught itself, and is how a React 18 boundary's `componentDidCatch`
@@ -712,9 +730,9 @@ does the same thing. See
712
730
  [`docs/replacing-sentry.md`](replacing-sentry.md) for what is and is not
713
731
  captured versus `@sentry/react`.
714
732
 
715
- SSR timing needs no configuration: `Railwatch::Patches::Inertia` prepends
733
+ SSR timing needs no configuration. `Railwatch::Patches::Inertia` prepends
716
734
  `InertiaRails::Renderer#ssr_render` whenever `inertia_rails` SSR is
717
- enabled, and the resulting `ssr_ms` lands on the `request` record's
735
+ enabled. The resulting `ssr_ms` lands on the `request` record's
718
736
  `inertia` field automatically.
719
737
 
720
738
  ## Manual reporting and instrumentation
@@ -728,16 +746,16 @@ Railwatch.instrument_outgoing(:get, url) { http_client.get(url) } # for HTTP cl
728
746
  `Railwatch.report` defaults `severity` to `:warning` when `handled: true`,
729
747
  `:error` otherwise, and tags `source: "railwatch.manual"`.
730
748
  `Railwatch.instrument_outgoing` records an `outgoing_request` only if the
731
- block's return value responds to `#status` for Faraday-alike client
732
- objects that aren't Net::HTTP and don't already go through
749
+ block's return value responds to `#status`. It is for Faraday-alike
750
+ client objects that aren't Net::HTTP and don't already go through
733
751
  `Railwatch::Faraday` middleware.
734
752
 
735
753
  ### Fingerprinting
736
754
 
737
755
  How an exception is bucketed into an issue. The default is class + top
738
- in-app frame + normalized message (see
756
+ in-app frame + normalized message. See
739
757
  [`docs/records.md`](records.md)'s `exception` section for what
740
- normalization removes). Three ways to override it, in precedence order:
758
+ normalization removes. Three ways to override it, in precedence order:
741
759
 
742
760
  ```ruby
743
761
  # 1. Per call, when you already know the bucket.
@@ -754,23 +772,24 @@ Railwatch.fingerprint do |error, default|
754
772
  end
755
773
  ```
756
774
 
757
- The block is called with the error and `default` the Array of parts
758
- Railwatch would have hashed (`[class, file, line, normalized message]`). It
759
- returns an Array of strings/symbols/numbers; the literal `:default`
760
- splices those default parts in wherever you put it (Sentry's
761
- `{{ default }}`). Parts are stringified, empty ones dropped, and the
762
- result capped at 10 parts of 200 chars. Returning nil or an empty Array —
763
- or raising falls back to the default, so a bad resolver can never lose
764
- an exception. Every `exception` record carries the parts it was hashed on
765
- (`fingerprint`) and where they came from (`fingerprint_source`), and an
766
- attachment filed against the error (`Railwatch.attach(..., exception:)`)
767
- follows the same rule, so it lands on the same issue.
775
+ The block is called with the error and `default`. `default` is the Array
776
+ of parts Railwatch would have hashed:
777
+ `[class, file, line, normalized message]`. It returns an Array of
778
+ strings/symbols/numbers. The literal `:default` splices those default
779
+ parts in wherever you put it, like Sentry's `{{ default }}`. Parts are
780
+ stringified, empty ones dropped, and the result capped at 10 parts of 200
781
+ chars. Returning nil or an empty Array, or raising, falls back to the
782
+ default, so a bad resolver can never lose an exception. Every `exception`
783
+ record carries the parts it was hashed on, as `fingerprint`, and where
784
+ they came from, as `fingerprint_source`. An attachment filed against the
785
+ error with `Railwatch.attach(..., exception:)` follows the same rule, so
786
+ it lands on the same issue.
768
787
 
769
788
  ### Attachments
770
789
 
771
- Ship an arbitrary blob the payload that failed to parse, a rendered PDF,
772
- the webhook body a customer swears they sent as its own `attachment`
773
- record (Sentry's `Sentry.add_attachment`):
790
+ Ship an arbitrary blob as its own `attachment` record, like Sentry's
791
+ `Sentry.add_attachment`. That might be the payload that failed to parse,
792
+ a rendered PDF, or the webhook body a customer swears they sent:
774
793
 
775
794
  ```ruby
776
795
  Railwatch.attach("payload.json", request.raw_post) # a String
@@ -780,13 +799,13 @@ Railwatch.attach("payload.json", body, exception: error) # file it ag
780
799
  Railwatch.report(error, attachments: { "payload.json" => body }) # capture + attach in one call
781
800
  ```
782
801
 
783
- `content_type` defaults to whatever Marcel makes of the name's extension
784
- (`application/octet-stream` if it can't tell). Passing `exception:` sets
785
- the record's `exception_group_hash` to the same group hash the `exception`
786
- record is filed under, so the platform shows the attachment on that issue.
787
- An attachment made inside a recording execution belongs to it; made with
788
- nothing executing, it ships standalone. Returns nil and records nothing
789
- when Railwatch is disabled or the payload is empty.
802
+ `content_type` defaults to whatever Marcel makes of the name's extension,
803
+ or `application/octet-stream` if it can't tell. Passing `exception:` sets
804
+ the record's `exception_group_hash` to the same group hash the
805
+ `exception` record is filed under, so the platform shows the attachment
806
+ on that issue. An attachment made inside a recording execution belongs to
807
+ it. Made with nothing executing, it ships standalone. Returns nil and
808
+ records nothing when Railwatch is disabled or the payload is empty.
790
809
 
791
810
  | Attribute | Env var | Default | Meaning |
792
811
  |---|---|---|---|
@@ -799,16 +818,17 @@ Railwatch.on_unrecoverable { |error| Rails.error.report(error, handled: true) }
799
818
  ```
800
819
 
801
820
  Called whenever Railwatch rescues one of its own internal errors, ingest
802
- permanently rejects a batch, or shutdown expires with retained records that
803
- could not be sent. Retryable delivery failures stay buffered and do not fire
804
- the callback on every attempt. With no callback registered, this falls back
805
- to `Railwatch.debug` (stderr, gated on `RAILWATCH_DEBUG`, never `Rails.logger` —
806
- so gem-internal failures can never themselves become `log` records).
821
+ permanently rejects a batch, or shutdown expires with retained records
822
+ that could not be sent. Retryable delivery failures stay buffered and do
823
+ not fire the callback on every attempt. With no callback registered, this
824
+ falls back to `Railwatch.debug`. That goes to stderr, gated on
825
+ `RAILWATCH_DEBUG`, never `Rails.logger`, so gem-internal failures can
826
+ never themselves become `log` records.
807
827
 
808
828
  ## Faraday
809
829
 
810
- Opt in per connection (only needed for a non-default Faraday adapter;
811
- the default adapter is Net::HTTP, already covered globally):
830
+ Opt in per connection. This is only needed for a non-default Faraday
831
+ adapter; the default adapter is Net::HTTP, already covered globally:
812
832
 
813
833
  ```ruby
814
834
  Faraday.new(url) { |f| f.use Railwatch::Faraday }
@@ -820,98 +840,104 @@ Faraday.new(url) { |f| f.use Railwatch::Faraday }
820
840
  |---|---|---|
821
841
  | `debug` | `RAILWATCH_DEBUG` | `false` |
822
842
 
823
- Internal diagnostics to stderr (`warn`, prefixed `[railwatch]`) — deliberately
824
- not `Rails.logger`, so turning this on can't create a feedback loop of
825
- `log` records about Railwatch's own failures.
843
+ Internal diagnostics to stderr, via `warn`, prefixed `[railwatch]`. This
844
+ is deliberately not `Rails.logger`, so turning this on can't create a
845
+ feedback loop of `log` records about Railwatch's own failures.
826
846
 
827
847
  ## Public facade — full method list
828
848
 
829
- Mirrors Laravel Nightwatch's facade shape. All on the `Railwatch` module
830
- (`lib/railwatch.rb`) unless noted:
849
+ Mirrors Laravel Nightwatch's facade shape. All on the `Railwatch` module,
850
+ in `lib/railwatch.rb`, unless noted:
831
851
 
832
852
  `configure`, `config`, `enabled?`, `sample(rate)`, `dont_sample`,
833
- `keep!`, `sampling?`, `span(name, **attributes) { }`, `ignore { }` / `pause` / `resume` / `paused?` (pause/resume
834
- are the ignore block's building blocks — nestable), `record(type, **fields)`,
853
+ `keep!`, `sampling?`, `span(name, **attributes) { }`, `ignore { }` /
854
+ `pause` / `resume` / `paused?`, `record(type, **fields)`,
835
855
  `report(error, ..., attachments: {}, fingerprint: [])`, `attach(name, data, ...)`, `context(**attrs)`, `user(&block)`,
836
856
  `fingerprint(&block)`, `redact_*`,
837
857
  `reject_*`, `reject_cache_keys`, `before_ingest`, `on_unrecoverable`,
838
- `instrument_outgoing`, `flush`, `debug { }`.
858
+ `instrument_outgoing`, `flush`, `debug { }`. `pause`/`resume` are the
859
+ ignore block's building blocks, and are nestable.
839
860
 
840
861
  ## Rake tasks
841
862
 
842
- Ship with the gem via Rails::Engine's default `lib/tasks` convention
843
- (`lib/tasks/railwatch_tasks.rake`):
844
-
845
- - **`railwatch:status`** pings `{ingest_url}/ingest/ping` with the
846
- configured token; aborts if `RAILWATCH_TOKEN` is unset or the ping fails.
847
- - **`railwatch:doctor`** — prints a ✓/✗ checklist of the whole install: token,
848
- ingest URL, `GET /ingest/ping`, `Railwatch::Middleware::Request` in the
849
- middleware stack, the mounted engine's beacon route, `config.deploy` and
850
- its environment, `REVISION`, Git, or initializer source, sample rates,
851
- ignored record types, the Kamal
852
- `post-deploy` hook, `app/frontend/lib/railwatch.ts`, and whether
853
- `railwatch/rspec` (or `railwatch/minitest`) is required by the test helper.
854
- The last five are informational; it exits non-zero only when the token is
855
- missing or the ping fails.
856
- - **`railwatch:deploy[ref,name,url]`** — POSTs `{deploy, ref, name, url,
863
+ Ship with the gem via Rails::Engine's default `lib/tasks` convention, in
864
+ `lib/tasks/railwatch_tasks.rake`:
865
+
866
+ - **`railwatch:status`** pings `{ingest_url}/ingest/ping` with the
867
+ configured token. It aborts if `RAILWATCH_TOKEN` is unset or the ping
868
+ fails.
869
+ - **`railwatch:doctor`** prints a ✓/✗ checklist of the whole install:
870
+ token, ingest URL, `GET /ingest/ping`, `Railwatch::Middleware::Request`
871
+ in the middleware stack, the mounted engine's beacon route,
872
+ `config.deploy` and its environment, `REVISION`, Git, or initializer
873
+ source, sample rates, ignored record types, the Kamal `post-deploy`
874
+ hook, `app/frontend/lib/railwatch.ts`, and whether `railwatch/rspec` or
875
+ `railwatch/minitest` is required by the test helper. The last five are
876
+ informational. It exits non-zero only when the token is missing or the
877
+ ping fails.
878
+ - **`railwatch:deploy[ref,name,url]`** POSTs `{deploy, ref, name, url,
857
879
  server, timestamp, performer, destination, service, commits}` to
858
- `{ingest_url}/ingest/deploys`. `deploy` comes from `config.deploy`; aborts
859
- if that's unset. `ref` defaults to `git rev-parse HEAD` when not passed.
860
- `performer`/`destination`/`service` come from `KAMAL_PERFORMER`,
880
+ `{ingest_url}/ingest/deploys`. `deploy` comes from `config.deploy`. It
881
+ aborts if that's unset. `ref` defaults to `git rev-parse HEAD` when not
882
+ passed. `performer`/`destination`/`service` come from `KAMAL_PERFORMER`,
861
883
  `KAMAL_DESTINATION`, and `KAMAL_SERVICE`. `commits` is up to 50
862
- `{sha, author, message, at}` objects, newest first, from `git log` — empty
863
- inside an app container, which has no `.git`, which is why the hook below
864
- posts from the deployer instead.
884
+ `{sha, author, message, at}` objects, newest first, from `git log`. It
885
+ is empty inside an app container, which has no `.git`. That is why the
886
+ hook below posts from the deployer instead.
865
887
 
866
888
  ## Kamal integration
867
889
 
868
- `bin/rails generate railwatch:install` writes `.kamal/hooks/post-deploy` (only
869
- if `config/deploy.yml` already exists). It no-ops when `RAILWATCH_TOKEN` isn't
870
- set, and never fails a deploy every network call ends in `|| true`.
871
-
872
- The hook runs on the **deployer machine**, not in a container, which is the
873
- whole point: that's where the git history lives and where Kamal exports its
874
- [`KAMAL_*` variables](https://kamal-deploy.org/docs/hooks/overview/)
875
- (`KAMAL_VERSION`, `KAMAL_HOSTS`, `KAMAL_PERFORMER`, `KAMAL_DESTINATION`,
876
- `KAMAL_SERVICE`, `KAMAL_RECORDED_AT`, `KAMAL_COMMAND`, `KAMAL_SUBCOMMAND`,
877
- `KAMAL_ROLE`). With `curl`, `ruby`, and `RAILWATCH_INGEST_URL` all present it
878
- POSTs directly, twice:
879
-
880
- 1. `POST $RAILWATCH_INGEST_URL/ingest/deploys` `{deploy, ref, name, url,
881
- server, timestamp, performer, destination, service, commits}`, where
890
+ `bin/rails generate railwatch:install` writes `.kamal/hooks/post-deploy`,
891
+ but only if `config/deploy.yml` already exists. It no-ops when
892
+ `RAILWATCH_TOKEN` isn't set, and never fails a deploy. Every network call
893
+ ends in `|| true`.
894
+
895
+ The hook runs on the **deployer machine**, not in a container, which is
896
+ the whole point. That's where the git history lives and where Kamal
897
+ exports its
898
+ [`KAMAL_*` variables](https://kamal-deploy.org/docs/hooks/overview/).
899
+ Those are `KAMAL_VERSION`, `KAMAL_HOSTS`, `KAMAL_PERFORMER`,
900
+ `KAMAL_DESTINATION`, `KAMAL_SERVICE`, `KAMAL_RECORDED_AT`,
901
+ `KAMAL_COMMAND`, `KAMAL_SUBCOMMAND`, `KAMAL_ROLE`. With `curl`, `ruby`,
902
+ and `RAILWATCH_INGEST_URL` all present it POSTs directly, twice:
903
+
904
+ 1. `POST $RAILWATCH_INGEST_URL/ingest/deploys` with `{deploy, ref, name,
905
+ url, server, timestamp, performer, destination, service, commits}`.
882
906
  `commits` is up to 50 `{sha, author, message, at}` objects built from
883
- `git log -n 50 --format='%H%x1f%an%x1f%s%x1f%cI'` piped through a one-line
884
- `ruby -rjson -e`. This is what lets the platform show a diff of what
885
- actually shipped. `name` is `KAMAL_SERVICE_VERSION`; set the optional
886
- `RAILWATCH_DEPLOY_URL` to link the marker at a CI run or release page.
887
- 2. `POST $RAILWATCH_INGEST_URL/ingest/kamal` — `{version, hosts, roles,
888
- performer, destination, service, recorded_at, command, subcommand}`, with
889
- `hosts` split out of the comma-separated `KAMAL_HOSTS`. The platform uses
890
- this to know which servers should be reporting.
891
-
892
- Without `curl`/`ruby`, or without `RAILWATCH_INGEST_URL`, it falls back to the
893
- original behaviour `bin/kamal app exec --primary --reuse "bin/rails
894
- railwatch:deploy[$KAMAL_VERSION]"` which records the same deploy minus the
907
+ `git log -n 50 --format='%H%x1f%an%x1f%s%x1f%cI'` piped through a
908
+ one-line `ruby -rjson -e`. This is what lets the platform show a diff
909
+ of what actually shipped. `name` is `KAMAL_SERVICE_VERSION`. Set the
910
+ optional `RAILWATCH_DEPLOY_URL` to link the marker at a CI run or
911
+ release page.
912
+ 2. `POST $RAILWATCH_INGEST_URL/ingest/kamal` with `{version, hosts, roles,
913
+ performer, destination, service, recorded_at, command, subcommand}`.
914
+ `hosts` is split out of the comma-separated `KAMAL_HOSTS`. The platform
915
+ uses this to know which servers should be reporting.
916
+
917
+ Without `curl`/`ruby`, or without `RAILWATCH_INGEST_URL`, it falls back to
918
+ the original behaviour: `bin/kamal app exec --primary --reuse "bin/rails
919
+ railwatch:deploy[$KAMAL_VERSION]"`. That records the same deploy minus the
895
920
  commit list.
896
921
 
897
- `config.deploy` itself auto-detects `KAMAL_VERSION` (and the other release
898
- sources listed under Core) with no configuration needed even without this
899
- hook the hook's job is the deploy marker, the commit diff, and the server
900
- inventory.
922
+ `config.deploy` itself auto-detects `KAMAL_VERSION`, and the other
923
+ release sources listed under Core, with no configuration needed even
924
+ without this hook. The hook's job is the deploy marker, the commit diff,
925
+ and the server inventory.
901
926
 
902
927
  ## Overhead gate
903
928
 
904
- `bench/overhead.rb` boots the dummy app on SQLite, drives three request
905
- shapes (no queries; 20 uncached queries; the N+1 widgets page) with
906
- Railwatch's subscribers unsubscribed and then subscribed, alternating every
907
- batch, and fails (exit 1) if instrumentation adds more than the per-shape
908
- budget in `LIMITS` (CPU time on the request thread, not wall — stable
909
- under CI load plus an allocation count). It also fails if the log
910
- capture has made `Rails.logger.debug?` true. Run it with `bundle exec ruby
911
- bench/overhead.rb`. Measured on a shared box the gem adds ~0.4ms fixed per
912
- request plus 40–80µs per real query; the limits leave headroom for slower
913
- CI hosts without letting a real regression through unnoticed. The numbers
914
- and how they were taken are in [`docs/faq.md`](faq.md).
929
+ `bench/overhead.rb` boots the dummy app on SQLite and drives three
930
+ request shapes: no queries; 20 uncached queries; the N+1 widgets page. It
931
+ runs them with Railwatch's subscribers unsubscribed and then subscribed,
932
+ alternating every batch. It fails with exit 1 if instrumentation adds
933
+ more than the per-shape budget in `LIMITS`. That budget is CPU time on
934
+ the request thread, not wall, which is stable under CI load, plus an
935
+ allocation count. It also fails if the log capture has made
936
+ `Rails.logger.debug?` true. Run it with `bundle exec ruby
937
+ bench/overhead.rb`. Measured on a shared box the gem adds ~0.4ms fixed
938
+ per request plus 40–80µs per real query. The limits leave headroom for
939
+ slower CI hosts without letting a real regression through unnoticed. The
940
+ numbers and how they were taken are in [`docs/faq.md`](faq.md).
915
941
 
916
942
  ## Testing your own app against Railwatch
917
943
 
@@ -920,12 +946,13 @@ and how they were taken are in [`docs/faq.md`](faq.md).
920
946
  require "railwatch/rspec"
921
947
  ```
922
948
 
923
- `railwatch_records(type = nil)` flushes and returns buffered records (as
924
- built hashes, filtered to `type` if given) without a real network call
925
- backed by `Railwatch::SpecHelper::MemoryTransport`, swapped in for
926
- `Railwatch.reporter` on first use. `require "railwatch/rspec"` also includes
927
- `Railwatch::SpecHelper` everywhere and adds the block matchers
928
- (`have_railwatch_queries`, `have_railwatch_n_plus_one`, ...) documented in
929
- [`testing.md`](testing.md); `require "railwatch/minitest"` is the Minitest
930
- equivalent. `require "railwatch/spec_helper"` on its own, plus your own
931
- `config.include Railwatch::SpecHelper`, still works.
949
+ `railwatch_records(type = nil)` flushes and returns buffered records
950
+ without a real network call. They come back as built hashes, filtered to
951
+ `type` if given. It is backed by `Railwatch::SpecHelper::MemoryTransport`,
952
+ swapped in for `Railwatch.reporter` on first use.
953
+ `require "railwatch/rspec"` also includes `Railwatch::SpecHelper`
954
+ everywhere and adds the block matchers documented in
955
+ [`testing.md`](testing.md), such as `have_railwatch_queries` and
956
+ `have_railwatch_n_plus_one`. `require "railwatch/minitest"` is the
957
+ Minitest equivalent. `require "railwatch/spec_helper"` on its own, plus
958
+ your own `config.include Railwatch::SpecHelper`, still works.