railwatch 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/AGENTS.md +1 -1
- data/CHANGELOG.md +29 -0
- data/README.md +44 -166
- data/docs/ai-and-mcp.md +9 -9
- data/docs/configuration.md +419 -392
- data/docs/getting-started.md +10 -8
- data/docs/records.md +306 -280
- data/docs/replacing-sentry.md +6 -6
- data/docs/source-maps.md +1 -1
- data/docs/testing.md +18 -21
- data/docs/troubleshooting.md +126 -87
- data/lib/railwatch/minitest.rb +4 -3
- data/lib/railwatch/secret_safety.rb +3 -1
- data/lib/railwatch/subscribers/exceptions.rb +20 -0
- data/lib/railwatch/subscribers/process_info.rb +5 -2
- data/lib/railwatch/version.rb +1 -1
- data/lib/tasks/railwatch_tasks.rake +4 -4
- metadata +2 -2
data/docs/replacing-sentry.md
CHANGED
|
@@ -12,12 +12,12 @@ the other.
|
|
|
12
12
|
|
|
13
13
|
## Decide whether Railwatch covers your workload
|
|
14
14
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
broader, multi-language managed platform with browser replay,
|
|
19
|
-
SDKs, a large integration catalog, and generic tracing.
|
|
20
|
-
|
|
15
|
+
Railwatch and Sentry aim at different targets. Railwatch is an
|
|
16
|
+
application-monitoring product built around Rails executions: a request,
|
|
17
|
+
job, scheduled task, command, or channel action and everything it did.
|
|
18
|
+
Sentry is a broader, multi-language managed platform with browser replay,
|
|
19
|
+
native/mobile SDKs, a large integration catalog, and generic tracing.
|
|
20
|
+
Railwatch does not claim those broader capabilities.
|
|
21
21
|
|
|
22
22
|
| Workload or capability | Status | Boundary |
|
|
23
23
|
|---|---|---|
|
data/docs/source-maps.md
CHANGED
|
@@ -54,7 +54,7 @@ them and cannot resolve minified locations accurately.
|
|
|
54
54
|
|
|
55
55
|
For a custom release uploader, POST the raw `.map` bytes to
|
|
56
56
|
`/ingest/sourcemaps` with `Content-Type: application/octet-stream`,
|
|
57
|
-
`Authorization: Bearer
|
|
57
|
+
`Authorization: Bearer rw_...`, `X-Railwatch-Deploy`, and
|
|
58
58
|
`X-Railwatch-Filename` (the generated JavaScript URL path without a leading
|
|
59
59
|
slash). A successful response is HTTP 201 with
|
|
60
60
|
`{"ok":true,"filename":"vite/assets/index-abc.js","bytes":1234}`.
|
data/docs/testing.md
CHANGED
|
@@ -2,13 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
Railwatch already watches every query, N+1, span, exception, and outgoing
|
|
4
4
|
request your app makes. The same instrumentation works in your test suite,
|
|
5
|
-
which means a spec can assert on them
|
|
5
|
+
which means a spec can assert on them. CI can fail a pull request that
|
|
6
6
|
adds an N+1 or doubles a page's query count.
|
|
7
7
|
|
|
8
8
|
## Set-up
|
|
9
9
|
|
|
10
|
-
RSpec
|
|
11
|
-
for you
|
|
10
|
+
RSpec: add one line to `spec/rails_helper.rb`. The install generator adds it
|
|
11
|
+
for you.
|
|
12
12
|
|
|
13
13
|
```ruby
|
|
14
14
|
require "rspec/rails"
|
|
@@ -18,20 +18,17 @@ require "railwatch/rspec"
|
|
|
18
18
|
That requires `railwatch/spec_helper`, includes `Railwatch::SpecHelper` into every
|
|
19
19
|
example group, and defines the matchers below.
|
|
20
20
|
|
|
21
|
-
Minitest
|
|
21
|
+
Minitest: the same thing in `test/test_helper.rb`. The require includes
|
|
22
|
+
`Railwatch::Minitest` into `ActiveSupport::TestCase` on its own.
|
|
22
23
|
|
|
23
24
|
```ruby
|
|
24
25
|
require "rails/test_help"
|
|
25
26
|
require "railwatch/minitest"
|
|
26
|
-
|
|
27
|
-
class ActiveSupport::TestCase
|
|
28
|
-
include Railwatch::Minitest
|
|
29
|
-
end
|
|
30
27
|
```
|
|
31
28
|
|
|
32
29
|
Railwatch must be *enabled* in the test environment or every block would look
|
|
33
30
|
empty. `config.enabled?` is true when `config.enabled` is set and a token is
|
|
34
|
-
present, so set any non-blank `RAILWATCH_TOKEN` for the test env
|
|
31
|
+
present, so set any non-blank `RAILWATCH_TOKEN` for the test env. Records go to
|
|
35
32
|
an in-memory transport, never over the network. If Railwatch is disabled, the
|
|
36
33
|
matchers raise `Railwatch::SpecHelper::Disabled` rather than quietly passing.
|
|
37
34
|
|
|
@@ -50,9 +47,9 @@ expect { user.reload }.to have_railwatch_queries(exactly: 1)
|
|
|
50
47
|
expect { Report.generate }.to have_railwatch_queries(at_least: 1)
|
|
51
48
|
```
|
|
52
49
|
|
|
53
|
-
|
|
54
|
-
raises `ArgumentError`. On failure the message lists every statement,
|
|
55
|
-
truncated to 120 characters, so CI output says what to go and fix:
|
|
50
|
+
Pass exactly one of `at_most:`, `exactly:`, `at_least:`. Passing two, or
|
|
51
|
+
none, raises `ArgumentError`. On failure the message lists every statement,
|
|
52
|
+
each truncated to 120 characters, so CI output says what to go and fix:
|
|
56
53
|
|
|
57
54
|
```
|
|
58
55
|
expected the block to run at most 1 database queries, but it ran 3:
|
|
@@ -61,7 +58,7 @@ expected the block to run at most 1 database queries, but it ran 3:
|
|
|
61
58
|
3. SELECT "gadgets".* FROM "gadgets" WHERE "gadgets"."id" = ?
|
|
62
59
|
```
|
|
63
60
|
|
|
64
|
-
Cached queries don't count
|
|
61
|
+
Cached queries don't count. They never become `query` records.
|
|
65
62
|
|
|
66
63
|
### `have_railwatch_n_plus_one`
|
|
67
64
|
|
|
@@ -88,7 +85,7 @@ expect { importer.run }.to record_railwatch_exception(ArgumentError)
|
|
|
88
85
|
expect { importer.run }.not_to record_railwatch_exceptions
|
|
89
86
|
```
|
|
90
87
|
|
|
91
|
-
These see anything that reaches `Rails.error
|
|
88
|
+
These see anything that reaches `Rails.error`: `Rails.error.handle`,
|
|
92
89
|
`Rails.error.report`, `Railwatch.report`, and unhandled exceptions a request
|
|
93
90
|
spec's middleware catches. A block that raises out of the matcher still
|
|
94
91
|
raises; nothing is swallowed.
|
|
@@ -116,8 +113,8 @@ produces the same statement listing on failure.
|
|
|
116
113
|
## Where the matchers work
|
|
117
114
|
|
|
118
115
|
Anywhere. A request spec's `get "/widgets"` opens and closes its own
|
|
119
|
-
execution
|
|
120
|
-
|
|
116
|
+
execution. So its whole tree is visible by the time the block returns:
|
|
117
|
+
queries, N+1s, outgoing HTTP.
|
|
121
118
|
|
|
122
119
|
```ruby
|
|
123
120
|
expect { get "/widgets" }.to have_railwatch_queries(at_most: 6)
|
|
@@ -129,7 +126,7 @@ execution for the duration of the assertion and closed afterwards. No parent
|
|
|
129
126
|
opened yourself has its records read straight off that execution's buffer.
|
|
130
127
|
|
|
131
128
|
Under the hood every matcher calls `Railwatch::SpecHelper#railwatch_capture`,
|
|
132
|
-
which is public
|
|
129
|
+
which is public. Use it directly for anything the matchers don't cover:
|
|
133
130
|
|
|
134
131
|
```ruby
|
|
135
132
|
records = railwatch_capture { get "/widgets" }
|
|
@@ -164,12 +161,12 @@ RSpec.describe "performance budgets", type: :request do
|
|
|
164
161
|
end
|
|
165
162
|
```
|
|
166
163
|
|
|
167
|
-
Two ways to run it
|
|
168
|
-
|
|
169
|
-
job list
|
|
164
|
+
Two ways to run it. Tag these examples and run them as their own CI step
|
|
165
|
+
with `bundle exec rspec --tag performance`, so a budget failure is obvious in
|
|
166
|
+
the job list. Or leave them in the main suite so any pull request that adds a
|
|
170
167
|
query fails immediately. Either way the failure message names the statements,
|
|
171
168
|
so the fix is usually an `includes` one line away.
|
|
172
169
|
|
|
173
170
|
Seed enough rows in `before` that an N+1 actually crosses
|
|
174
|
-
`config.n_plus_one_threshold
|
|
171
|
+
`config.n_plus_one_threshold`. With three records, a five-query threshold
|
|
175
172
|
never fires and the gate passes on code that would melt in production.
|
data/docs/troubleshooting.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Troubleshooting
|
|
2
2
|
|
|
3
3
|
Start with `bin/rails railwatch:doctor`. It checks every piece of the
|
|
4
|
-
install in one pass and prints a `✓`/`✗` line per piece
|
|
4
|
+
install in one pass and prints a `✓`/`✗` line per piece. The sections
|
|
5
5
|
below are keyed to those lines.
|
|
6
6
|
|
|
7
7
|
```sh
|
|
@@ -9,14 +9,14 @@ bin/rails railwatch:doctor
|
|
|
9
9
|
```
|
|
10
10
|
|
|
11
11
|
The task exits non-zero only when **token** or **ingest reachable**
|
|
12
|
-
fails. Everything else is informational
|
|
12
|
+
fails. Everything else is informational. A `✗` there means a feature
|
|
13
13
|
isn't wired, not that the install is broken.
|
|
14
14
|
|
|
15
15
|
| Doctor line | What a `✗` means |
|
|
16
16
|
|---|---|
|
|
17
17
|
| `token` | `RAILWATCH_TOKEN` is unset or empty. Fatal: nothing is recorded at all. |
|
|
18
18
|
| `ingest url` | `ingest_url` isn't a parseable HTTP(S) URL. |
|
|
19
|
-
| `ingest reachable` | `GET {ingest_url}/ingest/ping` didn't return success. Fatal. |
|
|
19
|
+
| `ingest reachable` | `GET {ingest_url}/ingest/ping` didn't return success. Fatal. The ping carries the token, so a missing or wrong token fails this line too; fix `token` first. |
|
|
20
20
|
| `request middleware` | `Railwatch::Middleware::Request` isn't in the stack, so requests aren't executions. |
|
|
21
21
|
| `engine mounted` | `mount Railwatch::Engine, at: "/railwatch"` is missing from `config/routes.rb`; the browser beacon has nowhere to post. |
|
|
22
22
|
| `deploy` | `config.deploy` is unset — records ship, charts get no deploy markers. |
|
|
@@ -33,37 +33,38 @@ isn't wired, not that the install is broken.
|
|
|
33
33
|
**Symptom.** The environment's pages stay empty however much traffic the
|
|
34
34
|
app takes.
|
|
35
35
|
|
|
36
|
-
Work down this list
|
|
37
|
-
different angles
|
|
36
|
+
Work down this list. The first five are the same root cause seen from
|
|
37
|
+
different angles: Railwatch decided not to record.
|
|
38
38
|
|
|
39
39
|
**The token is missing or blank.** `Railwatch.enabled?` is
|
|
40
40
|
`config.enabled && token.present?`. With no token the engine's
|
|
41
41
|
`railwatch.subscribe` initializer returns early, so no subscribers and no
|
|
42
|
-
patches are installed at all
|
|
43
|
-
development. Fix: set `RAILWATCH_TOKEN`, restart, re-run
|
|
44
|
-
|
|
45
|
-
enough to spot a truncated or quoted value
|
|
42
|
+
patches are installed at all. This is by design, so the gem is inert in
|
|
43
|
+
development. Fix: set `RAILWATCH_TOKEN`, restart, and re-run
|
|
44
|
+
`railwatch:doctor`. The `token` line prints the first 6 characters and
|
|
45
|
+
the length, which is enough to spot a truncated or quoted value.
|
|
46
46
|
|
|
47
47
|
**The token is wrong.** A 401 from the ingest marks the transport
|
|
48
48
|
permanently unauthorized: no further flush is attempted for the lifetime
|
|
49
|
-
of that process. Fixing the env var isn't enough
|
|
49
|
+
of that process. Fixing the env var isn't enough. Restart the process.
|
|
50
50
|
`railwatch:doctor`'s `ingest reachable` line catches this before you
|
|
51
51
|
deploy.
|
|
52
52
|
|
|
53
53
|
**`RAILWATCH_INGEST_URL` points somewhere else.** Records go where you sent
|
|
54
|
-
them. `railwatch:status` prints the URL it is actually using
|
|
54
|
+
them. `railwatch:status` prints the URL it is actually using. Compare it
|
|
55
55
|
against the platform you're looking at. Self-hosting: see
|
|
56
56
|
[`self-hosting.md`](self-hosting.md).
|
|
57
57
|
|
|
58
58
|
**`config.enabled` is false.** `RAILWATCH_ENABLED=0` (or `false`/`no`/`off`)
|
|
59
59
|
turns everything off with a valid token present.
|
|
60
60
|
|
|
61
|
-
**Sample rates are at zero.** `c.sample = { requests: 0.0 }`
|
|
62
|
-
per-route `railwatch_never_sample` macro on
|
|
63
|
-
|
|
64
|
-
values. Note that an unhandled exception still
|
|
65
|
-
execution,
|
|
66
|
-
signature of a low sample rate
|
|
61
|
+
**Sample rates are at zero.** `c.sample = { requests: 0.0 }` means no
|
|
62
|
+
request records. So does the per-route `railwatch_never_sample` macro on
|
|
63
|
+
a controller. The `sample rates` doctor line prints the effective
|
|
64
|
+
values. Note that an unhandled exception can still ship from a
|
|
65
|
+
sampled-out execution, subject to the `exceptions` rate. So "exceptions
|
|
66
|
+
arrive but nothing else does" is the signature of a low sample rate
|
|
67
|
+
rather than a broken install.
|
|
67
68
|
|
|
68
69
|
**The record type is ignored.** `c.ignore` drops a type before it is
|
|
69
70
|
built. The `ignored record types` doctor line prints the list. Ignoring
|
|
@@ -71,14 +72,50 @@ built. The `ignored record types` doctor line prints the list. Ignoring
|
|
|
71
72
|
|
|
72
73
|
**You're looking at the test environment.** Requiring `railwatch/rspec`
|
|
73
74
|
(or `railwatch/minitest`) swaps the reporter's transport for an in-memory
|
|
74
|
-
one
|
|
75
|
+
one. A suite records normally but never sends anything over the
|
|
75
76
|
network. Independently: the health sampler, the session flusher, and the
|
|
76
77
|
profiler all refuse to start when `Rails.env.test?`.
|
|
77
78
|
|
|
78
79
|
Still nothing? Set `RAILWATCH_DEBUG=1` and restart. Internal diagnostics go
|
|
79
|
-
to stderr prefixed `[railwatch]
|
|
80
|
-
become `log` records about themselves
|
|
81
|
-
... }` gets the same failures as a callback.
|
|
80
|
+
to stderr prefixed `[railwatch]`. They never go to `Rails.logger`, so they
|
|
81
|
+
can't become `log` records about themselves. `Railwatch.on_unrecoverable
|
|
82
|
+
{ |e| ... }` gets the same failures as a callback.
|
|
83
|
+
|
|
84
|
+
## `bin/jobs` dies in a loop with `wrong number of arguments (given 2, expected 1)`
|
|
85
|
+
|
|
86
|
+
**Symptom.** Every Solid Queue process crashes at boot from
|
|
87
|
+
`json/common.rb` in `parse`, and Railwatch reports the same
|
|
88
|
+
`ArgumentError` from source `application.solid_queue` hundreds of times.
|
|
89
|
+
The web server still answers until it touches a session or a JSON column.
|
|
90
|
+
|
|
91
|
+
**Cause.** Not Railwatch. `json` 3.0 (September 2026) made `JSON.parse`'s
|
|
92
|
+
options keyword-only, and Active Support up to 8.1.3.1 still passes them
|
|
93
|
+
positionally ([rails/rails#58685](https://github.com/rails/rails/issues/58685)).
|
|
94
|
+
A fresh `rails new` resolves the newest `json`, so a new app hits this
|
|
95
|
+
before Railwatch is even installed.
|
|
96
|
+
|
|
97
|
+
**Fix.** Pin `json` below 3 until a Rails release includes the fix:
|
|
98
|
+
|
|
99
|
+
```sh
|
|
100
|
+
bundle add json --version "< 3"
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## Deprecations are counted but never listed
|
|
104
|
+
|
|
105
|
+
**Symptom.** The `deprecations` counter on an execution is non-zero, but
|
|
106
|
+
the Deprecations page stays empty.
|
|
107
|
+
|
|
108
|
+
**Cause.** Railwatch listens to the `deprecation.rails` notification, and
|
|
109
|
+
Rails only emits it when the app's deprecation behavior includes
|
|
110
|
+
`:notify`. The default in development is `:log` and in production
|
|
111
|
+
`:silence`, neither of which notifies.
|
|
112
|
+
|
|
113
|
+
**Fix.** Add `:notify` alongside whatever the environment already does:
|
|
114
|
+
|
|
115
|
+
```ruby
|
|
116
|
+
# config/environments/production.rb
|
|
117
|
+
config.active_support.deprecation = [:silence, :notify]
|
|
118
|
+
```
|
|
82
119
|
|
|
83
120
|
## Doubled scheduled_task records
|
|
84
121
|
|
|
@@ -86,16 +123,16 @@ become `log` records about themselves). `Railwatch.on_unrecoverable { |e|
|
|
|
86
123
|
page, at the same minute, usually with different `drift`.
|
|
87
124
|
|
|
88
125
|
**Cause.** Two Solid Queue supervisors are running against the same
|
|
89
|
-
queue database
|
|
90
|
-
`SOLID_QUEUE_IN_PUMA=true` while a dedicated job role
|
|
91
|
-
two containers of the job role. Each supervisor has
|
|
92
|
-
scheduler and its own workers, so the job really is
|
|
93
|
-
Railwatch doesn't dedupe: it records one
|
|
94
|
-
`perform.active_job`, in whichever process performed
|
|
95
|
-
rows are a true report of a doubled run.
|
|
126
|
+
queue database. That could be a stale `bin/jobs` left over from a
|
|
127
|
+
previous `bin/dev`, `SOLID_QUEUE_IN_PUMA=true` while a dedicated job role
|
|
128
|
+
is also booted, or two containers of the job role. Each supervisor has
|
|
129
|
+
its own recurring scheduler and its own workers, so the job really is
|
|
130
|
+
performed twice. Railwatch doesn't dedupe: it records one
|
|
131
|
+
`scheduled_task` per `perform.active_job`, in whichever process performed
|
|
132
|
+
it. The doubled rows are a true report of a doubled run.
|
|
96
133
|
|
|
97
134
|
**Fix.** Run one supervisor. `SolidQueue::Process.where(kind:
|
|
98
|
-
"Supervisor")` tells you how many think they're alive
|
|
135
|
+
"Supervisor")` tells you how many think they're alive. `bin/kamal app
|
|
99
136
|
logs -r job` tells you which containers are booting one. The same
|
|
100
137
|
duplication also doubles the work itself, so this is worth fixing
|
|
101
138
|
regardless of what the dashboard says.
|
|
@@ -111,9 +148,8 @@ fine.
|
|
|
111
148
|
the real class never runs.
|
|
112
149
|
|
|
113
150
|
**Fix.** Re-prepend the patch onto the replacement, once, before the
|
|
114
|
-
suite. This is exactly what the gem's own suite does
|
|
115
|
-
|
|
116
|
-
same:
|
|
151
|
+
suite. This is exactly what the gem's own suite does, in
|
|
152
|
+
`spec/spec_helper.rb`. App suites using WebMock should do the same:
|
|
117
153
|
|
|
118
154
|
```ruby
|
|
119
155
|
# WebMock replaces ::Net::HTTP with a subclass whose #request short-circuits
|
|
@@ -132,35 +168,35 @@ end
|
|
|
132
168
|
have workers.
|
|
133
169
|
|
|
134
170
|
**What actually happens.** Ruby routes `fork`, `Process.fork`, and
|
|
135
|
-
`Kernel#fork` through `Process._fork
|
|
136
|
-
with Rails' own `ActiveSupport::ForkTracker
|
|
137
|
-
uses to reset its connection pools
|
|
171
|
+
`Kernel#fork` through `Process._fork`. Railwatch registers one callback
|
|
172
|
+
with Rails' own `ActiveSupport::ForkTracker`, the same hook Active Record
|
|
173
|
+
uses to reset its connection pools. Before the child returns from
|
|
138
174
|
`fork`, it replaces the inherited reporter buffer, drop accounting,
|
|
139
175
|
transport policy state, mutexes, condition variables, dead threads, and the
|
|
140
|
-
profiler's process-global state
|
|
141
|
-
otherwise leave the child permanently unable to profile
|
|
176
|
+
profiler's process-global state. A parent's in-flight profile would
|
|
177
|
+
otherwise leave the child permanently unable to profile.
|
|
142
178
|
The parent's half-finished session map is discarded too. The child then
|
|
143
179
|
emits its own `process` record and starts fresh health/session threads for
|
|
144
|
-
its role. Parent records remain owned by and delivered from the parent
|
|
145
|
-
|
|
180
|
+
its role. Parent records remain owned by and delivered from the parent.
|
|
181
|
+
They can never be replayed by every child. **No `on_worker_boot`
|
|
146
182
|
configuration is needed**, in Puma cluster mode or in Solid Queue's
|
|
147
183
|
forked workers.
|
|
148
184
|
|
|
149
185
|
The synchronization objects are replaced without locking them. That is
|
|
150
|
-
deliberate
|
|
186
|
+
deliberate. If another parent thread owned a mutex at the instant of
|
|
151
187
|
`fork`, Ruby preserves the locked mutex in the child but not the thread
|
|
152
188
|
that could unlock it.
|
|
153
189
|
|
|
154
190
|
**When a process legitimately reports nothing.** `Health.start!` returns
|
|
155
191
|
early unless the process's role is `web` or `worker`, and
|
|
156
192
|
`Sessions.start!` only runs for `web`. Role detection is
|
|
157
|
-
`Railwatch::Subscribers::ProcessInfo.role`, in this order
|
|
158
|
-
Solid Queue is loaded and `$PROGRAM_NAME` includes `"jobs"` (or the
|
|
159
|
-
command starts with `solid_queue:`)
|
|
160
|
-
`$PROGRAM_NAME` ends in `rake
|
|
161
|
-
`process`. The worker check comes first deliberately
|
|
162
|
-
a job container too. A console or a rake task ships no health
|
|
163
|
-
design, and both modules also return early in the `test` env.
|
|
193
|
+
`Railwatch::Subscribers::ProcessInfo.role`, in this order. First `worker`,
|
|
194
|
+
when Solid Queue is loaded and `$PROGRAM_NAME` includes `"jobs"` (or the
|
|
195
|
+
command starts with `solid_queue:`). Then `console`. Then `command`, when
|
|
196
|
+
`$PROGRAM_NAME` ends in `rake`. Then `web`, when Puma is defined. Else
|
|
197
|
+
`process`. The worker check comes first deliberately, because Puma is
|
|
198
|
+
loaded in a job container too. A console or a rake task ships no health
|
|
199
|
+
records by design, and both modules also return early in the `test` env.
|
|
164
200
|
|
|
165
201
|
## Memory growth with tail sampling on
|
|
166
202
|
|
|
@@ -169,68 +205,71 @@ design, and both modules also return early in the `test` env.
|
|
|
169
205
|
|
|
170
206
|
**Cause.** That is the trade-off, not a leak. With head sampling only, a
|
|
171
207
|
sampled-out execution builds and buffers nothing. With tail sampling on,
|
|
172
|
-
*every* execution buffers its child records
|
|
173
|
-
logs, view renders
|
|
174
|
-
|
|
208
|
+
*every* execution buffers its child records for its whole lifetime:
|
|
209
|
+
queries, cache events, logs, view renders. The keep-or-discard decision
|
|
210
|
+
can't be made until it ends.
|
|
175
211
|
|
|
176
212
|
**What to check.**
|
|
177
213
|
|
|
178
214
|
- Per execution, the buffer is capped at `Execution::MAX_RECORDS`
|
|
179
|
-
(10,000). Past that, records are dropped and counted
|
|
215
|
+
(10,000). Past that, records are dropped and counted. The count is
|
|
180
216
|
added to the reporter's drop counter so the loss is visible on the
|
|
181
217
|
platform rather than silent.
|
|
182
218
|
- `c.buffer_size` (default 10,000, the same as `MAX_RECORDS`) caps the
|
|
183
219
|
process-wide queue between the app and the reporter thread.
|
|
184
|
-
Oldest-dropped-first, also counted. Do not set it below `MAX_RECORDS
|
|
185
|
-
|
|
186
|
-
a tree larger than the queue loses its own first records
|
|
187
|
-
the outgoing requests a long job made before it started
|
|
188
|
-
Keeping far more executions than before means far more
|
|
189
|
-
arriving at this queue
|
|
190
|
-
- `c.profile_slow_ms` compounds it
|
|
220
|
+
Oldest-dropped-first, also counted. Do not set it below `MAX_RECORDS`.
|
|
221
|
+
An execution's tree is written to the queue in one go when it ends, so
|
|
222
|
+
a tree larger than the queue loses its own first records. Typically
|
|
223
|
+
those are the outgoing requests a long job made before it started
|
|
224
|
+
writing. Keeping far more executions than before means far more
|
|
225
|
+
records arriving at this queue. Raise it, or lower what you keep.
|
|
226
|
+
- `c.profile_slow_ms` compounds it. It profiles every tail-buffering
|
|
191
227
|
execution from its first line and throws away the fast ones, so the
|
|
192
228
|
profiler's stack table is held alongside the record buffer.
|
|
193
229
|
- `c.failure_context` buffers sampled-out executions too, but a ring of
|
|
194
230
|
that many records each rather than all of them. If RSS climbed after
|
|
195
|
-
setting it, lower the count
|
|
231
|
+
setting it, lower the count. It is a per-execution bound, so the
|
|
196
232
|
process-wide cost is that many records times the executions running
|
|
197
233
|
concurrently.
|
|
198
234
|
|
|
199
|
-
**Fix.**
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
235
|
+
**Fix.** The threshold decides what is kept, not what is buffered:
|
|
236
|
+
with tail sampling on, every execution buffers until it ends, so
|
|
237
|
+
lowering `tail_sample_slow_ms` keeps more trees rather than fewer. To
|
|
238
|
+
buffer less, turn tail sampling off and use `Railwatch.keep!` on the
|
|
239
|
+
specific paths you care about, or drop the highest-volume child types
|
|
240
|
+
with `c.ignore` so they are never buffered.
|
|
203
241
|
|
|
204
242
|
## Profiles never appear
|
|
205
243
|
|
|
206
244
|
**Symptom.** `c.profile_sample` is set but no `profile` records ship and
|
|
207
245
|
no request is marked `profiled`.
|
|
208
246
|
|
|
209
|
-
**Cause.** No profiler backend is installed. Railwatch doesn't vendor one
|
|
247
|
+
**Cause.** No profiler backend is installed. Railwatch doesn't vendor one.
|
|
210
248
|
`Railwatch::Profiler.available?` is false unless `vernier` or `stackprof`
|
|
211
249
|
loads, and every profiling setting is inert while it is. The doctor's
|
|
212
250
|
`profiler backend` line reports this.
|
|
213
251
|
|
|
214
252
|
**Fix.** Add `gem "vernier"` (Ruby ≥ 3.2, preferred) or
|
|
215
253
|
`gem "stackprof"`. Two other reasons a profile can be absent even with a
|
|
216
|
-
backend
|
|
217
|
-
stays off rather than falling back
|
|
218
|
-
is skipped unless `profile_sample` is explicitly non-zero. Both
|
|
219
|
-
are process-global, so an execution that starts while another
|
|
220
|
-
being profiled is simply not profiled
|
|
254
|
+
backend. One is `c.profiler` pinned to a name that doesn't load; profiling
|
|
255
|
+
stays off rather than falling back. The other is the `test` env, where
|
|
256
|
+
profiling is skipped unless `profile_sample` is explicitly non-zero. Both
|
|
257
|
+
backends are process-global, so an execution that starts while another
|
|
258
|
+
one is being profiled is simply not profiled. That is expected, not a
|
|
259
|
+
bug.
|
|
221
260
|
|
|
222
261
|
## No deploy marker on the charts
|
|
223
262
|
|
|
224
263
|
**Symptom.** Charts have no vertical deploy lines; the Releases page
|
|
225
264
|
groups everything under one blank release.
|
|
226
265
|
|
|
227
|
-
**Cause.** `config.deploy` is unset. The doctor's `deploy` line says so
|
|
228
|
-
|
|
229
|
-
or initializer it came from.
|
|
266
|
+
**Cause.** `config.deploy` is unset. The doctor's `deploy` line says so.
|
|
267
|
+
When it is set, the line names the environment variable, `REVISION`, Git
|
|
268
|
+
checkout, or initializer it came from.
|
|
230
269
|
|
|
231
270
|
**Fix.** Set one of them; they are read in this order:
|
|
232
271
|
|
|
233
|
-
1. `RAILWATCH_DEPLOY
|
|
272
|
+
1. `RAILWATCH_DEPLOY`, the explicit override on any platform.
|
|
234
273
|
2. `KAMAL_VERSION`.
|
|
235
274
|
3. `GIT_REV`, `GIT_SHA`, `SOURCE_VERSION`, `HEROKU_SLUG_COMMIT`,
|
|
236
275
|
`RENDER_GIT_COMMIT`, the tag from `FLY_IMAGE_REF`,
|
|
@@ -243,8 +282,8 @@ Full 40-character SHAs are shortened to 12 characters. Or assign `deploy` in
|
|
|
243
282
|
the initializer. Set `detect_deploy`/`RAILWATCH_DETECT_DEPLOY` to false to ignore
|
|
244
283
|
steps 3–5. The value is stamped on every record, so a change only affects
|
|
245
284
|
records shipped after the restart. Note that
|
|
246
|
-
`config.deploy` and the deploy *marker* are two different things
|
|
247
|
-
marker
|
|
285
|
+
`config.deploy` and the deploy *marker* are two different things. The
|
|
286
|
+
marker, with its commit list, comes from `railwatch:deploy` or the Kamal
|
|
248
287
|
hook below.
|
|
249
288
|
|
|
250
289
|
## The Kamal hook doesn't fire
|
|
@@ -255,14 +294,14 @@ hook below.
|
|
|
255
294
|
|
|
256
295
|
- **`RAILWATCH_TOKEN` isn't exported to the hook.** The first thing
|
|
257
296
|
`.kamal/hooks/post-deploy` does is `[ -z "$RAILWATCH_TOKEN" ] && exit 0`.
|
|
258
|
-
The hook runs on the deployer machine, in your shell
|
|
259
|
-
container
|
|
260
|
-
*app* isn't necessarily in the deployer's environment. Export it there
|
|
261
|
-
|
|
297
|
+
The hook runs on the deployer machine, in your shell, not in a
|
|
298
|
+
container. So a token that only exists in `.kamal/secrets` for the
|
|
299
|
+
*app* isn't necessarily in the deployer's environment. Export it there,
|
|
300
|
+
or source the same secret store your CI uses.
|
|
262
301
|
- **`curl` or `ruby` isn't on the deployer, or `RAILWATCH_INGEST_URL`
|
|
263
302
|
isn't set.** Then the hook falls back to
|
|
264
303
|
`bin/kamal app exec --primary --reuse "bin/rails railwatch:deploy[$KAMAL_VERSION]"`,
|
|
265
|
-
which records the same deploy **minus the commit list
|
|
304
|
+
which records the same deploy **minus the commit list**. A container
|
|
266
305
|
has the code but not the git history. If your deploys show up without
|
|
267
306
|
commits, this is the path you're on.
|
|
268
307
|
- **The hook isn't there.** The install generator only writes it when
|
|
@@ -278,8 +317,8 @@ it exits 0 regardless.
|
|
|
278
317
|
fewer lines and highlights nothing.
|
|
279
318
|
|
|
280
319
|
**Cause.** Full-text search uses SQLite's FTS5 (`logs_fts`). The
|
|
281
|
-
platform checks for both
|
|
282
|
-
|
|
320
|
+
platform checks for both a SQLite adapter *and* the `logs_fts` table.
|
|
321
|
+
When either is absent it falls back to `message LIKE '%...%'`. That
|
|
283
322
|
fallback is a plain substring match: no phrase or negation syntax, and no
|
|
284
323
|
snippet highlighting.
|
|
285
324
|
|
|
@@ -294,11 +333,11 @@ self-hosted platform operator should rebuild the documented search index.
|
|
|
294
333
|
**Symptom.** Every other page has data; Tenants shows nothing.
|
|
295
334
|
|
|
296
335
|
**Cause.** `tenant` is a column on every telemetry row, filled from
|
|
297
|
-
`Railwatch::Context.current_tenant
|
|
336
|
+
`Railwatch::Context.current_tenant`. A tenant only exists as a GROUP BY
|
|
298
337
|
over those rows. If nothing ever sets it, every row has a null tenant and
|
|
299
338
|
there is nothing to group.
|
|
300
339
|
|
|
301
|
-
**Fix.** Apps on `activerecord-tenanted` get it free
|
|
340
|
+
**Fix.** Apps on `activerecord-tenanted` get it free. Railwatch reads
|
|
302
341
|
`ActiveRecord::Base.current_tenant` / `TenantRecord.current_tenant` with
|
|
303
342
|
no configuration. Everyone else sets it explicitly, as early in the
|
|
304
343
|
request as the tenant is known:
|
|
@@ -313,7 +352,7 @@ does not retroactively apply to it.
|
|
|
313
352
|
|
|
314
353
|
## See also
|
|
315
354
|
|
|
316
|
-
- [`configuration.md`](configuration.md)
|
|
317
|
-
- [`records.md`](records.md)
|
|
318
|
-
- [`faq.md`](faq.md)
|
|
355
|
+
- [`configuration.md`](configuration.md): every option and its default.
|
|
356
|
+
- [`records.md`](records.md): what each record type contains.
|
|
357
|
+
- [`faq.md`](faq.md): overhead, retention, PII, and what happens when
|
|
319
358
|
the platform is unreachable.
|
data/lib/railwatch/minitest.rb
CHANGED
|
@@ -7,10 +7,9 @@ module Railwatch
|
|
|
7
7
|
# assert_/refute_. Add to test/test_helper.rb:
|
|
8
8
|
#
|
|
9
9
|
# require "railwatch/minitest"
|
|
10
|
-
# class ActiveSupport::TestCase
|
|
11
|
-
# include Railwatch::Minitest
|
|
12
|
-
# end
|
|
13
10
|
#
|
|
11
|
+
# That includes the module into ActiveSupport::TestCase (through its load
|
|
12
|
+
# hook, so the require order against rails/test_help does not matter).
|
|
14
13
|
# Includes Railwatch::SpecHelper, so `railwatch_records(:query)` is available
|
|
15
14
|
# too. See docs/testing.md.
|
|
16
15
|
module Minitest
|
|
@@ -41,3 +40,5 @@ module Railwatch
|
|
|
41
40
|
end
|
|
42
41
|
end
|
|
43
42
|
end
|
|
43
|
+
|
|
44
|
+
ActiveSupport.on_load(:active_support_test_case) { include Railwatch::Minitest } if defined?(ActiveSupport.on_load)
|
|
@@ -6,7 +6,9 @@ module Railwatch
|
|
|
6
6
|
# Read-only Git checks used by the installer and doctor. Tokens are never
|
|
7
7
|
# returned in diagnostics: callers get a path or a short prefix only.
|
|
8
8
|
module SecretSafety
|
|
9
|
-
|
|
9
|
+
# rw_ is the ingest token prefix; lt_ was the prefix before 0.1.1 and is
|
|
10
|
+
# still matched so a token minted earlier is still caught in a tracked file.
|
|
11
|
+
TOKEN_PATTERN = /\b(?:rw|lt)_[A-Za-z0-9_-]{6,}\b/
|
|
10
12
|
TOKEN_FILE_GLOBS = [ ".env", ".env.*", ".kamal/secrets", "config/deploy.yml",
|
|
11
13
|
"config/initializers/*.rb" ].freeze
|
|
12
14
|
|
|
@@ -87,6 +87,13 @@ module Railwatch
|
|
|
87
87
|
def capture(error, handled:, severity:, context: {}, source: nil, fingerprint: nil)
|
|
88
88
|
return unless Railwatch.enabled?
|
|
89
89
|
return if ignored?(error)
|
|
90
|
+
# Solid Queue re-raises a failed job's error out of the worker thread,
|
|
91
|
+
# where its executor reports it to Rails.error a second time
|
|
92
|
+
# (source application.solid_queue) after the job_attempt execution
|
|
93
|
+
# has already finished and reported it. The execution's own
|
|
94
|
+
# first_exception_report? bookkeeping is gone by then, so the error
|
|
95
|
+
# object itself remembers that its unhandled report has shipped.
|
|
96
|
+
return if !handled && reported_unhandled?(error)
|
|
90
97
|
|
|
91
98
|
exe = execution
|
|
92
99
|
if exe&.first_exception_observation?(error, handled)
|
|
@@ -151,9 +158,22 @@ module Railwatch
|
|
|
151
158
|
# children (Railwatch.tail_keep?).
|
|
152
159
|
exe.exception_reported = true if exe
|
|
153
160
|
Railwatch.record_now(:exception, group: group, **rec)
|
|
161
|
+
remember_reported(error)
|
|
154
162
|
end
|
|
155
163
|
end
|
|
156
164
|
|
|
165
|
+
def reported_unhandled?(error)
|
|
166
|
+
error.instance_variable_defined?(:@__railwatch_reported)
|
|
167
|
+
rescue StandardError
|
|
168
|
+
false
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
def remember_reported(error)
|
|
172
|
+
error.instance_variable_set(:@__railwatch_reported, true)
|
|
173
|
+
rescue StandardError
|
|
174
|
+
nil
|
|
175
|
+
end
|
|
176
|
+
|
|
157
177
|
# The group hash `capture` would assign this error. Public so
|
|
158
178
|
# Railwatch.attach can file an attachment against the same issue without
|
|
159
179
|
# having to re-derive the bucketing rule (source snippets are skipped:
|
|
@@ -84,9 +84,12 @@ module Railwatch
|
|
|
84
84
|
|
|
85
85
|
# Puma is loaded in every process of an app that bundles it, so a Solid
|
|
86
86
|
# Queue worker is recognised first, by how it was started (bin/jobs or
|
|
87
|
-
# `rake solid_queue:start`)
|
|
87
|
+
# `rake solid_queue:start`) or by the procline Solid Queue gives every
|
|
88
|
+
# process it forks ("solid-queue-worker(1.7.0): ..."), which replaces
|
|
89
|
+
# $PROGRAM_NAME after boot and would otherwise turn the supervisor,
|
|
90
|
+
# dispatcher, and scheduler into "web" on every health sample.
|
|
88
91
|
def role
|
|
89
|
-
if defined?(::SolidQueue) && ($PROGRAM_NAME.include?("jobs") || ARGV.first.to_s.start_with?("solid_queue:")) then "worker"
|
|
92
|
+
if defined?(::SolidQueue) && ($PROGRAM_NAME.include?("jobs") || $PROGRAM_NAME.start_with?("solid-queue-") || ARGV.first.to_s.start_with?("solid_queue:")) then "worker"
|
|
90
93
|
elsif defined?(::Rails::Console) then "console"
|
|
91
94
|
elsif $PROGRAM_NAME.end_with?("rake") then "command"
|
|
92
95
|
elsif defined?(::Puma) then "web"
|
data/lib/railwatch/version.rb
CHANGED
|
@@ -186,11 +186,11 @@ namespace :railwatch do
|
|
|
186
186
|
|
|
187
187
|
1. Sign in (or sign up) at #{base.url("/dashboard")}
|
|
188
188
|
2. New application, then New environment (production, staging, ...)
|
|
189
|
-
3. The environment's token (
|
|
189
|
+
3. The environment's token (rw_...) is shown once, right after it is created.
|
|
190
190
|
|
|
191
191
|
Then set it where this app reads its environment:
|
|
192
192
|
|
|
193
|
-
RAILWATCH_TOKEN=
|
|
193
|
+
RAILWATCH_TOKEN=rw_...#{"\n RAILWATCH_INGEST_URL=#{base.host_url}" if base.self_hosted?}
|
|
194
194
|
|
|
195
195
|
With Kamal: bin/rails generate railwatch:install --prompt-token --kamal-secrets
|
|
196
196
|
Verify: bin/rails railwatch:doctor
|
|
@@ -204,12 +204,12 @@ namespace :railwatch do
|
|
|
204
204
|
task mcp: :environment do
|
|
205
205
|
base = Railwatch::Endpoints.new(Railwatch.config)
|
|
206
206
|
mcp = base.url("/mcp")
|
|
207
|
-
token = "
|
|
207
|
+
token = "rwp_your_token_here"
|
|
208
208
|
puts <<~TEXT
|
|
209
209
|
Railwatch MCP server: #{mcp}
|
|
210
210
|
|
|
211
211
|
An MCP token is per person, not per app: Settings -> Profile -> "API & MCP
|
|
212
|
-
token" at #{base.url("/settings/profile")}. It starts with
|
|
212
|
+
token" at #{base.url("/settings/profile")}. It starts with rwp_ and is
|
|
213
213
|
shown once. Everything below is scoped to whatever accounts that user
|
|
214
214
|
belongs to.
|
|
215
215
|
|