railwatch-sidecar 0.2.0-x86_64-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 661d946a8cc373933fac81449919067596a853f1f3cbaa3ff56660bd6788d484
4
+ data.tar.gz: 3f1d9c271a89b1ba9b1342d97aae3c86e9bff2a6377d8f8560d3d5e4e8e5717d
5
+ SHA512:
6
+ metadata.gz: eb004ddfc093b4569aea30d3f39ebc3fad199d21f8012dd6801fff156b00f4d778fa357d2904f51c8276eeea870e6cd820f047684d275664b672cebe4839d2f9
7
+ data.tar.gz: 3708b7a60e4ac44b0e901fb73a398bd78e9bb789089a47a45d0ee59a57c5313b64f7e2d90994bdab321e710a9505f2a38396e9df7c6ee17636eebd7b0b3638a4
data/LICENSE ADDED
@@ -0,0 +1,105 @@
1
+ # Functional Source License, Version 1.1, ALv2 Future License
2
+
3
+ ## Abbreviation
4
+
5
+ FSL-1.1-ALv2
6
+
7
+ ## Notice
8
+
9
+ Copyright 2026 Cole Robertson and Rebulk
10
+
11
+ ## Terms and Conditions
12
+
13
+ ### Licensor ("We")
14
+
15
+ The party offering the Software under these Terms and Conditions.
16
+
17
+ ### The Software
18
+
19
+ The "Software" is each version of the software that we make available under
20
+ these Terms and Conditions, as indicated by our inclusion of these Terms and
21
+ Conditions with the Software.
22
+
23
+ ### License Grant
24
+
25
+ Subject to your compliance with this License Grant and the Patents,
26
+ Redistribution and Trademark clauses below, we hereby grant you the right to
27
+ use, copy, modify, create derivative works, publicly perform, publicly display
28
+ and redistribute the Software for any Permitted Purpose identified below.
29
+
30
+ ### Permitted Purpose
31
+
32
+ A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
33
+ means making the Software available to others in a commercial product or
34
+ service that:
35
+
36
+ 1. substitutes for the Software;
37
+
38
+ 2. substitutes for any other product or service we offer using the Software
39
+ that exists as of the date we make the Software available; or
40
+
41
+ 3. offers the same or substantially similar functionality as the Software.
42
+
43
+ Permitted Purposes specifically include using the Software:
44
+
45
+ 1. for your internal use and access;
46
+
47
+ 2. for non-commercial education;
48
+
49
+ 3. for non-commercial research; and
50
+
51
+ 4. in connection with professional services that you provide to a licensee
52
+ using the Software in accordance with these Terms and Conditions.
53
+
54
+ ### Patents
55
+
56
+ To the extent your use for a Permitted Purpose would necessarily infringe our
57
+ patents, the license grant above includes a license under our patents. If you
58
+ make a claim against any party that the Software infringes or contributes to
59
+ the infringement of any patent, then your patent license to the Software ends
60
+ immediately.
61
+
62
+ ### Redistribution
63
+
64
+ The Terms and Conditions apply to all copies, modifications and derivatives of
65
+ the Software.
66
+
67
+ If you redistribute any copies, modifications or derivatives of the Software,
68
+ you must include a copy of or a link to these Terms and Conditions and not
69
+ remove any copyright notices provided in or with the Software.
70
+
71
+ ### Disclaimer
72
+
73
+ THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
74
+ IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
75
+ PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
76
+
77
+ IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
78
+ SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
79
+ EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
80
+
81
+ ### Trademarks
82
+
83
+ Except for displaying the License Details and identifying us as the origin of
84
+ the Software, you have no right under these Terms and Conditions to use our
85
+ trademarks, trade names, service marks or product names.
86
+
87
+ ## Grant of Future License
88
+
89
+ We hereby irrevocably grant you an additional license to use the Software under
90
+ the Apache License, Version 2.0 that is effective on the second anniversary of
91
+ the date we make the Software available. On or after that date, you may use the
92
+ Software under the Apache License, Version 2.0, in which case the following
93
+ will apply:
94
+
95
+ Licensed under the Apache License, Version 2.0 (the "License"); you may not use
96
+ this file except in compliance with the License.
97
+
98
+ You may obtain a copy of the License at
99
+
100
+ http://www.apache.org/licenses/LICENSE-2.0
101
+
102
+ Unless required by applicable law or agreed to in writing, software distributed
103
+ under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
104
+ CONDITIONS OF ANY KIND, either express or implied. See the License for the
105
+ specific language governing permissions and limitations under the License.
data/NOTICE ADDED
@@ -0,0 +1,25 @@
1
+ railwatch-sidecar bundles the Railwatch server.
2
+
3
+ The platform gems of railwatch-sidecar (x86_64-linux, aarch64-linux,
4
+ x86_64-darwin, arm64-darwin) contain exe/<platform>/railwatch, a compiled
5
+ build of Railwatch (railwatch-rs), https://github.com/Rebulk/railwatch-rs,
6
+ which is under the Functional Source License, Version 1.1, ALv2 Future
7
+ License (FSL-1.1-ALv2). This gem's own Ruby code is under the same license;
8
+ see LICENSE.
9
+
10
+ The source of the bundled binary is the railwatch-rs commit
11
+ named in lib/railwatch/sidecar/version.rb (RAILWATCH_RS), unmodified. The
12
+ running server also links to its source on every response (the Link header and /source).
13
+ Third-party code compiled into the binary keeps its own licenses, listed in
14
+ railwatch-rs's THIRD-PARTY-LICENSES.
15
+
16
+ Running this gem in your application does not make your application a
17
+ derivative of Railwatch: the server is a separate program, started as its
18
+ own process and spoken to over HTTP. You may run it, and change it, to
19
+ monitor your own applications; the license does not allow offering it, or
20
+ something substantially like it, to others as a commercial product or
21
+ service.
22
+
23
+ The railwatch gem (https://github.com/Rebulk/railwatch), the agent that
24
+ reports your app's telemetry, is a separate gem under the MIT license. This
25
+ gem depends on it; it is not part of this gem.
data/README.md ADDED
@@ -0,0 +1,447 @@
1
+ # railwatch-sidecar
2
+
3
+ The Railwatch server, bundled as a gem. Add it to a Rails app and you get
4
+ the **full** Railwatch dashboard at `/railwatch` (issues, alerts, the API
5
+ and MCP included), behind your app's own login, with no server to install.
6
+ The server runs as **its own process** next to the app, started by Puma's
7
+ master. It never runs inside a Puma worker.
8
+
9
+ ## Coming from railwatch-server
10
+
11
+ This gem was called `railwatch-server` until 0.2.0. Replace the gem in your
12
+ Gemfile, then rename at your own pace: every old name keeps working until
13
+ 1.0 and warns once per process.
14
+
15
+ | before | now |
16
+ |---|---|
17
+ | `gem "railwatch-server"` | `gem "railwatch-sidecar"` |
18
+ | `plugin :railwatch_server` | `plugin :railwatch_sidecar` |
19
+ | `RAILWATCH_SERVER_*` | `RAILWATCH_SIDECAR_*` (the new name wins when both are set) |
20
+ | `Railwatch::Server.configure` | `Railwatch::Sidecar.configure` |
21
+ | `bin/rails railwatch_server:start` / `:status` | `bin/rails railwatch_sidecar:start` / `:status` |
22
+ | `credentials.railwatch_server` | `credentials.railwatch_sidecar` |
23
+
24
+ The data dir (`storage/railwatch/<env>`), the port and the dashboard path
25
+ are unchanged, so nothing is moved or re-provisioned.
26
+
27
+ ## How it relates to the other two
28
+
29
+ | | what it is | license |
30
+ |---|---|---|
31
+ | [`railwatch`](https://github.com/Rebulk/railwatch) | the agent in your app; on its own it reports to Railwatch Cloud or a self-hosted server | MIT |
32
+ | [`railwatch-rs`](https://github.com/Rebulk/railwatch-rs) | the server, one static binary; self-hosted on a VPS for many apps | FSL-1.1-ALv2 |
33
+ | **`railwatch-sidecar`** (this) | `railwatch-rs`, prebuilt per platform, run and wired for one app | FSL-1.1-ALv2 |
34
+
35
+ This gem depends on `railwatch` and points the agent at the bundled server.
36
+ It is the same server you would self-host, at the same version, just started
37
+ for you. Moving later to a shared self-hosted install means changing
38
+ `RAILWATCH_INGEST_URL` and `RAILWATCH_TOKEN` (and copying `storage/railwatch`
39
+ if you want the history).
40
+
41
+ ## Install
42
+
43
+ ```ruby
44
+ # Gemfile
45
+ gem "railwatch"
46
+ gem "railwatch-sidecar"
47
+ ```
48
+
49
+ ```ruby
50
+ # config/puma.rb
51
+ plugin :railwatch_sidecar
52
+ ```
53
+
54
+ ```ruby
55
+ # config/initializers/railwatch.rb: the dashboard's gate (the railwatch gem's settings)
56
+ Railwatch.configure do |c|
57
+ c.http_basic_auth_user = "admin" # or bin/rails railwatch:authentication:configure
58
+ c.http_basic_auth_password = ENV["RAILWATCH_DASHBOARD_PASSWORD"]
59
+ end
60
+ ```
61
+
62
+ `bin/rails s`, then open `/railwatch`. On first boot the gem runs
63
+ `railwatch setup` against `storage/railwatch/<Rails.env>`, so the account,
64
+ app, environment, ingest token and proxy secret are created once and kept
65
+ there. It then starts `railwatch serve` on `127.0.0.1:28790` and points the
66
+ agent at it.
67
+
68
+ Only a process that starts the server creates that directory: Puma with the
69
+ plugin (`bin/rails server` or `puma`) and `bin/rails railwatch_sidecar:start`.
70
+ Anything else (`assets:precompile` in a Dockerfile, a console, a Solid Queue
71
+ worker) reads it and, until it exists, leaves the agent alone and says so.
72
+ So an image build ships no token, proxy secret or `secret_key_base`; each
73
+ volume gets its own on first start. A job worker started before the first
74
+ server reports from its next restart. A `railwatch setup` that hangs never
75
+ holds up the boot past `RAILWATCH_SIDECAR_SETUP_TIMEOUT` (below).
76
+
77
+ Bundler picks the platform gem with the binary for your machine:
78
+ x86_64-linux, aarch64-linux, x86_64-darwin or arm64-darwin. On anything
79
+ else the plain gem installs and tells you to set `RAILWATCH_SIDECAR_BINARY`
80
+ to a `railwatch` you installed yourself.
81
+
82
+ ### Without Puma
83
+
84
+ Use `bin/rails railwatch_sidecar:start` (foreground, TERM to stop), e.g. as a
85
+ Procfile line:
86
+
87
+ ```
88
+ railwatch: bin/rails railwatch_sidecar:start
89
+ ```
90
+
91
+ `bin/rails railwatch_sidecar:status` reports the binary, the data dir,
92
+ whether the server is up, and where the agent reports.
93
+
94
+ ## Configuration
95
+
96
+ | env | default | |
97
+ |---|---|---|
98
+ | `RAILWATCH_SIDECAR_ENABLED` | on, off under `Rails.env.test` | off: no server, no route, the agent untouched |
99
+ | `RAILWATCH_SIDECAR_PORT` | `28790` | loopback port. Fixed, not picked at boot, so every process of the app finds the same server |
100
+ | `RAILWATCH_SIDECAR_DATA_DIR` | `storage/railwatch/<Rails.env>` | put it on the volume that survives deploys |
101
+ | `RAILWATCH_SIDECAR_MOUNT_PATH` | `/railwatch` | |
102
+ | `RAILWATCH_SIDECAR_URL` | the loopback URL | public URL of the app, for links in alert emails and Secure cookies |
103
+ | `RAILWATCH_SIDECAR_BINARY` | the platform gem's | your own `railwatch` |
104
+ | `RAILWATCH_SIDECAR_STOP_TIMEOUT` | `15` | seconds Puma's exit waits for the server to drain |
105
+ | `RAILWATCH_SIDECAR_SETUP_TIMEOUT` | `30` | seconds the first start of a data dir waits for `railwatch setup` (about a second), before Puma listens. Past it the setup is KILLed, the app boots without the server (agent not wired, the dashboard's 503 says why) and the next start retries |
106
+ | `RAILWATCH_SIDECAR_WIRE_AGENT` | `true` | off: you configure the agent yourself |
107
+ | `RAILWATCH_SIDECAR_MAX_LIVE_CONNECTIONS` | `64` | open live-update connections per app process (one per dashboard tab); past it the dashboard works without live updates |
108
+ | `RAILWATCH_SIDECAR_MAX_PROXIED_REQUESTS` | `64` | dashboard requests in flight per app process (waiting on the server or streaming back); past it a dashboard request gets 503 with `Retry-After: 1` at once. The app's own requests are never affected |
109
+ | `RAILWATCH_SIDECAR_PROXY_CONNECT_TIMEOUT` | `2` | seconds to connect to the server on loopback; past it, 503 |
110
+ | `RAILWATCH_SIDECAR_PROXY_FIRST_BYTE_TIMEOUT` | `25` | seconds for the server to start answering a dashboard request; past it, 504 with a message. Under 30 s on purpose: Thruster (its write timeout) and kamal-proxy drop a response that hasn't started after 30 s, and the visitor gets an empty reply instead |
111
+ | `RAILWATCH_SIDECAR_PROXY_IDLE_TIMEOUT` | `60` | seconds a response that has started may go without a byte moving; past it the connection is closed |
112
+ | `RAILWATCH_SIDECAR_AGENT_ACCESS` | on in development for this machine only, off elsewhere | whether `<mount>/mcp` and `<mount>/api/` (an AI agent's requests) are signed for the visitor like the dashboard, so the agent needs no token. See [AI agents](#ai-agents) |
113
+ | `RAILWATCH_SIDECAR_NICE` | `10` | how far below the app's CPU priority the server runs (0-19; 0: the same). A busy dashboard can keep every core busy; measured in a 2-CPU container under 30 dashboard clients at the app's priority, the app's `/up` p50 went from 2 to 15 ms. Not under a CPU quota (`docker --cpus`, Kubernetes CPU limits): the kernel throttles the container's processes together whatever their priority, and in that 2-CPU container under `--cpus 2` nice 10 measured no better than nice 0 (`/up` p99 190-580 ms at either); on 2 dedicated cores (`--cpuset-cpus`) it took `/up` p99 from 207-363 ms to 18-22 ms. In a quota-limited container give it real cores (a cpuset, or a VM) or keep dashboard use light |
114
+
115
+ **Environments.** One environment per data dir, named after `Rails.env`:
116
+ development, staging and production usually run on different machines,
117
+ and each keeps its own history in `storage/railwatch/<env>`. Under
118
+ `Rails.env.test` the server is off by default (a test run would otherwise
119
+ report into the development server). The data dir records the environment
120
+ it was set up for (`environment`), and an app booted in another
121
+ environment against it refuses to wire the agent and says so, rather than
122
+ filing staging's telemetry under production. To see several environments
123
+ in one dashboard, run one shared self-hosted `railwatch-rs` and point each
124
+ environment's agent at it (`RAILWATCH_INGEST_URL`, `RAILWATCH_TOKEN`).
125
+
126
+ **The server's own settings** (alert and digest email, backups, Google
127
+ sign-in, retention, whether alerts are sent) come from the app, written on
128
+ every server start to `<data dir>/server.env` (0600) and never logged
129
+ (the log names only what was set: "settings from the app: smtp, backup"):
130
+
131
+ | server setting | from |
132
+ |---|---|
133
+ | SMTP | `config.smtp`, else Action Mailer's `smtp_settings` when `delivery_method` is `:smtp` |
134
+ | From | `config.mail_from`, else `ActionMailer::Base.default[:from]` |
135
+ | S3 backups | `config.backup`, else `credentials.railwatch_sidecar.backup` (`endpoint bucket access_key_id secret_access_key`, optional `region prefix interval_seconds`) |
136
+ | Google sign-in | `config.google`, else `credentials.railwatch_sidecar.google` (`client_id client_secret`) |
137
+ | retention | `config.retention = { detail_days:, core_days_max: }` |
138
+ | alert delivery | `config.alert_delivery = false` records alerts without sending |
139
+ | anything else | `config.server_env = { "RAILWATCH_NEW_ISSUES_PER_HOUR" => 50 }` |
140
+
141
+ `false` or `{}` turns one off where the app would otherwise supply it. An
142
+ incomplete backup is refused whole (logged, no backups) rather than
143
+ half-applied. `railwatch doctor` with `RAILWATCH_ENV_FILE` pointed at the
144
+ file shows what the server reads.
145
+
146
+ The agent is wired only when the app has not configured a destination:
147
+ when `RAILWATCH_TOKEN` is set, it is left alone and the bundled server
148
+ serves only the dashboard.
149
+
150
+ **Who the server sees.** The dashboard's gate is the `railwatch` gem's:
151
+ HTTP Basic, on and closed by default; or, with Basic off,
152
+ `base_controller_class` (the proxy controller inherits from it, so your
153
+ `before_action :require_admin` runs first). Past the gate, the visitor is
154
+ named to the server by `Railwatch::Sidecar.config.user`:
155
+
156
+ ```ruby
157
+ Railwatch::Sidecar.configure do |c|
158
+ c.user = ->(controller) { u = controller.current_user; { email: u.email, name: u.name, role: u.admin? ? "admin" : "member" } }
159
+ end
160
+ ```
161
+
162
+ Without it, the HTTP Basic user name (or "Operator") is an admin. Users are
163
+ created in the server on first sight. Nobody signs in to the server itself.
164
+
165
+ ## AI agents
166
+
167
+ The server's MCP endpoint and JSON API are under the mount too
168
+ (`/railwatch/mcp`, `/railwatch/api/v1`). The bundled railwatch-rs takes the
169
+ proxy's signed user on them, so an agent on the same machine needs no token
170
+ in development:
171
+
172
+ ```sh
173
+ claude mcp add --transport http railwatch http://localhost:3000/railwatch/mcp
174
+ ```
175
+
176
+ (or `bin/rails generate railwatch:agents` from the railwatch gem, which
177
+ writes `.mcp.json` and the instructions). The proxy signs those requests
178
+ for the visitor exactly as it signs dashboard pages, and the server takes
179
+ the signed header as that user on MCP and the API: every read tool, and the
180
+ issue writes (`update_issue`, `add_comment`), attributed to the user and
181
+ "agent".
182
+
183
+ **What that opens, and to whom.** By default (`agent_access` unset) only
184
+ when all of these hold:
185
+
186
+ - `Rails.env.development?` (the server's environment, `config.environment`);
187
+ - the request comes from this machine (`request.local?`: `127.0.0.1` or
188
+ `::1`). A dev server bound to `0.0.0.0` still refuses the LAN;
189
+ - its `Host` is `localhost`, `127.0.0.1` or `::1`. A web page whose own name
190
+ was rebound to `127.0.0.1` (DNS rebinding) arrives under that name, not
191
+ these, and gets no signature;
192
+ - the dashboard's gate passes, as for any page (open in development unless
193
+ you configured HTTP Basic or a base controller).
194
+
195
+ The server also refuses a signed MCP or API request the browser marks as
196
+ coming from another site (`Origin` that is not the `Host`, or
197
+ `Sec-Fetch-Site: cross-site`), so a page you visit can't call them through
198
+ your browser. What is left: anything that can send a request to
199
+ `localhost:3000` from your machine (another local user, a local process)
200
+ can read the development app's telemetry and comment on its issues. That is
201
+ already true of the dashboard at `/railwatch` in development, and of the
202
+ dev app itself.
203
+
204
+ Everywhere else agent requests are forwarded without the visitor and the
205
+ server answers 401 until the agent sends a token (`Authorization: Bearer
206
+ rwp_...` from Settings -> Profile), which the proxy passes through
207
+ unchanged; a token always wins over the signature, in development too.
208
+ `config.agent_access = true` (or `RAILWATCH_SIDECAR_AGENT_ACCESS=1`) signs
209
+ agent requests for whoever passes the gate in any environment: only behind
210
+ a gate that authenticates people (`base_controller_class` with your login),
211
+ since the agent then acts as them. `false` turns it off in development too.
212
+
213
+ ## How it works
214
+
215
+ ```
216
+ browser ── /railwatch/* ──> Rails (gate) ──> ProxyController ── 127.0.0.1:28790 ──> railwatch serve
217
+ + X-Railwatch-Proxy-Auth (signed, 60 s) (RAILWATCH_BASE_PATH=/railwatch)
218
+ Puma workers, jobs ── railwatch agent ── /ingest ─────────────────────────────────────┘
219
+ ```
220
+
221
+ - **Process.** `plugin :railwatch_sidecar` spawns `railwatch serve` from
222
+ Puma's master after boot, in its own process group: a Ctrl-C reaches
223
+ Puma, Puma drains, and only then is the server stopped. A supervisor
224
+ thread checks every second and restarts a dead server with backoff
225
+ (1, 2, 4, 8, 15, 30 s), logging the death and the last lines of
226
+ `storage/railwatch/server.log`. Each check also asks the server's `/up`
227
+ (2 s timeout): one that is alive but stops answering (stopped, wedged)
228
+ is logged once as NOT ANSWERING, and after 8 s more of silence is
229
+ KILLed and restarted the same way (measured: a SIGSTOPped server was
230
+ replaced and answering in 11 s). At boot Puma waits at most 5 s for a
231
+ new server's `/up`, then boots without it and says so; a server that
232
+ hasn't answered once within 60 s of starting is killed and restarted. On stop: the railwatch agent's samplers
233
+ stop, the reporter flushes into the still-running server, then the
234
+ server gets SIGTERM (finishes in-flight requests, commits queued writes)
235
+ and KILL past the deadline. `server.pid` records "server parent"; a
236
+ server orphaned by a SIGKILLed Puma is stopped by the next start.
237
+ - **Proxy.** `/railwatch/*` is a prepended route to
238
+ `Railwatch::Sidecar::ProxyController`. It forwards the server's cookies
239
+ and Inertia headers, sets `Host`/`X-Forwarded-Proto`, sets
240
+ `X-Forwarded-For` to the visitor's `request.remote_ip` (replacing any the
241
+ browser sent; the server runs with `RAILWATCH_TRUST_PROXY`, so its per-IP
242
+ sign-in and MCP limits count visitors, not the app), adds the signed
243
+ header, and hands the request off: **no Puma thread waits for the
244
+ server.** Past the gate the controller connects to the server and leaves
245
+ the request to `ProxyHandoff`, a middleware outside every other one,
246
+ which takes the browser's socket with Rack hijack and gives both
247
+ sockets to the process's one `Relay` thread (the live channel's, below).
248
+ The Relay writes the request (Puma has already read the body; a large
249
+ one comes from Puma's tempfile) and passes the response back as it
250
+ arrives, byte for byte except the head: `Connection: close` (the browser,
251
+ or the proxy in front of the app, opens a new connection for the next
252
+ request) and the app's own `Set-Cookie` (a rolling session) added. A
253
+ response the server hasn't started within the first-byte timeout is a
254
+ 504 and one it drops before answering a 502, both written by the Relay;
255
+ one that goes quiet mid-body is cut off after the idle timeout.
256
+ Measured (`test/e2e/slow_dashboard.rb`, Puma 1 worker x 3 threads, pages
257
+ taking 3 s): before, 6 slow loads made the app's `/up` wait 5.7 s;
258
+ after, 4 ms. With 30 slow loads at once the worker kept its 13 threads,
259
+ RSS rose 131 -> 137 MB and `/up` stayed under 12 ms. Encodings pass
260
+ through untouched. Without Rack hijack, and under Puma's own TLS (`ssl_bind`;
261
+ its socket wrapper holds decrypted bytes `IO.select` can't see), the
262
+ response is streamed from the Puma thread instead, as before: headers
263
+ as soon as the server sends them, the body as Puma writes it. Behind
264
+ Thruster, kamal-proxy or nginx the app sees plain HTTP/1.1 and the
265
+ handoff applies; HTTP/2 ends at that proxy. It doesn't rewrite paths:
266
+ the server itself serves under `/railwatch` (below). CSRF is the
267
+ server's own (Origin check plus its token), so Rails' check is off for
268
+ this controller, and so is inertia_rails' (its `XSRF-TOKEN` at
269
+ `Path=/` shadowed the server's, refusing every write). inertia_rails'
270
+ middleware would also answer every dashboard click with a 409 full
271
+ reload, since the dashboard's Inertia version is the server's, not the
272
+ app's; the proxy removes `X-Inertia` from the request once it has been
273
+ forwarded, so the middleware leaves the response alone.
274
+ Commenting, resolving, saving a view and changing a setting are tested
275
+ through the proxy (`test/e2e/browser_writes.rb`, and by hand in Chromium).
276
+ - **Live channel.** `/railwatch/cable` is a Rack hijack too; the browser's
277
+ and the server's sockets are spliced by the same `IO.select` thread per
278
+ app process (`Relay`), not two threads per socket: 30 open dashboards
279
+ cost one thread. At most `max_live_connections` per process, counted
280
+ apart from dashboard requests; past it the handshake gets 503 and the
281
+ dashboard falls back to reloading.
282
+ - **Trust.** `storage/railwatch/proxy_secret` (0600, generated) is the
283
+ HMAC key. The header is `v1.<claims>.<HMAC-SHA256>` with
284
+ `email, name, role, exp`; the server refuses a bad signature, an expired
285
+ one, one valid for more than 5 minutes, and any role but admin/member
286
+ (401, logged). It never falls back to "signed out". The server binds
287
+ loopback only, so nothing outside the host can reach it unproxied.
288
+
289
+ ## The server side (Rebulk/railwatch-rs)
290
+
291
+ The server features this gem relies on are upstream, in Rebulk/railwatch-rs
292
+ `main` from `ef87f72` (PR #16), and the dashboard's mount handling is part of
293
+ the server's frontend. Every setting is off by default, so a self-hosted
294
+ install is unchanged. Documented in railwatch-rs `docs/SELF-HOST.md`,
295
+ "Running behind an app".
296
+
297
+ - **`RAILWATCH_BASE_PATH`**: pages are served under a prefix (`/railwatch`)
298
+ while `/ingest` and `/up` stay at the root, so the agent's URL doesn't
299
+ change. Redirects, cookie paths, asset/icon/entry URLs, emails, MCP
300
+ discovery and the Google callback all carry it.
301
+ - **`RAILWATCH_TRUSTED_PROXY_SECRET_FILE`**: the server accepts an
302
+ HMAC-signed `X-Railwatch-Proxy-Auth` header (email, name, admin/member
303
+ role, `exp` at most 300 s ahead) from this gem's proxy as the signed-in
304
+ user. The header is stripped from every request before any route sees it,
305
+ even with the mode off; when on, an invalid one is a 401, never a
306
+ fall-back. The proxy can't create or demote an owner.
307
+ - **`RAILWATCH_STANDBY_RELAY`**: the server holding `serve.lock` also
308
+ listens on `<data_dir>/serve.sock` (0600, peer uid checked); a second
309
+ `serve` on the same data dir binds its own loopback port at once and
310
+ relays to that socket until it gets the lock. This is what makes a
311
+ two-container deploy overlap lose nothing (see "Deploys" below).
312
+
313
+ **Why a base path, not rewriting in the proxy:** URLs are made in four
314
+ places (Rust redirects, Inertia's 409 `X-Inertia-Location`, js-routes in
315
+ the bundle, CSS/JS asset URLs) plus cookies and the cable URL. Rewriting
316
+ means parsing HTML, JSON page props and JS chunks, and it breaks the first
317
+ time the server adds a URL the proxy doesn't know about. With a base path
318
+ the server says the right URL once and the proxy stays byte-transparent.
319
+
320
+ ## Building
321
+
322
+ ```sh
323
+ script/build-server /path/to/railwatch-rs # railwatch-rs at or after ef87f72, frontend/dist built (bin/build, SKIP_CARGO=1)
324
+ rake gem:all # pkg/: ruby + 4 platform gems (~10 MB each)
325
+ ```
326
+
327
+ Builds use the pinned `cargo-zigbuild` image from `bin/release`: static musl
328
+ for Linux, and that image's MacOSX 11.3 SDK for both darwin targets. All
329
+ four built here (2-3 min each, 4 jobs).
330
+
331
+ **macOS: built natively, signed ad hoc, run in CI.** `.github/workflows/macos.yml`
332
+ builds the frontend once on Linux, then on `macos-14` (arm64) and
333
+ `macos-15-intel` (x86_64; GitHub retired the macos-13 runners) builds the
334
+ server from the pinned railwatch-rs commit (`MACOSX_DEPLOYMENT_TARGET=11.0`, checked
335
+ with `vtool`: minos 11.0), signs it `codesign -s -`, verifies the
336
+ signature and the arch, and runs it: `--version`, `setup` in a temp dir,
337
+ `serve`, `/up` and `/ingest/ping`, then a second `serve` on the same data
338
+ dir that comes up as a standby and takes over when the first gets TERM.
339
+ Then it builds the darwin platform gem, installs it, checks the installed
340
+ binary's signature and runs it. Artifacts: the binary and the `.gem` per
341
+ arch. The zigbuild cross-build in `script/build-server` still works for
342
+ Linux; for darwin, use the workflow's artifacts (they are signed).
343
+
344
+ ## Deploys and two processes
345
+
346
+ - **Same host, same data dir** (two Pumas, or `phased-restart` overlap):
347
+ the second app process finds the port already answered, starts its own
348
+ server anyway (it waits on `serve.lock`) and logs "standby". Its agent
349
+ reports to the first server, which owns the data. When the first app
350
+ process stops, its server drains and exits, the standby takes the lock
351
+ and the port (the log says "took over"). Measured: 10 + 10 requests
352
+ during the overlap and 10 after, 30 of 30 stored; one server writes at
353
+ any time.
354
+ - **Two containers sharing a volume** (Kamal blue/green): each container
355
+ has its own loopback, so the new container's agent can't reach the old
356
+ container's server. The new container's server (`RAILWATCH_STANDBY_RELAY`,
357
+ railwatch-rs `RAILWATCH_STANDBY_RELAY`) binds its own `127.0.0.1:28790` at once and splices
358
+ every connection, byte for byte, to `storage/railwatch/serve.sock`, where
359
+ the server holding `serve.lock` also listens. A Unix socket on the shared
360
+ volume crosses network namespaces (same kernel, same inode). When the old
361
+ container stops, its server drains and exits, the standby takes the lock
362
+ and serves the same listener itself; connections in that gap retry the
363
+ socket for up to 30 s. The log says "as a standby" then "took over".
364
+
365
+ Why this mattered (measured, `test/overlap/`, 3 runs before the fix): the
366
+ new container's first delivery was refused, the agent retained it and
367
+ retried on its ladder, and every retained retry doubled its backpressure
368
+ factor (1, 2, 4, 8). The sampler divides the request sample rate by that
369
+ factor, so for the ~12 s until the old server exited the new container
370
+ recorded 1 in 2, 1 in 4, then 1 in 8 of its requests. The dropped ones were
371
+ never recorded at all; nothing that reached the agent's buffer was lost.
372
+ 186, 184 and 197 of ~480 requests went unrecorded, every one at a factor
373
+ of 2, 4 or 8, none at 1. With `RAILWATCH_BACKPRESSURE=false` the same
374
+ overlap stored 476/476: the batches themselves all arrived on retry.
375
+ Starting the server before Puma accepts requests (under `bin/rails s`)
376
+ also removes the one refused delivery at boot that cost 5-9 requests.
377
+
378
+ | handover, ~800 requests | served | stored once | lost |
379
+ |---|---|---|---|
380
+ | before, `docker stop` | 795 | 602 | 193 (new container: 186) |
381
+ | before, SIGKILL | 793 | 582 | 211 |
382
+ | after, `docker stop` | 795 | 795 | 0 |
383
+ | after, SIGKILL of the old one | 789 | 771 | 18, all the old container's last second |
384
+ | after, fresh `rails new` app (Rails 8.1, Solid Queue), `docker stop` | 865 | 865 | 0 |
385
+ | after, fresh `rails new` app, SIGKILL of the old one | 845 | 828 | 17, the old container's last second |
386
+
387
+ A SIGKILLed app loses what its agent had not flushed yet (up to
388
+ `flush_interval`, here 1 s): the records were in that process's memory.
389
+ The new container lost nothing in either run, and no record was stored
390
+ twice. Volumes on two different hosts are not supported (flock and Unix
391
+ sockets don't cross hosts; each host would need its own data dir, which
392
+ splits the history).
393
+
394
+ ## Cost (same app, same 1000 requests, single-mode Puma)
395
+
396
+ | | idle RSS | RSS after 1000 req | CPU for 1000 req |
397
+ |---|---|---|---|
398
+ | this gem: `railwatch serve` | 12 MB | 51 MB | 0.15 s |
399
+ | railwatch gem embedded: writer process | 87 MB | 138 MB | 0.08 s |
400
+ | Puma itself (either way) | 126 MB | 162-166 MB | 3.2-3.8 s |
401
+
402
+ The Rust server is a third of the Ruby writer's memory. The Ruby writer
403
+ spends less CPU only because it writes rows and does no grouping,
404
+ rollups, alerts or the dashboard. Rendering the dashboard costs the
405
+ sidecar, not Puma (Puma only proxies), whereas the embedded dashboard
406
+ renders inside Puma's workers.
407
+
408
+ ## Limitations and TODOs
409
+
410
+ - Dashboard connections close after each response (no keep-alive
411
+ between the browser and the proxy for `/railwatch`). On loopback or
412
+ behind a reverse proxy that costs nothing measurable; a browser talking
413
+ straight to Puma over the internet opens a new TCP connection per
414
+ dashboard request.
415
+ - The cap is per process and checked before connecting, so a burst can
416
+ pass it by as many requests as Puma has threads.
417
+ - macOS binaries are ad-hoc signed only: fine for a gem installed with
418
+ Bundler (RubyGems sets no quarantine flag). A notarized release needs an
419
+ Apple Developer ID certificate in CI (`codesign -s "Developer ID
420
+ Application: ..." --options runtime --timestamp`, then `notarytool
421
+ submit --wait`); a bare executable can't be stapled, so Gatekeeper checks
422
+ it online the first time it runs from a quarantined download.
423
+ - One environment per data dir (above).
424
+ - The account owner is a placeholder user with a random password nobody
425
+ knows; the proxied users are admins/members. Transferring ownership
426
+ works in Settings > Members.
427
+ - Releases must keep building the platform gems from a published
428
+ railwatch-rs commit: `RAILWATCH_RS_SHA` in
429
+ `.github/workflows/macos.yml`, and the same commit for the Linux builds.
430
+
431
+ ## License
432
+
433
+ FSL-1.1-ALv2, the Functional Source License with an Apache 2.0 future
434
+ license (LICENSE). Use it, change it and run it for your own apps freely;
435
+ what it does not allow is a Competing Use: making Railwatch available to
436
+ others in a commercial product or service that substitutes for Railwatch,
437
+ offers substantially similar functionality, or substitutes for another
438
+ product or service Rebulk offers using Railwatch (one that existed when that
439
+ version was released). LICENSE has the exact terms. Each version
440
+ released under FSL also becomes available under Apache-2.0 two years after
441
+ its release. Releases up to and including 0.1.6 were published under
442
+ AGPL-3.0-only and stay available under it alone; the Apache-2.0 grant does
443
+ not apply to them.
444
+
445
+ The platform gems contain the Railwatch server binary; NOTICE explains what
446
+ that means for your app (nothing: it runs as a separate program) and where
447
+ its source is.