rails_mind 0.4.0 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +9 -0
- data/README.md +1 -1
- data/docs/compatibility.md +5 -4
- data/docs/sdk.md +15 -7
- data/lib/rails_mind/page_view.rb +109 -10
- data/lib/rails_mind/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 372c9336e641bc2e69af9ed38a5771fbc340bd67d4f416eff8faaa90258f2f8b
|
|
4
|
+
data.tar.gz: 16e987e172416a1cd6a18937da87f941c95355db87922e1a0c475c3af2130e0e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6af33efbf21560279d31b2fbf0c8729499afedfe5514658cef6a5d7b6edcb9922fe21067b49b9a14b4965c5adba32f4b4d5d3183840281e42986aadde3859f6d
|
|
7
|
+
data.tar.gz: 7bf58df1349bcb4740eaf44381e7878336ccf89ee9018ff1df573a157980f1bf116b3aa648fa1134486fd049f9e97e92d2bedb0c11785d5cc42db16f2e87d741
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.1 — 2026-11-06
|
|
4
|
+
|
|
5
|
+
Fixes for page views. Upgrade from 0.4.0 before deploying it.
|
|
6
|
+
|
|
7
|
+
- A route segment your `filter_parameters` masks is masked in the page view's `path`, as the query already was: `/passwords/:token/edit` (the Rails authentication generator's reset page) sends `/passwords/[FILTERED]/edit`, not the reset token. 0.4.0 sent the token. The whole segment goes, with any format suffix or text around the value; so do a glob value spanning segments, a long value repeated in the query, a back link to the masked page in the query, a campaign tag holding the value, and a masked mount segment in `route`. Filters receive string keys, as in Rails' own logs, and if masking fails the path is sent as `[FILTERED]`.
|
|
8
|
+
- A referrer inside the app sends only its origin, since its path can hold what the app masks in its own paths. Other sites' referrers keep their path.
|
|
9
|
+
- Only a GET whose `Accept` header asks for `text/html` counts. Pollers and scripts that accept anything (`*/*`), such as GoodJob's live dashboard and bare HTTP clients, no longer count as page views.
|
|
10
|
+
- The `Turbo::Native::` stub pages Hotwire Native apps load to recede, resume or refresh are not counted.
|
|
11
|
+
|
|
3
12
|
## 0.4.0 — 2026-11-06
|
|
4
13
|
|
|
5
14
|
Breaking: the Ahoy integration and the Ahoy-based browser adapter are gone. RailsMind counts page views itself. An app that keeps Ahoy for its own reports keeps it unchanged.
|
data/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
The MIT-licensed Ruby client for [RailsMind](https://railsmind.com). One bounded
|
|
4
4
|
collector for Rails page views, requests, errors, jobs, mailer attempts, Flipper
|
|
5
|
-
observations, and selected OpenTelemetry spans. Version 0.4.
|
|
5
|
+
observations, and selected OpenTelemetry spans. Version 0.4.1 is an early release; see
|
|
6
6
|
[compatibility and validation limits](docs/compatibility.md).
|
|
7
7
|
|
|
8
8
|
```ruby
|
data/docs/compatibility.md
CHANGED
|
@@ -112,17 +112,18 @@ limits, and real process fork resets. Fake transports verify the wire contract;
|
|
|
112
112
|
deployed server policy behavior, cross-process traffic forecasts, and a
|
|
113
113
|
customer app's production overhead require hosted/staging validation.
|
|
114
114
|
|
|
115
|
-
## 0.4.0 page view verification
|
|
115
|
+
## 0.4.0 and 0.4.1 page view verification
|
|
116
116
|
|
|
117
117
|
The Ruby suite passes locally with Ruby 4.0.9 and the locked Rails 8.1.4 and
|
|
118
|
-
Rack 3.2.7 libraries:
|
|
118
|
+
Rack 3.2.7 libraries: 167 tests with no failures or skips at 0.4.1. The packaged-gem
|
|
119
119
|
smoke check also runs the page-view, visitor and transport tests against the
|
|
120
120
|
built gem. Page views are tested by instrumenting `process_action.action_controller`
|
|
121
121
|
with real `ActionDispatch::Request` and `ActionDispatch::Response` objects inside
|
|
122
122
|
the SDK's request boundary: every counting rule, the properties and their caps,
|
|
123
123
|
the redactor keeping every property key, the visitor cookie (set, refreshed,
|
|
124
|
-
replaced, never on public responses), the daily hash, identity linking
|
|
125
|
-
time-zone header.
|
|
124
|
+
replaced, never on public responses), the daily hash, identity linking, the
|
|
125
|
+
time-zone header and, from 0.4.1, masked route segments, same-site referrers
|
|
126
|
+
cut to their origin and the `Accept: text/html` rule.
|
|
126
127
|
|
|
127
128
|
No Rails application is booted. Routing, `ActionDispatch::RemoteIp`, the cookie
|
|
128
129
|
middleware and a CDN in front of a real app, Rails 7.2, and Rack 2.2 are not
|
data/docs/sdk.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
The gem is `rails_mind`, namespace `RailsMind`. Install the MIT-licensed client
|
|
4
4
|
from RubyGems.org and commit the application lockfile to retain the selected
|
|
5
5
|
version. The hosted RailsMind service is a separate, proprietary application.
|
|
6
|
-
Version 0.4.
|
|
6
|
+
Version 0.4.1 is an early release with the [documented compatibility limits](compatibility.md).
|
|
7
7
|
|
|
8
8
|
## Install
|
|
9
9
|
|
|
@@ -147,13 +147,17 @@ ActiveJob serializes only the context identifier allowlist under `rails_mind_con
|
|
|
147
147
|
## Page views
|
|
148
148
|
|
|
149
149
|
With `analytics` and `page_views` on (the default), the SDK counts a page view
|
|
150
|
-
for each HTML page a controller renders for a person: a GET
|
|
151
|
-
`text/html` response.
|
|
150
|
+
for each HTML page a controller renders for a person: a GET whose `Accept`
|
|
151
|
+
header asks for `text/html`, answered with a 2xx `text/html` response. Browsers
|
|
152
|
+
and Turbo always ask for HTML when they load a page; pollers, scripts and bare
|
|
153
|
+
HTTP clients that accept anything (`*/*`) are not counted. These are not counted
|
|
154
|
+
either:
|
|
152
155
|
|
|
153
156
|
- Turbo Frame requests, prefetches (`Sec-Purpose` or `X-Sec-Purpose: prefetch`)
|
|
154
157
|
and XHRs;
|
|
155
|
-
- `Rails::` controllers (the `/up` health check included)
|
|
156
|
-
controllers
|
|
158
|
+
- `Rails::` controllers (the `/up` health check included), `ActiveStorage::`
|
|
159
|
+
controllers and the `Turbo::Native::` stub pages Hotwire Native apps load to
|
|
160
|
+
recede, resume or refresh;
|
|
157
161
|
- a blank or known-bot user agent (crawlers, link previews, LLM fetchers, uptime
|
|
158
162
|
monitors and bare HTTP clients);
|
|
159
163
|
- anything your `ignore_page_view` hook returns true for. It receives the
|
|
@@ -169,9 +173,9 @@ request IDs of the request event for the same action. Its properties:
|
|
|
169
173
|
| Property | What it holds |
|
|
170
174
|
| --- | --- |
|
|
171
175
|
| `route` | The route pattern, such as `/posts/:id`, without `(.:format)`; a mounted engine's starts with its mount path |
|
|
172
|
-
| `path` | The path and query string, with the app's `filter_parameters` applied (at most 1,000 characters) |
|
|
176
|
+
| `path` | The path and query string, with the app's `filter_parameters` applied to the query and to route segments, so `/passwords/:token/edit` sends `/passwords/[FILTERED]/edit` (at most 1,000 characters) |
|
|
173
177
|
| `host` | The request's host |
|
|
174
|
-
| `referrer` |
|
|
178
|
+
| `referrer` | Another site's scheme, host and path; its query and fragment are not sent. A link inside the app sends only its origin |
|
|
175
179
|
| `utm_source`, `utm_medium`, `utm_campaign`, `utm_term`, `utm_content`, `ref` | Those query parameters, at most 200 characters each; one your `filter_parameters` masks is not sent |
|
|
176
180
|
| `user_agent` | The user agent, at most 500 characters; RailsMind derives device, browser and OS from it |
|
|
177
181
|
| `country` | Two letters from `CF-IPCountry` or `CloudFront-Viewer-Country`, when present |
|
|
@@ -228,6 +232,10 @@ RailsMind counts days in that zone.
|
|
|
228
232
|
|
|
229
233
|
### Limits
|
|
230
234
|
|
|
235
|
+
- A referrer counts as inside the app when its host matches the request's host
|
|
236
|
+
(ignoring a leading `www.`). Behind a proxy that rewrites `Host`, forward the
|
|
237
|
+
original host (`X-Forwarded-Host`) so Rails sees it; otherwise an internal
|
|
238
|
+
referrer is sent with its path.
|
|
231
239
|
- Turbo broadcast page refreshes (`broadcasts_refreshes`) reload a page the way
|
|
232
240
|
a visit does, so each refresh counts as a page view in every open tab.
|
|
233
241
|
- Turbo 8 prefetches a link on hover (`X-Sec-Purpose: prefetch`) and, when the
|
data/lib/rails_mind/page_view.rb
CHANGED
|
@@ -22,7 +22,13 @@ module RailsMind
|
|
|
22
22
|
NO_COUNTRY = %w[XX T1].freeze
|
|
23
23
|
PREFETCH_HEADERS = %w[HTTP_SEC_PURPOSE HTTP_X_SEC_PURPOSE].freeze
|
|
24
24
|
NO_CONTENT = [ 204, 205 ].freeze
|
|
25
|
-
|
|
25
|
+
# Rails' own pages, Active Storage, and the stub pages turbo-rails serves a
|
|
26
|
+
# Hotwire Native app to recede, resume or refresh.
|
|
27
|
+
FRAMEWORK_CONTROLLER = /\A(?:Rails|ActiveStorage|Turbo::Native)::/
|
|
28
|
+
FILTERED = "[FILTERED]".freeze
|
|
29
|
+
# A masked value at least this long is also masked wherever it appears in
|
|
30
|
+
# the query string; a shorter one would mask unrelated values there.
|
|
31
|
+
QUERY_MASK_MIN = 8
|
|
26
32
|
# Crawlers, link previews, LLM fetchers, monitors and bare HTTP clients. A
|
|
27
33
|
# blank user agent is not counted either. "cloudfront" is what CloudFront
|
|
28
34
|
# sends when a distribution does not forward the caller's user agent.
|
|
@@ -64,6 +70,9 @@ module RailsMind
|
|
|
64
70
|
return false unless (payload[:method] || request.request_method) == "GET"
|
|
65
71
|
return false unless (200..299).cover?(payload[:status].to_i)
|
|
66
72
|
return false unless html?(payload, response)
|
|
73
|
+
# A browser asks for HTML when it loads a page; scripts, pollers and bare
|
|
74
|
+
# HTTP clients that take whatever comes back are not people reading it.
|
|
75
|
+
return false unless request.get_header("HTTP_ACCEPT").to_s.include?("text/html")
|
|
67
76
|
return false unless request.get_header("HTTP_TURBO_FRAME").to_s.empty?
|
|
68
77
|
return false if PREFETCH_HEADERS.any? { |header| request.get_header(header).to_s.downcase.include?("prefetch") }
|
|
69
78
|
return false if request.xhr?
|
|
@@ -117,11 +126,11 @@ module RailsMind
|
|
|
117
126
|
values = {}
|
|
118
127
|
route = route(request)
|
|
119
128
|
values["route"] = route if route
|
|
120
|
-
values["path"] = request
|
|
129
|
+
values["path"] = path(request)[0, PATH_LIMIT]
|
|
121
130
|
values["host"] = request.host.to_s.downcase[0, HOST_LIMIT]
|
|
122
|
-
referrer = referrer(request.referer)
|
|
131
|
+
referrer = referrer(request.referer, request.host)
|
|
123
132
|
values["referrer"] = referrer if referrer
|
|
124
|
-
values.merge!(tags(request))
|
|
133
|
+
values.merge!(tags(request, secret_values(request)))
|
|
125
134
|
values["user_agent"] = request.user_agent.to_s[0, USER_AGENT_LIMIT]
|
|
126
135
|
country = country(request)
|
|
127
136
|
values["country"] = country if country
|
|
@@ -134,29 +143,119 @@ module RailsMind
|
|
|
134
143
|
return unless request.respond_to?(:route_uri_pattern)
|
|
135
144
|
pattern = request.route_uri_pattern.to_s.delete_suffix("(.:format)")
|
|
136
145
|
return if pattern.empty?
|
|
137
|
-
|
|
146
|
+
script = request.script_name.to_s
|
|
147
|
+
secrets = filtered_path_values(request)
|
|
148
|
+
script = mask(script, secrets) unless script.empty? || secrets.empty?
|
|
149
|
+
pattern = "#{script}#{pattern}".chomp("/")
|
|
138
150
|
(pattern.empty? ? "/" : pattern)[0, ROUTE_LIMIT]
|
|
139
151
|
rescue StandardError
|
|
140
152
|
nil
|
|
141
153
|
end
|
|
142
154
|
|
|
143
|
-
#
|
|
144
|
-
|
|
155
|
+
# The query string goes through the app's filter_parameters, and so do
|
|
156
|
+
# the route's own segments: a token in /passwords/:token/edit is masked
|
|
157
|
+
# like one in ?token=.
|
|
158
|
+
# If masking fails, the path is not sent at all: never the unmasked one.
|
|
159
|
+
def path(request)
|
|
160
|
+
path = request.filtered_path.to_s
|
|
161
|
+
secrets = filtered_path_values(request)
|
|
162
|
+
secrets.empty? ? path : mask(path, secrets)
|
|
163
|
+
rescue StandardError
|
|
164
|
+
FILTERED
|
|
165
|
+
end
|
|
166
|
+
|
|
167
|
+
# Each path parameter the app's filter masks, as written and as escaped,
|
|
168
|
+
# compared as bytes so a path in any encoding can be searched. Keys go to
|
|
169
|
+
# the filter as strings, as Rails' own filtered_parameters gives them.
|
|
170
|
+
def filtered_path_values(request)
|
|
171
|
+
params = request.path_parameters.except(:controller, :action, :format).transform_keys(&:to_s)
|
|
172
|
+
return [] if params.empty?
|
|
173
|
+
filtered = request.parameter_filter.filter(params)
|
|
174
|
+
params.each_with_object([]) do |(key, value), secrets|
|
|
175
|
+
next unless value.is_a?(String) && !value.empty? && filtered[key] != value
|
|
176
|
+
secrets.push(value, Rack::Utils.escape_path(value), Rack::Utils.escape(value).gsub("+", "%20"))
|
|
177
|
+
end.map(&:b).uniq
|
|
178
|
+
end
|
|
179
|
+
|
|
180
|
+
# Every route segment holding a secret goes whole (with any .format or
|
|
181
|
+
# text around it), a secret spanning segments (a glob route) goes, and a
|
|
182
|
+
# long secret repeated in the query goes too. Masking too much costs a
|
|
183
|
+
# little detail; masking too little sends a credential.
|
|
184
|
+
def mask(text, secrets)
|
|
185
|
+
return text if text.empty?
|
|
186
|
+
route_part, query = text.b.split("?", 2)
|
|
187
|
+
filtered = FILTERED.b
|
|
188
|
+
globs = secrets.select { |secret| secret.include?("/".b) }
|
|
189
|
+
globs.each { |secret| route_part = route_part.gsub(secret, filtered) }
|
|
190
|
+
pieces = (secrets + globs.flat_map { |secret| secret.split("/".b) }).reject(&:empty?).uniq
|
|
191
|
+
masked = route_part.split("/", -1).map { |segment| secret_in?(segment, pieces) ? filtered : segment }.join("/")
|
|
192
|
+
masked = "#{masked}?#{mask_query(query, secrets)}".b if query
|
|
193
|
+
masked.force_encoding(Encoding::UTF_8).scrub
|
|
194
|
+
end
|
|
195
|
+
|
|
196
|
+
# A long secret goes wherever it appears; a query value that is, or holds
|
|
197
|
+
# as a path piece, any secret goes whole (a back link to the masked page).
|
|
198
|
+
def mask_query(query, secrets)
|
|
199
|
+
filtered = FILTERED.b
|
|
200
|
+
secrets.each { |secret| query = query.gsub(secret, filtered) if secret.bytesize >= QUERY_MASK_MIN }
|
|
201
|
+
query.split("&".b, -1).map do |pair|
|
|
202
|
+
key, value = pair.split("=".b, 2)
|
|
203
|
+
next pair unless value
|
|
204
|
+
decoded = begin
|
|
205
|
+
Rack::Utils.unescape(value).b
|
|
206
|
+
rescue StandardError
|
|
207
|
+
value
|
|
208
|
+
end
|
|
209
|
+
pieces = decoded.split("/".b)
|
|
210
|
+
secrets.any? { |secret| decoded == secret || pieces.include?(secret) } ? "#{key}=#{filtered}".b : pair
|
|
211
|
+
end.join("&".b)
|
|
212
|
+
end
|
|
213
|
+
|
|
214
|
+
# The route's masked values, for the path and the campaign tags.
|
|
215
|
+
def secret_values(request)
|
|
216
|
+
filtered_path_values(request)
|
|
217
|
+
rescue StandardError
|
|
218
|
+
[]
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
def secret_in?(segment, secrets)
|
|
222
|
+
decoded = begin
|
|
223
|
+
Rack::Utils.unescape_path(segment).b
|
|
224
|
+
rescue StandardError
|
|
225
|
+
segment
|
|
226
|
+
end
|
|
227
|
+
secrets.any? { |secret| segment.include?(secret) || decoded.include?(secret) }
|
|
228
|
+
end
|
|
229
|
+
|
|
230
|
+
# Another site's scheme, host and path; its query and fragment stay home.
|
|
231
|
+
# A link inside the app sends only its origin, since its path can hold
|
|
232
|
+
# what the app masks in its own paths (a reset token, for one).
|
|
233
|
+
def referrer(value, host = nil)
|
|
145
234
|
return unless value.is_a?(String) && !value.strip.empty?
|
|
146
235
|
uri = URI.parse(value.strip)
|
|
147
236
|
return unless uri.is_a?(URI::HTTP) && !uri.host.to_s.empty?
|
|
148
|
-
"#{uri.scheme.downcase}://#{uri.host.downcase}
|
|
237
|
+
origin = "#{uri.scheme.downcase}://#{uri.host.downcase}"
|
|
238
|
+
return origin if same_site?(uri.host, host)
|
|
239
|
+
"#{origin}#{uri.path}"[0, REFERRER_LIMIT]
|
|
149
240
|
rescue URI::Error
|
|
150
241
|
nil
|
|
151
242
|
end
|
|
152
243
|
|
|
153
|
-
|
|
154
|
-
|
|
244
|
+
def same_site?(referrer_host, host)
|
|
245
|
+
return false if host.to_s.empty?
|
|
246
|
+
bare = ->(name) { name.to_s.downcase.delete_prefix("www.") }
|
|
247
|
+
bare.call(referrer_host) == bare.call(host)
|
|
248
|
+
end
|
|
249
|
+
|
|
250
|
+
# A tag the app's filter_parameters would mask in the query is not sent,
|
|
251
|
+
# nor one holding a value it masks in the route.
|
|
252
|
+
def tags(request, secrets = [])
|
|
155
253
|
query = request.GET
|
|
156
254
|
raw = TAGS.each_with_object({}) { |key, values| values[key] = query[key] if query[key].is_a?(String) }
|
|
157
255
|
filtered = request.parameter_filter.filter(raw)
|
|
158
256
|
raw.each_with_object({}) do |(key, value), tags|
|
|
159
257
|
next unless filtered[key] == value
|
|
258
|
+
next if secrets.any? { |secret| value.b.include?(secret) }
|
|
160
259
|
value = value.strip
|
|
161
260
|
tags[key] = value[0, TAG_LIMIT] unless value.empty?
|
|
162
261
|
end
|
data/lib/rails_mind/version.rb
CHANGED