rails-profiler 0.30.10 → 0.30.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +72 -0
- data/app/controllers/profiler/api/cluster_controller.rb +37 -1
- data/app/controllers/profiler/api/slave_proxy_controller.rb +4 -2
- data/app/controllers/profiler/application_controller.rb +12 -1
- data/app/controllers/profiler/profiles_controller.rb +2 -0
- data/config/routes.rb +15 -8
- data/lib/profiler/cluster/master_client.rb +24 -6
- data/lib/profiler/cluster/security.rb +179 -0
- data/lib/profiler/cluster/slave_proxy.rb +24 -1
- data/lib/profiler/collectors/base_collector.rb +2 -0
- data/lib/profiler/collectors/console_collector.rb +2 -2
- data/lib/profiler/collectors/flamegraph_collector.rb +2 -2
- data/lib/profiler/collectors/i18n_collector.rb +1 -1
- data/lib/profiler/collectors/mailer_collector.rb +5 -5
- data/lib/profiler/configuration.rb +21 -1
- data/lib/profiler/console_profiler.rb +1 -1
- data/lib/profiler/instrumentation/net_http_instrumentation.rb +2 -1
- data/lib/profiler/job_profiler.rb +2 -2
- data/lib/profiler/mcp/http_guard.rb +73 -0
- data/lib/profiler/mcp/server.rb +4 -1
- data/lib/profiler/models/profile.rb +14 -4
- data/lib/profiler/railtie.rb +5 -1
- data/lib/profiler/redaction.rb +98 -0
- data/lib/profiler/test_runner/run_store.rb +39 -2
- data/lib/profiler/test_runner/runner.rb +9 -3
- data/lib/profiler/version.rb +1 -1
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e7bb3bed351c863507d670d29552cb6751bb023300728a48e9baf9c718a53659
|
|
4
|
+
data.tar.gz: 8af241a6ca0278f3ada6a4d0d88599e93b952f67057ba6b826ea36935976ba67
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 04d1ce9d5e1c0a2527ddb60280aa82e89e04ccc32a202a4f79e09b4a5fa87319b8f6d6c95a93f212a61cb4bc7213b92d15bb914567669a2f079eec502c075822
|
|
7
|
+
data.tar.gz: 91db3673cbcd3debdcf9fc1e02690601c4a70aaf10f4a17a070eed4b7f9359ae6261b5a9256ab43b9a13108aec0fa61677dddc9b735f72bf8103261f7e8b24f5
|
data/CHANGELOG.md
CHANGED
|
@@ -19,6 +19,78 @@ every commit of every tag interval is accounted for one way or the other.
|
|
|
19
19
|
|
|
20
20
|
## [Unreleased]
|
|
21
21
|
|
|
22
|
+
## [0.30.11] - 2026-10-04
|
|
23
|
+
|
|
24
|
+
<!-- stamped -->
|
|
25
|
+
|
|
26
|
+
### Fixed
|
|
27
|
+
|
|
28
|
+
- **Cluster:** A profile page served by the master now finds profiles held by a slave even before
|
|
29
|
+
the slave proxy has been used once.
|
|
30
|
+
|
|
31
|
+
### Security
|
|
32
|
+
|
|
33
|
+
- **MCP:** The HTTP transport at `/_profiler/mcp` is routed only when `mcp_enabled` is true and
|
|
34
|
+
`mcp_transport` is `:http`; it answers `404` otherwise. It used to be routed in every application,
|
|
35
|
+
whatever these options said, and to answer anyone: after the MCP handshake, any client could list
|
|
36
|
+
the tools and call those that write `ENV`, clear profiles or run the test suite. The stdio
|
|
37
|
+
transport (`rake profiler:mcp`) is unchanged.
|
|
38
|
+
- **MCP:** When it is routed, each request to `/_profiler/mcp` passes the profiler's own checks
|
|
39
|
+
before the MCP server sees it, and gets a `403` otherwise: the profiler must be enabled, the
|
|
40
|
+
request must pass `authorization_mode`, and a `POST` must carry `Content-Type: application/json`
|
|
41
|
+
with no foreign `Origin`, so that a page on another site cannot call the tools from your browser.
|
|
42
|
+
- **Cluster:** The cluster endpoints (`register`, `heartbeat`, `slaves` and the slave proxy) are
|
|
43
|
+
routed only on a node with the new `cluster_master = true`; they answer `404` otherwise. Every
|
|
44
|
+
application used to accept slave registrations.
|
|
45
|
+
- **Cluster:** The master and its slaves authenticate each other with the new shared
|
|
46
|
+
`cluster_secret`, sent in an `X-Profiler-Cluster-Secret` header and compared in constant time:
|
|
47
|
+
`register` and `heartbeat` refuse a missing or wrong secret, and everything while no secret is
|
|
48
|
+
configured. On a slave, the master's proxied calls are let in by the secret, so a master on
|
|
49
|
+
another machine no longer needs the slave to admit its address.
|
|
50
|
+
- **Cluster:** The master registers, and sends requests to, only the slave URLs allowed by the new
|
|
51
|
+
`cluster_allowed_slave_urls` (scheme, host and port, compared after normalization, and a path
|
|
52
|
+
prefix); the default refuses all. HTTPS is required, except for loopback addresses, for slave URLs
|
|
53
|
+
and for `master_url` (new `cluster_allow_insecure_http` to lift it), and the master no longer
|
|
54
|
+
returns the body of a redirect. Anyone who could register used to make the master fetch any host
|
|
55
|
+
and port it could reach and read the answer (server-side request forgery).
|
|
56
|
+
- **Cluster:** A path, profile token or MCP argument sent to a slave can no longer leave the slave's
|
|
57
|
+
`/_profiler/api/`: `?`, `#` and `%` are encoded into the path segment, and a `.`, `..` or `/`
|
|
58
|
+
segment is refused before any request, on the slave proxy, the profile pages and the MCP tools.
|
|
59
|
+
An encoded `..` or `?` used to reach other paths of the slave.
|
|
60
|
+
- **Cluster:** A `cluster_secret` shorter than 32 characters, or blank, is ignored as if none were
|
|
61
|
+
configured, with a warning at boot and an explicit error on registration. The master stores slave
|
|
62
|
+
URLs in their normalized form.
|
|
63
|
+
- **Cluster:** The `cluster_secret` is masked by value in everything a profile captures (params,
|
|
64
|
+
bodies, headers in and out, URLs, `ENV`, SQL binds, job and mailer arguments, and the free text of
|
|
65
|
+
every collector: log lines, exception messages, dumps, SQL text, console expressions), and in the
|
|
66
|
+
output of the test runner, whatever name it travels under and even with `redact_sensitive_data`
|
|
67
|
+
off, on top of the masking by name.
|
|
68
|
+
- **Cluster:** The `cluster_secret` is masked before text is shortened (flame graph names, I18n
|
|
69
|
+
values, console expressions and results, mail bodies and assigns, job arguments), so no prefix of
|
|
70
|
+
it is left, and the output of the test runner is masked across the pieces it is read in, so the
|
|
71
|
+
secret no longer comes back whole once they are joined. Only a secret the cluster accepts (32
|
|
72
|
+
characters or more) is masked by value.
|
|
73
|
+
- **Cluster:** The output of the test runner holds back an end that could start the secret until
|
|
74
|
+
the process has finished printing, a killed run included, and holds back nothing else, so
|
|
75
|
+
progress output is shown as it comes. Binary bodies stored in base64, incoming, response and
|
|
76
|
+
outbound, are masked on their raw bytes before they are encoded.
|
|
77
|
+
- **Upgrading:** Nothing to do if you use neither the MCP HTTP endpoint nor the cluster. Otherwise,
|
|
78
|
+
in `config/initializers/profiler.rb`:
|
|
79
|
+
- MCP over HTTP: set `config.mcp_enabled = true` and `config.mcp_transport = :http`. An
|
|
80
|
+
installation that used `/_profiler/mcp` while `mcp_transport` stayed at its default, `:stdio`,
|
|
81
|
+
must now set `:http`. The checks have no MCP-specific switch: `config.authorization_mode =
|
|
82
|
+
:allow_all` lets anybody who can reach the application call every tool, and
|
|
83
|
+
`config.api_forgery_protection = false` lets any website you visit make your browser call them.
|
|
84
|
+
- Cluster master: set `config.cluster_master = true`, `config.cluster_secret` (at least 32
|
|
85
|
+
characters, the same value on every node, from the environment) and
|
|
86
|
+
`config.cluster_allowed_slave_urls`. Slaves: set the same `config.cluster_secret`, and use HTTPS
|
|
87
|
+
URLs unless master and slave share the machine.
|
|
88
|
+
- To go back to the previous behavior: `config.cluster_require_secret = false` (with no
|
|
89
|
+
`cluster_secret`) accepts registrations from any client `authorization_mode` lets in, as in
|
|
90
|
+
0.30.6; `config.cluster_allowed_slave_urls = :any` accepts any slave URL, which reopens the
|
|
91
|
+
server-side request forgery; `config.cluster_allow_insecure_http = true` sends the secret and the
|
|
92
|
+
profiles in clear to remote hosts.
|
|
93
|
+
|
|
22
94
|
## [0.30.10] - 2026-10-04
|
|
23
95
|
|
|
24
96
|
<!-- stamped -->
|
|
@@ -1,8 +1,12 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "../../../../lib/profiler/cluster/security"
|
|
4
|
+
|
|
3
5
|
module Profiler
|
|
4
6
|
module Api
|
|
5
7
|
class ClusterController < Profiler::ApplicationController
|
|
8
|
+
SLAVE_ACTIONS = %w[register heartbeat].freeze
|
|
9
|
+
|
|
6
10
|
def register
|
|
7
11
|
name = params[:name].to_s
|
|
8
12
|
url = params[:url].to_s
|
|
@@ -11,7 +15,11 @@ module Profiler
|
|
|
11
15
|
return render json: { error: "name and url are required" }, status: :unprocessable_entity
|
|
12
16
|
end
|
|
13
17
|
|
|
14
|
-
|
|
18
|
+
if (reason = Cluster::Security.slave_url_denial(url))
|
|
19
|
+
return render json: { error: reason }, status: :unprocessable_entity
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
Profiler.slave_registry.register(name: name, url: Cluster::Security.normalized_url(url))
|
|
15
23
|
render json: { ok: true, name: name }
|
|
16
24
|
end
|
|
17
25
|
|
|
@@ -28,6 +36,34 @@ module Profiler
|
|
|
28
36
|
def slaves
|
|
29
37
|
render json: { slaves: Profiler.slave_registry.all }
|
|
30
38
|
end
|
|
39
|
+
|
|
40
|
+
private
|
|
41
|
+
|
|
42
|
+
# A slave is a server, often on another host, not a browser: it proves itself with the
|
|
43
|
+
# shared secret instead of the access guard and the forgery header. Without
|
|
44
|
+
# cluster_require_secret, and with no secret configured, the guard of the other
|
|
45
|
+
# endpoints applies, as in 0.30.6.
|
|
46
|
+
def authorize_request
|
|
47
|
+
return super unless secret_authenticated_action?
|
|
48
|
+
return if Cluster::Security.request_secret_valid?(request)
|
|
49
|
+
|
|
50
|
+
if (problem = Cluster::Security.secret_problem)
|
|
51
|
+
deny("#{problem} on this master: cluster requests are refused")
|
|
52
|
+
else
|
|
53
|
+
deny("Missing or invalid #{Cluster::Security::SECRET_HEADER} header")
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
# The secret is a custom header, which a cross-site page cannot send without a preflight.
|
|
58
|
+
def verified_request?
|
|
59
|
+
return Cluster::Security.request_secret_valid?(request) if secret_authenticated_action?
|
|
60
|
+
|
|
61
|
+
super
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def secret_authenticated_action?
|
|
65
|
+
SLAVE_ACTIONS.include?(action_name) && Cluster::Security.secret_required?
|
|
66
|
+
end
|
|
31
67
|
end
|
|
32
68
|
end
|
|
33
69
|
end
|
|
@@ -7,8 +7,10 @@ module Profiler
|
|
|
7
7
|
class SlaveProxyController < Profiler::ApplicationController
|
|
8
8
|
def forward
|
|
9
9
|
proxy = Cluster::SlaveProxy.new(params[:slave_name])
|
|
10
|
-
|
|
11
|
-
|
|
10
|
+
# Each segment is escaped: a decoded "?" or "#" stays in its segment, and a dot segment
|
|
11
|
+
# is refused, so the request stays under the slave's API.
|
|
12
|
+
segments = params[:path].to_s.split("/", -1).map { |segment| Cluster::Security.escape_segment(segment) }
|
|
13
|
+
full_path = "/_profiler/api/#{segments.join("/")}"
|
|
12
14
|
|
|
13
15
|
result = case request.method
|
|
14
16
|
when "GET"
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "../../../lib/profiler/cluster/security"
|
|
4
|
+
|
|
3
5
|
module Profiler
|
|
4
6
|
class ApplicationController < ActionController::Base
|
|
5
7
|
layout "profiler/application"
|
|
@@ -18,6 +20,8 @@ module Profiler
|
|
|
18
20
|
end
|
|
19
21
|
|
|
20
22
|
def authorize_request
|
|
23
|
+
# Checked first, so that :allow_local does not log a refusal for the master's calls.
|
|
24
|
+
return if cluster_master_request?
|
|
21
25
|
return if Profiler.configuration.authorized?(request)
|
|
22
26
|
|
|
23
27
|
deny("Not authorized to access the profiler")
|
|
@@ -32,7 +36,14 @@ module Profiler
|
|
|
32
36
|
# Rails' token, or the header every profiler client sends. A third-party page cannot add a
|
|
33
37
|
# custom header without a CORS preflight, which is refused unless an origin was allowed.
|
|
34
38
|
def verified_request?
|
|
35
|
-
super || request.headers[Profiler::FORGERY_PROTECTION_HEADER].present?
|
|
39
|
+
super || request.headers[Profiler::FORGERY_PROTECTION_HEADER].present? || cluster_master_request?
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
# On a slave, the master's proxied calls carry the shared secret, which stands for both the
|
|
43
|
+
# access guard and the forgery header. Never true on a node without master_url, or without
|
|
44
|
+
# a configured secret.
|
|
45
|
+
def cluster_master_request?
|
|
46
|
+
Profiler.configuration.slave? && Cluster::Security.request_secret_valid?(request)
|
|
36
47
|
end
|
|
37
48
|
|
|
38
49
|
def handle_unverified_request
|
data/config/routes.rb
CHANGED
|
@@ -3,7 +3,12 @@
|
|
|
3
3
|
require "profiler/mcp/server"
|
|
4
4
|
|
|
5
5
|
Profiler::Engine.routes.draw do
|
|
6
|
-
|
|
6
|
+
# Read on each request rather than when the routes are drawn: the application's initializer
|
|
7
|
+
# may set these options after the engine routes are loaded. A route that does not match
|
|
8
|
+
# answers 404, as if it were not there.
|
|
9
|
+
constraints(->(_request) { Profiler.configuration.mcp_http_enabled? }) do
|
|
10
|
+
mount Profiler::MCP::Server.rack_app, at: "mcp"
|
|
11
|
+
end
|
|
7
12
|
|
|
8
13
|
root to: "profiles#index"
|
|
9
14
|
|
|
@@ -52,14 +57,16 @@ Profiler::Engine.routes.draw do
|
|
|
52
57
|
delete "test_runner/runs/:id", to: "test_runner#destroy"
|
|
53
58
|
get "events/:token", to: "events#subscribe", as: :profile_events
|
|
54
59
|
|
|
55
|
-
# Cluster endpoints (master-side)
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
60
|
+
# Cluster endpoints (master-side), routed only on a node with cluster_master set
|
|
61
|
+
constraints(->(_request) { Profiler.configuration.cluster_master? }) do
|
|
62
|
+
post "cluster/register", to: "cluster#register"
|
|
63
|
+
post "cluster/heartbeat", to: "cluster#heartbeat"
|
|
64
|
+
get "cluster/slaves", to: "cluster#slaves"
|
|
59
65
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
66
|
+
# Slave proxy: must be last to avoid shadowing other api routes
|
|
67
|
+
scope "/slaves/:slave_name" do
|
|
68
|
+
match "*path", to: "slave_proxy#forward", via: :all
|
|
69
|
+
end
|
|
63
70
|
end
|
|
64
71
|
end
|
|
65
72
|
end
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "net/http"
|
|
4
4
|
require "json"
|
|
5
5
|
require "uri"
|
|
6
|
+
require_relative "security"
|
|
6
7
|
|
|
7
8
|
module Profiler
|
|
8
9
|
module Cluster
|
|
@@ -22,10 +23,9 @@ module Profiler
|
|
|
22
23
|
|
|
23
24
|
def register!
|
|
24
25
|
config = Profiler.configuration
|
|
25
|
-
uri =
|
|
26
|
+
uri = master_uri("register")
|
|
26
27
|
body = { name: config.resolved_name, url: config.self_url }.to_json
|
|
27
|
-
resp = Net::HTTP.post(uri, body,
|
|
28
|
-
Profiler::FORGERY_PROTECTION_HEADER => "1")
|
|
28
|
+
resp = Net::HTTP.post(uri, body, request_headers)
|
|
29
29
|
unless resp.code.to_i.between?(200, 299)
|
|
30
30
|
raise "Master returned #{resp.code}: #{resp.body.to_s.slice(0, 200)}"
|
|
31
31
|
end
|
|
@@ -35,13 +35,31 @@ module Profiler
|
|
|
35
35
|
|
|
36
36
|
def heartbeat!
|
|
37
37
|
config = Profiler.configuration
|
|
38
|
-
uri =
|
|
38
|
+
uri = master_uri("heartbeat")
|
|
39
39
|
body = { name: config.resolved_name }.to_json
|
|
40
|
-
resp = Net::HTTP.post(uri, body,
|
|
41
|
-
Profiler::FORGERY_PROTECTION_HEADER => "1")
|
|
40
|
+
resp = Net::HTTP.post(uri, body, request_headers)
|
|
42
41
|
raise "Heartbeat rejected #{resp.code}" unless resp.code.to_i.between?(200, 299)
|
|
43
42
|
end
|
|
44
43
|
|
|
44
|
+
# Refused before any request when the secret would cross the network in clear, or when
|
|
45
|
+
# there is no secret for the master to accept.
|
|
46
|
+
def master_uri(action)
|
|
47
|
+
config = Profiler.configuration
|
|
48
|
+
if (reason = Security.master_url_denial(config.master_url))
|
|
49
|
+
raise reason
|
|
50
|
+
end
|
|
51
|
+
if Security.secret_required? && (problem = Security.secret_problem)
|
|
52
|
+
raise "#{problem}: the master refuses registration without a secret"
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
URI("#{config.master_url}/_profiler/api/cluster/#{action}")
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
def request_headers
|
|
59
|
+
{ "Content-Type" => "application/json", Profiler::FORGERY_PROTECTION_HEADER => "1" }
|
|
60
|
+
.merge(Security.outgoing_headers)
|
|
61
|
+
end
|
|
62
|
+
|
|
45
63
|
def start_heartbeat_thread
|
|
46
64
|
Thread.new do
|
|
47
65
|
interval = Profiler.configuration.cluster_heartbeat_interval
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "ipaddr"
|
|
4
|
+
require "uri"
|
|
5
|
+
require "active_support/security_utils"
|
|
6
|
+
|
|
7
|
+
module Profiler
|
|
8
|
+
module Cluster
|
|
9
|
+
# The rules a master and its slaves apply to each other: the shared secret they exchange,
|
|
10
|
+
# and the URLs a master agrees to send requests to.
|
|
11
|
+
module Security
|
|
12
|
+
SECRET_HEADER = "X-Profiler-Cluster-Secret"
|
|
13
|
+
SECRET_ENV_KEY = "HTTP_X_PROFILER_CLUSTER_SECRET"
|
|
14
|
+
# A shorter secret, or a blank one, is treated as if none were configured.
|
|
15
|
+
MIN_SECRET_LENGTH = 32
|
|
16
|
+
API_PREFIX = "/_profiler/api/"
|
|
17
|
+
|
|
18
|
+
class << self
|
|
19
|
+
# Whether a secret has to be presented. False only with cluster_require_secret = false
|
|
20
|
+
# and no cluster_secret, the behaviour of 0.30.6.
|
|
21
|
+
def secret_required?
|
|
22
|
+
config = Profiler.configuration
|
|
23
|
+
config.cluster_require_secret || configured_secret?
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def configured_secret?
|
|
27
|
+
secret_problem.nil?
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# Why the configured secret cannot be used, or nil when it can.
|
|
31
|
+
def secret_problem
|
|
32
|
+
secret = Profiler.configuration.cluster_secret.to_s
|
|
33
|
+
return "No config.cluster_secret is configured" if secret.strip.empty?
|
|
34
|
+
return nil if secret.strip.length >= MIN_SECRET_LENGTH
|
|
35
|
+
|
|
36
|
+
"config.cluster_secret is shorter than #{MIN_SECRET_LENGTH} characters, so it is ignored " \
|
|
37
|
+
"as if none were configured"
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# Logged once at boot on a cluster node whose secret cannot be used, since every cluster
|
|
41
|
+
# request is then refused.
|
|
42
|
+
def warn_about_configuration(logger)
|
|
43
|
+
config = Profiler.configuration
|
|
44
|
+
return unless config.cluster_master? || config.slave?
|
|
45
|
+
return unless (problem = secret_problem) && secret_required?
|
|
46
|
+
|
|
47
|
+
logger.warn("[Profiler Cluster] #{problem}: registrations, heartbeats and proxied requests " \
|
|
48
|
+
"are refused. Generate one with `ruby -rsecurerandom -e 'puts SecureRandom.hex(32)'` " \
|
|
49
|
+
"and give the same value to every node.")
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# Constant-time comparison of the secret a request carries with the configured one.
|
|
53
|
+
# Always false when no usable secret is configured.
|
|
54
|
+
def valid_secret?(presented)
|
|
55
|
+
return false unless configured_secret?
|
|
56
|
+
|
|
57
|
+
expected = Profiler.configuration.cluster_secret.to_s
|
|
58
|
+
return false if presented.to_s.empty?
|
|
59
|
+
|
|
60
|
+
ActiveSupport::SecurityUtils.secure_compare(presented.to_s, expected)
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def request_secret_valid?(request)
|
|
64
|
+
valid_secret?(request.get_header(SECRET_ENV_KEY))
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# The headers to send with a request to another node of the cluster.
|
|
68
|
+
def outgoing_headers
|
|
69
|
+
configured_secret? ? { SECRET_HEADER => Profiler.configuration.cluster_secret.to_s } : {}
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# Returns nil when the master may send requests to this slave URL, otherwise the reason.
|
|
73
|
+
def slave_url_denial(url)
|
|
74
|
+
config = Profiler.configuration
|
|
75
|
+
target = normalize(url)
|
|
76
|
+
return "#{url.inspect} is not a valid http(s) URL" unless target
|
|
77
|
+
|
|
78
|
+
if target[:scheme] == "http" && !loopback_host?(target[:host]) && !config.cluster_allow_insecure_http
|
|
79
|
+
return "#{url} uses plain HTTP to a host that is not a loopback address: HTTPS is required " \
|
|
80
|
+
"(or set config.cluster_allow_insecure_http = true)"
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
allowed = config.cluster_allowed_slave_urls
|
|
84
|
+
return nil if allowed == :any
|
|
85
|
+
return nil if Array(allowed).any? { |entry| covers?(normalize(entry), target) }
|
|
86
|
+
|
|
87
|
+
"#{url} is not allowed by config.cluster_allowed_slave_urls"
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
# Same transport rule for the master URL a slave sends its secret to.
|
|
91
|
+
def master_url_denial(url)
|
|
92
|
+
target = normalize(url)
|
|
93
|
+
return "master_url #{url.inspect} is not a valid http(s) URL" unless target
|
|
94
|
+
return nil if target[:scheme] == "https" || loopback_host?(target[:host])
|
|
95
|
+
return nil if Profiler.configuration.cluster_allow_insecure_http
|
|
96
|
+
|
|
97
|
+
"master_url #{url} uses plain HTTP to a host that is not a loopback address: HTTPS is required " \
|
|
98
|
+
"(or set config.cluster_allow_insecure_http = true)"
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
# The URL as the registry keeps it: lower-case scheme and host, no default port, IPv6
|
|
102
|
+
# between brackets, path segments re-encoded, no trailing slash. Nil when not acceptable.
|
|
103
|
+
def normalized_url(url)
|
|
104
|
+
parts = normalize(url)
|
|
105
|
+
return nil unless parts
|
|
106
|
+
|
|
107
|
+
host = parts[:host].include?(":") ? "[#{parts[:host]}]" : parts[:host]
|
|
108
|
+
default_port = parts[:scheme] == "https" ? 443 : 80
|
|
109
|
+
port = parts[:port] == default_port ? "" : ":#{parts[:port]}"
|
|
110
|
+
path = parts[:segments].map { |segment| escape_segment(segment) }.join("/")
|
|
111
|
+
"#{parts[:scheme]}://#{host}#{port}#{path.empty? ? "" : "/#{path}"}"
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
# One path segment of a request to a slave, from a value that may come from a client (a
|
|
115
|
+
# proxied path, a profile token). "?", "#" and "%" are encoded, so they stay in the
|
|
116
|
+
# segment; ".", ".." and anything holding a "/" are refused, so the request cannot leave
|
|
117
|
+
# the slave's API.
|
|
118
|
+
def escape_segment(value)
|
|
119
|
+
segment = value.to_s
|
|
120
|
+
if segment.empty? || segment == "." || segment == ".." || segment.match?(%r{[/\\]})
|
|
121
|
+
raise Profiler::Error, "Refusing to send #{segment.inspect} as a path segment to a slave profiler"
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
URI.encode_www_form_component(segment).gsub("+", "%20")
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
# The final check on a path sent to a slave: under /_profiler/api/, with no dot or empty
|
|
128
|
+
# segment once decoded.
|
|
129
|
+
def api_path_denial(path)
|
|
130
|
+
raw = path.to_s.split("?", 2).first
|
|
131
|
+
return "#{path.inspect} is not under #{API_PREFIX}" unless raw.start_with?(API_PREFIX)
|
|
132
|
+
|
|
133
|
+
segments = raw.delete_prefix("/").split("/", -1)
|
|
134
|
+
decoded = segments.map { |segment| URI.decode_www_form_component(segment) }
|
|
135
|
+
return nil if decoded.none? { |segment| segment.empty? || segment == "." || segment == ".." || segment.include?("/") }
|
|
136
|
+
|
|
137
|
+
"#{path.inspect} holds an empty, \".\", \"..\" or encoded \"/\" path segment"
|
|
138
|
+
rescue ArgumentError
|
|
139
|
+
"#{path.inspect} is not a valid path"
|
|
140
|
+
end
|
|
141
|
+
|
|
142
|
+
# Scheme, host and port compared after normalization (case, default port, IPv6 brackets),
|
|
143
|
+
# and a path prefix compared segment by segment.
|
|
144
|
+
def covers?(entry, target)
|
|
145
|
+
return false unless entry
|
|
146
|
+
|
|
147
|
+
entry[:scheme] == target[:scheme] && entry[:host] == target[:host] && entry[:port] == target[:port] &&
|
|
148
|
+
(entry[:segments].empty? || target[:segments].first(entry[:segments].size) == entry[:segments])
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
# Returns { scheme:, host:, port:, segments: }, or nil for anything that is not a plain
|
|
152
|
+
# http(s) URL: user info, a query, a fragment, or a "." or ".." path segment are refused
|
|
153
|
+
# rather than interpreted.
|
|
154
|
+
def normalize(url)
|
|
155
|
+
uri = URI.parse(url.to_s.strip)
|
|
156
|
+
return nil unless uri.is_a?(URI::HTTP) && uri.host && !uri.host.empty?
|
|
157
|
+
return nil if uri.userinfo || uri.query || uri.fragment
|
|
158
|
+
|
|
159
|
+
segments = uri.path.to_s.split("/").reject(&:empty?)
|
|
160
|
+
decoded = segments.map { |segment| URI.decode_www_form_component(segment) }
|
|
161
|
+
return nil if decoded.any? { |segment| segment == "." || segment == ".." || segment.include?("/") }
|
|
162
|
+
|
|
163
|
+
{ scheme: uri.scheme.downcase, host: uri.hostname.downcase, port: uri.port, segments: decoded }
|
|
164
|
+
rescue URI::Error, ArgumentError
|
|
165
|
+
nil
|
|
166
|
+
end
|
|
167
|
+
|
|
168
|
+
def loopback_host?(host)
|
|
169
|
+
host = host.to_s.downcase
|
|
170
|
+
return true if host == "localhost"
|
|
171
|
+
|
|
172
|
+
IPAddr.new(host).loopback?
|
|
173
|
+
rescue IPAddr::Error
|
|
174
|
+
false
|
|
175
|
+
end
|
|
176
|
+
end
|
|
177
|
+
end
|
|
178
|
+
end
|
|
179
|
+
end
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "net/http"
|
|
4
4
|
require "json"
|
|
5
5
|
require "uri"
|
|
6
|
+
require_relative "security"
|
|
6
7
|
|
|
7
8
|
module Profiler
|
|
8
9
|
module Cluster
|
|
@@ -15,6 +16,15 @@ module Profiler
|
|
|
15
16
|
entry = Profiler.slave_registry.find!(slave_name)
|
|
16
17
|
raise Profiler::Error, "Slave profiler '#{slave_name}' is offline" if entry.status == "offline"
|
|
17
18
|
|
|
19
|
+
# Checked again on every use: the URL was registered earlier, under a configuration that
|
|
20
|
+
# may have changed since.
|
|
21
|
+
if (reason = Security.slave_url_denial(entry.url))
|
|
22
|
+
raise Profiler::Error, "Slave profiler '#{slave_name}' refused: #{reason}"
|
|
23
|
+
end
|
|
24
|
+
if Security.secret_required? && (problem = Security.secret_problem)
|
|
25
|
+
raise Profiler::Error, "#{problem}: requests to slave profilers are refused"
|
|
26
|
+
end
|
|
27
|
+
|
|
18
28
|
@base_url = entry.url.to_s.chomp("/")
|
|
19
29
|
@open_timeout = open_timeout || OPEN_TIMEOUT
|
|
20
30
|
@read_timeout = read_timeout || READ_TIMEOUT
|
|
@@ -30,7 +40,7 @@ module Profiler
|
|
|
30
40
|
end
|
|
31
41
|
|
|
32
42
|
def load(token)
|
|
33
|
-
data = get_json("/_profiler/api/profiles/#{token}")
|
|
43
|
+
data = get_json("/_profiler/api/profiles/#{Security.escape_segment(token)}")
|
|
34
44
|
return nil unless data["token"] || data["profile"]
|
|
35
45
|
|
|
36
46
|
raw = data["profile"] || data
|
|
@@ -69,7 +79,14 @@ module Profiler
|
|
|
69
79
|
|
|
70
80
|
private
|
|
71
81
|
|
|
82
|
+
# Every request carries the secret, so its path must stay under the slave's API: values
|
|
83
|
+
# from clients are escaped by Security.escape_segment where they are interpolated, and
|
|
84
|
+
# this refuses whatever still reaches here with a dot segment.
|
|
72
85
|
def build_uri(path, params = {})
|
|
86
|
+
if (reason = Security.api_path_denial(path))
|
|
87
|
+
raise Profiler::Error, "Refusing to send a request to a slave profiler: #{reason}"
|
|
88
|
+
end
|
|
89
|
+
|
|
73
90
|
uri = URI("#{@base_url}#{path}")
|
|
74
91
|
unless params.empty?
|
|
75
92
|
uri.query = URI.encode_www_form(params.compact.transform_values(&:to_s))
|
|
@@ -87,12 +104,18 @@ module Profiler
|
|
|
87
104
|
|
|
88
105
|
def request(uri, req)
|
|
89
106
|
req[Profiler::FORGERY_PROTECTION_HEADER] = "1"
|
|
107
|
+
Security.outgoing_headers.each { |name, value| req[name] = value }
|
|
108
|
+
# Net::HTTP never follows a redirect; a 3xx is reported, without its body, rather than
|
|
109
|
+
# handed back as if the slave had answered.
|
|
90
110
|
resp = Net::HTTP.start(uri.hostname, uri.port,
|
|
91
111
|
open_timeout: @open_timeout, read_timeout: @read_timeout,
|
|
92
112
|
use_ssl: uri.scheme == "https") do |http|
|
|
93
113
|
http.request(req)
|
|
94
114
|
end
|
|
95
115
|
return {} if resp.code == "204"
|
|
116
|
+
if resp.code.to_s.start_with?("3")
|
|
117
|
+
raise Profiler::Error, "Slave profiler answered #{resp.code} (redirect not followed)"
|
|
118
|
+
end
|
|
96
119
|
|
|
97
120
|
parse_response(resp)
|
|
98
121
|
end
|
|
@@ -13,7 +13,7 @@ module Profiler
|
|
|
13
13
|
end
|
|
14
14
|
|
|
15
15
|
def set_return_value(value)
|
|
16
|
-
@return_value = value.inspect
|
|
16
|
+
@return_value = Profiler::Redaction.truncate(value.inspect, 10_000, "")
|
|
17
17
|
@return_value_captured = true
|
|
18
18
|
rescue
|
|
19
19
|
@return_value = "(uninspectable)"
|
|
@@ -50,7 +50,7 @@ module Profiler
|
|
|
50
50
|
end
|
|
51
51
|
|
|
52
52
|
def toolbar_summary
|
|
53
|
-
{ text: @expression
|
|
53
|
+
{ text: Profiler::Redaction.truncate(@expression, 30, ""), color: "blue" }
|
|
54
54
|
end
|
|
55
55
|
end
|
|
56
56
|
end
|
|
@@ -86,7 +86,7 @@ module Profiler
|
|
|
86
86
|
|
|
87
87
|
sql = payload[:sql].to_s
|
|
88
88
|
add_event Models::TimelineEvent.new(
|
|
89
|
-
name:
|
|
89
|
+
name: Profiler::Redaction.truncate(sql, 80),
|
|
90
90
|
started_at: started,
|
|
91
91
|
finished_at: finished,
|
|
92
92
|
category: "sql",
|
|
@@ -100,7 +100,7 @@ module Profiler
|
|
|
100
100
|
op = name.split(".").first.sub("cache_", "")
|
|
101
101
|
key = payload[:key].to_s
|
|
102
102
|
add_event Models::TimelineEvent.new(
|
|
103
|
-
name: "cache_#{op}: #{
|
|
103
|
+
name: "cache_#{op}: #{Profiler::Redaction.truncate(key, 60)}",
|
|
104
104
|
started_at: started,
|
|
105
105
|
finished_at: finished,
|
|
106
106
|
category: "cache",
|
|
@@ -224,12 +224,12 @@ module Profiler
|
|
|
224
224
|
if mail.multipart?
|
|
225
225
|
html_part = mail.parts.find { |p| p.content_type.to_s.start_with?("text/html") }
|
|
226
226
|
text_part = mail.parts.find { |p| p.content_type.to_s.start_with?("text/plain") }
|
|
227
|
-
html = html_part&.body&.decoded&.then { |b| b
|
|
228
|
-
text = text_part&.body&.decoded&.then { |b| b
|
|
227
|
+
html = html_part&.body&.decoded&.then { |b| Profiler::Redaction.truncate(b, MAX_BODY_SIZE, "") }
|
|
228
|
+
text = text_part&.body&.decoded&.then { |b| Profiler::Redaction.truncate(b, MAX_BODY_SIZE, "") }
|
|
229
229
|
elsif mail.content_type.to_s.start_with?("text/html")
|
|
230
|
-
html = mail.body.decoded
|
|
230
|
+
html = Profiler::Redaction.truncate(mail.body.decoded, MAX_BODY_SIZE, "")
|
|
231
231
|
else
|
|
232
|
-
text = mail.body.decoded
|
|
232
|
+
text = Profiler::Redaction.truncate(mail.body.decoded, MAX_BODY_SIZE, "")
|
|
233
233
|
end
|
|
234
234
|
[html.nil? || html.empty? ? nil : html, text.nil? || text.empty? ? nil : text]
|
|
235
235
|
rescue StandardError
|
|
@@ -240,7 +240,7 @@ module Profiler
|
|
|
240
240
|
return "nil" if value.nil?
|
|
241
241
|
|
|
242
242
|
inspected = value.inspect
|
|
243
|
-
inspected
|
|
243
|
+
Profiler::Redaction.truncate(inspected, 300, "…")
|
|
244
244
|
rescue StandardError
|
|
245
245
|
value.to_s
|
|
246
246
|
end
|
|
@@ -41,7 +41,9 @@ module Profiler
|
|
|
41
41
|
:compress_bodies, :compress_body_threshold,
|
|
42
42
|
:redact_sensitive_data, :filter_parameters, :env_allowlist,
|
|
43
43
|
:name, :master_url, :self_url,
|
|
44
|
-
:cluster_heartbeat_interval, :cluster_offline_threshold
|
|
44
|
+
:cluster_heartbeat_interval, :cluster_offline_threshold,
|
|
45
|
+
:cluster_master, :cluster_secret, :cluster_require_secret,
|
|
46
|
+
:cluster_allowed_slave_urls, :cluster_allow_insecure_http
|
|
45
47
|
|
|
46
48
|
attr_writer :tmp_path
|
|
47
49
|
|
|
@@ -98,6 +100,12 @@ module Profiler
|
|
|
98
100
|
@self_url = nil
|
|
99
101
|
@cluster_heartbeat_interval = 15
|
|
100
102
|
@cluster_offline_threshold = 60
|
|
103
|
+
# Cluster security: see Profiler::Cluster::Security and the Cluster section of the README.
|
|
104
|
+
@cluster_master = false
|
|
105
|
+
@cluster_secret = nil
|
|
106
|
+
@cluster_require_secret = true
|
|
107
|
+
@cluster_allowed_slave_urls = []
|
|
108
|
+
@cluster_allow_insecure_http = false
|
|
101
109
|
end
|
|
102
110
|
|
|
103
111
|
def tmp_path
|
|
@@ -112,6 +120,18 @@ module Profiler
|
|
|
112
120
|
!slave?
|
|
113
121
|
end
|
|
114
122
|
|
|
123
|
+
# Whether this node serves the master-side cluster routes (register, heartbeat, slaves and
|
|
124
|
+
# the slave proxy). Read on each request by the route constraints.
|
|
125
|
+
def cluster_master?
|
|
126
|
+
cluster_master ? true : false
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
# Whether /_profiler/mcp is routed. The stdio transport (rake profiler:mcp) does not depend
|
|
130
|
+
# on it.
|
|
131
|
+
def mcp_http_enabled?
|
|
132
|
+
mcp_enabled && mcp_transport.to_s == "http" ? true : false
|
|
133
|
+
end
|
|
134
|
+
|
|
115
135
|
def resolved_name
|
|
116
136
|
@name || (defined?(Rails) ? Rails.application.class.module_parent_name.underscore : "profiler")
|
|
117
137
|
end
|
|
@@ -39,7 +39,7 @@ module Profiler
|
|
|
39
39
|
profile = Models::Profile.new
|
|
40
40
|
profile.profile_type = "console"
|
|
41
41
|
profile.gem_version = Profiler::VERSION
|
|
42
|
-
profile.path =
|
|
42
|
+
profile.path = Redaction.truncate(@expression, 200)
|
|
43
43
|
profile.method = "CONSOLE"
|
|
44
44
|
|
|
45
45
|
console_collector = Collectors::ConsoleCollector.new(profile, expression: @expression)
|
|
@@ -150,7 +150,8 @@ module Profiler
|
|
|
150
150
|
mime = content_type.split(";").first.to_s.strip
|
|
151
151
|
|
|
152
152
|
if mime.match?(BINARY_CONTENT_TYPES)
|
|
153
|
-
|
|
153
|
+
# Masked on the raw bytes, before the encoding hides them from any later search.
|
|
154
|
+
{ body: Base64.strict_encode64(Redaction.hide_credentials(body.b)), encoding: "base64" }
|
|
154
155
|
else
|
|
155
156
|
text = Redaction.filter_body(body, content_type)
|
|
156
157
|
.encode("UTF-8", invalid: :replace, undef: :replace, replace: "?")
|
|
@@ -118,11 +118,11 @@ module Profiler
|
|
|
118
118
|
|
|
119
119
|
Redaction.filter_value(args).map do |arg|
|
|
120
120
|
case arg
|
|
121
|
-
when String then
|
|
121
|
+
when String then Redaction.truncate(arg, 200)
|
|
122
122
|
when Numeric, TrueClass, FalseClass, NilClass then arg
|
|
123
123
|
else
|
|
124
124
|
inspected = arg.inspect
|
|
125
|
-
|
|
125
|
+
Redaction.truncate(inspected, 200)
|
|
126
126
|
end
|
|
127
127
|
rescue
|
|
128
128
|
arg.to_s
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
require "uri"
|
|
5
|
+
|
|
6
|
+
module Profiler
|
|
7
|
+
module MCP
|
|
8
|
+
# Checks a request to the MCP HTTP transport before the MCP server sees it, with the same
|
|
9
|
+
# rules as the profiler's controllers: the profiler must be enabled, the request authorized,
|
|
10
|
+
# and a POST must not be one that another site could forge from a browser.
|
|
11
|
+
#
|
|
12
|
+
# Forgery protection does not use the X-Profiler-Request header, which MCP clients do not
|
|
13
|
+
# send. A POST must carry Content-Type: application/json instead: a cross-site form or
|
|
14
|
+
# "simple" fetch can only send text/plain, form-urlencoded or multipart, and JSON needs a
|
|
15
|
+
# CORS preflight that the profiler grants to configured origins only. A browser also sends
|
|
16
|
+
# its Origin, which has to be the profiler's own or one of cors_allowed_origins. GET (the
|
|
17
|
+
# SSE stream) changes nothing, and DELETE always needs a preflight.
|
|
18
|
+
module HttpGuard
|
|
19
|
+
JSON_MEDIA_TYPE = "application/json"
|
|
20
|
+
|
|
21
|
+
class << self
|
|
22
|
+
# Returns nil when the request may proceed, otherwise a Rack response.
|
|
23
|
+
def call(env)
|
|
24
|
+
config = Profiler.configuration
|
|
25
|
+
return deny("Profiler is disabled") unless config.enabled
|
|
26
|
+
|
|
27
|
+
request = ActionDispatch::Request.new(env)
|
|
28
|
+
return deny("Not authorized to access the profiler") unless config.authorized?(request)
|
|
29
|
+
return nil unless config.api_forgery_protection
|
|
30
|
+
|
|
31
|
+
forgery_denial(request, config)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
private
|
|
35
|
+
|
|
36
|
+
def forgery_denial(request, config)
|
|
37
|
+
origin = request.get_header("HTTP_ORIGIN")
|
|
38
|
+
if origin && !origin.empty? && !allowed_origin?(origin, request, config)
|
|
39
|
+
return deny("Cross-origin request refused: Origin #{origin} is not allowed")
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
if request.post? && media_type(request) != JSON_MEDIA_TYPE
|
|
43
|
+
return deny("A POST to the MCP endpoint needs Content-Type: #{JSON_MEDIA_TYPE}")
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
nil
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def media_type(request)
|
|
50
|
+
request.get_header("CONTENT_TYPE").to_s.split(";").first.to_s.strip.downcase
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def allowed_origin?(origin, request, config)
|
|
54
|
+
return true if Array(config.cors_allowed_origins).include?(origin)
|
|
55
|
+
|
|
56
|
+
same_origin?(origin, request.base_url)
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def same_origin?(origin, base_url)
|
|
60
|
+
a = URI.parse(origin)
|
|
61
|
+
b = URI.parse(base_url)
|
|
62
|
+
a.scheme.to_s.casecmp?(b.scheme.to_s) && a.host.to_s.casecmp?(b.host.to_s) && a.port == b.port
|
|
63
|
+
rescue URI::InvalidURIError
|
|
64
|
+
false
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def deny(message)
|
|
68
|
+
[403, { "content-type" => "application/json" }, [{ error: message }.to_json]]
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
end
|
|
73
|
+
end
|
data/lib/profiler/mcp/server.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "mcp"
|
|
4
|
+
require_relative "http_guard"
|
|
4
5
|
|
|
5
6
|
module Profiler
|
|
6
7
|
module MCP
|
|
@@ -12,7 +13,9 @@ module Profiler
|
|
|
12
13
|
|
|
13
14
|
def rack_app
|
|
14
15
|
# Memoized at class level — survives route reloads in development
|
|
15
|
-
@rack_app ||=
|
|
16
|
+
@rack_app ||= lambda do |env|
|
|
17
|
+
HttpGuard.call(env) || instance.http_transport.handle_request(Rack::Request.new(env))
|
|
18
|
+
end
|
|
16
19
|
end
|
|
17
20
|
end
|
|
18
21
|
|
|
@@ -9,7 +9,8 @@ require_relative "../redaction"
|
|
|
9
9
|
module Profiler
|
|
10
10
|
module Models
|
|
11
11
|
class Profile
|
|
12
|
-
|
|
12
|
+
attr_reader :path
|
|
13
|
+
attr_accessor :token, :method, :status, :duration, :memory,
|
|
13
14
|
:started_at, :finished_at, :params, :headers,
|
|
14
15
|
:response_headers, :collectors_data, :collectors_metadata,
|
|
15
16
|
:parent_token, :is_ajax, :profile_type,
|
|
@@ -50,8 +51,15 @@ module Profiler
|
|
|
50
51
|
@response_headers = Redaction.filter_headers(response_headers)
|
|
51
52
|
end
|
|
52
53
|
|
|
54
|
+
# Every collector stores its data here: logs, exception messages, dumps and SQL text are free
|
|
55
|
+
# text with no name to filter on, so the profiler's own credentials are masked by value.
|
|
53
56
|
def add_collector_data(name, data)
|
|
54
|
-
@collectors_data[name.to_s] = data
|
|
57
|
+
@collectors_data[name.to_s] = Redaction.hide_credentials(data)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
# The path is free text for a console profile: the expression typed.
|
|
61
|
+
def path=(value)
|
|
62
|
+
@path = Redaction.hide_credentials(value)
|
|
55
63
|
end
|
|
56
64
|
|
|
57
65
|
def collector_data(name)
|
|
@@ -160,7 +168,9 @@ module Profiler
|
|
|
160
168
|
return { body: nil, encoding: "text" } if raw.nil? || raw.empty?
|
|
161
169
|
|
|
162
170
|
if binary_content_type?(content_type)
|
|
163
|
-
|
|
171
|
+
# Masked on the raw bytes, before the encoding hides them from any later search. A
|
|
172
|
+
# compressed format (zip, png, gzip...) does not hold the secret as it is.
|
|
173
|
+
{ body: Base64.strict_encode64(Redaction.hide_credentials(raw.b)), encoding: "base64" }
|
|
164
174
|
else
|
|
165
175
|
text = Redaction.filter_body(raw, content_type).encode("UTF-8", invalid: :replace, undef: :replace)
|
|
166
176
|
if compress_body?(text)
|
|
@@ -195,7 +205,7 @@ module Profiler
|
|
|
195
205
|
def sanitize_params(params)
|
|
196
206
|
return {} unless params
|
|
197
207
|
|
|
198
|
-
return params.to_h.except(*LEGACY_FILTERED_PARAMS) unless Redaction.enabled?
|
|
208
|
+
return Redaction.hide_credentials(params.to_h.except(*LEGACY_FILTERED_PARAMS)) unless Redaction.enabled?
|
|
199
209
|
|
|
200
210
|
Redaction.filter_hash(params.to_h)
|
|
201
211
|
end
|
data/lib/profiler/railtie.rb
CHANGED
|
@@ -92,7 +92,11 @@ module Profiler
|
|
|
92
92
|
# Check slave? inside on_load — the app's own initializers (config/initializers/profiler.rb)
|
|
93
93
|
# set master_url AFTER railtie initializers run, so the check must happen after_initialize.
|
|
94
94
|
ActiveSupport.on_load(:after_initialize) do
|
|
95
|
-
next unless Profiler.configuration.enabled
|
|
95
|
+
next unless Profiler.configuration.enabled
|
|
96
|
+
|
|
97
|
+
require_relative "cluster/security"
|
|
98
|
+
Profiler::Cluster::Security.warn_about_configuration(Rails.logger)
|
|
99
|
+
next unless Profiler.configuration.slave?
|
|
96
100
|
|
|
97
101
|
require_relative "cluster/master_client"
|
|
98
102
|
Profiler::Cluster::MasterClient.new.start
|
data/lib/profiler/redaction.rb
CHANGED
|
@@ -4,6 +4,7 @@ require "json"
|
|
|
4
4
|
require "set"
|
|
5
5
|
require "rack/utils"
|
|
6
6
|
require "active_support/parameter_filter"
|
|
7
|
+
require_relative "cluster/security"
|
|
7
8
|
|
|
8
9
|
module Profiler
|
|
9
10
|
# The single filter applied to everything the profiler captures: request
|
|
@@ -44,6 +45,18 @@ module Profiler
|
|
|
44
45
|
# fail on the same key inside the text.
|
|
45
46
|
CONTEXT_SENSITIVE_REGEXP = /\\[AzZbBG]|[\^$]|\(\?<?[=!]/
|
|
46
47
|
|
|
48
|
+
# The profiler's own credentials are masked wherever they appear, by value,
|
|
49
|
+
# whatever name they travel under, and even with redact_sensitive_data
|
|
50
|
+
# off: the cluster secret opens every slave from the network, and a
|
|
51
|
+
# profile can be read on any node. Wraps every public filter, so that each
|
|
52
|
+
# capture point gets it.
|
|
53
|
+
module CredentialMasking
|
|
54
|
+
%i[filter_hash filter_value filter_named filter_headers filter_body filter_query filter_url
|
|
55
|
+
env_snapshot env_value env_overrides].each do |name|
|
|
56
|
+
define_method(name) { |*args| hide_credentials(super(*args)) }
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
|
|
47
60
|
# Bound on the memo of names already tested against the filter.
|
|
48
61
|
KEY_CACHE_LIMIT = 10_000
|
|
49
62
|
|
|
@@ -52,6 +65,41 @@ module Profiler
|
|
|
52
65
|
PROBE = "x"
|
|
53
66
|
|
|
54
67
|
class << self
|
|
68
|
+
prepend CredentialMasking
|
|
69
|
+
|
|
70
|
+
# Masks every occurrence of the profiler's own credentials in a value:
|
|
71
|
+
# a string equal to one becomes MASK, a string holding one has it
|
|
72
|
+
# replaced, hashes and arrays are walked. Only what changes is copied:
|
|
73
|
+
# a value holding no credential is returned as it is.
|
|
74
|
+
def hide_credentials(value)
|
|
75
|
+
credentials = credential_values
|
|
76
|
+
credentials.empty? ? value : hide_in(value, credentials)
|
|
77
|
+
rescue StandardError => e
|
|
78
|
+
report(e, "masking a profiler credential")
|
|
79
|
+
MASK
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
# Text cut to +max+ characters, with +ellipsis+ when cut, once the
|
|
83
|
+
# credentials are masked: cutting first would leave a prefix of one, or
|
|
84
|
+
# all of it once the pieces are put back together. Every capture point
|
|
85
|
+
# that shortens text goes through here.
|
|
86
|
+
def truncate(text, max, ellipsis = "...")
|
|
87
|
+
text = hide_credentials(text.to_s)
|
|
88
|
+
text.length > max ? "#{text[0, max]}#{ellipsis}" : text
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# How many trailing bytes of +text+, a stream read in pieces, could be
|
|
92
|
+
# the start of a credential and have to wait for the next piece: the
|
|
93
|
+
# longest end of the text that is also the beginning of a credential,
|
|
94
|
+
# 0 when there is none, so that other output is shown as it comes.
|
|
95
|
+
def held_back_bytes(text)
|
|
96
|
+
bytes = text.to_s.b
|
|
97
|
+
credential_values.map do |credential|
|
|
98
|
+
start = credential.b
|
|
99
|
+
[start.bytesize - 1, bytes.bytesize].min.downto(1).find { |k| bytes.end_with?(start.byteslice(0, k)) } || 0
|
|
100
|
+
end.max || 0
|
|
101
|
+
end
|
|
102
|
+
|
|
55
103
|
def enabled?
|
|
56
104
|
Profiler.configuration.redact_sensitive_data != false
|
|
57
105
|
end
|
|
@@ -231,6 +279,56 @@ module Profiler
|
|
|
231
279
|
|
|
232
280
|
private
|
|
233
281
|
|
|
282
|
+
# Only a secret the cluster would use (Cluster::Security.secret_problem
|
|
283
|
+
# nil: 32 characters or more): a shorter one is ignored by the cluster,
|
|
284
|
+
# and masking "true" or "/" everywhere would hide the very data profiled.
|
|
285
|
+
def credential_values
|
|
286
|
+
return [] unless Cluster::Security.configured_secret?
|
|
287
|
+
|
|
288
|
+
[Profiler.configuration.cluster_secret.to_s.strip]
|
|
289
|
+
end
|
|
290
|
+
|
|
291
|
+
def hide_in(value, credentials)
|
|
292
|
+
case value
|
|
293
|
+
when String then hide_in_string(value, credentials)
|
|
294
|
+
when Hash
|
|
295
|
+
changes = nil
|
|
296
|
+
value.each do |key, v|
|
|
297
|
+
masked = hide_in(v, credentials)
|
|
298
|
+
(changes ||= {})[key] = masked unless masked.equal?(v)
|
|
299
|
+
end
|
|
300
|
+
changes ? value.merge(changes) : value
|
|
301
|
+
when Array
|
|
302
|
+
copy = nil
|
|
303
|
+
value.each_with_index do |v, i|
|
|
304
|
+
masked = hide_in(v, credentials)
|
|
305
|
+
next if masked.equal?(v)
|
|
306
|
+
|
|
307
|
+
(copy ||= value.dup)[i] = masked
|
|
308
|
+
end
|
|
309
|
+
copy || value
|
|
310
|
+
else value
|
|
311
|
+
end
|
|
312
|
+
end
|
|
313
|
+
|
|
314
|
+
# The string itself when it holds no credential, which is the common
|
|
315
|
+
# case and costs one search; otherwise a copy, compared as bytes so that
|
|
316
|
+
# a binary body never meets an incompatible encoding.
|
|
317
|
+
def hide_in_string(string, credentials)
|
|
318
|
+
credentials.reduce(string) do |text, credential|
|
|
319
|
+
next text unless contains?(text, credential)
|
|
320
|
+
next MASK if text.b.strip == credential.b
|
|
321
|
+
|
|
322
|
+
text.b.gsub(credential.b, MASK).force_encoding(text.encoding)
|
|
323
|
+
end
|
|
324
|
+
end
|
|
325
|
+
|
|
326
|
+
def contains?(text, credential)
|
|
327
|
+
text.include?(credential)
|
|
328
|
+
rescue Encoding::CompatibilityError, ArgumentError
|
|
329
|
+
text.b.include?(credential.b)
|
|
330
|
+
end
|
|
331
|
+
|
|
234
332
|
def compiled
|
|
235
333
|
filters = current_filters
|
|
236
334
|
state = @compiled
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
require "concurrent"
|
|
4
4
|
require "securerandom"
|
|
5
|
+
require_relative "../redaction"
|
|
5
6
|
|
|
6
7
|
module Profiler
|
|
7
8
|
module TestRunner
|
|
@@ -24,6 +25,9 @@ module Profiler
|
|
|
24
25
|
def initialize
|
|
25
26
|
@runs = Concurrent::Hash.new
|
|
26
27
|
@locks = Concurrent::Hash.new
|
|
28
|
+
@held = Concurrent::Hash.new
|
|
29
|
+
@output_finished = Concurrent::Hash.new
|
|
30
|
+
@held_lock = Mutex.new
|
|
27
31
|
end
|
|
28
32
|
|
|
29
33
|
def create(files:, framework:)
|
|
@@ -58,11 +62,38 @@ module Profiler
|
|
|
58
62
|
run
|
|
59
63
|
end
|
|
60
64
|
|
|
65
|
+
# Output is free text read back by the API, the SSE stream and run_tests, in the pieces it
|
|
66
|
+
# arrives in: the profiler's own credentials are masked by value, as in a profile. They
|
|
67
|
+
# are masked on the text joined with what was held back from the previous piece, and the
|
|
68
|
+
# last bytes that could start a credential wait for the next piece, so that none is ever
|
|
69
|
+
# shown cut in two. What is held back is released by finish_output, once the process has
|
|
70
|
+
# nothing more to print: a killed run still prints its summary after its status changed.
|
|
61
71
|
def append_output(id, chunk)
|
|
62
72
|
run = @runs[id]
|
|
63
73
|
return unless run
|
|
64
74
|
|
|
65
|
-
|
|
75
|
+
@held_lock.synchronize do
|
|
76
|
+
text = Profiler::Redaction.hide_credentials(@held.fetch(id, "".b) + chunk.to_s.b)
|
|
77
|
+
holdback = @output_finished[id] ? 0 : Profiler::Redaction.held_back_bytes(text)
|
|
78
|
+
cut = text.bytesize - holdback
|
|
79
|
+
@held[id] = text.byteslice(cut, text.bytesize - cut)
|
|
80
|
+
shown = text.byteslice(0, cut).force_encoding(chunk.to_s.encoding)
|
|
81
|
+
run.output_lines.push(shown) unless shown.empty?
|
|
82
|
+
end
|
|
83
|
+
signal(id)
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
# The end of the output, told by the reader of the process once it read everything: the
|
|
87
|
+
# bytes held back are shown, and anything appended later is shown as it comes.
|
|
88
|
+
def finish_output(id)
|
|
89
|
+
run = @runs[id]
|
|
90
|
+
return unless run
|
|
91
|
+
|
|
92
|
+
@held_lock.synchronize do
|
|
93
|
+
@output_finished[id] = true
|
|
94
|
+
rest = @held.delete(id)
|
|
95
|
+
run.output_lines.push(rest) if rest && !rest.empty?
|
|
96
|
+
end
|
|
66
97
|
signal(id)
|
|
67
98
|
end
|
|
68
99
|
|
|
@@ -97,6 +128,12 @@ module Profiler
|
|
|
97
128
|
lock[:mutex].synchronize { lock[:cond].broadcast }
|
|
98
129
|
end
|
|
99
130
|
|
|
131
|
+
def forget_output(id)
|
|
132
|
+
@held.delete(id)
|
|
133
|
+
@output_finished.delete(id)
|
|
134
|
+
true
|
|
135
|
+
end
|
|
136
|
+
|
|
100
137
|
def snapshot(run, position)
|
|
101
138
|
lines = run.output_lines.dup
|
|
102
139
|
{
|
|
@@ -109,7 +146,7 @@ module Profiler
|
|
|
109
146
|
|
|
110
147
|
def cleanup_old_runs
|
|
111
148
|
cutoff = Time.now - TTL
|
|
112
|
-
@runs.delete_if { |id, r| r.finished_at && r.finished_at < cutoff && @locks.delete(id) }
|
|
149
|
+
@runs.delete_if { |id, r| r.finished_at && r.finished_at < cutoff && @locks.delete(id) && forget_output(id) }
|
|
113
150
|
end
|
|
114
151
|
end
|
|
115
152
|
|
|
@@ -124,6 +124,7 @@ module Profiler
|
|
|
124
124
|
finished_at: Time.now
|
|
125
125
|
)
|
|
126
126
|
Profiler::TestRunner.run_store.append_output(run.id, "\n[Profiler] Error: #{e.message}\n")
|
|
127
|
+
Profiler::TestRunner.run_store.finish_output(run.id)
|
|
127
128
|
end
|
|
128
129
|
end
|
|
129
130
|
|
|
@@ -137,10 +138,15 @@ module Profiler
|
|
|
137
138
|
IO.popen([env, *cmd, unsetenv_others: true, err: [:child, :out]], "r") do |io|
|
|
138
139
|
Profiler::TestRunner.run_store.update(run.id, pid: io.pid)
|
|
139
140
|
|
|
140
|
-
|
|
141
|
-
|
|
141
|
+
begin
|
|
142
|
+
while (chunk = io.read(256))
|
|
143
|
+
break if chunk.empty?
|
|
142
144
|
|
|
143
|
-
|
|
145
|
+
Profiler::TestRunner.run_store.append_output(run.id, chunk)
|
|
146
|
+
end
|
|
147
|
+
ensure
|
|
148
|
+
# The process printed everything, killed or not: what was held back can be shown.
|
|
149
|
+
Profiler::TestRunner.run_store.finish_output(run.id)
|
|
144
150
|
end
|
|
145
151
|
end
|
|
146
152
|
|
data/lib/profiler/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rails-profiler
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.30.
|
|
4
|
+
version: 0.30.11
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Sébastien Duplessy
|
|
@@ -135,6 +135,7 @@ files:
|
|
|
135
135
|
- lib/profiler.rb
|
|
136
136
|
- lib/profiler/boot_env.rb
|
|
137
137
|
- lib/profiler/cluster/master_client.rb
|
|
138
|
+
- lib/profiler/cluster/security.rb
|
|
138
139
|
- lib/profiler/cluster/slave_proxy.rb
|
|
139
140
|
- lib/profiler/cluster/slave_registry.rb
|
|
140
141
|
- lib/profiler/cluster/token_cache.rb
|
|
@@ -172,6 +173,7 @@ files:
|
|
|
172
173
|
- lib/profiler/local_request.rb
|
|
173
174
|
- lib/profiler/mcp/body_formatter.rb
|
|
174
175
|
- lib/profiler/mcp/file_cache.rb
|
|
176
|
+
- lib/profiler/mcp/http_guard.rb
|
|
175
177
|
- lib/profiler/mcp/path_extractor.rb
|
|
176
178
|
- lib/profiler/mcp/resources/failing_tests.rb
|
|
177
179
|
- lib/profiler/mcp/resources/n1_patterns.rb
|