rails-profiler 0.30.7 → 0.30.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +72 -0
- data/app/controllers/profiler/api/env_vars_controller.rb +2 -1
- data/lib/profiler/configuration.rb +2 -0
- data/lib/profiler/env_override_store.rb +131 -32
- data/lib/profiler/explain_runner.rb +60 -13
- data/lib/profiler/mcp/server.rb +1 -1
- data/lib/profiler/mcp/tools/reset_env_var.rb +3 -5
- data/lib/profiler/railtie.rb +7 -4
- data/lib/profiler/sql_statement.rb +138 -0
- data/lib/profiler/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a3a3ebc22965fd603d63bafecef777474fc488912625ac2950e618566bce8f09
|
|
4
|
+
data.tar.gz: e82aa1ce06e0f24cfb65b86fe01a942557851735f596edb3172c3fd00d7941f3
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 857313cb42832f5992a26897cbe6cef4d7dd66d6f59b5fccc63c8758db55da54cd6e3587e254e3d1410db561b668782ef8f3fcda73e4535235efba1e9a34c9be
|
|
7
|
+
data.tar.gz: 1bd504a236e7c548011ceed8dcc1488b21bc89400d824306c2f90d8b1817290768a10264265ee093f78d295f873b72a250a00a96fc5fe04f54f85cd2d63597db
|
data/CHANGELOG.md
CHANGED
|
@@ -19,6 +19,78 @@ every commit of every tag interval is accounted for one way or the other.
|
|
|
19
19
|
|
|
20
20
|
## [Unreleased]
|
|
21
21
|
|
|
22
|
+
## [0.30.9] - 2026-10-04
|
|
23
|
+
|
|
24
|
+
<!-- stamped -->
|
|
25
|
+
|
|
26
|
+
### Security
|
|
27
|
+
|
|
28
|
+
- **Database:** Explain only read-only queries. The **Explain** button, `POST /_profiler/api/explain`
|
|
29
|
+
and the MCP `explain_query` tool ran `EXPLAIN ANALYZE` on any stored query, and PostgreSQL's
|
|
30
|
+
`EXPLAIN ANALYZE` runs the statement it explains: explaining a `DELETE`, `UPDATE` or `INSERT` the
|
|
31
|
+
application had just run ran it a second time, for real. Only statements starting with `SELECT`,
|
|
32
|
+
`WITH`, `TABLE` or `VALUES` are explained now, and a writing CTE, `SELECT ... INTO`, `FOR UPDATE`
|
|
33
|
+
or a second statement is refused too, with a 422 from the endpoint and an error from the MCP tool.
|
|
34
|
+
The probe also runs in a transaction that is always rolled back, read-only on PostgreSQL.
|
|
35
|
+
Upgrading: nothing to set, and no setting brings back the explaining of writes, since the old
|
|
36
|
+
behaviour was to run them again; to see the plan of a write, run it yourself in `psql` between
|
|
37
|
+
`BEGIN` and `ROLLBACK`, as the README shows.
|
|
38
|
+
- **Database:** Run the Explain probe on a connection of its own, opened outside the pool and closed
|
|
39
|
+
afterwards, so that nothing it does to its session outlives it: a `SELECT pg_try_advisory_lock(...)`
|
|
40
|
+
explained from a profile left the lock held by a connection of the pool. On PostgreSQL the probe
|
|
41
|
+
is also limited to 30 seconds (`statement_timeout`), and its EXPLAIN goes by the extended
|
|
42
|
+
protocol, which refuses a second statement.
|
|
43
|
+
|
|
44
|
+
### Fixed
|
|
45
|
+
|
|
46
|
+
- **Database:** Put bind values back correctly when explaining a query with ten binds or more
|
|
47
|
+
(`$10` was read as `$1` followed by `0`), and leave alone a `?` or `$1` inside a string literal or
|
|
48
|
+
inside a value already put back.
|
|
49
|
+
|
|
50
|
+
## [0.30.8] - 2026-10-04
|
|
51
|
+
|
|
52
|
+
<!-- stamped -->
|
|
53
|
+
|
|
54
|
+
### Fixed
|
|
55
|
+
|
|
56
|
+
- **Env overrides:** Keep every override when several are set, deleted or reset at the same
|
|
57
|
+
time, from threads of the web process or from Sidekiq processes. Each writer read the file,
|
|
58
|
+
changed it and saved it over the others' changes, and a reader could see a half-written file as
|
|
59
|
+
no override at all. The file is now changed under a lock (`env_overrides.json.lock`, next to
|
|
60
|
+
it) and replaced in one step.
|
|
61
|
+
|
|
62
|
+
### Security
|
|
63
|
+
|
|
64
|
+
- **Env overrides:** Stop applying the environment variable overrides saved from the Env tab or
|
|
65
|
+
the MCP env tools in production, and while the profiler is disabled. They were written into
|
|
66
|
+
`ENV` at every boot whatever `enabled` said and whatever the environment, so an
|
|
67
|
+
`env_overrides.json` left under `tmp/` and shipped with a deployment changed the environment of
|
|
68
|
+
the production application, and an application with `config.enabled = false` still had them
|
|
69
|
+
applied without a word. Sidekiq jobs and console evaluations, which apply them again, follow the
|
|
70
|
+
same rules. When overrides are left out, the boot logs one warning to `Rails.logger` with the
|
|
71
|
+
file, the number of overrides and the reason, never a name or a value.
|
|
72
|
+
- **Env overrides:** Apply them at boot once the application's `config/initializers` have run,
|
|
73
|
+
where `enabled` is decided, instead of before. An initializer of the application that reads
|
|
74
|
+
`ENV` while it runs no longer sees them; requests, jobs, eager loading and `after_initialize`
|
|
75
|
+
still do.
|
|
76
|
+
- **Env overrides:** Resetting one override, or all of them, from the Env tab or the MCP env
|
|
77
|
+
tools no longer writes into `ENV` the "original" values saved in `env_overrides.json` in
|
|
78
|
+
production, nor while the profiler is disabled: those values come from the machine that wrote
|
|
79
|
+
the file, so a reset could put a development value into a production variable, or delete it.
|
|
80
|
+
There, a reset restores only the variables the running process changed itself, to the values
|
|
81
|
+
they had before.
|
|
82
|
+
- **Env overrides:** In production, or with the profiler disabled, typing back in the Env tab the
|
|
83
|
+
original value the overrides file shows for a variable now records the running value first, so
|
|
84
|
+
a later reset puts it back instead of keeping the file's value. A reset also restores a variable
|
|
85
|
+
this process changed even when its entry has gone from the file. The `reset_env_var` MCP tool
|
|
86
|
+
now says whether the original value was restored in the process or `ENV` was left unchanged,
|
|
87
|
+
and no longer prints a value, masked or not.
|
|
88
|
+
- **Upgrading:** to apply the env overrides while the profiler is disabled outside production,
|
|
89
|
+
as before, set `config.apply_env_overrides_when_disabled = true`. Nothing applies them in
|
|
90
|
+
production. If an initializer of yours needs them, call `Profiler.env_override_store.apply!` at
|
|
91
|
+
the end of `config/initializers/profiler.rb`; initializers loaded after it then see them, under
|
|
92
|
+
the same rules.
|
|
93
|
+
|
|
22
94
|
## [0.30.7] - 2026-10-04
|
|
23
95
|
|
|
24
96
|
<!-- stamped -->
|
|
@@ -32,7 +32,8 @@ module Profiler
|
|
|
32
32
|
render json: { key: key, value: nil, deleted: true, override: redacted_override(key) }
|
|
33
33
|
else
|
|
34
34
|
current_original = Profiler.env_override_store.all_overrides.dig(key, "original")
|
|
35
|
-
|
|
35
|
+
# Where the overrides are blocked, set records this process's value, which reset would not.
|
|
36
|
+
if current_original == value.to_s && Profiler.env_override_store.blocked_reason.nil?
|
|
36
37
|
Profiler.env_override_store.reset(key)
|
|
37
38
|
else
|
|
38
39
|
Profiler.env_override_store.set(key, value.to_s)
|
|
@@ -35,6 +35,7 @@ module Profiler
|
|
|
35
35
|
:track_http, :slow_http_threshold, :http_skip_hosts, :http_backtrace_depth,
|
|
36
36
|
:track_jobs,
|
|
37
37
|
:track_console,
|
|
38
|
+
:apply_env_overrides_when_disabled,
|
|
38
39
|
:track_tests,
|
|
39
40
|
:track_mailers, :capture_mail_body, :sanitize_mailer_recipients, :mailer_skip_actions,
|
|
40
41
|
:compress_bodies, :compress_body_threshold,
|
|
@@ -77,6 +78,7 @@ module Profiler
|
|
|
77
78
|
@http_backtrace_depth = 40
|
|
78
79
|
@track_jobs = true
|
|
79
80
|
@track_console = true
|
|
81
|
+
@apply_env_overrides_when_disabled = false
|
|
80
82
|
@track_tests = false
|
|
81
83
|
@track_mailers = true
|
|
82
84
|
@capture_mail_body = false
|
|
@@ -8,63 +8,108 @@ module Profiler
|
|
|
8
8
|
DELETED_SENTINEL = "__profiler_deleted__"
|
|
9
9
|
RESTORE_SENTINEL = "__profiler_restore__"
|
|
10
10
|
|
|
11
|
+
# Why the persisted overrides are left out of ENV, as logged at boot.
|
|
12
|
+
BLOCKED_REASONS = {
|
|
13
|
+
production: "persisted overrides are never applied in production; " \
|
|
14
|
+
"delete the file if it was deployed by mistake",
|
|
15
|
+
disabled: "the profiler is disabled; set " \
|
|
16
|
+
"config.apply_env_overrides_when_disabled = true to apply them anyway"
|
|
17
|
+
}.freeze
|
|
18
|
+
|
|
11
19
|
# File format: { "KEY" => { "value" => "...", "original" => "..." } }
|
|
12
20
|
# Sentinels for "value":
|
|
13
21
|
# DELETED_SENTINEL → ENV.delete(key)
|
|
14
22
|
# RESTORE_SENTINEL → restore ENV[key] to "original" (cleaned up after apply!)
|
|
15
23
|
|
|
24
|
+
def initialize
|
|
25
|
+
# What ENV held in this process before set, delete or apply! changed a key. Where the
|
|
26
|
+
# overrides are blocked, a reset restores these and never the file's "original" values,
|
|
27
|
+
# which come from whatever machine wrote the file.
|
|
28
|
+
@process_originals = {}
|
|
29
|
+
@process_originals_lock = Mutex.new
|
|
30
|
+
end
|
|
31
|
+
|
|
16
32
|
def set(key, value)
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
33
|
+
remember_process_original(key)
|
|
34
|
+
update_overrides do |overrides|
|
|
35
|
+
overrides[key] = { "value" => value.to_s, "original" => original_for(overrides, key) }
|
|
36
|
+
end
|
|
21
37
|
rescue => e
|
|
22
38
|
warn "[Profiler] EnvOverrideStore: failed to set #{key}: #{e.message}"
|
|
23
39
|
end
|
|
24
40
|
|
|
25
41
|
def delete(key)
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
42
|
+
remember_process_original(key)
|
|
43
|
+
update_overrides do |overrides|
|
|
44
|
+
overrides[key] = { "value" => DELETED_SENTINEL, "original" => original_for(overrides, key) }
|
|
45
|
+
end
|
|
30
46
|
rescue => e
|
|
31
47
|
warn "[Profiler] EnvOverrideStore: failed to delete #{key}: #{e.message}"
|
|
32
48
|
end
|
|
33
49
|
|
|
50
|
+
# Returns whether ENV was written in this process.
|
|
34
51
|
def reset(key)
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
overrides[key] = { "value" => RESTORE_SENTINEL, "original" => original }
|
|
42
|
-
save_overrides(overrides)
|
|
52
|
+
entry = nil
|
|
53
|
+
update_overrides do |overrides|
|
|
54
|
+
entry = overrides[key]
|
|
55
|
+
# Keep a RESTORE entry so Sidekiq workers pick it up on next job
|
|
56
|
+
overrides[key] = { "value" => RESTORE_SENTINEL, "original" => entry["original"] } if entry
|
|
57
|
+
end
|
|
43
58
|
|
|
44
59
|
# Apply immediately to the current (web) process
|
|
45
|
-
|
|
60
|
+
restore_env(key, entry, blocked: blocked_reason)
|
|
46
61
|
rescue => e
|
|
47
62
|
warn "[Profiler] EnvOverrideStore: failed to reset #{key}: #{e.message}"
|
|
48
63
|
end
|
|
49
64
|
|
|
50
65
|
def reset_all
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
# Leave a RESTORE sentinel for Sidekiq workers to pick up
|
|
59
|
-
restore_overrides[key] = { "value" => RESTORE_SENTINEL, "original" => original }
|
|
66
|
+
originals = {}
|
|
67
|
+
update_overrides do |overrides|
|
|
68
|
+
overrides.each do |key, entry|
|
|
69
|
+
originals[key] = entry.is_a?(Hash) ? entry["original"] : nil
|
|
70
|
+
# Leave a RESTORE sentinel for Sidekiq workers to pick up
|
|
71
|
+
overrides[key] = { "value" => RESTORE_SENTINEL, "original" => originals[key] }
|
|
72
|
+
end
|
|
60
73
|
end
|
|
61
74
|
|
|
62
|
-
|
|
75
|
+
# Apply immediately to the current (web) process
|
|
76
|
+
blocked = blocked_reason
|
|
77
|
+
originals.each { |key, original| restore_env(key, { "original" => original }, blocked: blocked) }
|
|
63
78
|
rescue => e
|
|
64
79
|
warn "[Profiler] EnvOverrideStore: failed to reset all: #{e.message}"
|
|
65
80
|
end
|
|
66
81
|
|
|
82
|
+
# Why the persisted overrides must not reach ENV in this process, or nil when they may.
|
|
83
|
+
def blocked_reason
|
|
84
|
+
return :production if defined?(Rails) && Rails.respond_to?(:env) && Rails.env.production?
|
|
85
|
+
|
|
86
|
+
config = Profiler.configuration
|
|
87
|
+
return :disabled unless config.enabled || config.apply_env_overrides_when_disabled
|
|
88
|
+
|
|
89
|
+
nil
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# Applies the overrides at boot, or says once in the log why they are left out. The message
|
|
93
|
+
# carries the file and the number of keys only, never a key or a value.
|
|
94
|
+
def apply_at_boot!(logger)
|
|
95
|
+
reason = blocked_reason
|
|
96
|
+
return apply! unless reason
|
|
97
|
+
return if @boot_warning_logged
|
|
98
|
+
|
|
99
|
+
count = all_overrides.size
|
|
100
|
+
return if count.zero?
|
|
101
|
+
|
|
102
|
+
@boot_warning_logged = true
|
|
103
|
+
noun, verb = count == 1 ? %w[override was] : %w[overrides were]
|
|
104
|
+
logger&.warn("[Profiler] #{count} persisted environment #{noun} in #{override_file_path} " \
|
|
105
|
+
"#{verb} not applied: #{BLOCKED_REASONS.fetch(reason)}.")
|
|
106
|
+
rescue => e
|
|
107
|
+
warn "[Profiler] EnvOverrideStore: failed to check overrides at boot: #{e.message}"
|
|
108
|
+
end
|
|
109
|
+
|
|
67
110
|
def apply!
|
|
111
|
+
return if blocked_reason
|
|
112
|
+
|
|
68
113
|
overrides = load_overrides
|
|
69
114
|
restore_keys = []
|
|
70
115
|
|
|
@@ -72,6 +117,8 @@ module Profiler
|
|
|
72
117
|
value = entry.is_a?(Hash) ? entry["value"] : entry
|
|
73
118
|
original = entry.is_a?(Hash) ? entry["original"] : nil
|
|
74
119
|
|
|
120
|
+
remember_process_original(key) unless value == RESTORE_SENTINEL
|
|
121
|
+
|
|
75
122
|
case value
|
|
76
123
|
when DELETED_SENTINEL
|
|
77
124
|
ENV.delete(key)
|
|
@@ -84,8 +131,13 @@ module Profiler
|
|
|
84
131
|
end
|
|
85
132
|
|
|
86
133
|
if restore_keys.any?
|
|
87
|
-
|
|
88
|
-
|
|
134
|
+
# Read again under the lock: another writer may have changed the file since.
|
|
135
|
+
update_overrides do |current|
|
|
136
|
+
restore_keys.each do |k|
|
|
137
|
+
entry = current[k]
|
|
138
|
+
current.delete(k) if entry.is_a?(Hash) && entry["value"] == RESTORE_SENTINEL
|
|
139
|
+
end
|
|
140
|
+
end
|
|
89
141
|
end
|
|
90
142
|
rescue => e
|
|
91
143
|
warn "[Profiler] EnvOverrideStore: failed to apply overrides: #{e.message}"
|
|
@@ -105,13 +157,34 @@ module Profiler
|
|
|
105
157
|
end
|
|
106
158
|
|
|
107
159
|
def clear
|
|
108
|
-
FileUtils.rm_f(override_file_path)
|
|
160
|
+
with_lock { FileUtils.rm_f(override_file_path) }
|
|
109
161
|
rescue => e
|
|
110
162
|
warn "[Profiler] EnvOverrideStore: failed to clear: #{e.message}"
|
|
111
163
|
end
|
|
112
164
|
|
|
113
165
|
private
|
|
114
166
|
|
|
167
|
+
def remember_process_original(key)
|
|
168
|
+
@process_originals_lock.synchronize do
|
|
169
|
+
@process_originals[key] = ENV[key] unless @process_originals.key?(key)
|
|
170
|
+
end
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
# Puts a reset key back to its original value and returns whether ENV was written. Where the
|
|
174
|
+
# overrides are blocked, or the file has no entry for the key, only a key this process
|
|
175
|
+
# changed itself is touched, back to the value it had here; ENV is left alone for any other.
|
|
176
|
+
def restore_env(key, file_entry, blocked:)
|
|
177
|
+
changed_here, process_original = @process_originals_lock.synchronize do
|
|
178
|
+
[@process_originals.key?(key), @process_originals.delete(key)]
|
|
179
|
+
end
|
|
180
|
+
from_process = blocked || file_entry.nil?
|
|
181
|
+
return false if from_process && !changed_here
|
|
182
|
+
|
|
183
|
+
original = from_process ? process_original : file_entry["original"]
|
|
184
|
+
original.nil? ? ENV.delete(key) : ENV[key] = original
|
|
185
|
+
true
|
|
186
|
+
end
|
|
187
|
+
|
|
115
188
|
def original_for(overrides, key)
|
|
116
189
|
overrides.key?(key) ? overrides[key]["original"] : ENV[key]
|
|
117
190
|
end
|
|
@@ -129,10 +202,36 @@ module Profiler
|
|
|
129
202
|
{}
|
|
130
203
|
end
|
|
131
204
|
|
|
132
|
-
|
|
205
|
+
# Reads, changes and writes the file under the lock, so that the web process, its threads and
|
|
206
|
+
# the Sidekiq processes never save over each other's changes. The file is removed when the
|
|
207
|
+
# block leaves it empty.
|
|
208
|
+
def update_overrides
|
|
209
|
+
with_lock do
|
|
210
|
+
overrides = load_overrides
|
|
211
|
+
yield overrides
|
|
212
|
+
overrides.empty? ? FileUtils.rm_f(override_file_path) : save_overrides(overrides)
|
|
213
|
+
end
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
# An exclusive flock on a file next to the overrides file. Not reentrant: never nest it.
|
|
217
|
+
def with_lock
|
|
133
218
|
path = override_file_path
|
|
134
219
|
FileUtils.mkdir_p(path.dirname)
|
|
135
|
-
File.
|
|
220
|
+
File.open("#{path}.lock", File::RDWR | File::CREAT, 0o644) do |lock|
|
|
221
|
+
lock.flock(File::LOCK_EX)
|
|
222
|
+
yield
|
|
223
|
+
end
|
|
224
|
+
end
|
|
225
|
+
|
|
226
|
+
# Written to a temporary file then renamed over the old one, so that a reader never sees a
|
|
227
|
+
# half-written file, which would parse as no override at all.
|
|
228
|
+
def save_overrides(overrides)
|
|
229
|
+
path = override_file_path
|
|
230
|
+
tmp = "#{path}.#{Process.pid}.#{Thread.current.object_id}.tmp"
|
|
231
|
+
File.write(tmp, JSON.generate(overrides))
|
|
232
|
+
File.rename(tmp, path)
|
|
233
|
+
ensure
|
|
234
|
+
FileUtils.rm_f(tmp) if tmp
|
|
136
235
|
end
|
|
137
236
|
end
|
|
138
237
|
end
|
|
@@ -1,9 +1,23 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "profiler/sql_statement"
|
|
4
|
+
|
|
3
5
|
module Profiler
|
|
4
6
|
# Shared service for running EXPLAIN ANALYZE on a stored query.
|
|
5
7
|
# Used by both the HTTP API controller and the MCP explain_query tool.
|
|
8
|
+
#
|
|
9
|
+
# Only read-only statements are explained: PostgreSQL's EXPLAIN ANALYZE runs the
|
|
10
|
+
# statement, so explaining a write would replay it. The probe itself runs, for
|
|
11
|
+
# what no reading of the statement can rule out (a function with a side effect),
|
|
12
|
+
# on a connection of its own that is closed afterwards, in a transaction that is
|
|
13
|
+
# always rolled back, read-only and time-limited on PostgreSQL.
|
|
6
14
|
module ExplainRunner
|
|
15
|
+
# An ArgumentError, so callers answer it as a bad request (422, MCP error).
|
|
16
|
+
class UnsafeStatementError < ArgumentError; end
|
|
17
|
+
|
|
18
|
+
# PostgreSQL's EXPLAIN ANALYZE runs the query: a slow one is stopped after this.
|
|
19
|
+
PROBE_STATEMENT_TIMEOUT = "30s"
|
|
20
|
+
|
|
7
21
|
# @param profile_token [String]
|
|
8
22
|
# @param query_index [Integer]
|
|
9
23
|
# @return [Hash] { result:, format: "json"|"text", adapter: String }
|
|
@@ -35,11 +49,14 @@ module Profiler
|
|
|
35
49
|
|
|
36
50
|
conn = ActiveRecord::Base.connection
|
|
37
51
|
adapter = conn.adapter_name.downcase
|
|
52
|
+
refusal = SqlStatement.read_only_refusal(sql, dialect: dialect_for(adapter))
|
|
53
|
+
raise UnsafeStatementError, refusal if refusal
|
|
54
|
+
|
|
38
55
|
full_sql = reconstruct_sql(sql, binds, conn, adapter)
|
|
39
56
|
|
|
40
57
|
explain_sql, format = build_explain_statement(full_sql, adapter)
|
|
41
58
|
|
|
42
|
-
rows =
|
|
59
|
+
rows = probe(explain_sql, adapter)
|
|
43
60
|
|
|
44
61
|
result = if format == "json"
|
|
45
62
|
# PostgreSQL / MySQL return JSON in rows[0]["QUERY PLAN"] or rows[0]["EXPLAIN"]
|
|
@@ -55,21 +72,51 @@ module Profiler
|
|
|
55
72
|
private
|
|
56
73
|
|
|
57
74
|
def self.reconstruct_sql(sql, binds, conn, adapter)
|
|
58
|
-
|
|
75
|
+
SqlStatement.substitute(sql, binds, dialect: dialect_for(adapter)) { |value| conn.quote(value) }
|
|
76
|
+
end
|
|
59
77
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
78
|
+
def self.dialect_for(adapter)
|
|
79
|
+
if adapter.include?("postgresql") then :postgresql
|
|
80
|
+
elsif adapter.include?("mysql") || adapter.include?("trilogy") then :mysql
|
|
81
|
+
else :sqlite
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
# Runs on a connection of its own, opened outside the pool and closed
|
|
86
|
+
# afterwards, so that nothing the probe does to its session outlives it (an
|
|
87
|
+
# advisory lock, a setting, a LISTEN), the application's own transaction is
|
|
88
|
+
# never touched, and a full pool does not keep Explain waiting.
|
|
89
|
+
# The transaction is always rolled back. On PostgreSQL it is also read-only and
|
|
90
|
+
# time-limited, and the EXPLAIN goes by the extended protocol, which refuses
|
|
91
|
+
# more than one statement: a COMMIT slipped in could not end the read-only
|
|
92
|
+
# transaction. Neither MySQL's EXPLAIN nor SQLite's EXPLAIN QUERY PLAN runs the
|
|
93
|
+
# statement.
|
|
94
|
+
def self.probe(explain_sql, adapter)
|
|
95
|
+
conn = open_probe_connection
|
|
96
|
+
begin
|
|
97
|
+
rows = nil
|
|
98
|
+
conn.transaction do
|
|
99
|
+
if adapter.include?("postgresql")
|
|
100
|
+
conn.execute("SET TRANSACTION READ ONLY")
|
|
101
|
+
conn.execute("SET LOCAL statement_timeout = #{conn.quote(PROBE_STATEMENT_TIMEOUT)}")
|
|
102
|
+
rows = conn.raw_connection.exec_params(explain_sql, []).to_a
|
|
103
|
+
else
|
|
104
|
+
rows = conn.exec_query(explain_sql, "EXPLAIN").to_a
|
|
105
|
+
end
|
|
106
|
+
raise ActiveRecord::Rollback
|
|
64
107
|
end
|
|
65
|
-
|
|
108
|
+
rows
|
|
109
|
+
ensure
|
|
110
|
+
conn.disconnect!
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
def self.open_probe_connection
|
|
115
|
+
db_config = ActiveRecord::Base.connection_pool.db_config
|
|
116
|
+
if db_config.respond_to?(:new_connection) # Active Record 7.2 and later
|
|
117
|
+
db_config.new_connection
|
|
66
118
|
else
|
|
67
|
-
|
|
68
|
-
result = sql.dup
|
|
69
|
-
binds.each do |value|
|
|
70
|
-
result = result.sub("?", conn.quote(value))
|
|
71
|
-
end
|
|
72
|
-
result
|
|
119
|
+
ActiveRecord::Base.public_send(db_config.adapter_method, db_config.configuration_hash)
|
|
73
120
|
end
|
|
74
121
|
end
|
|
75
122
|
|
data/lib/profiler/mcp/server.rb
CHANGED
|
@@ -139,7 +139,7 @@ module Profiler
|
|
|
139
139
|
),
|
|
140
140
|
define_tool(
|
|
141
141
|
name: "explain_query",
|
|
142
|
-
description: "Run EXPLAIN ANALYZE on a specific query from a profile. Returns the query execution plan with cost and row estimates. Only available in development/test environments.",
|
|
142
|
+
description: "Run EXPLAIN ANALYZE on a specific query from a profile. Returns the query execution plan with cost and row estimates. Only read-only statements (SELECT, WITH ... SELECT, TABLE, VALUES) are explained, in a transaction always rolled back; other queries return an error. Only available in development/test environments.",
|
|
143
143
|
input_schema: {
|
|
144
144
|
properties: {
|
|
145
145
|
token: { type: "string", description: "Profile token, or 'latest' for the most recent profile (required)" },
|
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "../slave_support"
|
|
4
|
-
require_relative "../../redaction"
|
|
5
4
|
|
|
6
5
|
require "uri"
|
|
7
6
|
|
|
@@ -24,11 +23,10 @@ module Profiler
|
|
|
24
23
|
return [{ type: "text", text: "No active override for #{key}." }]
|
|
25
24
|
end
|
|
26
25
|
|
|
27
|
-
|
|
28
|
-
Profiler.env_override_store.reset(key)
|
|
26
|
+
restored = Profiler.env_override_store.reset(key)
|
|
29
27
|
|
|
30
|
-
|
|
31
|
-
[{ type: "text", text: "Reset #{key}
|
|
28
|
+
done = restored ? "original value restored" : "ENV left unchanged"
|
|
29
|
+
[{ type: "text", text: "Reset #{key}: override removed, #{done} in this process." }]
|
|
32
30
|
end
|
|
33
31
|
end
|
|
34
32
|
end
|
data/lib/profiler/railtie.rb
CHANGED
|
@@ -6,10 +6,6 @@ module Profiler
|
|
|
6
6
|
class Railtie < Rails::Railtie
|
|
7
7
|
config.profiler = ActiveSupport::OrderedOptions.new
|
|
8
8
|
|
|
9
|
-
initializer "profiler.apply_env_overrides" do
|
|
10
|
-
Profiler.env_override_store.apply!
|
|
11
|
-
end
|
|
12
|
-
|
|
13
9
|
initializer "profiler.set_configs" do |app|
|
|
14
10
|
# Set default configuration for Rails environment
|
|
15
11
|
Profiler.configure do |config|
|
|
@@ -103,6 +99,13 @@ module Profiler
|
|
|
103
99
|
end
|
|
104
100
|
end
|
|
105
101
|
|
|
102
|
+
# After the application's config/initializers, so that `enabled` is the application's own.
|
|
103
|
+
# Declared last: an initializer without `after:` runs after the one declared before it, so
|
|
104
|
+
# one declared below this one would move past the application's initializers too.
|
|
105
|
+
initializer "profiler.apply_env_overrides", after: :load_config_initializers do
|
|
106
|
+
Profiler.env_override_store.apply_at_boot!(Rails.logger)
|
|
107
|
+
end
|
|
108
|
+
|
|
106
109
|
console do
|
|
107
110
|
next unless Profiler.configuration.enabled && Profiler.configuration.track_console
|
|
108
111
|
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "strscan"
|
|
4
|
+
|
|
5
|
+
module Profiler
|
|
6
|
+
# Reads a stored SQL statement the way the database would, one token at a time:
|
|
7
|
+
# comments, string literals and quoted identifiers are never mistaken for keywords
|
|
8
|
+
# or placeholders. Used by ExplainRunner to refuse anything but a read, and to put
|
|
9
|
+
# the bind values back into the statement.
|
|
10
|
+
module SqlStatement
|
|
11
|
+
# The first keyword of a statement EXPLAIN may run.
|
|
12
|
+
READ_ONLY_VERBS = %w[SELECT WITH TABLE VALUES].freeze
|
|
13
|
+
|
|
14
|
+
# Keywords that make a read-only verb write or lock: a CTE that writes,
|
|
15
|
+
# SELECT ... INTO (and MySQL's INTO OUTFILE), FOR UPDATE / FOR SHARE,
|
|
16
|
+
# LOCK IN SHARE MODE. Matched as whole words outside literals, so a column
|
|
17
|
+
# named `updated_at` is fine and an unquoted column named `share` is refused.
|
|
18
|
+
WRITE_KEYWORDS = %w[INSERT UPDATE DELETE MERGE TRUNCATE DROP ALTER CREATE INTO LOCK SHARE].freeze
|
|
19
|
+
|
|
20
|
+
Token = Struct.new(:type, :text)
|
|
21
|
+
|
|
22
|
+
# 'it''s' everywhere; 'it\'s' too where a backslash escapes. An unterminated
|
|
23
|
+
# literal runs to the end: the database refuses such a statement anyway.
|
|
24
|
+
PLAIN_LITERAL = /[Nn]?'(?:[^']|'')*'?/m
|
|
25
|
+
BACKSLASH_LITERAL = /[Nn]?'(?:[^'\\]|''|\\.)*'?/m
|
|
26
|
+
ESCAPE_LITERAL = /[Ee]'(?:[^'\\]|''|\\.)*'?/m
|
|
27
|
+
|
|
28
|
+
# Dialects differ in what a literal or a comment is: a backslash escapes a quote
|
|
29
|
+
# in every MySQL string but only in PostgreSQL's E'...' strings, dollar quoting
|
|
30
|
+
# is PostgreSQL's, backticks quote identifiers in MySQL and SQLite, MySQL starts
|
|
31
|
+
# a comment with `#` and needs a space after `--`, and only PostgreSQL nests
|
|
32
|
+
# block comments.
|
|
33
|
+
DIALECTS = %i[postgresql mysql sqlite].freeze
|
|
34
|
+
|
|
35
|
+
# @param dialect [Symbol] one of DIALECTS
|
|
36
|
+
# @return [String, nil] why the statement must not be explained, or nil when it reads only
|
|
37
|
+
def self.read_only_refusal(sql, dialect:)
|
|
38
|
+
all = tokenize(sql, dialect)
|
|
39
|
+
if dialect == :mysql && all.any? { |t| t.type == :comment && t.text.start_with?("/*!") }
|
|
40
|
+
return refusal("it holds a MySQL executable comment (/*! ... */)")
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
tokens = all.reject { |t| %i[space comment].include?(t.type) }
|
|
44
|
+
first = tokens.find { |t| t.type != :punct || t.text != "(" }
|
|
45
|
+
return refusal("it is empty") if first.nil?
|
|
46
|
+
|
|
47
|
+
verb = first.text.upcase
|
|
48
|
+
return refusal("it starts with #{verb}") unless first.type == :word && READ_ONLY_VERBS.include?(verb)
|
|
49
|
+
|
|
50
|
+
tokens.each_with_index do |token, i|
|
|
51
|
+
if token.type == :word && WRITE_KEYWORDS.include?(token.text.upcase)
|
|
52
|
+
return refusal("it contains #{token.text.upcase}")
|
|
53
|
+
end
|
|
54
|
+
if token.type == :punct && token.text == ";" && i != tokens.size - 1
|
|
55
|
+
return refusal("it holds more than one statement")
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
nil
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# Replaces each placeholder by its quoted value in a single pass: `$1`, `$10`...
|
|
62
|
+
# by number (PostgreSQL), or `?` in order (MySQL, SQLite). Placeholders inside
|
|
63
|
+
# literals or comments stay as they are, and a substituted value is never read
|
|
64
|
+
# again. A placeholder without a value is left in place.
|
|
65
|
+
def self.substitute(sql, binds, dialect:, "e)
|
|
66
|
+
return sql if binds.empty?
|
|
67
|
+
|
|
68
|
+
position = 0
|
|
69
|
+
tokenize(sql, dialect).map do |token|
|
|
70
|
+
index = case token.type
|
|
71
|
+
when :numbered then token.text[1..].to_i - 1
|
|
72
|
+
when :question then (position += 1) - 1
|
|
73
|
+
end
|
|
74
|
+
index && index >= 0 && index < binds.size ? quote.call(binds[index]) : token.text
|
|
75
|
+
end.join
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
def self.refusal(reason)
|
|
79
|
+
"EXPLAIN refused: only read-only statements (SELECT, WITH ... SELECT, TABLE, VALUES) " \
|
|
80
|
+
"are explained, and #{reason}. PostgreSQL's EXPLAIN ANALYZE runs the statement it explains."
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
# Splits the statement into tokens whose texts, joined, give it back unchanged.
|
|
84
|
+
def self.tokenize(sql, dialect)
|
|
85
|
+
raise ArgumentError, "unknown SQL dialect: #{dialect.inspect}" unless DIALECTS.include?(dialect)
|
|
86
|
+
|
|
87
|
+
s = StringScanner.new(sql.to_s)
|
|
88
|
+
tokens = []
|
|
89
|
+
until s.eos?
|
|
90
|
+
tokens << if s.scan(/\s+/)
|
|
91
|
+
Token.new(:space, s.matched)
|
|
92
|
+
elsif s.scan(dialect == :mysql ? /(?:--(?=\s|\z)|#)[^\n]*/ : /--[^\n]*/)
|
|
93
|
+
Token.new(:comment, s.matched)
|
|
94
|
+
elsif s.check(%r{/\*})
|
|
95
|
+
Token.new(:comment, scan_block_comment(s, nested: dialect == :postgresql))
|
|
96
|
+
elsif s.scan(dialect == :mysql ? BACKSLASH_LITERAL : PLAIN_LITERAL) ||
|
|
97
|
+
(dialect == :postgresql && s.scan(ESCAPE_LITERAL))
|
|
98
|
+
Token.new(:literal, s.matched)
|
|
99
|
+
elsif s.scan(/"(?:[^"]|"")*"?/m) || (dialect != :postgresql && s.scan(/`(?:[^`]|``)*`?/m))
|
|
100
|
+
Token.new(:identifier, s.matched)
|
|
101
|
+
elsif dialect == :postgresql && s.scan(/\$\d+/)
|
|
102
|
+
Token.new(:numbered, s.matched)
|
|
103
|
+
elsif dialect == :postgresql && s.scan(/\$([A-Za-z_][A-Za-z0-9_]*)?\$/)
|
|
104
|
+
tag = s.matched
|
|
105
|
+
body = s.scan_until(/#{Regexp.escape(tag)}/) || s.rest.tap { s.terminate }
|
|
106
|
+
Token.new(:literal, tag + body)
|
|
107
|
+
elsif dialect != :postgresql && s.scan(/\?/)
|
|
108
|
+
Token.new(:question, s.matched)
|
|
109
|
+
elsif s.scan(/[A-Za-z_][A-Za-z0-9_$]*/)
|
|
110
|
+
Token.new(:word, s.matched)
|
|
111
|
+
else
|
|
112
|
+
Token.new(:punct, s.getch)
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
tokens
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
# An unterminated block comment runs to the end.
|
|
119
|
+
def self.scan_block_comment(s, nested:)
|
|
120
|
+
text = +""
|
|
121
|
+
depth = 0
|
|
122
|
+
until s.eos?
|
|
123
|
+
if s.scan(%r{/\*})
|
|
124
|
+
depth += 1 if nested || depth.zero?
|
|
125
|
+
elsif s.scan(%r{\*/})
|
|
126
|
+
depth -= 1
|
|
127
|
+
else
|
|
128
|
+
s.getch
|
|
129
|
+
end
|
|
130
|
+
text << s.matched
|
|
131
|
+
break if depth.zero?
|
|
132
|
+
end
|
|
133
|
+
text
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
private_class_method :refusal, :tokenize, :scan_block_comment
|
|
137
|
+
end
|
|
138
|
+
end
|
data/lib/profiler/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rails-profiler
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.30.
|
|
4
|
+
version: 0.30.9
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Sébastien Duplessy
|
|
@@ -210,6 +210,7 @@ files:
|
|
|
210
210
|
- lib/profiler/models/timeline_event.rb
|
|
211
211
|
- lib/profiler/railtie.rb
|
|
212
212
|
- lib/profiler/redaction.rb
|
|
213
|
+
- lib/profiler/sql_statement.rb
|
|
213
214
|
- lib/profiler/sse/bus.rb
|
|
214
215
|
- lib/profiler/sse/event_bus.rb
|
|
215
216
|
- lib/profiler/sse/redis_event_bus.rb
|