rails_pulse 0.4.0 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +27 -2
- data/app/models/rails_pulse/cards/base.rb +4 -4
- data/app/services/rails_pulse/sql_query_normalizer.rb +84 -6
- data/app/views/layouts/rails_pulse/application.html.erb +1 -1
- data/lib/rails_pulse/cleanup_service.rb +177 -90
- data/lib/rails_pulse/version.rb +1 -1
- metadata +11 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b54f69a078537dd6dabc4074623c3deaf4b0123f4c229b57c706583f104d1b62
|
|
4
|
+
data.tar.gz: 2cd90650cc8ceb9cf15dac48c4d4695113c73a52fdb4661153fa5c45cbf3b378
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 125303510a989e6de226b732d14fd76d001b5641feb4f93bd8ae5daf07ec73b2328c022097c36be6b4153cd270a9344bf8d4f14e1364843b96fdabd07beb0aab
|
|
7
|
+
data.tar.gz: d262a5d599007772f1cccc087b6166fd3a151f93f3cdf990bcd37b3ffed6dc6df3aa785a959ae0ae92ac56df545d9a299aed1ec41e3f3d605adb868ce9bb22d0
|
data/CHANGELOG.md
CHANGED
|
@@ -5,9 +5,34 @@ All notable changes to Rails Pulse will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
-
##
|
|
8
|
+
## 0.4.2 - 2025-10-01
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
### Changed
|
|
11
|
+
|
|
12
|
+
- Patch releases can be cut from a stable branch (`0-4-stable` for 0.4.x) while `main` carries the next release; `bin/release` and CI accept those branches.
|
|
13
|
+
|
|
14
|
+
### Fixed
|
|
15
|
+
|
|
16
|
+
- **Cleanup deletes in bounded batches.** A table far over its limit was trimmed with one statement whose size grew with the overage, which could exhaust the database server's memory on a busy app; every delete now removes at most 5,000 rows per statement. (#322)
|
|
17
|
+
- **One failed cleanup stage no longer stops the rest.** The remaining tables are still cleaned and the error is raised once they have finished. (#322)
|
|
18
|
+
|
|
19
|
+
## [0.4.1] - 2026-09-23
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- A test that fails, naming the key, when a default in `RailsPulse::Configuration` and the install template drift apart. (#272)
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
|
|
27
|
+
- Documented what a tracked request costs, measured against the bounded writer: about 25 microseconds of request-thread overhead per request on SQLite and PostgreSQL. (#275)
|
|
28
|
+
- The release gate and CI now fail if anything under `app/` branches on `Rails.env`, so a code path the test suite cannot reach can no longer ship. (#273)
|
|
29
|
+
- Loosened the `ransack` dependency from `~> 4.0` to `>= 4.0, < 6` so host apps can upgrade to ransack 5 without dropping Rails Pulse.
|
|
30
|
+
- `rake test` now refuses to run when the dummy database was last set up for a different adapter, instead of loading that adapter's `schema.rb` and failing two index tests. (#276)
|
|
31
|
+
|
|
32
|
+
### Fixed
|
|
33
|
+
|
|
34
|
+
- SQL normalization no longer raises `Regexp::TimeoutError` on long queries, which could fail the whole request when N+1 detection ran during collection.
|
|
35
|
+
- **Metric card headline numbers now honour the global tag filter.** Disabling a tag changed each card's sparkline but not the number beside it, so the two disagreed. Both are now computed over the same summaries. (#274)
|
|
11
36
|
|
|
12
37
|
## [0.4.0] - 2026-09-22
|
|
13
38
|
|
|
@@ -62,10 +62,10 @@ module RailsPulse
|
|
|
62
62
|
period_start: range_start..now
|
|
63
63
|
)
|
|
64
64
|
|
|
65
|
-
#
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
65
|
+
# The same tag filter the sparkline applies, so the headline number and
|
|
66
|
+
# the sparkline beside it are computed over the same summaries. Cards
|
|
67
|
+
# receive the filter as instance variables; nil means unfiltered.
|
|
68
|
+
query = query.with_tag_filters(@disabled_tags || [], @show_non_tagged != false)
|
|
69
69
|
|
|
70
70
|
# Filter to specific resource if provided
|
|
71
71
|
query = query.where(summarizable_id: subject_id) if subject_id
|
|
@@ -1,5 +1,15 @@
|
|
|
1
1
|
module RailsPulse
|
|
2
2
|
class SqlQueryNormalizer
|
|
3
|
+
# Above this length, skip the regex-based literal scan in favor of a
|
|
4
|
+
# slower-per-char but backtracking-free manual scan. Regex matching here
|
|
5
|
+
# is linear (benchmarked at ~0.4s for an 8,000,000-char literal under a
|
|
6
|
+
# 1s Regexp.timeout), but a host app can configure Regexp.timeout far
|
|
7
|
+
# below that, and very long queries are rare enough that the fast path
|
|
8
|
+
# isn't worth the risk. 100_000 chars leaves generous headroom below
|
|
9
|
+
# where even an aggressive timeout config could plausibly trip — see
|
|
10
|
+
# issue #286.
|
|
11
|
+
LONG_QUERY_THRESHOLD = 100_000
|
|
12
|
+
|
|
3
13
|
# Smart normalization: preserve table/column names, replace only literal values
|
|
4
14
|
def self.normalize(query_string)
|
|
5
15
|
new(query_string).normalize
|
|
@@ -72,6 +82,14 @@ module RailsPulse
|
|
|
72
82
|
def replace_literal_values(query)
|
|
73
83
|
normalized = query.dup
|
|
74
84
|
|
|
85
|
+
# Replace string literals (single quotes) FIRST so a very long literal
|
|
86
|
+
# collapses to a single "?" before any other regex in this method has
|
|
87
|
+
# to scan across it — see LONG_QUERY_THRESHOLD and issue #286.
|
|
88
|
+
normalized = replace_quoted_literals(normalized, "'")
|
|
89
|
+
|
|
90
|
+
# Replace double-quoted string literals (not protected identifiers)
|
|
91
|
+
normalized = replace_quoted_literals(normalized, '"')
|
|
92
|
+
|
|
75
93
|
# Replace floating-point numbers FIRST (before integers) to avoid double replacement
|
|
76
94
|
normalized = normalized.gsub(/(?<![a-zA-Z_])\b\d+\.\d+\b(?![a-zA-Z_])/, "?")
|
|
77
95
|
|
|
@@ -79,18 +97,78 @@ module RailsPulse
|
|
|
79
97
|
# Negative lookbehind/lookahead prevents replacing numbers in table/column names
|
|
80
98
|
normalized = normalized.gsub(/(?<![a-zA-Z_])\b\d+\b(?![a-zA-Z_])/, "?")
|
|
81
99
|
|
|
82
|
-
# Replace string literals (single quotes)
|
|
83
|
-
normalized = normalized.gsub(/'(?:[^']|'')*'/, "?")
|
|
84
|
-
|
|
85
|
-
# Replace double-quoted string literals (not protected identifiers)
|
|
86
|
-
normalized = normalized.gsub(/"(?:[^"]|"")*"/, "?")
|
|
87
|
-
|
|
88
100
|
# Handle boolean literals
|
|
89
101
|
normalized = normalized.gsub(/\b(true|false)\b/i, "?")
|
|
90
102
|
|
|
91
103
|
normalized
|
|
92
104
|
end
|
|
93
105
|
|
|
106
|
+
# Replaces quote_char-delimited string literals with a single placeholder.
|
|
107
|
+
# Uses a regex for the common case (fast — see benchmarks in
|
|
108
|
+
# LONG_QUERY_THRESHOLD's comment) and falls back to a manual scan for
|
|
109
|
+
# very long queries, where that regex risks Regexp::TimeoutError under
|
|
110
|
+
# an aggressively configured Regexp.timeout — see issue #286.
|
|
111
|
+
def replace_quoted_literals(query, quote_char)
|
|
112
|
+
if query.length < LONG_QUERY_THRESHOLD
|
|
113
|
+
pattern = quote_char == "'" ? /'(?:[^']|'')*'/ : /"(?:[^"]|"")*"/
|
|
114
|
+
query.gsub(pattern, "?")
|
|
115
|
+
else
|
|
116
|
+
scan_quoted_literals(query, quote_char)
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
# Linear, non-regex scan used above LONG_QUERY_THRESHOLD. A quote left
|
|
121
|
+
# unterminated for the rest of the string is left untouched rather than
|
|
122
|
+
# rescanned from every possible start position, which is what would make
|
|
123
|
+
# a naive manual reimplementation quadratic.
|
|
124
|
+
def scan_quoted_literals(query, quote_char)
|
|
125
|
+
result = +""
|
|
126
|
+
i = 0
|
|
127
|
+
len = query.length
|
|
128
|
+
|
|
129
|
+
while i < len
|
|
130
|
+
char = query[i]
|
|
131
|
+
|
|
132
|
+
unless char == quote_char
|
|
133
|
+
result << char
|
|
134
|
+
i += 1
|
|
135
|
+
next
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
close_index = find_closing_quote(query, i + 1, quote_char)
|
|
139
|
+
|
|
140
|
+
if close_index
|
|
141
|
+
result << "?"
|
|
142
|
+
i = close_index + 1
|
|
143
|
+
else
|
|
144
|
+
result << query[i..]
|
|
145
|
+
break
|
|
146
|
+
end
|
|
147
|
+
end
|
|
148
|
+
|
|
149
|
+
result
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
# Returns the index of the unescaped closing quote_char starting the
|
|
153
|
+
# search at `start`, treating a doubled quote_char (e.g. '') as an
|
|
154
|
+
# escaped literal quote rather than a terminator. Returns nil if the
|
|
155
|
+
# string is never closed.
|
|
156
|
+
def find_closing_quote(query, start, quote_char)
|
|
157
|
+
j = start
|
|
158
|
+
len = query.length
|
|
159
|
+
|
|
160
|
+
while j < len
|
|
161
|
+
if query[j] == quote_char
|
|
162
|
+
return j unless query[j + 1] == quote_char
|
|
163
|
+
j += 2
|
|
164
|
+
else
|
|
165
|
+
j += 1
|
|
166
|
+
end
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
nil
|
|
170
|
+
end
|
|
171
|
+
|
|
94
172
|
def handle_special_constructs(query)
|
|
95
173
|
normalized = normalize_in_clauses(query)
|
|
96
174
|
normalized.gsub(/\bBETWEEN\s+\?\s+AND\s+\?/i, "BETWEEN ? AND ?")
|
|
@@ -69,7 +69,7 @@
|
|
|
69
69
|
</div>
|
|
70
70
|
|
|
71
71
|
<div class="flex items-center gap">
|
|
72
|
-
<%= link_to rails_pulse.root_path, class: "flex items-center gap mie-2" do %>
|
|
72
|
+
<%= link_to rails_pulse.root_path, class: "flex items-center gap mie-2", title: "Rails Pulse v#{RailsPulse::VERSION}" do %>
|
|
73
73
|
<span class="font-bold">Rails Pulse</span>
|
|
74
74
|
<% end %>
|
|
75
75
|
<nav class="flex items-center gap text-sm text-subtle show@md" style="--column-gap: 1rem">
|
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
module RailsPulse
|
|
2
2
|
class CleanupService
|
|
3
|
+
# Rows removed per DELETE statement. Statement size, lock time and
|
|
4
|
+
# transaction length stay flat however far a table has outgrown its limits.
|
|
5
|
+
BATCH_SIZE = 5_000
|
|
6
|
+
|
|
7
|
+
# Parent rows handled per round when their child rows must go first. The
|
|
8
|
+
# parent ids are held in Ruby and sent back as a list, so this bounds it.
|
|
9
|
+
PARENT_BATCH_SIZE = 1_000
|
|
10
|
+
|
|
3
11
|
def self.perform
|
|
4
12
|
new.perform
|
|
5
13
|
end
|
|
@@ -11,6 +19,7 @@ module RailsPulse
|
|
|
11
19
|
count_based: {},
|
|
12
20
|
total_deleted: 0
|
|
13
21
|
}
|
|
22
|
+
@failures = []
|
|
14
23
|
end
|
|
15
24
|
|
|
16
25
|
def perform
|
|
@@ -23,11 +32,65 @@ module RailsPulse
|
|
|
23
32
|
perform_summary_cleanup
|
|
24
33
|
|
|
25
34
|
log_cleanup_summary
|
|
35
|
+
raise @failures.first[:error] if @failures.any?
|
|
36
|
+
|
|
26
37
|
@stats
|
|
27
38
|
end
|
|
28
39
|
|
|
29
40
|
private
|
|
30
41
|
|
|
42
|
+
# Runs one table's cleanup and records its count. A failure is logged and
|
|
43
|
+
# held until every stage has run, so one table that cannot be cleaned does
|
|
44
|
+
# not stop the others; each stage removes its own child rows or skips rows
|
|
45
|
+
# that still have them, so none depends on an earlier stage succeeding.
|
|
46
|
+
def run_stage(group, name)
|
|
47
|
+
@stats[group][name] = yield
|
|
48
|
+
rescue StandardError => e
|
|
49
|
+
@stats[group][name] = 0
|
|
50
|
+
@failures << { stage: "#{name} (#{group})", error: e }
|
|
51
|
+
RailsPulse.logger.error "Cleanup stage #{name} (#{group}) failed: #{e.class}: #{e.message}"
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
# Deletes the relation's rows in statements of at most BATCH_SIZE rows,
|
|
55
|
+
# choosing each batch inside the database. `order_column` decides which
|
|
56
|
+
# rows go first; `limit` stops after that many rows.
|
|
57
|
+
def delete_in_batches(relation, order_column: nil, limit: nil)
|
|
58
|
+
relation = relation.order(order_column => :asc) if order_column
|
|
59
|
+
deleted = 0
|
|
60
|
+
|
|
61
|
+
loop do
|
|
62
|
+
batch_size = limit ? [ BATCH_SIZE, limit - deleted ].min : BATCH_SIZE
|
|
63
|
+
break if batch_size <= 0
|
|
64
|
+
|
|
65
|
+
count = relation.limit(batch_size).delete_all
|
|
66
|
+
deleted += count
|
|
67
|
+
break if count < batch_size
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
deleted
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
# Deletes parent rows oldest first, removing each batch's child rows before
|
|
74
|
+
# the parents — there is no ON DELETE CASCADE on the foreign keys. Returns
|
|
75
|
+
# the number of parent rows deleted.
|
|
76
|
+
def delete_parents_in_batches(parents, order_column:, children:, foreign_key:, limit: nil)
|
|
77
|
+
deleted = 0
|
|
78
|
+
|
|
79
|
+
loop do
|
|
80
|
+
batch_size = limit ? [ PARENT_BATCH_SIZE, limit - deleted ].min : PARENT_BATCH_SIZE
|
|
81
|
+
break if batch_size <= 0
|
|
82
|
+
|
|
83
|
+
ids = parents.order(order_column => :asc).limit(batch_size).pluck(:id)
|
|
84
|
+
break if ids.empty?
|
|
85
|
+
|
|
86
|
+
delete_in_batches(children.where(foreign_key => ids))
|
|
87
|
+
deleted += parents.where(id: ids).delete_all
|
|
88
|
+
break if ids.size < batch_size
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
deleted
|
|
92
|
+
end
|
|
93
|
+
|
|
31
94
|
def cleanup_enabled?
|
|
32
95
|
@config.archiving_enabled
|
|
33
96
|
end
|
|
@@ -41,15 +104,15 @@ module RailsPulse
|
|
|
41
104
|
RailsPulse.logger.info "Time-based cleanup: removing records older than #{cutoff_time}"
|
|
42
105
|
|
|
43
106
|
# Clean up in order that respects foreign key constraints
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
107
|
+
run_stage(:time_based, :operations) { cleanup_operations_by_time(cutoff_time) }
|
|
108
|
+
run_stage(:time_based, :job_runs) { cleanup_job_runs_by_time(cutoff_time) }
|
|
109
|
+
run_stage(:time_based, :requests) { cleanup_requests_by_time(cutoff_time) }
|
|
110
|
+
run_stage(:time_based, :queries) { cleanup_queries_by_time(cutoff_time) }
|
|
111
|
+
run_stage(:time_based, :routes) { cleanup_routes_by_time(cutoff_time) }
|
|
112
|
+
run_stage(:time_based, :jobs) { cleanup_jobs_by_time(cutoff_time) }
|
|
50
113
|
if exception_tables_exist?
|
|
51
|
-
|
|
52
|
-
|
|
114
|
+
run_stage(:time_based, :exception_occurrences) { cleanup_exception_occurrences_by_time(cutoff_time) }
|
|
115
|
+
run_stage(:time_based, :exception_groups) { cleanup_orphaned_exception_groups }
|
|
53
116
|
end
|
|
54
117
|
end
|
|
55
118
|
|
|
@@ -58,28 +121,20 @@ module RailsPulse
|
|
|
58
121
|
|
|
59
122
|
RailsPulse.logger.info "Count-based cleanup: enforcing table record limits"
|
|
60
123
|
|
|
61
|
-
# Only delete records from periods that have already been summarized.
|
|
62
|
-
# This prevents count-based cleanup from removing data before the summary
|
|
63
|
-
# job has had a chance to aggregate it.
|
|
64
|
-
cutoff = summarized_cutoff
|
|
65
|
-
|
|
66
|
-
# Operations: only delete those before the summarization cutoff
|
|
67
|
-
ops_scope = cutoff ? RailsPulse::Operation.where("occurred_at < ?", cutoff) : RailsPulse::Operation.none
|
|
68
|
-
|
|
69
124
|
# Clean up in order that respects foreign key constraints
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
125
|
+
run_stage(:count_based, :operations) { cleanup_operations_by_count }
|
|
126
|
+
run_stage(:count_based, :job_runs) { cleanup_job_runs_by_count }
|
|
127
|
+
run_stage(:count_based, :requests) { cleanup_requests_by_count }
|
|
128
|
+
run_stage(:count_based, :queries) { cleanup_queries_by_count }
|
|
129
|
+
run_stage(:count_based, :routes) { cleanup_routes_by_count }
|
|
130
|
+
run_stage(:count_based, :jobs) { cleanup_jobs_by_count }
|
|
76
131
|
if exception_tables_exist?
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
132
|
+
run_stage(:count_based, :exception_occurrences) { cleanup_exception_occurrences_by_count }
|
|
133
|
+
run_stage(:count_based, :exception_groups) { cleanup_exception_groups_by_count }
|
|
134
|
+
run_stage(:count_based, :orphaned_exception_groups) { cleanup_orphaned_exception_groups }
|
|
80
135
|
end
|
|
81
136
|
if deployments_table_exists?
|
|
82
|
-
|
|
137
|
+
run_stage(:count_based, :deployments) { cleanup_deployments_by_count }
|
|
83
138
|
end
|
|
84
139
|
end
|
|
85
140
|
|
|
@@ -101,56 +156,77 @@ module RailsPulse
|
|
|
101
156
|
return 0 unless max_records
|
|
102
157
|
|
|
103
158
|
relation = scope || model_class
|
|
104
|
-
|
|
105
|
-
return 0 if
|
|
159
|
+
overage = relation.count - max_records
|
|
160
|
+
return 0 if overage <= 0
|
|
106
161
|
|
|
107
|
-
|
|
108
|
-
ids_to_delete = relation.order(order_column => :asc).limit(records_to_delete).pluck(:id)
|
|
109
|
-
model_class.where(id: ids_to_delete).delete_all
|
|
110
|
-
records_to_delete
|
|
162
|
+
delete_in_batches(relation, order_column: order_column, limit: overage)
|
|
111
163
|
end
|
|
112
164
|
|
|
113
165
|
# Time-based cleanup methods
|
|
114
166
|
|
|
115
167
|
def cleanup_operations_by_time(cutoff_time)
|
|
116
|
-
RailsPulse::Operation.where("occurred_at < ?", cutoff_time)
|
|
168
|
+
delete_in_batches(RailsPulse::Operation.where("occurred_at < ?", cutoff_time), order_column: :occurred_at)
|
|
117
169
|
end
|
|
118
170
|
|
|
119
171
|
def cleanup_requests_by_time(cutoff_time)
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
172
|
+
delete_parents_in_batches(
|
|
173
|
+
RailsPulse::Request.where("occurred_at < ?", cutoff_time),
|
|
174
|
+
order_column: :occurred_at,
|
|
175
|
+
children: RailsPulse::Operation,
|
|
176
|
+
foreign_key: :request_id
|
|
177
|
+
)
|
|
123
178
|
end
|
|
124
179
|
|
|
125
180
|
def cleanup_queries_by_time(cutoff_time)
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
181
|
+
delete_in_batches(
|
|
182
|
+
RailsPulse::Query
|
|
183
|
+
.where("created_at < ?", cutoff_time)
|
|
184
|
+
.where("NOT EXISTS (SELECT 1 FROM rails_pulse_operations WHERE rails_pulse_operations.query_id = rails_pulse_queries.id)")
|
|
185
|
+
)
|
|
130
186
|
end
|
|
131
187
|
|
|
132
188
|
def cleanup_routes_by_time(cutoff_time)
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
189
|
+
delete_in_batches(
|
|
190
|
+
RailsPulse::Route
|
|
191
|
+
.where("created_at < ?", cutoff_time)
|
|
192
|
+
.where("NOT EXISTS (SELECT 1 FROM rails_pulse_requests WHERE rails_pulse_requests.route_id = rails_pulse_routes.id)")
|
|
193
|
+
)
|
|
137
194
|
end
|
|
138
195
|
|
|
139
196
|
def cleanup_job_runs_by_time(cutoff_time)
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
197
|
+
delete_parents_in_batches(
|
|
198
|
+
RailsPulse::JobRun.where("occurred_at < ?", cutoff_time),
|
|
199
|
+
order_column: :occurred_at,
|
|
200
|
+
children: RailsPulse::Operation,
|
|
201
|
+
foreign_key: :job_run_id
|
|
202
|
+
)
|
|
143
203
|
end
|
|
144
204
|
|
|
145
205
|
def cleanup_jobs_by_time(cutoff_time)
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
206
|
+
delete_in_batches(
|
|
207
|
+
RailsPulse::Job
|
|
208
|
+
.where("created_at < ?", cutoff_time)
|
|
209
|
+
.where("NOT EXISTS (SELECT 1 FROM rails_pulse_job_runs WHERE rails_pulse_job_runs.job_id = rails_pulse_jobs.id)")
|
|
210
|
+
)
|
|
150
211
|
end
|
|
151
212
|
|
|
152
213
|
# Count-based cleanup methods (complex cases that need custom scoping)
|
|
153
214
|
|
|
215
|
+
# Only operations from periods that have already been summarized are
|
|
216
|
+
# deleted, so count-based cleanup never removes data the summary job has
|
|
217
|
+
# not yet aggregated.
|
|
218
|
+
def cleanup_operations_by_count
|
|
219
|
+
cutoff = summarized_cutoff
|
|
220
|
+
return 0 unless cutoff
|
|
221
|
+
|
|
222
|
+
cleanup_by_count(
|
|
223
|
+
RailsPulse::Operation,
|
|
224
|
+
:rails_pulse_operations,
|
|
225
|
+
order_column: :occurred_at,
|
|
226
|
+
scope: RailsPulse::Operation.where("occurred_at < ?", cutoff)
|
|
227
|
+
)
|
|
228
|
+
end
|
|
229
|
+
|
|
154
230
|
def cleanup_requests_by_count
|
|
155
231
|
max_records = @config.max_table_records[:rails_pulse_requests]
|
|
156
232
|
return 0 unless max_records
|
|
@@ -159,36 +235,32 @@ module RailsPulse
|
|
|
159
235
|
cutoff = summarized_cutoff
|
|
160
236
|
return 0 unless cutoff
|
|
161
237
|
|
|
162
|
-
|
|
163
|
-
return 0 if
|
|
164
|
-
|
|
165
|
-
records_to_delete = current_count - max_records
|
|
166
|
-
ids_to_delete = RailsPulse::Request
|
|
167
|
-
.where("occurred_at < ?", cutoff)
|
|
168
|
-
.order(occurred_at: :asc)
|
|
169
|
-
.limit(records_to_delete)
|
|
170
|
-
.pluck(:id)
|
|
171
|
-
return 0 if ids_to_delete.empty?
|
|
238
|
+
overage = RailsPulse::Request.count - max_records
|
|
239
|
+
return 0 if overage <= 0
|
|
172
240
|
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
241
|
+
delete_parents_in_batches(
|
|
242
|
+
RailsPulse::Request.where("occurred_at < ?", cutoff),
|
|
243
|
+
order_column: :occurred_at,
|
|
244
|
+
children: RailsPulse::Operation,
|
|
245
|
+
foreign_key: :request_id,
|
|
246
|
+
limit: overage
|
|
247
|
+
)
|
|
176
248
|
end
|
|
177
249
|
|
|
178
250
|
def cleanup_job_runs_by_count
|
|
179
251
|
max_records = @config.max_table_records[:rails_pulse_job_runs]
|
|
180
252
|
return 0 unless max_records
|
|
181
253
|
|
|
182
|
-
|
|
183
|
-
return 0 if
|
|
184
|
-
|
|
185
|
-
records_to_delete = current_count - max_records
|
|
186
|
-
ids_to_delete = RailsPulse::JobRun.order(occurred_at: :asc).limit(records_to_delete).pluck(:id)
|
|
187
|
-
return 0 if ids_to_delete.empty?
|
|
254
|
+
overage = RailsPulse::JobRun.count - max_records
|
|
255
|
+
return 0 if overage <= 0
|
|
188
256
|
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
257
|
+
delete_parents_in_batches(
|
|
258
|
+
RailsPulse::JobRun,
|
|
259
|
+
order_column: :occurred_at,
|
|
260
|
+
children: RailsPulse::Operation,
|
|
261
|
+
foreign_key: :job_run_id,
|
|
262
|
+
limit: overage
|
|
263
|
+
)
|
|
192
264
|
end
|
|
193
265
|
|
|
194
266
|
def cleanup_queries_by_count
|
|
@@ -219,7 +291,10 @@ module RailsPulse
|
|
|
219
291
|
end
|
|
220
292
|
|
|
221
293
|
def cleanup_exception_occurrences_by_time(cutoff_time)
|
|
222
|
-
|
|
294
|
+
delete_in_batches(
|
|
295
|
+
exception_occurrences_cleanup_scope.where("occurred_at < ?", cutoff_time),
|
|
296
|
+
order_column: :occurred_at
|
|
297
|
+
)
|
|
223
298
|
end
|
|
224
299
|
|
|
225
300
|
def cleanup_exception_occurrences_by_count
|
|
@@ -255,29 +330,33 @@ module RailsPulse
|
|
|
255
330
|
records_to_delete = [ overage, deletable_count ].min
|
|
256
331
|
return 0 if records_to_delete <= 0
|
|
257
332
|
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
333
|
+
deleted = delete_parents_in_batches(
|
|
334
|
+
scope,
|
|
335
|
+
order_column: :last_seen_at,
|
|
336
|
+
children: RailsPulse::ExceptionOccurrence,
|
|
337
|
+
foreign_key: :exception_group_id,
|
|
338
|
+
limit: records_to_delete
|
|
339
|
+
)
|
|
263
340
|
|
|
264
341
|
if overage > deletable_count
|
|
265
342
|
RailsPulse.logger.warn("[RailsPulse] Exception group cap #{max_records} cannot be met: " \
|
|
266
|
-
"#{current_count -
|
|
343
|
+
"#{current_count - deleted} remain (#{current_count - deletable_count} are preserved/ignored)")
|
|
267
344
|
end
|
|
268
345
|
|
|
269
|
-
|
|
346
|
+
deleted
|
|
270
347
|
end
|
|
271
348
|
|
|
272
|
-
# Delete groups whose last occurrence was removed —
|
|
273
|
-
# so no orphan can be deleted while
|
|
349
|
+
# Delete groups whose last occurrence was removed — the orphan check and
|
|
350
|
+
# the delete are one statement per batch, so no orphan can be deleted while
|
|
351
|
+
# a concurrent occurrence is being inserted.
|
|
274
352
|
# preserve: true and ignored groups are never deleted automatically.
|
|
275
353
|
def cleanup_orphaned_exception_groups
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
354
|
+
delete_in_batches(
|
|
355
|
+
RailsPulse::ExceptionGroup
|
|
356
|
+
.where(preserve: false)
|
|
357
|
+
.where.not(status: "ignored")
|
|
358
|
+
.where("NOT EXISTS (SELECT 1 FROM rails_pulse_exception_occurrences WHERE rails_pulse_exception_occurrences.exception_group_id = rails_pulse_exception_groups.id)")
|
|
359
|
+
)
|
|
281
360
|
end
|
|
282
361
|
|
|
283
362
|
def perform_summary_cleanup
|
|
@@ -287,8 +366,12 @@ module RailsPulse
|
|
|
287
366
|
# buys hour-accurate change points further back — at the cost of summary
|
|
288
367
|
# table growth. Day, week and month summaries are never pruned.
|
|
289
368
|
cutoff = @config.hourly_summary_retention.ago
|
|
290
|
-
|
|
291
|
-
|
|
369
|
+
run_stage(:time_based, :hourly_summaries) do
|
|
370
|
+
delete_in_batches(
|
|
371
|
+
RailsPulse::Summary.where(period_type: "hour").where("period_start < ?", cutoff),
|
|
372
|
+
order_column: :period_start
|
|
373
|
+
)
|
|
374
|
+
end
|
|
292
375
|
end
|
|
293
376
|
|
|
294
377
|
# Returns the period_end of the most recent completed hourly overall-request
|
|
@@ -305,6 +388,10 @@ module RailsPulse
|
|
|
305
388
|
total_count_based = @stats[:count_based].values.sum
|
|
306
389
|
@stats[:total_deleted] = total_time_based + total_count_based
|
|
307
390
|
|
|
391
|
+
if @failures.any?
|
|
392
|
+
RailsPulse.logger.error "Cleanup finished with #{@failures.size} failed stage(s): #{@failures.map { |failure| failure[:stage] }.join(', ')}"
|
|
393
|
+
end
|
|
394
|
+
|
|
308
395
|
RailsPulse.logger.info "Cleanup completed:"
|
|
309
396
|
RailsPulse.logger.info " Time-based: #{total_time_based} records deleted"
|
|
310
397
|
RailsPulse.logger.info " Count-based: #{total_count_based} records deleted"
|
data/lib/rails_pulse/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rails_pulse
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.4.
|
|
4
|
+
version: 0.4.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Scott Harvey
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-10-01 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: rails
|
|
@@ -48,16 +48,22 @@ dependencies:
|
|
|
48
48
|
name: ransack
|
|
49
49
|
requirement: !ruby/object:Gem::Requirement
|
|
50
50
|
requirements:
|
|
51
|
-
- - "
|
|
51
|
+
- - ">="
|
|
52
52
|
- !ruby/object:Gem::Version
|
|
53
53
|
version: '4.0'
|
|
54
|
+
- - "<"
|
|
55
|
+
- !ruby/object:Gem::Version
|
|
56
|
+
version: '6'
|
|
54
57
|
type: :runtime
|
|
55
58
|
prerelease: false
|
|
56
59
|
version_requirements: !ruby/object:Gem::Requirement
|
|
57
60
|
requirements:
|
|
58
|
-
- - "
|
|
61
|
+
- - ">="
|
|
59
62
|
- !ruby/object:Gem::Version
|
|
60
63
|
version: '4.0'
|
|
64
|
+
- - "<"
|
|
65
|
+
- !ruby/object:Gem::Version
|
|
66
|
+
version: '6'
|
|
61
67
|
- !ruby/object:Gem::Dependency
|
|
62
68
|
name: sqlite3
|
|
63
69
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -545,7 +551,7 @@ metadata:
|
|
|
545
551
|
bug_tracker_uri: https://github.com/railspulse/rails_pulse/issues
|
|
546
552
|
rubygems_mfa_required: 'true'
|
|
547
553
|
post_install_message: |
|
|
548
|
-
Rails Pulse 0.4.
|
|
554
|
+
Rails Pulse 0.4.2 installed.
|
|
549
555
|
|
|
550
556
|
UPGRADING from any 0.3.x? This release changes route identity and DROPS
|
|
551
557
|
rails_pulse_routes.method. Back up your database first, then run:
|