pg_reports 0.6.2 โ 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -0
- data/README.md +48 -0
- data/app/controllers/pg_reports/metrics_controller.rb +27 -0
- data/config/routes.rb +2 -0
- data/lib/pg_reports/configuration.rb +10 -0
- data/lib/pg_reports/grafana/dashboard_builder.rb +277 -0
- data/lib/pg_reports/grafana/exporter.rb +240 -0
- data/lib/pg_reports/module_generator.rb +29 -28
- data/lib/pg_reports/modules/schema_analysis.rb +1 -1
- data/lib/pg_reports/version.rb +1 -1
- data/lib/pg_reports.rb +4 -0
- data/lib/tasks/pg_reports.rake +36 -0
- metadata +15 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 67cc51703472362dd188536abd8196660d5e4bb928d9f9ee53d7e2939290772e
|
|
4
|
+
data.tar.gz: 99fe7733e13344ac1e6b0a9a981728525005437a36e6c17846da455de5365a9b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f408472b95f858f7770511b89885afd98b14074ee5fa3ed67c75ae36b29cb34a96adf1ac17ec0f6cf69bb9cbb51843def46483ff60ee121e2317d930da23deaf
|
|
7
|
+
data.tar.gz: fb4760d0e84fbe5a29b157cb37ee765454e7a26863a01b64aee1ba4262d83b28272e16664b7e797981f72594deb192e7f1a22a0c93ef6fb1399d584932446d91
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.7.0] - 2026-04-26
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- **Experimental Grafana / Prometheus support.** Selected reports can now be exposed at `<mount_point>/metrics` in Prometheus exposition format. Severity (`ok` / `warning` / `critical`) is derived automatically from the existing `REPORT_CONFIG` thresholds, including inverted ones (`cache_hit_ratio` etc.). Per-row data is emitted as `pg_reports_row` with each row column as a label, suitable for Grafana table panels via the "Labels to fields" transformation.
|
|
15
|
+
- New config: `grafana_favorites`, `grafana_metrics_token`, `grafana_cache_ttl`. Reports are cached via `Rails.cache` with a per-report TTL override to keep frequent scrapes from hammering the database.
|
|
16
|
+
- New endpoint: `MetricsController` with timing-safe bearer-token auth.
|
|
17
|
+
- New rake tasks: `pg_reports:grafana:dashboard` (writes a ready-to-import Grafana dashboard JSON) and `pg_reports:grafana:metrics` (prints the current metrics payload).
|
|
18
|
+
- New module `PgReports::Grafana::Exporter` and `PgReports::Grafana::DashboardBuilder`.
|
|
19
|
+
- Documentation: [docs/grafana.md](docs/grafana.md) (integration guide) and [docs/grafana-local-setup.md](docs/grafana-local-setup.md) (local Prometheus + Grafana setup without Docker).
|
|
20
|
+
- **Note:** the metric format may change in future minor versions until 1.0.
|
|
21
|
+
|
|
10
22
|
## [0.6.2] - 2026-04-25
|
|
11
23
|
|
|
12
24
|
### Added
|
data/README.md
CHANGED
|
@@ -18,6 +18,7 @@ A comprehensive PostgreSQL monitoring and analysis library for Rails application
|
|
|
18
18
|
- ๐ฅ๏ธ **System Overview** - Database sizes, PostgreSQL settings, installed extensions
|
|
19
19
|
- ๐ **Web Dashboard** - Beautiful dark-themed UI with sortable tables and expandable rows
|
|
20
20
|
- ๐จ **Telegram Integration** - Send reports directly to Telegram
|
|
21
|
+
- ๐ **Grafana / Prometheus Exporter** - Expose selected reports at `/metrics` with severity derived from configured thresholds
|
|
21
22
|
- ๐ฅ **Export** - Download reports in TXT, CSV, or JSON format
|
|
22
23
|
- ๐ **IDE Integration** - Open source locations in VS Code, Cursor, RubyMine, or IntelliJ (with WSL support)
|
|
23
24
|
- ๐ **Comparison Mode** - Save records to compare before/after optimization
|
|
@@ -299,6 +300,53 @@ The Export dropdown includes **Copy Prompt** (visible on actionable reports). It
|
|
|
299
300
|
|
|
300
301
|
</details>
|
|
301
302
|
|
|
303
|
+
<details>
|
|
304
|
+
<summary><strong>Grafana / Prometheus exporter</strong></summary>
|
|
305
|
+
|
|
306
|
+
Expose selected reports at `<mount_point>/metrics` in Prometheus exposition format. The default mount is `/pg_reports`, so the endpoint is typically `/pg_reports/metrics` โ but it follows whatever path you used in `mount PgReports::Engine, at: "..."`. Severity (`ok` / `warning` / `critical`) is derived automatically from the thresholds defined in [`Dashboard::ReportsRegistry::REPORT_CONFIG`](lib/pg_reports/dashboard/reports_registry.rb).
|
|
307
|
+
|
|
308
|
+
```ruby
|
|
309
|
+
PgReports.configure do |config|
|
|
310
|
+
config.grafana_favorites = [
|
|
311
|
+
:slow_queries,
|
|
312
|
+
:unused_indexes,
|
|
313
|
+
:bloated_tables,
|
|
314
|
+
:missing_validations,
|
|
315
|
+
:polymorphic_without_index
|
|
316
|
+
]
|
|
317
|
+
config.grafana_metrics_token = ENV["PG_REPORTS_METRICS_TOKEN"] # optional bearer token
|
|
318
|
+
config.grafana_cache_ttl = 60 # seconds
|
|
319
|
+
end
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
Scrape with Prometheus:
|
|
323
|
+
|
|
324
|
+
```yaml
|
|
325
|
+
scrape_configs:
|
|
326
|
+
- job_name: pg_reports
|
|
327
|
+
metrics_path: /pg_reports/metrics # adjust to your Engine mount point
|
|
328
|
+
scrape_interval: 60s
|
|
329
|
+
authorization: { credentials: "${PG_REPORTS_METRICS_TOKEN}" }
|
|
330
|
+
static_configs:
|
|
331
|
+
- targets: ["app.internal:3000"]
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
> [!WARNING]
|
|
335
|
+
> Reports are cached via `Rails.cache` for `grafana_cache_ttl` so frequent scrapes don't hammer the database. Without it, Prometheus' default 15s scrape interval against heavy reports like `missing_validations` will DDoS your own DB. Always set a TTL โฅ scrape interval, and consider a longer per-report TTL for expensive reports.
|
|
336
|
+
|
|
337
|
+
The exporter also emits a `pg_reports_row` series per report row (each column becomes a Prometheus label), so the auto-generated dashboard can show a **table panel** with the actual rows that need fixing โ not just an aggregate count.
|
|
338
|
+
|
|
339
|
+
Generate a matching Grafana dashboard from the same favorites:
|
|
340
|
+
|
|
341
|
+
```bash
|
|
342
|
+
bundle exec rake pg_reports:grafana:dashboard
|
|
343
|
+
# writes pg_reports.json in pwd; then Dashboards โ Import in Grafana
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
**[Full Grafana integration guide โ](docs/grafana.md)** ยท **[Local Prometheus + Grafana without Docker โ](docs/grafana-local-setup.md)**
|
|
347
|
+
|
|
348
|
+
</details>
|
|
349
|
+
|
|
302
350
|
<details>
|
|
303
351
|
<summary><strong>Telegram delivery</strong></summary>
|
|
304
352
|
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "digest/sha2"
|
|
4
|
+
|
|
5
|
+
module PgReports
|
|
6
|
+
class MetricsController < ActionController::Base
|
|
7
|
+
CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"
|
|
8
|
+
|
|
9
|
+
before_action :authenticate_metrics!, if: -> { PgReports.config.grafana_metrics_token.present? }
|
|
10
|
+
|
|
11
|
+
def show
|
|
12
|
+
render plain: PgReports::Grafana::Exporter.render, content_type: CONTENT_TYPE
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
private
|
|
16
|
+
|
|
17
|
+
def authenticate_metrics!
|
|
18
|
+
expected = PgReports.config.grafana_metrics_token.to_s
|
|
19
|
+
provided = request.headers["Authorization"].to_s.sub(/\Abearer\s+/i, "")
|
|
20
|
+
|
|
21
|
+
a = ::Digest::SHA256.digest(expected)
|
|
22
|
+
b = ::Digest::SHA256.digest(provided)
|
|
23
|
+
|
|
24
|
+
head :unauthorized unless ActiveSupport::SecurityUtils.secure_compare(a, b) && expected.present?
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
data/config/routes.rb
CHANGED
|
@@ -3,6 +3,8 @@
|
|
|
3
3
|
PgReports::Engine.routes.draw do
|
|
4
4
|
root to: "dashboard#index"
|
|
5
5
|
|
|
6
|
+
get "metrics", to: "metrics#show", as: :metrics
|
|
7
|
+
|
|
6
8
|
get "live_metrics", to: "dashboard#live_metrics", as: :live_metrics
|
|
7
9
|
|
|
8
10
|
post "enable_pg_stat_statements", to: "dashboard#enable_pg_stat_statements", as: :enable_pg_stat_statements
|
|
@@ -42,6 +42,11 @@ module PgReports
|
|
|
42
42
|
# Security settings
|
|
43
43
|
attr_accessor :allow_raw_query_execution # Allow execute_query and explain_analyze from dashboard
|
|
44
44
|
|
|
45
|
+
# Grafana / Prometheus exporter settings
|
|
46
|
+
attr_accessor :grafana_favorites # Reports exposed at /metrics (Array of keys or Hash with per-report opts)
|
|
47
|
+
attr_accessor :grafana_metrics_token # Bearer token required to access /metrics (nil = no auth)
|
|
48
|
+
attr_accessor :grafana_cache_ttl # Default cache TTL for collected reports, seconds
|
|
49
|
+
|
|
45
50
|
def initialize
|
|
46
51
|
# Telegram
|
|
47
52
|
@telegram_bot_token = ENV.fetch("PG_REPORTS_TELEGRAM_TOKEN", nil)
|
|
@@ -87,6 +92,11 @@ module PgReports
|
|
|
87
92
|
@allow_raw_query_execution = ActiveModel::Type::Boolean.new.cast(
|
|
88
93
|
ENV.fetch("PG_REPORTS_ALLOW_RAW_QUERY_EXECUTION", false)
|
|
89
94
|
)
|
|
95
|
+
|
|
96
|
+
# Grafana / Prometheus exporter
|
|
97
|
+
@grafana_favorites = []
|
|
98
|
+
@grafana_metrics_token = ENV.fetch("PG_REPORTS_METRICS_TOKEN", nil)
|
|
99
|
+
@grafana_cache_ttl = 60
|
|
90
100
|
end
|
|
91
101
|
|
|
92
102
|
def connection
|
|
@@ -0,0 +1,277 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module PgReports
|
|
6
|
+
module Grafana
|
|
7
|
+
# Builds an importable Grafana dashboard JSON from configured favorites.
|
|
8
|
+
# Each report becomes a row with two panels:
|
|
9
|
+
# - "rows" stat (total)
|
|
10
|
+
# - "issues by severity" timeseries (ok / warning / critical)
|
|
11
|
+
# Severity colours are wired so that any warning lights yellow, any critical lights red.
|
|
12
|
+
class DashboardBuilder
|
|
13
|
+
DATASOURCE_INPUT = "DS_PROMETHEUS"
|
|
14
|
+
|
|
15
|
+
DEFAULT_TITLE = "PgReports โ PostgreSQL Health"
|
|
16
|
+
DEFAULT_UID = "pg-reports"
|
|
17
|
+
|
|
18
|
+
SEVERITY_COLORS = {
|
|
19
|
+
"ok" => "green",
|
|
20
|
+
"warning" => "yellow",
|
|
21
|
+
"critical" => "red"
|
|
22
|
+
}.freeze
|
|
23
|
+
|
|
24
|
+
GRID_WIDTH = 24
|
|
25
|
+
ROW_HEIGHT = 1
|
|
26
|
+
TIMESERIES_HEIGHT = 8
|
|
27
|
+
TABLE_HEIGHT = 10
|
|
28
|
+
REPORT_BLOCK_HEIGHT = TIMESERIES_HEIGHT + TABLE_HEIGHT
|
|
29
|
+
|
|
30
|
+
def self.build(**opts)
|
|
31
|
+
new(**opts).build
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def initialize(favorites: PgReports.config.grafana_favorites,
|
|
35
|
+
title: DEFAULT_TITLE,
|
|
36
|
+
uid: DEFAULT_UID,
|
|
37
|
+
refresh: "1m",
|
|
38
|
+
time_from: "now-6h")
|
|
39
|
+
@favorites = normalize(favorites)
|
|
40
|
+
@title = title
|
|
41
|
+
@uid = uid
|
|
42
|
+
@refresh = refresh
|
|
43
|
+
@time_from = time_from
|
|
44
|
+
@panel_id = 0
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def build
|
|
48
|
+
if @favorites.empty?
|
|
49
|
+
raise ArgumentError,
|
|
50
|
+
"No favorites configured. Set PgReports.config.grafana_favorites or pass favorites:."
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
{
|
|
54
|
+
"__inputs" => [datasource_input],
|
|
55
|
+
"__requires" => [grafana_require, prometheus_require],
|
|
56
|
+
"annotations" => {"list" => []},
|
|
57
|
+
"editable" => true,
|
|
58
|
+
"graphTooltip" => 1,
|
|
59
|
+
"panels" => build_panels,
|
|
60
|
+
"refresh" => @refresh,
|
|
61
|
+
"schemaVersion" => 38,
|
|
62
|
+
"tags" => ["pg_reports", "postgresql"],
|
|
63
|
+
"templating" => {"list" => []},
|
|
64
|
+
"time" => {"from" => @time_from, "to" => "now"},
|
|
65
|
+
"timezone" => "browser",
|
|
66
|
+
"title" => @title,
|
|
67
|
+
"uid" => @uid,
|
|
68
|
+
"version" => 1,
|
|
69
|
+
"weekStart" => ""
|
|
70
|
+
}
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def to_json(*)
|
|
74
|
+
JSON.pretty_generate(build)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
private
|
|
78
|
+
|
|
79
|
+
def normalize(favorites)
|
|
80
|
+
case favorites
|
|
81
|
+
when Hash then favorites.keys.map(&:to_sym)
|
|
82
|
+
when Array then favorites.map(&:to_sym)
|
|
83
|
+
else []
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
def build_panels
|
|
88
|
+
panels = []
|
|
89
|
+
y = 0
|
|
90
|
+
|
|
91
|
+
grouped_favorites.each do |category, keys|
|
|
92
|
+
panels << row_panel(category_label(category), y)
|
|
93
|
+
y += ROW_HEIGHT
|
|
94
|
+
|
|
95
|
+
keys.each do |key|
|
|
96
|
+
info = report_info(category, key)
|
|
97
|
+
panels << timeseries_panel(key, info, y)
|
|
98
|
+
panels << table_panel(key, info, y + TIMESERIES_HEIGHT)
|
|
99
|
+
y += REPORT_BLOCK_HEIGHT
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
panels
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def grouped_favorites
|
|
107
|
+
groups = {}
|
|
108
|
+
@favorites.each do |key|
|
|
109
|
+
category = category_for(key)
|
|
110
|
+
next unless category # silently skip unknown keys; exporter logs them
|
|
111
|
+
groups[category] ||= []
|
|
112
|
+
groups[category] << key
|
|
113
|
+
end
|
|
114
|
+
groups
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
def category_for(key)
|
|
118
|
+
Dashboard::ReportsRegistry::REPORTS.each do |category, info|
|
|
119
|
+
return category if info[:reports].key?(key)
|
|
120
|
+
end
|
|
121
|
+
nil
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
def category_label(category)
|
|
125
|
+
Dashboard::ReportsRegistry::REPORTS.dig(category, :name) || category.to_s.humanize
|
|
126
|
+
end
|
|
127
|
+
|
|
128
|
+
def report_info(category, key)
|
|
129
|
+
Dashboard::ReportsRegistry::REPORTS.dig(category, :reports, key) || {name: key.to_s.humanize, description: ""}
|
|
130
|
+
end
|
|
131
|
+
|
|
132
|
+
def row_panel(title, y)
|
|
133
|
+
{
|
|
134
|
+
"id" => next_id,
|
|
135
|
+
"type" => "row",
|
|
136
|
+
"title" => title,
|
|
137
|
+
"collapsed" => false,
|
|
138
|
+
"gridPos" => {"h" => ROW_HEIGHT, "w" => GRID_WIDTH, "x" => 0, "y" => y},
|
|
139
|
+
"panels" => []
|
|
140
|
+
}
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
def timeseries_panel(key, info, y)
|
|
144
|
+
{
|
|
145
|
+
"id" => next_id,
|
|
146
|
+
"type" => "timeseries",
|
|
147
|
+
"title" => "#{info[:name]} โ issues by severity",
|
|
148
|
+
"description" => info[:description].to_s,
|
|
149
|
+
"datasource" => datasource_ref,
|
|
150
|
+
"gridPos" => {"h" => TIMESERIES_HEIGHT, "w" => GRID_WIDTH, "x" => 0, "y" => y},
|
|
151
|
+
"targets" => severity_targets(key),
|
|
152
|
+
"options" => {
|
|
153
|
+
"legend" => {"displayMode" => "table", "placement" => "right", "calcs" => ["lastNotNull"]},
|
|
154
|
+
"tooltip" => {"mode" => "multi", "sort" => "desc"}
|
|
155
|
+
},
|
|
156
|
+
"fieldConfig" => {
|
|
157
|
+
"defaults" => {
|
|
158
|
+
"custom" => {
|
|
159
|
+
"drawStyle" => "bars",
|
|
160
|
+
"stacking" => {"mode" => "normal", "group" => "A"},
|
|
161
|
+
"fillOpacity" => 60,
|
|
162
|
+
"lineWidth" => 1
|
|
163
|
+
},
|
|
164
|
+
"color" => {"mode" => "fixed"},
|
|
165
|
+
"mappings" => []
|
|
166
|
+
},
|
|
167
|
+
"overrides" => severity_color_overrides
|
|
168
|
+
}
|
|
169
|
+
}
|
|
170
|
+
end
|
|
171
|
+
|
|
172
|
+
def table_panel(key, info, y)
|
|
173
|
+
{
|
|
174
|
+
"id" => next_id,
|
|
175
|
+
"type" => "table",
|
|
176
|
+
"title" => "#{info[:name]} โ current rows",
|
|
177
|
+
"description" => "#{info[:description]}\n\nLatest snapshot of report rows. Each row's columns are unpacked from Prometheus labels via the Labels-to-fields transformation.".strip,
|
|
178
|
+
"datasource" => datasource_ref,
|
|
179
|
+
"gridPos" => {"h" => TABLE_HEIGHT, "w" => GRID_WIDTH, "x" => 0, "y" => y},
|
|
180
|
+
"targets" => [
|
|
181
|
+
{
|
|
182
|
+
"refId" => "A",
|
|
183
|
+
"expr" => %(pg_reports_row{report="#{key}"}),
|
|
184
|
+
"datasource" => datasource_ref,
|
|
185
|
+
"format" => "table",
|
|
186
|
+
"instant" => true,
|
|
187
|
+
"legendFormat" => "__auto"
|
|
188
|
+
}
|
|
189
|
+
],
|
|
190
|
+
"transformations" => [
|
|
191
|
+
{
|
|
192
|
+
"id" => "labelsToFields",
|
|
193
|
+
"options" => {"mode" => "columns"}
|
|
194
|
+
},
|
|
195
|
+
{
|
|
196
|
+
"id" => "organize",
|
|
197
|
+
"options" => {
|
|
198
|
+
"excludeByName" => {
|
|
199
|
+
"Time" => true,
|
|
200
|
+
"Value" => true,
|
|
201
|
+
"__name__" => true,
|
|
202
|
+
"instance" => true,
|
|
203
|
+
"job" => true,
|
|
204
|
+
"report" => true,
|
|
205
|
+
"row" => true
|
|
206
|
+
},
|
|
207
|
+
"indexByName" => {},
|
|
208
|
+
"renameByName" => {}
|
|
209
|
+
}
|
|
210
|
+
}
|
|
211
|
+
],
|
|
212
|
+
"options" => {
|
|
213
|
+
"showHeader" => true,
|
|
214
|
+
"cellHeight" => "sm",
|
|
215
|
+
"footer" => {"show" => false}
|
|
216
|
+
},
|
|
217
|
+
"fieldConfig" => {
|
|
218
|
+
"defaults" => {
|
|
219
|
+
"custom" => {"align" => "auto", "displayMode" => "auto"},
|
|
220
|
+
"mappings" => []
|
|
221
|
+
},
|
|
222
|
+
"overrides" => []
|
|
223
|
+
}
|
|
224
|
+
}
|
|
225
|
+
end
|
|
226
|
+
|
|
227
|
+
def severity_targets(key)
|
|
228
|
+
SEVERITY_COLORS.keys.each_with_index.map do |severity, i|
|
|
229
|
+
{
|
|
230
|
+
"refId" => ("A".ord + i).chr,
|
|
231
|
+
"expr" => %(pg_reports_issues{report="#{key}",severity="#{severity}"}),
|
|
232
|
+
"datasource" => datasource_ref,
|
|
233
|
+
"legendFormat" => severity
|
|
234
|
+
}
|
|
235
|
+
end
|
|
236
|
+
end
|
|
237
|
+
|
|
238
|
+
def severity_color_overrides
|
|
239
|
+
SEVERITY_COLORS.map do |severity, color|
|
|
240
|
+
{
|
|
241
|
+
"matcher" => {"id" => "byName", "options" => severity},
|
|
242
|
+
"properties" => [
|
|
243
|
+
{"id" => "color", "value" => {"mode" => "fixed", "fixedColor" => color}}
|
|
244
|
+
]
|
|
245
|
+
}
|
|
246
|
+
end
|
|
247
|
+
end
|
|
248
|
+
|
|
249
|
+
def datasource_input
|
|
250
|
+
{
|
|
251
|
+
"name" => DATASOURCE_INPUT,
|
|
252
|
+
"label" => "Prometheus",
|
|
253
|
+
"description" => "Datasource that scrapes /pg_reports/metrics",
|
|
254
|
+
"type" => "datasource",
|
|
255
|
+
"pluginId" => "prometheus",
|
|
256
|
+
"pluginName" => "Prometheus"
|
|
257
|
+
}
|
|
258
|
+
end
|
|
259
|
+
|
|
260
|
+
def datasource_ref
|
|
261
|
+
{"type" => "prometheus", "uid" => "${#{DATASOURCE_INPUT}}"}
|
|
262
|
+
end
|
|
263
|
+
|
|
264
|
+
def grafana_require
|
|
265
|
+
{"type" => "grafana", "id" => "grafana", "name" => "Grafana", "version" => "9.0.0"}
|
|
266
|
+
end
|
|
267
|
+
|
|
268
|
+
def prometheus_require
|
|
269
|
+
{"type" => "datasource", "id" => "prometheus", "name" => "Prometheus", "version" => "1.0.0"}
|
|
270
|
+
end
|
|
271
|
+
|
|
272
|
+
def next_id
|
|
273
|
+
@panel_id += 1
|
|
274
|
+
end
|
|
275
|
+
end
|
|
276
|
+
end
|
|
277
|
+
end
|
|
@@ -0,0 +1,240 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PgReports
|
|
4
|
+
module Grafana
|
|
5
|
+
# Renders selected reports in Prometheus exposition format.
|
|
6
|
+
# Severity is derived from REPORT_CONFIG thresholds in Dashboard::ReportsRegistry.
|
|
7
|
+
class Exporter
|
|
8
|
+
SEVERITY_ORDER = {"ok" => 0, "warning" => 1, "critical" => 2}.freeze
|
|
9
|
+
MAX_LABEL_VALUE_LENGTH = 200
|
|
10
|
+
RESERVED_LABEL_NAMES = %w[report severity row error].freeze
|
|
11
|
+
|
|
12
|
+
MODULES = {
|
|
13
|
+
queries: -> { Modules::Queries },
|
|
14
|
+
indexes: -> { Modules::Indexes },
|
|
15
|
+
tables: -> { Modules::Tables },
|
|
16
|
+
connections: -> { Modules::Connections },
|
|
17
|
+
system: -> { Modules::System },
|
|
18
|
+
schema_analysis: -> { Modules::SchemaAnalysis }
|
|
19
|
+
}.freeze
|
|
20
|
+
|
|
21
|
+
def self.render
|
|
22
|
+
new.render
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def initialize(favorites: PgReports.config.grafana_favorites,
|
|
26
|
+
cache_ttl: PgReports.config.grafana_cache_ttl,
|
|
27
|
+
clock: Time)
|
|
28
|
+
@favorites = normalize(favorites)
|
|
29
|
+
@cache_ttl = cache_ttl
|
|
30
|
+
@clock = clock
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def render
|
|
34
|
+
results = @favorites.map { |key, opts| collect(key, opts) }
|
|
35
|
+
|
|
36
|
+
lines = []
|
|
37
|
+
emit(lines, "pg_reports_issues", "Number of rows by severity for the report") do |emit|
|
|
38
|
+
results.each do |r|
|
|
39
|
+
next unless r[:ok]
|
|
40
|
+
r[:severities].each { |sev, count| emit.call({report: r[:key], severity: sev}, count) }
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
emit(lines, "pg_reports_rows", "Total rows returned by the report") do |emit|
|
|
45
|
+
results.each { |r| emit.call({report: r[:key]}, r[:rows]) if r[:ok] }
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
emit(lines, "pg_reports_run_seconds", "Time spent collecting the report") do |emit|
|
|
49
|
+
results.each { |r| emit.call({report: r[:key]}, r[:duration].round(4)) if r[:ok] }
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
emit(lines, "pg_reports_last_run_timestamp", "Unix timestamp of last collection") do |emit|
|
|
53
|
+
results.each { |r| emit.call({report: r[:key]}, r[:timestamp]) if r[:ok] }
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
emit(lines, "pg_reports_up", "Whether collection succeeded (1) or failed (0)") do |emit|
|
|
57
|
+
results.each do |r|
|
|
58
|
+
labels = {report: r[:key]}
|
|
59
|
+
labels[:error] = r[:error] unless r[:ok]
|
|
60
|
+
emit.call(labels, r[:ok] ? 1 : 0)
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
emit(lines, "pg_reports_row", "One series per row of the report (drives Grafana table panels). Each row column becomes a label.") do |emit|
|
|
65
|
+
results.each do |r|
|
|
66
|
+
next unless r[:ok] && r[:rows_data]
|
|
67
|
+
r[:rows_data].each_with_index do |row_labels, idx|
|
|
68
|
+
emit.call(row_labels.merge(report: r[:key], row: idx), 1)
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
(lines << "").join("\n")
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
private
|
|
77
|
+
|
|
78
|
+
def normalize(favorites)
|
|
79
|
+
case favorites
|
|
80
|
+
when Hash
|
|
81
|
+
favorites.each_with_object({}) { |(k, v), h| h[k.to_sym] = (v || {}).symbolize_keys }
|
|
82
|
+
when Array
|
|
83
|
+
favorites.each_with_object({}) { |k, h| h[k.to_sym] = {} }
|
|
84
|
+
else
|
|
85
|
+
{}
|
|
86
|
+
end
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
def collect(key, opts)
|
|
90
|
+
cached(key, opts) { run(key, opts) }
|
|
91
|
+
rescue => e
|
|
92
|
+
{key: key, ok: false, error: e.class.name, message: e.message}
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
def cached(key, opts)
|
|
96
|
+
ttl = opts[:ttl] || @cache_ttl
|
|
97
|
+
if ttl && defined?(Rails) && Rails.respond_to?(:cache) && Rails.cache
|
|
98
|
+
Rails.cache.fetch("pg_reports/grafana/#{key}", expires_in: ttl) { yield }
|
|
99
|
+
else
|
|
100
|
+
yield
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
def run(key, opts)
|
|
105
|
+
mod = module_for(key) or raise ArgumentError, "Unknown report: #{key}"
|
|
106
|
+
|
|
107
|
+
started = @clock.now
|
|
108
|
+
report = mod.public_send(key, **report_args(opts))
|
|
109
|
+
finished = @clock.now
|
|
110
|
+
|
|
111
|
+
{
|
|
112
|
+
key: key,
|
|
113
|
+
ok: true,
|
|
114
|
+
rows: report.size,
|
|
115
|
+
severities: severity_counts(key, report),
|
|
116
|
+
rows_data: opts.fetch(:expose_rows, true) ? row_label_sets(report) : nil,
|
|
117
|
+
duration: finished - started,
|
|
118
|
+
timestamp: finished.to_i
|
|
119
|
+
}
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
def row_label_sets(report)
|
|
123
|
+
report.map { |row| row_to_labels(row) }
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
def row_to_labels(row)
|
|
127
|
+
labels = {}
|
|
128
|
+
row.each do |column, value|
|
|
129
|
+
next if value.nil?
|
|
130
|
+
|
|
131
|
+
name = sanitize_label_name(column.to_s)
|
|
132
|
+
next if name.empty? || RESERVED_LABEL_NAMES.include?(name)
|
|
133
|
+
|
|
134
|
+
formatted = format_label_value(value)
|
|
135
|
+
next if formatted.length > MAX_LABEL_VALUE_LENGTH
|
|
136
|
+
|
|
137
|
+
labels[name] = formatted
|
|
138
|
+
end
|
|
139
|
+
labels
|
|
140
|
+
end
|
|
141
|
+
|
|
142
|
+
def sanitize_label_name(name)
|
|
143
|
+
cleaned = name.gsub(/[^a-zA-Z0-9_]/, "_")
|
|
144
|
+
cleaned = "_#{cleaned}" if cleaned.match?(/\A[0-9]/)
|
|
145
|
+
cleaned
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
def format_label_value(value)
|
|
149
|
+
case value
|
|
150
|
+
when Float then format("%g", value)
|
|
151
|
+
when Time, DateTime then value.iso8601
|
|
152
|
+
else value.to_s
|
|
153
|
+
end
|
|
154
|
+
end
|
|
155
|
+
|
|
156
|
+
def module_for(key)
|
|
157
|
+
Dashboard::ReportsRegistry::REPORTS.each do |category, info|
|
|
158
|
+
next unless info[:reports].key?(key.to_sym)
|
|
159
|
+
factory = MODULES[category] or return nil
|
|
160
|
+
return factory.call
|
|
161
|
+
end
|
|
162
|
+
nil
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
def report_args(opts)
|
|
166
|
+
# Only forward kwargs that report methods accept; keep the surface tiny.
|
|
167
|
+
opts.slice(:limit).compact
|
|
168
|
+
end
|
|
169
|
+
|
|
170
|
+
def severity_counts(key, report)
|
|
171
|
+
thresholds = Dashboard::ReportsRegistry.thresholds(key)
|
|
172
|
+
counts = Hash.new(0)
|
|
173
|
+
|
|
174
|
+
if thresholds.empty?
|
|
175
|
+
counts["ok"] = report.size
|
|
176
|
+
return counts
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
report.each { |row| counts[row_severity(row, thresholds)] += 1 }
|
|
180
|
+
counts
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
def row_severity(row, thresholds)
|
|
184
|
+
worst = "ok"
|
|
185
|
+
thresholds.each do |field, t|
|
|
186
|
+
value = row[field.to_s] || row[field]
|
|
187
|
+
next if value.nil?
|
|
188
|
+
|
|
189
|
+
worst = max_severity(worst, severity_for(value.to_f, t))
|
|
190
|
+
end
|
|
191
|
+
worst
|
|
192
|
+
end
|
|
193
|
+
|
|
194
|
+
def severity_for(value, thresholds)
|
|
195
|
+
critical = thresholds[:critical]
|
|
196
|
+
warning = thresholds[:warning]
|
|
197
|
+
|
|
198
|
+
if thresholds[:inverted]
|
|
199
|
+
return "critical" if critical && value <= critical
|
|
200
|
+
return "warning" if warning && value <= warning
|
|
201
|
+
else
|
|
202
|
+
return "critical" if critical && value >= critical
|
|
203
|
+
return "warning" if warning && value >= warning
|
|
204
|
+
end
|
|
205
|
+
"ok"
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
def max_severity(a, b)
|
|
209
|
+
(SEVERITY_ORDER[a] >= SEVERITY_ORDER[b]) ? a : b
|
|
210
|
+
end
|
|
211
|
+
|
|
212
|
+
def emit(lines, metric, help)
|
|
213
|
+
buffer = []
|
|
214
|
+
emitter = ->(labels, value) {
|
|
215
|
+
buffer << "#{metric}#{format_labels(labels)} #{value}"
|
|
216
|
+
}
|
|
217
|
+
yield emitter
|
|
218
|
+
return if buffer.empty?
|
|
219
|
+
|
|
220
|
+
lines << "# HELP #{metric} #{help}"
|
|
221
|
+
lines << "# TYPE #{metric} gauge"
|
|
222
|
+
lines.concat(buffer)
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
def format_labels(labels)
|
|
226
|
+
return "" if labels.nil? || labels.empty?
|
|
227
|
+
|
|
228
|
+
pairs = labels.map { |k, v| %(#{k}="#{escape_label(v)}") }
|
|
229
|
+
"{#{pairs.join(",")}}"
|
|
230
|
+
end
|
|
231
|
+
|
|
232
|
+
def escape_label(value)
|
|
233
|
+
value.to_s
|
|
234
|
+
.gsub("\\", "\\\\\\\\")
|
|
235
|
+
.gsub('"', '\\"')
|
|
236
|
+
.gsub("\n", '\\n')
|
|
237
|
+
end
|
|
238
|
+
end
|
|
239
|
+
end
|
|
240
|
+
end
|
|
@@ -3,42 +3,43 @@
|
|
|
3
3
|
module PgReports
|
|
4
4
|
# Generates module methods dynamically from YAML report definitions
|
|
5
5
|
class ModuleGenerator
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
6
|
+
class << self
|
|
7
|
+
def generate!
|
|
8
|
+
ReportLoader.load_all.each do |module_name, reports|
|
|
9
|
+
module_class = get_module(module_name)
|
|
10
|
+
next unless module_class
|
|
11
|
+
|
|
12
|
+
reports.each do |report_name, definition|
|
|
13
|
+
define_report_method(module_class, report_name, definition)
|
|
14
|
+
end
|
|
13
15
|
end
|
|
14
16
|
end
|
|
15
|
-
end
|
|
16
17
|
|
|
17
|
-
|
|
18
|
+
private
|
|
18
19
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
20
|
+
def get_module(module_name)
|
|
21
|
+
const_name = module_name.to_s.split("_").map(&:capitalize).join
|
|
22
|
+
PgReports::Modules.const_get(const_name)
|
|
23
|
+
rescue NameError
|
|
24
|
+
# Module doesn't exist, skip it
|
|
25
|
+
# We don't auto-create modules to avoid conflicts
|
|
26
|
+
nil
|
|
27
|
+
end
|
|
27
28
|
|
|
28
|
-
|
|
29
|
-
|
|
29
|
+
def define_report_method(module_class, report_name, definition)
|
|
30
|
+
params_config = definition.config["parameters"] || {}
|
|
30
31
|
|
|
31
|
-
|
|
32
|
-
|
|
32
|
+
# Extract default parameter values
|
|
33
|
+
defaults = params_config.transform_values { |v| v["default"] }
|
|
33
34
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
captured_defaults = defaults
|
|
35
|
+
# Capture definition + defaults so the singleton method closes over them
|
|
36
|
+
captured_definition = definition
|
|
37
|
+
captured_defaults = defaults
|
|
38
38
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
39
|
+
module_class.define_singleton_method(report_name) do |**params|
|
|
40
|
+
merged_params = captured_defaults.merge(params)
|
|
41
|
+
captured_definition.generate_report(**merged_params)
|
|
42
|
+
end
|
|
42
43
|
end
|
|
43
44
|
end
|
|
44
45
|
end
|
|
@@ -112,7 +112,7 @@ module PgReports
|
|
|
112
112
|
counter_belongs_to(model).each do |assoc|
|
|
113
113
|
counter_col = counter_cache_column_name(model, assoc)
|
|
114
114
|
parent = parent_class_for(assoc)
|
|
115
|
-
next unless parent
|
|
115
|
+
next unless parent&.table_exists?
|
|
116
116
|
|
|
117
117
|
unless parent.column_names.include?(counter_col)
|
|
118
118
|
results << {
|
data/lib/pg_reports/version.rb
CHANGED
data/lib/pg_reports.rb
CHANGED
|
@@ -33,6 +33,10 @@ require_relative "pg_reports/modules/schema_analysis"
|
|
|
33
33
|
# Dashboard
|
|
34
34
|
require_relative "pg_reports/dashboard/reports_registry"
|
|
35
35
|
|
|
36
|
+
# Grafana / Prometheus exporter
|
|
37
|
+
require_relative "pg_reports/grafana/exporter"
|
|
38
|
+
require_relative "pg_reports/grafana/dashboard_builder"
|
|
39
|
+
|
|
36
40
|
# Rails Engine
|
|
37
41
|
require_relative "pg_reports/engine" if defined?(Rails::Engine)
|
|
38
42
|
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
namespace :pg_reports do
|
|
6
|
+
namespace :grafana do
|
|
7
|
+
desc "Write importable Grafana dashboard JSON for the configured grafana_favorites. " \
|
|
8
|
+
"Defaults to pg_reports.json in pwd. Override with OUTPUT=, FAVORITES=, TITLE=, UID=, REFRESH=, TIME_FROM=."
|
|
9
|
+
task dashboard: :environment do
|
|
10
|
+
favorites = if ENV["FAVORITES"]
|
|
11
|
+
ENV["FAVORITES"].split(",").map { |k| k.strip.to_sym }
|
|
12
|
+
else
|
|
13
|
+
PgReports.config.grafana_favorites
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
builder = PgReports::Grafana::DashboardBuilder.new(
|
|
17
|
+
favorites: favorites,
|
|
18
|
+
title: ENV.fetch("TITLE", PgReports::Grafana::DashboardBuilder::DEFAULT_TITLE),
|
|
19
|
+
uid: ENV.fetch("UID", PgReports::Grafana::DashboardBuilder::DEFAULT_UID),
|
|
20
|
+
refresh: ENV.fetch("REFRESH", "1m"),
|
|
21
|
+
time_from: ENV.fetch("TIME_FROM", "now-6h")
|
|
22
|
+
)
|
|
23
|
+
|
|
24
|
+
output_path = ENV.fetch("OUTPUT", "pg_reports.json")
|
|
25
|
+
File.write(output_path, JSON.pretty_generate(builder.build))
|
|
26
|
+
warn "Wrote #{output_path}"
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
desc "Write the current /metrics payload to a file. Defaults to pg_reports.metrics in pwd. Override with OUTPUT=."
|
|
30
|
+
task metrics: :environment do
|
|
31
|
+
output_path = ENV.fetch("OUTPUT", "pg_reports.metrics")
|
|
32
|
+
File.write(output_path, PgReports::Grafana::Exporter.render)
|
|
33
|
+
warn "Wrote #{output_path}"
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: pg_reports
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.7.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Eldar Avatov
|
|
@@ -148,6 +148,7 @@ files:
|
|
|
148
148
|
- LICENSE.txt
|
|
149
149
|
- README.md
|
|
150
150
|
- app/controllers/pg_reports/dashboard_controller.rb
|
|
151
|
+
- app/controllers/pg_reports/metrics_controller.rb
|
|
151
152
|
- app/views/layouts/pg_reports/application.html.erb
|
|
152
153
|
- app/views/pg_reports/dashboard/_fake_source_data.html.erb
|
|
153
154
|
- app/views/pg_reports/dashboard/_show_modals.html.erb
|
|
@@ -214,6 +215,8 @@ files:
|
|
|
214
215
|
- lib/pg_reports/executor.rb
|
|
215
216
|
- lib/pg_reports/explain_analyzer.rb
|
|
216
217
|
- lib/pg_reports/filter.rb
|
|
218
|
+
- lib/pg_reports/grafana/dashboard_builder.rb
|
|
219
|
+
- lib/pg_reports/grafana/exporter.rb
|
|
217
220
|
- lib/pg_reports/module_generator.rb
|
|
218
221
|
- lib/pg_reports/modules/connections.rb
|
|
219
222
|
- lib/pg_reports/modules/indexes.rb
|
|
@@ -278,6 +281,7 @@ files:
|
|
|
278
281
|
- lib/pg_reports/sql_loader.rb
|
|
279
282
|
- lib/pg_reports/telegram_sender.rb
|
|
280
283
|
- lib/pg_reports/version.rb
|
|
284
|
+
- lib/tasks/pg_reports.rake
|
|
281
285
|
homepage: https://github.com/deadalice/pg_reports
|
|
282
286
|
licenses:
|
|
283
287
|
- MIT
|
|
@@ -285,6 +289,16 @@ metadata:
|
|
|
285
289
|
homepage_uri: https://github.com/deadalice/pg_reports
|
|
286
290
|
source_code_uri: https://github.com/deadalice/pg_reports
|
|
287
291
|
changelog_uri: https://github.com/deadalice/pg_reports/blob/main/CHANGELOG.md
|
|
292
|
+
post_install_message: |
|
|
293
|
+
Thanks for installing pg_reports v0.7.0!
|
|
294
|
+
|
|
295
|
+
New in 0.7.0 โ experimental Grafana / Prometheus support
|
|
296
|
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
|
297
|
+
Expose selected reports at <mount_point>/metrics in Prometheus exposition
|
|
298
|
+
format and import a ready-to-use Grafana dashboard:
|
|
299
|
+
|
|
300
|
+
Setup guide:
|
|
301
|
+
https://github.com/deadalice/pg_reports/blob/main/docs/grafana.md
|
|
288
302
|
rdoc_options: []
|
|
289
303
|
require_paths:
|
|
290
304
|
- lib
|