schema_reaper 1.0.16 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +56 -0
- data/lib/generators/schema_reaper/install/install_generator.rb +194 -0
- data/lib/generators/schema_reaper/install/templates/initializer.rb.tt +18 -0
- data/lib/generators/schema_reaper/install/templates/schema_reaper_controller.rb.tt +69 -0
- data/lib/schema_reaper/alert_config.rb +55 -0
- data/lib/schema_reaper/mailer.rb +31 -0
- data/lib/schema_reaper/notifier.rb +97 -0
- data/lib/schema_reaper/scan_job.rb +17 -0
- data/lib/schema_reaper/tasks/schema_reaper.rake +5 -0
- data/lib/schema_reaper/version.rb +1 -1
- data/lib/schema_reaper.rb +4 -0
- data/schema_reaper.gemspec +4 -0
- metadata +83 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 27824319f6c1b0a06c833b1de1e6c9f627f042964f12920083899aebea81c3ee
|
|
4
|
+
data.tar.gz: 4a5a7b872774fbb6a4fad373206baa54878d2a1bf77e31f8d24232a45e8e65cb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 95bfcabf3a810c80deab66508b5f75c04df8149f15e5d134c2977749a893b01e1ba5f471c9774dddad2d431dbe26e6178bb348d973c370b99bdb42f5e1249c18
|
|
7
|
+
data.tar.gz: 95f99230df34ab0f09f2ccf713552cf02c513d6c67d27715224283d2af107c60d0929c01a102232c468774766f644f1c831ed97db39d61675591a1aa4032244d
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,61 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [2.0.0] - 2026-09-23
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
- **`rails generate schema_reaper:install`** — one-time setup for running
|
|
7
|
+
`schema_reaper` unattended in production, with zero ongoing DevOps
|
|
8
|
+
dependency: a scheduled scan, results delivered automatically, and an
|
|
9
|
+
on-demand manual trigger, none of it gated behind infrastructure a given
|
|
10
|
+
project may not have (Slack, a CI pipeline with prod network access,
|
|
11
|
+
etc.).
|
|
12
|
+
- `SchemaReaper::AlertConfig` / `config/initializers/schema_reaper.rb` —
|
|
13
|
+
where reports go (`emails`, `webhook_url`). Plain committed config, not
|
|
14
|
+
secrets; commented out by default so an unfilled-in initializer is a
|
|
15
|
+
safe no-op.
|
|
16
|
+
- `SchemaReaper::Notifier` — delivers a scan's findings through whichever
|
|
17
|
+
channels are configured. Both fire when both are set (two audiences —
|
|
18
|
+
a dev-facing chat channel and a formal inbox record — not a fallback
|
|
19
|
+
chain). Webhook: a Slack-compatible `{"text": ...}` POST, with a 10s
|
|
20
|
+
timeout so a dead host can't hang a queue worker indefinitely. Email:
|
|
21
|
+
reuses the host app's own ActionMailer setup via `SchemaReaper::Mailer`
|
|
22
|
+
(inherits `ApplicationMailer` when the host app defines one, falls back
|
|
23
|
+
to a placeholder `default from:` otherwise so delivery degrades to a
|
|
24
|
+
normal SMTP rejection instead of a hard crash). Delivery failures are
|
|
25
|
+
logged, never raised, so a flaky endpoint can't fail the scan job.
|
|
26
|
+
- `SchemaReaper::ScanJob` / `rake schema_reaper:alert` — the one job both
|
|
27
|
+
the schedule and the manual trigger enqueue.
|
|
28
|
+
- Generated `app/controllers/schema_reaper_controller.rb` + a route for
|
|
29
|
+
the manual trigger — bearer-token protected via
|
|
30
|
+
`Rails.application.credentials` (decrypted with the `RAILS_MASTER_KEY`
|
|
31
|
+
every Rails app already has, so this introduces zero new production
|
|
32
|
+
configuration), checked with `ActiveSupport::SecurityUtils
|
|
33
|
+
.secure_compare`, and a shared 5-minute cooldown via `Rails.cache` so a
|
|
34
|
+
valid or leaked token can't be POSTed repeatedly to stack up scans
|
|
35
|
+
against production.
|
|
36
|
+
- A schedule entry in whichever of `config/schedule.rb` (`whenever`) or
|
|
37
|
+
`config/schedule.yml` (`sidekiq-cron`) the app's `Gemfile.lock` says it
|
|
38
|
+
has; prints manual instructions instead of guessing when neither gem is
|
|
39
|
+
present. Warns at install time if the Gemfile scopes `schema_reaper` to
|
|
40
|
+
`group: :development` (most production deploy pipelines strip dev/test
|
|
41
|
+
groups) or if no `ApplicationMailer` `default from:` is detected (email
|
|
42
|
+
reports would otherwise silently never arrive).
|
|
43
|
+
- Re-running the generator is safe — every step checks for its own
|
|
44
|
+
marker before writing, so nothing gets duplicated.
|
|
45
|
+
|
|
46
|
+
CI/SARIF stays a separate, already-documented path for PR/staging checks
|
|
47
|
+
— this is for production, where GitHub-hosted runners don't have prod
|
|
48
|
+
network access by default.
|
|
49
|
+
|
|
50
|
+
Verified against a real booted `Rails::Application` (not just specs):
|
|
51
|
+
the manual-trigger endpoint correctly rejects requests without hitting
|
|
52
|
+
Rails' CSRF protection, a valid token enqueues and a second one within
|
|
53
|
+
the cooldown 429s without enqueuing twice, and wrong-token requests
|
|
54
|
+
during the cooldown still 401 rather than being masked by the lock.
|
|
55
|
+
Reviewed by @aksshatt; three follow-up findings (a stale doc comment, the
|
|
56
|
+
mailer `from:` gap, and the missing rate limit) addressed and verified
|
|
57
|
+
the same way. (#20, mitkush)
|
|
58
|
+
|
|
3
59
|
## [1.0.16] - 2026-09-22
|
|
4
60
|
|
|
5
61
|
### Fixed
|
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "rails/generators"
|
|
4
|
+
require "securerandom"
|
|
5
|
+
|
|
6
|
+
module SchemaReaper
|
|
7
|
+
module Generators
|
|
8
|
+
# `rails generate schema_reaper:install` -- one-time setup for unattended
|
|
9
|
+
# production scanning: an initializer for where reports go, a scheduled
|
|
10
|
+
# job (whenever or sidekiq-cron, whichever the app already uses), and a
|
|
11
|
+
# token-protected manual-trigger route. See the plan this implements:
|
|
12
|
+
# production automation with zero ongoing DevOps dependency -- a normal
|
|
13
|
+
# `git commit` + deploy is the only step after this generator runs.
|
|
14
|
+
class InstallGenerator < Rails::Generators::Base
|
|
15
|
+
source_root File.expand_path("templates", __dir__)
|
|
16
|
+
|
|
17
|
+
def create_initializer
|
|
18
|
+
template "initializer.rb.tt", "config/initializers/schema_reaper.rb"
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def create_manual_trigger
|
|
22
|
+
template "schema_reaper_controller.rb.tt", "app/controllers/schema_reaper_controller.rb"
|
|
23
|
+
|
|
24
|
+
# `route` (unlike `template`/`create_file`) has no conflict
|
|
25
|
+
# detection of its own -- it unconditionally injects, so re-running
|
|
26
|
+
# this generator would duplicate the line every time without this
|
|
27
|
+
# guard.
|
|
28
|
+
routes_path = File.join(destination_root, "config/routes.rb")
|
|
29
|
+
if File.exist?(routes_path) && File.read(routes_path).include?("schema_reaper#trigger")
|
|
30
|
+
say_status :skip, "config/routes.rb already routes to schema_reaper#trigger -- not duplicating it", :yellow
|
|
31
|
+
else
|
|
32
|
+
route "post '/internal/schema_scan', to: 'schema_reaper#trigger'"
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def add_schedule
|
|
37
|
+
case detected_scheduler
|
|
38
|
+
when :whenever then add_whenever_schedule
|
|
39
|
+
when :sidekiq_cron then add_sidekiq_cron_schedule
|
|
40
|
+
else
|
|
41
|
+
say_status :skip, "no `whenever` or `sidekiq-cron` gem detected -- add a cron entry yourself " \
|
|
42
|
+
"that runs `rake schema_reaper:alert` on whatever cadence you want", :yellow
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def warn_if_dev_scoped
|
|
47
|
+
return unless gemfile_scopes_schema_reaper_to_dev?
|
|
48
|
+
|
|
49
|
+
say_status :warning,
|
|
50
|
+
"schema_reaper is Gemfile-scoped to group: :development. Most production deploy " \
|
|
51
|
+
"pipelines strip dev/test groups (`bundle install --without development test`), so " \
|
|
52
|
+
"the scheduled scan will not run in production until you remove that restriction.",
|
|
53
|
+
:red
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def warn_if_no_mailer_from
|
|
57
|
+
return if mailer_from_configured?
|
|
58
|
+
|
|
59
|
+
say_status :warning,
|
|
60
|
+
"no ApplicationMailer default `from:` detected. SchemaReaper::Mailer falls back to a " \
|
|
61
|
+
"placeholder sender in that case, which most real SMTP relays reject or spam-flag -- " \
|
|
62
|
+
"email reports would silently fail to arrive. If you plan to use the email channel " \
|
|
63
|
+
"(config.emails in the initializer this generator just wrote), set " \
|
|
64
|
+
"`default from: \"...\"` on ApplicationMailer first.",
|
|
65
|
+
:red
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
def print_token_instructions
|
|
69
|
+
token = "#{app_identifier}-#{SecureRandom.hex(32)}"
|
|
70
|
+
|
|
71
|
+
say ""
|
|
72
|
+
say "Manual trigger token (shown once -- store it now):", :green
|
|
73
|
+
say " #{token}"
|
|
74
|
+
say ""
|
|
75
|
+
say "This is genuinely sensitive (it authorizes a production action), so it does not go in a " \
|
|
76
|
+
"committed file. Store it in encrypted credentials instead:"
|
|
77
|
+
say ""
|
|
78
|
+
say " rails credentials:edit"
|
|
79
|
+
say ""
|
|
80
|
+
say "and add:"
|
|
81
|
+
say ""
|
|
82
|
+
say " schema_reaper:"
|
|
83
|
+
say " trigger_token: #{token}"
|
|
84
|
+
say ""
|
|
85
|
+
say "Trigger a scan on demand with:"
|
|
86
|
+
say ""
|
|
87
|
+
say " curl -X POST https://your-app.example.com/internal/schema_scan \\"
|
|
88
|
+
say " -H \"Authorization: Bearer #{token}\""
|
|
89
|
+
say ""
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
private
|
|
93
|
+
|
|
94
|
+
def detected_scheduler
|
|
95
|
+
return :whenever if gem_locked?("whenever")
|
|
96
|
+
return :sidekiq_cron if gem_locked?("sidekiq-cron")
|
|
97
|
+
|
|
98
|
+
nil
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
def gem_locked?(name)
|
|
102
|
+
Bundler.locked_gems&.specs&.any? { |s| s.name == name } || false
|
|
103
|
+
rescue StandardError
|
|
104
|
+
false
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def add_whenever_schedule
|
|
108
|
+
path = "config/schedule.rb"
|
|
109
|
+
marker = 'rake "schema_reaper:alert"'
|
|
110
|
+
entry = <<~RUBY
|
|
111
|
+
|
|
112
|
+
every 3.months do
|
|
113
|
+
#{marker}
|
|
114
|
+
end
|
|
115
|
+
RUBY
|
|
116
|
+
|
|
117
|
+
full_path = File.join(destination_root, path)
|
|
118
|
+
if File.exist?(full_path) && File.read(full_path).include?(marker)
|
|
119
|
+
say_status :skip, "#{path} already has a schema_reaper entry -- not duplicating it", :yellow
|
|
120
|
+
elsif File.exist?(full_path)
|
|
121
|
+
append_to_file path, entry
|
|
122
|
+
else
|
|
123
|
+
# No `require "whenever"` here -- real wheneverize-generated files
|
|
124
|
+
# don't have one; whenever's own CLI evaluates this file through
|
|
125
|
+
# its DSL, not as a plain script that needs to load itself.
|
|
126
|
+
create_file path, entry.sub("\n\n", "")
|
|
127
|
+
end
|
|
128
|
+
end
|
|
129
|
+
|
|
130
|
+
def add_sidekiq_cron_schedule
|
|
131
|
+
path = "config/schedule.yml"
|
|
132
|
+
marker = "schema_reaper_scan:"
|
|
133
|
+
entry = <<~YAML
|
|
134
|
+
|
|
135
|
+
#{marker}
|
|
136
|
+
cron: "0 4 1 */3 *" # 4am on the 1st, every 3 months
|
|
137
|
+
class: "SchemaReaper::ScanJob"
|
|
138
|
+
queue: default
|
|
139
|
+
active_job: true # explicit, not relying on class-ancestry auto-detection
|
|
140
|
+
YAML
|
|
141
|
+
|
|
142
|
+
full_path = File.join(destination_root, path)
|
|
143
|
+
if File.exist?(full_path) && File.read(full_path).include?(marker)
|
|
144
|
+
say_status :skip, "#{path} already has a schema_reaper_scan entry -- not duplicating it", :yellow
|
|
145
|
+
elsif File.exist?(full_path)
|
|
146
|
+
append_to_file path, entry
|
|
147
|
+
else
|
|
148
|
+
create_file path, entry.sub("\n\n", "")
|
|
149
|
+
end
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
# `gem "schema_reaper", group: :development` (or the equivalent
|
|
153
|
+
# `group :development do ... end` block form) in the app's own Gemfile
|
|
154
|
+
# -- see README's current install snippet. A dev-scoped gem is absent
|
|
155
|
+
# from `bundle install --without development test`, which most
|
|
156
|
+
# production deploy pipelines run, so the scheduled scan silently
|
|
157
|
+
# never runs.
|
|
158
|
+
#
|
|
159
|
+
# Parsed with Bundler's own DSL rather than line-scanning for
|
|
160
|
+
# "group: :development" -- a regex over raw lines misses the block
|
|
161
|
+
# form entirely (the gem's own line never mentions :development; the
|
|
162
|
+
# `group :development do` line above it does), and that block form is
|
|
163
|
+
# the more common style in practice, not an edge case.
|
|
164
|
+
def gemfile_scopes_schema_reaper_to_dev?
|
|
165
|
+
gemfile = File.join(destination_root, "Gemfile")
|
|
166
|
+
return false unless File.exist?(gemfile)
|
|
167
|
+
|
|
168
|
+
dep = Bundler::Dsl.evaluate(gemfile, nil, {}).dependencies.find { |d| d.name == "schema_reaper" }
|
|
169
|
+
dep && !dep.groups.include?(:default)
|
|
170
|
+
rescue StandardError
|
|
171
|
+
false # a Gemfile we can't parse shouldn't block the rest of the generator
|
|
172
|
+
end
|
|
173
|
+
|
|
174
|
+
# Checked against ApplicationMailer specifically, not SchemaReaper::Mailer
|
|
175
|
+
# -- at generate time the app is already booted (ActionMailer, and
|
|
176
|
+
# ApplicationMailer if the app has one, are loaded), so this reads the
|
|
177
|
+
# same `default_params[:from]` chain SchemaReaper::Mailer itself will
|
|
178
|
+
# inherit from once it's loaded.
|
|
179
|
+
def mailer_from_configured?
|
|
180
|
+
return false unless defined?(::ApplicationMailer) # no mailer to inherit from at all
|
|
181
|
+
|
|
182
|
+
::ApplicationMailer.default_params[:from].present?
|
|
183
|
+
rescue StandardError
|
|
184
|
+
true # can't determine -- don't nag over something we're unsure about
|
|
185
|
+
end
|
|
186
|
+
|
|
187
|
+
def app_identifier
|
|
188
|
+
Rails.application.class.module_parent_name.underscore
|
|
189
|
+
rescue StandardError
|
|
190
|
+
"app"
|
|
191
|
+
end
|
|
192
|
+
end
|
|
193
|
+
end
|
|
194
|
+
end
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# schema_reaper production automation.
|
|
4
|
+
#
|
|
5
|
+
# Neither of these is a secret -- leaking a teammate's email address or a
|
|
6
|
+
# webhook URL isn't a meaningful security event -- so this file is meant to
|
|
7
|
+
# be committed to git directly. The manual-trigger token is not: see the
|
|
8
|
+
# instructions `rails generate schema_reaper:install` printed for that one
|
|
9
|
+
# (it lives in Rails.application.credentials, not here).
|
|
10
|
+
#
|
|
11
|
+
# Uncomment and fill in at least one of these to start receiving reports.
|
|
12
|
+
# Both fire on every scheduled run when both are set -- this isn't a
|
|
13
|
+
# fallback, it's two audiences: a dev-facing chat channel and a formal inbox
|
|
14
|
+
# record for people who don't watch chat.
|
|
15
|
+
SchemaReaper::AlertConfig.configure do |config|
|
|
16
|
+
# config.emails = %w[dev1@example.com dev2@example.com]
|
|
17
|
+
# config.webhook_url = "https://hooks.slack.com/services/T000/B000/XXXX"
|
|
18
|
+
end
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Manual, on-demand trigger for the production schema_reaper scan -- enqueues
|
|
4
|
+
# the exact same job the schedule uses, so there is one code path to keep
|
|
5
|
+
# correct. Protected by a bearer token stored in encrypted credentials
|
|
6
|
+
# (config/credentials.yml.enc, key schema_reaper.trigger_token) rather than a
|
|
7
|
+
# DevOps-provisioned ENV var -- every Rails app already has the
|
|
8
|
+
# RAILS_MASTER_KEY needed to decrypt it, so this introduces zero new
|
|
9
|
+
# production configuration.
|
|
10
|
+
#
|
|
11
|
+
# Trigger it with (the token is printed once by the install generator --
|
|
12
|
+
# store it somewhere your team can find it, e.g. a password manager. It is
|
|
13
|
+
# deliberately NOT embedded in the scheduled report itself: that report goes
|
|
14
|
+
# to Slack/email, and a bearer token that authorizes a production action
|
|
15
|
+
# doesn't belong in either):
|
|
16
|
+
#
|
|
17
|
+
# curl -X POST https://your-app.example.com/internal/schema_scan \
|
|
18
|
+
# -H "Authorization: Bearer <token>"
|
|
19
|
+
class SchemaReaperController < ActionController::Base
|
|
20
|
+
# This is a token-authenticated API endpoint, not a session-backed form --
|
|
21
|
+
# Rails' CSRF protection is meant for the latter. Deliberately does NOT
|
|
22
|
+
# inherit from ::ApplicationController, and explicitly skips the
|
|
23
|
+
# authenticity-token check (`raise: false` so this doesn't itself blow up
|
|
24
|
+
# on an app where default_protect_from_forgery is off and the callback was
|
|
25
|
+
# never registered). Without this, a plain curl POST is rejected with
|
|
26
|
+
# ActionController::InvalidAuthenticityToken before the token check below
|
|
27
|
+
# even runs -- Rails wires `protect_from_forgery with: :exception` onto
|
|
28
|
+
# ActionController::Base itself for every app using Rails' post-5.2
|
|
29
|
+
# defaults, so every subclass inherits it, this one included.
|
|
30
|
+
skip_before_action :verify_authenticity_token, raise: false
|
|
31
|
+
|
|
32
|
+
# A valid (or leaked) token could otherwise be POSTed repeatedly, each hit
|
|
33
|
+
# enqueueing a full schema introspection against production. This is not
|
|
34
|
+
# per-caller rate limiting -- it's a single shared cooldown so a burst of
|
|
35
|
+
# triggers (accidental double-click, a leaked token being hammered) can't
|
|
36
|
+
# stack up scans faster than they can plausibly be useful. Rails.cache is
|
|
37
|
+
# whatever the app already has configured; on the default NullStore (no
|
|
38
|
+
# caching configured at all) this is a no-op, same as before this guard
|
|
39
|
+
# existed -- configure a real cache store to get actual protection.
|
|
40
|
+
TRIGGER_COOLDOWN = 5.minutes
|
|
41
|
+
|
|
42
|
+
def trigger
|
|
43
|
+
return head(:unauthorized) unless token_valid?
|
|
44
|
+
return head(:too_many_requests) unless acquire_trigger_lock
|
|
45
|
+
|
|
46
|
+
SchemaReaper::ScanJob.perform_later
|
|
47
|
+
head :accepted
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
private
|
|
51
|
+
|
|
52
|
+
def acquire_trigger_lock
|
|
53
|
+
Rails.cache.write("schema_reaper:manual_trigger_lock", true, unless_exist: true, expires_in: TRIGGER_COOLDOWN)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def token_valid?
|
|
57
|
+
expected = Rails.application.credentials.dig(:schema_reaper, :trigger_token)
|
|
58
|
+
return false if expected.blank?
|
|
59
|
+
|
|
60
|
+
provided = request.headers["Authorization"].to_s.delete_prefix("Bearer ").strip
|
|
61
|
+
return false if provided.empty?
|
|
62
|
+
|
|
63
|
+
ActiveSupport::SecurityUtils.secure_compare(provided, expected)
|
|
64
|
+
rescue ActiveSupport::EncryptedFile::MissingKeyError, ActiveSupport::MessageEncryptor::InvalidMessage
|
|
65
|
+
# RAILS_MASTER_KEY absent or credentials undecryptable in this
|
|
66
|
+
# environment -- fail closed (unauthorized), don't 500.
|
|
67
|
+
false
|
|
68
|
+
end
|
|
69
|
+
end
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SchemaReaper
|
|
4
|
+
# Where the production scan's report goes. Set once from a Rails
|
|
5
|
+
# initializer (config/initializers/schema_reaper.rb), typically written by
|
|
6
|
+
# `rails generate schema_reaper:install`:
|
|
7
|
+
#
|
|
8
|
+
# SchemaReaper::AlertConfig.configure do |config|
|
|
9
|
+
# config.emails = %w[dev1@example.com dev2@example.com]
|
|
10
|
+
# config.webhook_url = "https://hooks.slack.com/services/T000/B000/XXXX"
|
|
11
|
+
# end
|
|
12
|
+
#
|
|
13
|
+
# Neither field is a secret -- leaking a teammate's email address or a
|
|
14
|
+
# webhook URL isn't a meaningful security event -- so this is meant to be
|
|
15
|
+
# committed to git directly. The manual-trigger token is not: that one
|
|
16
|
+
# lives in Rails.application.credentials (see the install generator).
|
|
17
|
+
class AlertConfig
|
|
18
|
+
class << self
|
|
19
|
+
def configure
|
|
20
|
+
yield instance
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def instance
|
|
24
|
+
@instance ||= new
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
# Test helper -- config set in one example must not leak into the next.
|
|
28
|
+
def reset!
|
|
29
|
+
@instance = new
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
attr_accessor :emails, :webhook_url
|
|
34
|
+
|
|
35
|
+
def initialize
|
|
36
|
+
@emails = []
|
|
37
|
+
@webhook_url = nil
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def emails?
|
|
41
|
+
!emails.to_a.empty?
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def webhook?
|
|
45
|
+
!webhook_url.to_s.strip.empty?
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
# Whether Notifier has anywhere to send a report. A generator-scaffolded
|
|
49
|
+
# but unfilled-in config (both fields left commented out) is a safe
|
|
50
|
+
# no-op, not an error.
|
|
51
|
+
def configured?
|
|
52
|
+
emails? || webhook?
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
end
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SchemaReaper
|
|
4
|
+
# Inherits ApplicationMailer when the host app defines one (the normal
|
|
5
|
+
# case), so it picks up the app's own `default from:`, layout and delivery
|
|
6
|
+
# settings automatically -- "reuse the app's already-configured mailer,
|
|
7
|
+
# don't require a new SMTP setup" is the whole point of the email channel.
|
|
8
|
+
# Falls back to ActionMailer::Base directly for apps that don't.
|
|
9
|
+
class Mailer < (defined?(::ApplicationMailer) ? ::ApplicationMailer : ActionMailer::Base)
|
|
10
|
+
# Only applied when nothing upstream (ApplicationMailer or its own
|
|
11
|
+
# ancestors) already set one. Without this, `mail()` raises "SMTP From
|
|
12
|
+
# address may not be blank" -- and Notifier's rescue never sees it:
|
|
13
|
+
# deliver_later only enqueues, the actual render/from-validation happens
|
|
14
|
+
# later inside ActionMailer::MailDeliveryJob#perform, so an app that
|
|
15
|
+
# hits this path gets a bare, unbranded ActiveJob failure with no
|
|
16
|
+
# `[schema_reaper]` log line anywhere near it. This placeholder isn't a
|
|
17
|
+
# real deliverable address -- most SMTP relays will still reject or
|
|
18
|
+
# spam-flag mail from an unconfigured sender -- it only turns a hard
|
|
19
|
+
# crash into a normal delivery failure. The install generator warns at
|
|
20
|
+
# setup time when this fallback would apply, so the real fix (configure
|
|
21
|
+
# a `default from:`) doesn't have to be discovered from a stack trace.
|
|
22
|
+
default from: "schema_reaper@localhost" unless default_params[:from]
|
|
23
|
+
|
|
24
|
+
def report_email(to:, report:)
|
|
25
|
+
@report = report
|
|
26
|
+
mail(to: to, subject: "[schema_reaper] scan report") do |format|
|
|
27
|
+
format.text { render plain: @report }
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
end
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "net/http"
|
|
4
|
+
require "json"
|
|
5
|
+
require "stringio"
|
|
6
|
+
require_relative "reporters/markdown"
|
|
7
|
+
|
|
8
|
+
module SchemaReaper
|
|
9
|
+
# Delivers a completed scan's findings through whichever channels are
|
|
10
|
+
# configured in AlertConfig -- a Slack-compatible webhook POST and/or email
|
|
11
|
+
# via the host app's own ActionMailer setup. Both fire when both are
|
|
12
|
+
# configured; this isn't a fallback chain (schema reports aren't critical
|
|
13
|
+
# enough to need one), it's reaching two different audiences: a dev-facing
|
|
14
|
+
# chat channel and a more formal inbox record for people who don't watch
|
|
15
|
+
# chat.
|
|
16
|
+
#
|
|
17
|
+
# Delivery failures are logged, never raised -- a webhook endpoint being
|
|
18
|
+
# briefly down should not fail the scan job or retry-storm it.
|
|
19
|
+
class Notifier
|
|
20
|
+
# `mailer:` defaults to :auto rather than nil so an explicit
|
|
21
|
+
# `mailer: nil` (forcing "no mailer available", e.g. to test that path,
|
|
22
|
+
# or because a caller genuinely wants email delivery skipped) is
|
|
23
|
+
# respected rather than silently falling back to the auto-detected
|
|
24
|
+
# SchemaReaper::Mailer via `||`.
|
|
25
|
+
def initialize(findings, config: AlertConfig.instance, http: Net::HTTP, mailer: :auto)
|
|
26
|
+
@findings = findings
|
|
27
|
+
@config = config
|
|
28
|
+
@http = http
|
|
29
|
+
@mailer = mailer == :auto ? default_mailer : mailer
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def deliver
|
|
33
|
+
return unless @config.configured?
|
|
34
|
+
|
|
35
|
+
deliver_webhook if @config.webhook?
|
|
36
|
+
deliver_email if @config.emails?
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
private
|
|
40
|
+
|
|
41
|
+
def default_mailer
|
|
42
|
+
defined?(Mailer) ? Mailer : nil
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def report_text
|
|
46
|
+
@report_text ||= begin
|
|
47
|
+
io = StringIO.new
|
|
48
|
+
Reporters::Markdown.new(@findings, io: io).render
|
|
49
|
+
io.string
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
# Slack's incoming-webhook shape ({"text": ...}) is the most common
|
|
54
|
+
# receiver in practice; Discord's legacy webhook path also accepts a
|
|
55
|
+
# plain "content" body but that's a different key, and structured-alert
|
|
56
|
+
# services like PagerDuty need an entirely different schema (their
|
|
57
|
+
# Events API v2, not a text webhook) -- those need a different endpoint,
|
|
58
|
+
# not this one.
|
|
59
|
+
def deliver_webhook
|
|
60
|
+
uri = URI.parse(@config.webhook_url)
|
|
61
|
+
request = Net::HTTP::Post.new(uri)
|
|
62
|
+
request["Content-Type"] = "application/json"
|
|
63
|
+
request.body = JSON.generate(text: report_text)
|
|
64
|
+
|
|
65
|
+
client = @http.new(uri.host, uri.port)
|
|
66
|
+
client.use_ssl = uri.scheme == "https"
|
|
67
|
+
# A background job blocking indefinitely on a dead webhook host would
|
|
68
|
+
# back up the queue behind it -- bound the wait instead of relying on
|
|
69
|
+
# Net::HTTP's own (version-dependent) defaults.
|
|
70
|
+
client.open_timeout = 10
|
|
71
|
+
client.read_timeout = 10
|
|
72
|
+
client.request(request)
|
|
73
|
+
rescue StandardError => e
|
|
74
|
+
log_error("webhook delivery failed", e)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def deliver_email
|
|
78
|
+
unless @mailer
|
|
79
|
+
log_error("email delivery skipped", "SchemaReaper::Mailer is not loaded (ActionMailer not present?)")
|
|
80
|
+
return
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
@mailer.report_email(to: @config.emails, report: report_text).deliver_later
|
|
84
|
+
rescue StandardError => e
|
|
85
|
+
log_error("email delivery failed", e)
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
def log_error(message, error)
|
|
89
|
+
detail = error.is_a?(Exception) ? "#{error.class}: #{error.message}" : error.to_s
|
|
90
|
+
if defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger
|
|
91
|
+
Rails.logger.error("[schema_reaper] #{message}: #{detail}")
|
|
92
|
+
else
|
|
93
|
+
warn("[schema_reaper] #{message}: #{detail}")
|
|
94
|
+
end
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
end
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SchemaReaper
|
|
4
|
+
# Runs a scan and delivers the report through whatever channels AlertConfig
|
|
5
|
+
# has configured. Enqueued two ways: on the schedule an app generates via
|
|
6
|
+
# `rails generate schema_reaper:install` (whenever/sidekiq-cron), and
|
|
7
|
+
# on-demand by the generated manual-trigger controller -- same job either
|
|
8
|
+
# way, so there is exactly one code path to keep correct.
|
|
9
|
+
class ScanJob < ActiveJob::Base
|
|
10
|
+
queue_as :default
|
|
11
|
+
|
|
12
|
+
def perform
|
|
13
|
+
findings = Runner.new.run
|
|
14
|
+
Notifier.new(findings).deliver
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
end
|
|
@@ -23,4 +23,9 @@ namespace :schema_reaper do
|
|
|
23
23
|
history.record(findings)
|
|
24
24
|
pp history.trend
|
|
25
25
|
end
|
|
26
|
+
|
|
27
|
+
desc "Enqueue the production scan+alert job (this is what the scheduled cron entry calls)"
|
|
28
|
+
task alert: :environment do
|
|
29
|
+
SchemaReaper::ScanJob.perform_later
|
|
30
|
+
end
|
|
26
31
|
end
|
data/lib/schema_reaper.rb
CHANGED
|
@@ -29,6 +29,10 @@ require_relative "schema_reaper/baseline"
|
|
|
29
29
|
require_relative "schema_reaper/history"
|
|
30
30
|
require_relative "schema_reaper/migration_generator"
|
|
31
31
|
require_relative "schema_reaper/runner"
|
|
32
|
+
require_relative "schema_reaper/alert_config"
|
|
33
|
+
require_relative "schema_reaper/notifier"
|
|
34
|
+
require_relative "schema_reaper/mailer" if defined?(ActionMailer::Base)
|
|
35
|
+
require_relative "schema_reaper/scan_job" if defined?(ActiveJob::Base)
|
|
32
36
|
require_relative "schema_reaper/railtie" if defined?(Rails::Railtie)
|
|
33
37
|
|
|
34
38
|
# Finds columns, indexes and tables that a Rails/ActiveRecord app no longer
|
data/schema_reaper.gemspec
CHANGED
|
@@ -47,6 +47,10 @@ Gem::Specification.new do |spec|
|
|
|
47
47
|
spec.add_dependency "prism", ">= 0.19", "< 2.0"
|
|
48
48
|
spec.add_dependency "thor", "~> 1.3"
|
|
49
49
|
|
|
50
|
+
spec.add_development_dependency "actionmailer", ">= 6.1", "< 9.0"
|
|
51
|
+
spec.add_development_dependency "activejob", ">= 6.1", "< 9.0"
|
|
50
52
|
spec.add_development_dependency "activerecord", ">= 6.1", "< 9.0"
|
|
51
53
|
spec.add_development_dependency "pg", "~> 1.5"
|
|
54
|
+
spec.add_development_dependency "railties", ">= 6.1", "< 9.0"
|
|
55
|
+
spec.add_development_dependency "simplecov", "~> 0.22"
|
|
52
56
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: schema_reaper
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version:
|
|
4
|
+
version: 2.0.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- aksshatt
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: exe
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2026-09-
|
|
12
|
+
date: 2026-09-23 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
name: prism
|
|
@@ -45,6 +45,46 @@ dependencies:
|
|
|
45
45
|
- - "~>"
|
|
46
46
|
- !ruby/object:Gem::Version
|
|
47
47
|
version: '1.3'
|
|
48
|
+
- !ruby/object:Gem::Dependency
|
|
49
|
+
name: actionmailer
|
|
50
|
+
requirement: !ruby/object:Gem::Requirement
|
|
51
|
+
requirements:
|
|
52
|
+
- - ">="
|
|
53
|
+
- !ruby/object:Gem::Version
|
|
54
|
+
version: '6.1'
|
|
55
|
+
- - "<"
|
|
56
|
+
- !ruby/object:Gem::Version
|
|
57
|
+
version: '9.0'
|
|
58
|
+
type: :development
|
|
59
|
+
prerelease: false
|
|
60
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
61
|
+
requirements:
|
|
62
|
+
- - ">="
|
|
63
|
+
- !ruby/object:Gem::Version
|
|
64
|
+
version: '6.1'
|
|
65
|
+
- - "<"
|
|
66
|
+
- !ruby/object:Gem::Version
|
|
67
|
+
version: '9.0'
|
|
68
|
+
- !ruby/object:Gem::Dependency
|
|
69
|
+
name: activejob
|
|
70
|
+
requirement: !ruby/object:Gem::Requirement
|
|
71
|
+
requirements:
|
|
72
|
+
- - ">="
|
|
73
|
+
- !ruby/object:Gem::Version
|
|
74
|
+
version: '6.1'
|
|
75
|
+
- - "<"
|
|
76
|
+
- !ruby/object:Gem::Version
|
|
77
|
+
version: '9.0'
|
|
78
|
+
type: :development
|
|
79
|
+
prerelease: false
|
|
80
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
81
|
+
requirements:
|
|
82
|
+
- - ">="
|
|
83
|
+
- !ruby/object:Gem::Version
|
|
84
|
+
version: '6.1'
|
|
85
|
+
- - "<"
|
|
86
|
+
- !ruby/object:Gem::Version
|
|
87
|
+
version: '9.0'
|
|
48
88
|
- !ruby/object:Gem::Dependency
|
|
49
89
|
name: activerecord
|
|
50
90
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -79,6 +119,40 @@ dependencies:
|
|
|
79
119
|
- - "~>"
|
|
80
120
|
- !ruby/object:Gem::Version
|
|
81
121
|
version: '1.5'
|
|
122
|
+
- !ruby/object:Gem::Dependency
|
|
123
|
+
name: railties
|
|
124
|
+
requirement: !ruby/object:Gem::Requirement
|
|
125
|
+
requirements:
|
|
126
|
+
- - ">="
|
|
127
|
+
- !ruby/object:Gem::Version
|
|
128
|
+
version: '6.1'
|
|
129
|
+
- - "<"
|
|
130
|
+
- !ruby/object:Gem::Version
|
|
131
|
+
version: '9.0'
|
|
132
|
+
type: :development
|
|
133
|
+
prerelease: false
|
|
134
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
135
|
+
requirements:
|
|
136
|
+
- - ">="
|
|
137
|
+
- !ruby/object:Gem::Version
|
|
138
|
+
version: '6.1'
|
|
139
|
+
- - "<"
|
|
140
|
+
- !ruby/object:Gem::Version
|
|
141
|
+
version: '9.0'
|
|
142
|
+
- !ruby/object:Gem::Dependency
|
|
143
|
+
name: simplecov
|
|
144
|
+
requirement: !ruby/object:Gem::Requirement
|
|
145
|
+
requirements:
|
|
146
|
+
- - "~>"
|
|
147
|
+
- !ruby/object:Gem::Version
|
|
148
|
+
version: '0.22'
|
|
149
|
+
type: :development
|
|
150
|
+
prerelease: false
|
|
151
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
152
|
+
requirements:
|
|
153
|
+
- - "~>"
|
|
154
|
+
- !ruby/object:Gem::Version
|
|
155
|
+
version: '0.22'
|
|
82
156
|
description: |-
|
|
83
157
|
schema_reaper scans a Rails + PostgreSQL app for schema debt that's easy to accumulate and hard to find by hand.
|
|
84
158
|
|
|
@@ -108,7 +182,11 @@ files:
|
|
|
108
182
|
- RELEASE_CHECKLIST.md
|
|
109
183
|
- Rakefile
|
|
110
184
|
- exe/schema_reaper
|
|
185
|
+
- lib/generators/schema_reaper/install/install_generator.rb
|
|
186
|
+
- lib/generators/schema_reaper/install/templates/initializer.rb.tt
|
|
187
|
+
- lib/generators/schema_reaper/install/templates/schema_reaper_controller.rb.tt
|
|
111
188
|
- lib/schema_reaper.rb
|
|
189
|
+
- lib/schema_reaper/alert_config.rb
|
|
112
190
|
- lib/schema_reaper/analyzers/always_null_column.rb
|
|
113
191
|
- lib/schema_reaper/analyzers/base.rb
|
|
114
192
|
- lib/schema_reaper/analyzers/dead_column.rb
|
|
@@ -126,7 +204,9 @@ files:
|
|
|
126
204
|
- lib/schema_reaper/gem_awareness.rb
|
|
127
205
|
- lib/schema_reaper/history.rb
|
|
128
206
|
- lib/schema_reaper/introspect/postgres.rb
|
|
207
|
+
- lib/schema_reaper/mailer.rb
|
|
129
208
|
- lib/schema_reaper/migration_generator.rb
|
|
209
|
+
- lib/schema_reaper/notifier.rb
|
|
130
210
|
- lib/schema_reaper/railtie.rb
|
|
131
211
|
- lib/schema_reaper/reporters/ansi.rb
|
|
132
212
|
- lib/schema_reaper/reporters/bytes.rb
|
|
@@ -140,6 +220,7 @@ files:
|
|
|
140
220
|
- lib/schema_reaper/reporters/trend.rb
|
|
141
221
|
- lib/schema_reaper/runner.rb
|
|
142
222
|
- lib/schema_reaper/runtime.rb
|
|
223
|
+
- lib/schema_reaper/scan_job.rb
|
|
143
224
|
- lib/schema_reaper/schema.rb
|
|
144
225
|
- lib/schema_reaper/static/scanner.rb
|
|
145
226
|
- lib/schema_reaper/tasks/schema_reaper.rake
|