schema_reaper 1.0.16 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,7 +1,18 @@
1
- # schema_reaper
1
+ <div align="center">
2
2
 
3
- Find dead columns, dead tables, unused indexes and other schema dead-weight in
4
- Rails / ActiveRecord apps — then remove them safely.
3
+ <img src="docs/assets/hero.svg" width="100%" alt="schema_reaper — find and safely remove the dead columns, tables and indexes your Rails + PostgreSQL app no longer uses" />
4
+
5
+ [![Gem Version](https://img.shields.io/gem/v/schema_reaper?color=cc342d&logo=rubygems&logoColor=white&label=gem)](https://rubygems.org/gems/schema_reaper)
6
+ [![Gem Downloads](https://img.shields.io/gem/dt/schema_reaper?color=cc342d&logo=rubygems&logoColor=white&label=downloads)](https://rubygems.org/gems/schema_reaper)
7
+ [![CI](https://github.com/aksshatt/schema_reaper/actions/workflows/main.yml/badge.svg)](https://github.com/aksshatt/schema_reaper/actions/workflows/main.yml)
8
+ [![License: MIT](https://img.shields.io/badge/license-MIT-yellow.svg)](LICENSE.txt)
9
+ [![Ruby](https://img.shields.io/badge/ruby-%3E%3D%202.7-CC342D?logo=ruby&logoColor=white)](schema_reaper.gemspec)
10
+ [![PostgreSQL](https://img.shields.io/badge/postgres-only%20(for%20now)-336791?logo=postgresql&logoColor=white)](#install)
11
+
12
+ | ⚡ [Quickstart](#quickstart) | ⚙️ [Usage](#usage) | 🤖 [Production automation](#production-automation) | 🔍 [Analyzers](#analyzers) | 🛡️ [Safety model](#safety-model) | ✅ [CI](#ci) | 🧩 [Configuration](#configuration) | 💼 [Pro](#pro-for-teams) |
13
+ |:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
14
+
15
+ </div>
5
16
 
6
17
  `schema_reaper` reads your **live PostgreSQL schema and planner statistics** and
7
18
  cross-references them against a **static scan of your codebase** (Ruby via the
@@ -10,129 +21,408 @@ Prism AST, plus views and SQL string literals). Optionally it also fuses in a
10
21
  production. Every finding is scored by confidence and severity, carries an
11
22
  estimate of the disk it reclaims, and comes with a concrete fix.
12
23
 
13
- ## Install
24
+ It finds three kinds of schema debt:
25
+
26
+ <picture>
27
+ <source media="(prefers-color-scheme: dark)" srcset="docs/assets/debt-dark.svg">
28
+ <img src="docs/assets/debt-light.svg" width="100%" alt="Dead weight — columns and tables nothing references any more (dead_column, dead_table). Index trouble — indexes nobody queries, indexes a wider one already covers, and foreign keys with no index (unused_index, duplicate_index, missing_fk_index). Degenerate data — columns that are always NULL or hold one value in every row (always_null_column, single_value_column).">
29
+ </picture>
30
+
31
+ ## Quickstart
14
32
 
15
33
  ```ruby
16
34
  # Gemfile
17
- gem "schema_reaper", group: :development
35
+ gem "schema_reaper"
18
36
  ```
19
37
 
20
- ```
38
+ ```sh
21
39
  bundle install
40
+ bundle exec schema_reaper scan
41
+ ```
42
+
43
+ In a Rails app that's it — the connection comes from `config/database.yml`.
44
+ You get a report like this:
45
+
46
+ <p align="center">
47
+ <img src="docs/assets/terminal.svg" width="100%" alt="Terminal output of bundle exec schema_reaper scan: 3 findings across 2 tables — users.team_id has no covering index (90%, medium, fix: add_index :users, :team_id); users.api_key is NULL in every row (85%, high, 46.9 KB reclaimable); table stale_exports has no model or query reference and holds 0 rows (85%, high, fix: drop_table :stale_exports after confirming no external consumer)." />
48
+ </p>
49
+
50
+ > [!TIP]
51
+ > Scan a database that has served real traffic — a production read-replica or a
52
+ > recent snapshot. `unused_index` needs query history (it skips itself on a
53
+ > fresh database and tells you why), and the data analyzers read `pg_stats`,
54
+ > which PostgreSQL only fills in after `ANALYZE`.
55
+
56
+ ## How it works
57
+
58
+ <picture>
59
+ <source media="(prefers-color-scheme: dark)" srcset="docs/assets/pipeline-dark.svg">
60
+ <img src="docs/assets/pipeline-light.svg" width="100%" alt="Three inputs — the live PostgreSQL schema and statistics, a static scan of your codebase, and an optional runtime signal — feed seven analyzers. Their scored findings become reports in four formats, a CI baseline gate, scheduled webhook and email reports, and staged removal migrations.">
61
+ </picture>
62
+
63
+ Findings that say the same thing about different tables are rolled up into one
64
+ entry; when several analyzers flag the same column, the strongest finding wins
65
+ and notes the others that agreed. `schema_reaper` never changes your database —
66
+ it only reads, and hands you migrations to review.
67
+
68
+ ## Install
69
+
70
+ ```ruby
71
+ # Gemfile
72
+ gem "schema_reaper"
22
73
  ```
23
74
 
24
- Requires Ruby >= 2.7 and PostgreSQL. The database connection is resolved in
25
- this order:
75
+ | requirement | notes |
76
+ |---|---|
77
+ | Ruby >= 2.7 | CI-tested on 2.7 – 3.3 |
78
+ | PostgreSQL + the `pg` gem | uses your app's own `pg` (every Rails + PostgreSQL app already bundles it); add `gem "pg"` for standalone CLI use. Tables in the `public` schema are scanned. MySQL is on the [roadmap](#roadmap). |
79
+ | Rails | optional for the CLI; the rake tasks and production automation need it, and the runtime tracker needs ActiveRecord |
80
+
81
+ > [!IMPORTANT]
82
+ > **Only scanning locally or in CI?** Scope it out of your production bundle:
83
+ > `gem "schema_reaper", group: :development`.
84
+ >
85
+ > **Planning to use [production automation](#production-automation)?** Leave it
86
+ > unscoped. Most deploys exclude the development and test groups
87
+ > (`BUNDLE_WITHOUT=development:test`), so a dev-scoped gem is simply absent when
88
+ > the scheduled scan tries to run. The install generator warns you about this.
89
+
90
+ <details>
91
+ <summary><b>How the database connection is resolved</b></summary>
92
+
93
+ <br>
26
94
 
27
95
  1. `database_url:` in `.schema_reaper.yml`
28
96
  2. `ENV["DATABASE_URL"]`
29
- 3. `config/database.yml` for the current environment (`SCHEMA_REAPER_ENV` /
30
- `RAILS_ENV`, default `development`) — ERB and YAML aliases are handled, as
31
- are Rails 6+ multi-database sections
97
+ 3. `config/database.yml` for the current environment (`SCHEMA_REAPER_ENV`, then
98
+ `RAILS_ENV`, then `RACK_ENV`, default `development`) — ERB and YAML aliases
99
+ are handled, as are Rails 6+ multi-database sections (the `primary` entry is
100
+ used)
32
101
 
33
- So in a Rails app, `bundle exec schema_reaper scan` works with no setup.
102
+ </details>
34
103
 
35
104
  ## Usage
36
105
 
37
- ```
106
+ ```sh
38
107
  bundle exec schema_reaper scan # grouped terminal report
39
108
  bundle exec schema_reaper scan --format markdown # PR-comment table
40
109
  bundle exec schema_reaper scan --format sarif # GitHub code scanning
41
110
  bundle exec schema_reaper scan --format json
42
- bundle exec schema_reaper scan --ci # exit 1 on new findings
111
+ bundle exec schema_reaper scan --ci # exit 1 on findings not in the baseline
43
112
  bundle exec schema_reaper scan --min-confidence 0.8
44
- bundle exec schema_reaper scan --no-color # force plain output
45
113
  bundle exec schema_reaper baseline # accept current findings
46
114
  bundle exec schema_reaper trend # snapshot + progress delta
47
115
  bundle exec schema_reaper generate-migration users legacy_api_token
48
116
  ```
49
117
 
50
- The `scan` report rolls up findings that say the same thing about different
51
- tables, then groups the rest by table and sorts by confidence:
118
+ <details>
119
+ <summary><b>All <code>scan</code> options</b></summary>
120
+
121
+ <br>
122
+
123
+ | option | default | what it does |
124
+ |---|---|---|
125
+ | `--format` | `table` | `table`, `json`, `markdown` or `sarif` |
126
+ | `--ci` | off | exit 1 when a finding isn't in `.schema_reaper/baseline.json` |
127
+ | `--min-confidence N` | `0.0` | hide findings below this confidence (0.0 – 1.0) |
128
+ | `--record` | off | also append this run to the history log used by `trend` |
129
+ | `--color` / `--no-color` | auto | force colour on or off for the table report |
130
+ | `--config PATH` | `.schema_reaper.yml` | config file to load (works on every command) |
131
+
132
+ Colour is automatic on a terminal, and off when output is piped or `NO_COLOR` is set.
133
+
134
+ </details>
135
+
136
+ <details>
137
+ <summary><b>Rake tasks (Rails)</b></summary>
138
+
139
+ <br>
140
+
141
+ The railtie adds these to any Rails app with the gem in its bundle:
142
+
143
+ | task | what it does |
144
+ |---|---|
145
+ | `rake schema_reaper:scan` | same as `schema_reaper scan`; pick a format with `FORMAT=json` etc. |
146
+ | `rake schema_reaper:baseline` | write the current findings to the baseline file |
147
+ | `rake schema_reaper:trend` | record a snapshot and print the trend |
148
+ | `rake schema_reaper:alert` | run one production scan and send the report — what the [schedule](#production-automation) calls |
149
+
150
+ </details>
151
+
152
+ ## Production automation
153
+
154
+ *New in v2.0.* Run `schema_reaper` unattended in production and have the report
155
+ land in your team's chat and inbox — no DevOps ticket, and no server access
156
+ beyond a normal deploy.
157
+
158
+ ```sh
159
+ bin/rails generate schema_reaper:install
160
+ ```
161
+
162
+ <picture>
163
+ <source media="(prefers-color-scheme: dark)" srcset="docs/assets/automation-dark.svg">
164
+ <img src="docs/assets/automation-light.svg" width="100%" alt="whenever or cron runs rake schema_reaper:alert, sidekiq-cron enqueues the job directly, and the token-protected manual trigger POST /internal/schema_scan enqueues it on demand. All three run SchemaReaper::ScanJob, which scans the live database and your code; the Notifier sends the report to a Slack-compatible webhook and by email through your ApplicationMailer.">
165
+ </picture>
166
+
167
+ | generated | purpose |
168
+ |---|---|
169
+ | `config/initializers/schema_reaper.rb` | where reports go (webhook and/or email) |
170
+ | an entry in `config/schedule.rb` **or** `config/schedule.yml` | the quarterly scheduled scan (whenever or sidekiq-cron) |
171
+ | `app/controllers/schema_reaper_controller.rb` + a route | a token-protected "scan now" endpoint |
172
+ | a trigger token, printed once | you paste it into your encrypted credentials |
173
+
174
+ Re-running the generator is safe: every step checks for its own marker and
175
+ skips itself rather than duplicating anything. Commit the files, deploy, and
176
+ check it works end to end with one manual run in production:
177
+
178
+ ```sh
179
+ bin/rails schema_reaper:alert
180
+ ```
181
+
182
+ ### 1. Where reports go
52
183
 
184
+ The generated initializer is commented out, so an unfilled config is a safe
185
+ no-op, not an error. Uncomment one or both:
186
+
187
+ ```ruby
188
+ SchemaReaper::AlertConfig.configure do |config|
189
+ # config.emails = %w[dev1@example.com dev2@example.com]
190
+ # config.webhook_url = "https://hooks.slack.com/services/T000/B000/XXXX"
191
+ end
192
+ ```
193
+
194
+ Both channels fire when both are set — not a fallback chain, but two audiences:
195
+ a dev-facing chat channel, and an inbox record for people who don't watch chat.
196
+
197
+ - **Webhook** — a Slack-compatible `{"text": "..."}` JSON POST carrying the
198
+ Markdown report, with a 10-second timeout.
199
+ - **Email** — sent by `SchemaReaper::Mailer`, which inherits your app's
200
+ `ApplicationMailer` (its `default from:`, delivery settings and so on), or
201
+ `ActionMailer::Base` if you don't have one. Delivered with `deliver_later`.
202
+
203
+ Delivery failures are logged with a `[schema_reaper]` prefix and never raised,
204
+ so a flaky endpoint can't fail or retry-storm the scan.
205
+
206
+ ### 2. The schedule
207
+
208
+ The generator looks at your `Gemfile.lock` and uses whichever scheduler you
209
+ already run. The default cadence is **quarterly**; edit the generated entry to
210
+ change it.
211
+
212
+ | you use | the generator adds | also needed |
213
+ |---|---|---|
214
+ | [whenever](https://github.com/javan/whenever) | an `every 3.months` block running `rake "schema_reaper:alert"` in `config/schedule.rb` | whenever only writes your crontab when `whenever --update-crontab` runs on the server — usually via its Capistrano recipe. Platforms without a crontab (Heroku, most container hosts) need one of the other rows. |
215
+ | [sidekiq-cron](https://github.com/sidekiq-cron/sidekiq-cron) | a `schema_reaper_scan` entry in `config/schedule.yml` (`0 4 1 */3 *` — 04:00 on the 1st, every 3 months) that enqueues `SchemaReaper::ScanJob` | nothing — sidekiq-cron ≥ 1.6 loads `config/schedule.yml` automatically |
216
+ | neither | nothing — it prints the command instead of guessing | point any scheduler (cron, Heroku Scheduler, a Kubernetes CronJob) at `bin/rails schema_reaper:alert` |
217
+
218
+ `rake schema_reaper:alert` enqueues the scan on your ActiveJob backend. If that
219
+ backend is Rails' in-process `:async` adapter (the default before Rails 8 unless
220
+ you've set one up), the job would die with the short-lived rake process — so
221
+ the task runs the scan inline instead, email included.
222
+
223
+ ### 3. The manual trigger
224
+
225
+ For "scan now" without shelling into a box:
226
+
227
+ ```sh
228
+ curl -X POST https://your-app.example.com/internal/schema_scan \
229
+ -H "Authorization: Bearer <token>"
53
230
  ```
54
- schema_reaper 5 findings across 2 tables
55
- missing_fk_index 3 · always_null_column 1 · dead_column 1
56
- ~93.8 KB reclaimable
57
-
58
- users
59
- █████ 90% medium missing_fk_index team_id
60
- team_id is a foreign key with no covering index
61
- → add_index :users, :team_id
62
- ████░ 85% high always_null_column api_key 46.9 KB
63
- pg_stats.null_frac = 1.0 across ~3000 row(s) · column carries no data
64
- → verify with `SELECT count(api_key) FROM users` then stage a removal
65
-
66
- stale_exports
67
- ████░ 85% high dead_table
68
- no model or query reference · table holds ~0 row(s)
69
- → confirm no external consumer, then `drop_table :stale_exports`
70
-
71
- high 2 medium 1 low 2
231
+
232
+ | response | meaning |
233
+ |---|---|
234
+ | `202 Accepted` | scan enqueued — the report arrives through the channels above |
235
+ | `401 Unauthorized` | missing or wrong token, or no token configured in this environment |
236
+ | `429 Too Many Requests` | a scan was already triggered in the last 5 minutes |
237
+
238
+ The generator prints the token once, with the exact snippet to paste into
239
+ `bin/rails credentials:edit`:
240
+
241
+ ```yaml
242
+ schema_reaper:
243
+ trigger_token: <token>
72
244
  ```
73
245
 
74
- Colour is automatic on a terminal, off when piped or `NO_COLOR` is set.
246
+ Credentials are decrypted with the `RAILS_MASTER_KEY` your app already has, so
247
+ this adds no new production configuration. The token is checked with a
248
+ constant-time comparison. The controller deliberately doesn't inherit your
249
+ `ApplicationController` and skips CSRF (it's a token-authenticated API endpoint,
250
+ not a form), so your own auth filters don't apply to it.
251
+
252
+ > [!NOTE]
253
+ > The 5-minute cooldown lives in `Rails.cache`, so it's only as shared as your
254
+ > cache store. Redis, Memcached or Solid Cache give one cooldown for the whole
255
+ > app; a file or memory store gives one per host or process; `:null_store`
256
+ > turns it off.
257
+
258
+ ### 4. Install-time warnings
259
+
260
+ Two misconfigurations would otherwise fail silently much later, so the
261
+ generator flags them when it runs:
75
262
 
76
- In a Rails app the railtie also gives you
77
- `rake schema_reaper:scan|baseline|trend` (with `FORMAT=`).
263
+ - the gem is scoped to `group: :development` — the scheduled scan would never
264
+ run in production
265
+ - neither `ApplicationMailer` nor `config.action_mailer.default_options` sets a
266
+ `from:` address — the mailer would fall back to a placeholder sender that most
267
+ SMTP relays reject, so email reports would never arrive
268
+
269
+ > [!WARNING]
270
+ > **Still on 2.0.0?** Two production-automation bugs are fixed on `main` and
271
+ > ship in the next release:
272
+ > - Report emails ignore `ApplicationMailer` and are sent from
273
+ > `schema_reaper@localhost`. Work around it by adding
274
+ > `SchemaReaper::Mailer.default from: "you@your-domain.com"` to
275
+ > `config/initializers/schema_reaper.rb`.
276
+ > - With the `:async` ActiveJob adapter, `rake schema_reaper:alert` enqueues a
277
+ > job that never runs. Use a persistent backend (Sidekiq, GoodJob, Solid
278
+ > Queue, …) for the whenever and plain-cron paths.
279
+
280
+ <details>
281
+ <summary><b>Why not just run it in CI?</b></summary>
282
+
283
+ <br>
284
+
285
+ [`--format sarif` in CI](#ci) is for PR- and staging-level checks. GitHub-hosted
286
+ runners have no network path to your production database by default, and a CI
287
+ database has no production traffic statistics. Production automation is the path
288
+ for watching the real, live database.
289
+
290
+ </details>
78
291
 
79
292
  ## Analyzers
80
293
 
81
294
  | type | what it flags | main signal |
82
295
  |---|---|---|
83
296
  | `dead_column` | column no code path references | static scan (+ runtime) |
84
- | `dead_table` | table with no model/query reference | static scan + row count |
85
- | `unused_index` | non-unique index, `idx_scan = 0` | `pg_stat_user_indexes` |
86
- | `duplicate_index` | index that is a prefix of a wider one | schema shape |
87
- | `missing_fk_index` | `*_id` / FK column with no index | schema shape |
297
+ | `dead_table` | table with no model or query reference | static scan + row count |
298
+ | `unused_index` | non-unique index with `idx_scan = 0` (needs query history) | `pg_stat_user_indexes` |
299
+ | `duplicate_index` | index that exactly duplicates another, or is a prefix of a wider one | schema shape |
300
+ | `missing_fk_index` | `*_id` / foreign-key column with no index (polymorphic pairs need a `(type, id)` index) | schema shape |
88
301
  | `always_null_column` | `null_frac = 1.0` — no data at all | `pg_stats` |
89
- | `single_value_column` | one distinct value on a large table | `pg_stats` |
302
+ | `single_value_column` | one distinct value on a table of 500+ rows | `pg_stats` |
90
303
 
91
- Columns owned by common gems (devise, paper_trail, activestorage, actiontext,
92
- friendly_id, audited, pg_search, ahoy_matey, paranoia family) are whitelisted
93
- automatically when the gem is in your bundle.
304
+ Columns and tables owned by common gems are whitelisted automatically when the
305
+ gem is in your bundle: **devise**, **devise-api**, **paper_trail**,
306
+ **audited**, **friendly_id**, **paranoia** / **acts_as_paranoid**,
307
+ **activestorage**, **actiontext**, **pg_search**, **ahoy_matey** and
308
+ **activeadmin**.
94
309
 
95
310
  ## Runtime signal (optional, raises confidence)
96
311
 
97
312
  Static analysis alone can't see metaprogrammed access, so `dead_column`
98
- confidence is capped at **0.6** without runtime data. To lift the cap:
313
+ confidence is capped at **0.6** without runtime data. To lift the cap, sample
314
+ real column reads:
99
315
 
100
316
  ```ruby
101
- # config/initializers or manually
317
+ # config/initializers/schema_reaper_tracker.rb
102
318
  SchemaReaper::Runtime::Tracker.install!(
103
319
  store: SchemaReaper::Runtime::Store.new(path: ".schema_reaper/runtime.jsonl"),
104
320
  sample_rate: 0.05
105
321
  )
106
322
  ```
107
323
 
108
- or, in Rails, boot with `SCHEMA_REAPER_TRACK=1`. Let it run in staging or
324
+ or, in Rails, boot with `SCHEMA_REAPER_TRACK=1` (and optionally
325
+ `SCHEMA_REAPER_SAMPLE=0.05` for the sample rate). Let it run in staging or
109
326
  production for a couple of weeks. A column unseen in **both** code and
110
- >= 14 observed days of runtime data reaches ~0.9 confidence.
327
+ ≥ 14 observed days of runtime data (`min_age_days` in
328
+ [`.schema_reaper.yml`](#configuration)) reaches **0.9** confidence (0.8 if it's
329
+ `NOT NULL`).
330
+
331
+ > [!NOTE]
332
+ > This is separate from [production automation](#production-automation): the
333
+ > tracker raises confidence in the findings a scan already makes, while
334
+ > production automation runs the scan on a schedule and delivers the report.
335
+ > Most teams want both.
111
336
 
112
337
  ## Safety model
113
338
 
114
- `schema_reaper` never drops anything itself. `generate-migration` emits a pair:
339
+ <picture>
340
+ <source media="(prefers-color-scheme: dark)" srcset="docs/assets/safety-dark.svg">
341
+ <img src="docs/assets/safety-light.svg" width="100%" alt="generate-migration writes two migrations. Step 1 adds the column to ignored_columns and is deployed; nothing is dropped. Once it has soaked in production and nothing reads the column, step 2 runs remove_column, which is irreversible.">
342
+ </picture>
115
343
 
116
- 1. **Ignore** — you add `self.ignored_columns += %w[col]` to the model and
117
- deploy. Nothing is dropped.
118
- 2. **Drop** — run only after step 1 has soaked in production and nothing broke.
344
+ `schema_reaper` never drops anything itself. For a dead column,
345
+ `generate-migration users legacy_api_token` writes a pair:
119
346
 
120
- `always_null_column` / `single_value_column` fixes ask you to confirm with a
121
- `SELECT` first.
347
+ 1. **`…_ignore_users_legacy_api_token.rb`** — a no-op migration that reminds
348
+ you to add `self.ignored_columns += %w[legacy_api_token]` to the model. Deploy
349
+ that. Nothing is dropped; ActiveRecord just stops selecting the column.
350
+ 2. **`…_drop_users_legacy_api_token.rb`** — the `remove_column`. Run it only
351
+ after step 1 has soaked in production and nothing broke. Its `down` raises
352
+ `ActiveRecord::IrreversibleMigration` on purpose.
353
+
354
+ The data-driven fixes are cautious too: `always_null_column` asks you to
355
+ confirm with a `SELECT count(...)` first, `single_value_column` to check the
356
+ value isn't a meaningful default, and `dead_table` to rule out external
357
+ consumers.
122
358
 
123
359
  ## CI
124
360
 
361
+ <picture>
362
+ <source media="(prefers-color-scheme: dark)" srcset="docs/assets/ci-dark.svg">
363
+ <img src="docs/assets/ci-light.svg" width="100%" alt="On each pull request, schema_reaper scan --ci compares the findings with the committed baseline.json. It passes when every finding is already in the baseline and exits 1 when the change adds new dead weight.">
364
+ </picture>
365
+
125
366
  ```yaml
126
- # .github/workflows/schema_reaper.yml
127
- - run: bundle exec schema_reaper scan --ci --format sarif > reaper.sarif
128
- - uses: github/codeql-action/upload-sarif@v3
129
- with: { sarif_file: reaper.sarif }
367
+ # .github/workflows/schema_reaper.yml (the relevant parts)
368
+ permissions:
369
+ contents: read
370
+ security-events: write # required by upload-sarif
371
+
372
+ # ...job setup: Ruby, a PostgreSQL service, DATABASE_URL...
373
+ steps:
374
+ - run: bin/rails db:schema:load
375
+ - run: bundle exec schema_reaper scan --ci --format sarif > reaper.sarif
376
+ - uses: github/codeql-action/upload-sarif@v4
377
+ if: always() # upload even when --ci fails the step above
378
+ with: { sarif_file: reaper.sarif }
130
379
  ```
131
380
 
132
- Commit `.schema_reaper/baseline.json` so the job fails only when a change adds
133
- *new* dead weight.
381
+ Commit `.schema_reaper/baseline.json` (from `schema_reaper baseline`) so the job
382
+ fails only when a change adds *new* dead weight.
383
+
384
+ > [!NOTE]
385
+ > A freshly loaded CI database has no rows, statistics or query history, so CI
386
+ > catches the schema- and code-shaped findings (`dead_column`, `dead_table`,
387
+ > `duplicate_index`, `missing_fk_index`). The data-driven ones need a real
388
+ > database — that's what [production automation](#production-automation) is for.
389
+
390
+ ## Configuration
391
+
392
+ Every key is optional. Drop a `.schema_reaper.yml` in the project root to
393
+ override any of these defaults.
134
394
 
135
- ## Custom analyzers
395
+ > [!CAUTION]
396
+ > A list you set **replaces** the default list rather than adding to it. Setting
397
+ > `ignore: { tables: [legacy_audit] }` stops ignoring `schema_migrations` and
398
+ > `ar_internal_metadata`, and `always_keep_columns: [uuid]` stops protecting
399
+ > `created_at`, `updated_at` and `type`. Repeat the defaults you still want.
400
+
401
+ <details>
402
+ <summary><b>.schema_reaper.yml — all keys with their defaults</b></summary>
403
+
404
+ <br>
405
+
406
+ ```yaml
407
+ database_url: # falls back to DATABASE_URL, then config/database.yml
408
+ database_yml: config/database.yml
409
+ scan_paths: [app, lib, config]
410
+ view_globs: ["app/**/*.erb", "app/**/*.haml", "app/**/*.slim", "app/**/*.jbuilder"]
411
+ ignore:
412
+ tables: [schema_migrations, ar_internal_metadata]
413
+ columns: [] # exact names, or "/regex/" patterns
414
+ always_keep_columns: [id, created_at, updated_at, type]
415
+ gem_awareness: true # auto-whitelist columns owned by known gems
416
+ min_age_days: 14 # days of runtime data before dead_column trusts it
417
+ runtime_log: .schema_reaper/runtime.jsonl
418
+ history_log: .schema_reaper/history.jsonl
419
+ baseline: .schema_reaper/baseline.json
420
+ require: [] # extra files to load, e.g. custom analyzers
421
+ ```
422
+
423
+ </details>
424
+
425
+ ### Custom analyzers
136
426
 
137
427
  ```ruby
138
428
  # lib/schema_reaper/analyzers/my_check.rb
@@ -153,6 +443,10 @@ require:
153
443
 
154
444
  ## Roadmap
155
445
 
446
+ > [!TIP]
447
+ > ✅ Self-hosted scheduled scans + alerts shipped in **v2.0** — see
448
+ > [Production automation](#production-automation).
449
+
156
450
  - Runtime verdict fusion for index and table findings
157
451
  - Orphan-row and `schema.rb`↔DB drift analyzers
158
452
  - Disk/$ reclaim from real `pg_total_relation_size`
@@ -161,12 +455,12 @@ require:
161
455
 
162
456
  ## Pro (for teams)
163
457
 
164
- The gem is free and complete for a single app. **schema_reaper Pro** adds the
165
- team-scale layer: MySQL adapter, multi-database fan-out, orphan-row and
166
- schema-drift analyzers, Slack/Jira/PR-comment reporters, real
167
- `pg_total_relation_size` + $ estimates, scheduled scans with alerts, and a
168
- mountable dashboard engine. See [PRO.md](PRO.md). Waitlist / early access:
169
- open an issue tagged `pro`.
458
+ The gem is free and complete for a single app — including the scheduled scans
459
+ and alerts above. **schema_reaper Pro** adds the team-scale layer: MySQL
460
+ adapter, multi-database fan-out, orphan-row and schema-drift analyzers, native
461
+ Slack/Jira/PR-comment reporters, real `pg_total_relation_size` + $ estimates, a
462
+ hosted zero-install scan runner, and a mountable dashboard engine. See
463
+ [PRO.md](PRO.md). Waitlist / early access: open an issue tagged `pro`.
170
464
 
171
465
  ## Sponsor
172
466
 
@@ -176,11 +470,14 @@ repo.
176
470
 
177
471
  ## Development
178
472
 
179
- ```
473
+ ```sh
180
474
  bin/setup
181
475
  bundle exec rake # rspec + rubocop
476
+
477
+ # opt-in: live-database introspection specs
478
+ SCHEMA_REAPER_TEST_DATABASE_URL=postgres://localhost/schema_reaper_test bundle exec rspec
182
479
  ```
183
480
 
184
481
  ## License
185
482
 
186
- MIT.
483
+ [MIT](LICENSE.txt).