schema_reaper 2.0.0 → 2.1.0

data/README.md

@@ -1,7 +1,18 @@
-# schema_reaper
+<div align="center">
 
-Find dead columns, dead tables, unused indexes and other schema dead-weight in
-Rails / ActiveRecord apps — then remove them safely.
+<img src="docs/assets/hero.svg" width="100%" alt="schema_reaper — find and safely remove the dead columns, tables and indexes your Rails + PostgreSQL app no longer uses" />
+
+[![Gem Version](https://img.shields.io/gem/v/schema_reaper?color=cc342d&logo=rubygems&logoColor=white&label=gem)](https://rubygems.org/gems/schema_reaper)
+[![Gem Downloads](https://img.shields.io/gem/dt/schema_reaper?color=cc342d&logo=rubygems&logoColor=white&label=downloads)](https://rubygems.org/gems/schema_reaper)
+[![CI](https://github.com/aksshatt/schema_reaper/actions/workflows/main.yml/badge.svg)](https://github.com/aksshatt/schema_reaper/actions/workflows/main.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-yellow.svg)](LICENSE.txt)
+[![Ruby](https://img.shields.io/badge/ruby-%3E%3D%202.7-CC342D?logo=ruby&logoColor=white)](schema_reaper.gemspec)
+[![PostgreSQL](https://img.shields.io/badge/postgres-only%20(for%20now)-336791?logo=postgresql&logoColor=white)](#install)
+
+| ⚡ [Quickstart](#quickstart) | ⚙️ [Usage](#usage) | 🤖 [Production automation](#production-automation) | 🔍 [Analyzers](#analyzers) | 🛡️ [Safety model](#safety-model) | ✅ [CI](#ci) | 🧩 [Configuration](#configuration) | 💼 [Pro](#pro-for-teams) |
+|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
+
+</div>
 
 `schema_reaper` reads your **live PostgreSQL schema and planner statistics** and
 cross-references them against a **static scan of your codebase** (Ruby via the
@@ -10,129 +21,461 @@ Prism AST, plus views and SQL string literals). Optionally it also fuses in a
 production. Every finding is scored by confidence and severity, carries an
 estimate of the disk it reclaims, and comes with a concrete fix.
 
-## Install
+It finds three kinds of schema debt:
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/debt-dark.svg">
+  <img src="docs/assets/debt-light.svg" width="100%" alt="Dead weight — columns and tables nothing references any more (dead_column, dead_table). Index trouble — indexes nobody queries, indexes a wider one already covers, and foreign keys with no index (unused_index, duplicate_index, missing_fk_index). Degenerate data — columns that are always NULL or hold one value in every row (always_null_column, single_value_column).">
+</picture>
+
+## Quickstart
 
 ```ruby
 # Gemfile
-gem "schema_reaper", group: :development
+gem "schema_reaper"
 ```
 
-```
+```sh
 bundle install
+bundle exec schema_reaper scan
+```
+
+In a Rails app that's it — the connection comes from `config/database.yml`.
+You get a report like this:
+
+<p align="center">
+  <img src="docs/assets/terminal.svg" width="100%" alt="Terminal output of bundle exec schema_reaper scan: 3 findings across 2 tables — users.team_id has no covering index (90%, medium, fix: add_index :users, :team_id); users.api_key is NULL in every row (85%, high, 46.9 KB reclaimable); table stale_exports has no model or query reference and holds 0 rows (85%, high, fix: drop_table :stale_exports after confirming no external consumer)." />
+</p>
+
+> [!TIP]
+> Scan a database that has served real traffic — a production read-replica or a
+> recent snapshot. `unused_index` needs query history (it skips itself on a
+> fresh database and tells you why), and the data analyzers read `pg_stats`,
+> which PostgreSQL only fills in after `ANALYZE`.
+
+## How it works
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/pipeline-dark.svg">
+  <img src="docs/assets/pipeline-light.svg" width="100%" alt="Three inputs — the live PostgreSQL schema and statistics, a static scan of your codebase, and an optional runtime signal — feed seven analyzers. Their scored findings become reports in four formats, a CI baseline gate, scheduled webhook and email reports, and staged removal migrations.">
+</picture>
+
+Findings that say the same thing about different tables are rolled up into one
+entry; when several analyzers flag the same column, the strongest finding wins
+and notes the others that agreed. `schema_reaper` never changes your database —
+it only reads, and hands you migrations to review.
+
+## Install
+
+```ruby
+# Gemfile
+gem "schema_reaper"
 ```
 
-Requires Ruby >= 2.7 and PostgreSQL. The database connection is resolved in
-this order:
+| requirement | notes |
+|---|---|
+| Ruby >= 2.7 | CI-tested on 2.7 – 3.3 |
+| PostgreSQL + the `pg` gem | uses your app's own `pg` (every Rails + PostgreSQL app already bundles it); add `gem "pg"` for standalone CLI use. Partitioned tables are reported once, as the parent. Tables in the `public` schema are scanned by default; list others under `schemas:` in the [config](#configuration). MySQL is on the [roadmap](#roadmap). |
+| Rails | optional for the CLI; the rake tasks and production automation need it, and the runtime tracker needs ActiveRecord |
+
+> [!IMPORTANT]
+> **Only scanning locally or in CI?** Scope it out of your production bundle:
+> `gem "schema_reaper", group: :development`.
+>
+> **Planning to use [production automation](#production-automation)?** Leave it
+> unscoped. Most deploys exclude the development and test groups
+> (`BUNDLE_WITHOUT=development:test`), so a dev-scoped gem is simply absent when
+> the scheduled scan tries to run. The install generator warns you about this.
+
+<details>
+<summary><b>How the database connection is resolved</b></summary>
+
+<br>
 
 1. `database_url:` in `.schema_reaper.yml`
 2. `ENV["DATABASE_URL"]`
-3. `config/database.yml` for the current environment (`SCHEMA_REAPER_ENV` /
-   `RAILS_ENV`, default `development`) — ERB and YAML aliases are handled, as
-   are Rails 6+ multi-database sections
+3. `config/database.yml` for the current environment (`SCHEMA_REAPER_ENV`, then
+   `RAILS_ENV`, then `RACK_ENV`, default `development`) — ERB and YAML aliases
+   are handled, as are Rails 6+ multi-database sections (the `primary` entry is
+   used)
 
-So in a Rails app, `bundle exec schema_reaper scan` works with no setup.
+</details>
 
 ## Usage
 
-```
+```sh
 bundle exec schema_reaper scan                    # grouped terminal report
 bundle exec schema_reaper scan --format markdown  # PR-comment table
 bundle exec schema_reaper scan --format sarif     # GitHub code scanning
 bundle exec schema_reaper scan --format json
-bundle exec schema_reaper scan --ci               # exit 1 on new findings
+bundle exec schema_reaper scan --ci               # exit 1 on findings not in the baseline
 bundle exec schema_reaper scan --min-confidence 0.8
-bundle exec schema_reaper scan --no-color         # force plain output
+bundle exec schema_reaper scan --only dead_column unused_index   # or --except ...
 bundle exec schema_reaper baseline                # accept current findings
+bundle exec schema_reaper baseline --reason "billing ETL" --expires 2027-03-31
+bundle exec schema_reaper baseline --prune        # drop entries that no longer occur
+bundle exec schema_reaper explain users/legacy_api_token   # one finding in full + next step
+bundle exec schema_reaper doctor                  # is the setup good enough to trust a scan?
 bundle exec schema_reaper trend                   # snapshot + progress delta
-bundle exec schema_reaper generate-migration users legacy_api_token
+bundle exec schema_reaper generate-migration users legacy_api_token   # column: staged pair
+bundle exec schema_reaper generate-migration users --index index_users_on_legacy_token
+bundle exec schema_reaper generate-migration old_audit_logs           # whole table
+```
+
+<details>
+<summary><b>All <code>scan</code> options</b></summary>
+
+<br>
+
+| option | default | what it does |
+|---|---|---|
+| `--format` | `table` | `table`, `json`, `markdown` or `sarif` |
+| `--ci` | off | exit 1 when a finding isn't in `.schema_reaper/baseline.json` |
+| `--min-confidence N` | `min_confidence` in config, else `0.0` | hide findings below this confidence (0.0 – 1.0) |
+| `--only A B` / `--except A B` | all | run only, or skip, these analyzers (also on `baseline` and `trend`) |
+| `--record` | off | also append this run to the history log used by `trend` |
+| `--color` / `--no-color` | auto | force colour on or off (works on every command) |
+| `--config PATH` | `.schema_reaper.yml` | config file to load (works on every command) |
+
+Colour is automatic on a terminal, and off when output is piped or `NO_COLOR` is set.
+
+</details>
+
+<details>
+<summary><b>Rake tasks (Rails)</b></summary>
+
+<br>
+
+The railtie adds these to any Rails app with the gem in its bundle:
+
+| task | what it does |
+|---|---|
+| `rake schema_reaper:scan` | same as `schema_reaper scan`; pick a format with `FORMAT=json` etc. |
+| `rake schema_reaper:baseline` | write the current findings to the baseline file |
+| `rake schema_reaper:trend` | record a snapshot and print the trend |
+| `rake schema_reaper:alert` | run one production scan and send the report — what the [schedule](#production-automation) calls |
+
+</details>
+
+## Production automation
+
+*New in v2.0.* Run `schema_reaper` unattended in production and have the report
+land in your team's chat and inbox — no DevOps ticket, and no server access
+beyond a normal deploy.
+
+```sh
+bin/rails generate schema_reaper:install
+```
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/automation-dark.svg">
+  <img src="docs/assets/automation-light.svg" width="100%" alt="whenever or cron runs rake schema_reaper:alert, sidekiq-cron enqueues the job directly, and the token-protected manual trigger POST /internal/schema_scan enqueues it on demand. All three run SchemaReaper::ScanJob, which scans the live database and your code; the Notifier sends the report to a Slack-compatible webhook and by email through your ApplicationMailer.">
+</picture>
+
+| generated | purpose |
+|---|---|
+| `config/initializers/schema_reaper.rb` | where reports go (webhook and/or email) |
+| an entry in `config/schedule.rb` **or** `config/schedule.yml` | the quarterly scheduled scan (whenever or sidekiq-cron) |
+| `app/controllers/schema_reaper_controller.rb` + a route | a token-protected "scan now" endpoint |
+| a trigger token, printed once | you paste it into your encrypted credentials |
+
+Re-running the generator is safe: every step checks for its own marker and
+skips itself rather than duplicating anything. Commit the files, deploy, and
+check it works end to end with one manual run in production:
+
+```sh
+bin/rails schema_reaper:alert
+```
+
+### 1. Where reports go
+
+The generated initializer is commented out, so an unfilled config is a safe
+no-op, not an error. Uncomment one or both:
+
+```ruby
+SchemaReaper::AlertConfig.configure do |config|
+  # config.emails = %w[dev1@example.com dev2@example.com]
+  # config.webhook_url = "https://hooks.slack.com/services/T000/B000/XXXX"
+end
 ```
 
-The `scan` report rolls up findings that say the same thing about different
-tables, then groups the rest by table and sorts by confidence:
+Both channels fire when both are set — not a fallback chain, but two audiences:
+a dev-facing chat channel, and an inbox record for people who don't watch chat.
+
+Every report says what changed since the previous one (remembered in
+`Rails.cache`). Set `config.alert_on = :changes` to send only when findings
+appeared or were resolved; the default, `:always`, sends every run.
+
+- **Webhook** — for a `hooks.slack.com` URL, a Slack Block Kit summary (counts,
+  the top or newly added findings, what was resolved); for any other URL, a
+  `{"text": "..."}` JSON POST carrying the full Markdown report. Force either
+  with `config.webhook_format = :slack` or `:text`. 10-second timeout.
+- **Email** — sent by `SchemaReaper::Mailer`, which inherits your app's
+  `ApplicationMailer` (its `default from:`, delivery settings and so on), or
+  `ActionMailer::Base` if you don't have one. Delivered with `deliver_later`.
+
+Delivery failures are logged with a `[schema_reaper]` prefix and never raised,
+so a flaky endpoint can't fail or retry-storm the scan.
+
+### 2. The schedule
+
+The generator looks at your `Gemfile.lock` and uses whichever scheduler you
+already run. The default cadence is **quarterly**; edit the generated entry to
+change it.
+
+| you use | the generator adds | also needed |
+|---|---|---|
+| [whenever](https://github.com/javan/whenever) | an `every 3.months` block running `rake "schema_reaper:alert"` in `config/schedule.rb` | whenever only writes your crontab when `whenever --update-crontab` runs on the server — usually via its Capistrano recipe. Platforms without a crontab (Heroku, most container hosts) need one of the other rows. |
+| [sidekiq-cron](https://github.com/sidekiq-cron/sidekiq-cron) | a `schema_reaper_scan` entry in `config/schedule.yml` (`0 4 1 */3 *` — 04:00 on the 1st, every 3 months) that enqueues `SchemaReaper::ScanJob` | nothing — sidekiq-cron ≥ 1.6 loads `config/schedule.yml` automatically |
+| neither | nothing — it prints the command instead of guessing | point any scheduler (cron, Heroku Scheduler, a Kubernetes CronJob) at `bin/rails schema_reaper:alert` |
+
+`rake schema_reaper:alert` enqueues the scan on your ActiveJob backend. If that
+backend is Rails' in-process `:async` adapter (the default before Rails 8 unless
+you've set one up), the job would die with the short-lived rake process — so
+the task runs the scan inline instead, email included.
+
+### 3. The manual trigger
+
+For "scan now" without shelling into a box:
 
+```sh
+curl -X POST https://your-app.example.com/internal/schema_scan \
+  -H "Authorization: Bearer <token>"
 ```
-  schema_reaper  5 findings across 2 tables
-  missing_fk_index 3 · always_null_column 1 · dead_column 1
-  ~93.8 KB reclaimable
 
-  users
-    █████  90%  medium  missing_fk_index     team_id
-           team_id is a foreign key with no covering index
-           → add_index :users, :team_id
-    ████░  85%  high    always_null_column   api_key  46.9 KB
-           pg_stats.null_frac = 1.0 across ~3000 row(s) · column carries no data
-           → verify with `SELECT count(api_key) FROM users` then stage a removal
+| response | meaning |
+|---|---|
+| `202 Accepted` | scan enqueued — the report arrives through the channels above |
+| `401 Unauthorized` | missing or wrong token, or no token configured in this environment |
+| `429 Too Many Requests` | a scan was already triggered in the last 5 minutes |
 
-  stale_exports
-    ████░  85%  high    dead_table
-           no model or query reference · table holds ~0 row(s)
-           → confirm no external consumer, then `drop_table :stale_exports`
+The generator prints the token once, with the exact snippet to paste into
+`bin/rails credentials:edit`:
 
-  high 2   medium 1   low 2
+```yaml
+schema_reaper:
+  trigger_token: <token>
 ```
 
-Colour is automatic on a terminal, off when piped or `NO_COLOR` is set.
+Credentials are decrypted with the `RAILS_MASTER_KEY` your app already has, so
+this adds no new production configuration. The token is checked with a
+constant-time comparison. The controller deliberately doesn't inherit your
+`ApplicationController` and skips CSRF (it's a token-authenticated API endpoint,
+not a form), so your own auth filters don't apply to it.
+
+> [!NOTE]
+> The 5-minute cooldown lives in `Rails.cache`, so it's only as shared as your
+> cache store. Redis, Memcached or Solid Cache give one cooldown for the whole
+> app; a file or memory store gives one per host or process; `:null_store`
+> turns it off.
+
+### 4. Install-time warnings
+
+Two misconfigurations would otherwise fail silently much later, so the
+generator flags them when it runs:
+
+- the gem is scoped to `group: :development` — the scheduled scan would never
+  run in production
+- neither `ApplicationMailer` nor `config.action_mailer.default_options` sets a
+  `from:` address — the mailer would fall back to a placeholder sender that most
+  SMTP relays reject, so email reports would never arrive
+
+> [!WARNING]
+> **Still on 2.0.0?** Two production-automation bugs are fixed on `main` and
+> ship in the next release:
+> - Report emails ignore `ApplicationMailer` and are sent from
+>   `schema_reaper@localhost`. Work around it by adding
+>   `SchemaReaper::Mailer.default from: "you@your-domain.com"` to
+>   `config/initializers/schema_reaper.rb`.
+> - With the `:async` ActiveJob adapter, `rake schema_reaper:alert` enqueues a
+>   job that never runs. Use a persistent backend (Sidekiq, GoodJob, Solid
+>   Queue, …) for the whenever and plain-cron paths.
 
-In a Rails app the railtie also gives you
-`rake schema_reaper:scan|baseline|trend` (with `FORMAT=`).
+<details>
+<summary><b>Why not just run it in CI?</b></summary>
+
+<br>
+
+[`--format sarif` in CI](#ci) is for PR- and staging-level checks. GitHub-hosted
+runners have no network path to your production database by default, and a CI
+database has no production traffic statistics. Production automation is the path
+for watching the real, live database.
+
+</details>
 
 ## Analyzers
 
 | type | what it flags | main signal |
 |---|---|---|
 | `dead_column` | column no code path references | static scan (+ runtime) |
-| `dead_table` | table with no model/query reference | static scan + row count |
-| `unused_index` | non-unique index, `idx_scan = 0` | `pg_stat_user_indexes` |
-| `duplicate_index` | index that is a prefix of a wider one | schema shape |
-| `missing_fk_index` | `*_id` / FK column with no index | schema shape |
+| `dead_table` | table with no model or query reference | static scan + row count |
+| `unused_index` | non-unique index with `idx_scan = 0` (needs query history; stats are per server, so check replicas too) | `pg_stat_user_indexes` |
+| `duplicate_index` | index that exactly duplicates another, or is a prefix of a wider one | schema shape |
+| `missing_fk_index` | `*_id` / foreign-key column with no index (polymorphic pairs need a `(type, id)` index) | schema shape |
 | `always_null_column` | `null_frac = 1.0` — no data at all | `pg_stats` |
-| `single_value_column` | one distinct value on a large table | `pg_stats` |
-
-Columns owned by common gems (devise, paper_trail, activestorage, actiontext,
-friendly_id, audited, pg_search, ahoy_matey, paranoia family) are whitelisted
-automatically when the gem is in your bundle.
+| `single_value_column` | one distinct value on a table of 500+ rows | `pg_stats` |
+| `sequence_exhaustion` | serial / IDENTITY sequence past 50% of its column type's range (high at 75%) | `pg_sequences` |
+| `duplicate_foreign_key` | two FK constraints on the same columns to the same table | `pg_constraint` |
+
+Columns and tables owned by common gems are whitelisted automatically when the
+gem is in your bundle: **devise**, **devise-api**, **paper_trail**,
+**audited**, **friendly_id**, **paranoia** / **acts_as_paranoid**,
+**activestorage**, **actiontext**, **pg_search**, **ahoy_matey**,
+**activeadmin**, **solid_queue**, **solid_cache**, **solid_cable**,
+**good_job**, **delayed_job_active_record**, **flipper-active_record**,
+**acts-as-taggable-on**, **doorkeeper**, **noticed**,
+**activerecord-session_store**, **discard**, **acts_as_list** and
+**ancestry**. A table a gem owns outright is left alone by every analyzer.
 
 ## Runtime signal (optional, raises confidence)
 
 Static analysis alone can't see metaprogrammed access, so `dead_column`
-confidence is capped at **0.6** without runtime data. To lift the cap:
+confidence is capped at **0.6** without runtime data. To lift the cap, sample
+real column reads:
 
 ```ruby
-# config/initializers or manually
+# config/initializers/schema_reaper_tracker.rb
 SchemaReaper::Runtime::Tracker.install!(
   store: SchemaReaper::Runtime::Store.new(path: ".schema_reaper/runtime.jsonl"),
   sample_rate: 0.05
 )
 ```
 
-or, in Rails, boot with `SCHEMA_REAPER_TRACK=1`. Let it run in staging or
+or, in Rails, boot with `SCHEMA_REAPER_TRACK=1` (and optionally
+`SCHEMA_REAPER_SAMPLE=0.05` for the sample rate). The tracker also samples
+the tables named in executed SQL (pass `track_sql: false` to opt out), so a
+table read only through raw SQL isn't reported as dead. The log rotates to
+`runtime.jsonl.1` past `runtime_log_max_bytes` (50 MB), and both generations
+are read. Let it run in staging or
 production for a couple of weeks. A column unseen in **both** code and
->= 14 observed days of runtime data reaches ~0.9 confidence.
+≥ 14 observed days of runtime data (`min_age_days` in
+[`.schema_reaper.yml`](#configuration)) reaches **0.9** confidence (0.8 if it's
+`NOT NULL`).
+
+> [!NOTE]
+> This is separate from [production automation](#production-automation): the
+> tracker raises confidence in the findings a scan already makes, while
+> production automation runs the scan on a schedule and delivers the report.
+> Most teams want both.
 
 ## Safety model
 
-`schema_reaper` never drops anything itself. `generate-migration` emits a pair:
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/safety-dark.svg">
+  <img src="docs/assets/safety-light.svg" width="100%" alt="generate-migration writes two migrations. Step 1 adds the column to ignored_columns and is deployed; nothing is dropped. Once it has soaked in production and nothing reads the column, step 2 runs remove_column, which is irreversible.">
+</picture>
+
+`schema_reaper` never drops anything itself. For a dead column,
+`generate-migration users legacy_api_token` writes a pair:
 
-1. **Ignore** — you add `self.ignored_columns += %w[col]` to the model and
-   deploy. Nothing is dropped.
-2. **Drop** — run only after step 1 has soaked in production and nothing broke.
+1. **`…_ignore_users_legacy_api_token.rb`** — a no-op migration that reminds
+   you to add `self.ignored_columns += %w[legacy_api_token]` to the model. Deploy
+   that. Nothing is dropped; ActiveRecord just stops selecting the column.
+2. **`…_drop_users_legacy_api_token.rb`** — the `remove_column`. Run it only
+   after step 1 has soaked in production and nothing broke. Its `down` raises
+   `ActiveRecord::IrreversibleMigration` on purpose.
 
-`always_null_column` / `single_value_column` fixes ask you to confirm with a
-`SELECT` first.
+The data-driven fixes are cautious too: `always_null_column` asks you to
+confirm with a `SELECT count(...)` first, `single_value_column` to check the
+value isn't a meaningful default, and `dead_table` to rule out external
+consumers.
 
 ## CI
 
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/ci-dark.svg">
+  <img src="docs/assets/ci-light.svg" width="100%" alt="On each pull request, schema_reaper scan --ci compares the findings with the committed baseline.json. It passes when every finding is already in the baseline and exits 1 when the change adds new dead weight.">
+</picture>
+
+```yaml
+# .github/workflows/schema_reaper.yml (the relevant parts)
+permissions:
+  contents: read
+  security-events: write          # required by upload-sarif
+
+# ...job setup: Ruby, a PostgreSQL service, DATABASE_URL...
+    steps:
+      - run: bin/rails db:schema:load
+      - run: bundle exec schema_reaper scan --ci --format sarif > reaper.sarif
+      - uses: github/codeql-action/upload-sarif@v4
+        if: always()              # upload even when --ci fails the step above
+        with: { sarif_file: reaper.sarif }
+```
+
+Commit `.schema_reaper/baseline.json` (from `schema_reaper baseline`) so the job
+fails only when a change adds *new* dead weight.
+
+> [!NOTE]
+> A freshly loaded CI database has no rows, statistics or query history, so CI
+> catches the schema- and code-shaped findings (`dead_column`, `dead_table`,
+> `duplicate_index`, `missing_fk_index`). The data-driven ones need a real
+> database — that's what [production automation](#production-automation) is for.
+
+## Configuration
+
+Every key is optional. Drop a `.schema_reaper.yml` in the project root to
+override any of these defaults.
+
+> [!CAUTION]
+> A list you set **replaces** the default list rather than adding to it. Setting
+> `ignore: { tables: [legacy_audit] }` stops ignoring `schema_migrations` and
+> `ar_internal_metadata`, and `always_keep_columns: [uuid]` stops protecting
+> `created_at`, `updated_at` and `type`. Repeat the defaults you still want.
+
+<details>
+<summary><b>.schema_reaper.yml — all keys with their defaults</b></summary>
+
+<br>
+
 ```yaml
-# .github/workflows/schema_reaper.yml
-- run: bundle exec schema_reaper scan --ci --format sarif > reaper.sarif
-- uses: github/codeql-action/upload-sarif@v3
-  with: { sarif_file: reaper.sarif }
+database_url:          # falls back to DATABASE_URL, then config/database.yml
+database_yml: config/database.yml
+schemas: [public]      # PostgreSQL schemas to scan; non-public tables appear as schema.table
+scan_paths: [app, lib, config]
+view_globs: ["app/**/*.erb", "app/**/*.haml", "app/**/*.slim", "app/**/*.jbuilder"]
+ignore:
+  tables: [schema_migrations, ar_internal_metadata]
+  columns: []          # exact names, or "/regex/" patterns
+always_keep_columns: [id, created_at, updated_at, type]
+gem_awareness: true    # auto-whitelist columns owned by known gems
+min_age_days: 14       # days of runtime data before dead_column trusts it
+min_confidence: 0.0    # hide findings below this on every command
+runtime_log_max_bytes: 52428800  # rotate the runtime log past this size (null: never)
+runtime_log: .schema_reaper/runtime.jsonl
+history_log: .schema_reaper/history.jsonl
+baseline: .schema_reaper/baseline.json
+require: []            # extra files to load, e.g. custom analyzers
+scan_cache: .schema_reaper/scan_cache.json  # per-file scan cache (null: off)
+column_scope: global   # or model -- see "Stricter column matching"
 ```
 
-Commit `.schema_reaper/baseline.json` so the job fails only when a change adds
-*new* dead weight.
+</details>
+
+### Keeping a finding from code
 
-## Custom analyzers
+A `schema_reaper:keep` comment anywhere in scanned Ruby or view files silences
+matching findings, with the reason kept next to the code it's about:
+
+```ruby
+class User < ApplicationRecord
+  # schema_reaper:keep legacy_api_token, users.import_ref -- read by the billing ETL
+end
+```
+
+A target is a column (on any table), `table.column`, a table (every finding on
+it), or an index name.
+
+### Stricter column matching
+
+By default a column counts as used if its name appears anywhere in scanned
+code, so `status` in `Order` also hides a dead `status` on `users`. With
+`column_scope: model`, a name seen only in an `app/models` file counts for that
+model's table and for the tables the model mentions (associations, class
+names); names in controllers, views, jobs and so on still count everywhere.
+It finds more dead columns, at some risk of flagging one used through
+metaprogramming -- review before acting.
+
+### Custom analyzers
 
 ```ruby
 # lib/schema_reaper/analyzers/my_check.rb
@@ -153,7 +496,11 @@ require:
 
 ## Roadmap
 
-- Runtime verdict fusion for index and table findings
+> [!TIP]
+> ✅ Self-hosted scheduled scans + alerts shipped in **v2.0** — see
+> [Production automation](#production-automation).
+
+- Runtime verdict fusion for index findings
 - Orphan-row and `schema.rb`↔DB drift analyzers
 - Disk/$ reclaim from real `pg_total_relation_size`
 - Mountable dashboard engine, trend charts
@@ -161,12 +508,12 @@ require:
 
 ## Pro (for teams)
 
-The gem is free and complete for a single app. **schema_reaper Pro** adds the
-team-scale layer: MySQL adapter, multi-database fan-out, orphan-row and
-schema-drift analyzers, Slack/Jira/PR-comment reporters, real
-`pg_total_relation_size` + $ estimates, scheduled scans with alerts, and a
-mountable dashboard engine. See [PRO.md](PRO.md). Waitlist / early access:
-open an issue tagged `pro`.
+The gem is free and complete for a single app — including the scheduled scans
+and alerts above. **schema_reaper Pro** adds the team-scale layer: MySQL
+adapter, multi-database fan-out, orphan-row and schema-drift analyzers, native
+Slack/Jira/PR-comment reporters, real `pg_total_relation_size` + $ estimates, a
+hosted zero-install scan runner, and a mountable dashboard engine. See
+[PRO.md](PRO.md). Waitlist / early access: open an issue tagged `pro`.
 
 ## Sponsor
 
@@ -176,11 +523,14 @@ repo.
 
 ## Development
 
-```
+```sh
 bin/setup
 bundle exec rake        # rspec + rubocop
+
+# opt-in: live-database introspection specs
+SCHEMA_REAPER_TEST_DATABASE_URL=postgres://localhost/schema_reaper_test bundle exec rspec
 ```
 
 ## License
 
-MIT.
+[MIT](LICENSE.txt).

data/RELEASE_CHECKLIST.md

@@ -152,8 +152,19 @@ a throwaway PostgreSQL with a deliberately dirty schema.
 
 ## 6. Publish the gem
 
-RubyGems requires account MFA; `gem push` prompts for an OTP that only the
-maintainer can enter.
+Pushing the `vX.Y.Z` tag (§5) runs `.github/workflows/release.yml`: it checks
+the tag matches `VERSION`, is on `main`, has a CHANGELOG section and no AI
+trailers in history; runs the full suite against PostgreSQL; builds the gem
+and diffs the packaged `lib/` against `git ls-files`; then (after approval, if
+the `release` environment requires reviewers) pushes to RubyGems via Trusted
+Publishing and creates the GitHub release from the CHANGELOG section.
+
+One-time setup: rubygems.org -> schema_reaper -> Trusted publishers -> add
+repository `aksshatt/schema_reaper`, workflow `release.yml`, environment
+`release`.
+
+Manual fallback -- RubyGems requires account MFA; `gem push` prompts for an
+OTP that only the maintainer can enter.
 
 ```sh
 gem build schema_reaper.gemspec