schema_reaper 1.0.8 → 1.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +43 -0
- data/README.md +35 -3
- data/RELEASE_CHECKLIST.md +218 -0
- data/lib/schema_reaper/analyzers/dead_table.rb +5 -1
- data/lib/schema_reaper/cli.rb +39 -9
- data/lib/schema_reaper/config.rb +12 -3
- data/lib/schema_reaper/database_url.rb +78 -0
- data/lib/schema_reaper/finding.rb +8 -0
- data/lib/schema_reaper/history.rb +2 -0
- data/lib/schema_reaper/introspect/postgres.rb +16 -3
- data/lib/schema_reaper/reporters/ansi.rb +53 -0
- data/lib/schema_reaper/reporters/console.rb +65 -0
- data/lib/schema_reaper/reporters/table.rb +76 -18
- data/lib/schema_reaper/reporters/trend.rb +70 -0
- data/lib/schema_reaper/runner.rb +5 -3
- data/lib/schema_reaper/version.rb +1 -1
- data/lib/schema_reaper.rb +3 -0
- metadata +7 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e86ae3d2cb33da4bdf545809519f27fb5104e73f4996bb650e2fd3d3f9232067
|
|
4
|
+
data.tar.gz: fa0faef4c064a923bd2d699d6fbc3b58d364f7d94a3a0cf67efc93e86ec889b1
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4a49eb0d8ff830df84543e945ae809af391f86fc3259cf69d3189a5c60eff90478d59b2bf914af7eed57104f924ed1034c26b94eabd662e8d5a73192e44b38b1
|
|
7
|
+
data.tar.gz: f83a196f89bda5b52ce31598c80e4300e6816b93519e810b60febaa588647a4d3432b049567baff48482d2ab362226581a02a918284fc823a0bcccebb828604c
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,48 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [1.0.9] - 2026-09-09
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
- Automatic database connection in a Rails app. If `database_url:` is not set in
|
|
7
|
+
`.schema_reaper.yml` and `DATABASE_URL` is not in the environment,
|
|
8
|
+
schema_reaper now reads `config/database.yml` — rendering ERB, resolving YAML
|
|
9
|
+
aliases, honouring `SCHEMA_REAPER_ENV` / `RAILS_ENV` (default `development`),
|
|
10
|
+
and handling Rails 6+ multi-database sections. `bundle exec schema_reaper
|
|
11
|
+
scan` works with no setup. PostgreSQL adapters only.
|
|
12
|
+
- Connection and missing-config errors now print a one-line `✗` message
|
|
13
|
+
instead of a Ruby backtrace.
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
- `pg_class.reltuples` is `-1` on PostgreSQL 14+ for a table that has never
|
|
17
|
+
been analysed. It was being treated as a real row count: `dead_table`
|
|
18
|
+
printed "table holds ~-1 row(s)", scored it at the 0.4 "non-empty" level,
|
|
19
|
+
and the reclaimable-bytes estimate went negative. A negative `reltuples` is
|
|
20
|
+
now mapped to "unknown" — the finding drops to 0.5 confidence, the evidence
|
|
21
|
+
says the count is unknown and suggests running `ANALYZE`, and the byte
|
|
22
|
+
estimate stays at 0. Found by running against a real production schema.
|
|
23
|
+
|
|
24
|
+
### Changed
|
|
25
|
+
- Redesigned the terminal report (`scan` / `scan --format table`):
|
|
26
|
+
- one summary line (finding count + total reclaimable),
|
|
27
|
+
- findings grouped under their table, sorted by confidence,
|
|
28
|
+
- a five-cell confidence bar with the percentage, padded severity and
|
|
29
|
+
analyzer name, the target column/index, and the reclaimable size,
|
|
30
|
+
- evidence on one dim `·`-joined line, the fix on a green `→` line,
|
|
31
|
+
- a footer with a severity tally and a per-analyzer breakdown.
|
|
32
|
+
- Colour is automatic on an interactive terminal and suppressed when output
|
|
33
|
+
is piped or `NO_COLOR` is set. New `scan --color` / `--no-color` to force it.
|
|
34
|
+
- JSON, SARIF and Markdown reporters are unchanged.
|
|
35
|
+
- `trend` now prints a readable progress block — finding count and reclaimable
|
|
36
|
+
total with signed deltas since the first and previous run, a list of findings
|
|
37
|
+
newly introduced and resolved, and the snapshot dates — instead of a raw
|
|
38
|
+
`pp` hash dump.
|
|
39
|
+
- `baseline` and `generate-migration` print a short titled block with a `✓`
|
|
40
|
+
line and next steps; `generate-migration` spells out the two-step deploy.
|
|
41
|
+
- `scan --ci` reports new findings as a `✗` block on stderr with the finding
|
|
42
|
+
ids indented under it.
|
|
43
|
+
- The "unused_index skipped / no query history" message is now a single
|
|
44
|
+
formatted `!` notice on stderr.
|
|
45
|
+
|
|
3
46
|
## [1.0.8] - 2026-09-04
|
|
4
47
|
|
|
5
48
|
### Fixed
|
data/README.md
CHANGED
|
@@ -21,23 +21,55 @@ gem "schema_reaper", group: :development
|
|
|
21
21
|
bundle install
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
-
Requires Ruby >=
|
|
25
|
-
|
|
24
|
+
Requires Ruby >= 2.7 and PostgreSQL. The database connection is resolved in
|
|
25
|
+
this order:
|
|
26
|
+
|
|
27
|
+
1. `database_url:` in `.schema_reaper.yml`
|
|
28
|
+
2. `ENV["DATABASE_URL"]`
|
|
29
|
+
3. `config/database.yml` for the current environment (`SCHEMA_REAPER_ENV` /
|
|
30
|
+
`RAILS_ENV`, default `development`) — ERB and YAML aliases are handled, as
|
|
31
|
+
are Rails 6+ multi-database sections
|
|
32
|
+
|
|
33
|
+
So in a Rails app, `bundle exec schema_reaper scan` works with no setup.
|
|
26
34
|
|
|
27
35
|
## Usage
|
|
28
36
|
|
|
29
37
|
```
|
|
30
|
-
bundle exec schema_reaper scan #
|
|
38
|
+
bundle exec schema_reaper scan # grouped terminal report
|
|
31
39
|
bundle exec schema_reaper scan --format markdown # PR-comment table
|
|
32
40
|
bundle exec schema_reaper scan --format sarif # GitHub code scanning
|
|
33
41
|
bundle exec schema_reaper scan --format json
|
|
34
42
|
bundle exec schema_reaper scan --ci # exit 1 on new findings
|
|
35
43
|
bundle exec schema_reaper scan --min-confidence 0.8
|
|
44
|
+
bundle exec schema_reaper scan --no-color # force plain output
|
|
36
45
|
bundle exec schema_reaper baseline # accept current findings
|
|
37
46
|
bundle exec schema_reaper trend # snapshot + progress delta
|
|
38
47
|
bundle exec schema_reaper generate-migration users legacy_api_token
|
|
39
48
|
```
|
|
40
49
|
|
|
50
|
+
The `scan` report groups findings by table and sorts by confidence:
|
|
51
|
+
|
|
52
|
+
```
|
|
53
|
+
schema_reaper 5 findings ~93.8 KB reclaimable
|
|
54
|
+
|
|
55
|
+
users
|
|
56
|
+
█████ 90% medium missing_fk_index team_id
|
|
57
|
+
team_id is a foreign key with no covering index
|
|
58
|
+
→ add_index :users, :team_id
|
|
59
|
+
████░ 85% high always_null_column api_key 46.9 KB
|
|
60
|
+
pg_stats.null_frac = 1.0 across ~3000 row(s) · column carries no data
|
|
61
|
+
→ verify with `SELECT count(api_key) FROM users` then stage a removal
|
|
62
|
+
|
|
63
|
+
stale_exports
|
|
64
|
+
████░ 85% high dead_table
|
|
65
|
+
no model or query reference · table holds ~0 row(s)
|
|
66
|
+
→ confirm no external consumer, then `drop_table :stale_exports`
|
|
67
|
+
|
|
68
|
+
high 2 medium 1 low 2
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Colour is automatic on a terminal, off when piped or `NO_COLOR` is set.
|
|
72
|
+
|
|
41
73
|
In a Rails app the railtie also gives you
|
|
42
74
|
`rake schema_reaper:scan|baseline|trend` (with `FORMAT=`).
|
|
43
75
|
|
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
# Release checklist
|
|
2
|
+
|
|
3
|
+
Run top to bottom for every release. Nothing here is optional — each item has
|
|
4
|
+
burned us at least once.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## 1. History & attribution hygiene
|
|
9
|
+
|
|
10
|
+
Do this **before** committing, and again before force-pushing anything.
|
|
11
|
+
|
|
12
|
+
- [ ] **No AI / wrong-author trailers.** Commit messages must not contain
|
|
13
|
+
`Co-Authored-By: Claude`, `Claude-Session:`, or any `Co-Authored-By` for
|
|
14
|
+
someone who did not write the change.
|
|
15
|
+
|
|
16
|
+
```sh
|
|
17
|
+
git log --all --format='%B' | grep -iE 'co-authored|claude-session|claude' && echo "DIRTY" || echo "clean"
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
- [ ] **Authors are only the real people.** Expect `aksshatt` and `mitkush`
|
|
21
|
+
only (GitHub's merge-commit `GitHub <noreply@github.com>` committer is
|
|
22
|
+
fine).
|
|
23
|
+
|
|
24
|
+
```sh
|
|
25
|
+
git log --all --format='%an <%ae>%n%cn <%ce>' | sort -u
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
- [ ] **Working tree identity is right** (so new commits are attributed
|
|
29
|
+
correctly):
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
git config user.name # aksshatt
|
|
33
|
+
git config user.email # akshatpegwar5@gmail.com
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
- [ ] If a bad trailer or author already landed: strip it, then force-push
|
|
37
|
+
`main` **and** move the affected tag.
|
|
38
|
+
|
|
39
|
+
```sh
|
|
40
|
+
FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch -f \
|
|
41
|
+
--msg-filter 'grep -v -iE "^(Co-Authored-By: Claude|Claude-Session:)"' \
|
|
42
|
+
--tag-name-filter cat -- <range>
|
|
43
|
+
git tag -f -a vX.Y.Z -m "schema_reaper X.Y.Z"
|
|
44
|
+
git push --force <remote> main
|
|
45
|
+
git push --force <remote> refs/tags/vX.Y.Z
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
- [ ] After any force-push, re-verify with a **fresh full clone** (not
|
|
49
|
+
`--depth 1` — shallow only checks HEAD):
|
|
50
|
+
|
|
51
|
+
```sh
|
|
52
|
+
git clone <url> /tmp/verify && cd /tmp/verify
|
|
53
|
+
git log --all --format='%an <%ae>' | sort -u
|
|
54
|
+
git log --all --format='%B' | grep -iE 'claude|rahul' || echo clean
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## 2. Code checks
|
|
60
|
+
|
|
61
|
+
- [ ] Full CI locally — must be **0 failures**, **no offenses**:
|
|
62
|
+
|
|
63
|
+
```sh
|
|
64
|
+
bundle exec rake # rspec + rubocop
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
- [ ] **Ruby 2.7 floor holds** (the gem supports `>= 2.7`; endless methods and
|
|
68
|
+
3.0+ syntax are a syntax error there):
|
|
69
|
+
|
|
70
|
+
```sh
|
|
71
|
+
for f in $(find lib spec exe -name '*.rb'); do ruby2.7 -c "$f" >/dev/null || echo "FAIL $f"; done
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
RuboCop's `TargetRubyVersion: 2.7` also catches this via `Lint/Syntax`.
|
|
75
|
+
|
|
76
|
+
- [ ] `gem build schema_reaper.gemspec` — **no warnings**, correct version.
|
|
77
|
+
|
|
78
|
+
- [ ] Packaged file list is complete (every `lib/**` file present):
|
|
79
|
+
|
|
80
|
+
```sh
|
|
81
|
+
tar -xf schema_reaper-X.Y.Z.gem -C /tmp/g && tar -tzf /tmp/g/data.tar.gz | grep '^lib/'
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## 3. Live database smoke test
|
|
87
|
+
|
|
88
|
+
Static-only unit specs do not exercise the real introspection SQL. Run against
|
|
89
|
+
a throwaway PostgreSQL with a deliberately dirty schema.
|
|
90
|
+
|
|
91
|
+
- [ ] Seed a DB with: a dead column, an always-NULL FK column, a single-value
|
|
92
|
+
column, a dead (0-row) table, a composite-PK table, a polymorphic
|
|
93
|
+
`(_type, _id)` pair, a duplicate/prefix index, an unindexed FK **that has
|
|
94
|
+
data**.
|
|
95
|
+
|
|
96
|
+
- [ ] Run every command and eyeball the output:
|
|
97
|
+
|
|
98
|
+
```sh
|
|
99
|
+
schema_reaper scan # grouped report, no duplicate rows per column/index
|
|
100
|
+
schema_reaper scan --format json | ruby -rjson -e 'JSON.parse(STDIN.read)' # valid
|
|
101
|
+
schema_reaper scan --format sarif | ruby -rjson -e 'p JSON.parse(STDIN.read)["version"]' # "2.1.0"
|
|
102
|
+
schema_reaper scan --format markdown
|
|
103
|
+
schema_reaper scan --no-color # plain output, no ANSI
|
|
104
|
+
schema_reaper baseline && schema_reaper scan --ci ; echo $? # 0 when nothing new
|
|
105
|
+
# add a dead column, then:
|
|
106
|
+
schema_reaper scan --ci ; echo $? # 1, names the new finding
|
|
107
|
+
schema_reaper generate-migration <table> <col> # two files, both `ruby -c` clean
|
|
108
|
+
schema_reaper trend
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
- [ ] Assertions that have regressed before:
|
|
112
|
+
- `missing_fk_index` does **not** suggest `add_index` on an always-NULL column.
|
|
113
|
+
- polymorphic `*_id` is **not** flagged when a `(*_type, *_id)` index exists.
|
|
114
|
+
- `unused_index` **skips** (with a stderr notice) when the DB has no query
|
|
115
|
+
history — it does not flag every index.
|
|
116
|
+
- a finding for one physical column/index appears **once** (highest
|
|
117
|
+
confidence wins; others listed as `also flagged by:`).
|
|
118
|
+
- `--format json` on stdout is clean JSON — the `unused_index` notice is on
|
|
119
|
+
**stderr**.
|
|
120
|
+
|
|
121
|
+
- [ ] Live introspection spec:
|
|
122
|
+
|
|
123
|
+
```sh
|
|
124
|
+
SCHEMA_REAPER_TEST_DATABASE_URL=postgres:///throwaway bundle exec rspec spec/introspect/postgres_spec.rb
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
- [ ] `dropdb` the throwaway database.
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## 4. Version & changelog
|
|
132
|
+
|
|
133
|
+
- [ ] Bump `lib/schema_reaper/version.rb`.
|
|
134
|
+
- [ ] `CHANGELOG.md` — new dated section, **Fixed / Changed / Added**, credit
|
|
135
|
+
PRs and their authors.
|
|
136
|
+
- [ ] README still accurate (Ruby floor, flags, sample output, analyzer list).
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## 5. Commit, tag, push
|
|
141
|
+
|
|
142
|
+
- [ ] Commit message: plain, no AI trailer (see §1).
|
|
143
|
+
- [ ] Annotated tag `vX.Y.Z`.
|
|
144
|
+
|
|
145
|
+
```sh
|
|
146
|
+
git tag -a vX.Y.Z -m "schema_reaper X.Y.Z"
|
|
147
|
+
git push <remote> main
|
|
148
|
+
git push <remote> refs/tags/vX.Y.Z
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## 6. Publish the gem
|
|
154
|
+
|
|
155
|
+
RubyGems requires account MFA; `gem push` prompts for an OTP that only the
|
|
156
|
+
maintainer can enter.
|
|
157
|
+
|
|
158
|
+
```sh
|
|
159
|
+
gem build schema_reaper.gemspec
|
|
160
|
+
gem push schema_reaper-X.Y.Z.gem --otp <6-digit code>
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
- [ ] Confirm it went live (API cache lags ~1 min):
|
|
164
|
+
|
|
165
|
+
```sh
|
|
166
|
+
curl -s https://rubygems.org/api/v1/gems/schema_reaper.json \
|
|
167
|
+
| ruby -rjson -e 'd=JSON.parse(STDIN.read); puts "#{d["version"]} #{d["authors"]}"'
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
- [ ] Published versions are **immutable**. A metadata/description typo needs a
|
|
171
|
+
new version — never assume you can edit a live one.
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## 7. Post-publish
|
|
176
|
+
|
|
177
|
+
- [ ] **Clean-room install** from RubyGems (not the local checkout):
|
|
178
|
+
|
|
179
|
+
```sh
|
|
180
|
+
gem install schema_reaper -v X.Y.Z --install-dir /tmp/cr --no-document
|
|
181
|
+
GEM_HOME=/tmp/cr /tmp/cr/bin/schema_reaper version
|
|
182
|
+
# then run §3 against a live DB using that binary
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
- [ ] **Contributors graph.** GitHub's sidebar/Insights widget caches hard and
|
|
186
|
+
can lag ~24h behind the real data. The source of truth is:
|
|
187
|
+
|
|
188
|
+
```sh
|
|
189
|
+
gh api repos/aksshatt/schema_reaper/contributors --jq '.[].login'
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
If the widget still shows a removed name after the API is clean: it is stale
|
|
193
|
+
cache of force-pushed-away commits, clears on GitHub's next `gc`. A brand-new
|
|
194
|
+
repo populated by `git push --mirror` of the clean history never shows the
|
|
195
|
+
ghost.
|
|
196
|
+
|
|
197
|
+
- [ ] Rotate any personal access token that was pasted into a chat/log.
|
|
198
|
+
|
|
199
|
+
---
|
|
200
|
+
|
|
201
|
+
## Repo-move runbook (only when starting a fresh repo to shed a cache)
|
|
202
|
+
|
|
203
|
+
```sh
|
|
204
|
+
git clone --bare <old-url> /tmp/bare # --bare, not --mirror: skips refs/pull/*
|
|
205
|
+
git -C /tmp/bare branch -D <stale/pr-branches>
|
|
206
|
+
gh api --method PATCH repos/aksshatt/schema_reaper -f name=schema_reaper_old
|
|
207
|
+
gh repo create aksshatt/schema_reaper --public --description "<desc>"
|
|
208
|
+
git -C /tmp/bare push --mirror https://<user>:<token>@github.com/aksshatt/schema_reaper.git
|
|
209
|
+
gh api --method PUT repos/aksshatt/schema_reaper/collaborators/mitkush -f permission=push
|
|
210
|
+
gh api --method PATCH repos/aksshatt/schema_reaper -f has_issues=true -f has_wiki=true -f has_projects=true
|
|
211
|
+
gh api --method PATCH repos/aksshatt/schema_reaper_old -F archived=true
|
|
212
|
+
git remote set-url origin https://github.com/aksshatt/schema_reaper.git
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
`push --mirror` copies commit objects **byte-for-byte** — authors, dates and
|
|
216
|
+
SHAs are unchanged. It carries branches, tags and deletions; it does **not**
|
|
217
|
+
carry issues, PRs, stars, releases text, settings, webhooks or the wiki. Set
|
|
218
|
+
those manually on the new repo.
|
|
@@ -76,7 +76,11 @@ module SchemaReaper
|
|
|
76
76
|
def evidence_for(table)
|
|
77
77
|
ev = ["no model or query reference to `#{table.name}` in scanned code"]
|
|
78
78
|
ev << "runtime data shows no access" if runtime.present?
|
|
79
|
-
ev <<
|
|
79
|
+
ev << if table.row_count.nil?
|
|
80
|
+
"row count unknown — run ANALYZE for a confidence boost"
|
|
81
|
+
else
|
|
82
|
+
"table holds ~#{table.row_count} row(s)"
|
|
83
|
+
end
|
|
80
84
|
ev
|
|
81
85
|
end
|
|
82
86
|
|
data/lib/schema_reaper/cli.rb
CHANGED
|
@@ -21,9 +21,11 @@ module SchemaReaper
|
|
|
21
21
|
option :record, type: :boolean, default: false,
|
|
22
22
|
desc: "append this run to the history log"
|
|
23
23
|
option :min_confidence, type: :numeric, default: 0.0
|
|
24
|
+
option :color, type: :boolean, default: nil,
|
|
25
|
+
desc: "force colour on/off for the table report (default: auto)"
|
|
24
26
|
def scan
|
|
25
27
|
findings = run.select { |f| f.confidence >= options[:min_confidence] }
|
|
26
|
-
|
|
28
|
+
render_report(findings)
|
|
27
29
|
|
|
28
30
|
History.new(config.history_log).record(findings) if options[:record]
|
|
29
31
|
enforce_baseline(findings) if options[:ci]
|
|
@@ -33,20 +35,31 @@ module SchemaReaper
|
|
|
33
35
|
def baseline
|
|
34
36
|
findings = run
|
|
35
37
|
Baseline.new(config.baseline_path).write(findings)
|
|
36
|
-
|
|
38
|
+
console.title("baseline")
|
|
39
|
+
console.ok("recorded #{findings.size} finding#{"s" unless findings.size == 1} " \
|
|
40
|
+
"as the accepted baseline")
|
|
41
|
+
console.info("file: #{config.baseline_path}")
|
|
42
|
+
console.info("`scan --ci` now fails only on findings that appear after this point.")
|
|
37
43
|
end
|
|
38
44
|
|
|
39
45
|
desc "trend", "Append a snapshot and print progress over time"
|
|
40
46
|
def trend
|
|
41
|
-
History.new(config.history_log)
|
|
42
|
-
|
|
43
|
-
|
|
47
|
+
history = History.new(config.history_log)
|
|
48
|
+
history.record(run)
|
|
49
|
+
Reporters::Trend.new(history.trend, color: options[:color]).render
|
|
44
50
|
end
|
|
45
51
|
|
|
46
52
|
desc "generate-migration TABLE COLUMN", "Emit a staged removal migration pair"
|
|
47
53
|
def generate_migration(table, column)
|
|
48
|
-
MigrationGenerator.new(table: table, column: column).call
|
|
49
|
-
|
|
54
|
+
paths = MigrationGenerator.new(table: table, column: column).call
|
|
55
|
+
console.title("generate-migration", "#{table}.#{column}")
|
|
56
|
+
console.ok("created two migrations:")
|
|
57
|
+
paths.each { |p| console.info(" #{p}") }
|
|
58
|
+
console.blank
|
|
59
|
+
console.list("next:", [
|
|
60
|
+
"deploy step 1 (adds `#{column}` to ignored_columns) and let it soak",
|
|
61
|
+
"run step 2 (`remove_column`) only once nothing has broken"
|
|
62
|
+
])
|
|
50
63
|
end
|
|
51
64
|
|
|
52
65
|
desc "version", "Print version"
|
|
@@ -62,14 +75,31 @@ module SchemaReaper
|
|
|
62
75
|
|
|
63
76
|
def run
|
|
64
77
|
Runner.new(config: config).run
|
|
78
|
+
rescue SchemaReaper::Error => e
|
|
79
|
+
console.problem(e.message)
|
|
80
|
+
exit 1
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
def render_report(findings)
|
|
84
|
+
if options[:format] == "table"
|
|
85
|
+
Reporters::Table.new(findings, color: options[:color]).render
|
|
86
|
+
else
|
|
87
|
+
SchemaReaper.reporter(options[:format]).new(findings).render
|
|
88
|
+
end
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
def console
|
|
92
|
+
@console ||= Reporters::Console.new(color: options[:color])
|
|
65
93
|
end
|
|
66
94
|
|
|
67
95
|
def enforce_baseline(findings)
|
|
68
96
|
new_ones = Baseline.new(config.baseline_path).new_among(findings)
|
|
69
97
|
return if new_ones.empty?
|
|
70
98
|
|
|
71
|
-
|
|
72
|
-
|
|
99
|
+
console.problem(
|
|
100
|
+
"#{new_ones.size} new finding#{"s" unless new_ones.size == 1} since the baseline",
|
|
101
|
+
items: new_ones.map(&:id)
|
|
102
|
+
)
|
|
73
103
|
exit 1
|
|
74
104
|
end
|
|
75
105
|
end
|
data/lib/schema_reaper/config.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "yaml"
|
|
4
|
+
require_relative "database_url"
|
|
4
5
|
|
|
5
6
|
module SchemaReaper
|
|
6
7
|
# Loaded from .schema_reaper.yml at the project root. Every key has a default
|
|
@@ -22,12 +23,13 @@ module SchemaReaper
|
|
|
22
23
|
"runtime_log" => ".schema_reaper/runtime.jsonl",
|
|
23
24
|
"history_log" => ".schema_reaper/history.jsonl",
|
|
24
25
|
"baseline" => ".schema_reaper/baseline.json",
|
|
26
|
+
"database_yml" => "config/database.yml", # Rails fallback for the connection
|
|
25
27
|
"require" => [] # extra files to load (custom analyzers)
|
|
26
28
|
}.freeze
|
|
27
29
|
|
|
28
30
|
def self.load(path = ".schema_reaper.yml")
|
|
29
31
|
raw = File.exist?(path) ? (YAML.safe_load_file(path) || {}) : {}
|
|
30
|
-
new(deep_merge(DEFAULTS, raw))
|
|
32
|
+
new(deep_merge(DEFAULTS, raw), root: File.dirname(File.expand_path(path)))
|
|
31
33
|
end
|
|
32
34
|
|
|
33
35
|
def self.deep_merge(base, override)
|
|
@@ -36,12 +38,19 @@ module SchemaReaper
|
|
|
36
38
|
end
|
|
37
39
|
end
|
|
38
40
|
|
|
39
|
-
def initialize(data)
|
|
41
|
+
def initialize(data, root: Dir.pwd)
|
|
40
42
|
@data = data
|
|
43
|
+
@root = root
|
|
41
44
|
end
|
|
42
45
|
|
|
46
|
+
# Resolution order:
|
|
47
|
+
# 1. database_url: in .schema_reaper.yml
|
|
48
|
+
# 2. ENV["DATABASE_URL"]
|
|
49
|
+
# 3. config/database.yml for the current environment (Rails apps)
|
|
43
50
|
def database_url
|
|
44
|
-
@data["database_url"] ||
|
|
51
|
+
@data["database_url"] ||
|
|
52
|
+
ENV.fetch("DATABASE_URL", nil) ||
|
|
53
|
+
DatabaseUrl.from_rails(root: @root, path: @data["database_yml"])
|
|
45
54
|
end
|
|
46
55
|
|
|
47
56
|
def scan_paths
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "yaml"
|
|
4
|
+
require "erb"
|
|
5
|
+
|
|
6
|
+
module SchemaReaper
|
|
7
|
+
# Best-effort resolution of a PostgreSQL connection URL from a Rails project,
|
|
8
|
+
# so `schema_reaper scan` works in an app without `DATABASE_URL` exported or a
|
|
9
|
+
# `database_url:` in .schema_reaper.yml.
|
|
10
|
+
#
|
|
11
|
+
# Order of preference is applied by Config; this module only handles the
|
|
12
|
+
# config/database.yml case.
|
|
13
|
+
module DatabaseUrl
|
|
14
|
+
module_function
|
|
15
|
+
|
|
16
|
+
POSTGRES_ADAPTERS = %w[postgresql postgis postgres].freeze
|
|
17
|
+
|
|
18
|
+
# @return [String, nil]
|
|
19
|
+
def from_rails(root: Dir.pwd, path: "config/database.yml", env: nil)
|
|
20
|
+
file = File.expand_path(path, root)
|
|
21
|
+
return nil unless File.file?(file)
|
|
22
|
+
|
|
23
|
+
section = section_for(load_yaml(file), env || rails_env)
|
|
24
|
+
return nil unless section.is_a?(Hash)
|
|
25
|
+
|
|
26
|
+
section["url"] || build_url(section)
|
|
27
|
+
rescue StandardError
|
|
28
|
+
nil
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def rails_env
|
|
32
|
+
ENV["SCHEMA_REAPER_ENV"] || ENV["RAILS_ENV"] || ENV["RACK_ENV"] || "development"
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def load_yaml(file)
|
|
36
|
+
rendered = ERB.new(File.read(file)).result
|
|
37
|
+
begin
|
|
38
|
+
YAML.safe_load(rendered, aliases: true)
|
|
39
|
+
rescue ArgumentError # older Psych without the aliases: kwarg
|
|
40
|
+
YAML.safe_load(rendered, [], [], true)
|
|
41
|
+
end
|
|
42
|
+
rescue StandardError
|
|
43
|
+
YAML.load_file(file)
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# Rails 6+ allows `development: { primary: {...}, replica: {...} }`. Pick the
|
|
47
|
+
# primary, or the first sub-config, when the env section is nested.
|
|
48
|
+
def section_for(data, env)
|
|
49
|
+
return nil unless data.is_a?(Hash)
|
|
50
|
+
|
|
51
|
+
section = data[env] || data[env.to_s]
|
|
52
|
+
return section unless nested?(section)
|
|
53
|
+
|
|
54
|
+
section["primary"] || section.values.find { |v| v.is_a?(Hash) }
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def nested?(section)
|
|
58
|
+
section.is_a?(Hash) &&
|
|
59
|
+
!section.key?("adapter") && !section.key?("url") && !section.key?("database") &&
|
|
60
|
+
section.values.any?(Hash)
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def build_url(section)
|
|
64
|
+
adapter = section["adapter"].to_s
|
|
65
|
+
return nil unless POSTGRES_ADAPTERS.include?(adapter)
|
|
66
|
+
|
|
67
|
+
db = section["database"]
|
|
68
|
+
return nil if db.to_s.empty?
|
|
69
|
+
|
|
70
|
+
userinfo = [section["username"], section["password"]].compact.join(":")
|
|
71
|
+
host = section["host"].to_s
|
|
72
|
+
hostport = host.empty? ? "" : "#{host}#{":#{section["port"]}" if section["port"]}"
|
|
73
|
+
auth = userinfo.empty? ? "" : "#{userinfo}@"
|
|
74
|
+
|
|
75
|
+
"postgresql://#{auth}#{hostport}/#{db}"
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
end
|
|
@@ -36,6 +36,14 @@ module SchemaReaper
|
|
|
36
36
|
self[:reclaimable_bytes] || 0
|
|
37
37
|
end
|
|
38
38
|
|
|
39
|
+
# Column and/or index this finding points at, without the table name
|
|
40
|
+
# (callers that group by table already show it). nil for a whole-table
|
|
41
|
+
# finding.
|
|
42
|
+
def target_label
|
|
43
|
+
parts = [column, index].compact
|
|
44
|
+
parts.empty? ? nil : parts.join(" · ")
|
|
45
|
+
end
|
|
46
|
+
|
|
39
47
|
def to_h
|
|
40
48
|
super.merge(id: id, reclaimable_bytes: reclaimable_bytes)
|
|
41
49
|
end
|
|
@@ -49,6 +49,8 @@ module SchemaReaper
|
|
|
49
49
|
snapshots: snaps.size,
|
|
50
50
|
first_at: first.at,
|
|
51
51
|
last_at: last.at,
|
|
52
|
+
latest_count: last.count,
|
|
53
|
+
latest_bytes: last.reclaimable_bytes,
|
|
52
54
|
count_change_total: last.count - first.count,
|
|
53
55
|
count_change_last: last.count - prev.count,
|
|
54
56
|
newly_introduced: (last.ids - prev.ids),
|
|
@@ -12,11 +12,17 @@ module SchemaReaper
|
|
|
12
12
|
"timestamp with time zone" => 8, "uuid" => 16
|
|
13
13
|
}.freeze
|
|
14
14
|
|
|
15
|
+
NO_URL = "no database connection found. schema_reaper looks, in order, for: " \
|
|
16
|
+
"database_url: in .schema_reaper.yml; the DATABASE_URL env var; " \
|
|
17
|
+
"config/database.yml for RAILS_ENV (default: development, Postgres only)."
|
|
18
|
+
|
|
15
19
|
def initialize(url)
|
|
16
|
-
|
|
20
|
+
require "pg" # load first so the PG::Error rescue below can resolve
|
|
21
|
+
raise Error, NO_URL if url.nil? || url.empty?
|
|
17
22
|
|
|
18
|
-
require "pg"
|
|
19
23
|
@conn = PG.connect(url)
|
|
24
|
+
rescue PG::Error => e
|
|
25
|
+
raise Error, "could not connect to the database: #{e.message.strip}"
|
|
20
26
|
end
|
|
21
27
|
|
|
22
28
|
def call
|
|
@@ -140,10 +146,17 @@ module SchemaReaper
|
|
|
140
146
|
SQL
|
|
141
147
|
end
|
|
142
148
|
|
|
149
|
+
# pg_class.reltuples is an estimate maintained by ANALYZE/VACUUM. On
|
|
150
|
+
# PostgreSQL 14+ it is -1 for a relation that has never been analysed,
|
|
151
|
+
# which is "unknown", not "minus one row". Map anything negative to nil so
|
|
152
|
+
# analyzers treat the count as unavailable.
|
|
143
153
|
def row_count_for(table)
|
|
144
|
-
exec(<<~SQL, [table]).first&.fetch("reltuples")&.to_f
|
|
154
|
+
raw = exec(<<~SQL, [table]).first&.fetch("reltuples")&.to_f
|
|
145
155
|
SELECT reltuples FROM pg_class WHERE relname = $1
|
|
146
156
|
SQL
|
|
157
|
+
return nil if raw.nil? || raw.negative?
|
|
158
|
+
|
|
159
|
+
raw.round
|
|
147
160
|
end
|
|
148
161
|
|
|
149
162
|
def exec(sql, params = nil)
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SchemaReaper
|
|
4
|
+
module Reporters
|
|
5
|
+
# Minimal ANSI styling. Colour is emitted only for an interactive terminal
|
|
6
|
+
# and stays off when NO_COLOR is set, when output is redirected, or when
|
|
7
|
+
# the caller forces it off.
|
|
8
|
+
class Ansi
|
|
9
|
+
CODES = {
|
|
10
|
+
reset: 0, bold: 1, dim: 2,
|
|
11
|
+
red: 31, green: 32, yellow: 33, blue: 34, magenta: 35, cyan: 36, grey: 90
|
|
12
|
+
}.freeze
|
|
13
|
+
|
|
14
|
+
SEVERITY_COLOUR = { high: :red, medium: :yellow, low: :cyan }.freeze
|
|
15
|
+
|
|
16
|
+
def initialize(io: $stdout, enabled: nil)
|
|
17
|
+
@on = enabled.nil? ? auto?(io) : enabled
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def on?
|
|
21
|
+
@on
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
# paint(:bold, :red) { "text" } or paint("text", :bold, :red)
|
|
25
|
+
def paint(text, *styles)
|
|
26
|
+
return text unless @on && !styles.empty?
|
|
27
|
+
|
|
28
|
+
seq = styles.filter_map { |s| CODES[s] }.join(";")
|
|
29
|
+
"\e[#{seq}m#{text}\e[0m"
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def severity(sev, text = sev.to_s)
|
|
33
|
+
paint(text, :bold, SEVERITY_COLOUR.fetch(sev, :grey))
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# A five-cell bar for a 0.0..1.0 value, coloured by severity.
|
|
37
|
+
def confidence_bar(value, sev)
|
|
38
|
+
filled = (value * 5).round.clamp(0, 5)
|
|
39
|
+
bar = ("█" * filled) + ("░" * (5 - filled))
|
|
40
|
+
paint(bar, SEVERITY_COLOUR.fetch(sev, :grey))
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
private
|
|
44
|
+
|
|
45
|
+
def auto?(io)
|
|
46
|
+
return false if ENV["NO_COLOR"] && !ENV["NO_COLOR"].empty?
|
|
47
|
+
return false unless io.respond_to?(:tty?) && io.tty?
|
|
48
|
+
|
|
49
|
+
true
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
end
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "ansi"
|
|
4
|
+
|
|
5
|
+
module SchemaReaper
|
|
6
|
+
module Reporters
|
|
7
|
+
# Shared styling for the CLI's short command output (baseline, trend,
|
|
8
|
+
# generate-migration, the CI gate and stderr notices). Keeps every command
|
|
9
|
+
# looking like it belongs to the same tool.
|
|
10
|
+
class Console
|
|
11
|
+
def initialize(out: $stdout, err: $stderr, color: nil)
|
|
12
|
+
@out = out
|
|
13
|
+
@err = err
|
|
14
|
+
@a = Ansi.new(io: out, enabled: color)
|
|
15
|
+
@ae = Ansi.new(io: err, enabled: color)
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def blank
|
|
19
|
+
@out.puts
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
# " schema_reaper trend · 3 snapshots"
|
|
23
|
+
def title(command, meta = nil)
|
|
24
|
+
line = " #{@a.paint("schema_reaper", :bold)} #{@a.paint(command, :bold)}"
|
|
25
|
+
line << " #{@a.paint("· #{meta}", :dim)}" if meta
|
|
26
|
+
@out.puts
|
|
27
|
+
@out.puts line
|
|
28
|
+
@out.puts
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def ok(message)
|
|
32
|
+
@out.puts " #{@a.paint("✓", :green, :bold)} #{message}"
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def info(message)
|
|
36
|
+
@out.puts " #{message}"
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
# aligned "label value (aside)" row
|
|
40
|
+
def row(label, value, aside = nil)
|
|
41
|
+
text = format(" %-15s %s", label, @a.paint(value.to_s, :bold))
|
|
42
|
+
text << " #{@a.paint("(#{aside})", :dim)}" if aside && !aside.empty?
|
|
43
|
+
@out.puts text
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def list(heading, items, colour: :dim)
|
|
47
|
+
return if items.empty?
|
|
48
|
+
|
|
49
|
+
@out.puts " #{@a.paint(heading, colour)}" if heading && !heading.empty?
|
|
50
|
+
items.each { |i| @out.puts " #{@a.paint("- #{i}", colour)}" }
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
# A single stderr notice, e.g. the unused-index skip.
|
|
54
|
+
def notice(message)
|
|
55
|
+
@err.puts " #{@ae.paint("!", :yellow, :bold)} #{message}"
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
def problem(message, items: [])
|
|
59
|
+
@err.puts
|
|
60
|
+
@err.puts " #{@ae.paint("✗", :red, :bold)} #{@ae.paint(message, :bold)}"
|
|
61
|
+
items.each { |i| @err.puts " #{@ae.paint("- #{i}", :red)}" }
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
end
|
|
65
|
+
end
|
|
@@ -1,42 +1,100 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "bytes"
|
|
4
|
+
require_relative "ansi"
|
|
4
5
|
|
|
5
6
|
module SchemaReaper
|
|
6
7
|
module Reporters
|
|
7
|
-
# Human-readable terminal
|
|
8
|
+
# Human-readable terminal report: a summary line, findings grouped by table
|
|
9
|
+
# and sorted by confidence, then a severity/type tally. Colour is used only
|
|
10
|
+
# on an interactive terminal (see Ansi).
|
|
8
11
|
class Table
|
|
9
12
|
SEV_ORDER = { high: 0, medium: 1, low: 2 }.freeze
|
|
10
13
|
|
|
11
|
-
def initialize(findings, io: $stdout)
|
|
14
|
+
def initialize(findings, io: $stdout, color: nil)
|
|
12
15
|
@findings = findings
|
|
13
16
|
@io = io
|
|
17
|
+
@a = Ansi.new(io: io, enabled: color)
|
|
14
18
|
end
|
|
15
19
|
|
|
16
20
|
def render
|
|
17
|
-
if @findings.empty?
|
|
18
|
-
@io.puts "schema_reaper: no findings. Schema is lean."
|
|
19
|
-
return
|
|
20
|
-
end
|
|
21
|
+
return render_clean if @findings.empty?
|
|
21
22
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
"~#{Bytes.human(total_reclaimable)} reclaimable."
|
|
23
|
+
header
|
|
24
|
+
grouped.each { |table, group| render_table(table, group) }
|
|
25
|
+
footer
|
|
26
26
|
end
|
|
27
27
|
|
|
28
28
|
private
|
|
29
29
|
|
|
30
|
-
def
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
@io.puts
|
|
30
|
+
def render_clean
|
|
31
|
+
@io.puts @a.paint(" ✓ no findings — schema is lean", :green, :bold)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def header
|
|
35
|
+
@io.puts
|
|
36
|
+
@io.puts " #{@a.paint("schema_reaper", :bold)} " \
|
|
37
|
+
"#{@a.paint("#{@findings.size} finding#{"s" unless @findings.size == 1}", :bold)} " \
|
|
38
|
+
"#{@a.paint("~#{Bytes.human(total_reclaimable)} reclaimable", :dim)}"
|
|
39
|
+
@io.puts
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def grouped
|
|
43
|
+
@findings
|
|
44
|
+
.sort_by { |f| [-f.confidence, SEV_ORDER.fetch(f.severity, 9)] }
|
|
45
|
+
.group_by(&:table)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def render_table(table, group)
|
|
49
|
+
@io.puts " #{@a.paint(table, :bold, :magenta)}"
|
|
50
|
+
group.each { |f| render_finding(f) }
|
|
51
|
+
@io.puts
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def render_finding(finding)
|
|
55
|
+
@io.puts finding_head(finding)
|
|
56
|
+
@io.puts " #{@a.paint(evidence_line(finding), :dim)}"
|
|
57
|
+
fix = "→ #{finding.suggested_fix}"
|
|
58
|
+
@io.puts " #{@a.paint(fix, :green)}"
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def finding_head(finding)
|
|
62
|
+
pct = (finding.confidence * 100).round
|
|
63
|
+
bar = @a.confidence_bar(finding.confidence, finding.severity)
|
|
64
|
+
sev = @a.severity(finding.severity, format("%-6s", finding.severity))
|
|
65
|
+
label = finding.target_label
|
|
66
|
+
type_text = label ? format("%-19s", finding.type) : finding.type.to_s
|
|
67
|
+
|
|
68
|
+
head = format(" %s %3d%% %s %s", bar, pct, sev, @a.paint(type_text, :bold))
|
|
69
|
+
head << " #{label}" if label
|
|
70
|
+
head << rjust_bytes(finding)
|
|
71
|
+
head
|
|
36
72
|
end
|
|
37
73
|
|
|
38
|
-
def
|
|
39
|
-
|
|
74
|
+
def rjust_bytes(finding)
|
|
75
|
+
return "" unless finding.reclaimable_bytes.positive?
|
|
76
|
+
|
|
77
|
+
human = Bytes.human(finding.reclaimable_bytes)
|
|
78
|
+
" #{@a.paint(human, :dim)}"
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def evidence_line(finding)
|
|
82
|
+
finding.evidence.join(" · ")
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
def footer
|
|
86
|
+
by_sev = @findings.group_by(&:severity)
|
|
87
|
+
tally = SEV_ORDER.keys.filter_map do |sev|
|
|
88
|
+
n = by_sev[sev]&.size
|
|
89
|
+
@a.severity(sev, "#{sev} #{n}") if n
|
|
90
|
+
end.join(" ")
|
|
91
|
+
|
|
92
|
+
by_type = @findings.group_by(&:type).transform_values(&:size)
|
|
93
|
+
.sort_by { |_t, n| -n }
|
|
94
|
+
.map { |t, n| "#{t} #{n}" }.join(" · ")
|
|
95
|
+
|
|
96
|
+
@io.puts " #{tally}"
|
|
97
|
+
@io.puts " #{@a.paint(by_type, :dim)}"
|
|
40
98
|
end
|
|
41
99
|
|
|
42
100
|
def total_reclaimable
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "bytes"
|
|
4
|
+
require_relative "console"
|
|
5
|
+
|
|
6
|
+
module SchemaReaper
|
|
7
|
+
module Reporters
|
|
8
|
+
# Formats History#trend into a readable progress block instead of a raw
|
|
9
|
+
# hash dump.
|
|
10
|
+
class Trend
|
|
11
|
+
def initialize(data, io: $stdout, color: nil)
|
|
12
|
+
@d = data
|
|
13
|
+
@c = Console.new(out: io, color: color)
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def render
|
|
17
|
+
snaps = @d[:snapshots].to_i
|
|
18
|
+
@c.title("trend", "#{snaps} snapshot#{"s" unless snaps == 1}")
|
|
19
|
+
|
|
20
|
+
if snaps.zero?
|
|
21
|
+
@c.info("no snapshots yet — run `schema_reaper trend` to record the first.")
|
|
22
|
+
return
|
|
23
|
+
end
|
|
24
|
+
if snaps == 1
|
|
25
|
+
@c.info("first snapshot recorded. Run this again later to see the delta.")
|
|
26
|
+
dates
|
|
27
|
+
return
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
counts
|
|
31
|
+
bytes
|
|
32
|
+
@c.blank
|
|
33
|
+
@c.list("new since last run:", @d[:newly_introduced].to_a, colour: :yellow)
|
|
34
|
+
@c.list("resolved since last run:", @d[:resolved_since_prev].to_a, colour: :green)
|
|
35
|
+
@c.blank
|
|
36
|
+
dates
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
private
|
|
40
|
+
|
|
41
|
+
def counts
|
|
42
|
+
total = signed(@d[:count_change_total])
|
|
43
|
+
last = signed(@d[:count_change_last])
|
|
44
|
+
@c.row("findings", @d.fetch(:latest_count, "—"),
|
|
45
|
+
"#{total} since first run · #{last} since last")
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def bytes
|
|
49
|
+
delta = @d[:bytes_change_total].to_i
|
|
50
|
+
sign = delta.positive? ? "+" : "-"
|
|
51
|
+
@c.row("reclaimable", Bytes.human(@d.fetch(:latest_bytes, 0)),
|
|
52
|
+
"#{sign}#{Bytes.human(delta.abs)} since first run")
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def dates
|
|
56
|
+
@c.row("first snapshot", short(@d[:first_at]))
|
|
57
|
+
@c.row("latest", short(@d[:last_at]))
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def signed(number)
|
|
61
|
+
n = number.to_i
|
|
62
|
+
n.positive? ? "+#{n}" : n.to_s
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
def short(iso)
|
|
66
|
+
iso.to_s.split("T").first
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
data/lib/schema_reaper/runner.rb
CHANGED
|
@@ -37,9 +37,11 @@ module SchemaReaper
|
|
|
37
37
|
def warn_missing_query_history(db)
|
|
38
38
|
return if db.query_history?
|
|
39
39
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
40
|
+
Reporters::Console.new.notice(
|
|
41
|
+
"unused_index skipped — only #{db.index_scan_total} scan(s) across " \
|
|
42
|
+
"#{db.index_count} indexes, so there is no query history to judge by. " \
|
|
43
|
+
"Run against a database that has served production traffic."
|
|
44
|
+
)
|
|
43
45
|
end
|
|
44
46
|
|
|
45
47
|
def dedupe(findings)
|
data/lib/schema_reaper.rb
CHANGED
|
@@ -18,7 +18,10 @@ require_relative "schema_reaper/analyzers/missing_fk_index"
|
|
|
18
18
|
require_relative "schema_reaper/analyzers/always_null_column"
|
|
19
19
|
require_relative "schema_reaper/analyzers/single_value_column"
|
|
20
20
|
require_relative "schema_reaper/reporters/bytes"
|
|
21
|
+
require_relative "schema_reaper/reporters/ansi"
|
|
22
|
+
require_relative "schema_reaper/reporters/console"
|
|
21
23
|
require_relative "schema_reaper/reporters/table"
|
|
24
|
+
require_relative "schema_reaper/reporters/trend"
|
|
22
25
|
require_relative "schema_reaper/reporters/json"
|
|
23
26
|
require_relative "schema_reaper/reporters/markdown"
|
|
24
27
|
require_relative "schema_reaper/reporters/sarif"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: schema_reaper
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.0.
|
|
4
|
+
version: 1.0.9
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- aksshatt
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: exe
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2026-09-
|
|
12
|
+
date: 2026-09-09 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
name: prism
|
|
@@ -103,6 +103,7 @@ files:
|
|
|
103
103
|
- LICENSE.txt
|
|
104
104
|
- PRO.md
|
|
105
105
|
- README.md
|
|
106
|
+
- RELEASE_CHECKLIST.md
|
|
106
107
|
- Rakefile
|
|
107
108
|
- exe/schema_reaper
|
|
108
109
|
- lib/schema_reaper.rb
|
|
@@ -118,17 +119,21 @@ files:
|
|
|
118
119
|
- lib/schema_reaper/baseline.rb
|
|
119
120
|
- lib/schema_reaper/cli.rb
|
|
120
121
|
- lib/schema_reaper/config.rb
|
|
122
|
+
- lib/schema_reaper/database_url.rb
|
|
121
123
|
- lib/schema_reaper/finding.rb
|
|
122
124
|
- lib/schema_reaper/gem_awareness.rb
|
|
123
125
|
- lib/schema_reaper/history.rb
|
|
124
126
|
- lib/schema_reaper/introspect/postgres.rb
|
|
125
127
|
- lib/schema_reaper/migration_generator.rb
|
|
126
128
|
- lib/schema_reaper/railtie.rb
|
|
129
|
+
- lib/schema_reaper/reporters/ansi.rb
|
|
127
130
|
- lib/schema_reaper/reporters/bytes.rb
|
|
131
|
+
- lib/schema_reaper/reporters/console.rb
|
|
128
132
|
- lib/schema_reaper/reporters/json.rb
|
|
129
133
|
- lib/schema_reaper/reporters/markdown.rb
|
|
130
134
|
- lib/schema_reaper/reporters/sarif.rb
|
|
131
135
|
- lib/schema_reaper/reporters/table.rb
|
|
136
|
+
- lib/schema_reaper/reporters/trend.rb
|
|
132
137
|
- lib/schema_reaper/runner.rb
|
|
133
138
|
- lib/schema_reaper/runtime.rb
|
|
134
139
|
- lib/schema_reaper/schema.rb
|