tewip-ai 0.30.0 → 0.31.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -552,6 +552,52 @@ With `--json` the verdict is **sealed**: editing `not-ready` to `ready` in the f
552
552
  fail, so a verdict that travels into a ticket or a release note can be checked rather than trusted. It is
553
553
  also an MCP tool, so an agent can ask whether its own work is done before saying that it is.
554
554
 
555
+ ## What will this migration do to a live database? (Phase 26)
556
+
557
+ ```bash
558
+ tewip schema --diff main...HEAD # or: tewip schema migrations/0002_drop_nickname.sql
559
+ tewip schema --strict # fails the build on a change that breaks the running version
560
+ ```
561
+
562
+ The most expensive change in a pull request is usually not a test — it is a migration. A column dropped
563
+ while the previous deployment is still selecting it. An index built without `CONCURRENTLY`, blocking every
564
+ write to the busiest table. A `NOT NULL` added to a populated table. These are known, documented, and a
565
+ tool can name every one of them from the text of the migration.
566
+
567
+ ```
568
+ 1 of 1 migration file(s) read. 1 change(s) would break the running version or destroy data
569
+
570
+ ## Breaks the version that is still running (1)
571
+ - drops the column nickname from orders — migrations/0002_drop_nickname.sql:1
572
+ - Instead: deploy code that no longer selects or writes the column, then drop it in a later
573
+ migration. During a rolling deploy the old version is still running and will fail on a column
574
+ that is gone
575
+
576
+ ## Code that still uses what this takes away (1)
577
+ - orders.nickname is still named in 1 file(s): app/orders.py
578
+ ```
579
+
580
+ That last section is the part no SQL linter does: it searches your own code for what the migration takes
581
+ away. "This drops `users.nickname`, and four files still read it" is a different sentence from "dropping a
582
+ column is risky", and it is the one that stops the outage.
583
+
584
+ It reads **SQL, Django, Rails and Alembic** migrations, judges **only the forward path** (a rollback step
585
+ contains the mirror image of the change, so reading it says a migration that creates a table destroys
586
+ one), and stays quiet when the team already did the safe thing — `atomic = False`, `algorithm:
587
+ :concurrently`, `NOT VALID`, `postgresql_concurrently=True`. When it cannot read a migration it says so
588
+ rather than passing it, and "read and nothing found" never looks like "not read".
589
+
590
+ **Tried on real projects.** 200 Zulip migrations: 199 read, 19 changes that break the running version or
591
+ destroy data, each sampled finding correct when read against the file — including a `DROP COLUMN` buried
592
+ in raw SQL inside a Python data migration. 60 Mastodon migrations: it caught a `change_column` that
593
+ rewrites a table, in a migration whose *name* says it only adds one.
594
+
595
+ What it does not know, and says so: your table sizes and how busy the database is, so it reports what a
596
+ statement **does** — rewrites the table, scans every row, blocks writes — and leaves the duration to the
597
+ person who knows the data. In CI the findings go into the pull request comment, and anything that breaks
598
+ the running version enters the evidence bundle as a refusal: a passing test suite does not verify a
599
+ migration.
600
+
555
601
  ## Run only the tests that could notice this change (Phase 25)
556
602
 
557
603
  ```bash
@@ -588,6 +634,22 @@ test that would have caught the bug; on a monolith where every test imports the
588
634
  there is nothing to skip, rather than inventing a saving. It is also an MCP tool, so an agent can ask what
589
635
  to run before running anything.
590
636
 
637
+ **In CI**, `mode: pick` runs before your tests and hands the workflow the files to run:
638
+
639
+ ```yaml
640
+ - id: pick
641
+ uses: hmamut39/tewip@main
642
+ with: { mode: pick } # needs fetch-depth: 0 on actions/checkout
643
+ - run: npx jest ${{ steps.pick.outputs.tests }}
644
+ if: steps.pick.outputs.narrowed == 'true'
645
+ - run: npx jest
646
+ if: steps.pick.outputs.narrowed != 'true'
647
+ ```
648
+
649
+ Whatever it leaves out is written into the job summary with the reason beside it, so "why did CI not run
650
+ that test" is answered by the run itself. When it will not narrow safely, `narrowed` is `false` and
651
+ `tests` is empty - so a workflow that ignores the flag still runs everything, which is the safe default.
652
+
591
653
  ## Would your tests catch a bug? (Phase 23)
592
654
 
593
655
  ```bash
package/dist/cli/help.js CHANGED
@@ -118,6 +118,13 @@ Usage:
118
118
  is what was asked for covered. Answers ready, not ready, or cannot
119
119
  say - and cannot say is never rounded up to ready. --json writes a
120
120
  sealed verdict that can be checked later. No model, no cost.
121
+ tewip schema [<files>] [--diff main...HEAD] [--strict] [--out schema.md]
122
+ What will this migration do to a live database? Names the changes
123
+ that break the version still running during a deploy, destroy data,
124
+ block writes while they run, or cannot be undone - and searches your
125
+ code for what still uses the column it takes away. Reads SQL, Django,
126
+ Rails and Alembic migrations, and says so when it cannot read one.
127
+ No model, no cost.
121
128
  tewip pick [--diff main...HEAD] [--list] [--out picked.md]
122
129
  Which tests could notice this change? Narrows a test run using
123
130
  facts rather than a prediction: a test runs when it imports a
package/dist/cli/tewip.js CHANGED
@@ -199,6 +199,17 @@ async function main() {
199
199
  });
200
200
  return;
201
201
  }
202
+ case 'schema': {
203
+ const { runSchema } = await import('../engine/schema/schema-command.js');
204
+ runSchema({
205
+ root: process.cwd(),
206
+ diff: flag(args, 'diff') ?? 'main...HEAD',
207
+ files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
208
+ out: flag(args, 'out'),
209
+ strict: args.includes('--strict'),
210
+ });
211
+ return;
212
+ }
202
213
  case 'pick': {
203
214
  const { runPick } = await import('../engine/pick/pick-command.js');
204
215
  runPick({
@@ -0,0 +1,230 @@
1
+ export function dialectOf(file, text) {
2
+ const path = file.split('\\').join('/').toLowerCase();
3
+ if (path.endsWith('.sql'))
4
+ return 'sql';
5
+ // `class Migration(migrations.Migration)` is what makes a file a Django migration. Keying only on the
6
+ // operation names left Zulip's data migrations unread - and a data migration is exactly the kind that
7
+ // runs for an hour with a transaction held open.
8
+ if (/\bmigrations\.Migration\b|\bmigrations\.(?:CreateModel|AddField|RemoveField|AlterField|RenameField|RunSQL|RunPython|DeleteModel|RenameModel)\b/.test(text))
9
+ return 'django';
10
+ if (/\bActiveRecord::Migration\b|\bdef\s+(?:up|change)\b[\s\S]*\b(?:add_column|remove_column|add_index|change_column|rename_column|drop_table)\b/.test(text))
11
+ return 'rails';
12
+ if (/\bop\.(?:drop_column|add_column|create_index|alter_column|drop_table|rename_table|execute)\b/.test(text))
13
+ return 'alembic';
14
+ if (/(^|\/)(migrations?|db\/migrate)\//.test(path))
15
+ return 'unknown';
16
+ return 'unknown';
17
+ }
18
+ const STR = String.raw `['"]([\w.]+)['"]`;
19
+ /** Django's operations. `model_name` and `name` are the table and the column. */
20
+ const DJANGO = [
21
+ {
22
+ find: new RegExp(String.raw `migrations\.RemoveField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?name=${STR}`, 'i'),
23
+ risk: 'breaks-old-code',
24
+ what: (m) => `removes the field ${m[2]} from ${m[1]}`,
25
+ instead: 'stop reading and writing the field, deploy that, then remove it in a later migration. The version still running during a rolling deploy will select a column that no longer exists',
26
+ table: (m) => m[1],
27
+ column: (m) => m[2],
28
+ },
29
+ {
30
+ find: new RegExp(String.raw `migrations\.RenameField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?old_name=${STR}`, 'i'),
31
+ risk: 'breaks-old-code',
32
+ what: (m) => `renames ${m[1]}.${m[2]}`,
33
+ instead: 'add the new field, write to both, move the readers, then remove the old one',
34
+ table: (m) => m[1],
35
+ column: (m) => m[2],
36
+ },
37
+ {
38
+ find: new RegExp(String.raw `migrations\.DeleteModel\([\s\S]{0,120}?${STR}`, 'i'),
39
+ risk: 'destroys-data',
40
+ what: (m) => `deletes the model ${m[1]} and its table`,
41
+ instead: 'stop using it, ship that, confirm nothing reads it, and delete it later',
42
+ table: (m) => m[1],
43
+ },
44
+ {
45
+ find: new RegExp(String.raw `migrations\.AddField\([\s\S]{0,400}?null=False(?![\s\S]{0,400}?default=)`, 'i'),
46
+ risk: 'breaks-old-code',
47
+ what: () => 'adds a field that cannot be null and has no default',
48
+ instead: 'add it with null=True or a default. Without one, Django rewrites the table and every insert from the running version fails',
49
+ },
50
+ {
51
+ find: new RegExp(String.raw `migrations\.AlterField\([\s\S]{0,400}?null=False`, 'i'),
52
+ risk: 'locks-writes',
53
+ what: () => 'makes an existing field NOT NULL, which reads every row under a lock',
54
+ instead: 'backfill first, add a validated constraint, then alter - or use SeparateDatabaseAndState with the safe SQL',
55
+ },
56
+ {
57
+ // A data migration. Django wraps a migration in one transaction, so this holds it open for as long as
58
+ // the work takes - fine for a hundred rows, an outage for a hundred million.
59
+ find: /migrations\.RunPython\(/,
60
+ unless: /atomic\s*=\s*False/,
61
+ risk: 'slow',
62
+ what: () => 'runs application code against the data inside the migration transaction',
63
+ instead: 'for a large table, work in batches with atomic = False on the migration and make it resumable. For a small one this is fine - Tewip cannot see how many rows there are',
64
+ },
65
+ {
66
+ find: /AddIndex\(|AddConstraint\(/i,
67
+ unless: /AddIndexConcurrently|AddConstraintNotValid|atomic\s*=\s*False/,
68
+ risk: 'locks-writes',
69
+ what: () => 'adds an index or constraint, which Django builds with a lock unless told otherwise',
70
+ instead: 'AddIndexConcurrently / AddConstraintNotValid from django.contrib.postgres.operations, with atomic = False on the migration',
71
+ },
72
+ ];
73
+ /** Rails migrations. */
74
+ const RAILS = [
75
+ {
76
+ find: new RegExp(String.raw `remove_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
77
+ risk: 'breaks-old-code',
78
+ what: (m) => `removes ${m[1]}.${m[2]}`,
79
+ instead: "ignore the column in the model (self.ignored_columns), deploy that, then remove it. Rails caches the column list, so the running version keeps selecting it",
80
+ table: (m) => m[1],
81
+ column: (m) => m[2],
82
+ },
83
+ {
84
+ find: new RegExp(String.raw `rename_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
85
+ risk: 'breaks-old-code',
86
+ what: (m) => `renames ${m[1]}.${m[2]}`,
87
+ instead: 'add the new column, write to both, move the readers, then remove the old one',
88
+ table: (m) => m[1],
89
+ column: (m) => m[2],
90
+ },
91
+ {
92
+ find: new RegExp(String.raw `drop_table\s*[\s(]\s*:?(\w+)`, 'i'),
93
+ risk: 'destroys-data',
94
+ what: (m) => `drops the table ${m[1]}`,
95
+ instead: 'stop using it, ship that, then drop it later',
96
+ table: (m) => m[1],
97
+ },
98
+ {
99
+ find: /add_index\s*[\s(]/i,
100
+ unless: /algorithm:\s*:concurrently/,
101
+ risk: 'locks-writes',
102
+ what: () => 'adds an index without algorithm: :concurrently, so writes to the table block while it builds',
103
+ instead: 'add_index ..., algorithm: :concurrently, with disable_ddl_transaction! on the migration',
104
+ },
105
+ {
106
+ find: new RegExp(String.raw `change_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
107
+ risk: 'locks-writes',
108
+ what: (m) => `changes the type of ${m[1]}.${m[2]}, which usually rewrites the table`,
109
+ instead: 'add a new column, backfill in batches, switch the readers, then drop the old one',
110
+ table: (m) => m[1],
111
+ column: (m) => m[2],
112
+ },
113
+ {
114
+ find: new RegExp(String.raw `change_column_null\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)\s*,\s*false`, 'i'),
115
+ risk: 'locks-writes',
116
+ what: (m) => `makes ${m[1]}.${m[2]} NOT NULL, which reads every row under a lock`,
117
+ instead: 'add a validated check constraint first, then set the column NOT NULL',
118
+ table: (m) => m[1],
119
+ column: (m) => m[2],
120
+ },
121
+ ];
122
+ /** Alembic. */
123
+ const ALEMBIC = [
124
+ {
125
+ find: new RegExp(String.raw `op\.drop_column\(\s*${STR}\s*,\s*${STR}`, 'i'),
126
+ risk: 'breaks-old-code',
127
+ what: (m) => `drops ${m[1]}.${m[2]}`,
128
+ instead: 'stop reading and writing it, deploy that, then drop it in a later revision',
129
+ table: (m) => m[1],
130
+ column: (m) => m[2],
131
+ },
132
+ {
133
+ find: new RegExp(String.raw `op\.drop_table\(\s*${STR}`, 'i'),
134
+ risk: 'destroys-data',
135
+ what: (m) => `drops the table ${m[1]}`,
136
+ instead: 'stop using it, ship that, then drop it later',
137
+ table: (m) => m[1],
138
+ },
139
+ {
140
+ find: new RegExp(String.raw `op\.alter_column\([\s\S]{0,300}?nullable=False`, 'i'),
141
+ risk: 'locks-writes',
142
+ what: () => 'makes a column NOT NULL, which reads every row under a lock',
143
+ instead: 'backfill, add a validated constraint, then alter the column',
144
+ },
145
+ {
146
+ find: /op\.create_index\(/i,
147
+ unless: /postgresql_concurrently\s*=\s*True/,
148
+ risk: 'locks-writes',
149
+ what: () => 'builds an index without postgresql_concurrently, so writes block while it builds',
150
+ instead: 'op.create_index(..., postgresql_concurrently=True) with a non-transactional revision',
151
+ },
152
+ {
153
+ find: new RegExp(String.raw `op\.rename_table\(\s*${STR}`, 'i'),
154
+ risk: 'breaks-old-code',
155
+ what: (m) => `renames the table ${m[1]}`,
156
+ instead: 'move the readers first, or keep a view with the old name',
157
+ table: (m) => m[1],
158
+ },
159
+ ];
160
+ const BY_DIALECT = { django: DJANGO, rails: RAILS, alembic: ALEMBIC };
161
+ /** Findings from a framework migration, with the line each one sits on. */
162
+ export function readFramework(dialect, text) {
163
+ const lines = text.split(String.fromCharCode(10));
164
+ const out = [];
165
+ for (const operation of BY_DIALECT[dialect]) {
166
+ const hit = operation.find.exec(text);
167
+ if (!hit)
168
+ continue;
169
+ if (operation.unless?.test(text))
170
+ continue;
171
+ // The line the match starts on, so the reader can go straight to it.
172
+ const before = text.slice(0, hit.index ?? 0).split(String.fromCharCode(10)).length;
173
+ out.push({
174
+ statement: (lines[before - 1] ?? hit[0]).trim().slice(0, 300),
175
+ line: before,
176
+ risk: operation.risk,
177
+ what: operation.what(hit),
178
+ instead: operation.instead,
179
+ table: operation.table?.(hit) ?? '',
180
+ column: operation.column?.(hit) ?? '',
181
+ });
182
+ }
183
+ return out;
184
+ }
185
+ /** A migration that cannot be rolled back by running its own down step. */
186
+ export function irreversible(dialect, text) {
187
+ if (dialect === 'django')
188
+ return /migrations\.RunPython\([^)]*\)/.test(text) && !/reverse_code|RunPython\.noop/.test(text);
189
+ if (dialect === 'alembic')
190
+ return /def\s+downgrade\(\)[\s\S]{0,120}?(?:pass|raise\s+NotImplementedError)/.test(text);
191
+ if (dialect === 'rails')
192
+ return /\braise\s+ActiveRecord::IrreversibleMigration\b/.test(text);
193
+ return false;
194
+ }
195
+ /**
196
+ * The forward path only, with the rollback blanked out.
197
+ *
198
+ * A migration's `down` step is what runs if the change is undone, and it necessarily contains the mirror
199
+ * image of the change - a drop_table for a create_table, a remove_column for an add_column. Reading it as
200
+ * part of the change produces exactly the wrong answer, and Mastodon's own migration proved it: a file
201
+ * that creates a table was reported as destroying one, because its rollback drops it.
202
+ *
203
+ * The region is blanked rather than removed so every remaining line keeps its number.
204
+ */
205
+ export function forwardOnly(dialect, text) {
206
+ const lines = text.split(String.fromCharCode(10));
207
+ const blankFrom = (start, until, indent) => {
208
+ for (let i = start; i < lines.length; i += 1) {
209
+ if (i > start && until(lines[i], indent))
210
+ return;
211
+ lines[i] = '';
212
+ }
213
+ };
214
+ const indentOf = (line) => line.length - line.trimStart().length;
215
+ for (let i = 0; i < lines.length; i += 1) {
216
+ const line = lines[i];
217
+ const down = (dialect === 'rails' && /^\s*def\s+down\b/.test(line)) ||
218
+ (dialect === 'alembic' && /^\s*def\s+downgrade\s*\(/.test(line)) ||
219
+ (dialect === 'sql' && /^\s*--\s*(?:\+migrate\s+Down|\+goose\s+Down|\/\/@UNDO|Down\b)/i.test(line));
220
+ if (down) {
221
+ // To the end of the block: the next line at the same indentation or less that starts something new.
222
+ blankFrom(i, (l, indent) => l.trim() !== '' && indentOf(l) <= indent && !/^\s*(end|\})\s*$/.test(l), indentOf(line));
223
+ continue;
224
+ }
225
+ // Rails's `direction.down { ... }` is usually one line.
226
+ if (dialect === 'rails' && /direction\.down\b/.test(line))
227
+ lines[i] = '';
228
+ }
229
+ return lines.join(String.fromCharCode(10));
230
+ }
@@ -0,0 +1,174 @@
1
+ // `tewip schema --diff main...HEAD`: what this migration will do to a live database.
2
+ //
3
+ // The part that no SQL linter does, and the reason this belongs in Tewip rather than in a linter: it
4
+ // looks for the code that still uses what the migration takes away. "This drops users.nickname, and four
5
+ // files still read it" is a different sentence from "dropping a column is risky", and it is the one that
6
+ // stops the outage.
7
+ //
8
+ // That check is a search of the repository, so it is evidence rather than proof, and it is worded as
9
+ // such. A mention in a comment counts as a mention; Tewip says where it looked and lets the reader judge.
10
+ import { spawnSync } from 'node:child_process';
11
+ import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
12
+ import { dirname, resolve } from 'node:path';
13
+ import { changeSummary } from '../quality/grounding.js';
14
+ import { dialectOf, forwardOnly, irreversible, readFramework } from './frameworks.js';
15
+ import { readSql } from './statements.js';
16
+ const MIGRATION = /(^|\/)(migrations?|db\/migrate|prisma\/migrations|alembic|flyway|sql)\//i;
17
+ const SCHEMA_FILE = /\.(sql)$|(^|\/)(migrations?|db\/migrate)\/.*\.(py|rb|js|ts)$/i;
18
+ /** The schema files in this change. */
19
+ export function schemaFiles(cmd) {
20
+ const given = cmd.files && cmd.files.length > 0 ? cmd.files : (changeSummary(cmd.root, cmd.diff)?.files ?? []);
21
+ return given
22
+ .map((f) => f.split('\\').join('/'))
23
+ .filter((f) => (SCHEMA_FILE.test(f) || MIGRATION.test(f)) && existsSync(resolve(cmd.root, f)));
24
+ }
25
+ export function readMigration(root, file) {
26
+ let text = '';
27
+ try {
28
+ text = readFileSync(resolve(root, file), 'utf8');
29
+ }
30
+ catch {
31
+ return { file, dialect: 'unknown', findings: [], unread: 'the file could not be read', irreversible: false };
32
+ }
33
+ const dialect = dialectOf(file, text);
34
+ // Only the forward path: a rollback step contains the mirror image of the change, and reading it as
35
+ // part of the change says a migration that creates a table destroys one.
36
+ const forward = forwardOnly(dialect, text);
37
+ if (dialect === 'sql')
38
+ return { file, dialect, findings: readSql(forward), unread: '', irreversible: false };
39
+ if (dialect === 'unknown') {
40
+ return {
41
+ file,
42
+ dialect,
43
+ findings: [],
44
+ unread: 'Tewip does not recognise how this migration is written, so it says nothing about it rather than passing it. It reads SQL, Django, Rails and Alembic migrations',
45
+ irreversible: false,
46
+ };
47
+ }
48
+ const findings = readFramework(dialect, forward);
49
+ // A migration that only runs raw SQL through the framework is still readable: the SQL is the migration.
50
+ // Raw SQL, however it reaches the database: through the framework, or through a cursor inside a data
51
+ // migration, which is where most large projects put the dangerous parts.
52
+ const raw = [...forward.matchAll(/(?:RunSQL|op\.execute|execute|cursor\.execute)\(\s*(?:r?["'`]{1,3})([\s\S]*?)["'`]{1,3}\s*[,)]/g)].map((m) => m[1]).join(';');
53
+ if (raw.trim())
54
+ findings.push(...readSql(raw));
55
+ return { file, dialect, findings, unread: '', irreversible: irreversible(dialect, text) };
56
+ }
57
+ /**
58
+ * Code outside the migrations that still names a table or column this change takes away.
59
+ *
60
+ * `git grep` because it respects the repository's own ignore rules and is fast on a large tree. A column
61
+ * named `name` or `id` is not searched for: everything matches it, and a finding that always fires teaches
62
+ * a reader to ignore the whole report.
63
+ */
64
+ export function stillUsed(root, findings) {
65
+ const out = [];
66
+ const tooCommon = new Set(['id', 'name', 'type', 'value', 'data', 'key', 'status', 'state', 'date', 'time', 'user', 'count', 'index', 'order', 'group', 'text', 'url', 'size', 'code']);
67
+ for (const finding of findings) {
68
+ if (finding.risk !== 'breaks-old-code' && finding.risk !== 'destroys-data')
69
+ continue;
70
+ const what = finding.column || finding.table;
71
+ if (!what || what.length < 4 || tooCommon.has(what.toLowerCase()))
72
+ continue;
73
+ if (out.some((u) => u.table === finding.table && u.column === finding.column))
74
+ continue;
75
+ let files = grepFor(root, what);
76
+ // A bare column name is too weak on its own. Dropping `fasp_providers.capabilities` from Mastodon
77
+ // matched 37 files including a Dockerfile and an emoji module, because "capabilities" is an ordinary
78
+ // word - and noise in the one section meant to be the differentiator is worse than silence. So when a
79
+ // column is named, the file has to mention the table as well.
80
+ if (finding.column && finding.table) {
81
+ const named = new Set([...grepFor(root, finding.table), ...grepFor(root, singular(finding.table))]);
82
+ files = files.filter((f) => named.has(f));
83
+ }
84
+ if (files.length > 0)
85
+ out.push({ table: finding.table, column: finding.column, files });
86
+ }
87
+ return out;
88
+ }
89
+ /** Files naming this word, excluding migrations and documentation. */
90
+ function grepFor(root, word) {
91
+ if (!word || word.length < 4)
92
+ return [];
93
+ const found = spawnSync('git', ['grep', '-l', '-w', '--', word], { cwd: root, encoding: 'utf8', timeout: 60_000 });
94
+ return (found.stdout ?? '')
95
+ .split(String.fromCharCode(10))
96
+ .map((l) => l.trim().split(String.fromCharCode(92)).join('/'))
97
+ .filter((l) => l && !MIGRATION.test(l) && !/\.(md|txt|lock|json)$/i.test(l));
98
+ }
99
+ /** `fasp_providers` is the table; the code calls the model `FaspProvider`, so the singular is searched too. */
100
+ const singular = (table) => (table.endsWith('ies') ? table.slice(0, -3) + 'y' : table.endsWith('s') && !table.endsWith('ss') ? table.slice(0, -1) : table);
101
+ const ORDER = ['breaks-old-code', 'destroys-data', 'locks-writes', 'slow', 'irreversible', 'fine'];
102
+ const TITLES = {
103
+ 'breaks-old-code': 'Breaks the version that is still running',
104
+ 'destroys-data': 'Destroys data',
105
+ 'locks-writes': 'Blocks writes while it runs',
106
+ slow: 'Holds a long transaction',
107
+ irreversible: 'Cannot be undone',
108
+ fine: 'Worth knowing',
109
+ };
110
+ export function renderSchema(migrations, used) {
111
+ const all = migrations.flatMap((m) => m.findings);
112
+ const serious = all.filter((f) => f.risk === 'breaks-old-code' || f.risk === 'destroys-data');
113
+ const lines = [
114
+ '# What this migration does to a live database',
115
+ '',
116
+ migrations.length === 0
117
+ ? 'No migration or schema file in this change.'
118
+ : `${migrations.filter((m) => !m.unread).length} of ${migrations.length} migration file(s) read. **${serious.length}** change(s) would break the running version or destroy data; ${all.length - serious.length} other finding(s).`,
119
+ '',
120
+ // "Read, and nothing found" must never look like "not read". A reader who cannot tell them apart will
121
+ // take silence for safety, which is the one thing this report must not allow.
122
+ ...(migrations.length > 0 && all.length === 0 && migrations.every((m) => !m.unread)
123
+ ? [`All ${migrations.length} were read, and nothing in them breaks the running version, destroys data, blocks writes or holds a long transaction.`]
124
+ : []),
125
+ '',
126
+ ];
127
+ for (const risk of ORDER) {
128
+ const group = all.filter((f) => f.risk === risk);
129
+ if (group.length === 0)
130
+ continue;
131
+ lines.push(`## ${TITLES[risk]} (${group.length})`, '');
132
+ for (const finding of group) {
133
+ const where = migrations.find((m) => m.findings.includes(finding));
134
+ lines.push(`- **${finding.what}** — \`${where?.file ?? ''}:${finding.line}\``);
135
+ lines.push(` - \`${finding.statement.replace(/`/g, "'")}\``);
136
+ lines.push(` - Instead: ${finding.instead}`);
137
+ }
138
+ lines.push('');
139
+ }
140
+ if (used.length > 0) {
141
+ lines.push(`## Code that still uses what this takes away (${used.length})`, '', 'Found by searching the repository, so it is evidence rather than proof - a match may be a comment or', 'an unrelated name. Each one is worth opening before this ships.', '');
142
+ for (const u of used) {
143
+ lines.push(`- **${u.column ? `${u.table}.${u.column}` : u.table}** is still named in ${u.files.length} file(s): ${u.files.slice(0, 6).join(', ')}${u.files.length > 6 ? `, and ${u.files.length - 6} more` : ''}`);
144
+ }
145
+ lines.push('');
146
+ }
147
+ const irreversibles = migrations.filter((m) => m.irreversible);
148
+ if (irreversibles.length > 0) {
149
+ lines.push(`## Cannot be rolled back by its own down step (${irreversibles.length})`, '', ...irreversibles.map((m) => `- \`${m.file}\``), '');
150
+ }
151
+ const unread = migrations.filter((m) => m.unread);
152
+ if (unread.length > 0) {
153
+ lines.push(`## Not read (${unread.length})`, '', ...unread.map((m) => `- \`${m.file}\`: ${m.unread}`), '');
154
+ }
155
+ lines.push('---', '', 'Every line above is read from the migration itself; no model was called and nothing was guessed.', '', 'What this does not know: how large your tables are, how busy the database is, and therefore how long', 'any of this takes. It says what a statement does - rewrites the table, scans every row, blocks writes -', 'and leaves the duration to the person who knows the data. It also cannot see a deployment order, so', '"breaks the running version" means during a rolling deploy, which is the usual case.');
156
+ return lines.join(String.fromCharCode(10));
157
+ }
158
+ export function runSchema(cmd) {
159
+ const files = schemaFiles(cmd);
160
+ const migrations = files.map((file) => readMigration(cmd.root, file));
161
+ const used = stillUsed(cmd.root, migrations.flatMap((m) => m.findings));
162
+ const report = renderSchema(migrations, used);
163
+ console.log(report);
164
+ if (cmd.out) {
165
+ mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
166
+ writeFileSync(resolve(cmd.out), report + String.fromCharCode(10), 'utf8');
167
+ }
168
+ const serious = migrations.flatMap((m) => m.findings).filter((f) => f.risk === 'breaks-old-code' || f.risk === 'destroys-data').length;
169
+ if (cmd.strict && serious > 0) {
170
+ console.error(String.fromCharCode(10) + `${serious} change(s) here would break the running version or destroy data.`);
171
+ process.exitCode = 1;
172
+ }
173
+ return { serious };
174
+ }
@@ -0,0 +1,164 @@
1
+ const CLEAN = (s) => s.replace(/\s+/g, ' ').trim();
2
+ const NAME = String.raw `[\w."\[\]` + "`" + String.raw `]+`;
3
+ /**
4
+ * What a statement does. Ordered: the first rule that matches decides, so the most serious reading of a
5
+ * statement wins over a milder one.
6
+ */
7
+ const RULES = [
8
+ {
9
+ find: new RegExp(String.raw `\bDROP\s+TABLE\s+(?:IF\s+EXISTS\s+)?(${NAME})`, 'i'),
10
+ risk: 'destroys-data',
11
+ what: (m) => `drops the table ${bare(m[1])} and everything in it`,
12
+ instead: 'stop writing to it, ship that, confirm nothing reads it, and drop it in a later release - a drop cannot be rolled back by deploying the old code again',
13
+ table: (m) => bare(m[1]),
14
+ },
15
+ {
16
+ find: new RegExp(String.raw `\bTRUNCATE\s+(?:TABLE\s+)?(${NAME})`, 'i'),
17
+ risk: 'destroys-data',
18
+ what: (m) => `empties the table ${bare(m[1])}`,
19
+ instead: 'delete in batches if this is cleanup, or keep the rows and stop reading them',
20
+ table: (m) => bare(m[1]),
21
+ },
22
+ {
23
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})[\s\S]*?\bDROP\s+(?:COLUMN\s+)?(?:IF\s+EXISTS\s+)?(${NAME})`, 'i'),
24
+ risk: 'breaks-old-code',
25
+ what: (m) => `drops the column ${bare(m[2])} from ${bare(m[1])}`,
26
+ instead: 'deploy code that no longer selects or writes the column, then drop it in a later migration. During a rolling deploy the old version is still running and will fail on a column that is gone',
27
+ table: (m) => bare(m[1]),
28
+ column: (m) => bare(m[2]),
29
+ },
30
+ {
31
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+RENAME\s+(?:COLUMN\s+)?(${NAME})\s+TO\s+(${NAME})`, 'i'),
32
+ risk: 'breaks-old-code',
33
+ what: (m) => `renames ${bare(m[1])}.${bare(m[2])} to ${bare(m[3])}`,
34
+ instead: 'add the new column, write to both, move readers over, then remove the old one. A rename is a drop and an add at the same instant, and no deployment is that instant',
35
+ table: (m) => bare(m[1]),
36
+ column: (m) => bare(m[2]),
37
+ },
38
+ {
39
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+RENAME\s+TO\s+(${NAME})`, 'i'),
40
+ risk: 'breaks-old-code',
41
+ what: (m) => `renames the table ${bare(m[1])} to ${bare(m[2])}`,
42
+ instead: 'create a view with the old name, or move readers first. Every query naming the old table fails the moment this runs',
43
+ table: (m) => bare(m[1]),
44
+ },
45
+ {
46
+ find: new RegExp(String.raw `\bCREATE\s+(?:UNIQUE\s+)?INDEX\s+(?!CONCURRENTLY)(?:IF\s+NOT\s+EXISTS\s+)?(${NAME})?\s*ON\s+(${NAME})`, 'i'),
47
+ risk: 'locks-writes',
48
+ what: (m) => `builds an index on ${bare(m[2])} while holding a lock that blocks writes to it`,
49
+ instead: 'CREATE INDEX CONCURRENTLY (Postgres), or ALGORITHM=INPLACE LOCK=NONE (MySQL). On a large table the plain form blocks every write until it finishes',
50
+ table: (m) => bare(m[2]),
51
+ },
52
+ {
53
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+(?:ALTER|MODIFY)\s+(?:COLUMN\s+)?(${NAME})[\s\S]*?\bSET\s+NOT\s+NULL`, 'i'),
54
+ risk: 'locks-writes',
55
+ what: (m) => `makes ${bare(m[1])}.${bare(m[2])} NOT NULL, which reads every existing row under a lock`,
56
+ instead: 'add a CHECK (column IS NOT NULL) NOT VALID, VALIDATE CONSTRAINT separately, then set NOT NULL (Postgres 12+). The direct form scans the whole table while blocking writes',
57
+ table: (m) => bare(m[1]),
58
+ column: (m) => bare(m[2]),
59
+ },
60
+ {
61
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+(?:ALTER|MODIFY|CHANGE)\s+(?:COLUMN\s+)?(${NAME})[\s\S]*?\b(?:TYPE|SET\s+DATA\s+TYPE)\b`, 'i'),
62
+ risk: 'locks-writes',
63
+ what: (m) => `changes the type of ${bare(m[1])}.${bare(m[2])}, which usually rewrites the whole table`,
64
+ instead: 'add a new column of the new type, backfill in batches, switch the readers, then drop the old one. A type change holds an exclusive lock for the rewrite',
65
+ table: (m) => bare(m[1]),
66
+ column: (m) => bare(m[2]),
67
+ },
68
+ {
69
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+ADD\s+(?:CONSTRAINT\s+${NAME}\s+)?FOREIGN\s+KEY(?![\s\S]*NOT\s+VALID)`, 'i'),
70
+ risk: 'locks-writes',
71
+ what: (m) => `adds a foreign key to ${bare(m[1])}, which checks every existing row under a lock on both tables`,
72
+ instead: 'ADD CONSTRAINT ... NOT VALID, then VALIDATE CONSTRAINT, which takes a weaker lock',
73
+ table: (m) => bare(m[1]),
74
+ },
75
+ {
76
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+ADD\s+(?:COLUMN\s+)?(${NAME})[\s\S]*?\bNOT\s+NULL\b(?![\s\S]*\bDEFAULT\b)`, 'i'),
77
+ risk: 'breaks-old-code',
78
+ what: (m) => `adds ${bare(m[2])} to ${bare(m[1])} as NOT NULL with no default`,
79
+ instead: 'add it nullable or with a default. Without one, every insert from code that does not know about the column fails - including the version still running during the deploy',
80
+ table: (m) => bare(m[1]),
81
+ column: (m) => bare(m[2]),
82
+ },
83
+ {
84
+ find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+ADD\s+(?:CONSTRAINT\s+${NAME}\s+)?UNIQUE`, 'i'),
85
+ risk: 'locks-writes',
86
+ what: (m) => `adds a unique constraint to ${bare(m[1])}, which builds an index under a lock`,
87
+ instead: 'CREATE UNIQUE INDEX CONCURRENTLY first, then add the constraint using that index',
88
+ table: (m) => bare(m[1]),
89
+ },
90
+ {
91
+ find: /\bVACUUM\s+FULL\b|\bCLUSTER\b|\bREINDEX\b(?!\s+CONCURRENTLY)/i,
92
+ risk: 'locks-writes',
93
+ what: () => 'rewrites the table or index while holding an exclusive lock on it',
94
+ instead: 'run it in a maintenance window, or use the CONCURRENTLY form where the database offers one',
95
+ },
96
+ {
97
+ find: new RegExp(String.raw `\bUPDATE\s+(${NAME})\s+SET\b(?![\s\S]*\bWHERE\b)`, 'i'),
98
+ risk: 'slow',
99
+ what: (m) => `updates every row of ${bare(m[1])} in one statement`,
100
+ instead: 'update in batches with a WHERE on the primary key. One statement over a large table holds its locks and its transaction open until it finishes',
101
+ table: (m) => bare(m[1]),
102
+ },
103
+ {
104
+ find: new RegExp(String.raw `\bDELETE\s+FROM\s+(${NAME})\b(?![\s\S]*\bWHERE\b)`, 'i'),
105
+ risk: 'destroys-data',
106
+ what: (m) => `deletes every row of ${bare(m[1])}`,
107
+ instead: 'say what should go with a WHERE, or keep the rows and stop reading them',
108
+ table: (m) => bare(m[1]),
109
+ },
110
+ ];
111
+ /** `"users"` and `` `users` `` and `[users]` are all the table users. */
112
+ const bare = (name) => name.replace(/["`[\]]/g, '').trim();
113
+ /** The statements of a SQL file, with the line each one starts on and comments removed. */
114
+ export function statementsOf(sql) {
115
+ // Comments are blanked rather than removed, so every remaining character keeps its original position
116
+ // and a statement can be reported on the line a reader will actually find it. Deleting them instead
117
+ // shifted everything upwards: a statement on line 3 was reported as line 1.
118
+ const blanked = sql
119
+ .split(String.fromCharCode(10))
120
+ .map((line) => line.replace(/--.*$/, (m) => ' '.repeat(m.length)))
121
+ .join(String.fromCharCode(10))
122
+ // A block comment keeps its newlines so later lines are still numbered correctly.
123
+ .replace(/\/\*[\s\S]*?\*\//g, (m) => m.split(String.fromCharCode(10)).map((l) => ' '.repeat(l.length)).join(String.fromCharCode(10)));
124
+ const lineOf = (index) => blanked.slice(0, index).split(String.fromCharCode(10)).length;
125
+ const out = [];
126
+ let start = 0;
127
+ for (let i = 0; i <= blanked.length; i += 1) {
128
+ if (i < blanked.length && blanked[i] !== ';')
129
+ continue;
130
+ const segment = blanked.slice(start, i);
131
+ const text = CLEAN(segment);
132
+ if (text) {
133
+ // Where the statement itself begins, not where the segment did.
134
+ const offset = segment.length - segment.replace(/^\s+/, '').length;
135
+ out.push({ text, line: lineOf(start + offset) });
136
+ }
137
+ start = i + 1;
138
+ }
139
+ return out;
140
+ }
141
+ /** What this one statement does, or nothing when it is not a statement worth a word. */
142
+ export function judge(statement, line) {
143
+ for (const rule of RULES) {
144
+ const hit = rule.find.exec(statement);
145
+ if (!hit)
146
+ continue;
147
+ return {
148
+ statement: statement.slice(0, 300),
149
+ line,
150
+ risk: rule.risk,
151
+ what: rule.what(hit),
152
+ instead: rule.instead,
153
+ table: rule.table?.(hit) ?? '',
154
+ column: rule.column?.(hit) ?? '',
155
+ };
156
+ }
157
+ return null;
158
+ }
159
+ /** Every finding in a SQL migration. */
160
+ export function readSql(sql) {
161
+ return statementsOf(sql)
162
+ .map(({ text, line }) => judge(text, line))
163
+ .filter((f) => f !== null);
164
+ }
@@ -197,4 +197,23 @@ server.registerTool('which_tests_to_run', {
197
197
  return `Nothing changed in ${range}, or that range could not be read here.`;
198
198
  return renderPick(chosen.picked, chosen.total, chosen.command, chosen.noHistory);
199
199
  }));
200
+ server.registerTool('what_will_this_migration_do', {
201
+ title: 'What will this migration do to a live database?',
202
+ description: "Reads the migrations in a change and names what will happen when they run against a live database: a column dropped while the version still running keeps selecting it, a table dropped, an index built under a lock that blocks writes, a NOT NULL added to a populated table, a type change that rewrites it, a foreign key validated against every row, a data migration holding a transaction open, a step that cannot be undone. It also searches the repository for code that still uses the column being taken away. Reads SQL, Django, Rails and Alembic, judges only the forward path, and says plainly when it cannot read a migration. It does not know your table sizes, so it says what a statement does rather than how long it will take.",
203
+ inputSchema: {
204
+ project,
205
+ diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
206
+ files: z.array(z.string()).optional().describe('Migration files to read, when they do not come from a git range'),
207
+ },
208
+ annotations: readOnly,
209
+ }, ({ project: root, diff, files }) => answer(async () => {
210
+ if (!isAbsolute(root) || !existsSync(root))
211
+ throw new Error(`project must be an existing absolute path, got "${root}"`);
212
+ const { readMigration, renderSchema, schemaFiles, stillUsed } = await import('../engine/schema/schema-command.js');
213
+ const cmd = { root, diff: diff ?? 'main...HEAD', files };
214
+ const migrations = schemaFiles(cmd).map((file) => readMigration(root, file));
215
+ if (migrations.length === 0)
216
+ return 'No migration or schema file in that change.';
217
+ return renderSchema(migrations, stillUsed(root, migrations.flatMap((m) => m.findings)));
218
+ }));
200
219
  await server.connect(new StdioServerTransport());
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tewip-ai",
3
- "version": "0.30.0",
3
+ "version": "0.31.0",
4
4
  "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
5
5
  "keywords": [
6
6
  "test-failures",