tewip-ai 0.41.0 → 0.43.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -870,7 +870,7 @@ That last section is the part no SQL linter does: it searches your own code for
870
870
  away. "This drops `users.nickname`, and four files still read it" is a different sentence from "dropping a
871
871
  column is risky", and it is the one that stops the outage.
872
872
 
873
- It reads **SQL, Django, Rails, Alembic and TypeORM** migrations — which covers Prisma, Drizzle, Flyway
873
+ It reads **SQL, Django, Rails, Alembic, TypeORM, Sequelize, Knex, EF Core and Liquibase** migrations — which covers Prisma, Drizzle, Flyway
874
874
  and golang-migrate too, since those all emit plain `.sql`. It judges **only the forward path** (a rollback step
875
875
  contains the mirror image of the change, so reading it says a migration that creates a table destroys
876
876
  one), and stays quiet when the team already did the safe thing — `atomic = False`, `algorithm:
@@ -995,6 +995,9 @@ tewip vouch tests/ # or a file, or the whole project
995
995
  tewip vouch tests/ --strict # exits non-zero, for a merge gate
996
996
  ```
997
997
 
998
+ Reads **JavaScript, TypeScript, Python, Java, Kotlin, Scala, Go, C#, Ruby, PHP and Rust** — and says
999
+ so, rather than clearing a file it cannot read.
1000
+
998
1001
  Most tests in a pull request are now written by an agent, and the question that decides whether they
999
1002
  are worth anything goes unasked: **could this test ever fail?** Coverage counts a test that checks
1000
1003
  nothing. CI goes green on it. A reviewer skims something plausible and approves it.
package/dist/cli/help.js CHANGED
@@ -157,9 +157,11 @@ Usage:
157
157
  What will this migration do to a live database? Names the changes
158
158
  that break the version still running during a deploy, destroy data,
159
159
  block writes while they run, or cannot be undone - and searches your
160
- code for what still uses the column it takes away. Reads SQL, Django,
161
- Rails and Alembic migrations, and says so when it cannot read one.
162
- No model, no cost.
160
+ code for what still uses the column it takes away. Reads SQL,
161
+ Django, Rails, Alembic, TypeORM, Sequelize, Knex, EF Core and
162
+ Liquibase - so Prisma, Drizzle, Flyway and golang-migrate too,
163
+ since those emit plain SQL - and says so when it cannot read one.
164
+ It judges only the forward path, never the rollback. No cost.
163
165
  tewip pick [--diff main...HEAD] [--list] [--out picked.md]
164
166
  Which tests could notice this change? Narrows a test run using
165
167
  facts rather than a prediction: a test runs when it imports a
@@ -183,8 +185,9 @@ Usage:
183
185
  that check nothing, compare a value with itself, leave an
184
186
  expectation unfinished, swallow their own failure, or never run.
185
187
  Where the checking is in a helper it cannot follow it says so
186
- instead of accusing. --strict exits non-zero on a finding, for a
187
- gate. No model, no cost.
188
+ instead of accusing. Reads JavaScript, TypeScript, Python, Java,
189
+ Kotlin, Scala, Go, C#, Ruby, PHP and Rust. --strict exits
190
+ non-zero on a finding, for a gate. No model, no cost.
188
191
  tewip trace [--only "<ticket or title>"] [--out trace.md]
189
192
  Is this requirement covered, and is it passing right now? Joins
190
193
  the plans Tewip wrote to the tests that cover them and the runs
@@ -51,7 +51,7 @@ export const ANSWERABLE = [
51
51
  id: 'migration-safety',
52
52
  verb: 'schema',
53
53
  name: 'What will this migration do to a live database?',
54
- description: 'Names what breaks the version still running during a rolling deploy, destroys data, blocks writes, or cannot be undone - and searches the repository for code that still uses a column being dropped. Reads SQL, Django, Rails, Alembic and TypeORM, which covers Prisma, Drizzle and Flyway.',
54
+ description: 'Names what breaks the version still running during a rolling deploy, destroys data, blocks writes, or cannot be undone - and searches the repository for code that still uses a column being dropped. Reads SQL, Django, Rails, Alembic, TypeORM, Sequelize, Knex, EF Core and Liquibase, which covers Prisma, Drizzle and Flyway too.',
55
55
  tags: ['database', 'migrations', 'deployment', 'risk'],
56
56
  examples: ['Is this migration safe to deploy?', 'Will this drop break the running version?'],
57
57
  },
@@ -0,0 +1,190 @@
1
+ const STR = String.raw `['"]([\w.]+)['"]`;
2
+ /** Alembic. */
3
+ /**
4
+ * Sequelize. `queryInterface.removeColumn('Users', 'passwordHash')` names the table and the column
5
+ * in one call, which makes it the simplest of these to read.
6
+ *
7
+ * Written against opencollective-api's real migrations, which also show why the forward path
8
+ * matters more here than anywhere else: a Sequelize migration almost always *adds* in `up` and
9
+ * *removes* in `down`, so a reader that took the whole file would report every harmless migration
10
+ * in the ecosystem as dropping a column.
11
+ */
12
+ const SEQUELIZE = [
13
+ {
14
+ find: new RegExp(String.raw `removeColumn\s*\(\s*${STR}\s*,\s*${STR}`, 'i'),
15
+ risk: 'breaks-old-code',
16
+ what: (m) => `removes ${m[1]}.${m[2]}`,
17
+ instead: 'deploy code that no longer selects or writes the column, then remove it in a later migration',
18
+ table: (m) => m[1],
19
+ column: (m) => m[2],
20
+ },
21
+ {
22
+ find: new RegExp(String.raw `renameColumn\s*\(\s*${STR}\s*,\s*${STR}`, 'i'),
23
+ risk: 'breaks-old-code',
24
+ what: (m) => `renames ${m[1]}.${m[2]}`,
25
+ instead: 'add the new column, write to both, move the readers, then remove the old one',
26
+ table: (m) => m[1],
27
+ column: (m) => m[2],
28
+ },
29
+ {
30
+ find: new RegExp(String.raw `dropTable\s*\(\s*${STR}`, 'i'),
31
+ risk: 'destroys-data',
32
+ what: (m) => `drops the table ${m[1]}`,
33
+ instead: 'stop reading the table, deploy that, then drop it once nothing refers to it',
34
+ table: (m) => m[1],
35
+ },
36
+ {
37
+ // Sequelize has no CONCURRENTLY option, so an index on a populated table takes the lock.
38
+ find: new RegExp(String.raw `addIndex\s*\(\s*${STR}`, 'i'),
39
+ unless: /concurrently\s*:\s*true/i,
40
+ risk: 'locks-writes',
41
+ what: (m) => `builds an index on ${m[1]} while holding a lock that blocks writes to it`,
42
+ instead: 'build it concurrently with a raw query, or accept the lock if the table is small',
43
+ table: (m) => m[1],
44
+ },
45
+ {
46
+ find: new RegExp(String.raw `changeColumn\s*\(\s*${STR}\s*,\s*${STR}`, 'i'),
47
+ risk: 'locks-writes',
48
+ what: (m) => `changes the type of ${m[1]}.${m[2]}, which can rewrite the whole table`,
49
+ instead: 'add a new column, backfill it in batches, and switch the readers over',
50
+ table: (m) => m[1],
51
+ column: (m) => m[2],
52
+ },
53
+ ];
54
+ /**
55
+ * Knex. The awkward one: the table is named by `knex.schema.table('articles', ...)` and the
56
+ * operation happens inside the callback, so the two are matched together across the lines between
57
+ * them. Matterwiki's real migrations are exactly this shape.
58
+ */
59
+ const KNEX = [
60
+ {
61
+ find: new RegExp(String.raw `\.(?:table|alterTable)\s*\(\s*${STR}[\s\S]{0,400}?\.dropColumn\s*\(\s*${STR}`, 'i'),
62
+ risk: 'breaks-old-code',
63
+ what: (m) => `drops ${m[1]}.${m[2]}`,
64
+ instead: 'deploy code that no longer selects or writes the column, then drop it in a later migration',
65
+ table: (m) => m[1],
66
+ column: (m) => m[2],
67
+ },
68
+ {
69
+ find: new RegExp(String.raw `\.(?:table|alterTable)\s*\(\s*${STR}[\s\S]{0,400}?\.renameColumn\s*\(\s*${STR}`, 'i'),
70
+ risk: 'breaks-old-code',
71
+ what: (m) => `renames ${m[1]}.${m[2]}`,
72
+ instead: 'add the new column, write to both, move the readers, then remove the old one',
73
+ table: (m) => m[1],
74
+ column: (m) => m[2],
75
+ },
76
+ {
77
+ find: new RegExp(String.raw `\.dropTable(?:IfExists)?\s*\(\s*${STR}`, 'i'),
78
+ risk: 'destroys-data',
79
+ what: (m) => `drops the table ${m[1]}`,
80
+ instead: 'stop reading the table, deploy that, then drop it once nothing refers to it',
81
+ table: (m) => m[1],
82
+ },
83
+ {
84
+ find: new RegExp(String.raw `\.(?:table|alterTable)\s*\(\s*${STR}[\s\S]{0,400}?\.notNullable\s*\(`, 'i'),
85
+ risk: 'locks-writes',
86
+ what: (m) => `makes a column on ${m[1]} NOT NULL, which scans every row under a lock`,
87
+ instead: 'add a CHECK ... NOT VALID, validate it, then set NOT NULL - or accept the scan if the table is small',
88
+ table: (m) => m[1],
89
+ },
90
+ ];
91
+ /**
92
+ * EF Core. Its MigrationBuilder calls use named arguments over several lines and in either order,
93
+ * so each operation is matched twice - once for `name` before `table`, once for the reverse -
94
+ * rather than assuming the order Coflnet's real migrations happen to use.
95
+ */
96
+ const NAMED = (first, second) => String.raw `(?:${first}:\s*${STR}[\s\S]{0,200}?${second}:\s*${STR}|${second}:\s*${STR}[\s\S]{0,200}?${first}:\s*${STR})`;
97
+ /** Whichever order the arguments came in, the two names are in 1,2 or in 4,3. */
98
+ const pair = (m) => (m[1] !== undefined ? [m[1], m[2]] : [m[4], m[3]]);
99
+ const EFCORE = [
100
+ {
101
+ find: new RegExp(String.raw `migrationBuilder\.DropColumn\s*\(\s*${NAMED('name', 'table')}`, 'i'),
102
+ risk: 'breaks-old-code',
103
+ what: (m) => `drops ${pair(m)[1]}.${pair(m)[0]}`,
104
+ instead: 'deploy code that no longer selects or writes the column, then drop it in a later migration',
105
+ table: (m) => pair(m)[1],
106
+ column: (m) => pair(m)[0],
107
+ },
108
+ {
109
+ find: new RegExp(String.raw `migrationBuilder\.DropTable\s*\(\s*name:\s*${STR}`, 'i'),
110
+ risk: 'destroys-data',
111
+ what: (m) => `drops the table ${m[1]}`,
112
+ instead: 'stop reading the table, deploy that, then drop it once nothing refers to it',
113
+ table: (m) => m[1],
114
+ },
115
+ {
116
+ find: new RegExp(String.raw `migrationBuilder\.RenameColumn\s*\(\s*${NAMED('name', 'table')}`, 'i'),
117
+ risk: 'breaks-old-code',
118
+ what: (m) => `renames ${pair(m)[1]}.${pair(m)[0]}`,
119
+ instead: 'add the new column, write to both, move the readers, then remove the old one',
120
+ table: (m) => pair(m)[1],
121
+ column: (m) => pair(m)[0],
122
+ },
123
+ {
124
+ find: new RegExp(String.raw `migrationBuilder\.CreateIndex\s*\(\s*${NAMED('name', 'table')}`, 'i'),
125
+ risk: 'locks-writes',
126
+ what: (m) => `builds an index on ${pair(m)[1]} while holding a lock that blocks writes to it`,
127
+ instead: 'build it concurrently with migrationBuilder.Sql, or accept the lock if the table is small',
128
+ table: (m) => pair(m)[1],
129
+ },
130
+ {
131
+ find: new RegExp(String.raw `migrationBuilder\.AlterColumn<[^>]*>\s*\(\s*${NAMED('name', 'table')}`, 'i'),
132
+ risk: 'locks-writes',
133
+ what: (m) => `alters ${pair(m)[1]}.${pair(m)[0]}, which can rewrite the whole table`,
134
+ instead: 'add a new column, backfill it in batches, and switch the readers over',
135
+ table: (m) => pair(m)[1],
136
+ column: (m) => pair(m)[0],
137
+ },
138
+ ];
139
+ /**
140
+ * Liquibase. A changelog is XML or YAML rather than code, and the operation is the element name
141
+ * with the table on `tableName`. DEFRA's real changelogs are the shape this was written against.
142
+ */
143
+ const XML_TABLE = String.raw `tableName\s*=\s*["']([\w.]+)["']`;
144
+ const LIQUIBASE = [
145
+ {
146
+ find: new RegExp(String.raw `<dropColumn[^>]*${XML_TABLE}[\s\S]{0,300}?<column[^>]*name\s*=\s*["']([\w.]+)["']`, 'i'),
147
+ risk: 'breaks-old-code',
148
+ what: (m) => `drops ${m[1]}.${m[2]}`,
149
+ instead: 'deploy code that no longer selects or writes the column, then drop it in a later changeSet',
150
+ table: (m) => m[1],
151
+ column: (m) => m[2],
152
+ },
153
+ {
154
+ find: new RegExp(String.raw `<dropColumn[^>]*columnName\s*=\s*["']([\w.]+)["'][^>]*${XML_TABLE}`, 'i'),
155
+ risk: 'breaks-old-code',
156
+ what: (m) => `drops ${m[2]}.${m[1]}`,
157
+ instead: 'deploy code that no longer selects or writes the column, then drop it in a later changeSet',
158
+ table: (m) => m[2],
159
+ column: (m) => m[1],
160
+ },
161
+ {
162
+ find: new RegExp(String.raw `<dropTable[^>]*${XML_TABLE}`, 'i'),
163
+ risk: 'destroys-data',
164
+ what: (m) => `drops the table ${m[1]}`,
165
+ instead: 'stop reading the table, deploy that, then drop it once nothing refers to it',
166
+ table: (m) => m[1],
167
+ },
168
+ {
169
+ find: new RegExp(String.raw `<renameColumn[^>]*${XML_TABLE}`, 'i'),
170
+ risk: 'breaks-old-code',
171
+ what: (m) => `renames a column on ${m[1]}`,
172
+ instead: 'add the new column, write to both, move the readers, then remove the old one',
173
+ table: (m) => m[1],
174
+ },
175
+ {
176
+ find: new RegExp(String.raw `<addNotNullConstraint[^>]*${XML_TABLE}`, 'i'),
177
+ risk: 'locks-writes',
178
+ what: (m) => `makes a column on ${m[1]} NOT NULL, which scans every row under a lock`,
179
+ instead: 'add a CHECK ... NOT VALID, validate it, then set NOT NULL - or accept the scan if the table is small',
180
+ table: (m) => m[1],
181
+ },
182
+ {
183
+ find: new RegExp(String.raw `<createIndex[^>]*${XML_TABLE}`, 'i'),
184
+ risk: 'locks-writes',
185
+ what: (m) => `builds an index on ${m[1]} while holding a lock that blocks writes to it`,
186
+ instead: 'build it concurrently with a <sql> changeSet, or accept the lock if the table is small',
187
+ table: (m) => m[1],
188
+ },
189
+ ];
190
+ export { SEQUELIZE, KNEX, EFCORE, LIQUIBASE };
@@ -0,0 +1,147 @@
1
+ const STR = String.raw `['"]([\w.]+)['"]`;
2
+ /** Django's operations. `model_name` and `name` are the table and the column. */
3
+ const DJANGO = [
4
+ {
5
+ find: new RegExp(String.raw `migrations\.RemoveField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?name=${STR}`, 'i'),
6
+ risk: 'breaks-old-code',
7
+ what: (m) => `removes the field ${m[2]} from ${m[1]}`,
8
+ instead: 'stop reading and writing the field, deploy that, then remove it in a later migration. The version still running during a rolling deploy will select a column that no longer exists',
9
+ table: (m) => m[1],
10
+ column: (m) => m[2],
11
+ },
12
+ {
13
+ find: new RegExp(String.raw `migrations\.RenameField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?old_name=${STR}`, 'i'),
14
+ risk: 'breaks-old-code',
15
+ what: (m) => `renames ${m[1]}.${m[2]}`,
16
+ instead: 'add the new field, write to both, move the readers, then remove the old one',
17
+ table: (m) => m[1],
18
+ column: (m) => m[2],
19
+ },
20
+ {
21
+ find: new RegExp(String.raw `migrations\.DeleteModel\([\s\S]{0,120}?${STR}`, 'i'),
22
+ risk: 'destroys-data',
23
+ what: (m) => `deletes the model ${m[1]} and its table`,
24
+ instead: 'stop using it, ship that, confirm nothing reads it, and delete it later',
25
+ table: (m) => m[1],
26
+ },
27
+ {
28
+ find: new RegExp(String.raw `migrations\.AddField\([\s\S]{0,400}?null=False(?![\s\S]{0,400}?default=)`, 'i'),
29
+ risk: 'breaks-old-code',
30
+ what: () => 'adds a field that cannot be null and has no default',
31
+ instead: 'add it with null=True or a default. Without one, Django rewrites the table and every insert from the running version fails',
32
+ },
33
+ {
34
+ find: new RegExp(String.raw `migrations\.AlterField\([\s\S]{0,400}?null=False`, 'i'),
35
+ risk: 'locks-writes',
36
+ what: () => 'makes an existing field NOT NULL, which reads every row under a lock',
37
+ instead: 'backfill first, add a validated constraint, then alter - or use SeparateDatabaseAndState with the safe SQL',
38
+ },
39
+ {
40
+ // A data migration. Django wraps a migration in one transaction, so this holds it open for as long as
41
+ // the work takes - fine for a hundred rows, an outage for a hundred million.
42
+ find: /migrations\.RunPython\(/,
43
+ unless: /atomic\s*=\s*False/,
44
+ risk: 'slow',
45
+ what: () => 'runs application code against the data inside the migration transaction',
46
+ instead: 'for a large table, work in batches with atomic = False on the migration and make it resumable. For a small one this is fine - Tewip cannot see how many rows there are',
47
+ },
48
+ {
49
+ find: /AddIndex\(|AddConstraint\(/i,
50
+ unless: /AddIndexConcurrently|AddConstraintNotValid|atomic\s*=\s*False/,
51
+ risk: 'locks-writes',
52
+ what: () => 'adds an index or constraint, which Django builds with a lock unless told otherwise',
53
+ instead: 'AddIndexConcurrently / AddConstraintNotValid from django.contrib.postgres.operations, with atomic = False on the migration',
54
+ },
55
+ ];
56
+ /** Rails migrations. */
57
+ const RAILS = [
58
+ {
59
+ find: new RegExp(String.raw `remove_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
60
+ risk: 'breaks-old-code',
61
+ what: (m) => `removes ${m[1]}.${m[2]}`,
62
+ instead: "ignore the column in the model (self.ignored_columns), deploy that, then remove it. Rails caches the column list, so the running version keeps selecting it",
63
+ table: (m) => m[1],
64
+ column: (m) => m[2],
65
+ },
66
+ {
67
+ find: new RegExp(String.raw `rename_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
68
+ risk: 'breaks-old-code',
69
+ what: (m) => `renames ${m[1]}.${m[2]}`,
70
+ instead: 'add the new column, write to both, move the readers, then remove the old one',
71
+ table: (m) => m[1],
72
+ column: (m) => m[2],
73
+ },
74
+ {
75
+ find: new RegExp(String.raw `drop_table\s*[\s(]\s*:?(\w+)`, 'i'),
76
+ risk: 'destroys-data',
77
+ what: (m) => `drops the table ${m[1]}`,
78
+ instead: 'stop using it, ship that, then drop it later',
79
+ table: (m) => m[1],
80
+ },
81
+ {
82
+ find: /add_index\s*[\s(]/i,
83
+ unless: /algorithm:\s*:concurrently/,
84
+ risk: 'locks-writes',
85
+ what: () => 'adds an index without algorithm: :concurrently, so writes to the table block while it builds',
86
+ instead: 'add_index ..., algorithm: :concurrently, with disable_ddl_transaction! on the migration',
87
+ },
88
+ {
89
+ find: new RegExp(String.raw `change_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
90
+ risk: 'locks-writes',
91
+ what: (m) => `changes the type of ${m[1]}.${m[2]}, which usually rewrites the table`,
92
+ instead: 'add a new column, backfill in batches, switch the readers, then drop the old one',
93
+ table: (m) => m[1],
94
+ column: (m) => m[2],
95
+ },
96
+ {
97
+ find: new RegExp(String.raw `change_column_null\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)\s*,\s*false`, 'i'),
98
+ risk: 'locks-writes',
99
+ what: (m) => `makes ${m[1]}.${m[2]} NOT NULL, which reads every row under a lock`,
100
+ instead: 'add a validated check constraint first, then set the column NOT NULL',
101
+ table: (m) => m[1],
102
+ column: (m) => m[2],
103
+ },
104
+ ];
105
+ const ALEMBIC = [
106
+ {
107
+ find: new RegExp(String.raw `op\.drop_column\(\s*${STR}\s*,\s*${STR}`, 'i'),
108
+ risk: 'breaks-old-code',
109
+ what: (m) => `drops ${m[1]}.${m[2]}`,
110
+ instead: 'stop reading and writing it, deploy that, then drop it in a later revision',
111
+ table: (m) => m[1],
112
+ column: (m) => m[2],
113
+ },
114
+ {
115
+ find: new RegExp(String.raw `op\.drop_table\(\s*${STR}`, 'i'),
116
+ risk: 'destroys-data',
117
+ what: (m) => `drops the table ${m[1]}`,
118
+ instead: 'stop using it, ship that, then drop it later',
119
+ table: (m) => m[1],
120
+ },
121
+ {
122
+ find: new RegExp(String.raw `op\.alter_column\([\s\S]{0,300}?nullable=False`, 'i'),
123
+ risk: 'locks-writes',
124
+ what: () => 'makes a column NOT NULL, which reads every row under a lock',
125
+ instead: 'backfill, add a validated constraint, then alter the column',
126
+ },
127
+ {
128
+ find: /op\.create_index\(/i,
129
+ unless: /postgresql_concurrently\s*=\s*True/,
130
+ risk: 'locks-writes',
131
+ what: () => 'builds an index without postgresql_concurrently, so writes block while it builds',
132
+ instead: 'op.create_index(..., postgresql_concurrently=True) with a non-transactional revision',
133
+ },
134
+ {
135
+ find: new RegExp(String.raw `op\.rename_table\(\s*${STR}`, 'i'),
136
+ risk: 'breaks-old-code',
137
+ what: (m) => `renames the table ${m[1]}`,
138
+ instead: 'move the readers first, or keep a view with the old name',
139
+ table: (m) => m[1],
140
+ },
141
+ ];
142
+ // TypeORM has no operations of its own: unlike Django's `RemoveField` or Rails's `remove_column`,
143
+ // a TypeORM migration contains SQL and nothing else, handed to queryRunner.query(). So its list is
144
+ // empty on purpose and every finding comes from reading that SQL, which is the same reader the
145
+ // `.sql` dialect uses. The exhaustive Record is what made this a compile error rather than a
146
+ // TypeORM migration silently producing no findings at all.
147
+ export { DJANGO, RAILS, ALEMBIC };
@@ -1,3 +1,5 @@
1
+ import { ALEMBIC, DJANGO, RAILS } from './dialect-rules.js';
2
+ import { EFCORE, KNEX, LIQUIBASE, SEQUELIZE } from './dialect-rules-node.js';
1
3
  export function dialectOf(file, text) {
2
4
  const path = file.split('\\').join('/').toLowerCase();
3
5
  if (path.endsWith('.sql'))
@@ -17,158 +19,28 @@ export function dialectOf(file, text) {
17
19
  // at all - and on a Node backend that is most of them.
18
20
  if (/\bMigrationInterface\b|\bqueryRunner\.query\s*\(/.test(text))
19
21
  return 'typeorm';
22
+ if (/\bqueryInterface\.\w+\s*\(/.test(text))
23
+ return 'sequelize';
24
+ if (/\bknex\.schema\b|\bexports\.(?:up|down)\s*=\s*(?:async\s*)?function\s*\(\s*knex\b/.test(text))
25
+ return 'knex';
26
+ if (/\bmigrationBuilder\.\w+|\b:\s*Migration\b[\s\S]*\bMigrationBuilder\b/.test(text))
27
+ return 'efcore';
28
+ if (/<databaseChangeLog\b|\bdatabaseChangeLog\s*:/.test(text))
29
+ return 'liquibase';
20
30
  if (/(^|\/)(migrations?|db\/migrate)\//.test(path))
21
31
  return 'unknown';
22
32
  return 'unknown';
23
33
  }
24
- const STR = String.raw `['"]([\w.]+)['"]`;
25
- /** Django's operations. `model_name` and `name` are the table and the column. */
26
- const DJANGO = [
27
- {
28
- find: new RegExp(String.raw `migrations\.RemoveField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?name=${STR}`, 'i'),
29
- risk: 'breaks-old-code',
30
- what: (m) => `removes the field ${m[2]} from ${m[1]}`,
31
- instead: 'stop reading and writing the field, deploy that, then remove it in a later migration. The version still running during a rolling deploy will select a column that no longer exists',
32
- table: (m) => m[1],
33
- column: (m) => m[2],
34
- },
35
- {
36
- find: new RegExp(String.raw `migrations\.RenameField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?old_name=${STR}`, 'i'),
37
- risk: 'breaks-old-code',
38
- what: (m) => `renames ${m[1]}.${m[2]}`,
39
- instead: 'add the new field, write to both, move the readers, then remove the old one',
40
- table: (m) => m[1],
41
- column: (m) => m[2],
42
- },
43
- {
44
- find: new RegExp(String.raw `migrations\.DeleteModel\([\s\S]{0,120}?${STR}`, 'i'),
45
- risk: 'destroys-data',
46
- what: (m) => `deletes the model ${m[1]} and its table`,
47
- instead: 'stop using it, ship that, confirm nothing reads it, and delete it later',
48
- table: (m) => m[1],
49
- },
50
- {
51
- find: new RegExp(String.raw `migrations\.AddField\([\s\S]{0,400}?null=False(?![\s\S]{0,400}?default=)`, 'i'),
52
- risk: 'breaks-old-code',
53
- what: () => 'adds a field that cannot be null and has no default',
54
- instead: 'add it with null=True or a default. Without one, Django rewrites the table and every insert from the running version fails',
55
- },
56
- {
57
- find: new RegExp(String.raw `migrations\.AlterField\([\s\S]{0,400}?null=False`, 'i'),
58
- risk: 'locks-writes',
59
- what: () => 'makes an existing field NOT NULL, which reads every row under a lock',
60
- instead: 'backfill first, add a validated constraint, then alter - or use SeparateDatabaseAndState with the safe SQL',
61
- },
62
- {
63
- // A data migration. Django wraps a migration in one transaction, so this holds it open for as long as
64
- // the work takes - fine for a hundred rows, an outage for a hundred million.
65
- find: /migrations\.RunPython\(/,
66
- unless: /atomic\s*=\s*False/,
67
- risk: 'slow',
68
- what: () => 'runs application code against the data inside the migration transaction',
69
- instead: 'for a large table, work in batches with atomic = False on the migration and make it resumable. For a small one this is fine - Tewip cannot see how many rows there are',
70
- },
71
- {
72
- find: /AddIndex\(|AddConstraint\(/i,
73
- unless: /AddIndexConcurrently|AddConstraintNotValid|atomic\s*=\s*False/,
74
- risk: 'locks-writes',
75
- what: () => 'adds an index or constraint, which Django builds with a lock unless told otherwise',
76
- instead: 'AddIndexConcurrently / AddConstraintNotValid from django.contrib.postgres.operations, with atomic = False on the migration',
77
- },
78
- ];
79
- /** Rails migrations. */
80
- const RAILS = [
81
- {
82
- find: new RegExp(String.raw `remove_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
83
- risk: 'breaks-old-code',
84
- what: (m) => `removes ${m[1]}.${m[2]}`,
85
- instead: "ignore the column in the model (self.ignored_columns), deploy that, then remove it. Rails caches the column list, so the running version keeps selecting it",
86
- table: (m) => m[1],
87
- column: (m) => m[2],
88
- },
89
- {
90
- find: new RegExp(String.raw `rename_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
91
- risk: 'breaks-old-code',
92
- what: (m) => `renames ${m[1]}.${m[2]}`,
93
- instead: 'add the new column, write to both, move the readers, then remove the old one',
94
- table: (m) => m[1],
95
- column: (m) => m[2],
96
- },
97
- {
98
- find: new RegExp(String.raw `drop_table\s*[\s(]\s*:?(\w+)`, 'i'),
99
- risk: 'destroys-data',
100
- what: (m) => `drops the table ${m[1]}`,
101
- instead: 'stop using it, ship that, then drop it later',
102
- table: (m) => m[1],
103
- },
104
- {
105
- find: /add_index\s*[\s(]/i,
106
- unless: /algorithm:\s*:concurrently/,
107
- risk: 'locks-writes',
108
- what: () => 'adds an index without algorithm: :concurrently, so writes to the table block while it builds',
109
- instead: 'add_index ..., algorithm: :concurrently, with disable_ddl_transaction! on the migration',
110
- },
111
- {
112
- find: new RegExp(String.raw `change_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
113
- risk: 'locks-writes',
114
- what: (m) => `changes the type of ${m[1]}.${m[2]}, which usually rewrites the table`,
115
- instead: 'add a new column, backfill in batches, switch the readers, then drop the old one',
116
- table: (m) => m[1],
117
- column: (m) => m[2],
118
- },
119
- {
120
- find: new RegExp(String.raw `change_column_null\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)\s*,\s*false`, 'i'),
121
- risk: 'locks-writes',
122
- what: (m) => `makes ${m[1]}.${m[2]} NOT NULL, which reads every row under a lock`,
123
- instead: 'add a validated check constraint first, then set the column NOT NULL',
124
- table: (m) => m[1],
125
- column: (m) => m[2],
126
- },
127
- ];
128
- /** Alembic. */
129
- const ALEMBIC = [
130
- {
131
- find: new RegExp(String.raw `op\.drop_column\(\s*${STR}\s*,\s*${STR}`, 'i'),
132
- risk: 'breaks-old-code',
133
- what: (m) => `drops ${m[1]}.${m[2]}`,
134
- instead: 'stop reading and writing it, deploy that, then drop it in a later revision',
135
- table: (m) => m[1],
136
- column: (m) => m[2],
137
- },
138
- {
139
- find: new RegExp(String.raw `op\.drop_table\(\s*${STR}`, 'i'),
140
- risk: 'destroys-data',
141
- what: (m) => `drops the table ${m[1]}`,
142
- instead: 'stop using it, ship that, then drop it later',
143
- table: (m) => m[1],
144
- },
145
- {
146
- find: new RegExp(String.raw `op\.alter_column\([\s\S]{0,300}?nullable=False`, 'i'),
147
- risk: 'locks-writes',
148
- what: () => 'makes a column NOT NULL, which reads every row under a lock',
149
- instead: 'backfill, add a validated constraint, then alter the column',
150
- },
151
- {
152
- find: /op\.create_index\(/i,
153
- unless: /postgresql_concurrently\s*=\s*True/,
154
- risk: 'locks-writes',
155
- what: () => 'builds an index without postgresql_concurrently, so writes block while it builds',
156
- instead: 'op.create_index(..., postgresql_concurrently=True) with a non-transactional revision',
157
- },
158
- {
159
- find: new RegExp(String.raw `op\.rename_table\(\s*${STR}`, 'i'),
160
- risk: 'breaks-old-code',
161
- what: (m) => `renames the table ${m[1]}`,
162
- instead: 'move the readers first, or keep a view with the old name',
163
- table: (m) => m[1],
164
- },
165
- ];
166
- // TypeORM has no operations of its own: unlike Django's `RemoveField` or Rails's `remove_column`,
167
- // a TypeORM migration contains SQL and nothing else, handed to queryRunner.query(). So its list is
168
- // empty on purpose and every finding comes from reading that SQL, which is the same reader the
169
- // `.sql` dialect uses. The exhaustive Record is what made this a compile error rather than a
170
- // TypeORM migration silently producing no findings at all.
171
- const BY_DIALECT = { django: DJANGO, rails: RAILS, alembic: ALEMBIC, typeorm: [] };
34
+ const BY_DIALECT = {
35
+ django: DJANGO,
36
+ rails: RAILS,
37
+ alembic: ALEMBIC,
38
+ typeorm: [],
39
+ sequelize: SEQUELIZE,
40
+ knex: KNEX,
41
+ efcore: EFCORE,
42
+ liquibase: LIQUIBASE,
43
+ };
172
44
  /** Findings from a framework migration, with the line each one sits on. */
173
45
  export function readFramework(dialect, text) {
174
46
  const lines = text.split(String.fromCharCode(10));
@@ -230,10 +102,24 @@ export function forwardOnly(dialect, text) {
230
102
  (dialect === 'sql' && /^\s*--\s*(?:\+migrate\s+Down|\+goose\s+Down|\/\/@UNDO|Down\b)/i.test(line)) ||
231
103
  // TypeORM's rollback is `public async down(queryRunner: QueryRunner)`. Reading it as part of
232
104
  // the change would report a migration that adds a column as one that drops it.
233
- (dialect === 'typeorm' && /^\s*(?:public\s+)?async\s+down\s*\(/.test(line));
105
+ (dialect === 'typeorm' && /^\s*(?:public\s+)?async\s+down\s*\(/.test(line)) ||
106
+ // These three matter more than the rest. A Sequelize or Knex migration almost always *adds*
107
+ // in up and *removes* in down, and an EF Core Down() is the mirror image of its Up(), so a
108
+ // reader that took the whole file would report nearly every harmless migration in those
109
+ // ecosystems as dropping a column. Verified on real migrations from opencollective-api,
110
+ // Matterwiki and Coflnet, all three of which have their only destructive call in down.
111
+ (dialect === 'sequelize' && /^\s*down\s*:/.test(line)) ||
112
+ (dialect === 'knex' && /^\s*(?:exports\.down|module\.exports\.down)\s*=/.test(line)) ||
113
+ (dialect === 'efcore' && /^\s*protected\s+override\s+void\s+Down\s*\(/.test(line)) ||
114
+ // Liquibase puts the reverse inside the change itself rather than in a second method.
115
+ (dialect === 'liquibase' && /^\s*<rollback\b/.test(line));
234
116
  if (down) {
235
117
  // To the end of the block: the next line at the same indentation or less that starts something new.
236
- blankFrom(i, (l, indent) => l.trim() !== '' && indentOf(l) <= indent && !/^\s*(end|\})\s*$/.test(l), indentOf(line));
118
+ // A lone brace is punctuation, not the start of something new. C# puts the opening `{` of a
119
+ // method on its own line at the same indent as the signature, so without `{` here the block
120
+ // ended one line after it began and the whole of EF Core's Down() was read as part of the
121
+ // change - which reported Coflnet's real migration as dropping three columns it adds.
122
+ blankFrom(i, (l, indent) => l.trim() !== '' && indentOf(l) <= indent && !/^\s*(end|\}|\{)\s*$/.test(l), indentOf(line));
237
123
  continue;
238
124
  }
239
125
  // Rails's `direction.down { ... }` is usually one line.
@@ -70,7 +70,7 @@ export function readMigration(root, file) {
70
70
  file,
71
71
  dialect,
72
72
  findings: [],
73
- unread: 'Tewip does not recognise how this migration is written, so it says nothing about it rather than passing it. It reads SQL, Django, Rails, Alembic and TypeORM migrations',
73
+ unread: 'Tewip does not recognise how this migration is written, so it says nothing about it rather than passing it. It reads SQL, Django, Rails, Alembic, TypeORM, Sequelize, Knex, EF Core and Liquibase migrations',
74
74
  irreversible: false,
75
75
  };
76
76
  }
@@ -38,10 +38,35 @@ const CHECKS = [
38
38
  // Go.
39
39
  /\bt\s*\.\s*(Error|Errorf|Fatal|Fatalf|Fail|FailNow)\b/,
40
40
  /\b(require|assert)\s*\.\s*[A-Z]\w+\s*\(/,
41
- // Ruby, PHP, Rust.
42
- /\bexpect\s*\(.*\)\s*\.\s*to\b/,
43
- /\b\$this\s*->\s*assert/,
41
+ // Ruby. The snake_case family is the whole of minitest and most of Rails's own suite, and
42
+ // `assert_equal "x", y` has no parentheses - so the generic `assert` patterns above, which all
43
+ // want a `.` or a `(`, matched none of it. Without this, 121 of the 154 real tests in Rails's
44
+ // activesupport string_ext_test.rb were reported as comparing no value. That is the Phase 8
45
+ // lesson for the fourth time: a missing entry here does not produce a smaller answer, it
46
+ // produces a confident wrong one.
47
+ /\b(assert|refute)_\w+/,
48
+ /\bassert\s+[^=\n]/,
49
+ /\bexpect\s*\(.*\)\s*\.\s*(to|not_to|to_not)\b/,
50
+ // RSpec's block form, which is how every raise_error, change and output matcher is written:
51
+ // `expect { thing }.to raise_error(...)`. Missing it reported rspec-core's own specs - the
52
+ // reference implementation - as containing tests that cannot fail.
53
+ /\bexpect\s*\{[\s\S]*?\}\s*\.\s*(to|not_to|to_not)\b/,
54
+ /\bis_expected\s*\.\s*to\b/,
55
+ /\.\s*(must_equal|must_be|wont_equal|wont_be)\b/,
56
+ // PHP: PHPUnit on the instance or statically, and Pest's expectation chain.
57
+ //
58
+ // No `\b` before the `$`. A word boundary needs a word character on one side, `$` is not one,
59
+ // and the character before it in real code is a space - so `/\b\$this/` matched nothing, ever.
60
+ // This pattern was on the list from the start and had never once fired: 88 of the 128 real tests
61
+ // in laravel/framework's SupportStrTest came back as "Tewip would not judge" because the
62
+ // `assertSame(...)` in them was being read as an unknown helper that might assert.
63
+ /\$this\s*->\s*assert/,
64
+ /\b(?:self|static|Assert)\s*::\s*assert/,
65
+ /\bexpect\s*\(.*\)\s*->\s*\w+/,
66
+ // Rust. A #[should_panic] test asserts by panicking, so it can fail without an assert in it.
44
67
  /\bassert(_eq|_ne|_matches)?!\s*\(/,
68
+ /\bdebug_assert(_eq|_ne)?!\s*\(/,
69
+ /#\s*\[\s*should_panic/,
45
70
  // Robot Framework and other keyword styles: a line that reads as a check.
46
71
  /^\s*Should\s+(Be|Not|Contain|Match)\b/im,
47
72
  ];
@@ -0,0 +1,274 @@
1
+ // Phase 22 and after: finding the tests inside a file, in each language's own shape.
2
+ //
3
+ // Split out of tests-in.ts, which passed this project's 300-line rule when Ruby, PHP and Rust were
4
+ // added. tests-in.ts keeps the questions every language shares - which language is this, does it
5
+ // look like tests at all, which parts of it are text rather than code - and this file holds the
6
+ // four readers: callbacks (JavaScript), indentation (Python), `end` (Ruby), and braces (the rest).
7
+ import { braced, isText, lineAt, textRanges } from './tests-in.js';
8
+ /**
9
+ * `it('...', async () => { ... })` and its many spellings, including `test.each`, `it.skip` and a
10
+ * plain `function ()` callback. The body is taken by matching braces from the callback's opening one.
11
+ */
12
+ function jsTests(text) {
13
+ const found = [];
14
+ const ranges = textRanges(text);
15
+ const declaration = /\b(it|test|specify)\s*(\.\s*(skip|only|todo|failing|concurrent|each|for)\s*(\([^)]*\)|`[^`]*`)?\s*)*\(\s*(['"`])((?:\\.|(?!\5)[^\\])*)\5/g;
16
+ for (const hit of text.matchAll(declaration)) {
17
+ const at = hit.index ?? 0;
18
+ if (isText(ranges, at))
19
+ continue;
20
+ const open = text.indexOf('{', at + hit[0].length);
21
+ if (open === -1)
22
+ continue;
23
+ const body = braced(text, open);
24
+ if (body === null)
25
+ continue;
26
+ const signature = text.slice(at + hit[0].length, open);
27
+ found.push({
28
+ name: hit[6],
29
+ line: lineAt(text, at),
30
+ body,
31
+ skipped: /\.\s*(skip|todo|failing)\b/.test(hit[0]) || /\bxit\s*\(/.test(hit[0]),
32
+ callback: /\(\s*([A-Za-z_$][\w$]*)\s*\)\s*(=>)?\s*$/.exec(signature.trim())?.[1] ?? '',
33
+ });
34
+ }
35
+ // `xit(...)` and `xtest(...)`, which the pattern above deliberately does not match.
36
+ for (const hit of text.matchAll(/\bx(it|test)\s*\(\s*(['"`])((?:\\.|(?!\2)[^\\])*)\2/g)) {
37
+ const at = hit.index ?? 0;
38
+ if (isText(ranges, at))
39
+ continue;
40
+ const open = text.indexOf('{', at + hit[0].length);
41
+ const body = open === -1 ? null : braced(text, open);
42
+ found.push({ name: hit[3], line: lineAt(text, at), body: body ?? '', skipped: true, callback: '' });
43
+ }
44
+ return found.sort((a, b) => a.line - b.line);
45
+ }
46
+ /** Python has no braces: a test's body is everything indented further than its `def`. */
47
+ function pythonTests(text) {
48
+ const lines = text.split(String.fromCharCode(10));
49
+ const ranges = textRanges(text);
50
+ const offsets = [];
51
+ let seen = 0;
52
+ for (const line of lines) {
53
+ offsets.push(seen);
54
+ seen += line.length + 1;
55
+ }
56
+ const found = [];
57
+ for (let i = 0; i < lines.length; i += 1) {
58
+ const hit = /^(\s*)(?:async\s+)?def\s+(test_\w+|test)\s*\(/.exec(lines[i]);
59
+ if (!hit)
60
+ continue;
61
+ if (isText(ranges, offsets[i] + hit[1].length + 1))
62
+ continue;
63
+ const indent = hit[1].length;
64
+ // Nested in another def? Then it is a helper or, in Click's tests, a CLI command that happens to
65
+ // be named `test`. Only a module-level def, or a method of a test class, is a test.
66
+ if (nestedInFunction(lines, i, indent))
67
+ continue;
68
+ if (hit[2] === 'test' && indent > 0)
69
+ continue;
70
+ // A def whose parameters run over several lines ends at the line that closes them, and the body
71
+ // starts after that. Without this, the parameter list was read as the body and the real body was
72
+ // thrown away: psf/requests has several, and each came back as a test that compares no value.
73
+ let signatureEnds = i;
74
+ let open = 0;
75
+ for (let j = i; j < lines.length && j < i + 20; j += 1) {
76
+ open += (lines[j].match(/\(/g) ?? []).length - (lines[j].match(/\)/g) ?? []).length;
77
+ signatureEnds = j;
78
+ if (open <= 0)
79
+ break;
80
+ }
81
+ const body = [];
82
+ for (let j = signatureEnds + 1; j < lines.length; j += 1) {
83
+ const line = lines[j];
84
+ if (line.trim() === '') {
85
+ body.push(line);
86
+ continue;
87
+ }
88
+ const width = line.length - line.trimStart().length;
89
+ if (width <= indent)
90
+ break;
91
+ body.push(line);
92
+ }
93
+ // Decorators sit above the def, so skipping is read upwards.
94
+ const above = lines.slice(Math.max(0, i - 4), i).join(String.fromCharCode(10));
95
+ // A conditional skip runs somewhere - `skipif` on Windows registry tests in psf/requests runs
96
+ // on Windows - so only an unconditional skip means the test never runs. `xfail` runs too.
97
+ const skipped = /@\s*(pytest\s*\.\s*mark\s*\.\s*skip|unittest\s*\.\s*skip)\b/.test(above) && !/skipif|skipUnless|xfail/.test(above);
98
+ found.push({ name: hit[2], line: i + 1, body: body.join(String.fromCharCode(10)), skipped, callback: '' });
99
+ }
100
+ return found;
101
+ }
102
+ /** Is the def on line `i` nested inside another def, rather than inside a class or at the top? */
103
+ function nestedInFunction(lines, i, indent) {
104
+ for (let j = i - 1; j >= 0; j -= 1) {
105
+ const line = lines[j];
106
+ if (line.trim() === '')
107
+ continue;
108
+ const width = line.length - line.trimStart().length;
109
+ if (width >= indent)
110
+ continue;
111
+ return /^\s*(async\s+)?def\s/.test(line);
112
+ }
113
+ return false;
114
+ }
115
+ /**
116
+ * Ruby, in both of the two shapes that cover nearly all of it: minitest's `def test_name ... end`
117
+ * and RSpec's `it "name" do ... end`. Rails's own activesupport tests are the first; rspec-core's
118
+ * own specs are the second.
119
+ *
120
+ * The body runs to the `end` that lines up with the opener's indentation. Counting `do`, `def`,
121
+ * `if`, `case` and the rest against `end` would be the thorough way and is a poor trade: Ruby has
122
+ * one-line `if` modifiers, `end` inside strings, and heredocs, so a counter gets it wrong on real
123
+ * code in ways that are hard to see. Matching the indentation is right for conventionally
124
+ * formatted Ruby, which is nearly all Ruby, and when it cannot find the closing `end` the test is
125
+ * skipped rather than judged on a body that was guessed at.
126
+ */
127
+ function rubyTests(text) {
128
+ /** Each accepted test with the line its `end` sits on, so a nested one can be recognised. */
129
+ const found = [];
130
+ const lines = text.split(String.fromCharCode(10));
131
+ const ranges = textRanges(text);
132
+ const offsets = [];
133
+ let seen = 0;
134
+ for (const line of lines) {
135
+ offsets.push(seen);
136
+ seen += line.length + 1;
137
+ }
138
+ for (let i = 0; i < lines.length; i += 1) {
139
+ // def test_foo minitest
140
+ // it "does a thing" do RSpec
141
+ // test "does a thing" do Rails and minitest's spec syntax
142
+ // specify { ... } is deliberately not here: a one-line specify has no body to read.
143
+ const hit = /^(\s*)(?:(?:x|f)?(it|test|scenario|example)\s*[("']|def\s+(test_\w+))/.exec(lines[i]);
144
+ if (!hit)
145
+ continue;
146
+ if (isText(ranges, offsets[i] + hit[1].length))
147
+ continue;
148
+ const indent = hit[1].length;
149
+ let close = -1;
150
+ for (let j = i + 1; j < lines.length; j += 1) {
151
+ const line = lines[j];
152
+ if (line.trim() === '')
153
+ continue;
154
+ if (/^\s*end\b/.test(line) && line.length - line.trimStart().length === indent) {
155
+ close = j;
156
+ break;
157
+ }
158
+ }
159
+ if (close === -1)
160
+ continue;
161
+ // An `it` inside another `it` is not a test. rspec-core's own specs define one as a fixture -
162
+ // `describe_successfully do it "..." do sequence << :example end end` - and the assertion that
163
+ // matters is in the enclosing test. Python's finder skips nested defs for the same reason.
164
+ if (found.some((earlier) => i > earlier.line - 1 && i < earlier.endsAt))
165
+ continue;
166
+ const body = lines.slice(i + 1, close).join(String.fromCharCode(10));
167
+ const named = hit[3] ?? /^\s*(?:x|f)?(?:it|test|scenario|example)\s*[("']\s*["']?([^"')]*)/.exec(lines[i])?.[1]?.trim();
168
+ found.push({
169
+ endsAt: close,
170
+ name: (named ?? '').trim() === '' ? 'unnamed' : named,
171
+ line: i + 1,
172
+ body,
173
+ // `xit` turns a test off. A bare `skip` does too - but only at the top level of the body.
174
+ // rspec-core's own specs pass a block containing `skip "because"` to a helper, as the
175
+ // fixture whose behaviour the test is checking, and the test around it asserts plenty.
176
+ // Reading that as "this test never runs" condemned the reference implementation.
177
+ skipped: /^\s*(?:x(?:it|specify|example)|skip\b)/.test(lines[i]) || skipsAtTopLevel(body),
178
+ callback: '',
179
+ });
180
+ }
181
+ return found;
182
+ }
183
+ /**
184
+ * How each language says "this is a test".
185
+ *
186
+ * Rust and PHP join the braced finder rather than getting one of their own, because both mark a
187
+ * test the way the JVM languages do: an attribute or a naming convention, then a braced body. Only
188
+ * the marker differs.
189
+ */
190
+ const DECLARES = {
191
+ // Rust marks a test with an attribute. `#[bench]` is left out for the same reason Go's Benchmark
192
+ // is: it measures rather than checks, and calling it assertion-free is true and useless.
193
+ rust: /#\s*\[\s*(?:test|tokio::test|async_std::test|rstest)\s*\]/g,
194
+ // PHPUnit accepts three markers and real suites use all of them: the testFoo naming convention
195
+ // (laravel/framework does this), the #[Test] attribute of PHPUnit 10, and the @test docblock.
196
+ php: /(?:#\s*\[\s*Test[^\]]*\]|\*\s*@test\b|(?:public|protected|private)?\s*function\s+test\w*\s*\()/g,
197
+ };
198
+ /** JUnit, Kotlin, Scala, Go, C#, PHP and Rust: an annotation or a naming convention, then a braced body. */
199
+ function bracedTests(language, text) {
200
+ const found = [];
201
+ const ranges = textRanges(text, language);
202
+ const declaration = DECLARES[language] ??
203
+ (language === 'go'
204
+ ? // Only a Test function. A Benchmark asserts nothing by design, an Example is checked against
205
+ // its own Output comment, and a Fuzz target is not a test - all three were condemned as
206
+ // "checks nothing" in gorilla/mux before this narrowed.
207
+ /\bfunc\s+(Test)(\w+)\s*\(\s*\w+\s+\*testing\.[TB]\s*\)\s*\{/g
208
+ : /(@Test\b[^\n]*|\[\s*(?:Test|Fact|Theory)[^\]]*\]|@(?:ParameterizedTest|RepeatedTest)\b)/g);
209
+ for (const hit of text.matchAll(declaration)) {
210
+ const at = hit.index ?? 0;
211
+ if (isText(ranges, at))
212
+ continue;
213
+ // Where the body starts. Go's marker already includes the brace. Rust and PHP put a signature
214
+ // after the marker, and in PHP the marker may itself *be* the signature, so searching from the
215
+ // end of the match covers both.
216
+ const open = language === 'go'
217
+ ? text.indexOf('{', at)
218
+ : language === 'rust' || language === 'php'
219
+ ? text.indexOf('{', at + hit[0].length)
220
+ : text.indexOf('{', text.indexOf('(', at + hit[0].length));
221
+ if (open === -1)
222
+ continue;
223
+ const body = braced(text, open);
224
+ if (body === null)
225
+ continue;
226
+ const signature = text.slice(at, open);
227
+ // `void foo() throws IOException` still names foo. Without allowing the throws clause, every
228
+ // checked-exception test in gson was reported as 'unnamed'.
229
+ const name = language === 'go'
230
+ ? hit[1] + hit[2]
231
+ : language === 'rust'
232
+ ? // ripgrep generates whole families of tests from a macro, where the name in the source
233
+ // is literally `$name`. Reporting that is more honest than inventing one.
234
+ /\bfn\s+(\$?\w+)/.exec(text.slice(at, open))?.[1] ?? 'unnamed'
235
+ : language === 'php'
236
+ ? /\bfunction\s+(\w+)/.exec(text.slice(at, open))?.[1] ?? 'unnamed'
237
+ : (/([\w]+)\s*\([^)]*\)\s*(throws\s+[\w.,\s]+)?$/.exec(signature.trim())?.[1] ?? 'unnamed');
238
+ const around = text.slice(Math.max(0, at - 200), open);
239
+ // An annotation disables a test outright. A `t.Skip()` may be conditional - cobra's
240
+ // TestDeadcodeElimination skips on Windows and runs everywhere else - so it counts only when it
241
+ // is unconditional, at the top level of the body.
242
+ const skipped = /@(Disabled|Ignore)\b|\[\s*Ignore\b/.test(around) || skipsUnconditionally(body);
243
+ found.push({ name, line: lineAt(text, at), body, skipped, callback: '' });
244
+ }
245
+ return found;
246
+ }
247
+ /**
248
+ * A `t.Skip()` that runs whatever happens: at the top level of the body, not inside an `if`. Go
249
+ * tests commonly skip on one platform and run on the others, and calling those "never runs" is
250
+ * wrong.
251
+ */
252
+ function skipsUnconditionally(body) {
253
+ let depth = 0;
254
+ for (const line of body.split(String.fromCharCode(10))) {
255
+ const text = line.trim();
256
+ if (depth === 0 && /^\w+\s*\.\s*Skip(Now)?\s*\(/.test(text))
257
+ return true;
258
+ depth += (line.match(/\{/g) ?? []).length - (line.match(/\}/g) ?? []).length;
259
+ }
260
+ return false;
261
+ }
262
+ /**
263
+ * A Ruby `skip` that turns this test off, rather than one sitting inside a block the test is
264
+ * handing to something else. Only a call at the body's own indentation counts: anything deeper is
265
+ * inside a block, and in rspec-core's specs that block is the fixture under test.
266
+ */
267
+ function skipsAtTopLevel(body) {
268
+ const lines = body.split(String.fromCharCode(10)).filter((l) => l.trim() !== '');
269
+ if (lines.length === 0)
270
+ return false;
271
+ const base = Math.min(...lines.map((l) => l.length - l.trimStart().length));
272
+ return lines.some((l) => l.length - l.trimStart().length === base && /^\s*(?:skip|pending)\b/.test(l));
273
+ }
274
+ export { jsTests, pythonTests, rubyTests, bracedTests };
@@ -1,3 +1,10 @@
1
+ // Finding the individual tests in a file, and their bodies, without a parser per language.
2
+ //
3
+ // A parser for each of six languages is not the job, and would be the whole job. What is needed is
4
+ // narrower: where each test starts, what text belongs to it, and whether it was marked as skipped.
5
+ // Everything after this reads that text. Where the shape is not recognised, the file is reported as
6
+ // unread rather than judged - a file Tewip cannot see into is not a file full of worthless tests.
7
+ import { bracedTests, jsTests, pythonTests, rubyTests } from './find-tests.js';
1
8
  /** Which language this file is, by extension. Unknown extensions are not guessed at. */
2
9
  export function languageOf(file) {
3
10
  const lower = file.toLowerCase();
@@ -11,12 +18,27 @@ export function languageOf(file) {
11
18
  return 'go';
12
19
  if (/\.(cs|fs)$/.test(lower))
13
20
  return 'csharp';
21
+ // Phase 36. The assertion vocabulary already knew how these three check things - `expect(x).to`,
22
+ // `$this->assert`, `assert_eq!` were all on the list - so the only reason a Ruby, PHP or Rust
23
+ // file was never judged is that it never got this far. Triage has claimed all three for months.
24
+ if (lower.endsWith('.rb'))
25
+ return 'ruby';
26
+ if (lower.endsWith('.php'))
27
+ return 'php';
28
+ if (lower.endsWith('.rs'))
29
+ return 'rust';
14
30
  return null;
15
31
  }
16
32
  /** Does this file look like tests at all? Vouching for a source file is a category error. */
17
33
  export function looksLikeTests(file, text) {
18
34
  const named = /(^|[\\/])(tests?|spec|specs|__tests__|e2e|it|integration)[\\/]|[._-](test|tests|spec|specs|it)\.[a-z]+$|(^|[\\/])test_[^\\/]+\.py$|_test\.(go|py|rb)$|Tests?\.(java|kt|cs|scala)$/i.test(file);
19
- const declares = /\b(it|test|describe|Scenario)\s*\(|^\s*def\s+test_|@Test\b|\[(Test|Fact|Theory)\]|\bfunc\s+Test[A-Z]/m.test(text);
35
+ const declares = /\b(it|test|describe|Scenario)\s*\(|^\s*def\s+test_|@Test\b|\[(Test|Fact|Theory)\]|\bfunc\s+Test[A-Z]/m.test(text) ||
36
+ // Ruby, PHP and Rust, added in Phase 36. PHPUnit names a test `testSomething`, so the `test\s*\(`
37
+ // above never matched it, and Rust marks one with an attribute no pattern here knew. Both came
38
+ // back as "0 test(s) read in 0 file(s)", which reads like a clean answer and is not one.
39
+ /#\s*\[\s*(?:test|tokio::test|rstest)\s*\]/.test(text) ||
40
+ /\bfunction\s+test\w*\s*\(|#\s*\[\s*Test\b|\*\s*@test\b/.test(text) ||
41
+ /^\s*(?:RSpec\.)?(?:describe|context)\s+|^\s*(?:x|f)?it\s+["'].*\bdo\b/m.test(text);
20
42
  return named || declares;
21
43
  }
22
44
  /**
@@ -26,7 +48,9 @@ export function looksLikeTests(file, text) {
26
48
  * a checker ends up telling somebody that a string in their fixture checks nothing. Found by
27
49
  * pointing this at Tewip's own test suite, where it produced 27 confident false accusations.
28
50
  */
29
- export function textRanges(text) {
51
+ /** A Rust char literal: `'a'`, `'\n'`, `'\''`. Anything else beginning with a quote is a lifetime. */
52
+ const RUST_CHAR = new RegExp(`^'(?:${String.raw `\\`}.|[^'${String.raw `\\`}])'`);
53
+ export function textRanges(text, language) {
30
54
  const ranges = [];
31
55
  const TRIPLES = ['"'.repeat(3), "'".repeat(3)];
32
56
  for (let i = 0; i < text.length; i += 1) {
@@ -47,6 +71,14 @@ export function textRanges(text) {
47
71
  ranges.push([i, end]);
48
72
  i = end;
49
73
  }
74
+ else if (ch === "'" && language === 'rust' && !RUST_CHAR.test(text.slice(i, i + 4))) {
75
+ // A lone quote in Rust is a lifetime - `&'a str`, `Cow<'_, str>` - and not the start of
76
+ // anything. Treating it as a quote made the detector swallow the rest of the file, so all
77
+ // six of ripgrep's #[test] attributes were discarded as being inside a string and the file
78
+ // was reported as having no tests at all. A char literal is still text, so it is matched
79
+ // first: a char closes within three characters and a lifetime never closes.
80
+ continue;
81
+ }
50
82
  else if (ch === '"' || ch === "'" || ch === '`') {
51
83
  // A Python docstring runs to the next triple quote, not to the next single one.
52
84
  const triple = TRIPLES.find((t) => text.startsWith(t, i));
@@ -58,171 +90,18 @@ export function textRanges(text) {
58
90
  return ranges;
59
91
  }
60
92
  /** Is this offset inside text rather than code? */
61
- const isText = (ranges, at) => ranges.some(([from, to]) => at > from && at < to);
93
+ export const isText = (ranges, at) => ranges.some(([from, to]) => at > from && at < to);
62
94
  export function testsIn(language, text) {
63
95
  if (language === 'python')
64
96
  return pythonTests(text);
65
97
  if (language === 'js')
66
98
  return jsTests(text);
99
+ if (language === 'ruby')
100
+ return rubyTests(text);
67
101
  return bracedTests(language, text);
68
102
  }
69
103
  /** The line number of an offset, 1-based. */
70
- const lineAt = (text, index) => text.slice(0, index).split(String.fromCharCode(10)).length;
71
- /**
72
- * `it('...', async () => { ... })` and its many spellings, including `test.each`, `it.skip` and a
73
- * plain `function ()` callback. The body is taken by matching braces from the callback's opening one.
74
- */
75
- function jsTests(text) {
76
- const found = [];
77
- const ranges = textRanges(text);
78
- const declaration = /\b(it|test|specify)\s*(\.\s*(skip|only|todo|failing|concurrent|each|for)\s*(\([^)]*\)|`[^`]*`)?\s*)*\(\s*(['"`])((?:\\.|(?!\5)[^\\])*)\5/g;
79
- for (const hit of text.matchAll(declaration)) {
80
- const at = hit.index ?? 0;
81
- if (isText(ranges, at))
82
- continue;
83
- const open = text.indexOf('{', at + hit[0].length);
84
- if (open === -1)
85
- continue;
86
- const body = braced(text, open);
87
- if (body === null)
88
- continue;
89
- const signature = text.slice(at + hit[0].length, open);
90
- found.push({
91
- name: hit[6],
92
- line: lineAt(text, at),
93
- body,
94
- skipped: /\.\s*(skip|todo|failing)\b/.test(hit[0]) || /\bxit\s*\(/.test(hit[0]),
95
- callback: /\(\s*([A-Za-z_$][\w$]*)\s*\)\s*(=>)?\s*$/.exec(signature.trim())?.[1] ?? '',
96
- });
97
- }
98
- // `xit(...)` and `xtest(...)`, which the pattern above deliberately does not match.
99
- for (const hit of text.matchAll(/\bx(it|test)\s*\(\s*(['"`])((?:\\.|(?!\2)[^\\])*)\2/g)) {
100
- const at = hit.index ?? 0;
101
- if (isText(ranges, at))
102
- continue;
103
- const open = text.indexOf('{', at + hit[0].length);
104
- const body = open === -1 ? null : braced(text, open);
105
- found.push({ name: hit[3], line: lineAt(text, at), body: body ?? '', skipped: true, callback: '' });
106
- }
107
- return found.sort((a, b) => a.line - b.line);
108
- }
109
- /** Python has no braces: a test's body is everything indented further than its `def`. */
110
- function pythonTests(text) {
111
- const lines = text.split(String.fromCharCode(10));
112
- const ranges = textRanges(text);
113
- const offsets = [];
114
- let seen = 0;
115
- for (const line of lines) {
116
- offsets.push(seen);
117
- seen += line.length + 1;
118
- }
119
- const found = [];
120
- for (let i = 0; i < lines.length; i += 1) {
121
- const hit = /^(\s*)(?:async\s+)?def\s+(test_\w+|test)\s*\(/.exec(lines[i]);
122
- if (!hit)
123
- continue;
124
- if (isText(ranges, offsets[i] + hit[1].length + 1))
125
- continue;
126
- const indent = hit[1].length;
127
- // Nested in another def? Then it is a helper or, in Click's tests, a CLI command that happens to
128
- // be named `test`. Only a module-level def, or a method of a test class, is a test.
129
- if (nestedInFunction(lines, i, indent))
130
- continue;
131
- if (hit[2] === 'test' && indent > 0)
132
- continue;
133
- // A def whose parameters run over several lines ends at the line that closes them, and the body
134
- // starts after that. Without this, the parameter list was read as the body and the real body was
135
- // thrown away: psf/requests has several, and each came back as a test that compares no value.
136
- let signatureEnds = i;
137
- let open = 0;
138
- for (let j = i; j < lines.length && j < i + 20; j += 1) {
139
- open += (lines[j].match(/\(/g) ?? []).length - (lines[j].match(/\)/g) ?? []).length;
140
- signatureEnds = j;
141
- if (open <= 0)
142
- break;
143
- }
144
- const body = [];
145
- for (let j = signatureEnds + 1; j < lines.length; j += 1) {
146
- const line = lines[j];
147
- if (line.trim() === '') {
148
- body.push(line);
149
- continue;
150
- }
151
- const width = line.length - line.trimStart().length;
152
- if (width <= indent)
153
- break;
154
- body.push(line);
155
- }
156
- // Decorators sit above the def, so skipping is read upwards.
157
- const above = lines.slice(Math.max(0, i - 4), i).join(String.fromCharCode(10));
158
- // A conditional skip runs somewhere - `skipif` on Windows registry tests in psf/requests runs
159
- // on Windows - so only an unconditional skip means the test never runs. `xfail` runs too.
160
- const skipped = /@\s*(pytest\s*\.\s*mark\s*\.\s*skip|unittest\s*\.\s*skip)\b/.test(above) && !/skipif|skipUnless|xfail/.test(above);
161
- found.push({ name: hit[2], line: i + 1, body: body.join(String.fromCharCode(10)), skipped, callback: '' });
162
- }
163
- return found;
164
- }
165
- /** Is the def on line `i` nested inside another def, rather than inside a class or at the top? */
166
- function nestedInFunction(lines, i, indent) {
167
- for (let j = i - 1; j >= 0; j -= 1) {
168
- const line = lines[j];
169
- if (line.trim() === '')
170
- continue;
171
- const width = line.length - line.trimStart().length;
172
- if (width >= indent)
173
- continue;
174
- return /^\s*(async\s+)?def\s/.test(line);
175
- }
176
- return false;
177
- }
178
- /** JUnit, Kotlin, Scala, Go and C#: an annotation or a naming convention, then a braced body. */
179
- function bracedTests(language, text) {
180
- const found = [];
181
- const ranges = textRanges(text);
182
- const declaration = language === 'go'
183
- ? // Only a Test function. A Benchmark asserts nothing by design, an Example is checked against
184
- // its own Output comment, and a Fuzz target is not a test - all three were condemned as
185
- // "checks nothing" in gorilla/mux before this narrowed.
186
- /\bfunc\s+(Test)(\w+)\s*\(\s*\w+\s+\*testing\.[TB]\s*\)\s*\{/g
187
- : /(@Test\b[^\n]*|\[\s*(?:Test|Fact|Theory)[^\]]*\]|@(?:ParameterizedTest|RepeatedTest)\b)/g;
188
- for (const hit of text.matchAll(declaration)) {
189
- const at = hit.index ?? 0;
190
- if (isText(ranges, at))
191
- continue;
192
- const open = language === 'go' ? text.indexOf('{', at) : text.indexOf('{', text.indexOf('(', at + hit[0].length));
193
- if (open === -1)
194
- continue;
195
- const body = braced(text, open);
196
- if (body === null)
197
- continue;
198
- const signature = text.slice(at, open);
199
- // `void foo() throws IOException` still names foo. Without allowing the throws clause, every
200
- // checked-exception test in gson was reported as 'unnamed'.
201
- const name = language === 'go' ? hit[1] + hit[2] : /([\w]+)\s*\([^)]*\)\s*(throws\s+[\w.,\s]+)?$/.exec(signature.trim())?.[1] ?? 'unnamed';
202
- const around = text.slice(Math.max(0, at - 200), open);
203
- // An annotation disables a test outright. A `t.Skip()` may be conditional - cobra's
204
- // TestDeadcodeElimination skips on Windows and runs everywhere else - so it counts only when it
205
- // is unconditional, at the top level of the body.
206
- const skipped = /@(Disabled|Ignore)\b|\[\s*Ignore\b/.test(around) || skipsUnconditionally(body);
207
- found.push({ name, line: lineAt(text, at), body, skipped, callback: '' });
208
- }
209
- return found;
210
- }
211
- /**
212
- * A `t.Skip()` that runs whatever happens: at the top level of the body, not inside an `if`. Go
213
- * tests commonly skip on one platform and run on the others, and calling those "never runs" is
214
- * wrong.
215
- */
216
- function skipsUnconditionally(body) {
217
- let depth = 0;
218
- for (const line of body.split(String.fromCharCode(10))) {
219
- const text = line.trim();
220
- if (depth === 0 && /^\w+\s*\.\s*Skip(Now)?\s*\(/.test(text))
221
- return true;
222
- depth += (line.match(/\{/g) ?? []).length - (line.match(/\}/g) ?? []).length;
223
- }
224
- return false;
225
- }
104
+ export const lineAt = (text, index) => text.slice(0, index).split(String.fromCharCode(10)).length;
226
105
  /**
227
106
  * The text between the brace at `open` and the one that closes it, skipping over strings, template
228
107
  * literals, comments and regular expressions - all of which can hold a brace that closes nothing.
@@ -34,7 +34,10 @@ export function testFiles(root, paths) {
34
34
  }
35
35
  if (stat.isDirectory())
36
36
  walk(full, depth + 1);
37
- else if (/\.(js|jsx|mjs|cjs|ts|tsx|mts|cts|py|java|kt|kts|scala|groovy|go|cs|fs)$/i.test(entry))
37
+ // rb, php and rs added in Phase 36. They were missing here as well as in languageOf, so even
38
+ // a file named outright came back "0 test(s) read in 0 file(s)" - which reads like a clean
39
+ // answer and is not one.
40
+ else if (/\.(js|jsx|mjs|cjs|ts|tsx|mts|cts|py|java|kt|kts|scala|groovy|go|cs|fs|rb|php|rs)$/i.test(entry))
38
41
  found.push(full);
39
42
  }
40
43
  };
@@ -95,6 +95,17 @@ function judge(file, test, helpers) {
95
95
  const unknown = called.find((name) => /^(check|verify|ensure|validate|assert|expect|should|confirm)/i.test(name));
96
96
  if (unknown)
97
97
  return { ...at, verdict: 'cannot-tell', evidence: `${unknown}(…)`, why: `${unknown}() may assert, and it is not in this file, so Tewip will not call this either way` };
98
+ // A body whose whole content is one call to something defined in another file. The check is very
99
+ // likely inside it, and Tewip cannot follow it there, so "it compares no value" would be a
100
+ // confident wrong answer. Rails's own `test_constantize` is exactly this: one line calling
101
+ // `run_constantize_tests_on`, which lives in a shared helper and asserts plenty.
102
+ const statements = body
103
+ .split(String.fromCharCode(10))
104
+ .map((l) => l.trim())
105
+ .filter((l) => l !== '' && !l.startsWith('#') && !l.startsWith('//'));
106
+ if (statements.length === 1 && /^[A-Za-z_][\w.:]*\s*[({]/.test(statements[0])) {
107
+ return { ...at, verdict: 'cannot-tell', evidence: statements[0].slice(0, 80), why: 'its whole body is one call to something defined elsewhere, which may assert; Tewip will not call this either way' };
108
+ }
98
109
  if (unfinished.length > 0)
99
110
  return { ...at, verdict: 'unfinished-expectation', evidence: unfinished[0], why: MEANS['unfinished-expectation'] };
100
111
  if (tautology.length > 0)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tewip-ai",
3
- "version": "0.41.0",
3
+ "version": "0.43.0",
4
4
  "mcpName": "io.github.hmamut39/tewip",
5
5
  "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
6
6
  "keywords": [