tewip-ai 0.30.0 → 0.32.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +99 -0
- package/dist/cli/help.js +14 -0
- package/dist/cli/tewip.js +22 -0
- package/dist/engine/quality/write-command.js +2 -1
- package/dist/engine/quality/write.js +2 -0
- package/dist/engine/review/review-command.js +6 -1
- package/dist/engine/schema/frameworks.js +230 -0
- package/dist/engine/schema/schema-command.js +174 -0
- package/dist/engine/schema/statements.js +164 -0
- package/dist/engine/standards/standards-command.js +126 -0
- package/dist/engine/standards/standards.js +140 -0
- package/dist/engine/verdict/verdict-command.js +4 -0
- package/dist/engine/verdict/verdict.js +17 -1
- package/dist/mcp/server.js +37 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -552,6 +552,89 @@ With `--json` the verdict is **sealed**: editing `not-ready` to `ready` in the f
|
|
|
552
552
|
fail, so a verdict that travels into a ticket or a release note can be checked rather than trusted. It is
|
|
553
553
|
also an MCP tool, so an agent can ask whether its own work is done before saying that it is.
|
|
554
554
|
|
|
555
|
+
## Teach it your house rules (Phase 27)
|
|
556
|
+
|
|
557
|
+
```bash
|
|
558
|
+
tewip standards # what this project has taught Tewip
|
|
559
|
+
tewip standards --diff main...HEAD --strict
|
|
560
|
+
```
|
|
561
|
+
|
|
562
|
+
Tewip knows a lot that is true of software in general and nothing that is true of *your company*. Every
|
|
563
|
+
IT department has its own rules — "never drop a column in the release that stops writing to it",
|
|
564
|
+
"locators use data-testid", "no test may sleep" — and a tool that cannot be taught them is a tool that
|
|
565
|
+
argues with the house.
|
|
566
|
+
|
|
567
|
+
A standard is a markdown file in `.tewip/standards/`:
|
|
568
|
+
|
|
569
|
+
```markdown
|
|
570
|
+
---
|
|
571
|
+
description: A fixed wait hides a race instead of fixing it.
|
|
572
|
+
applies-to: tests/, e2e/
|
|
573
|
+
forbid: /waitForTimeout|sleep\(/ because a fixed wait passes by luck and fails in CI
|
|
574
|
+
---
|
|
575
|
+
|
|
576
|
+
Wait for the thing you are actually waiting for — an element, a response, a state —
|
|
577
|
+
never for a number of milliseconds.
|
|
578
|
+
```
|
|
579
|
+
|
|
580
|
+
**The split is the point.** The `forbid` and `require` lines are checked and enforced, with the offending
|
|
581
|
+
line quoted and your own reason printed beside it. The prose below is **guidance**: Tewip carries it into
|
|
582
|
+
whatever it writes or fixes, and quotes it in reports — but it is never counted as checked and never
|
|
583
|
+
reported as kept. A team that believes an unenforceable rule is being enforced is worse off than a team
|
|
584
|
+
with no tool at all, because they have stopped watching for it themselves. A rule Tewip cannot parse is
|
|
585
|
+
reported as unusable rather than silently dropped.
|
|
586
|
+
|
|
587
|
+
Once written, the rules appear everywhere the work is judged: in `tewip review`, in the pull request
|
|
588
|
+
comment, and as a question in `tewip done` — *"Does it keep this project's own standards?"* — which
|
|
589
|
+
answers **no** and names the rule. The guidance goes into the prompt when `tewip write` drafts a test, so
|
|
590
|
+
what comes back is in your conventions rather than Tewip's.
|
|
591
|
+
|
|
592
|
+
## What will this migration do to a live database? (Phase 26)
|
|
593
|
+
|
|
594
|
+
```bash
|
|
595
|
+
tewip schema --diff main...HEAD # or: tewip schema migrations/0002_drop_nickname.sql
|
|
596
|
+
tewip schema --strict # fails the build on a change that breaks the running version
|
|
597
|
+
```
|
|
598
|
+
|
|
599
|
+
The most expensive change in a pull request is usually not a test — it is a migration. A column dropped
|
|
600
|
+
while the previous deployment is still selecting it. An index built without `CONCURRENTLY`, blocking every
|
|
601
|
+
write to the busiest table. A `NOT NULL` added to a populated table. These are known, documented, and a
|
|
602
|
+
tool can name every one of them from the text of the migration.
|
|
603
|
+
|
|
604
|
+
```
|
|
605
|
+
1 of 1 migration file(s) read. 1 change(s) would break the running version or destroy data
|
|
606
|
+
|
|
607
|
+
## Breaks the version that is still running (1)
|
|
608
|
+
- drops the column nickname from orders — migrations/0002_drop_nickname.sql:1
|
|
609
|
+
- Instead: deploy code that no longer selects or writes the column, then drop it in a later
|
|
610
|
+
migration. During a rolling deploy the old version is still running and will fail on a column
|
|
611
|
+
that is gone
|
|
612
|
+
|
|
613
|
+
## Code that still uses what this takes away (1)
|
|
614
|
+
- orders.nickname is still named in 1 file(s): app/orders.py
|
|
615
|
+
```
|
|
616
|
+
|
|
617
|
+
That last section is the part no SQL linter does: it searches your own code for what the migration takes
|
|
618
|
+
away. "This drops `users.nickname`, and four files still read it" is a different sentence from "dropping a
|
|
619
|
+
column is risky", and it is the one that stops the outage.
|
|
620
|
+
|
|
621
|
+
It reads **SQL, Django, Rails and Alembic** migrations, judges **only the forward path** (a rollback step
|
|
622
|
+
contains the mirror image of the change, so reading it says a migration that creates a table destroys
|
|
623
|
+
one), and stays quiet when the team already did the safe thing — `atomic = False`, `algorithm:
|
|
624
|
+
:concurrently`, `NOT VALID`, `postgresql_concurrently=True`. When it cannot read a migration it says so
|
|
625
|
+
rather than passing it, and "read and nothing found" never looks like "not read".
|
|
626
|
+
|
|
627
|
+
**Tried on real projects.** 200 Zulip migrations: 199 read, 19 changes that break the running version or
|
|
628
|
+
destroy data, each sampled finding correct when read against the file — including a `DROP COLUMN` buried
|
|
629
|
+
in raw SQL inside a Python data migration. 60 Mastodon migrations: it caught a `change_column` that
|
|
630
|
+
rewrites a table, in a migration whose *name* says it only adds one.
|
|
631
|
+
|
|
632
|
+
What it does not know, and says so: your table sizes and how busy the database is, so it reports what a
|
|
633
|
+
statement **does** — rewrites the table, scans every row, blocks writes — and leaves the duration to the
|
|
634
|
+
person who knows the data. In CI the findings go into the pull request comment, and anything that breaks
|
|
635
|
+
the running version enters the evidence bundle as a refusal: a passing test suite does not verify a
|
|
636
|
+
migration.
|
|
637
|
+
|
|
555
638
|
## Run only the tests that could notice this change (Phase 25)
|
|
556
639
|
|
|
557
640
|
```bash
|
|
@@ -588,6 +671,22 @@ test that would have caught the bug; on a monolith where every test imports the
|
|
|
588
671
|
there is nothing to skip, rather than inventing a saving. It is also an MCP tool, so an agent can ask what
|
|
589
672
|
to run before running anything.
|
|
590
673
|
|
|
674
|
+
**In CI**, `mode: pick` runs before your tests and hands the workflow the files to run:
|
|
675
|
+
|
|
676
|
+
```yaml
|
|
677
|
+
- id: pick
|
|
678
|
+
uses: hmamut39/tewip@main
|
|
679
|
+
with: { mode: pick } # needs fetch-depth: 0 on actions/checkout
|
|
680
|
+
- run: npx jest ${{ steps.pick.outputs.tests }}
|
|
681
|
+
if: steps.pick.outputs.narrowed == 'true'
|
|
682
|
+
- run: npx jest
|
|
683
|
+
if: steps.pick.outputs.narrowed != 'true'
|
|
684
|
+
```
|
|
685
|
+
|
|
686
|
+
Whatever it leaves out is written into the job summary with the reason beside it, so "why did CI not run
|
|
687
|
+
that test" is answered by the run itself. When it will not narrow safely, `narrowed` is `false` and
|
|
688
|
+
`tests` is empty - so a workflow that ignores the flag still runs everything, which is the safe default.
|
|
689
|
+
|
|
591
690
|
## Would your tests catch a bug? (Phase 23)
|
|
592
691
|
|
|
593
692
|
```bash
|
package/dist/cli/help.js
CHANGED
|
@@ -118,6 +118,20 @@ Usage:
|
|
|
118
118
|
is what was asked for covered. Answers ready, not ready, or cannot
|
|
119
119
|
say - and cannot say is never rounded up to ready. --json writes a
|
|
120
120
|
sealed verdict that can be checked later. No model, no cost.
|
|
121
|
+
tewip standards [--diff main...HEAD] [--strict] [--out standards.md]
|
|
122
|
+
Teach Tewip your house rules. Markdown files in
|
|
123
|
+
.tewip/standards/ - a forbid or require line is checked and
|
|
124
|
+
enforced on the files a change touches; the prose below it is
|
|
125
|
+
guidance Tewip carries into whatever it writes or fixes. It never
|
|
126
|
+
reports guidance as verified, because a rule nobody enforces is
|
|
127
|
+
worse than no rule. No model, no cost.
|
|
128
|
+
tewip schema [<files>] [--diff main...HEAD] [--strict] [--out schema.md]
|
|
129
|
+
What will this migration do to a live database? Names the changes
|
|
130
|
+
that break the version still running during a deploy, destroy data,
|
|
131
|
+
block writes while they run, or cannot be undone - and searches your
|
|
132
|
+
code for what still uses the column it takes away. Reads SQL, Django,
|
|
133
|
+
Rails and Alembic migrations, and says so when it cannot read one.
|
|
134
|
+
No model, no cost.
|
|
121
135
|
tewip pick [--diff main...HEAD] [--list] [--out picked.md]
|
|
122
136
|
Which tests could notice this change? Narrows a test run using
|
|
123
137
|
facts rather than a prediction: a test runs when it imports a
|
package/dist/cli/tewip.js
CHANGED
|
@@ -199,6 +199,28 @@ async function main() {
|
|
|
199
199
|
});
|
|
200
200
|
return;
|
|
201
201
|
}
|
|
202
|
+
case 'standards': {
|
|
203
|
+
const { runStandards } = await import('../engine/standards/standards-command.js');
|
|
204
|
+
runStandards({
|
|
205
|
+
root: process.cwd(),
|
|
206
|
+
diff: flag(args, 'diff'),
|
|
207
|
+
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
|
|
208
|
+
out: flag(args, 'out'),
|
|
209
|
+
strict: args.includes('--strict'),
|
|
210
|
+
});
|
|
211
|
+
return;
|
|
212
|
+
}
|
|
213
|
+
case 'schema': {
|
|
214
|
+
const { runSchema } = await import('../engine/schema/schema-command.js');
|
|
215
|
+
runSchema({
|
|
216
|
+
root: process.cwd(),
|
|
217
|
+
diff: flag(args, 'diff') ?? 'main...HEAD',
|
|
218
|
+
files: args.filter((a, i) => !a.startsWith('--') && args[i - 1] !== '--diff' && args[i - 1] !== '--out'),
|
|
219
|
+
out: flag(args, 'out'),
|
|
220
|
+
strict: args.includes('--strict'),
|
|
221
|
+
});
|
|
222
|
+
return;
|
|
223
|
+
}
|
|
202
224
|
case 'pick': {
|
|
203
225
|
const { runPick } = await import('../engine/pick/pick-command.js');
|
|
204
226
|
runPick({
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
// `tewip write`: turn a plan's "to automate next" scenarios into tests in the project's own
|
|
2
2
|
// framework. Kept tests stay beside the project's tests for the team to review and commit; every
|
|
3
3
|
// other draft is moved to .tewip/drafts with the reason, and nothing is committed or pushed.
|
|
4
|
+
import { guidanceFor, loadStandards } from '../standards/standards-command.js';
|
|
4
5
|
import { existsSync, mkdirSync, readdirSync, readFileSync, renameSync, statSync, writeFileSync } from 'node:fs';
|
|
5
6
|
import { dirname, extname, join, relative } from 'node:path';
|
|
6
7
|
import { noModelHelp } from '../llm/providers.js';
|
|
@@ -76,7 +77,7 @@ ${noModelHelp()}`);
|
|
|
76
77
|
process.once('SIGINT', leaveNothing).once('SIGTERM', leaveNothing);
|
|
77
78
|
let outcome;
|
|
78
79
|
try {
|
|
79
|
-
({ outcome } = await writeAndVerify(model, { framework, fileName, pages, feature: saved.feature, code }, scenario, file, { runs: cmd.runs, repairs: cmd.repairs, mutateSource: cmd.mutateSource, log: (l) => console.error(l) }));
|
|
80
|
+
({ outcome } = await writeAndVerify(model, { framework, fileName, pages, feature: saved.feature, code, houseRules: guidanceFor(loadStandards(cmd.root), fileName) }, scenario, file, { runs: cmd.runs, repairs: cmd.repairs, mutateSource: cmd.mutateSource, log: (l) => console.error(l) }));
|
|
80
81
|
}
|
|
81
82
|
catch (error) {
|
|
82
83
|
outcome = { kind: 'rejected', reason: `stopped: ${error.message}`, repairs: 0, passedReliably: false };
|
|
@@ -7,6 +7,7 @@ import { readFileSync } from 'node:fs';
|
|
|
7
7
|
export const WRITE_PROMPT_VERSION = 'write-6';
|
|
8
8
|
const RULES = `Rules:
|
|
9
9
|
- Write in the given framework and language, in the style of the example test: the same imports, helpers, and way of opening pages (if it uses a base URL with relative paths, do the same).
|
|
10
|
+
- When houseRules is given, it is this team's own written guidance. Follow it as you would the example test; where it conflicts with your habits, the team wins.
|
|
10
11
|
- Find elements only by what the accessibility snapshots show: role and accessible name, label, placeholder or visible text. Never invent ids, classes or test ids that no snapshot shows.
|
|
11
12
|
- Check the scenario's expected result specifically. A check that is always true (a page merely loads, a list merely exists) is not a test.
|
|
12
13
|
- Wait for conditions, never for fixed times.
|
|
@@ -42,6 +43,7 @@ function contextFor(ctx, scenario) {
|
|
|
42
43
|
exampleTest: example,
|
|
43
44
|
pages: ctx.pages.map((p) => ({ readAt: p.url, title: p.title, accessibilitySnapshot: p.aria })),
|
|
44
45
|
codeUnderTest: ctx.code?.length ? ctx.code : undefined,
|
|
46
|
+
houseRules: ctx.houseRules || undefined,
|
|
45
47
|
};
|
|
46
48
|
}
|
|
47
49
|
export async function draftTest(model, ctx, scenario) {
|
|
@@ -9,6 +9,8 @@ import { renderReview, reviewChange } from './review.js';
|
|
|
9
9
|
import { changedExpectations } from './expectations.js';
|
|
10
10
|
import { changedAgreements } from './agreements.js';
|
|
11
11
|
import { changedTestLines, vouchChanged } from '../vouch/changed.js';
|
|
12
|
+
import { checkChanged, loadStandards } from '../standards/standards-command.js';
|
|
13
|
+
import { renderViolations } from '../standards/standards.js';
|
|
12
14
|
/** Every failure Tewip has diagnosed in this project, from the history it keeps. */
|
|
13
15
|
function history(root, only) {
|
|
14
16
|
const dir = join(root, '.tewip', 'history');
|
|
@@ -37,7 +39,10 @@ export async function runReview(cmd) {
|
|
|
37
39
|
// Whether the tests in this change could ever fail. Costs nothing, calls no model, and is the one
|
|
38
40
|
// question an agent-written diff most needs asked of it.
|
|
39
41
|
const vouched = changedTestLines(vouchChanged(cmd.root, files));
|
|
40
|
-
|
|
42
|
+
// The project's own rules, which Tewip only knows because this team wrote them down.
|
|
43
|
+
const standards = loadStandards(cmd.root);
|
|
44
|
+
const broken = renderViolations(checkChanged(cmd.root, standards, files), standards);
|
|
45
|
+
const report = [renderReview(review, runCommand), ...(vouched.length > 0 ? ['', ...vouched] : []), ...(broken.length > 0 ? ['', ...broken] : [])].join(String.fromCharCode(10));
|
|
41
46
|
console.log(report);
|
|
42
47
|
if (past.length === 0) {
|
|
43
48
|
console.log('\nNo failure history yet, so the "what has gone wrong here before" part is empty. It fills itself: run `tewip explain` or `tewip report` on your CI reports and every later review knows more.');
|
|
@@ -0,0 +1,230 @@
|
|
|
1
|
+
export function dialectOf(file, text) {
|
|
2
|
+
const path = file.split('\\').join('/').toLowerCase();
|
|
3
|
+
if (path.endsWith('.sql'))
|
|
4
|
+
return 'sql';
|
|
5
|
+
// `class Migration(migrations.Migration)` is what makes a file a Django migration. Keying only on the
|
|
6
|
+
// operation names left Zulip's data migrations unread - and a data migration is exactly the kind that
|
|
7
|
+
// runs for an hour with a transaction held open.
|
|
8
|
+
if (/\bmigrations\.Migration\b|\bmigrations\.(?:CreateModel|AddField|RemoveField|AlterField|RenameField|RunSQL|RunPython|DeleteModel|RenameModel)\b/.test(text))
|
|
9
|
+
return 'django';
|
|
10
|
+
if (/\bActiveRecord::Migration\b|\bdef\s+(?:up|change)\b[\s\S]*\b(?:add_column|remove_column|add_index|change_column|rename_column|drop_table)\b/.test(text))
|
|
11
|
+
return 'rails';
|
|
12
|
+
if (/\bop\.(?:drop_column|add_column|create_index|alter_column|drop_table|rename_table|execute)\b/.test(text))
|
|
13
|
+
return 'alembic';
|
|
14
|
+
if (/(^|\/)(migrations?|db\/migrate)\//.test(path))
|
|
15
|
+
return 'unknown';
|
|
16
|
+
return 'unknown';
|
|
17
|
+
}
|
|
18
|
+
const STR = String.raw `['"]([\w.]+)['"]`;
|
|
19
|
+
/** Django's operations. `model_name` and `name` are the table and the column. */
|
|
20
|
+
const DJANGO = [
|
|
21
|
+
{
|
|
22
|
+
find: new RegExp(String.raw `migrations\.RemoveField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?name=${STR}`, 'i'),
|
|
23
|
+
risk: 'breaks-old-code',
|
|
24
|
+
what: (m) => `removes the field ${m[2]} from ${m[1]}`,
|
|
25
|
+
instead: 'stop reading and writing the field, deploy that, then remove it in a later migration. The version still running during a rolling deploy will select a column that no longer exists',
|
|
26
|
+
table: (m) => m[1],
|
|
27
|
+
column: (m) => m[2],
|
|
28
|
+
},
|
|
29
|
+
{
|
|
30
|
+
find: new RegExp(String.raw `migrations\.RenameField\([\s\S]{0,200}?model_name=${STR}[\s\S]{0,200}?old_name=${STR}`, 'i'),
|
|
31
|
+
risk: 'breaks-old-code',
|
|
32
|
+
what: (m) => `renames ${m[1]}.${m[2]}`,
|
|
33
|
+
instead: 'add the new field, write to both, move the readers, then remove the old one',
|
|
34
|
+
table: (m) => m[1],
|
|
35
|
+
column: (m) => m[2],
|
|
36
|
+
},
|
|
37
|
+
{
|
|
38
|
+
find: new RegExp(String.raw `migrations\.DeleteModel\([\s\S]{0,120}?${STR}`, 'i'),
|
|
39
|
+
risk: 'destroys-data',
|
|
40
|
+
what: (m) => `deletes the model ${m[1]} and its table`,
|
|
41
|
+
instead: 'stop using it, ship that, confirm nothing reads it, and delete it later',
|
|
42
|
+
table: (m) => m[1],
|
|
43
|
+
},
|
|
44
|
+
{
|
|
45
|
+
find: new RegExp(String.raw `migrations\.AddField\([\s\S]{0,400}?null=False(?![\s\S]{0,400}?default=)`, 'i'),
|
|
46
|
+
risk: 'breaks-old-code',
|
|
47
|
+
what: () => 'adds a field that cannot be null and has no default',
|
|
48
|
+
instead: 'add it with null=True or a default. Without one, Django rewrites the table and every insert from the running version fails',
|
|
49
|
+
},
|
|
50
|
+
{
|
|
51
|
+
find: new RegExp(String.raw `migrations\.AlterField\([\s\S]{0,400}?null=False`, 'i'),
|
|
52
|
+
risk: 'locks-writes',
|
|
53
|
+
what: () => 'makes an existing field NOT NULL, which reads every row under a lock',
|
|
54
|
+
instead: 'backfill first, add a validated constraint, then alter - or use SeparateDatabaseAndState with the safe SQL',
|
|
55
|
+
},
|
|
56
|
+
{
|
|
57
|
+
// A data migration. Django wraps a migration in one transaction, so this holds it open for as long as
|
|
58
|
+
// the work takes - fine for a hundred rows, an outage for a hundred million.
|
|
59
|
+
find: /migrations\.RunPython\(/,
|
|
60
|
+
unless: /atomic\s*=\s*False/,
|
|
61
|
+
risk: 'slow',
|
|
62
|
+
what: () => 'runs application code against the data inside the migration transaction',
|
|
63
|
+
instead: 'for a large table, work in batches with atomic = False on the migration and make it resumable. For a small one this is fine - Tewip cannot see how many rows there are',
|
|
64
|
+
},
|
|
65
|
+
{
|
|
66
|
+
find: /AddIndex\(|AddConstraint\(/i,
|
|
67
|
+
unless: /AddIndexConcurrently|AddConstraintNotValid|atomic\s*=\s*False/,
|
|
68
|
+
risk: 'locks-writes',
|
|
69
|
+
what: () => 'adds an index or constraint, which Django builds with a lock unless told otherwise',
|
|
70
|
+
instead: 'AddIndexConcurrently / AddConstraintNotValid from django.contrib.postgres.operations, with atomic = False on the migration',
|
|
71
|
+
},
|
|
72
|
+
];
|
|
73
|
+
/** Rails migrations. */
|
|
74
|
+
const RAILS = [
|
|
75
|
+
{
|
|
76
|
+
find: new RegExp(String.raw `remove_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
|
|
77
|
+
risk: 'breaks-old-code',
|
|
78
|
+
what: (m) => `removes ${m[1]}.${m[2]}`,
|
|
79
|
+
instead: "ignore the column in the model (self.ignored_columns), deploy that, then remove it. Rails caches the column list, so the running version keeps selecting it",
|
|
80
|
+
table: (m) => m[1],
|
|
81
|
+
column: (m) => m[2],
|
|
82
|
+
},
|
|
83
|
+
{
|
|
84
|
+
find: new RegExp(String.raw `rename_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
|
|
85
|
+
risk: 'breaks-old-code',
|
|
86
|
+
what: (m) => `renames ${m[1]}.${m[2]}`,
|
|
87
|
+
instead: 'add the new column, write to both, move the readers, then remove the old one',
|
|
88
|
+
table: (m) => m[1],
|
|
89
|
+
column: (m) => m[2],
|
|
90
|
+
},
|
|
91
|
+
{
|
|
92
|
+
find: new RegExp(String.raw `drop_table\s*[\s(]\s*:?(\w+)`, 'i'),
|
|
93
|
+
risk: 'destroys-data',
|
|
94
|
+
what: (m) => `drops the table ${m[1]}`,
|
|
95
|
+
instead: 'stop using it, ship that, then drop it later',
|
|
96
|
+
table: (m) => m[1],
|
|
97
|
+
},
|
|
98
|
+
{
|
|
99
|
+
find: /add_index\s*[\s(]/i,
|
|
100
|
+
unless: /algorithm:\s*:concurrently/,
|
|
101
|
+
risk: 'locks-writes',
|
|
102
|
+
what: () => 'adds an index without algorithm: :concurrently, so writes to the table block while it builds',
|
|
103
|
+
instead: 'add_index ..., algorithm: :concurrently, with disable_ddl_transaction! on the migration',
|
|
104
|
+
},
|
|
105
|
+
{
|
|
106
|
+
find: new RegExp(String.raw `change_column\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)`, 'i'),
|
|
107
|
+
risk: 'locks-writes',
|
|
108
|
+
what: (m) => `changes the type of ${m[1]}.${m[2]}, which usually rewrites the table`,
|
|
109
|
+
instead: 'add a new column, backfill in batches, switch the readers, then drop the old one',
|
|
110
|
+
table: (m) => m[1],
|
|
111
|
+
column: (m) => m[2],
|
|
112
|
+
},
|
|
113
|
+
{
|
|
114
|
+
find: new RegExp(String.raw `change_column_null\s*[\s(]\s*:?(\w+)\s*,\s*:?(\w+)\s*,\s*false`, 'i'),
|
|
115
|
+
risk: 'locks-writes',
|
|
116
|
+
what: (m) => `makes ${m[1]}.${m[2]} NOT NULL, which reads every row under a lock`,
|
|
117
|
+
instead: 'add a validated check constraint first, then set the column NOT NULL',
|
|
118
|
+
table: (m) => m[1],
|
|
119
|
+
column: (m) => m[2],
|
|
120
|
+
},
|
|
121
|
+
];
|
|
122
|
+
/** Alembic. */
|
|
123
|
+
const ALEMBIC = [
|
|
124
|
+
{
|
|
125
|
+
find: new RegExp(String.raw `op\.drop_column\(\s*${STR}\s*,\s*${STR}`, 'i'),
|
|
126
|
+
risk: 'breaks-old-code',
|
|
127
|
+
what: (m) => `drops ${m[1]}.${m[2]}`,
|
|
128
|
+
instead: 'stop reading and writing it, deploy that, then drop it in a later revision',
|
|
129
|
+
table: (m) => m[1],
|
|
130
|
+
column: (m) => m[2],
|
|
131
|
+
},
|
|
132
|
+
{
|
|
133
|
+
find: new RegExp(String.raw `op\.drop_table\(\s*${STR}`, 'i'),
|
|
134
|
+
risk: 'destroys-data',
|
|
135
|
+
what: (m) => `drops the table ${m[1]}`,
|
|
136
|
+
instead: 'stop using it, ship that, then drop it later',
|
|
137
|
+
table: (m) => m[1],
|
|
138
|
+
},
|
|
139
|
+
{
|
|
140
|
+
find: new RegExp(String.raw `op\.alter_column\([\s\S]{0,300}?nullable=False`, 'i'),
|
|
141
|
+
risk: 'locks-writes',
|
|
142
|
+
what: () => 'makes a column NOT NULL, which reads every row under a lock',
|
|
143
|
+
instead: 'backfill, add a validated constraint, then alter the column',
|
|
144
|
+
},
|
|
145
|
+
{
|
|
146
|
+
find: /op\.create_index\(/i,
|
|
147
|
+
unless: /postgresql_concurrently\s*=\s*True/,
|
|
148
|
+
risk: 'locks-writes',
|
|
149
|
+
what: () => 'builds an index without postgresql_concurrently, so writes block while it builds',
|
|
150
|
+
instead: 'op.create_index(..., postgresql_concurrently=True) with a non-transactional revision',
|
|
151
|
+
},
|
|
152
|
+
{
|
|
153
|
+
find: new RegExp(String.raw `op\.rename_table\(\s*${STR}`, 'i'),
|
|
154
|
+
risk: 'breaks-old-code',
|
|
155
|
+
what: (m) => `renames the table ${m[1]}`,
|
|
156
|
+
instead: 'move the readers first, or keep a view with the old name',
|
|
157
|
+
table: (m) => m[1],
|
|
158
|
+
},
|
|
159
|
+
];
|
|
160
|
+
const BY_DIALECT = { django: DJANGO, rails: RAILS, alembic: ALEMBIC };
|
|
161
|
+
/** Findings from a framework migration, with the line each one sits on. */
|
|
162
|
+
export function readFramework(dialect, text) {
|
|
163
|
+
const lines = text.split(String.fromCharCode(10));
|
|
164
|
+
const out = [];
|
|
165
|
+
for (const operation of BY_DIALECT[dialect]) {
|
|
166
|
+
const hit = operation.find.exec(text);
|
|
167
|
+
if (!hit)
|
|
168
|
+
continue;
|
|
169
|
+
if (operation.unless?.test(text))
|
|
170
|
+
continue;
|
|
171
|
+
// The line the match starts on, so the reader can go straight to it.
|
|
172
|
+
const before = text.slice(0, hit.index ?? 0).split(String.fromCharCode(10)).length;
|
|
173
|
+
out.push({
|
|
174
|
+
statement: (lines[before - 1] ?? hit[0]).trim().slice(0, 300),
|
|
175
|
+
line: before,
|
|
176
|
+
risk: operation.risk,
|
|
177
|
+
what: operation.what(hit),
|
|
178
|
+
instead: operation.instead,
|
|
179
|
+
table: operation.table?.(hit) ?? '',
|
|
180
|
+
column: operation.column?.(hit) ?? '',
|
|
181
|
+
});
|
|
182
|
+
}
|
|
183
|
+
return out;
|
|
184
|
+
}
|
|
185
|
+
/** A migration that cannot be rolled back by running its own down step. */
|
|
186
|
+
export function irreversible(dialect, text) {
|
|
187
|
+
if (dialect === 'django')
|
|
188
|
+
return /migrations\.RunPython\([^)]*\)/.test(text) && !/reverse_code|RunPython\.noop/.test(text);
|
|
189
|
+
if (dialect === 'alembic')
|
|
190
|
+
return /def\s+downgrade\(\)[\s\S]{0,120}?(?:pass|raise\s+NotImplementedError)/.test(text);
|
|
191
|
+
if (dialect === 'rails')
|
|
192
|
+
return /\braise\s+ActiveRecord::IrreversibleMigration\b/.test(text);
|
|
193
|
+
return false;
|
|
194
|
+
}
|
|
195
|
+
/**
|
|
196
|
+
* The forward path only, with the rollback blanked out.
|
|
197
|
+
*
|
|
198
|
+
* A migration's `down` step is what runs if the change is undone, and it necessarily contains the mirror
|
|
199
|
+
* image of the change - a drop_table for a create_table, a remove_column for an add_column. Reading it as
|
|
200
|
+
* part of the change produces exactly the wrong answer, and Mastodon's own migration proved it: a file
|
|
201
|
+
* that creates a table was reported as destroying one, because its rollback drops it.
|
|
202
|
+
*
|
|
203
|
+
* The region is blanked rather than removed so every remaining line keeps its number.
|
|
204
|
+
*/
|
|
205
|
+
export function forwardOnly(dialect, text) {
|
|
206
|
+
const lines = text.split(String.fromCharCode(10));
|
|
207
|
+
const blankFrom = (start, until, indent) => {
|
|
208
|
+
for (let i = start; i < lines.length; i += 1) {
|
|
209
|
+
if (i > start && until(lines[i], indent))
|
|
210
|
+
return;
|
|
211
|
+
lines[i] = '';
|
|
212
|
+
}
|
|
213
|
+
};
|
|
214
|
+
const indentOf = (line) => line.length - line.trimStart().length;
|
|
215
|
+
for (let i = 0; i < lines.length; i += 1) {
|
|
216
|
+
const line = lines[i];
|
|
217
|
+
const down = (dialect === 'rails' && /^\s*def\s+down\b/.test(line)) ||
|
|
218
|
+
(dialect === 'alembic' && /^\s*def\s+downgrade\s*\(/.test(line)) ||
|
|
219
|
+
(dialect === 'sql' && /^\s*--\s*(?:\+migrate\s+Down|\+goose\s+Down|\/\/@UNDO|Down\b)/i.test(line));
|
|
220
|
+
if (down) {
|
|
221
|
+
// To the end of the block: the next line at the same indentation or less that starts something new.
|
|
222
|
+
blankFrom(i, (l, indent) => l.trim() !== '' && indentOf(l) <= indent && !/^\s*(end|\})\s*$/.test(l), indentOf(line));
|
|
223
|
+
continue;
|
|
224
|
+
}
|
|
225
|
+
// Rails's `direction.down { ... }` is usually one line.
|
|
226
|
+
if (dialect === 'rails' && /direction\.down\b/.test(line))
|
|
227
|
+
lines[i] = '';
|
|
228
|
+
}
|
|
229
|
+
return lines.join(String.fromCharCode(10));
|
|
230
|
+
}
|
|
@@ -0,0 +1,174 @@
|
|
|
1
|
+
// `tewip schema --diff main...HEAD`: what this migration will do to a live database.
|
|
2
|
+
//
|
|
3
|
+
// The part that no SQL linter does, and the reason this belongs in Tewip rather than in a linter: it
|
|
4
|
+
// looks for the code that still uses what the migration takes away. "This drops users.nickname, and four
|
|
5
|
+
// files still read it" is a different sentence from "dropping a column is risky", and it is the one that
|
|
6
|
+
// stops the outage.
|
|
7
|
+
//
|
|
8
|
+
// That check is a search of the repository, so it is evidence rather than proof, and it is worded as
|
|
9
|
+
// such. A mention in a comment counts as a mention; Tewip says where it looked and lets the reader judge.
|
|
10
|
+
import { spawnSync } from 'node:child_process';
|
|
11
|
+
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
|
|
12
|
+
import { dirname, resolve } from 'node:path';
|
|
13
|
+
import { changeSummary } from '../quality/grounding.js';
|
|
14
|
+
import { dialectOf, forwardOnly, irreversible, readFramework } from './frameworks.js';
|
|
15
|
+
import { readSql } from './statements.js';
|
|
16
|
+
const MIGRATION = /(^|\/)(migrations?|db\/migrate|prisma\/migrations|alembic|flyway|sql)\//i;
|
|
17
|
+
const SCHEMA_FILE = /\.(sql)$|(^|\/)(migrations?|db\/migrate)\/.*\.(py|rb|js|ts)$/i;
|
|
18
|
+
/** The schema files in this change. */
|
|
19
|
+
export function schemaFiles(cmd) {
|
|
20
|
+
const given = cmd.files && cmd.files.length > 0 ? cmd.files : (changeSummary(cmd.root, cmd.diff)?.files ?? []);
|
|
21
|
+
return given
|
|
22
|
+
.map((f) => f.split('\\').join('/'))
|
|
23
|
+
.filter((f) => (SCHEMA_FILE.test(f) || MIGRATION.test(f)) && existsSync(resolve(cmd.root, f)));
|
|
24
|
+
}
|
|
25
|
+
export function readMigration(root, file) {
|
|
26
|
+
let text = '';
|
|
27
|
+
try {
|
|
28
|
+
text = readFileSync(resolve(root, file), 'utf8');
|
|
29
|
+
}
|
|
30
|
+
catch {
|
|
31
|
+
return { file, dialect: 'unknown', findings: [], unread: 'the file could not be read', irreversible: false };
|
|
32
|
+
}
|
|
33
|
+
const dialect = dialectOf(file, text);
|
|
34
|
+
// Only the forward path: a rollback step contains the mirror image of the change, and reading it as
|
|
35
|
+
// part of the change says a migration that creates a table destroys one.
|
|
36
|
+
const forward = forwardOnly(dialect, text);
|
|
37
|
+
if (dialect === 'sql')
|
|
38
|
+
return { file, dialect, findings: readSql(forward), unread: '', irreversible: false };
|
|
39
|
+
if (dialect === 'unknown') {
|
|
40
|
+
return {
|
|
41
|
+
file,
|
|
42
|
+
dialect,
|
|
43
|
+
findings: [],
|
|
44
|
+
unread: 'Tewip does not recognise how this migration is written, so it says nothing about it rather than passing it. It reads SQL, Django, Rails and Alembic migrations',
|
|
45
|
+
irreversible: false,
|
|
46
|
+
};
|
|
47
|
+
}
|
|
48
|
+
const findings = readFramework(dialect, forward);
|
|
49
|
+
// A migration that only runs raw SQL through the framework is still readable: the SQL is the migration.
|
|
50
|
+
// Raw SQL, however it reaches the database: through the framework, or through a cursor inside a data
|
|
51
|
+
// migration, which is where most large projects put the dangerous parts.
|
|
52
|
+
const raw = [...forward.matchAll(/(?:RunSQL|op\.execute|execute|cursor\.execute)\(\s*(?:r?["'`]{1,3})([\s\S]*?)["'`]{1,3}\s*[,)]/g)].map((m) => m[1]).join(';');
|
|
53
|
+
if (raw.trim())
|
|
54
|
+
findings.push(...readSql(raw));
|
|
55
|
+
return { file, dialect, findings, unread: '', irreversible: irreversible(dialect, text) };
|
|
56
|
+
}
|
|
57
|
+
/**
|
|
58
|
+
* Code outside the migrations that still names a table or column this change takes away.
|
|
59
|
+
*
|
|
60
|
+
* `git grep` because it respects the repository's own ignore rules and is fast on a large tree. A column
|
|
61
|
+
* named `name` or `id` is not searched for: everything matches it, and a finding that always fires teaches
|
|
62
|
+
* a reader to ignore the whole report.
|
|
63
|
+
*/
|
|
64
|
+
export function stillUsed(root, findings) {
|
|
65
|
+
const out = [];
|
|
66
|
+
const tooCommon = new Set(['id', 'name', 'type', 'value', 'data', 'key', 'status', 'state', 'date', 'time', 'user', 'count', 'index', 'order', 'group', 'text', 'url', 'size', 'code']);
|
|
67
|
+
for (const finding of findings) {
|
|
68
|
+
if (finding.risk !== 'breaks-old-code' && finding.risk !== 'destroys-data')
|
|
69
|
+
continue;
|
|
70
|
+
const what = finding.column || finding.table;
|
|
71
|
+
if (!what || what.length < 4 || tooCommon.has(what.toLowerCase()))
|
|
72
|
+
continue;
|
|
73
|
+
if (out.some((u) => u.table === finding.table && u.column === finding.column))
|
|
74
|
+
continue;
|
|
75
|
+
let files = grepFor(root, what);
|
|
76
|
+
// A bare column name is too weak on its own. Dropping `fasp_providers.capabilities` from Mastodon
|
|
77
|
+
// matched 37 files including a Dockerfile and an emoji module, because "capabilities" is an ordinary
|
|
78
|
+
// word - and noise in the one section meant to be the differentiator is worse than silence. So when a
|
|
79
|
+
// column is named, the file has to mention the table as well.
|
|
80
|
+
if (finding.column && finding.table) {
|
|
81
|
+
const named = new Set([...grepFor(root, finding.table), ...grepFor(root, singular(finding.table))]);
|
|
82
|
+
files = files.filter((f) => named.has(f));
|
|
83
|
+
}
|
|
84
|
+
if (files.length > 0)
|
|
85
|
+
out.push({ table: finding.table, column: finding.column, files });
|
|
86
|
+
}
|
|
87
|
+
return out;
|
|
88
|
+
}
|
|
89
|
+
/** Files naming this word, excluding migrations and documentation. */
|
|
90
|
+
function grepFor(root, word) {
|
|
91
|
+
if (!word || word.length < 4)
|
|
92
|
+
return [];
|
|
93
|
+
const found = spawnSync('git', ['grep', '-l', '-w', '--', word], { cwd: root, encoding: 'utf8', timeout: 60_000 });
|
|
94
|
+
return (found.stdout ?? '')
|
|
95
|
+
.split(String.fromCharCode(10))
|
|
96
|
+
.map((l) => l.trim().split(String.fromCharCode(92)).join('/'))
|
|
97
|
+
.filter((l) => l && !MIGRATION.test(l) && !/\.(md|txt|lock|json)$/i.test(l));
|
|
98
|
+
}
|
|
99
|
+
/** `fasp_providers` is the table; the code calls the model `FaspProvider`, so the singular is searched too. */
|
|
100
|
+
const singular = (table) => (table.endsWith('ies') ? table.slice(0, -3) + 'y' : table.endsWith('s') && !table.endsWith('ss') ? table.slice(0, -1) : table);
|
|
101
|
+
const ORDER = ['breaks-old-code', 'destroys-data', 'locks-writes', 'slow', 'irreversible', 'fine'];
|
|
102
|
+
const TITLES = {
|
|
103
|
+
'breaks-old-code': 'Breaks the version that is still running',
|
|
104
|
+
'destroys-data': 'Destroys data',
|
|
105
|
+
'locks-writes': 'Blocks writes while it runs',
|
|
106
|
+
slow: 'Holds a long transaction',
|
|
107
|
+
irreversible: 'Cannot be undone',
|
|
108
|
+
fine: 'Worth knowing',
|
|
109
|
+
};
|
|
110
|
+
export function renderSchema(migrations, used) {
|
|
111
|
+
const all = migrations.flatMap((m) => m.findings);
|
|
112
|
+
const serious = all.filter((f) => f.risk === 'breaks-old-code' || f.risk === 'destroys-data');
|
|
113
|
+
const lines = [
|
|
114
|
+
'# What this migration does to a live database',
|
|
115
|
+
'',
|
|
116
|
+
migrations.length === 0
|
|
117
|
+
? 'No migration or schema file in this change.'
|
|
118
|
+
: `${migrations.filter((m) => !m.unread).length} of ${migrations.length} migration file(s) read. **${serious.length}** change(s) would break the running version or destroy data; ${all.length - serious.length} other finding(s).`,
|
|
119
|
+
'',
|
|
120
|
+
// "Read, and nothing found" must never look like "not read". A reader who cannot tell them apart will
|
|
121
|
+
// take silence for safety, which is the one thing this report must not allow.
|
|
122
|
+
...(migrations.length > 0 && all.length === 0 && migrations.every((m) => !m.unread)
|
|
123
|
+
? [`All ${migrations.length} were read, and nothing in them breaks the running version, destroys data, blocks writes or holds a long transaction.`]
|
|
124
|
+
: []),
|
|
125
|
+
'',
|
|
126
|
+
];
|
|
127
|
+
for (const risk of ORDER) {
|
|
128
|
+
const group = all.filter((f) => f.risk === risk);
|
|
129
|
+
if (group.length === 0)
|
|
130
|
+
continue;
|
|
131
|
+
lines.push(`## ${TITLES[risk]} (${group.length})`, '');
|
|
132
|
+
for (const finding of group) {
|
|
133
|
+
const where = migrations.find((m) => m.findings.includes(finding));
|
|
134
|
+
lines.push(`- **${finding.what}** — \`${where?.file ?? ''}:${finding.line}\``);
|
|
135
|
+
lines.push(` - \`${finding.statement.replace(/`/g, "'")}\``);
|
|
136
|
+
lines.push(` - Instead: ${finding.instead}`);
|
|
137
|
+
}
|
|
138
|
+
lines.push('');
|
|
139
|
+
}
|
|
140
|
+
if (used.length > 0) {
|
|
141
|
+
lines.push(`## Code that still uses what this takes away (${used.length})`, '', 'Found by searching the repository, so it is evidence rather than proof - a match may be a comment or', 'an unrelated name. Each one is worth opening before this ships.', '');
|
|
142
|
+
for (const u of used) {
|
|
143
|
+
lines.push(`- **${u.column ? `${u.table}.${u.column}` : u.table}** is still named in ${u.files.length} file(s): ${u.files.slice(0, 6).join(', ')}${u.files.length > 6 ? `, and ${u.files.length - 6} more` : ''}`);
|
|
144
|
+
}
|
|
145
|
+
lines.push('');
|
|
146
|
+
}
|
|
147
|
+
const irreversibles = migrations.filter((m) => m.irreversible);
|
|
148
|
+
if (irreversibles.length > 0) {
|
|
149
|
+
lines.push(`## Cannot be rolled back by its own down step (${irreversibles.length})`, '', ...irreversibles.map((m) => `- \`${m.file}\``), '');
|
|
150
|
+
}
|
|
151
|
+
const unread = migrations.filter((m) => m.unread);
|
|
152
|
+
if (unread.length > 0) {
|
|
153
|
+
lines.push(`## Not read (${unread.length})`, '', ...unread.map((m) => `- \`${m.file}\`: ${m.unread}`), '');
|
|
154
|
+
}
|
|
155
|
+
lines.push('---', '', 'Every line above is read from the migration itself; no model was called and nothing was guessed.', '', 'What this does not know: how large your tables are, how busy the database is, and therefore how long', 'any of this takes. It says what a statement does - rewrites the table, scans every row, blocks writes -', 'and leaves the duration to the person who knows the data. It also cannot see a deployment order, so', '"breaks the running version" means during a rolling deploy, which is the usual case.');
|
|
156
|
+
return lines.join(String.fromCharCode(10));
|
|
157
|
+
}
|
|
158
|
+
export function runSchema(cmd) {
|
|
159
|
+
const files = schemaFiles(cmd);
|
|
160
|
+
const migrations = files.map((file) => readMigration(cmd.root, file));
|
|
161
|
+
const used = stillUsed(cmd.root, migrations.flatMap((m) => m.findings));
|
|
162
|
+
const report = renderSchema(migrations, used);
|
|
163
|
+
console.log(report);
|
|
164
|
+
if (cmd.out) {
|
|
165
|
+
mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
|
|
166
|
+
writeFileSync(resolve(cmd.out), report + String.fromCharCode(10), 'utf8');
|
|
167
|
+
}
|
|
168
|
+
const serious = migrations.flatMap((m) => m.findings).filter((f) => f.risk === 'breaks-old-code' || f.risk === 'destroys-data').length;
|
|
169
|
+
if (cmd.strict && serious > 0) {
|
|
170
|
+
console.error(String.fromCharCode(10) + `${serious} change(s) here would break the running version or destroy data.`);
|
|
171
|
+
process.exitCode = 1;
|
|
172
|
+
}
|
|
173
|
+
return { serious };
|
|
174
|
+
}
|
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
const CLEAN = (s) => s.replace(/\s+/g, ' ').trim();
|
|
2
|
+
const NAME = String.raw `[\w."\[\]` + "`" + String.raw `]+`;
|
|
3
|
+
/**
|
|
4
|
+
* What a statement does. Ordered: the first rule that matches decides, so the most serious reading of a
|
|
5
|
+
* statement wins over a milder one.
|
|
6
|
+
*/
|
|
7
|
+
const RULES = [
|
|
8
|
+
{
|
|
9
|
+
find: new RegExp(String.raw `\bDROP\s+TABLE\s+(?:IF\s+EXISTS\s+)?(${NAME})`, 'i'),
|
|
10
|
+
risk: 'destroys-data',
|
|
11
|
+
what: (m) => `drops the table ${bare(m[1])} and everything in it`,
|
|
12
|
+
instead: 'stop writing to it, ship that, confirm nothing reads it, and drop it in a later release - a drop cannot be rolled back by deploying the old code again',
|
|
13
|
+
table: (m) => bare(m[1]),
|
|
14
|
+
},
|
|
15
|
+
{
|
|
16
|
+
find: new RegExp(String.raw `\bTRUNCATE\s+(?:TABLE\s+)?(${NAME})`, 'i'),
|
|
17
|
+
risk: 'destroys-data',
|
|
18
|
+
what: (m) => `empties the table ${bare(m[1])}`,
|
|
19
|
+
instead: 'delete in batches if this is cleanup, or keep the rows and stop reading them',
|
|
20
|
+
table: (m) => bare(m[1]),
|
|
21
|
+
},
|
|
22
|
+
{
|
|
23
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})[\s\S]*?\bDROP\s+(?:COLUMN\s+)?(?:IF\s+EXISTS\s+)?(${NAME})`, 'i'),
|
|
24
|
+
risk: 'breaks-old-code',
|
|
25
|
+
what: (m) => `drops the column ${bare(m[2])} from ${bare(m[1])}`,
|
|
26
|
+
instead: 'deploy code that no longer selects or writes the column, then drop it in a later migration. During a rolling deploy the old version is still running and will fail on a column that is gone',
|
|
27
|
+
table: (m) => bare(m[1]),
|
|
28
|
+
column: (m) => bare(m[2]),
|
|
29
|
+
},
|
|
30
|
+
{
|
|
31
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+RENAME\s+(?:COLUMN\s+)?(${NAME})\s+TO\s+(${NAME})`, 'i'),
|
|
32
|
+
risk: 'breaks-old-code',
|
|
33
|
+
what: (m) => `renames ${bare(m[1])}.${bare(m[2])} to ${bare(m[3])}`,
|
|
34
|
+
instead: 'add the new column, write to both, move readers over, then remove the old one. A rename is a drop and an add at the same instant, and no deployment is that instant',
|
|
35
|
+
table: (m) => bare(m[1]),
|
|
36
|
+
column: (m) => bare(m[2]),
|
|
37
|
+
},
|
|
38
|
+
{
|
|
39
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+RENAME\s+TO\s+(${NAME})`, 'i'),
|
|
40
|
+
risk: 'breaks-old-code',
|
|
41
|
+
what: (m) => `renames the table ${bare(m[1])} to ${bare(m[2])}`,
|
|
42
|
+
instead: 'create a view with the old name, or move readers first. Every query naming the old table fails the moment this runs',
|
|
43
|
+
table: (m) => bare(m[1]),
|
|
44
|
+
},
|
|
45
|
+
{
|
|
46
|
+
find: new RegExp(String.raw `\bCREATE\s+(?:UNIQUE\s+)?INDEX\s+(?!CONCURRENTLY)(?:IF\s+NOT\s+EXISTS\s+)?(${NAME})?\s*ON\s+(${NAME})`, 'i'),
|
|
47
|
+
risk: 'locks-writes',
|
|
48
|
+
what: (m) => `builds an index on ${bare(m[2])} while holding a lock that blocks writes to it`,
|
|
49
|
+
instead: 'CREATE INDEX CONCURRENTLY (Postgres), or ALGORITHM=INPLACE LOCK=NONE (MySQL). On a large table the plain form blocks every write until it finishes',
|
|
50
|
+
table: (m) => bare(m[2]),
|
|
51
|
+
},
|
|
52
|
+
{
|
|
53
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+(?:ALTER|MODIFY)\s+(?:COLUMN\s+)?(${NAME})[\s\S]*?\bSET\s+NOT\s+NULL`, 'i'),
|
|
54
|
+
risk: 'locks-writes',
|
|
55
|
+
what: (m) => `makes ${bare(m[1])}.${bare(m[2])} NOT NULL, which reads every existing row under a lock`,
|
|
56
|
+
instead: 'add a CHECK (column IS NOT NULL) NOT VALID, VALIDATE CONSTRAINT separately, then set NOT NULL (Postgres 12+). The direct form scans the whole table while blocking writes',
|
|
57
|
+
table: (m) => bare(m[1]),
|
|
58
|
+
column: (m) => bare(m[2]),
|
|
59
|
+
},
|
|
60
|
+
{
|
|
61
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+(?:ALTER|MODIFY|CHANGE)\s+(?:COLUMN\s+)?(${NAME})[\s\S]*?\b(?:TYPE|SET\s+DATA\s+TYPE)\b`, 'i'),
|
|
62
|
+
risk: 'locks-writes',
|
|
63
|
+
what: (m) => `changes the type of ${bare(m[1])}.${bare(m[2])}, which usually rewrites the whole table`,
|
|
64
|
+
instead: 'add a new column of the new type, backfill in batches, switch the readers, then drop the old one. A type change holds an exclusive lock for the rewrite',
|
|
65
|
+
table: (m) => bare(m[1]),
|
|
66
|
+
column: (m) => bare(m[2]),
|
|
67
|
+
},
|
|
68
|
+
{
|
|
69
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+ADD\s+(?:CONSTRAINT\s+${NAME}\s+)?FOREIGN\s+KEY(?![\s\S]*NOT\s+VALID)`, 'i'),
|
|
70
|
+
risk: 'locks-writes',
|
|
71
|
+
what: (m) => `adds a foreign key to ${bare(m[1])}, which checks every existing row under a lock on both tables`,
|
|
72
|
+
instead: 'ADD CONSTRAINT ... NOT VALID, then VALIDATE CONSTRAINT, which takes a weaker lock',
|
|
73
|
+
table: (m) => bare(m[1]),
|
|
74
|
+
},
|
|
75
|
+
{
|
|
76
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+ADD\s+(?:COLUMN\s+)?(${NAME})[\s\S]*?\bNOT\s+NULL\b(?![\s\S]*\bDEFAULT\b)`, 'i'),
|
|
77
|
+
risk: 'breaks-old-code',
|
|
78
|
+
what: (m) => `adds ${bare(m[2])} to ${bare(m[1])} as NOT NULL with no default`,
|
|
79
|
+
instead: 'add it nullable or with a default. Without one, every insert from code that does not know about the column fails - including the version still running during the deploy',
|
|
80
|
+
table: (m) => bare(m[1]),
|
|
81
|
+
column: (m) => bare(m[2]),
|
|
82
|
+
},
|
|
83
|
+
{
|
|
84
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})\s+ADD\s+(?:CONSTRAINT\s+${NAME}\s+)?UNIQUE`, 'i'),
|
|
85
|
+
risk: 'locks-writes',
|
|
86
|
+
what: (m) => `adds a unique constraint to ${bare(m[1])}, which builds an index under a lock`,
|
|
87
|
+
instead: 'CREATE UNIQUE INDEX CONCURRENTLY first, then add the constraint using that index',
|
|
88
|
+
table: (m) => bare(m[1]),
|
|
89
|
+
},
|
|
90
|
+
{
|
|
91
|
+
find: /\bVACUUM\s+FULL\b|\bCLUSTER\b|\bREINDEX\b(?!\s+CONCURRENTLY)/i,
|
|
92
|
+
risk: 'locks-writes',
|
|
93
|
+
what: () => 'rewrites the table or index while holding an exclusive lock on it',
|
|
94
|
+
instead: 'run it in a maintenance window, or use the CONCURRENTLY form where the database offers one',
|
|
95
|
+
},
|
|
96
|
+
{
|
|
97
|
+
find: new RegExp(String.raw `\bUPDATE\s+(${NAME})\s+SET\b(?![\s\S]*\bWHERE\b)`, 'i'),
|
|
98
|
+
risk: 'slow',
|
|
99
|
+
what: (m) => `updates every row of ${bare(m[1])} in one statement`,
|
|
100
|
+
instead: 'update in batches with a WHERE on the primary key. One statement over a large table holds its locks and its transaction open until it finishes',
|
|
101
|
+
table: (m) => bare(m[1]),
|
|
102
|
+
},
|
|
103
|
+
{
|
|
104
|
+
find: new RegExp(String.raw `\bDELETE\s+FROM\s+(${NAME})\b(?![\s\S]*\bWHERE\b)`, 'i'),
|
|
105
|
+
risk: 'destroys-data',
|
|
106
|
+
what: (m) => `deletes every row of ${bare(m[1])}`,
|
|
107
|
+
instead: 'say what should go with a WHERE, or keep the rows and stop reading them',
|
|
108
|
+
table: (m) => bare(m[1]),
|
|
109
|
+
},
|
|
110
|
+
];
|
|
111
|
+
/** `"users"` and `` `users` `` and `[users]` are all the table users. */
|
|
112
|
+
const bare = (name) => name.replace(/["`[\]]/g, '').trim();
|
|
113
|
+
/** The statements of a SQL file, with the line each one starts on and comments removed. */
|
|
114
|
+
export function statementsOf(sql) {
|
|
115
|
+
// Comments are blanked rather than removed, so every remaining character keeps its original position
|
|
116
|
+
// and a statement can be reported on the line a reader will actually find it. Deleting them instead
|
|
117
|
+
// shifted everything upwards: a statement on line 3 was reported as line 1.
|
|
118
|
+
const blanked = sql
|
|
119
|
+
.split(String.fromCharCode(10))
|
|
120
|
+
.map((line) => line.replace(/--.*$/, (m) => ' '.repeat(m.length)))
|
|
121
|
+
.join(String.fromCharCode(10))
|
|
122
|
+
// A block comment keeps its newlines so later lines are still numbered correctly.
|
|
123
|
+
.replace(/\/\*[\s\S]*?\*\//g, (m) => m.split(String.fromCharCode(10)).map((l) => ' '.repeat(l.length)).join(String.fromCharCode(10)));
|
|
124
|
+
const lineOf = (index) => blanked.slice(0, index).split(String.fromCharCode(10)).length;
|
|
125
|
+
const out = [];
|
|
126
|
+
let start = 0;
|
|
127
|
+
for (let i = 0; i <= blanked.length; i += 1) {
|
|
128
|
+
if (i < blanked.length && blanked[i] !== ';')
|
|
129
|
+
continue;
|
|
130
|
+
const segment = blanked.slice(start, i);
|
|
131
|
+
const text = CLEAN(segment);
|
|
132
|
+
if (text) {
|
|
133
|
+
// Where the statement itself begins, not where the segment did.
|
|
134
|
+
const offset = segment.length - segment.replace(/^\s+/, '').length;
|
|
135
|
+
out.push({ text, line: lineOf(start + offset) });
|
|
136
|
+
}
|
|
137
|
+
start = i + 1;
|
|
138
|
+
}
|
|
139
|
+
return out;
|
|
140
|
+
}
|
|
141
|
+
/** What this one statement does, or nothing when it is not a statement worth a word. */
|
|
142
|
+
export function judge(statement, line) {
|
|
143
|
+
for (const rule of RULES) {
|
|
144
|
+
const hit = rule.find.exec(statement);
|
|
145
|
+
if (!hit)
|
|
146
|
+
continue;
|
|
147
|
+
return {
|
|
148
|
+
statement: statement.slice(0, 300),
|
|
149
|
+
line,
|
|
150
|
+
risk: rule.risk,
|
|
151
|
+
what: rule.what(hit),
|
|
152
|
+
instead: rule.instead,
|
|
153
|
+
table: rule.table?.(hit) ?? '',
|
|
154
|
+
column: rule.column?.(hit) ?? '',
|
|
155
|
+
};
|
|
156
|
+
}
|
|
157
|
+
return null;
|
|
158
|
+
}
|
|
159
|
+
/** Every finding in a SQL migration. */
|
|
160
|
+
export function readSql(sql) {
|
|
161
|
+
return statementsOf(sql)
|
|
162
|
+
.map(({ text, line }) => judge(text, line))
|
|
163
|
+
.filter((f) => f !== null);
|
|
164
|
+
}
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
// `tewip standards`: what this project has taught Tewip, and what of it can actually be checked.
|
|
2
|
+
//
|
|
3
|
+
// Loading is on demand and cheap: only the name and description of each standard are read until
|
|
4
|
+
// something needs the rest, which is the same discipline a skill file uses. The report leads with the
|
|
5
|
+
// split that matters - rules Tewip enforces, and guidance it can only carry - because a team that
|
|
6
|
+
// believes an unenforceable rule is being enforced is worse off than a team with no tool at all.
|
|
7
|
+
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
|
|
8
|
+
import { dirname, join, resolve } from 'node:path';
|
|
9
|
+
import { changeSummary } from '../quality/grounding.js';
|
|
10
|
+
import { check, readStandard, renderViolations } from './standards.js';
|
|
11
|
+
export const STANDARDS_DIR = join('.tewip', 'standards');
|
|
12
|
+
/** Every standard this project has written, newest name order, with unreadable ones reported. */
|
|
13
|
+
export function loadStandards(root) {
|
|
14
|
+
const dir = join(root, STANDARDS_DIR);
|
|
15
|
+
if (!existsSync(dir))
|
|
16
|
+
return [];
|
|
17
|
+
const out = [];
|
|
18
|
+
for (const entry of readdirSync(dir).filter((f) => /\.(md|markdown)$/i.test(f)).sort()) {
|
|
19
|
+
try {
|
|
20
|
+
out.push(readStandard(entry, readFileSync(join(dir, entry), 'utf8')));
|
|
21
|
+
}
|
|
22
|
+
catch {
|
|
23
|
+
out.push({ name: entry, description: '', appliesTo: [], rules: [], guidance: '', unusable: 'the file could not be read' });
|
|
24
|
+
}
|
|
25
|
+
}
|
|
26
|
+
return out;
|
|
27
|
+
}
|
|
28
|
+
/**
|
|
29
|
+
* The guidance to hand a model when writing or fixing something in this file.
|
|
30
|
+
*
|
|
31
|
+
* Only the standards that apply, and only their prose: the checkable rules are enforced afterwards by
|
|
32
|
+
* `check`, so repeating them in a prompt would be asking politely for something that is about to be
|
|
33
|
+
* verified anyway.
|
|
34
|
+
*/
|
|
35
|
+
export function guidanceFor(standards, file) {
|
|
36
|
+
const relevant = standards.filter((s) => s.guidance.trim() && covers(s.appliesTo, file));
|
|
37
|
+
if (relevant.length === 0)
|
|
38
|
+
return '';
|
|
39
|
+
return relevant.map((s) => `## ${s.name}${s.description ? ` - ${s.description}` : ''}${String.fromCharCode(10)}${s.guidance.trim()}`).join(String.fromCharCode(10) + String.fromCharCode(10)).slice(0, 6000);
|
|
40
|
+
}
|
|
41
|
+
const covers = (where, file) => where.length === 0 || where.some((p) => file.split('\\').join('/').includes(p.replace(/^\*\*?\//, '').replace(/\*/g, '')));
|
|
42
|
+
/** Violations in the files a change touched. */
|
|
43
|
+
export function checkChanged(root, standards, files) {
|
|
44
|
+
const out = [];
|
|
45
|
+
for (const file of files) {
|
|
46
|
+
const full = resolve(root, file);
|
|
47
|
+
if (!existsSync(full))
|
|
48
|
+
continue;
|
|
49
|
+
try {
|
|
50
|
+
out.push(...check(standards, file.split('\\').join('/'), readFileSync(full, 'utf8')));
|
|
51
|
+
}
|
|
52
|
+
catch {
|
|
53
|
+
// Unreadable: nothing can be said about it.
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
return out;
|
|
57
|
+
}
|
|
58
|
+
export function renderStandards(standards, violations, checkedFiles) {
|
|
59
|
+
if (standards.length === 0) {
|
|
60
|
+
return [
|
|
61
|
+
'# This project has taught Tewip nothing yet',
|
|
62
|
+
'',
|
|
63
|
+
`Put a markdown file in \`${STANDARDS_DIR.split('\\').join('/')}/\` and Tewip will read it. For example, \`no-fixed-waits.md\`:`,
|
|
64
|
+
'',
|
|
65
|
+
'```markdown',
|
|
66
|
+
'---',
|
|
67
|
+
'description: A fixed wait hides a race instead of fixing it.',
|
|
68
|
+
'applies-to: tests/, e2e/',
|
|
69
|
+
'forbid: /waitForTimeout|sleep\\(/ because a fixed wait passes by luck and fails in CI',
|
|
70
|
+
'---',
|
|
71
|
+
'',
|
|
72
|
+
'Wait for the thing you are actually waiting for - an element, a response, a state -',
|
|
73
|
+
'never for a number of milliseconds.',
|
|
74
|
+
'```',
|
|
75
|
+
'',
|
|
76
|
+
'The `forbid` and `require` lines are checked and enforced. The prose below them is guidance: Tewip',
|
|
77
|
+
'passes it on when it writes or fixes something, and never claims to have verified it.',
|
|
78
|
+
].join(String.fromCharCode(10));
|
|
79
|
+
}
|
|
80
|
+
const checkable = standards.filter((s) => s.rules.length > 0);
|
|
81
|
+
const guidanceOnly = standards.filter((s) => s.rules.length === 0 && s.guidance.trim());
|
|
82
|
+
const lines = [
|
|
83
|
+
'# What this project has taught Tewip',
|
|
84
|
+
'',
|
|
85
|
+
`${standards.length} standard(s) loaded: ${checkable.length} with rules Tewip can check, ${guidanceOnly.length} guidance only.`,
|
|
86
|
+
...(checkedFiles > 0 ? ['', `${checkedFiles} changed file(s) checked. **${violations.length}** broke a rule.`] : []),
|
|
87
|
+
'',
|
|
88
|
+
'## Checked and enforced',
|
|
89
|
+
'',
|
|
90
|
+
];
|
|
91
|
+
if (checkable.length === 0)
|
|
92
|
+
lines.push('None. No standard here carries a `forbid:` or `require:` rule, so nothing is enforced.', '');
|
|
93
|
+
for (const standard of checkable) {
|
|
94
|
+
lines.push(`- **${standard.name}**${standard.description ? ` — ${standard.description}` : ''}`);
|
|
95
|
+
for (const rule of standard.rules) {
|
|
96
|
+
lines.push(` - ${rule.kind} \`${rule.pattern}\`${rule.where.length > 0 ? ` in ${rule.where.join(', ')}` : ''}${rule.because ? ` — ${rule.because}` : ''}`);
|
|
97
|
+
}
|
|
98
|
+
}
|
|
99
|
+
lines.push('');
|
|
100
|
+
if (guidanceOnly.length > 0) {
|
|
101
|
+
lines.push('## Carried, but not checked', '', 'Tewip passes these to the model when it writes or fixes something, and quotes them in reports. It', 'cannot verify them, and it will never report them as kept.', '', ...guidanceOnly.map((s) => `- **${s.name}**${s.description ? ` — ${s.description}` : ''}`), '');
|
|
102
|
+
}
|
|
103
|
+
const unusable = standards.filter((s) => s.unusable);
|
|
104
|
+
if (unusable.length > 0) {
|
|
105
|
+
lines.push(`## Could not be used (${unusable.length})`, '', ...unusable.map((s) => `- **${s.name}**: ${s.unusable}`), '');
|
|
106
|
+
}
|
|
107
|
+
lines.push(...renderViolations(violations, standards));
|
|
108
|
+
lines.push('---', '', 'A rule is enforced only if it appears above under "checked and enforced". Everything else is prose', 'Tewip carries for you - useful, and not the same thing as a check.');
|
|
109
|
+
return lines.join(String.fromCharCode(10));
|
|
110
|
+
}
|
|
111
|
+
export function runStandards(cmd) {
|
|
112
|
+
const standards = loadStandards(cmd.root);
|
|
113
|
+
const files = cmd.files && cmd.files.length > 0 ? cmd.files : cmd.diff ? (changeSummary(cmd.root, cmd.diff)?.files ?? []) : [];
|
|
114
|
+
const violations = files.length > 0 ? checkChanged(cmd.root, standards, files.map((f) => f.split('\\').join('/'))) : [];
|
|
115
|
+
const report = renderStandards(standards, violations, files.length);
|
|
116
|
+
console.log(report);
|
|
117
|
+
if (cmd.out) {
|
|
118
|
+
mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
|
|
119
|
+
writeFileSync(resolve(cmd.out), report + String.fromCharCode(10), 'utf8');
|
|
120
|
+
}
|
|
121
|
+
if (cmd.strict && violations.length > 0) {
|
|
122
|
+
console.error(String.fromCharCode(10) + `${violations.length} violation(s) of this project's own standards.`);
|
|
123
|
+
process.exitCode = 1;
|
|
124
|
+
}
|
|
125
|
+
return { violations: violations.length };
|
|
126
|
+
}
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Reads one standard.
|
|
3
|
+
*
|
|
4
|
+
* The format is deliberately plain: a markdown file with a small front matter block. A team should be
|
|
5
|
+
* able to write one without reading a manual, and anything Tewip cannot understand is reported as
|
|
6
|
+
* unusable rather than silently ignored - a rule nobody is enforcing is worse than no rule, because
|
|
7
|
+
* somebody believes in it.
|
|
8
|
+
*/
|
|
9
|
+
export function readStandard(file, text) {
|
|
10
|
+
const name = file.split(/[\\/]/).pop()?.replace(/\.md$/i, '') ?? file;
|
|
11
|
+
const front = /^---\r?\n([\s\S]*?)\r?\n---/.exec(text);
|
|
12
|
+
const body = front ? text.slice(front[0].length).trim() : text.trim();
|
|
13
|
+
const standard = { name, description: '', appliesTo: [], rules: [], guidance: body, unusable: '' };
|
|
14
|
+
if (!front) {
|
|
15
|
+
// Prose only. That is allowed, and it is guidance - but say so, so nobody thinks it is enforced.
|
|
16
|
+
standard.description = firstLine(body);
|
|
17
|
+
return standard;
|
|
18
|
+
}
|
|
19
|
+
for (const line of front[1].split(String.fromCharCode(10))) {
|
|
20
|
+
const entry = /^\s*([a-z-]+)\s*:\s*(.*)$/i.exec(line);
|
|
21
|
+
if (!entry)
|
|
22
|
+
continue;
|
|
23
|
+
const [, key, rawValue] = entry;
|
|
24
|
+
const value = rawValue.trim();
|
|
25
|
+
if (key === 'name' && value)
|
|
26
|
+
standard.name = value;
|
|
27
|
+
else if (key === 'description')
|
|
28
|
+
standard.description = value;
|
|
29
|
+
else if (key === 'applies-to')
|
|
30
|
+
standard.appliesTo = list(value);
|
|
31
|
+
else if (key === 'forbid' || key === 'require') {
|
|
32
|
+
const rule = parseRule(key, value, standard.appliesTo);
|
|
33
|
+
if (rule)
|
|
34
|
+
standard.rules.push(rule);
|
|
35
|
+
else
|
|
36
|
+
standard.unusable = `the ${key} on this line could not be read: ${line.trim().slice(0, 120)}`;
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
if (!standard.description)
|
|
40
|
+
standard.description = firstLine(body);
|
|
41
|
+
return standard;
|
|
42
|
+
}
|
|
43
|
+
/** `forbid: /waitForTimeout/ in tests because a fixed wait hides a race` */
|
|
44
|
+
function parseRule(kind, value, scope) {
|
|
45
|
+
const shape = /^\/(.+?)\/([a-z]*)\s*(?:\bin\s+([^]*?))?\s*(?:\bbecause\s+([^]*))?$/i.exec(value.trim());
|
|
46
|
+
if (!shape)
|
|
47
|
+
return null;
|
|
48
|
+
try {
|
|
49
|
+
new RegExp(shape[1], shape[2] || undefined); // it has to be a pattern before it is a rule
|
|
50
|
+
}
|
|
51
|
+
catch {
|
|
52
|
+
return null;
|
|
53
|
+
}
|
|
54
|
+
return {
|
|
55
|
+
kind,
|
|
56
|
+
pattern: shape[2] ? `/${shape[1]}/${shape[2]}` : shape[1],
|
|
57
|
+
where: shape[3] ? list(shape[3]) : scope,
|
|
58
|
+
because: (shape[4] ?? '').trim(),
|
|
59
|
+
};
|
|
60
|
+
}
|
|
61
|
+
const list = (value) => value
|
|
62
|
+
.replace(/^\[|\]$/g, '')
|
|
63
|
+
.split(/[,\s]+/)
|
|
64
|
+
.map((v) => v.trim().replace(/^["']|["']$/g, ''))
|
|
65
|
+
.filter(Boolean);
|
|
66
|
+
const firstLine = (text) => text.split(String.fromCharCode(10)).map((l) => l.replace(/^#+\s*/, '').trim()).find(Boolean)?.slice(0, 160) ?? '';
|
|
67
|
+
/** Does this file fall under that scope? A suffix, an extension, or a path fragment. */
|
|
68
|
+
export function covers(where, file) {
|
|
69
|
+
if (where.length === 0)
|
|
70
|
+
return true;
|
|
71
|
+
const path = file.split('\\').join('/');
|
|
72
|
+
return where.some((pattern) => {
|
|
73
|
+
const clean = pattern.replace(/^\*\*?\//, '').replace(/\*/g, '');
|
|
74
|
+
return clean !== '' && path.includes(clean);
|
|
75
|
+
});
|
|
76
|
+
}
|
|
77
|
+
const regexOf = (rule) => {
|
|
78
|
+
const slashed = /^\/(.+)\/([a-z]*)$/.exec(rule.pattern);
|
|
79
|
+
return slashed ? new RegExp(slashed[1], slashed[2]) : new RegExp(rule.pattern);
|
|
80
|
+
};
|
|
81
|
+
/**
|
|
82
|
+
* The rules this file breaks.
|
|
83
|
+
*
|
|
84
|
+
* `forbid` is checked line by line, so the violation can be quoted. `require` is checked over the whole
|
|
85
|
+
* file, because "this file must mention X" is a statement about the file and pointing at a line would be
|
|
86
|
+
* inventing a location.
|
|
87
|
+
*/
|
|
88
|
+
export function check(standards, file, text) {
|
|
89
|
+
const out = [];
|
|
90
|
+
const lines = text.split(String.fromCharCode(10));
|
|
91
|
+
for (const standard of standards) {
|
|
92
|
+
if (!covers(standard.appliesTo, file))
|
|
93
|
+
continue;
|
|
94
|
+
for (const rule of standard.rules) {
|
|
95
|
+
if (!covers(rule.where, file))
|
|
96
|
+
continue;
|
|
97
|
+
let pattern;
|
|
98
|
+
try {
|
|
99
|
+
pattern = regexOf(rule);
|
|
100
|
+
}
|
|
101
|
+
catch {
|
|
102
|
+
continue;
|
|
103
|
+
}
|
|
104
|
+
if (rule.kind === 'forbid') {
|
|
105
|
+
lines.forEach((line, index) => {
|
|
106
|
+
if (pattern.test(line))
|
|
107
|
+
out.push({ standard: standard.name, file, line: index + 1, text: line.trim().slice(0, 160), rule });
|
|
108
|
+
});
|
|
109
|
+
}
|
|
110
|
+
else if (!pattern.test(text)) {
|
|
111
|
+
out.push({ standard: standard.name, file, line: 0, text: '', rule });
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
return out;
|
|
116
|
+
}
|
|
117
|
+
/** The section for a review, a pull request comment or a verdict. Empty when nothing is broken. */
|
|
118
|
+
export function renderViolations(violations, loaded) {
|
|
119
|
+
if (violations.length === 0)
|
|
120
|
+
return [];
|
|
121
|
+
const lines = [`## Your own standards (${violations.length})`, ''];
|
|
122
|
+
for (const v of violations.slice(0, 20)) {
|
|
123
|
+
const where = v.line > 0 ? `${v.file}:${v.line}` : v.file;
|
|
124
|
+
lines.push(v.rule.kind === 'forbid'
|
|
125
|
+
? `- **${v.standard}** — \`${where}\` has \`${v.rule.pattern}\`, which this project forbids`
|
|
126
|
+
: `- **${v.standard}** — \`${where}\` does not have \`${v.rule.pattern}\`, which this project requires`);
|
|
127
|
+
if (v.text)
|
|
128
|
+
lines.push(` - \`${v.text.replace(/`/g, "'")}\``);
|
|
129
|
+
if (v.rule.because)
|
|
130
|
+
lines.push(` - ${v.rule.because}`);
|
|
131
|
+
}
|
|
132
|
+
if (violations.length > 20)
|
|
133
|
+
lines.push(`- …and ${violations.length - 20} more`);
|
|
134
|
+
const guidance = loaded.filter((s) => s.guidance.trim().length > 0);
|
|
135
|
+
if (guidance.length > 0) {
|
|
136
|
+
lines.push('', `${guidance.length} standard(s) also carry guidance Tewip cannot check: ${guidance.map((s) => s.name).join(', ')}. It is passed on when Tewip writes or fixes something, and it is not counted here as either kept or broken.`);
|
|
137
|
+
}
|
|
138
|
+
lines.push('');
|
|
139
|
+
return lines;
|
|
140
|
+
}
|
|
@@ -13,6 +13,7 @@ import { coverageFor, testFiles } from '../review/coverage.js';
|
|
|
13
13
|
import { importingTests } from '../prove/prove.js';
|
|
14
14
|
import { testsReaching } from '../prove/reaches.js';
|
|
15
15
|
import { vouchChanged } from '../vouch/changed.js';
|
|
16
|
+
import { checkChanged, loadStandards } from '../standards/standards-command.js';
|
|
16
17
|
import { datasetFiles, readJsonl, readTriageRecords } from '../../reporters/store.js';
|
|
17
18
|
import { decide, renderDecision } from './verdict.js';
|
|
18
19
|
const SOURCE = /\.(js|jsx|mjs|cjs|ts|tsx|mts|cts|py|java|kt|go|cs|rb|php)$/i;
|
|
@@ -97,6 +98,8 @@ export function gather(cmd) {
|
|
|
97
98
|
return touched.has(where) || [...touched].some((t) => r.testId.includes(t.replace(/\.[a-z]+$/, '')));
|
|
98
99
|
})
|
|
99
100
|
.map((r) => ({ testId: r.testId, category: String(r.diagnosis?.category ?? 'UNKNOWN') }));
|
|
101
|
+
const standards = loadStandards(cmd.root);
|
|
102
|
+
const brokenRules = standards.length === 0 ? null : checkChanged(cmd.root, standards, changed).map((v) => ({ standard: v.standard, file: v.file, line: v.line, why: v.rule.because }));
|
|
100
103
|
const proved = cmd.prove ? proveChanged(cmd.root, changedSource, { budgetSeconds: cmd.proveBudgetSeconds ?? 300, onStep: (m) => console.error(` ${m}`) }) : null;
|
|
101
104
|
return {
|
|
102
105
|
changed,
|
|
@@ -110,6 +113,7 @@ export function gather(cmd) {
|
|
|
110
113
|
failingNow,
|
|
111
114
|
noHistory: runs.length === 0,
|
|
112
115
|
uncovered: uncoveredFor(cmd.root, changed, Object.keys(newest?.attempts ?? {})),
|
|
116
|
+
brokenRules,
|
|
113
117
|
},
|
|
114
118
|
};
|
|
115
119
|
}
|
|
@@ -76,7 +76,23 @@ export function decide(evidence) {
|
|
|
76
76
|
else {
|
|
77
77
|
checks.push({ question: 'Would the tests catch a bug in this code?', standing: 'met', answer: 'every deliberate change made to this code was caught by a covering test', decidedBy: 'tewip prove' });
|
|
78
78
|
}
|
|
79
|
-
// 5.
|
|
79
|
+
// 5. Does it keep the rules this team wrote down? Only a project that has written some can break them.
|
|
80
|
+
if (evidence.brokenRules === null) {
|
|
81
|
+
checks.push({ question: "Does it keep this project's own standards?", standing: 'not-checked', answer: 'this project has written none for Tewip to read, so there is nothing to keep or break', decidedBy: 'tewip standards' });
|
|
82
|
+
}
|
|
83
|
+
else if (evidence.brokenRules.length > 0) {
|
|
84
|
+
const first = evidence.brokenRules[0];
|
|
85
|
+
checks.push({
|
|
86
|
+
question: "Does it keep this project's own standards?",
|
|
87
|
+
standing: 'not-met',
|
|
88
|
+
answer: `${evidence.brokenRules.length} rule(s) this team wrote are broken, starting with ${first.standard} at ${first.file}${first.line > 0 ? `:${first.line}` : ''}${first.why ? ` - ${first.why}` : ''}`,
|
|
89
|
+
decidedBy: 'tewip standards, which enforces only the rules a pattern can check',
|
|
90
|
+
});
|
|
91
|
+
}
|
|
92
|
+
else {
|
|
93
|
+
checks.push({ question: "Does it keep this project's own standards?", standing: 'met', answer: 'every rule this team wrote and Tewip can check is kept', decidedBy: 'tewip standards' });
|
|
94
|
+
}
|
|
95
|
+
// 6. Is what was asked for covered? Only where the project keeps plans to compare against.
|
|
80
96
|
if (evidence.uncovered === null) {
|
|
81
97
|
checks.push({ question: 'Is what was asked for covered?', standing: 'not-checked', answer: 'this project keeps no test plan Tewip can read, so there is nothing to compare the change against', decidedBy: 'tewip trace' });
|
|
82
98
|
}
|
package/dist/mcp/server.js
CHANGED
|
@@ -197,4 +197,41 @@ server.registerTool('which_tests_to_run', {
|
|
|
197
197
|
return `Nothing changed in ${range}, or that range could not be read here.`;
|
|
198
198
|
return renderPick(chosen.picked, chosen.total, chosen.command, chosen.noHistory);
|
|
199
199
|
}));
|
|
200
|
+
server.registerTool('what_will_this_migration_do', {
|
|
201
|
+
title: 'What will this migration do to a live database?',
|
|
202
|
+
description: "Reads the migrations in a change and names what will happen when they run against a live database: a column dropped while the version still running keeps selecting it, a table dropped, an index built under a lock that blocks writes, a NOT NULL added to a populated table, a type change that rewrites it, a foreign key validated against every row, a data migration holding a transaction open, a step that cannot be undone. It also searches the repository for code that still uses the column being taken away. Reads SQL, Django, Rails and Alembic, judges only the forward path, and says plainly when it cannot read a migration. It does not know your table sizes, so it says what a statement does rather than how long it will take.",
|
|
203
|
+
inputSchema: {
|
|
204
|
+
project,
|
|
205
|
+
diff: z.string().optional().describe('A git range, e.g. main...HEAD (the default)'),
|
|
206
|
+
files: z.array(z.string()).optional().describe('Migration files to read, when they do not come from a git range'),
|
|
207
|
+
},
|
|
208
|
+
annotations: readOnly,
|
|
209
|
+
}, ({ project: root, diff, files }) => answer(async () => {
|
|
210
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
211
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
212
|
+
const { readMigration, renderSchema, schemaFiles, stillUsed } = await import('../engine/schema/schema-command.js');
|
|
213
|
+
const cmd = { root, diff: diff ?? 'main...HEAD', files };
|
|
214
|
+
const migrations = schemaFiles(cmd).map((file) => readMigration(root, file));
|
|
215
|
+
if (migrations.length === 0)
|
|
216
|
+
return 'No migration or schema file in that change.';
|
|
217
|
+
return renderSchema(migrations, stillUsed(root, migrations.flatMap((m) => m.findings)));
|
|
218
|
+
}));
|
|
219
|
+
server.registerTool('project_standards', {
|
|
220
|
+
title: "What are this project's own rules?",
|
|
221
|
+
description: "The house rules this team has written down in .tewip/standards/, and whether a change keeps them. Returns two things kept carefully apart: rules Tewip checks and enforces (a forbid or require pattern over a set of files), and guidance it can only carry - prose for the parts no pattern captures, which is passed on when something is written but never reported as verified. Use it before writing or changing code in somebody's project, so what you produce matches their conventions rather than yours.",
|
|
222
|
+
inputSchema: {
|
|
223
|
+
project,
|
|
224
|
+
files: z.array(z.string()).optional().describe('Files to check against the rules. Omit to list what the project has written.'),
|
|
225
|
+
diff: z.string().optional().describe('A git range whose changed files should be checked instead'),
|
|
226
|
+
},
|
|
227
|
+
annotations: readOnly,
|
|
228
|
+
}, ({ project: root, files, diff }) => answer(async () => {
|
|
229
|
+
if (!isAbsolute(root) || !existsSync(root))
|
|
230
|
+
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
231
|
+
const { checkChanged, loadStandards, renderStandards } = await import('../engine/standards/standards-command.js');
|
|
232
|
+
const { changeSummary } = await import('../engine/quality/grounding.js');
|
|
233
|
+
const standards = loadStandards(root);
|
|
234
|
+
const changed = files && files.length > 0 ? files : diff ? (changeSummary(root, diff)?.files ?? []) : [];
|
|
235
|
+
return renderStandards(standards, changed.length > 0 ? checkChanged(root, standards, changed) : [], changed.length);
|
|
236
|
+
}));
|
|
200
237
|
await server.connect(new StdioServerTransport());
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tewip-ai",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.32.0",
|
|
4
4
|
"description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"test-failures",
|