tewip-ai 0.38.0 → 0.40.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -1
- package/dist/cli/help.js +8 -6
- package/dist/engine/access/access-command.js +27 -126
- package/dist/engine/access/axe-rules.js +128 -0
- package/dist/engine/access/from-lighthouse.js +74 -0
- package/dist/engine/access/render.js +133 -0
- package/dist/engine/access/reports.js +1 -1
- package/dist/engine/perf/lighthouse.js +6 -0
- package/dist/engine/schema/frameworks.js +16 -2
- package/dist/engine/schema/schema-command.js +37 -6
- package/dist/engine/schema/statements.js +6 -1
- package/dist/mcp/tools-prove.js +4 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -750,6 +750,26 @@ A source finding is set aside only when the run opened **that same file** and re
|
|
|
750
750
|
In a React or Vue project that means nothing is set aside — the source and the rendered output are two
|
|
751
751
|
different artefacts, and neither is evidence about the other.
|
|
752
752
|
|
|
753
|
+
**Already run Lighthouse instead? That works too** (Phase 32, new):
|
|
754
|
+
|
|
755
|
+
```bash
|
|
756
|
+
tewip access --report lighthouse-report.json
|
|
757
|
+
```
|
|
758
|
+
|
|
759
|
+
Lighthouse's accessibility audits *are* axe rules — same rule ids — so the same triage applies. Most
|
|
760
|
+
teams have Lighthouse in CI and no separate axe step; they get the whole of the above from a tool they
|
|
761
|
+
already run.
|
|
762
|
+
|
|
763
|
+
The catch was that a Lighthouse report strips axe's tags, so it says `landmark-one-main` and nothing
|
|
764
|
+
about which WCAG criterion that is, or whether it is one at all. That distinction covers **30 of axe's
|
|
765
|
+
105 rules**, and a remembered table of a hundred mappings is a table with mistakes in it. So the table
|
|
766
|
+
is **generated from axe-core itself** (`scripts/axe-rules.ts`) — 105 rules, 75 carrying a criterion, 30
|
|
767
|
+
best-practice only, none unclassified.
|
|
768
|
+
|
|
769
|
+
It also surfaces the **ten audits Lighthouse marks `manual`** — logical tab order, focus traps, whether
|
|
770
|
+
visual order follows the DOM. Those become questions, never passes. A page with no failures and ten
|
|
771
|
+
unanswered manual checks has not passed.
|
|
772
|
+
|
|
753
773
|
## Teach it your house rules (Phase 27)
|
|
754
774
|
|
|
755
775
|
```bash
|
|
@@ -816,7 +836,8 @@ That last section is the part no SQL linter does: it searches your own code for
|
|
|
816
836
|
away. "This drops `users.nickname`, and four files still read it" is a different sentence from "dropping a
|
|
817
837
|
column is risky", and it is the one that stops the outage.
|
|
818
838
|
|
|
819
|
-
It reads **SQL, Django, Rails and
|
|
839
|
+
It reads **SQL, Django, Rails, Alembic and TypeORM** migrations — which covers Prisma, Drizzle, Flyway
|
|
840
|
+
and golang-migrate too, since those all emit plain `.sql`. It judges **only the forward path** (a rollback step
|
|
820
841
|
contains the mirror image of the change, so reading it says a migration that creates a table destroys
|
|
821
842
|
one), and stays quiet when the team already did the safe thing — `atomic = False`, `algorithm:
|
|
822
843
|
:concurrently`, `NOT VALID`, `postgresql_concurrently=True`. When it cannot read a migration it says so
|
package/dist/cli/help.js
CHANGED
|
@@ -130,7 +130,7 @@ Usage:
|
|
|
130
130
|
--record adds a run to the baseline, and is deliberately not
|
|
131
131
|
automatic: a regression recorded by accident becomes next week's
|
|
132
132
|
normal. It never predicts a duration or a capacity. No cost.
|
|
133
|
-
tewip access [<file or directory>] [--report axe.json] [--strict]
|
|
133
|
+
tewip access [<file or directory>] [--report axe.json|lighthouse.json] [--strict]
|
|
134
134
|
Can everybody use this? Reads your markup and names the barriers
|
|
135
135
|
that reading alone can prove - a picture with no alt text, a field
|
|
136
136
|
with no label, a div only a mouse can use - each against its WCAG
|
|
@@ -139,11 +139,13 @@ Usage:
|
|
|
139
139
|
criterion it cannot reach without rendering the page (contrast,
|
|
140
140
|
focus order, whether the alt text is any good) is printed every
|
|
141
141
|
time. It never says a page is accessible, only what was checked.
|
|
142
|
-
Give it --report with the JSON from an axe run
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
142
|
+
Give it --report with the JSON from an axe run or a Lighthouse
|
|
143
|
+
report and it reads that too - Lighthouse runs axe rules, so
|
|
144
|
+
either works. Contrast moves out of the unchecked list, rules
|
|
145
|
+
that map to no WCAG criterion are reported as advice rather than
|
|
146
|
+
barriers (51 of 174 findings on nine real pages), and what the
|
|
147
|
+
tool could not determine becomes a question. Lighthouse's ten
|
|
148
|
+
manual checks are questions too, never passes. No model, no cost.
|
|
147
149
|
tewip standards [--diff main...HEAD] [--strict] [--out standards.md]
|
|
148
150
|
Teach Tewip your house rules. Markdown files in
|
|
149
151
|
.tewip/standards/ - a forbid or require line is checked and
|
|
@@ -14,9 +14,13 @@ import { sourceFiles } from '../secure/secrets.js';
|
|
|
14
14
|
import { JS_LIKE, MARKUP, tags } from './markup.js';
|
|
15
15
|
import { checkFile } from './checks.js';
|
|
16
16
|
import { readAxeReport } from './reports.js';
|
|
17
|
-
import {
|
|
18
|
-
import {
|
|
19
|
-
|
|
17
|
+
import { isLighthouseFile, readLighthouse } from '../perf/lighthouse.js';
|
|
18
|
+
import { accessibilityFromLighthouse, criteriaFromLighthouse, rulesRunByLighthouse } from './from-lighthouse.js';
|
|
19
|
+
import { merge, pagesIn } from './merge.js';
|
|
20
|
+
import { renderAccess, worstFirst } from './render.js';
|
|
21
|
+
// Re-exported so callers do not have to know the report moved into its own file.
|
|
22
|
+
export { renderAccess, worstFirst };
|
|
23
|
+
import { blocks } from './wcag.js';
|
|
20
24
|
/** One line per finding a person has already judged: "<rule> <file>:<line> - reason". */
|
|
21
25
|
export function readAccepted(root) {
|
|
22
26
|
const file = join(root, '.tewip', 'access-accepted.txt');
|
|
@@ -30,15 +34,6 @@ export function readAccepted(root) {
|
|
|
30
34
|
export function keyOf(f) {
|
|
31
35
|
return `${f.rule} ${f.file}:${f.line}`;
|
|
32
36
|
}
|
|
33
|
-
export function worstFirst(a, b) {
|
|
34
|
-
const sa = ORDER[RULES[a.rule]?.severity ?? 'minor'];
|
|
35
|
-
const sb = ORDER[RULES[b.rule]?.severity ?? 'minor'];
|
|
36
|
-
if (sa !== sb)
|
|
37
|
-
return sa - sb;
|
|
38
|
-
if (a.file !== b.file)
|
|
39
|
-
return a.file < b.file ? -1 : 1;
|
|
40
|
-
return a.line - b.line;
|
|
41
|
-
}
|
|
42
37
|
/** Reads every markup file under `root` and returns what the markup alone can settle. */
|
|
43
38
|
/**
|
|
44
39
|
* Markup written to exercise code, not to be used by a person. An `<img>` with no alt in a
|
|
@@ -71,119 +66,24 @@ export function findBarriers(root, only) {
|
|
|
71
66
|
}
|
|
72
67
|
return { findings, filesRead };
|
|
73
68
|
}
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
lines.push(`${criterionOf(rule)}, ${meta?.severity ?? 'unrated'}.`, '');
|
|
93
|
-
if (meta)
|
|
94
|
-
lines.push(`Who it stops: ${meta.stops}.`, '');
|
|
95
|
-
for (const f of found.slice(0, 20)) {
|
|
96
|
-
lines.push(`- \`${f.file}:${f.line}\` ${f.element}`);
|
|
97
|
-
lines.push(` - ${f.why}`);
|
|
98
|
-
lines.push(` - do this: ${f.next}`);
|
|
99
|
-
}
|
|
100
|
-
if (found.length > 20)
|
|
101
|
-
lines.push(`- ...and ${found.length - 20} more of the same, in the JSON beside this report.`);
|
|
102
|
-
lines.push('');
|
|
103
|
-
}
|
|
104
|
-
}
|
|
105
|
-
export function renderAccess(result, scanned) {
|
|
106
|
-
const { barriers, questions, accepted, filesRead, browser } = result;
|
|
107
|
-
const lines = ['# Tewip accessibility check', ''];
|
|
108
|
-
lines.push(`Read ${filesRead} markup ${filesRead === 1 ? 'file' : 'files'} under \`${scanned}\`, without rendering any of them.`, '');
|
|
109
|
-
if (browser) {
|
|
110
|
-
lines.push(`Also read an axe run over ${browser.pages.length} rendered ${browser.pages.length === 1 ? 'page' : 'pages'}, ` +
|
|
111
|
-
'which settles what reading markup cannot.', '');
|
|
112
|
-
}
|
|
113
|
-
if (barriers.length === 0 && questions.length === 0 && !browser) {
|
|
114
|
-
lines.push('No barrier was found that reading the markup can settle, and nothing needs a decision.', '');
|
|
115
|
-
}
|
|
116
|
-
else {
|
|
117
|
-
const stopping = barriers.filter((f) => blocks(f.rule)).length;
|
|
118
|
-
const counts = [
|
|
119
|
-
`${barriers.length} ${barriers.length === 1 ? 'barrier' : 'barriers'} (${stopping} that stop somebody outright)`,
|
|
120
|
-
`${questions.length} for a person to settle`,
|
|
121
|
-
];
|
|
122
|
-
if (accepted.length > 0)
|
|
123
|
-
counts.push(`${accepted.length} already accepted`);
|
|
124
|
-
if (browser) {
|
|
125
|
-
counts.unshift(`${browser.barriers.length} barriers the browser run proves`);
|
|
126
|
-
if (browser.questions.length > 0)
|
|
127
|
-
counts.push(`${browser.questions.length} the browser could not determine`);
|
|
128
|
-
}
|
|
129
|
-
lines.push(`**${counts.join(', ')}.**`, '');
|
|
130
|
-
}
|
|
131
|
-
if (browser) {
|
|
132
|
-
renderBrowserSection('Barriers seen in the browser', browser.barriers, lines);
|
|
133
|
-
renderBrowserSection('The browser could not determine these', browser.questions, lines);
|
|
134
|
-
}
|
|
135
|
-
section('Barriers read from the source', [...barriers].sort(worstFirst), lines);
|
|
136
|
-
section('Only a person can settle these', [...questions].sort(worstFirst), lines);
|
|
137
|
-
if (browser && browser.advice.length > 0) {
|
|
138
|
-
// Kept out of the counts above on purpose. These are axe's own best-practice rules, which test
|
|
139
|
-
// no standard: on nine real pages they were 51 of 174 findings, outnumbering every real barrier.
|
|
140
|
-
// Reporting them as equals is how a team learns to ignore the whole report.
|
|
141
|
-
const rules = [...new Set(browser.advice.map((f) => f.rule))].sort();
|
|
142
|
-
lines.push('## Advice, not barriers', '');
|
|
143
|
-
lines.push(`${browser.advice.length} findings come from axe rules that map to no WCAG criterion - ` +
|
|
144
|
-
`${rules.join(', ')}. They are somebody's good idea rather than a standard, so they are not counted as barriers ` +
|
|
145
|
-
'and never fail a build. Worth reading once; not worth a release.', '');
|
|
146
|
-
}
|
|
147
|
-
if (browser && browser.superseded.length > 0) {
|
|
148
|
-
lines.push('## Set aside, because the browser run checked them', '');
|
|
149
|
-
lines.push(`${browser.superseded.length} source ${browser.superseded.length === 1 ? 'finding' : 'findings'} ` +
|
|
150
|
-
'are not listed above: the axe run checked their criterion on a rendered page, so its answer is the better ' +
|
|
151
|
-
'one and repeating them here would count the same barrier twice. They are in the JSON beside this report.', '');
|
|
152
|
-
}
|
|
153
|
-
if (accepted.length > 0) {
|
|
154
|
-
lines.push('## Already accepted', '');
|
|
155
|
-
lines.push(`${accepted.length} ${accepted.length === 1 ? 'finding is' : 'findings are'} listed in \`.tewip/access-accepted.txt\`, so they are not repeated here.`, '');
|
|
156
|
-
}
|
|
157
|
-
// Coverage comes from every rule that ran, not from the ones that found something: a contrast
|
|
158
|
-
// check that ran and passed has still checked contrast.
|
|
159
|
-
const ran = browser?.rulesRun ?? new Set();
|
|
160
|
-
const settled = nowChecked(ran);
|
|
161
|
-
const remaining = stillNotChecked(ran);
|
|
162
|
-
if (settled.length > 0) {
|
|
163
|
-
lines.push('## Checked here only because a browser run was given', '');
|
|
164
|
-
for (const n of settled)
|
|
165
|
-
lines.push(`- **${n.criterion}** - would otherwise be unreachable: ${n.why}`);
|
|
166
|
-
lines.push('');
|
|
167
|
-
}
|
|
168
|
-
lines.push('## What was not checked', '');
|
|
169
|
-
lines.push(browser
|
|
170
|
-
? 'Even with the browser run, none of the following was tested:'
|
|
171
|
-
: 'This read the markup. It did not render a page, so none of the following was tested:', '');
|
|
172
|
-
for (const n of remaining)
|
|
173
|
-
lines.push(`- **${n.criterion}** - ${n.why}`);
|
|
174
|
-
if (browser) {
|
|
175
|
-
lines.push('');
|
|
176
|
-
lines.push(`The run covered ${browser.pages.length} ${browser.pages.length === 1 ? 'page' : 'pages'}. ` +
|
|
177
|
-
'Nothing is known about a page it did not open, or about a state it did not reach - a dialog that was never ' +
|
|
178
|
-
'opened, a form that was never submitted, an error that was never triggered.');
|
|
179
|
-
}
|
|
180
|
-
lines.push('');
|
|
181
|
-
lines.push('A clean result here means these checks found nothing, not that the product is accessible. ' +
|
|
182
|
-
'Roughly a third of WCAG can be judged by a machine at all' +
|
|
183
|
-
(browser ? '' : ', and reading source rather than a rendered page reaches less than that') +
|
|
184
|
-
'. The rest needs a person, ideally one who uses a screen reader.', '');
|
|
185
|
-
lines.push('To accept a finding, add its key to `.tewip/access-accepted.txt`, one per line, with a reason after ` - `.', '');
|
|
186
|
-
return lines.join('\n');
|
|
69
|
+
/**
|
|
70
|
+
* Reads whichever browser evidence the team actually has.
|
|
71
|
+
*
|
|
72
|
+
* axe and Lighthouse are the same rules - Lighthouse runs axe - so both become one shape and are
|
|
73
|
+
* triaged identically. A team with Lighthouse in CI and no separate axe step gets the same answers
|
|
74
|
+
* without adding a tool (Phase 32).
|
|
75
|
+
*/
|
|
76
|
+
export function readBrowserRun(path) {
|
|
77
|
+
if (!isLighthouseFile(path))
|
|
78
|
+
return readAxeReport(path);
|
|
79
|
+
const report = readLighthouse(path);
|
|
80
|
+
return {
|
|
81
|
+
source: 'lighthouse',
|
|
82
|
+
findings: accessibilityFromLighthouse(report),
|
|
83
|
+
pages: new Set(report.url === '' ? [] : [report.url]),
|
|
84
|
+
rulesRun: rulesRunByLighthouse(report),
|
|
85
|
+
criteriaCovered: criteriaFromLighthouse(report),
|
|
86
|
+
};
|
|
187
87
|
}
|
|
188
88
|
export function runAccess(cmd) {
|
|
189
89
|
const root = resolve(cmd.root);
|
|
@@ -193,7 +93,7 @@ export function runAccess(cmd) {
|
|
|
193
93
|
const open = findings.filter((f) => !alreadyJudged.has(keyOf(f)));
|
|
194
94
|
// Phase 29: a browser run, where one was given. It is the better witness for every criterion it
|
|
195
95
|
// checked, so the source findings for those criteria are set aside rather than repeated.
|
|
196
|
-
const run = cmd.report ?
|
|
96
|
+
const run = cmd.report ? readBrowserRun(resolve(root, cmd.report)) : undefined;
|
|
197
97
|
const graded = run ? merge(open, run, root) : undefined;
|
|
198
98
|
const kept = graded ? graded.kept : open;
|
|
199
99
|
const barriers = kept.filter((f) => f.confidence === 'barrier');
|
|
@@ -206,6 +106,7 @@ export function runAccess(cmd) {
|
|
|
206
106
|
superseded: graded.superseded,
|
|
207
107
|
pages: pagesIn(run),
|
|
208
108
|
rulesRun: run.rulesRun,
|
|
109
|
+
tool: run.source,
|
|
209
110
|
}
|
|
210
111
|
: undefined;
|
|
211
112
|
const result = { barriers, questions, accepted, filesRead, ...(browser ? { browser } : {}) };
|
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
// Phase 32: what each axe rule tests, taken from axe-core itself.
|
|
2
|
+
//
|
|
3
|
+
// Lighthouse's accessibility audits are axe rules - the ids are identical - but a Lighthouse report
|
|
4
|
+
// does not carry axe's tags, so it says "landmark-one-main" and nothing about which criterion that
|
|
5
|
+
// is, or whether it is a criterion at all. Phase 29 needs that to tell a legal failure from somebody's
|
|
6
|
+
// good idea, and 42% of axe's output is the latter.
|
|
7
|
+
//
|
|
8
|
+
// Writing the table by hand was the reason Phase 31 left this undone: a remembered mapping between a
|
|
9
|
+
// hundred rule ids and WCAG numbers is a mapping with mistakes in it, and a wrong criterion beside a
|
|
10
|
+
// finding is worse than no criterion. So the table is **generated**: axe-core is loaded in a browser
|
|
11
|
+
// and asked for its own rule catalogue, and these are its tags, parsed the same way Phase 29 parses
|
|
12
|
+
// the tags on a live run.
|
|
13
|
+
//
|
|
14
|
+
// Generated from axe-core 4.14.0. 105 rules: 75 carry a WCAG criterion,
|
|
15
|
+
// 30 are tagged best-practice only and test no standard at all.
|
|
16
|
+
// Regenerate with: node scripts/axe-rules.ts - never by editing this file.
|
|
17
|
+
export const AXE_CORE_VERSION = '4.14.0';
|
|
18
|
+
export const AXE_RULES = {
|
|
19
|
+
'accesskeys': { criterion: null, level: null, en301549: null },
|
|
20
|
+
'area-alt': { criterion: '2.4.4', level: 'A', en301549: '9.2.4.4' },
|
|
21
|
+
'aria-allowed-attr': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
22
|
+
'aria-allowed-role': { criterion: null, level: null, en301549: null },
|
|
23
|
+
'aria-braille-equivalent': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
24
|
+
'aria-command-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
25
|
+
'aria-conditional-attr': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
26
|
+
'aria-deprecated-role': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
27
|
+
'aria-dialog-name': { criterion: null, level: null, en301549: null },
|
|
28
|
+
'aria-hidden-body': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
29
|
+
'aria-hidden-focus': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
30
|
+
'aria-input-field-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
31
|
+
'aria-meter-name': { criterion: '1.1.1', level: 'A', en301549: '9.1.1.1' },
|
|
32
|
+
'aria-progressbar-name': { criterion: '1.1.1', level: 'A', en301549: '9.1.1.1' },
|
|
33
|
+
'aria-prohibited-attr': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
34
|
+
'aria-required-attr': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
35
|
+
'aria-required-children': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
36
|
+
'aria-required-parent': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
37
|
+
'aria-roledescription': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
38
|
+
'aria-roles': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
39
|
+
'aria-tab-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
40
|
+
'aria-text': { criterion: null, level: null, en301549: null },
|
|
41
|
+
'aria-toggle-field-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
42
|
+
'aria-tooltip-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
43
|
+
'aria-treeitem-name': { criterion: null, level: null, en301549: null },
|
|
44
|
+
'aria-valid-attr': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
45
|
+
'aria-valid-attr-value': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
46
|
+
'audio-caption': { criterion: '1.2.1', level: 'A', en301549: '9.1.2.1' },
|
|
47
|
+
'autocomplete-valid': { criterion: '1.3.5', level: 'AA', en301549: '9.1.3.5' },
|
|
48
|
+
'avoid-inline-spacing': { criterion: '1.4.12', level: 'AA', en301549: '9.1.4.12' },
|
|
49
|
+
'blink': { criterion: '2.2.2', level: 'A', en301549: '9.2.2.2' },
|
|
50
|
+
'button-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
51
|
+
'bypass': { criterion: '2.4.1', level: 'A', en301549: '9.2.4.1' },
|
|
52
|
+
'color-contrast': { criterion: '1.4.3', level: 'AA', en301549: '9.1.4.3' },
|
|
53
|
+
'color-contrast-enhanced': { criterion: '1.4.6', level: 'AAA', en301549: null },
|
|
54
|
+
'css-orientation-lock': { criterion: '1.3.4', level: 'AA', en301549: '9.1.3.4' },
|
|
55
|
+
'definition-list': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
56
|
+
'dlitem': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
57
|
+
'document-title': { criterion: '2.4.2', level: 'A', en301549: '9.2.4.2' },
|
|
58
|
+
'duplicate-id': { criterion: '4.1.1', level: null, en301549: null },
|
|
59
|
+
'duplicate-id-active': { criterion: '4.1.1', level: null, en301549: null },
|
|
60
|
+
'duplicate-id-aria': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
61
|
+
'empty-heading': { criterion: null, level: null, en301549: null },
|
|
62
|
+
'empty-table-header': { criterion: null, level: null, en301549: null },
|
|
63
|
+
'focus-order-semantics': { criterion: null, level: null, en301549: null },
|
|
64
|
+
'form-field-multiple-labels': { criterion: '3.3.2', level: 'A', en301549: '9.3.3.2' },
|
|
65
|
+
'frame-focusable-content': { criterion: '2.1.1', level: 'A', en301549: '9.2.1.1' },
|
|
66
|
+
'frame-tested': { criterion: null, level: null, en301549: null },
|
|
67
|
+
'frame-title': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
68
|
+
'frame-title-unique': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
69
|
+
'heading-order': { criterion: null, level: null, en301549: null },
|
|
70
|
+
'hidden-content': { criterion: null, level: null, en301549: null },
|
|
71
|
+
'html-has-lang': { criterion: '3.1.1', level: 'A', en301549: '9.3.1.1' },
|
|
72
|
+
'html-lang-valid': { criterion: '3.1.1', level: 'A', en301549: '9.3.1.1' },
|
|
73
|
+
'html-xml-lang-mismatch': { criterion: '3.1.1', level: 'A', en301549: '9.3.1.1' },
|
|
74
|
+
'identical-links-same-purpose': { criterion: '2.4.9', level: 'AAA', en301549: null },
|
|
75
|
+
'image-alt': { criterion: '1.1.1', level: 'A', en301549: '9.1.1.1' },
|
|
76
|
+
'image-redundant-alt': { criterion: null, level: null, en301549: null },
|
|
77
|
+
'input-button-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
78
|
+
'input-image-alt': { criterion: '1.1.1', level: 'A', en301549: '9.1.1.1' },
|
|
79
|
+
'label': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
80
|
+
'label-content-name-mismatch': { criterion: '2.5.3', level: 'A', en301549: '9.2.5.3' },
|
|
81
|
+
'label-title-only': { criterion: null, level: null, en301549: null },
|
|
82
|
+
'landmark-banner-is-top-level': { criterion: null, level: null, en301549: null },
|
|
83
|
+
'landmark-complementary-is-top-level': { criterion: null, level: null, en301549: null },
|
|
84
|
+
'landmark-contentinfo-is-top-level': { criterion: null, level: null, en301549: null },
|
|
85
|
+
'landmark-main-is-top-level': { criterion: null, level: null, en301549: null },
|
|
86
|
+
'landmark-no-duplicate-banner': { criterion: null, level: null, en301549: null },
|
|
87
|
+
'landmark-no-duplicate-contentinfo': { criterion: null, level: null, en301549: null },
|
|
88
|
+
'landmark-no-duplicate-main': { criterion: null, level: null, en301549: null },
|
|
89
|
+
'landmark-one-main': { criterion: null, level: null, en301549: null },
|
|
90
|
+
'landmark-unique': { criterion: null, level: null, en301549: null },
|
|
91
|
+
'link-in-text-block': { criterion: '1.4.1', level: 'A', en301549: '9.1.4.1' },
|
|
92
|
+
'link-name': { criterion: '2.4.4', level: 'A', en301549: '9.2.4.4' },
|
|
93
|
+
'list': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
94
|
+
'listitem': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
95
|
+
'marquee': { criterion: '2.2.2', level: 'A', en301549: '9.2.2.2' },
|
|
96
|
+
'meta-refresh': { criterion: '2.2.1', level: 'A', en301549: '9.2.2.1' },
|
|
97
|
+
'meta-refresh-no-exceptions': { criterion: '2.2.4', level: 'AAA', en301549: null },
|
|
98
|
+
'meta-viewport': { criterion: '1.4.4', level: 'AA', en301549: '9.1.4.4' },
|
|
99
|
+
'meta-viewport-large': { criterion: null, level: null, en301549: null },
|
|
100
|
+
'nested-interactive': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
101
|
+
'no-autoplay-audio': { criterion: '1.4.2', level: 'A', en301549: '9.1.4.2' },
|
|
102
|
+
'object-alt': { criterion: '1.1.1', level: 'A', en301549: '9.1.1.1' },
|
|
103
|
+
'p-as-heading': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
104
|
+
'page-has-heading-one': { criterion: null, level: null, en301549: null },
|
|
105
|
+
'presentation-role-conflict': { criterion: null, level: null, en301549: null },
|
|
106
|
+
'region': { criterion: null, level: null, en301549: null },
|
|
107
|
+
'role-img-alt': { criterion: '1.1.1', level: 'A', en301549: '9.1.1.1' },
|
|
108
|
+
'scope-attr-valid': { criterion: null, level: null, en301549: null },
|
|
109
|
+
'scrollable-region-focusable': { criterion: '2.1.1', level: 'A', en301549: '9.2.1.1' },
|
|
110
|
+
'select-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
111
|
+
'server-side-image-map': { criterion: '2.1.1', level: 'A', en301549: '9.2.1.1' },
|
|
112
|
+
'skip-link': { criterion: null, level: null, en301549: null },
|
|
113
|
+
'summary-name': { criterion: '4.1.2', level: 'A', en301549: '9.4.1.2' },
|
|
114
|
+
'svg-img-alt': { criterion: '1.1.1', level: 'A', en301549: '9.1.1.1' },
|
|
115
|
+
'tabindex': { criterion: null, level: null, en301549: null },
|
|
116
|
+
'table-duplicate-name': { criterion: null, level: null, en301549: null },
|
|
117
|
+
'table-fake-caption': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
118
|
+
'target-size': { criterion: '2.5.8', level: 'AA', en301549: '9.2.5.8' },
|
|
119
|
+
'td-has-header': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
120
|
+
'td-headers-attr': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
121
|
+
'th-has-data-cells': { criterion: '1.3.1', level: 'A', en301549: '9.1.3.1' },
|
|
122
|
+
'valid-lang': { criterion: '3.1.2', level: 'AA', en301549: '9.3.1.2' },
|
|
123
|
+
'video-caption': { criterion: '1.2.2', level: 'A', en301549: '9.1.2.2' },
|
|
124
|
+
};
|
|
125
|
+
/** What a rule tests, or undefined when axe has no rule by that name. */
|
|
126
|
+
export function factsFor(ruleId) {
|
|
127
|
+
return AXE_RULES[ruleId];
|
|
128
|
+
}
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
import { factsFor } from './axe-rules.js';
|
|
2
|
+
/** WCAG's own ordering: level A is the most basic requirement, so its failures come first. */
|
|
3
|
+
const BY_LEVEL = { A: 'blocker', AA: 'serious', AAA: 'moderate' };
|
|
4
|
+
/**
|
|
5
|
+
* The accessibility half of a Lighthouse report, as findings the Phase 29 report can print.
|
|
6
|
+
*
|
|
7
|
+
* Three kinds come out, the same three an axe run produces:
|
|
8
|
+
* - a failing audit whose rule tests a WCAG criterion -> a barrier;
|
|
9
|
+
* - a failing audit whose rule tests nobody's standard -> advice;
|
|
10
|
+
* - an audit Lighthouse marks `manual` -> a question, because Lighthouse itself
|
|
11
|
+
* declines to answer it. Those are not passes, and a page with no failures and ten unanswered
|
|
12
|
+
* manual checks has not passed.
|
|
13
|
+
*/
|
|
14
|
+
export function accessibilityFromLighthouse(report) {
|
|
15
|
+
const out = [];
|
|
16
|
+
const page = report.url;
|
|
17
|
+
for (const failure of report.accessibilityFailures) {
|
|
18
|
+
const facts = factsFor(failure.id);
|
|
19
|
+
const criterion = facts?.criterion ?? null;
|
|
20
|
+
out.push({
|
|
21
|
+
rule: failure.id,
|
|
22
|
+
tool: 'axe',
|
|
23
|
+
confidence: criterion === null ? 'advice' : 'barrier',
|
|
24
|
+
criterion,
|
|
25
|
+
level: facts?.level ?? null,
|
|
26
|
+
en301549: facts?.en301549 ?? null,
|
|
27
|
+
severity: criterion === null ? 'minor' : (BY_LEVEL[facts?.level ?? 'A'] ?? 'serious'),
|
|
28
|
+
html: failure.snippet,
|
|
29
|
+
selector: failure.selector,
|
|
30
|
+
page,
|
|
31
|
+
why: failure.title,
|
|
32
|
+
next: failure.explanation || failure.title,
|
|
33
|
+
});
|
|
34
|
+
}
|
|
35
|
+
for (const check of report.manual) {
|
|
36
|
+
const facts = factsFor(check.id);
|
|
37
|
+
out.push({
|
|
38
|
+
rule: check.id,
|
|
39
|
+
tool: 'axe',
|
|
40
|
+
confidence: 'ask-a-person',
|
|
41
|
+
criterion: facts?.criterion ?? null,
|
|
42
|
+
level: facts?.level ?? null,
|
|
43
|
+
en301549: facts?.en301549 ?? null,
|
|
44
|
+
severity: 'moderate',
|
|
45
|
+
html: '',
|
|
46
|
+
selector: '',
|
|
47
|
+
page,
|
|
48
|
+
why: `${check.title}. Lighthouse does not score this one and hands it to a person`,
|
|
49
|
+
next: 'check it yourself, ideally with a keyboard and a screen reader; it is neither a pass nor a failure until somebody looks',
|
|
50
|
+
});
|
|
51
|
+
}
|
|
52
|
+
return out;
|
|
53
|
+
}
|
|
54
|
+
/**
|
|
55
|
+
* Which axe rules a Lighthouse run actually exercised.
|
|
56
|
+
*
|
|
57
|
+
* Everything in the accessibility category ran, including the audits that passed and the ones that
|
|
58
|
+
* did not apply to this page - both are evidence the rule was checked. This is what lets a
|
|
59
|
+
* Lighthouse run move contrast off the "not checked" list, exactly as an axe run does, since
|
|
60
|
+
* `color-contrast` is one of the rules it runs.
|
|
61
|
+
*/
|
|
62
|
+
export function rulesRunByLighthouse(report) {
|
|
63
|
+
return new Set(report.accessibilityAuditIds);
|
|
64
|
+
}
|
|
65
|
+
/** The criteria those rules test, for shrinking what the report has to call unchecked. */
|
|
66
|
+
export function criteriaFromLighthouse(report) {
|
|
67
|
+
const out = new Set();
|
|
68
|
+
for (const id of report.accessibilityAuditIds) {
|
|
69
|
+
const criterion = factsFor(id)?.criterion;
|
|
70
|
+
if (criterion)
|
|
71
|
+
out.add(criterion);
|
|
72
|
+
}
|
|
73
|
+
return out;
|
|
74
|
+
}
|
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
// Phase 28-32: writing the accessibility report.
|
|
2
|
+
//
|
|
3
|
+
// Split out of access-command.ts to keep every file in this project under 300 lines, which the
|
|
4
|
+
// Lighthouse reader pushed it past. The rendering is the part most worth keeping readable: it is
|
|
5
|
+
// where the distinction between a barrier, a question and a piece of advice is actually made
|
|
6
|
+
// visible to a person, and where what was *not* checked gets said.
|
|
7
|
+
import { nowChecked, renderBrowserSection, stillNotChecked } from './merge.js';
|
|
8
|
+
import { blocks, criterionOf, RULES } from './wcag.js';
|
|
9
|
+
const ORDER = { blocker: 0, serious: 1, moderate: 2, minor: 3 };
|
|
10
|
+
export function worstFirst(a, b) {
|
|
11
|
+
const sa = ORDER[RULES[a.rule]?.severity ?? 'minor'];
|
|
12
|
+
const sb = ORDER[RULES[b.rule]?.severity ?? 'minor'];
|
|
13
|
+
if (sa !== sb)
|
|
14
|
+
return sa - sb;
|
|
15
|
+
if (a.file !== b.file)
|
|
16
|
+
return a.file < b.file ? -1 : 1;
|
|
17
|
+
return a.line - b.line;
|
|
18
|
+
}
|
|
19
|
+
function group(findings) {
|
|
20
|
+
const by = new Map();
|
|
21
|
+
for (const f of findings) {
|
|
22
|
+
const list = by.get(f.rule);
|
|
23
|
+
if (list)
|
|
24
|
+
list.push(f);
|
|
25
|
+
else
|
|
26
|
+
by.set(f.rule, [f]);
|
|
27
|
+
}
|
|
28
|
+
return by;
|
|
29
|
+
}
|
|
30
|
+
function section(title, findings, lines) {
|
|
31
|
+
if (findings.length === 0)
|
|
32
|
+
return;
|
|
33
|
+
lines.push(`## ${title}`, '');
|
|
34
|
+
for (const [rule, found] of group(findings)) {
|
|
35
|
+
const meta = RULES[rule];
|
|
36
|
+
lines.push(`### ${rule} - ${found.length} ${found.length === 1 ? 'place' : 'places'}`, '');
|
|
37
|
+
lines.push(`${criterionOf(rule)}, ${meta?.severity ?? 'unrated'}.`, '');
|
|
38
|
+
if (meta)
|
|
39
|
+
lines.push(`Who it stops: ${meta.stops}.`, '');
|
|
40
|
+
for (const f of found.slice(0, 20)) {
|
|
41
|
+
lines.push(`- \`${f.file}:${f.line}\` ${f.element}`);
|
|
42
|
+
lines.push(` - ${f.why}`);
|
|
43
|
+
lines.push(` - do this: ${f.next}`);
|
|
44
|
+
}
|
|
45
|
+
if (found.length > 20)
|
|
46
|
+
lines.push(`- ...and ${found.length - 20} more of the same, in the JSON beside this report.`);
|
|
47
|
+
lines.push('');
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
export function renderAccess(result, scanned) {
|
|
51
|
+
const { barriers, questions, accepted, filesRead, browser } = result;
|
|
52
|
+
const lines = ['# Tewip accessibility check', ''];
|
|
53
|
+
lines.push(`Read ${filesRead} markup ${filesRead === 1 ? 'file' : 'files'} under \`${scanned}\`, without rendering any of them.`, '');
|
|
54
|
+
if (browser) {
|
|
55
|
+
lines.push(`Also read a ${browser.tool === 'lighthouse' ? 'Lighthouse' : 'axe'} run over ${browser.pages.length} rendered ` +
|
|
56
|
+
`${browser.pages.length === 1 ? 'page' : 'pages'}, ` +
|
|
57
|
+
'which settles what reading markup cannot.', '');
|
|
58
|
+
}
|
|
59
|
+
if (barriers.length === 0 && questions.length === 0 && !browser) {
|
|
60
|
+
lines.push('No barrier was found that reading the markup can settle, and nothing needs a decision.', '');
|
|
61
|
+
}
|
|
62
|
+
else {
|
|
63
|
+
const stopping = barriers.filter((f) => blocks(f.rule)).length;
|
|
64
|
+
const counts = [
|
|
65
|
+
`${barriers.length} ${barriers.length === 1 ? 'barrier' : 'barriers'} (${stopping} that stop somebody outright)`,
|
|
66
|
+
`${questions.length} for a person to settle`,
|
|
67
|
+
];
|
|
68
|
+
if (accepted.length > 0)
|
|
69
|
+
counts.push(`${accepted.length} already accepted`);
|
|
70
|
+
if (browser) {
|
|
71
|
+
counts.unshift(`${browser.barriers.length} barriers the browser run proves`);
|
|
72
|
+
if (browser.questions.length > 0)
|
|
73
|
+
counts.push(`${browser.questions.length} the browser could not determine`);
|
|
74
|
+
}
|
|
75
|
+
lines.push(`**${counts.join(', ')}.**`, '');
|
|
76
|
+
}
|
|
77
|
+
if (browser) {
|
|
78
|
+
renderBrowserSection('Barriers seen in the browser', browser.barriers, lines);
|
|
79
|
+
renderBrowserSection('The browser could not determine these', browser.questions, lines);
|
|
80
|
+
}
|
|
81
|
+
section('Barriers read from the source', [...barriers].sort(worstFirst), lines);
|
|
82
|
+
section('Only a person can settle these', [...questions].sort(worstFirst), lines);
|
|
83
|
+
if (browser && browser.advice.length > 0) {
|
|
84
|
+
// Kept out of the counts above on purpose. These are axe's own best-practice rules, which test
|
|
85
|
+
// no standard: on nine real pages they were 51 of 174 findings, outnumbering every real barrier.
|
|
86
|
+
// Reporting them as equals is how a team learns to ignore the whole report.
|
|
87
|
+
const rules = [...new Set(browser.advice.map((f) => f.rule))].sort();
|
|
88
|
+
lines.push('## Advice, not barriers', '');
|
|
89
|
+
lines.push(`${browser.advice.length} ${browser.advice.length === 1 ? 'finding comes' : 'findings come'} from axe rules that map to no WCAG criterion - ` +
|
|
90
|
+
`${rules.join(', ')}. They are somebody's good idea rather than a standard, so they are not counted as barriers ` +
|
|
91
|
+
'and never fail a build. Worth reading once; not worth a release.', '');
|
|
92
|
+
}
|
|
93
|
+
if (browser && browser.superseded.length > 0) {
|
|
94
|
+
lines.push('## Set aside, because the browser run checked them', '');
|
|
95
|
+
lines.push(`${browser.superseded.length} source ${browser.superseded.length === 1 ? 'finding' : 'findings'} ` +
|
|
96
|
+
'are not listed above: the axe run checked their criterion on a rendered page, so its answer is the better ' +
|
|
97
|
+
'one and repeating them here would count the same barrier twice. They are in the JSON beside this report.', '');
|
|
98
|
+
}
|
|
99
|
+
if (accepted.length > 0) {
|
|
100
|
+
lines.push('## Already accepted', '');
|
|
101
|
+
lines.push(`${accepted.length} ${accepted.length === 1 ? 'finding is' : 'findings are'} listed in \`.tewip/access-accepted.txt\`, so they are not repeated here.`, '');
|
|
102
|
+
}
|
|
103
|
+
// Coverage comes from every rule that ran, not from the ones that found something: a contrast
|
|
104
|
+
// check that ran and passed has still checked contrast.
|
|
105
|
+
const ran = browser?.rulesRun ?? new Set();
|
|
106
|
+
const settled = nowChecked(ran);
|
|
107
|
+
const remaining = stillNotChecked(ran);
|
|
108
|
+
if (settled.length > 0) {
|
|
109
|
+
lines.push('## Checked here only because a browser run was given', '');
|
|
110
|
+
for (const n of settled)
|
|
111
|
+
lines.push(`- **${n.criterion}** - would otherwise be unreachable: ${n.why}`);
|
|
112
|
+
lines.push('');
|
|
113
|
+
}
|
|
114
|
+
lines.push('## What was not checked', '');
|
|
115
|
+
lines.push(browser
|
|
116
|
+
? 'Even with the browser run, none of the following was tested:'
|
|
117
|
+
: 'This read the markup. It did not render a page, so none of the following was tested:', '');
|
|
118
|
+
for (const n of remaining)
|
|
119
|
+
lines.push(`- **${n.criterion}** - ${n.why}`);
|
|
120
|
+
if (browser) {
|
|
121
|
+
lines.push('');
|
|
122
|
+
lines.push(`The run covered ${browser.pages.length} ${browser.pages.length === 1 ? 'page' : 'pages'}. ` +
|
|
123
|
+
'Nothing is known about a page it did not open, or about a state it did not reach - a dialog that was never ' +
|
|
124
|
+
'opened, a form that was never submitted, an error that was never triggered.');
|
|
125
|
+
}
|
|
126
|
+
lines.push('');
|
|
127
|
+
lines.push('A clean result here means these checks found nothing, not that the product is accessible. ' +
|
|
128
|
+
'Roughly a third of WCAG can be judged by a machine at all' +
|
|
129
|
+
(browser ? '' : ', and reading source rather than a rendered page reaches less than that') +
|
|
130
|
+
'. The rest needs a person, ideally one who uses a screen reader.', '');
|
|
131
|
+
lines.push('To accept a finding, add its key to `.tewip/access-accepted.txt`, one per line, with a reason after ` - `.', '');
|
|
132
|
+
return lines.join('\n');
|
|
133
|
+
}
|
|
@@ -145,7 +145,7 @@ export function readAxeReport(path) {
|
|
|
145
145
|
catch {
|
|
146
146
|
throw new Error(`${path} is not JSON. Tewip reads the JSON an axe run writes.`);
|
|
147
147
|
}
|
|
148
|
-
const run = { findings: [], pages: new Set(), rulesRun: new Set(), criteriaCovered: new Set() };
|
|
148
|
+
const run = { source: 'axe', findings: [], pages: new Set(), rulesRun: new Set(), criteriaCovered: new Set() };
|
|
149
149
|
const pageOf = (v, fallback) => text(v.url) || text(v.page) || fallback;
|
|
150
150
|
if (looksLikeAxe(parsed)) {
|
|
151
151
|
fromResult(parsed, pageOf(parsed, path), run);
|
|
@@ -102,12 +102,17 @@ export function readLighthouse(path) {
|
|
|
102
102
|
}
|
|
103
103
|
const accessibilityFailures = [];
|
|
104
104
|
const manual = [];
|
|
105
|
+
const accessibilityAuditIds = [];
|
|
105
106
|
const a11yRefs = (categories.accessibility?.auditRefs ?? []);
|
|
106
107
|
for (const ref of a11yRefs) {
|
|
107
108
|
const audit = audits[text(ref.id)];
|
|
108
109
|
if (!audit)
|
|
109
110
|
continue;
|
|
110
111
|
const id = text(audit.id);
|
|
112
|
+
// Recorded before the outcome is looked at: a rule that passed, or did not apply to this page,
|
|
113
|
+
// has still been checked, and that is what coverage means.
|
|
114
|
+
if (id !== '')
|
|
115
|
+
accessibilityAuditIds.push(id);
|
|
111
116
|
if (audit.scoreDisplayMode === 'manual') {
|
|
112
117
|
manual.push({ id, title: text(audit.title) });
|
|
113
118
|
continue;
|
|
@@ -133,6 +138,7 @@ export function readLighthouse(path) {
|
|
|
133
138
|
categories: Object.fromEntries(Object.entries(categories).map(([k, v]) => [k, num(v?.score)])),
|
|
134
139
|
accessibilityFailures,
|
|
135
140
|
manual,
|
|
141
|
+
accessibilityAuditIds,
|
|
136
142
|
url: text(o.finalDisplayedUrl) || text(o.finalUrl) || text(o.requestedUrl),
|
|
137
143
|
lighthouseVersion: text(o.lighthouseVersion),
|
|
138
144
|
};
|
|
@@ -11,6 +11,12 @@ export function dialectOf(file, text) {
|
|
|
11
11
|
return 'rails';
|
|
12
12
|
if (/\bop\.(?:drop_column|add_column|create_index|alter_column|drop_table|rename_table|execute)\b/.test(text))
|
|
13
13
|
return 'alembic';
|
|
14
|
+
// TypeORM writes a migration as a class with up() and down(), and puts the real DDL in template
|
|
15
|
+
// strings passed to queryRunner.query(). The SQL is right there and perfectly readable; only the
|
|
16
|
+
// wrapper was unfamiliar, so every TypeORM project was told Tewip could not read its migrations
|
|
17
|
+
// at all - and on a Node backend that is most of them.
|
|
18
|
+
if (/\bMigrationInterface\b|\bqueryRunner\.query\s*\(/.test(text))
|
|
19
|
+
return 'typeorm';
|
|
14
20
|
if (/(^|\/)(migrations?|db\/migrate)\//.test(path))
|
|
15
21
|
return 'unknown';
|
|
16
22
|
return 'unknown';
|
|
@@ -157,7 +163,12 @@ const ALEMBIC = [
|
|
|
157
163
|
table: (m) => m[1],
|
|
158
164
|
},
|
|
159
165
|
];
|
|
160
|
-
|
|
166
|
+
// TypeORM has no operations of its own: unlike Django's `RemoveField` or Rails's `remove_column`,
|
|
167
|
+
// a TypeORM migration contains SQL and nothing else, handed to queryRunner.query(). So its list is
|
|
168
|
+
// empty on purpose and every finding comes from reading that SQL, which is the same reader the
|
|
169
|
+
// `.sql` dialect uses. The exhaustive Record is what made this a compile error rather than a
|
|
170
|
+
// TypeORM migration silently producing no findings at all.
|
|
171
|
+
const BY_DIALECT = { django: DJANGO, rails: RAILS, alembic: ALEMBIC, typeorm: [] };
|
|
161
172
|
/** Findings from a framework migration, with the line each one sits on. */
|
|
162
173
|
export function readFramework(dialect, text) {
|
|
163
174
|
const lines = text.split(String.fromCharCode(10));
|
|
@@ -216,7 +227,10 @@ export function forwardOnly(dialect, text) {
|
|
|
216
227
|
const line = lines[i];
|
|
217
228
|
const down = (dialect === 'rails' && /^\s*def\s+down\b/.test(line)) ||
|
|
218
229
|
(dialect === 'alembic' && /^\s*def\s+downgrade\s*\(/.test(line)) ||
|
|
219
|
-
(dialect === 'sql' && /^\s*--\s*(?:\+migrate\s+Down|\+goose\s+Down|\/\/@UNDO|Down\b)/i.test(line))
|
|
230
|
+
(dialect === 'sql' && /^\s*--\s*(?:\+migrate\s+Down|\+goose\s+Down|\/\/@UNDO|Down\b)/i.test(line)) ||
|
|
231
|
+
// TypeORM's rollback is `public async down(queryRunner: QueryRunner)`. Reading it as part of
|
|
232
|
+
// the change would report a migration that adds a column as one that drops it.
|
|
233
|
+
(dialect === 'typeorm' && /^\s*(?:public\s+)?async\s+down\s*\(/.test(line));
|
|
220
234
|
if (down) {
|
|
221
235
|
// To the end of the block: the next line at the same indentation or less that starts something new.
|
|
222
236
|
blankFrom(i, (l, indent) => l.trim() !== '' && indentOf(l) <= indent && !/^\s*(end|\})\s*$/.test(l), indentOf(line));
|
|
@@ -17,10 +17,39 @@ const MIGRATION = /(^|\/)(migrations?|db\/migrate|prisma\/migrations|alembic|fly
|
|
|
17
17
|
const SCHEMA_FILE = /\.(sql)$|(^|\/)(migrations?|db\/migrate)\/.*\.(py|rb|js|ts)$/i;
|
|
18
18
|
/** The schema files in this change. */
|
|
19
19
|
export function schemaFiles(cmd) {
|
|
20
|
-
const
|
|
20
|
+
const named = cmd.files !== undefined && cmd.files.length > 0;
|
|
21
|
+
const given = named ? cmd.files : (changeSummary(cmd.root, cmd.diff)?.files ?? []);
|
|
21
22
|
return given
|
|
22
23
|
.map((f) => f.split('\\').join('/'))
|
|
23
|
-
|
|
24
|
+
// Asking for a file by name overrides the path filter: the filter exists to find migrations in
|
|
25
|
+
// a diff of a hundred files, not to argue with somebody who pointed at one. A TypeORM migration
|
|
26
|
+
// sitting outside a migrations/ directory was being answered with "no migration in this change".
|
|
27
|
+
.filter((f) => (named || SCHEMA_FILE.test(f) || MIGRATION.test(f)) && existsSync(resolve(cmd.root, f)));
|
|
28
|
+
}
|
|
29
|
+
/**
|
|
30
|
+
* Tables this migration creates itself.
|
|
31
|
+
*
|
|
32
|
+
* A lock is only expensive because of the rows it holds up, and a table created three statements
|
|
33
|
+
* earlier has none. This matters enormously on the tools that generate their SQL: a Prisma project
|
|
34
|
+
* emits `CREATE TABLE` followed by an `ADD CONSTRAINT ... FOREIGN KEY` for every relation, so
|
|
35
|
+
* documenso's 165 real migrations produced 275 "blocks writes while it runs" findings - 1.7 per
|
|
36
|
+
* migration, nearly all of them about tables that did not exist a moment before. That is the volume
|
|
37
|
+
* at which a team stops reading the report, which costs more than the warnings are worth.
|
|
38
|
+
*/
|
|
39
|
+
function createdHere(text) {
|
|
40
|
+
const names = new Set();
|
|
41
|
+
for (const m of text.matchAll(/\bCREATE\s+TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?"?([\w$.]+)"?/gi)) {
|
|
42
|
+
names.add(m[1].toLowerCase().replace(/^.*\./, ''));
|
|
43
|
+
}
|
|
44
|
+
return names;
|
|
45
|
+
}
|
|
46
|
+
/** Risks that are only risks because rows already exist. */
|
|
47
|
+
const ONLY_HURTS_EXISTING_ROWS = new Set(['locks-writes', 'slow']);
|
|
48
|
+
export function droppingWhatIsNew(findings, text) {
|
|
49
|
+
const fresh = createdHere(text);
|
|
50
|
+
if (fresh.size === 0)
|
|
51
|
+
return findings;
|
|
52
|
+
return findings.filter((f) => !(ONLY_HURTS_EXISTING_ROWS.has(f.risk) && fresh.has(f.table.toLowerCase().replace(/^.*\./, ''))));
|
|
24
53
|
}
|
|
25
54
|
export function readMigration(root, file) {
|
|
26
55
|
let text = '';
|
|
@@ -35,13 +64,13 @@ export function readMigration(root, file) {
|
|
|
35
64
|
// part of the change says a migration that creates a table destroys one.
|
|
36
65
|
const forward = forwardOnly(dialect, text);
|
|
37
66
|
if (dialect === 'sql')
|
|
38
|
-
return { file, dialect, findings: readSql(forward), unread: '', irreversible: false };
|
|
67
|
+
return { file, dialect, findings: droppingWhatIsNew(readSql(forward), forward), unread: '', irreversible: false };
|
|
39
68
|
if (dialect === 'unknown') {
|
|
40
69
|
return {
|
|
41
70
|
file,
|
|
42
71
|
dialect,
|
|
43
72
|
findings: [],
|
|
44
|
-
unread: 'Tewip does not recognise how this migration is written, so it says nothing about it rather than passing it. It reads SQL, Django, Rails and
|
|
73
|
+
unread: 'Tewip does not recognise how this migration is written, so it says nothing about it rather than passing it. It reads SQL, Django, Rails, Alembic and TypeORM migrations',
|
|
45
74
|
irreversible: false,
|
|
46
75
|
};
|
|
47
76
|
}
|
|
@@ -49,10 +78,12 @@ export function readMigration(root, file) {
|
|
|
49
78
|
// A migration that only runs raw SQL through the framework is still readable: the SQL is the migration.
|
|
50
79
|
// Raw SQL, however it reaches the database: through the framework, or through a cursor inside a data
|
|
51
80
|
// migration, which is where most large projects put the dangerous parts.
|
|
52
|
-
|
|
81
|
+
// `queryRunner.query` is TypeORM's, and is the whole of a TypeORM migration rather than an
|
|
82
|
+
// escape hatch inside one: the class is a wrapper and every statement arrives this way.
|
|
83
|
+
const raw = [...forward.matchAll(/(?:RunSQL|op\.execute|queryRunner\.query|execute|cursor\.execute)\(\s*(?:r?["'`]{1,3})([\s\S]*?)["'`]{1,3}\s*[,)]/g)].map((m) => m[1]).join(';');
|
|
53
84
|
if (raw.trim())
|
|
54
85
|
findings.push(...readSql(raw));
|
|
55
|
-
return { file, dialect, findings, unread: '', irreversible: irreversible(dialect, text) };
|
|
86
|
+
return { file, dialect, findings: droppingWhatIsNew(findings, forward), unread: '', irreversible: irreversible(dialect, text) };
|
|
56
87
|
}
|
|
57
88
|
/**
|
|
58
89
|
* Code outside the migrations that still names a table or column this change takes away.
|
|
@@ -20,7 +20,12 @@ const RULES = [
|
|
|
20
20
|
table: (m) => bare(m[1]),
|
|
21
21
|
},
|
|
22
22
|
{
|
|
23
|
-
|
|
23
|
+
// The word COLUMN is optional in SQL, so the name after DROP has to be checked against the
|
|
24
|
+
// other things an ALTER TABLE can drop. Without the lookahead, Prisma's own
|
|
25
|
+
// `ALTER TABLE "Share" DROP CONSTRAINT "Share_userId_fkey"` - which every Prisma project is full
|
|
26
|
+
// of - was read as dropping a column called CONSTRAINT, and reported as breaking the running
|
|
27
|
+
// version, which dropping a foreign key does not do. Found on documenso's real migrations.
|
|
28
|
+
find: new RegExp(String.raw `\bALTER\s+TABLE\s+(${NAME})[\s\S]*?\bDROP\s+(?:COLUMN\s+)?(?:IF\s+EXISTS\s+)?(?!CONSTRAINT\b|INDEX\b|DEFAULT\b|NOT\s+NULL\b|PRIMARY\b|FOREIGN\b|CHECK\b|UNIQUE\b|PARTITION\b)(${NAME})`, 'i'),
|
|
24
29
|
risk: 'breaks-old-code',
|
|
25
30
|
what: (m) => `drops the column ${bare(m[2])} from ${bare(m[1])}`,
|
|
26
31
|
instead: 'deploy code that no longer selects or writes the column, then drop it in a later migration. During a rolling deploy the old version is still running and will fail on a column that is gone',
|
package/dist/mcp/tools-prove.js
CHANGED
|
@@ -131,20 +131,19 @@ export function registerProofTools(server, { answer, readOnly, project }) {
|
|
|
131
131
|
axeReport: z
|
|
132
132
|
.string()
|
|
133
133
|
.optional()
|
|
134
|
-
.describe('The JSON from an axe run, if the project has
|
|
134
|
+
.describe('The JSON from an axe run OR a Lighthouse report, if the project has either - Lighthouse runs axe rules, so both work. It settles contrast, which reading markup cannot, and its findings are triaged rather than relayed: rules that map to no WCAG criterion are reported as advice, and what the tool could not determine becomes a question.'),
|
|
135
135
|
},
|
|
136
136
|
annotations: readOnly,
|
|
137
137
|
}, ({ project: root, path, axeReport }) => answer(async () => {
|
|
138
138
|
if (!isAbsolute(root) || !existsSync(root))
|
|
139
139
|
throw new Error(`project must be an existing absolute path, got "${root}"`);
|
|
140
140
|
// The pure functions, not runAccess: that one prints, and anything on stdout breaks JSON-RPC.
|
|
141
|
-
const { findBarriers, keyOf, readAccepted, renderAccess } = await import('../engine/access/access-command.js');
|
|
141
|
+
const { findBarriers, keyOf, readAccepted, readBrowserRun, renderAccess } = await import('../engine/access/access-command.js');
|
|
142
142
|
const { merge, pagesIn } = await import('../engine/access/merge.js');
|
|
143
|
-
const { readAxeReport } = await import('../engine/access/reports.js');
|
|
144
143
|
const { findings, filesRead } = findBarriers(root, path);
|
|
145
144
|
const judged = readAccepted(root);
|
|
146
145
|
const open = findings.filter((f) => !judged.has(keyOf(f)));
|
|
147
|
-
const run = axeReport ?
|
|
146
|
+
const run = axeReport ? readBrowserRun(isAbsolute(axeReport) ? axeReport : `${root}/${axeReport}`) : undefined;
|
|
148
147
|
const graded = run ? merge(open, run, root) : undefined;
|
|
149
148
|
const kept = graded ? graded.kept : open;
|
|
150
149
|
return renderAccess({
|
|
@@ -161,6 +160,7 @@ export function registerProofTools(server, { answer, readOnly, project }) {
|
|
|
161
160
|
superseded: graded.superseded,
|
|
162
161
|
pages: pagesIn(run),
|
|
163
162
|
rulesRun: run.rulesRun,
|
|
163
|
+
tool: run.source,
|
|
164
164
|
},
|
|
165
165
|
}
|
|
166
166
|
: {}),
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tewip-ai",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.40.0",
|
|
4
4
|
"mcpName": "io.github.hmamut39/tewip",
|
|
5
5
|
"description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
|
|
6
6
|
"keywords": [
|