qualm-a11y 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +201 -0
- package/RESEARCH.md +139 -0
- package/dist/analyser.d.ts +3 -0
- package/dist/analyser.d.ts.map +1 -0
- package/dist/analyser.js +77 -0
- package/dist/analyser.js.map +1 -0
- package/dist/cli.d.ts +3 -0
- package/dist/cli.d.ts.map +1 -0
- package/dist/cli.js +128 -0
- package/dist/cli.js.map +1 -0
- package/dist/diff.d.ts +3 -0
- package/dist/diff.d.ts.map +1 -0
- package/dist/diff.js +60 -0
- package/dist/diff.js.map +1 -0
- package/dist/parser.d.ts +3 -0
- package/dist/parser.d.ts.map +1 -0
- package/dist/parser.js +22 -0
- package/dist/parser.js.map +1 -0
- package/dist/reporters/json.d.ts +4 -0
- package/dist/reporters/json.d.ts.map +1 -0
- package/dist/reporters/json.js +20 -0
- package/dist/reporters/json.js.map +1 -0
- package/dist/reporters/sarif.d.ts +3 -0
- package/dist/reporters/sarif.d.ts.map +1 -0
- package/dist/reporters/sarif.js +40 -0
- package/dist/reporters/sarif.js.map +1 -0
- package/dist/reporters/terminal.d.ts +5 -0
- package/dist/reporters/terminal.d.ts.map +1 -0
- package/dist/reporters/terminal.js +83 -0
- package/dist/reporters/terminal.js.map +1 -0
- package/dist/rules/aria-correctness.d.ts +3 -0
- package/dist/rules/aria-correctness.d.ts.map +1 -0
- package/dist/rules/aria-correctness.js +52 -0
- package/dist/rules/aria-correctness.js.map +1 -0
- package/dist/rules/complexity.d.ts +4 -0
- package/dist/rules/complexity.d.ts.map +1 -0
- package/dist/rules/complexity.js +86 -0
- package/dist/rules/complexity.js.map +1 -0
- package/dist/rules/document-structure.d.ts +3 -0
- package/dist/rules/document-structure.d.ts.map +1 -0
- package/dist/rules/document-structure.js +51 -0
- package/dist/rules/document-structure.js.map +1 -0
- package/dist/rules/form-semantics.d.ts +3 -0
- package/dist/rules/form-semantics.d.ts.map +1 -0
- package/dist/rules/form-semantics.js +105 -0
- package/dist/rules/form-semantics.js.map +1 -0
- package/dist/rules/heading-hierarchy.d.ts +3 -0
- package/dist/rules/heading-hierarchy.d.ts.map +1 -0
- package/dist/rules/heading-hierarchy.js +37 -0
- package/dist/rules/heading-hierarchy.js.map +1 -0
- package/dist/rules/index.d.ts +4 -0
- package/dist/rules/index.d.ts.map +1 -0
- package/dist/rules/index.js +20 -0
- package/dist/rules/index.js.map +1 -0
- package/dist/rules/interactive-semantics.d.ts +3 -0
- package/dist/rules/interactive-semantics.d.ts.map +1 -0
- package/dist/rules/interactive-semantics.js +65 -0
- package/dist/rules/interactive-semantics.js.map +1 -0
- package/dist/rules/landmark-structure.d.ts +3 -0
- package/dist/rules/landmark-structure.d.ts.map +1 -0
- package/dist/rules/landmark-structure.js +60 -0
- package/dist/rules/landmark-structure.js.map +1 -0
- package/dist/types.d.ts +73 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +37 -0
- package/dist/types.js.map +1 -0
- package/package.json +58 -0
package/README.md
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
# qualm
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/qualm)
|
|
4
|
+
[](https://doi.org/10.5281/zenodo.20482307)
|
|
5
|
+
[](LICENSE)
|
|
6
|
+
[](https://github.com/SomilKSharma/qualm/actions)
|
|
7
|
+
|
|
8
|
+
**Static AST-level quality analyser for LLM-generated React/TypeScript code.** Operationalises the empirical findings of [Sharma (2026)](https://doi.org/10.5281/zenodo.20482307) as a working static analyser.
|
|
9
|
+
|
|
10
|
+
> _AI coding tools do not produce an immediate, dramatic accessibility regression—but a slow accumulation effect, particularly in structural HTML semantics, warrants further investigation._
|
|
11
|
+
> — Sharma (2026), Abstract
|
|
12
|
+
|
|
13
|
+
qualm detects the violation patterns that the Sharma (2026) empirical study found most associated with AI-assisted code: generic containers substituted for semantic HTML, landmark elements rendered as divs, and structural HTML degradation that accumulates silently over months.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Quick Start
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
# Run directly on a directory
|
|
21
|
+
npx qualm-a11y ./src
|
|
22
|
+
|
|
23
|
+
# Or install globally
|
|
24
|
+
npm install -g qualm-a11y
|
|
25
|
+
qualm-a11y ./src
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## All CLI Flags
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
Usage: qualm [options] <paths...>
|
|
34
|
+
|
|
35
|
+
Arguments:
|
|
36
|
+
paths Files or directories to analyse (.tsx, .ts, .jsx, .js)
|
|
37
|
+
|
|
38
|
+
Options:
|
|
39
|
+
-f, --format <format> Output format: terminal, json, sarif (default: "terminal")
|
|
40
|
+
-o, --output <file> Write output to file instead of stdout
|
|
41
|
+
--diff-branch <branch> Compare against git branch to detect regressions
|
|
42
|
+
--fail-on <level> Exit 1 if violations of this level exist: error, warning (default: "error")
|
|
43
|
+
--research-mode Output metrics in Sharma (2026) taxonomy format
|
|
44
|
+
-V, --version Output version number
|
|
45
|
+
-h, --help Display help
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
### Examples
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
# Analyse a single file
|
|
52
|
+
qualm src/components/Button.tsx
|
|
53
|
+
|
|
54
|
+
# Analyse entire src directory, JSON output
|
|
55
|
+
qualm src/ --format json --output results.json
|
|
56
|
+
|
|
57
|
+
# SARIF output for GitHub Code Scanning
|
|
58
|
+
qualm src/ --format sarif --output results.sarif
|
|
59
|
+
|
|
60
|
+
# Detect regressions vs main branch
|
|
61
|
+
qualm src/ --diff-branch main
|
|
62
|
+
|
|
63
|
+
# Fail CI on warnings too
|
|
64
|
+
qualm src/ --fail-on warning
|
|
65
|
+
|
|
66
|
+
# Research mode — Sharma (2026) taxonomy table
|
|
67
|
+
qualm src/ --research-mode
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## Research Background
|
|
73
|
+
|
|
74
|
+
This tool directly implements the measurement framework from:
|
|
75
|
+
|
|
76
|
+
**Sharma, S. (2026).** _Accessibility and Semantic Quality Regressions in AI-Assisted React Development: An Empirical Study._ arXiv preprint.
|
|
77
|
+
|
|
78
|
+
**Study design**: Longitudinal difference-in-differences across 74 open-source React/TypeScript repositories (41 treated with AI tools, 33 matched controls), 2,374 repo-months.
|
|
79
|
+
|
|
80
|
+
**Key findings (Table 5 — DiD estimates by violation category):**
|
|
81
|
+
|
|
82
|
+
| Violation Category | β (paper) | p-value | Interpretation |
|
|
83
|
+
| -------------------- | --------- | ------- | ------------------------------------------------ |
|
|
84
|
+
| `document_structure` | +0.007 | 0.151 | Largest point estimate — dominant AI-gen failure |
|
|
85
|
+
| `aria_specific` | +0.002 | 0.561 | Moderate ARIA degradation |
|
|
86
|
+
| `semantic_naming` | −0.003 | 0.824 | AI may slightly improve naming visibility |
|
|
87
|
+
|
|
88
|
+
**AST semantic score (Table A1):** Treated repos show treated-pre mean = 0.989, treated-post mean = 0.983, consistent with marginal but accumulating structural degradation (DiD β = +0.005, p = 0.075).
|
|
89
|
+
|
|
90
|
+
qualm's semantic score is computed using these β values as weights — `document_structure` violations deduct proportionally more than `aria_correctness` violations because the empirical evidence assigns them greater causal weight.
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Violation Categories
|
|
95
|
+
|
|
96
|
+
| Category | β Weight | Severity | Description | Example Violation |
|
|
97
|
+
| ----------------------- | --------- | -------- | ------------------------------------------------------------- | ---------------------------------------------- |
|
|
98
|
+
| `document_structure` | **0.007** | error | Interactive `<div>`/`<span>` without semantic element or role | `<div onClick={fn}>` → use `<button>` |
|
|
99
|
+
| `landmark_structure` | 0.004 | warning | Generic container with landmark-suggesting class/id | `<div className="navbar">` → use `<nav>` |
|
|
100
|
+
| `heading_hierarchy` | 0.003 | warning | Skipped heading level | `<h1>` → `<h3>` without `<h2>` |
|
|
101
|
+
| `interactive_semantics` | 0.003 | error | Missing `alt` on image; icon button without label | `<img src="...">` (no alt) |
|
|
102
|
+
| `aria_correctness` | 0.002 | error | Invalid boolean ARIA value | `aria-expanded="yes"` → `"true"` |
|
|
103
|
+
| `form_semantics` | 0.002 | error | Form control without associated label | `<input id="x">` with no `<label htmlFor="x">` |
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## `--research-mode` Output
|
|
108
|
+
|
|
109
|
+
```
|
|
110
|
+
qualm Research Mode — Sharma (2026) Taxonomy
|
|
111
|
+
|
|
112
|
+
────────────────────────────────────────────────────────────────────────────────
|
|
113
|
+
Category | Violations | β (paper) | Weighted Score
|
|
114
|
+
────────────────────────────────────────────────────────────────────────────────
|
|
115
|
+
document_structure | 3 | +0.007 | 0.0500
|
|
116
|
+
landmark_structure | 2 | +0.004 | 0.0190
|
|
117
|
+
heading_hierarchy | 0 | +0.003 | 0.0000
|
|
118
|
+
interactive_semantics | 1 | +0.003 | 0.0143
|
|
119
|
+
aria_correctness | 0 | +0.002 | 0.0000
|
|
120
|
+
form_semantics | 1 | +0.002 | 0.0095
|
|
121
|
+
────────────────────────────────────────────────────────────────────────────────
|
|
122
|
+
Composite Regression Score | 0.0928 | Baseline | 0.9334
|
|
123
|
+
────────────────────────────────────────────────────────────────────────────────
|
|
124
|
+
Paper baseline (Table A1): treated-pre AST score = 0.989, treated-post = 0.983
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## GitHub Action
|
|
130
|
+
|
|
131
|
+
```yaml
|
|
132
|
+
# .github/workflows/qualm.yml
|
|
133
|
+
name: qualm accessibility check
|
|
134
|
+
|
|
135
|
+
on: [pull_request]
|
|
136
|
+
|
|
137
|
+
jobs:
|
|
138
|
+
qualm:
|
|
139
|
+
runs-on: ubuntu-latest
|
|
140
|
+
steps:
|
|
141
|
+
- uses: actions/checkout@v4
|
|
142
|
+
with:
|
|
143
|
+
fetch-depth: 0 # needed for --diff-branch
|
|
144
|
+
|
|
145
|
+
- name: Run qualm
|
|
146
|
+
run: npx qualm src/ --diff-branch main --fail-on error
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
For SARIF upload to GitHub Code Scanning:
|
|
150
|
+
|
|
151
|
+
```yaml
|
|
152
|
+
- name: Run qualm (SARIF)
|
|
153
|
+
run: npx qualm src/ --format sarif --output qualm.sarif || true
|
|
154
|
+
|
|
155
|
+
- name: Upload SARIF
|
|
156
|
+
uses: github/codeql-action/upload-sarif@v3
|
|
157
|
+
with:
|
|
158
|
+
sarif_file: qualm.sarif
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
---
|
|
162
|
+
|
|
163
|
+
## Contributing
|
|
164
|
+
|
|
165
|
+
qualm is explicitly designed for researcher extension. Each rule in `src/rules/` is a self-contained module — adding a new rule requires touching only three files:
|
|
166
|
+
|
|
167
|
+
1. `src/rules/your-rule.ts` — implement the `Rule` interface
|
|
168
|
+
2. `src/rules/index.ts` — add to `activeRules`
|
|
169
|
+
3. `src/types.ts` — add β coefficient to `PAPER_BETA_COEFFICIENTS`
|
|
170
|
+
|
|
171
|
+
See [RESEARCH.md](RESEARCH.md) for the full extension guide, β coefficient derivation, and known limitations.
|
|
172
|
+
|
|
173
|
+
Researchers extending this tool are encouraged to:
|
|
174
|
+
|
|
175
|
+
- Add rules targeting violation types identified in replication studies
|
|
176
|
+
- Update β coefficients with your own DiD estimates
|
|
177
|
+
- Contribute fixtures from real AI-generated code samples
|
|
178
|
+
|
|
179
|
+
---
|
|
180
|
+
|
|
181
|
+
## Citation
|
|
182
|
+
|
|
183
|
+
If you use qualm in research, please cite the underlying empirical study:
|
|
184
|
+
|
|
185
|
+
```bibtex
|
|
186
|
+
@article{sharma2026qualm,
|
|
187
|
+
title = {Accessibility and Semantic Quality Regressions in AI-Assisted React Development: An Empirical Study},
|
|
188
|
+
author = {Sharma, Somil},
|
|
189
|
+
year = {2026},
|
|
190
|
+
month = {May},
|
|
191
|
+
journal = {arXiv preprint},
|
|
192
|
+
note = {Independent Researcher, Gurugram, India},
|
|
193
|
+
url = {https://doi.org/10.5281/zenodo.20482307}
|
|
194
|
+
}
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
---
|
|
198
|
+
|
|
199
|
+
## License
|
|
200
|
+
|
|
201
|
+
MIT © Somil Sharma
|
package/RESEARCH.md
ADDED
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
# RESEARCH.md — qualm Methodology
|
|
2
|
+
|
|
3
|
+
qualm operationalises the empirical findings of:
|
|
4
|
+
|
|
5
|
+
> Sharma, S. (2026). *Accessibility and Semantic Quality Regressions in AI-Assisted React Development: An Empirical Study.* arXiv preprint.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Rule-to-Taxonomy Mapping
|
|
10
|
+
|
|
11
|
+
The paper defines three primary violation categories (Table 2, Table 5). qualm extends these to six rules, all of which are detectable via static AST analysis without a DOM runtime.
|
|
12
|
+
|
|
13
|
+
| qualm Rule ID | Paper Category | Paper β (Table 5) | Detection Method |
|
|
14
|
+
|-----------------------|---------------------|--------------------|------------------|
|
|
15
|
+
| `document-structure` | `document_structure`| **+0.007** (dominant) | Interactive handler on generic container (`<div>`, `<span>`) without `role` |
|
|
16
|
+
| `landmark-structure` | `document_structure`| **+0.007** (shared) | Generic container with landmark-suggesting `className`/`id` |
|
|
17
|
+
| `aria-correctness` | `aria_specific` | +0.002 | Invalid string literal on boolean ARIA attribute |
|
|
18
|
+
| `interactive-semantics` | `semantic_naming` | −0.003 | Missing `alt` on `<img>`; icon-only button without `aria-label` |
|
|
19
|
+
| `heading-hierarchy` | `document_structure`| — | Skipped heading level within component |
|
|
20
|
+
| `form-semantics` | `semantic_naming` | — | Form control without associated `<label>` or `aria-label` |
|
|
21
|
+
|
|
22
|
+
The paper's `semantic_naming` category has β = −0.003 (Table 5), indicating that AI adoption may slightly *improve* naming violations — possibly because AI tools generate boilerplate `aria-label` attributes. This is reflected in the lower β weight assigned to `interactive_semantics` and `form_semantics` in qualm's semantic score.
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Semantic Score Operationalisation
|
|
27
|
+
|
|
28
|
+
The paper reports AST-based semantic score as a key outcome variable (Section 3.6.2):
|
|
29
|
+
|
|
30
|
+
- **Treated PRE mean**: 0.989 (SD 0.022) — Table A1
|
|
31
|
+
- **Treated POST mean**: 0.983 (SD 0.027) — Table A1
|
|
32
|
+
- **DiD estimate**: β = +0.005, p = 0.075 (marginal significance)
|
|
33
|
+
- **Tobit-corrected**: β = +0.005, p = 0.092
|
|
34
|
+
|
|
35
|
+
qualm computes a file-level semantic score in [0, 1] using:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
semanticScore = 1.0 - Σ_c [ (β_c / Σβ) × count_c × 0.05 ]
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Where:
|
|
42
|
+
- `β_c` = paper β coefficient for category `c` (from Table 5)
|
|
43
|
+
- `count_c` = violation count in category `c`
|
|
44
|
+
- `0.05` = per-violation deduction constant, calibrated against paper baselines
|
|
45
|
+
- Result is clamped to [0.0, 1.0]
|
|
46
|
+
|
|
47
|
+
This formula weights violations by their empirically estimated contribution to semantic HTML degradation — `document_structure` violations (β = 0.007) deduct proportionally more than `aria_correctness` violations (β = 0.002).
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## How `--diff-branch` Simulates the DiD Estimator
|
|
52
|
+
|
|
53
|
+
The paper uses a two-way fixed effects DiD design (Section 3.2):
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
y_{it} = α_i + γ_t + β·(Treated_i × Post_{it}) + X_{it}δ + ε_{it}
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
qualm's `--diff-branch` command simulates the "before/after" component of this estimator at the single-file level:
|
|
60
|
+
|
|
61
|
+
- **Before** = file content at the specified git branch (Pre-treatment)
|
|
62
|
+
- **After** = current working tree content (Post-treatment)
|
|
63
|
+
- **deltaSemanticScore** = after.semanticScore − before.semanticScore (analogous to β)
|
|
64
|
+
- **regressionDetected** = true when deltaSemanticScore < 0 OR new `document_structure`/`landmark_structure` violations appear
|
|
65
|
+
|
|
66
|
+
This is a per-file approximation — not a panel estimate — but it provides the same directional signal: has this code change *introduced* accessibility debt?
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## β Coefficients and Semantic Score Weights
|
|
71
|
+
|
|
72
|
+
From Table 5 of the paper (DiD estimates by violation category):
|
|
73
|
+
|
|
74
|
+
| Category | β (paper) | Normalised weight |
|
|
75
|
+
|-----------------------|-----------|-------------------|
|
|
76
|
+
| `document_structure` | +0.007 | 0.333 |
|
|
77
|
+
| `landmark_structure` | +0.004 | 0.190 |
|
|
78
|
+
| `heading_hierarchy` | +0.003 | 0.143 |
|
|
79
|
+
| `interactive_semantics` | +0.003 | 0.143 |
|
|
80
|
+
| `aria_correctness` | +0.002 | 0.095 |
|
|
81
|
+
| `form_semantics` | +0.002 | 0.095 |
|
|
82
|
+
| **Sum** | **0.021** | **1.000** |
|
|
83
|
+
|
|
84
|
+
The three categories without a direct paper estimate (`landmark_structure`, `heading_hierarchy`, `form_semantics`) use conservative values set between the two confirmed paper values (0.002–0.007). Researchers should update these with their own empirical estimates as replication data becomes available.
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
## Extending qualm with New Rules
|
|
89
|
+
|
|
90
|
+
1. Create `src/rules/your-rule.ts` implementing the `Rule` interface from `src/types.ts`
|
|
91
|
+
2. Export a single named constant: `export const yourRule: Rule = { ... }`
|
|
92
|
+
3. Add it to `activeRules` array in `src/rules/index.ts`
|
|
93
|
+
4. Choose an existing `ViolationCategory` or extend the type in `src/types.ts`
|
|
94
|
+
5. Add corresponding β to `PAPER_BETA_COEFFICIENTS` in `src/types.ts`
|
|
95
|
+
6. Write tests in `tests/rules/your-rule.test.ts`
|
|
96
|
+
7. Add fixture files in `tests/fixtures/`
|
|
97
|
+
|
|
98
|
+
Rule `create(context)` receives a `RuleContext` with:
|
|
99
|
+
- `context.report({ message, fixSuggestion, location, snippet? })` — emit a violation
|
|
100
|
+
- `context.getSourceCode(node)` — get source text for a node
|
|
101
|
+
- `context.getLoc(node)` — get `{ line, column, lineEnd, columnEnd }` for a node
|
|
102
|
+
|
|
103
|
+
Listeners are keyed by AST node type. Use `simpleTraverse` visitor keys:
|
|
104
|
+
- `JSXOpeningElement` — fires on `<Tag` opening
|
|
105
|
+
- `JSXElement` — fires on the full `<Tag>...</Tag>` (gives access to children)
|
|
106
|
+
- `JSXAttribute` — fires on each attribute
|
|
107
|
+
- `Program:exit` — fires after full traversal (use for post-collection checks)
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## Known Limitations
|
|
112
|
+
|
|
113
|
+
1. **Single-file scope**: Rules operate within one file at a time. Cross-component prop drilling and component-tree accessibility errors (e.g. context-dependent ARIA relationships) are not detectable.
|
|
114
|
+
|
|
115
|
+
2. **No runtime DOM**: qualm is purely static. Runtime violations that only manifest after rendering (e.g. dynamic `aria-hidden` state, focus management) are outside scope. The paper (Section 3.6.1) notes 60.4% of component-snapshot pairs failed to render — qualm's static approach avoids this limitation entirely.
|
|
116
|
+
|
|
117
|
+
3. **JavaScript only**: qualm analyses JSX/TSX as written. Components using headless UI libraries (Radix, Ariakit, Chakra UI) that wrap semantic elements may score lower than their actual HTML output warrants — exactly the ambiguity noted in the paper's Appendix D.
|
|
118
|
+
|
|
119
|
+
4. **Form label matching is file-scoped**: `<label htmlFor="x">` and `<input id="x">` must appear in the same file for the association to be detected. Split across files → false positive.
|
|
120
|
+
|
|
121
|
+
5. **Single-rater construct validity**: The paper's AST score has Spearman ρ = 0.751 with expert ratings (Appendix D, N=53, single rater). qualm's semantic score inherits this validity ceiling and the single-rater limitation.
|
|
122
|
+
|
|
123
|
+
6. **β coefficients not replicated yet**: The paper's Table 5 estimates are from 74 repos across 2,374 repo-months. qualm uses these as calibration weights but they should be treated as priors pending independent replication.
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## Citation
|
|
128
|
+
|
|
129
|
+
```bibtex
|
|
130
|
+
@article{sharma2026qualm,
|
|
131
|
+
title = {Accessibility and Semantic Quality Regressions in AI-Assisted React Development: An Empirical Study},
|
|
132
|
+
author = {Sharma, Somil},
|
|
133
|
+
year = {2026},
|
|
134
|
+
month = {May},
|
|
135
|
+
journal = {arXiv preprint},
|
|
136
|
+
note = {Independent Researcher, Gurugram, India},
|
|
137
|
+
url = {https://doi.org/10.5281/zenodo.20482307}
|
|
138
|
+
}
|
|
139
|
+
```
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"analyser.d.ts","sourceRoot":"","sources":["../src/analyser.ts"],"names":[],"mappings":"AAIA,OAAO,EACL,kBAAkB,EAKnB,MAAM,SAAS,CAAC;AAEjB,wBAAgB,WAAW,CAAC,UAAU,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,kBAAkB,CAiFpF"}
|
package/dist/analyser.js
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.analyseFile = analyseFile;
|
|
4
|
+
const typescript_estree_1 = require("@typescript-eslint/typescript-estree");
|
|
5
|
+
const parser_1 = require("./parser");
|
|
6
|
+
const index_1 = require("./rules/index");
|
|
7
|
+
const complexity_1 = require("./rules/complexity");
|
|
8
|
+
const types_1 = require("./types");
|
|
9
|
+
function analyseFile(sourceCode, filePath) {
|
|
10
|
+
const ast = (0, parser_1.parseTSX)(sourceCode, filePath);
|
|
11
|
+
const violations = [];
|
|
12
|
+
const createContext = (ruleId, category, severity) => ({
|
|
13
|
+
report(v) {
|
|
14
|
+
violations.push({ ...v, ruleId, category, severity });
|
|
15
|
+
},
|
|
16
|
+
getSourceCode(node) {
|
|
17
|
+
if (node.range) {
|
|
18
|
+
return sourceCode.slice(node.range[0], node.range[1]);
|
|
19
|
+
}
|
|
20
|
+
return '';
|
|
21
|
+
},
|
|
22
|
+
getLoc(node) {
|
|
23
|
+
return {
|
|
24
|
+
line: node.loc.start.line,
|
|
25
|
+
column: node.loc.start.column,
|
|
26
|
+
lineEnd: node.loc.end.line,
|
|
27
|
+
columnEnd: node.loc.end.column
|
|
28
|
+
};
|
|
29
|
+
}
|
|
30
|
+
});
|
|
31
|
+
// Build a listener map: nodeType -> list of callbacks
|
|
32
|
+
const listeners = {};
|
|
33
|
+
for (const rule of index_1.activeRules) {
|
|
34
|
+
const ctx = createContext(rule.id, rule.category, rule.severity);
|
|
35
|
+
const ruleListeners = rule.create(ctx);
|
|
36
|
+
for (const [nodeType, callback] of Object.entries(ruleListeners)) {
|
|
37
|
+
if (!listeners[nodeType])
|
|
38
|
+
listeners[nodeType] = [];
|
|
39
|
+
listeners[nodeType].push(callback);
|
|
40
|
+
}
|
|
41
|
+
}
|
|
42
|
+
// Collect Program:exit callbacks separately — they run after traversal
|
|
43
|
+
const exitCallbacks = listeners['Program:exit'] ?? [];
|
|
44
|
+
delete listeners['Program:exit'];
|
|
45
|
+
(0, typescript_estree_1.simpleTraverse)(ast, {
|
|
46
|
+
enter(node) {
|
|
47
|
+
const callbacks = listeners[node.type];
|
|
48
|
+
if (callbacks) {
|
|
49
|
+
for (const cb of callbacks)
|
|
50
|
+
cb(node);
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
});
|
|
54
|
+
// Fire Program:exit listeners
|
|
55
|
+
for (const cb of exitCallbacks) {
|
|
56
|
+
cb(ast);
|
|
57
|
+
}
|
|
58
|
+
const metrics = (0, complexity_1.calculateComplexityMetrics)(ast, sourceCode);
|
|
59
|
+
// Weighted semantic score using paper β coefficients from Table 5 (Sharma 2026).
|
|
60
|
+
// Each violation in a category deducts (β_i / Σβ) * 0.05 from the score.
|
|
61
|
+
// The 0.05-per-violation constant is calibrated so that the treated-post mean
|
|
62
|
+
// of ~0.983 (Table A1) is reachable with a small number of violations.
|
|
63
|
+
const totalBeta = Object.values(types_1.PAPER_BETA_COEFFICIENTS).reduce((a, b) => a + b, 0);
|
|
64
|
+
const violationsByCategory = violations.reduce((acc, v) => {
|
|
65
|
+
acc[v.category] = (acc[v.category] ?? 0) + 1;
|
|
66
|
+
return acc;
|
|
67
|
+
}, {});
|
|
68
|
+
let totalDeduction = 0;
|
|
69
|
+
for (const [category, count] of Object.entries(violationsByCategory)) {
|
|
70
|
+
const beta = types_1.PAPER_BETA_COEFFICIENTS[category] ?? 0;
|
|
71
|
+
const weight = beta / totalBeta;
|
|
72
|
+
totalDeduction += weight * count * 0.05;
|
|
73
|
+
}
|
|
74
|
+
const semanticScore = Math.max(0.0, Math.min(1.0, 1.0 - totalDeduction));
|
|
75
|
+
return { filePath, violations, metrics, semanticScore };
|
|
76
|
+
}
|
|
77
|
+
//# sourceMappingURL=analyser.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"analyser.js","sourceRoot":"","sources":["../src/analyser.ts"],"names":[],"mappings":";;AAYA,kCAiFC;AA7FD,4EAAsE;AACtE,qCAAoC;AACpC,yCAA4C;AAC5C,mDAAgE;AAChE,mCAMiB;AAEjB,SAAgB,WAAW,CAAC,UAAkB,EAAE,QAAgB;IAC9D,MAAM,GAAG,GAAG,IAAA,iBAAQ,EAAC,UAAU,EAAE,QAAQ,CAAC,CAAC;IAC3C,MAAM,UAAU,GAAgB,EAAE,CAAC;IAEnC,MAAM,aAAa,GAAG,CACpB,MAAc,EACd,QAA2B,EAC3B,QAA+B,EAClB,EAAE,CAAC,CAAC;QACjB,MAAM,CAAC,CAAC;YACN,UAAU,CAAC,IAAI,CAAC,EAAE,GAAG,CAAC,EAAE,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,CAAC,CAAC;QACxD,CAAC;QACD,aAAa,CAAC,IAAS;YACrB,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;gBACf,OAAO,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;YACxD,CAAC;YACD,OAAO,EAAE,CAAC;QACZ,CAAC;QACD,MAAM,CAAC,IAAS;YACd,OAAO;gBACL,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI;gBACzB,MAAM,EAAE,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM;gBAC7B,OAAO,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI;gBAC1B,SAAS,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM;aAC/B,CAAC;QACJ,CAAC;KACF,CAAC,CAAC;IAEH,sDAAsD;IACtD,MAAM,SAAS,GAA4C,EAAE,CAAC;IAE9D,KAAK,MAAM,IAAI,IAAI,mBAAW,EAAE,CAAC;QAC/B,MAAM,GAAG,GAAG,aAAa,CAAC,IAAI,CAAC,EAAE,EAAE,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC;QACjE,MAAM,aAAa,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;QACvC,KAAK,MAAM,CAAC,QAAQ,EAAE,QAAQ,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,aAAa,CAAC,EAAE,CAAC;YACjE,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC;gBAAE,SAAS,CAAC,QAAQ,CAAC,GAAG,EAAE,CAAC;YACnD,SAAS,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,QAA+B,CAAC,CAAC;QAC5D,CAAC;IACH,CAAC;IAED,uEAAuE;IACvE,MAAM,aAAa,GAAG,SAAS,CAAC,cAAc,CAAC,IAAI,EAAE,CAAC;IACtD,OAAO,SAAS,CAAC,cAAc,CAAC,CAAC;IAEjC,IAAA,kCAAc,EAAC,GAAG,EAAE;QAClB,KAAK,CAAC,IAAI;YACR,MAAM,SAAS,GAAG,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YACvC,IAAI,SAAS,EAAE,CAAC;gBACd,KAAK,MAAM,EAAE,IAAI,SAAS;oBAAE,EAAE,CAAC,IAAI,CAAC,CAAC;YACvC,CAAC;QACH,CAAC;KACF,CAAC,CAAC;IAEH,8BAA8B;IAC9B,KAAK,MAAM,EAAE,IAAI,aAAa,EAAE,CAAC;QAC/B,EAAE,CAAC,GAAG,CAAC,CAAC;IACV,CAAC;IAED,MAAM,OAAO,GAAG,IAAA,uCAA0B,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;IAE5D,iFAAiF;IACjF,yEAAyE;IACzE,8EAA8E;IAC9E,uEAAuE;IACvE,MAAM,SAAS,GAAG,MAAM,CAAC,MAAM,CAAC,+BAAuB,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC;IAEpF,MAAM,oBAAoB,GAAG,UAAU,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE;QACxD,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC;QAC7C,OAAO,GAAG,CAAC;IACb,CAAC,EAAE,EAAgD,CAAC,CAAC;IAErD,IAAI,cAAc,GAAG,CAAC,CAAC;IACvB,KAAK,MAAM,CAAC,QAAQ,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,oBAAoB,CAAC,EAAE,CAAC;QACrE,MAAM,IAAI,GAAG,+BAAuB,CAAC,QAA6B,CAAC,IAAI,CAAC,CAAC;QACzE,MAAM,MAAM,GAAG,IAAI,GAAG,SAAS,CAAC;QAChC,cAAc,IAAI,MAAM,GAAI,KAAgB,GAAG,IAAI,CAAC;IACtD,CAAC;IAED,MAAM,aAAa,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,GAAG,cAAc,CAAC,CAAC,CAAC;IAEzE,OAAO,EAAE,QAAQ,EAAE,UAAU,EAAE,OAAO,EAAE,aAAa,EAAE,CAAC;AAC1D,CAAC"}
|
package/dist/cli.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":""}
|
package/dist/cli.js
ADDED
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
"use strict";
|
|
3
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
4
|
+
const commander_1 = require("commander");
|
|
5
|
+
const fs_1 = require("fs");
|
|
6
|
+
const path_1 = require("path");
|
|
7
|
+
const child_process_1 = require("child_process");
|
|
8
|
+
const glob_1 = require("glob");
|
|
9
|
+
const analyser_1 = require("./analyser");
|
|
10
|
+
const diff_1 = require("./diff");
|
|
11
|
+
const terminal_1 = require("./reporters/terminal");
|
|
12
|
+
const json_1 = require("./reporters/json");
|
|
13
|
+
const sarif_1 = require("./reporters/sarif");
|
|
14
|
+
// eslint-disable-next-line @typescript-eslint/no-var-requires
|
|
15
|
+
const pkg = require('../package.json');
|
|
16
|
+
const program = new commander_1.Command();
|
|
17
|
+
program
|
|
18
|
+
.name('qualm')
|
|
19
|
+
.description('Static AST-level quality analyser for LLM-generated React/TypeScript code.\n' +
|
|
20
|
+
'Based on Sharma (2026) empirical study: https://doi.org/10.5281/zenodo.20482307')
|
|
21
|
+
.version(pkg.version);
|
|
22
|
+
program
|
|
23
|
+
.argument('<paths...>', 'Files or directories to analyse (supports .tsx, .ts, .jsx, .js)')
|
|
24
|
+
.option('-f, --format <format>', 'Output format: terminal, json, sarif', 'terminal')
|
|
25
|
+
.option('-o, --output <file>', 'Write output to file instead of stdout')
|
|
26
|
+
.option('--diff-branch <branch>', 'Compare current files against git branch to detect regressions')
|
|
27
|
+
.option('--fail-on <level>', 'Exit with code 1 if violations of this severity exist: error, warning', 'error')
|
|
28
|
+
.option('--research-mode', 'Output metrics in Sharma (2026) taxonomy format')
|
|
29
|
+
.action(async (paths, options) => {
|
|
30
|
+
try {
|
|
31
|
+
const files = [];
|
|
32
|
+
for (const p of paths) {
|
|
33
|
+
const resolved = (0, path_1.resolve)(p);
|
|
34
|
+
const ext = (0, path_1.extname)(resolved);
|
|
35
|
+
if (['.tsx', '.ts', '.jsx', '.js'].includes(ext)) {
|
|
36
|
+
files.push(resolved);
|
|
37
|
+
}
|
|
38
|
+
else {
|
|
39
|
+
// Directory — glob for React/TS files
|
|
40
|
+
const found = await (0, glob_1.glob)(`${resolved}/**/*.{tsx,ts,jsx,js}`, {
|
|
41
|
+
ignore: ['**/node_modules/**', '**/dist/**', '**/.next/**']
|
|
42
|
+
});
|
|
43
|
+
files.push(...found);
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
if (files.length === 0) {
|
|
47
|
+
console.error('qualm: No TypeScript/React files found at the specified paths.');
|
|
48
|
+
process.exit(1);
|
|
49
|
+
}
|
|
50
|
+
const results = [];
|
|
51
|
+
for (const file of files) {
|
|
52
|
+
if (!(0, fs_1.existsSync)(file))
|
|
53
|
+
continue;
|
|
54
|
+
const source = (0, fs_1.readFileSync)(file, 'utf-8');
|
|
55
|
+
if (options.diffBranch) {
|
|
56
|
+
let beforeContent = null;
|
|
57
|
+
try {
|
|
58
|
+
beforeContent = (0, child_process_1.execSync)(`git show ${options.diffBranch}:${file}`, {
|
|
59
|
+
encoding: 'utf-8',
|
|
60
|
+
stdio: ['pipe', 'pipe', 'pipe']
|
|
61
|
+
});
|
|
62
|
+
}
|
|
63
|
+
catch {
|
|
64
|
+
beforeContent = null;
|
|
65
|
+
}
|
|
66
|
+
const diff = (0, diff_1.diffFiles)(beforeContent, source, file);
|
|
67
|
+
if (options.format === 'json') {
|
|
68
|
+
const out = (0, json_1.renderDiffJSON)(diff);
|
|
69
|
+
if (options.output) {
|
|
70
|
+
(0, fs_1.writeFileSync)(options.output, out, 'utf-8');
|
|
71
|
+
}
|
|
72
|
+
else {
|
|
73
|
+
console.log(out);
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
else {
|
|
77
|
+
(0, terminal_1.renderDiffTerminal)(diff);
|
|
78
|
+
}
|
|
79
|
+
if (diff.regressionDetected) {
|
|
80
|
+
process.exit(1);
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
else {
|
|
84
|
+
results.push((0, analyser_1.analyseFile)(source, file));
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
if (!options.diffBranch && results.length > 0) {
|
|
88
|
+
let output = null;
|
|
89
|
+
if (options.researchMode) {
|
|
90
|
+
(0, terminal_1.renderResearchMode)(results);
|
|
91
|
+
}
|
|
92
|
+
else if (options.format === 'json') {
|
|
93
|
+
output = (0, json_1.renderJSON)(results);
|
|
94
|
+
}
|
|
95
|
+
else if (options.format === 'sarif') {
|
|
96
|
+
output = (0, sarif_1.renderSARIF)(results);
|
|
97
|
+
}
|
|
98
|
+
else {
|
|
99
|
+
(0, terminal_1.renderTerminal)(results);
|
|
100
|
+
}
|
|
101
|
+
if (output !== null) {
|
|
102
|
+
if (options.output) {
|
|
103
|
+
(0, fs_1.writeFileSync)(options.output, output, 'utf-8');
|
|
104
|
+
}
|
|
105
|
+
else {
|
|
106
|
+
console.log(output);
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
// Determine exit code
|
|
111
|
+
if (!options.diffBranch) {
|
|
112
|
+
const failLevel = options.failOn;
|
|
113
|
+
const hasFailure = results.some(r => r.violations.some(v => {
|
|
114
|
+
if (failLevel === 'warning') {
|
|
115
|
+
return v.severity === 'error' || v.severity === 'warning';
|
|
116
|
+
}
|
|
117
|
+
return v.severity === 'error';
|
|
118
|
+
}));
|
|
119
|
+
process.exit(hasFailure ? 1 : 0);
|
|
120
|
+
}
|
|
121
|
+
}
|
|
122
|
+
catch (err) {
|
|
123
|
+
console.error('qualm error:', err.message);
|
|
124
|
+
process.exit(2);
|
|
125
|
+
}
|
|
126
|
+
});
|
|
127
|
+
program.parse();
|
|
128
|
+
//# sourceMappingURL=cli.js.map
|
package/dist/cli.js.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";;;AACA,yCAAoC;AACpC,2BAA6D;AAC7D,+BAAwC;AACxC,iDAAyC;AACzC,+BAA4B;AAC5B,yCAAyC;AACzC,iCAAmC;AACnC,mDAA8F;AAC9F,2CAA8D;AAC9D,6CAAgD;AAGhD,8DAA8D;AAC9D,MAAM,GAAG,GAAG,OAAO,CAAC,iBAAiB,CAAwB,CAAC;AAE9D,MAAM,OAAO,GAAG,IAAI,mBAAO,EAAE,CAAC;AAE9B,OAAO;KACJ,IAAI,CAAC,OAAO,CAAC;KACb,WAAW,CACV,8EAA8E;IAC9E,iFAAiF,CAClF;KACA,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;AAExB,OAAO;KACJ,QAAQ,CAAC,YAAY,EAAE,iEAAiE,CAAC;KACzF,MAAM,CAAC,uBAAuB,EAAE,sCAAsC,EAAE,UAAU,CAAC;KACnF,MAAM,CAAC,qBAAqB,EAAE,wCAAwC,CAAC;KACvE,MAAM,CAAC,wBAAwB,EAAE,gEAAgE,CAAC;KAClG,MAAM,CAAC,mBAAmB,EAAE,uEAAuE,EAAE,OAAO,CAAC;KAC7G,MAAM,CAAC,iBAAiB,EAAE,iDAAiD,CAAC;KAC5E,MAAM,CAAC,KAAK,EAAE,KAAe,EAAE,OAM/B,EAAE,EAAE;IACH,IAAI,CAAC;QACH,MAAM,KAAK,GAAa,EAAE,CAAC;QAE3B,KAAK,MAAM,CAAC,IAAI,KAAK,EAAE,CAAC;YACtB,MAAM,QAAQ,GAAG,IAAA,cAAO,EAAC,CAAC,CAAC,CAAC;YAC5B,MAAM,GAAG,GAAG,IAAA,cAAO,EAAC,QAAQ,CAAC,CAAC;YAE9B,IAAI,CAAC,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,KAAK,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;gBACjD,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;YACvB,CAAC;iBAAM,CAAC;gBACN,sCAAsC;gBACtC,MAAM,KAAK,GAAG,MAAM,IAAA,WAAI,EAAC,GAAG,QAAQ,uBAAuB,EAAE;oBAC3D,MAAM,EAAE,CAAC,oBAAoB,EAAE,YAAY,EAAE,aAAa,CAAC;iBAC5D,CAAC,CAAC;gBACH,KAAK,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,CAAC;YACvB,CAAC;QACH,CAAC;QAED,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YACvB,OAAO,CAAC,KAAK,CAAC,gEAAgE,CAAC,CAAC;YAChF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;QAED,MAAM,OAAO,GAAyB,EAAE,CAAC;QAEzC,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,CAAC,IAAA,eAAU,EAAC,IAAI,CAAC;gBAAE,SAAS;YAChC,MAAM,MAAM,GAAG,IAAA,iBAAY,EAAC,IAAI,EAAE,OAAO,CAAC,CAAC;YAE3C,IAAI,OAAO,CAAC,UAAU,EAAE,CAAC;gBACvB,IAAI,aAAa,GAAkB,IAAI,CAAC;gBACxC,IAAI,CAAC;oBACH,aAAa,GAAG,IAAA,wBAAQ,EAAC,YAAY,OAAO,CAAC,UAAU,IAAI,IAAI,EAAE,EAAE;wBACjE,QAAQ,EAAE,OAAO;wBACjB,KAAK,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC;qBAChC,CAAC,CAAC;gBACL,CAAC;gBAAC,MAAM,CAAC;oBACP,aAAa,GAAG,IAAI,CAAC;gBACvB,CAAC;gBAED,MAAM,IAAI,GAAG,IAAA,gBAAS,EAAC,aAAa,EAAE,MAAM,EAAE,IAAI,CAAC,CAAC;gBAEpD,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;oBAC9B,MAAM,GAAG,GAAG,IAAA,qBAAc,EAAC,IAAI,CAAC,CAAC;oBACjC,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;wBACnB,IAAA,kBAAa,EAAC,OAAO,CAAC,MAAM,EAAE,GAAG,EAAE,OAAO,CAAC,CAAC;oBAC9C,CAAC;yBAAM,CAAC;wBACN,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;oBACnB,CAAC;gBACH,CAAC;qBAAM,CAAC;oBACN,IAAA,6BAAkB,EAAC,IAAI,CAAC,CAAC;gBAC3B,CAAC;gBAED,IAAI,IAAI,CAAC,kBAAkB,EAAE,CAAC;oBAC5B,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;gBAClB,CAAC;YACH,CAAC;iBAAM,CAAC;gBACN,OAAO,CAAC,IAAI,CAAC,IAAA,sBAAW,EAAC,MAAM,EAAE,IAAI,CAAC,CAAC,CAAC;YAC1C,CAAC;QACH,CAAC;QAED,IAAI,CAAC,OAAO,CAAC,UAAU,IAAI,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC9C,IAAI,MAAM,GAAkB,IAAI,CAAC;YAEjC,IAAI,OAAO,CAAC,YAAY,EAAE,CAAC;gBACzB,IAAA,6BAAkB,EAAC,OAAO,CAAC,CAAC;YAC9B,CAAC;iBAAM,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;gBACrC,MAAM,GAAG,IAAA,iBAAU,EAAC,OAAO,CAAC,CAAC;YAC/B,CAAC;iBAAM,IAAI,OAAO,CAAC,MAAM,KAAK,OAAO,EAAE,CAAC;gBACtC,MAAM,GAAG,IAAA,mBAAW,EAAC,OAAO,CAAC,CAAC;YAChC,CAAC;iBAAM,CAAC;gBACN,IAAA,yBAAc,EAAC,OAAO,CAAC,CAAC;YAC1B,CAAC;YAED,IAAI,MAAM,KAAK,IAAI,EAAE,CAAC;gBACpB,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;oBACnB,IAAA,kBAAa,EAAC,OAAO,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,CAAC,CAAC;gBACjD,CAAC;qBAAM,CAAC;oBACN,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;gBACtB,CAAC;YACH,CAAC;QACH,CAAC;QAED,sBAAsB;QACtB,IAAI,CAAC,OAAO,CAAC,UAAU,EAAE,CAAC;YACxB,MAAM,SAAS,GAAG,OAAO,CAAC,MAAM,CAAC;YACjC,MAAM,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAClC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE;gBACpB,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;oBAC5B,OAAO,CAAC,CAAC,QAAQ,KAAK,OAAO,IAAI,CAAC,CAAC,QAAQ,KAAK,SAAS,CAAC;gBAC5D,CAAC;gBACD,OAAO,CAAC,CAAC,QAAQ,KAAK,OAAO,CAAC;YAChC,CAAC,CAAC,CACH,CAAC;YAEF,OAAO,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACnC,CAAC;IACH,CAAC;IAAC,OAAO,GAAQ,EAAE,CAAC;QAClB,OAAO,CAAC,KAAK,CAAC,cAAc,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC;QAC3C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAClB,CAAC;AACH,CAAC,CAAC,CAAC;AAEL,OAAO,CAAC,KAAK,EAAE,CAAC"}
|
package/dist/diff.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"diff.d.ts","sourceRoot":"","sources":["../src/diff.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,UAAU,EAAgC,MAAM,SAAS,CAAC;AAUnE,wBAAgB,SAAS,CACvB,aAAa,EAAE,MAAM,GAAG,IAAI,EAC5B,YAAY,EAAE,MAAM,EACpB,QAAQ,EAAE,MAAM,GACf,UAAU,CA8DZ"}
|
package/dist/diff.js
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.diffFiles = diffFiles;
|
|
4
|
+
const analyser_1 = require("./analyser");
|
|
5
|
+
// Fingerprint uses ruleId + column offset + snippet to identify a violation
|
|
6
|
+
// across refactors. Line numbers are intentionally excluded because they are
|
|
7
|
+
// unstable when code is reformatted; column + snippet provides sufficient
|
|
8
|
+
// structural identity within a file.
|
|
9
|
+
function fingerprintViolation(v) {
|
|
10
|
+
return `${v.ruleId}::${v.location.column}::${(v.snippet ?? '').slice(0, 60)}`;
|
|
11
|
+
}
|
|
12
|
+
function diffFiles(beforeContent, afterContent, filePath) {
|
|
13
|
+
const afterResult = (0, analyser_1.analyseFile)(afterContent, filePath);
|
|
14
|
+
if (!beforeContent) {
|
|
15
|
+
return {
|
|
16
|
+
filePath,
|
|
17
|
+
before: null,
|
|
18
|
+
after: {
|
|
19
|
+
violationsCount: afterResult.violations.length,
|
|
20
|
+
semanticScore: afterResult.semanticScore,
|
|
21
|
+
metrics: afterResult.metrics
|
|
22
|
+
},
|
|
23
|
+
deltaSemanticScore: 0,
|
|
24
|
+
addedViolations: afterResult.violations,
|
|
25
|
+
removedViolations: [],
|
|
26
|
+
regressionDetected: afterResult.violations.length > 0,
|
|
27
|
+
regressionCategories: [...new Set(afterResult.violations.map(v => v.category))]
|
|
28
|
+
};
|
|
29
|
+
}
|
|
30
|
+
const beforeResult = (0, analyser_1.analyseFile)(beforeContent, filePath);
|
|
31
|
+
const beforeFingerprints = new Set(beforeResult.violations.map(fingerprintViolation));
|
|
32
|
+
const afterFingerprints = new Set(afterResult.violations.map(fingerprintViolation));
|
|
33
|
+
const addedViolations = afterResult.violations.filter(v => !beforeFingerprints.has(fingerprintViolation(v)));
|
|
34
|
+
const removedViolations = beforeResult.violations.filter(v => !afterFingerprints.has(fingerprintViolation(v)));
|
|
35
|
+
const deltaSemanticScore = afterResult.semanticScore - beforeResult.semanticScore;
|
|
36
|
+
const regressionDetected = deltaSemanticScore < 0 ||
|
|
37
|
+
addedViolations.some(v => v.category === 'document_structure' || v.category === 'landmark_structure');
|
|
38
|
+
const regressionCategories = [
|
|
39
|
+
...new Set(addedViolations.map(v => v.category))
|
|
40
|
+
];
|
|
41
|
+
return {
|
|
42
|
+
filePath,
|
|
43
|
+
before: {
|
|
44
|
+
violationsCount: beforeResult.violations.length,
|
|
45
|
+
semanticScore: beforeResult.semanticScore,
|
|
46
|
+
metrics: beforeResult.metrics
|
|
47
|
+
},
|
|
48
|
+
after: {
|
|
49
|
+
violationsCount: afterResult.violations.length,
|
|
50
|
+
semanticScore: afterResult.semanticScore,
|
|
51
|
+
metrics: afterResult.metrics
|
|
52
|
+
},
|
|
53
|
+
deltaSemanticScore,
|
|
54
|
+
addedViolations,
|
|
55
|
+
removedViolations,
|
|
56
|
+
regressionDetected,
|
|
57
|
+
regressionCategories
|
|
58
|
+
};
|
|
59
|
+
}
|
|
60
|
+
//# sourceMappingURL=diff.js.map
|
package/dist/diff.js.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"diff.js","sourceRoot":"","sources":["../src/diff.ts"],"names":[],"mappings":";;AAWA,8BAkEC;AA7ED,yCAAyC;AAGzC,4EAA4E;AAC5E,6EAA6E;AAC7E,0EAA0E;AAC1E,qCAAqC;AACrC,SAAS,oBAAoB,CAAC,CAAY;IACxC,OAAO,GAAG,CAAC,CAAC,MAAM,KAAK,CAAC,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,EAAE,CAAC;AAChF,CAAC;AAED,SAAgB,SAAS,CACvB,aAA4B,EAC5B,YAAoB,EACpB,QAAgB;IAEhB,MAAM,WAAW,GAAG,IAAA,sBAAW,EAAC,YAAY,EAAE,QAAQ,CAAC,CAAC;IAExD,IAAI,CAAC,aAAa,EAAE,CAAC;QACnB,OAAO;YACL,QAAQ;YACR,MAAM,EAAE,IAAI;YACZ,KAAK,EAAE;gBACL,eAAe,EAAE,WAAW,CAAC,UAAU,CAAC,MAAM;gBAC9C,aAAa,EAAE,WAAW,CAAC,aAAa;gBACxC,OAAO,EAAE,WAAW,CAAC,OAAO;aAC7B;YACD,kBAAkB,EAAE,CAAC;YACrB,eAAe,EAAE,WAAW,CAAC,UAAU;YACvC,iBAAiB,EAAE,EAAE;YACrB,kBAAkB,EAAE,WAAW,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC;YACrD,oBAAoB,EAAE,CAAC,GAAG,IAAI,GAAG,CAAC,WAAW,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAwB;SACvG,CAAC;IACJ,CAAC;IAED,MAAM,YAAY,GAAG,IAAA,sBAAW,EAAC,aAAa,EAAE,QAAQ,CAAC,CAAC;IAE1D,MAAM,kBAAkB,GAAG,IAAI,GAAG,CAAC,YAAY,CAAC,UAAU,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC,CAAC;IACtF,MAAM,iBAAiB,GAAG,IAAI,GAAG,CAAC,WAAW,CAAC,UAAU,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC,CAAC;IAEpF,MAAM,eAAe,GAAG,WAAW,CAAC,UAAU,CAAC,MAAM,CACnD,CAAC,CAAC,EAAE,CAAC,CAAC,kBAAkB,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC,CAAC,CAAC,CACtD,CAAC;IACF,MAAM,iBAAiB,GAAG,YAAY,CAAC,UAAU,CAAC,MAAM,CACtD,CAAC,CAAC,EAAE,CAAC,CAAC,iBAAiB,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC,CAAC,CAAC,CACrD,CAAC;IAEF,MAAM,kBAAkB,GAAG,WAAW,CAAC,aAAa,GAAG,YAAY,CAAC,aAAa,CAAC;IAElF,MAAM,kBAAkB,GACtB,kBAAkB,GAAG,CAAC;QACtB,eAAe,CAAC,IAAI,CAClB,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,QAAQ,KAAK,oBAAoB,IAAI,CAAC,CAAC,QAAQ,KAAK,oBAAoB,CAChF,CAAC;IAEJ,MAAM,oBAAoB,GAAG;QAC3B,GAAG,IAAI,GAAG,CAAC,eAAe,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC;KAC1B,CAAC;IAEzB,OAAO;QACL,QAAQ;QACR,MAAM,EAAE;YACN,eAAe,EAAE,YAAY,CAAC,UAAU,CAAC,MAAM;YAC/C,aAAa,EAAE,YAAY,CAAC,aAAa;YACzC,OAAO,EAAE,YAAY,CAAC,OAAO;SAC9B;QACD,KAAK,EAAE;YACL,eAAe,EAAE,WAAW,CAAC,UAAU,CAAC,MAAM;YAC9C,aAAa,EAAE,WAAW,CAAC,aAAa;YACxC,OAAO,EAAE,WAAW,CAAC,OAAO;SAC7B;QACD,kBAAkB;QAClB,eAAe;QACf,iBAAiB;QACjB,kBAAkB;QAClB,oBAAoB;KACrB,CAAC;AACJ,CAAC"}
|
package/dist/parser.d.ts
ADDED