falsegreen-js 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +140 -1
- package/README.md +72 -13
- package/dist/cases.d.ts +51 -3
- package/dist/cases.js +91 -43
- package/dist/cfg.d.ts +32 -0
- package/dist/cfg.js +237 -0
- package/dist/cli.d.ts +27 -1
- package/dist/cli.js +136 -41
- package/dist/oracles.d.ts +52 -0
- package/dist/oracles.js +87 -0
- package/dist/report.d.ts +31 -0
- package/dist/report.js +177 -0
- package/dist/rules.js +538 -90
- package/dist/scan.d.ts +1 -0
- package/dist/scan.js +0 -0
- package/dist/types.js +2 -2
- package/package.json +34 -9
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,143 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.5.0] - 2026-06-28
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
- `JS23` (high, J1): `expect.assertions(N)` with a numeric `N` higher than the unconditional,
|
|
13
|
+
reachable, non-nested `expect()` calls that can run. The guard can never be met, so the test
|
|
14
|
+
passes without ever exercising the count it claims. Fires only when `N` is a numeric literal
|
|
15
|
+
and the shortfall is provable: an `expect` in a loop, a branch, a `.then`/callback, or a
|
|
16
|
+
helper makes the count indeterminate and suppresses the finding. `expect.hasAssertions()`
|
|
17
|
+
carries no count and is skipped. This is the implemented sibling of the still-skipped JS16.
|
|
18
|
+
- `JS24` (low, J4): a Cypress query chain (`cy.get`/`cy.find`/`cy.contains`) used as a statement
|
|
19
|
+
with no terminating `.should`/`.and` and no `expect` inside a `.then` callback. The query
|
|
20
|
+
produces a subject that is never asserted, the cy.* analogue of JS13. Action commands
|
|
21
|
+
(`click`/`type`/`visit`/...) do work rather than just query, so a chain ending in one stays
|
|
22
|
+
clean, as does a chain that ends in `.should`/`.and` or asserts in `.then`.
|
|
23
|
+
- CLI `--enable <codes>` (and `--enable=...`): re-activates listed off or opt-in codes at their
|
|
24
|
+
catalog severity, flipping a default-off code on. It cannot raise a code above catalog
|
|
25
|
+
severity. `--disable` wins over `--enable`, so a code passed to both stays off.
|
|
26
|
+
- `examples/` tree (#47): a worked sample for every emitted code, a BAD test the scanner flags
|
|
27
|
+
paired with a CLEAN look-alike one token away that it leaves alone. Files are grouped by
|
|
28
|
+
RiskGroup (`effectiveness`, `execution`, `nondeterminism`, `dependency`), with `cypress.cy.ts`
|
|
29
|
+
for the Cypress code and `diagnostics.test.ts` for the opt-in maintainability group. C16 keeps
|
|
30
|
+
a separate frozen-clock file because the fake-timer signal is file-wide. `vitest.config.ts`
|
|
31
|
+
excludes `examples/**` from collection, and `test/examples.test.ts` scans each file with
|
|
32
|
+
`analyze(parse(...))` to assert every code fires in its file, with a drift guard that fails if a
|
|
33
|
+
new default-on code lands without an example. The config-audit-only PL series scans Jest/Vitest
|
|
34
|
+
config rather than a test file, so it has no test-file example.
|
|
35
|
+
|
|
36
|
+
### Changed
|
|
37
|
+
- `JS8` now also catches the `jest.spyOn`/`vi.spyOn` form: a spy with a canned return
|
|
38
|
+
(`mockReturnValue`/`mockResolvedValue`/`mockImplementation`) whose spied target root is also an
|
|
39
|
+
`expect` subject. The test asserts the canned value, not real behaviour. Conservative
|
|
40
|
+
same-binding guard: spying a collaborator (a different object) stays clean, and asserting on
|
|
41
|
+
the spy handle itself (`expect(spy).toHaveBeenCalled()`) is not treated as the subject.
|
|
42
|
+
- `JS3` gains a distinct detail when the snapshot is an empty inline baseline:
|
|
43
|
+
`toMatchInlineSnapshot()` with no argument, or an empty or whitespace-only string baseline,
|
|
44
|
+
passes by writing itself on the first run. A populated inline snapshot keeps the existing
|
|
45
|
+
detail; the snapshot-only detection logic is unchanged.
|
|
46
|
+
|
|
47
|
+
### Docs
|
|
48
|
+
- `CONTRIBUTING.md` documents the FP-boundary decisions that previously lived only in
|
|
49
|
+
source comments: the admission criteria for a new code (statically provable, FP-guarded,
|
|
50
|
+
ships with a CLEAN look-alike one token from the BAD, carries a catalog entry, clears the
|
|
51
|
+
panel and principal-reviewer gate) and the standing per-code rules for C44, C6, JS5, C16,
|
|
52
|
+
JS23, JS24, and JS8 (#51).
|
|
53
|
+
|
|
54
|
+
## [0.4.0] - 2026-06-28
|
|
55
|
+
|
|
56
|
+
### Fixed
|
|
57
|
+
- C21 no longer false-positives on a `do { expect } while(c)`: a do/while body always runs at least
|
|
58
|
+
once, so its assertion is unconditional (#60).
|
|
59
|
+
- C16 crypto match is anchored to a crypto root (`crypto.randomUUID`, `globalThis/window/self.crypto`,
|
|
60
|
+
or the bare node:crypto import), so a user method named `randomUUID()`/`getRandomValues()` is no
|
|
61
|
+
longer flagged (#61).
|
|
62
|
+
- C20 and C21 no longer double-report on a dead-code-only assertion: an assertion already flagged
|
|
63
|
+
C20 (unreachable) is excluded from the C21 set, so C20 owns the line. C21 still fires when a live
|
|
64
|
+
conditional assertion remains (#62).
|
|
65
|
+
|
|
66
|
+
### Added
|
|
67
|
+
- `C16` nondeterminism now also flags `new Date()` (zero-arg, reads the system clock),
|
|
68
|
+
`crypto.randomUUID()`, and `crypto.getRandomValues()`. `new Date(<literal/expr>)` is a fixed
|
|
69
|
+
instant and stays clean, and the file-wide fake-timer suppression applies. Aliased/destructured
|
|
70
|
+
clock reads (`const now = Date.now; now()`) stay out: tracking them would trade a rare miss for
|
|
71
|
+
a frequent false positive on user `now()` helpers (#46).
|
|
72
|
+
- `C48` (dark patch): a test that flips a known test-mode flag into test mode and then
|
|
73
|
+
asserts is exercising the product's test-only branch, not real behaviour. Detection-only;
|
|
74
|
+
v1 covers raw writes (`process.env.NODE_ENV = "test"`, `process.env.TESTING = "1"`,
|
|
75
|
+
`settings.TESTING = true`). `NODE_ENV` only counts as `"test"`; config values and product
|
|
76
|
+
feature flags are not flagged; a flag write with no assertion after it is setup, not a dark
|
|
77
|
+
patch. Parity with falsegreen (Python) `C48`, same id and J1/low (#39).
|
|
78
|
+
|
|
79
|
+
### Tests
|
|
80
|
+
- Lock the floating `expect(p).resolves`/`.rejects.<matcher>()` case for `JS5`: a non-awaited
|
|
81
|
+
promise matcher is already flagged through the oracle registry (the matcher builds a promise
|
|
82
|
+
that never settles before the test ends), and tests now pin that, including an exact-count
|
|
83
|
+
guard so a future change cannot double-report it. Awaited/returned forms stay clean (#43).
|
|
84
|
+
- Characterization tests for the cfg reachability edge cases: for-in (C20 after / C21 inside),
|
|
85
|
+
labeled `break outer`, a switch case that falls through without escaping, an IIFE holding the
|
|
86
|
+
only assertion (no phantom C21), and `performance.now()` C16 (#63).
|
|
87
|
+
|
|
88
|
+
### Changed
|
|
89
|
+
- `C20` and `C21` now use a structured intra-test reachability walk (`src/cfg.ts`) instead of
|
|
90
|
+
a top-level-only scan. `C20` (dead code) catches an assertion after any non-falling-through
|
|
91
|
+
statement: a `return`/`throw`, `process.exit()`, a `break`/`continue`, an `if` whose both
|
|
92
|
+
arms terminate, a terminating block, or an exhaustive `switch` (every case plus a `default`
|
|
93
|
+
escapes). `C21` (no unconditional assertion) fires only when no assertion is on the guaranteed
|
|
94
|
+
spine; an assertion in an `if(true)` branch, a `finally`, or a `try` block now counts as
|
|
95
|
+
guaranteed, and an assertion only in a `catch` or a loop body is correctly flagged. The walk
|
|
96
|
+
stops at nested functions, so a `return` inside a `forEach`/IIFE callback no longer reads as
|
|
97
|
+
dead code. False-positive-averse: anything unmodeled is treated as reachable/guaranteed (#35).
|
|
98
|
+
- README Status no longer pins a stale `0.1.0` literal; it points to STATUS.md for the current
|
|
99
|
+
version and coverage. Removed two boolean sub-clauses fully subsumed by their first disjunct
|
|
100
|
+
in `isTestBlock` and the JS6 suite guard (behavior-preserving) (#64, #65).
|
|
101
|
+
|
|
102
|
+
### Fixed (earlier in the 0.4.0 cycle)
|
|
103
|
+
- JS5 surfaces a floating promise observed only by a non-assignment binary op (||, &&, ===); only a real assignment with the call as RHS counts as observed.
|
|
104
|
+
- JS7 timer control now consults lifecycle hooks that install/flush fake timers at both the enclosing-describe and source-file top level (#41).
|
|
105
|
+
|
|
106
|
+
|
|
107
|
+
### Added
|
|
108
|
+
- C44 (numeric tautology, high, J2): `expect(x.length).toBeGreaterThanOrEqual(0)`. A
|
|
109
|
+
`.length` is never negative and never NaN, so `>= 0` holds for every input and checks
|
|
110
|
+
nothing — the JS/TS mirror of the Python `len(x) >= 0`. The subject must be a direct
|
|
111
|
+
`.length` property access: a derived expression that only mentions `.length`
|
|
112
|
+
(`a.length - b.length`) can be negative and is not flagged, and a bound that can still
|
|
113
|
+
fail (`>= 1`, `> 0`) is a real check. Finiteness/NaN guards (`toBeLessThan(Infinity)`,
|
|
114
|
+
`toBeGreaterThan(-Infinity)`) are deliberately not flagged: they are false for `NaN`, so
|
|
115
|
+
they catch divide-by-zero and invalid-number bugs.
|
|
116
|
+
- Output-format parity with the Python scanner: `--format text|json|sarif|junit`
|
|
117
|
+
(default `text`; `--json` stays as an alias for `--format json`). SARIF 2.1.0
|
|
118
|
+
emits one rule per code and one result per finding, maps high to `error`, low to
|
|
119
|
+
`warning`, off to `note`, and tags each result with its judgment, `risk:<group>`,
|
|
120
|
+
and `level:<conf>`. JUnit XML turns high findings into `<failure>` and the rest
|
|
121
|
+
into `<skipped>`. `--output` writes any of the four formats (sarif -> `.sarif`,
|
|
122
|
+
junit -> `.xml`).
|
|
123
|
+
- Baseline ratchet: `--baseline [PATH]` suppresses findings already recorded (so CI
|
|
124
|
+
fails only on net-new ones), and `--write-baseline [PATH]` records the current
|
|
125
|
+
findings and exits 0. Both default to `.falsegreen-baseline.json`. A finding's
|
|
126
|
+
identity is a content fingerprint (`sha1` of relative path + code + detail, no
|
|
127
|
+
line number), stable across unrelated line shifts. The fingerprint omits the
|
|
128
|
+
source snippet the Python scanner folds in, since the js `Finding` carries none.
|
|
129
|
+
- Risk-group taxonomy: every code now carries an explicit conceptual failure mode
|
|
130
|
+
(`effectiveness`, `execution`, `nondeterminism`, `dependency`, `structure`,
|
|
131
|
+
`diagnostic`), read from a closed per-code table (`riskGroupOf`) rather than the
|
|
132
|
+
code prefix. An unknown code is rejected instead of defaulted. The JSON report
|
|
133
|
+
gains a `riskGroup` field; the legacy `group` field stays for transition compatibility.
|
|
134
|
+
- A code's metadata is split into independent axes: `group` (taxonomy), `severity`
|
|
135
|
+
(`high`/`low`), `defaultOn` (whether the default scan emits it), and `judgment`
|
|
136
|
+
(J1-J6). The taxonomy no longer depends on whether a finding blocks.
|
|
137
|
+
- Oracle registry (`oracles.ts`): the assertion-API vocabulary is one versioned
|
|
138
|
+
table, each family classified by how its failure reaches the runner (`sync-fail`,
|
|
139
|
+
`promise`, `runner-registered`, `value-only`). The JSON report records the
|
|
140
|
+
`oracleRegistryVersion` that classified it.
|
|
141
|
+
|
|
142
|
+
### Fixed
|
|
143
|
+
- `--version` and the JSON report's `version` field read from `package.json` at
|
|
144
|
+
runtime; they were pinned to a stale `0.2.0` literal while the package was `0.3.0`.
|
|
145
|
+
|
|
9
146
|
## [0.3.0] - 2026-06-23
|
|
10
147
|
|
|
11
148
|
### Added
|
|
@@ -84,7 +221,9 @@ All notable changes to this project are documented here. The format is based on
|
|
|
84
221
|
- pre-commit hook (`.pre-commit-hooks.yaml`), CI matrix (Node 18/20/22), and an npm
|
|
85
222
|
trusted-publishing release workflow.
|
|
86
223
|
|
|
87
|
-
[Unreleased]: https://github.com/vinicq/falsegreen-js/compare/v0.
|
|
224
|
+
[Unreleased]: https://github.com/vinicq/falsegreen-js/compare/v0.5.0...HEAD
|
|
225
|
+
[0.5.0]: https://github.com/vinicq/falsegreen-js/compare/v0.4.0...v0.5.0
|
|
226
|
+
[0.4.0]: https://github.com/vinicq/falsegreen-js/compare/v0.3.0...v0.4.0
|
|
88
227
|
[0.3.0]: https://github.com/vinicq/falsegreen-js/compare/v0.2.0...v0.3.0
|
|
89
228
|
[0.2.0]: https://github.com/vinicq/falsegreen-js/compare/v0.1.0...v0.2.0
|
|
90
229
|
[0.1.0]: https://github.com/vinicq/falsegreen-js/releases/tag/v0.1.0
|
package/README.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# falsegreen-js
|
|
2
2
|
|
|
3
|
+
[](https://github.com/vinicq/falsegreen-js/actions/workflows/ci.yml)
|
|
4
|
+
[](https://www.npmjs.com/package/falsegreen-js)
|
|
5
|
+
[](https://www.npmjs.com/package/falsegreen-js)
|
|
6
|
+
[](https://www.npmjs.com/package/falsegreen-js)
|
|
7
|
+
[](LICENSE)
|
|
8
|
+
[](CONTRIBUTING.md)
|
|
9
|
+
[](https://vinicq.github.io/falsegreen-docs/)
|
|
10
|
+
|
|
3
11
|
Find JavaScript/TypeScript unit tests that give false positives: green tests that
|
|
4
12
|
protect nothing, and tests that pass while asserting the wrong thing. Deterministic
|
|
5
13
|
AST scan, no code execution. Sibling of [`falsegreen`](https://github.com/vinicq/falsegreen)
|
|
@@ -7,6 +15,15 @@ AST scan, no code execution. Sibling of [`falsegreen`](https://github.com/vinicq
|
|
|
7
15
|
|
|
8
16
|
Covers `.js`, `.jsx`, `.ts`, `.tsx`, `.mjs`, `.cjs`, `.mts`, `.cts`.
|
|
9
17
|
|
|
18
|
+
**The falsegreen family** (install the one for your stack):
|
|
19
|
+
|
|
20
|
+
| Tool | Stack | Install | Package |
|
|
21
|
+
|---|---|---|---|
|
|
22
|
+
| [falsegreen](https://github.com/vinicq/falsegreen) | Python / pytest | `pip install falsegreen` | [PyPI](https://pypi.org/project/falsegreen/) |
|
|
23
|
+
| **falsegreen-js** | JS / TS | `npm i -D falsegreen-js` (`npx falsegreen-js`) | [npm](https://www.npmjs.com/package/falsegreen-js) |
|
|
24
|
+
| [robotframework-falsegreen](https://github.com/vinicq/robotframework-falsegreen) | Robot Framework | `pip install robotframework-falsegreen` | [PyPI](https://pypi.org/project/robotframework-falsegreen/) |
|
|
25
|
+
| [falsegreen-skill](https://github.com/vinicq/falsegreen-skill) | semantic LLM pass | `npx falsegreen-skill analyze <path>` | [npm](https://www.npmjs.com/package/falsegreen-skill) |
|
|
26
|
+
|
|
10
27
|
## Why
|
|
11
28
|
|
|
12
29
|
A test can be green and still protect nothing: an empty body, an assertion that is
|
|
@@ -27,15 +44,36 @@ npm install -D falsegreen-js
|
|
|
27
44
|
npx falsegreen-js # scan cwd
|
|
28
45
|
npx falsegreen-js src test # scan paths
|
|
29
46
|
npx falsegreen-js --staged # only test files staged in git (pre-commit)
|
|
30
|
-
npx falsegreen-js --json # machine-readable output
|
|
47
|
+
npx falsegreen-js --json # machine-readable output (alias for --format json)
|
|
48
|
+
npx falsegreen-js --format sarif # SARIF 2.1.0 for GitHub code scanning
|
|
49
|
+
npx falsegreen-js --format junit # JUnit XML for CI test reporters
|
|
31
50
|
npx falsegreen-js --output report.json # write to a file
|
|
32
51
|
npx falsegreen-js --output .falsegreen/ # write report.<ext> into a directory
|
|
33
52
|
npx falsegreen-js --config-audit # audit Jest/Vitest config (project-layer PL codes)
|
|
34
53
|
npx falsegreen-js --disable C7,JS3
|
|
54
|
+
npx falsegreen-js --enable D8,M2 # re-activate off/opt-in codes at catalog severity
|
|
35
55
|
```
|
|
36
56
|
|
|
37
57
|
Each finding is reported with its pyramid level (unit / integration / e2e, read from the file's imports) and a one-line fix hint, and the summary breaks the findings down by level and lists the most common fixes. `--output` takes a file or a directory: an extension-less or trailing-slash path (e.g. `.falsegreen/`) receives `report.<ext>` for the chosen format. Reports are run artifacts; keep the output directory gitignored.
|
|
38
58
|
|
|
59
|
+
### Output formats
|
|
60
|
+
|
|
61
|
+
`--format text|json|sarif|junit` (default `text`; `--json` stays as an alias for `--format json`). These match the [Python sibling](https://github.com/vinicq/falsegreen) byte-for-concept, so a pipeline can swap one scanner for the other.
|
|
62
|
+
|
|
63
|
+
- **`sarif`**: SARIF 2.1.0. One rule per code present, one result per finding, with `error` for high-severity findings, `warning` for low, and `note` for off. Result tags carry the judgment (J1-J6), the risk group (`risk:effectiveness`...), and the level (`level:high`). Upload it to GitHub code scanning to see findings inline on the PR.
|
|
64
|
+
- **`junit`**: JUnit XML. High-severity findings become `<failure>`, everything else `<skipped>`, so a CI test reporter surfaces them as a failing suite.
|
|
65
|
+
|
|
66
|
+
### Baseline (ratchet)
|
|
67
|
+
|
|
68
|
+
Adopting the scanner on a large codebase without fixing every legacy finding at once:
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
npx falsegreen-js --write-baseline # record current findings to .falsegreen-baseline.json, exit 0
|
|
72
|
+
npx falsegreen-js --baseline # report and fail only on findings not in the baseline
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
`--baseline [PATH]` and `--write-baseline [PATH]` default to `.falsegreen-baseline.json`. A finding's identity is a content fingerprint (`sha1` of relative path + code + detail, no line number), so it survives unrelated line shifts in the file. Commit the baseline, then let CI block only on net-new findings. (The fingerprint omits the source snippet the Python scanner folds in, since the js scanner does not carry one; two findings with the same code and detail in one file share an id.)
|
|
76
|
+
|
|
39
77
|
`--config-audit` is a separate mode: instead of scanning test files, it reads the Jest/Vitest config (`package.json` `jest` field, `jest.config.*`, `vitest.config.*`) and reports the project-layer ways a suite stays green by configuration: `PL10` (`passWithNoTests` passes an empty or filtered-to-nothing run), `PL7` (no `coverageThreshold` / `coverage.thresholds`), `PL8` (`bail` stops the run early). The per-file scan cannot see config.
|
|
40
78
|
|
|
41
79
|
For the layer no static scan reaches (does a green test fail when the code is wrong?), run a **mutation tester** like [Stryker](https://stryker-mutator.io/). falsegreen-js is the cheap pre-filter on every commit; mutation testing is the deeper audit.
|
|
@@ -95,7 +133,8 @@ line up in the research. `JS*` codes are ecosystem-specific.
|
|
|
95
133
|
| C5 | high | always-true check (`expect(true).toBe(true)`, `assert(1)`) |
|
|
96
134
|
| C6 | low | weak check — only verifies something came back (`toBeTruthy`/`toBeDefined`, `length > 0`) |
|
|
97
135
|
| C7 | high | compares a thing to itself (`expect(x).toBe(x)`) |
|
|
98
|
-
|
|
|
136
|
+
| C44 | high | numeric tautology — a length compared so the result is always true (`x.length >= 0`) |
|
|
137
|
+
| C20 | high | assertion in unreachable code (after a `return`/`throw`/`process.exit`, a `break`, a both-arms-terminating `if`, or an exhaustive `switch`) — it never runs |
|
|
99
138
|
| C23 | low | reads a real file at a literal path, or a hard-coded URL (mystery guest) |
|
|
100
139
|
| C8 | low | exact equality on a float (use `toBeCloseTo`) |
|
|
101
140
|
| C9 | low | `toThrow()` with no error type or message — accepts any error |
|
|
@@ -103,6 +142,7 @@ line up in the research. `JS*` codes are ecosystem-specific.
|
|
|
103
142
|
| C18 | low | compares `String(x)` / `JSON.stringify(x)` / `` `${x}` `` to a literal (formatting, not value) |
|
|
104
143
|
| C21 | low | every assertion is conditional — none runs unconditionally |
|
|
105
144
|
| C37 | low | duplicate case in `it.each`/`test.each` — the same scenario runs twice |
|
|
145
|
+
| C48 | low | dark patch — the test flips a test-mode flag (`process.env.NODE_ENV = "test"`, `process.env.TESTING`, a `TESTING` flag) then asserts, exercising the product's test-only branch |
|
|
106
146
|
| CC | low | commented-out assertion |
|
|
107
147
|
| JS1 | high | focused test (`it.only` / `fit`) silently skips the rest of the suite |
|
|
108
148
|
| JS2 | high | `expect(x)` with no matcher — the assertion never runs |
|
|
@@ -120,13 +160,15 @@ line up in the research. `JS*` codes are ecosystem-specific.
|
|
|
120
160
|
| JS18 | low | test takes a `done` callback instead of async/await — a mistimed `done` passes early |
|
|
121
161
|
| JS21 | high | matcher referenced but never called (`expect(x).toBe` with no `()`) — the assertion never runs |
|
|
122
162
|
| JS22 | high | empty `it.each`/`test.each` table — generated with zero cases, never runs |
|
|
163
|
+
| JS23 | high | `expect.assertions(N)` with fewer unconditional `expect()` calls than N — the guard can never be met |
|
|
164
|
+
| JS24 | low | Cypress query (`cy.get`/`cy.find`/`cy.contains`) as a loose statement with no terminating `.should`/`.and` and no `expect` in `.then` — its result is never asserted |
|
|
123
165
|
|
|
124
166
|
Each code carries a judgment tag (J1-J6) shared with the
|
|
125
167
|
[falsegreen-skill](https://github.com/vinicq/falsegreen-skill) semantic framework.
|
|
126
168
|
|
|
127
169
|
### Opt-in: maintainability group (default off)
|
|
128
170
|
|
|
129
|
-
These are **not** false-green
|
|
171
|
+
These are **not** false-green - the test still protects something - so they are off by
|
|
130
172
|
default. Enable them with `--diagnostics`, or per code via config `severity`. They are a
|
|
131
173
|
"plus" for test-code health, mirroring falsegreen's diagnostic/coupling groups.
|
|
132
174
|
|
|
@@ -149,18 +191,32 @@ npx falsegreen-js --diagnostics # include D*/M* as warnings
|
|
|
149
191
|
Some catalog codes were reviewed and left out, on purpose:
|
|
150
192
|
|
|
151
193
|
- **JS19** (`toBe` on an object/array literal): `expect(x).toBe({...})` compares by reference,
|
|
152
|
-
so it always fails. That is
|
|
194
|
+
so it always fails. That is the false-red axis (a test that always fails), the opposite of
|
|
195
|
+
what this scanner looks for, and out of scope on principle.
|
|
153
196
|
- **JS20** (a Promise compared without `resolves`/`rejects`): telling that a value is a
|
|
154
|
-
Promise needs type information the
|
|
197
|
+
Promise needs type information the AST does not carry, so it would be too noisy.
|
|
155
198
|
- **JS12** (a floating promise whose `expect` is never returned): already covered by JS7.
|
|
156
|
-
- **JS16** (`async` test with no `expect.assertions(n)`): the absence of a guard is not a
|
|
157
|
-
smell on its own; flagging it would fire on most async tests.
|
|
199
|
+
- **JS16** (`async` test with no `expect.assertions(n)`): the *absence* of a guard is not a
|
|
200
|
+
smell on its own; flagging it would fire on most async tests. The implemented sibling is
|
|
201
|
+
`JS23`, which fires on a present-but-unsatisfiable guard: `expect.assertions(N)` with a
|
|
202
|
+
numeric `N` higher than the unconditional `expect()` calls that can run, so the count can
|
|
203
|
+
never be met.
|
|
204
|
+
- **JS14** (a giant inline snapshot): a readability and review-noise concern, not a
|
|
205
|
+
false-green one. The snapshot still protects, so it belongs to the diagnostic group and is
|
|
206
|
+
better served by `eslint-plugin-jest` (`no-large-snapshots`) as an opt-in lint rule.
|
|
158
207
|
- **JS10** (any conditional in a test body): handled by `eslint-plugin-jest`
|
|
159
208
|
(`no-conditional-in-test`); JS9 and C21 already cover the false-green subset.
|
|
209
|
+
- **C1** (an assertion under an `if`/`for` that may not run): redundant once C21 and JS9
|
|
210
|
+
exist, and high-FP on its own. C21 already fires the actual false-green case, where
|
|
211
|
+
*every* assertion is conditional and the test can pass with nothing checked. A test that
|
|
212
|
+
mixes a conditional assertion with an unconditional one is not false-green: the
|
|
213
|
+
unconditional assertion still protects. JS9 covers the dead-branch form (`if(false)`).
|
|
214
|
+
Flagging every conditional assertion (C1's full scope) is the linter concern JS10 already
|
|
215
|
+
names (`no-conditional-in-test`), so C1 would add noise without a new false-green signal.
|
|
160
216
|
|
|
161
217
|
### What carries over from falsegreen, what does not
|
|
162
218
|
|
|
163
|
-
Ported (same concept): C2, C2b, C5, C7, C8, C16, CC.
|
|
219
|
+
Ported (same concept): C2, C2b, C5, C7, C8, C16, C44, C48, CC.
|
|
164
220
|
|
|
165
221
|
Python-only, not applicable to JS/TS: pytest collection rules (C4 family), `pytest.raises`
|
|
166
222
|
breadth (C9/C19/C27/C28), fixtures and `os.environ`/global-state codes (C23/C24/C29),
|
|
@@ -183,7 +239,7 @@ Optional. `falsegreen.json`, `.falsegreenrc.json`, or a `"falsegreen"` key in
|
|
|
183
239
|
}
|
|
184
240
|
```
|
|
185
241
|
|
|
186
|
-
Precedence: CLI `--disable` > config `disable`/`severity` > catalog default.
|
|
242
|
+
Precedence: CLI `--disable` > CLI `--enable` > config `disable`/`severity` > catalog default. `--enable <codes>` re-activates listed off or opt-in codes at their catalog severity (it flips a default-off code on; it cannot raise a code above catalog). A code passed to both `--enable` and `--disable` stays off — `--disable` wins.
|
|
187
243
|
|
|
188
244
|
## Scope and honesty
|
|
189
245
|
|
|
@@ -193,20 +249,23 @@ test re-implements the production logic. Those are semantic and belong to the
|
|
|
193
249
|
`falsegreen-skill` LLM pass. Precision over recall: a softened heuristic that misses a
|
|
194
250
|
case is preferred to one that flags correct code.
|
|
195
251
|
|
|
252
|
+
Measured against the [Open Catalog of Test Smells](https://test-smell-catalog.readthedocs.io/) (517 documented smells), only the false-green slice is in scope. What stays out, on purpose: **brittleness / false-red** (sensitive equality, brittle assertions - the opposite axis), **hygiene / maintainability** (assertion roulette, magic numbers, long tests - linter territory, a few surfaced as opt-in diagnostics), and **slow, design, naming, duplication, runtime/culture** (none about whether the test protects). The boundary is deliberate: where a smell has a statically provable false-green form, that form is a code here - uncontrolled `Date.now`/`Math.random` is `C16`, a hard-coded path or URL is `C23`, an assertion that may never run is `C21`/`C20`, and JS-specific forms (focused tests, missing matchers) are the `JS*` codes. See [CREDITS.md](CREDITS.md) for the full cross-walk.
|
|
253
|
+
|
|
196
254
|
## References
|
|
197
255
|
|
|
198
256
|
The catalog is grounded in the test-smell literature. Direct influences: the
|
|
199
257
|
rotten-green-test work that names this whole family (Delplanque et al., ICSE 2019),
|
|
200
258
|
the founding test-smell refactoring catalog (van Deursen et al., XP 2001), the
|
|
201
|
-
JS/TS empirical studies (Jorge, UFCG 2023; Silva, PUC Minas 2022
|
|
259
|
+
JS/TS empirical studies (Jorge, UFCG 2023; Silva, PUC Minas 2022 - the academic
|
|
202
260
|
precedent for the focused-test and snapshot codes; Oliveira et al., SBES 2024/2025),
|
|
203
261
|
and the detection-tool baselines (tsDetect, Peruma et al., 2020). Full list and the
|
|
204
262
|
code-to-source mapping in [CREDITS.md](CREDITS.md).
|
|
205
263
|
|
|
206
264
|
## Status
|
|
207
265
|
|
|
208
|
-
|
|
209
|
-
|
|
266
|
+
The rule set is a deterministic core; the full JS/TS smell catalog is tracked as
|
|
267
|
+
research in the private audit hub. See [STATUS.md](STATUS.md) for the current version
|
|
268
|
+
and rule coverage. Issues and PRs welcome.
|
|
210
269
|
|
|
211
270
|
## License
|
|
212
271
|
|
|
@@ -227,7 +286,7 @@ Thanks to the people who keep false-green tests out of real suites ([emoji key](
|
|
|
227
286
|
<tbody>
|
|
228
287
|
<tr>
|
|
229
288
|
<td align="center" valign="top" width="14.28%"><a href="https://vinicq.github.io/md-bridge/"><img src="https://avatars.githubusercontent.com/u/78210890?v=4?s=100" width="100px;" alt="Vinicius Queiroz"/><br /><sub><b>Vinicius Queiroz</b></sub></a><br /><a href="https://github.com/vinicq/falsegreen-js/commits?author=vinicq" title="Code">💻</a> <a href="https://github.com/vinicq/falsegreen-js/commits?author=vinicq" title="Documentation">📖</a> <a href="#ideas-vinicq" title="Ideas, Planning, & Feedback">🤔</a> <a href="#maintenance-vinicq" title="Maintenance">🚧</a> <a href="#infra-vinicq" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a> <a href="https://github.com/vinicq/falsegreen-js/commits?author=vinicq" title="Tests">⚠️</a> <a href="#research-vinicq" title="Research">🔬</a></td>
|
|
230
|
-
<td align="center" valign="top" width="14.28%"><a href="https://github.com/homesellerq-coder"><img src="https://avatars.githubusercontent.com/u/294912019?v=4?s=100" width="100px;" alt="Home Seller"/><br /><sub><b>Home Seller</b></sub></a><br /><a href="https://github.com/vinicq/falsegreen-js/commits?author=homesellerq-coder" title="Code">💻</a></td>
|
|
289
|
+
<td align="center" valign="top" width="14.28%"><a href="https://github.com/homesellerq-coder"><img src="https://avatars.githubusercontent.com/u/294912019?v=4?s=100" width="100px;" alt="Home Seller"/><br /><sub><b>Home Seller</b></sub></a><br /><a href="https://github.com/vinicq/falsegreen-js/commits?author=homesellerq-coder" title="Code">💻</a> <a href="https://github.com/vinicq/falsegreen-js/commits?author=homesellerq-coder" title="Documentation">📖</a> <a href="https://github.com/vinicq/falsegreen-js/commits?author=homesellerq-coder" title="Tests">⚠️</a> <a href="#infra-homesellerq-coder" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a></td>
|
|
231
290
|
</tr>
|
|
232
291
|
</tbody>
|
|
233
292
|
</table>
|
package/dist/cases.d.ts
CHANGED
|
@@ -3,14 +3,42 @@
|
|
|
3
3
|
* the same concept (shared C-codes, so cross-language paper comparison lines up),
|
|
4
4
|
* plus JS/TS-specific codes (JS-prefix) for ecosystem-only patterns.
|
|
5
5
|
*
|
|
6
|
-
*
|
|
7
|
-
*
|
|
6
|
+
* Each code carries four independent axes (none derived from another or from the
|
|
7
|
+
* code prefix):
|
|
8
|
+
* group conceptual failure mode (RiskGroup, closed taxonomy).
|
|
9
|
+
* severity how serious the finding is when it fires ("high" | "low").
|
|
10
|
+
* defaultOn whether the default scan emits it (false for the opt-in
|
|
11
|
+
* diagnostic group, surfaced only via --diagnostics).
|
|
12
|
+
* judgment which semantic question (J1-J6, see falsegreen-skill) it answers.
|
|
13
|
+
*
|
|
14
|
+
* The effective "confidence" used downstream (high/low/off) is derived from
|
|
15
|
+
* severity + defaultOn by baseConfidence(); the exit code is derived from the
|
|
16
|
+
* severity of the findings that are actually emitted. Keeping the axes apart is
|
|
17
|
+
* the point: a finding's taxonomy must not depend on whether it blocks.
|
|
8
18
|
*/
|
|
9
19
|
export type Confidence = "high" | "low" | "off";
|
|
20
|
+
export type Severity = "high" | "low";
|
|
21
|
+
/**
|
|
22
|
+
* Conceptual failure mode — a closed taxonomy condensing the F1-F8 families to
|
|
23
|
+
* six axes. Driven by the per-code table below (riskGroupOf), never by the code
|
|
24
|
+
* prefix, and never defaulted: an unknown code is an error, not a guess.
|
|
25
|
+
*
|
|
26
|
+
* effectiveness no oracle, a trivial oracle, or the wrong oracle (F1/F3/F4).
|
|
27
|
+
* execution the check exists but does not run, or the test vanishes from
|
|
28
|
+
* the count (F2/F5).
|
|
29
|
+
* nondeterminism passes or fails by luck — time, randomness, timers (F6).
|
|
30
|
+
* dependency real I/O or a stand-in for the unit under test: mystery
|
|
31
|
+
* guest, self-mock (isolation, J3/J6).
|
|
32
|
+
* structure size/readability; the test still protects (F8 maintainability).
|
|
33
|
+
* diagnostic opt-in health signal, off by default.
|
|
34
|
+
*/
|
|
35
|
+
export type RiskGroup = "effectiveness" | "execution" | "nondeterminism" | "dependency" | "structure" | "diagnostic";
|
|
10
36
|
export declare const JUDGMENTS: Record<string, string>;
|
|
11
37
|
export interface CaseDef {
|
|
12
38
|
title: string;
|
|
13
|
-
|
|
39
|
+
group: RiskGroup;
|
|
40
|
+
severity: Severity;
|
|
41
|
+
defaultOn: boolean;
|
|
14
42
|
judgment: keyof typeof JUDGMENTS;
|
|
15
43
|
}
|
|
16
44
|
export declare const CASES: Record<string, CaseDef>;
|
|
@@ -19,6 +47,26 @@ export declare const DIAGNOSTIC_THRESHOLDS: {
|
|
|
19
47
|
assertionRoulette: number;
|
|
20
48
|
longTest: number;
|
|
21
49
|
};
|
|
50
|
+
/**
|
|
51
|
+
* Effective default state of a code as a single value: its severity when the
|
|
52
|
+
* default scan emits it, or "off" when it is opt-in. Derives the legacy
|
|
53
|
+
* three-valued "confidence" from the independent severity + defaultOn axes, so
|
|
54
|
+
* the rest of the pipeline (makeFinding, effectiveConf, exit code) keeps working
|
|
55
|
+
* unchanged while the taxonomy stays separate from the blocking decision.
|
|
56
|
+
*/
|
|
57
|
+
export declare function baseConfidence(code: string): Confidence;
|
|
58
|
+
/**
|
|
59
|
+
* Primary taxonomy: the conceptual failure mode, read from the closed per-code
|
|
60
|
+
* table. Rejects an unknown code instead of defaulting, so a typo or a code that
|
|
61
|
+
* was added to the rules but never classified fails loudly.
|
|
62
|
+
*/
|
|
63
|
+
export declare function riskGroupOf(code: string): RiskGroup;
|
|
64
|
+
/**
|
|
65
|
+
* Legacy product grouping (false-positive / diagnostic / coupling / project),
|
|
66
|
+
* kept only as a transition-compat field in the JSON report. New consumers
|
|
67
|
+
* should read `riskGroup` (riskGroupOf). Prefix-based by design: it mirrors the
|
|
68
|
+
* pre-0.3 output exactly so downstream filters do not break across the upgrade.
|
|
69
|
+
*/
|
|
22
70
|
export declare function groupOf(code: string): "false-positive" | "diagnostic" | "coupling" | "project";
|
|
23
71
|
/** Test-pyramid level, detected from a file's import roots (see level.ts).
|
|
24
72
|
* `project` is the config-audit layer (--config-audit), not a file level. */
|