falsegreen-js 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,45 @@ All notable changes to this project are documented here. The format is based on
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.3.0] - 2026-06-23
10
+
11
+ ### Added
12
+ - New codes: JS21 (matcher referenced but never called, `expect(x).toBe` with no `()`),
13
+ JS22 (empty `it.each`/`test.each` table), JS17 (commented-out test block), JS18 (`done`
14
+ callback instead of async/await).
15
+ - supertest / chai-http `.expect()` is recognized as an assertion, so API integration tests
16
+ built with `request(app).get(...).expect(200)` are no longer flagged C2b.
17
+ - Documented test-pyramid coverage: unit, integration (API and database), and E2E.
18
+ - `--config-audit` mode (project layer): reads the Jest/Vitest config (`package.json` `jest`
19
+ field, `jest.config.*`, `vitest.config.*`; JSON directly, JS/TS via the TypeScript parser)
20
+ and reports PL10 (`passWithNoTests`), PL7 (no `coverageThreshold` / `coverage.thresholds`),
21
+ PL8 (`bail`). Findings carry level `project` and a fix hint. README now recommends Stryker
22
+ for the mutation-testing layer the static scan cannot reach.
23
+ - Status report output: every finding now carries its pyramid level (unit / integration /
24
+ e2e, detected from the file's import roots) and a one-line fix hint. The text summary adds
25
+ a per-level breakdown and the top fixes by frequency; JSON gains `level` and `fix` fields.
26
+ - `--output` flag: write to a file, or pass a directory (e.g. `.falsegreen/`) to get
27
+ `report.<ext>` for the chosen format. Parent directories are created as needed.
28
+
29
+ ### Added
30
+ - Cross-language parity with the Python scanner: C6 (weak check — toBeTruthy/toBeDefined/length>0),
31
+ C20 (assertion in dead code after return/throw), C23 (mystery guest — real file at a literal
32
+ path / hard-coded URL), and JS8 (mocks the unit under test and asserts it directly).
33
+
34
+ ### Added
35
+ - JS3 now covers visual snapshots (Playwright `toHaveScreenshot`/`toMatchScreenshot`): a test whose only check is a visual snapshot is snapshot-only (the baseline comes from the output). Percy `percySnapshot()`/`cy.percySnapshot()` is not a runtime assertion, so a percy-only test surfaces as no-assertion (C2b).
36
+
37
+ ### Fixed
38
+ - Test-file discovery now matches more JS/TS naming conventions: Cypress `.cy.*`,
39
+ Deno/Go `_test.*`, Jasmine `*Spec.*`, Angular/Protractor `.e2e-spec.*`, and `.e2e.*`
40
+ (plus the `cypress`/`e2e` directories). Previously only `.test`/`.spec` were
41
+ discovered, so Cypress/Deno/Jasmine/Angular specs were silently skipped.
42
+
43
+ ### Added
44
+ - Vue/Svelte test-utils coverage: JS5 now flags non-awaited `flushPromises`/`nextTick`/
45
+ `tick`; JS13 now flags Vue Test Utils `findComponent`/`findAllComponents` and
46
+ `find`/`findAll` with a string selector used as a loose statement.
47
+
9
48
  ## [0.2.0] - 2026-06-22
10
49
 
11
50
  ### Added
@@ -45,6 +84,7 @@ All notable changes to this project are documented here. The format is based on
45
84
  - pre-commit hook (`.pre-commit-hooks.yaml`), CI matrix (Node 18/20/22), and an npm
46
85
  trusted-publishing release workflow.
47
86
 
48
- [Unreleased]: https://github.com/vinicq/falsegreen-js/compare/v0.2.0...HEAD
87
+ [Unreleased]: https://github.com/vinicq/falsegreen-js/compare/v0.3.0...HEAD
88
+ [0.3.0]: https://github.com/vinicq/falsegreen-js/compare/v0.2.0...v0.3.0
49
89
  [0.2.0]: https://github.com/vinicq/falsegreen-js/compare/v0.1.0...v0.2.0
50
90
  [0.1.0]: https://github.com/vinicq/falsegreen-js/releases/tag/v0.1.0
package/README.md CHANGED
@@ -28,9 +28,18 @@ npx falsegreen-js # scan cwd
28
28
  npx falsegreen-js src test # scan paths
29
29
  npx falsegreen-js --staged # only test files staged in git (pre-commit)
30
30
  npx falsegreen-js --json # machine-readable output
31
+ npx falsegreen-js --output report.json # write to a file
32
+ npx falsegreen-js --output .falsegreen/ # write report.<ext> into a directory
33
+ npx falsegreen-js --config-audit # audit Jest/Vitest config (project-layer PL codes)
31
34
  npx falsegreen-js --disable C7,JS3
32
35
  ```
33
36
 
37
+ Each finding is reported with its pyramid level (unit / integration / e2e, read from the file's imports) and a one-line fix hint, and the summary breaks the findings down by level and lists the most common fixes. `--output` takes a file or a directory: an extension-less or trailing-slash path (e.g. `.falsegreen/`) receives `report.<ext>` for the chosen format. Reports are run artifacts; keep the output directory gitignored.
38
+
39
+ `--config-audit` is a separate mode: instead of scanning test files, it reads the Jest/Vitest config (`package.json` `jest` field, `jest.config.*`, `vitest.config.*`) and reports the project-layer ways a suite stays green by configuration: `PL10` (`passWithNoTests` passes an empty or filtered-to-nothing run), `PL7` (no `coverageThreshold` / `coverage.thresholds`), `PL8` (`bail` stops the run early). The per-file scan cannot see config.
40
+
41
+ For the layer no static scan reaches (does a green test fail when the code is wrong?), run a **mutation tester** like [Stryker](https://stryker-mutator.io/). falsegreen-js is the cheap pre-filter on every commit; mutation testing is the deeper audit.
42
+
34
43
  Exit code: `0` clean, `10` low-confidence only, `20` high-confidence present. Wire it
35
44
  into CI or a pre-commit hook and let exit `20` block the commit.
36
45
 
@@ -44,8 +53,9 @@ expect(x); // falsegreen: ignore
44
53
  ## Runner coverage
45
54
 
46
55
  Runner-agnostic. The assertion and test vocabulary spans Jest, Vitest, Mocha + Chai,
47
- Jasmine, AVA, `node:test`, tap, Cypress, Playwright, and Testing Library
48
- (`@testing-library/*` with `jest-dom` / `jasmine-dom` matchers and `user-event`).
56
+ Jasmine, AVA, `node:test`, tap, Cypress, Playwright, Testing Library
57
+ (`@testing-library/*` with `jest-dom` / `jasmine-dom` matchers and `user-event`),
58
+ and Vue Test Utils (`mount`/`wrapper.find`/`flushPromises`/`nextTick`).
49
59
  `expect().matcher()`, chai `expect().to`, `assert`, `x.should`, and AVA `t.is` all
50
60
  count as real assertions, so a Mocha or AVA test is not mistaken for one that never
51
61
  checks anything.
@@ -54,6 +64,25 @@ Note: component files (`.vue`, `.svelte`, `.astro`, `.marko`) and templates (`.h
54
64
  are not test files. Tests for those frameworks are written in `.spec`/`.test` files in
55
65
  the eight extensions above, which is what the scanner reads.
56
66
 
67
+ ## Test levels (the pyramid)
68
+
69
+ falsegreen-js scans tests at every level of the pyramid. Discovery is level-agnostic - it
70
+ reads any test file - but a few codes are read in light of the level, so a valid pattern at
71
+ one level is not flagged at another.
72
+
73
+ - **Unit:** a function or component with its boundaries doubled. The oracle is `expect`.
74
+ - **Integration (API and database):** API tests through supertest / chai-http
75
+ (`request(app).get("/").expect(200)`, recognized as an assertion) or `fetch`, and database
76
+ tests through Prisma / TypeORM / Knex against a real datastore. These cross the I/O
77
+ boundary on purpose, so the response or row IS the verification at that level.
78
+ - **E2E:** Cypress (`.cy.*`) and Playwright (`.e2e.*`). `cy.get().should(...)` and
79
+ `expect(page).toHaveURL(...)` are the oracle; a visible element is a real check here, not a
80
+ weak one.
81
+
82
+ A real API or database call inside a test that claims to be a unit test is itself the smell
83
+ (mystery guest, environment coupling), not the level of the test. C23 flags the hard-coded
84
+ file path or URL form.
85
+
57
86
  ## Case catalog
58
87
 
59
88
  Codes shared with `falsegreen` (Python) keep the same id, so cross-language results
@@ -64,7 +93,10 @@ line up in the research. `JS*` codes are ecosystem-specific.
64
93
  | C2 | high | test with no check at all (empty body) |
65
94
  | C2b | low | test calls code but asserts nothing |
66
95
  | C5 | high | always-true check (`expect(true).toBe(true)`, `assert(1)`) |
96
+ | C6 | low | weak check — only verifies something came back (`toBeTruthy`/`toBeDefined`, `length > 0`) |
67
97
  | C7 | high | compares a thing to itself (`expect(x).toBe(x)`) |
98
+ | C20 | high | assertion in dead code after a `return`/`throw` — it never runs |
99
+ | C23 | low | reads a real file at a literal path, or a hard-coded URL (mystery guest) |
68
100
  | C8 | low | exact equality on a float (use `toBeCloseTo`) |
69
101
  | C9 | low | `toThrow()` with no error type or message — accepts any error |
70
102
  | C16 | low | result depends on `Date.now`, `Math.random`, or a fixed timer |
@@ -79,10 +111,15 @@ line up in the research. `JS*` codes are ecosystem-specific.
79
111
  | JS5 | low | async query/event not awaited (`findBy*` / `waitFor` / `user-event`) |
80
112
  | JS6 | high | empty `describe`/`suite` — the suite is green but runs nothing |
81
113
  | JS7 | low | assertion inside a non-awaited `setTimeout`/`then` callback — may run after the test ends |
114
+ | JS8 | low | mocks the unit under test (`jest.mock`/`vi.mock` of an imported module asserted directly) |
82
115
  | JS9 | high | assertion in a dead branch (`if(false)` / `if(true){}else`) — never runs |
83
116
  | JS11 | low | `try/catch` swallows the assertion — a failing `expect` is caught, test stays green |
84
117
  | JS13 | low | query (`getBy*`/`queryBy*`) as a loose statement — its result is never asserted |
85
118
  | JS15 | low | inappropriate assertion — comparison wrapped in a boolean (`expect(a===b).toBe(true)`), blind failure message |
119
+ | JS17 | low | commented-out test block (`// it(...)` / `// test(...)`) — disabled, no longer runs |
120
+ | JS18 | low | test takes a `done` callback instead of async/await — a mistimed `done` passes early |
121
+ | JS21 | high | matcher referenced but never called (`expect(x).toBe` with no `()`) — the assertion never runs |
122
+ | JS22 | high | empty `it.each`/`test.each` table — generated with zero cases, never runs |
86
123
 
87
124
  Each code carries a judgment tag (J1-J6) shared with the
88
125
  [falsegreen-skill](https://github.com/vinicq/falsegreen-skill) semantic framework.
@@ -107,12 +144,19 @@ default. Enable them with `--diagnostics`, or per code via config `severity`. Th
107
144
  npx falsegreen-js --diagnostics # include D*/M* as warnings
108
145
  ```
109
146
 
110
- ### Roadmap (researched, not yet active)
147
+ ### Deliberately not implemented
148
+
149
+ Some catalog codes were reviewed and left out, on purpose:
111
150
 
112
- Tracked in the research hub, pending implementation: JS8 (mocking the unit under test),
113
- JS10 (async test with no `expect.assertions` and the assertion only in a `catch`), JS12
114
- (`render(<C/>)`/`mount()` with no query or assertion), JS13 (a `getBy*`/`queryBy*` query
115
- as a loose statement), and a general Mystery Guest / external-resource code.
151
+ - **JS19** (`toBe` on an object/array literal): `expect(x).toBe({...})` compares by reference,
152
+ so it always fails. That is a loud red test, the opposite of false-green, and out of scope.
153
+ - **JS20** (a Promise compared without `resolves`/`rejects`): telling that a value is a
154
+ Promise needs type information the parser does not have, so it would be too noisy.
155
+ - **JS12** (a floating promise whose `expect` is never returned): already covered by JS7.
156
+ - **JS16** (`async` test with no `expect.assertions(n)`): the absence of a guard is not a
157
+ smell on its own; flagging it would fire on most async tests.
158
+ - **JS10** (any conditional in a test body): handled by `eslint-plugin-jest`
159
+ (`no-conditional-in-test`); JS9 and C21 already cover the false-green subset.
116
160
 
117
161
  ### What carries over from falsegreen, what does not
118
162
 
@@ -167,3 +211,30 @@ tracked as research in the private audit hub. Issues and PRs welcome.
167
211
  ## License
168
212
 
169
213
  MIT, Vinicius Queiroz.
214
+
215
+ ## Contributors ✨
216
+
217
+ Thanks to the people who keep false-green tests out of real suites ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
218
+
219
+ <!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
220
+ [![All Contributors](https://img.shields.io/badge/all_contributors-2-orange.svg?style=flat-square)](#contributors-)
221
+ <!-- ALL-CONTRIBUTORS-BADGE:END -->
222
+
223
+ <!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
224
+ <!-- prettier-ignore-start -->
225
+ <!-- markdownlint-disable -->
226
+ <table>
227
+ <tbody>
228
+ <tr>
229
+ <td align="center" valign="top" width="14.28%"><a href="https://vinicq.github.io/md-bridge/"><img src="https://avatars.githubusercontent.com/u/78210890?v=4?s=100" width="100px;" alt="Vinicius Queiroz"/><br /><sub><b>Vinicius Queiroz</b></sub></a><br /><a href="https://github.com/vinicq/falsegreen-js/commits?author=vinicq" title="Code">💻</a> <a href="https://github.com/vinicq/falsegreen-js/commits?author=vinicq" title="Documentation">📖</a> <a href="#ideas-vinicq" title="Ideas, Planning, & Feedback">🤔</a> <a href="#maintenance-vinicq" title="Maintenance">🚧</a> <a href="#infra-vinicq" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a> <a href="https://github.com/vinicq/falsegreen-js/commits?author=vinicq" title="Tests">⚠️</a> <a href="#research-vinicq" title="Research">🔬</a></td>
230
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/homesellerq-coder"><img src="https://avatars.githubusercontent.com/u/294912019?v=4?s=100" width="100px;" alt="Home Seller"/><br /><sub><b>Home Seller</b></sub></a><br /><a href="https://github.com/vinicq/falsegreen-js/commits?author=homesellerq-coder" title="Code">💻</a></td>
231
+ </tr>
232
+ </tbody>
233
+ </table>
234
+
235
+ <!-- markdownlint-restore -->
236
+ <!-- prettier-ignore-end -->
237
+
238
+ <!-- ALL-CONTRIBUTORS-LIST:END -->
239
+
240
+ New contributors are added automatically; the table also recognizes non-code work (docs, ideas, infrastructure, tests, research) via the [all-contributors](https://allcontributors.org) spec.
@@ -0,0 +1,5 @@
1
+ import { Finding } from "./types.js";
2
+ /** Read the Jest/Vitest config and report the project-layer PL codes: ways the
3
+ * suite can report green by configuration. Findings carry level `project`.
4
+ * Returns [] when no Jest/Vitest config is found. */
5
+ export declare function auditConfig(start?: string): Finding[];
package/dist/audit.js ADDED
@@ -0,0 +1,120 @@
1
+ import * as fs from "node:fs";
2
+ import * as path from "node:path";
3
+ import ts from "typescript";
4
+ import { makeFinding } from "./types.js";
5
+ import { parse } from "./parse.js";
6
+ /** JSON config: package.json `jest` field, or jest.config.json. */
7
+ function readJsonConfig(start) {
8
+ const pkg = path.join(start, "package.json");
9
+ if (fs.existsSync(pkg)) {
10
+ try {
11
+ const j = JSON.parse(fs.readFileSync(pkg, "utf-8"));
12
+ if (j && typeof j.jest === "object")
13
+ return { path: pkg, cfg: j.jest };
14
+ }
15
+ catch { /* unreadable */ }
16
+ }
17
+ const jcj = path.join(start, "jest.config.json");
18
+ if (fs.existsSync(jcj)) {
19
+ try {
20
+ return { path: jcj, cfg: JSON.parse(fs.readFileSync(jcj, "utf-8")) };
21
+ }
22
+ catch { /* unreadable */ }
23
+ }
24
+ return null;
25
+ }
26
+ /** JS/TS config (jest.config.*, vitest.config.*, vite.config.*): AST-walk for
27
+ * the property assignments of interest. A heuristic, not full evaluation. */
28
+ function readAstConfig(start) {
29
+ const candidates = [
30
+ "jest.config.ts", "jest.config.js", "jest.config.mjs", "jest.config.cjs",
31
+ "vitest.config.ts", "vitest.config.js", "vitest.config.mts",
32
+ "vite.config.ts", "vite.config.js",
33
+ ];
34
+ for (const name of candidates) {
35
+ const p = path.join(start, name);
36
+ if (!fs.existsSync(p))
37
+ continue;
38
+ let text;
39
+ try {
40
+ text = fs.readFileSync(p, "utf-8");
41
+ }
42
+ catch {
43
+ continue;
44
+ }
45
+ const sf = parse(p, text);
46
+ const props = new Set();
47
+ let passWithNoTests = false;
48
+ let bail = false;
49
+ const visit = (node) => {
50
+ if (ts.isPropertyAssignment(node) &&
51
+ (ts.isIdentifier(node.name) || ts.isStringLiteral(node.name))) {
52
+ const key = node.name.text;
53
+ props.add(key);
54
+ const init = node.initializer;
55
+ if (key === "passWithNoTests" && init.kind === ts.SyntaxKind.TrueKeyword) {
56
+ passWithNoTests = true;
57
+ }
58
+ if (key === "bail") {
59
+ if (init.kind === ts.SyntaxKind.TrueKeyword)
60
+ bail = true;
61
+ else if (ts.isNumericLiteral(init) && Number(init.text) > 0)
62
+ bail = true;
63
+ }
64
+ }
65
+ ts.forEachChild(node, visit);
66
+ };
67
+ visit(sf);
68
+ return { path: p, props, passWithNoTests, bail };
69
+ }
70
+ return null;
71
+ }
72
+ function collect(start) {
73
+ const json = readJsonConfig(start);
74
+ const ast = readAstConfig(start);
75
+ if (!json && !ast)
76
+ return null;
77
+ let passWithNoTests = false;
78
+ let bail = false;
79
+ let hasCovGate = false;
80
+ if (json) {
81
+ const c = json.cfg;
82
+ if (c.passWithNoTests === true)
83
+ passWithNoTests = true;
84
+ if (c.bail === true || (typeof c.bail === "number" && c.bail > 0))
85
+ bail = true;
86
+ const cov = c.coverage;
87
+ if (c.coverageThreshold || c.thresholds || (cov && cov.thresholds))
88
+ hasCovGate = true;
89
+ }
90
+ if (ast) {
91
+ if (ast.passWithNoTests)
92
+ passWithNoTests = true;
93
+ if (ast.bail)
94
+ bail = true;
95
+ if (ast.props.has("coverageThreshold") || ast.props.has("thresholds"))
96
+ hasCovGate = true;
97
+ }
98
+ return { where: json?.path ?? ast.path, passWithNoTests, bail, hasCovGate };
99
+ }
100
+ /** Read the Jest/Vitest config and report the project-layer PL codes: ways the
101
+ * suite can report green by configuration. Findings carry level `project`.
102
+ * Returns [] when no Jest/Vitest config is found. */
103
+ export function auditConfig(start = process.cwd()) {
104
+ const sig = collect(start);
105
+ if (!sig)
106
+ return [];
107
+ const findings = [];
108
+ const mk = (code) => {
109
+ const f = makeFinding(sig.where, 1, code);
110
+ f.level = "project";
111
+ return f;
112
+ };
113
+ if (!sig.hasCovGate)
114
+ findings.push(mk("PL7"));
115
+ if (sig.bail)
116
+ findings.push(mk("PL8"));
117
+ if (sig.passWithNoTests)
118
+ findings.push(mk("PL10"));
119
+ return findings;
120
+ }
package/dist/cases.d.ts CHANGED
@@ -19,4 +19,13 @@ export declare const DIAGNOSTIC_THRESHOLDS: {
19
19
  assertionRoulette: number;
20
20
  longTest: number;
21
21
  };
22
- export declare function groupOf(code: string): "false-positive" | "diagnostic" | "coupling";
22
+ export declare function groupOf(code: string): "false-positive" | "diagnostic" | "coupling" | "project";
23
+ /** Test-pyramid level, detected from a file's import roots (see level.ts).
24
+ * `project` is the config-audit layer (--config-audit), not a file level. */
25
+ export type PyramidLevel = "unit" | "integration" | "e2e" | "project";
26
+ /**
27
+ * One-line remediation per case: what to change so the test protects something.
28
+ * Short, imperative, no trailing period. Surfaced in the status report (text +
29
+ * JSON `fix` field). A code missing here renders no fix line, never throws.
30
+ */
31
+ export declare const FIX_HINTS: Record<string, string>;
package/dist/cases.js CHANGED
@@ -19,7 +19,10 @@ export const CASES = {
19
19
  C2: { title: "test with no check at all (empty body)", confidence: "high", judgment: "J1" },
20
20
  C2b: { title: "test calls things but checks nothing", confidence: "low", judgment: "J1" },
21
21
  C5: { title: "always-true check (expect(true).toBe(true), assert(1))", confidence: "high", judgment: "J2" },
22
+ C6: { title: "weak check — only verifies something came back (toBeTruthy/toBeDefined, length > 0)", confidence: "low", judgment: "J4" },
22
23
  C7: { title: "compares a thing to itself (expect(x).toBe(x))", confidence: "high", judgment: "J2" },
24
+ C20: { title: "assertion in dead code after a return/throw — it never runs", confidence: "high", judgment: "J1" },
25
+ C23: { title: "reads a real file at a literal path or hits a hard-coded URL (mystery guest)", confidence: "low", judgment: "J6" },
23
26
  C8: { title: "exact equality on a float (fails on rounding, not bugs)", confidence: "low", judgment: "J4" },
24
27
  C16: { title: "result depends on time, randomness or a fixed timer", confidence: "low", judgment: "J1" },
25
28
  C18: { title: "compares String()/JSON.stringify()/`${x}` of a value to a literal (checks formatting, not the value)", confidence: "low", judgment: "J2" },
@@ -35,10 +38,15 @@ export const CASES = {
35
38
  JS5: { title: "async query/event not awaited (findBy* / waitFor / user-event) — the assertion may never settle", confidence: "low", judgment: "J1" },
36
39
  JS6: { title: "empty describe/suite block — the suite reports green but runs nothing", confidence: "high", judgment: "J1" },
37
40
  JS7: { title: "assertion inside a non-awaited setTimeout/setInterval/then callback — it may run after the test ends", confidence: "low", judgment: "J1" },
41
+ JS8: { title: "mocks the unit under test (jest.mock/vi.mock of an imported module asserted directly) — tests the mock, not the code", confidence: "low", judgment: "J3" },
38
42
  JS9: { title: "assertion in a dead branch (if(false) / if(true){}else) — it never runs", confidence: "high", judgment: "J1" },
39
43
  JS11: { title: "try/catch swallows the assertion — a failing expect is caught and the test stays green", confidence: "low", judgment: "J1" },
40
44
  JS13: { title: "query (getBy*/queryBy*/wrapper.find) as a loose statement — its result is never asserted", confidence: "low", judgment: "J4" },
41
45
  JS15: { title: "inappropriate assertion — the comparison is wrapped in a boolean (expect(a===b).toBe(true)), so the failure message is blind", confidence: "low", judgment: "J4" },
46
+ JS17: { title: "commented-out test block (// it(...) / // test(...)) — a disabled test that no longer runs", confidence: "low", judgment: "J1" },
47
+ JS18: { title: "test takes a done callback instead of async/await — a done called too early (or in a floating promise) passes before the assertions run", confidence: "low", judgment: "J1" },
48
+ JS21: { title: "matcher referenced but never called (expect(x).toBe with no ()) — the assertion never executes", confidence: "high", judgment: "J1" },
49
+ JS22: { title: "empty it.each/test.each table — the test is generated with zero cases and never runs", confidence: "high", judgment: "J1" },
42
50
  // --- diagnostic group (maintainability; default off, opt-in via --diagnostics
43
51
  // or config severity). These are NOT false-green: the test still protects. They
44
52
  // are a "plus" for test-code health, mirroring falsegreen's D/M group. -------
@@ -49,13 +57,68 @@ export const CASES = {
49
57
  D7: { title: "anonymous test — empty or missing description", confidence: "off", judgment: "J4" },
50
58
  D8: { title: "magic number in an assertion — a bare numeric literal instead of a named constant", confidence: "off", judgment: "J4" },
51
59
  M2: { title: "test body exceeds the line-count threshold — hard to read and maintain", confidence: "off", judgment: "J5" },
60
+ // --- project layer (config-audit only; emitted by --config-audit, never by
61
+ // the per-file scan). The suite goes green by configuration, not by a smell
62
+ // inside any one test file. ------------------------------------------------
63
+ PL7: { title: "no coverage gate (coverageThreshold / coverage.thresholds) - coverage can fall to zero and the suite still passes", confidence: "low", judgment: "J5" },
64
+ PL8: { title: "bail stops the run early (bail) - the reported test count is incomplete", confidence: "low", judgment: "J5" },
65
+ PL10: { title: "passWithNoTests lets an empty or fully-filtered suite report green", confidence: "low", judgment: "J1" },
52
66
  };
53
67
  /** Default thresholds for the diagnostic group (overridable later via config). */
54
68
  export const DIAGNOSTIC_THRESHOLDS = { assertionRoulette: 5, longTest: 50 };
55
69
  export function groupOf(code) {
70
+ if (code.startsWith("PL"))
71
+ return "project";
56
72
  if (code.startsWith("D"))
57
73
  return "diagnostic";
58
74
  if (code.startsWith("M"))
59
75
  return "coupling";
60
76
  return "false-positive";
61
77
  }
78
+ /**
79
+ * One-line remediation per case: what to change so the test protects something.
80
+ * Short, imperative, no trailing period. Surfaced in the status report (text +
81
+ * JSON `fix` field). A code missing here renders no fix line, never throws.
82
+ */
83
+ export const FIX_HINTS = {
84
+ C2: "add an assertion that checks the behaviour under test",
85
+ C2b: "assert the result of the call, not just that it ran",
86
+ C5: "assert the real behaviour, not a constant or tautology",
87
+ C6: "assert the actual value, not just that something came back",
88
+ C7: "compare against an independent expected value, not the subject itself",
89
+ C20: "move the assertion before the return/throw so it runs",
90
+ C23: "use a fixture or temp file instead of a real path or hard-coded URL",
91
+ C8: "use toBeCloseTo() or a tolerance instead of exact float equality",
92
+ C16: "freeze time and seed randomness so the result is deterministic",
93
+ C18: "assert the value, not its String()/JSON.stringify() form",
94
+ C21: "add at least one assertion that runs unconditionally",
95
+ C9: "pass an error type or message to toThrow()",
96
+ C37: "remove the duplicate it.each/test.each case",
97
+ CC: "restore the commented-out assertion, or delete it",
98
+ JS1: "remove .only (it.only/fit/describe.only) so the whole suite runs",
99
+ JS2: "add a matcher (expect(x).toBe(...)) so the assertion runs",
100
+ JS3: "add a real assertion; don't rely only on a self-generated snapshot",
101
+ JS4: "remove .skip/xit/todo, or implement the test",
102
+ JS5: "await the async query/event before asserting",
103
+ JS6: "add tests to the describe block, or remove it",
104
+ JS7: "await the promise/timer, or assert synchronously",
105
+ JS8: "unmock the unit under test; mock only its collaborators",
106
+ JS9: "remove the dead branch so the assertion runs",
107
+ JS11: "let the assertion error propagate; don't catch it",
108
+ JS13: "assert on the query result, not just query it",
109
+ JS15: "expect the value directly (expect(a).toBe(b)), not a boolean",
110
+ JS17: "restore the commented-out test, or delete it",
111
+ JS18: "use async/await instead of the done callback",
112
+ JS21: "call the matcher (add ()) so the assertion executes",
113
+ JS22: "add at least one row to the it.each/test.each table",
114
+ D1: "give each assertion a message, or split the test",
115
+ D3: "remove the duplicate assertion",
116
+ D4: "add titled cases to it.each/test.each",
117
+ D6: "remove console.* or replace it with an assertion",
118
+ D7: "give the test a description",
119
+ D8: "name the magic number with a constant",
120
+ M2: "split the long test into focused cases",
121
+ PL7: "set coverageThreshold (Jest) or coverage.thresholds (Vitest) to gate coverage",
122
+ PL8: "remove bail so the whole suite runs and the count is complete",
123
+ PL10: "drop passWithNoTests so an empty or filtered-to-nothing run fails",
124
+ };
package/dist/cli.d.ts CHANGED
@@ -1,2 +1,13 @@
1
1
  #!/usr/bin/env node
2
- export {};
2
+ import { Finding } from "./types.js";
3
+ /** Turn --output into a concrete file path. A directory (existing dir, a
4
+ * trailing separator, or an extension-less name like ".falsegreen") receives
5
+ * "report.<ext>" for the chosen format; anything else is treated as a file.
6
+ * Missing parent directories are created either way. */
7
+ export declare function resolveOutputPath(p: string, fmt: "json" | "text"): string;
8
+ export declare function renderText(findings: Finding[]): string;
9
+ /** True when this module is the process entry point. Package managers expose the
10
+ * `bin` through a symlink (node_modules/.bin/falsegreen-js), so process.argv[1]
11
+ * is the symlink while import.meta.url is the real dist/cli.js path; resolve the
12
+ * realpath before comparing, or `npx falsegreen-js` would exit without scanning. */
13
+ export declare function isDirectRun(invokedPath: string | undefined, moduleUrl: string): boolean;
package/dist/cli.js CHANGED
@@ -1,6 +1,10 @@
1
1
  #!/usr/bin/env node
2
- import { JUDGMENTS, groupOf } from "./cases.js";
2
+ import * as fs from "node:fs";
3
+ import * as path from "node:path";
4
+ import { pathToFileURL } from "node:url";
5
+ import { JUDGMENTS, CASES, groupOf, FIX_HINTS } from "./cases.js";
3
6
  import { scanPaths, scanFile, stagedFiles, loadConfig, } from "./scan.js";
7
+ import { auditConfig } from "./audit.js";
4
8
  const VERSION = "0.2.0";
5
9
  const TOOL_URI = "https://github.com/vinicq/falsegreen-js";
6
10
  const HELP = `falsegreen-js ${VERSION} - find false-positive JS/TS tests (static AST scan)
@@ -9,17 +13,23 @@ Usage:
9
13
  falsegreen-js [paths...] files/dirs; no args = scan cwd
10
14
  falsegreen-js --staged only test files staged in git
11
15
  falsegreen-js --json JSON output
16
+ falsegreen-js --output PATH write to a file, or report.<ext> into a directory
17
+ falsegreen-js --config-audit audit Jest/Vitest config (project-layer PL codes)
12
18
  falsegreen-js --diagnostics also report the opt-in maintainability group (D*/M*)
13
19
  falsegreen-js --disable C7,JS3 turn off specific codes
14
20
  falsegreen-js --version
15
21
  falsegreen-js --help
16
22
 
23
+ Each finding carries its pyramid level (unit/integration/e2e, read from imports)
24
+ and a one-line fix hint; the summary breaks findings down by level.
17
25
  Exit codes: 0 clean, 10 low-confidence only, 20 high-confidence present.
18
26
  Suppress inline: expect(x).toBe(x); // falsegreen: ignore[C7]
19
27
  Covers: .js .jsx .ts .tsx .mjs .cjs .mts .cts`;
20
28
  function parseArgs(argv) {
21
29
  const paths = [];
22
30
  let json = false, staged = false, help = false, version = false, diagnostics = false;
31
+ let configAudit = false;
32
+ let output;
23
33
  const disable = new Set();
24
34
  for (let i = 0; i < argv.length; i++) {
25
35
  const a = argv[i];
@@ -27,12 +37,18 @@ function parseArgs(argv) {
27
37
  json = true;
28
38
  else if (a === "--staged")
29
39
  staged = true;
40
+ else if (a === "--config-audit")
41
+ configAudit = true;
30
42
  else if (a === "--diagnostics")
31
43
  diagnostics = true;
32
44
  else if (a === "--help" || a === "-h")
33
45
  help = true;
34
46
  else if (a === "--version" || a === "-V")
35
47
  version = true;
48
+ else if (a === "--output")
49
+ output = argv[++i] ?? "";
50
+ else if (a.startsWith("--output="))
51
+ output = a.slice("--output=".length);
36
52
  else if (a === "--disable") {
37
53
  const v = argv[++i] ?? "";
38
54
  v.split(",").map((s) => s.trim()).filter(Boolean).forEach((c) => disable.add(c));
@@ -48,7 +64,30 @@ function parseArgs(argv) {
48
64
  else
49
65
  paths.push(a);
50
66
  }
51
- return { paths, json, staged, help, version, diagnostics, disable };
67
+ return { paths, json, staged, help, version, diagnostics, configAudit, disable, output };
68
+ }
69
+ /** Turn --output into a concrete file path. A directory (existing dir, a
70
+ * trailing separator, or an extension-less name like ".falsegreen") receives
71
+ * "report.<ext>" for the chosen format; anything else is treated as a file.
72
+ * Missing parent directories are created either way. */
73
+ export function resolveOutputPath(p, fmt) {
74
+ const ext = fmt === "json" ? "json" : "txt";
75
+ const trimmed = p.replace(/[/\\]+$/, "");
76
+ const base = path.basename(trimmed);
77
+ let isDir = /[/\\]$/.test(p) || path.extname(base) === "";
78
+ try {
79
+ if (fs.statSync(p).isDirectory())
80
+ isDir = true;
81
+ }
82
+ catch { /* missing path */ }
83
+ if (isDir) {
84
+ fs.mkdirSync(p, { recursive: true });
85
+ return path.join(p, `report.${ext}`);
86
+ }
87
+ const parent = path.dirname(p);
88
+ if (parent)
89
+ fs.mkdirSync(parent, { recursive: true });
90
+ return p;
52
91
  }
53
92
  function exitCode(findings) {
54
93
  if (findings.some((f) => f.confidence === "high"))
@@ -57,7 +96,7 @@ function exitCode(findings) {
57
96
  return 10;
58
97
  return 0;
59
98
  }
60
- function renderText(findings) {
99
+ export function renderText(findings) {
61
100
  if (findings.length === 0)
62
101
  return "falsegreen-js: no false-positive patterns found.";
63
102
  const byFile = new Map();
@@ -76,9 +115,30 @@ function renderText(findings) {
76
115
  low++;
77
116
  lines.push(` ${tag} ${f.code.padEnd(4)} L${f.line} ${f.title}` +
78
117
  (f.detail ? `\n ${f.detail}` : ""));
118
+ const hint = FIX_HINTS[f.code];
119
+ lines.push(` level: ${f.level}` + (hint ? ` fix: ${hint}` : ""));
79
120
  }
80
121
  }
81
122
  lines.push(`\n${high} high, ${low} low. ${TOOL_URI}`);
123
+ // Test-pyramid breakdown + the most common fixes, over every finding shown.
124
+ const byLevel = new Map();
125
+ const byCode = new Map();
126
+ for (const f of findings) {
127
+ byLevel.set(f.level, (byLevel.get(f.level) ?? 0) + 1);
128
+ byCode.set(f.code, (byCode.get(f.code) ?? 0) + 1);
129
+ }
130
+ const order = ["unit", "integration", "e2e"];
131
+ const levels = [
132
+ ...order.filter((l) => byLevel.has(l)),
133
+ ...[...byLevel.keys()].filter((l) => !order.includes(l)).sort(),
134
+ ];
135
+ lines.push("By level: " + levels.map((l) => `${l}:${byLevel.get(l)}`).join(", "));
136
+ const top = [...byCode.entries()]
137
+ .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])).slice(0, 3);
138
+ lines.push("Top fixes:");
139
+ for (const [code, n] of top) {
140
+ lines.push(` ${code} (${n}): ${FIX_HINTS[code] ?? CASES[code].title}`);
141
+ }
82
142
  return lines.join("\n");
83
143
  }
84
144
  function main() {
@@ -91,6 +151,26 @@ function main() {
91
151
  process.stdout.write(VERSION + "\n");
92
152
  process.exit(0);
93
153
  }
154
+ if (opt.configAudit) {
155
+ const base = opt.paths.find((p) => { try {
156
+ return fs.statSync(p).isDirectory();
157
+ }
158
+ catch {
159
+ return false;
160
+ } }) ?? ".";
161
+ const findings = auditConfig(base);
162
+ const rendered = opt.json
163
+ ? JSON.stringify({
164
+ tool: "falsegreen-js", version: VERSION, judgments: JUDGMENTS,
165
+ findings: findings.map((f) => ({ ...f, group: groupOf(f.code), fix: FIX_HINTS[f.code] ?? "" })),
166
+ }, null, 2)
167
+ : renderText(findings);
168
+ if (opt.output)
169
+ fs.writeFileSync(resolveOutputPath(opt.output, opt.json ? "json" : "text"), rendered + "\n");
170
+ else
171
+ process.stdout.write(rendered + "\n");
172
+ process.exit(findings.length ? 10 : 0);
173
+ }
94
174
  const config = loadConfig();
95
175
  const scanOpts = { config, cliDisable: opt.disable, diagnostics: opt.diagnostics };
96
176
  let findings;
@@ -100,17 +180,41 @@ function main() {
100
180
  else {
101
181
  findings = scanPaths(opt.paths.length ? opt.paths : ["."], scanOpts);
102
182
  }
103
- if (opt.json) {
104
- process.stdout.write(JSON.stringify({
183
+ const rendered = opt.json
184
+ ? JSON.stringify({
105
185
  tool: "falsegreen-js",
106
186
  version: VERSION,
107
187
  judgments: JUDGMENTS,
108
- findings: findings.map((f) => ({ ...f, group: groupOf(f.code) })),
109
- }, null, 2) + "\n");
188
+ findings: findings.map((f) => ({
189
+ ...f, group: groupOf(f.code), fix: FIX_HINTS[f.code] ?? "",
190
+ })),
191
+ }, null, 2)
192
+ : renderText(findings);
193
+ if (opt.output) {
194
+ const dest = resolveOutputPath(opt.output, opt.json ? "json" : "text");
195
+ fs.writeFileSync(dest, rendered + "\n");
110
196
  }
111
197
  else {
112
- process.stdout.write(renderText(findings) + "\n");
198
+ process.stdout.write(rendered + "\n");
113
199
  }
114
200
  process.exit(exitCode(findings));
115
201
  }
116
- main();
202
+ /** True when this module is the process entry point. Package managers expose the
203
+ * `bin` through a symlink (node_modules/.bin/falsegreen-js), so process.argv[1]
204
+ * is the symlink while import.meta.url is the real dist/cli.js path; resolve the
205
+ * realpath before comparing, or `npx falsegreen-js` would exit without scanning. */
206
+ export function isDirectRun(invokedPath, moduleUrl) {
207
+ if (!invokedPath)
208
+ return false;
209
+ let resolved = invokedPath;
210
+ try {
211
+ resolved = fs.realpathSync(invokedPath);
212
+ }
213
+ catch { /* keep raw path */ }
214
+ return moduleUrl === pathToFileURL(resolved).href;
215
+ }
216
+ // Run only when invoked as the CLI, so the module can be imported in tests
217
+ // without triggering a scan and process.exit.
218
+ if (isDirectRun(process.argv[1], import.meta.url)) {
219
+ main();
220
+ }
@@ -0,0 +1,10 @@
1
+ import ts from "typescript";
2
+ import { PyramidLevel } from "./cases.js";
3
+ /**
4
+ * Map a test file to a pyramid level from its import roots: "e2e" (browser
5
+ * driver / e2e framework), "integration" (HTTP client or database driver:
6
+ * API and DB tests), or "unit" (neither). Broadest wins. A real API/DB import
7
+ * in a test the author treats as a unit test is itself the smell, surfaced by
8
+ * the level mismatch.
9
+ */
10
+ export declare function detectPyramidLevel(sf: ts.SourceFile): PyramidLevel;
package/dist/level.js ADDED
@@ -0,0 +1,75 @@
1
+ import ts from "typescript";
2
+ // Browser drivers and end-to-end frameworks. A test file importing one of these
3
+ // drives a real browser or a full stack: it is an E2E test.
4
+ const E2E_ROOTS = new Set([
5
+ "cypress", "@playwright/test", "playwright", "playwright-core",
6
+ "selenium-webdriver", "webdriverio", "@wdio/globals", "puppeteer",
7
+ "puppeteer-core", "protractor", "nightwatch", "testcafe",
8
+ ]);
9
+ // HTTP clients / API mocks and real datastore drivers or ORMs. A test importing
10
+ // one of these crosses an I/O boundary (API or database): it is an integration
11
+ // test, where the response or the row is the oracle.
12
+ const INTEGRATION_ROOTS = new Set([
13
+ // API / HTTP
14
+ "supertest", "axios", "node-fetch", "cross-fetch", "got", "undici",
15
+ "superagent", "request", "nock", "msw", "pactum",
16
+ // database drivers / ORMs
17
+ "@prisma/client", "prisma", "typeorm", "sequelize", "mongoose", "mongodb",
18
+ "pg", "mysql", "mysql2", "redis", "ioredis", "knex", "better-sqlite3",
19
+ "sqlite3", "drizzle-orm", "testcontainers",
20
+ ]);
21
+ /** The package root of a module specifier, or null for a relative import.
22
+ * Scoped packages keep two segments (`@playwright/test`); others keep one. */
23
+ function packageRoot(spec) {
24
+ if (!spec || spec.startsWith(".") || spec.startsWith("/"))
25
+ return null;
26
+ const parts = spec.split("/");
27
+ return spec.startsWith("@") ? parts.slice(0, 2).join("/") : parts[0];
28
+ }
29
+ /** Every module specifier imported or required in the file. */
30
+ function importRoots(sf) {
31
+ const roots = new Set();
32
+ const add = (spec) => {
33
+ const root = spec ? packageRoot(spec) : null;
34
+ if (root)
35
+ roots.add(root);
36
+ };
37
+ const visit = (node) => {
38
+ if (ts.isImportDeclaration(node) && ts.isStringLiteral(node.moduleSpecifier)) {
39
+ add(node.moduleSpecifier.text);
40
+ }
41
+ else if (ts.isImportEqualsDeclaration(node) &&
42
+ ts.isExternalModuleReference(node.moduleReference) &&
43
+ ts.isStringLiteral(node.moduleReference.expression)) {
44
+ add(node.moduleReference.expression.text);
45
+ }
46
+ else if (ts.isCallExpression(node)) {
47
+ const fn = node.expression;
48
+ const isRequire = ts.isIdentifier(fn) && fn.text === "require";
49
+ const isDynImport = fn.kind === ts.SyntaxKind.ImportKeyword;
50
+ const arg = node.arguments[0];
51
+ if ((isRequire || isDynImport) && arg && ts.isStringLiteral(arg))
52
+ add(arg.text);
53
+ }
54
+ ts.forEachChild(node, visit);
55
+ };
56
+ visit(sf);
57
+ return roots;
58
+ }
59
+ /**
60
+ * Map a test file to a pyramid level from its import roots: "e2e" (browser
61
+ * driver / e2e framework), "integration" (HTTP client or database driver:
62
+ * API and DB tests), or "unit" (neither). Broadest wins. A real API/DB import
63
+ * in a test the author treats as a unit test is itself the smell, surfaced by
64
+ * the level mismatch.
65
+ */
66
+ export function detectPyramidLevel(sf) {
67
+ const roots = importRoots(sf);
68
+ for (const r of roots)
69
+ if (E2E_ROOTS.has(r))
70
+ return "e2e";
71
+ for (const r of roots)
72
+ if (INTEGRATION_ROOTS.has(r))
73
+ return "integration";
74
+ return "unit";
75
+ }
package/dist/rules.js CHANGED
@@ -23,6 +23,8 @@ const ASSERT_METHODS = new Set([
23
23
  const SNAPSHOT_MATCHERS = new Set([
24
24
  "toMatchSnapshot", "toMatchInlineSnapshot",
25
25
  "toThrowErrorMatchingSnapshot", "toThrowErrorMatchingInlineSnapshot",
26
+ // visual snapshots (Playwright): the baseline is generated from the output too
27
+ "toHaveScreenshot", "toMatchScreenshot",
26
28
  ]);
27
29
  const EQUALITY_MATCHERS = new Set(["toBe", "toEqual", "toStrictEqual"]);
28
30
  // --- name helpers ----------------------------------------------------------
@@ -39,6 +41,44 @@ function calleeName(expr) {
39
41
  return calleeName(expr.expression);
40
42
  return "";
41
43
  }
44
+ /** Leftmost identifier of a property/call/element chain: `a.b().c` -> "a". */
45
+ function rootIdent(e) {
46
+ let cur = e;
47
+ while (cur) {
48
+ if (ts.isIdentifier(cur))
49
+ return cur.text;
50
+ if (ts.isPropertyAccessExpression(cur) || ts.isCallExpression(cur) || ts.isElementAccessExpression(cur)) {
51
+ cur = cur.expression;
52
+ }
53
+ else if (ts.isParenthesizedExpression(cur) || ts.isNonNullExpression(cur)) {
54
+ cur = cur.expression;
55
+ }
56
+ else {
57
+ return null;
58
+ }
59
+ }
60
+ return null;
61
+ }
62
+ /** True if the expression chain bottoms out in an `expect(...)` call, walking
63
+ * through property access, calls, `.not`/`.resolves`/`.rejects`, and wrappers. */
64
+ function expectRooted(e) {
65
+ let cur = e;
66
+ while (cur) {
67
+ if (ts.isCallExpression(cur) && ts.isIdentifier(cur.expression) && cur.expression.text === "expect") {
68
+ return true;
69
+ }
70
+ if (ts.isPropertyAccessExpression(cur) || ts.isCallExpression(cur) || ts.isElementAccessExpression(cur)) {
71
+ cur = cur.expression;
72
+ }
73
+ else if (ts.isParenthesizedExpression(cur) || ts.isNonNullExpression(cur)) {
74
+ cur = cur.expression;
75
+ }
76
+ else {
77
+ return false;
78
+ }
79
+ }
80
+ return false;
81
+ }
42
82
  function literalTruthiness(e) {
43
83
  if (!e)
44
84
  return null;
@@ -141,6 +181,27 @@ function expectChain(call) {
141
181
  }
142
182
  return null;
143
183
  }
184
+ /** True if a call's result is observed: awaited, returned, assigned, or the
185
+ * implicit-return body of an arrow. A bare floating call (its enclosing
186
+ * statement is a plain ExpressionStatement) is NOT observed. Used to gate
187
+ * supertest `.expect()` so a floating API request still surfaces as C2b. */
188
+ function isObservedAsync(node) {
189
+ let cur = node;
190
+ let p = node.parent;
191
+ while (p) {
192
+ if (ts.isAwaitExpression(p) || ts.isReturnStatement(p))
193
+ return true;
194
+ if (ts.isVariableDeclaration(p) || ts.isBinaryExpression(p))
195
+ return true;
196
+ if (ts.isArrowFunction(p) && p.body === cur)
197
+ return true;
198
+ if (ts.isExpressionStatement(p))
199
+ return false;
200
+ cur = p;
201
+ p = p.parent;
202
+ }
203
+ return false;
204
+ }
144
205
  // --- assertion presence ----------------------------------------------------
145
206
  function isAssertionNode(node) {
146
207
  if (ts.isCallExpression(node)) {
@@ -151,6 +212,13 @@ function isAssertionNode(node) {
151
212
  // expect(x).matcher(...) — Jest, Vitest, Jasmine, Playwright, jest-dom, chai expect
152
213
  if (root === "expect" && ts.isPropertyAccessExpression(node.expression))
153
214
  return true;
215
+ // <chain>.expect(...) — supertest / chai-http API tests: request(app).get("/").expect(200)
216
+ // is the assertion (it throws on mismatch), but only when the request is awaited or
217
+ // returned. A floating `request(app).get("/").expect(200);` can finish after the test
218
+ // ends, so it stays uncovered (C2b) instead of scanning clean.
219
+ if (ts.isPropertyAccessExpression(node.expression) && node.expression.name.text === "expect"
220
+ && isObservedAsync(node))
221
+ return true;
154
222
  if (root === "assert")
155
223
  return true; // node:test, chai assert
156
224
  if (root === "sinon" && name.includes("assert"))
@@ -236,6 +304,8 @@ function getTestCallback(call) {
236
304
  }
237
305
  // --- JS5: async query/event not awaited (Testing Library) ------------------
238
306
  const ASYNC_AWAIT_LEAVES = new Set(["waitFor", "waitForElementToBeRemoved"]);
307
+ // Vue/Svelte async test helpers that return a promise and must be awaited.
308
+ const VUE_SVELTE_ASYNC = new Set(["flushPromises", "nextTick", "$nextTick", "tick"]);
239
309
  function isAsyncQueryCall(name) {
240
310
  const parts = name.split(".");
241
311
  const root = parts[0];
@@ -271,10 +341,30 @@ export function analyze(sf) {
271
341
  const push = (line, code, detail = "") => {
272
342
  findings.push(makeFinding(file, line, code, detail));
273
343
  };
344
+ // JS8 (self-mock) file-level state
345
+ const mockedAt = new Map(); // module -> line of jest.mock/vi.mock
346
+ const importBinding = new Map(); // binding name -> module
347
+ const expectRoots = new Set(); // root identifier used as expect subject
274
348
  const visit = (node) => {
349
+ if (ts.isImportDeclaration(node) && ts.isStringLiteral(node.moduleSpecifier)) {
350
+ const mod = node.moduleSpecifier.text;
351
+ const clause = node.importClause;
352
+ if (clause?.name)
353
+ importBinding.set(clause.name.text, mod);
354
+ const nb = clause?.namedBindings;
355
+ if (nb && ts.isNamespaceImport(nb))
356
+ importBinding.set(nb.name.text, mod);
357
+ if (nb && ts.isNamedImports(nb))
358
+ for (const el of nb.elements)
359
+ importBinding.set(el.name.text, mod);
360
+ }
275
361
  if (ts.isCallExpression(node)) {
276
362
  const name = calleeName(node.expression);
277
363
  const root = name.split(".")[0];
364
+ if ((name === "jest.mock" || name === "vi.mock" || name === "jest.doMock" || name === "vi.doMock") &&
365
+ node.arguments[0] && ts.isStringLiteral(node.arguments[0])) {
366
+ mockedAt.set(node.arguments[0].text, lineOf(sf, node));
367
+ }
278
368
  const modifier = name.split(".")[1] ?? "";
279
369
  // JS1 focused / JS4 skipped
280
370
  if (name.endsWith(".only") ||
@@ -294,9 +384,27 @@ export function analyze(sf) {
294
384
  ((TEST_BLOCK_ROOTS.has(root)) && (modifier === "only" || modifier === "skip" || modifier === "each"));
295
385
  if (isTestBlock) {
296
386
  const cb = getTestCallback(node);
387
+ // JS18: the test takes a `done` callback instead of async/await. A done
388
+ // called too early, or inside a floating promise, lets the test pass
389
+ // before the assertions run.
390
+ if (cb && cb.parameters.length > 0) {
391
+ const p0 = cb.parameters[0].name;
392
+ if (ts.isIdentifier(p0) && p0.text === "done") {
393
+ push(lineOf(sf, node), "JS18", "uses a done callback; prefer async/await");
394
+ }
395
+ }
297
396
  if (cb && cb.body && ts.isBlock(cb.body)) {
298
397
  const stmts = cb.body.statements;
299
398
  const line = lineOf(sf, node);
399
+ // C20: an assertion after a return/throw in the test body is dead code
400
+ let terminated = false;
401
+ for (const st of stmts) {
402
+ if (terminated && containsAssertion(st)) {
403
+ push(lineOf(sf, st), "C20", "assertion after a return/throw never runs");
404
+ }
405
+ if (ts.isReturnStatement(st) || ts.isThrowStatement(st))
406
+ terminated = true;
407
+ }
300
408
  if (stmts.length === 0) {
301
409
  push(line, "C2", "test body is empty");
302
410
  }
@@ -370,6 +478,11 @@ export function analyze(sf) {
370
478
  }
371
479
  // expect-chain matchers (C5, C7, C8)
372
480
  const chain = expectChain(node);
481
+ if (chain && chain.subject) {
482
+ const r = rootIdent(chain.subject);
483
+ if (r)
484
+ expectRoots.add(r);
485
+ }
373
486
  if (chain && !chain.negated) {
374
487
  const subj = chain.subject;
375
488
  const arg = chain.args[0];
@@ -403,6 +516,18 @@ export function analyze(sf) {
403
516
  push(lineOf(sf, node), "D8", `magic number ${arg.text} in the assertion`);
404
517
  }
405
518
  }
519
+ // C6 weak check: truthiness/defined-only, or length > 0, on a real (non-literal) value
520
+ if (subj && !isLiteral(subj) && !subjIsComparison) {
521
+ if (chain.matcher === "toBeTruthy" || chain.matcher === "toBeFalsy" || chain.matcher === "toBeDefined") {
522
+ push(lineOf(sf, node), "C6", "only checks the value is present, not the expected result");
523
+ }
524
+ else if (arg && ts.isNumericLiteral(arg) &&
525
+ ((chain.matcher === "toBeGreaterThan" && Number(arg.text) === 0) ||
526
+ (chain.matcher === "toBeGreaterThanOrEqual" && Number(arg.text) === 1)) &&
527
+ /\.length\b/.test(subj.getText(sf))) {
528
+ push(lineOf(sf, node), "C6", "only checks it is not empty");
529
+ }
530
+ }
406
531
  // C5 always-true
407
532
  if (chain.matcher === "toBeTruthy" && literalTruthiness(subj) === true) {
408
533
  push(lineOf(sf, node), "C5", "toBeTruthy on a constant truthy literal");
@@ -439,10 +564,29 @@ export function analyze(sf) {
439
564
  if (detail)
440
565
  push(lineOf(sf, node), "C16", detail);
441
566
  }
442
- // JS5: Testing Library async query/event used without await
567
+ // C23 mystery guest: real file at a literal path, or a hard-coded URL
568
+ {
569
+ const leaf = name.split(".").pop() ?? "";
570
+ const a0 = node.arguments[0];
571
+ const lit = a0 && ts.isStringLiteral(a0) ? a0.text : null;
572
+ if (lit && /^(readFileSync|readFile|openSync|createReadStream)$/.test(leaf) && /[\\/]/.test(lit)) {
573
+ push(lineOf(sf, node), "C23", "reads a real file at a literal path");
574
+ }
575
+ else if (lit && (leaf === "fetch" || name === "fetch" || leaf === "get") && /^https?:\/\//i.test(lit)) {
576
+ push(lineOf(sf, node), "C23", "hard-coded URL (mystery guest)");
577
+ }
578
+ }
579
+ // JS5: async query/event used without await. Testing Library (findBy*/waitFor/
580
+ // user-event) plus Vue/Svelte async helpers (flushPromises/nextTick/tick) in
581
+ // their promise form (no callback arg) used as a bare, non-awaited statement.
443
582
  if (isAsyncQueryCall(name) && ts.isExpressionStatement(node.parent)) {
444
583
  push(lineOf(sf, node), "JS5", `${name} is not awaited`);
445
584
  }
585
+ else if (ts.isExpressionStatement(node.parent) && node.arguments.length === 0 &&
586
+ VUE_SVELTE_ASYNC.has(name.split(".").pop() ?? "")) {
587
+ const leaf = name.split(".").pop();
588
+ push(lineOf(sf, node), "JS5", `${leaf}() is not awaited`);
589
+ }
446
590
  // JS7: assertion inside a non-awaited setTimeout/setInterval/then callback
447
591
  {
448
592
  const leaf = name.split(".").pop() ?? "";
@@ -461,15 +605,35 @@ export function analyze(sf) {
461
605
  }
462
606
  }
463
607
  }
464
- // JS13: a sync RTL query used as a loose statement (result never asserted)
608
+ // JS13: a sync query used as a loose statement (result never asserted).
609
+ // Testing Library getBy*/queryBy*, and Vue Test Utils findComponent /
610
+ // findAllComponents always, or find/findAll with a string selector (which
611
+ // distinguishes wrapper.find('.btn') from Array.prototype.find(fn)).
465
612
  if (ts.isExpressionStatement(node.parent)) {
466
613
  const qleaf = name.split(".").pop() ?? "";
467
- if (/^(getBy|getAllBy|queryBy|queryAllBy)/.test(qleaf)) {
614
+ const isRtlQuery = /^(getBy|getAllBy|queryBy|queryAllBy)/.test(qleaf);
615
+ const isVueComponentQuery = qleaf === "findComponent" || qleaf === "findAllComponents";
616
+ const isVueSelectorQuery = (qleaf === "find" || qleaf === "findAll") &&
617
+ node.arguments.length > 0 && ts.isStringLiteral(node.arguments[0]);
618
+ if (isRtlQuery || isVueComponentQuery || isVueSelectorQuery) {
468
619
  push(lineOf(sf, node), "JS13", `${qleaf}() result is not asserted`);
469
620
  }
470
621
  }
622
+ // A runner `.each` table: it.each / test.each / describe.each (and fit/xit
623
+ // variants). Gate the each-table codes on a runner root so a plain helper
624
+ // like `_.each([], fn)` or `lodash.each([])` is never mistaken for a test table.
625
+ const eachRoot = name.split(".")[0];
626
+ const isRunnerEach = name.endsWith(".each") &&
627
+ (TEST_BLOCK_ROOTS.has(eachRoot) || SUITE_ROOTS.has(eachRoot) ||
628
+ eachRoot === "fit" || eachRoot === "xit");
629
+ // JS22: empty it.each/test.each table — zero cases are generated, so the
630
+ // test is collected but never runs and the suite stays green.
631
+ if (isRunnerEach && node.arguments.length > 0 &&
632
+ ts.isArrayLiteralExpression(node.arguments[0]) && node.arguments[0].elements.length === 0) {
633
+ push(lineOf(sf, node), "JS22", "empty .each table — the test runs zero times");
634
+ }
471
635
  // C37: duplicate case in it.each/test.each table
472
- if (name.endsWith(".each") && node.arguments.length > 0 && ts.isArrayLiteralExpression(node.arguments[0])) {
636
+ if (isRunnerEach && node.arguments.length > 0 && ts.isArrayLiteralExpression(node.arguments[0])) {
473
637
  const seen = new Set();
474
638
  for (const el of node.arguments[0].elements) {
475
639
  const t = el.getText(sf).replace(/\s+/g, " ").trim();
@@ -508,18 +672,42 @@ export function analyze(sf) {
508
672
  push(lineOf(sf, node), "JS11", "a failing assertion in try is swallowed by catch");
509
673
  }
510
674
  }
675
+ // JS21: a matcher referenced but never called — `expect(x).toBe;` with no (),
676
+ // so the assertion object is built and dropped; nothing executes. The chain
677
+ // must be a bare statement (a call would make node.parent a CallExpression).
678
+ if (ts.isPropertyAccessExpression(node) &&
679
+ node.name.text.startsWith("to") &&
680
+ ts.isExpressionStatement(node.parent) &&
681
+ expectRooted(node.expression)) {
682
+ push(lineOf(sf, node), "JS21", `${node.name.text} is referenced but never called`);
683
+ }
511
684
  ts.forEachChild(node, visit);
512
685
  };
513
686
  visit(sf);
687
+ // JS8: a mocked module's imported binding is asserted directly -> testing the mock,
688
+ // not the real unit. Conservative: same module both mocked and imported, and that
689
+ // import used as an expect subject.
690
+ for (const [binding, mod] of importBinding) {
691
+ if (mockedAt.has(mod) && expectRoots.has(binding)) {
692
+ findings.push(makeFinding(file, mockedAt.get(mod), "JS8", `${binding} (from ${mod}) is mocked and asserted directly`));
693
+ break;
694
+ }
695
+ }
514
696
  // CC: commented-out assertion (text scan over single-line comments)
515
697
  // CC: a single-line `//` comment that is a commented-out assertion call.
516
698
  // Requires the call paren (expect(/assert(/assert.x() or a .should chain) so it
517
699
  // does not match JSDoc prose like ` * assert that ...`.
518
700
  const lines = text.split(/\r?\n/);
519
701
  const ccRe = /^\s*\/\/\s*(?:await\s+)?(?:expect\s*\(|assert(?:\.\w+)?\s*\(|[\w.]+\.should\b)/;
702
+ // JS17: a commented-out test block (// it('...', / // test(...) / // describe(...)),
703
+ // optionally with .skip/.only/.each. A disabled test that no longer runs and no
704
+ // longer shows up as skipped.
705
+ const js17Re = /^\s*\/\/\s*(?:it|test|describe|context|specify)(?:\.\w+)?\s*\(/;
520
706
  lines.forEach((ln, i) => {
521
707
  if (ccRe.test(ln))
522
708
  push(i + 1, "CC", "assertion is commented out");
709
+ else if (js17Re.test(ln))
710
+ push(i + 1, "JS17", "test block is commented out");
523
711
  });
524
712
  return findings;
525
713
  }
package/dist/scan.js CHANGED
Binary file
package/dist/types.d.ts CHANGED
@@ -1,4 +1,4 @@
1
- import { Confidence } from "./cases.js";
1
+ import { Confidence, PyramidLevel } from "./cases.js";
2
2
  export interface Finding {
3
3
  file: string;
4
4
  line: number;
@@ -6,5 +6,6 @@ export interface Finding {
6
6
  detail: string;
7
7
  confidence: Confidence;
8
8
  title: string;
9
+ level: PyramidLevel;
9
10
  }
10
11
  export declare function makeFinding(file: string, line: number, code: string, detail?: string, confidence?: Confidence): Finding;
package/dist/types.js CHANGED
@@ -7,5 +7,6 @@ export function makeFinding(file, line, code, detail = "", confidence) {
7
7
  detail,
8
8
  confidence: confidence ?? CASES[code].confidence,
9
9
  title: CASES[code].title,
10
+ level: "unit",
10
11
  };
11
12
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "falsegreen-js",
3
- "version": "0.2.0",
3
+ "version": "0.3.0",
4
4
  "description": "Find JavaScript/TypeScript unit tests that give false positives: green tests that protect nothing, and tests that pass while asserting the wrong thing. Deterministic AST scanner, sibling of falsegreen (Python).",
5
5
  "keywords": [
6
6
  "jest", "vitest", "mocha", "testing", "test-smells", "false-positive",