falsegreen-js 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +52 -0
- package/README.md +8 -0
- package/dist/cases.js +16 -0
- package/dist/rules.js +343 -2
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,58 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.6.0] - 2026-06-29
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
- `JS25` (high, J1): the only assertion sits inside an array-iterator callback
|
|
13
|
+
(`forEach`/`map`/`filter`/`some`/`every`/`flatMap`). On an empty collection the
|
|
14
|
+
callback runs zero times, so nothing is checked and the test still goes green.
|
|
15
|
+
Fills the verified gap between C2/C2b (whose `hasAssertion` descends into the
|
|
16
|
+
callback and finds the assertion) and C21 (whose own-scope scan stops at the
|
|
17
|
+
callback and sees none). FP guards: an own-scope assertion, a non-empty
|
|
18
|
+
array-literal receiver, or an `expect.assertions`/`hasAssertions` guard suppress it.
|
|
19
|
+
- `JS30` (high, J2): literal-vs-literal assertion such as `expect(2).toBe(3)` or
|
|
20
|
+
chai `expect(x).to.equal(y)`, where both operands are literal nodes through an
|
|
21
|
+
equality matcher (`toBe`/`toEqual`/`toStrictEqual`/`toBeCloseTo`/`equal`/`equals`/
|
|
22
|
+
`eql`/`is`). The comparison is fixed at parse time, independent of any code. The
|
|
23
|
+
same-token case (`expect(1).toBe(1)`) stays with C5; object/array literals
|
|
24
|
+
(reference equality) and template literals with substitutions are excluded.
|
|
25
|
+
- `JS31` (low, J1): a `try/catch` whose try calls code that may throw and whose
|
|
26
|
+
catch neither asserts on the exception, re-raises, nor calls `fail()`. A unit that
|
|
27
|
+
stops throwing (a real regression) still passes green. Complement of JS11, which
|
|
28
|
+
owns the swallowed-assertion case; JS31 fires only when the try has a call but no
|
|
29
|
+
assertion and the catch is harmless.
|
|
30
|
+
- `JS27` (low, J3): `toHaveBeenCalled*` is the sole oracle on a locally-created
|
|
31
|
+
double (`jest.fn`/`vi.fn`/`spyOn`). The test confirms it called the double it set
|
|
32
|
+
up, not the unit's output or state. Gated to the unit level (a logger-spy call
|
|
33
|
+
check is legitimate at integration/e2e) and suppressed when any non-call-tracking
|
|
34
|
+
assertion is present. Sibling of JS8.
|
|
35
|
+
- `JS26` (low, J1): fake timers installed but never advanced. A `setTimeout`/
|
|
36
|
+
`setInterval` is armed under frozen fake timers and nothing in the test scope (nor
|
|
37
|
+
a sibling `before`/`after` hook) calls `runAllTimers`/`advanceTimersByTime`/`tick`,
|
|
38
|
+
so the scheduled callback never fires and the assertion reads un-mutated state. The
|
|
39
|
+
opposite of C16 (uncontrolled timer). Requires an assertion in scope; a flush in
|
|
40
|
+
the body or an enclosing hook suppresses it.
|
|
41
|
+
- `JS29` (low, J6): an `expect(...).resolves`/`.rejects` chain that is a bare
|
|
42
|
+
statement, not awaited or returned. The matcher settles asynchronously, so a
|
|
43
|
+
floating chain finishes green before it resolves. The statically-provable subset of
|
|
44
|
+
the skipped JS20, keyed on the explicit `.resolves`/`.rejects` marker so no type
|
|
45
|
+
inference is needed.
|
|
46
|
+
- `C8b` (low, J4): `toBeCloseTo` called with no precision argument, so the default
|
|
47
|
+
2-digit tolerance applies. The js analogue of `assertAlmostEqual`/`pytest.approx`
|
|
48
|
+
with no tolerance, ported from `falsegreen`. Implicit-precision only; a
|
|
49
|
+
literal-vs-literal `toBeCloseTo` stays with JS30.
|
|
50
|
+
- `C11a` (low, J2): self-confirming literal. The expected value is bound from the
|
|
51
|
+
same call under test (`const e = foo(); expect(foo()).toBe(e)`), so the oracle
|
|
52
|
+
confirms the code against itself. Ported from `falsegreen`; the bound initializer
|
|
53
|
+
must provably be the SUT call (its source text equals the expect subject's), so a
|
|
54
|
+
literal or a different-call binding stays clean.
|
|
55
|
+
|
|
56
|
+
### Changed
|
|
57
|
+
- `C8` (exact float) now fires only when the subject is a real (non-literal) value;
|
|
58
|
+
a literal-vs-literal float (`expect(0.1).toBe(0.3)`) is owned by the stronger JS30
|
|
59
|
+
lane, so the two no longer double-report.
|
|
60
|
+
|
|
9
61
|
## [0.5.0] - 2026-06-28
|
|
10
62
|
|
|
11
63
|
### Added
|
package/README.md
CHANGED
|
@@ -137,7 +137,9 @@ line up in the research. `JS*` codes are ecosystem-specific.
|
|
|
137
137
|
| C20 | high | assertion in unreachable code (after a `return`/`throw`/`process.exit`, a `break`, a both-arms-terminating `if`, or an exhaustive `switch`) — it never runs |
|
|
138
138
|
| C23 | low | reads a real file at a literal path, or a hard-coded URL (mystery guest) |
|
|
139
139
|
| C8 | low | exact equality on a float (use `toBeCloseTo`) |
|
|
140
|
+
| C8b | low | `toBeCloseTo` with no precision argument — the default 2-digit tolerance may be too loose |
|
|
140
141
|
| C9 | low | `toThrow()` with no error type or message — accepts any error |
|
|
142
|
+
| C11a | low | self-confirming literal — the expected value is bound from the same call under test (`const e = foo(); expect(foo()).toBe(e)`) |
|
|
141
143
|
| C16 | low | result depends on `Date.now`, `Math.random`, or a fixed timer |
|
|
142
144
|
| C18 | low | compares `String(x)` / `JSON.stringify(x)` / `` `${x}` `` to a literal (formatting, not value) |
|
|
143
145
|
| C21 | low | every assertion is conditional — none runs unconditionally |
|
|
@@ -162,6 +164,12 @@ line up in the research. `JS*` codes are ecosystem-specific.
|
|
|
162
164
|
| JS22 | high | empty `it.each`/`test.each` table — generated with zero cases, never runs |
|
|
163
165
|
| JS23 | high | `expect.assertions(N)` with fewer unconditional `expect()` calls than N — the guard can never be met |
|
|
164
166
|
| JS24 | low | Cypress query (`cy.get`/`cy.find`/`cy.contains`) as a loose statement with no terminating `.should`/`.and` and no `expect` in `.then` — its result is never asserted |
|
|
167
|
+
| JS25 | high | the only assertion sits inside an array-iterator callback (`forEach`/`map`/`filter`/`some`/`every`/`flatMap`) — runs zero times on an empty collection |
|
|
168
|
+
| JS26 | low | fake timers installed but never advanced (`runAllTimers`/`advanceTimersByTime`/`tick`) — the scheduled callback never fires, so the assertion reads un-mutated state |
|
|
169
|
+
| JS27 | low | `toHaveBeenCalled*` is the sole oracle on a locally-created double — verifies wiring, not behaviour |
|
|
170
|
+
| JS29 | low | `expect(...).resolves`/`.rejects` chain is a bare statement, not awaited or returned — the test finishes green before the matcher settles |
|
|
171
|
+
| JS30 | high | literal-vs-literal assertion (`expect(2).toBe(3)`, chai `expect(x).to.equal(y)`) — both operands are fixed at parse time |
|
|
172
|
+
| JS31 | low | `try/catch` swallows a possible throw with no assertion on the exception — a unit that stops throwing still passes green |
|
|
165
173
|
|
|
166
174
|
Each code carries a judgment tag (J1-J6) shared with the
|
|
167
175
|
[falsegreen-skill](https://github.com/vinicq/falsegreen-skill) semantic framework.
|
package/dist/cases.js
CHANGED
|
@@ -61,6 +61,14 @@ export const CASES = {
|
|
|
61
61
|
JS22: { title: "empty it.each/test.each table — the test is generated with zero cases and never runs", group: "execution", severity: "high", defaultOn: true, judgment: "J1" },
|
|
62
62
|
JS23: { title: "expect.assertions(N) with fewer unconditional expect() calls than N — the guard can never be met", group: "execution", severity: "high", defaultOn: true, judgment: "J1" },
|
|
63
63
|
JS24: { title: "Cypress query (cy.get/find/contains) with no terminating .should/.and and no expect in .then — its result is never asserted", group: "effectiveness", severity: "low", defaultOn: true, judgment: "J4" },
|
|
64
|
+
JS25: { title: "the only assertion sits inside an array-iterator callback (forEach/map/filter/some/every/flatMap) — runs zero times on an empty collection", group: "execution", severity: "high", defaultOn: true, judgment: "J1" },
|
|
65
|
+
JS26: { title: "fake timers installed but never advanced — the scheduled callback never fires, so the assertion runs against un-mutated state", group: "execution", severity: "low", defaultOn: true, judgment: "J1" },
|
|
66
|
+
JS27: { title: "toHaveBeenCalled* is the sole oracle on a locally-created double — verifies wiring, not behaviour", group: "dependency", severity: "low", defaultOn: true, judgment: "J3" },
|
|
67
|
+
JS29: { title: "expect(...).resolves/.rejects chain is a bare statement, not awaited or returned — the test finishes green before the matcher settles", group: "execution", severity: "low", defaultOn: true, judgment: "J6" },
|
|
68
|
+
JS30: { title: "literal-vs-literal assertion (expect(1).toBe(1)) — both operands are fixed at parse time, independent of any code", group: "effectiveness", severity: "high", defaultOn: true, judgment: "J2" },
|
|
69
|
+
JS31: { title: "try/catch swallows a SUT throw with no assertion on the exception — a unit that stops throwing still passes green", group: "execution", severity: "low", defaultOn: true, judgment: "J1" },
|
|
70
|
+
C8b: { title: "toBeCloseTo with no precision argument — the default 2-digit tolerance may be too loose for the value under test", group: "effectiveness", severity: "low", defaultOn: true, judgment: "J4" },
|
|
71
|
+
C11a: { title: "self-confirming literal — the expected value is bound from the same call/expression under test (const e = foo(); expect(foo()).toBe(e))", group: "effectiveness", severity: "low", defaultOn: true, judgment: "J2" },
|
|
64
72
|
// --- diagnostic group (maintainability; default off, opt-in via --diagnostics
|
|
65
73
|
// or config severity). These are NOT false-green: the test still protects. They
|
|
66
74
|
// are a "plus" for test-code health, mirroring falsegreen's D/M group. -------
|
|
@@ -159,6 +167,14 @@ export const FIX_HINTS = {
|
|
|
159
167
|
JS22: "add at least one row to the it.each/test.each table",
|
|
160
168
|
JS23: "make the unconditional expect() count match expect.assertions(N), or remove the guard",
|
|
161
169
|
JS24: "end the cy query in .should()/.and(), or assert in .then()",
|
|
170
|
+
JS25: "add an assertion outside the iterator callback so it runs even when the collection is empty",
|
|
171
|
+
JS26: "advance the fake timers (runAllTimers/advanceTimersByTime/tick) before asserting",
|
|
172
|
+
JS27: "assert the unit's output or state, not just that the double was called",
|
|
173
|
+
JS29: "await or return the resolves/rejects assertion so the matcher settles",
|
|
174
|
+
JS30: "assert a real value against an independent expected one, not literal against literal",
|
|
175
|
+
JS31: "assert on the caught error, re-throw, call fail(), or wrap the call in expect().toThrow()",
|
|
176
|
+
C8b: "pass a precision argument to toBeCloseTo(), or choose an explicit tolerance",
|
|
177
|
+
C11a: "bind the expected value independently of the call under test",
|
|
162
178
|
D1: "give each assertion a message, or split the test",
|
|
163
179
|
D3: "remove the duplicate assertion",
|
|
164
180
|
D4: "add titled cases to it.each/test.each",
|
package/dist/rules.js
CHANGED
|
@@ -4,6 +4,7 @@ import { DIAGNOSTIC_THRESHOLDS } from "./cases.js";
|
|
|
4
4
|
import { ASSERT_ROOTS, ASSERT_METHODS, SNAPSHOT_MATCHERS, EQUALITY_MATCHERS, VUE_SVELTE_ASYNC, oracleKind, } from "./oracles.js";
|
|
5
5
|
import { lineOf } from "./parse.js";
|
|
6
6
|
import { assertionsInDeadCode, hasUnconditionalAssertion } from "./cfg.js";
|
|
7
|
+
import { detectPyramidLevel } from "./level.js";
|
|
7
8
|
// --- test framework vocabulary (runner-agnostic) ---------------------------
|
|
8
9
|
// it/test/specify (Jest, Vitest, Mocha, Jasmine, AVA, node:test, Cypress,
|
|
9
10
|
// Playwright, tap). describe/context/suite are suites. fit/fdescribe focus and
|
|
@@ -14,6 +15,23 @@ import { assertionsInDeadCode, hasUnconditionalAssertion } from "./cfg.js";
|
|
|
14
15
|
const TEST_BLOCK_ROOTS = new Set(["it", "test", "specify"]);
|
|
15
16
|
const SUITE_ROOTS = new Set(["describe", "context", "suite", "fdescribe", "xdescribe", "fcontext", "xcontext"]);
|
|
16
17
|
const FOCUS_NAMES = new Set(["fit", "fdescribe", "fcontext"]);
|
|
18
|
+
// Array-iterator methods whose callback runs once per element — zero times on an
|
|
19
|
+
// empty collection. An assertion that lives ONLY inside one of these callbacks
|
|
20
|
+
// (JS25) runs zero times when the receiver is empty: green with nothing checked.
|
|
21
|
+
const ARRAY_ITERATOR_METHODS = new Set(["forEach", "map", "filter", "some", "every", "flatMap"]);
|
|
22
|
+
// Equality matchers across runners for the literal-vs-literal (JS30) and
|
|
23
|
+
// self-confirming-literal (C11a) lanes: Jest/Vitest toBe family + toBeCloseTo,
|
|
24
|
+
// plus chai/AVA equal/equals/eql/is. Broader than EQUALITY_MATCHERS (which gates
|
|
25
|
+
// the Jest-only C5/C7/C8 lanes) on purpose.
|
|
26
|
+
const EQ_MATCHERS_ANY = new Set([
|
|
27
|
+
"toBe", "toEqual", "toStrictEqual", "toBeCloseTo", "equal", "equals", "eql", "is",
|
|
28
|
+
]);
|
|
29
|
+
// toHaveBeenCalled* family (JS27): matchers that only assert a double was invoked.
|
|
30
|
+
const CALL_TRACKING_MATCHERS = new Set([
|
|
31
|
+
"toHaveBeenCalled", "toHaveBeenCalledTimes", "toHaveBeenCalledWith",
|
|
32
|
+
"toHaveBeenLastCalledWith", "toHaveBeenNthCalledWith", "toBeCalled",
|
|
33
|
+
"toBeCalledTimes", "toBeCalledWith", "toHaveBeenCalledOnce",
|
|
34
|
+
]);
|
|
17
35
|
const SKIP_NAMES = new Set(["xit", "xdescribe", "xcontext", "xspecify"]);
|
|
18
36
|
// --- C48 dark-patch: a test that flips a known test-mode flag then asserts ---
|
|
19
37
|
// env keys (process.env.<KEY> = <test-mode value>) whose name means "we are under
|
|
@@ -203,6 +221,23 @@ function expectChain(call) {
|
|
|
203
221
|
}
|
|
204
222
|
return null;
|
|
205
223
|
}
|
|
224
|
+
/** True if `call` is the terminal matcher call of an `expect(...).resolves`/
|
|
225
|
+
* `.rejects` chain (e.g. `expect(p).resolves.toBe(1)`). Walks the callee base for a
|
|
226
|
+
* `.resolves`/`.rejects` property access that bottoms out in an `expect(...)` call.
|
|
227
|
+
* Only the explicit resolves/rejects marker counts (JS29); a plain promise does not. */
|
|
228
|
+
function expectIsResolvesRejects(call) {
|
|
229
|
+
if (!ts.isPropertyAccessExpression(call.expression))
|
|
230
|
+
return false;
|
|
231
|
+
let base = call.expression.expression;
|
|
232
|
+
let sawSettle = false;
|
|
233
|
+
while (ts.isPropertyAccessExpression(base) || ts.isCallExpression(base)) {
|
|
234
|
+
if (ts.isPropertyAccessExpression(base) &&
|
|
235
|
+
(base.name.text === "resolves" || base.name.text === "rejects"))
|
|
236
|
+
sawSettle = true;
|
|
237
|
+
base = base.expression;
|
|
238
|
+
}
|
|
239
|
+
return sawSettle && expectRooted(call.expression);
|
|
240
|
+
}
|
|
206
241
|
/** True if a call's result is observed: awaited, returned, assigned, the
|
|
207
242
|
* implicit-return body of an arrow, or explicitly discarded with `void` (an
|
|
208
243
|
* author signalling "I am dropping this on purpose"). A bare floating call (its
|
|
@@ -366,6 +401,22 @@ function isHarmlessCatch(block) {
|
|
|
366
401
|
}
|
|
367
402
|
return true;
|
|
368
403
|
}
|
|
404
|
+
/** Stricter than isHarmlessCatch, for JS31: the catch truly swallows the throw,
|
|
405
|
+
* doing NOTHING with it. Only an empty body or `console.*` log-only statements
|
|
406
|
+
* qualify. An assignment (recovery flag like `supported = false`), a return, a
|
|
407
|
+
* fail()/throw, an assertion, or any other call is meaningful handling, not a
|
|
408
|
+
* silent swallow, so JS31 must not fire. Keeps JS31 precision-first. */
|
|
409
|
+
function catchSilentlySwallows(block) {
|
|
410
|
+
for (const stmt of block.statements) {
|
|
411
|
+
if (!ts.isExpressionStatement(stmt))
|
|
412
|
+
return false; // return/var/if/etc: handling
|
|
413
|
+
if (!ts.isCallExpression(stmt.expression))
|
|
414
|
+
return false; // a bare assignment/expr
|
|
415
|
+
if (!calleeName(stmt.expression.expression).startsWith("console."))
|
|
416
|
+
return false;
|
|
417
|
+
}
|
|
418
|
+
return true;
|
|
419
|
+
}
|
|
369
420
|
/** Matcher names of every expect-chain assertion under scope (for snapshot-only). */
|
|
370
421
|
function matchersUnder(scope) {
|
|
371
422
|
const out = [];
|
|
@@ -513,6 +564,176 @@ function expectSubjectRoots(scope) {
|
|
|
513
564
|
walk(scope);
|
|
514
565
|
return out;
|
|
515
566
|
}
|
|
567
|
+
/** JS25: every assertion under `scope` sits inside an array-iterator callback
|
|
568
|
+
* (arr.forEach/map/filter/some/every/flatMap(cb)), and at least one such
|
|
569
|
+
* assertion exists. Walk the body; when entering an iterator callback, any
|
|
570
|
+
* assertion inside counts as "iterator-bound"; an assertion reached outside one
|
|
571
|
+
* is an own-scope assertion that disproves JS25. Returns whether the only
|
|
572
|
+
* assertions are iterator-bound (and there is at least one). FP-averse: a single
|
|
573
|
+
* own-scope assertion anywhere suppresses it. */
|
|
574
|
+
function assertionsOnlyInArrayIterator(scope) {
|
|
575
|
+
let iteratorBound = 0;
|
|
576
|
+
let ownScope = 0;
|
|
577
|
+
// The callback nodes of array-iterator calls under scope, so the walk knows when
|
|
578
|
+
// an assertion is inside one without re-deriving the call shape per node.
|
|
579
|
+
const iterCallbacks = new Set();
|
|
580
|
+
const collect = (n) => {
|
|
581
|
+
if (ts.isCallExpression(n) && ts.isPropertyAccessExpression(n.expression) &&
|
|
582
|
+
ARRAY_ITERATOR_METHODS.has(n.expression.name.text)) {
|
|
583
|
+
for (const a of n.arguments) {
|
|
584
|
+
if (ts.isArrowFunction(a) || ts.isFunctionExpression(a))
|
|
585
|
+
iterCallbacks.add(a);
|
|
586
|
+
}
|
|
587
|
+
}
|
|
588
|
+
ts.forEachChild(n, collect);
|
|
589
|
+
};
|
|
590
|
+
collect(scope);
|
|
591
|
+
const walk = (n, insideIter) => {
|
|
592
|
+
const nowInside = insideIter || iterCallbacks.has(n);
|
|
593
|
+
if (isAssertionNode(n)) {
|
|
594
|
+
if (nowInside)
|
|
595
|
+
iteratorBound++;
|
|
596
|
+
else
|
|
597
|
+
ownScope++;
|
|
598
|
+
}
|
|
599
|
+
ts.forEachChild(n, (c) => walk(c, nowInside));
|
|
600
|
+
};
|
|
601
|
+
ts.forEachChild(scope, (c) => walk(c, false));
|
|
602
|
+
return ownScope === 0 && iteratorBound > 0;
|
|
603
|
+
}
|
|
604
|
+
/** True if `scope` carries an expect.assertions(N) / expect.hasAssertions() guard
|
|
605
|
+
* (JS23 territory) — used as a JS25 FP guard: the author already declared a count. */
|
|
606
|
+
function hasAssertionCountGuard(scope) {
|
|
607
|
+
let found = false;
|
|
608
|
+
const walk = (n) => {
|
|
609
|
+
if (found)
|
|
610
|
+
return;
|
|
611
|
+
if (ts.isCallExpression(n)) {
|
|
612
|
+
const nm = calleeName(n.expression);
|
|
613
|
+
if (nm === "expect.assertions" || nm === "expect.hasAssertions") {
|
|
614
|
+
found = true;
|
|
615
|
+
return;
|
|
616
|
+
}
|
|
617
|
+
}
|
|
618
|
+
ts.forEachChild(n, walk);
|
|
619
|
+
};
|
|
620
|
+
walk(scope);
|
|
621
|
+
return found;
|
|
622
|
+
}
|
|
623
|
+
/** True if any array-iterator call under `scope` iterates a non-empty array
|
|
624
|
+
* literal receiver (arr `[1, 2].forEach(...)`) — JS25 FP guard: a non-empty
|
|
625
|
+
* literal always runs the callback at least once, so the assertion does run. */
|
|
626
|
+
function iteratesNonEmptyArrayLiteral(scope) {
|
|
627
|
+
let found = false;
|
|
628
|
+
const walk = (n) => {
|
|
629
|
+
if (found)
|
|
630
|
+
return;
|
|
631
|
+
if (ts.isCallExpression(n) && ts.isPropertyAccessExpression(n.expression) &&
|
|
632
|
+
ARRAY_ITERATOR_METHODS.has(n.expression.name.text)) {
|
|
633
|
+
const recv = n.expression.expression;
|
|
634
|
+
if (ts.isArrayLiteralExpression(recv) && recv.elements.length > 0) {
|
|
635
|
+
found = true;
|
|
636
|
+
return;
|
|
637
|
+
}
|
|
638
|
+
}
|
|
639
|
+
ts.forEachChild(n, walk);
|
|
640
|
+
};
|
|
641
|
+
walk(scope);
|
|
642
|
+
return found;
|
|
643
|
+
}
|
|
644
|
+
/** JS27: the matcher names of every expect-chain assertion under `scope`, each
|
|
645
|
+
* paired with the root identifier of its subject. Used to decide whether the ONLY
|
|
646
|
+
* oracle is a toHaveBeenCalled* check on a locally-created double. */
|
|
647
|
+
function expectMatcherSubjects(scope) {
|
|
648
|
+
const out = [];
|
|
649
|
+
const walk = (n) => {
|
|
650
|
+
if (ts.isCallExpression(n)) {
|
|
651
|
+
const chain = expectChain(n);
|
|
652
|
+
if (chain)
|
|
653
|
+
out.push({ matcher: chain.matcher, root: chain.subject ? rootIdent(chain.subject) : null });
|
|
654
|
+
}
|
|
655
|
+
ts.forEachChild(n, walk);
|
|
656
|
+
};
|
|
657
|
+
walk(scope);
|
|
658
|
+
return out;
|
|
659
|
+
}
|
|
660
|
+
/** Root identifiers in `scope` bound to a freshly-created test double:
|
|
661
|
+
* jest.fn()/vi.fn()/jest.spyOn()/vi.spyOn() in a const/let, or a mockReturnValue/
|
|
662
|
+
* mockImplementation-decorated handle. Used by JS27 to confirm the call-tracking
|
|
663
|
+
* subject is a local double, not a real collaborator. */
|
|
664
|
+
function localDoubleRoots(scope) {
|
|
665
|
+
const out = new Set();
|
|
666
|
+
const isDoubleInit = (e) => {
|
|
667
|
+
let cur = e;
|
|
668
|
+
while (ts.isCallExpression(cur) || ts.isPropertyAccessExpression(cur)) {
|
|
669
|
+
if (ts.isCallExpression(cur)) {
|
|
670
|
+
const nm = calleeName(cur.expression);
|
|
671
|
+
if (nm === "jest.fn" || nm === "vi.fn" || nm === "jest.spyOn" || nm === "vi.spyOn" ||
|
|
672
|
+
nm === "sinon.spy" || nm === "sinon.stub")
|
|
673
|
+
return true;
|
|
674
|
+
}
|
|
675
|
+
cur = ts.isCallExpression(cur) ? cur.expression : cur.expression;
|
|
676
|
+
}
|
|
677
|
+
return false;
|
|
678
|
+
};
|
|
679
|
+
const walk = (n) => {
|
|
680
|
+
if (ts.isVariableDeclaration(n) && n.initializer && ts.isIdentifier(n.name) &&
|
|
681
|
+
isDoubleInit(n.initializer)) {
|
|
682
|
+
out.add(n.name.text);
|
|
683
|
+
}
|
|
684
|
+
ts.forEachChild(n, walk);
|
|
685
|
+
};
|
|
686
|
+
walk(scope);
|
|
687
|
+
return out;
|
|
688
|
+
}
|
|
689
|
+
/** JS26: a fake-timer install and a setTimeout/setInterval arm both live under
|
|
690
|
+
* `scope`, but no flush/advance does (in scope or a sibling lifecycle hook). The
|
|
691
|
+
* scheduled callback never fires, so any assertion runs against un-mutated state. */
|
|
692
|
+
function timerInstalledNeverAdvanced(scope, timer, sf) {
|
|
693
|
+
// install present in scope, no flush in scope — order does not matter (a frozen
|
|
694
|
+
// timer that is never advanced is vacuous regardless of where the install sits).
|
|
695
|
+
if (!callMatchesUnder(scope, sf, FAKE_TIMER_INSTALL))
|
|
696
|
+
return false;
|
|
697
|
+
if (callMatchesUnder(scope, sf, FAKE_TIMER_FLUSH))
|
|
698
|
+
return false;
|
|
699
|
+
// a sibling hook may install or flush; reuse the JS-wide hook scan. If a hook
|
|
700
|
+
// flushes, the callback fires, so suppress.
|
|
701
|
+
if (hookControlsTimerFlush(timer, sf))
|
|
702
|
+
return false;
|
|
703
|
+
return true;
|
|
704
|
+
}
|
|
705
|
+
/** True if an enclosing describe/top-level afterEach/afterAll flushes timers — the
|
|
706
|
+
* JS26 suppression twin of hookControlsTimer (which also accepts a before-install).
|
|
707
|
+
* Here only a teardown flush matters: an install in a hook does not advance. */
|
|
708
|
+
function hookControlsTimerFlush(timer, sf) {
|
|
709
|
+
const scan = (statements) => {
|
|
710
|
+
for (const stmt of statements) {
|
|
711
|
+
if (!ts.isExpressionStatement(stmt) || !ts.isCallExpression(stmt.expression))
|
|
712
|
+
continue;
|
|
713
|
+
const hook = calleeName(stmt.expression.expression).split(".")[0];
|
|
714
|
+
const cb = getTestCallback(stmt.expression);
|
|
715
|
+
if (!cb)
|
|
716
|
+
continue;
|
|
717
|
+
if (callMatchesUnder(cb, sf, FAKE_TIMER_FLUSH) &&
|
|
718
|
+
(SETUP_HOOKS.has(hook) || TEARDOWN_HOOKS.has(hook)))
|
|
719
|
+
return true;
|
|
720
|
+
}
|
|
721
|
+
return false;
|
|
722
|
+
};
|
|
723
|
+
if (scan(sf.statements))
|
|
724
|
+
return true;
|
|
725
|
+
let pn = timer.parent;
|
|
726
|
+
while (pn) {
|
|
727
|
+
if ((ts.isArrowFunction(pn) || ts.isFunctionExpression(pn) || ts.isFunctionDeclaration(pn)) &&
|
|
728
|
+
pn.parent && ts.isCallExpression(pn.parent) && pn.parent.arguments.includes(pn)) {
|
|
729
|
+
const suiteRoot = calleeName(pn.parent.expression).split(".")[0];
|
|
730
|
+
if (SUITE_ROOTS.has(suiteRoot) && pn.body && ts.isBlock(pn.body) && scan(pn.body.statements))
|
|
731
|
+
return true;
|
|
732
|
+
}
|
|
733
|
+
pn = pn.parent;
|
|
734
|
+
}
|
|
735
|
+
return false;
|
|
736
|
+
}
|
|
516
737
|
function getTestCallback(call) {
|
|
517
738
|
for (const arg of call.arguments) {
|
|
518
739
|
if (ts.isArrowFunction(arg) || ts.isFunctionExpression(arg))
|
|
@@ -665,6 +886,7 @@ export function analyze(sf) {
|
|
|
665
886
|
const findings = [];
|
|
666
887
|
const file = sf.fileName;
|
|
667
888
|
const text = sf.getFullText();
|
|
889
|
+
const level = detectPyramidLevel(sf);
|
|
668
890
|
// Fake-timer / flush presence suppresses the JS7 timer arm: if the test fakes
|
|
669
891
|
// or drives timers anywhere, a setTimeout/setInterval callback is flushed
|
|
670
892
|
// synchronously and its assertion does run. Covers Jest, Vitest and Sinon
|
|
@@ -819,6 +1041,68 @@ export function analyze(sf) {
|
|
|
819
1041
|
}
|
|
820
1042
|
}
|
|
821
1043
|
}
|
|
1044
|
+
// JS25: every assertion lives inside an array-iterator callback
|
|
1045
|
+
// (forEach/map/filter/some/every/flatMap) and none on the test's own
|
|
1046
|
+
// spine — so on an empty collection the callback never runs and zero
|
|
1047
|
+
// assertions execute. The verified hole between C2/C2b (hasAssertion
|
|
1048
|
+
// descends into callbacks, so it finds one) and C21 (its ownAsserts stop
|
|
1049
|
+
// at callbacks, so it sees none). FP guards: any own-scope assertion, a
|
|
1050
|
+
// non-empty array-literal receiver, or an expect.assertions/hasAssertions
|
|
1051
|
+
// guard suppress it.
|
|
1052
|
+
if (hasAssertion(cb) && assertionsOnlyInArrayIterator(cb.body) &&
|
|
1053
|
+
!iteratesNonEmptyArrayLiteral(cb.body) && !hasAssertionCountGuard(cb.body)) {
|
|
1054
|
+
push(line, "JS25", "the only assertion is inside an array-iterator callback; it runs zero times on an empty collection");
|
|
1055
|
+
}
|
|
1056
|
+
// JS27: every expect-chain matcher is in the toHaveBeenCalled* family AND
|
|
1057
|
+
// each such subject root is a locally-created double (jest.fn/vi.fn/spyOn).
|
|
1058
|
+
// The test confirms it called the double it set up, never the unit's output
|
|
1059
|
+
// or state (J3). FP guards: any non-call-tracking assertion suppresses it,
|
|
1060
|
+
// and it is gated to the unit level (a logger-spy call check is legit at
|
|
1061
|
+
// integration/e2e). Sibling of JS8.
|
|
1062
|
+
if (level === "unit") {
|
|
1063
|
+
const ms27 = expectMatcherSubjects(cb.body);
|
|
1064
|
+
if (ms27.length > 0 && ms27.every((m) => CALL_TRACKING_MATCHERS.has(m.matcher))) {
|
|
1065
|
+
const doubles = localDoubleRoots(cb.body);
|
|
1066
|
+
if (ms27.every((m) => m.root !== null && doubles.has(m.root))) {
|
|
1067
|
+
push(line, "JS27", "the only oracle is a toHaveBeenCalled* check on a local double; assert the unit's output or state");
|
|
1068
|
+
}
|
|
1069
|
+
}
|
|
1070
|
+
}
|
|
1071
|
+
// C11a: self-confirming literal — the expected value is bound from the
|
|
1072
|
+
// same call/expression under test. `const e = foo(); expect(foo()).toBe(e)`
|
|
1073
|
+
// can never fail: both sides evaluate the SUT, so the oracle confirms the
|
|
1074
|
+
// code against itself (J2). Static, low-FP corner of the circular-oracle
|
|
1075
|
+
// family. FP guard: the bound initializer must be provably the SUT call,
|
|
1076
|
+
// i.e. its source text equals the expect subject's source text exactly, and
|
|
1077
|
+
// it contains a call (a plain literal binding is not self-confirming).
|
|
1078
|
+
{
|
|
1079
|
+
const inits = new Map(); // var name -> initializer text
|
|
1080
|
+
forEachNoNesting(cb.body, (n) => {
|
|
1081
|
+
if (ts.isVariableDeclaration(n) && ts.isIdentifier(n.name) && n.initializer &&
|
|
1082
|
+
containsCall(n.initializer)) {
|
|
1083
|
+
inits.set(n.name.text, n.initializer.getText(sf).replace(/\s+/g, " ").trim());
|
|
1084
|
+
}
|
|
1085
|
+
});
|
|
1086
|
+
if (inits.size > 0) {
|
|
1087
|
+
forEachNoNesting(cb.body, (n) => {
|
|
1088
|
+
if (!ts.isCallExpression(n))
|
|
1089
|
+
return;
|
|
1090
|
+
const ch = expectChain(n);
|
|
1091
|
+
if (!ch || ch.negated || !ch.subject)
|
|
1092
|
+
return;
|
|
1093
|
+
if (!EQ_MATCHERS_ANY.has(ch.matcher))
|
|
1094
|
+
return;
|
|
1095
|
+
const a0 = ch.args[0];
|
|
1096
|
+
if (!a0 || !ts.isIdentifier(a0))
|
|
1097
|
+
return;
|
|
1098
|
+
const initText = inits.get(a0.text);
|
|
1099
|
+
if (initText && containsCall(ch.subject) &&
|
|
1100
|
+
ch.subject.getText(sf).replace(/\s+/g, " ").trim() === initText) {
|
|
1101
|
+
push(lineOf(sf, n), "C11a", "expected value is bound from the same call under test");
|
|
1102
|
+
}
|
|
1103
|
+
});
|
|
1104
|
+
}
|
|
1105
|
+
}
|
|
822
1106
|
// D7: anonymous test (empty or missing description)
|
|
823
1107
|
const desc = node.arguments[0];
|
|
824
1108
|
const emptyStr = (d) => d !== undefined &&
|
|
@@ -960,8 +1244,9 @@ export function analyze(sf) {
|
|
|
960
1244
|
// C7 self-compare
|
|
961
1245
|
push(lineOf(sf, node), "C7", "expected value is the same expression as the subject");
|
|
962
1246
|
}
|
|
963
|
-
else if (EQUALITY_MATCHERS.has(chain.matcher) && arg && ts.isNumericLiteral(arg)) {
|
|
964
|
-
// C8 exact float
|
|
1247
|
+
else if (EQUALITY_MATCHERS.has(chain.matcher) && arg && ts.isNumericLiteral(arg) && !isLiteral(subj)) {
|
|
1248
|
+
// C8 exact float — on a real (non-literal) subject. A literal-vs-literal
|
|
1249
|
+
// float (expect(0.1).toBe(0.3)) is JS30, the stronger both-literals lane.
|
|
965
1250
|
const v = Number(arg.text);
|
|
966
1251
|
if (!Number.isInteger(v) && v !== 0 && v !== 1) {
|
|
967
1252
|
push(lineOf(sf, node), "C8", "exact equality on a float; use toBeCloseTo");
|
|
@@ -971,6 +1256,25 @@ export function analyze(sf) {
|
|
|
971
1256
|
// C18 sensitive equality: compares the stringified form to a literal
|
|
972
1257
|
push(lineOf(sf, node), "C18", "compares the stringified form of a value to a literal");
|
|
973
1258
|
}
|
|
1259
|
+
// JS30: literal-vs-literal through an equality matcher — both operands are
|
|
1260
|
+
// fixed at parse time, so the comparison is independent of any production
|
|
1261
|
+
// code. Different-token only (the same-token case is C5's "both sides are
|
|
1262
|
+
// the same literal"); object/array literals (reference-equality, false-red)
|
|
1263
|
+
// and template literals with substitutions (C18 lane) are excluded by
|
|
1264
|
+
// isLiteral. Broader matcher set than C5/C7 (adds toBeCloseTo + chai/AVA
|
|
1265
|
+
// equal/equals/eql/is). Negation already filtered (chain.negated).
|
|
1266
|
+
if (EQ_MATCHERS_ANY.has(chain.matcher) && isLiteral(subj) && isLiteral(arg) &&
|
|
1267
|
+
subj.getText(sf) !== arg.getText(sf)) {
|
|
1268
|
+
push(lineOf(sf, node), "JS30", "both operands are literals; the comparison is fixed at parse time");
|
|
1269
|
+
}
|
|
1270
|
+
// C8b: toBeCloseTo called with no precision argument — only the expected
|
|
1271
|
+
// value, so the default 2-digit tolerance applies. The js analogue of
|
|
1272
|
+
// assertAlmostEqual/pytest.approx with no tolerance: implicit-precision only.
|
|
1273
|
+
// A literal-vs-literal toBeCloseTo is JS30's stronger lane, so skip it here.
|
|
1274
|
+
if (chain.matcher === "toBeCloseTo" && chain.args.length === 1 &&
|
|
1275
|
+
!(isLiteral(subj) && isLiteral(arg))) {
|
|
1276
|
+
push(lineOf(sf, node), "C8b", "toBeCloseTo with no precision; the default 2-digit tolerance may be too loose");
|
|
1277
|
+
}
|
|
974
1278
|
}
|
|
975
1279
|
// C16 nondeterminism
|
|
976
1280
|
if (!fakeTimers) {
|
|
@@ -1032,8 +1336,32 @@ export function analyze(sf) {
|
|
|
1032
1336
|
push(lineOf(sf, node), "JS7", `assertion deferred into a floating .${leaf}(); may not run before the test ends`);
|
|
1033
1337
|
}
|
|
1034
1338
|
}
|
|
1339
|
+
// JS26: fake timers installed but never advanced. The setTimeout/setInterval
|
|
1340
|
+
// is armed, fake timers freeze it, and nothing in the same test scope (nor a
|
|
1341
|
+
// sibling before/after hook) calls runAllTimers/advanceTimersByTime/tick — so
|
|
1342
|
+
// the scheduled callback never fires and any assertion runs against the
|
|
1343
|
+
// un-mutated initial state. Opposite of C16 (uncontrolled timer): here the
|
|
1344
|
+
// timer is controlled but not advanced. Requires an assertion in scope so a
|
|
1345
|
+
// pure setup arm with no oracle is not flagged. Kept low (legit "assert
|
|
1346
|
+
// nothing happened yet" exists). FP guard: any flush in the body or a
|
|
1347
|
+
// before/afterEach of the enclosing describe suppresses it.
|
|
1348
|
+
if (isTimer) {
|
|
1349
|
+
const scope = enclosingTestScope(node);
|
|
1350
|
+
if (timerInstalledNeverAdvanced(scope, node, sf) && hasAssertion(scope)) {
|
|
1351
|
+
push(lineOf(sf, node), "JS26", `${name} scheduled under fake timers that are never advanced; the callback never fires`);
|
|
1352
|
+
}
|
|
1353
|
+
}
|
|
1035
1354
|
}
|
|
1036
1355
|
}
|
|
1356
|
+
// JS29: an expect(...).resolves/.rejects chain that is a bare statement, not
|
|
1357
|
+
// awaited, returned, or collected. The matcher only settles asynchronously, so
|
|
1358
|
+
// a floating chain finishes green before it resolves. The statically-provable
|
|
1359
|
+
// subset of JS20 (no type inference needed: the explicit .resolves/.rejects
|
|
1360
|
+
// marker is the signal). FP guard: only the explicit resolves/rejects member
|
|
1361
|
+
// (a plain promise stays JS20-out); awaited/returned/collected suppresses it.
|
|
1362
|
+
if (expectIsResolvesRejects(node) && !isObservedAsync(node)) {
|
|
1363
|
+
push(lineOf(sf, node), "JS29", "resolves/rejects assertion is not awaited or returned; it settles after the test ends");
|
|
1364
|
+
}
|
|
1037
1365
|
// JS13: a sync query used as a loose statement (result never asserted).
|
|
1038
1366
|
// Testing Library getBy*/queryBy*, and Vue Test Utils findComponent /
|
|
1039
1367
|
// findAllComponents always, or find/findAll with a string selector (which
|
|
@@ -1109,6 +1437,19 @@ export function analyze(sf) {
|
|
|
1109
1437
|
if (containsAssertion(node.tryBlock) && isHarmlessCatch(node.catchClause.block)) {
|
|
1110
1438
|
push(lineOf(sf, node), "JS11", "a failing assertion in try is swallowed by catch");
|
|
1111
1439
|
}
|
|
1440
|
+
else if (
|
|
1441
|
+
// JS31: the try calls production code that may throw, the catch neither
|
|
1442
|
+
// asserts on the exception, re-raises, nor calls fail() — so a unit that
|
|
1443
|
+
// STOPS throwing (a real regression) still passes green. Complement of JS11
|
|
1444
|
+
// (which owns the swallowed-assertion case): JS31 fires only when the try
|
|
1445
|
+
// has a call but NO assertion (otherwise JS11), and the catch is harmless.
|
|
1446
|
+
// FP guard: a catch that asserts on e / re-throws / calls fail() makes
|
|
1447
|
+
// isHarmlessCatch false; a toThrow/assert.throws oracle in the try would be
|
|
1448
|
+
// an assertion (so JS31 is skipped, the throw is covered).
|
|
1449
|
+
containsCall(node.tryBlock) &&
|
|
1450
|
+
catchSilentlySwallows(node.catchClause.block)) {
|
|
1451
|
+
push(lineOf(sf, node), "JS31", "try/catch swallows a possible throw with no assertion on the exception");
|
|
1452
|
+
}
|
|
1112
1453
|
}
|
|
1113
1454
|
// JS21: a matcher referenced but never called — `expect(x).toBe;` with no (),
|
|
1114
1455
|
// so the assertion object is built and dropped; nothing executes. The chain
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "falsegreen-js",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.6.0",
|
|
4
4
|
"description": "Find JavaScript/TypeScript unit tests that give false positives: green tests that protect nothing, and tests that pass while asserting the wrong thing. Deterministic AST scanner, sibling of falsegreen (Python).",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"jest",
|