eslint-plugin-test-governance 0.2.1 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,47 @@ All notable changes to this project are documented here. The format follows
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.3.1] - 2026-10-06
10
+
11
+ ### Fixed
12
+
13
+ - A `test` bound to another test runner without an import, as in `const test = require('node:test')`,
14
+ `const test = require('ava')` or `const test = anyTest as TestFn<Context>`, is no longer mistaken
15
+ for Playwright's `test`. Before, skipped tests in those files were reported as missing a ticket.
16
+ - `require-ticket-in-comments` reads a ticket in parentheses after a space, as in `// TODO (WEB-123)`.
17
+ Before, such comments were reported as having no ticket at all.
18
+ - A test or describe whose body is not written inline, such as `test.skip('pays', payWithCard)` or
19
+ `test('pays @new', withPage(async () => {}))`, is checked like any other. Before, it was ignored,
20
+ so a skipped test written this way needed no ticket.
21
+
22
+ ## [0.3.0] - 2026-10-06
23
+
24
+ ### Added
25
+
26
+ - `require-ticket-in-comments` accepts the ticket in parentheses: `// TODO(WEB-123)`. A name in
27
+ parentheses, as in `// TODO(alice): refactor`, is not a ticket and is reported. Before, comments
28
+ written this way were never checked.
29
+ - Configuration mistakes that used to be accepted silently now fail with a clear message: lowercase
30
+ Jira `projects` or Linear `teams` (`'web'` never matches a ticket like `WEB-1`), options that belong
31
+ to another ticket preset (such as `host` on the `any` preset), and a custom state that reuses a
32
+ built-in state's name (`customStates: { fixme: ... }`).
33
+
34
+ ### Fixed
35
+
36
+ - A skip in a `catch` block (`catch { test.skip(true, 'Mail server is down') }`) or after an early
37
+ exit such as `if (ready) return;` counts as conditional, like one under an `if`, so it no longer
38
+ needs a ticket by default.
39
+ - A plain `// FIXME: some text` above `test.skip(...)` no longer gets a confusing `marker-matches-state`
40
+ error claiming the test is also in the `fixme` state. `require-ticket-in-comments` and
41
+ `require-ticket` report what is actually wrong. The `marker-matches-state` message for a test in
42
+ two states now starts its second sentence with a capital letter.
43
+ - `@stable`, `@untestable` and other tags two or more letters longer or shorter than `@unstable`
44
+ (or any other state tag) are no longer reported as typos of it.
45
+ - A broken marker next to a valid one for the same state, as in `// SKIP: WEB-1` followed by
46
+ `// SKIP: nope`, is reported. Before, the valid marker hid it.
47
+ - A full stop or colon after the last ticket, as in `// SKIP: WEB-1.`, is no longer reported as
48
+ extra text.
49
+
9
50
  ## [0.2.1] - 2026-10-03
10
51
 
11
52
  ### Added
@@ -93,7 +134,9 @@ First release.
93
134
  - Support for ESLint 9 and 10 flat config, JavaScript and TypeScript test files, `.extend()`
94
135
  fixtures, describe blocks, runtime skips and tags in titles or the `tag` option.
95
136
 
96
- [Unreleased]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.2.1...HEAD
137
+ [Unreleased]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.3.1...HEAD
138
+ [0.3.1]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.3.0...v0.3.1
139
+ [0.3.0]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.2.1...v0.3.0
97
140
  [0.2.1]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.2.0...v0.2.1
98
141
  [0.2.0]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.1.1...v0.2.0
99
142
  [0.1.1]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.1.0...v0.1.1
package/README.md CHANGED
@@ -47,7 +47,7 @@ Any flat config file works. The package is ESM, and `require('eslint-plugin-test
47
47
  [`jiti`](https://github.com/unjs/jiti), so install it as a dev dependency too.
48
48
 
49
49
  To change the defaults, use `configure()`. It returns the same recommended config with your options
50
- stored in `settings['test-governance']`, which all four rules read:
50
+ stored in `settings['test-governance']`, which all the rules read:
51
51
 
52
52
  ```js
53
53
  export default [
@@ -100,20 +100,20 @@ eslint-plugin-test-governance: invalid options:
100
100
 
101
101
  The defaults follow how most teams already work, so `recommended` is useful with no options.
102
102
 
103
- | Setting | Default | Why |
104
- | -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
105
- | States that need a ticket | `skip`, `fixme` | Every team skips tests. Chromium, GitLab and Slack all require a bug link on disabled tests. |
106
- | `@new`, `@unstable` | off; `lifecycleTags: true` turns both on | A promotion workflow is not universal. Teams that call it `@flaky` or `@quarantine` use `customStates`. |
107
- | Conditional skips (`test.skip(browserName === 'webkit', ...)`, `if (...) test.skip()`) | no ticket needed | These are usually permanent platform limits, not bugs. `requireTicketForConditional: true` changes this. |
108
- | `test.fail`, `test.slow` | off | They don't remove coverage. Turn them on with `states: { fail: true, slow: true }`. |
109
- | Marker format | `// SKIP: WEB-123` or `// SKIP: WEB-123, WEB-124` | Uppercase keyword and colon, like `TODO:`. Easy to grep. |
110
- | Notes after the ticket | not allowed; `allowNotes: true` allows them | The ticket is the source of truth. A note in a comment goes stale while the ticket stays current. |
111
- | `FIXME` / `TODO` comments | must start with a ticket, e.g. `// TODO: WEB-123` | The ticket is the source of truth; an untracked `TODO` is never done. `comments: false` turns this off. |
112
- | Placement | comment block directly above; other comments allowed; a blank line breaks it | Survives `eslint-disable-next-line` and Prettier. |
113
- | Ticket format | `any`: `PROJ-123`, `#123`, `owner/repo#123`, or an http(s) URL | Works for Jira, Linear, GitHub and GitLab out of the box. Narrow it with a preset. |
114
- | Placeholder tickets | rejected: `TODO`, `TBD`, `XXX-*`, `0`, `123`, `1234`, `12345`, and any ticket numbered 0 (`PROJ-0`, `#0`) | A fake ticket is worse than none, and no tracker issues number 0. |
115
- | Describe blocks | a marker on a skipped or tagged describe covers its tests | One ticket per skipped suite. |
116
- | Severity | `require-ticket`, `marker-matches-state`, `no-orphaned-marker`: error; `no-conflicting-states`: warn | A marker must always match a real state, so a missing, wrong or leftover marker fails the lint. Tag spelling is cleanup. |
103
+ | Setting | Default | Why |
104
+ | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
105
+ | States that need a ticket | `skip`, `fixme` | Every team skips tests. Chromium, GitLab and Slack all require a bug link on disabled tests. |
106
+ | `@new`, `@unstable` | off; `lifecycleTags: true` turns both on | A promotion workflow is not universal. Teams that call it `@flaky` or `@quarantine` use `customStates`. |
107
+ | Conditional skips (`test.skip(browserName === 'webkit', ...)`, `if (...) test.skip()`, a skip in `catch`) | no ticket needed | These are usually permanent platform limits, not bugs. `requireTicketForConditional: true` changes this. |
108
+ | `test.fail`, `test.slow` | off | They don't remove coverage. Turn them on with `states: { fail: true, slow: true }`. |
109
+ | Marker format | `// SKIP: WEB-123` or `// SKIP: WEB-123, WEB-124` | Uppercase keyword and colon, like `TODO:`. Easy to grep. |
110
+ | Notes after the ticket | not allowed; `allowNotes: true` allows them | The ticket is the source of truth. A note in a comment goes stale while the ticket stays current. |
111
+ | `FIXME` / `TODO` comments | must start with a ticket, e.g. `// TODO: WEB-123` or `// TODO(WEB-123)` | The ticket is the source of truth; an untracked `TODO` is never done. `comments: false` turns this off. |
112
+ | Placement | comment block directly above; other comments allowed; a blank line breaks it | Survives `eslint-disable-next-line` and Prettier. |
113
+ | Ticket format | `any`: `PROJ-123`, `#123`, `owner/repo#123`, or an http(s) URL | Works for Jira, Linear, GitHub and GitLab out of the box. Narrow it with a preset. |
114
+ | Placeholder tickets | rejected: `TODO`, `TBD`, `XXX-*`, `0`, `123`, `1234`, `12345`, and any ticket numbered 0 (`PROJ-0`, `#0`) | A fake ticket is worse than none, and no tracker issues number 0. |
115
+ | Describe blocks | a marker on a skipped or tagged describe covers its tests | One ticket per skipped suite. |
116
+ | Severity | `require-ticket`, `marker-matches-state`, `no-orphaned-marker`: error; `no-conflicting-states`: warn | A marker must always match a real state, so a missing, wrong or leftover marker fails the lint. Tag spelling is cleanup. |
117
117
 
118
118
  ## Options
119
119
 
@@ -172,8 +172,9 @@ so keep it free of nested quantifiers.
172
172
  - Any name in `testFunctions`, for fixtures imported from your own modules. The default `['test']`
173
173
  covers the common `import { test } from './fixtures'`.
174
174
 
175
- A name imported from another test runner (`vitest`, `@jest/globals`, `node:test`, `bun:test`,
176
- `mocha`, `ava`, `tap`, `uvu`) is never treated as Playwright's `test`, and neither is a local
175
+ A name imported or required from another test runner (`vitest`, `@jest/globals`, `node:test`,
176
+ `bun:test`, `mocha`, `ava`, `tap`, `uvu`), or a variable set from one such as
177
+ `const test = require('ava')`, is never treated as Playwright's `test`, and neither is a local
177
178
  variable declared inside a function that happens to be called `test`.
178
179
 
179
180
  ### Syntax the rules understand
package/dist/index.d.ts CHANGED
@@ -1,7 +1,7 @@
1
1
  import type { ESLint, Rule } from 'eslint';
2
- import { type GovernanceOptions } from './utils/options.js';
3
- export type { CustomState, GovernanceOptions, StateOverride } from './utils/options.js';
4
- export type { TicketSpec } from './utils/tickets.js';
2
+ import { type GovernanceOptions } from './utils/options.ts';
3
+ export type { CustomState, GovernanceOptions, StateOverride } from './utils/options.ts';
4
+ export type { TicketSpec } from './utils/tickets.ts';
5
5
  export type RuleName = 'require-ticket' | 'marker-matches-state' | 'no-orphaned-marker' | 'no-conflicting-states' | 'require-ticket-in-comments';
6
6
  export declare const rules: Record<RuleName, Rule.RuleModule>;
7
7
  export interface FlatConfig {
package/dist/index.js CHANGED
@@ -1,10 +1,10 @@
1
1
  import { createRequire } from 'node:module';
2
- import markerMatchesState from './rules/marker-matches-state.js';
3
- import noConflictingStates from './rules/no-conflicting-states.js';
4
- import noOrphanedMarker from './rules/no-orphaned-marker.js';
5
- import requireTicketInComments from './rules/require-ticket-in-comments.js';
6
- import requireTicket from './rules/require-ticket.js';
7
- import { compileOptions, SETTINGS_KEY } from './utils/options.js';
2
+ import markerMatchesState from "./rules/marker-matches-state.js";
3
+ import noConflictingStates from "./rules/no-conflicting-states.js";
4
+ import noOrphanedMarker from "./rules/no-orphaned-marker.js";
5
+ import requireTicketInComments from "./rules/require-ticket-in-comments.js";
6
+ import requireTicket from "./rules/require-ticket.js";
7
+ import { compileOptions, SETTINGS_KEY } from "./utils/options.js";
8
8
  const PLUGIN_NAME = 'test-governance';
9
9
  // Read at load time so the reported version always matches the published package.
10
10
  const { version: VERSION } = createRequire(import.meta.url)('../package.json');
@@ -1,3 +1,3 @@
1
- declare const _default: import("../utils/create-rule.js").Rule<"wrongMarker" | "sharedMarker" | "renameMarker" | "addMarker">;
1
+ declare const _default: import("../utils/create-rule.ts").Rule<"wrongMarker" | "sharedMarker" | "renameMarker" | "addMarker">;
2
2
  export default _default;
3
3
  //# sourceMappingURL=marker-matches-state.d.ts.map
@@ -1,6 +1,6 @@
1
- import { describeSubject, isStateMarker, missingStates, strayMarkers } from '../utils/analyze.js';
2
- import { createRule } from '../utils/create-rule.js';
3
- import { indentOf, keywordRange } from '../utils/fix.js';
1
+ import { appliesTo, describeSubject, isStateMarker, missingStates, strayMarkers } from "../utils/analyze.js";
2
+ import { createRule } from "../utils/create-rule.js";
3
+ import { indentOf, keywordRange } from "../utils/fix.js";
4
4
  export default createRule({
5
5
  name: 'marker-matches-state',
6
6
  meta: {
@@ -24,12 +24,17 @@ export default createRule({
24
24
  if (missing.length === 0)
25
25
  continue;
26
26
  const stray = strayMarkers(subject, options);
27
- const subjectName = describeSubject(subject).replace(/^This/, 'this');
27
+ const subjectName = describeSubject(subject);
28
28
  for (const marker of stray) {
29
29
  context.report({
30
30
  loc: marker.comment.loc,
31
31
  messageId: 'wrongMarker',
32
- data: { found: marker.keyword, expected: missing[0].marker, state: missing[0].name, subject: subjectName },
32
+ data: {
33
+ found: marker.keyword,
34
+ expected: missing[0].marker,
35
+ state: missing[0].name,
36
+ subject: subjectName.replace(/^This/, 'this'),
37
+ },
33
38
  suggest: missing.map((state) => ({
34
39
  messageId: 'renameMarker',
35
40
  data: { expected: state.marker },
@@ -39,8 +44,12 @@ export default createRule({
39
44
  }
40
45
  if (stray.length > 0)
41
46
  continue;
42
- // One marker written for a test that is in two states.
43
- const present = subject.markers.filter(isStateMarker).find((m) => m.state !== missing[0]);
47
+ // One marker written for a test that is in two states. A marker for a state the test isn't in
48
+ // is not "shared": with no stray markers left, it can only be a work comment such as
49
+ // `// FIXME: flaky` above `test.skip(...)`, which require-ticket-in-comments reports.
50
+ const present = subject.markers
51
+ .filter(isStateMarker)
52
+ .find((m) => m.state !== missing[0] && appliesTo(subject, m.state.name));
44
53
  if (!present)
45
54
  continue;
46
55
  for (const state of missing) {
@@ -1,3 +1,3 @@
1
- declare const _default: import("../utils/create-rule.js").Rule<"newAndUnstable" | "newSkipped" | "tagCase" | "tagTypo">;
1
+ declare const _default: import("../utils/create-rule.ts").Rule<"newAndUnstable" | "newSkipped" | "tagCase" | "tagTypo">;
2
2
  export default _default;
3
3
  //# sourceMappingURL=no-conflicting-states.d.ts.map
@@ -1,8 +1,8 @@
1
- import { headLoc } from '../utils/analyze.js';
2
- import { createRule } from '../utils/create-rule.js';
3
- import { isTagState } from '../utils/options.js';
4
- import { editDistance, typoThreshold } from '../utils/distance.js';
5
- import { tagRange } from '../utils/fix.js';
1
+ import { headLoc } from "../utils/analyze.js";
2
+ import { createRule } from "../utils/create-rule.js";
3
+ import { isTagState } from "../utils/options.js";
4
+ import { editDistance, typoThreshold } from "../utils/distance.js";
5
+ import { tagRange } from "../utils/fix.js";
6
6
  export default createRule({
7
7
  name: 'no-conflicting-states',
8
8
  meta: {
@@ -71,9 +71,12 @@ export default createRule({
71
71
  });
72
72
  return;
73
73
  }
74
+ // A typo changes, adds, drops or swaps a letter or two. A tag that is two or more letters
75
+ // shorter or longer, like `@stable` or `@untestable` next to `@unstable`, is a different word.
74
76
  const near = tagStates.find((s) => {
75
77
  const expected = s.tag.toLowerCase();
76
- return editDistance(lower, expected) <= typoThreshold(expected.length - 1);
78
+ return (Math.abs(lower.length - expected.length) < 2 &&
79
+ editDistance(lower, expected) <= typoThreshold(expected.length - 1));
77
80
  });
78
81
  if (near) {
79
82
  context.report({ node: occurrence.node, messageId: 'tagTypo', data: { found: tag, expected: near.tag } });
@@ -1,3 +1,3 @@
1
- declare const _default: import("../utils/create-rule.js").Rule<"orphaned" | "detached" | "removeMarker">;
1
+ declare const _default: import("../utils/create-rule.ts").Rule<"orphaned" | "detached" | "removeMarker">;
2
2
  export default _default;
3
3
  //# sourceMappingURL=no-orphaned-marker.d.ts.map
@@ -1,6 +1,6 @@
1
- import { describeSubject, isProse, missingStates, strayMarkers } from '../utils/analyze.js';
2
- import { createRule } from '../utils/create-rule.js';
3
- import { removeComment } from '../utils/fix.js';
1
+ import { describeSubject, isProse, missingStates, strayMarkers } from "../utils/analyze.js";
2
+ import { createRule } from "../utils/create-rule.js";
3
+ import { removeComment } from "../utils/fix.js";
4
4
  export default createRule({
5
5
  name: 'no-orphaned-marker',
6
6
  meta: {
@@ -1,3 +1,3 @@
1
- declare const _default: import("../utils/create-rule.js").Rule<"missingTicket" | "invalidTicket" | "placeholderTicket" | "extraText" | "removeExtraText">;
1
+ declare const _default: import("../utils/create-rule.ts").Rule<"missingTicket" | "invalidTicket" | "placeholderTicket" | "extraText" | "removeExtraText">;
2
2
  export default _default;
3
3
  //# sourceMappingURL=require-ticket-in-comments.d.ts.map
@@ -1,6 +1,6 @@
1
- import { appliesTo, isStateMarker, parseTickets } from '../utils/analyze.js';
2
- import { createRule } from '../utils/create-rule.js';
3
- import { compileTicketSpec } from '../utils/tickets.js';
1
+ import { appliesTo, isStateMarker, parseTickets } from "../utils/analyze.js";
2
+ import { createRule } from "../utils/create-rule.js";
3
+ import { compileTicketSpec } from "../utils/tickets.js";
4
4
  /** Recognises anything shaped like a ticket, to tell a wrong-format ticket from plain prose. */
5
5
  const ticketShape = compileTicketSpec([{ preset: 'any' }], []);
6
6
  function escapeRegExp(text) {
@@ -38,9 +38,26 @@ export default createRule({
38
38
  }
39
39
  }
40
40
  // `TODO`/`FIXME` in capitals counts with or without a colon. Other casings need the colon, so a
41
- // sentence that happens to start with "Todo" is not mistaken for a work comment.
41
+ // sentence that happens to start with "Todo" is not mistaken for a work comment. The keyword can
42
+ // carry its ticket in parentheses, `TODO(WEB-123): ...`, a common style that tools also use for
43
+ // a name, `TODO(alice)`, which is not a ticket. Some teams put a space before the parenthesis.
42
44
  const alternatives = keywords.map(escapeRegExp).join('|');
43
- const keywordRe = new RegExp(`^(\\s*\\*?\\s*)(${alternatives})(?:\\s*:|(?=\\s|$))`, 'i');
45
+ const keywordRe = new RegExp(`^(\\s*\\*?\\s*)(${alternatives})(?:\\s*\\(([^)]*)\\))?(?:\\s*:|(?=\\s|$))`, 'i');
46
+ /**
47
+ * `TODO(WEB-1, WEB-2): text`: the tickets are in the parentheses, and anything after them, from the
48
+ * closing parenthesis on, is a note.
49
+ */
50
+ function parseParenthesized(line, start, open, inParens) {
51
+ const parsed = parseTickets(inParens, start + open + 1, matcher);
52
+ const close = open + inParens.length + 2;
53
+ const note = line
54
+ .slice(close)
55
+ .replace(/^\s*:?/, '')
56
+ .trim();
57
+ if (note === '' || parsed.extra)
58
+ return parsed;
59
+ return { ...parsed, extra: { text: note, range: [start + close, start + line.trimEnd().length] } };
60
+ }
44
61
  for (const comment of sourceCode.getAllComments()) {
45
62
  if (owned.has(comment))
46
63
  continue;
@@ -54,10 +71,14 @@ export default createRule({
54
71
  if (!match)
55
72
  continue;
56
73
  const [whole, indent, keyword] = match;
74
+ // The parenthesised group is optional, so it can be undefined despite RegExpExecArray's typing.
75
+ const inParens = match[3];
57
76
  const upper = keyword.toUpperCase();
58
77
  if (keyword !== upper && !whole.endsWith(':'))
59
78
  continue;
60
- const { tickets, extra } = parseTickets(line.slice(whole.length), start + whole.length, matcher);
79
+ const { tickets, extra } = inParens === undefined
80
+ ? parseTickets(line.slice(whole.length), start + whole.length, matcher)
81
+ : parseParenthesized(line, start, whole.indexOf('(', indent.length + keyword.length), inParens);
61
82
  const loc = {
62
83
  start: sourceCode.getLocFromIndex(start + indent.length),
63
84
  end: sourceCode.getLocFromIndex(start + line.trimEnd().length),
@@ -1,3 +1,3 @@
1
- declare const _default: import("../utils/create-rule.js").Rule<"missingTicket" | "invalidTicket" | "placeholderTicket" | "extraText" | "removeExtraText" | "missingMarker" | "markerCase" | "dynamicTitle">;
1
+ declare const _default: import("../utils/create-rule.ts").Rule<"missingTicket" | "invalidTicket" | "placeholderTicket" | "extraText" | "removeExtraText" | "missingMarker" | "markerCase" | "dynamicTitle">;
2
2
  export default _default;
3
3
  //# sourceMappingURL=require-ticket.d.ts.map
@@ -1,5 +1,5 @@
1
- import { appliesTo, describeSubject, evaluate, headLoc } from '../utils/analyze.js';
2
- import { createRule } from '../utils/create-rule.js';
1
+ import { appliesTo, describeSubject, evaluate, headLoc } from "../utils/analyze.js";
2
+ import { createRule } from "../utils/create-rule.js";
3
3
  export default createRule({
4
4
  name: 'require-ticket',
5
5
  meta: {
@@ -1,6 +1,6 @@
1
1
  import type { TSESLint, TSESTree } from '@typescript-eslint/utils';
2
- import type { ResolvedOptions, StateDef } from './options.js';
3
- import type { TicketCheck, TicketMatcher } from './tickets.js';
2
+ import type { ResolvedOptions, StateDef } from './options.ts';
3
+ import type { TicketCheck, TicketMatcher } from './tickets.ts';
4
4
  type Node = TSESTree.Node;
5
5
  type Comment = TSESTree.Comment;
6
6
  type SourceCode = Readonly<TSESLint.SourceCode>;
@@ -49,6 +49,8 @@ function isFunction(node) {
49
49
  function collectTestNames(program, configured) {
50
50
  const names = new Set(configured);
51
51
  const mergeTests = new Set();
52
+ /** Names bound to another test runner, such as `ava` in `import ava from 'ava'`. */
53
+ const others = new Set();
52
54
  const isTestExpression = (node) => {
53
55
  const target = unwrap(node);
54
56
  if (target.type === 'Identifier')
@@ -62,12 +64,24 @@ function collectTestNames(program, configured) {
62
64
  return false;
63
65
  return memberName(target.callee) === 'extend' && isTestExpression(target.callee.object);
64
66
  };
67
+ // require('node:test'), ava, ava.serial or `ava as TestFn<Context>`
68
+ const isOtherFramework = (node) => {
69
+ const target = unwrap(node);
70
+ const source = requireSource(target);
71
+ if (source !== undefined)
72
+ return OTHER_FRAMEWORKS.has(source);
73
+ if (target.type === 'Identifier')
74
+ return others.has(target.name);
75
+ return target.type === 'MemberExpression' && isOtherFramework(target.object);
76
+ };
65
77
  for (const statement of program.body) {
66
78
  if (statement.type === 'ImportDeclaration') {
67
79
  const source = statement.source.value;
68
80
  if (OTHER_FRAMEWORKS.has(source)) {
69
- for (const spec of statement.specifiers)
81
+ for (const spec of statement.specifiers) {
70
82
  names.delete(spec.local.name);
83
+ others.add(spec.local.name);
84
+ }
71
85
  }
72
86
  else if (isPlaywrightModule(source)) {
73
87
  for (const spec of statement.specifiers) {
@@ -91,6 +105,11 @@ function collectTestNames(program, configured) {
91
105
  if (declarator.id.type === 'Identifier' && init.type === 'CallExpression' && isTestExpression(init)) {
92
106
  names.add(declarator.id.name);
93
107
  }
108
+ else if (declarator.id.type === 'Identifier' && isOtherFramework(init)) {
109
+ // `const test = require('node:test')` is another runner's `test`, like an import from it.
110
+ names.delete(declarator.id.name);
111
+ others.add(declarator.id.name);
112
+ }
94
113
  else if (declarator.id.type === 'ObjectPattern') {
95
114
  const source = requireSource(init);
96
115
  if (source === undefined)
@@ -98,8 +117,10 @@ function collectTestNames(program, configured) {
98
117
  for (const prop of declarator.id.properties) {
99
118
  if (prop.type !== 'Property' || prop.key.type !== 'Identifier' || prop.value.type !== 'Identifier')
100
119
  continue;
101
- if (OTHER_FRAMEWORKS.has(source))
120
+ if (OTHER_FRAMEWORKS.has(source)) {
102
121
  names.delete(prop.value.name);
122
+ others.add(prop.value.name);
123
+ }
103
124
  else if (isPlaywrightModule(source) && prop.key.name === 'test')
104
125
  names.add(prop.value.name);
105
126
  }
@@ -183,7 +204,8 @@ export function parseTickets(text, offset, matcher) {
183
204
  index += separator.length;
184
205
  }
185
206
  const rest = text.slice(lastEnd).trimEnd();
186
- if (tickets.length === 0 || rest.trim() === '')
207
+ // A sentence-ending `WEB-1.` has nothing after the ticket; `WEB-1: flaky` has a note.
208
+ if (tickets.length === 0 || /^[.:;]*$/.test(rest.trim()))
187
209
  return { tickets };
188
210
  return { tickets, extra: { text: rest.trim(), range: [offset + lastEnd, offset + lastEnd + rest.length] } };
189
211
  }
@@ -253,10 +275,22 @@ function isUnconditional(call) {
253
275
  return (first === undefined || (first.type === 'Literal' && first.value === true)) && !isGuarded(call);
254
276
  }
255
277
  const FUNCTIONS = new Set(['FunctionDeclaration', 'FunctionExpression', 'ArrowFunctionExpression']);
278
+ /** Whether a statement always leaves the function or block: `return`, `throw`, or a block containing one. */
279
+ function exits(statement) {
280
+ if (statement.type === 'ReturnStatement' || statement.type === 'ThrowStatement')
281
+ return true;
282
+ return statement.type === 'BlockStatement' && statement.body.some(exits);
283
+ }
284
+ /** `if (ready) return;` or `if (!ok) { throw error; }`: code after it only runs on some paths. */
285
+ function isEarlyExit(statement) {
286
+ return (statement.type === 'IfStatement' &&
287
+ (exits(statement.consequent) || (statement.alternate !== null && exits(statement.alternate))));
288
+ }
256
289
  /**
257
- * Whether the call only runs on some paths of its function: under an `if`, a `switch` case, a ternary
258
- * or the right side of `&&`, `||` or `??`. `if (!enabled) test.skip()` is the same as
259
- * `test.skip(!enabled)`.
290
+ * Whether the call only runs on some paths of its function: under an `if`, a `switch` case, a ternary,
291
+ * the right side of `&&`, `||` or `??`, in a `catch` block, or after an early exit such as
292
+ * `if (ready) return;`. `if (!enabled) test.skip()` is the same as `test.skip(!enabled)`, and
293
+ * `catch { test.skip(true, 'service is down') }` only skips when the service is down.
260
294
  */
261
295
  function isGuarded(call) {
262
296
  let child = call;
@@ -267,6 +301,13 @@ function isGuarded(call) {
267
301
  return true;
268
302
  if (node.type === 'LogicalExpression' && child === node.right)
269
303
  return true;
304
+ if (node.type === 'CatchClause')
305
+ return true;
306
+ if (node.type === 'BlockStatement') {
307
+ const index = node.body.indexOf(child);
308
+ if (node.body.slice(0, index).some(isEarlyExit))
309
+ return true;
310
+ }
270
311
  }
271
312
  return false;
272
313
  }
@@ -354,11 +395,16 @@ function runAnalysis(sourceCode, options) {
354
395
  if (param?.type === 'Identifier')
355
396
  testInfo = param.name;
356
397
  }
357
- const isDeclaration = args.length >= 2 && isFunction(last) && !isFunction(args[0]);
398
+ // The body is usually written inline, but can be a function defined elsewhere:
399
+ // `test.skip('pays', payWithCard)`. A string title tells that apart from `test.skip(cond, 'why')`.
400
+ const titled = args.length >= 2 && isStaticText(args[0]);
401
+ const isDeclaration = args.length >= 2 && !isFunction(args[0]) && (isFunction(last) || titled);
358
402
  if (path.every((p) => TEST_MODIFIERS.has(p)) && isDeclaration) {
359
403
  return { subject: makeSubject('test', call, path, collectTags(call, true)), testInfo };
360
404
  }
361
- if (path[0] === 'describe' && path.slice(1).every((p) => DESCRIBE_MODIFIERS.has(p)) && isFunction(last)) {
405
+ if (path[0] === 'describe' &&
406
+ path.slice(1).every((p) => DESCRIBE_MODIFIERS.has(p)) &&
407
+ (isFunction(last) || titled)) {
362
408
  const tagInfo = collectTags(call, args.length >= 2);
363
409
  return { subject: makeSubject('describe', call, path.slice(1), tagInfo), testInfo };
364
410
  }
@@ -413,7 +459,7 @@ function runAnalysis(sourceCode, options) {
413
459
  }
414
460
  }
415
461
  // Parsers provide visitor keys for every node type they produce; the fallback is only defensive.
416
- /* v8 ignore next */
462
+ /* node:coverage ignore next */
417
463
  for (const key of keys[node.type] ?? []) {
418
464
  const child = node[key];
419
465
  if (Array.isArray(child)) {
@@ -463,10 +509,15 @@ export function evaluate(subject, state) {
463
509
  const marker = subject.markers.find((m) => m.caseOf === state);
464
510
  return marker ? { kind: 'case', marker } : { kind: 'missing' };
465
511
  }
466
- if (markers.some((m) => m.tickets.length > 0 && m.tickets.every((t) => t.result === 'ok')))
512
+ const isValid = (m) => m.tickets.length > 0 && m.tickets.every((t) => t.result === 'ok');
513
+ // A broken marker in the subject's own block is reported even when another marker is valid: in
514
+ // `// SKIP: WEB-1` + `// SKIP: nope`, the second line is still wrong. Broken markers further up
515
+ // are reported for the test or describe they sit above.
516
+ const brokenOwn = markers.find((m) => subject.markers.includes(m) && !isValid(m));
517
+ if (!brokenOwn && markers.some(isValid))
467
518
  return { kind: 'ok' };
468
- // Report the problem with the closest marker.
469
- const [marker] = markers;
519
+ // Report the problem with the closest broken marker.
520
+ const marker = brokenOwn ?? markers[0];
470
521
  const bad = marker.tickets.find((t) => t.result !== 'ok');
471
522
  return bad ? { kind: 'bad-ticket', marker, ticket: bad } : { kind: 'no-ticket', marker };
472
523
  }
@@ -1,6 +1,6 @@
1
1
  import type { TSESLint } from '@typescript-eslint/utils';
2
- import { type Analysis } from './analyze.js';
3
- import { type ResolvedOptions } from './options.js';
2
+ import { type Analysis } from './analyze.ts';
3
+ import { type ResolvedOptions } from './options.ts';
4
4
  export declare const REPO_URL = "https://github.com/Huldoser/eslint-plugin-test-governance";
5
5
  /** Rules take no options of their own: everything is shared through `settings['test-governance']`. */
6
6
  export type RuleOptions = [];
@@ -1,5 +1,5 @@
1
- import { analyze } from './analyze.js';
2
- import { resolveOptions } from './options.js';
1
+ import { analyze } from "./analyze.js";
2
+ import { resolveOptions } from "./options.js";
3
3
  export const REPO_URL = 'https://github.com/Huldoser/eslint-plugin-test-governance';
4
4
  /**
5
5
  * Builds a rule that reads its options from `settings['test-governance']`.
@@ -1,5 +1,5 @@
1
- import { type BuiltinStateName } from './constants.js';
2
- import { type TicketMatcher, type TicketSpec } from './tickets.js';
1
+ import { type BuiltinStateName } from './constants.ts';
2
+ import { type TicketMatcher, type TicketSpec } from './tickets.ts';
3
3
  export declare const SETTINGS_KEY = "test-governance";
4
4
  /** A state override in `states`. `false` turns a state off, `true` turns it on. */
5
5
  export interface StateOverride {
@@ -14,7 +14,7 @@ export interface CustomState {
14
14
  marker: string;
15
15
  ticket?: TicketSpec | TicketSpec[];
16
16
  }
17
- export type { BuiltinStateName } from './constants.js';
17
+ export type { BuiltinStateName } from './constants.ts';
18
18
  /** Options shared by every rule. Set them once in `settings['test-governance']`. */
19
19
  export interface GovernanceOptions {
20
20
  /** Names that are treated as the Playwright `test` function. Defaults to `['test']`. */
@@ -78,7 +78,7 @@ export interface ResolvedOptions {
78
78
  /** Ticket format for comments: the shared default format. */
79
79
  commentTicket: TicketMatcher;
80
80
  }
81
- export { ConfigError } from './errors.js';
81
+ export { ConfigError } from './errors.ts';
82
82
  /**
83
83
  * Compiles `settings['test-governance']`. Every rule reads the same settings, so they always agree,
84
84
  * and the result is cached per settings object, so the rules share one compile.
@@ -1,8 +1,8 @@
1
- import { DEFAULT_COMMENT_KEYWORDS } from './constants.js';
2
- import { ConfigError } from './errors.js';
3
- import { optionsSchema } from './schema.js';
4
- import { compileTicketSpec, DEFAULT_PLACEHOLDERS } from './tickets.js';
5
- import { validate } from './validate.js';
1
+ import { BUILTIN_STATE_NAMES, DEFAULT_COMMENT_KEYWORDS } from "./constants.js";
2
+ import { ConfigError } from "./errors.js";
3
+ import { optionsSchema } from "./schema.js";
4
+ import { compileTicketSpec, DEFAULT_PLACEHOLDERS } from "./tickets.js";
5
+ import { validate } from "./validate.js";
6
6
  export const SETTINGS_KEY = 'test-governance';
7
7
  export function isTagState(state) {
8
8
  return state.tag !== undefined;
@@ -15,7 +15,7 @@ const BUILTIN_STATES = {
15
15
  new: { marker: 'NEW', tag: '@new', lifecycle: true, enabled: false },
16
16
  unstable: { marker: 'UNSTABLE', tag: '@unstable', lifecycle: true, enabled: false },
17
17
  };
18
- export { ConfigError } from './errors.js';
18
+ export { ConfigError } from "./errors.js";
19
19
  function toSpecs(spec) {
20
20
  return Array.isArray(spec) ? spec : [spec];
21
21
  }
@@ -34,8 +34,51 @@ export function resolveOptions(settings) {
34
34
  }
35
35
  return resolved;
36
36
  }
37
+ /** The options each ticket preset takes besides `preset`. */
38
+ const PRESET_OPTIONS = {
39
+ any: [],
40
+ jira: ['projects', 'host'],
41
+ github: ['host'],
42
+ gitlab: ['host'],
43
+ linear: ['teams'],
44
+ 'azure-devops': ['host'],
45
+ numeric: ['minLength', 'maxLength'],
46
+ pattern: ['pattern', 'flags'],
47
+ };
48
+ /** Options set on a ticket spec that its preset ignores, such as `host` on the `any` preset. */
49
+ function presetProblems(ticket, path) {
50
+ if (ticket === undefined)
51
+ return [];
52
+ const specs = Array.isArray(ticket) ? ticket : [ticket];
53
+ return specs.flatMap((spec, index) => {
54
+ const at = Array.isArray(ticket) ? `${path}[${index}]` : path;
55
+ const allowed = PRESET_OPTIONS[spec.preset];
56
+ return Object.keys(spec)
57
+ .filter((key) => key !== 'preset' && spec[key] !== undefined && !allowed.includes(key))
58
+ .map((key) => {
59
+ const presets = Object.entries(PRESET_OPTIONS).filter(([, keys]) => keys.includes(key));
60
+ return `${at}.${key} is not an option of the "${spec.preset}" preset (it applies to ${presets.map(([name]) => `"${name}"`).join(', ')})`;
61
+ });
62
+ });
63
+ }
64
+ /** Problems the schema can't express: options that belong to another preset, and reused state names. */
65
+ function crossFieldProblems(options) {
66
+ const problems = presetProblems(options.ticket, 'ticket');
67
+ for (const [name, override] of Object.entries(options.states ?? {})) {
68
+ if (typeof override === 'object')
69
+ problems.push(...presetProblems(override.ticket, `states.${name}.ticket`));
70
+ }
71
+ for (const [name, custom] of Object.entries(options.customStates ?? {})) {
72
+ if (BUILTIN_STATE_NAMES.includes(name)) {
73
+ problems.push(`customStates.${name} reuses the name of the built-in "${name}" state; give it another name, or use states.${name} to change the built-in`);
74
+ }
75
+ problems.push(...presetProblems(custom.ticket, `customStates.${name}.ticket`));
76
+ }
77
+ return problems;
78
+ }
37
79
  export function compileOptions(options) {
38
- const problems = validate(options, optionsSchema);
80
+ const schemaProblems = validate(options, optionsSchema);
81
+ const problems = schemaProblems.length > 0 ? schemaProblems : crossFieldProblems(options);
39
82
  if (problems.length > 0) {
40
83
  throw new ConfigError(`invalid options:\n${problems.map((problem) => ` - ${problem}`).join('\n')}`);
41
84
  }
@@ -1,4 +1,4 @@
1
- import type { Schema } from './validate.js';
1
+ import type { Schema } from './validate.ts';
2
2
  /** The shape of `settings['test-governance']`, the options of `configure()`. */
3
3
  export declare const optionsSchema: Schema;
4
4
  //# sourceMappingURL=schema.d.ts.map
@@ -1,12 +1,15 @@
1
- import { BUILTIN_STATE_NAMES, MARKER_RE, TAG_RE, TICKET_PRESETS } from './constants.js';
1
+ import { BUILTIN_STATE_NAMES, MARKER_RE, TAG_RE, TICKET_PRESETS } from "./constants.js";
2
2
  const MARKER_HINT = 'an uppercase keyword such as NEEDS-DATA (letters, digits, "-" and "_")';
3
3
  const TAG_HINT = 'a tag such as @needs-data';
4
+ /** Jira project and Linear team keys, as they appear in tickets: `WEB` in `WEB-123`. */
5
+ const KEY_PATTERN = '^[A-Z][A-Z0-9_]+$';
6
+ const KEY_HINT = 'an uppercase key as it appears in tickets, such as WEB for WEB-123';
4
7
  const ticketSpecSchema = {
5
8
  type: 'object',
6
9
  properties: {
7
10
  preset: { type: 'string', enum: [...TICKET_PRESETS] },
8
- projects: { type: 'array', items: { type: 'string' } },
9
- teams: { type: 'array', items: { type: 'string' } },
11
+ projects: { type: 'array', items: { type: 'string', pattern: KEY_PATTERN, patternHint: KEY_HINT } },
12
+ teams: { type: 'array', items: { type: 'string', pattern: KEY_PATTERN, patternHint: KEY_HINT } },
10
13
  host: { type: 'string' },
11
14
  minLength: { type: 'integer', minimum: 1 },
12
15
  maxLength: { type: 'integer', minimum: 1 },
@@ -1,4 +1,4 @@
1
- import { ConfigError } from './errors.js';
1
+ import { ConfigError } from "./errors.js";
2
2
  export const DEFAULT_PLACEHOLDERS = ['TODO', 'TBD', 'XXX-*', '0', '123', '1234', '12345'];
3
3
  // Every pattern below is anchored and has no nested or overlapping quantifiers, so matching is
4
4
  // linear in the token length. URLs are parsed with `new URL` rather than a regex.
@@ -1,4 +1,4 @@
1
- import { editDistance } from './distance.js';
1
+ import { editDistance } from "./distance.js";
2
2
  function kindOf(value) {
3
3
  if (value === null)
4
4
  return 'null';
@@ -16,8 +16,15 @@ A leftover marker makes searches for open tickets lie. This rule reports:
16
16
 
17
17
  `FIXME:` and `TODO:` are also everyday work comments, so this rule leaves them to
18
18
  [`require-ticket-in-comments`](require-ticket-in-comments.md), which requires a ticket in them anywhere
19
- in a test file. A `// FIXME: WEB-12` above a helper is a tracked work comment, not a stray marker. A
20
- `// FIXME: refactor this` gets one clear error from that rule rather than a confusing one from this.
19
+ in a test file. A `// FIXME: refactor this` gets one clear error from that rule rather than a confusing
20
+ one from this.
21
+
22
+ A `FIXME:` or `TODO:` comment that starts with a valid ticket is treated by where it sits:
23
+
24
+ - Above a helper or any other code that isn't a test, it is a tracked work comment and is allowed.
25
+ - Above a test, describe or skip call that is not in the `fixme` state, it reads as a leftover marker
26
+ and is reported, since `// FIXME: WEB-12` above a test usually means the test was fixed and the
27
+ marker forgotten. Move a work comment into the test body, or onto the line of code it is about.
21
28
 
22
29
  Other markers that read as a sentence, such as `// SKIP: this one is flaky`, are left alone too: the rule
23
30
  counts a comment as prose when its first word is not a valid ticket and more text follows.
@@ -15,6 +15,8 @@ source of truth and nothing is forgotten in a comment.
15
15
  ```ts
16
16
  // TODO: WEB-512 ✓
17
17
  // FIXME: WEB-77, WEB-78 ✓ several tickets, separated by commas
18
+ // TODO(WEB-512) ✓ the ticket can go in parentheses
19
+ // TODO(alice): refactor this ✖ a name is not a ticket
18
20
  // TODO: refactor this ✖ no ticket
19
21
  // FIXME: WEB-77 flaky locator ✖ only tickets are allowed (allowNotes: true allows the note)
20
22
  ```
@@ -24,6 +26,8 @@ How comments are read:
24
26
  - The keyword starts the comment (or a line of a block comment). `TODO` and `FIXME` in capitals
25
27
  count with or without a colon. Other casings, such as `todo:`, need the colon, so a sentence that
26
28
  happens to start with "Todo" is left alone.
29
+ - The tickets can also go in parentheses after the keyword, `TODO(WEB-512)` or `TODO (WEB-512)`, a
30
+ style many teams already use. Anything after the parentheses is a note, as it would be after `TODO: WEB-512`.
27
31
  - The tickets follow the same rules as markers: the shared ticket format, placeholders such as `TBD`
28
32
  rejected, and no text after the tickets unless `allowNotes` is set.
29
33
  - A `// FIXME:` marker directly above a `test.fixme(...)` is checked by
@@ -53,7 +53,8 @@ How ticket lists are read:
53
53
  | `// SKIP: WEB-1 , WEB-2 ` | two tickets; extra spaces are fine |
54
54
  | `// SKIP: https://acme.atlassian.net/browse/WEB-1?focusedCommentId=5#comment-5` | one ticket; query strings and fragments are part of a URL |
55
55
  | `// SKIP: WEB-1 flaky on CI`, `// SKIP: WEB-1: flaky` | extra text `flaky on CI` / `: flaky` |
56
- | `// SKIP: WEB-1,` or `// SKIP: WEB-1.` | extra text `,` / `.` |
56
+ | `// SKIP: WEB-1.` or `// SKIP: WEB-1:` | one ticket; a full stop or colon at the end is fine |
57
+ | `// SKIP: WEB-1,` | extra text `,` |
57
58
  | `// SKIP: WEB-1 and WEB-2` | extra text `and WEB-2`; separate tickets with commas |
58
59
  | `// SKIP: WEB-1;WEB-2`, `// SKIP: WEB-1/WEB-2` | invalid ticket, since only commas separate tickets |
59
60
  | `// SKIP: flaky on CI` | invalid ticket `flaky`: the first word must be a ticket |
@@ -64,7 +65,12 @@ lines. Extra text is reported only once the tickets on that line are valid.
64
65
  Conditional skips such as `test.skip(browserName === 'webkit', 'Not supported')` usually describe a
65
66
  permanent platform limit, so they need no ticket unless you set `requireTicketForConditional: true`.
66
67
  A runtime skip under an `if`, a `switch` case, a ternary or `&&` counts as conditional too, since
67
- `if (!enabled) test.skip()` does the same as `test.skip(!enabled)`.
68
+ `if (!enabled) test.skip()` does the same as `test.skip(!enabled)`. So does a skip in a `catch` block,
69
+ such as `catch { test.skip(true, 'Mail server is down') }`, and a skip after an early exit such as
70
+ `if (ready) return;` or `if (!ok) throw error;` earlier in the same function.
71
+
72
+ If one test or describe has several markers for the same state, each must be valid: in
73
+ `// SKIP: WEB-1` followed by `// SKIP: nope`, the second line is reported.
68
74
 
69
75
  ## Examples
70
76
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "eslint-plugin-test-governance",
3
- "version": "0.2.1",
3
+ "version": "0.3.1",
4
4
  "description": "ESLint rules that require a ticket comment above skipped, fixme and tagged Playwright tests.",
5
5
  "keywords": [
6
6
  "eslint",
@@ -45,8 +45,8 @@
45
45
  "scripts": {
46
46
  "build": "node -e \"require('node:fs').rmSync('dist',{recursive:true,force:true})\" && tsc -p tsconfig.build.json",
47
47
  "typecheck": "tsc -p tsconfig.json",
48
- "test": "vitest run",
49
- "test:coverage": "vitest run --coverage",
48
+ "test": "node --test 'tests/**/*.test.ts'",
49
+ "test:coverage": "node --test --experimental-test-coverage --test-coverage-include='src/rules/**' --test-coverage-include='src/utils/**' --test-coverage-branches=100 --test-coverage-functions=100 --test-coverage-lines=100 'tests/**/*.test.ts'",
50
50
  "test:pack": "node scripts/pack-smoke.mjs",
51
51
  "docs": "npm run build && eslint-doc-generator",
52
52
  "docs:check": "npm run build && eslint-doc-generator --check",
@@ -63,15 +63,11 @@
63
63
  "@types/node": "^26.6.3",
64
64
  "@typescript-eslint/parser": "^8.71.0",
65
65
  "@typescript-eslint/utils": "^8.71.0",
66
- "@vitest/coverage-v8": "^5.0.2",
67
66
  "eslint": "^10.11.0",
68
- "eslint-config-prettier": "^10.1.8",
69
67
  "eslint-doc-generator": "^3.7.1",
70
68
  "eslint-plugin-eslint-plugin": "^7.6.2",
71
- "globals": "^17.13.0",
72
69
  "prettier": "^3.9.9",
73
70
  "typescript": "6.0",
74
- "typescript-eslint": "^8.71.0",
75
- "vitest": "^5.0.2"
71
+ "typescript-eslint": "^8.71.0"
76
72
  }
77
73
  }