eslint-plugin-test-governance 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -2,7 +2,7 @@
2
2
 
3
3
  📝 Disallow state tags that contradict each other, differ in case or look like typos.
4
4
 
5
- ⚠️ This rule _warns_ in the ✅ `recommended` [config](https://github.com/Huldoser/eslint-plugin-test-governance#usage).
5
+ ⚠️ This rule _warns_ in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
6
6
 
7
7
  🔧 This rule is automatically fixable by the [`--fix` CLI option](https://eslint.org/docs/latest/user-guide/command-line-interface#--fix).
8
8
 
@@ -17,7 +17,9 @@ rule reports:
17
17
  - a near-miss spelling of a state tag, such as `@unstabel`.
18
18
 
19
19
  The `@new` checks run when the `new` state is on (`lifecycleTags: true`). Case and typo checks cover
20
- every tag state that is on, including custom states.
20
+ every tag state that is on, including custom states. A tag kept in a `const` in the same file is
21
+ reported where the test uses it, without an autofix, since other tests may share the constant. Vitest
22
+ tag names are written without the `@`, so `{ tags: ['New'] }` is reported, and fixed, as `'new'`.
21
23
 
22
24
  ## Examples
23
25
 
@@ -2,7 +2,7 @@
2
2
 
3
3
  📝 Disallow marker comments that no longer match a test state.
4
4
 
5
- 💼 This rule is enabled in the ✅ `recommended` [config](https://github.com/Huldoser/eslint-plugin-test-governance#usage).
5
+ 💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
6
6
 
7
7
  💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
8
8
 
@@ -33,6 +33,16 @@ A `FIXME:` or `TODO:` comment that starts with a valid ticket is treated by wher
33
33
  Other markers that read as a sentence, such as `// SKIP: this one is flaky`, are left alone too: the rule
34
34
  counts a comment as prose when its first word is not a valid ticket and more text follows.
35
35
 
36
+ A marker written in the wrong case, such as `// skip: TRADE-1`, is reported once the test is no longer
37
+ in that state, as long as a valid ticket follows the colon. Without one, as in `// skip: not on mobile`,
38
+ it reads as a sentence.
39
+
40
+ When a test's tags come from another module, as in `{ tag: TAGS.NEW }` with `TAGS` imported, the rule
41
+ can't tell which tags it has, so it leaves markers for tag states such as `// NEW:` alone on that test
42
+ and the tests inside it. The same goes for Vitest's state options, such as `// SKIP:`, when the options
43
+ object itself comes from another module. Tags kept in a `const` or an enum in the same file are read
44
+ and checked as usual.
45
+
36
46
  Files without tests, such as application code, are not checked.
37
47
 
38
48
  The rule offers a suggestion to delete the comment.
@@ -57,6 +67,23 @@ test('places a limit order', async ({ page }) => {
57
67
  });
58
68
  ```
59
69
 
70
+ <!-- example: invalid -->
71
+
72
+ ```js
73
+ // skip: TRADE-82
74
+ test('cancels an open order', async ({ page }) => {});
75
+ ```
76
+
77
+ A Vitest test with `skipIf(false)`, `runIf(true)` or `{ skip: false }` always runs, so its marker is
78
+ left over:
79
+
80
+ <!-- example: invalid settings={"framework":"vitest"} -->
81
+
82
+ ```js
83
+ // SKIP: TRADE-83
84
+ test.skipIf(false)('fills a stop order when the market opens', () => {});
85
+ ```
86
+
60
87
  Correct:
61
88
 
62
89
  <!-- example: valid settings={"lifecycleTags":true} -->
@@ -2,7 +2,7 @@
2
2
 
3
3
  📝 Require FIXME and TODO comments in test files to start with a ticket.
4
4
 
5
- 💼 This rule is enabled in the ✅ `recommended` [config](https://github.com/Huldoser/eslint-plugin-test-governance#usage).
5
+ 💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
6
6
 
7
7
  💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
8
8
 
@@ -2,7 +2,7 @@
2
2
 
3
3
  📝 Require a ticket marker comment above skipped, fixme and tagged tests.
4
4
 
5
- 💼 This rule is enabled in the ✅ `recommended` [config](https://github.com/Huldoser/eslint-plugin-test-governance#usage).
5
+ 💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
6
6
 
7
7
  💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
8
8
 
@@ -23,12 +23,14 @@ A test is in a state when it is:
23
23
  | `fail`, `slow` | `test.fail(...)`, `test.slow()`, `testInfo.fail()`, `testInfo.slow()` | `// FAIL:`, `// SLOW:` | no |
24
24
  | custom | any tag you list in `customStates` | your keyword | when configured |
25
25
 
26
- In Jest and Vitest, `xit`, `xdescribe`, Vitest's `{ skip: true }` and `context.skip()` skip a test
27
- too, and `test.failing` and `test.fails` put it in the `fail` state. The
28
- [README](../../README.md#jest-and-vitest) lists every form.
26
+ In Jest and Vitest, `xit`, `xdescribe`, Vitest's `{ skip: true }`, `skipIf(true)` and `context.skip()`,
27
+ in a test or in `beforeEach`, skip a test too, and `test.failing` and `test.fails` put it in the `fail` state. The
28
+ [README](../../README.md#frameworks) lists every form.
29
29
 
30
- Tags are read from the title (`'places a limit order @new'`) and from `{ tag: '@new' }` or `{ tag: ['@new', '@smoke'] }`.
31
- Tags on a `describe` apply to every test inside it.
30
+ Tags are read from the title (`'places a limit order @new'`) and from `{ tag: '@new' }` or `{ tag: ['@new', '@smoke'] }`,
31
+ also when they come from a `const` or an enum declared in the same file, as in `{ tag: TAGS.NEW }`.
32
+ Vitest writes tag names without the `@`, so `{ tags: ['new'] }` is the `@new` tag. Tags on a
33
+ `describe` apply to every test inside it.
32
34
 
33
35
  ## Marker format
34
36
 
@@ -80,14 +82,15 @@ such as `catch { test.skip(true, 'Market data feed is down') }`, and a skip afte
80
82
  If one test or describe has several markers for the same state, each must be valid: in
81
83
  `// SKIP: TRADE-1` followed by `// SKIP: nope`, the second line is reported.
82
84
 
83
- ## Dynamic titles
85
+ ## Titles and tags that can't be read
84
86
 
85
- Tags can only be read from a title that is text: a string or a template literal such as
86
- `` `buys ${symbol} @new` ``. A title built another way, such as `'buys ' + symbol` or a variable, may
87
- hide a tag. With `reportDynamicTitles: true`, the rule reports those titles whenever a tag state
88
- (`@new`, `@unstable` or a custom state) is on, so no tagged test slips past.
87
+ Tags can only be read from text: a string or a template literal such as `` `buys ${symbol} @new` ``,
88
+ written in the call or kept in a `const` or an enum in the same file. A title built another way, such
89
+ as `'buys ' + symbol`, and a tag or details object imported from another module may hide a tag. With
90
+ `reportUnreadableTags: true`, the rule reports those whenever a tag state (`@new`, `@unstable` or a
91
+ custom state) is on, so no tagged test slips past.
89
92
 
90
- <!-- example: invalid settings={"lifecycleTags":true,"reportDynamicTitles":true} -->
93
+ <!-- example: invalid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
91
94
 
92
95
  ```js
93
96
  for (const symbol of ['AAPL', 'MSFT']) {
@@ -95,7 +98,15 @@ for (const symbol of ['AAPL', 'MSFT']) {
95
98
  }
96
99
  ```
97
100
 
98
- <!-- example: valid settings={"lifecycleTags":true,"reportDynamicTitles":true} -->
101
+ <!-- example: invalid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
102
+
103
+ ```js
104
+ import { TAGS } from './tags';
105
+
106
+ test('closes all positions with one click', { tag: TAGS.NEW }, async ({ page }) => {});
107
+ ```
108
+
109
+ <!-- example: valid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
99
110
 
100
111
  ```js
101
112
  for (const symbol of ['AAPL', 'MSFT']) {
@@ -103,6 +114,15 @@ for (const symbol of ['AAPL', 'MSFT']) {
103
114
  }
104
115
  ```
105
116
 
117
+ <!-- example: valid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
118
+
119
+ ```js
120
+ const TAGS = { NEW: '@new', SMOKE: '@smoke' };
121
+
122
+ // NEW: TRADE-77
123
+ test('closes all positions with one click', { tag: [TAGS.NEW, TAGS.SMOKE] }, async ({ page }) => {});
124
+ ```
125
+
106
126
  ## Examples
107
127
 
108
128
  Incorrect:
@@ -146,6 +166,12 @@ test('closes all positions with one click @new', async ({ page }) => {});
146
166
  it.todo('caps position size at 2% of equity');
147
167
  ```
148
168
 
169
+ <!-- example: invalid settings={"framework":"vitest"} -->
170
+
171
+ ```js
172
+ test.skipIf(true)('fills a stop order when the market opens', () => {});
173
+ ```
174
+
149
175
  Correct:
150
176
 
151
177
  <!-- example: valid -->
@@ -212,5 +238,5 @@ change what it reports:
212
238
  - `requireTicketForConditional`: conditional skips need a ticket too.
213
239
  - `allowBlankLine`: blank lines may separate a marker from its test.
214
240
  - `allowNotes`: text may follow the tickets.
215
- - `reportDynamicTitles`: report titles whose tags can't be read (see [Dynamic titles](#dynamic-titles)).
241
+ - `reportUnreadableTags`: report titles and tags that can't be read (see [Titles and tags that can't be read](#titles-and-tags-that-cant-be-read)).
216
242
  - `framework` and `testFunctions`: which functions count as the framework's `test`.
@@ -0,0 +1,131 @@
1
+ # Upgrading
2
+
3
+ ## Upgrading to 0.5
4
+
5
+ 0.5.0 changes how the plugin is configured, ahead of 1.0. Most projects change one or two lines. A
6
+ config that still uses an old name fails when ESLint loads it, with a message that says what to
7
+ change, so no check is turned off without a word.
8
+
9
+ | What changed | Who it affects | What to do |
10
+ | ------------------------------------------------------------------- | -------------------------------------------------------------------------------- | ----------------------------------------------------- |
11
+ | [`configs.recommended` is removed](#the-recommended-config) | Playwright projects that use it | Use `configs.playwright` |
12
+ | [`framework` is required](#the-framework-option) | `configure()` calls and hand-written `settings['test-governance']` without one | Add `framework: 'playwright'`, `'jest'` or `'vitest'` |
13
+ | [`reportDynamicTitles` is renamed](#reportunreadabletags) | Configs that set it | Call it `reportUnreadableTags` |
14
+ | [Vitest's `skipIf(true)` needs a ticket](#vitests-skipif-and-runif) | Vitest tests skipped with a literal `true` in `skipIf()` or `false` in `runIf()` | Add a `// SKIP:` marker, or remove the skip |
15
+
16
+ Jest and Vitest projects set up with `configs.jest`, `configs.vitest` or `configure({ framework })` in
17
+ 0.4.0 can skip the first two rows.
18
+
19
+ ### The recommended config
20
+
21
+ `configs.recommended` checked every file as Playwright tests. It is gone, so use the config for your
22
+ framework. The rules and their severities are the same.
23
+
24
+ Before:
25
+
26
+ ```js
27
+ export default [{ files: ['tests/**/*.spec.ts'], ...testGovernance.configs.recommended }];
28
+ ```
29
+
30
+ After:
31
+
32
+ ```js
33
+ export default [{ files: ['tests/**/*.spec.ts'], ...testGovernance.configs.playwright }];
34
+ ```
35
+
36
+ With `defineConfig()`, change `extends` the same way: `extends: [testGovernance.configs.recommended]`
37
+ becomes `extends: [testGovernance.configs.playwright]`, and `extends: ['test-governance/recommended']`
38
+ becomes `extends: ['test-governance/playwright']`.
39
+
40
+ A config that still reads `configs.recommended` fails when it loads:
41
+
42
+ ```text
43
+ eslint-plugin-test-governance: configs.recommended was removed in 0.5.0. Use the config for your framework: configs.playwright, configs.jest or configs.vitest. See https://github.com/Huldoser/eslint-plugin-test-governance/blob/main/docs/upgrading.md
44
+ ```
45
+
46
+ ### The framework option
47
+
48
+ Before 0.5.0, a config without a `framework` checked files as Playwright tests. Now `configure()`
49
+ needs one.
50
+
51
+ Before:
52
+
53
+ ```js
54
+ testGovernance.configure({
55
+ ticket: { preset: 'jira', projects: ['TRADE', 'RISK'] },
56
+ lifecycleTags: true,
57
+ });
58
+ ```
59
+
60
+ After:
61
+
62
+ ```js
63
+ testGovernance.configure({
64
+ framework: 'playwright',
65
+ ticket: { preset: 'jira', projects: ['TRADE', 'RISK'] },
66
+ lifecycleTags: true,
67
+ });
68
+ ```
69
+
70
+ Without it, the config fails when it loads:
71
+
72
+ ```text
73
+ eslint-plugin-test-governance: invalid options:
74
+ - framework is required: set it to "playwright", "jest" or "vitest", or use configs.playwright, configs.jest or configs.vitest
75
+ ```
76
+
77
+ If you write `settings['test-governance']` yourself instead of calling `configure()`, add `framework`
78
+ there too. That config fails on the first lint instead, with the same message after
79
+ `Error while loading rule 'test-governance/require-ticket':`.
80
+
81
+ To share options between frameworks, pass the same options to a `configure()` for each one, as in
82
+ [Frameworks](../README.md#frameworks).
83
+
84
+ ### reportUnreadableTags
85
+
86
+ `reportDynamicTitles` is now called `reportUnreadableTags`, because it also reports tags and details
87
+ objects imported from another module, not only titles. It works the same way.
88
+
89
+ Before:
90
+
91
+ ```js
92
+ testGovernance.configure({ framework: 'playwright', lifecycleTags: true, reportDynamicTitles: true });
93
+ ```
94
+
95
+ After:
96
+
97
+ ```js
98
+ testGovernance.configure({ framework: 'playwright', lifecycleTags: true, reportUnreadableTags: true });
99
+ ```
100
+
101
+ The old name fails when the config loads:
102
+
103
+ ```text
104
+ eslint-plugin-test-governance: invalid options:
105
+ - reportDynamicTitles was renamed to "reportUnreadableTags"
106
+ ```
107
+
108
+ ### Vitest's skipIf and runIf
109
+
110
+ A literal condition now decides whether the test is skipped, as it does for `{ skip: true }` and
111
+ Playwright's `test.skip(true)`:
112
+
113
+ - `test.skipIf(true)` and `test.runIf(false)` always skip, so they need a `// SKIP:` marker.
114
+ - `test.skipIf(false)` and `test.runIf(true)` never skip, so a `// SKIP:` marker above one is reported
115
+ as left over.
116
+ - Any other condition, such as `test.skipIf(process.env.CI)`, is a conditional skip as before. It needs
117
+ a ticket only with `requireTicketForConditional: true`.
118
+
119
+ The same goes for `describe.skipIf()` and `describe.runIf()`.
120
+
121
+ ```ts
122
+ // SKIP: TRADE-1482
123
+ test.skipIf(true)('fills a stop order when the market opens', () => {});
124
+ ```
125
+
126
+ ### Checking the upgrade
127
+
128
+ Run ESLint on your tests as usual. Once the config loads, the upgrade is done. 0.5.0 also reads more
129
+ of your tests than 0.4.0, so a few new reports are expected: tags kept in a `const` in the same file,
130
+ Vitest tag names written without the `@`, and `context.skip()` in Vitest's `beforeEach`. The
131
+ [changelog](../CHANGELOG.md) lists them all.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "eslint-plugin-test-governance",
3
- "version": "0.4.0",
3
+ "version": "0.5.0",
4
4
  "description": "ESLint rules that require a ticket comment above skipped, fixme, todo and tagged Playwright, Jest and Vitest tests.",
5
5
  "keywords": [
6
6
  "eslint",