eslint-plugin-test-governance 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +53 -1
- package/README.md +68 -50
- package/dist/index.d.ts +8 -9
- package/dist/index.js +17 -13
- package/dist/rules/no-conflicting-states.js +9 -3
- package/dist/rules/no-orphaned-marker.js +1 -1
- package/dist/rules/require-ticket.d.ts +1 -1
- package/dist/rules/require-ticket.js +6 -2
- package/dist/utils/analyze.d.ts +18 -4
- package/dist/utils/analyze.js +200 -46
- package/dist/utils/frameworks.d.ts +13 -4
- package/dist/utils/frameworks.js +8 -2
- package/dist/utils/options.d.ts +11 -5
- package/dist/utils/options.js +2 -2
- package/dist/utils/schema.js +8 -2
- package/dist/utils/validate.d.ts +4 -0
- package/dist/utils/validate.js +7 -2
- package/docs/rules/marker-matches-state.md +1 -1
- package/docs/rules/no-conflicting-states.md +4 -2
- package/docs/rules/no-orphaned-marker.md +28 -1
- package/docs/rules/require-ticket-in-comments.md +1 -1
- package/docs/rules/require-ticket.md +40 -14
- package/docs/upgrading.md +131 -0
- package/package.json +1 -1
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Disallow state tags that contradict each other, differ in case or look like typos.
|
|
4
4
|
|
|
5
|
-
⚠️ This rule _warns_ in the
|
|
5
|
+
⚠️ This rule _warns_ in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
6
6
|
|
|
7
7
|
🔧 This rule is automatically fixable by the [`--fix` CLI option](https://eslint.org/docs/latest/user-guide/command-line-interface#--fix).
|
|
8
8
|
|
|
@@ -17,7 +17,9 @@ rule reports:
|
|
|
17
17
|
- a near-miss spelling of a state tag, such as `@unstabel`.
|
|
18
18
|
|
|
19
19
|
The `@new` checks run when the `new` state is on (`lifecycleTags: true`). Case and typo checks cover
|
|
20
|
-
every tag state that is on, including custom states.
|
|
20
|
+
every tag state that is on, including custom states. A tag kept in a `const` in the same file is
|
|
21
|
+
reported where the test uses it, without an autofix, since other tests may share the constant. Vitest
|
|
22
|
+
tag names are written without the `@`, so `{ tags: ['New'] }` is reported, and fixed, as `'new'`.
|
|
21
23
|
|
|
22
24
|
## Examples
|
|
23
25
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Disallow marker comments that no longer match a test state.
|
|
4
4
|
|
|
5
|
-
💼 This rule is enabled in the
|
|
5
|
+
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
6
6
|
|
|
7
7
|
💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
8
8
|
|
|
@@ -33,6 +33,16 @@ A `FIXME:` or `TODO:` comment that starts with a valid ticket is treated by wher
|
|
|
33
33
|
Other markers that read as a sentence, such as `// SKIP: this one is flaky`, are left alone too: the rule
|
|
34
34
|
counts a comment as prose when its first word is not a valid ticket and more text follows.
|
|
35
35
|
|
|
36
|
+
A marker written in the wrong case, such as `// skip: TRADE-1`, is reported once the test is no longer
|
|
37
|
+
in that state, as long as a valid ticket follows the colon. Without one, as in `// skip: not on mobile`,
|
|
38
|
+
it reads as a sentence.
|
|
39
|
+
|
|
40
|
+
When a test's tags come from another module, as in `{ tag: TAGS.NEW }` with `TAGS` imported, the rule
|
|
41
|
+
can't tell which tags it has, so it leaves markers for tag states such as `// NEW:` alone on that test
|
|
42
|
+
and the tests inside it. The same goes for Vitest's state options, such as `// SKIP:`, when the options
|
|
43
|
+
object itself comes from another module. Tags kept in a `const` or an enum in the same file are read
|
|
44
|
+
and checked as usual.
|
|
45
|
+
|
|
36
46
|
Files without tests, such as application code, are not checked.
|
|
37
47
|
|
|
38
48
|
The rule offers a suggestion to delete the comment.
|
|
@@ -57,6 +67,23 @@ test('places a limit order', async ({ page }) => {
|
|
|
57
67
|
});
|
|
58
68
|
```
|
|
59
69
|
|
|
70
|
+
<!-- example: invalid -->
|
|
71
|
+
|
|
72
|
+
```js
|
|
73
|
+
// skip: TRADE-82
|
|
74
|
+
test('cancels an open order', async ({ page }) => {});
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
A Vitest test with `skipIf(false)`, `runIf(true)` or `{ skip: false }` always runs, so its marker is
|
|
78
|
+
left over:
|
|
79
|
+
|
|
80
|
+
<!-- example: invalid settings={"framework":"vitest"} -->
|
|
81
|
+
|
|
82
|
+
```js
|
|
83
|
+
// SKIP: TRADE-83
|
|
84
|
+
test.skipIf(false)('fills a stop order when the market opens', () => {});
|
|
85
|
+
```
|
|
86
|
+
|
|
60
87
|
Correct:
|
|
61
88
|
|
|
62
89
|
<!-- example: valid settings={"lifecycleTags":true} -->
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Require FIXME and TODO comments in test files to start with a ticket.
|
|
4
4
|
|
|
5
|
-
💼 This rule is enabled in the
|
|
5
|
+
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
6
6
|
|
|
7
7
|
💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
8
8
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Require a ticket marker comment above skipped, fixme and tagged tests.
|
|
4
4
|
|
|
5
|
-
💼 This rule is enabled in the
|
|
5
|
+
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
6
6
|
|
|
7
7
|
💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
8
8
|
|
|
@@ -23,12 +23,14 @@ A test is in a state when it is:
|
|
|
23
23
|
| `fail`, `slow` | `test.fail(...)`, `test.slow()`, `testInfo.fail()`, `testInfo.slow()` | `// FAIL:`, `// SLOW:` | no |
|
|
24
24
|
| custom | any tag you list in `customStates` | your keyword | when configured |
|
|
25
25
|
|
|
26
|
-
In Jest and Vitest, `xit`, `xdescribe`, Vitest's `{ skip: true }` and `context.skip()
|
|
27
|
-
too, and `test.failing` and `test.fails` put it in the `fail` state. The
|
|
28
|
-
[README](../../README.md#
|
|
26
|
+
In Jest and Vitest, `xit`, `xdescribe`, Vitest's `{ skip: true }`, `skipIf(true)` and `context.skip()`,
|
|
27
|
+
in a test or in `beforeEach`, skip a test too, and `test.failing` and `test.fails` put it in the `fail` state. The
|
|
28
|
+
[README](../../README.md#frameworks) lists every form.
|
|
29
29
|
|
|
30
|
-
Tags are read from the title (`'places a limit order @new'`) and from `{ tag: '@new' }` or `{ tag: ['@new', '@smoke'] }
|
|
31
|
-
|
|
30
|
+
Tags are read from the title (`'places a limit order @new'`) and from `{ tag: '@new' }` or `{ tag: ['@new', '@smoke'] }`,
|
|
31
|
+
also when they come from a `const` or an enum declared in the same file, as in `{ tag: TAGS.NEW }`.
|
|
32
|
+
Vitest writes tag names without the `@`, so `{ tags: ['new'] }` is the `@new` tag. Tags on a
|
|
33
|
+
`describe` apply to every test inside it.
|
|
32
34
|
|
|
33
35
|
## Marker format
|
|
34
36
|
|
|
@@ -80,14 +82,15 @@ such as `catch { test.skip(true, 'Market data feed is down') }`, and a skip afte
|
|
|
80
82
|
If one test or describe has several markers for the same state, each must be valid: in
|
|
81
83
|
`// SKIP: TRADE-1` followed by `// SKIP: nope`, the second line is reported.
|
|
82
84
|
|
|
83
|
-
##
|
|
85
|
+
## Titles and tags that can't be read
|
|
84
86
|
|
|
85
|
-
Tags can only be read from
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
87
|
+
Tags can only be read from text: a string or a template literal such as `` `buys ${symbol} @new` ``,
|
|
88
|
+
written in the call or kept in a `const` or an enum in the same file. A title built another way, such
|
|
89
|
+
as `'buys ' + symbol`, and a tag or details object imported from another module may hide a tag. With
|
|
90
|
+
`reportUnreadableTags: true`, the rule reports those whenever a tag state (`@new`, `@unstable` or a
|
|
91
|
+
custom state) is on, so no tagged test slips past.
|
|
89
92
|
|
|
90
|
-
<!-- example: invalid settings={"lifecycleTags":true,"
|
|
93
|
+
<!-- example: invalid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
|
|
91
94
|
|
|
92
95
|
```js
|
|
93
96
|
for (const symbol of ['AAPL', 'MSFT']) {
|
|
@@ -95,7 +98,15 @@ for (const symbol of ['AAPL', 'MSFT']) {
|
|
|
95
98
|
}
|
|
96
99
|
```
|
|
97
100
|
|
|
98
|
-
<!-- example:
|
|
101
|
+
<!-- example: invalid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
|
|
102
|
+
|
|
103
|
+
```js
|
|
104
|
+
import { TAGS } from './tags';
|
|
105
|
+
|
|
106
|
+
test('closes all positions with one click', { tag: TAGS.NEW }, async ({ page }) => {});
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
<!-- example: valid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
|
|
99
110
|
|
|
100
111
|
```js
|
|
101
112
|
for (const symbol of ['AAPL', 'MSFT']) {
|
|
@@ -103,6 +114,15 @@ for (const symbol of ['AAPL', 'MSFT']) {
|
|
|
103
114
|
}
|
|
104
115
|
```
|
|
105
116
|
|
|
117
|
+
<!-- example: valid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
|
|
118
|
+
|
|
119
|
+
```js
|
|
120
|
+
const TAGS = { NEW: '@new', SMOKE: '@smoke' };
|
|
121
|
+
|
|
122
|
+
// NEW: TRADE-77
|
|
123
|
+
test('closes all positions with one click', { tag: [TAGS.NEW, TAGS.SMOKE] }, async ({ page }) => {});
|
|
124
|
+
```
|
|
125
|
+
|
|
106
126
|
## Examples
|
|
107
127
|
|
|
108
128
|
Incorrect:
|
|
@@ -146,6 +166,12 @@ test('closes all positions with one click @new', async ({ page }) => {});
|
|
|
146
166
|
it.todo('caps position size at 2% of equity');
|
|
147
167
|
```
|
|
148
168
|
|
|
169
|
+
<!-- example: invalid settings={"framework":"vitest"} -->
|
|
170
|
+
|
|
171
|
+
```js
|
|
172
|
+
test.skipIf(true)('fills a stop order when the market opens', () => {});
|
|
173
|
+
```
|
|
174
|
+
|
|
149
175
|
Correct:
|
|
150
176
|
|
|
151
177
|
<!-- example: valid -->
|
|
@@ -212,5 +238,5 @@ change what it reports:
|
|
|
212
238
|
- `requireTicketForConditional`: conditional skips need a ticket too.
|
|
213
239
|
- `allowBlankLine`: blank lines may separate a marker from its test.
|
|
214
240
|
- `allowNotes`: text may follow the tickets.
|
|
215
|
-
- `
|
|
241
|
+
- `reportUnreadableTags`: report titles and tags that can't be read (see [Titles and tags that can't be read](#titles-and-tags-that-cant-be-read)).
|
|
216
242
|
- `framework` and `testFunctions`: which functions count as the framework's `test`.
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
# Upgrading
|
|
2
|
+
|
|
3
|
+
## Upgrading to 0.5
|
|
4
|
+
|
|
5
|
+
0.5.0 changes how the plugin is configured, ahead of 1.0. Most projects change one or two lines. A
|
|
6
|
+
config that still uses an old name fails when ESLint loads it, with a message that says what to
|
|
7
|
+
change, so no check is turned off without a word.
|
|
8
|
+
|
|
9
|
+
| What changed | Who it affects | What to do |
|
|
10
|
+
| ------------------------------------------------------------------- | -------------------------------------------------------------------------------- | ----------------------------------------------------- |
|
|
11
|
+
| [`configs.recommended` is removed](#the-recommended-config) | Playwright projects that use it | Use `configs.playwright` |
|
|
12
|
+
| [`framework` is required](#the-framework-option) | `configure()` calls and hand-written `settings['test-governance']` without one | Add `framework: 'playwright'`, `'jest'` or `'vitest'` |
|
|
13
|
+
| [`reportDynamicTitles` is renamed](#reportunreadabletags) | Configs that set it | Call it `reportUnreadableTags` |
|
|
14
|
+
| [Vitest's `skipIf(true)` needs a ticket](#vitests-skipif-and-runif) | Vitest tests skipped with a literal `true` in `skipIf()` or `false` in `runIf()` | Add a `// SKIP:` marker, or remove the skip |
|
|
15
|
+
|
|
16
|
+
Jest and Vitest projects set up with `configs.jest`, `configs.vitest` or `configure({ framework })` in
|
|
17
|
+
0.4.0 can skip the first two rows.
|
|
18
|
+
|
|
19
|
+
### The recommended config
|
|
20
|
+
|
|
21
|
+
`configs.recommended` checked every file as Playwright tests. It is gone, so use the config for your
|
|
22
|
+
framework. The rules and their severities are the same.
|
|
23
|
+
|
|
24
|
+
Before:
|
|
25
|
+
|
|
26
|
+
```js
|
|
27
|
+
export default [{ files: ['tests/**/*.spec.ts'], ...testGovernance.configs.recommended }];
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
After:
|
|
31
|
+
|
|
32
|
+
```js
|
|
33
|
+
export default [{ files: ['tests/**/*.spec.ts'], ...testGovernance.configs.playwright }];
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
With `defineConfig()`, change `extends` the same way: `extends: [testGovernance.configs.recommended]`
|
|
37
|
+
becomes `extends: [testGovernance.configs.playwright]`, and `extends: ['test-governance/recommended']`
|
|
38
|
+
becomes `extends: ['test-governance/playwright']`.
|
|
39
|
+
|
|
40
|
+
A config that still reads `configs.recommended` fails when it loads:
|
|
41
|
+
|
|
42
|
+
```text
|
|
43
|
+
eslint-plugin-test-governance: configs.recommended was removed in 0.5.0. Use the config for your framework: configs.playwright, configs.jest or configs.vitest. See https://github.com/Huldoser/eslint-plugin-test-governance/blob/main/docs/upgrading.md
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### The framework option
|
|
47
|
+
|
|
48
|
+
Before 0.5.0, a config without a `framework` checked files as Playwright tests. Now `configure()`
|
|
49
|
+
needs one.
|
|
50
|
+
|
|
51
|
+
Before:
|
|
52
|
+
|
|
53
|
+
```js
|
|
54
|
+
testGovernance.configure({
|
|
55
|
+
ticket: { preset: 'jira', projects: ['TRADE', 'RISK'] },
|
|
56
|
+
lifecycleTags: true,
|
|
57
|
+
});
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
After:
|
|
61
|
+
|
|
62
|
+
```js
|
|
63
|
+
testGovernance.configure({
|
|
64
|
+
framework: 'playwright',
|
|
65
|
+
ticket: { preset: 'jira', projects: ['TRADE', 'RISK'] },
|
|
66
|
+
lifecycleTags: true,
|
|
67
|
+
});
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
Without it, the config fails when it loads:
|
|
71
|
+
|
|
72
|
+
```text
|
|
73
|
+
eslint-plugin-test-governance: invalid options:
|
|
74
|
+
- framework is required: set it to "playwright", "jest" or "vitest", or use configs.playwright, configs.jest or configs.vitest
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
If you write `settings['test-governance']` yourself instead of calling `configure()`, add `framework`
|
|
78
|
+
there too. That config fails on the first lint instead, with the same message after
|
|
79
|
+
`Error while loading rule 'test-governance/require-ticket':`.
|
|
80
|
+
|
|
81
|
+
To share options between frameworks, pass the same options to a `configure()` for each one, as in
|
|
82
|
+
[Frameworks](../README.md#frameworks).
|
|
83
|
+
|
|
84
|
+
### reportUnreadableTags
|
|
85
|
+
|
|
86
|
+
`reportDynamicTitles` is now called `reportUnreadableTags`, because it also reports tags and details
|
|
87
|
+
objects imported from another module, not only titles. It works the same way.
|
|
88
|
+
|
|
89
|
+
Before:
|
|
90
|
+
|
|
91
|
+
```js
|
|
92
|
+
testGovernance.configure({ framework: 'playwright', lifecycleTags: true, reportDynamicTitles: true });
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
After:
|
|
96
|
+
|
|
97
|
+
```js
|
|
98
|
+
testGovernance.configure({ framework: 'playwright', lifecycleTags: true, reportUnreadableTags: true });
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
The old name fails when the config loads:
|
|
102
|
+
|
|
103
|
+
```text
|
|
104
|
+
eslint-plugin-test-governance: invalid options:
|
|
105
|
+
- reportDynamicTitles was renamed to "reportUnreadableTags"
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
### Vitest's skipIf and runIf
|
|
109
|
+
|
|
110
|
+
A literal condition now decides whether the test is skipped, as it does for `{ skip: true }` and
|
|
111
|
+
Playwright's `test.skip(true)`:
|
|
112
|
+
|
|
113
|
+
- `test.skipIf(true)` and `test.runIf(false)` always skip, so they need a `// SKIP:` marker.
|
|
114
|
+
- `test.skipIf(false)` and `test.runIf(true)` never skip, so a `// SKIP:` marker above one is reported
|
|
115
|
+
as left over.
|
|
116
|
+
- Any other condition, such as `test.skipIf(process.env.CI)`, is a conditional skip as before. It needs
|
|
117
|
+
a ticket only with `requireTicketForConditional: true`.
|
|
118
|
+
|
|
119
|
+
The same goes for `describe.skipIf()` and `describe.runIf()`.
|
|
120
|
+
|
|
121
|
+
```ts
|
|
122
|
+
// SKIP: TRADE-1482
|
|
123
|
+
test.skipIf(true)('fills a stop order when the market opens', () => {});
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### Checking the upgrade
|
|
127
|
+
|
|
128
|
+
Run ESLint on your tests as usual. Once the config loads, the upgrade is done. 0.5.0 also reads more
|
|
129
|
+
of your tests than 0.4.0, so a few new reports are expected: tags kept in a `const` in the same file,
|
|
130
|
+
Vitest tag names written without the `@`, and `context.skip()` in Vitest's `beforeEach`. The
|
|
131
|
+
[changelog](../CHANGELOG.md) lists them all.
|
package/package.json
CHANGED