eslint-plugin-test-governance 0.6.2 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -1
- package/README.md +62 -41
- package/dist/index.d.ts +3 -3
- package/dist/index.js +1 -1
- package/dist/utils/analyze.js +67 -16
- package/dist/utils/constants.d.ts +1 -1
- package/dist/utils/constants.js +1 -1
- package/dist/utils/frameworks.d.ts +4 -0
- package/dist/utils/frameworks.js +53 -1
- package/dist/utils/options.d.ts +5 -5
- package/docs/rules/marker-matches-state.md +1 -1
- package/docs/rules/no-conflicting-states.md +1 -1
- package/docs/rules/no-orphaned-marker.md +3 -3
- package/docs/rules/require-ticket-in-comments.md +3 -3
- package/docs/rules/require-ticket.md +38 -7
- package/docs/upgrading.md +2 -2
- package/package.json +5 -2
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,30 @@ All notable changes to this project are documented here. The format follows
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.7.0] - 2026-10-20
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- Mocha, Cypress and WebdriverIO support. `configs.mocha`, `configs.cypress` and `configs.webdriverio`,
|
|
14
|
+
or `configure({ framework })`, check those frameworks' tests with the same rules and markers:
|
|
15
|
+
`it.skip`, `describe.skip`, `context.skip`, `xit`, `xdescribe` and the other `x` forms, the TDD
|
|
16
|
+
interface's `suite.skip` and `test.skip`, with globals or imports from `mocha`, and `this.skip()` in a
|
|
17
|
+
test or hook written as a `function`. Under a condition, as in `if (!broker) this.skip()`, it is a
|
|
18
|
+
conditional skip, so it needs no ticket by default. Cypress and WebdriverIO run their tests on Mocha,
|
|
19
|
+
so their configs read the same calls; `webdriverio` is for WebdriverIO's Mocha framework.
|
|
20
|
+
- A Mocha test with only a title, `it('settles at T+1')`, which Mocha runs as pending, is in the `todo`
|
|
21
|
+
state and needs a `// TODO:` marker, like Jest's `test.todo`. `it.skip('title')` with no body is a
|
|
22
|
+
skip.
|
|
23
|
+
- In Cypress files, tags from @cypress/grep's `{ tags: '@smoke' }` count like tags in the title, and
|
|
24
|
+
`{ browser: '!webkit' }`, which runs the test only in some browsers, is a conditional skip.
|
|
25
|
+
|
|
26
|
+
### Fixed
|
|
27
|
+
|
|
28
|
+
- A runtime skip after other code in a `try` block that has a `catch`, as in
|
|
29
|
+
`try { await stat(target); test.skip(); } catch {}`, is a conditional skip: an error before it jumps to
|
|
30
|
+
the `catch`, so the test runs. Before, it needed a ticket like an unconditional skip. A skip that starts
|
|
31
|
+
the `try` block, or one in a `try` with only `finally`, still does. This covers Mocha's `this.skip()` too.
|
|
32
|
+
|
|
9
33
|
## [0.6.2] - 2026-10-14
|
|
10
34
|
|
|
11
35
|
### Fixed
|
|
@@ -305,7 +329,8 @@ First release.
|
|
|
305
329
|
- Support for ESLint 9 and 10 flat config, JavaScript and TypeScript test files, `.extend()`
|
|
306
330
|
fixtures, describe blocks, runtime skips and tags in titles or the `tag` option.
|
|
307
331
|
|
|
308
|
-
[Unreleased]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.
|
|
332
|
+
[Unreleased]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.7.0...HEAD
|
|
333
|
+
[0.7.0]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.6.2...v0.7.0
|
|
309
334
|
[0.6.2]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.6.1...v0.6.2
|
|
310
335
|
[0.6.1]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.6.0...v0.6.1
|
|
311
336
|
[0.6.0]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.5.1...v0.6.0
|
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
[](https://github.com/Huldoser/eslint-plugin-test-governance/actions/workflows/ci.yml)
|
|
5
5
|
[](LICENSE)
|
|
6
6
|
|
|
7
|
-
ESLint rules that keep a Playwright, Jest or
|
|
7
|
+
ESLint rules that keep a Playwright, Jest, Vitest, Mocha, Cypress or WebdriverIO suite honest. Every skipped, fixme, todo, `@new`
|
|
8
8
|
or `@quarantined` test must point at the ticket that tracks it:
|
|
9
9
|
|
|
10
10
|
<!-- example: valid -->
|
|
@@ -34,8 +34,9 @@ Requires ESLint 9 or 10 with flat config, and Node.js 22.12 or later. For TypeSc
|
|
|
34
34
|
|
|
35
35
|
## Usage
|
|
36
36
|
|
|
37
|
-
Pick the config for the framework your tests use, `configs.playwright`, `configs.jest
|
|
38
|
-
`configs.vitest`, and point it at those
|
|
37
|
+
Pick the config for the framework your tests use, `configs.playwright`, `configs.jest`,
|
|
38
|
+
`configs.vitest`, `configs.mocha`, `configs.cypress` or `configs.webdriverio`, and point it at those
|
|
39
|
+
test files with `files`:
|
|
39
40
|
|
|
40
41
|
```js
|
|
41
42
|
// eslint.config.js
|
|
@@ -52,9 +53,14 @@ export default [
|
|
|
52
53
|
```
|
|
53
54
|
|
|
54
55
|
There is no default config. Each framework declares, skips and tags tests its own way, so the rules
|
|
55
|
-
need to know which one a file uses. In a Jest or
|
|
56
|
-
that framework, whether they are globals or imported from `@jest/globals` or `
|
|
57
|
-
Jest and
|
|
56
|
+
need to know which one a file uses. In a Jest, Vitest or Mocha config, `test`, `it` and `describe` belong
|
|
57
|
+
to that framework, whether they are globals or imported from `@jest/globals`, `vitest` or `mocha`, the
|
|
58
|
+
way the Jest, Vitest and Mocha ESLint plugins are set up.
|
|
59
|
+
|
|
60
|
+
Cypress and WebdriverIO run their tests on Mocha, so `configs.cypress` and `configs.webdriverio` read the
|
|
61
|
+
same calls as `configs.mocha`. `webdriverio` means WebdriverIO's Mocha framework; its Jasmine framework
|
|
62
|
+
isn't supported yet. Puppeteer has no test functions of its own, so use the config of the runner its
|
|
63
|
+
tests run on: `configs.jest` for jest-puppeteer, or `configs.vitest` or `configs.mocha`.
|
|
58
64
|
|
|
59
65
|
With ESLint's `defineConfig()` (ESLint 9.22 or later), put the config in `extends`:
|
|
60
66
|
|
|
@@ -125,26 +131,29 @@ export default [
|
|
|
125
131
|
];
|
|
126
132
|
```
|
|
127
133
|
|
|
128
|
-
|
|
129
|
-
`configure()` for the same files keeps that config's options and changes only the framework.
|
|
134
|
+
Each framework's config, such as `configs.playwright`, sets only the framework, so one listed after a
|
|
135
|
+
`configure()` for the same files keeps that config's options and changes only the framework. They all
|
|
130
136
|
turn on the same rules. What puts a test in a state in each framework:
|
|
131
137
|
|
|
132
|
-
| State (marker) | Playwright | Jest | Vitest |
|
|
133
|
-
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
134
|
-
| `skip` (`// SKIP:`) | `test.skip(...)`, `test.describe.skip`, `test.skip()` in a body, `testInfo.skip()`, `test.step.skip`, `step.skip()` | `test.skip`, `it.skip`, `describe.skip`, `xit`, `xtest`, `xdescribe` | `test.skip`, `it.skip`, `describe.skip`, `suite.skip`, `skipIf(true)`, `runIf(false)`, `{ skip: true }`, `context.skip()`, `context.skip('note')` or `skip()` in a body or in `beforeEach` |
|
|
135
|
-
| `todo` (`// TODO:`) | | `test.todo`, `it.todo` | `test.todo`, `describe.todo`, `{ todo: true }` |
|
|
136
|
-
| `fixme` (`// FIXME:`) | `test.fixme(...)`, `test.describe.fixme`, `testInfo.fixme()` | | |
|
|
137
|
-
| `fail` (`// FAIL:`), off by default | `test.fail(...)`, `testInfo.fail()` | `test.failing` | `test.fails`, `{ fails: true }` |
|
|
138
|
-
| `slow` (`// SLOW:`), off by default | `test.slow()`, `testInfo.slow()` | | |
|
|
139
|
-
| conditional skip, no ticket by default | `test.skip(condition)`, `if (...) test.skip()`, `(condition ? test.skip : test)(...)` | `(condition ? it.skip : it)(...)` | `skipIf(condition)`, `runIf(condition)`, `{ skip: condition }`, `context.skip(condition)`, `if (...) context.skip()`, `(condition ? it.skip : it)(...)` |
|
|
140
|
-
| tags such as `@new` (`// NEW:`) | the title, `{ tag }` | the title | the title, `{ tags }`, where `'new'` and `'@new'` both mean `@new` |
|
|
138
|
+
| State (marker) | Playwright | Jest | Vitest | Mocha | Cypress | WebdriverIO |
|
|
139
|
+
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | ----------- |
|
|
140
|
+
| `skip` (`// SKIP:`) | `test.skip(...)`, `test.describe.skip`, `test.skip()` in a body, `testInfo.skip()`, `test.step.skip`, `step.skip()` | `test.skip`, `it.skip`, `describe.skip`, `xit`, `xtest`, `xdescribe` | `test.skip`, `it.skip`, `describe.skip`, `suite.skip`, `skipIf(true)`, `runIf(false)`, `{ skip: true }`, `context.skip()`, `context.skip('note')` or `skip()` in a body or in `beforeEach` | `it.skip`, `specify.skip`, `describe.skip`, `context.skip`, `xit`, `xspecify`, `xdescribe`, `xcontext`, TDD `test.skip` and `suite.skip`, `this.skip()` in a test or hook | as Mocha | as Mocha |
|
|
141
|
+
| `todo` (`// TODO:`) | | `test.todo`, `it.todo` | `test.todo`, `describe.todo`, `{ todo: true }` | a test with only a title, `it('title')`, which Mocha runs as pending | as Mocha | as Mocha |
|
|
142
|
+
| `fixme` (`// FIXME:`) | `test.fixme(...)`, `test.describe.fixme`, `testInfo.fixme()` | | | | | |
|
|
143
|
+
| `fail` (`// FAIL:`), off by default | `test.fail(...)`, `testInfo.fail()` | `test.failing` | `test.fails`, `{ fails: true }` | | | |
|
|
144
|
+
| `slow` (`// SLOW:`), off by default | `test.slow()`, `testInfo.slow()` | | | | | |
|
|
145
|
+
| conditional skip, no ticket by default | `test.skip(condition)`, `if (...) test.skip()`, `(condition ? test.skip : test)(...)` | `(condition ? it.skip : it)(...)` | `skipIf(condition)`, `runIf(condition)`, `{ skip: condition }`, `context.skip(condition)`, `if (...) context.skip()`, `(condition ? it.skip : it)(...)` | `if (...) this.skip()`, `(condition ? it.skip : it)(...)` | as Mocha, and `{ browser: '!webkit' }` | as Mocha |
|
|
146
|
+
| tags such as `@new` (`// NEW:`) | the title, `{ tag }` | the title | the title, `{ tags }`, where `'new'` and `'@new'` both mean `@new` | the title | the title, `{ tags }` from @cypress/grep | the title |
|
|
141
147
|
|
|
142
148
|
Table-driven tests count too: `test.skip.each(table)(...)`, `describe.skip.each` and Vitest's `.for`.
|
|
143
149
|
A test or describe whose function a condition picks, as in `(isMobile ? test.skip : test)('title', fn)`,
|
|
144
150
|
is a conditional skip in every framework.
|
|
145
151
|
A literal condition decides on its own: `skipIf(true)` and `runIf(false)` always skip, and `skipIf(false)`,
|
|
146
152
|
`runIf(true)` and `{ skip: false }` never do.
|
|
147
|
-
In Vitest, `test.skip('title')` and `describe.skip('title')` with no body are skipped too.
|
|
153
|
+
In Vitest and Mocha, `test.skip('title')` and `describe.skip('title')` with no body are skipped too.
|
|
154
|
+
In Mocha, `this.skip()` counts in a test or hook written as a `function`; an arrow function gets no
|
|
155
|
+
Mocha context, so its `this` is not the test's. `this.slow(ms)` only sets Mocha's speed threshold, so
|
|
156
|
+
it isn't the `slow` state.
|
|
148
157
|
Vitest writes tag names without the `@`, as in `{ tags: ['flaky'] }`, so a name in `tags` counts as the
|
|
149
158
|
tag with an `@` in front: `'flaky'` is `@flaky`, and a custom state with `when: '@flaky'` matches it.
|
|
150
159
|
|
|
@@ -154,19 +163,22 @@ tag with an `@` in front: `'flaky'` is `@flaky`, and a custom state with `when:
|
|
|
154
163
|
|
|
155
164
|
💼 [Configurations](https://github.com/Huldoser/eslint-plugin-test-governance#usage) enabled in.\
|
|
156
165
|
⚠️ [Configurations](https://github.com/Huldoser/eslint-plugin-test-governance#usage) set to warn in.\
|
|
166
|
+
🌲 Set in the `cypress` [configuration](https://github.com/Huldoser/eslint-plugin-test-governance#usage).\
|
|
157
167
|
🃏 Set in the `jest` [configuration](https://github.com/Huldoser/eslint-plugin-test-governance#usage).\
|
|
168
|
+
☕ Set in the `mocha` [configuration](https://github.com/Huldoser/eslint-plugin-test-governance#usage).\
|
|
158
169
|
🎭 Set in the `playwright` [configuration](https://github.com/Huldoser/eslint-plugin-test-governance#usage).\
|
|
159
170
|
⚡ Set in the `vitest` [configuration](https://github.com/Huldoser/eslint-plugin-test-governance#usage).\
|
|
171
|
+
🤖 Set in the `webdriverio` [configuration](https://github.com/Huldoser/eslint-plugin-test-governance#usage).\
|
|
160
172
|
🔧 Automatically fixable by the [`--fix` CLI option](https://eslint.org/docs/user-guide/command-line-interface#--fix).\
|
|
161
173
|
💡 Manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
162
174
|
|
|
163
|
-
| Name | Description | 💼
|
|
164
|
-
| :--------------------------------------------------------------------- | :-------------------------------------------------------------------------------- |
|
|
165
|
-
| [marker-matches-state](docs/rules/marker-matches-state.md) | Require the marker keyword to match the test's state | 🃏 🎭 ⚡ |
|
|
166
|
-
| [no-conflicting-states](docs/rules/no-conflicting-states.md) | Disallow state tags that contradict each other, differ in case or look like typos |
|
|
167
|
-
| [no-orphaned-marker](docs/rules/no-orphaned-marker.md) | Disallow marker comments that no longer match a test state | 🃏 🎭 ⚡ |
|
|
168
|
-
| [require-ticket](docs/rules/require-ticket.md) | Require a ticket marker comment above skipped, fixme and tagged tests | 🃏 🎭 ⚡ |
|
|
169
|
-
| [require-ticket-in-comments](docs/rules/require-ticket-in-comments.md) | Require FIXME and TODO comments in test files to start with a ticket | 🃏 🎭 ⚡ |
|
|
175
|
+
| Name | Description | 💼 | ⚠️ | 🔧 | 💡 |
|
|
176
|
+
| :--------------------------------------------------------------------- | :-------------------------------------------------------------------------------- | :---------------- | :---------------- | :-- | :-- |
|
|
177
|
+
| [marker-matches-state](docs/rules/marker-matches-state.md) | Require the marker keyword to match the test's state | 🌲 🃏 ☕ 🎭 ⚡ 🤖 | | | 💡 |
|
|
178
|
+
| [no-conflicting-states](docs/rules/no-conflicting-states.md) | Disallow state tags that contradict each other, differ in case or look like typos | | 🌲 🃏 ☕ 🎭 ⚡ 🤖 | 🔧 | |
|
|
179
|
+
| [no-orphaned-marker](docs/rules/no-orphaned-marker.md) | Disallow marker comments that no longer match a test state | 🌲 🃏 ☕ 🎭 ⚡ 🤖 | | | 💡 |
|
|
180
|
+
| [require-ticket](docs/rules/require-ticket.md) | Require a ticket marker comment above skipped, fixme and tagged tests | 🌲 🃏 ☕ 🎭 ⚡ 🤖 | | | 💡 |
|
|
181
|
+
| [require-ticket-in-comments](docs/rules/require-ticket-in-comments.md) | Require FIXME and TODO comments in test files to start with a ticket | 🌲 🃏 ☕ 🎭 ⚡ 🤖 | | | 💡 |
|
|
170
182
|
|
|
171
183
|
<!-- end auto-generated rules list -->
|
|
172
184
|
|
|
@@ -176,7 +188,7 @@ The defaults follow how most teams already work, so each framework config is use
|
|
|
176
188
|
|
|
177
189
|
| Setting | Default | Why |
|
|
178
190
|
| --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
|
179
|
-
| States that need a ticket | `skip`, `fixme`; in
|
|
191
|
+
| States that need a ticket | `skip`, `fixme`; in the other frameworks `skip`, `todo` | Every team skips tests. Chromium DevTools, GitLab and Slack all require a bug link on disabled tests. |
|
|
180
192
|
| `@new`, `@quarantined` | off; `lifecycleTags: true` turns both on | A promotion workflow is not universal. Teams that say `@flaky` instead [rename the tag](#tag-names). |
|
|
181
193
|
| Conditional skips (`test.skip(browserName === 'webkit', ...)`, `if (...) test.skip()`, a skip in `catch`) | no ticket needed | These are usually permanent platform limits, not bugs. `requireTicketForConditional: true` changes this. |
|
|
182
194
|
| `test.fail`, `test.slow`, Jest's `test.failing`, Vitest's `test.fails` | off | They don't remove coverage. Turn them on with `states: { fail: true, slow: true }`. |
|
|
@@ -196,20 +208,20 @@ yourself. `framework` is required, and every other option has a default. The rul
|
|
|
196
208
|
of their own: they all read these shared settings, so they always agree on ticket formats, states
|
|
197
209
|
and markers. To turn a rule off, set it to `'off'` as usual.
|
|
198
210
|
|
|
199
|
-
| Option | Type
|
|
200
|
-
| ----------------------------- |
|
|
201
|
-
| `framework` | `'playwright' \| 'jest' \| 'vitest'`
|
|
202
|
-
| `ticket` | `TicketSpec \| TicketSpec[]`
|
|
203
|
-
| `placeholders` | `string[]`
|
|
204
|
-
| `lifecycleTags` | `boolean`
|
|
205
|
-
| `states` | `{ [state]: boolean \| { enabled?, tag?, marker?, ticket? } }`
|
|
206
|
-
| `customStates` | `{ [name]: { when, marker, ticket? } }`
|
|
207
|
-
| `requireTicketForConditional` | `boolean`
|
|
208
|
-
| `allowBlankLine` | `boolean`
|
|
209
|
-
| `allowNotes` | `boolean`
|
|
210
|
-
| `comments` | `false \| { keywords?: string[] }`
|
|
211
|
-
| `testFunctions` | `string[]`
|
|
212
|
-
| `reportUnreadableTags` | `boolean`
|
|
211
|
+
| Option | Type | Default | Description |
|
|
212
|
+
| ----------------------------- | ----------------------------------------------------------------------------- | --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
213
|
+
| `framework` | `'playwright' \| 'jest' \| 'vitest' \| 'mocha' \| 'cypress' \| 'webdriverio'` | required | The test framework of the files the config applies to. Each framework's config, such as `configs.playwright`, sets it, and `configure()` needs it. See [Frameworks](#frameworks). |
|
|
214
|
+
| `ticket` | `TicketSpec \| TicketSpec[]` | `{ preset: 'any' }` | Accepted ticket formats. An array accepts a ticket that matches any entry. See [Ticket presets](#ticket-presets). |
|
|
215
|
+
| `placeholders` | `string[]` | see above | Tickets rejected as placeholders. `*` matches any characters. Case-insensitive; a leading `#` is ignored. Your list replaces the default one, so repeat the defaults you want to keep. Tickets numbered 0 are always rejected. |
|
|
216
|
+
| `lifecycleTags` | `boolean` | `false` | Turns on the `new` (`@new`) and `quarantined` (`@quarantined`) states. |
|
|
217
|
+
| `states` | `{ [state]: boolean \| { enabled?, tag?, marker?, ticket? } }` | `skip`, `fixme` and `todo` on | Turns built-in states (`skip`, `fixme`, `fail`, `slow`, `todo`, `new`, `quarantined`) on or off, renames their tag or marker, or gives them their own ticket format. `fail` and `slow` are off by default; a setting for `new` or `quarantined` here wins over `lifecycleTags`. Only `new` and `quarantined` have a tag to rename; see [Tag names](#tag-names). `fixme` and `slow` exist only in Playwright, `todo` in every framework but Playwright, and `fail` in Playwright, Jest and Vitest, so turning one on elsewhere has no effect. |
|
|
218
|
+
| `customStates` | `{ [name]: { when, marker, ticket? } }` | `{}` | Adds a state for a tag, e.g. `{ 'needs-data': { when: '@needs-data', marker: 'NEEDS-DATA' } }`. `when` is a tag and `marker` an uppercase keyword; neither can be shared with another state. |
|
|
219
|
+
| `requireTicketForConditional` | `boolean` | `false` | Require a ticket for conditional skips too: `test.skip(condition, ...)`, `if (...) test.skip()`, `(condition ? test.skip : test)(...)`, a skip in a `catch` block, after other code in a `try` block with a `catch`, or after an early `return` or `throw`. |
|
|
220
|
+
| `allowBlankLine` | `boolean` | `false` | Let blank lines separate a marker from its test. |
|
|
221
|
+
| `allowNotes` | `boolean` | `false` | Allow free text after the tickets, e.g. `// SKIP: TRADE-123 flaky on CI`. Applies to markers and to `FIXME` / `TODO` comments. |
|
|
222
|
+
| `comments` | `false \| { keywords?: string[] }` | `{ keywords: ['FIXME', 'TODO'] }` | Comment keywords that must start with a ticket anywhere in a test file, e.g. `// TODO: TRADE-123`. Your keywords replace the default ones. `false` allows free-form comments. |
|
|
223
|
+
| `testFunctions` | `string[]` | `['test']` | Names treated as the framework's `test`. Jest and Vitest default to `['test', 'it']`, and Mocha, Cypress and WebdriverIO to `['it', 'specify', 'test']`. Your list replaces the default one, so include those names if you still need them. See below. |
|
|
224
|
+
| `reportUnreadableTags` | `boolean` | `false` | When a tag state is on, report titles that aren't static text, and tags or details that can't be read in the file, such as ones imported from another module, since their tags can't be checked. |
|
|
213
225
|
|
|
214
226
|
### Tag names
|
|
215
227
|
|
|
@@ -293,6 +305,14 @@ In Jest and Vitest files:
|
|
|
293
305
|
- Vitest fixtures made with `test.extend()` in the same file.
|
|
294
306
|
- Any name in `testFunctions`, which defaults to `['test', 'it']`.
|
|
295
307
|
|
|
308
|
+
In Mocha, Cypress and WebdriverIO files:
|
|
309
|
+
|
|
310
|
+
- The globals `it`, `specify`, `describe` and `context`, their `x` forms, the TDD names `suite` and
|
|
311
|
+
`test`, and the hooks `before`, `beforeEach`, `after`, `afterEach`, `suiteSetup`, `setup`,
|
|
312
|
+
`suiteTeardown` and `teardown`.
|
|
313
|
+
- The names imported or required from `mocha`, including aliases and `import * as mocha from 'mocha'`.
|
|
314
|
+
- Any name in `testFunctions`, which defaults to `['it', 'specify', 'test']`.
|
|
315
|
+
|
|
296
316
|
A name imported or required from another test runner (Playwright, `vitest`, `@jest/globals`,
|
|
297
317
|
`node:test`, `bun:test`, `mocha`, `ava`, `tap`, `uvu`), or a variable set from one such as
|
|
298
318
|
`const test = require('ava')`, is never treated as the framework's `test`, and neither is a local
|
|
@@ -321,7 +341,8 @@ The [Frameworks](#frameworks) table lists what each framework's tests can do. In
|
|
|
321
341
|
Not covered in this version: tests generated in loops with dynamic titles, and tags or details imported
|
|
322
342
|
from other modules (see `reportUnreadableTags`), fixtures imported from other modules under a name not in
|
|
323
343
|
`testFunctions`, a Vitest test with only a title, which Vitest runs as a todo but which looks the same as a
|
|
324
|
-
helper called `test`, Playwright annotations as markers,
|
|
344
|
+
helper called `test`, Playwright annotations as markers, Cypress tests skipped with @cypress/skip-test's
|
|
345
|
+
`skipOn()` and `onlyOn()`, and other runners such as Jasmine, Bun and `node:test`.
|
|
325
346
|
|
|
326
347
|
## TypeScript
|
|
327
348
|
|
package/dist/index.d.ts
CHANGED
|
@@ -13,8 +13,8 @@ export interface FlatConfig {
|
|
|
13
13
|
settings?: Record<string, unknown>;
|
|
14
14
|
}
|
|
15
15
|
/**
|
|
16
|
-
* One config per framework.
|
|
17
|
-
*
|
|
16
|
+
* One config per framework. Each turns on the same rules for that framework's tests; point each one at its
|
|
17
|
+
* test files with `files`.
|
|
18
18
|
*/
|
|
19
19
|
export type ConfigName = FrameworkName;
|
|
20
20
|
export interface TestGovernancePlugin extends ESLint.Plugin {
|
|
@@ -33,5 +33,5 @@ declare const plugin: TestGovernancePlugin;
|
|
|
33
33
|
* so all rules read the same ticket formats and states.
|
|
34
34
|
*/
|
|
35
35
|
export declare function configure(options: GovernanceOptions): FlatConfig;
|
|
36
|
-
export declare const configs: Record<"playwright" | "jest" | "vitest", FlatConfig>;
|
|
36
|
+
export declare const configs: Record<"playwright" | "jest" | "vitest" | "mocha" | "cypress" | "webdriverio", FlatConfig>;
|
|
37
37
|
export default plugin;
|
package/dist/index.js
CHANGED
|
@@ -52,7 +52,7 @@ for (const framework of FRAMEWORK_NAMES) {
|
|
|
52
52
|
// tools that list the configs don't see it.
|
|
53
53
|
Object.defineProperty(plugin.configs, 'recommended', {
|
|
54
54
|
get() {
|
|
55
|
-
throw new ConfigError('configs.recommended was removed in 0.5.0. Use the config for your framework: configs.playwright, configs.jest or configs.
|
|
55
|
+
throw new ConfigError('configs.recommended was removed in 0.5.0. Use the config for your framework: configs.playwright, configs.jest, configs.vitest, configs.mocha, configs.cypress or configs.webdriverio. See https://github.com/Huldoser/eslint-plugin-test-governance/blob/main/docs/upgrading.md');
|
|
56
56
|
},
|
|
57
57
|
});
|
|
58
58
|
export const configs = plugin.configs;
|
package/dist/utils/analyze.js
CHANGED
|
@@ -492,10 +492,17 @@ function collectTags(sourceCode, call, hasTitle, framework, details) {
|
|
|
492
492
|
unread.push(details.unread);
|
|
493
493
|
return { tags, dynamic: title.unread.size > 0, unread, opaqueDetails: details.unread !== undefined };
|
|
494
494
|
}
|
|
495
|
+
/** Whether a title is text, written out or held in a constant declared in this file. */
|
|
496
|
+
function isTextTitle(sourceCode, node) {
|
|
497
|
+
const title = unwrap(node);
|
|
498
|
+
const value = constantValue(sourceCode, title);
|
|
499
|
+
return isStaticText(title) || (value !== undefined && isStaticText(unwrap(value)));
|
|
500
|
+
}
|
|
495
501
|
/**
|
|
496
502
|
* Whether a call declares a test or describe block: a title and a body, as in `test('title', fn)`,
|
|
497
503
|
* `test('title', details, fn)` or `test.skip(title, payWithCard)` with the body defined elsewhere, a
|
|
498
|
-
* describe with only a body, or a `todo` with only a title,
|
|
504
|
+
* describe with only a body, or a `todo` with only a title, in Vitest and Mocha a `skip` with only a
|
|
505
|
+
* title, and in Mocha a test with only a title written out, which it runs as pending.
|
|
499
506
|
* Only Playwright has a runtime `test.skip(isMobile, 'why')`; there a title, written out or held in a
|
|
500
507
|
* constant, or a body written in place tells the two apart.
|
|
501
508
|
*/
|
|
@@ -505,13 +512,22 @@ function isDeclaration(sourceCode, framework, kind, args, members) {
|
|
|
505
512
|
return kind === 'describe';
|
|
506
513
|
if (first === undefined)
|
|
507
514
|
return false;
|
|
508
|
-
if (args.length === 1)
|
|
509
|
-
|
|
515
|
+
if (args.length === 1) {
|
|
516
|
+
if (members.includes('todo') || (framework.skipWithoutBody && members.includes('skip')))
|
|
517
|
+
return true;
|
|
518
|
+
return framework.todoWithoutBody && kind === 'test' && isTextTitle(sourceCode, first);
|
|
519
|
+
}
|
|
510
520
|
if (!framework.playwright)
|
|
511
521
|
return true;
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
522
|
+
return isTextTitle(sourceCode, first) || args.slice(1).some(isFunction);
|
|
523
|
+
}
|
|
524
|
+
/** The function whose `this` a call sees: the nearest one around it that isn't an arrow function. */
|
|
525
|
+
function thisFunction(node) {
|
|
526
|
+
let current = node.parent;
|
|
527
|
+
while (current && (current.type === 'ArrowFunctionExpression' || !FUNCTIONS.has(current.type))) {
|
|
528
|
+
current = current.parent;
|
|
529
|
+
}
|
|
530
|
+
return current;
|
|
515
531
|
}
|
|
516
532
|
/** The value of a `true` or `false` literal; undefined for anything else. */
|
|
517
533
|
function booleanOf(node) {
|
|
@@ -540,8 +556,8 @@ function isEarlyExit(statement) {
|
|
|
540
556
|
}
|
|
541
557
|
/**
|
|
542
558
|
* Whether the call only runs on some paths of its function: under an `if`, a `switch` case, a ternary,
|
|
543
|
-
* the right side of `&&`, `||` or `??`, in a `catch` block,
|
|
544
|
-
* `if (ready) return;`. `if (!enabled) test.skip()` is the same as `test.skip(!enabled)`, and
|
|
559
|
+
* the right side of `&&`, `||` or `??`, in a `catch` block, after other code in a `try` block that has a
|
|
560
|
+
* `catch`, or after an early exit such as `if (ready) return;`. `if (!enabled) test.skip()` is the same as `test.skip(!enabled)`, and
|
|
545
561
|
* `catch { test.skip(true, 'service is down') }` only skips when the service is down.
|
|
546
562
|
*/
|
|
547
563
|
function isGuarded(call) {
|
|
@@ -559,6 +575,10 @@ function isGuarded(call) {
|
|
|
559
575
|
const index = node.body.indexOf(child);
|
|
560
576
|
if (node.body.slice(0, index).some(isEarlyExit))
|
|
561
577
|
return true;
|
|
578
|
+
// `try { await stat(target); test.skip(); } catch {}`: an error before the skip jumps to the catch.
|
|
579
|
+
const owner = node.parent;
|
|
580
|
+
if (index > 0 && owner.type === 'TryStatement' && owner.block === node && owner.handler)
|
|
581
|
+
return true;
|
|
562
582
|
}
|
|
563
583
|
}
|
|
564
584
|
return false;
|
|
@@ -617,6 +637,8 @@ function runAnalysis(sourceCode, options) {
|
|
|
617
637
|
const keys = sourceCode.visitorKeys;
|
|
618
638
|
/** States an options object can put a test in, such as `skip` for Vitest's `{ skip: true }`. */
|
|
619
639
|
const optionModifiers = new Set(framework.optionEffects.values());
|
|
640
|
+
/** Bodies of Mocha tests and hooks written as a `function`, where `this.skip()` skips at runtime. */
|
|
641
|
+
const thisBodies = new Set();
|
|
620
642
|
/**
|
|
621
643
|
* `modifiers` maps each state modifier the call has to whether it always applies; a conditional one,
|
|
622
644
|
* like `test.skip(isMobile)` or Vitest's `skipIf(...)`, needs a ticket only with `requireTicketForConditional`.
|
|
@@ -722,6 +744,22 @@ function runAnalysis(sourceCode, options) {
|
|
|
722
744
|
}
|
|
723
745
|
return modifiers;
|
|
724
746
|
};
|
|
747
|
+
/**
|
|
748
|
+
* The modifiers of a declaration, with `todo` for a Mocha test that has only a title, which Mocha runs as
|
|
749
|
+
* pending: someone has yet to write it. A skipped one, as `it.skip('title')`, stays a skip.
|
|
750
|
+
*/
|
|
751
|
+
const declarationModifiers = (kind, args, modifiers) => {
|
|
752
|
+
if (framework.todoWithoutBody && kind === 'test' && args.length === 1 && !modifiers.has('skip')) {
|
|
753
|
+
modifiers.set('todo', true);
|
|
754
|
+
}
|
|
755
|
+
return modifiers;
|
|
756
|
+
};
|
|
757
|
+
/** Keeps the body of a Mocha test or hook written as a `function`, whose `this` can skip it. */
|
|
758
|
+
const addThisBody = (args) => {
|
|
759
|
+
const body = args.findLast(isFunction);
|
|
760
|
+
if (framework.thisSkip && body?.type === 'FunctionExpression')
|
|
761
|
+
thisBodies.add(body);
|
|
762
|
+
};
|
|
725
763
|
/** What a body can change the test's state with: Playwright's `testInfo` or Vitest's test context. */
|
|
726
764
|
const bodyInfos = (args, path) => {
|
|
727
765
|
const body = args.findLast(isFunction);
|
|
@@ -767,7 +805,9 @@ function runAnalysis(sourceCode, options) {
|
|
|
767
805
|
usesTest = true;
|
|
768
806
|
const details = detailsOf(sourceCode, call);
|
|
769
807
|
const tagInfo = collectTags(sourceCode, call, !isFunction(args[0]), framework, details);
|
|
770
|
-
const [ifYes, ifNo] = [yes, no].map(({ members, args: conditions }) => modifiersOf(members, details.entries, conditions));
|
|
808
|
+
const [ifYes, ifNo] = [yes, no].map(({ members, args: conditions }) => declarationModifiers(yes.kind, args, modifiersOf(members, details.entries, conditions)));
|
|
809
|
+
if (yes.kind === 'test')
|
|
810
|
+
addThisBody(args);
|
|
771
811
|
const modifiers = new Map();
|
|
772
812
|
for (const modifier of new Set([...ifYes.keys(), ...ifNo.keys()])) {
|
|
773
813
|
modifiers.set(modifier, ifYes.get(modifier) === true && ifNo.get(modifier) === true);
|
|
@@ -782,9 +822,12 @@ function runAnalysis(sourceCode, options) {
|
|
|
782
822
|
const args = call.arguments;
|
|
783
823
|
const { path } = callee;
|
|
784
824
|
const declaration = declarationOf(callee);
|
|
785
|
-
// Vitest's `beforeEach((context) => context.skip())`
|
|
786
|
-
|
|
825
|
+
// Vitest's `beforeEach((context) => context.skip())` and Mocha's `before(function () { this.skip(); })`
|
|
826
|
+
// skip the tests they run for.
|
|
827
|
+
if (declaration === undefined) {
|
|
828
|
+
addThisBody(args);
|
|
787
829
|
return { infos: bodyInfos(args, path) };
|
|
830
|
+
}
|
|
788
831
|
if (framework.playwright) {
|
|
789
832
|
// `test.step('fills the order', async (step) => {...})` or `test.step.skip(...)`. The step's body
|
|
790
833
|
// gets a `TestStepInfo` that can skip the step at runtime.
|
|
@@ -802,10 +845,10 @@ function runAnalysis(sourceCode, options) {
|
|
|
802
845
|
// A declaration always has a first argument: its title, or the body of a describe without one.
|
|
803
846
|
const details = detailsOf(sourceCode, call);
|
|
804
847
|
const tagInfo = collectTags(sourceCode, call, !isFunction(args[0]), framework, details);
|
|
805
|
-
|
|
806
|
-
|
|
807
|
-
|
|
808
|
-
};
|
|
848
|
+
const modifiers = declarationModifiers(kind, args, modifiersOf(members, details.entries, callee.args));
|
|
849
|
+
if (kind === 'test')
|
|
850
|
+
addThisBody(args);
|
|
851
|
+
return { subject: makeSubject(kind, call, modifiers, tagInfo), infos: bodyInfo };
|
|
809
852
|
}
|
|
810
853
|
// test.skip(), test.fixme(), test.fail() and test.slow() in a test, a describe or a hook.
|
|
811
854
|
if (framework.playwright && members.length === 1 && RUNTIME_MODIFIERS.has(members[0])) {
|
|
@@ -813,7 +856,10 @@ function runAnalysis(sourceCode, options) {
|
|
|
813
856
|
}
|
|
814
857
|
return { infos: bodyInfo };
|
|
815
858
|
};
|
|
816
|
-
/**
|
|
859
|
+
/**
|
|
860
|
+
* `testInfo.skip()`, `step.skip()`, `test.info().skip()`, Vitest's `context.skip()` or `skip()`, and
|
|
861
|
+
* Mocha's `this.skip()`.
|
|
862
|
+
*/
|
|
817
863
|
const runtimeCall = (call) => {
|
|
818
864
|
const { callee } = call;
|
|
819
865
|
// The innermost parameter with a name is the one the call uses.
|
|
@@ -828,6 +874,11 @@ function runAnalysis(sourceCode, options) {
|
|
|
828
874
|
if (name === undefined || !RUNTIME_MODIFIERS.has(name))
|
|
829
875
|
return undefined;
|
|
830
876
|
const { object } = callee;
|
|
877
|
+
// Mocha's `this.skip()`, in a test or hook whose `function` gives it the test's `this`.
|
|
878
|
+
if (object.type === 'ThisExpression') {
|
|
879
|
+
const owner = name === 'skip' ? thisFunction(call) : undefined;
|
|
880
|
+
return owner !== undefined && thisBodies.has(owner) ? runtimeSubject(call, name) : undefined;
|
|
881
|
+
}
|
|
831
882
|
if (object.type === 'Identifier') {
|
|
832
883
|
const info = innermost(object.name);
|
|
833
884
|
return info?.methods?.has(name) ? runtimeSubject(call, name, info.step) : undefined;
|
|
@@ -8,7 +8,7 @@ export type BuiltinStateName = (typeof BUILTIN_STATE_NAMES)[number];
|
|
|
8
8
|
export declare const TAG_STATE_NAMES: readonly ["new", "quarantined"];
|
|
9
9
|
export type TagStateName = (typeof TAG_STATE_NAMES)[number];
|
|
10
10
|
/** The test frameworks the rules understand. */
|
|
11
|
-
export declare const FRAMEWORK_NAMES: readonly ["playwright", "jest", "vitest"];
|
|
11
|
+
export declare const FRAMEWORK_NAMES: readonly ["playwright", "jest", "vitest", "mocha", "cypress", "webdriverio"];
|
|
12
12
|
export type FrameworkName = (typeof FRAMEWORK_NAMES)[number];
|
|
13
13
|
export declare const TICKET_PRESETS: readonly ["any", "jira", "github", "gitlab", "linear", "azure-devops", "numeric", "pattern"];
|
|
14
14
|
/** Comment keywords that must reference a ticket by default (`// TODO: TRADE-123`). */
|
package/dist/utils/constants.js
CHANGED
|
@@ -6,7 +6,7 @@ export const BUILTIN_STATE_NAMES = ['skip', 'fixme', 'fail', 'slow', 'todo', 'ne
|
|
|
6
6
|
/** The built-in states a tag puts a test in, so the only ones whose tag can be renamed. */
|
|
7
7
|
export const TAG_STATE_NAMES = ['new', 'quarantined'];
|
|
8
8
|
/** The test frameworks the rules understand. */
|
|
9
|
-
export const FRAMEWORK_NAMES = ['playwright', 'jest', 'vitest'];
|
|
9
|
+
export const FRAMEWORK_NAMES = ['playwright', 'jest', 'vitest', 'mocha', 'cypress', 'webdriverio'];
|
|
10
10
|
export const TICKET_PRESETS = [
|
|
11
11
|
'any',
|
|
12
12
|
'jira',
|
|
@@ -62,6 +62,10 @@ export interface Framework {
|
|
|
62
62
|
* `context.skip('why')`. In Playwright the first argument is always a condition.
|
|
63
63
|
*/
|
|
64
64
|
skipNotes: boolean;
|
|
65
|
+
/** Whether a test or hook written as a `function` can skip at runtime with `this.skip()`, as in Mocha. */
|
|
66
|
+
thisSkip: boolean;
|
|
67
|
+
/** Whether a test with only a title written out is a todo, as Mocha's pending `it('title')`. */
|
|
68
|
+
todoWithoutBody: boolean;
|
|
65
69
|
/** Built-in states a test can be put in with this framework. */
|
|
66
70
|
states: ReadonlySet<BuiltinStateName>;
|
|
67
71
|
/** Playwright's own API on `test`: `test.describe`, `test.step`, `test.info()` and runtime `test.skip()`. */
|
package/dist/utils/frameworks.js
CHANGED
|
@@ -32,6 +32,8 @@ const playwright = {
|
|
|
32
32
|
info: { param: 1, methods: RUNTIME_MODIFIERS },
|
|
33
33
|
skipWithoutBody: false,
|
|
34
34
|
skipNotes: false,
|
|
35
|
+
thisSkip: false,
|
|
36
|
+
todoWithoutBody: false,
|
|
35
37
|
states: new Set(['skip', 'fixme', 'fail', 'slow', 'new', 'quarantined']),
|
|
36
38
|
playwright: true,
|
|
37
39
|
};
|
|
@@ -63,6 +65,8 @@ const jest = {
|
|
|
63
65
|
plainTags: false,
|
|
64
66
|
skipWithoutBody: false,
|
|
65
67
|
skipNotes: false,
|
|
68
|
+
thisSkip: false,
|
|
69
|
+
todoWithoutBody: false,
|
|
66
70
|
states: new Set(['skip', 'todo', 'fail', 'new', 'quarantined']),
|
|
67
71
|
playwright: false,
|
|
68
72
|
};
|
|
@@ -103,10 +107,58 @@ const vitest = {
|
|
|
103
107
|
info: { param: 0, methods: new Set(['skip']) },
|
|
104
108
|
skipWithoutBody: true,
|
|
105
109
|
skipNotes: true,
|
|
110
|
+
thisSkip: false,
|
|
111
|
+
todoWithoutBody: false,
|
|
106
112
|
states: new Set(['skip', 'todo', 'fail', 'new', 'quarantined']),
|
|
107
113
|
playwright: false,
|
|
108
114
|
};
|
|
109
|
-
|
|
115
|
+
const SKIPPED_TEST = { kind: 'test', modifiers: ['skip'] };
|
|
116
|
+
const SKIPPED_DESCRIBE = { kind: 'describe', modifiers: ['skip'] };
|
|
117
|
+
/** What the `mocha` module exports, the BDD and TDD names alike. `context` and `specify` are only globals. */
|
|
118
|
+
const MOCHA_EXPORTS = new Map([
|
|
119
|
+
['it', TEST],
|
|
120
|
+
['test', TEST],
|
|
121
|
+
['xit', SKIPPED_TEST],
|
|
122
|
+
['describe', DESCRIBE],
|
|
123
|
+
['suite', DESCRIBE],
|
|
124
|
+
['xdescribe', SKIPPED_DESCRIBE],
|
|
125
|
+
...['before', 'beforeEach', 'after', 'afterEach', 'suiteSetup', 'setup', 'suiteTeardown', 'teardown'].map((name) => [name, HOOK]),
|
|
126
|
+
]);
|
|
127
|
+
/** Mocha, and Cypress and WebdriverIO, which run their tests on Mocha. */
|
|
128
|
+
const mocha = {
|
|
129
|
+
ownsModule: (source) => source === 'mocha',
|
|
130
|
+
exports: MOCHA_EXPORTS,
|
|
131
|
+
globals: new Map([
|
|
132
|
+
...[...MOCHA_EXPORTS].filter(([name]) => name !== 'it' && name !== 'test'),
|
|
133
|
+
['context', DESCRIBE],
|
|
134
|
+
['xcontext', SKIPPED_DESCRIBE],
|
|
135
|
+
['xspecify', SKIPPED_TEST],
|
|
136
|
+
]),
|
|
137
|
+
testFunctions: ['it', 'specify', 'test'],
|
|
138
|
+
testMembers: new Set(['only', 'skip']),
|
|
139
|
+
describeMembers: new Set(['only', 'skip']),
|
|
140
|
+
factories: new Set(),
|
|
141
|
+
effects: effects([['skip', 'skip']]),
|
|
142
|
+
// Cypress's `{ browser: '!webkit' }` runs the test only in some browsers.
|
|
143
|
+
optionEffects: new Map([['browser', 'skip']]),
|
|
144
|
+
// @cypress/grep's `{ tags: '@smoke' }`, written with the `@`.
|
|
145
|
+
tagKey: 'tags',
|
|
146
|
+
plainTags: false,
|
|
147
|
+
skipWithoutBody: true,
|
|
148
|
+
skipNotes: false,
|
|
149
|
+
thisSkip: true,
|
|
150
|
+
todoWithoutBody: true,
|
|
151
|
+
states: new Set(['skip', 'todo', 'new', 'quarantined']),
|
|
152
|
+
playwright: false,
|
|
153
|
+
};
|
|
154
|
+
export const FRAMEWORKS = {
|
|
155
|
+
playwright,
|
|
156
|
+
jest,
|
|
157
|
+
vitest,
|
|
158
|
+
mocha: { name: 'mocha', ...mocha },
|
|
159
|
+
cypress: { name: 'cypress', ...mocha },
|
|
160
|
+
webdriverio: { name: 'webdriverio', ...mocha },
|
|
161
|
+
};
|
|
110
162
|
/** Whether `source` belongs to a test runner other than `framework`, so its names are never tests here. */
|
|
111
163
|
export function isOtherRunner(framework, source) {
|
|
112
164
|
return (isPlaywrightModule(source) || OTHER_RUNNERS.has(source)) && !framework.ownsModule(source);
|
package/dist/utils/options.d.ts
CHANGED
|
@@ -20,14 +20,14 @@ export interface CustomState {
|
|
|
20
20
|
/** Options shared by every rule. Set them once in `settings['test-governance']`. */
|
|
21
21
|
export interface GovernanceOptions {
|
|
22
22
|
/**
|
|
23
|
-
* The test framework of the files the config applies to: `'playwright'`, `'jest'
|
|
24
|
-
* `
|
|
23
|
+
* The test framework of the files the config applies to: `'playwright'`, `'jest'`, `'vitest'`, `'mocha'`,
|
|
24
|
+
* `'cypress'` or `'webdriverio'`. Required; each framework's config, such as `configs.mocha`, sets it.
|
|
25
25
|
*/
|
|
26
26
|
framework: FrameworkName;
|
|
27
27
|
/**
|
|
28
|
-
* Names that are treated as the framework's `test` function. Defaults to `['test']` for Playwright
|
|
29
|
-
* `['test', 'it']` for Jest and Vitest
|
|
30
|
-
* you still need them.
|
|
28
|
+
* Names that are treated as the framework's `test` function. Defaults to `['test']` for Playwright,
|
|
29
|
+
* `['test', 'it']` for Jest and Vitest, and `['it', 'specify', 'test']` for Mocha, Cypress and
|
|
30
|
+
* WebdriverIO. Your list replaces the default one, so include those names if you still need them.
|
|
31
31
|
*/
|
|
32
32
|
testFunctions?: string[];
|
|
33
33
|
/** Accepted ticket formats. Defaults to `{ preset: 'any' }`. */
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Require the marker keyword to match the test's state.
|
|
4
4
|
|
|
5
|
-
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
5
|
+
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🌲 `cypress`, 🃏 `jest`, ☕ `mocha`, 🎭 `playwright`, ⚡ `vitest`, 🤖 `webdriverio`.
|
|
6
6
|
|
|
7
7
|
💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
8
8
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Disallow state tags that contradict each other, differ in case or look like typos.
|
|
4
4
|
|
|
5
|
-
⚠️ This rule _warns_ in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
5
|
+
⚠️ This rule _warns_ in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🌲 `cypress`, 🃏 `jest`, ☕ `mocha`, 🎭 `playwright`, ⚡ `vitest`, 🤖 `webdriverio`.
|
|
6
6
|
|
|
7
7
|
🔧 This rule is automatically fixable by the [`--fix` CLI option](https://eslint.org/docs/latest/user-guide/command-line-interface#--fix).
|
|
8
8
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Disallow marker comments that no longer match a test state.
|
|
4
4
|
|
|
5
|
-
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
5
|
+
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🌲 `cypress`, 🃏 `jest`, ☕ `mocha`, 🎭 `playwright`, ⚡ `vitest`, 🤖 `webdriverio`.
|
|
6
6
|
|
|
7
7
|
💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
8
8
|
|
|
@@ -26,9 +26,9 @@ A `FIXME:` or `TODO:` comment that starts with a valid ticket is treated by wher
|
|
|
26
26
|
leftover marker and is reported, since `// FIXME: TRADE-12` above a test usually means the test was
|
|
27
27
|
fixed and the marker forgotten. Move a work comment into the test body, or onto the line of code it
|
|
28
28
|
is about.
|
|
29
|
-
- Above a
|
|
29
|
+
- Above a test that is not a todo, in every framework but Playwright, a `TODO:` comment is a tracked work comment about
|
|
30
30
|
the test and is allowed. `TODO` is the everyday keyword for such comments, so there it is far more
|
|
31
|
-
often a note than the forgotten marker of a
|
|
31
|
+
often a note than the forgotten marker of a todo test.
|
|
32
32
|
|
|
33
33
|
Other markers that read as a sentence, such as `// SKIP: this one is flaky`, are left alone too: the rule
|
|
34
34
|
counts a comment as prose when its first word is not a valid ticket and more text follows.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Require FIXME and TODO comments in test files to start with a ticket.
|
|
4
4
|
|
|
5
|
-
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
5
|
+
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🌲 `cypress`, 🃏 `jest`, ☕ `mocha`, 🎭 `playwright`, ⚡ `vitest`, 🤖 `webdriverio`.
|
|
6
6
|
|
|
7
7
|
💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
8
8
|
|
|
@@ -30,8 +30,8 @@ How comments are read:
|
|
|
30
30
|
style many teams already use. Anything after the parentheses is a note, as it would be after `TODO: TRADE-512`.
|
|
31
31
|
- The tickets follow the same rules as markers: the shared ticket format, placeholders such as `TBD`
|
|
32
32
|
rejected, and no text after the tickets unless `allowNotes` is set.
|
|
33
|
-
- A `// FIXME:` marker directly above a `test.fixme(...)`, or a `// TODO:` marker above a
|
|
34
|
-
|
|
33
|
+
- A `// FIXME:` marker directly above a `test.fixme(...)`, or a `// TODO:` marker above a todo test,
|
|
34
|
+
such as Jest's `test.todo(...)` or Mocha's `it('title')`, is checked by [`require-ticket`](require-ticket.md) instead, so it is
|
|
35
35
|
reported once.
|
|
36
36
|
- Only test files (a test framework import or a call to a test function) are checked.
|
|
37
37
|
Application code is left alone, even if your config applies the plugin to it.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
📝 Require a ticket marker comment above skipped, fixme and tagged tests.
|
|
4
4
|
|
|
5
|
-
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🃏 `jest`, 🎭 `playwright`, ⚡ `vitest`.
|
|
5
|
+
💼 This rule is enabled in the following [configs](https://github.com/Huldoser/eslint-plugin-test-governance#usage): 🌲 `cypress`, 🃏 `jest`, ☕ `mocha`, 🎭 `playwright`, ⚡ `vitest`, 🤖 `webdriverio`.
|
|
6
6
|
|
|
7
7
|
💡 This rule is manually fixable by [editor suggestions](https://eslint.org/docs/latest/use/core-concepts#rule-suggestions).
|
|
8
8
|
|
|
@@ -17,19 +17,21 @@ A test is in a state when it is:
|
|
|
17
17
|
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ---------------------- | -------------------------- |
|
|
18
18
|
| `skip` | `test.skip('title', fn)`, `test.describe.skip`, `test.skip()` in a body, `testInfo.skip()`, `test.step.skip`, `step.skip()` in a step | `// SKIP:` | yes |
|
|
19
19
|
| `fixme` | `test.fixme(...)`, `test.describe.fixme`, `testInfo.fixme()` | `// FIXME:` | yes |
|
|
20
|
-
| `todo` | Jest's and Vitest's `test.todo`
|
|
20
|
+
| `todo` | Jest's and Vitest's `test.todo`, and a Mocha test with only a title, `it('title')` | `// TODO:` | yes, except in Playwright |
|
|
21
21
|
| `new` | the `@new` tag | `// NEW:` | with `lifecycleTags: true` |
|
|
22
22
|
| `quarantined` | the `@quarantined` tag | `// QUARANTINED:` | with `lifecycleTags: true` |
|
|
23
23
|
| `fail`, `slow` | `test.fail(...)`, `test.slow()`, `testInfo.fail()`, `testInfo.slow()` | `// FAIL:`, `// SLOW:` | no |
|
|
24
24
|
| custom | any tag you list in `customStates` | your keyword | when configured |
|
|
25
25
|
|
|
26
26
|
In Jest and Vitest, `xit`, `xdescribe`, Vitest's `{ skip: true }`, `skipIf(true)`, `context.skip()` and
|
|
27
|
-
`context.skip('note')`, in a test or in `beforeEach`, skip a test too, and `test.failing` and `test.fails` put it in the `fail` state.
|
|
28
|
-
|
|
27
|
+
`context.skip('note')`, in a test or in `beforeEach`, skip a test too, and `test.failing` and `test.fails` put it in the `fail` state. In Mocha, Cypress and WebdriverIO, `xit`, `xdescribe` and
|
|
28
|
+
`this.skip()` in a test or hook skip a test too, and Cypress's `{ browser: '!webkit' }` skips it in other
|
|
29
|
+
browsers, which counts as a conditional skip. The [README](../../README.md#frameworks) lists every form.
|
|
29
30
|
|
|
30
31
|
Tags are read from the title (`'places a limit order @new'`) and from `{ tag: '@new' }` or `{ tag: ['@new', '@smoke'] }`,
|
|
31
32
|
also when they come from a `const` or an enum declared in the same file, as in `{ tag: TAGS.NEW }`.
|
|
32
|
-
Vitest writes tag names without the `@`, so `{ tags: ['new'] }` is the `@new` tag.
|
|
33
|
+
Vitest writes tag names without the `@`, so `{ tags: ['new'] }` is the `@new` tag. Cypress tags from
|
|
34
|
+
@cypress/grep keep it: `{ tags: '@new' }`. Tags on a
|
|
33
35
|
`describe` apply to every test inside it. `states` can rename the tag and marker of `new` and
|
|
34
36
|
`quarantined`, as in `states: { quarantined: { tag: '@flaky', marker: 'FLAKY' } }`.
|
|
35
37
|
|
|
@@ -77,8 +79,10 @@ Conditional skips such as `test.skip(browserName === 'webkit', 'Not supported')`
|
|
|
77
79
|
permanent platform limit, so they need no ticket unless you set `requireTicketForConditional: true`.
|
|
78
80
|
A runtime skip under an `if`, a `switch` case, a ternary or `&&` counts as conditional too, since
|
|
79
81
|
`if (!enabled) test.skip()` does the same as `test.skip(!enabled)`. So does a skip in a `catch` block,
|
|
80
|
-
such as `catch { test.skip(true, 'Market data feed is down') }`,
|
|
81
|
-
|
|
82
|
+
such as `catch { test.skip(true, 'Market data feed is down') }`, a skip after other code in a `try`
|
|
83
|
+
block that has a `catch`, as in `try { await stat(target); test.skip(); } catch {}`, where an error
|
|
84
|
+
skips past it, and a skip after an early exit such as `if (ready) return;` or `if (!ok) throw error;`
|
|
85
|
+
earlier in the same function.
|
|
82
86
|
|
|
83
87
|
A test or describe whose function a condition picks is a conditional skip too, in every framework. Its
|
|
84
88
|
title and tags count as usual, and a marker goes above it, or in code without semicolons above the
|
|
@@ -191,6 +195,20 @@ it.todo('caps position size at 2% of equity');
|
|
|
191
195
|
test.skipIf(true)('fills a stop order when the market opens', () => {});
|
|
192
196
|
```
|
|
193
197
|
|
|
198
|
+
<!-- example: invalid settings={"framework":"mocha"} -->
|
|
199
|
+
|
|
200
|
+
```js
|
|
201
|
+
it('settles a trade at T+1', function () {
|
|
202
|
+
this.skip();
|
|
203
|
+
});
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
<!-- example: invalid settings={"framework":"cypress","lifecycleTags":true} -->
|
|
207
|
+
|
|
208
|
+
```js
|
|
209
|
+
it('places a stop order', { tags: '@quarantined' }, () => {});
|
|
210
|
+
```
|
|
211
|
+
|
|
194
212
|
Correct:
|
|
195
213
|
|
|
196
214
|
<!-- example: valid -->
|
|
@@ -245,6 +263,19 @@ describe('risk limits', () => {
|
|
|
245
263
|
});
|
|
246
264
|
```
|
|
247
265
|
|
|
266
|
+
<!-- example: valid settings={"framework":"mocha"} -->
|
|
267
|
+
|
|
268
|
+
```js
|
|
269
|
+
describe('settlement', function () {
|
|
270
|
+
// TODO: TRADE-91
|
|
271
|
+
it('settles a trade on a market holiday');
|
|
272
|
+
|
|
273
|
+
it('settles in the clearing house currency', function () {
|
|
274
|
+
if (!process.env.CLEARING_URL) this.skip();
|
|
275
|
+
});
|
|
276
|
+
});
|
|
277
|
+
```
|
|
278
|
+
|
|
248
279
|
## Options
|
|
249
280
|
|
|
250
281
|
This rule has no options of its own. It reads the shared options described in the
|
package/docs/upgrading.md
CHANGED
|
@@ -90,7 +90,7 @@ becomes `extends: ['test-governance/playwright']`.
|
|
|
90
90
|
A config that still reads `configs.recommended` fails when it loads:
|
|
91
91
|
|
|
92
92
|
```text
|
|
93
|
-
eslint-plugin-test-governance: configs.recommended was removed in 0.5.0. Use the config for your framework: configs.playwright, configs.jest or configs.
|
|
93
|
+
eslint-plugin-test-governance: configs.recommended was removed in 0.5.0. Use the config for your framework: configs.playwright, configs.jest, configs.vitest, configs.mocha, configs.cypress or configs.webdriverio. See https://github.com/Huldoser/eslint-plugin-test-governance/blob/main/docs/upgrading.md
|
|
94
94
|
```
|
|
95
95
|
|
|
96
96
|
### The framework option
|
|
@@ -121,7 +121,7 @@ Without it, the config fails when it loads:
|
|
|
121
121
|
|
|
122
122
|
```text
|
|
123
123
|
eslint-plugin-test-governance: invalid options:
|
|
124
|
-
- framework is required: set it to "playwright", "jest" or "
|
|
124
|
+
- framework is required: set it to "playwright", "jest", "vitest", "mocha", "cypress" or "webdriverio", or use configs.playwright, configs.jest, configs.vitest, configs.mocha, configs.cypress or configs.webdriverio
|
|
125
125
|
```
|
|
126
126
|
|
|
127
127
|
If you write `settings['test-governance']` yourself instead of calling `configure()`, add `framework`
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "eslint-plugin-test-governance",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "ESLint rules that require a ticket comment above skipped, fixme, todo and tagged Playwright, Jest and
|
|
3
|
+
"version": "0.7.0",
|
|
4
|
+
"description": "ESLint rules that require a ticket comment above skipped, fixme, todo and tagged Playwright, Jest, Vitest, Mocha, Cypress and WebdriverIO tests.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"eslint",
|
|
7
7
|
"eslintplugin",
|
|
@@ -9,6 +9,9 @@
|
|
|
9
9
|
"playwright",
|
|
10
10
|
"jest",
|
|
11
11
|
"vitest",
|
|
12
|
+
"mocha",
|
|
13
|
+
"cypress",
|
|
14
|
+
"webdriverio",
|
|
12
15
|
"testing"
|
|
13
16
|
],
|
|
14
17
|
"homepage": "https://github.com/Huldoser/eslint-plugin-test-governance#readme",
|