eslint-plugin-test-governance 0.5.1 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -1
- package/README.md +47 -20
- package/dist/rules/no-conflicting-states.d.ts +1 -1
- package/dist/rules/no-conflicting-states.js +7 -7
- package/dist/utils/constants.d.ts +4 -1
- package/dist/utils/constants.js +3 -1
- package/dist/utils/frameworks.js +4 -5
- package/dist/utils/options.d.ts +9 -5
- package/dist/utils/options.js +4 -2
- package/dist/utils/schema.js +10 -4
- package/docs/rules/marker-matches-state.md +3 -3
- package/docs/rules/no-conflicting-states.md +9 -8
- package/docs/rules/require-ticket.md +6 -5
- package/docs/upgrading.md +50 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,37 @@ All notable changes to this project are documented here. The format follows
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.6.1] - 2026-10-14
|
|
10
|
+
|
|
11
|
+
### Fixed
|
|
12
|
+
|
|
13
|
+
- In Playwright, `test.slow('title', body)` is no longer read as a slow test declared with a title.
|
|
14
|
+
Playwright has no such form: `slow` only marks a running test, and a string before the description
|
|
15
|
+
is its condition, so the call is checked like `test.slow(condition, description)`. Before, with the
|
|
16
|
+
`slow` state on, it needed a `// SLOW:` marker even without `requireTicketForConditional`.
|
|
17
|
+
|
|
18
|
+
## [0.6.0] - 2026-10-14
|
|
19
|
+
|
|
20
|
+
This release renames the `unstable` state to `quarantined`.
|
|
21
|
+
[Upgrading to 0.6](https://github.com/Huldoser/eslint-plugin-test-governance/blob/main/docs/upgrading.md#upgrading-to-06)
|
|
22
|
+
shows what to change.
|
|
23
|
+
|
|
24
|
+
### Added
|
|
25
|
+
|
|
26
|
+
- `states` can rename the tag of the `new` and `quarantined` states, for teams that call them
|
|
27
|
+
something else: `states: { quarantined: { tag: '@flaky', marker: 'FLAKY' } }` makes `@flaky` tests
|
|
28
|
+
need a `// FLAKY:` marker. The rules check the new tag as they did the default one, and
|
|
29
|
+
`@quarantined` no longer counts.
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- **Breaking:** the `unstable` state is renamed to `quarantined`. `lifecycleTags: true` now turns on
|
|
34
|
+
`@quarantined`, with the `// QUARANTINED:` marker, instead of `@unstable` and `// UNSTABLE:`, so rename
|
|
35
|
+
those in your tests, or keep them with
|
|
36
|
+
`states: { quarantined: { tag: '@unstable', marker: 'UNSTABLE' } }`. A config that sets
|
|
37
|
+
`states.unstable` fails with a message that names `states.quarantined`. A config that only sets
|
|
38
|
+
`lifecycleTags: true` still loads, but no longer checks `@unstable` tests.
|
|
39
|
+
|
|
9
40
|
## [0.5.1] - 2026-10-13
|
|
10
41
|
|
|
11
42
|
### Fixed
|
|
@@ -263,7 +294,9 @@ First release.
|
|
|
263
294
|
- Support for ESLint 9 and 10 flat config, JavaScript and TypeScript test files, `.extend()`
|
|
264
295
|
fixtures, describe blocks, runtime skips and tags in titles or the `tag` option.
|
|
265
296
|
|
|
266
|
-
[Unreleased]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.
|
|
297
|
+
[Unreleased]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.6.1...HEAD
|
|
298
|
+
[0.6.1]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.6.0...v0.6.1
|
|
299
|
+
[0.6.0]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.5.1...v0.6.0
|
|
267
300
|
[0.5.1]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.5.0...v0.5.1
|
|
268
301
|
[0.5.0]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.4.0...v0.5.0
|
|
269
302
|
[0.4.0]: https://github.com/Huldoser/eslint-plugin-test-governance/compare/v0.3.1...v0.4.0
|
package/README.md
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
[](LICENSE)
|
|
6
6
|
|
|
7
7
|
ESLint rules that keep a Playwright, Jest or Vitest suite honest. Every skipped, fixme, todo, `@new`
|
|
8
|
-
or `@
|
|
8
|
+
or `@quarantined` test must point at the ticket that tracks it:
|
|
9
9
|
|
|
10
10
|
<!-- example: valid -->
|
|
11
11
|
|
|
@@ -20,8 +20,8 @@ Without the comment, `test-governance/require-ticket` fails the lint. When the t
|
|
|
20
20
|
skip removed, `no-orphaned-marker` flags the leftover comment. The markers are plain
|
|
21
21
|
comments, so `grep -rn "SKIP:" tests/` lists every disabled test and its ticket.
|
|
22
22
|
|
|
23
|
-
> **Status:** early release (0.x). Options may still change before 1.0. Coming from 0.
|
|
24
|
-
> [Upgrading to 0.
|
|
23
|
+
> **Status:** early release (0.x). Options may still change before 1.0. Coming from 0.5?
|
|
24
|
+
> [Upgrading to 0.6](docs/upgrading.md#upgrading-to-06) shows what to change.
|
|
25
25
|
|
|
26
26
|
## Install
|
|
27
27
|
|
|
@@ -90,7 +90,7 @@ export default [
|
|
|
90
90
|
framework: 'playwright',
|
|
91
91
|
// Only accept Jira keys from these projects (or Jira URLs).
|
|
92
92
|
ticket: { preset: 'jira', projects: ['TRADE', 'RISK'] },
|
|
93
|
-
// Turn on the @new and @
|
|
93
|
+
// Turn on the @new and @quarantined states.
|
|
94
94
|
lifecycleTags: true,
|
|
95
95
|
// Any tag your team uses, with the same rules as the built-ins.
|
|
96
96
|
customStates: {
|
|
@@ -175,7 +175,7 @@ The defaults follow how most teams already work, so each framework config is use
|
|
|
175
175
|
| Setting | Default | Why |
|
|
176
176
|
| --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
|
177
177
|
| States that need a ticket | `skip`, `fixme`; in Jest and Vitest `skip`, `todo` | Every team skips tests. Chromium DevTools, GitLab and Slack all require a bug link on disabled tests. |
|
|
178
|
-
| `@new`, `@
|
|
178
|
+
| `@new`, `@quarantined` | off; `lifecycleTags: true` turns both on | A promotion workflow is not universal. Teams that say `@flaky` instead [rename the tag](#tag-names). |
|
|
179
179
|
| Conditional skips (`test.skip(browserName === 'webkit', ...)`, `if (...) test.skip()`, a skip in `catch`) | no ticket needed | These are usually permanent platform limits, not bugs. `requireTicketForConditional: true` changes this. |
|
|
180
180
|
| `test.fail`, `test.slow`, Jest's `test.failing`, Vitest's `test.fails` | off | They don't remove coverage. Turn them on with `states: { fail: true, slow: true }`. |
|
|
181
181
|
| Marker format | `// SKIP: TRADE-123` or `// SKIP: TRADE-123, TRADE-124` | Uppercase keyword and colon, like `TODO:`. Easy to grep. |
|
|
@@ -194,20 +194,47 @@ yourself. `framework` is required, and every other option has a default. The rul
|
|
|
194
194
|
of their own: they all read these shared settings, so they always agree on ticket formats, states
|
|
195
195
|
and markers. To turn a rule off, set it to `'off'` as usual.
|
|
196
196
|
|
|
197
|
-
| Option | Type
|
|
198
|
-
| ----------------------------- |
|
|
199
|
-
| `framework` | `'playwright' \| 'jest' \| 'vitest'`
|
|
200
|
-
| `ticket` | `TicketSpec \| TicketSpec[]`
|
|
201
|
-
| `placeholders` | `string[]`
|
|
202
|
-
| `lifecycleTags` | `boolean`
|
|
203
|
-
| `states` | `{ [state]: boolean \| { enabled?, marker?, ticket? } }` | `skip`, `fixme` and `todo` on | Turns built-in states (`skip`, `fixme`, `fail`, `slow`, `todo`, `new`, `
|
|
204
|
-
| `customStates` | `{ [name]: { when, marker, ticket? } }`
|
|
205
|
-
| `requireTicketForConditional` | `boolean`
|
|
206
|
-
| `allowBlankLine` | `boolean`
|
|
207
|
-
| `allowNotes` | `boolean`
|
|
208
|
-
| `comments` | `false \| { keywords?: string[] }`
|
|
209
|
-
| `testFunctions` | `string[]`
|
|
210
|
-
| `reportUnreadableTags` | `boolean`
|
|
197
|
+
| Option | Type | Default | Description |
|
|
198
|
+
| ----------------------------- | -------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
199
|
+
| `framework` | `'playwright' \| 'jest' \| 'vitest'` | required | The test framework of the files the config applies to. `configs.playwright`, `configs.jest` and `configs.vitest` set it, and `configure()` needs it. See [Frameworks](#frameworks). |
|
|
200
|
+
| `ticket` | `TicketSpec \| TicketSpec[]` | `{ preset: 'any' }` | Accepted ticket formats. An array accepts a ticket that matches any entry. See [Ticket presets](#ticket-presets). |
|
|
201
|
+
| `placeholders` | `string[]` | see above | Tickets rejected as placeholders. `*` matches any characters. Case-insensitive; a leading `#` is ignored. Your list replaces the default one, so repeat the defaults you want to keep. Tickets numbered 0 are always rejected. |
|
|
202
|
+
| `lifecycleTags` | `boolean` | `false` | Turns on the `new` (`@new`) and `quarantined` (`@quarantined`) states. |
|
|
203
|
+
| `states` | `{ [state]: boolean \| { enabled?, tag?, marker?, ticket? } }` | `skip`, `fixme` and `todo` on | Turns built-in states (`skip`, `fixme`, `fail`, `slow`, `todo`, `new`, `quarantined`) on or off, renames their tag or marker, or gives them their own ticket format. `fail` and `slow` are off by default; a setting for `new` or `quarantined` here wins over `lifecycleTags`. Only `new` and `quarantined` have a tag to rename; see [Tag names](#tag-names). `fixme` and `slow` exist only in Playwright and `todo` only in Jest and Vitest, so turning one on elsewhere has no effect. |
|
|
204
|
+
| `customStates` | `{ [name]: { when, marker, ticket? } }` | `{}` | Adds a state for a tag, e.g. `{ 'needs-data': { when: '@needs-data', marker: 'NEEDS-DATA' } }`. `when` is a tag and `marker` an uppercase keyword; neither can be shared with another state. |
|
|
205
|
+
| `requireTicketForConditional` | `boolean` | `false` | Require a ticket for conditional skips too: `test.skip(condition, ...)`, `if (...) test.skip()`, a skip in a `catch` block or after an early `return` or `throw`. |
|
|
206
|
+
| `allowBlankLine` | `boolean` | `false` | Let blank lines separate a marker from its test. |
|
|
207
|
+
| `allowNotes` | `boolean` | `false` | Allow free text after the tickets, e.g. `// SKIP: TRADE-123 flaky on CI`. Applies to markers and to `FIXME` / `TODO` comments. |
|
|
208
|
+
| `comments` | `false \| { keywords?: string[] }` | `{ keywords: ['FIXME', 'TODO'] }` | Comment keywords that must start with a ticket anywhere in a test file, e.g. `// TODO: TRADE-123`. Your keywords replace the default ones. `false` allows free-form comments. |
|
|
209
|
+
| `testFunctions` | `string[]` | `['test']` | Names treated as the framework's `test`. Jest and Vitest default to `['test', 'it']`. Your list replaces the default one, so include those names if you still need them. See below. |
|
|
210
|
+
| `reportUnreadableTags` | `boolean` | `false` | When a tag state is on, report titles that aren't static text, and tags or details that can't be read in the file, such as ones imported from another module, since their tags can't be checked. |
|
|
211
|
+
|
|
212
|
+
### Tag names
|
|
213
|
+
|
|
214
|
+
Teams name the stages of a test differently, so `states` can rename the tag of `new` and
|
|
215
|
+
`quarantined`. The marker keeps its name unless you rename it too:
|
|
216
|
+
|
|
217
|
+
```js
|
|
218
|
+
testGovernance.configure({
|
|
219
|
+
framework: 'playwright',
|
|
220
|
+
lifecycleTags: true,
|
|
221
|
+
states: { quarantined: { tag: '@flaky', marker: 'FLAKY' } },
|
|
222
|
+
});
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
<!-- example: valid settings={"lifecycleTags":true,"states":{"quarantined":{"tag":"@flaky","marker":"FLAKY"}}} -->
|
|
226
|
+
|
|
227
|
+
```ts
|
|
228
|
+
// FLAKY: TRADE-311
|
|
229
|
+
test('streams level 2 quotes @flaky', async ({ page }) => {
|
|
230
|
+
// ...
|
|
231
|
+
});
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
The new tag replaces the default one, so `@quarantined` no longer puts a test in the state, and the
|
|
235
|
+
rules check `@flaky` as they checked `@quarantined`: `@new` together with `@flaky` is reported, and so
|
|
236
|
+
is a near-miss such as `@flakey`. For a tag that isn't one of these stages, such as `@needs-data`, add
|
|
237
|
+
a `customStates` entry.
|
|
211
238
|
|
|
212
239
|
### Ticket presets
|
|
213
240
|
|
|
@@ -235,7 +262,7 @@ testGovernance.configure({
|
|
|
235
262
|
framework: 'playwright',
|
|
236
263
|
ticket: { preset: 'jira', projects: ['TRADE'] },
|
|
237
264
|
lifecycleTags: true,
|
|
238
|
-
states: {
|
|
265
|
+
states: { quarantined: { ticket: { preset: 'github' } } },
|
|
239
266
|
});
|
|
240
267
|
```
|
|
241
268
|
|
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
declare const _default: import("../utils/create-rule.ts").Rule<"
|
|
1
|
+
declare const _default: import("../utils/create-rule.ts").Rule<"newAndQuarantined" | "newSkipped" | "tagCase" | "tagTypo">;
|
|
2
2
|
export default _default;
|
|
@@ -13,7 +13,7 @@ export default createRule({
|
|
|
13
13
|
fixable: 'code',
|
|
14
14
|
schema: [],
|
|
15
15
|
messages: {
|
|
16
|
-
|
|
16
|
+
newAndQuarantined: '`{{new}}` and `{{quarantined}}` are separate stages; a test should be in only one of them.',
|
|
17
17
|
newSkipped: "`{{new}}` on a skipped test: a new test that doesn't run can't be promoted. Fix the test or drop the tag.",
|
|
18
18
|
tagCase: '`{{found}}` should be written `{{expected}}`.',
|
|
19
19
|
tagTypo: '`{{found}}` looks like a typo of `{{expected}}`.',
|
|
@@ -22,7 +22,7 @@ export default createRule({
|
|
|
22
22
|
create(context, analysis, options) {
|
|
23
23
|
const tagStates = options.states.filter(isTagState);
|
|
24
24
|
const newTag = options.states.find((s) => s.name === 'new')?.tag;
|
|
25
|
-
const
|
|
25
|
+
const quarantinedTag = options.states.find((s) => s.name === 'quarantined')?.tag;
|
|
26
26
|
for (const subject of analysis.subjects) {
|
|
27
27
|
for (const occurrence of subject.tags)
|
|
28
28
|
checkSpelling(occurrence);
|
|
@@ -31,13 +31,13 @@ export default createRule({
|
|
|
31
31
|
const own = (tag) => subject.tags.find((t) => t.tag === tag);
|
|
32
32
|
const has = (tag) => own(tag) !== undefined || subject.inheritedTags.has(tag);
|
|
33
33
|
const ownNew = own(newTag);
|
|
34
|
-
if (
|
|
35
|
-
const occurrence = ownNew ?? own(
|
|
34
|
+
if (quarantinedTag !== undefined && has(newTag) && has(quarantinedTag)) {
|
|
35
|
+
const occurrence = ownNew ?? own(quarantinedTag);
|
|
36
36
|
if (occurrence) {
|
|
37
37
|
context.report({
|
|
38
38
|
node: occurrence.node,
|
|
39
|
-
messageId: '
|
|
40
|
-
data: { new: newTag,
|
|
39
|
+
messageId: 'newAndQuarantined',
|
|
40
|
+
data: { new: newTag, quarantined: quarantinedTag },
|
|
41
41
|
});
|
|
42
42
|
}
|
|
43
43
|
}
|
|
@@ -67,7 +67,7 @@ export default createRule({
|
|
|
67
67
|
return;
|
|
68
68
|
}
|
|
69
69
|
// A typo changes, adds, drops or swaps a letter or two. A tag that is two or more letters
|
|
70
|
-
// shorter or longer, like `@
|
|
70
|
+
// shorter or longer, like `@unquarantined` next to `@quarantined`, is a different word.
|
|
71
71
|
const near = tagStates.find((s) => {
|
|
72
72
|
const expected = s.tag.toLowerCase();
|
|
73
73
|
return (Math.abs(lower.length - expected.length) < 2 &&
|
|
@@ -2,8 +2,11 @@
|
|
|
2
2
|
export declare const MARKER_RE: RegExp;
|
|
3
3
|
/** A tag as written in options: `@` followed by letters, digits, `-` and `_`. */
|
|
4
4
|
export declare const TAG_RE: RegExp;
|
|
5
|
-
export declare const BUILTIN_STATE_NAMES: readonly ["skip", "fixme", "fail", "slow", "todo", "new", "
|
|
5
|
+
export declare const BUILTIN_STATE_NAMES: readonly ["skip", "fixme", "fail", "slow", "todo", "new", "quarantined"];
|
|
6
6
|
export type BuiltinStateName = (typeof BUILTIN_STATE_NAMES)[number];
|
|
7
|
+
/** The built-in states a tag puts a test in, so the only ones whose tag can be renamed. */
|
|
8
|
+
export declare const TAG_STATE_NAMES: readonly ["new", "quarantined"];
|
|
9
|
+
export type TagStateName = (typeof TAG_STATE_NAMES)[number];
|
|
7
10
|
/** The test frameworks the rules understand. */
|
|
8
11
|
export declare const FRAMEWORK_NAMES: readonly ["playwright", "jest", "vitest"];
|
|
9
12
|
export type FrameworkName = (typeof FRAMEWORK_NAMES)[number];
|
package/dist/utils/constants.js
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
export const MARKER_RE = /^[A-Z][A-Z0-9_-]*$/;
|
|
3
3
|
/** A tag as written in options: `@` followed by letters, digits, `-` and `_`. */
|
|
4
4
|
export const TAG_RE = /^@[\w-]+$/;
|
|
5
|
-
export const BUILTIN_STATE_NAMES = ['skip', 'fixme', 'fail', 'slow', 'todo', 'new', '
|
|
5
|
+
export const BUILTIN_STATE_NAMES = ['skip', 'fixme', 'fail', 'slow', 'todo', 'new', 'quarantined'];
|
|
6
|
+
/** The built-in states a tag puts a test in, so the only ones whose tag can be renamed. */
|
|
7
|
+
export const TAG_STATE_NAMES = ['new', 'quarantined'];
|
|
6
8
|
/** The test frameworks the rules understand. */
|
|
7
9
|
export const FRAMEWORK_NAMES = ['playwright', 'jest', 'vitest'];
|
|
8
10
|
export const TICKET_PRESETS = [
|
package/dist/utils/frameworks.js
CHANGED
|
@@ -18,14 +18,13 @@ const playwright = {
|
|
|
18
18
|
defaultExport: TEST,
|
|
19
19
|
globals: new Map(),
|
|
20
20
|
testFunctions: ['test'],
|
|
21
|
-
testMembers: new Set(['only', 'skip', 'fixme', 'fail'
|
|
21
|
+
testMembers: new Set(['only', 'skip', 'fixme', 'fail']),
|
|
22
22
|
describeMembers: new Set(['only', 'skip', 'fixme', 'serial', 'parallel']),
|
|
23
23
|
factories: new Set(),
|
|
24
24
|
effects: effects([
|
|
25
25
|
['skip', 'skip'],
|
|
26
26
|
['fixme', 'fixme'],
|
|
27
27
|
['fail', 'fail'],
|
|
28
|
-
['slow', 'slow'],
|
|
29
28
|
]),
|
|
30
29
|
optionEffects: new Map(),
|
|
31
30
|
tagKey: 'tag',
|
|
@@ -33,7 +32,7 @@ const playwright = {
|
|
|
33
32
|
info: { param: 1, methods: RUNTIME_MODIFIERS },
|
|
34
33
|
skipWithoutBody: false,
|
|
35
34
|
skipNotes: false,
|
|
36
|
-
states: new Set(['skip', 'fixme', 'fail', 'slow', 'new', '
|
|
35
|
+
states: new Set(['skip', 'fixme', 'fail', 'slow', 'new', 'quarantined']),
|
|
37
36
|
playwright: true,
|
|
38
37
|
};
|
|
39
38
|
const JEST_FUNCTIONS = new Map([
|
|
@@ -64,7 +63,7 @@ const jest = {
|
|
|
64
63
|
plainTags: false,
|
|
65
64
|
skipWithoutBody: false,
|
|
66
65
|
skipNotes: false,
|
|
67
|
-
states: new Set(['skip', 'todo', 'fail', 'new', '
|
|
66
|
+
states: new Set(['skip', 'todo', 'fail', 'new', 'quarantined']),
|
|
68
67
|
playwright: false,
|
|
69
68
|
};
|
|
70
69
|
const VITEST_FACTORIES = ['each()', 'for()', 'skipIf()', 'runIf()'];
|
|
@@ -104,7 +103,7 @@ const vitest = {
|
|
|
104
103
|
info: { param: 0, methods: new Set(['skip']) },
|
|
105
104
|
skipWithoutBody: true,
|
|
106
105
|
skipNotes: true,
|
|
107
|
-
states: new Set(['skip', 'todo', 'fail', 'new', '
|
|
106
|
+
states: new Set(['skip', 'todo', 'fail', 'new', 'quarantined']),
|
|
108
107
|
playwright: false,
|
|
109
108
|
};
|
|
110
109
|
export const FRAMEWORKS = { playwright, jest, vitest };
|
package/dist/utils/options.d.ts
CHANGED
|
@@ -5,6 +5,8 @@ export declare const SETTINGS_KEY = "test-governance";
|
|
|
5
5
|
/** A state override in `states`. `false` turns a state off, `true` turns it on. */
|
|
6
6
|
export interface StateOverride {
|
|
7
7
|
enabled?: boolean;
|
|
8
|
+
/** The tag that puts a test in the state, e.g. `@flaky`. Only `new` and `quarantined` have a tag. */
|
|
9
|
+
tag?: string;
|
|
8
10
|
marker?: string;
|
|
9
11
|
ticket?: TicketSpec | TicketSpec[];
|
|
10
12
|
}
|
|
@@ -35,15 +37,17 @@ export interface GovernanceOptions {
|
|
|
35
37
|
* one (`TODO`, `TBD`, `XXX-*`, ...); tickets numbered 0 are always rejected.
|
|
36
38
|
*/
|
|
37
39
|
placeholders?: string[];
|
|
38
|
-
/** Turns on the `@new` and `@
|
|
40
|
+
/** Turns on the `@new` and `@quarantined` states. */
|
|
39
41
|
lifecycleTags?: boolean;
|
|
40
42
|
/**
|
|
41
|
-
* Turns built-in states on or off, renames their marker or gives them their own ticket format.
|
|
42
|
-
* `fixme` and `todo` are on by default, `fail` and `slow` off; a setting for `new` or `
|
|
43
|
-
* `lifecycleTags`.
|
|
43
|
+
* Turns built-in states on or off, renames their tag or marker, or gives them their own ticket format.
|
|
44
|
+
* `skip`, `fixme` and `todo` are on by default, `fail` and `slow` off; a setting for `new` or `quarantined`
|
|
45
|
+
* wins over `lifecycleTags`. Only `new` and `quarantined` have a tag, e.g.
|
|
46
|
+
* `{ quarantined: { tag: '@flaky', marker: 'FLAKY' } }`. `fixme` and `slow` exist only in Playwright and
|
|
47
|
+
* `todo` only in Jest and Vitest.
|
|
44
48
|
*/
|
|
45
49
|
states?: Partial<Record<BuiltinStateName, boolean | StateOverride>>;
|
|
46
|
-
/** Adds a state for a tag, e.g. `{
|
|
50
|
+
/** Adds a state for a tag, e.g. `{ 'needs-data': { when: '@needs-data', marker: 'NEEDS-DATA' } }`. */
|
|
47
51
|
customStates?: Record<string, CustomState>;
|
|
48
52
|
/** Require a ticket for conditional skips such as `test.skip(browserName === 'webkit')`. */
|
|
49
53
|
requireTicketForConditional?: boolean;
|
package/dist/utils/options.js
CHANGED
|
@@ -8,6 +8,8 @@ export const SETTINGS_KEY = 'test-governance';
|
|
|
8
8
|
export function isTagState(state) {
|
|
9
9
|
return state.tag !== undefined;
|
|
10
10
|
}
|
|
11
|
+
// A state in `TAG_STATE_NAMES` must have a tag and every other one a modifier, so the schema offers `tag`
|
|
12
|
+
// to the right states.
|
|
11
13
|
const BUILTIN_STATES = {
|
|
12
14
|
skip: { marker: 'SKIP', modifier: 'skip', enabled: true },
|
|
13
15
|
fixme: { marker: 'FIXME', modifier: 'fixme', enabled: true },
|
|
@@ -15,7 +17,7 @@ const BUILTIN_STATES = {
|
|
|
15
17
|
slow: { marker: 'SLOW', modifier: 'slow', enabled: false },
|
|
16
18
|
todo: { marker: 'TODO', modifier: 'todo', enabled: true },
|
|
17
19
|
new: { marker: 'NEW', tag: '@new', lifecycle: true, enabled: false },
|
|
18
|
-
|
|
20
|
+
quarantined: { marker: 'QUARANTINED', tag: '@quarantined', lifecycle: true, enabled: false },
|
|
19
21
|
};
|
|
20
22
|
export { ConfigError } from "./errors.js";
|
|
21
23
|
function toSpecs(spec) {
|
|
@@ -103,7 +105,7 @@ export function compileOptions(options) {
|
|
|
103
105
|
marker: override.marker ?? def.marker,
|
|
104
106
|
ticket: override.ticket === undefined ? defaultMatcher : compile(override.ticket),
|
|
105
107
|
};
|
|
106
|
-
states.push(def.modifier === undefined ? { ...base, tag: def.tag } : { ...base, modifier: def.modifier });
|
|
108
|
+
states.push(def.modifier === undefined ? { ...base, tag: override.tag ?? def.tag } : { ...base, modifier: def.modifier });
|
|
107
109
|
}
|
|
108
110
|
for (const [name, custom] of Object.entries(options.customStates ?? {})) {
|
|
109
111
|
states.push({
|
package/dist/utils/schema.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { BUILTIN_STATE_NAMES, FRAMEWORK_NAMES, MARKER_RE, TAG_RE, TICKET_PRESETS } from "./constants.js";
|
|
1
|
+
import { BUILTIN_STATE_NAMES, FRAMEWORK_NAMES, MARKER_RE, TAG_RE, TAG_STATE_NAMES, TICKET_PRESETS, } from "./constants.js";
|
|
2
2
|
const MARKER_HINT = 'an uppercase keyword such as NEEDS-DATA (letters, digits, "-" and "_")';
|
|
3
3
|
const TAG_HINT = 'a tag such as @needs-data';
|
|
4
4
|
/** Jira project and Linear team keys, as they appear in tickets: `WEB` in `WEB-123`. */
|
|
@@ -25,20 +25,22 @@ const ticketSpecSchema = {
|
|
|
25
25
|
const ticketSchema = {
|
|
26
26
|
anyOf: [ticketSpecSchema, { type: 'array', items: ticketSpecSchema, minItems: 1 }],
|
|
27
27
|
};
|
|
28
|
-
|
|
28
|
+
/** A setting in `states`. Only a state that a tag puts a test in, such as `new`, can rename its tag. */
|
|
29
|
+
const stateOverrideSchema = (hasTag) => ({
|
|
29
30
|
anyOf: [
|
|
30
31
|
{ type: 'boolean' },
|
|
31
32
|
{
|
|
32
33
|
type: 'object',
|
|
33
34
|
properties: {
|
|
34
35
|
enabled: { type: 'boolean' },
|
|
36
|
+
...(hasTag && { tag: { type: 'string', pattern: TAG_RE.source, patternHint: TAG_HINT } }),
|
|
35
37
|
marker: { type: 'string', pattern: MARKER_RE.source, patternHint: MARKER_HINT },
|
|
36
38
|
ticket: ticketSchema,
|
|
37
39
|
},
|
|
38
40
|
additionalProperties: false,
|
|
39
41
|
},
|
|
40
42
|
],
|
|
41
|
-
};
|
|
43
|
+
});
|
|
42
44
|
/** The shape of `settings['test-governance']`, the options of `configure()`. */
|
|
43
45
|
export const optionsSchema = {
|
|
44
46
|
type: 'object',
|
|
@@ -50,8 +52,12 @@ export const optionsSchema = {
|
|
|
50
52
|
lifecycleTags: { type: 'boolean' },
|
|
51
53
|
states: {
|
|
52
54
|
type: 'object',
|
|
53
|
-
properties: Object.fromEntries(BUILTIN_STATE_NAMES.map((name) => [
|
|
55
|
+
properties: Object.fromEntries(BUILTIN_STATE_NAMES.map((name) => [
|
|
56
|
+
name,
|
|
57
|
+
stateOverrideSchema(TAG_STATE_NAMES.includes(name)),
|
|
58
|
+
])),
|
|
54
59
|
additionalProperties: false,
|
|
60
|
+
renamed: { unstable: 'quarantined' },
|
|
55
61
|
},
|
|
56
62
|
customStates: {
|
|
57
63
|
type: 'object',
|
|
@@ -30,7 +30,7 @@ test.fixme('shows the unfilled quantity of a partly filled order', async () => {
|
|
|
30
30
|
|
|
31
31
|
```js
|
|
32
32
|
// SKIP: TRADE-500
|
|
33
|
-
test.skip('shows the unfilled quantity of a partly filled order @
|
|
33
|
+
test.skip('shows the unfilled quantity of a partly filled order @quarantined', async () => {});
|
|
34
34
|
```
|
|
35
35
|
|
|
36
36
|
Correct:
|
|
@@ -46,8 +46,8 @@ test.fixme('shows the unfilled quantity of a partly filled order', async () => {
|
|
|
46
46
|
|
|
47
47
|
```js
|
|
48
48
|
// SKIP: TRADE-500
|
|
49
|
-
//
|
|
50
|
-
test.skip('shows the unfilled quantity of a partly filled order @
|
|
49
|
+
// QUARANTINED: TRADE-500
|
|
50
|
+
test.skip('shows the unfilled quantity of a partly filled order @quarantined', async () => {});
|
|
51
51
|
```
|
|
52
52
|
|
|
53
53
|
## Options
|
|
@@ -8,13 +8,13 @@
|
|
|
8
8
|
|
|
9
9
|
<!-- end auto-generated rule header -->
|
|
10
10
|
|
|
11
|
-
Tags are matched exactly, so `@New` or `@
|
|
11
|
+
Tags are matched exactly, so `@New` or `@quarantiend` silently puts a test in no state at all. This
|
|
12
12
|
rule reports:
|
|
13
13
|
|
|
14
|
-
- `@new` together with `@
|
|
14
|
+
- `@new` together with `@quarantined` on the same test, including tags inherited from a `describe`;
|
|
15
15
|
- `@new` on a test that is skipped or fixme'd, since a test that doesn't run can't be promoted;
|
|
16
16
|
- a state tag written in the wrong case, such as `@New` (autofixed to `@new`);
|
|
17
|
-
- a near-miss spelling of a state tag, such as `@
|
|
17
|
+
- a near-miss spelling of a state tag, such as `@quarantiend`.
|
|
18
18
|
|
|
19
19
|
The `@new` checks run when the `new` state is on (`lifecycleTags: true`). Case and typo checks cover
|
|
20
20
|
every tag state that is on, including custom states. A tag kept in a `const` in the same file is
|
|
@@ -28,7 +28,7 @@ Incorrect:
|
|
|
28
28
|
<!-- example: invalid settings={"lifecycleTags":true} -->
|
|
29
29
|
|
|
30
30
|
```js
|
|
31
|
-
test('places a limit order @new @
|
|
31
|
+
test('places a limit order @new @quarantined', async () => {});
|
|
32
32
|
```
|
|
33
33
|
|
|
34
34
|
<!-- example: invalid settings={"lifecycleTags":true} -->
|
|
@@ -48,7 +48,7 @@ test('exports the trade history as CSV @New', async () => {});
|
|
|
48
48
|
<!-- example: invalid settings={"lifecycleTags":true} -->
|
|
49
49
|
|
|
50
50
|
```js
|
|
51
|
-
test('shows dividend payments in the account history @
|
|
51
|
+
test('shows dividend payments in the account history @quarantiend', async () => {});
|
|
52
52
|
```
|
|
53
53
|
|
|
54
54
|
Correct:
|
|
@@ -56,7 +56,7 @@ Correct:
|
|
|
56
56
|
<!-- example: valid settings={"lifecycleTags":true} -->
|
|
57
57
|
|
|
58
58
|
```js
|
|
59
|
-
test('places a limit order @
|
|
59
|
+
test('places a limit order @quarantined', async () => {});
|
|
60
60
|
```
|
|
61
61
|
|
|
62
62
|
## Options
|
|
@@ -64,7 +64,8 @@ test('places a limit order @unstable', async () => {});
|
|
|
64
64
|
This rule has no options of its own. It reads the shared options described in the
|
|
65
65
|
[README](../../README.md#options). These change what it reports:
|
|
66
66
|
|
|
67
|
-
- `lifecycleTags`, or `states` for `new` and `
|
|
68
|
-
is on, and `@new` together with `@
|
|
67
|
+
- `lifecycleTags`, or `states` for `new` and `quarantined`: the `@new` checks run when the `new` state
|
|
68
|
+
is on, and `@new` together with `@quarantined` is reported when both are on. A tag renamed in
|
|
69
|
+
`states`, such as `@flaky`, is checked in place of the default one.
|
|
69
70
|
- `customStates`: the tags of custom states get the same case and typo checks.
|
|
70
71
|
- `framework` and `testFunctions`: which functions count as the framework's `test`.
|
|
@@ -19,7 +19,7 @@ A test is in a state when it is:
|
|
|
19
19
|
| `fixme` | `test.fixme(...)`, `test.describe.fixme`, `testInfo.fixme()` | `// FIXME:` | yes |
|
|
20
20
|
| `todo` | Jest's and Vitest's `test.todo` | `// TODO:` | yes, in Jest and Vitest |
|
|
21
21
|
| `new` | the `@new` tag | `// NEW:` | with `lifecycleTags: true` |
|
|
22
|
-
| `
|
|
22
|
+
| `quarantined` | the `@quarantined` tag | `// QUARANTINED:` | with `lifecycleTags: true` |
|
|
23
23
|
| `fail`, `slow` | `test.fail(...)`, `test.slow()`, `testInfo.fail()`, `testInfo.slow()` | `// FAIL:`, `// SLOW:` | no |
|
|
24
24
|
| custom | any tag you list in `customStates` | your keyword | when configured |
|
|
25
25
|
|
|
@@ -30,7 +30,8 @@ In Jest and Vitest, `xit`, `xdescribe`, Vitest's `{ skip: true }`, `skipIf(true)
|
|
|
30
30
|
Tags are read from the title (`'places a limit order @new'`) and from `{ tag: '@new' }` or `{ tag: ['@new', '@smoke'] }`,
|
|
31
31
|
also when they come from a `const` or an enum declared in the same file, as in `{ tag: TAGS.NEW }`.
|
|
32
32
|
Vitest writes tag names without the `@`, so `{ tags: ['new'] }` is the `@new` tag. Tags on a
|
|
33
|
-
`describe` apply to every test inside it.
|
|
33
|
+
`describe` apply to every test inside it. `states` can rename the tag and marker of `new` and
|
|
34
|
+
`quarantined`, as in `states: { quarantined: { tag: '@flaky', marker: 'FLAKY' } }`.
|
|
34
35
|
|
|
35
36
|
## Marker format
|
|
36
37
|
|
|
@@ -87,7 +88,7 @@ If one test or describe has several markers for the same state, each must be val
|
|
|
87
88
|
Tags can only be read from text: a string or a template literal such as `` `buys ${symbol} @new` ``,
|
|
88
89
|
written in the call or kept in a `const` or an enum in the same file. A title built another way, such
|
|
89
90
|
as `'buys ' + symbol`, and a tag or details object imported from another module may hide a tag. With
|
|
90
|
-
`reportUnreadableTags: true`, the rule reports those whenever a tag state (`@new`, `@
|
|
91
|
+
`reportUnreadableTags: true`, the rule reports those whenever a tag state (`@new`, `@quarantined` or a
|
|
91
92
|
custom state) is on, so no tagged test slips past.
|
|
92
93
|
|
|
93
94
|
<!-- example: invalid settings={"lifecycleTags":true,"reportUnreadableTags":true} -->
|
|
@@ -233,8 +234,8 @@ This rule has no options of its own. It reads the shared options described in th
|
|
|
233
234
|
change what it reports:
|
|
234
235
|
|
|
235
236
|
- `ticket` and `placeholders`: which tickets are accepted.
|
|
236
|
-
- `lifecycleTags`, `states` and `customStates`: which states need a marker,
|
|
237
|
-
called, and, for a state with its own `ticket`, which tickets it accepts.
|
|
237
|
+
- `lifecycleTags`, `states` and `customStates`: which states need a marker, which tag puts a test in
|
|
238
|
+
each, what the marker is called, and, for a state with its own `ticket`, which tickets it accepts.
|
|
238
239
|
- `requireTicketForConditional`: conditional skips need a ticket too.
|
|
239
240
|
- `allowBlankLine`: blank lines may separate a marker from its test.
|
|
240
241
|
- `allowNotes`: text may follow the tickets.
|
package/docs/upgrading.md
CHANGED
|
@@ -1,5 +1,55 @@
|
|
|
1
1
|
# Upgrading
|
|
2
2
|
|
|
3
|
+
## Upgrading to 0.6
|
|
4
|
+
|
|
5
|
+
0.6.0 renames the `unstable` state to `quarantined`. Only projects that turn it on, with
|
|
6
|
+
`lifecycleTags: true` or `states.unstable`, need to change anything.
|
|
7
|
+
|
|
8
|
+
| What changed | Who it affects | What to do |
|
|
9
|
+
| ----------------------------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------- |
|
|
10
|
+
| [`@unstable` is now `@quarantined`](#the-quarantined-state) | Projects with `lifecycleTags: true` or `states.unstable` | Rename the tags and markers, or keep them with `states` |
|
|
11
|
+
|
|
12
|
+
### The quarantined state
|
|
13
|
+
|
|
14
|
+
The state's tag is now `@quarantined` and its marker `// QUARANTINED:`.
|
|
15
|
+
|
|
16
|
+
Before:
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
// UNSTABLE: TRADE-311
|
|
20
|
+
test('streams level 2 quotes @unstable', async ({ page }) => {});
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
After:
|
|
24
|
+
|
|
25
|
+
```ts
|
|
26
|
+
// QUARANTINED: TRADE-311
|
|
27
|
+
test('streams level 2 quotes @quarantined', async ({ page }) => {});
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
In Vitest, `{ tags: ['unstable'] }` becomes `{ tags: ['quarantined'] }` too. To keep the tags and
|
|
31
|
+
markers you have instead, name them in `states`:
|
|
32
|
+
|
|
33
|
+
```js
|
|
34
|
+
testGovernance.configure({
|
|
35
|
+
framework: 'playwright',
|
|
36
|
+
lifecycleTags: true,
|
|
37
|
+
states: { quarantined: { tag: '@unstable', marker: 'UNSTABLE' } },
|
|
38
|
+
});
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
A config that sets `states.unstable` fails when it loads, so rename it to `states.quarantined`:
|
|
42
|
+
|
|
43
|
+
```text
|
|
44
|
+
eslint-plugin-test-governance: invalid options:
|
|
45
|
+
- states.unstable was renamed to "quarantined"
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
A config that only sets `lifecycleTags: true` still loads, and nothing reports the old tag: an
|
|
49
|
+
`@unstable` test no longer needs a ticket, and its `// UNSTABLE:` marker is an ordinary comment. Before
|
|
50
|
+
you upgrade, list them with `grep -rni unstable tests/` (or wherever your tests are), then rename them
|
|
51
|
+
or keep them as above.
|
|
52
|
+
|
|
3
53
|
## Upgrading to 0.5
|
|
4
54
|
|
|
5
55
|
0.5.0 changes how the plugin is configured, ahead of 1.0. Most projects change one or two lines. A
|
package/package.json
CHANGED