@vaubaan/harness 2.0.0 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -21
- package/dist/cli/index.js +543 -76
- package/dist/cli/index.js.map +1 -1
- package/dist/commands/doc-audit.cmd.md +1 -1
- package/dist/commands/handbook.cmd.md +4 -3
- package/dist/commands/journey.cmd.md +22 -0
- package/dist/commands/story.cmd.md +24 -0
- package/dist/commands/test-plan.cmd.md +4 -4
- package/dist/index.d.ts +2 -2
- package/dist/index.js +304 -56
- package/dist/index.js.map +1 -1
- package/dist/init.d.ts +2 -3
- package/dist/init.js +301 -53
- package/dist/init.js.map +1 -1
- package/dist/linters/index.js +523 -56
- package/dist/linters/index.js.map +1 -1
- package/dist/parser/index.d.ts +110 -1
- package/dist/parser/index.js +229 -2
- package/dist/parser/index.js.map +1 -1
- package/dist/{render-C-FWmUfX.d.ts → render-BzrBb_l4.d.ts} +3 -1
- package/dist/render.d.ts +1 -1
- package/dist/render.js +89 -10
- package/dist/render.js.map +1 -1
- package/dist/skills/decision-record-writer.skill.md +93 -130
- package/dist/skills/doc-audit.skill.md +37 -7
- package/dist/skills/drift-detector.skill.md +41 -11
- package/dist/skills/handbook-writer.skill.md +71 -159
- package/dist/skills/journey-writer.skill.md +159 -0
- package/dist/skills/spec-writer.skill.md +25 -3
- package/dist/skills/test-plan-writer.skill.md +46 -53
- package/dist/skills/workflow-writer.skill.md +20 -3
- package/dist/templates/decision.md.tmpl +20 -47
- package/dist/templates/e2e-tests.md.tmpl +16 -45
- package/dist/templates/handbook.md.tmpl +20 -38
- package/dist/templates/integration-tests.md.tmpl +16 -34
- package/dist/templates/journey.md.tmpl +45 -0
- package/package.json +1 -1
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
id: test-plan-writer
|
|
3
3
|
description: |
|
|
4
4
|
Updates `docs/tests/integration-tests.md` and/or
|
|
5
|
-
`docs/tests/e2e-tests.md
|
|
6
|
-
|
|
7
|
-
`tracks: [...]` frontmatter for drift detection.
|
|
5
|
+
`docs/tests/e2e-tests.md` (scope, stack, conventions, known gaps).
|
|
6
|
+
Never lists the covered scenarios: the test code is their source of
|
|
7
|
+
truth. Maintains the `tracks: [...]` frontmatter for drift detection.
|
|
8
8
|
engines: [claude-code, codex-cli, opencode]
|
|
9
9
|
requires: [multi-step]
|
|
10
10
|
tags: [tests, doc, writer]
|
|
@@ -17,12 +17,25 @@ You are the Vaubaan harness's test plan writing wizard. The
|
|
|
17
17
|
required files: `integration-tests.md` and `e2e-tests.md`. Unit tests
|
|
18
18
|
live next to the code and do not need a separate plan.
|
|
19
19
|
|
|
20
|
+
The plans do **not** list the covered scenarios: the `describe()` /
|
|
21
|
+
`it()` titles of the test files already say what is tested, and a copy
|
|
22
|
+
in the docs only drifts. A plan documents what the code cannot tell:
|
|
23
|
+
the scope of the level, the stack, the conventions, and the known gaps.
|
|
24
|
+
|
|
25
|
+
The same rule applies to everything else: keep a sentence only if an
|
|
26
|
+
agent would not get it quickly from the code or another doc. Do not
|
|
27
|
+
restate what the runner config says (browsers, reporters, web server,
|
|
28
|
+
timeouts), what `docs/quality/code-quality.md` already says (file naming),
|
|
29
|
+
or placeholders ("to be defined"). Keep the *why* of a non-obvious choice,
|
|
30
|
+
the conventions that protect shared state, and the known gaps. Target 20-50
|
|
31
|
+
lines per plan, in bullets.
|
|
32
|
+
|
|
20
33
|
## When to invoke you
|
|
21
34
|
|
|
22
35
|
- The agent adds / modifies an integration or E2E test →
|
|
23
36
|
update the corresponding test plan.
|
|
24
|
-
- `doc-audit` shows that a spec has no associated test
|
|
25
|
-
|
|
37
|
+
- `doc-audit` shows that a spec has no associated test → add it to
|
|
38
|
+
the known gaps.
|
|
26
39
|
- `/test-plan [integration|e2e]` invoked by the human.
|
|
27
40
|
- Overhaul of the test scope: re-split integration vs e2e.
|
|
28
41
|
|
|
@@ -39,16 +52,14 @@ live next to the code and do not need a separate plan.
|
|
|
39
52
|
`**/*.int.test.ts`, or defined by the `package.json` scripts).
|
|
40
53
|
- For e2e: list `tests/e2e/`, `e2e/`, `playwright/`, or the
|
|
41
54
|
equivalent.
|
|
42
|
-
-
|
|
43
|
-
|
|
44
|
-
3. **Read the existing plan** if it exists.
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
- Scenarios in the plan but not in the code → mark as a "known
|
|
48
|
-
gap" or remove (ask the human).
|
|
55
|
+
- Use them to check that the documented stack and conventions
|
|
56
|
+
still match, not to copy the scenarios into the plan.
|
|
57
|
+
3. **Read the existing plan** if it exists. Flag the stack /
|
|
58
|
+
conventions statements the code contradicts, and known gaps that
|
|
59
|
+
are now covered by a test (→ remove, ask the human).
|
|
49
60
|
4. **Read the specs** under `docs/spec/` to identify the
|
|
50
|
-
subsystems/features
|
|
51
|
-
|
|
61
|
+
subsystems/features without any test at this level → known gap.
|
|
62
|
+
Spec neither tested nor listed as a gap = `doc-audit` warning.
|
|
52
63
|
5. **Load the appropriate template**.
|
|
53
64
|
6. **Draft** the required sections:
|
|
54
65
|
- **Scope**: what is tested at this level vs what is not
|
|
@@ -57,13 +68,11 @@ live next to the code and do not need a separate plan.
|
|
|
57
68
|
fixtures.
|
|
58
69
|
- **Conventions**: naming, file structure, shared fixtures,
|
|
59
70
|
cleanup.
|
|
60
|
-
- **
|
|
61
|
-
its source spec or handbook (`tracks` cross-link).
|
|
62
|
-
- **Known gaps**: scenarios identified as to be added, with
|
|
71
|
+
- **Known gaps**: what is not tested yet, with the linked spec,
|
|
63
72
|
priority and the reason for postponing.
|
|
64
|
-
7. **`tracks:` frontmatter**:
|
|
65
|
-
|
|
66
|
-
CI
|
|
73
|
+
7. **`tracks:` frontmatter**: the configs and test infrastructure the
|
|
74
|
+
plan describes (runner configs, shared fixtures, setup scripts, E2E
|
|
75
|
+
CI workflow) — not the individual test files.
|
|
67
76
|
8. **Submit** to the human. Iterate.
|
|
68
77
|
9. **Validate the `tracks:` before writing.** For each entry, check
|
|
69
78
|
that the path exists (file or matching glob). If not — it is
|
|
@@ -76,12 +85,11 @@ live next to the code and do not need a separate plan.
|
|
|
76
85
|
- **Clear integration vs E2E distinction**. Integration tests test the
|
|
77
86
|
interaction between several modules of the same process, with mocks
|
|
78
87
|
or in-memory DBs. E2E tests launch the full system (browser + live
|
|
79
|
-
backend + test DB) on golden paths.
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
- **Explicit known gaps**. Listing what we *do not test yet* is as
|
|
88
|
+
backend + test DB) on golden paths.
|
|
89
|
+
- **No covered-scenario list**. Not even a summary table: point to the
|
|
90
|
+
test files' location / naming convention instead.
|
|
91
|
+
- **Explicit known gaps**, each linked to its spec
|
|
92
|
+
(`docs/spec/<name>.md`). Listing what we *do not test yet* is as
|
|
85
93
|
important as listing what we test. Avoids false confidence.
|
|
86
94
|
- **E2E-specific conventions** (see harness spec §9.2): for
|
|
87
95
|
`e2e-tests.md`, document the e2e org (dedicated Stripe org /
|
|
@@ -96,7 +104,6 @@ live next to the code and do not need a separate plan.
|
|
|
96
104
|
---
|
|
97
105
|
title: Integration Tests Plan # or: E2E Tests Plan
|
|
98
106
|
tracks:
|
|
99
|
-
- packages/<name>/**/*.int.test.ts
|
|
100
107
|
- vitest.config.ts
|
|
101
108
|
- package.json
|
|
102
109
|
last_reviewed: YYYY-MM-DD
|
|
@@ -126,19 +133,6 @@ or e2e).>
|
|
|
126
133
|
- Dedicated test org / project: `<name>`
|
|
127
134
|
- Orphaned resource sweeper: `<script>`
|
|
128
135
|
|
|
129
|
-
## Covered scenarios
|
|
130
|
-
|
|
131
|
-
### <Subsystem / feature 1>
|
|
132
|
-
> Covers: docs/spec/<name>.md §X
|
|
133
|
-
|
|
134
|
-
- **scenario-name** — <1-line description>
|
|
135
|
-
- File: `path/to/test.int.test.ts`
|
|
136
|
-
- Setup: <precondition>
|
|
137
|
-
- Assert: <what is checked>
|
|
138
|
-
|
|
139
|
-
### <Subsystem / feature 2>
|
|
140
|
-
...
|
|
141
|
-
|
|
142
136
|
## Known gaps
|
|
143
137
|
|
|
144
138
|
- ⚠️ **<scenario-name>** (spec `<name>.md`) — not tested yet.
|
|
@@ -154,26 +148,25 @@ or e2e).>
|
|
|
154
148
|
|
|
155
149
|
Finish by:
|
|
156
150
|
- Listing the updated files.
|
|
157
|
-
- Listing the added / removed
|
|
151
|
+
- Listing the added / removed known gaps.
|
|
158
152
|
- Listing the specs still without tests (suggest a `/test-plan`
|
|
159
153
|
follow-up).
|
|
160
154
|
|
|
161
155
|
## Guardrails
|
|
162
156
|
|
|
163
|
-
- **Do not
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
- **No integration ↔ e2e duplication**. A scenario belongs to a
|
|
157
|
+
- **Do not duplicate the tests**. Scenarios live in the test code
|
|
158
|
+
only; the plan never lists them (only the untested ones, under
|
|
159
|
+
"Known gaps").
|
|
160
|
+
- **No integration ↔ e2e duplication**. A known gap belongs to a
|
|
168
161
|
single file. If you hesitate, ask the human.
|
|
169
162
|
- **No unit docs here**. Unit tests live next to the code and speak
|
|
170
163
|
for themselves. If someone asks to document them here, push back:
|
|
171
164
|
this is not the pillar.
|
|
172
|
-
- **Exhaustive `tracks:`**. Include
|
|
173
|
-
the
|
|
174
|
-
workflow). Without that,
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
165
|
+
- **Exhaustive `tracks:`**. Include every config and piece of test
|
|
166
|
+
infrastructure the plan describes (`vitest.config.ts`,
|
|
167
|
+
`playwright.config.ts`, fixtures, E2E CI workflow). Without that,
|
|
168
|
+
drift detection misses changes.
|
|
169
|
+
- **Do not remove a known gap without confirmation**. If a gap now
|
|
170
|
+
seems covered by a test, propose its removal and let the human
|
|
171
|
+
decide.
|
|
179
172
|
- **No auto-commit**.
|
|
@@ -17,6 +17,23 @@ You are the Vaubaan harness's workflow writing wizard. The
|
|
|
17
17
|
environment variants, PR process, branch model, release. One required
|
|
18
18
|
file in v1: `coding-workflow.md`.
|
|
19
19
|
|
|
20
|
+
## Content test
|
|
21
|
+
|
|
22
|
+
The doc is read by an agent about to change CI, deploy or the process.
|
|
23
|
+
Keep a sentence only if the agent would not get it quickly from the
|
|
24
|
+
configs or another doc:
|
|
25
|
+
|
|
26
|
+
- **Keep**: team conventions (PR, branches, commits), what lives outside
|
|
27
|
+
the repo (Vercel / Supabase / GitHub settings, integrations, secrets),
|
|
28
|
+
their traps, rollback, and the rules nobody can enforce mechanically.
|
|
29
|
+
- **Drop**: CI job steps, path lists, triggers or secrets readable in
|
|
30
|
+
`.github/workflows/`; rules already in `AGENTS.md` (language,
|
|
31
|
+
doc-first); architecture (→ ADR); feature config (→ handbook);
|
|
32
|
+
placeholders ("to be defined", "to be measured").
|
|
33
|
+
|
|
34
|
+
Target 60-100 lines, in bullets. The 5 required sections stay, even
|
|
35
|
+
short.
|
|
36
|
+
|
|
20
37
|
## When to invoke you
|
|
21
38
|
|
|
22
39
|
- The agent modifies a CI workflow, a deployment config, or a
|
|
@@ -45,9 +62,9 @@ file in v1: `coding-workflow.md`.
|
|
|
45
62
|
then confirm with the human.
|
|
46
63
|
3. **Load the template** `coding-workflow.md.tmpl`.
|
|
47
64
|
4. **Draft** the 5 required sections:
|
|
48
|
-
- **CI pipeline**:
|
|
49
|
-
|
|
50
|
-
|
|
65
|
+
- **CI pipeline**: link the workflows; state only which checks are
|
|
66
|
+
required, how to reproduce them locally, and the non-obvious
|
|
67
|
+
behaviours (conditional jobs, why a filter sits where it does).
|
|
51
68
|
- **Env variants**: `prod` / `staging` / `preview` / `local` —
|
|
52
69
|
where they run, which env vars distinguish them, how to switch
|
|
53
70
|
locally.
|
|
@@ -1,60 +1,33 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
1
|
+
---
|
|
2
|
+
title: <The decision in one sentence>
|
|
3
|
+
status: <accepted | proposed>
|
|
4
|
+
date: <DATE>
|
|
5
|
+
author: <AUTHOR>
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# <The decision in one sentence>
|
|
9
|
+
|
|
10
|
+
<!--
|
|
11
|
+
Only a structural decision: costly to reverse, a real alternative was
|
|
12
|
+
weighed, it outlives the ticket. About 40 lines, 60 at most. A decision
|
|
13
|
+
that changes is rewritten as a new file and the old one is deleted; the
|
|
14
|
+
old option goes to Rejected alternatives.
|
|
15
|
+
-->
|
|
7
16
|
|
|
8
17
|
## Context
|
|
9
18
|
|
|
10
|
-
|
|
11
|
-
are we solving? What constraints apply (technical, organizational,
|
|
12
|
-
time-related)? What signals triggered the discussion?
|
|
13
|
-
|
|
14
|
-
This section must be readable on its own: a reader discovering the
|
|
15
|
-
ADR six months later must understand why the question came up.
|
|
19
|
+
<2-4 sentences: the forces that made the choice necessary. No chronology.>
|
|
16
20
|
|
|
17
21
|
## Decision
|
|
18
22
|
|
|
19
|
-
|
|
20
|
-
"we will probably" — it is settled.
|
|
21
|
-
|
|
22
|
-
Then detail the conditions of application: scope, exceptions,
|
|
23
|
-
thresholds, effective dates.
|
|
23
|
+
<1-3 sentences in the present tense, stated as a rule.>
|
|
24
24
|
|
|
25
25
|
## Consequences
|
|
26
26
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
**Positive:**
|
|
30
|
-
|
|
31
|
-
- <POSITIVE_CONSEQUENCE_1>
|
|
32
|
-
- <POSITIVE_CONSEQUENCE_N>
|
|
33
|
-
|
|
34
|
-
**Negative:**
|
|
35
|
-
|
|
36
|
-
- <NEGATIVE_CONSEQUENCE_1>
|
|
37
|
-
- <NEGATIVE_CONSEQUENCE_N>
|
|
38
|
-
|
|
39
|
-
**Neutral / to watch:**
|
|
40
|
-
|
|
41
|
-
- <NEUTRAL_CONSEQUENCE_1>
|
|
27
|
+
- <constraint on future work>
|
|
42
28
|
|
|
43
29
|
## Rejected alternatives
|
|
44
30
|
|
|
45
|
-
|
|
46
|
-
with the reason for rejecting each one. This section is as important
|
|
47
|
-
as the decision itself — it avoids re-debating the same
|
|
48
|
-
alternatives later.
|
|
49
|
-
|
|
50
|
-
### Alternative A — <ALTERNATIVE_A_NAME>
|
|
51
|
-
|
|
52
|
-
<DESCRIPTION_A>
|
|
53
|
-
|
|
54
|
-
Rejected because: <REJECTION_REASON_A>.
|
|
55
|
-
|
|
56
|
-
### Alternative B — <ALTERNATIVE_B_NAME>
|
|
57
|
-
|
|
58
|
-
<DESCRIPTION_B>
|
|
31
|
+
### <Alternative>
|
|
59
32
|
|
|
60
|
-
|
|
33
|
+
<1-2 sentences: what it is, why not in this project.>
|
|
@@ -1,62 +1,33 @@
|
|
|
1
1
|
---
|
|
2
|
+
title: E2E test plan
|
|
2
3
|
tracks:
|
|
3
|
-
- <
|
|
4
|
-
- <E2E_TEST_DIR_OR_FILE_2>
|
|
4
|
+
- <RUNNER_CONFIG_AND_TEST_INFRASTRUCTURE>
|
|
5
5
|
last_reviewed: <DATE>
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# E2E test plan
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
10
|
+
<!--
|
|
11
|
+
Never list the covered scenarios: the describe()/it() titles of the test
|
|
12
|
+
files are their source of truth. Document what the code cannot tell.
|
|
13
|
+
Target 20-50 lines, bullets. Do not restate the runner config.
|
|
14
|
+
-->
|
|
14
15
|
|
|
15
|
-
|
|
16
|
-
pass on every release. Exhaustive coverage required.
|
|
17
|
-
- **Edge cases**: less frequent but high-value scenarios
|
|
18
|
-
(errors, permissions, boundary states). Selective coverage.
|
|
16
|
+
## Scope
|
|
19
17
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
E2E tests are expensive, we reserve them for journeys that cannot
|
|
23
|
-
be validated otherwise.
|
|
18
|
+
- The complete system against real or staging dependencies, on the journeys that cannot be validated in [integration](./integration-tests.md).
|
|
19
|
+
- <project-specific boundary>
|
|
24
20
|
|
|
25
21
|
## Stack
|
|
26
22
|
|
|
27
|
-
-
|
|
28
|
-
- **Drivers:** <BROWSER_OR_API_DRIVER>
|
|
29
|
-
- **Target environment:** <ENV_TARGET>
|
|
30
|
-
- **Reporting:** <REPORT_TOOL>
|
|
31
|
-
|
|
32
|
-
## Fixture conventions
|
|
33
|
-
|
|
34
|
-
- **E2E organization:** each scenario runs in an isolated
|
|
35
|
-
fixture (`<E2E_ORG_NAMING_CONVENTION>`) to avoid
|
|
36
|
-
interference between parallel runs.
|
|
37
|
-
- **Naming:** <FIXTURE_NAMING_RULE>
|
|
38
|
-
- **Seed data:** <SEED_STRATEGY>
|
|
39
|
-
- **Authentication:** <AUTH_STRATEGY>
|
|
40
|
-
|
|
41
|
-
## Sweeper / cleanup
|
|
42
|
-
|
|
43
|
-
E2E fixtures inevitably leave artifacts behind (created orgs,
|
|
44
|
-
uploaded files, in-flight jobs). The sweeper:
|
|
45
|
-
|
|
46
|
-
- **Cadence:** <SWEEPER_FREQUENCY>
|
|
47
|
-
- **Purge criterion:** <PURGE_CRITERIA>
|
|
48
|
-
- **Logs:** <SWEEPER_LOG_LOCATION>
|
|
49
|
-
- **Dry-run mode:** <DRY_RUN_SWITCH>
|
|
23
|
+
- <framework, test environment, core mocks / fixtures — with the *why* of a non-obvious choice>
|
|
50
24
|
|
|
51
|
-
##
|
|
25
|
+
## Conventions
|
|
52
26
|
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
| E2E-02 | <SCENARIO_2_NAME> | edge | [[<SPEC_NAME_2>]] | <TEST_FILE_PATH_2> |
|
|
57
|
-
| E2E-NN | <SCENARIO_N_NAME> | golden | edge | [[<SPEC_NAME_N>]] | <TEST_FILE_PATH_N> |
|
|
27
|
+
- <where the tests live and how they are found>
|
|
28
|
+
- <shared fixtures, cleanup, rules that protect shared state>
|
|
29
|
+
- Dedicated test org / project: `<name>`; orphaned resource sweeper: `<script>`
|
|
58
30
|
|
|
59
31
|
## Known gaps
|
|
60
32
|
|
|
61
|
-
-
|
|
62
|
-
- <GAP_N> — ...
|
|
33
|
+
- ⚠️ **<what is not tested>** (spec [<name>](../spec/<name>.md)) — <reason, priority>
|
|
@@ -1,58 +1,40 @@
|
|
|
1
1
|
---
|
|
2
|
+
title: <FEATURE_NAME>
|
|
2
3
|
tracks:
|
|
3
|
-
- <
|
|
4
|
-
- <RELEVANT_CODE_PATH_2>
|
|
4
|
+
- <PATHS_WHOSE_CHANGE_CAN_MAKE_THIS_HANDBOOK_WRONG>
|
|
5
5
|
last_reviewed: <DATE>
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# <FEATURE_NAME> — handbook
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
<One sentence on the scope. Spec: [<name>](../spec/<name>.md).>
|
|
11
11
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
12
|
+
<!--
|
|
13
|
+
Only what an agent cannot read quickly in the code. Target 30-60 lines,
|
|
14
|
+
80 at most, bullets. Keep only the sections that have content, in this
|
|
15
|
+
order. Never describe files, functions, routes or fields one by one.
|
|
16
|
+
-->
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
handbook depends on. The `frontmatter-tracks` linter and the
|
|
18
|
-
`drift-detector` skill rely on it to flag drifts.
|
|
18
|
+
## Start here
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
- [`<path/to/entry>`](../../<path/to/entry>) — <what it is the entry to>
|
|
21
21
|
|
|
22
|
-
|
|
23
|
-
files (`[<path/to/file.ext>](../../<path/to/file.ext>)`). Cover:
|
|
22
|
+
## Invariants
|
|
24
23
|
|
|
25
|
-
-
|
|
26
|
-
- The control or data flow through the modules listed
|
|
27
|
-
in `tracks:`.
|
|
28
|
-
- The data structures / entities handled.
|
|
29
|
-
- The implicit assumptions that are not visible in the spec.
|
|
24
|
+
- <a rule that holds across several files and that no single file states>
|
|
30
25
|
|
|
31
|
-
|
|
26
|
+
## Traps
|
|
32
27
|
|
|
33
|
-
<
|
|
28
|
+
- <non-obvious behaviour, and why the next reader must not "fix" it>
|
|
34
29
|
|
|
35
|
-
|
|
30
|
+
## Outside the repo
|
|
36
31
|
|
|
37
|
-
<
|
|
32
|
+
- <setting in a dashboard, external registration, secret — and how to mirror it>
|
|
38
33
|
|
|
39
|
-
##
|
|
34
|
+
## Procedures
|
|
40
35
|
|
|
41
|
-
|
|
42
|
-
minimal invocation example. Mention the prerequisites (env
|
|
43
|
-
variables, initial state, dependencies).
|
|
36
|
+
- <manual checklist, rotation, step nobody can automate>
|
|
44
37
|
|
|
45
|
-
|
|
46
|
-
<USAGE_EXAMPLE>
|
|
47
|
-
```
|
|
38
|
+
## Debug
|
|
48
39
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
List the typical failures and their diagnosis.
|
|
52
|
-
|
|
53
|
-
- **Symptom:** <SYMPTOM_1> — **Likely cause:** <CAUSE_1> —
|
|
54
|
-
**Fix:** <FIX_1>.
|
|
55
|
-
- **Symptom:** <SYMPTOM_2> — ...
|
|
56
|
-
|
|
57
|
-
Also mention: where to look at the logs, which flags to enable for
|
|
58
|
-
verbose output, which tests reproduce the bug fastest.
|
|
40
|
+
- **<symptom>** — <cause, when it is not visible from the symptom>
|
|
@@ -1,50 +1,32 @@
|
|
|
1
1
|
---
|
|
2
|
+
title: Integration test plan
|
|
2
3
|
tracks:
|
|
3
|
-
- <
|
|
4
|
-
- <TEST_DIR_OR_FILE_2>
|
|
4
|
+
- <RUNNER_CONFIG_AND_TEST_INFRASTRUCTURE>
|
|
5
5
|
last_reviewed: <DATE>
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# Integration test plan
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
Not documented here.
|
|
16
|
-
- **Integration**: several modules assembled, real I/O against
|
|
17
|
-
local dependencies (ephemeral DB, service mocked at the network
|
|
18
|
-
level, temporary filesystem). This is the scope of this document.
|
|
19
|
-
- **E2E**: complete system against real or staging dependencies,
|
|
20
|
-
walking through a user journey. Documented in
|
|
21
|
-
[e2e-tests.md](./e2e-tests.md).
|
|
10
|
+
<!--
|
|
11
|
+
Never list the covered scenarios: the describe()/it() titles of the test
|
|
12
|
+
files are their source of truth. Document what the code cannot tell.
|
|
13
|
+
Target 20-50 lines, bullets. Do not restate the runner config.
|
|
14
|
+
-->
|
|
22
15
|
|
|
23
|
-
|
|
16
|
+
## Scope
|
|
24
17
|
|
|
25
|
-
|
|
18
|
+
- Several modules assembled in one process, real I/O against local dependencies (ephemeral DB, network-level mocks, temporary filesystem). Unit tests live next to the code; full-system journeys are in [e2e-tests.md](./e2e-tests.md).
|
|
19
|
+
- <project-specific boundary>
|
|
26
20
|
|
|
27
|
-
|
|
28
|
-
- **Assertions:** <ASSERTION_LIB>
|
|
29
|
-
- **Fixtures:** <FIXTURE_STRATEGY>
|
|
30
|
-
- **Setup / teardown:** <SETUP_TEARDOWN>
|
|
31
|
-
- **Data:** <DATA_STRATEGY>
|
|
21
|
+
## Stack
|
|
32
22
|
|
|
33
|
-
|
|
23
|
+
- <framework, test environment, core mocks / fixtures — with the *why* of a non-obvious choice>
|
|
34
24
|
|
|
35
|
-
|
|
36
|
-
|-------|-------------------------------------|-------------------------|------------------------------------|
|
|
37
|
-
| IT-01 | <SCENARIO_1_NAME> | [[<SPEC_NAME_1>]] | <TEST_FILE_PATH_1> |
|
|
38
|
-
| IT-02 | <SCENARIO_2_NAME> | [[<SPEC_NAME_2>]] | <TEST_FILE_PATH_2> |
|
|
39
|
-
| IT-NN | <SCENARIO_N_NAME> | [[<SPEC_NAME_N>]] | <TEST_FILE_PATH_N> |
|
|
25
|
+
## Conventions
|
|
40
26
|
|
|
41
|
-
|
|
42
|
-
|
|
27
|
+
- <where the tests live and how they are found>
|
|
28
|
+
- <shared fixtures, cleanup, rules that protect shared state>
|
|
43
29
|
|
|
44
30
|
## Known gaps
|
|
45
31
|
|
|
46
|
-
|
|
47
|
-
risk estimate and a pointer to the tracking ticket, if any.
|
|
48
|
-
|
|
49
|
-
- <GAP_1> — risk <LOW | MEDIUM | HIGH> — <TRACKING_REF>
|
|
50
|
-
- <GAP_N> — ...
|
|
32
|
+
- ⚠️ **<what is not tested>** (spec [<name>](../spec/<name>.md)) — <reason, priority>
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: <VERB + OBJECT>
|
|
3
|
+
status: proposed
|
|
4
|
+
actors: [<ACTOR>]
|
|
5
|
+
spec: ../spec/<SUBSYSTEM>.md
|
|
6
|
+
last_reviewed: <DATE>
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# <VERB + OBJECT>
|
|
10
|
+
|
|
11
|
+
As a <actor>, I want <intention> so that <outcome>.
|
|
12
|
+
|
|
13
|
+
<!--
|
|
14
|
+
One file per level-1 intention; the levels below are nested headings. A
|
|
15
|
+
leaf is a heading with no sub-heading: it carries its fields on the lines
|
|
16
|
+
right under the title, one field per line, values comma-separated, and no
|
|
17
|
+
prose. An internal node may carry one sentence of user story.
|
|
18
|
+
|
|
19
|
+
Fields of a leaf:
|
|
20
|
+
requires: / creates: entities of the linked spec's Vocabulary (kebab-case)
|
|
21
|
+
anchors: paths or globs of the code and tests (implemented)
|
|
22
|
+
acceptance: observable criteria as a list (proposed)
|
|
23
|
+
status: proposed | implemented | deprecated, when it differs
|
|
24
|
+
from the file's
|
|
25
|
+
spec: link to the spec section that holds the rule
|
|
26
|
+
|
|
27
|
+
Targets: 2-9 children per node, 40-120 lines per file. Longer: split into
|
|
28
|
+
two journeys. The rule lives in the spec, the how in the handbook.
|
|
29
|
+
-->
|
|
30
|
+
|
|
31
|
+
## <Verb + object>
|
|
32
|
+
|
|
33
|
+
requires: <entity>
|
|
34
|
+
creates: <entity>
|
|
35
|
+
anchors: <path or glob>, <path or glob>
|
|
36
|
+
|
|
37
|
+
## <Verb + object>
|
|
38
|
+
|
|
39
|
+
### <Verb + object>
|
|
40
|
+
|
|
41
|
+
status: proposed
|
|
42
|
+
requires: <entity>
|
|
43
|
+
acceptance:
|
|
44
|
+
- <observable criterion>
|
|
45
|
+
- <observable criterion>
|
package/package.json
CHANGED