@vaubaan/harness 2.0.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -15
- package/dist/cli/index.js +3 -3
- package/dist/cli/index.js.map +1 -1
- package/dist/commands/handbook.cmd.md +4 -3
- package/dist/commands/test-plan.cmd.md +4 -4
- package/dist/index.d.ts +1 -1
- package/dist/index.js +3 -3
- package/dist/index.js.map +1 -1
- package/dist/init.d.ts +1 -2
- package/dist/init.js +3 -3
- package/dist/init.js.map +1 -1
- package/dist/linters/index.js +2 -2
- package/dist/linters/index.js.map +1 -1
- package/dist/render.js +2 -2
- package/dist/render.js.map +1 -1
- package/dist/skills/decision-record-writer.skill.md +93 -130
- package/dist/skills/doc-audit.skill.md +3 -3
- package/dist/skills/drift-detector.skill.md +5 -6
- package/dist/skills/handbook-writer.skill.md +68 -159
- package/dist/skills/spec-writer.skill.md +25 -3
- package/dist/skills/test-plan-writer.skill.md +46 -53
- package/dist/skills/workflow-writer.skill.md +20 -3
- package/dist/templates/decision.md.tmpl +20 -47
- package/dist/templates/e2e-tests.md.tmpl +16 -45
- package/dist/templates/handbook.md.tmpl +20 -38
- package/dist/templates/integration-tests.md.tmpl +16 -34
- package/package.json +1 -1
|
@@ -1,62 +1,33 @@
|
|
|
1
1
|
---
|
|
2
|
+
title: E2E test plan
|
|
2
3
|
tracks:
|
|
3
|
-
- <
|
|
4
|
-
- <E2E_TEST_DIR_OR_FILE_2>
|
|
4
|
+
- <RUNNER_CONFIG_AND_TEST_INFRASTRUCTURE>
|
|
5
5
|
last_reviewed: <DATE>
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# E2E test plan
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
10
|
+
<!--
|
|
11
|
+
Never list the covered scenarios: the describe()/it() titles of the test
|
|
12
|
+
files are their source of truth. Document what the code cannot tell.
|
|
13
|
+
Target 20-50 lines, bullets. Do not restate the runner config.
|
|
14
|
+
-->
|
|
14
15
|
|
|
15
|
-
|
|
16
|
-
pass on every release. Exhaustive coverage required.
|
|
17
|
-
- **Edge cases**: less frequent but high-value scenarios
|
|
18
|
-
(errors, permissions, boundary states). Selective coverage.
|
|
16
|
+
## Scope
|
|
19
17
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
E2E tests are expensive, we reserve them for journeys that cannot
|
|
23
|
-
be validated otherwise.
|
|
18
|
+
- The complete system against real or staging dependencies, on the journeys that cannot be validated in [integration](./integration-tests.md).
|
|
19
|
+
- <project-specific boundary>
|
|
24
20
|
|
|
25
21
|
## Stack
|
|
26
22
|
|
|
27
|
-
-
|
|
28
|
-
- **Drivers:** <BROWSER_OR_API_DRIVER>
|
|
29
|
-
- **Target environment:** <ENV_TARGET>
|
|
30
|
-
- **Reporting:** <REPORT_TOOL>
|
|
31
|
-
|
|
32
|
-
## Fixture conventions
|
|
33
|
-
|
|
34
|
-
- **E2E organization:** each scenario runs in an isolated
|
|
35
|
-
fixture (`<E2E_ORG_NAMING_CONVENTION>`) to avoid
|
|
36
|
-
interference between parallel runs.
|
|
37
|
-
- **Naming:** <FIXTURE_NAMING_RULE>
|
|
38
|
-
- **Seed data:** <SEED_STRATEGY>
|
|
39
|
-
- **Authentication:** <AUTH_STRATEGY>
|
|
40
|
-
|
|
41
|
-
## Sweeper / cleanup
|
|
42
|
-
|
|
43
|
-
E2E fixtures inevitably leave artifacts behind (created orgs,
|
|
44
|
-
uploaded files, in-flight jobs). The sweeper:
|
|
45
|
-
|
|
46
|
-
- **Cadence:** <SWEEPER_FREQUENCY>
|
|
47
|
-
- **Purge criterion:** <PURGE_CRITERIA>
|
|
48
|
-
- **Logs:** <SWEEPER_LOG_LOCATION>
|
|
49
|
-
- **Dry-run mode:** <DRY_RUN_SWITCH>
|
|
23
|
+
- <framework, test environment, core mocks / fixtures — with the *why* of a non-obvious choice>
|
|
50
24
|
|
|
51
|
-
##
|
|
25
|
+
## Conventions
|
|
52
26
|
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
| E2E-02 | <SCENARIO_2_NAME> | edge | [[<SPEC_NAME_2>]] | <TEST_FILE_PATH_2> |
|
|
57
|
-
| E2E-NN | <SCENARIO_N_NAME> | golden | edge | [[<SPEC_NAME_N>]] | <TEST_FILE_PATH_N> |
|
|
27
|
+
- <where the tests live and how they are found>
|
|
28
|
+
- <shared fixtures, cleanup, rules that protect shared state>
|
|
29
|
+
- Dedicated test org / project: `<name>`; orphaned resource sweeper: `<script>`
|
|
58
30
|
|
|
59
31
|
## Known gaps
|
|
60
32
|
|
|
61
|
-
-
|
|
62
|
-
- <GAP_N> — ...
|
|
33
|
+
- ⚠️ **<what is not tested>** (spec [<name>](../spec/<name>.md)) — <reason, priority>
|
|
@@ -1,58 +1,40 @@
|
|
|
1
1
|
---
|
|
2
|
+
title: <FEATURE_NAME>
|
|
2
3
|
tracks:
|
|
3
|
-
- <
|
|
4
|
-
- <RELEVANT_CODE_PATH_2>
|
|
4
|
+
- <PATHS_WHOSE_CHANGE_CAN_MAKE_THIS_HANDBOOK_WRONG>
|
|
5
5
|
last_reviewed: <DATE>
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# <FEATURE_NAME> — handbook
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
<One sentence on the scope. Spec: [<name>](../spec/<name>.md).>
|
|
11
11
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
12
|
+
<!--
|
|
13
|
+
Only what an agent cannot read quickly in the code. Target 30-60 lines,
|
|
14
|
+
80 at most, bullets. Keep only the sections that have content, in this
|
|
15
|
+
order. Never describe files, functions, routes or fields one by one.
|
|
16
|
+
-->
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
handbook depends on. The `frontmatter-tracks` linter and the
|
|
18
|
-
`drift-detector` skill rely on it to flag drifts.
|
|
18
|
+
## Start here
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
- [`<path/to/entry>`](../../<path/to/entry>) — <what it is the entry to>
|
|
21
21
|
|
|
22
|
-
|
|
23
|
-
files (`[<path/to/file.ext>](../../<path/to/file.ext>)`). Cover:
|
|
22
|
+
## Invariants
|
|
24
23
|
|
|
25
|
-
-
|
|
26
|
-
- The control or data flow through the modules listed
|
|
27
|
-
in `tracks:`.
|
|
28
|
-
- The data structures / entities handled.
|
|
29
|
-
- The implicit assumptions that are not visible in the spec.
|
|
24
|
+
- <a rule that holds across several files and that no single file states>
|
|
30
25
|
|
|
31
|
-
|
|
26
|
+
## Traps
|
|
32
27
|
|
|
33
|
-
<
|
|
28
|
+
- <non-obvious behaviour, and why the next reader must not "fix" it>
|
|
34
29
|
|
|
35
|
-
|
|
30
|
+
## Outside the repo
|
|
36
31
|
|
|
37
|
-
<
|
|
32
|
+
- <setting in a dashboard, external registration, secret — and how to mirror it>
|
|
38
33
|
|
|
39
|
-
##
|
|
34
|
+
## Procedures
|
|
40
35
|
|
|
41
|
-
|
|
42
|
-
minimal invocation example. Mention the prerequisites (env
|
|
43
|
-
variables, initial state, dependencies).
|
|
36
|
+
- <manual checklist, rotation, step nobody can automate>
|
|
44
37
|
|
|
45
|
-
|
|
46
|
-
<USAGE_EXAMPLE>
|
|
47
|
-
```
|
|
38
|
+
## Debug
|
|
48
39
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
List the typical failures and their diagnosis.
|
|
52
|
-
|
|
53
|
-
- **Symptom:** <SYMPTOM_1> — **Likely cause:** <CAUSE_1> —
|
|
54
|
-
**Fix:** <FIX_1>.
|
|
55
|
-
- **Symptom:** <SYMPTOM_2> — ...
|
|
56
|
-
|
|
57
|
-
Also mention: where to look at the logs, which flags to enable for
|
|
58
|
-
verbose output, which tests reproduce the bug fastest.
|
|
40
|
+
- **<symptom>** — <cause, when it is not visible from the symptom>
|
|
@@ -1,50 +1,32 @@
|
|
|
1
1
|
---
|
|
2
|
+
title: Integration test plan
|
|
2
3
|
tracks:
|
|
3
|
-
- <
|
|
4
|
-
- <TEST_DIR_OR_FILE_2>
|
|
4
|
+
- <RUNNER_CONFIG_AND_TEST_INFRASTRUCTURE>
|
|
5
5
|
last_reviewed: <DATE>
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# Integration test plan
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
Not documented here.
|
|
16
|
-
- **Integration**: several modules assembled, real I/O against
|
|
17
|
-
local dependencies (ephemeral DB, service mocked at the network
|
|
18
|
-
level, temporary filesystem). This is the scope of this document.
|
|
19
|
-
- **E2E**: complete system against real or staging dependencies,
|
|
20
|
-
walking through a user journey. Documented in
|
|
21
|
-
[e2e-tests.md](./e2e-tests.md).
|
|
10
|
+
<!--
|
|
11
|
+
Never list the covered scenarios: the describe()/it() titles of the test
|
|
12
|
+
files are their source of truth. Document what the code cannot tell.
|
|
13
|
+
Target 20-50 lines, bullets. Do not restate the runner config.
|
|
14
|
+
-->
|
|
22
15
|
|
|
23
|
-
|
|
16
|
+
## Scope
|
|
24
17
|
|
|
25
|
-
|
|
18
|
+
- Several modules assembled in one process, real I/O against local dependencies (ephemeral DB, network-level mocks, temporary filesystem). Unit tests live next to the code; full-system journeys are in [e2e-tests.md](./e2e-tests.md).
|
|
19
|
+
- <project-specific boundary>
|
|
26
20
|
|
|
27
|
-
|
|
28
|
-
- **Assertions:** <ASSERTION_LIB>
|
|
29
|
-
- **Fixtures:** <FIXTURE_STRATEGY>
|
|
30
|
-
- **Setup / teardown:** <SETUP_TEARDOWN>
|
|
31
|
-
- **Data:** <DATA_STRATEGY>
|
|
21
|
+
## Stack
|
|
32
22
|
|
|
33
|
-
|
|
23
|
+
- <framework, test environment, core mocks / fixtures — with the *why* of a non-obvious choice>
|
|
34
24
|
|
|
35
|
-
|
|
36
|
-
|-------|-------------------------------------|-------------------------|------------------------------------|
|
|
37
|
-
| IT-01 | <SCENARIO_1_NAME> | [[<SPEC_NAME_1>]] | <TEST_FILE_PATH_1> |
|
|
38
|
-
| IT-02 | <SCENARIO_2_NAME> | [[<SPEC_NAME_2>]] | <TEST_FILE_PATH_2> |
|
|
39
|
-
| IT-NN | <SCENARIO_N_NAME> | [[<SPEC_NAME_N>]] | <TEST_FILE_PATH_N> |
|
|
25
|
+
## Conventions
|
|
40
26
|
|
|
41
|
-
|
|
42
|
-
|
|
27
|
+
- <where the tests live and how they are found>
|
|
28
|
+
- <shared fixtures, cleanup, rules that protect shared state>
|
|
43
29
|
|
|
44
30
|
## Known gaps
|
|
45
31
|
|
|
46
|
-
|
|
47
|
-
risk estimate and a pointer to the tracking ticket, if any.
|
|
48
|
-
|
|
49
|
-
- <GAP_1> — risk <LOW | MEDIUM | HIGH> — <TRACKING_REF>
|
|
50
|
-
- <GAP_N> — ...
|
|
32
|
+
- ⚠️ **<what is not tested>** (spec [<name>](../spec/<name>.md)) — <reason, priority>
|
package/package.json
CHANGED