@boopkit/cap-testing 0.5.2 → 0.5.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -58,6 +58,35 @@ A skip carrying neither a ticket reference nor a `SKIP-OK:` reason is a conventi
|
|
|
58
58
|
|
|
59
59
|
---
|
|
60
60
|
|
|
61
|
+
## Integration tests never touch the shared dev database
|
|
62
|
+
|
|
63
|
+
A test or migration that needs a real database must not run against the checkout's inherited connection string.
|
|
64
|
+
In a worktree that is the operator's live dev database, and **file isolation is not data isolation**.
|
|
65
|
+
Measured cost: a loop's cutover migration ran against the shared dev database and destroyed the operator's live data.
|
|
66
|
+
An agent that applies migrations by hand, outside the test harness, must still create and point at a throwaway database first, and say so in its run record.
|
|
67
|
+
|
|
68
|
+
Make the harness handle this so nobody has to remember it.
|
|
69
|
+
Ask git whether the run is in a linked worktree, and in one, derive a per-worktree test database name from the worktree's absolute path — the same name on every run in that worktree, so the one-time migration cost is paid once.
|
|
70
|
+
The main checkout keeps the ordinary shared test database, and an explicit test-database override still wins over both, so it remains the escape hatch when you want a particular database.
|
|
71
|
+
|
|
72
|
+
Without that derivation, two concurrent runs both resolve to one test database and each suite's setup truncates the other's rows.
|
|
73
|
+
What that produces is not an obvious collision: a suite passes alone, fails in a full run, and fails on unmodified `HEAD` too, which reads as pre-existing flakiness in your own code and invites an agent to weaken real assertions until the phantom failures stop.
|
|
74
|
+
If you see that pattern, check which database the run resolved to before you touch an assertion.
|
|
75
|
+
|
|
76
|
+
## A suite that passes alone and fails in a full run is a suspect, not a verdict
|
|
77
|
+
|
|
78
|
+
Inter-suite interference — leaked state, a shared fixture, a revision conflict under concurrency — produces exactly the shape of a real regression.
|
|
79
|
+
Rerun before you touch an assertion, and never make a red test green by weakening what it asserts.
|
|
80
|
+
The failure that invites that is exactly the failure that is lying to you.
|
|
81
|
+
|
|
82
|
+
If a suite is a known intermittent offender, say so in writing where the next reader will look, for the same reason the shared-database hazard is written down: an undocumented intermittent failure gets "fixed" by deleting the assertion that caught it.
|
|
83
|
+
|
|
84
|
+
## A guard is only a guard if it can tell it is talking to your application
|
|
85
|
+
|
|
86
|
+
A test that gates its live cases on a generic liveness probe — a plain health endpoint returning ok — is trusting that nothing else could answer.
|
|
87
|
+
Measured: an unrelated process on the operator's machine answered that probe while returning 404 for every real route, so cases that should have skipped ran and failed.
|
|
88
|
+
Make the guard assert something only this application would say.
|
|
89
|
+
|
|
61
90
|
## Generic Testing Approach
|
|
62
91
|
|
|
63
92
|
These principles apply across all subsystems.
|