@solaqua/ghosts 0.3.1 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/common/AGENTS.md +0 -11
- package/engineering/skills/code-review/SKILL.md +14 -7
- package/engineering/skills/coding/SKILL.md +15 -0
- package/engineering/skills/github-issue/SKILL.md +35 -0
- package/engineering/skills/slop-shaking/SKILL.md +35 -0
- package/engineering/skills/stack-pr/SKILL.md +39 -0
- package/engineering/skills/test-review/SKILL.md +26 -12
- package/engineering/skills/work-breakdown/SKILL.md +37 -5
- package/package.json +1 -1
- package/research/skills/deep-research/SKILL.md +30 -0
- package/writing/skills/make-skill/SKILL.md +15 -0
- package/engineering/skills/loop-coding/SKILL.md +0 -48
- package/engineering/skills/loop-coding/templates/plan.md +0 -14
- package/engineering/skills/loop-plan/SKILL.md +0 -220
- package/engineering/skills/loop-plan/templates/plan.md +0 -39
- package/engineering/skills/test-review/references/backend.md +0 -145
- package/engineering/skills/test-review/references/common.md +0 -92
- package/engineering/skills/test-review/references/frontend.md +0 -272
- package/engineering/skills/test-review/templates/review.md +0 -21
- package/research/skills/loop-research/SKILL.md +0 -129
- package/research/skills/loop-research/templates/decision-memo.md +0 -30
- package/writing/skills/loop-skills/SKILL.md +0 -101
|
@@ -1,220 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: ghosts:loop-plan
|
|
3
|
-
description: Planning-only loop for complex development and design problems, adaptable to other domains; uses evidence, alternatives, counterarguments, and premortems to resolve material uncertainty or costly tradeoffs. Skip routine implementation planning.
|
|
4
|
-
disable-model-invocation: true
|
|
5
|
-
---
|
|
6
|
-
# Loop Plan
|
|
7
|
-
|
|
8
|
-
Create a plan another person or agent can execute without rediscovering key
|
|
9
|
-
decisions. Inspect context and write planning artifacts only; stop before
|
|
10
|
-
implementation or invoking an execution skill. Scale depth to uncertainty and
|
|
11
|
-
the cost of a wrong decision, not the number of tasks or document length. For
|
|
12
|
-
other domains, replace files, interfaces, and tests with the relevant
|
|
13
|
-
deliverables, dependencies, and verification; omit software-only checks.
|
|
14
|
-
Write responses and the plan in plain, natural English, or in Korean when the
|
|
15
|
-
user writes in Korean, including template headings.
|
|
16
|
-
|
|
17
|
-
## Runtime loop
|
|
18
|
-
|
|
19
|
-
Before acting, create `/tmp/loop-plan-<safe-task-slug>.md` with a unique slug
|
|
20
|
-
or timestamp, `Current state: Pass 1 / Next: Understand`, the checklist below,
|
|
21
|
-
and an empty append-only `Repair log:`. Process items in order, finish each
|
|
22
|
-
before moving on, record a short outcome, and update `Current state`. Mark
|
|
23
|
-
`[x]` only after completion and `[-]` only with a reason; never erase or
|
|
24
|
-
uncheck historical rows. After a repair, append a numbered `[ ] Recheck P2: ...`
|
|
25
|
-
row for the earliest affected item and record the trigger, change, rechecked
|
|
26
|
-
items, and result in the repair log. Finish only after final validation.
|
|
27
|
-
|
|
28
|
-
```text
|
|
29
|
-
- [ ] Understand the outcome, constraints, and relevant evidence
|
|
30
|
-
- [ ] Resolve research and user-decision dependencies before framing
|
|
31
|
-
- [ ] Frame success, scope, principles, and decision drivers
|
|
32
|
-
- [ ] Set the review budget from uncertainty, impact, and reversibility
|
|
33
|
-
- [ ] Compare viable approaches and choose a provisional direction
|
|
34
|
-
- [ ] Draft ordered work packages and validation
|
|
35
|
-
- [ ] Challenge the design and run a premortem
|
|
36
|
-
- [ ] Any unsupported assumption, unresolved decision, or stronger alternative that changes the plan?
|
|
37
|
-
- [ ] Any unaddressed failure, dependency, or unverifiable completion criterion?
|
|
38
|
-
- [ ] Any unnecessary scope, complexity, or detail?
|
|
39
|
-
- [ ] Final validation and delivery of the plan
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
Answer every `Any ...?` row Yes or No. On Yes, repair the smallest material
|
|
43
|
-
issue and return to the earliest affected step before repeating the review. If
|
|
44
|
-
progress depends on missing evidence or a user decision, identify the exact
|
|
45
|
-
blocker instead of repeating the review or claiming readiness.
|
|
46
|
-
|
|
47
|
-
## Adaptive review
|
|
48
|
-
|
|
49
|
-
Do not make the user choose a mode. After framing, keep a decision ledger with
|
|
50
|
-
each material decision's impact, uncertainty, reversibility, and needed
|
|
51
|
-
evidence. Use qualitative `low`, `medium`, or `high` values and states such as
|
|
52
|
-
`open`, `investigating`, `supported`, `disputed`, `gated`, and `closed`.
|
|
53
|
-
Require at least one independent targeted review for a decision with high
|
|
54
|
-
impact and high uncertainty, low reversibility, or cross-component effects. A
|
|
55
|
-
request for `ralplan` or subagents raises the review budget without imposing a
|
|
56
|
-
fixed ceremony.
|
|
57
|
-
|
|
58
|
-
```text
|
|
59
|
-
[Understand] → [Route unknowns]
|
|
60
|
-
├─ evidence → [$ghosts:loop-research]┐
|
|
61
|
-
├─ user decision → [$ghosts:grill-me] ├→ [Frame + ledger] → [Draft]
|
|
62
|
-
└─ known →──────────────────────────┘ ↓
|
|
63
|
-
[Any high-impact uncertainty?]
|
|
64
|
-
├─ No → [Self challenge]
|
|
65
|
-
└─ Yes → [Targeted subagent questions]
|
|
66
|
-
→ [Synthesize] → [Revise] ↺
|
|
67
|
-
→ [Premortem] → [Any material issue?]
|
|
68
|
-
→ [Validate + deliver]
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
For each unresolved ledger item, record a question ID, covered decision IDs,
|
|
72
|
-
capability, required inputs, `depends_on` question IDs, decision criterion,
|
|
73
|
-
and required evidence. Dispatch only ready questions; run independent ones in
|
|
74
|
-
parallel and refresh the snapshot after upstream results.
|
|
75
|
-
|
|
76
|
-
Choose a narrow capability rather than a permanent role:
|
|
77
|
-
|
|
78
|
-
- **Investigation:** establish a missing fact, constraint, dependency, or
|
|
79
|
-
feasibility signal.
|
|
80
|
-
- **Design challenge:** test boundaries, responsibilities, dependency
|
|
81
|
-
direction, and the strongest case against the chosen approach.
|
|
82
|
-
- **Execution challenge:** test prerequisites, rollout, recovery,
|
|
83
|
-
observability, and demonstrable completion.
|
|
84
|
-
- **Adversarial challenge:** identify a credible failure or attack path when
|
|
85
|
-
relevant.
|
|
86
|
-
|
|
87
|
-
Give each subagent the exact question, context snapshot, candidate options,
|
|
88
|
-
decision criterion, and required evidence. Keep it read-only and request
|
|
89
|
-
`finding`, `evidence`, `consequence`, and `recommendation`. Record each call as
|
|
90
|
-
`complete`, `failed`, `timed_out`, or `invalid`. Retry or reassign once when
|
|
91
|
-
useful, counting that attempt against the single follow-up budget; then use
|
|
92
|
-
self-review only when the risk permits. An unresolved high-impact item stays
|
|
93
|
-
`conditional` or `blocked`.
|
|
94
|
-
|
|
95
|
-
When two independent reviews are warranted, give them the same snapshot without
|
|
96
|
-
either review's findings, then record disagreements and their resolution. The
|
|
97
|
-
primary agent alone synthesizes the findings, records which decision changed
|
|
98
|
-
and why, and revises the draft.
|
|
99
|
-
|
|
100
|
-
When a finding changes a material decision, scope, or acceptance criterion,
|
|
101
|
-
send the revised delta to the relevant completed reviewer for one confirmation
|
|
102
|
-
pass. Ask whether the original concern is resolved, what new consequence the
|
|
103
|
-
revision creates, and whether a material objection remains. Record the response
|
|
104
|
-
and any objection in the question record and decision ledger. Skip the pass
|
|
105
|
-
when no material change occurred; when used, it counts against the single
|
|
106
|
-
follow-up budget and is review evidence, not user approval.
|
|
107
|
-
Preserve blind independence for reviews that have not run. If confirmation is
|
|
108
|
-
unavailable or a material objection remains, keep the item `conditional` or
|
|
109
|
-
`blocked`.
|
|
110
|
-
|
|
111
|
-
Set a bounded budget at delegation: one initial question per high-impact item,
|
|
112
|
-
one follow-up per item for conflicting, insufficient, or materially changed
|
|
113
|
-
evidence or drafts (including retry, reassignment, or confirmation), and at
|
|
114
|
-
most five synthesis passes. Repeat only while a material item lacks support or
|
|
115
|
-
a credible objection remains. Stop when high-impact decisions have evidence or
|
|
116
|
-
explicit gates, no unresolved objection changes the recommendation, and
|
|
117
|
-
acceptance and verification criteria are concrete. If the budget is exhausted
|
|
118
|
-
or the same material objection survives two revisions, deliver `conditional`
|
|
119
|
-
or `blocked` with the exact reason. If delegation is unavailable, perform the
|
|
120
|
-
same questions as self-review and state that no independent review occurred.
|
|
121
|
-
Subagent findings are review evidence, never user approval or authorization to
|
|
122
|
-
implement.
|
|
123
|
-
|
|
124
|
-
For high-risk work, deepen relevant challenges with three premortem scenarios
|
|
125
|
-
and the verification levels the problem needs. Do not add irrelevant test
|
|
126
|
-
layers to a design-only problem.
|
|
127
|
-
|
|
128
|
-
## Understand and frame
|
|
129
|
-
|
|
130
|
-
Inspect relevant code, documents, existing plans, and constraints before asking
|
|
131
|
-
questions that inspection can answer. Distinguish observed facts, inferences,
|
|
132
|
-
and assumptions; cite supporting files or sources. Identify the observable
|
|
133
|
-
outcome, non-goals, affected users or systems, and material constraints.
|
|
134
|
-
|
|
135
|
-
Route unresolved inputs before making design decisions. Use `$ghosts:loop-research`
|
|
136
|
-
for repository or document investigation, current or contested evidence,
|
|
137
|
-
comparisons, feasibility checks, or source citations; it returns verified
|
|
138
|
-
claims, provenance, uncertainty, and leads that could change the plan. Use
|
|
139
|
-
`$ghosts:grill-me` for choices only the user can make, such as outcome, scope,
|
|
140
|
-
priority, risk tolerance, or external action. It asks the whole ready frontier
|
|
141
|
-
in each round and returns a short, grouped decision summary. Wait for the user
|
|
142
|
-
to confirm the shared understanding before carrying its decisions into the plan.
|
|
143
|
-
Do not
|
|
144
|
-
ask the user for facts that inspection or research can establish, or infer a
|
|
145
|
-
material preference. Carry both outputs into the decision ledger.
|
|
146
|
-
|
|
147
|
-
Resolve material user choices through the interview's dependency-aware rounds;
|
|
148
|
-
continue independent inspection while waiting. State ordinary assumptions only
|
|
149
|
-
when they do not replace a material user decision. If feasibility remains
|
|
150
|
-
unestablished, put the cheapest useful investigation first with a decision
|
|
151
|
-
criterion and keep dependent work conditional rather than inventing certainty.
|
|
152
|
-
State task-specific principles, rank decision drivers, resolve conflicts
|
|
153
|
-
explicitly, and define observable success before choosing.
|
|
154
|
-
|
|
155
|
-
## Compare and draft
|
|
156
|
-
|
|
157
|
-
Compare at least two genuinely viable approaches against the same principles
|
|
158
|
-
and drivers, including a smaller change or no change when viable. Give each its
|
|
159
|
-
strongest case, cost, and downside. If only one survives the constraints,
|
|
160
|
-
explain why the alternatives fail. Recommend a direction, name the accepted
|
|
161
|
-
tradeoff, and state what evidence would reverse it.
|
|
162
|
-
|
|
163
|
-
Break the direction into dependency-ordered, independently verifiable work
|
|
164
|
-
packages. For each, name the outcome, affected components or files, important
|
|
165
|
-
interfaces, prerequisites, and completion evidence. Prefer end-to-end slices;
|
|
166
|
-
mark proposed paths as new and verify existing references. Specify correctness
|
|
167
|
-
contracts and ordering without prescribing incidental implementation details.
|
|
168
|
-
Do not invent precise estimates or parallelism without a basis.
|
|
169
|
-
|
|
170
|
-
Map success criteria to concrete checks, including material failure paths and
|
|
171
|
-
relevant existing tests. Add tests only for meaningful behavior. Include
|
|
172
|
-
rollout, observability, recovery, or stop conditions when operational
|
|
173
|
-
consequences require them.
|
|
174
|
-
|
|
175
|
-
## Challenge and repair
|
|
176
|
-
|
|
177
|
-
Review a fixed draft in two separate self-review passes before revising:
|
|
178
|
-
|
|
179
|
-
1. **Design challenge:** make the strongest case against the recommendation;
|
|
180
|
-
check principle violations, responsibility boundaries, dependency direction,
|
|
181
|
-
and whether a simpler alternative achieves the outcome. Resolve a real
|
|
182
|
-
tension through evidence or an explicit tradeoff.
|
|
183
|
-
2. **Execution challenge:** walk through representative work packages as an
|
|
184
|
-
implementer, including a failure path; check prerequisites, missing
|
|
185
|
-
decisions, feasibility, and demonstrable completion.
|
|
186
|
-
|
|
187
|
-
These are not independent agent consensus. If the user requests separate
|
|
188
|
-
reviewers, give them the same draft and source context without the other
|
|
189
|
-
review's conclusions, combine findings only after both finish, and do not claim
|
|
190
|
-
independent review unless it occurred.
|
|
191
|
-
|
|
192
|
-
Run a premortem: assume failure at a meaningful milestone and trace the most
|
|
193
|
-
consequential plausible failures to plan details. For each, capture the causal
|
|
194
|
-
sequence, enabling assumption, likelihood and impact separately, earliest
|
|
195
|
-
warning, check time, mitigation, and revision or stop trigger. Label unverified
|
|
196
|
-
scenarios as hypotheses and thresholds as provisional; do not invent numerical
|
|
197
|
-
probabilities or promise to eliminate every risk. Include an adversary's
|
|
198
|
-
response only when adversarial behavior is relevant to the actual problem.
|
|
199
|
-
|
|
200
|
-
Use findings to change the plan and give a short reason for each material
|
|
201
|
-
revision. Put cheap checks of critical assumptions before expensive dependent
|
|
202
|
-
work. Resolve material objections or expose them as blockers; do not bury them
|
|
203
|
-
in a risk list. Recheck affected decisions and packages after each repair.
|
|
204
|
-
|
|
205
|
-
## Deliver
|
|
206
|
-
|
|
207
|
-
Save the plan to the user-specified path if one is given; otherwise use
|
|
208
|
-
`/tmp/loop-plan-<safe-task-slug>-result.md` (append a timestamp when a collision
|
|
209
|
-
is possible). Keep the runtime log separate. Make the plan understandable
|
|
210
|
-
without the preceding conversation. Follow
|
|
211
|
-
[templates/plan.md](templates/plan.md), replacing each placeholder or omitting
|
|
212
|
-
the section as it directs.
|
|
213
|
-
|
|
214
|
-
Before delivery, verify references are accurate, critical assumptions are
|
|
215
|
-
validated or explicitly gated, acceptance checks cover intended behavior, and
|
|
216
|
-
another implementer can proceed without hidden design decisions.
|
|
217
|
-
Readiness describes the plan; it is not user approval or evidence that planned
|
|
218
|
-
checks passed. If handing the document to `ghosts:loop-coding`, preserve explicit
|
|
219
|
-
approval of that plan and let its implementation workflow own execution. End
|
|
220
|
-
with the plan path, chosen direction, and remaining decisions.
|
|
@@ -1,39 +0,0 @@
|
|
|
1
|
-
# <Plan title>
|
|
2
|
-
|
|
3
|
-
**Readiness:** <ready for approval | conditional | blocked> — <reason>
|
|
4
|
-
**Direction:** <Chosen approach in one sentence>
|
|
5
|
-
**Alternative:** <Strongest alternative and the tradeoff that ruled it out; omit when none is viable>
|
|
6
|
-
|
|
7
|
-
## Outcome
|
|
8
|
-
- <Observable outcome and success criteria>
|
|
9
|
-
- **Out of scope:** <Explicit non-goals>
|
|
10
|
-
|
|
11
|
-
## Work packages
|
|
12
|
-
1. **<Outcome of the package>**
|
|
13
|
-
- `<path>` — <What changes and why, without implementation details; mark new files>
|
|
14
|
-
- **Depends on:** <Earlier package or decision; omit when none>
|
|
15
|
-
- **Done when:** <Completion evidence>
|
|
16
|
-
|
|
17
|
-
## Validation
|
|
18
|
-
- <Check to run and what it proves>
|
|
19
|
-
|
|
20
|
-
---
|
|
21
|
-
|
|
22
|
-
## Details
|
|
23
|
-
Supporting context for reviewers; omit any subsection with nothing material to add.
|
|
24
|
-
|
|
25
|
-
### Open decisions
|
|
26
|
-
- <Decision only the user can make, with options and a recommendation>
|
|
27
|
-
|
|
28
|
-
### Constraints and assumptions
|
|
29
|
-
- <Constraint, or assumption with its source or the check that will verify it>
|
|
30
|
-
|
|
31
|
-
### Alternatives
|
|
32
|
-
- **<Alternative>** — <Its strongest case, and why it lost>
|
|
33
|
-
- **Would change if:** <Evidence that would reverse the chosen direction>
|
|
34
|
-
|
|
35
|
-
### Risks
|
|
36
|
-
- **<Plausible failure>** — <Early signal>; <mitigation>; <stop or revise trigger>
|
|
37
|
-
|
|
38
|
-
### Decision ledger
|
|
39
|
-
- **<ID> <Decision>** (<status>) — <Evidence and review call IDs>; <unresolved disagreement, only if any>
|
|
@@ -1,145 +0,0 @@
|
|
|
1
|
-
# Backend Testing Guide
|
|
2
|
-
|
|
3
|
-
This guide adds backend-specific rules and examples to the shared standards in `common.md`.
|
|
4
|
-
|
|
5
|
-
Repository structure, fixture names, and test commands may differ by repository. This guide focuses on shared judgment criteria for writing and reviewing backend tests, rather than repository-specific usage.
|
|
6
|
-
|
|
7
|
-
## 1. Test Targets
|
|
8
|
-
|
|
9
|
-
High-priority targets:
|
|
10
|
-
|
|
11
|
-
- Business-critical flows such as payment, authentication, authorization, subscriptions, and event publishing
|
|
12
|
-
- Tasks and workers that are expensive to recover from when they fail
|
|
13
|
-
- Complex queries, pagination, cursors, ordering, locks, and idempotency
|
|
14
|
-
- Logic that interprets external system responses and mutates local state
|
|
15
|
-
- Edge-case-heavy logic such as dates, timezones, market holidays, exchange rates, decimals, and discounts
|
|
16
|
-
|
|
17
|
-
## 2. Test Scope Strategy
|
|
18
|
-
|
|
19
|
-
API, service, repository, DB, and cache integration tests can catch wiring, transaction, serialization, and dependency override issues.
|
|
20
|
-
|
|
21
|
-
## 3. Test Naming And Grouping
|
|
22
|
-
|
|
23
|
-
Basic rules:
|
|
24
|
-
|
|
25
|
-
- Use `test_<situation>_<expected_outcome>` for test function names.
|
|
26
|
-
- Every test should have a docstring.
|
|
27
|
-
- Use `pytest.mark.parametrize` with readable `ids` for cases that share an execution flow.
|
|
28
|
-
|
|
29
|
-
Example:
|
|
30
|
-
|
|
31
|
-
```python
|
|
32
|
-
async def test_expired_subscription_removes_user_role(...) -> None:
|
|
33
|
-
"""An expired subscription removes the user's role."""
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
Avoid implementation-shaped names:
|
|
37
|
-
|
|
38
|
-
```python
|
|
39
|
-
async def test_calls_delete_role(...) -> None:
|
|
40
|
-
...
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
### 3.1 Class Grouping
|
|
44
|
-
|
|
45
|
-
Use classes when related tests are easier to read as one group. Standalone test functions are appropriate when grouping adds no clarity.
|
|
46
|
-
|
|
47
|
-
- Group tests by feature or public entrypoint.
|
|
48
|
-
- Do not use classes to hide important setup or build complex lifecycles.
|
|
49
|
-
|
|
50
|
-
## 4. AAA Pattern
|
|
51
|
-
|
|
52
|
-
Write test bodies in the `given / when / then` flow.
|
|
53
|
-
|
|
54
|
-
- `given`: Data, fixtures, mocks, and fixed time required for the scenario
|
|
55
|
-
- `when`: The behavior under test
|
|
56
|
-
- `then`: Observable results of `when`, such as the response, DB state, events, and external calls
|
|
57
|
-
|
|
58
|
-
Example:
|
|
59
|
-
|
|
60
|
-
```python
|
|
61
|
-
async def test_expired_subscription_removes_user_role(...) -> None:
|
|
62
|
-
"""An expired subscription removes the user's role."""
|
|
63
|
-
# given
|
|
64
|
-
subscription = create_subscription(status="active", expired=True)
|
|
65
|
-
|
|
66
|
-
# when
|
|
67
|
-
await expire_subscriptions()
|
|
68
|
-
|
|
69
|
-
# then
|
|
70
|
-
assert not user_has_role(subscription.user_id, "premium")
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
## 5. Layer-Specific Guidelines
|
|
74
|
-
|
|
75
|
-
Use the largest layer that still gives a clear signal.
|
|
76
|
-
|
|
77
|
-
- API tests are a good default when behavior is observable through HTTP. Verify the response contract and important side effects.
|
|
78
|
-
- Service or domain tests are useful when the business rule is the main concern and an API test would be too heavy.
|
|
79
|
-
- Repository or DAO behavior is usually covered through a higher-level test. Test it directly only when the query contract itself is the main risk.
|
|
80
|
-
- Task and worker tests should focus on operational concerns such as duplicate handling, retries, partial failures, and idempotency.
|
|
81
|
-
- Avoid API loops for large setup. Insert data directly or use helpers/factories when setup is not the behavior under test.
|
|
82
|
-
|
|
83
|
-
## 6. Fixture And Test Data
|
|
84
|
-
|
|
85
|
-
Backend fixtures commonly provide DB sessions, clients, caches, repositories, and services; scenario factories can use names such as `create_expired_subscription` or `create_verified_user`.
|
|
86
|
-
|
|
87
|
-
### 6.1 DB Setup
|
|
88
|
-
|
|
89
|
-
- Isolate DB changes with transaction rollback whenever possible.
|
|
90
|
-
- Use `flush` when DB-generated values or query-visible rows are needed.
|
|
91
|
-
- Use `commit` only when verifying behavior that is visible only after commit, across separate transactions, or through task/worker entrypoints.
|
|
92
|
-
- Do not clean tables manually in tests. Before adding cleanup fixtures, confirm why transaction rollback cannot solve the problem.
|
|
93
|
-
- Prefer DB bulk inserts over API loops for large setup.
|
|
94
|
-
|
|
95
|
-
### 6.2 Cache And External State
|
|
96
|
-
|
|
97
|
-
- Isolate cache state and restore dependency overrides, monkeypatches, and environment overrides through the established fixture lifecycle.
|
|
98
|
-
- Verify cache keys, TTLs, and session values directly when they are part of the contract.
|
|
99
|
-
|
|
100
|
-
## 7. Test Double Policy
|
|
101
|
-
|
|
102
|
-
Good targets for test doubles:
|
|
103
|
-
|
|
104
|
-
- Payment providers, Google Play, Apple, Firebase, S3, Discord, email, and Kafka producers
|
|
105
|
-
- Other servers or external HTTP clients
|
|
106
|
-
- Very slow or nondeterministic library calls
|
|
107
|
-
|
|
108
|
-
Avoid replacing when possible:
|
|
109
|
-
|
|
110
|
-
- Services, repositories, and DAOs in the same process
|
|
111
|
-
- Internal helpers called by the function under test
|
|
112
|
-
- Private properties or private methods
|
|
113
|
-
- The DB session itself
|
|
114
|
-
- The entire service behind an endpoint
|
|
115
|
-
|
|
116
|
-
Guidelines:
|
|
117
|
-
|
|
118
|
-
- If a dependency is already mocked at an upper boundary, do not also mock the lower DAO unnecessarily.
|
|
119
|
-
- Use `spec` or real-object-based `patch.object` for `Mock` and `AsyncMock` whenever possible.
|
|
120
|
-
- Patch where the object is used, not where it is defined.
|
|
121
|
-
|
|
122
|
-
Partial replacement can be better than replacing the whole object. For example, instead of replacing an entire external client, create the client through the real factory and patch only the failing method. This keeps more framework wiring intact.
|
|
123
|
-
|
|
124
|
-
## 8. Time, Randomness, And Determinism
|
|
125
|
-
|
|
126
|
-
For ordering-sensitive queries and pagination, make `created_at`, IDs, and cursor values explicit. Verify timezone conversion or epoch values when they form the storage or response contract.
|
|
127
|
-
|
|
128
|
-
## 9. Assertions
|
|
129
|
-
|
|
130
|
-
- In API tests, assert `status_code` first.
|
|
131
|
-
- Verify DB side effects by reading them back through a repository or query.
|
|
132
|
-
- Assert downstream contracts directly, such as event payloads, unique keys, and published flags.
|
|
133
|
-
- Verify authentication and authorization failures at the server boundary, including rejected requests and the absence of unauthorized side effects.
|
|
134
|
-
|
|
135
|
-
## 10. Snapshot Usage
|
|
136
|
-
|
|
137
|
-
Response JSON and event payloads can be snapshot targets when their full shape is a stable backend contract.
|
|
138
|
-
|
|
139
|
-
## 11. Review Checklist
|
|
140
|
-
|
|
141
|
-
Check these backend-specific concerns:
|
|
142
|
-
|
|
143
|
-
- Can DB, cache, and dependency overrides leak between tests?
|
|
144
|
-
- Are server authentication and authorization enforced, including rejected requests and unauthorized side effects?
|
|
145
|
-
- Are backend-specific concerns such as pagination, cursors, idempotency, and transactions covered?
|
|
@@ -1,92 +0,0 @@
|
|
|
1
|
-
# Common Testing Standards
|
|
2
|
-
|
|
3
|
-
Apply these standards to every test review. Frontend and backend guides add stack-specific rules and examples.
|
|
4
|
-
|
|
5
|
-
## Meaningful Failure
|
|
6
|
-
|
|
7
|
-
Tests should fail when the behavior they protect is broken and make the broken requirement clear. Improve the name, setup, or assertions when a failure is hard to interpret. If a refactor breaks a test without changing observable behavior, reconsider its implementation coupling.
|
|
8
|
-
|
|
9
|
-
## Test Target Value
|
|
10
|
-
|
|
11
|
-
Do not try to test all code at the same density. Evaluate the protection each test adds to the existing suite against the cost of writing, reading, running, and maintaining it. The goal is reliable protection of important behavior, not the fewest tests or the highest coverage number.
|
|
12
|
-
|
|
13
|
-
- Before adding or retaining a test, identify the important regression it catches that existing tests do not adequately cover.
|
|
14
|
-
- When a new feature reuses shared behavior, check what existing tests already protect. Add tests for how the feature uses that behavior and for any differences in its contract, rather than copying all tests for the shared behavior.
|
|
15
|
-
- When an integration test already protects a behavior, add narrower tests only when they provide independent protection or clearer checks of important edge cases.
|
|
16
|
-
- Before removing a test, confirm that important requirements and failure cases remain protected.
|
|
17
|
-
- Give lower priority to simple getters, setters, type declarations, and behavior already guaranteed by libraries or static analysis. Test application requirements those guarantees do not cover.
|
|
18
|
-
|
|
19
|
-
## Test Scope
|
|
20
|
-
|
|
21
|
-
Start from an existing public entrypoint and preserve the real collaboration needed to deliver the behavior. A test can involve several components while protecting one focused concern. Prefer integration tests when they give a clear signal; unit tests suit naturally small rules such as calculations, parsers, and validators.
|
|
22
|
-
|
|
23
|
-
- Narrow the scope when setup obscures the scenario, failures are difficult to isolate, dependency replacement dominates the test, execution cost hurts feedback, or one rule needs many focused edge cases. Keep the reason clear and check what integration protection remains.
|
|
24
|
-
- Replace external or unstable dependencies at the test boundary without replacing the behavior under test. If internal mocks keep increasing or make refactoring fragile, reconsider a larger scope.
|
|
25
|
-
- Before changing production code for testability, check existing public interfaces. Do not expose private members or add abstractions solely for testing convenience; require a clear behavioral testing benefit for design changes.
|
|
26
|
-
|
|
27
|
-
## Focused Scenarios
|
|
28
|
-
|
|
29
|
-
Keep each scenario focused on its intended behavior. Combine multiple actions when their sequence is itself the requirement, not merely because they can be placed in one workflow. Do not extend an existing test with unrelated assertions for a new feature when that behavior belongs in another test.
|
|
30
|
-
|
|
31
|
-
- Make each test understandable on its own, including when setup is shared. Names and descriptions should reveal the scenario and observable outcome; do not claim to verify behavior replaced by a test double.
|
|
32
|
-
- Keep setup, action, and observable results in the Given / When / Then flow. Multiple assertions should describe results of the same behavior.
|
|
33
|
-
- Parameterize cases of the same behavior with readable case names when they share an execution flow. Split cases when branches obscure their distinct scenarios.
|
|
34
|
-
|
|
35
|
-
## Fixtures And Isolation
|
|
36
|
-
|
|
37
|
-
- Reuse established setup for infrastructure outside the test's concern, while keeping scenario-specific conditions and actions visible.
|
|
38
|
-
- Create only the data required for the scenario, with enough fields and variation to represent its edge cases.
|
|
39
|
-
- Avoid helpers that create excessive data, accept many scenario-specific parameters, or hide conditions behind deep dependency chains.
|
|
40
|
-
- Extract helpers for clarity; a little duplication is preferable to hiding the scenario. Use names that reveal the provided state, share helpers when multiple features reuse them, and keep feature-specific helpers near the feature.
|
|
41
|
-
- Ensure mutable state and test overrides cannot leak between tests. Choose isolation or restoration appropriate to the resource; reuse established cleanup instead of repeating it when the environment already handles it.
|
|
42
|
-
|
|
43
|
-
## Deterministic Inputs
|
|
44
|
-
|
|
45
|
-
Control unstable inputs when they affect the behavior or assertions.
|
|
46
|
-
|
|
47
|
-
- Use fixed or controlled time, randomness, and UUIDs when their values matter to the scenario.
|
|
48
|
-
- For ordering-sensitive behavior, make sort inputs explicit and verify the required order.
|
|
49
|
-
- When timezone behavior is contractual, verify the relevant timezone, conversion, or epoch values.
|
|
50
|
-
|
|
51
|
-
## Assertions And Failure Cases
|
|
52
|
-
|
|
53
|
-
- Derive expected outcomes from requirements, independently of production logic. Avoid tautological tests that repeat production logic or assert only mock setup.
|
|
54
|
-
- Assert the observable results and outgoing interactions that form the contract. Check specific fields when only those fields matter, and exact objects when the whole payload is contractual.
|
|
55
|
-
- For lists, verify length, order, and item identities when those properties are requirements.
|
|
56
|
-
- Existence checks or mock call counts are sufficient only when existence or call count is itself the contract, such as preventing duplicate sends. Otherwise verify the intended result or payload.
|
|
57
|
-
- Cover failure, permission, and authentication cases relevant to the changed behavior through observable outcomes at the tested boundary.
|
|
58
|
-
|
|
59
|
-
## Snapshots
|
|
60
|
-
|
|
61
|
-
Use snapshots when the full shape of a focused output is a stable contract and reviewers can understand its diff.
|
|
62
|
-
|
|
63
|
-
- Stabilize volatile values and ordering using appropriate matchers or controlled inputs.
|
|
64
|
-
- Update snapshots intentionally and review their changes with the code change.
|
|
65
|
-
- Prefer explicit assertions when only a few fields matter. Avoid snapshots covering unrelated output or producing too much data to interpret.
|
|
66
|
-
|
|
67
|
-
## Coverage
|
|
68
|
-
|
|
69
|
-
Coverage is a signal for finding test gaps, not a quality goal. Protection of important branches and failure cases matters more than the percentage. Use uncovered code to identify candidate scenarios, explain protection elsewhere or low-value targets when relevant, and do not add meaningless tests to raise coverage.
|
|
70
|
-
|
|
71
|
-
## Review Checklist
|
|
72
|
-
|
|
73
|
-
Start with the test's purpose and contribution to the suite:
|
|
74
|
-
|
|
75
|
-
1. What important behavior does this test protect beyond the protection already provided by other tests?
|
|
76
|
-
2. Does it contain only the setup, actions, and assertions needed for that behavior?
|
|
77
|
-
3. Can a reader immediately understand the conditions, action, and expected outcome?
|
|
78
|
-
|
|
79
|
-
Then check:
|
|
80
|
-
|
|
81
|
-
- Would this test actually fail if that behavior broke, or do its assertions merely repeat production logic or mock setup?
|
|
82
|
-
- Does it preserve useful collaboration between real components while keeping its concern focused?
|
|
83
|
-
- If the scope was narrowed, is the reason clear and is necessary integration protection retained?
|
|
84
|
-
- Do the name and description accurately reflect the scenario and observable outcome, with a clear Given / When / Then flow?
|
|
85
|
-
- Does the test verify externally observable behavior instead of internal implementation?
|
|
86
|
-
- Do test doubles replace external or unstable dependencies without replacing the behavior under test?
|
|
87
|
-
- Do fixtures expose the important conditions with only the necessary data, or should hidden setup move into the test body?
|
|
88
|
-
- Can mutable state or test overrides leak between tests?
|
|
89
|
-
- Are inputs that affect the result controlled, including time, UUIDs, randomness, and ordering?
|
|
90
|
-
- Do assertions verify the required results, payloads, or interactions rather than incidental properties?
|
|
91
|
-
- Are relevant failure, permission, and authentication cases covered?
|
|
92
|
-
- If a snapshot is used, does it protect a focused contract with a readable diff?
|