@ervis/skills 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +335 -0
- package/bin/install.js +135 -0
- package/package.json +39 -0
- package/skills/README.md +18 -0
- package/skills/engineering/commit/SKILL.md +47 -0
- package/skills/engineering/implement-ruby/template.md +75 -0
- package/skills/engineering/ticket-grooming/SKILL.md +58 -0
- package/skills/incubator/qrspi-design/SKILL.md +131 -0
- package/skills/incubator/qrspi-implement/SKILL.md +123 -0
- package/skills/incubator/qrspi-plan/SKILL.md +113 -0
- package/skills/incubator/qrspi-question/SKILL.md +184 -0
- package/skills/incubator/qrspi-research/SKILL.md +109 -0
- package/skills/incubator/qrspi-research-resolve/SKILL.md +94 -0
- package/skills/incubator/qrspi-structure/SKILL.md +116 -0
- package/skills/incubator/qrspi-test-plan/SKILL.md +253 -0
- package/skills/incubator/qrspi-test-plan/references/example-test-plan.md +151 -0
- package/skills/productivity/brainstorm/SKILL.md +49 -0
- package/skills/productivity/brainstorm/references/assumption-mapping.md +24 -0
- package/skills/productivity/brainstorm/references/five-whys.md +20 -0
- package/skills/productivity/brainstorm/references/pre-mortem.md +21 -0
- package/skills/productivity/brainstorm/references/question-burst.md +23 -0
- package/skills/productivity/brainstorm/references/question-formulation-technique.md +27 -0
- package/skills/productivity/brainstorm/references/six-thinking-hats.md +24 -0
- package/skills/productivity/brainstorm/references/starbursting.md +19 -0
- package/skills/productivity/caveman/SKILL.md +49 -0
- package/skills/productivity/simple-english/SKILL.md +15 -0
- package/skills/research/.gitkeep +0 -0
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qrspi-research
|
|
3
|
+
description: Second step of the QRSPI workflow. Answers the research questions in questions.md with facts only. Each fact has a file:line reference or a URL. The answers contain no opinions. The researcher does not know what the team will build. The skill reads questions.md from the task directory and writes the answers to research-draft.md. The qrspi-research-resolve step then settles the gaps and writes the final research.md. Use it after qrspi-question, or when the user says "qrspi research", "/qrspi-research", or "research the questions".
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# QRSPI Research
|
|
7
|
+
|
|
8
|
+
## Dependencies
|
|
9
|
+
|
|
10
|
+
- **`simple-english`** (required). Follow it for all text that you write for the user. You can find it in `skills/productivity/` of this repository. If it is not available, stop. Tell the user to install it.
|
|
11
|
+
- **`codebase-locator`**, **`codebase-analyzer`**, **`codebase-pattern-finder`** (required). These agents answer the `[code]` questions in step 2. You can find them in the `agents/` directory of this repository. If one of them is not available, stop. Tell the user to install it.
|
|
12
|
+
- **`web-search-researcher`** (required). This agent answers the `[web]` questions in step 2. You can find it in the `agents/` directory of this repository. If it is not available, stop. Tell the user to install it.
|
|
13
|
+
|
|
14
|
+
This is the second step of QRSPI: Question, Research, Design, Structure, Plan, Test plan, Implement.
|
|
15
|
+
|
|
16
|
+
You only describe what exists. You answer the research questions with facts, code references, and the patterns that you see. You do not know what the team will build. You do not propose solutions.
|
|
17
|
+
|
|
18
|
+
Why: a researcher who knows the goal finds the facts that support the goal. That researcher misses the other facts. The `qrspi-question` step wrote the questions so that they do not show the goal. Keep the goal hidden.
|
|
19
|
+
|
|
20
|
+
## Input
|
|
21
|
+
|
|
22
|
+
The user starts the skill like this: `qrspi-research <task-dir>`. For example: `qrspi-research ~/wiki/rate-limit`.
|
|
23
|
+
|
|
24
|
+
- **`<task-dir>`.** The first argument. This is the directory for all files of this task. If the user does not give it, ask for it.
|
|
25
|
+
- **`<task-dir>/questions.md`.** This file is your only input.
|
|
26
|
+
|
|
27
|
+
Do not read `task.md`, `question-sources.md`, or other files in `<task-dir>`. Do not ask the user what the team will build.
|
|
28
|
+
|
|
29
|
+
Run this skill in a new session. If the session already knows the task, the research is not blind.
|
|
30
|
+
|
|
31
|
+
## 1. Read the questions
|
|
32
|
+
|
|
33
|
+
Read all of `questions.md`.
|
|
34
|
+
|
|
35
|
+
## 2. Send the questions to agents
|
|
36
|
+
|
|
37
|
+
Send the questions to agents. Start the agents at the same time:
|
|
38
|
+
|
|
39
|
+
- **`[code]` questions:**
|
|
40
|
+
- `codebase-locator` finds the locations of the files and components.
|
|
41
|
+
- `codebase-analyzer` traces how the code works. It gives `file:line` references.
|
|
42
|
+
- `codebase-pattern-finder` finds examples of the patterns in the question.
|
|
43
|
+
- **`[web]` questions:** `web-search-researcher` finds docs, specs, and other sources. It gives URLs.
|
|
44
|
+
|
|
45
|
+
Give one or two questions to each agent. Give the agent only the text of these questions. Do not give it other files or this conversation. Tell each agent: "Describe what exists. Do not suggest changes or solutions."
|
|
46
|
+
|
|
47
|
+
Wait until all agents are done.
|
|
48
|
+
|
|
49
|
+
## 3. Put the findings together
|
|
50
|
+
|
|
51
|
+
Connect the findings across the components. If two agents do not agree, read the code yourself. Find which agent is correct.
|
|
52
|
+
|
|
53
|
+
Done when each question has an answer or a clear "could not answer".
|
|
54
|
+
|
|
55
|
+
## 4. Write the findings
|
|
56
|
+
|
|
57
|
+
Write `<task-dir>/research-draft.md`.
|
|
58
|
+
|
|
59
|
+
```markdown
|
|
60
|
+
# Research draft
|
|
61
|
+
|
|
62
|
+
## Q1: <question text>
|
|
63
|
+
|
|
64
|
+
### Findings
|
|
65
|
+
- <fact, with `file:line` or URL>
|
|
66
|
+
- <how the components connect>
|
|
67
|
+
- <patterns that you see>
|
|
68
|
+
|
|
69
|
+
### Also found
|
|
70
|
+
- <a related fact that the question did not ask for, with `file:line` or URL>
|
|
71
|
+
|
|
72
|
+
## Q2: <question text>
|
|
73
|
+
...
|
|
74
|
+
|
|
75
|
+
## Across the questions
|
|
76
|
+
<patterns, conventions, or architecture that two or more questions touch>
|
|
77
|
+
|
|
78
|
+
## Open
|
|
79
|
+
<each question that you could not answer fully, and why>
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Keep it short. Use `file:line` references, not long explanations.
|
|
83
|
+
|
|
84
|
+
## 5. Show the summary
|
|
85
|
+
|
|
86
|
+
Give the user a short summary. Then ask:
|
|
87
|
+
|
|
88
|
+
```
|
|
89
|
+
1. Add a question that is missing.
|
|
90
|
+
2. Continue to `qrspi-research-resolve`.
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
If the user chooses 1:
|
|
94
|
+
|
|
95
|
+
1. Add the question to `questions.md`. Give it the next number. Put the tag `[user]` next to its `[code]` or `[web]` tag.
|
|
96
|
+
2. Research it as in steps 2 and 3.
|
|
97
|
+
3. Add the answer to `research-draft.md`.
|
|
98
|
+
4. Ask the two options again.
|
|
99
|
+
|
|
100
|
+
If the user chooses 2, tell the user: "Next: run `qrspi-research-resolve` with `<task-dir>`."
|
|
101
|
+
|
|
102
|
+
## Rules
|
|
103
|
+
|
|
104
|
+
- You describe. You do not criticize. Describe what is, not what should be.
|
|
105
|
+
- Do not suggest improvements, refactoring, or solutions.
|
|
106
|
+
- Each finding has a `file:line` reference or a URL.
|
|
107
|
+
- If you cannot answer a question, say so clearly. Do not guess.
|
|
108
|
+
- "Also found" contains only facts. It does not contain opinions or ideas for changes.
|
|
109
|
+
- If a question is too vague or looks at the wrong area, answer what you can. Write the problem under "Open". `qrspi-research-resolve` settles it with the user.
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qrspi-research-resolve
|
|
3
|
+
description: Step of the QRSPI workflow between research and design. Grades each research answer in research-draft.md. If some answers are weak, contradictory, or open, it interviews the user, who answers or skips each one. If no answer has a gap, it does not interrupt the user. It then writes the final research.md for the design step. After this step, the research is done. Use it after qrspi-research, or when the user says "qrspi research resolve", "/qrspi-research-resolve", "resolve the research", or "settle the research gaps".
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# QRSPI Research Resolve
|
|
7
|
+
|
|
8
|
+
## Dependencies
|
|
9
|
+
|
|
10
|
+
- **`simple-english`** (required). Follow it for all text that you write for the user. You can find it in `skills/productivity/` of this repository. If it is not available, stop. Tell the user to install it.
|
|
11
|
+
- **`mattpocock-skills/grilling`** (required). This skill does the interview in step 2. You can find it in the `mattpocock-skills` plugin of Matt Pocock. If it is not available, stop. Tell the user to install it.
|
|
12
|
+
|
|
13
|
+
This step is between research and design in QRSPI: Question, Research, Design, Structure, Plan, Test plan, Implement.
|
|
14
|
+
|
|
15
|
+
Research writes its answers blind to `research-draft.md`. Some answers are strong. Others are weak, do not agree, or are missing. In this step you find these gaps. The user answers or skips each gap. Then you write `research.md`, the final facts that design uses. After this step, the research is done. There are no more research questions.
|
|
16
|
+
|
|
17
|
+
You know the goal. Only the goal tells you which answers are strong enough for the task. Thus the grading happens here, on the side of the wall that knows the goal.
|
|
18
|
+
|
|
19
|
+
## Input
|
|
20
|
+
|
|
21
|
+
The user starts the skill like this: `qrspi-research-resolve <task-dir>`. For example: `qrspi-research-resolve ~/wiki/rate-limit`.
|
|
22
|
+
|
|
23
|
+
- **`<task-dir>`.** The first argument. This is the directory for all files of this task. If the user does not give it, ask for it.
|
|
24
|
+
- **`<task-dir>/task.md`.** What the user wants.
|
|
25
|
+
- **`<task-dir>/questions.md`.** The research questions.
|
|
26
|
+
- **`<task-dir>/question-sources.md`.** Why each question exists.
|
|
27
|
+
- **`<task-dir>/research-draft.md`.** The answers from `qrspi-research`.
|
|
28
|
+
|
|
29
|
+
If `research-draft.md` does not exist, stop. Tell the user to run `qrspi-research` first.
|
|
30
|
+
|
|
31
|
+
## 1. Grade each answer
|
|
32
|
+
|
|
33
|
+
Read all input files. Give one grade to each answer:
|
|
34
|
+
|
|
35
|
+
- **Strong.** It has `file:line` or URL evidence. The evidence proves the claim. It does not only mention the claim.
|
|
36
|
+
- **Weak.** It is thin, only inferred, or has only one source.
|
|
37
|
+
- **Contradictory.** It does not agree with another answer.
|
|
38
|
+
- **Open.** The researcher could not answer it.
|
|
39
|
+
|
|
40
|
+
An answer is strong enough when the task can use it to decide how to build. An answer that the task does not need has no gap, whatever its grade.
|
|
41
|
+
|
|
42
|
+
The tag `[user]` shows a question that the user added during research. Such a question did not go through the leak test. If its words showed the goal, its answer can lean toward the goal. Grade it with more care.
|
|
43
|
+
|
|
44
|
+
Also read the "Also found" items. An item is important if the task or an invariant can depend on it. Treat each important item as an open answer.
|
|
45
|
+
|
|
46
|
+
Done when each answer and each important item has a grade.
|
|
47
|
+
|
|
48
|
+
## 2. Interview the user
|
|
49
|
+
|
|
50
|
+
Ask the user only when it is necessary. If no answer is weak, contradictory, or open, and no important "Also found" item exists, do not interview the user. Go to step 3.
|
|
51
|
+
|
|
52
|
+
Otherwise, run `grilling` on these items: each weak, contradictory, or open answer, and each important "Also found" item. For each item, show what research found, with its `file:line` references or URLs. Then the user resolves the item in one of two ways:
|
|
53
|
+
|
|
54
|
+
1. **Answer.** The user gives the correct fact.
|
|
55
|
+
2. **Skip.** The user decides that the task does not need this fact.
|
|
56
|
+
|
|
57
|
+
Recommend one of the two, and tell why. Sometimes the user is not sure. Then help the user decide. Tell what the task loses without the fact. Suggest who can know it, for example a teammate or the owner of the product. Each item must end with an answer or a skip, because design needs a clear set of facts.
|
|
58
|
+
|
|
59
|
+
Do not look up facts yourself, and do not send an agent. The user resolves the items. Where `grilling` and this skill do not agree, this skill wins.
|
|
60
|
+
|
|
61
|
+
Done when each item has an answer or a skip.
|
|
62
|
+
|
|
63
|
+
## 3. Write the final research
|
|
64
|
+
|
|
65
|
+
Write `<task-dir>/research.md`. Take the facts from `research-draft.md` and from the user. Do not change `research-draft.md`.
|
|
66
|
+
|
|
67
|
+
```markdown
|
|
68
|
+
# Research
|
|
69
|
+
|
|
70
|
+
## Q1: <question text>
|
|
71
|
+
- <fact, with `file:line` or URL>
|
|
72
|
+
- <fact from the user> (from the user)
|
|
73
|
+
|
|
74
|
+
## Q2: <question text>
|
|
75
|
+
...
|
|
76
|
+
|
|
77
|
+
## Also found
|
|
78
|
+
- <important related fact, with `file:line` or URL>
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Keep only facts that the evidence proves or that the user gave. Mark each fact that the user gave with "(from the user)". Do not include the items that the user skipped.
|
|
82
|
+
|
|
83
|
+
Tell the user:
|
|
84
|
+
|
|
85
|
+
- The number of gaps, and how the user resolved each one.
|
|
86
|
+
|
|
87
|
+
Tell the user: "Next: run `qrspi-design` with `<task-dir>`."
|
|
88
|
+
|
|
89
|
+
## Rules
|
|
90
|
+
|
|
91
|
+
- Do not research in this step. The user resolves the gaps.
|
|
92
|
+
- After this step, the research is done. Do not send new questions to research.
|
|
93
|
+
- Never give research `task.md`, `question-sources.md`, or `research.md`.
|
|
94
|
+
- The user resolves each gap with an answer or a skip. You recommend, then you wait for the user.
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qrspi-structure
|
|
3
|
+
description: Structure step of the QRSPI workflow. Breaks the approved design into vertical slices, where each slice works end to end and has a check. For each slice, it lists the files, the key signatures and types, and the invariants and dependents that it touches. It reads design.md and research.md from the task directory and writes structure.md for the plan step. Use it after qrspi-design, or when the user says "qrspi structure", "/qrspi-structure", or "break the design into slices".
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# QRSPI Structure
|
|
7
|
+
|
|
8
|
+
## Dependencies
|
|
9
|
+
|
|
10
|
+
- **`simple-english`** (required). Follow it for all text that you write for the user. You can find it in `skills/productivity/` of this repository. If it is not available, stop. Tell the user to install it.
|
|
11
|
+
|
|
12
|
+
This is the structure step of QRSPI: Question, Research, Design, Structure, Plan, Test plan, Implement.
|
|
13
|
+
|
|
14
|
+
The design tells where the work goes. In this step you tell how the work gets there, in slices. Each slice works end to end and has a check. You show the signatures, the types, and the slice boundaries. You do not write the implementation. Plan adds that detail.
|
|
15
|
+
|
|
16
|
+
## Input
|
|
17
|
+
|
|
18
|
+
The user starts the skill like this: `qrspi-structure <task-dir>`. For example: `qrspi-structure ~/wiki/rate-limit`.
|
|
19
|
+
|
|
20
|
+
- **`<task-dir>`.** The first argument. This is the directory for all files of this task. If the user does not give it, ask for it.
|
|
21
|
+
- **`<task-dir>/design.md`.** The approved design.
|
|
22
|
+
- **`<task-dir>/research.md`.** The research facts, with `file:line` references.
|
|
23
|
+
|
|
24
|
+
If `design.md` does not exist, stop. Tell the user to run `qrspi-design` first.
|
|
25
|
+
|
|
26
|
+
## 1. Read the input
|
|
27
|
+
|
|
28
|
+
Read all of `design.md` and `research.md`.
|
|
29
|
+
|
|
30
|
+
Done when you know each decision in the design, and the invariants and dependents in its current state.
|
|
31
|
+
|
|
32
|
+
## 2. Make vertical slices
|
|
33
|
+
|
|
34
|
+
Break the work into vertical slices. Each slice delivers one piece of the work end to end:
|
|
35
|
+
|
|
36
|
+
- It goes through all layers that it needs, for example database, service, API, and UI.
|
|
37
|
+
- You can check it alone, after it is done.
|
|
38
|
+
- It has a clear check.
|
|
39
|
+
|
|
40
|
+
**Vertical** (correct):
|
|
41
|
+
|
|
42
|
+
> Slice 1: Add the "reticulate" endpoint: migration, store method, API handler, and a basic UI button. Check: the endpoint returns 200, and the button calls it.
|
|
43
|
+
|
|
44
|
+
**Horizontal** (not correct):
|
|
45
|
+
|
|
46
|
+
> Slice 1: All database migrations. Slice 2: All service methods. Slice 3: All API endpoints. Slice 4: All UI changes.
|
|
47
|
+
|
|
48
|
+
Follow the patterns and the decisions in `design.md`. Use the names, signatures, and file locations that the existing code uses. Do not make a new structure when the code already has one.
|
|
49
|
+
|
|
50
|
+
If a part of the design cannot be sliced vertically, say so in `structure.md`.
|
|
51
|
+
|
|
52
|
+
## 3. Put the slices in order
|
|
53
|
+
|
|
54
|
+
Put the slices that others build on first. If a later slice fails, the earlier slices must still have value alone.
|
|
55
|
+
|
|
56
|
+
## 4. Describe each slice
|
|
57
|
+
|
|
58
|
+
For each slice, write:
|
|
59
|
+
|
|
60
|
+
- **Delivers.** What it does end to end, in one or two sentences.
|
|
61
|
+
- **Files.** Each file that it creates or changes.
|
|
62
|
+
- **Key changes.** New or changed signatures and types. Signatures only, not the code.
|
|
63
|
+
- **Touches.** The invariants, cascades, and dependents from the current state of `design.md` that this slice touches. The implementor must keep each invariant, and must check each dependent.
|
|
64
|
+
- **Check.** How to see that the slice works: a command, and what to look at by hand.
|
|
65
|
+
|
|
66
|
+
## 5. Handle a problem in the design
|
|
67
|
+
|
|
68
|
+
Sometimes you find that the design missed a constraint or used a wrong fact. Do not work around it without a word. Tell the user what you found. Ask the user how to resolve it. Write the answer in `structure.md`, and mark it with "(from the user)".
|
|
69
|
+
|
|
70
|
+
## 6. Write the structure
|
|
71
|
+
|
|
72
|
+
Write `<task-dir>/structure.md`:
|
|
73
|
+
|
|
74
|
+
```markdown
|
|
75
|
+
# Structure
|
|
76
|
+
|
|
77
|
+
## Approach
|
|
78
|
+
<1-2 sentences: the approach of design.md, in short.>
|
|
79
|
+
|
|
80
|
+
## Slice 1: <name>
|
|
81
|
+
<What this slice delivers end to end.>
|
|
82
|
+
|
|
83
|
+
**Files**: `path/to/file.ext`, `path/to/other.ext`
|
|
84
|
+
**Key changes**:
|
|
85
|
+
- `functionName(param: Type): ReturnType` - new or changed
|
|
86
|
+
- `NewType { field: Type }` - new type
|
|
87
|
+
**Touches**:
|
|
88
|
+
- Invariant: <the invariant> - keep it.
|
|
89
|
+
- Dependent: <the dependent> - check it.
|
|
90
|
+
**Check**: <command>; <what to look at by hand>
|
|
91
|
+
|
|
92
|
+
## Slice 2: <name>
|
|
93
|
+
...
|
|
94
|
+
|
|
95
|
+
## Checkpoints
|
|
96
|
+
<What must be true after each slice. This helps to continue after a break.>
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Keep it short. Show what changes, not how. If it gets long, you are writing the plan.
|
|
100
|
+
|
|
101
|
+
Show `structure.md` to the user. The user can change the order of the slices, split a slice, or ask for more detail. Change it until the user approves it.
|
|
102
|
+
|
|
103
|
+
Done when the user approves the structure.
|
|
104
|
+
|
|
105
|
+
## 7. Hand off
|
|
106
|
+
|
|
107
|
+
Tell the user: "Next: run `qrspi-plan` with `<task-dir>`."
|
|
108
|
+
|
|
109
|
+
## Rules
|
|
110
|
+
|
|
111
|
+
- Vertical slices, not horizontal layers. Each slice goes through all the layers that it needs.
|
|
112
|
+
- Signatures and types, not the implementation.
|
|
113
|
+
- Each slice has a check.
|
|
114
|
+
- Each slice lists the invariants and dependents that it touches.
|
|
115
|
+
- Follow the patterns and decisions of `design.md`. Do not add decisions that the design did not make.
|
|
116
|
+
- Do not research in this step. If a fact is missing, ask the user, as in step 5.
|
|
@@ -0,0 +1,253 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qrspi-test-plan
|
|
3
|
+
description: Test plan step of the QRSPI workflow. Writes the black-box test cases for each slice of structure.md, in plain words. Each slice gets one goal case. Each public interface gets a case for each rule, in Given / When / Then. The cases guard the scope and the design, so the agent can change the inner code freely. The implement step writes the code and the tests from them. It reads structure.md, design.md, and research.md from the task directory. It does not read plan.md. It writes test-plan.md. Use it after qrspi-plan, or when the user says "qrspi test plan", "/qrspi-test-plan", "write the test plan", "write the test cases", or "plan the tests".
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# QRSPI Test Plan
|
|
7
|
+
|
|
8
|
+
## Dependencies
|
|
9
|
+
|
|
10
|
+
- **`simple-english`** (required). Follow it for all text that you write for the user. You can find it in `skills/productivity/` of this repository. If it is not available, stop. Tell the user to install it.
|
|
11
|
+
|
|
12
|
+
This is the test plan step of QRSPI: Question, Research, Design, Structure, Plan, Test plan, Implement.
|
|
13
|
+
|
|
14
|
+
The test plan is a contract. Its test cases record the scope and the design. The implement step writes the code and the tests. When the code changes, the tests show if the scope or the design broke.
|
|
15
|
+
|
|
16
|
+
An agent can rewrite inner code quickly and at low cost. Thus, the plan has only black-box test cases for public interfaces. It has no test cases for inner code. The implement step decides about tests for inner code.
|
|
17
|
+
|
|
18
|
+
The test cases also teach. A reader of the test cases must learn what the feature does. A reader must also learn how to use each public interface.
|
|
19
|
+
|
|
20
|
+
## Input
|
|
21
|
+
|
|
22
|
+
The user starts the skill like this: `qrspi-test-plan <task-dir>`. For example: `qrspi-test-plan ~/wiki/rate-limit`.
|
|
23
|
+
|
|
24
|
+
- **`<task-dir>`.** The first argument. This is the directory for all files of this task. If the user does not give it, ask for it.
|
|
25
|
+
- **`<task-dir>/structure.md`.** The approved slices.
|
|
26
|
+
- **`<task-dir>/design.md`.** The approved design: current state, patterns, and decisions.
|
|
27
|
+
- **`<task-dir>/research.md`.** The research facts, with `file:line` references.
|
|
28
|
+
|
|
29
|
+
If `structure.md` does not exist, stop. Tell the user to run `qrspi-structure` first.
|
|
30
|
+
|
|
31
|
+
Do not read `plan.md`, even if it exists. The plan tells how the code works. A test case from the plan checks how the code works. It also copies the mistakes of the plan. A test case from the design and the slices checks what must be true. Thus, it is an independent check of the code.
|
|
32
|
+
|
|
33
|
+
## 1. Read the input
|
|
34
|
+
|
|
35
|
+
Read all of `structure.md`, `design.md`, and `research.md`.
|
|
36
|
+
|
|
37
|
+
Do not search the code for new facts. Research is done.
|
|
38
|
+
|
|
39
|
+
Done when you know each slice and each design decision. You must also know the clients of the code that the work changes.
|
|
40
|
+
|
|
41
|
+
## 2. Find the public interfaces
|
|
42
|
+
|
|
43
|
+
A public interface is a contract with a client. A client is a person, a system, or code that uses the interface:
|
|
44
|
+
|
|
45
|
+
- An end user or an outside system. It uses HTTP, a UI, a job, or a CLI.
|
|
46
|
+
- A different module. It imports our classes or our methods. We cannot change this API without care.
|
|
47
|
+
- A future client. `design.md` makes the interface for clients on purpose.
|
|
48
|
+
|
|
49
|
+
The contract has two parts. Write test cases for the two parts.
|
|
50
|
+
|
|
51
|
+
- **Behavior.** What the interface does. For example: rules, errors, and side effects such as an email.
|
|
52
|
+
- **Data.** What the interface gives to clients or keeps for clients. For example: response fields, arguments, return values, event payloads, tables that a different module reads, and email content.
|
|
53
|
+
|
|
54
|
+
Code without a client is inner code. Do not write test cases for it. The agent can change it freely.
|
|
55
|
+
|
|
56
|
+
For each public interface, write this data:
|
|
57
|
+
|
|
58
|
+
- **Level.** Write `top` for the outermost entry point of a feature. Write `interface` for a public interface below it.
|
|
59
|
+
- **Clients.** Write each client of the interface. Get them from `research.md` and `design.md`.
|
|
60
|
+
- **Contract.** Write its behavior and its data in plain words.
|
|
61
|
+
- **Expected contract change.** Write the change that `design.md` makes on purpose. For a new interface, write "none". For a change to inner code only, write "none". Step 6 tells more.
|
|
62
|
+
|
|
63
|
+
Done when each public interface of the slices has a level, clients, a contract, and an expected change.
|
|
64
|
+
|
|
65
|
+
## 3. Find the outermost layer of each slice
|
|
66
|
+
|
|
67
|
+
The size of the box is different for each request. For each slice, find the outermost layer that the change touches. That layer is the box.
|
|
68
|
+
|
|
69
|
+
A black-box test case has two sides. The two sides must be on the outermost layer.
|
|
70
|
+
|
|
71
|
+
- **Input side.** This is where the actor starts the action. For example, a user shares a report with `POST /reports/:id/shares`. The user does not call a mailer.
|
|
72
|
+
- **Output side.** This is where a person outside the system sees the result. For example, the email that the member gets. "The mailer was called" is not an output. "A job is in the queue" is not an output. These are inner parts.
|
|
73
|
+
|
|
74
|
+
Some changes touch only a job. Then the job is the box. Some changes touch only a method that a different module calls. Then that method is the box.
|
|
75
|
+
|
|
76
|
+
## 4. Write the test cases
|
|
77
|
+
|
|
78
|
+
Put the test cases on two levels:
|
|
79
|
+
|
|
80
|
+
- **Top level.** Write one goal case for each slice. The slice is done when the test for its goal case passes. Also write the cases that show the connections. For example: "A rule fails. The response is 422."
|
|
81
|
+
- **Interface level.** Write all other cases here. Put each rule on the lowest public interface. There, tests are fast and exact.
|
|
82
|
+
|
|
83
|
+
Each test case is a black-box case for its own interface.
|
|
84
|
+
|
|
85
|
+
Write a case for each rule and each decision of a slice. For each public interface, look for these cases:
|
|
86
|
+
|
|
87
|
+
- the usual path
|
|
88
|
+
- each rule that fails
|
|
89
|
+
- each limit value
|
|
90
|
+
- a duplicate action
|
|
91
|
+
- each permission
|
|
92
|
+
- empty input
|
|
93
|
+
- each change of state
|
|
94
|
+
- each data field of the contract
|
|
95
|
+
|
|
96
|
+
Each case has these parts:
|
|
97
|
+
|
|
98
|
+
- **Name.** Use words of the business domain. The name tells the intent. For example: "A non-owner cannot share".
|
|
99
|
+
- **Interface.** Write the interface and its level.
|
|
100
|
+
- **Guards.** Write the rule or the decision that the case guards. Copy its text from `design.md` or `structure.md`.
|
|
101
|
+
- **Given / When / Then.** Write **Then** as a list of exact checks. Each check must be visible on the output side of the box. Logs and calls to inner parts are not outputs.
|
|
102
|
+
|
|
103
|
+
Write the top level as business rules. Use words of the business domain. A person who does not write code must understand it.
|
|
104
|
+
|
|
105
|
+
Write the interface level as examples of use for a developer of a client module. Tell what to call, what to send, and what comes back.
|
|
106
|
+
|
|
107
|
+
Write the test cases in plain words. Do not write test code. The implement step writes the tests.
|
|
108
|
+
|
|
109
|
+
## 5. Make the test cases repeatable
|
|
110
|
+
|
|
111
|
+
A test must give the same result each time.
|
|
112
|
+
|
|
113
|
+
Each **Given** makes only the data for its case:
|
|
114
|
+
|
|
115
|
+
- Cases do not share setup.
|
|
116
|
+
- The order of the cases is not important.
|
|
117
|
+
- The database is clean at the start of each case.
|
|
118
|
+
|
|
119
|
+
A small **Given** also teaches. Each line of setup shows a fact that is important for the rule.
|
|
120
|
+
|
|
121
|
+
A real database is correct in tests.
|
|
122
|
+
|
|
123
|
+
Some values change from one run to the next. Write each source of change, and tell how the tests must control it:
|
|
124
|
+
|
|
125
|
+
- **Time.** A rule can use the time. Then the **Given** sets a fixed date.
|
|
126
|
+
- **Background jobs.** The test runs the jobs. Then it checks the result on the output side.
|
|
127
|
+
- **Generated values.** For example: IDs, tokens, and the order of results. Do not check them. Check them only if they are part of the contract.
|
|
128
|
+
- **Outside services.** Use the rules for replacements below.
|
|
129
|
+
- **Two actions at the same time.** Tell if this is in scope. If it is in scope, tell how to test it. If it is not in scope, write it in **Known gaps**.
|
|
130
|
+
|
|
131
|
+
Do not use mocks, if possible. Use the real code in the box. With a mock, a test can pass when the real code fails.
|
|
132
|
+
|
|
133
|
+
Replace a dependency only for one of these reasons:
|
|
134
|
+
|
|
135
|
+
- We cannot control its behavior. For example, an outside HTTP service.
|
|
136
|
+
- A real dependency is too slow or too expensive for a test.
|
|
137
|
+
|
|
138
|
+
Replace a dependency only on the outer edge of the system. Do not replace a part in the box.
|
|
139
|
+
|
|
140
|
+
Each replacement must have a contract test, if possible. A contract test makes sure that the replacement and the real service agree. For example:
|
|
141
|
+
|
|
142
|
+
- The test runs the same checks on the test system of the real service.
|
|
143
|
+
- The test is a contract that the client writes, for example with Pact.
|
|
144
|
+
|
|
145
|
+
A contract test is not always possible. Then tell why. Also tell the source of truth for the replacement. For example: the API documents, or a recorded real response.
|
|
146
|
+
|
|
147
|
+
## 6. Write the expected contract changes
|
|
148
|
+
|
|
149
|
+
Some work changes a contract on purpose. Then some existing tests must fail. The implement step must know which failures are expected. All other failures are probably bugs.
|
|
150
|
+
|
|
151
|
+
For each public interface, get the planned change from `design.md`. Write two things in plain words:
|
|
152
|
+
|
|
153
|
+
- what changes
|
|
154
|
+
- which existing tests can fail because of it
|
|
155
|
+
|
|
156
|
+
For a change to inner code only, write "none". Then no existing test can fail.
|
|
157
|
+
|
|
158
|
+
Do not search the code for the existing tests. The implement step compares each real failure with this list.
|
|
159
|
+
|
|
160
|
+
## 7. Record a problem in the previous steps
|
|
161
|
+
|
|
162
|
+
Exact checks can show a problem that the previous steps did not find. For example:
|
|
163
|
+
|
|
164
|
+
- The design does not tell what occurs in a case.
|
|
165
|
+
- A slice has no clear goal case.
|
|
166
|
+
- A case cannot pass with the design as it is.
|
|
167
|
+
|
|
168
|
+
If you find a problem, stop. Do not solve it. Do not change `design.md` or `structure.md`.
|
|
169
|
+
|
|
170
|
+
Write the problem in the **Findings** section of `test-plan.md`:
|
|
171
|
+
|
|
172
|
+
- what you found
|
|
173
|
+
- the case that shows it
|
|
174
|
+
- the file that it affects: `design.md` or `structure.md`
|
|
175
|
+
|
|
176
|
+
Tell the user about the problem. The user solves it and updates the previous steps. Then the user starts this skill again.
|
|
177
|
+
|
|
178
|
+
## 8. Write the test plan
|
|
179
|
+
|
|
180
|
+
Write `<task-dir>/test-plan.md`.
|
|
181
|
+
|
|
182
|
+
The file must be complete alone. The implement step must be able to write all the tests from this file only. Thus, copy the rule text and the contracts into the file. Do not refer to other files for them.
|
|
183
|
+
|
|
184
|
+
For a full example, read `references/example-test-plan.md`.
|
|
185
|
+
|
|
186
|
+
````markdown
|
|
187
|
+
# Test plan
|
|
188
|
+
|
|
189
|
+
## Approach
|
|
190
|
+
<1-2 sentences: what the work gives, from design.md.>
|
|
191
|
+
|
|
192
|
+
## Public interfaces
|
|
193
|
+
|
|
194
|
+
### <interface, for example POST /reports/:id/shares>
|
|
195
|
+
**Level**: top | interface
|
|
196
|
+
**Clients**: <who uses it>
|
|
197
|
+
**Contract**: <behavior and data, in plain words>
|
|
198
|
+
**Expected contract change**: <what changes on purpose, and which existing tests can fail> | none
|
|
199
|
+
|
|
200
|
+
## Outside dependencies
|
|
201
|
+
| Dependency | How tests control it | Contract test or source of truth |
|
|
202
|
+
|---|---|---|
|
|
203
|
+
| <dependency> | <real, or replaced on the edge> | <contract test, or source and reason> |
|
|
204
|
+
|
|
205
|
+
## Known gaps
|
|
206
|
+
<What the test cases do not cover, and why. "None" if there are no gaps.>
|
|
207
|
+
|
|
208
|
+
## Slice 1: <name from structure.md>
|
|
209
|
+
<What the slice gives, copied from structure.md.>
|
|
210
|
+
|
|
211
|
+
### Goal case
|
|
212
|
+
**Interface**: <interface> (top)
|
|
213
|
+
**Given** <the state before the action>
|
|
214
|
+
**When** <the action>
|
|
215
|
+
**Then**:
|
|
216
|
+
- <exact check>
|
|
217
|
+
|
|
218
|
+
### Cases
|
|
219
|
+
#### TC-1.1 <intent, in words of the business domain>
|
|
220
|
+
**Interface**: <interface> (<level>)
|
|
221
|
+
**Guards**: "<rule or decision, copied>"
|
|
222
|
+
**Given** ...
|
|
223
|
+
**When** ...
|
|
224
|
+
**Then**:
|
|
225
|
+
- <exact check>
|
|
226
|
+
|
|
227
|
+
## Slice 2: <name>
|
|
228
|
+
...
|
|
229
|
+
|
|
230
|
+
## Findings
|
|
231
|
+
<Problems for the user to solve in the previous steps. "None" if there are no problems.>
|
|
232
|
+
````
|
|
233
|
+
|
|
234
|
+
Show `test-plan.md` to the user. The user can add, change, or remove cases. Change the file until the user approves it.
|
|
235
|
+
|
|
236
|
+
Done when the user approves the test plan and **Findings** is "None".
|
|
237
|
+
|
|
238
|
+
## 9. Hand off
|
|
239
|
+
|
|
240
|
+
Tell the user: "Next: run `qrspi-implement` with `<task-dir>`."
|
|
241
|
+
|
|
242
|
+
## Rules
|
|
243
|
+
|
|
244
|
+
- Write only black-box test cases for public interfaces. Do not write test cases for inner code.
|
|
245
|
+
- Do not read `plan.md`.
|
|
246
|
+
- Write one goal case for each slice, on the top level.
|
|
247
|
+
- Put the cases for each rule on the lowest public interface.
|
|
248
|
+
- Each case guards a rule or a decision. Copy its text.
|
|
249
|
+
- Write **Then** as a list of exact checks on the output side of the box.
|
|
250
|
+
- Do not use mocks, if possible. Replace only on the outer edge. Use a contract test, if possible.
|
|
251
|
+
- Make `test-plan.md` complete alone.
|
|
252
|
+
- Write test cases, not tests. The implement step writes the code and the tests.
|
|
253
|
+
- Do not do research. Do not change the previous files. Write a problem in **Findings**, and stop.
|