codeassembly-guidance 0.0.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.meta/changelog.json +1320 -0
- package/CHANGELOG.md +315 -0
- package/LICENSE +7 -0
- package/README.md +195 -1
- package/bin/codeassembly-validate-preferences.js +18 -0
- package/content/_partials/README.md +192 -0
- package/content/_partials/code-descriptions.md +4 -0
- package/content/_partials/code-style.md +7 -0
- package/content/_partials/comment-discipline.md +89 -0
- package/content/_partials/concision.md +3 -0
- package/content/_partials/diff-audit-checklist.md +9 -0
- package/content/_partials/file-access.md +4 -0
- package/content/_partials/live-repo-writes.md +13 -0
- package/content/_partials/plain-speech-calibration.md +70 -0
- package/content/_partials/plain-speech.md +13 -0
- package/content/_partials/prose-line-breaks.md +3 -0
- package/content/_partials/record-marker.md +1 -0
- package/content/_partials/reduced-object-relative.md +28 -0
- package/content/_partials/shell-commands.md +3 -0
- package/content/_partials/target-file-set-resolution.md +22 -0
- package/content/_partials/technical-recommendations.md +6 -0
- package/content/_partials/test-structure-audit-checklist.md +7 -0
- package/content/_partials/ticket-assessment-baseline.md +12 -0
- package/content/_partials/ticket-assessment.md +202 -0
- package/content/_partials/work-type-choice.md +13 -0
- package/content/codeassembly-content.yaml +59 -0
- package/content/collections/all.md +11 -0
- package/content/collections/atlassian.md +17 -0
- package/content/collections/recommended.md +41 -0
- package/content/collections/triage.md +13 -0
- package/content/collections/williamthorsen.md +26 -0
- package/content/guidance/README.md +22 -0
- package/content/guidance/_harnesses/claude/CLAUDE.md +6 -0
- package/content/guidance/_harnesses/rovo/AGENTS.md +5 -0
- package/content/guidance/_harnesses/rovo/codeassembly-guidance.md +4 -0
- package/content/guidance/rulebooks/accessibility-conventions.md +38 -0
- package/content/guidance/rulebooks/cli-conventions.md +192 -0
- package/content/guidance/rulebooks/codeassembly-content-specification.md +184 -0
- package/content/guidance/rulebooks/command-output-conventions.md +14 -0
- package/content/guidance/rulebooks/commit-conventions.md +72 -0
- package/content/guidance/rulebooks/generated-content-policy.md +10 -0
- package/content/guidance/rulebooks/live-worktree-policy.md +19 -0
- package/content/guidance/rulebooks/readme-conventions.md +74 -0
- package/content/guidance/rulebooks/shared-data-conventions.md +14 -0
- package/content/guidance/rulebooks/shell-conventions.md +273 -0
- package/content/guidance/rulebooks/ui-copy-conventions.md +34 -0
- package/content/guidance/rulebooks/williamthorsen-code-layout-preferences.md +102 -0
- package/content/guidance/rulebooks/williamthorsen-collaboration-preferences.md +60 -0
- package/content/guidance/rulebooks/williamthorsen-comment-preferences.md +30 -0
- package/content/guidance/rulebooks/williamthorsen-subagent-model-preferences.md +15 -0
- package/content/guidance/rulebooks/williamthorsen-ticketing-preferences.md +22 -0
- package/content/guidance/rulebooks/williamthorsen-tooling-preferences.md +36 -0
- package/content/guidance/rulebooks/williamthorsen-typescript-preferences.md +90 -0
- package/content/guidance/rulebooks/williamthorsen-workflow-preferences.md +58 -0
- package/content/guidance/rulebooks/williamthorsen-writing-preferences.md +80 -0
- package/content/guidance/shared/AGENTS.md +51 -0
- package/content/scripts/README.md +44 -0
- package/content/scripts/check-branch-currency.mjs +11 -0
- package/content/scripts/check-prose.mjs +171 -0
- package/content/scripts/describe-change.mjs +165 -0
- package/content/scripts/get-ticket-id.sh +131 -0
- package/content/scripts/manage-jira-ticket.mjs +214 -0
- package/content/scripts/resolve-frontmatter.sh +540 -0
- package/content/scripts/resolve-review-run.mjs +12 -0
- package/content/scripts/select-lede-exemplars.mjs +208 -0
- package/content/scripts/validate-preferences.mjs +146 -0
- package/content/skills/_data/action-items.md +83 -0
- package/content/skills/_data/artifact-conventions.md +1001 -0
- package/content/skills/_data/bitbucket-pr-access.md +89 -0
- package/content/skills/_data/branch-currency.md +36 -0
- package/content/skills/_data/branch-format.md +34 -0
- package/content/skills/_data/change-record.md +163 -0
- package/content/skills/_data/complexity-classification.md +58 -0
- package/content/skills/_data/concision.md +23 -0
- package/content/skills/_data/deployed-file-provenance.md +46 -0
- package/content/skills/_data/design-priorities.md +76 -0
- package/content/skills/_data/gh-body-file.md +30 -0
- package/content/skills/_data/git-commands.md +39 -0
- package/content/skills/_data/naming-conventions.md +50 -0
- package/content/skills/_data/pr-resolution.md +25 -0
- package/content/skills/_data/pr-source-resolution.md +36 -0
- package/content/skills/_data/project-stage.md +32 -0
- package/content/skills/_data/recommendation-gradient.md +92 -0
- package/content/skills/_data/scope-and-deferral.md +83 -0
- package/content/skills/_data/spike-conventions.md +89 -0
- package/content/skills/_data/ticket-evaluation.md +13 -0
- package/content/skills/_data/ticket-id-extraction.md +43 -0
- package/content/skills/_data/ticket-source-resolution.md +180 -0
- package/content/skills/_data/title-templates.md +632 -0
- package/content/skills/_data/title-voice.md +17 -0
- package/content/skills/_data/work-types.json +168 -0
- package/content/skills/_partials/acceptance-criteria-scaffold.md +15 -0
- package/content/skills/_partials/action-items.md +22 -0
- package/content/skills/_partials/bitbucket-delete-both-refusal.md +3 -0
- package/content/skills/_partials/commit-title-rendering.md +14 -0
- package/content/skills/_partials/gh-body-file.md +22 -0
- package/content/skills/_partials/jira-task-list-flattening.md +1 -0
- package/content/skills/_partials/nested-list-indent.md +9 -0
- package/content/skills/_partials/next-steps-after-plan.md +99 -0
- package/content/skills/_partials/next-steps-after-review.md +350 -0
- package/content/skills/_partials/option-format.md +85 -0
- package/content/skills/_partials/plan-template.md +97 -0
- package/content/skills/_partials/split-launch-prompts.md +5 -0
- package/content/skills/_partials/split-ticket-compose.md +5 -0
- package/content/skills/_partials/split-ticket-create.md +3 -0
- package/content/skills/_partials/ticket-and-plan-completeness.md +33 -0
- package/content/skills/_partials/ticket-concision.md +18 -0
- package/content/skills/_partials/ticket-criteria-conventions.md +7 -0
- package/content/skills/_partials/ticket-placement.md +14 -0
- package/content/skills/_partials/ticket-skeleton-tiers.md +6 -0
- package/content/skills/_partials/ticket-skeleton.md +13 -0
- package/content/skills/add-change-record/SKILL.md +136 -0
- package/content/skills/add-test-ids/SKILL.md +51 -0
- package/content/skills/align-ticket-with-implementation/SKILL.md +115 -0
- package/content/skills/anti-patterns/SKILL.md +57 -0
- package/content/skills/assess-ticket/SKILL.md +46 -0
- package/content/skills/assess-ticket/next-steps-after-assessment.md +98 -0
- package/content/skills/assess-usability/SKILL.md +96 -0
- package/content/skills/brainstorming/SKILL.md +107 -0
- package/content/skills/capture-event/SKILL.md +108 -0
- package/content/skills/capture-event/capture-event.mjs +216 -0
- package/content/skills/capture-feedback/SKILL.md +71 -0
- package/content/skills/capture-lede-decision/SKILL.md +163 -0
- package/content/skills/capture-lede-decision/capture-lede-decision.mjs +231 -0
- package/content/skills/classify-complexity/SKILL.md +92 -0
- package/content/skills/collaborate/SKILL.md +86 -0
- package/content/skills/condense-branch/SKILL.md +83 -0
- package/content/skills/create-bitbucket-pr/SKILL.md +87 -0
- package/content/skills/create-commit/SKILL.md +65 -0
- package/content/skills/create-devlog/SKILL.md +108 -0
- package/content/skills/create-gh-pr/SKILL.md +121 -0
- package/content/skills/create-pr/SKILL.md +150 -0
- package/content/skills/create-ticket/SKILL.md +395 -0
- package/content/skills/create-ticket/resolve-ticket-target.mjs +143 -0
- package/content/skills/derive-session-context/derive-session-context.mjs +149 -0
- package/content/skills/design-and-plan/SKILL.md +197 -0
- package/content/skills/development-workflows/SKILL.md +41 -0
- package/content/skills/get-ticket-id/SKILL.md +67 -0
- package/content/skills/groom-backlog/SKILL.md +247 -0
- package/content/skills/groom-backlog/groom-backlog.mjs +108 -0
- package/content/skills/implement-plan/SKILL.md +168 -0
- package/content/skills/index-prototypes/SKILL.md +200 -0
- package/content/skills/index-prototypes/index-prototypes.mjs +388 -0
- package/content/skills/kb-add/SKILL.md +157 -0
- package/content/skills/kb-add/kb-add.mjs +216 -0
- package/content/skills/kb-curate/SKILL.md +119 -0
- package/content/skills/kb-curate/kb-curate.mjs +213 -0
- package/content/skills/kb-edit/SKILL.md +128 -0
- package/content/skills/kb-edit/kb-edit.mjs +219 -0
- package/content/skills/kb-retrieve/SKILL.md +100 -0
- package/content/skills/kb-retrieve/kb-retrieve.mjs +207 -0
- package/content/skills/kb-retrieve-events/SKILL.md +98 -0
- package/content/skills/kb-retrieve-events/kb-retrieve-events.mjs +207 -0
- package/content/skills/kb-update-events/SKILL.md +77 -0
- package/content/skills/kb-update-events/kb-update-events.mjs +215 -0
- package/content/skills/merge-bb-pr/SKILL.md +136 -0
- package/content/skills/merge-gh-pr/SKILL.md +167 -0
- package/content/skills/merge-gh-pr/merge-gh-pr.mjs +10 -0
- package/content/skills/merge-pr/SKILL.md +368 -0
- package/content/skills/migrate-feedback-memories/SKILL.md +137 -0
- package/content/skills/migrate-feedback-memories/feedback-memories.mjs +161 -0
- package/content/skills/people-report/SKILL.md +208 -0
- package/content/skills/plan/SKILL.md +87 -0
- package/content/skills/prepare-prototype-brief/SKILL.md +178 -0
- package/content/skills/pull-from-backlog/SKILL.md +83 -0
- package/content/skills/pull-from-backlog/pull-from-backlog.mjs +209 -0
- package/content/skills/refine-plan/SKILL.md +263 -0
- package/content/skills/refine-prototype/SKILL.md +113 -0
- package/content/skills/respond-to-review/SKILL.md +292 -0
- package/content/skills/review-bb-pr/SKILL.md +161 -0
- package/content/skills/review-branch/SKILL.md +303 -0
- package/content/skills/review-criteria/SKILL.md +141 -0
- package/content/skills/review-gh-pr/SKILL.md +155 -0
- package/content/skills/review-gh-pr/select-pr-ticket.mjs +8 -0
- package/content/skills/review-permissions/SKILL.md +39 -0
- package/content/skills/review-pr/SKILL.md +123 -0
- package/content/skills/revise-comments/SKILL.md +148 -0
- package/content/skills/revise-prose/SKILL.md +197 -0
- package/content/skills/revise-prose/revise-prose.mjs +229 -0
- package/content/skills/save-artifact/SKILL.md +71 -0
- package/content/skills/save-plan/SKILL.md +55 -0
- package/content/skills/software-engineering/SKILL.md +74 -0
- package/content/skills/streamline-guidance/SKILL.md +174 -0
- package/content/skills/streamline-guidance/streamline-guidance.mjs +211 -0
- package/content/skills/summarize-change/SKILL.md +317 -0
- package/content/skills/summarize-chat/SKILL.md +100 -0
- package/content/skills/systematic-debugging/SKILL.md +308 -0
- package/content/skills/systematic-debugging/condition-based-waiting-example.ts +160 -0
- package/content/skills/systematic-debugging/condition-based-waiting.md +122 -0
- package/content/skills/systematic-debugging/defense-in-depth.md +130 -0
- package/content/skills/systematic-debugging/find-polluter.sh +68 -0
- package/content/skills/systematic-debugging/root-cause-tracing.md +185 -0
- package/content/skills/systematic-debugging/test-academic.md +16 -0
- package/content/skills/systematic-debugging/test-pressure-1.md +67 -0
- package/content/skills/systematic-debugging/test-pressure-2.md +77 -0
- package/content/skills/systematic-debugging/test-pressure-3.md +77 -0
- package/content/skills/testing-conventions/SKILL.md +235 -0
- package/content/skills/triage-spike-findings/SKILL.md +113 -0
- package/content/skills/triage-spike-findings/triage-spike-findings.mjs +71 -0
- package/content/skills/typescript-testing-conventions/SKILL.md +178 -0
- package/content/skills/update-jira-ticket/SKILL.md +257 -0
- package/content/skills/update-jira-ticket/update-jira-ticket.mjs +9 -0
- package/content/skills/update-project-guidance/SKILL.md +300 -0
- package/content/skills/upgrade-dependencies/SKILL.md +139 -0
- package/content/skills/wrap-up/SKILL.md +431 -0
- package/content/subagents/_data/claude.yaml +25 -0
- package/content/subagents/_data/rovo.yaml +41 -0
- package/content/subagents/canary.md +12 -0
- package/content/subagents/disposition-challenger.md +78 -0
- package/content/subagents/entry-drafter.md +191 -0
- package/content/subagents/handoff-reviewer.md +75 -0
- package/content/subagents/plan-reviewer.md +184 -0
- package/content/subagents/plan-reviser.md +88 -0
- package/content/subagents/prose-reviser.md +132 -0
- package/content/subagents/ticket-assessor.md +142 -0
- package/content/subagents/usability-assessor.md +157 -0
- package/package.json +74 -2
- package/schemas/preferences.v1.json +321 -0
- package/schemas/work-types.v1.json +142 -0
|
@@ -0,0 +1,235 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: testing-conventions
|
|
3
|
+
description: What a test is for, how to name and structure it, and how it calls git and other external programs. Consult before writing or changing a test.
|
|
4
|
+
user-invocable: false
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Testing conventions
|
|
8
|
+
|
|
9
|
+
## What a test is for
|
|
10
|
+
|
|
11
|
+
A test guards a behavior against accidental breakage. A **behavior** is an outcome that a caller or a user relies on: what a function returns, what a user can do, what the system refuses. A **value** is a thing that the code contains today while producing it: the text of a message, the contents of a config, the members of a list. A test that pins a value fails on every intended change to that value and catches nothing, so each such change is made twice.
|
|
12
|
+
|
|
13
|
+
Before writing a test, ask what would go uncaught if it did not exist. Three answers rule the test out.
|
|
14
|
+
|
|
15
|
+
- **A change that someone would have to make on purpose.** Rewording a message, editing a config, adding a member to a list: Each is a decision, and review already guards decisions. A test is justified when it catches an accident, a change made without intending to touch this outcome, such as a route added without its guard or a refactor that drops a case. A scenario constructed so that the test has something to catch is not an accident.
|
|
16
|
+
- **Nothing that this repository authored.** Machine state, the user's environment, and a dependency's behavior are not under test, and enabling a dependency's feature does not make that feature's behavior ours. If the right response to red would be a bug report upstream, the test is not ours.
|
|
17
|
+
- **A difference that costs nothing until noticed.** A relaxed lint rule, a setting that drifts from its best value, a directory that departs from convention: Maintaining a standing test takes more effort than noticing the difference would. A caught failure has to be worse than noticing it: a wrong answer returned, a task that a user cannot finish, data lost or exposed.
|
|
18
|
+
|
|
19
|
+
Then ask where the expected result comes from. A specification, a format, or a rule that someone could state without opening the code makes it an expectation. If pasting in whatever the code now produces would make the test pass again, the test pins a value, whatever its name says. When a test compares two sets, derive the expected one from its source; a set typed into the test drifts.
|
|
20
|
+
|
|
21
|
+
Three cases recur. Configuration is verified by the tool that reads it, which the build and lint gates run; if an accident is real and the tool is silent, prevent it at the tool rather than detect it in a test. Wording is asserted only when the text is the sole thing that distinguishes two code paths that exist, and then by a durable fragment. A process that this repository does not author is called through an adapter and replaced by a fake, as [External programs](#external-programs) states.
|
|
22
|
+
|
|
23
|
+
If a rule above forbids a test and the case seems exceptional, stop before writing it, state the case in one sentence, and wait.
|
|
24
|
+
|
|
25
|
+
## When tests are required
|
|
26
|
+
|
|
27
|
+
Every change to a behavior, as defined above, is covered by a test. A change that does not add runtime behavior does not need a new test: a deletion, a type consumed only within this compile, prose, styling.
|
|
28
|
+
|
|
29
|
+
A test that guards a behavior still has to clear two checks:
|
|
30
|
+
|
|
31
|
+
1. **Nothing else fails first.** If the compiler, the linter, or a gate that already reproduces the failure condition on every run would catch the regression, that gate is the guard, and a narrower test is a weaker and more brittle duplicate. A gate that merely exercises the same area does not qualify; it has to reproduce the condition.
|
|
32
|
+
2. **It would catch the failure that it guards against.** Run it against what it rejects: Break the guarded thing, watch the test fail, restore it. A test that still passes is not a guard. If reaching the cause means replicating a dependency's private shape, the test breaks or silently stops testing at the next upgrade.
|
|
33
|
+
|
|
34
|
+
If a change needs coverage and these sections rule out every candidate test, it is merged without one; say which answer ruled the candidate out.
|
|
35
|
+
|
|
36
|
+
## Where a test goes
|
|
37
|
+
|
|
38
|
+
A test's **tier** is what it starts or builds to observe a behavior. The cheapest tier calls the code in the test's own process. An **expensive tier** spawns an entry point as a process, starts a runtime or an engine such as a database, or builds a full fixture. An engine counts as expensive whether it runs in the test's process or in another. A project's own tier names may classify tests on another axis, such as what they reach, and a test takes its label from the project's definition; nmr's, for example, labels a test that runs an embedded engine `tool`. The rules below concern cost.
|
|
39
|
+
|
|
40
|
+
- **Tier placement.** Test each case at the cheapest tier that can observe it. An expensive tier covers only what a cheaper tier cannot observe: that the entry starts, parses its arguments, wires its dependencies, and reports its exit status, with one smoke example per entry or verb. A branch of logic, a message, or a field of output is a case for the cheapest tier, even when the entry is the only caller that exercises it.
|
|
41
|
+
- **Consolidate before adding.** Write a new example only for a behavior that the existing examples cannot be extended to cover. Cases that share setup and action are one table-driven test, as [Test structure](#test-structure) states. When the examples do not mutate a fixture, build it once per file, not once per example.
|
|
42
|
+
- **Delete what is superseded, in the same change.** A higher-tier example is superseded when cheaper coverage observes each of its assertions, apart from the entry wiring that the smoke example covers. A change that adds the cheaper coverage deletes the examples that it supersedes, once the new tests pass the second check in [When tests are required](#when-tests-are-required). A port or a rewrite follows the same rule: The change that contains the port deletes the old suite rather than keeping it to catch regressions.
|
|
43
|
+
- **Justify each expensive addition.** A change that adds an example at an expensive tier states, in its plan or its pull-request description, why a cheaper tier cannot observe that example. The justification does not belong in a test comment.
|
|
44
|
+
- **Use the cheapest setup that fits.** Give an expensive tier the cheapest setup that fits its time budget, such as an in-process engine when few files need one. Add infrastructure, such as a server, a container, or a shared test package, only after a run of the suite at CI's parallelism shows that the cheaper setup does not fit.
|
|
45
|
+
- **Keep setup cost independent of the number of files.** Design expensive setup so that adding or splitting a test file does not add to it, for example by starting one engine for the suite rather than one per file.
|
|
46
|
+
- **Fix the design, not the timeout.** When a hook, a test, or a tier needs a longer timeout, in the test or in config, treat that as a sign of bad design and make it do less work: smaller data and injected dependencies, as [Test data](#test-data) states, or setup shared across files. A raised timeout hides a cost that grows with each file and with CI load. Raise one only when none of these fixes applies, and state why in the plan or the pull-request description, not in a test comment.
|
|
47
|
+
|
|
48
|
+
## Test data
|
|
49
|
+
|
|
50
|
+
- **Use the smallest data that proves the behavior.** A test renders or builds only the records that its assertion needs. Production data, such as a full catalog, appears only in a smoke test whose subject is that data.
|
|
51
|
+
- **Make data that a test cannot replace injectable.** When the code under test reads data or a dependency from a module-level singleton, fix the code: Take the value as a parameter, or read it from a context or provider whose default is the real value. Do not test against the real data, and do not mock this repository's module by its path, which keeps the singleton and ties the test to a file layout.
|
|
52
|
+
|
|
53
|
+
```tsx
|
|
54
|
+
// ❌ Avoid: The component reads the singleton, so every test renders the full catalog
|
|
55
|
+
function ScopePicker() {
|
|
56
|
+
return <ul>{scopeCatalog.map((scope) => <li key={scope.id}>{scope.label}</li>)}</ul>;
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
// ✅ Good: The context defaults to the real catalog, and the test supplies two scopes
|
|
60
|
+
const ScopeCatalogContext = createContext(scopeCatalog);
|
|
61
|
+
|
|
62
|
+
function ScopePicker() {
|
|
63
|
+
const catalog = useContext(ScopeCatalogContext);
|
|
64
|
+
return <ul>{catalog.map((scope) => <li key={scope.id}>{scope.label}</li>)}</ul>;
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
it('renders one item per scope', () => {
|
|
68
|
+
const scopes = [
|
|
69
|
+
{ id: 'read', label: 'Read' },
|
|
70
|
+
{ id: 'write', label: 'Write' },
|
|
71
|
+
];
|
|
72
|
+
render(
|
|
73
|
+
<ScopeCatalogContext.Provider value={scopes}>
|
|
74
|
+
<ScopePicker />
|
|
75
|
+
</ScopeCatalogContext.Provider>,
|
|
76
|
+
);
|
|
77
|
+
expect(screen.getAllByRole('listitem')).toHaveLength(2);
|
|
78
|
+
});
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## Do not test that removed things stay removed
|
|
82
|
+
|
|
83
|
+
This is the commonest test that guards a decision. When a change removes code, text, or behavior, never add a permanent test asserting the removed thing is absent (a `not.toContain` guard against a deleted string, `expect(isEventType('input.received')).toBe(false)` against a removed variant). The assertion encodes history, not contract: It can fail only if someone reverts that exact line, so it does not guard any regression class, and such assertions accumulate without limit. The positive assertion describing the replacement behavior is the real guard.
|
|
84
|
+
|
|
85
|
+
Diagnostic: Would this test exist if the deleted code had never existed? If no, don't write it.
|
|
86
|
+
|
|
87
|
+
Verify the removal is complete once, as a pre-merge check (a `grep`, a plan Verification step), not a standing test. This applies to any change that removes something, not only removal-only changes.
|
|
88
|
+
|
|
89
|
+
When a test guards that a set is closed, the witness is an arbitrary non-member, never a member that was removed. The removed name guards nothing that the arbitrary one does not, and it records history.
|
|
90
|
+
|
|
91
|
+
## Loosen a test broken by a wording-only change
|
|
92
|
+
|
|
93
|
+
When a wording-only change forces a test update, don't re-pin the new wording: Match just the part that identifies the behavior, or drop the assertion if the behavior does not depend on the text.
|
|
94
|
+
|
|
95
|
+
## Naming of tests
|
|
96
|
+
|
|
97
|
+
**Always** use test names that describe the specific behavior being tested.
|
|
98
|
+
**Never** use vague or meaningless terms.
|
|
99
|
+
|
|
100
|
+
### Avoid these meaningless patterns
|
|
101
|
+
|
|
102
|
+
- ❌ **"handles X"**: Too vague, doesn't describe what happens
|
|
103
|
+
- ❌ **"does X correctly"**: Meaningless (all tests verify correctness)
|
|
104
|
+
- ❌ **"works with Y"**: Doesn't specify the expected outcome
|
|
105
|
+
- ❌ **"processes Z properly"**: "Properly" is subjective and unclear
|
|
106
|
+
|
|
107
|
+
### Use specific behavioral descriptions
|
|
108
|
+
|
|
109
|
+
- ✅ **"returns 0 when map is empty"**: Clear condition and outcome
|
|
110
|
+
- ✅ **"throws error for invalid input"**: Specific behavior under specific condition
|
|
111
|
+
- ✅ **"preserves existing data when adding new item"**: Describes what remains unchanged
|
|
112
|
+
- ✅ **"removes only items matching criteria"**: Specifies selective behavior
|
|
113
|
+
|
|
114
|
+
### Test name formula
|
|
115
|
+
|
|
116
|
+
Use this pattern: `"when/if [condition], [action/outcome]"`
|
|
117
|
+
|
|
118
|
+
Examples:
|
|
119
|
+
|
|
120
|
+
- `"if no items match filter, returns empty array"`
|
|
121
|
+
- `"if email format is invalid, throws ValidationError"`
|
|
122
|
+
- `"when saving user preferences, updates timestamp"`
|
|
123
|
+
- `"preserves order when removing middle element"`
|
|
124
|
+
|
|
125
|
+
### What makes a good test name
|
|
126
|
+
|
|
127
|
+
1. **Specific**: Describes exact behavior, not general capability
|
|
128
|
+
2. **Conditional**: States the circumstances that trigger the behavior
|
|
129
|
+
3. **Outcome-focused**: What happens, not just that something happens
|
|
130
|
+
4. **Readable**: Anyone can understand the test purpose without reading the code
|
|
131
|
+
|
|
132
|
+
<!-- include: ../../_partials/comment-discipline.md / -->
|
|
133
|
+
|
|
134
|
+
## Comments in test files
|
|
135
|
+
|
|
136
|
+
The same rules apply to test files as to source. Test names already communicate intent and assertions communicate the check, so everything outside the test-comment carve-out is over-commenting.
|
|
137
|
+
|
|
138
|
+
## Test structure
|
|
139
|
+
|
|
140
|
+
Tests should make their variation easy to see. When N adjacent tests differ only in one input but share a large block of identical setup, the reader has to diff three or four nearly-identical render calls to find what's actually being tested. The fix is not "delete things"; it's "factor the shared part out so that the variation reads as variation."
|
|
141
|
+
|
|
142
|
+
This is the same signal-buried-in-noise failure mode that [comment discipline](#comment-discipline) addresses on the comment side. Different mechanism, same principle.
|
|
143
|
+
|
|
144
|
+
### Rules
|
|
145
|
+
|
|
146
|
+
1. **Test bodies should be mostly variation and assertion, not setup.** Any prop, fixture, or boilerplate identical across N adjacent tests is noise. Factor it into a helper that supplies defaults, or collapse the tests into a single parameterized test in which the variation reads as a table.
|
|
147
|
+
2. **Use `it.each` when N adjacent tests differ only in a small set of inputs and the body is structurally identical.** Use a named helper when the variation is bigger, when shared setup should move into a helper signature, or when assertion shapes differ per case.
|
|
148
|
+
3. **Test the rule, not the data.** Prefer counts, predicates, and structural assertions (`expect(getVisibleChipCount()).toBe(N)`, `expect(queryPillElement()).toBeInTheDocument()`) over enumerating specific fixture labels in every row. Assert specific data flow _once_, in a focused test, not per row of a table.
|
|
149
|
+
|
|
150
|
+
### Diagnostic
|
|
151
|
+
|
|
152
|
+
Before writing a third test in the same `describe` block, scan the previous two: How many tokens does a reader have to diff to find what's actually different between them? If the answer is more than a handful, the signal is buried. Parameterize the trio or extract a helper before continuing.
|
|
153
|
+
|
|
154
|
+
### When N copies are right
|
|
155
|
+
|
|
156
|
+
The smell is shared _setup with one variable_, not shared _shape with different intents_. Three tests that read as genuinely distinct behavioral claims should stay as three `it` blocks even if their bodies superficially resemble each other. Do not collapse distinct intents into a table.
|
|
157
|
+
|
|
158
|
+
### Audit before save
|
|
159
|
+
|
|
160
|
+
<!-- include: ../../_partials/test-structure-audit-checklist.md / -->
|
|
161
|
+
|
|
162
|
+
## Test organization
|
|
163
|
+
|
|
164
|
+
- **Use function/class reference as describe argument**: `describe(myFunction, ...)` instead of `describe('myFunction', ...)`
|
|
165
|
+
|
|
166
|
+
### Test file length
|
|
167
|
+
|
|
168
|
+
A test file runs in one test worker, and a file that keeps its fixtures makes an append cheaper than a split, so a file grows until it is the slowest unit in the suite. Before adding tests to a file that the addition would leave past the ceiling, split the file by aspect, then add the tests to the file that their aspect names.
|
|
169
|
+
|
|
170
|
+
- **Ceiling.** The `max-lines` setting that the project's lint configuration applies to test files. When the lint does not set one, 500 lines, counting every line.
|
|
171
|
+
- **Aspect.** A group of the subject's behaviors that shares setup and reads without the rest of the file. Each split file contains one aspect.
|
|
172
|
+
- **Naming.** Each split file keeps the subject's name and adds the aspect after it, followed by any segment that the project's naming requires: `sync.test.ts` becomes `sync.retirement.test.ts`, and `sync.tool.test.ts` becomes `sync.retirement.tool.test.ts`.
|
|
173
|
+
- **Setup.** A split does not add expensive setup. When the file starts an engine or another expensive fixture of its own, share that setup across files first, as [Where a test goes](#where-a-test-goes) states, then split.
|
|
174
|
+
- **Fixtures.** Move the fixture builders that the split files share out of the test files, into a module in a `test-utils/` directory beside the tests (beside `__tests__/` when the tests live in one, never inside it), so that a new aspect file needs only one import.
|
|
175
|
+
|
|
176
|
+
## Keep passing tests quiet
|
|
177
|
+
|
|
178
|
+
A passing test prints nothing beyond the runner's report. Output that the code under test writes to the console, stdout, or stderr during a passing run appears between the results, and it hides the warning or stack trace that matters when a run fails.
|
|
179
|
+
|
|
180
|
+
When the code under test writes output, intercept it in the test that triggers it, and restore it when that test ends:
|
|
181
|
+
|
|
182
|
+
- **Capture** the output when it is the behavior, such as a CLI's report or a warning on which a caller relies, and assert on a durable fragment of it, as [What a test is for](#what-a-test-is-for) states for wording.
|
|
183
|
+
- **Silence** the output when it is incidental to the behavior under test.
|
|
184
|
+
|
|
185
|
+
Intercept only the channels that the test expects to fire. A silencer installed for a whole file or suite, or one that covers every channel, hides the unexpected output that this rule exists to surface.
|
|
186
|
+
|
|
187
|
+
The `typescript-testing-conventions` skill names the TypeScript techniques.
|
|
188
|
+
|
|
189
|
+
## Mocking principles
|
|
190
|
+
|
|
191
|
+
Mock only what matters in component tests. Don't forward irrelevant props or replicate complex implementation details.
|
|
192
|
+
|
|
193
|
+
```typescript
|
|
194
|
+
// ✅ Good: Focused on test needs
|
|
195
|
+
jest.mock('@atlaskit/component', () => ({
|
|
196
|
+
Component: ({ children, isOpen }: Props) =>
|
|
197
|
+
isOpen ? <div data-testid="component">{children}</div> : null
|
|
198
|
+
}));
|
|
199
|
+
|
|
200
|
+
// ❌ Avoid: Unnecessary complexity
|
|
201
|
+
jest.mock('@atlaskit/component', () => ({
|
|
202
|
+
Component: ({ children, isOpen, width, onClose, ...props }: ComplexProps) =>
|
|
203
|
+
<div {...props} style={{ width }}>{isOpen && children}</div>
|
|
204
|
+
}));
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
### External programs
|
|
208
|
+
|
|
209
|
+
A test that runs an external program to produce its inputs verifies the program rather than this code, and it depends on the copy installed on the machine that runs it: its version, its config, and its background processes. A `git commit` in a test repository, for example, can start a background maintenance run that is still writing into `.git/` when the test removes the directory.
|
|
210
|
+
|
|
211
|
+
- Code calls an external program whose behavior is not the subject of the test, such as git, through a small adapter that the code receives as a dependency.
|
|
212
|
+
- Tests of that code pass a fake adapter that returns fixed answers.
|
|
213
|
+
- Only the adapter's own test runs the real program, and it checks that the adapter parses the program's output as the code expects.
|
|
214
|
+
- A test does not create a repository, or any other state of a real program, to produce an input that a fake could return.
|
|
215
|
+
|
|
216
|
+
A test whose subject is the program's effect on real state, such as a command that initializes a repository, runs the program because that effect is the behavior under test.
|
|
217
|
+
|
|
218
|
+
The `typescript-testing-conventions` skill shows the TypeScript shape.
|
|
219
|
+
|
|
220
|
+
### Code that uses a database
|
|
221
|
+
|
|
222
|
+
A database is a program like any other in [External programs](#external-programs): A test starts one only when its subject is SQL.
|
|
223
|
+
|
|
224
|
+
- Logic that only passes through SQL, such as assembling a view, gating a command, or ordering and reporting, is tested at the cheapest tier against a fake client that returns fixed rows and errors, such as a SQLSTATE. Mock at the boundary of the module containing the SQL when that is simpler than faking the client.
|
|
225
|
+
- The database tests worth writing cover what the schema does not already state: that the migrations apply, transactional bookkeeping, upserts and idempotence, and queries.
|
|
226
|
+
- Do not write a test per constraint, such as inserting a duplicate row to expect the error. The test that the migrations apply covers the schema.
|
|
227
|
+
|
|
228
|
+
## Additional patterns
|
|
229
|
+
|
|
230
|
+
### Omit "should" from test names
|
|
231
|
+
|
|
232
|
+
Describe what the code actually does when the test passes:
|
|
233
|
+
|
|
234
|
+
- ✅ "returns false for unknown gates"
|
|
235
|
+
- ❌ "should return false for unknown gates"
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: triage-spike-findings
|
|
3
|
+
description: Dispose of the findings that a spike or a usability assessment left without a ticket. Inventories every undisposed item, grades each one's confidence and value, has a subagent without session context challenge every grade, and then creates the labelled tickets under an umbrella and records every disposition on the triage ticket. Use when a findings artifact has proposals or ideas that the existing tickets have not absorbed.
|
|
4
|
+
user-invocable: true
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Triage spike findings
|
|
8
|
+
|
|
9
|
+
Dispose of every item that a findings artifact leaves undisposed: the proposals, groupings, findings, and ideas that the filed tickets and comments did not absorb. You inventory them, recommend a disposition and two grades for each, have a {subagent:disposition-challenger} argue against every recommendation, and then, once the developer approves, create the tickets and post the record. A bundled helper checks the repository's labels, validates the dispositions record, and renders the comments.
|
|
10
|
+
|
|
11
|
+
**Announce at start:** "Using triage-spike-findings to dispose of the findings of {source} on {ticket_ref}."
|
|
12
|
+
|
|
13
|
+
## Inputs
|
|
14
|
+
|
|
15
|
+
- **The findings artifact**: A spike's findings or a `usability-findings` artifact, by local path or by artifact URL. Read a URL with the Artifact tool and save the content to `{ticket_dir}/triage/findings.md`, so that the challenger can read it from a path; on a harness without that tool, the input is a local path.
|
|
16
|
+
- **The triage ticket**: The ticket on which the dispositions are posted and under which the working files are kept, resolved per [ticket source resolution](../_data/ticket-source-resolution.md); when none is given, the session's ticket. The umbrella is created as a separate ticket, so the triage ticket may be the spike's own ticket.
|
|
17
|
+
|
|
18
|
+
GitHub alone is supported, through `gh`. When the session-context manifest's `scm` is not `github`, stop at preflight and report that the skill does not support the platform.
|
|
19
|
+
|
|
20
|
+
## Grades and dispositions
|
|
21
|
+
|
|
22
|
+
Each item takes a disposition and two grades:
|
|
23
|
+
|
|
24
|
+
- **Disposition**: `ticket` (a child of the umbrella), `comment` (a note on an existing issue, which must be open and must own the file or the behavior), `merge` (folded into another item's ticket; the target is a `ticket` item), or `drop`.
|
|
25
|
+
- **Confidence** states advisability: `high` is clearly worth doing, `medium` is worth doing but arguable, `low` is not worth a ticket now. A borderline item takes the lower grade. Only an item graded `high` or `medium` gets a ticket; the helper refuses a record that gives a `low` item one.
|
|
26
|
+
- **Value** states how much the item matters: `must-have`, `should-have`, `nice-to-have`, `questionable`. Value never decides whether an item gets a ticket; it is recorded on every item and labels every child.
|
|
27
|
+
|
|
28
|
+
The labels come from the common preset of `release-kit sync-labels`: the `value:` family, and the `confidence:` family that williamthorsen/node-monorepo-tools#1017 added. A repository whose labels lack either family cannot take the children, because `gh issue create` fails on a label that the repository does not have.
|
|
29
|
+
|
|
30
|
+
## Process
|
|
31
|
+
|
|
32
|
+
Every helper command prints one JSON result. When a result has `ok: false`, stop and report its `error` and `message`, except when a step below says otherwise. Invoke the helper as:
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
node {harness_home_dir}/skills/triage-spike-findings/triage-spike-findings.mjs {command} {flags}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Resolve `{ticket_dir}` per {skill:save-artifact}, and write every working file under `{ticket_dir}/triage/`.
|
|
39
|
+
|
|
40
|
+
### 1. Preflight
|
|
41
|
+
|
|
42
|
+
1. Run `gh auth status` and `gh repo view --json nameWithOwner`. If either fails, or `scm` is not `github`, stop and report why.
|
|
43
|
+
2. Run the helper's `check-labels`. When `complete` is false, stop: Name each missing label, and state that the common preset of `release-kit sync-labels` supplies both families (the `confidence:` family from williamthorsen/node-monorepo-tools#1017), so the repository takes them by upgrading release-kit and regenerating its label file rather than by creating them by hand.
|
|
44
|
+
3. Create `{ticket_dir}/triage/`, and save the findings there when they came from a URL.
|
|
45
|
+
|
|
46
|
+
### 2. Inventory
|
|
47
|
+
|
|
48
|
+
Read the findings in full. List every item that the artifact leaves without a disposition, where a disposition is a ticket or a comment that the artifact itself names: a proposed ticket that was not filed, a grouping, a finding outside every grouping, and an idea that the filed tickets did not absorb. Verify each item's claim against the current code and against the open issues (`gh issue list --search`, `gh issue view`): The finding may describe a problem that a merge has since fixed, a decision that the repository records, or work that an open issue already covers.
|
|
49
|
+
|
|
50
|
+
Write `inventory.md` as a table: the item's id (the artifact's own id when it has one), its title, where it appears in the artifact, and the verification outcome (holds, partly holds, or does not hold, with the evidence).
|
|
51
|
+
|
|
52
|
+
### 3. Recommend
|
|
53
|
+
|
|
54
|
+
Write `recommendations.md`: one line per item, with its id, title, disposition, confidence, value, and a one-sentence reason. Nothing else goes in the file, because the challenger reads it and a sentence of your reasoning there would bias its judgment toward yours. A `merge` names a `ticket` item as its target; a `comment` names its issue number.
|
|
55
|
+
|
|
56
|
+
### 4. Challenge
|
|
57
|
+
|
|
58
|
+
Send one {tool:Task} call, `subagent_type: disposition-challenger`, with this block and nothing else composed into it. `findings` is the copy saved in step 1, or the local path that was given:
|
|
59
|
+
|
|
60
|
+
```dispatch
|
|
61
|
+
findings: {ticket_dir}/triage/findings.md
|
|
62
|
+
recommendations: {ticket_dir}/triage/recommendations.md
|
|
63
|
+
root: {repository root}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
**A reply that lacks the `Phase: disposition-challenge` line is a challenger stopped by its turn limit.** Resume it once, asking only for its report; on Claude Code, send it a message with `SendMessage`, addressed by the agent ID that its dispatch returned. When the resumed challenger still does not return a report, do not dispatch again: Record the challenge as not returned, treat every item as unchallenged, and say so in the record's `challengeSummary`.
|
|
67
|
+
|
|
68
|
+
Save the reply as `challenge.md`. Then take each item in turn: Defend the recommendation when the argument fails on the evidence, modify it when the argument stands in part, and abandon it when the argument stands.
|
|
69
|
+
|
|
70
|
+
Decide the blockers of every `ticket` item under the threshold in the ticketing preferences rulebook: An item is blocked by another ticket when there is some probability that the other ticket affects it and should be merged first, and a ticket whose design takes another ticket's outcome as an input is blocked by it. A blocker is another `ticket` item, by id, or an open issue in this repository outside the triage, by number; check each issue with `gh issue view {issue} --json state`, and omit a closed one, which does not block. Record every such dependency in `blockedBy`, never in `notes`.
|
|
71
|
+
|
|
72
|
+
Write `dispositions.json` in the helper's record shape: `source` (the findings as given), `triageTicket`, `umbrella: null`, `items`, `challengeSummary` (what the challenge refuted or narrowed and what it did not), and `notes` (the lines of the comment's "Labels and follow-ups" section, empty when there are none). Each item has `id`, `title`, `disposition`, `confidence`, `value`, `reason`, `target` (the issue number of a `comment`, the target id of a `merge`, null otherwise), `ticket: null`, `blockedBy` on a `ticket` item that has blockers (item ids and issue numbers), and `challenge` with `argument`, `outcome` (`defended`, `modified`, or `abandoned`), and `note`. Run the helper's `validate`, which reports every schema and gate violation at once, a cycle among the blockers included, and fix the record until it passes.
|
|
73
|
+
|
|
74
|
+
### 5. Checkpoint: The dispositions
|
|
75
|
+
|
|
76
|
+
Present the dispositions as a table (item, disposition, confidence, value, blocked by, reason, challenge outcome), in the order that `validate` reports as `order`, which places every blocker before the items that it blocks, and the challenge summary, and ask in the action-items block for approval. Revise the record until the developer approves it; re-run `validate` after each change.
|
|
77
|
+
|
|
78
|
+
<!-- include: ../_partials/action-items.md / -->
|
|
79
|
+
|
|
80
|
+
### 6. Compose and confirm
|
|
81
|
+
|
|
82
|
+
Compose the bodies:
|
|
83
|
+
|
|
84
|
+
- **The umbrella**, when at least one item is a `ticket`: its problem (the follow-up work of the findings), its context (a link to the triage ticket, and the tickets that the findings filed without adopting them, listed and not made children), and "Every child is closed" as its acceptance criterion. It does not take a graded label.
|
|
85
|
+
- **One child per `ticket` item**, in the ticket skeleton that {skill:create-ticket} step 2 states: problem, context, and acceptance criteria, with the title and reason of every `merge` item that targets it folded in. Its labels are `confidence:{grade}` and `value:{grade}`.
|
|
86
|
+
|
|
87
|
+
Run the helper's `render --preview` to `{ticket_dir}/triage/preview/`, which writes the dispositions comment with `(pending)` where a number is not yet known, and one comment file per issue that a `comment` disposition names.
|
|
88
|
+
|
|
89
|
+
Show everything that step 7 will create or post in one block: every body with its title and labels, the parent relationship and the blockers of each child, every per-issue comment with its issue, and the dispositions comment. Confirm once in the action-items block. This confirmation is the only one: {skill:create-ticket} does not re-confirm a relationship given as an argument.
|
|
90
|
+
|
|
91
|
+
### 7. Write
|
|
92
|
+
|
|
93
|
+
Nothing before this step writes anything remote. On a failed write, stop and report which item failed; every number already recorded stays in `dispositions.json`, so that a re-run of this step creates only what is still null and checks each issue before posting on it.
|
|
94
|
+
|
|
95
|
+
1. When at least one item is a `ticket` and `umbrella` is null, create the umbrella with {skill:create-ticket}, and write its number into `umbrella`.
|
|
96
|
+
2. For each `ticket` item, in the order that `validate` reports as `order` for the approved record, run {skill:create-ticket} with `--parent {umbrella}` and `--label confidence:{grade},value:{grade}`, adding `--blocked-by` with the item's blockers as `#{number}` (the recorded `ticket` of each item id, and each issue number) when it has any, and write the returned number into the item's `ticket` as soon as it is returned. Because `order` places every blocker first, each item id that `--blocked-by` names already has its number.
|
|
97
|
+
3. For each `ticket` item, read `gh issue view {ticket} --json parent,blockedBy`, and add what is missing in one `gh issue edit {ticket} --parent {umbrella} --add-blocked-by {numbers}` call, omitting a flag that has nothing to add. {skill:create-ticket} sets the parent and the blockers in one call, so a rejected blocker can drop the parent as well, and this read-back is how a failed link is found. On a re-run, it completes the links of the items that an earlier run created.
|
|
98
|
+
4. Run the helper's `render` to `{ticket_dir}/triage/rendered/`, without `--preview`.
|
|
99
|
+
5. Post each `comment-{issue}.md` on its issue, and then `dispositions-comment.md` on the triage ticket. Before each post, read the issue's comments with `gh issue view {issue} --json comments`, and skip the post when a comment already opens with the first line of the rendered file:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
body_path="{ticket_dir}/triage/rendered/comment-{issue}.md"
|
|
103
|
+
[ -s "$body_path" ] || { echo "Body file missing or empty: $body_path" >&2; exit 1; }
|
|
104
|
+
gh issue comment {issue} --body-file "$body_path"
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
6. Save `dispositions-comment.md`, with frontmatter, as the ticket-level `dispositions` artifact per {skill:save-artifact}.
|
|
108
|
+
|
|
109
|
+
### 8. Report
|
|
110
|
+
|
|
111
|
+
Report the umbrella when one was created, each child with its grades and blockers in the order in which step 7 created them, each issue commented on, and the artifact's path. The triage ticket stays open: Closing it is the developer's call, once they have read the record.
|
|
112
|
+
|
|
113
|
+
<!-- include: ../_partials/gh-body-file.md / -->
|