@assuredloop/orca-bot-kit 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +545 -3
  3. package/SETUP.md +270 -0
  4. package/package.json +32 -13
  5. package/rules/evidence.md +15 -0
  6. package/rules/finishing.md +20 -0
  7. package/rules/limits.md +18 -0
  8. package/rules/mail.md +17 -0
  9. package/rules/profiles.md +9 -0
  10. package/rules/review.md +14 -0
  11. package/rules/scope.md +20 -0
  12. package/rules/simple.md +14 -0
  13. package/rules/talk.md +13 -0
  14. package/rules/tests-first.md +15 -0
  15. package/rules/the-ask.md +19 -0
  16. package/skills/obk-arch/LICENSES.md +45 -0
  17. package/skills/obk-arch/NOTICE.md +134 -0
  18. package/skills/obk-arch/SKILL.md +387 -0
  19. package/skills/obk-bot-building/LICENSES.md +21 -0
  20. package/skills/obk-bot-building/NOTICE.md +113 -0
  21. package/skills/obk-bot-building/SKILL.md +235 -0
  22. package/skills/obk-debugging/LICENSES.md +67 -0
  23. package/skills/obk-debugging/NOTICE.md +115 -0
  24. package/skills/obk-debugging/SKILL.md +419 -0
  25. package/skills/obk-decision-memo/LICENSES.md +49 -0
  26. package/skills/obk-decision-memo/NOTICE.md +181 -0
  27. package/skills/obk-decision-memo/SKILL.md +241 -0
  28. package/skills/obk-finops/LICENSES.md +19 -0
  29. package/skills/obk-finops/NOTICE.md +77 -0
  30. package/skills/obk-finops/SKILL.md +146 -0
  31. package/skills/obk-fleet-review/LICENSES.md +52 -0
  32. package/skills/obk-fleet-review/NOTICE.md +104 -0
  33. package/skills/obk-fleet-review/SKILL.md +219 -0
  34. package/skills/obk-grilling/LICENSES.md +42 -0
  35. package/skills/obk-grilling/NOTICE.md +103 -0
  36. package/skills/obk-grilling/SKILL.md +217 -0
  37. package/skills/obk-grooming/LICENSES.md +52 -0
  38. package/skills/obk-grooming/NOTICE.md +111 -0
  39. package/skills/obk-grooming/SKILL.md +219 -0
  40. package/skills/obk-handoff/LICENSES.md +45 -0
  41. package/skills/obk-handoff/NOTICE.md +51 -0
  42. package/skills/obk-handoff/SKILL.md +119 -0
  43. package/skills/obk-personal-facilitation/LICENSES.md +53 -0
  44. package/skills/obk-personal-facilitation/NOTICE.md +62 -0
  45. package/skills/obk-personal-facilitation/SKILL.md +86 -0
  46. package/skills/obk-recall/LICENSES.md +42 -0
  47. package/skills/obk-recall/NOTICE.md +53 -0
  48. package/skills/obk-recall/SKILL.md +129 -0
  49. package/skills/obk-researching/LICENSES.md +67 -0
  50. package/skills/obk-researching/NOTICE.md +170 -0
  51. package/skills/obk-researching/SKILL.md +259 -0
  52. package/skills/obk-reviewing/LICENSES.md +66 -0
  53. package/skills/obk-reviewing/NOTICE.md +58 -0
  54. package/skills/obk-reviewing/SKILL.md +408 -0
  55. package/skills/obk-tdd/LICENSES.md +68 -0
  56. package/skills/obk-tdd/NOTICE.md +54 -0
  57. package/skills/obk-tdd/SKILL.md +516 -0
  58. package/skills/obk-tdd/mutation-tools.md +80 -0
  59. package/skills/obk-teach/LICENSES.md +42 -0
  60. package/skills/obk-teach/NOTICE.md +65 -0
  61. package/skills/obk-teach/SKILL.md +106 -0
  62. package/skills/obk-why/LICENSES.md +42 -0
  63. package/skills/obk-why/NOTICE.md +74 -0
  64. package/skills/obk-why/SKILL.md +169 -0
  65. package/skills/obk-writing/LICENSES.md +272 -0
  66. package/skills/obk-writing/NOTICE.md +161 -0
  67. package/skills/obk-writing/SKILL.md +254 -0
  68. package/src/book.js +268 -0
  69. package/src/bot.js +451 -0
  70. package/src/cli.js +1130 -0
  71. package/src/conversations.js +150 -0
  72. package/src/groom.js +157 -0
  73. package/src/health.js +309 -0
  74. package/src/hooks.js +202 -0
  75. package/src/init.js +314 -0
  76. package/src/launch.js +276 -0
  77. package/src/message.js +380 -0
  78. package/src/orca.js +291 -0
  79. package/src/pause.js +58 -0
  80. package/src/record.js +209 -0
  81. package/src/restart.js +192 -0
  82. package/src/retire.js +92 -0
  83. package/src/roster.js +71 -0
  84. package/src/rules.js +408 -0
  85. package/src/skills.js +629 -0
  86. package/src/sources.js +370 -0
  87. package/src/up.js +475 -0
  88. package/src/usage.js +401 -0
  89. package/bin/okb.js +0 -4
  90. package/src/index.js +0 -11
@@ -0,0 +1,516 @@
1
+ ---
2
+ name: obk-tdd
3
+ description: >-
4
+ Test-driven work: a failing test before the code, one vertical slice at a
5
+ time, through the public interface; a separate author for the tests, whose
6
+ tests the implementer does not edit; the shapes of tests that catch nothing;
7
+ and the hand mutation check that asks whether the tests would catch a real
8
+ mistake. Use whenever you are about to change what code does (a feature, a
9
+ fix, a one-line change, a bug someone reported) and whenever you are
10
+ writing, changing or judging tests, deciding what to test, or asking whether
11
+ a suite would catch anything. Reach for it before writing the code, not
12
+ after: a small change is exactly where the order gets skipped.
13
+ ---
14
+
15
+ # Test-driven work
16
+
17
+ The order is the point: the check exists, and has been seen to fail, before the
18
+ code that satisfies it. What that buys is a test written against the
19
+ requirement rather than against the implementation you already had in mind.
20
+
21
+ Three things hold the tests honest, in this order of value: the author is
22
+ someone other than the implementer, a reviewer reads the tests, and the
23
+ implementer breaks its own logic on purpose to see whether a test notices.
24
+
25
+ These are defaults for work where nothing says otherwise. A user who asks for
26
+ something else gets what they asked for; say which of these you left and why.
27
+
28
+ ## Find out how this project tests
29
+
30
+ The loop is the same everywhere; the commands are not. Before the first test,
31
+ learn how *this* project tests: the build file and whether it has a checked-in
32
+ wrapper to use instead of a globally installed tool, the framework and its
33
+ configuration, how it runs one focused test as against the whole suite, where
34
+ tests live and how they are named, what the tests next door already do, and
35
+ which commands the project's own checks actually run. Never assume a default
36
+ command because it is the usual one for the language.
37
+
38
+ Use the focused command through the loop and the whole-suite command before you
39
+ finish.
40
+
41
+ ## Start from the requirement, not the code
42
+
43
+ Write the acceptance criteria before any test: one line each, each one
44
+ checkable on its own, each phrased as behaviour someone could observe. A
45
+ criterion you cannot check is a criterion nobody can hold you to.
46
+
47
+ A requirement's silence is not permission. What a reasonable reader would
48
+ expect is part of it: the invalid input, the denied permission, the empty
49
+ collection. List the input classes the requirement implies but no test
50
+ exercises, and give the few most likely to bite someone using it a test each.
51
+
52
+ You cannot test everything, and the point of choosing is that the effort lands
53
+ on the paths people depend on and the logic that is easy to get wrong, not on
54
+ every edge case there is.
55
+
56
+ Expected values come from the requirement, a worked example or a known-good
57
+ literal, never from the code under test or its helpers. An expectation the
58
+ code computes passes whatever the code does.
59
+
60
+ Say which requirement a test covers, in its name or in a line beside it. A test
61
+ that names a requirement but does not assert its behaviour does not cover it.
62
+
63
+ Test your own contract, not the library's. What the framework does is its
64
+ maintainers' test to write; yours is the route you register, the query you
65
+ emit, the payload you produce. A constructor, an accessor, a constant or a
66
+ plain hand-off earns a test only where it validates, normalises, defaults,
67
+ derives, enforces or causes an effect; otherwise assert the first result a
68
+ caller can see that depends on it. When upstream behaviour genuinely surprised
69
+ you, one narrow test that names the assumption is worth keeping.
70
+
71
+ Behaviour, not text. Asserting that a script, a configuration or a document
72
+ contains a particular line proves only that the source is the source. Run the
73
+ thing against a controlled input and assert what comes out: the output, the
74
+ effect, the exit status. Prose written for people earns no test at all.
75
+
76
+ ## One vertical slice at a time
77
+
78
+ One test, then the smallest implementation that passes it, then the next. Not
79
+ all the tests first and then all the code: tests written in bulk describe
80
+ behaviour you imagined, and they all go green in one step that nobody watched.
81
+
82
+ A separate author writing the acceptance tests up front is not that, although
83
+ it looks like it: those are a few tests at the top seam, taken from the
84
+ requirement rather than from a plan. You still work through them one at a time,
85
+ watching each one fail and then pass. When a slice turns out to need a narrower
86
+ test (a boundary the top seam does not reach), that one goes to the author
87
+ too; the brief is smaller, not different.
88
+
89
+ Test at a seam: the public boundary where you can observe behaviour without
90
+ reaching inside. Prefer a seam that already exists, and the highest one that
91
+ still shows the behaviour; the fewer seams a codebase has, the better. State
92
+ the seam you are testing at. Ask about it only when the interface itself is
93
+ what is in question.
94
+
95
+ Do not anticipate the next test. Code that no failing test asked for is
96
+ untested code, whatever it does.
97
+
98
+ Before you settle a test, run the behaviour past the kinds of mistake that get
99
+ made (a boundary off by one, a condition the wrong way round, an equality that
100
+ should be an inequality, an arithmetic identity, an empty collection, a side
101
+ effect that never happens) and add the case you find missing there and then.
102
+ The inputs that tell those apart: just below, at and just above a boundary; one
103
+ condition true and the other false; and values that are not identities, since
104
+ 0, 1 and empty hide a swapped operator.
105
+ It is the same question the mutation check asks at the end, asked while it is
106
+ still cheap to answer.
107
+
108
+ Use the least substitute that does the job, because the more of the real thing
109
+ a test runs, the more it tells you. In order: the real implementation; a
110
+ working in-memory version of it; something that returns fixed answers; and last
111
+ a stand-in you assert the calls of. Reach past the real thing only when it is
112
+ slow, unpredictable, or has effects you cannot control: an external service,
113
+ the clock, randomness, a database sometimes, the filesystem sometimes. Do not
114
+ stand in for your own modules or internal collaborators.
115
+
116
+ What settles this for a system outside your code is not who owns it but
117
+ whether you can have one to yourself. A database you start for the test, a
118
+ repository you create in a temporary directory, a server you run and throw
119
+ away: those are the real thing, isolated, and they are what the order above
120
+ asks for; a stand-in there would hide exactly the incompatibility you want to
121
+ find. What rules the real one out is sharing it: other people's work in it,
122
+ state you cannot reset, or simply not having permission to touch it. Then an
123
+ author works against a stand-in, and the project's rules usually name which
124
+ systems those are.
125
+
126
+ When a test is hard to write, that is information about the design, not about
127
+ testing. Hard to set up usually means hard to use. Having to stand in for
128
+ everything means the pieces are too tightly bound. Pass what a function
129
+ depends on in rather than letting it reach out and build its own. A single
130
+ function per external operation is easier to substitute than one general one
131
+ with a switch inside it, because each substitute then answers one question
132
+ instead of branching.
133
+
134
+ Four things about a stand-in, when you do need one. Learn what the real thing
135
+ does before you replace it, and stand in at the slow or external level below
136
+ what the test depends on, so you do not swallow an effect the test was there to
137
+ see. Mirror the real shape completely, not only the fields this test reads: a
138
+ partial one passes while the real thing breaks on a field it left out. Assert
139
+ the real behaviour, never the stand-in itself. An assertion that the stand-in
140
+ is present passes because it is present and says nothing about your code. And
141
+ when the setup grows past the test it serves, or you cannot say why it is
142
+ there, drop it and test against the real components.
143
+
144
+ Cleanup that only tests need belongs in the test helpers, not as a method on
145
+ the thing being tested.
146
+
147
+ ## The test author is someone else
148
+
149
+ An agent that writes the code and the tests shares one set of blind spots. It
150
+ writes tests it already knows will pass, and when one goes red the cheapest
151
+ road is to soften it. What separation buys is a context that holds the
152
+ requirement and not the implementation. The isolated context is the point, not
153
+ a role or a persona.
154
+
155
+ Who counts: a fresh subagent, another session, another bot, or a person. In
156
+ Claude Code it must be a subagent with its own context, not a fork that
157
+ inherits the conversation. Codex spawns a subagent only when asked in so many
158
+ words, so ask in so many words.
159
+
160
+ What the author gets: the requirement, the acceptance criteria, and the public
161
+ interface (names, signatures, types, the shape of inputs and outputs). Not your
162
+ plan, not your diff, not the implementation. What the author returns: the test
163
+ files, the command, the failing output, and one line per test naming the
164
+ requirement it covers.
165
+
166
+ A brief that works, adjusted to the job:
167
+
168
+ > Write tests for this requirement. You are not writing the implementation and
169
+ > you will not see it.
170
+ > Requirement: <what the behaviour is, and the acceptance criteria>.
171
+ > Out of scope: <what this change does not cover, if anything>.
172
+ > Public interface: <signatures and data shapes, no bodies>.
173
+ > Test at <seam>, in <the repo's test style and framework>; follow the
174
+ > repo's existing tests for shape. Write them the way obk-tdd's parts on
175
+ > writing tests say, if you can load it.
176
+ > Test observable behaviour through that interface. Take expected values from
177
+ > the requirement or a worked example, never by calling the code under test.
178
+ > Cover the unhappy inputs the requirement implies as well as the happy one.
179
+ > Run them. They are all expected to fail, and to fail because the behaviour is
180
+ > missing rather than because of a typo or an import error.
181
+ > Return: the files you wrote, the command, its failing output, and one line
182
+ > per test naming the requirement it covers.
183
+
184
+ Say what state the code is in, and be exact about it. The author cannot see it
185
+ and will believe what the brief says. When you send an author at code you have
186
+ deliberately broken (killing a surviving mutant, see below), say so in the
187
+ brief: *the code has a deliberate break in it and the test is expected to
188
+ fail*. Left out, the author reads your own break as a defect and spends a round
189
+ reporting a bug that does not exist.
190
+
191
+ Then, as the implementer: every test that ships is the author's, the narrow
192
+ ones as much as the first ones. A scratch check you run to see what the code is
193
+ doing is yours and is thrown away; it is not part of the suite, and it is not
194
+ the red you report. You do not edit, weaken, skip or delete the author's tests
195
+ to get to green. A test you believe
196
+ is wrong goes back to its author with what you think is wrong with it. That is
197
+ a real and expected outcome, not a defeat. The tests verify the requirement;
198
+ they do not define the solution, so do not write code that works only for the
199
+ inputs a test happens to use.
200
+
201
+ The author's tests are part of the change, not a stage before it. Commit them
202
+ as soon as they are red, before the implementation: that way a later diff over
203
+ the test paths shows any change to them, and (the part that actually bites)
204
+ they cannot be left behind. Where the repo allows it, keeping those paths out
205
+ of the implementer's write scope makes tampering visible rather than merely
206
+ forbidden.
207
+
208
+ An author you have dispatched and not heard back from is outstanding work. The
209
+ piece is not done while it is still writing, and nothing goes out on the
210
+ strength of tests that have not arrived. When the hand-back comes, read it
211
+ against what is actually there: every file it names, present in the working
212
+ tree, and present in the change you are about to hand on. A finished test file
213
+ still sitting in a working tree is the ordinary way this goes wrong, and it
214
+ looks exactly like success until someone else clones the branch.
215
+
216
+ When you can neither start an author nor reach one, that is where the work
217
+ stops: say so and ask what to do. The separation is the check, and an
218
+ implementer writing its own tests is not a weaker version of it, it is the
219
+ absence of it.
220
+
221
+ ## Red is evidence, not a formality
222
+
223
+ Run the test and read the output before you write any code:
224
+
225
+ - It fails, rather than errors.
226
+ - The failure message is the one you expected.
227
+ - It fails because the behaviour is missing, not a typo, a bad import, a
228
+ missing fixture.
229
+
230
+ If it passes, you are testing behaviour that already exists; that is a finding
231
+ about the test, not a step you may skip past. If it errors, fix the error and
232
+ run again until it fails cleanly.
233
+
234
+ Behaviour that already exists is the case this is easiest to fudge, because
235
+ there is no fix to revert and nothing obvious to run the test against. The red
236
+ has to come from somewhere real: an archive of the code as it stood before this
237
+ work (a commit, a tag, a copy) with the test run against that. Name which one
238
+ in the hand-over, because "it would have failed before" is not a run.
239
+
240
+ Sometimes there is no such baseline to be had: the code never existed in that
241
+ shape, or what you would have to reconstruct is not worth the day. Then say so
242
+ in the hand-over, and in a line beside the test marking it as a description of
243
+ what the code does today rather than a guard proven to catch a change. The two
244
+ look identical afterwards, and only one of them has been shown to fail.
245
+
246
+ Reaching a clean red is one step at a time, and none of them is implementation:
247
+ the test cannot find the symbol, so add an empty stub; the call does not match
248
+ the signature, so fix the signature and leave the body empty; now it fails on
249
+ the assertion, and only now does logic get written.
250
+
251
+ Keep the failing output as it was printed. Commit order proves nothing about
252
+ what ran first. Without a captured failing run, say the chronology is unproven
253
+ rather than implying it.
254
+
255
+ Things that look like green and are not: a test that passed on its first run; a
256
+ suite reported as passing that nobody ran; a default test command guessed
257
+ instead of the one this repo actually uses. And your test passing is not the
258
+ suite passing. Any failure the run showed, including one you did not cause,
259
+ goes in your report by name. Green means the output is clean too: warnings and
260
+ errors that scroll past on a passing run are results, not decoration.
261
+
262
+ Where the change reaches further than the test you wrote, run what is next to
263
+ it as well (the neighbouring tests, the type check, whatever the project uses
264
+ to say a thing still holds) and say in your report what you ran.
265
+
266
+ An assertion that already existed is not yours to loosen. It weakens only when
267
+ the behaviour it describes genuinely changed, and then the reason is stated out
268
+ loud, not left to be inferred from the diff.
269
+
270
+ ## Green, then refactor outside the loop
271
+
272
+ Write the smallest code that passes the failing test, and stop there.
273
+
274
+ Faking it is allowed and often right. If returning the answer the test asks for
275
+ passes, return it; then write the second case that the fake cannot satisfy, and
276
+ let that one force the real implementation. Generalise when a test makes you,
277
+ not before.
278
+
279
+ Refactoring is not a phase of the red/green loop. A small refactor is part of
280
+ the change and keeps the same contract, so the tests that were green stay
281
+ green; do it as its own step after green rather than while you are trying to
282
+ get there, and keep the tests green on both sides of it. Pulling out something
283
+ whose behaviour already exists elsewhere is refactoring; a function whose
284
+ behaviour appears nowhere else is new, and it starts with a failing test like
285
+ anything else.
286
+
287
+ Do not manufacture a red. A failing test written so a step looks like it
288
+ started with one, a break invented so a restructuring can be called a change of
289
+ behaviour, commits reshaped afterwards to tell a tidier story: all of that
290
+ costs the time of doing it and buys nothing, because none of it was ever a
291
+ check on anything. A step that did not need a red is reported as what it was.
292
+
293
+ When a change is large enough that the old tests cannot hold, that is still not
294
+ a licence to edit them. The author writes them again from the new requirement,
295
+ usually deleting the old ones first. A large refactor is planned as a piece of
296
+ work in its own right, not carried inside another change.
297
+
298
+ Tests against an inner module become waste once the same behaviour is covered
299
+ at the interface above it. Deleting them is the author's call, and deliberate.
300
+
301
+ ## Bugs
302
+
303
+ A bug fix starts with a test that reproduces it, written before the fix, at a
304
+ seam where the test exercises the bug as it actually happens at the call site.
305
+ A seam below that gives a green test and no protection. If no correct seam
306
+ exists, that is itself the finding: report it, because the shape of the code is
307
+ what is stopping the bug from being pinned down.
308
+
309
+ Minimise the reproduction until every part of it is load-bearing, turn it into
310
+ a failing test, watch it fail, fix the cause, watch it pass, then run the
311
+ original unminimised scenario again.
312
+
313
+ Two tests are often better than one here: one at the interface, saying what a
314
+ caller should have got, and the smallest one that shows the fault where it
315
+ lives. Get both to pass. The first is what stops the bug coming back in a way
316
+ anyone would notice; the second says where it was.
317
+
318
+ A regression test written after the fix has proved nothing yet. Prove it:
319
+ revert the fix, run it and watch it fail, restore the fix, run it again.
320
+
321
+ When the bug is intermittent, make the test deterministic if you can, and say
322
+ which signal you pinned down. When it is one of a class, land the test for the
323
+ one in front of you first, then say what else has the same shape.
324
+
325
+ When a failing test really is impractical, say so out loud, say why, and name
326
+ the closest executable check you used instead. Quiet omission is the part that
327
+ does damage: a reviewer treats a missing red receipt as a finding.
328
+
329
+ ## Tests that catch nothing
330
+
331
+ Before writing the body, answer one question: what change to the production
332
+ code should make this test fail, and is that change a bug or a decision? A test
333
+ earns its place by catching a wrong branch, a missing side effect, a wrong
334
+ argument, a boundary or a broken contract. A test that only fails on a
335
+ deliberate decision fires on every redesign and sleeps through every bug.
336
+
337
+ The cheap check on a test you already have: would it still pass if every
338
+ function it calls returned nothing? If yes, it observes no behaviour.
339
+
340
+ Five shapes that still pass that way, and what to do with each:
341
+
342
+ - **No real assertion.** Nothing asserted, or only "is defined", "is truthy",
343
+ "did not throw". Call the subject with one concrete input and assert the
344
+ literal output or the observable effect.
345
+ - **An absence with nothing to contrast it against.** Only "was not called",
346
+ "is undefined", "is empty", and no case in the same test where something
347
+ *is* produced. Assert the presence on the other input beside it, so the pair
348
+ can tell an empty result from a subject that never ran. For a stand-in,
349
+ assert the payload it received or the state after the call, not only that it
350
+ was called.
351
+ - **Self-referential.** The expected value is produced by the code under test.
352
+ Replace it with a literal you worked out by hand.
353
+ - **Constant pinned.** The assertion restates a constant, a config default or
354
+ the exact wording of a message. Test the mechanism that reads the constant
355
+ with one input instead.
356
+ - **Fixture asserts fixture.** The assertion reads data the test itself built,
357
+ and the subject never runs. Run the subject inside the test body.
358
+
359
+ An empty or absent result is not by itself a weak assertion. "This user sees
360
+ none of the private records" and "this call is never made" are contracts, and
361
+ a test that pins one catches a real defect. So are the arguments a call
362
+ carries, how many times it happens and in what order, where those are the
363
+ contract rather than an accident of how the code is written today. What makes
364
+ the shape weak is an absence with no positive case beside it, or a stub
365
+ asserted instead of the behaviour. Judge by the question that governs the
366
+ whole list: would this catch a defect someone could ship?
367
+
368
+ When no honest assertion exists, delete the test. Prefer no new test to a bad
369
+ one: one that mostly tests stubs, that encodes today's implementation, that
370
+ depends on timing or global state, or that you would delete the moment the fix
371
+ is proven.
372
+
373
+ Two that the five shapes do not condemn: a test that checks a relation holds
374
+ across rows of data (a key present in two tables, a parent that exists), and a
375
+ check the language itself makes at build time.
376
+
377
+ Other tells: the test breaks when you refactor although the behaviour did not
378
+ change; it reaches around the interface to check the result (reading the
379
+ database directly instead of asking for the record back), which couples it to
380
+ the inside just as surely as a stub does; the stub setup is more than half of
381
+ it; the expected values are hidden behind loops or builders; the only way it
382
+ can fail is a crash; it exists to raise a coverage number. Coverage is a
383
+ diagnostic and not a target: a high number says the lines ran, not that
384
+ anything would have noticed them being wrong.
385
+
386
+ One behaviour per test, so that a failure names one thing. That is not a limit
387
+ on how many assertions you write: a table of cases for one behaviour is still
388
+ one test, and the several assertions that together pin one outcome are one
389
+ claim. It is a limit on how many unrelated claims share a name.
390
+
391
+ Repetition between tests is not a fault. A test that reads on its own, with its
392
+ input and its expected value in front of you, is worth more than one that sends
393
+ you through three helpers to find out what it claims. Share setup when it makes
394
+ the test clearer, not to avoid typing.
395
+
396
+ What is missing is usually on the unhappy side: the denied case of a permission
397
+ check, untested guard clauses and early returns, error branches, boundaries
398
+ (zero, empty, negative, one element, the maximum), and the second half of any
399
+ "if". And tests that pass together but not alone are already broken: shared
400
+ mutable state, order dependence, the clock, the timezone, the real network, a
401
+ sleep, an assertion on the order of something unordered.
402
+
403
+ ## The mutation check before you call it done
404
+
405
+ The question is the whole point: if a real mistake were made in this code,
406
+ would a test fail? There is no score to reach.
407
+
408
+ **Everyday: your own hand check.** Once per piece of work, at the end, only
409
+ where it earns its place. Five to eight deliberate breaks in the risky logic
410
+ the work changed (money, permissions, eligibility, safety, data that could be
411
+ lost, and whatever else the requirement cares most about), chosen before you
412
+ look at the tests, each one run, each one expected to make a test fail, each
413
+ one put back. Reported in three lines.
414
+
415
+ One break at a time, never two at once: note the original, apply the break, run
416
+ the tests, record killed or survived, restore the code immediately, then the
417
+ next one.
418
+
419
+ What to break, in this order: boundaries (`<` for `<=`, one off), boolean logic
420
+ (`&&` for `||`, a dropped `not`), returned values (an empty or default return,
421
+ a removed early return), a removed statement or side effect, a wrong constant
422
+ or argument. Avoid a change that cannot show: adding zero, multiplying by one, or a value
423
+ identical to the original.
424
+
425
+ Others worth reaching for when the code has them: an arithmetic operator
426
+ swapped; equality turned into its opposite; a string emptied or altered; a
427
+ collection emptied, or a filter or a sort taken out; a safe-navigation step
428
+ removed so an absent value reaches further than it should; a call to a
429
+ collaborator deleted.
430
+
431
+ Skip it, and say that you skipped it, for work with no real logic in it: docs,
432
+ config, wiring, renames, small fixes, prototypes, and code with no runnable
433
+ tests.
434
+
435
+ **A survivor**, a break no test objected to, gets the equivalence question
436
+ first: is there any input for which the changed code returns a different value,
437
+ stores different state, or emits a different message? If there is none it is
438
+ equivalent, and no test can catch it; write the one-line argument for why.
439
+ "No test covers this" is not the same as "no input can reach this". Every
440
+ survivor ends as exactly one of three things: killed by a new test, equivalent
441
+ with that argument, or a judgment call you name and leave. A list of survivors
442
+ with nothing said about them is not a report, and neither is treating each one
443
+ as a defect.
444
+
445
+ When you cannot tell whether a survivor matters (the behaviour is genuinely
446
+ unspecified, the test would be expensive or brittle, or you are not sure it is
447
+ equivalent), name it as that in the report, with the question whoever decides
448
+ the behaviour would have to answer, rather than deciding quietly in either
449
+ direction.
450
+ And if a lot of the breaks turn out to be equivalent, that is a finding about
451
+ the code, not about the tests: code with that much slack in it can usually be
452
+ made simpler.
453
+
454
+ Kill a survivor only where it shows a gap in behaviour the requirement cares
455
+ about. Wording of messages, logging and changes with no visible difference are
456
+ not worth a test; say so in a line and move on. To kill one: keep the break in
457
+ place, write the smallest behaviour test that fails against it for the right
458
+ reason, restore the code, check the new test still passes.
459
+
460
+ The implementer runs the check. A test that needs strengthening goes to the
461
+ separate author, like any other test.
462
+
463
+ **A mutation tool is an audit, not everyday work.** It is never a requirement,
464
+ and it is not worth a user's time on a suite it does not fit. Where a project
465
+ has one, run it over the whole suite occasionally (at a milestone, or when the
466
+ user asks) in the background at a quiet time, in its own tab or session. Read
467
+ the result once; turn the few real gaps into test work; ignore the rest. Not
468
+ once per change, and never twice for the same change. For a piece of work at
469
+ the core of a product, one run narrowed to the logic that changed, in the
470
+ background, around twenty minutes; if it will not fit in that, do the hand
471
+ check instead. Then stop. A run heading towards hours is stopped.
472
+
473
+ When you have decided to run a tool, read [mutation-tools.md](mutation-tools.md)
474
+ beside this file: the scoped run for each language, what older advice gets
475
+ wrong, how to run one and read what it prints, and how far each line there has
476
+ been verified. There is nothing in it you need before that point.
477
+
478
+ Keep it in proportion. This check is the third safety net, after the separate
479
+ author and the reviewer's reading of the tests, and it stays the cheapest of
480
+ the three.
481
+
482
+ ## Green that was bought
483
+
484
+ Tightening the bar should be silent; loosening it should be loud. In your own
485
+ diff and in anyone else's, these are the cheap roads to green, and each one is
486
+ worth a sentence when it happens: a threshold moved, a test marked skipped, a
487
+ test file deleted, assertions removed from a test that stayed, a check disabled
488
+ by a comment, a value hard-coded to match one test's input and still there when
489
+ the work is handed on.
490
+
491
+ ## What you report
492
+
493
+ - Red: the command, the failing output, why that failure was expected.
494
+ - Green: the command, the passing output, and any other failure the run showed.
495
+ - Who wrote the tests, whether they were changed after they were written, and
496
+ that every file the author handed back is in the change.
497
+ - For a test of behaviour that already existed: which baseline its red came
498
+ from, or that there was none and it stands as a description rather than a
499
+ proven guard.
500
+ - The mutation check in three lines: what you broke, what survived, what you
501
+ did about it, or the line that says you skipped it and why.
502
+ - What you did not check, and the command that would check it.
503
+
504
+ ## Away from code
505
+
506
+ The same three moves work on documentation, configuration, prompts and data,
507
+ where no tool will help: state the check before you make the change, see the
508
+ check fail first, and at the end break the artefact on purpose to find out
509
+ whether the check notices.
510
+
511
+ They apply where there is something that can be wrong: a configuration that
512
+ decides who gets what, a prompt whose branches behave differently, a
513
+ transformation over data. The skip rule is the same as for code, and prose
514
+ written for people earns no test and no mutant.
515
+
516
+ Sources and licences: [NOTICE.md](NOTICE.md).
@@ -0,0 +1,80 @@
1
+ # Mutation tools, by stack
2
+
3
+ For the occasional audit described in the skill. No project needs one of these,
4
+ and the hand check is the everyday method; this page exists so that a run, when
5
+ there is a reason for one, is scoped to what changed instead of the whole repo.
6
+
7
+ How far to trust each line. The StrykerJS flags below (`--incremental`,
8
+ `--force` and `-m`/`--mutate`) were checked against `stryker run --help` on
9
+ version 10.0.0 on 2026-09-20. Everything else on this page, the line-range form
10
+ included, was read from each tool's own documentation on 2026-09-19 and has not
11
+ been run. Flags change: check the version the project actually has before
12
+ trusting a line here, and prefer a command the project already ships over one
13
+ from this page.
14
+
15
+ **JavaScript, TypeScript: StrykerJS** (Apache-2.0)
16
+
17
+ ```sh
18
+ npx stryker run --incremental
19
+ npx stryker run --mutate src/x.ts # one file
20
+ npx stryker run --mutate src/x.ts:10-40 # one range
21
+ ```
22
+
23
+ **C#: Stryker.NET** (Apache-2.0): `dotnet stryker --since:main`
24
+
25
+ **Rust: cargo-mutants** (MIT)
26
+
27
+ ```sh
28
+ git diff origin/main.. > git.diff && cargo mutants --in-diff git.diff
29
+ ```
30
+
31
+ **Go: gremlins** (Apache-2.0): `gremlins unleash --diff origin/main`, with the
32
+ full history fetched.
33
+
34
+ **Python: mutmut 3** (BSD-3-Clause): `mutmut run "module.function*"`;
35
+ incremental by default; `mutmut browse` to work through the survivors.
36
+
37
+ **Python: Cosmic Ray** (MIT): `cosmic-ray init`, then `cr-filter-git`, then
38
+ `cosmic-ray exec`, then `cr-report`.
39
+
40
+ **Java: PIT** (Apache-2.0):
41
+ `mvn -DwithHistory test-compile org.pitest:pitest-maven:mutationCoverage`, with
42
+ `targetClasses` and `targetTests` globs.
43
+
44
+ **Anything else**, shell, SQL, configuration, prompts, a language with no such
45
+ tool: the hand check in the skill is the method, and it is the same method,
46
+ including the rule about when to skip it. Something with no logic in it does
47
+ not get broken on purpose just because no tool would have run anyway.
48
+
49
+ ## Four things that older advice gets wrong
50
+
51
+ - PIT's `scmMutationCoverage` goal was deprecated in 1.17.1 and removed in
52
+ 1.18.0. Diff scoping in PIT now comes from a commercial plugin; with the free
53
+ one, scope with the globs above and `-DwithHistory`.
54
+ - mutmut 3 has no `--paths-to-mutate`. Scope it in the config (`only_mutate`,
55
+ `do_not_mutate`, `mutate_only_covered_lines = true`) or with a positional
56
+ glob.
57
+ - `cargo mutants --in-diff` takes a diff file, not a git ref: write the diff
58
+ out first.
59
+ - StrykerJS has no git-diff flag; feed it the changed files yourself.
60
+
61
+ ## Three caveats that apply to all of them
62
+
63
+ - A diff-scoped run matches the diff against the code under test, not the test
64
+ code. A change that only weakens a test produces no mutants at all and comes
65
+ back clean. When test files change, mutate the code those tests cover.
66
+ - A run that picks its targets from a committed diff sees only committed work,
67
+ so on a dirty tree it reports on the wrong code: commit first, or name the
68
+ files yourself instead. Naming them is the better answer when the point is to
69
+ check work that is not committed yet. Do not make someone stash the code
70
+ they asked about.
71
+ - Whatever selects the targets, a run that dies in the middle can leave a
72
+ mutant in the working copy. Look at the tree afterwards.
73
+
74
+ ## Running one and reading it
75
+
76
+ Prove the setup with one small scoped run before spending a long one. Capture
77
+ the output once and read it from the copy rather than re-running to re-read it.
78
+ Set no failing threshold before a measured baseline exists. Paste what the tool
79
+ printed: a mutation result reported from memory has been wrong. A compile error
80
+ is not a kill, and a timeout is inconclusive rather than evidence.
@@ -0,0 +1,42 @@
1
+ <!-- The licences of the sources this skill was consolidated from. They travel
2
+ with the skill directory because the licences require it of anyone who
3
+ distributes the material. NOTICE.md says what was taken from each source;
4
+ this file carries the permission notices themselves. -->
5
+
6
+ # Licences of the sources
7
+
8
+ `obk-teach` consolidates material from the sources named in
9
+ [NOTICE.md](NOTICE.md). Their licences are reproduced below in full.
10
+
11
+ ## MIT
12
+
13
+ The MIT permission notice below applies to the work of each of these copyright
14
+ holders, and is reproduced with each of their notices. The path after each one
15
+ is the licence file it was read from, in the research pack's source clones.
16
+
17
+ - **Cursor `plugins`, the pstack plugin**, Copyright (c) 2026 Lauren Tan
18
+ (`cursor_plugins/pstack/LICENSE`)
19
+
20
+ ```
21
+ MIT License
22
+
23
+ Copyright (c) 2026 Lauren Tan
24
+
25
+ Permission is hereby granted, free of charge, to any person obtaining a copy
26
+ of this software and associated documentation files (the "Software"), to deal
27
+ in the Software without restriction, including without limitation the rights
28
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
29
+ copies of the Software, and to permit persons to whom the Software is
30
+ furnished to do so, subject to the following conditions:
31
+
32
+ The above copyright notice and this permission notice shall be included in all
33
+ copies or substantial portions of the Software.
34
+
35
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
36
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
37
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
38
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
39
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
40
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
41
+ SOFTWARE.
42
+ ```