@complexthings/superpowers-agent 9.2.1 → 10.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (193) hide show
  1. package/.agents/skills/claude-handoff/SKILL.md +18 -0
  2. package/.agents/skills/code-review/SKILL.md +89 -0
  3. package/.agents/skills/{improve-codebase-architecture → codebase-design}/DEEPENING.md +1 -1
  4. package/.agents/skills/{improve-codebase-architecture/INTERFACE-DESIGN.md → codebase-design/DESIGN-IT-TWICE.md} +3 -3
  5. package/.agents/skills/codebase-design/SKILL.md +114 -0
  6. package/.agents/skills/design-an-interface/SKILL.md +94 -0
  7. package/.agents/skills/{diagnose → diagnosing-bugs}/SKILL.md +29 -12
  8. package/.agents/skills/{grill-with-docs → domain-modeling}/CONTEXT-FORMAT.md +1 -4
  9. package/.agents/skills/domain-modeling/SKILL.md +74 -0
  10. package/.agents/skills/fable-mode/SKILL.md +95 -0
  11. package/.agents/skills/git-guardrails-claude-code/SKILL.md +95 -0
  12. package/.agents/skills/git-guardrails-claude-code/scripts/block-dangerous-git.sh +25 -0
  13. package/.agents/skills/grill-me/SKILL.md +7 -0
  14. package/.agents/skills/grill-with-docs/SKILL.md +3 -86
  15. package/.agents/skills/grilling/SKILL.md +14 -0
  16. package/.agents/skills/handoff/SKILL.md +2 -1
  17. package/.agents/skills/i-have-adhd/SKILL.md +120 -0
  18. package/.agents/skills/implement/SKILL.md +11 -0
  19. package/.agents/skills/improve-codebase-architecture/HTML-REPORT.md +3 -3
  20. package/.agents/skills/improve-codebase-architecture/SKILL.md +13 -28
  21. package/.agents/skills/loop-me/SKILL.md +32 -0
  22. package/.agents/skills/prototype/SKILL.md +1 -1
  23. package/.agents/skills/qa/SKILL.md +130 -0
  24. package/.agents/skills/request-refactor-plan/SKILL.md +68 -0
  25. package/.agents/skills/research/SKILL.md +12 -0
  26. package/.agents/skills/resolving-merge-conflicts/SKILL.md +14 -0
  27. package/.agents/skills/scaffold-exercises/SKILL.md +106 -0
  28. package/.agents/skills/setup-matt-pocock-skills/SKILL.md +11 -9
  29. package/.agents/skills/setup-matt-pocock-skills/domain.md +2 -2
  30. package/.agents/skills/setup-matt-pocock-skills/issue-tracker-github.md +23 -0
  31. package/.agents/skills/setup-matt-pocock-skills/issue-tracker-gitlab.md +23 -0
  32. package/.agents/skills/setup-matt-pocock-skills/issue-tracker-local.md +11 -0
  33. package/.agents/skills/skill-creator/LICENSE.txt +202 -0
  34. package/.agents/skills/skill-creator/SKILL.md +485 -0
  35. package/.agents/skills/skill-creator/agents/analyzer.md +274 -0
  36. package/.agents/skills/skill-creator/agents/comparator.md +202 -0
  37. package/.agents/skills/skill-creator/agents/grader.md +223 -0
  38. package/.agents/skills/skill-creator/assets/eval_review.html +146 -0
  39. package/.agents/skills/skill-creator/eval-viewer/generate_review.py +471 -0
  40. package/.agents/skills/skill-creator/eval-viewer/viewer.html +1325 -0
  41. package/.agents/skills/skill-creator/references/schemas.md +430 -0
  42. package/.agents/skills/skill-creator/scripts/__init__.py +0 -0
  43. package/.agents/skills/skill-creator/scripts/__pycache__/__init__.cpython-314.pyc +0 -0
  44. package/.agents/skills/skill-creator/scripts/__pycache__/run_eval.cpython-314.pyc +0 -0
  45. package/.agents/skills/skill-creator/scripts/__pycache__/utils.cpython-314.pyc +0 -0
  46. package/.agents/skills/skill-creator/scripts/aggregate_benchmark.py +401 -0
  47. package/.agents/skills/skill-creator/scripts/generate_report.py +326 -0
  48. package/.agents/skills/skill-creator/scripts/improve_description.py +247 -0
  49. package/.agents/skills/skill-creator/scripts/package_skill.py +136 -0
  50. package/.agents/skills/skill-creator/scripts/quick_validate.py +103 -0
  51. package/.agents/skills/skill-creator/scripts/run_eval.py +310 -0
  52. package/.agents/skills/skill-creator/scripts/run_loop.py +328 -0
  53. package/.agents/skills/skill-creator/scripts/utils.py +47 -0
  54. package/.agents/skills/tdd/SKILL.md +17 -90
  55. package/.agents/skills/tdd/tests.md +16 -0
  56. package/.agents/skills/teach/GLOSSARY-FORMAT.md +35 -0
  57. package/.agents/skills/teach/LEARNING-RECORD-FORMAT.md +46 -0
  58. package/.agents/skills/teach/MISSION-FORMAT.md +31 -0
  59. package/.agents/skills/teach/RESOURCES-FORMAT.md +32 -0
  60. package/.agents/skills/teach/SKILL.md +140 -0
  61. package/.agents/skills/{to-prd → to-spec}/SKILL.md +11 -12
  62. package/.agents/skills/to-tickets/SKILL.md +114 -0
  63. package/.agents/skills/triage/AGENT-BRIEF.md +40 -1
  64. package/.agents/skills/triage/OUT-OF-SCOPE.md +5 -1
  65. package/.agents/skills/triage/SKILL.md +20 -11
  66. package/.agents/skills/wayfinder/SKILL.md +127 -0
  67. package/.agents/skills/writing-great-skills/GLOSSARY.md +201 -0
  68. package/.agents/skills/writing-great-skills/SKILL.md +83 -0
  69. package/.agents/superpowers-agent +103 -222
  70. package/.agents/superpowers-bootstrap.md +3 -3
  71. package/.agents/templates/AGENTS.md.template +11 -34
  72. package/.agents/templates/SUPERPOWERS.md.template +4 -4
  73. package/.github/copilot-instructions.md +23 -99
  74. package/.github/hooks/rtk-rewrite.json +22 -0
  75. package/AGENTS.md +7 -6
  76. package/README.md +53 -174
  77. package/package.json +2 -2
  78. package/skills/collaboration/brainstorming/SKILL.md +39 -139
  79. package/skills/collaboration/brainstorming/skill.json +2 -2
  80. package/skills/collaboration/leveraging-cli-tools/SKILL.md +70 -71
  81. package/skills/collaboration/leveraging-cli-tools/references/copilot-instructions.md +30 -0
  82. package/skills/collaboration/leveraging-cli-tools/scripts/setup-ponytail.sh +185 -0
  83. package/skills/collaboration/leveraging-cli-tools/scripts/setup-rtk.sh +217 -0
  84. package/skills/collaboration/leveraging-cli-tools/skill.json +1 -1
  85. package/skills/meta/create-skill-json/SKILL.md +4 -4
  86. package/skills/meta/create-skill-json/skill.json +1 -1
  87. package/skills/meta/create-skill-json/test-scenarios.md +1 -1
  88. package/skills/setup-skills/SKILL.md +18 -11
  89. package/skills/setup-skills/skill.json +8 -0
  90. package/.agents/skills/caveman/SKILL.md +0 -49
  91. package/.agents/skills/improve-codebase-architecture/LANGUAGE.md +0 -53
  92. package/.agents/skills/karpathy-guidelines/SKILL.md +0 -75
  93. package/.agents/skills/review/SKILL.md +0 -78
  94. package/.agents/skills/tdd/deep-modules.md +0 -33
  95. package/.agents/skills/tdd/interface-design.md +0 -31
  96. package/.agents/skills/tdd/refactoring.md +0 -10
  97. package/.agents/skills/to-issues/SKILL.md +0 -83
  98. package/.agents/skills/zoom-out/SKILL.md +0 -7
  99. package/skills/architecture/ABOUT.md +0 -20
  100. package/skills/architecture/preserving-productive-tensions/SKILL.md +0 -146
  101. package/skills/architecture/preserving-productive-tensions/skill.json +0 -9
  102. package/skills/collaboration/brainstorming/spec-document-reviewer-prompt.md +0 -50
  103. package/skills/collaboration/brainstorming/visual-companion.md +0 -277
  104. package/skills/collaboration/dispatching-parallel-agents/SKILL.md +0 -174
  105. package/skills/collaboration/dispatching-parallel-agents/skill.json +0 -9
  106. package/skills/collaboration/executing-plans/SKILL.md +0 -130
  107. package/skills/collaboration/executing-plans/skill.json +0 -9
  108. package/skills/collaboration/finishing-a-development-branch/SKILL.md +0 -261
  109. package/skills/collaboration/finishing-a-development-branch/skill.json +0 -9
  110. package/skills/collaboration/leveraging-cli-tools/scripts/slim.py +0 -167
  111. package/skills/collaboration/receiving-code-review/SKILL.md +0 -233
  112. package/skills/collaboration/receiving-code-review/skill.json +0 -9
  113. package/skills/collaboration/requesting-code-review/SKILL.md +0 -110
  114. package/skills/collaboration/requesting-code-review/code-reviewer.md +0 -146
  115. package/skills/collaboration/requesting-code-review/skill.json +0 -12
  116. package/skills/collaboration/subagent-driven-development/SKILL.md +0 -255
  117. package/skills/collaboration/subagent-driven-development/code-quality-reviewer-prompt.md +0 -26
  118. package/skills/collaboration/subagent-driven-development/implementer-prompt.md +0 -113
  119. package/skills/collaboration/subagent-driven-development/skill.json +0 -15
  120. package/skills/collaboration/subagent-driven-development/spec-reviewer-prompt.md +0 -61
  121. package/skills/collaboration/using-git-worktrees/SKILL.md +0 -366
  122. package/skills/collaboration/using-git-worktrees/skill.json +0 -9
  123. package/skills/collaboration/writing-plans/SKILL.md +0 -121
  124. package/skills/collaboration/writing-plans/plan-document-reviewer-prompt.md +0 -52
  125. package/skills/collaboration/writing-plans/skill.json +0 -9
  126. package/skills/debugging/defense-in-depth/SKILL.md +0 -380
  127. package/skills/debugging/defense-in-depth/skill.json +0 -9
  128. package/skills/debugging/root-cause-tracing/SKILL.md +0 -361
  129. package/skills/debugging/root-cause-tracing/find-polluter.sh +0 -63
  130. package/skills/debugging/root-cause-tracing/skill.json +0 -12
  131. package/skills/debugging/systematic-debugging/SKILL.md +0 -299
  132. package/skills/debugging/systematic-debugging/condition-based-waiting-example.ts +0 -158
  133. package/skills/debugging/systematic-debugging/condition-based-waiting.md +0 -115
  134. package/skills/debugging/systematic-debugging/defense-in-depth.md +0 -122
  135. package/skills/debugging/systematic-debugging/find-polluter.sh +0 -63
  136. package/skills/debugging/systematic-debugging/root-cause-tracing.md +0 -169
  137. package/skills/debugging/systematic-debugging/skill.json +0 -9
  138. package/skills/debugging/systematic-debugging/test-academic.md +0 -14
  139. package/skills/debugging/systematic-debugging/test-pressure-1.md +0 -58
  140. package/skills/debugging/systematic-debugging/test-pressure-2.md +0 -68
  141. package/skills/debugging/systematic-debugging/test-pressure-3.md +0 -69
  142. package/skills/debugging/verification-before-completion/SKILL.md +0 -143
  143. package/skills/debugging/verification-before-completion/skill.json +0 -9
  144. package/skills/finding-skills/SKILL.md +0 -101
  145. package/skills/finding-skills/skill.json +0 -8
  146. package/skills/meta/create-agents-md/SKILL.md +0 -182
  147. package/skills/meta/create-agents-md/skill.json +0 -9
  148. package/skills/meta/creating-prompts/SKILL.md +0 -349
  149. package/skills/meta/creating-prompts/examples/do-example.md +0 -65
  150. package/skills/meta/creating-prompts/examples/plan-example.md +0 -75
  151. package/skills/meta/creating-prompts/examples/refine-example.md +0 -65
  152. package/skills/meta/creating-prompts/examples/research-example.md +0 -63
  153. package/skills/meta/creating-prompts/scripts/get-next-number.sh +0 -27
  154. package/skills/meta/creating-prompts/skill.json +0 -20
  155. package/skills/meta/creating-prompts/templates/do-template.md +0 -59
  156. package/skills/meta/creating-prompts/templates/plan-template.md +0 -58
  157. package/skills/meta/creating-prompts/templates/refine-template.md +0 -54
  158. package/skills/meta/creating-prompts/templates/research-template.md +0 -56
  159. package/skills/meta/using-superpowers/SKILL.md +0 -108
  160. package/skills/meta/using-superpowers/skill.json +0 -5
  161. package/skills/meta/writing-prompts/SKILL.md +0 -122
  162. package/skills/meta/writing-prompts/references/platforms.md +0 -114
  163. package/skills/meta/writing-prompts/skill.json +0 -9
  164. package/skills/problem-solving/ABOUT.md +0 -40
  165. package/skills/problem-solving/collision-zone-thinking/SKILL.md +0 -188
  166. package/skills/problem-solving/collision-zone-thinking/references/historical-examples.md +0 -393
  167. package/skills/problem-solving/collision-zone-thinking/skill.json +0 -9
  168. package/skills/problem-solving/inversion-exercise/SKILL.md +0 -174
  169. package/skills/problem-solving/inversion-exercise/skill.json +0 -9
  170. package/skills/problem-solving/meta-pattern-recognition/SKILL.md +0 -116
  171. package/skills/problem-solving/meta-pattern-recognition/skill.json +0 -9
  172. package/skills/problem-solving/scale-game/SKILL.md +0 -222
  173. package/skills/problem-solving/scale-game/skill.json +0 -9
  174. package/skills/problem-solving/simplification-cascades/SKILL.md +0 -113
  175. package/skills/problem-solving/simplification-cascades/skill.json +0 -9
  176. package/skills/problem-solving/when-stuck/SKILL.md +0 -69
  177. package/skills/problem-solving/when-stuck/skill.json +0 -9
  178. package/skills/research/ABOUT.md +0 -20
  179. package/skills/research/tracing-knowledge-lineages/SKILL.md +0 -241
  180. package/skills/research/tracing-knowledge-lineages/skill.json +0 -9
  181. package/skills/testing/condition-based-waiting/SKILL.md +0 -359
  182. package/skills/testing/condition-based-waiting/example.ts +0 -158
  183. package/skills/testing/condition-based-waiting/skill.json +0 -12
  184. package/skills/testing/test-driven-development/SKILL.md +0 -434
  185. package/skills/testing/test-driven-development/skill.json +0 -9
  186. package/skills/testing/testing-anti-patterns/SKILL.md +0 -298
  187. package/skills/testing/testing-anti-patterns/skill.json +0 -9
  188. package/skills/testing/verification-before-completion/SKILL.md +0 -246
  189. package/skills/testing/verification-before-completion/skill.json +0 -10
  190. package/skills/using-a-skill/SKILL.md +0 -101
  191. package/skills/using-a-skill/skill.json +0 -8
  192. /package/.agents/skills/{diagnose → diagnosing-bugs}/scripts/hitl-loop.template.sh +0 -0
  193. /package/.agents/skills/{grill-with-docs → domain-modeling}/ADR-FORMAT.md +0 -0
@@ -0,0 +1,328 @@
1
+ #!/usr/bin/env python3
2
+ """Run the eval + improve loop until all pass or max iterations reached.
3
+
4
+ Combines run_eval.py and improve_description.py in a loop, tracking history
5
+ and returning the best description found. Supports train/test split to prevent
6
+ overfitting.
7
+ """
8
+
9
+ import argparse
10
+ import json
11
+ import random
12
+ import sys
13
+ import tempfile
14
+ import time
15
+ import webbrowser
16
+ from pathlib import Path
17
+
18
+ from scripts.generate_report import generate_html
19
+ from scripts.improve_description import improve_description
20
+ from scripts.run_eval import find_project_root, run_eval
21
+ from scripts.utils import parse_skill_md
22
+
23
+
24
+ def split_eval_set(eval_set: list[dict], holdout: float, seed: int = 42) -> tuple[list[dict], list[dict]]:
25
+ """Split eval set into train and test sets, stratified by should_trigger."""
26
+ random.seed(seed)
27
+
28
+ # Separate by should_trigger
29
+ trigger = [e for e in eval_set if e["should_trigger"]]
30
+ no_trigger = [e for e in eval_set if not e["should_trigger"]]
31
+
32
+ # Shuffle each group
33
+ random.shuffle(trigger)
34
+ random.shuffle(no_trigger)
35
+
36
+ # Calculate split points
37
+ n_trigger_test = max(1, int(len(trigger) * holdout))
38
+ n_no_trigger_test = max(1, int(len(no_trigger) * holdout))
39
+
40
+ # Split
41
+ test_set = trigger[:n_trigger_test] + no_trigger[:n_no_trigger_test]
42
+ train_set = trigger[n_trigger_test:] + no_trigger[n_no_trigger_test:]
43
+
44
+ return train_set, test_set
45
+
46
+
47
+ def run_loop(
48
+ eval_set: list[dict],
49
+ skill_path: Path,
50
+ description_override: str | None,
51
+ num_workers: int,
52
+ timeout: int,
53
+ max_iterations: int,
54
+ runs_per_query: int,
55
+ trigger_threshold: float,
56
+ holdout: float,
57
+ model: str,
58
+ verbose: bool,
59
+ live_report_path: Path | None = None,
60
+ log_dir: Path | None = None,
61
+ ) -> dict:
62
+ """Run the eval + improvement loop."""
63
+ project_root = find_project_root()
64
+ name, original_description, content = parse_skill_md(skill_path)
65
+ current_description = description_override or original_description
66
+
67
+ # Split into train/test if holdout > 0
68
+ if holdout > 0:
69
+ train_set, test_set = split_eval_set(eval_set, holdout)
70
+ if verbose:
71
+ print(f"Split: {len(train_set)} train, {len(test_set)} test (holdout={holdout})", file=sys.stderr)
72
+ else:
73
+ train_set = eval_set
74
+ test_set = []
75
+
76
+ history = []
77
+ exit_reason = "unknown"
78
+
79
+ for iteration in range(1, max_iterations + 1):
80
+ if verbose:
81
+ print(f"\n{'='*60}", file=sys.stderr)
82
+ print(f"Iteration {iteration}/{max_iterations}", file=sys.stderr)
83
+ print(f"Description: {current_description}", file=sys.stderr)
84
+ print(f"{'='*60}", file=sys.stderr)
85
+
86
+ # Evaluate train + test together in one batch for parallelism
87
+ all_queries = train_set + test_set
88
+ t0 = time.time()
89
+ all_results = run_eval(
90
+ eval_set=all_queries,
91
+ skill_name=name,
92
+ description=current_description,
93
+ num_workers=num_workers,
94
+ timeout=timeout,
95
+ project_root=project_root,
96
+ runs_per_query=runs_per_query,
97
+ trigger_threshold=trigger_threshold,
98
+ model=model,
99
+ )
100
+ eval_elapsed = time.time() - t0
101
+
102
+ # Split results back into train/test by matching queries
103
+ train_queries_set = {q["query"] for q in train_set}
104
+ train_result_list = [r for r in all_results["results"] if r["query"] in train_queries_set]
105
+ test_result_list = [r for r in all_results["results"] if r["query"] not in train_queries_set]
106
+
107
+ train_passed = sum(1 for r in train_result_list if r["pass"])
108
+ train_total = len(train_result_list)
109
+ train_summary = {"passed": train_passed, "failed": train_total - train_passed, "total": train_total}
110
+ train_results = {"results": train_result_list, "summary": train_summary}
111
+
112
+ if test_set:
113
+ test_passed = sum(1 for r in test_result_list if r["pass"])
114
+ test_total = len(test_result_list)
115
+ test_summary = {"passed": test_passed, "failed": test_total - test_passed, "total": test_total}
116
+ test_results = {"results": test_result_list, "summary": test_summary}
117
+ else:
118
+ test_results = None
119
+ test_summary = None
120
+
121
+ history.append({
122
+ "iteration": iteration,
123
+ "description": current_description,
124
+ "train_passed": train_summary["passed"],
125
+ "train_failed": train_summary["failed"],
126
+ "train_total": train_summary["total"],
127
+ "train_results": train_results["results"],
128
+ "test_passed": test_summary["passed"] if test_summary else None,
129
+ "test_failed": test_summary["failed"] if test_summary else None,
130
+ "test_total": test_summary["total"] if test_summary else None,
131
+ "test_results": test_results["results"] if test_results else None,
132
+ # For backward compat with report generator
133
+ "passed": train_summary["passed"],
134
+ "failed": train_summary["failed"],
135
+ "total": train_summary["total"],
136
+ "results": train_results["results"],
137
+ })
138
+
139
+ # Write live report if path provided
140
+ if live_report_path:
141
+ partial_output = {
142
+ "original_description": original_description,
143
+ "best_description": current_description,
144
+ "best_score": "in progress",
145
+ "iterations_run": len(history),
146
+ "holdout": holdout,
147
+ "train_size": len(train_set),
148
+ "test_size": len(test_set),
149
+ "history": history,
150
+ }
151
+ live_report_path.write_text(generate_html(partial_output, auto_refresh=True, skill_name=name))
152
+
153
+ if verbose:
154
+ def print_eval_stats(label, results, elapsed):
155
+ pos = [r for r in results if r["should_trigger"]]
156
+ neg = [r for r in results if not r["should_trigger"]]
157
+ tp = sum(r["triggers"] for r in pos)
158
+ pos_runs = sum(r["runs"] for r in pos)
159
+ fn = pos_runs - tp
160
+ fp = sum(r["triggers"] for r in neg)
161
+ neg_runs = sum(r["runs"] for r in neg)
162
+ tn = neg_runs - fp
163
+ total = tp + tn + fp + fn
164
+ precision = tp / (tp + fp) if (tp + fp) > 0 else 1.0
165
+ recall = tp / (tp + fn) if (tp + fn) > 0 else 1.0
166
+ accuracy = (tp + tn) / total if total > 0 else 0.0
167
+ print(f"{label}: {tp+tn}/{total} correct, precision={precision:.0%} recall={recall:.0%} accuracy={accuracy:.0%} ({elapsed:.1f}s)", file=sys.stderr)
168
+ for r in results:
169
+ status = "PASS" if r["pass"] else "FAIL"
170
+ rate_str = f"{r['triggers']}/{r['runs']}"
171
+ print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:60]}", file=sys.stderr)
172
+
173
+ print_eval_stats("Train", train_results["results"], eval_elapsed)
174
+ if test_summary:
175
+ print_eval_stats("Test ", test_results["results"], 0)
176
+
177
+ if train_summary["failed"] == 0:
178
+ exit_reason = f"all_passed (iteration {iteration})"
179
+ if verbose:
180
+ print(f"\nAll train queries passed on iteration {iteration}!", file=sys.stderr)
181
+ break
182
+
183
+ if iteration == max_iterations:
184
+ exit_reason = f"max_iterations ({max_iterations})"
185
+ if verbose:
186
+ print(f"\nMax iterations reached ({max_iterations}).", file=sys.stderr)
187
+ break
188
+
189
+ # Improve the description based on train results
190
+ if verbose:
191
+ print(f"\nImproving description...", file=sys.stderr)
192
+
193
+ t0 = time.time()
194
+ # Strip test scores from history so improvement model can't see them
195
+ blinded_history = [
196
+ {k: v for k, v in h.items() if not k.startswith("test_")}
197
+ for h in history
198
+ ]
199
+ new_description = improve_description(
200
+ skill_name=name,
201
+ skill_content=content,
202
+ current_description=current_description,
203
+ eval_results=train_results,
204
+ history=blinded_history,
205
+ model=model,
206
+ log_dir=log_dir,
207
+ iteration=iteration,
208
+ )
209
+ improve_elapsed = time.time() - t0
210
+
211
+ if verbose:
212
+ print(f"Proposed ({improve_elapsed:.1f}s): {new_description}", file=sys.stderr)
213
+
214
+ current_description = new_description
215
+
216
+ # Find the best iteration by TEST score (or train if no test set)
217
+ if test_set:
218
+ best = max(history, key=lambda h: h["test_passed"] or 0)
219
+ best_score = f"{best['test_passed']}/{best['test_total']}"
220
+ else:
221
+ best = max(history, key=lambda h: h["train_passed"])
222
+ best_score = f"{best['train_passed']}/{best['train_total']}"
223
+
224
+ if verbose:
225
+ print(f"\nExit reason: {exit_reason}", file=sys.stderr)
226
+ print(f"Best score: {best_score} (iteration {best['iteration']})", file=sys.stderr)
227
+
228
+ return {
229
+ "exit_reason": exit_reason,
230
+ "original_description": original_description,
231
+ "best_description": best["description"],
232
+ "best_score": best_score,
233
+ "best_train_score": f"{best['train_passed']}/{best['train_total']}",
234
+ "best_test_score": f"{best['test_passed']}/{best['test_total']}" if test_set else None,
235
+ "final_description": current_description,
236
+ "iterations_run": len(history),
237
+ "holdout": holdout,
238
+ "train_size": len(train_set),
239
+ "test_size": len(test_set),
240
+ "history": history,
241
+ }
242
+
243
+
244
+ def main():
245
+ parser = argparse.ArgumentParser(description="Run eval + improve loop")
246
+ parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file")
247
+ parser.add_argument("--skill-path", required=True, help="Path to skill directory")
248
+ parser.add_argument("--description", default=None, help="Override starting description")
249
+ parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers")
250
+ parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds")
251
+ parser.add_argument("--max-iterations", type=int, default=5, help="Max improvement iterations")
252
+ parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query")
253
+ parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold")
254
+ parser.add_argument("--holdout", type=float, default=0.4, help="Fraction of eval set to hold out for testing (0 to disable)")
255
+ parser.add_argument("--model", required=True, help="Model for improvement")
256
+ parser.add_argument("--verbose", action="store_true", help="Print progress to stderr")
257
+ parser.add_argument("--report", default="auto", help="Generate HTML report at this path (default: 'auto' for temp file, 'none' to disable)")
258
+ parser.add_argument("--results-dir", default=None, help="Save all outputs (results.json, report.html, log.txt) to a timestamped subdirectory here")
259
+ args = parser.parse_args()
260
+
261
+ eval_set = json.loads(Path(args.eval_set).read_text())
262
+ skill_path = Path(args.skill_path)
263
+
264
+ if not (skill_path / "SKILL.md").exists():
265
+ print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr)
266
+ sys.exit(1)
267
+
268
+ name, _, _ = parse_skill_md(skill_path)
269
+
270
+ # Set up live report path
271
+ if args.report != "none":
272
+ if args.report == "auto":
273
+ timestamp = time.strftime("%Y%m%d_%H%M%S")
274
+ live_report_path = Path(tempfile.gettempdir()) / f"skill_description_report_{skill_path.name}_{timestamp}.html"
275
+ else:
276
+ live_report_path = Path(args.report)
277
+ # Open the report immediately so the user can watch
278
+ live_report_path.write_text("<html><body><h1>Starting optimization loop...</h1><meta http-equiv='refresh' content='5'></body></html>")
279
+ webbrowser.open(str(live_report_path))
280
+ else:
281
+ live_report_path = None
282
+
283
+ # Determine output directory (create before run_loop so logs can be written)
284
+ if args.results_dir:
285
+ timestamp = time.strftime("%Y-%m-%d_%H%M%S")
286
+ results_dir = Path(args.results_dir) / timestamp
287
+ results_dir.mkdir(parents=True, exist_ok=True)
288
+ else:
289
+ results_dir = None
290
+
291
+ log_dir = results_dir / "logs" if results_dir else None
292
+
293
+ output = run_loop(
294
+ eval_set=eval_set,
295
+ skill_path=skill_path,
296
+ description_override=args.description,
297
+ num_workers=args.num_workers,
298
+ timeout=args.timeout,
299
+ max_iterations=args.max_iterations,
300
+ runs_per_query=args.runs_per_query,
301
+ trigger_threshold=args.trigger_threshold,
302
+ holdout=args.holdout,
303
+ model=args.model,
304
+ verbose=args.verbose,
305
+ live_report_path=live_report_path,
306
+ log_dir=log_dir,
307
+ )
308
+
309
+ # Save JSON output
310
+ json_output = json.dumps(output, indent=2)
311
+ print(json_output)
312
+ if results_dir:
313
+ (results_dir / "results.json").write_text(json_output)
314
+
315
+ # Write final HTML report (without auto-refresh)
316
+ if live_report_path:
317
+ live_report_path.write_text(generate_html(output, auto_refresh=False, skill_name=name))
318
+ print(f"\nReport: {live_report_path}", file=sys.stderr)
319
+
320
+ if results_dir and live_report_path:
321
+ (results_dir / "report.html").write_text(generate_html(output, auto_refresh=False, skill_name=name))
322
+
323
+ if results_dir:
324
+ print(f"Results saved to: {results_dir}", file=sys.stderr)
325
+
326
+
327
+ if __name__ == "__main__":
328
+ main()
@@ -0,0 +1,47 @@
1
+ """Shared utilities for skill-creator scripts."""
2
+
3
+ from pathlib import Path
4
+
5
+
6
+
7
+ def parse_skill_md(skill_path: Path) -> tuple[str, str, str]:
8
+ """Parse a SKILL.md file, returning (name, description, full_content)."""
9
+ content = (skill_path / "SKILL.md").read_text()
10
+ lines = content.split("\n")
11
+
12
+ if lines[0].strip() != "---":
13
+ raise ValueError("SKILL.md missing frontmatter (no opening ---)")
14
+
15
+ end_idx = None
16
+ for i, line in enumerate(lines[1:], start=1):
17
+ if line.strip() == "---":
18
+ end_idx = i
19
+ break
20
+
21
+ if end_idx is None:
22
+ raise ValueError("SKILL.md missing frontmatter (no closing ---)")
23
+
24
+ name = ""
25
+ description = ""
26
+ frontmatter_lines = lines[1:end_idx]
27
+ i = 0
28
+ while i < len(frontmatter_lines):
29
+ line = frontmatter_lines[i]
30
+ if line.startswith("name:"):
31
+ name = line[len("name:"):].strip().strip('"').strip("'")
32
+ elif line.startswith("description:"):
33
+ value = line[len("description:"):].strip()
34
+ # Handle YAML multiline indicators (>, |, >-, |-)
35
+ if value in (">", "|", ">-", "|-"):
36
+ continuation_lines: list[str] = []
37
+ i += 1
38
+ while i < len(frontmatter_lines) and (frontmatter_lines[i].startswith(" ") or frontmatter_lines[i].startswith("\t")):
39
+ continuation_lines.append(frontmatter_lines[i].strip())
40
+ i += 1
41
+ description = " ".join(continuation_lines)
42
+ continue
43
+ else:
44
+ description = value.strip('"').strip("'")
45
+ i += 1
46
+
47
+ return name, description, content
@@ -1,109 +1,36 @@
1
1
  ---
2
2
  name: tdd
3
- description: Test-driven development with red-green-refactor loop. Use when user wants to build features or fix bugs using TDD, mentions "red-green-refactor", wants integration tests, or asks for test-first development.
3
+ description: Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
4
4
  ---
5
5
 
6
6
  # Test-Driven Development
7
7
 
8
- ## Philosophy
8
+ TDD is the red → green loop. This skill is the reference that makes that loop produce tests worth keeping: what a good test is, where tests go, the anti-patterns, and the rules of the loop. Every section applies on every cycle — consult them before and during the loop, not after.
9
9
 
10
- **Core principle**: Tests should verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't.
10
+ When exploring the codebase, read `CONTEXT.md` (if it exists) so test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
11
11
 
12
- **Good tests** are integration-style: they exercise real code paths through public APIs. They describe _what_ the system does, not _how_ it does it. A good test reads like a specification - "user can checkout with valid cart" tells you exactly what capability exists. These tests survive refactors because they don't care about internal structure.
12
+ ## What a good test is
13
13
 
14
- **Bad tests** are coupled to implementation. They mock internal collaborators, test private methods, or verify through external means (like querying a database directly instead of using the interface). The warning sign: your test breaks when you refactor, but behavior hasn't changed. If you rename an internal function and tests fail, those tests were testing implementation, not behavior.
14
+ Tests verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't. A good test reads like a specification "user can checkout with valid cart" tells you exactly what capability exists and survives refactors because it doesn't care about internal structure.
15
15
 
16
16
  See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
17
17
 
18
- ## Anti-Pattern: Horizontal Slices
18
+ ## Seams where tests go
19
19
 
20
- **DO NOT write all tests first, then all implementation.** This is "horizontal slicing" - treating RED as "write all tests" and GREEN as "write all code."
20
+ A **seam** is the public boundary you test at: the interface where you observe behavior without reaching inside. Tests live at seams, never against internals.
21
21
 
22
- This produces **crap tests**:
22
+ **Test only at pre-agreed seams.** Before writing any test, write down the seams under test and confirm them with the user. No test is written at an unconfirmed seam. You can't test everything — agreeing the seams up front is how testing effort lands on the critical paths and complex logic instead of every edge case.
23
23
 
24
- - Tests written in bulk test _imagined_ behavior, not _actual_ behavior
25
- - You end up testing the _shape_ of things (data structures, function signatures) rather than user-facing behavior
26
- - Tests become insensitive to real changes - they pass when behavior breaks, fail when behavior is fine
27
- - You outrun your headlights, committing to test structure before understanding the implementation
24
+ Ask: "What's the public interface, and which seams should we test?"
28
25
 
29
- **Correct approach**: Vertical slices via tracer bullets. One test → one implementation → repeat. Each test responds to what you learned from the previous cycle. Because you just wrote the code, you know exactly what behavior matters and how to verify it.
26
+ ## Anti-patterns
30
27
 
31
- ```
32
- WRONG (horizontal):
33
- RED: test1, test2, test3, test4, test5
34
- GREEN: impl1, impl2, impl3, impl4, impl5
28
+ - **Implementation-coupled** — mocks internal collaborators, tests private methods, or verifies through a side channel (querying the database instead of using the interface). The tell: the test breaks when you refactor but behavior hasn't changed.
29
+ - **Tautological** — the assertion recomputes the expected value the way the code does (`expect(add(a, b)).toBe(a + b)`, a snapshot derived by hand the same way, a constant asserted equal to itself), so it passes by construction and can never disagree with the code. Expected values must come from an independent source of truth — a known-good literal, a worked example, the spec.
30
+ - **Horizontal slicing** — writing all tests first, then all implementation. Bulk tests verify _imagined_ behavior: you test the _shape_ of things rather than user-facing behavior, the tests go insensitive to real changes, and you commit to test structure before understanding the implementation. Work in **vertical slices** instead — one test → one implementation → repeat, each test a **tracer bullet** that responds to what the last cycle taught you.
35
31
 
36
- RIGHT (vertical):
37
- RED→GREEN: test1→impl1
38
- RED→GREEN: test2→impl2
39
- RED→GREEN: test3→impl3
40
- ...
41
- ```
32
+ ## Rules of the loop
42
33
 
43
- ## Workflow
44
-
45
- ### 1. Planning
46
-
47
- When exploring the codebase, use the project's domain glossary so that test names and interface vocabulary match the project's language, and respect ADRs in the area you're touching.
48
-
49
- Before writing any code:
50
-
51
- - [ ] Confirm with user what interface changes are needed
52
- - [ ] Confirm with user which behaviors to test (prioritize)
53
- - [ ] Identify opportunities for [deep modules](deep-modules.md) (small interface, deep implementation)
54
- - [ ] Design interfaces for [testability](interface-design.md)
55
- - [ ] List the behaviors to test (not implementation steps)
56
- - [ ] Get user approval on the plan
57
-
58
- Ask: "What should the public interface look like? Which behaviors are most important to test?"
59
-
60
- **You can't test everything.** Confirm with the user exactly which behaviors matter most. Focus testing effort on critical paths and complex logic, not every possible edge case.
61
-
62
- ### 2. Tracer Bullet
63
-
64
- Write ONE test that confirms ONE thing about the system:
65
-
66
- ```
67
- RED: Write test for first behavior → test fails
68
- GREEN: Write minimal code to pass → test passes
69
- ```
70
-
71
- This is your tracer bullet - proves the path works end-to-end.
72
-
73
- ### 3. Incremental Loop
74
-
75
- For each remaining behavior:
76
-
77
- ```
78
- RED: Write next test → fails
79
- GREEN: Minimal code to pass → passes
80
- ```
81
-
82
- Rules:
83
-
84
- - One test at a time
85
- - Only enough code to pass current test
86
- - Don't anticipate future tests
87
- - Keep tests focused on observable behavior
88
-
89
- ### 4. Refactor
90
-
91
- After all tests pass, look for [refactor candidates](refactoring.md):
92
-
93
- - [ ] Extract duplication
94
- - [ ] Deepen modules (move complexity behind simple interfaces)
95
- - [ ] Apply SOLID principles where natural
96
- - [ ] Consider what new code reveals about existing code
97
- - [ ] Run tests after each refactor step
98
-
99
- **Never refactor while RED.** Get to GREEN first.
100
-
101
- ## Checklist Per Cycle
102
-
103
- ```
104
- [ ] Test describes behavior, not implementation
105
- [ ] Test uses public interface only
106
- [ ] Test would survive internal refactor
107
- [ ] Code is minimal for this test
108
- [ ] No speculative features added
109
- ```
34
+ - **Red before green.** Write the failing test first, then only enough code to pass it. Don't anticipate future tests or add speculative features.
35
+ - **One slice at a time.** One seam, one test, one minimal implementation per cycle.
36
+ - **Refactoring is not part of the loop.** It belongs to the review stage (see the `code-review` skill), not the red → green implementation cycle.
@@ -59,3 +59,19 @@ test("createUser makes user retrievable", async () => {
59
59
  expect(retrieved.name).toBe("Alice");
60
60
  });
61
61
  ```
62
+
63
+ **Tautological tests**: Expected value restates the implementation, so the test passes by construction.
64
+
65
+ ```typescript
66
+ // BAD: Expected value is recomputed the way the code computes it
67
+ test("calculateTotal sums line items", () => {
68
+ const items = [{ price: 10 }, { price: 5 }];
69
+ const expected = items.reduce((sum, i) => sum + i.price, 0);
70
+ expect(calculateTotal(items)).toBe(expected);
71
+ });
72
+
73
+ // GOOD: Expected value is an independent, known literal
74
+ test("calculateTotal sums line items", () => {
75
+ expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
76
+ });
77
+ ```
@@ -0,0 +1,35 @@
1
+ # GLOSSARY.md Format
2
+
3
+ `GLOSSARY.md` is the canonical language for this teaching workspace. All explainers, exercises, and learning records should adhere to its terminology. Building it is itself part of learning: compressing a concept into a tight definition is evidence the user understands it.
4
+
5
+ ## Structure
6
+
7
+ ```md
8
+ # {Topic} Glossary
9
+
10
+ {One or two sentence description of the topic this glossary covers.}
11
+
12
+ ## Terms
13
+
14
+ **Hypertrophy**:
15
+ Muscle growth driven by mechanical tension and metabolic stress over repeated training sessions.
16
+ _Avoid_: Bulking, getting big
17
+
18
+ **Progressive overload**:
19
+ Systematically increasing the demand on a muscle over time — via load, volume, or intensity.
20
+ _Avoid_: Pushing harder, levelling up
21
+
22
+ **RPE (Rate of Perceived Exertion)**:
23
+ A 1–10 self-rating of how hard a set felt, where 10 is failure and 8 means two reps left in the tank.
24
+ _Avoid_: Effort score, intensity rating
25
+ ```
26
+
27
+ ## Rules
28
+
29
+ - **Add a term only when the user understands it.** The glossary is a record of compressed knowledge, not a dictionary the user reads to learn. If the user has just been introduced to a concept, wait until they can use it correctly before promoting it here.
30
+ - **Be opinionated.** When several words exist for the same concept, pick the best one and list the rest as aliases to avoid. This is how language compresses.
31
+ - **Keep definitions tight.** One or two sentences. Define what the term IS, not what it does or how to do it.
32
+ - **Use the glossary's own terms inside definitions.** Once a term is in the glossary, prefer it everywhere — including inside other definitions. This is what makes complex terms easier to grasp later.
33
+ - **Group under subheadings** when natural clusters emerge (e.g. `## Anatomy`, `## Programming`). A flat list is fine when terms cohere.
34
+ - **Flag ambiguities explicitly.** If a term is used loosely in the wider field, note the resolution: "In this workspace, 'set' always means a working set — warm-ups are tracked separately."
35
+ - **Revise as understanding deepens.** A definition the user wrote in week one may be wrong by week six. Update in place; do not leave stale entries.
@@ -0,0 +1,46 @@
1
+ # Learning Record Format
2
+
3
+ Learning records live in `./learning-records/` and use sequential numbering: `0001-slug.md`, `0002-slug.md`, etc. Create the directory lazily — only when the first record is written.
4
+
5
+ They are the teaching equivalent of ADRs: they capture non-obvious lessons, key insights, and stated prior knowledge that will steer future sessions. They are used to calculate the zone of proximal development.
6
+
7
+ ## Template
8
+
9
+ ```md
10
+ # {Short title of what was learned or established}
11
+
12
+ {1-3 sentences: what was learned (or what prior knowledge was established), and why it matters for future sessions.}
13
+ ```
14
+
15
+ That is the whole format. A learning record can be a single paragraph. The value is recording _that_ this is now known and _why_ it changes what to teach next — not in filling out sections.
16
+
17
+ ## Optional sections
18
+
19
+ Only include these when they add genuine value. Most records won't need them.
20
+
21
+ - **Status** frontmatter (`active | superseded by LR-NNNN`) — useful when an earlier understanding turns out to be wrong and is replaced.
22
+ - **Evidence** — how the user demonstrated the understanding (a question answered, an exercise completed, prior experience cited). Useful when the claim might be revisited.
23
+ - **Implications** — what this unlocks or rules out for future sessions. Worth recording when non-obvious.
24
+
25
+ ## Numbering
26
+
27
+ Scan `./learning-records/` for the highest existing number and increment by one.
28
+
29
+ ## When to write a learning record
30
+
31
+ Write one when any of these is true:
32
+
33
+ 1. **The user demonstrated genuine understanding of something non-trivial** — not just exposure, but evidence they can use the concept correctly. This sets a new floor for what to teach next.
34
+ 2. **The user disclosed prior knowledge** — "I already know X." Record it so future sessions don't re-teach it. Also record the _depth_ claimed.
35
+ 3. **A misconception was corrected** — the user previously believed something wrong and now sees why. These are high-value: they predict future stumbling blocks for related topics.
36
+ 4. **The mission shifted in response to learning** — the user discovered they cared about something different than they thought. Cross-link to [[MISSION.md]] and update it.
37
+
38
+ ### What does _not_ qualify
39
+
40
+ - Material that was merely covered. Coverage is not learning. Wait for evidence.
41
+ - Anything already captured tersely in [[GLOSSARY.md]] as a term definition. Don't duplicate.
42
+ - Session-by-session activity logs. Learning records are not a journal — they are decision-grade insights.
43
+
44
+ ## Supersession
45
+
46
+ When a later record contradicts an earlier one (the user's understanding deepened or corrected), mark the old record `Status: superseded by LR-NNNN` rather than deleting it. The history of how understanding evolved is itself useful signal.
@@ -0,0 +1,31 @@
1
+ # MISSION.md Format
2
+
3
+ `MISSION.md` lives at the workspace root. It captures the _reason_ the user is learning this topic. Every teaching decision — what to teach next, which resources to surface, which exercises to design — should trace back to this document.
4
+
5
+ ## Template
6
+
7
+ ```md
8
+ # Mission: {Topic}
9
+
10
+ ## Why
11
+ {1-3 sentences. The concrete real-world goal the user is chasing. What changes in their life or work when they have this skill? Avoid abstract framings like "to understand X" — push for the underlying outcome.}
12
+
13
+ ## Success looks like
14
+ - {A specific, observable thing the user will be able to do}
15
+ - {Another specific thing}
16
+ - {…}
17
+
18
+ ## Constraints
19
+ - {Time, budget, prior commitments, learning preferences, anything that bounds the approach}
20
+
21
+ ## Out of scope
22
+ - {Adjacent topics the user explicitly does not want to chase right now — protects the zone of proximal development}
23
+ ```
24
+
25
+ ## Rules
26
+
27
+ - **One mission per workspace.** If the user wants to learn two unrelated things, that is two workspaces.
28
+ - **Concrete over abstract.** "Run a half marathon by October" beats "get fitter." "Ship a Rust CLI to my team" beats "learn Rust."
29
+ - **Push back on vagueness.** If the user cannot articulate why, interview them before writing anything. A bad mission is worse than no mission.
30
+ - **Revise when reality shifts.** Missions change. When the user's goal moves, update this file — don't leave a stale mission steering future sessions.
31
+ - **Keep it short.** If `MISSION.md` runs past a screen, it has stopped being a compass and started being a plan.