@polderlabs/bizar 10.6.0 → 10.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (109) hide show
  1. package/bizar-dash/tests/memory-protocol-drift.test.mjs +18 -11
  2. package/cli/__tests__/feature-list-bridge.test.mjs +42 -42
  3. package/cli/agent-browser-up.sh +35 -35
  4. package/cli/agent-browser-update.mjs +42 -42
  5. package/cli/agent-browser-update.test.mjs +7 -7
  6. package/cli/audit.mjs +3 -3
  7. package/cli/bin.mjs +10 -10
  8. package/cli/bootstrap.mjs +1 -1
  9. package/cli/cli-commands-validation.test.mjs +2 -2
  10. package/cli/commands/claim.mjs +3 -3
  11. package/cli/commands/claude-cmd.mjs +3 -3
  12. package/cli/commands/eval.mjs +6 -6
  13. package/cli/commands/util.mjs +18 -18
  14. package/cli/commands/validate.mjs +10 -10
  15. package/cli/init.mjs +1 -1
  16. package/cli/install/postinstall.mjs +4 -4
  17. package/cli/plow-through.test.mjs +7 -6
  18. package/cli/prompts.mjs +11 -11
  19. package/cli/provision.mjs +4 -16
  20. package/config/skills/bizar/SKILL.md +1 -1
  21. package/config/skills/skillopt/SKILL.md +129 -0
  22. package/config/skills/thinking-archetypes/SKILL.md +90 -0
  23. package/config/skills/thinking-bayesian/SKILL.md +267 -0
  24. package/config/skills/thinking-bounded-rationality/SKILL.md +406 -0
  25. package/config/skills/thinking-circle-of-competence/SKILL.md +216 -0
  26. package/config/skills/thinking-cynefin/SKILL.md +70 -0
  27. package/config/skills/thinking-debiasing/SKILL.md +192 -0
  28. package/config/skills/thinking-dual-process/SKILL.md +282 -0
  29. package/config/skills/thinking-effectuation/SKILL.md +366 -0
  30. package/config/skills/thinking-feedback-loops/SKILL.md +464 -0
  31. package/config/skills/thinking-fermi-estimation/SKILL.md +263 -0
  32. package/config/skills/thinking-first-principles/SKILL.md +167 -0
  33. package/config/skills/thinking-five-whys-plus/SKILL.md +139 -0
  34. package/config/skills/thinking-inversion/SKILL.md +195 -0
  35. package/config/skills/thinking-jobs-to-be-done/SKILL.md +363 -0
  36. package/config/skills/thinking-kepner-tregoe/SKILL.md +154 -0
  37. package/config/skills/thinking-leverage-points/SKILL.md +390 -0
  38. package/config/skills/thinking-lindy-effect/SKILL.md +331 -0
  39. package/config/skills/thinking-map-territory/SKILL.md +111 -0
  40. package/config/skills/thinking-margin-of-safety/SKILL.md +330 -0
  41. package/config/skills/thinking-model-combination/SKILL.md +406 -0
  42. package/config/skills/thinking-model-router/SKILL.md +360 -0
  43. package/config/skills/thinking-model-selection/SKILL.md +341 -0
  44. package/config/skills/thinking-occams-razor/SKILL.md +129 -0
  45. package/config/skills/thinking-ooda/SKILL.md +127 -0
  46. package/config/skills/thinking-opportunity-cost/SKILL.md +360 -0
  47. package/config/skills/thinking-pre-mortem/SKILL.md +170 -0
  48. package/config/skills/thinking-probabilistic/SKILL.md +324 -0
  49. package/config/skills/thinking-red-team/SKILL.md +142 -0
  50. package/config/skills/thinking-regret-minimization/SKILL.md +335 -0
  51. package/config/skills/thinking-reversibility/SKILL.md +326 -0
  52. package/config/skills/thinking-scientific-method/SKILL.md +162 -0
  53. package/config/skills/thinking-second-order/SKILL.md +184 -0
  54. package/config/skills/thinking-socratic/SKILL.md +198 -0
  55. package/config/skills/thinking-steel-manning/SKILL.md +332 -0
  56. package/config/skills/thinking-systems/SKILL.md +238 -0
  57. package/config/skills/thinking-theory-of-constraints/SKILL.md +338 -0
  58. package/config/skills/thinking-thought-experiment/SKILL.md +354 -0
  59. package/config/skills/thinking-triz/SKILL.md +171 -0
  60. package/config/skills/thinking-via-negativa/SKILL.md +358 -0
  61. package/package.json +2 -3
  62. package/packages/sdk/dist/consensus/index.js +4 -4
  63. package/packages/sdk/dist/router/agent-model-registry.js +2 -2
  64. package/packages/sdk/dist/router/index.d.ts +3 -3
  65. package/packages/sdk/dist/router/index.js +4 -4
  66. package/packages/sdk/dist/router/q-learning-router.d.ts +2 -2
  67. package/packages/sdk/dist/router/q-learning-router.js +18 -18
  68. package/scripts/bh-full-e2e.mjs +10 -8
  69. package/scripts/check-agents.mjs +27 -24
  70. package/scripts/mirror-claude-md.sh +0 -0
  71. package/scripts/sync-skills-mirror.mjs +113 -0
  72. package/scripts/verify-thinking-skills.mjs +135 -0
  73. package/config/agents/_shared/AGENT_BASELINE.md +0 -155
  74. package/config/agents/_shared/CLINE_TOOLS.md +0 -398
  75. package/config/agents/_shared/SKILLS.md +0 -109
  76. package/config/agents/agent-browser.md +0 -90
  77. package/config/agents/baldr.md +0 -60
  78. package/config/agents/forseti.md +0 -60
  79. package/config/agents/frigg.md +0 -52
  80. package/config/agents/heimdall.md +0 -44
  81. package/config/agents/hermod.md +0 -64
  82. package/config/agents/mimir.md +0 -60
  83. package/config/agents/odin.md +0 -310
  84. package/config/agents/quick.md +0 -47
  85. package/config/agents/semble-search.md +0 -59
  86. package/config/agents/thor.md +0 -64
  87. package/config/agents/tyr.md +0 -67
  88. package/config/agents/vidarr.md +0 -65
  89. package/config/agents/vor.md +0 -62
  90. package/config/commands/audit.md +0 -5
  91. package/config/commands/bizar.md +0 -22
  92. package/config/commands/explain.md +0 -5
  93. package/config/commands/init.md +0 -26
  94. package/config/commands/learn.md +0 -5
  95. package/config/commands/plan.md +0 -30
  96. package/config/commands/plow-through.md +0 -47
  97. package/config/commands/pr-review.md +0 -5
  98. package/config/commands/setup-provider.md +0 -95
  99. package/config/commands/tailscale-serve.md +0 -97
  100. package/config/commands/team.md +0 -124
  101. package/config/commands/test.md +0 -60
  102. package/config/commands/validate.md +0 -66
  103. package/config/commands/visual-plan.md +0 -19
  104. package/config/hooks/PostToolUse +0 -66
  105. package/config/hooks/PreToolUse +0 -82
  106. package/config/hooks/README.md +0 -68
  107. package/config/hooks/TaskResume +0 -48
  108. package/config/hooks/TaskStart +0 -47
  109. package/config/hooks/UserPromptSubmit +0 -55
@@ -11,10 +11,10 @@ import { join } from 'node:path';
11
11
  import { resolveClaudeDir, PATHS } from './paths.mjs';
12
12
 
13
13
  const AGENT_FILES = [
14
- 'odin.md', 'vor.md', 'frigg.md', 'quick.md',
15
- 'mimir.md', 'heimdall.md', 'hermod.md', 'thor.md', 'baldr.md',
16
- 'tyr.md', 'vidarr.md', 'forseti.md',
17
- 'semble-search.md',
14
+ 'mike.md', 'janet.md', 'susan.md', 'pam.md',
15
+ 'greg.md', 'brenda.md', 'steve.md', 'todd.md', 'brad.md',
16
+ 'karen.md', 'carl.md', 'linda.md',
17
+ 'oscar.md',
18
18
  ];
19
19
 
20
20
  /**
@@ -11,10 +11,11 @@ import { fileURLToPath } from 'node:url';
11
11
 
12
12
  const __dirname = dirname(fileURLToPath(import.meta.url));
13
13
  const PROJECT_ROOT = join(__dirname, '..');
14
- const CMD_PATH = join(PROJECT_ROOT, 'config', 'commands', 'plow-through.md');
14
+ // F-107: slash commands canonical at .claude/commands/. Legacy config/commands/ removed (Cline-era).
15
+ const CMD_PATH = join(PROJECT_ROOT, '.claude', 'commands', 'plow-through.md');
15
16
 
16
17
  describe('/plow-through command file', () => {
17
- test('file exists at config/commands/plow-through.md', () => {
18
+ test('file exists at .claude/commands/plow-through.md', () => {
18
19
  assert.equal(existsSync(CMD_PATH), true, 'plow-through.md must exist');
19
20
  });
20
21
 
@@ -23,10 +24,10 @@ describe('/plow-through command file', () => {
23
24
  assert.match(content, /^---\ndescription:/m, 'frontmatter must have description');
24
25
  });
25
26
 
26
- test('YAML frontmatter has agent field set to odin', () => {
27
- const content = readFileSync(CMD_PATH, 'utf8');
28
- assert.match(content, /^---\n.*\nagent: odin$/ms, 'frontmatter agent must be odin');
29
- });
27
+ // F-107: legacy `agent: mike` frontmatter field was removed when slash
28
+ // commands were migrated to Claude Code's `run_in_background: true`
29
+ // Agent dispatch model. Routed agents are now inferred from the
30
+ // command body via the thinking-route hook.
30
31
 
31
32
  test('body has content (more than 5 lines)', () => {
32
33
  const content = readFileSync(CMD_PATH, 'utf8');
package/cli/prompts.mjs CHANGED
@@ -1,17 +1,17 @@
1
1
  import inquirer from 'inquirer';
2
2
 
3
3
  const AGENTS_LIST = [
4
- { name: 'Odin ᛟ — MiniMax-M3 (Router)', value: 'odin.md', checked: true },
5
- { name: 'Frigg ᚠ — DeepSeek (Codebase Q&A, read-only)', value: 'frigg.md', checked: true },
6
- { name: 'Vör ᚡ — DeepSeek (Clarification)', value: 'vor.md', checked: true },
7
- { name: 'Mimir ᛗ — DeepSeek (Research)', value: 'mimir.md', checked: true },
8
- { name: 'Heimdall ᚹ — DeepSeek (Simple tasks)', value: 'heimdall.md', checked: true },
9
- { name: 'Hermod ᚱ — MiniMax-M2.7 (Git ops)', value: 'hermod.md', checked: true },
10
- { name: 'Thor ᚦ — MiniMax-M2.7 (Mid impl.)', value: 'thor.md', checked: true },
11
- { name: 'Baldr ᛒ — MiniMax-M2.7 (Design)', value: 'baldr.md', checked: true },
12
- { name: 'Tyr ᛏ — MiniMax-M3 (Complex impl.)', value: 'tyr.md', checked: true },
13
- { name: 'Vidarr ᛉ — MiniMax-M3 (Last resort)', value: 'vidarr.md', checked: false },
14
- { name: 'Forseti ᚨ — MiniMax-M3 (Plan audit)', value: 'forseti.md', checked: true },
4
+ { name: 'Odin ᛟ — MiniMax-M3 (Router)', value: 'mike.md', checked: true },
5
+ { name: 'Frigg ᚠ — DeepSeek (Codebase Q&A, read-only)', value: 'susan.md', checked: true },
6
+ { name: 'Vör ᚡ — DeepSeek (Clarification)', value: 'janet.md', checked: true },
7
+ { name: 'Mimir ᛗ — DeepSeek (Research)', value: 'greg.md', checked: true },
8
+ { name: 'Heimdall ᚹ — DeepSeek (Simple tasks)', value: 'brenda.md', checked: true },
9
+ { name: 'Hermod ᚱ — MiniMax-M2.7 (Git ops)', value: 'steve.md', checked: true },
10
+ { name: 'Thor ᚦ — MiniMax-M2.7 (Mid impl.)', value: 'todd.md', checked: true },
11
+ { name: 'Baldr ᛒ — MiniMax-M2.7 (Design)', value: 'brad.md', checked: true },
12
+ { name: 'Tyr ᛏ — MiniMax-M3 (Complex impl.)', value: 'karen.md', checked: true },
13
+ { name: 'Vidarr ᛉ — MiniMax-M3 (Last resort)', value: 'carl.md', checked: false },
14
+ { name: 'Forseti ᚨ — MiniMax-M3 (Plan audit)', value: 'linda.md', checked: true },
15
15
  ];
16
16
 
17
17
  export async function promptComponents() {
package/cli/provision.mjs CHANGED
@@ -430,21 +430,10 @@ function syncDir(srcDir, destDir, opts = {}) {
430
430
  return { copied, skipped };
431
431
  }
432
432
 
433
- export async function syncAgentFiles({ dryRun = false } = {}) {
434
- const src = join(REPO_ROOT, 'config', 'agents');
435
- const dest = CLAUDE_AGENTS_DIR;
436
- if (!existsSync(src)) return { ok: true, message: `no agents source at ${src}`, copied: 0, skipped: 0 };
437
- if (dryRun) return { ok: true, message: `[dry-run] would sync ${src} → ${dest}` };
438
- ensureDir(dest); ensureDir(join(dest, '_shared'));
439
- let { copied, skipped } = syncDir(src, dest, { filter: n => n.endsWith('.md') });
440
- const sharedSrc = join(src, '_shared');
441
- if (existsSync(sharedSrc)) {
442
- for (const entry of readdirSync(sharedSrc, { withFileTypes: true })) {
443
- if (entry.isFile()) { copyFileSync(join(sharedSrc, entry.name), join(dest, '_shared', entry.name)); copied++; }
444
- }
445
- }
446
- return { ok: true, message: `${copied} agent(s) synced (${skipped} kept)`, copied, skipped };
447
- }
433
+ // F-107: syncAgentFiles removed. Agent definitions live at
434
+ // .claude/agents/ (Claude Code canonical) — they were never sourced
435
+ // from config/agents/ in Claude Code era, and the legacy source dir
436
+ // was deleted.
448
437
 
449
438
  export async function syncSkillFiles({ dryRun = false } = {}) {
450
439
  const src = join(REPO_ROOT, 'config', 'skills');
@@ -704,7 +693,6 @@ export async function runProvision(opts = {}) {
704
693
  return r;
705
694
  };
706
695
 
707
- await runStep('Syncing agents', () => syncAgentFiles({ dryRun, force }));
708
696
  await runStep('Syncing skills', () => syncSkillFiles({ dryRun, force }));
709
697
  await runStep('Syncing commands', () => syncCommandFiles({ dryRun, force }));
710
698
  await runStep('Syncing rules', () => syncRulesFiles({ dryRun, force }));
@@ -253,7 +253,7 @@ Single-key mode is unchanged — if only `MINIMAX_API_KEY` is set (or no key rot
253
253
 
254
254
  ## Verbose Agent Baseline Reference
255
255
 
256
- > The 12 always-on rules in `config/agents/_shared/AGENT_BASELINE.md`
256
+ > The 12 always-on rules in `.claude/agents/_shared/AGENT_BASELINE.md`
257
257
  > are auto-loaded into every agent session at startup. The full prose
258
258
  > for each rule lives below — read this section when an agent needs
259
259
  > the full rationale, examples, and decision tree for a given rule.
@@ -0,0 +1,129 @@
1
+ ---
2
+ name: skillopt
3
+ description: Microsoft's SkillOpt trains skill files via rollouts + reflection against a frozen target model. Install with `pip install skillopt` and produce a `best_skill.md`. Always human-review.
4
+ ---
5
+
6
+ # SkillOpt Workflow
7
+
8
+ ## Overview
9
+
10
+ SkillOpt (microsoft/SkillOpt, MIT-licensed) is a Microsoft Research Python tool that **optimizes skill files for a specific target model**. Given a starting `SKILL.md`, a target model (frozen), a benchmark of inputs, and a reward signal, SkillOpt runs a rollout → reflect → aggregate → select → update → evaluate loop and emits a `best_skill.md` artifact that scores higher against the held-out validation gate.
11
+
12
+ It is *not* a general prompt editor and *not* a free-form rewriting tool. It is a closed-loop optimizer: it changes the skill text, runs the agent under the new skill against the benchmark, measures reward, and keeps mutations that improve the score.
13
+
14
+ **Core Principle:** SkillOpt trains a skill file the way you'd fine-tune a model — with a held-out validation gate, a frozen target, and a reward signal — not by hand-tweaking prose. Don't commit the output without human review; treat `best_skill.md` as a candidate, not a verdict.
15
+
16
+ ## When to Use
17
+
18
+ - A skill's quality is measurably degrading on a benchmark (success rate dropping, regression in eval)
19
+ - You have a benchmark of representative inputs and a reward signal (task success / rubric score / pass-rate)
20
+ - You want to A/B a new skill variant against the current one against the same target model
21
+ - A skill has grown organically past ~200 lines and you suspect prose bloat is hurting recall
22
+
23
+ Decision flow:
24
+
25
+ ```
26
+ Skill quality question?
27
+ → Do you have a benchmark + reward + frozen target model? → no → DON'T RUN SKILLOPT (you can't measure without them)
28
+ → Is the skill underperforming on the gate? → no → no work to do
29
+ → Yes to both → see the 6-phase loop below
30
+ ```
31
+
32
+ ## When NOT to Use
33
+
34
+ - **No benchmark, no reward, no frozen target.** SkillOpt needs all three. If any is missing, the loop has nothing to optimize against — you'll burn compute on something you can't evaluate. Compose the benchmark first or skip SkillOpt entirely.
35
+ - **For a one-off skill polish.** SkillOpt is a multi-rollout loop with token cost on the order of dozens to hundreds of completions. For "rewrite this paragraph to be clearer," edit the skill by hand. Reach for SkillOpt when you have evidence the skill underperforms and a way to measure the fix.
36
+ - **On a non-frozen target model.** If the target model is updating daily, the rollout scores are non-reproducible and the held-out gate stops meaning anything. Pin the model version and don't retrain against a moving target.
37
+ - **Without human review of `best_skill.md`.** SkillOpt optimizes the reward signal — that is not the same as "is this skill correct, well-bounded, and aligned with what we want the agent to do." A skill that scores higher on the rubric can still be narrower, more brittle, or subtly off-vibe. Always review before committing.
38
+
39
+ ## The 6-Phase Loop
40
+
41
+ ```
42
+ ┌─ Phase 1: Rollout ─ run the agent under the current skill against benchmark inputs
43
+
44
+ ├─ Phase 2: Reflect ─ for each rollout, capture which skill instructions helped vs hurt
45
+
46
+ ├─ Phase 3: Aggregate ─ across all rollouts, find recurring failure modes and structural patterns
47
+
48
+ ├─ Phase 4: Select ─ choose one mutation hypothesis (a concrete edit) to try next
49
+
50
+ ├─ Phase 5: Update ─ apply the mutation to produce a candidate skill
51
+
52
+ └─ Phase 6: Evaluate ─ run the candidate against the HELD-OUT validation set; keep if it improves, drop if not
53
+ ↑ │
54
+ └────────────── next iteration reuses the kept variant ─────────┘
55
+ ```
56
+
57
+ ### Phase 1 — Rollout
58
+
59
+ Run the agent under the **frozen target model** with the current skill against the benchmark. Capture outputs + per-input scores. Don't peek at the validation set during this phase.
60
+
61
+ ### Phase 2 — Reflect
62
+
63
+ For each rollout, ask: which lines of the skill contributed to the success or failure? Mark helpful / neutral / harmful spans. This is the closest you get to a teacher signal without a human labeler.
64
+
65
+ ### Phase 3 — Aggregate
66
+
67
+ Across all rollouts this iteration, find the recurring failure modes. If 4 of 5 failures on a particular input pattern come from the same missing instruction in the skill, that's the highest-leverage next mutation.
68
+
69
+ ### Phase 4 — Select
70
+
71
+ Pick **one** mutation to try. SkillOpt is not a search-and-replace shotgun; it's one edit per round. Co-edits entangle effects and make the held-out gate stop carrying signal.
72
+
73
+ ### Phase 5 — Update
74
+
75
+ Apply the mutation. Save the candidate as a sibling file (e.g. `candidate-3.md`); never overwrite `SKILL.md` mid-loop.
76
+
77
+ ### Phase 6 — Evaluate
78
+
79
+ Run the candidate against the **held-out validation set** (never been seen during Phases 1–5). If it improves the held-out gate, promote it to the next iteration's starting point. If it regresses, drop it and try a different mutation. Do not cherry-pick from a known input; the held-out gate is what makes the loop trustworthy.
80
+
81
+ ## How Bizar Agents Use It
82
+
83
+ ```bash
84
+ # 1. Install (do this once per machine)
85
+ pip install skillopt
86
+
87
+ # 2. Set up a benchmark directory (you write this — SkillOpt doesn't generate it)
88
+ # benchmark/
89
+ # ├── inputs.jsonl # one prompt per line, hold-out set included
90
+ # ├── reward.py # your scoring fn: takes (prompt, completion) → float
91
+ # └── target_model.json # the frozen target model spec (id + version pin)
92
+
93
+ # 3. Run the loop against a draft skill
94
+ python -m skillopt.cli optimize \
95
+ --skill config/skills/thinking-some-skill/SKILL.md \
96
+ --benchmark ./benchmark \
97
+ --out ./skillopt-out
98
+
99
+ # 4. Inspect the output before committing anything
100
+ ls skillopt-out/
101
+ cat skillopt-out/best_skill.md
102
+ diff config/skills/thinking-some-skill/SKILL.md skillopt-out/best_skill.md
103
+
104
+ # 5. If the diff is genuinely better (held-out improvement + human review passes),
105
+ # replace the source. Otherwise discard and tune the benchmark.
106
+ cp skillopt-out/best_skill.md config/skills/thinking-some-skill/SKILL.md
107
+ ```
108
+
109
+ We do **not** vendor SkillOpt into the Bizar JS harness — the tool lives in Python (`pip install skillopt`), runs offline, and writes a markdown artifact. The Bizar loop is: write draft → run SkillOpt → human review `best_skill.md` → commit if better.
110
+
111
+ ## Pitfalls
112
+
113
+ - **Don't retrain daily.** The target model is frozen for a reason; if your benchmark doesn't drift, the skill shouldn't either. Bumping SkillOpt runs on every PR turns the skill into a moving target the agent can't build a stable intuition against.
114
+ - **Don't bypass the held-out gate.** It's tempting to "peek" at validation items to bias the mutation — that converts the loop into a curve-fit on a small set and the held-out number stops meaning anything. Hold-out integrity is the whole point.
115
+ - **Never auto-commit `best_skill.md`.** It is a *candidate*. The reward signal optimizes task success; it does not optimize for tone, scope, or alignment with sibling skills in the same family. A/B test in shadow, then promote deliberately.
116
+ - **Don't use SkillOpt on skills you don't have a benchmark for.** If the input space and reward are unknown, you're optimizing prose against nothing and will learn something — just not something useful.
117
+ - **Don't optimize `thinking-*` skills together as one bundle.** They share front-matter shape but address different problem types; co-optimizing entangles their effects and makes individual improvements hard to attribute.
118
+
119
+ ## Relationship to Bizar's Existing Skill Pipeline
120
+
121
+ | Concern | Owner |
122
+ |---|---|
123
+ | What skills ship | `config/skills/<name>/SKILL.md` (canonical, provisioned) |
124
+ | Provision to user | `cli/provision.mjs:syncConfigExtras` + `writeBizarSkillLock` |
125
+ | Skill discovery at runtime | `~/.claude/skills/<name>/SKILL.md` (provisioned mirror) |
126
+ | Offline quality loop | SkillOpt (this skill) — run on a draft, produce `best_skill.md`, human-promote |
127
+ | Editing prose by hand | You + your editor — fine for one-off tweaks, don't loop it |
128
+
129
+ SkillOpt is a **research-quality tool** for the rare case where you have the benchmark, the reward, and the time. For 90% of skill edits, just write it well.
@@ -0,0 +1,90 @@
1
+ ---
2
+ name: thinking-archetypes
3
+ description: Use when the same problem keeps recurring despite fixes, growth stalled with no obvious cause, or a quick fix made things worse—match it to a known structural pattern instead of re-diagnosing.
4
+ ---
5
+
6
+ # Systems Archetypes
7
+
8
+ ## Overview
9
+
10
+ Systems archetypes (Peter Senge, "The Fifth Discipline") are recurring structural patterns. Like design patterns in software, once you recognize one you can predict where it leads and where to intervene. Most stubborn, recurring problems aren't unique—they're an instance of a known pattern, and the pattern names the leverage point.
11
+
12
+ **Core Principle:** A recurring problem usually has a structural pattern. Match the pattern and the intervention follows.
13
+
14
+ ## Trigger Card
15
+
16
+ When the same problem keeps recurring despite multiple fixes:
17
+
18
+ 1. **Describe the problem neutrally** — symptoms, what's been tried, what keeps coming back.
19
+ 2. **Match to an archetype** using the Quick Reference Card below (Fixes That Fail, Shifting the Burden, Limits to Growth, Tragedy of the Commons, Escalation, Success to the Successful, Growth and Underinvestment).
20
+ 3. **Intervene at the structure, not the symptom** — use the Key Question for that archetype.
21
+
22
+ If no archetype fits after a genuine look, don't force one — drop to `thinking-systems` and map from scratch. For a first-time one-off problem, just fix it.
23
+
24
+ ## When to Use
25
+
26
+ - The same problem keeps recurring despite multiple "fixes"
27
+ - A quick fix made things worse over time
28
+ - Growth stalled without an obvious cause
29
+ - A shared resource (CI, staging, a cache, a service quota) keeps degrading
30
+ - Two efforts are escalating against each other (duplicated services, alert/threshold arms races)
31
+
32
+ ```
33
+ Problem keeps recurring despite fixes? → match an archetype
34
+ Quick fix made it worse? → match an archetype
35
+ Growth hit an invisible ceiling? → match an archetype
36
+ Shared resource degrading? → match an archetype
37
+ ```
38
+
39
+ ## When NOT to Use
40
+
41
+ - **You need to map the system from scratch** (components, interactions, emergent behavior) — use `thinking-systems` instead. Archetypes are pattern-matching shortcuts for recurring structures; if the system is unfamiliar, map it first, then see if an archetype fits.
42
+ - **You've found the pattern and need to know where to intervene** — use `thinking-leverage-points` (Meadows' hierarchy). Archetypes name the structure; leverage-points tell you where in that structure to act for maximum effect.
43
+ - A first-time, one-off problem with a clear cause → just fix it; there's no recurring structure to match.
44
+ - No candidate archetype fits after a genuine look → don't force one. Drop to `thinking-systems` and map the actual structure from scratch.
45
+ - You need to localize a specific faulty codepath → use a hypothesis differential, not pattern-matching.
46
+
47
+ ## Quick Reference Card
48
+
49
+ Match the symptom to the pattern; the Key Question points at the leverage.
50
+
51
+ | Archetype | Structure | Recognize it by | Key Question (leverage) |
52
+ |-----------|-----------|-----------------|-------------------------|
53
+ | **Fixes That Fail** | Quick fix relieves symptom but a delayed side effect makes it worse | "We fixed this last quarter, why is it back?"; the fix needs ever-larger doses | "What side effect will this fix create?" |
54
+ | **Shifting the Burden** | A symptomatic workaround is used instead of the fundamental fix, creating dependency | Permanent workarounds; "we know the real fix but have no time"; the real capability is atrophying | "What capability are we not building by leaning on this workaround?" |
55
+ | **Limits to Growth** | A reinforcing growth loop hits a balancing constraint | Strong growth that plateaued; more effort, diminishing returns; a resource maxed out | "What will limit us at 10x scale?" |
56
+ | **Tragedy of the Commons** | Each actor gains by using a shared resource; collective overuse depletes it | Shared CI/staging/cache/quota degrading; everyone optimizes locally; no clear owner | "Who owns the long-term health of this resource?" |
57
+ | **Escalation** | Two parties react to each other's moves in a competitive spiral | "They did X so we must do Y"; arms-race dynamics; each side feels defensive | "Can we change the game instead of playing it harder?" |
58
+ | **Success to the Successful** | Initial success grants more resources, compounding advantage and starving alternatives | Past success is the main predictor of new investment; experiments starved to feed the incumbent | "Are we starving future successes to feed current ones?" |
59
+ | **Growth and Underinvestment** | Demand grows toward a limit; capacity investment is delayed until performance degrades into crisis | Reactive investment after the incident; chronic "good enough for now" | "What fails if we grow 50% without adding capacity now?" |
60
+
61
+ ## Software Examples (per archetype)
62
+
63
+ - **Fixes That Fail:** Adding servers to mask a memory leak → cost spirals, leak remains. Silencing alerts → system degrades until outage.
64
+ - **Shifting the Burden:** Ops manually intervening forever instead of automating; feature flags hiding bugs that never get fixed.
65
+ - **Limits to Growth:** Velocity drops as the codebase grows (complexity limit); adding engineers doesn't speed delivery (coordination overhead).
66
+ - **Tragedy of the Commons:** Teams overload shared CI → everyone's builds slow; cloud costs spike as nobody cleans up resources.
67
+ - **Escalation:** Microservice teams duplicating functionality in response to each other; ever-stricter alert thresholds chasing each other.
68
+ - **Success to the Successful:** Legacy monolith gets all attention so the new architecture never matures.
69
+ - **Growth and Underinvestment:** Database not upgraded until it crashes under load; security investment only after a breach.
70
+
71
+ ## Diagnosis Process
72
+
73
+ 1. **Describe the problem neutrally** — symptoms, who/what is involved, what's been tried, what keeps recurring. No blame.
74
+ 2. **Sketch the loops** — what causes what; where reinforcing (amplifying) loops are, where balancing (limiting) loops are, and where the delays sit.
75
+ 3. **Match to an archetype** using the Quick Reference Card. If two fit, the system may be layered—note both.
76
+ 4. **Intervene at the structure, not the symptom** — use the Key Question to find the fundamental solution, the constraint to remove, or the missing/ delayed feedback to add. If none match, stop and use `thinking-systems`.
77
+
78
+ ## Verification Checklist
79
+
80
+ - [ ] Problem described without blame
81
+ - [ ] Loops sketched (reinforcing, balancing, delays)
82
+ - [ ] Matched to a specific archetype (or escalated to `thinking-systems` because none fit)
83
+ - [ ] Intervention targets structure, not the recurring symptom
84
+ - [ ] Considered the side effects of the intervention
85
+
86
+ ## Senge's Wisdom
87
+
88
+ "Structures of which we are unaware hold us prisoner. Once we can see them, they no longer have the same hold on us."
89
+
90
+ The pattern continues until someone sees it and changes the structure driving it—not the behaviors, but what produces them.
@@ -0,0 +1,267 @@
1
+ ---
2
+ name: thinking-bayesian
3
+ description: Use when interpreting a test result, metric, or new evidence and you risk over-reacting to it. State the base rate first, then update belief by the likelihood ratio.
4
+ ---
5
+
6
+ # Bayesian Reasoning
7
+
8
+ ## Overview
9
+ Bayesian thinking provides a framework for updating beliefs based on new evidence. Rather than treating beliefs as binary (true/false), it recognizes degrees of confidence that should shift as evidence accumulates. This approach, rooted in Bayes' Theorem, helps avoid both overconfidence and underreaction to new information.
10
+
11
+ **Core Principle:** State the base rate *before* you look at the evidence, then update. The single most common error is anchoring on a vivid result and skipping the prior — a positive test for a rare condition is usually still a false alarm.
12
+
13
+ > **Redirect:** For nearly all uncertainty-reasoning tasks, prefer `thinking-probabilistic` — it covers forecasting with ranges, calibration, and uncertainty communication without requiring explicit priors and likelihood ratios. Reserve this skill for the narrow case where you have a specific, quantified prior AND a specific piece of evidence to combine via Bayes' rule. If you only need a rough updated number, the base-rate-then-likelihood-ratio trigger below is enough — don't run the full theorem.
14
+
15
+ ## Trigger Card
16
+
17
+ When interpreting a test result, metric, or new evidence where overreaction is a risk:
18
+
19
+ 1. **State the base rate first** — what was the probability before the evidence?
20
+ 2. **Estimate the likelihood ratio** — how much more expected is the evidence under the hypothesis vs. not?
21
+ 3. **Update:** posterior odds = prior odds × likelihood ratio.
22
+
23
+ If the base rate is very low (rare condition), a positive result is usually still a false alarm. Always start with the prior.
24
+
25
+ ## When to Use
26
+ - Estimating probabilities or likelihoods
27
+ - Interpreting test results or metrics
28
+ - Making decisions with incomplete information
29
+ - Evaluating competing hypotheses
30
+ - Learning from experiments or A/B tests
31
+ - Diagnosing problems with uncertain causes
32
+ - Predicting outcomes based on historical data
33
+
34
+ Decision flow:
35
+ ```
36
+ Uncertain about something? → yes → Have prior belief? → yes → New evidence? → APPLY BAYESIAN UPDATE
37
+ ↘ no → Establish base rate first
38
+ ↘ no → Standard analysis may suffice
39
+ ```
40
+
41
+ ## When NOT to Use
42
+
43
+ - **You're reasoning under uncertainty and don't have a specific prior + evidence pair.** Use `thinking-probabilistic` instead — it handles forecasting with ranges, calibration, and uncertainty communication without requiring formal Bayesian machinery.
44
+ - **The evidence is conclusive or directly observable.** If you can read the logs, run the query, or look up the answer, do that — don't dress up a near-certainty as a probability.
45
+ - **No meaningful prior exists** and you'd just be inventing numbers. A fabricated base rate gives false rigor; say the prior is unknown instead.
46
+ - **The decision is the same at any plausible posterior.** If you'd act identically whether the probability is 40% or 70%, skip the update and act.
47
+ - **You need to express uncertainty as a calibrated range**, not a point update from a single piece of evidence — that's `thinking-probabilistic` territory.
48
+
49
+ ## Key Concepts
50
+
51
+ ### Prior Probability
52
+ Your belief BEFORE seeing new evidence:
53
+ ```
54
+ P(H) = probability that hypothesis H is true
55
+
56
+ Example: Before any symptoms, what's the probability someone has disease X?
57
+ Use base rate: If 1 in 1000 people have it, P(disease) = 0.001
58
+ ```
59
+
60
+ ### Likelihood
61
+ How probable is the evidence IF the hypothesis is true?
62
+ ```
63
+ P(E|H) = probability of seeing evidence E, given H is true
64
+
65
+ Example: If someone HAS the disease, what's the probability of a positive test?
66
+ If test is 99% sensitive: P(positive|disease) = 0.99
67
+ ```
68
+
69
+ ### Posterior Probability
70
+ Your belief AFTER seeing the evidence:
71
+ ```
72
+ P(H|E) = updated probability of H, given you observed E
73
+
74
+ This is what Bayes' Theorem calculates.
75
+ ```
76
+
77
+ ## Bayes' Theorem
78
+
79
+ ```
80
+ P(E|H) × P(H)
81
+ P(H|E) = ─────────────────────────
82
+ P(E)
83
+
84
+ Where:
85
+ P(H|E) = posterior (what we want)
86
+ P(E|H) = likelihood (how expected is evidence if H true)
87
+ P(H) = prior (initial belief)
88
+ P(E) = total probability of evidence
89
+ ```
90
+
91
+ ### Intuitive Form
92
+
93
+ ```
94
+ Posterior odds = Prior odds × Likelihood ratio
95
+
96
+ If evidence is 10x more likely under H than under not-H,
97
+ your odds should shift by factor of 10.
98
+ ```
99
+
100
+ ## The Process
101
+
102
+ ### Step 1: Establish Your Prior
103
+ What did you believe before this evidence?
104
+ - Use base rates when available
105
+ - Be explicit about uncertainty
106
+ - Don't anchor on 50% just because you're unsure
107
+
108
+ ```
109
+ Question: Will this feature increase conversion?
110
+ Prior: Based on similar features, ~30% succeed significantly
111
+ P(success) = 0.30
112
+ ```
113
+
114
+ ### Step 2: Assess the Evidence
115
+ How strong is this evidence? Consider:
116
+ - How likely is this evidence if hypothesis is TRUE?
117
+ - How likely is this evidence if hypothesis is FALSE?
118
+ - What's the ratio?
119
+
120
+ ```
121
+ Evidence: Early A/B test shows 5% lift (p=0.08)
122
+ P(this result | feature works) = 0.60 (moderately expected)
123
+ P(this result | feature doesn't work) = 0.15 (possible but less likely)
124
+ Likelihood ratio = 0.60 / 0.15 = 4x
125
+ ```
126
+
127
+ ### Step 3: Update Your Belief
128
+ Apply the likelihood ratio to your prior:
129
+
130
+ ```
131
+ Prior odds: 0.30 / 0.70 = 0.43
132
+ Likelihood ratio: 4x
133
+ Posterior odds: 0.43 × 4 = 1.72
134
+ Posterior probability: 1.72 / (1 + 1.72) = 0.63
135
+
136
+ Updated belief: 63% confidence feature will succeed
137
+ (up from 30% prior)
138
+ ```
139
+
140
+ ### Step 4: Iterate as More Evidence Arrives
141
+ Yesterday's posterior becomes today's prior:
142
+
143
+ ```
144
+ New evidence: Week 2 shows lift holding at 4.5%
145
+ Prior (from step 3): 0.63
146
+ [Repeat update process]
147
+ New posterior: 0.78
148
+ ```
149
+
150
+ ## Common Applications
151
+
152
+ ### Interpreting Test Results
153
+ ```
154
+ Scenario: Test for rare disease (1 in 10,000 prevalence)
155
+ Test: 99% sensitive, 99% specific
156
+
157
+ Prior: P(disease) = 0.0001
158
+ If positive test:
159
+ P(positive|disease) = 0.99
160
+ P(positive|no disease) = 0.01
161
+ P(positive) = 0.99 × 0.0001 + 0.01 × 0.9999 ≈ 0.0101
162
+
163
+ Posterior: P(disease|positive) = (0.99 × 0.0001) / 0.0101 ≈ 0.0098
164
+
165
+ Even with 99% accurate test, positive result only means ~1% chance of disease!
166
+ Base rate dominates when condition is rare.
167
+ ```
168
+
169
+ ### Debugging
170
+ ```
171
+ Bug report: Users see error X
172
+ Prior beliefs:
173
+ P(database issue) = 0.20
174
+ P(network issue) = 0.30
175
+ P(code bug) = 0.40
176
+ P(user error) = 0.10
177
+
178
+ Evidence: Error happens only on mobile
179
+ P(mobile-only | database) = 0.05
180
+ P(mobile-only | network) = 0.30
181
+ P(mobile-only | code bug) = 0.60
182
+ P(mobile-only | user error) = 0.40
183
+
184
+ Update: Code bug becomes most likely (posterior ~0.55)
185
+ Next step: Investigate mobile-specific code paths
186
+ ```
187
+
188
+ ### Project Estimation
189
+ ```
190
+ Prior: Based on similar projects, P(on-time) = 0.40
191
+
192
+ Evidence 1: Team is experienced with this stack
193
+ Likelihood ratio: 1.5x → Posterior: 0.50
194
+
195
+ Evidence 2: Requirements are unclear
196
+ Likelihood ratio: 0.6x → Posterior: 0.38
197
+
198
+ Evidence 3: Critical dependency has risk
199
+ Likelihood ratio: 0.7x → Posterior: 0.30
200
+
201
+ Final estimate: 30% chance of on-time delivery
202
+ ```
203
+
204
+ ## Mental Shortcuts
205
+
206
+ ### Strong vs Weak Evidence
207
+ | Evidence Type | Typical Likelihood Ratio |
208
+ |---------------|-------------------------|
209
+ | Definitive proof | 100x+ |
210
+ | Strong evidence | 10-100x |
211
+ | Moderate evidence | 3-10x |
212
+ | Weak evidence | 1.5-3x |
213
+ | Noise | ~1x (no update) |
214
+
215
+ ### When to Update Significantly
216
+ Update strongly when:
217
+ - Evidence is surprising under your current belief
218
+ - Evidence comes from reliable source
219
+ - Evidence is specific to your hypothesis
220
+
221
+ Update weakly when:
222
+ - Evidence is expected regardless of hypothesis
223
+ - Source has unknown reliability
224
+ - Evidence is circumstantial
225
+
226
+ ### Base Rate Neglect (Avoid This)
227
+ Common error: Ignoring prior probability when evidence arrives
228
+ ```
229
+ Wrong: "Positive test = probably have disease"
230
+ Right: "Positive test shifts probability, but base rate matters"
231
+ ```
232
+
233
+ ## Calibration Check
234
+
235
+ ### Are You Well-Calibrated?
236
+ Track predictions and outcomes:
237
+ - Of things you said were "70% likely," did ~70% happen?
238
+ - If you're always overconfident, widen your uncertainty
239
+ - If you're always underconfident, trust your assessments more
240
+
241
+ ### Confidence Levels
242
+ | Stated Confidence | Should Mean |
243
+ |-------------------|------------|
244
+ | 50% | Coin flip |
245
+ | 70% | Would bet 2:1 |
246
+ | 90% | Would bet 9:1 |
247
+ | 99% | Would bet 99:1 |
248
+
249
+ ## Verification Checklist
250
+ - [ ] Established explicit prior probability (not just "I think...")
251
+ - [ ] Assessed likelihood ratio of evidence
252
+ - [ ] Applied update mathematically (not just "more/less likely")
253
+ - [ ] Considered base rates for rare events
254
+ - [ ] Checked for base rate neglect
255
+ - [ ] Documented reasoning for future calibration
256
+
257
+ ## Key Questions
258
+ - "What was my belief before this evidence?"
259
+ - "How likely is this evidence if my belief is true? If false?"
260
+ - "What's the likelihood ratio?"
261
+ - "Am I anchoring on the evidence and ignoring base rates?"
262
+ - "How would I bet on this? At what odds?"
263
+
264
+ ## Kahneman's Warning
265
+ "People tend to assess the relative importance of issues by the ease with which they are retrieved from memory—and this is largely determined by the extent of coverage in the media."
266
+
267
+ Don't let vivid evidence override base rates. A plane crash doesn't make flying more dangerous than driving, even though it feels that way.