@ak--47/dungeon-master 1.5.3 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (146) hide show
  1. package/.claude/skills/analyze-soup/SKILL.md +21 -12
  2. package/.claude/skills/create-dungeon/SKILL.md +65 -5
  3. package/.claude/skills/create-project/SKILL.md +96 -0
  4. package/.claude/skills/create-project/provision.mjs +336 -0
  5. package/.claude/skills/verify-dungeon/SKILL.md +56 -25
  6. package/.claude/skills/verify-dungeon/references/counting-semantics.md +26 -6
  7. package/.claude/skills/verify-dungeon/references/report-format.md +14 -8
  8. package/.claude/skills/write-hooks/SKILL.md +156 -18
  9. package/CHANGELOG.md +211 -0
  10. package/HOOKS.md +566 -112
  11. package/README.md +28 -9
  12. package/docs/guides/1.3.0-upgrade-guide.md +262 -0
  13. package/docs/guides/1.3.1-upgrade-guide.md +328 -0
  14. package/docs/guides/1.4.0-upgrade-guide.md +130 -0
  15. package/docs/guides/1.4.1-upgrade-guide.md +125 -0
  16. package/docs/guides/1.4.3-upgrade-guide.md +200 -0
  17. package/docs/guides/1.4.4-upgrade-guide.md +95 -0
  18. package/docs/guides/1.5.0-upgrade-guide.md +426 -0
  19. package/docs/guides/1.5.1-upgrade-guide.md +178 -0
  20. package/docs/guides/1.5.3-upgrade-guide.md +117 -0
  21. package/docs/guides/1.6.0-upgrade-guide.md +204 -0
  22. package/dungeons/technical/stories-verify.js +152 -0
  23. package/dungeons/vertical/README.md +154 -0
  24. package/dungeons/vertical/ai-platform/ai-platform.js +1424 -0
  25. package/dungeons/vertical/ai-platform/ai-platform.sql +184 -0
  26. package/dungeons/vertical/ai-platform/ai-platform.verify.mjs +71 -0
  27. package/dungeons/vertical/community/community.js +1589 -0
  28. package/dungeons/vertical/community/community.sql +216 -0
  29. package/dungeons/vertical/community/community.verify.mjs +72 -0
  30. package/dungeons/vertical/crypto/crypto.js +1483 -0
  31. package/dungeons/vertical/crypto/crypto.sql +194 -0
  32. package/dungeons/vertical/crypto/crypto.verify.mjs +78 -0
  33. package/dungeons/vertical/dating/dating.js +1540 -0
  34. package/dungeons/vertical/dating/dating.sql +288 -0
  35. package/dungeons/vertical/dating/dating.verify.mjs +72 -0
  36. package/dungeons/vertical/devtools/devtools.js +1524 -0
  37. package/dungeons/vertical/devtools/devtools.sql +161 -0
  38. package/dungeons/vertical/devtools/devtools.verify.mjs +73 -0
  39. package/dungeons/vertical/ecommerce/ecommerce.js +1480 -0
  40. package/dungeons/vertical/ecommerce/ecommerce.sql +218 -0
  41. package/dungeons/vertical/ecommerce/ecommerce.verify.mjs +70 -0
  42. package/dungeons/vertical/education/education.js +1763 -0
  43. package/dungeons/vertical/education/education.sql +243 -0
  44. package/dungeons/vertical/education/education.verify.mjs +75 -0
  45. package/dungeons/vertical/fintech/fintech.js +1410 -0
  46. package/dungeons/vertical/fintech/fintech.sql +175 -0
  47. package/dungeons/vertical/fintech/fintech.verify.mjs +70 -0
  48. package/dungeons/vertical/fitness/fitness.js +1495 -0
  49. package/dungeons/vertical/fitness/fitness.sql +206 -0
  50. package/dungeons/vertical/fitness/fitness.verify.mjs +71 -0
  51. package/dungeons/vertical/food-delivery/food-delivery.js +1641 -0
  52. package/dungeons/vertical/food-delivery/food-delivery.sql +183 -0
  53. package/dungeons/vertical/food-delivery/food-delivery.verify.mjs +75 -0
  54. package/dungeons/vertical/gaming/gaming.js +1741 -0
  55. package/dungeons/vertical/gaming/gaming.sql +249 -0
  56. package/dungeons/vertical/gaming/gaming.verify.mjs +70 -0
  57. package/dungeons/vertical/healthcare/healthcare.js +1488 -0
  58. package/dungeons/vertical/healthcare/healthcare.sql +193 -0
  59. package/dungeons/vertical/healthcare/healthcare.verify.mjs +72 -0
  60. package/dungeons/vertical/insurance-application/insurance-application.js +1852 -0
  61. package/dungeons/vertical/insurance-application/insurance-application.sql +174 -0
  62. package/dungeons/vertical/insurance-application/insurance-application.verify.mjs +74 -0
  63. package/dungeons/vertical/logistics/logistics.js +1541 -0
  64. package/dungeons/vertical/logistics/logistics.sql +205 -0
  65. package/dungeons/vertical/logistics/logistics.verify.mjs +73 -0
  66. package/dungeons/vertical/marketplace/marketplace.js +1438 -0
  67. package/dungeons/vertical/marketplace/marketplace.sql +148 -0
  68. package/dungeons/vertical/marketplace/marketplace.verify.mjs +74 -0
  69. package/dungeons/vertical/{media.js → media/media.js} +557 -58
  70. package/dungeons/vertical/media/media.sql +173 -0
  71. package/dungeons/vertical/media/media.verify.mjs +74 -0
  72. package/dungeons/vertical/real-estate/real-estate.js +1629 -0
  73. package/dungeons/vertical/real-estate/real-estate.sql +167 -0
  74. package/dungeons/vertical/real-estate/real-estate.verify.mjs +75 -0
  75. package/dungeons/vertical/sass/sass.js +1733 -0
  76. package/dungeons/vertical/sass/sass.sql +173 -0
  77. package/dungeons/vertical/sass/sass.verify.mjs +74 -0
  78. package/dungeons/vertical/social/social.js +1456 -0
  79. package/dungeons/vertical/social/social.sql +160 -0
  80. package/dungeons/vertical/social/social.verify.mjs +76 -0
  81. package/dungeons/vertical/streaming/streaming.js +677 -0
  82. package/dungeons/vertical/streaming/streaming.sql +101 -0
  83. package/dungeons/vertical/streaming/streaming.verify.mjs +75 -0
  84. package/dungeons/vertical/support-desk/support-desk.js +741 -0
  85. package/dungeons/vertical/support-desk/support-desk.sql +137 -0
  86. package/dungeons/vertical/support-desk/support-desk.verify.mjs +75 -0
  87. package/dungeons/vertical/travel/travel.js +1603 -0
  88. package/dungeons/vertical/travel/travel.sql +236 -0
  89. package/dungeons/vertical/travel/travel.verify.mjs +72 -0
  90. package/index.js +2 -2
  91. package/lib/core/config-validator.js +37 -2
  92. package/lib/core/context.js +4 -2
  93. package/lib/core/dungeon-loader.js +20 -15
  94. package/lib/core/dungeon-to-json.js +22 -4
  95. package/lib/core/storage.js +3 -3
  96. package/lib/generators/funnels.js +36 -9
  97. package/lib/hook-helpers/cohort.js +53 -0
  98. package/lib/hook-helpers/index.js +8 -0
  99. package/lib/hook-helpers/mutate.js +1 -1
  100. package/lib/hook-helpers/shape.js +319 -0
  101. package/lib/hook-patterns/aggregate-per-user-by-bin.js +13 -5
  102. package/lib/hook-patterns/attributed-by-source.js +60 -58
  103. package/lib/hook-patterns/frequency-by-frequency.js +14 -4
  104. package/lib/hook-patterns/funnel-frequency-breakdown.js +24 -12
  105. package/lib/hook-patterns/index.js +1 -1
  106. package/lib/hook-patterns/time-to-convert-by-segment.js +75 -13
  107. package/lib/orchestrators/user-loop.js +41 -0
  108. package/lib/templates/story-spec.schema.json +175 -0
  109. package/lib/utils/json-evaluator.js +1 -1
  110. package/lib/utils/utils.js +5 -4
  111. package/lib/verify/coerce.js +186 -0
  112. package/lib/verify/counting.js +270 -32
  113. package/lib/verify/emulate-breakdown.js +1285 -203
  114. package/lib/verify/first-time.js +89 -0
  115. package/lib/verify/flows.js +679 -0
  116. package/lib/verify/formula.js +259 -0
  117. package/lib/verify/funnel-engine.js +697 -113
  118. package/lib/verify/index.js +32 -4
  119. package/lib/verify/sessionize.js +275 -0
  120. package/lib/verify/story-runner.js +483 -0
  121. package/lib/verify/verify-dungeon.js +59 -37
  122. package/package.json +3 -2
  123. package/scripts/extract-dungeon-schema.mjs +2 -2
  124. package/scripts/run-many.mjs +2 -2
  125. package/scripts/verify-runner.mjs +2 -2
  126. package/scripts/verify-stories.mjs +244 -0
  127. package/types.d.ts +296 -30
  128. package/dungeons/vertical/ai-platform.js +0 -818
  129. package/dungeons/vertical/community.js +0 -966
  130. package/dungeons/vertical/crypto.js +0 -895
  131. package/dungeons/vertical/dating.js +0 -830
  132. package/dungeons/vertical/devtools.js +0 -1050
  133. package/dungeons/vertical/ecommerce.js +0 -811
  134. package/dungeons/vertical/education.js +0 -964
  135. package/dungeons/vertical/fintech.js +0 -946
  136. package/dungeons/vertical/fitness.js +0 -935
  137. package/dungeons/vertical/food-delivery.js +0 -934
  138. package/dungeons/vertical/gaming.js +0 -1120
  139. package/dungeons/vertical/healthcare.js +0 -945
  140. package/dungeons/vertical/insurance-application.js +0 -1039
  141. package/dungeons/vertical/logistics.js +0 -986
  142. package/dungeons/vertical/marketplace.js +0 -1019
  143. package/dungeons/vertical/real-estate.js +0 -872
  144. package/dungeons/vertical/sass.js +0 -995
  145. package/dungeons/vertical/social.js +0 -904
  146. package/dungeons/vertical/travel.js +0 -930
@@ -1,16 +1,16 @@
1
1
  ---
2
2
  name: verify-dungeon
3
- description: Use when a dungeon's hooks need verification before pushing data to Mixpanel — runs the dungeon at full scale, validates schema integrity (catches flag-stamping), queries with DuckDB using Mixpanel-accurate counting semantics, and reports per-hook NAILED/STRONG/WEAK/NONE/INVERSE verdicts.
4
- argument-hint: [dungeon path(s), e.g. dungeons/gaming.js or dungeons/fintech.js]
3
+ description: Use when a dungeon's hooks need verification before pushing data to Mixpanel — runs the dungeon at full scale, evaluates the dungeon's `stories` export with the mechanical story runner (verify-stories.mjs), validates schema integrity (catches flag-stamping), and reports per-hook NAILED/STRONG/WEAK/NONE/INVERSE verdicts. LLM investigation is reserved for failures and duckdb-type assertions.
4
+ argument-hint: [dungeon path(s), e.g. dungeons/vertical/gaming/gaming.js or dungeons/vertical/fintech/fintech.js]
5
5
  model: claude-opus-4-6
6
6
  effort: max
7
7
  ---
8
8
 
9
9
  # Verify Dungeon
10
10
 
11
- Verify a dungeon at full scale: validate schema integrity, query the output with the Mixpanel emulator (preferred) or DuckDB, write a single consolidated `hook-results.md`.
11
+ Verify a dungeon at full scale: run the story runner (`scripts/verify-stories.mjs`) as the primary mechanical check, validate schema integrity, investigate only what the runner can't settle (failures + `duckdb`-type assertions), write a single consolidated `hook-results.md`.
12
12
 
13
- **Dungeon file(s):** `$ARGUMENTS` — single path, multiple space-separated paths, or glob pattern. In batch mode, process each dungeon sequentially through Steps 1–3, then write one consolidated report in Step 4. Use a unique `name` prefix per dungeon (e.g., `verify-fintech`, `verify-gaming`) so output files don't collide.
13
+ **Dungeon file(s):** `$ARGUMENTS` — single path, multiple space-separated paths, or glob pattern. In batch mode, process each dungeon sequentially through Steps 1–5, then write one consolidated report in Step 7. Use a unique `name` prefix per dungeon (e.g., `verify-fintech`, `verify-gaming`) so output files don't collide.
14
14
 
15
15
  ## Reference files
16
16
 
@@ -28,9 +28,10 @@ Also: `HOOKS.md` (recipe encyclopedia) and `types.d.ts` (source of truth for hoo
28
28
 
29
29
  Read the dungeon at `$ARGUMENTS`. If it's a bare filename (no `/`), check `dungeons/`. Find:
30
30
 
31
- 1. **The `hook:` function** — read the full body
32
- 2. **The documentation comment block** describing engineered patterns
33
- 3. **Module-level closure state** (Map / Set / tracking variables defined outside the hook function but used inside)
31
+ 1. **The `stories` named export** — the machine-checkable contract (see `lib/templates/story-spec.schema.json` and the `DungeonStory` typedef in `types.d.ts`). If present, the story runner in Step 3 does the heavy lifting. If absent (legacy dungeon), the full per-hook flow in Step 5 applies to every hook.
32
+ 2. **The `hook:` function** read the full body
33
+ 3. **The documentation comment block** describing engineered patterns
34
+ 4. **Module-level closure state** (Map / Set / tracking variables defined outside the hook function but used inside)
34
35
 
35
36
  For each hook/pattern, catalog:
36
37
  - Hook number and name (e.g., "Hook #1: Compass users have 3x quest completion")
@@ -52,7 +53,7 @@ node scripts/verify-runner.mjs <dungeon-path> <run-name>
52
53
 
53
54
  Example:
54
55
  ```bash
55
- node scripts/verify-runner.mjs dungeons/vertical/gaming.js verify-gaming
56
+ node scripts/verify-runner.mjs dungeons/vertical/gaming/gaming.js verify-gaming
56
57
  ```
57
58
 
58
59
  Full-fidelity runs can take minutes (50K+ user dungeons). Plan accordingly — kick off the run, do other reading, return when the file lands. If a run takes longer than your budget, report that as a finding ("dungeon too large to verify in current session") rather than falling back to `--small`.
@@ -65,7 +66,27 @@ Full-fidelity runs can take minutes (50K+ user dungeons). Plan accordingly — k
65
66
 
66
67
  For batched output (>2M events), the runner writes `<run-name>-EVENTS-part-*.json`. See [sql-recipes.md "Multi-part EVENTS files"](references/sql-recipes.md#multi-part-events-files-batch-mode).
67
68
 
68
- ### Step 3: Validate schema (BEFORE per-hook checks)
69
+ ### Step 3: Run the story runner (primary mechanical check)
70
+
71
+ For any dungeon with a `stories` export, the runner — not the LLM — computes the verdicts:
72
+
73
+ ```bash
74
+ node scripts/verify-stories.mjs <dungeon-path> --data-prefix <run-name>
75
+ node scripts/verify-stories.mjs <dungeon-path> --data-prefix <run-name> --json # machine-readable, for hook-results.md
76
+ ```
77
+
78
+ The runner streams the shards from Step 2, evaluates every assertion against its declared `target` / `floor` / `minCohort`, substitutes `{{PREFIX}}` into `duckdb`-type assertions and shells them out, enforces hook coverage (every numbered hook in the HOOK STORIES comment block must be targeted by at least one story), and prints a five-tier verdict table. Exit code is non-zero when any story lands WEAK / NONE / INVERSE or coverage is incomplete.
79
+
80
+ **Verdicts are computed, not judged.** They include the population floor: a cohort smaller than the assertion's `minCohort` caps at WEAK — a 12-user cohort can no longer score NAILED regardless of how clean its ratio looks. See [report-format.md "Verdict criteria"](references/report-format.md#verdict-criteria-5-tier) for the mechanical definitions.
81
+
82
+ **What the LLM investigates after this step — and nothing else:**
83
+ 1. **Stories below STRONG** (WEAK / NONE / INVERSE) — root-cause via Step 5's decision table. A miss means fixing the hook or the assertion's derivation, never relaxing the number to match output.
84
+ 2. **`duckdb`-type assertions** — the runner executes them but can't interpret bespoke shapes; sanity-check their output against the story narrative.
85
+ 3. **Dungeons without a `stories` export** — legacy fallback: full per-hook flow (Step 5) for every documented hook.
86
+
87
+ Do NOT re-derive verdicts the runner already computed as passing. `hook-results.md` (Step 7) renders the runner's JSON.
88
+
89
+ ### Step 4: Validate schema (BEFORE per-hook checks)
69
90
 
70
91
  Catches hooks that introduce undeclared columns (flag stamping). For each unique event type, compare actual columns against config-declared properties. See [sql-recipes.md "Schema validation queries"](references/sql-recipes.md#schema-validation-queries) for the SQL and the expected-schema source table.
71
92
 
@@ -75,7 +96,9 @@ Catches hooks that introduce undeclared columns (flag stamping). For each unique
75
96
 
76
97
  If any event type has SCHEMA-FAIL, flag it prominently in the report header with specific remediation: which hook line adds the property and how to remove it while preserving the intended pattern.
77
98
 
78
- ### Step 4: Verify each hook
99
+ ### Step 5: Investigate failures (and legacy no-stories dungeons)
100
+
101
+ Applies only to the investigation targets from Step 3 — failing stories, `duckdb`-type assertions, and dungeons with no `stories` export.
79
102
 
80
103
  **Decision: emulator vs DuckDB**
81
104
 
@@ -84,14 +107,22 @@ If any event type has SCHEMA-FAIL, flag it prominently in the report header with
84
107
  | Funnel completion / step conversion | `emulateBreakdown({type: 'funnelFrequency'})` |
85
108
  | Insights frequency-distribution | `emulateBreakdown({type: 'frequencyByFrequency'})` |
86
109
  | Avg(prop) by per-user count(B) | `emulateBreakdown({type: 'aggregatePerUser'})` |
87
- | Funnel TTC by segment | `emulateBreakdown({type: 'timeToConvert'})` |
110
+ | Funnel TTC by segment (steps-based or funnel-based) | `emulateBreakdown({type: 'timeToConvert'})` |
88
111
  | First/last touch attribution | `emulateBreakdown({type: 'attributedBy'})` |
89
- | Retention curves | `emulateBreakdown({type: 'retention'})` |
112
+ | Retention curves — birth or compounded | `emulateBreakdown({type: 'retention'})` (`compounded: true` for "DAU coming back") |
90
113
  | Per-session metrics | `emulateBreakdown({type: 'sessionMetrics'})` |
114
+ | Lifecycle (new / retained / resurrected / dormant) | `emulateBreakdown({type: 'lifecycle'})` |
115
+ | Flows / top paths (Sankey) | `emulateBreakdown({type: 'topPaths'})` |
116
+ | Event totals segmented by property | `emulateBreakdown({type: 'eventBreakdown'})` (`countType: 'general' \| 'sessions'`) |
117
+ | Uniques per segment | `emulateBreakdown({type: 'uniques'})` |
118
+ | COUNT DISTINCT of a property + top values | `emulateBreakdown({type: 'distinctCount'})` |
119
+ | Ratio / composite metrics (conversion %, ARPU, blends) | `evaluateFormula` (`lib/verify/formula.js`) over emulator rows |
91
120
  | Schema integrity / column coverage | DuckDB |
92
121
  | Identity-model invariants | DuckDB |
93
122
  | Experiment invariants | DuckDB |
94
- | Bespoke time-window patterns | DuckDB |
123
+ | True bespokes (no emulator analysis fits) | DuckDB |
124
+
125
+ The emulator now covers lifecycle, flows, sessions, event breakdowns, formulas, and compounded retention — DuckDB's remit is schema / identity / experiment invariants plus true bespoke shapes. If a "bespoke" check is really a funnel, frequency, path, or breakdown in disguise, it belongs in the emulator.
95
126
 
96
127
  **Hand-written DuckDB funnel SQL diverges from Mixpanel — never hand-roll.** If you find yourself writing `WITH step1 AS ..., step2 AS ...` for a funnel, STOP — use `emulateBreakdown` with `funnelFrequency` instead.
97
128
 
@@ -107,18 +138,18 @@ For emulator details, identity-model dungeons (must pass `profiles`), and time-s
107
138
  **Everything about a dungeon lives in its folder.** When the dungeon being
108
139
  verified is a user dungeon at `dungeons/user/<name>/<name>.js`, write ALL
109
140
  generated artifacts into `dungeons/user/<name>/`:
110
- - `hook-results.md` (Step 6)
111
- - `hook-query-log.txt` (Step 5)
112
- - `<name>-verifications.sql` (Step 6b)
141
+ - `hook-results.md` (Step 7)
142
+ - `hook-query-log.txt` (Step 6)
143
+ - `<name>-verifications.sql` (Step 7b)
113
144
 
114
145
  The ONLY exception is the throwaway verification data the run writes to
115
146
  `./data/` (`verify-*` event/user files) — that stays in `./data/` and is
116
- deleted in Step 7.
147
+ deleted in Step 8.
117
148
 
118
149
  For non-user dungeons (technical/vertical) or batch runs across many dungeons,
119
150
  fall back to `./research/` for `hook-results.md` / `hook-query-log.txt`.
120
151
 
121
- ### Step 5: Stash query log
152
+ ### Step 6: Stash query log
122
153
 
123
154
  Write every DuckDB query execution to `hook-query-log.txt`:
124
155
  - **User dungeon:** always write to `dungeons/user/<name>/hook-query-log.txt`.
@@ -126,21 +157,21 @@ Write every DuckDB query execution to `hook-query-log.txt`:
126
157
 
127
158
  Format and conventions: see [report-format.md "Query log format"](references/report-format.md#query-log-format).
128
159
 
129
- ### Step 6: Write `hook-results.md`
160
+ ### Step 7: Write `hook-results.md`
130
161
 
131
- Write to `dungeons/user/<name>/hook-results.md` for a user dungeon, else `./research/hook-results.md`. Use the templates in [report-format.md](references/report-format.md):
162
+ Write to `dungeons/user/<name>/hook-results.md` for a user dungeon, else `./research/hook-results.md`. For story-backed dungeons, **the report renders the runner's JSON** (`verify-stories.mjs --json`): the hook summary table comes straight from the runner's per-story verdicts (story id, hook, archetype, observed vs target, verdict), and detailed blocks exist only for the Step-3 investigation targets. Use the templates in [report-format.md](references/report-format.md):
132
163
  - Single-dungeon report structure
133
164
  - Multi-dungeon report structure (when batch mode)
134
165
  - Per-hook detail block
135
- - Verdict criteria (5-tier)
166
+ - Verdict criteria (5-tier, mechanical)
136
167
 
137
168
  **Order failures first** within each dungeon section: INVERSE → NONE → WEAK → STRONG → NAILED. Sort the summary table the same way. Actionable issues at the top.
138
169
 
139
- ### Step 6b: Write verification SQL (mandatory for user dungeons)
170
+ ### Step 7b: Write verification SQL (mandatory for user dungeons)
140
171
 
141
- When verifying a dungeon in `dungeons/user/`, also write a standalone DuckDB SQL file alongside the dungeon in its folder at `dungeons/user/<name>/<name>-verifications.sql`. Vertical dungeons already have their SQL in `verification/verticals/`. Format: see [report-format.md "Verification SQL file"](references/report-format.md#verification-sql-file-mandatory-for-user-dungeons).
172
+ When verifying a dungeon in `dungeons/user/`, also write a standalone DuckDB SQL file alongside the dungeon in its folder at `dungeons/user/<name>/<name>-verifications.sql`. Vertical dungeons already have their SQL co-located at `dungeons/vertical/<name>/<name>.sql`. Format: see [report-format.md "Verification SQL file"](references/report-format.md#verification-sql-file-mandatory-for-user-dungeons).
142
173
 
143
- ### Step 7: Cleanup
174
+ ### Step 8: Cleanup
144
175
 
145
176
  ```bash
146
177
  rm -f ./data/verify-* ./verify-*
@@ -171,7 +202,7 @@ Tell the user:
171
202
  1. Report path: `dungeons/user/<name>/hook-results.md` (user dungeon) or `./research/hook-results.md`
172
203
  2. Verification SQL path (for user dungeons): `dungeons/user/<name>/<name>-verifications.sql`
173
204
  3. Query log path (if written): alongside the report (`dungeons/user/<name>/hook-query-log.txt`, else `./research/hook-query-log.txt`)
174
- 4. Pass/weak/fail counts (per dungeon if batch mode)
205
+ 4. Verdict counts from the story runner (per dungeon if batch mode), plus which stories needed LLM investigation
175
206
  5. One-line summary of the most interesting finding
176
207
 
177
208
  If hooks failed, note that `hook-results.md` can be used as context for fixing them: "read hook-results.md and fix the failing hooks in <dungeon-file>".
@@ -14,13 +14,10 @@ Mixpanel does NOT count the way naive SQL does. The verifier (and any DuckDB que
14
14
  | Sessions | 3-trigger split: timeout `>`, max duration `>`, day-idx change (`session_query.cpp:906-911`) | Trust pre-stamped `session_id`; group by `(user, session_id)` |
15
15
  | Retention | Birth-anchored, ms-strict gate (default `birth_can_retain=false` → `<`), bucketed by `floor((ret−birth)/unit)` (`retention_query.cpp:1097-1109,1228-1231`) | Use `emulateBreakdown` with `retention` |
16
16
 
17
- **Known divergences from Mixpanel C++** (1.5.1):
17
+ **Known divergences from Mixpanel C++** (1.6.0):
18
18
  - `countDistinctPeriods` default = `algorithm: 'calendar'` (UTC bucket).
19
19
  Mixpanel C++ (`addiction_query.cpp:359`) uses ROLLING window. Pass
20
20
  `algorithm: 'rolling'` for exact Frequency-Distribution parity.
21
- - COMPOUNDED retention is NOT implemented — verifier silently ignores
22
- `compounded: true`. Use DuckDB or query Mixpanel directly for "DAU
23
- coming back" reports.
24
21
  - Touchpoint sampling: generator stamps uniform-random across user
25
22
  lifetime; verifier reads last-N before conversion (matches C++).
26
23
  For users with ≤10 attribution events lifetime, no divergence.
@@ -31,7 +28,7 @@ Full rules: see [HOOKS.md Section 2](../../../../HOOKS.md#2-how-mixpanel-counts-
31
28
 
32
29
  ## When to use the emulator vs DuckDB
33
30
 
34
- The emulator (`emulateBreakdown` from `@ak--47/dungeon-master/verify`) implements Mixpanel's rules natively. **ALWAYS use the emulator for funnel, frequency, aggregate, TTC, and attribution patterns.** Hand-written DuckDB queries for these pattern types diverge from what Mixpanel shows in reports — even when they look correct.
31
+ The emulator (`emulateBreakdown` from `@ak--47/dungeon-master/verify`) implements Mixpanel's rules natively. **ALWAYS use the emulator for funnel, frequency, aggregate, TTC, attribution, retention, lifecycle, flows, sessions, and event-breakdown patterns.** Hand-written DuckDB queries for these pattern types diverge from what Mixpanel shows in reports — even when they look correct.
35
32
 
36
33
  Use DuckDB ONLY for:
37
34
  - Schema integrity checks (column coverage, flag detection)
@@ -51,10 +48,33 @@ If you find yourself writing `WITH step1 AS ..., step2 AS ...` for a funnel, STO
51
48
  | Funnel TTC by user property | `timeToConvert` | "Trial users take 4× longer than enterprise" |
52
49
  | First/last touch attribution | `attributedBy` | "Conversions by Source" |
53
50
  | Birth retention curves | `retention` | "Sign Up → Login on day N" — requires `cohortEvent`, `returnEvent`, `dayBuckets` |
51
+ | Compounded retention ("DAU coming back") | `retention` + `compounded: true` | Sets `returnEvent := cohortEvent` internally; throws on a conflicting `returnEvent` |
54
52
  | Per-session metrics | `sessionMetrics` | Count / duration / events distributions per session |
53
+ | Lifecycle states (new / retained / resurrected / dormant) | `lifecycle` | "Resurrection wave in week N" — per-period user-state classification |
54
+ | Flows / top paths | `topPaths` | "What do users do after X" — Sankey top-path shares |
55
+ | Event totals segmented by property | `eventBreakdown` | "Purchases by plan tier" — `countType: 'general' \| 'sessions'`; list props fan out per item |
56
+ | Uniques per segment | `uniques` | "Unique users by plan tier" |
57
+ | Distinct property values | `distinctCount` | "How many distinct SKUs" — see below |
58
+ | Ratio / composite metrics | `evaluateFormula` (`lib/verify/formula.js`) | Conversion %, ARPU, any PEMDAS formula over emulator series |
55
59
 
56
60
  Cross-cutting on EVERY type: `timeBucket: 'day' | 'week' | 'month'` partitions events into UTC buckets and emits one row per period.
57
61
 
62
+ ### `distinctCount`
63
+
64
+ Counts distinct values of a flat event property, Mixpanel COUNT_DISTINCT-style, with a top-values breakdown:
65
+
66
+ ```js
67
+ emulateBreakdown(events, {
68
+ type: 'distinctCount',
69
+ property: 'sku', // required — flat property name (dot-paths not supported)
70
+ event: 'Purchase', // optional — restrict to one event type; omit = all events
71
+ topN: 25, // optional — top values by count (default 25)
72
+ });
73
+ // → [{ distinct_count: 143, top_values: [{ value: 'SKU-001', count: 812 }, ...] }]
74
+ ```
75
+
76
+ Single-row result. Use it for catalog-breadth stories ("power sellers list 5× more distinct SKUs") where `uniques` / `eventBreakdown` answer a different question (users or totals per bucket, not distinct values of the property itself).
77
+
58
78
  Quick emulator script:
59
79
 
60
80
  ```js
@@ -98,7 +118,7 @@ For CI-style assertions, use `verifyDungeon` with a checks array; see `tests/e2e
98
118
  | `readFileSync` ENOMEM on shards >500MB | Node string cap at ~512MB | Stream-load with `readline.createInterface` over `data/PREFIX-EVENTS*.json` glob |
99
119
  | Hook reads `e.event === 'login'` but cohort empty | `login` is a funnel-step event auto-promoted to `isStrictEvent: true` | Add `isStrictEvent: false` to the event config to keep standalone occurrences |
100
120
 
101
- When writing per-dungeon verify scripts, follow the template in HOOKS.md §9.9. Reference proofs for all 20 vertical dungeons live at `verification/verticals/` — consult them as exemplars before authoring a new one.
121
+ When writing per-dungeon verify scripts, follow the template in HOOKS.md §9.9. Reference proofs for all 22 vertical dungeons live at `dungeons/vertical/<name>/` — consult them as exemplars before authoring a new one.
102
122
 
103
123
  **Coverage discipline:** count documented hooks in the dungeon's top-level comment block; count `check()` calls in your verify script; the two MUST match. A "NAILED 7/7" claim against an 11-hook dungeon is misleading — either add the missing checks or document the limitation explicitly in the verify script and status file.
104
124
 
@@ -4,13 +4,17 @@ Templates and conventions for writing `hook-results.md` and per-dungeon verifica
4
4
 
5
5
  ## Verdict criteria (5-tier)
6
6
 
7
- - **NAILED** — Within 10% of expected value/ratio. Direction correct, magnitude precise. The story reads exactly as documented.
8
- - **STRONG** — Within 25% of expected. Direction correct, clearly visible. An analyst would find this pattern immediately.
9
- - **WEAK** — Within 50% of expected. Directionally correct but magnitude is off, OR sample size is too small to be conclusive.
10
- - **NONE** — No statistically meaningful difference between cohorts. The hook has no observable effect.
11
- - **INVERSE** — Effect goes the opposite direction from intended. The story is backwards.
7
+ Verdicts are **mechanical** — computed by `scripts/verify-stories.mjs` from each story assertion's declared `target` / `floor` / `minCohort` (see the `StoryVerdict` typedef in `types.d.ts`), not eyeballed percentages:
12
8
 
13
- NAILED and STRONG are passing verdicts. WEAK, NONE, and INVERSE are failing verdicts that require investigation.
9
+ - **NAILED** observed within ±10% of `target`.
10
+ - **STRONG** — passes `floor` (or `target` when no floor is declared).
11
+ - **WEAK** — fails `floor` but effect direction is correct, **or** the selected cohort is smaller than `minCohort`. The population floor is a hard cap: a 12-user cohort can never score NAILED, no matter how clean its ratio.
12
+ - **NONE** — no measurable effect, or the selection matched no rows.
13
+ - **INVERSE** — effect direction is opposite the assertion.
14
+
15
+ Story verdict = worst assertion verdict. NAILED and STRONG are passing; WEAK, NONE, and INVERSE fail and require investigation.
16
+
17
+ Hand-assigned verdicts appear only in the legacy no-stories fallback and MUST follow the same definitions: derive a target from the hook's knob constants, compute the band the observed value lands in, and state the derivation in the detail block — never assign a tier by feel.
14
18
 
15
19
  ## Ordering: failures first
16
20
 
@@ -25,6 +29,8 @@ The summary table should also be sorted this way (INVERSE → NONE → WEAK →
25
29
 
26
30
  ## Single-dungeon report structure
27
31
 
32
+ For story-backed dungeons, `hook-results.md` **renders the runner's JSON**: run `verify-stories.mjs --json` and build the Hook Summary table directly from its per-story records (story id, hook number, archetype, observed vs target per assertion, computed verdict). Do not recompute verdicts the runner already settled. Detailed Results blocks exist only for stories below STRONG, `duckdb`-type assertions, and legacy no-stories hooks.
33
+
28
34
  ```markdown
29
35
  # Dungeon Verification Report
30
36
 
@@ -191,7 +197,7 @@ grep "^DUNGEON:" research/hook-query-log.txt # list of dungeons querie
191
197
 
192
198
  When verifying a dungeon in `dungeons/user/`, write a standalone DuckDB SQL file alongside the dungeon in its folder at `dungeons/user/<name>/<name>-verifications.sql`. This file is the reproducible verification artifact — anyone can re-run it against fresh data.
193
199
 
194
- Follow the format in `verification/verticals/`:
200
+ Follow the format in `dungeons/vertical/<name>/`:
195
201
 
196
202
  ```sql
197
203
  -- ============================================================================
@@ -211,4 +217,4 @@ Follow the format in `verification/verticals/`:
211
217
 
212
218
  Each query block includes the pattern description, observed result, and verdict as SQL comments. This makes the file self-documenting and grep-friendly.
213
219
 
214
- **This step is mandatory for user dungeons.** Vertical dungeons already have their SQL in `verification/verticals/`. User dungeons keep theirs co-located with the dungeon file.
220
+ **This step is mandatory for user dungeons.** Vertical dungeons already have their SQL co-located at `dungeons/vertical/<name>/<name>.sql`. User dungeons keep theirs co-located with the dungeon file.
@@ -20,7 +20,11 @@ already complete (produced by `create-dungeon`). After writing, hand off to
20
20
  In scope:
21
21
  - The `hook: function(record, type, meta) { ... }` body
22
22
  - Documentation comments above the hook explaining each engineered pattern,
23
- including a reference Mixpanel report block per pattern
23
+ including a reference Mixpanel report block per pattern and the mandatory
24
+ EXPECTED METRICS SUMMARY table
25
+ - The `stories` named export — one machine-checkable story per engineered
26
+ pattern (see "Stories export" below). A hook without stories is
27
+ unverifiable; this skill is not done until the stories exist.
24
28
 
25
29
  Out of scope:
26
30
  - Schema changes (events, properties, funnels, superProps, userProps).
@@ -29,6 +33,18 @@ Out of scope:
29
33
  - New top-level config knobs.
30
34
  - Removing the `hook: function...` body to start over with a new schema.
31
35
 
36
+ **Before writing a hook, ask: is this trend structural?** Between-path
37
+ comparisons (path X converts worse / slower than path Y, detour-takers drop
38
+ off, mix shift drags the blended rate) are better architected as initial
39
+ conditions — duplicate funnels with swapped steps/props/`conversionRate`/
40
+ `timeToConvert`/`weight` (see the "Structural trend engineering" section in
41
+ `create-dungeon`). If a story reduces to structure, recommend the funnel
42
+ change back to the schema instead of writing a hook to fight the engine —
43
+ the knob IS the expected value, which makes the story band knob-derivable
44
+ (NAILED-capable) instead of confounded (STRONG-capped). Hooks are for
45
+ within-cohort behavior: segments doing more/less of something over time,
46
+ property values that differ by cohort, injected bursts, lifecycle waves.
47
+
32
48
  ## Reference reading
33
49
 
34
50
  - `lib/hook-helpers/index.js` — atoms (cohort, mutate, timing, inject,
@@ -36,8 +52,11 @@ Out of scope:
36
52
  - `lib/hook-patterns/index.js` — high-level recipes (one per Mixpanel
37
53
  analysis type).
38
54
  - `lib/verify/emulate-breakdown.js` — what `verify-dungeon` will check.
39
- - `dungeons/user/my-buddy/my-buddy.js` reference dungeon using a mix of atoms
40
- and hand-rolled logic.
55
+ - `lib/templates/story-spec.schema.json` + `DungeonStory` in `types.d.ts`
56
+ the story-spec grammar the `stories` export must follow.
57
+ - `dungeons/vertical/ecommerce/ecommerce.js` — reference dungeon using a mix
58
+ of atoms and hand-rolled logic, with the mandatory EXPECTED METRICS SUMMARY
59
+ table and a full `stories` export.
41
60
  - `dungeons/technical/pattern-*.js` — five minimal pattern fixtures, one per
42
61
  recipe.
43
62
  - `HOOKS.md` — encyclopedia of hook recipes organized by story pattern. Contains
@@ -134,6 +153,10 @@ if (type === 'funnel-post' && meta.experiment) {
134
153
  | inject | `injectOnNewDays(events, eventName, targetDistinctDays)` | **Cohort-only.** Spreads injections across previously-empty days. Use for cohort-conditional active-day boosts; for global active-day shape, use `Dungeon.avgActiveDaysPerUser` config knob. |
135
154
  | identity | `isPreAuthEvent(event, authTime)` | Standalone variant of meta.isPreAuth |
136
155
  | identity | `splitByAuth(events, authTime)` | { preAuth, postAuth, stitch } partition |
156
+ | cohort | `hashCohort(id, pct)` | Deterministic pct% cohort (0–100 scale). **Use this first for hidden cohorts** — replaces ad-hoc `charCodeAt % N`. When one dungeon needs several NON-overlapping cohorts, gate on disjoint `hashFloat(uid)` bands instead (e.g. `[0, 0.45)`, `[0.45, 0.70)`) |
157
+ | shape | `applyLifecycleWave(events, uid, opts)` | Dormancy window + resurrection burst for Lifecycle reports. When-to-use: the story is "users go quiet, then come back". Gap discipline: ONE stray value moment inside the window destroys the Resurrected read — size `dormantDays` to cover ≥2 whole lifecycle periods, keep the window inside the user's lifespan |
158
+ | shape | `applyPathBias(events, uid, opts)` | Inject a Flows path after the user's FIRST anchor occurrence. When-to-use: the story is "X% of users take this route". `share` is a 0–1 FRACTION (not `hashCohort`'s pct scale); needs ~≥0.20–0.25 to survive Sankey top-3-per-level pruning; per-step gaps clamped ≥1s so ordering survives |
159
+ | shape | `applySessionShape(events, uid, opts)` | Retime the whole stream into `sessionsPerWeek` clusters of `sessionMinutes`. When-to-use: session-duration/cadence stories (sessionMetrics reads). Retiming ONLY — no adds/drops; intra-session gaps stay <28min, inter-session >30min, no cluster crosses UTC midnight. Combine with `hashCohort` for per-role shapes; call BEFORE `applyPathBias` so injected paths keep their own tight gaps |
137
160
 
138
161
  ### Hook anti-patterns
139
162
 
@@ -214,17 +237,51 @@ on undeclared entries.
214
237
  Higher-level recipes. Each maps to ONE Mixpanel analysis the verify-dungeon
215
238
  emulator can re-derive.
216
239
 
217
- | Pattern | Mixpanel analysis | Hook type |
218
- |---------|-------------------|-----------|
219
- | `applyFrequencyByFrequency` | Insights — count(A) by per-user count(B) | everything |
220
- | `applyFunnelFrequencyBreakdown` | Funnels — completion by per-user count(X) | funnel-post |
221
- | `applyAggregateByBin` | Insights — avg(prop X) by per-user count(B) | everything |
222
- | `applyTTCBySegment` | Funnel TTC — broken down by user-property segment | funnel-post |
223
- | `applyAttributedBySource` | Conversions by Source (first/last touch) | everything |
240
+ | Pattern | Mixpanel analysis | Hook type | Caveat (HOOKS.md) |
241
+ |---------|-------------------|-----------|-------------------|
242
+ | `applyFrequencyByFrequency` | Insights — count(A) by per-user count(B) | everything | `binBy` defaults to `'distinctDays'` (v1.6) — bins match Mixpanel's per-user distinct-day counting, not raw event totals |
243
+ | `applyFunnelFrequencyBreakdown` | Funnels — completion by per-user count(X) | funnel-post | When funnels share a step prefix, restrict scaling to the target funnel — scaling every instance lets first-occurrence funnel evaluation assemble chains across unscaled instances and the ratio never reaches the report |
244
+ | `applyAggregateByBin` | Insights — avg(prop X) by per-user count(B) | everything | Same `binBy: 'distinctDays'` default as above |
245
+ | `applyTTCBySegmentV2` | Funnel TTC — broken down by user-property segment | everything | v1 (`applyTTCBySegment`, funnel-post) is **deprecated**: Mixpanel TTC reads each step's FIRST occurrence per user, so per-run gap scaling only reaches the report on `isFirstFunnel` runs. V2 finds the greedy first sequence (`findFirstSequence`) and scales that |
246
+ | `applyAttributedBySource` | Conversions by Source (first/last touch) | everything | OVERWRITES the engine-stamped touch the chosen model reads; never stamps UTMs onto unstamped events (would blow the `maxTouchpointsPerUser` cap and land outside the last-10 lookback) |
224
247
 
225
248
  Use a pattern when the trend matches its analysis 1:1. Drop down to atoms when
226
249
  the trend is bespoke or composite.
227
250
 
251
+ ### New story archetypes (v1.6) — design rules per report family
252
+
253
+ Four archetypes joined the story-spec enum in v1.6. Each has a
254
+ non-negotiable design rule learned the hard way:
255
+
256
+ - **`lifecycle-wave`** (`applyLifecycleWave`) — GAP DISCIPLINE. Mixpanel's
257
+ "dormant" state is an `EqualTo 0` filter over the whole period: one stray
258
+ value-moment event inside the dormancy window (including events OTHER hooks
259
+ injected earlier in the same `everything` pass) flips the user out of
260
+ Resurrected. Run the wave AFTER every injecting hook, size `dormantDays` to
261
+ ≥2 lifecycle periods, and keep `dormantFromDay + dormantDays` inside the
262
+ user's lifespan (the future-time guard eats bursts past dataset end).
263
+ - **`path-share`** (`applyPathBias`) — FIRST-FLOW ANCHORING. Flows' unique
264
+ counting reads only each user's FIRST flow past the anchor, so the injected
265
+ path must follow the FIRST anchor occurrence (the atom does this; don't
266
+ hand-roll a later anchor). Sankey prunes to the top ~3 branches per level:
267
+ an engineered branch below ~20–25% share silently disappears from the
268
+ visualization even though the data is there. Label-only path reads can
269
+ INVERT when a busier cohort glues extra visible events between path steps —
270
+ assert the share on the cohort you engineered, not globally.
271
+ - **`session-shape`** (`applySessionShape`) — 30-MIN STRADDLING + MIDNIGHT
272
+ RULE. Mixpanel derives sessions with a 30-min idle timeout and splits at
273
+ UTC midnight. Engineered cadences must keep intra-session gaps clearly
274
+ UNDER 30min and inter-session gaps clearly OVER it — a gap that straddles
275
+ the timeout makes session counts jitter across runs. Never let an
276
+ engineered session cross UTC midnight (the day split cuts it in two). The
277
+ atom guarantees all three; hand-rolled retiming must too.
278
+ - **`composition-drift`** — the breakdown's SHARE of a segment moves over
279
+ time while totals stay flat (e.g. plan-mix shifts toward premium). Engineer
280
+ by flipping an existing property value on a date-gated cohort, never by
281
+ changing volumes — volume changes read as `temporal-inflection` instead.
282
+ Assert with a `timeBucket` breakdown comparing first-window vs last-window
283
+ share.
284
+
228
285
  ## Anti-flag-stamping rule (HARD WALL)
229
286
 
230
287
  Hooks MUST NOT add new properties to records. The schema (config) defines what
@@ -318,6 +375,71 @@ checks against and what consumers read to understand the dataset.
318
375
  * Expected ratio: bin>=15 / bin<5 ≈ 3x (within ±15%)
319
376
  ```
320
377
 
378
+ ### EXPECTED METRICS SUMMARY table (MANDATORY)
379
+
380
+ The doc block MUST end with an EXPECTED METRICS SUMMARY table — one row per
381
+ verifiable read, with the DERIVATION of each expected number from the hook's
382
+ knob constants (never a bare number someone has to trust). Follow the style
383
+ in `dungeons/vertical/ecommerce/ecommerce.js`:
384
+
385
+ ```
386
+ * EXPECTED METRICS SUMMARY
387
+ * ============================================================================
388
+ *
389
+ * Hook | Metric | Derivation | Expected | Measured (full)
390
+ * -----|---------------------------------|---------------------|----------|----------------
391
+ * H2 | avg watchTimeSec post/pre | 1.52/0.48 | 3.17x | 3.19x
392
+ * H6 | sweet/over avg cart item amount | SWEET_CART_BOOST | 1.25x | 1.233x
393
+ * H9 | dark/light checkouts per user | 20/13 diluted | ~1.48x | 1.412x
394
+ * ============================================================================
395
+ ```
396
+
397
+ Fill the Measured column from the verification run (a reduced-scale iteration
398
+ number is fine if labeled). "Diluted" derivations must say WHAT dilutes
399
+ (organic events, cohort mixing) — the Derivation column is the anchor
400
+ `/verify-dungeon` uses when a read misses.
401
+
402
+ ## Stories export (MANDATORY — the machine contract)
403
+
404
+ Every engineered pattern ships with one story in a `stories` named export —
405
+ the machine-checkable form of the doc block. `scripts/verify-stories.mjs`
406
+ evaluates them; `/verify-dungeon` runs it as step 1. Grammar:
407
+ `lib/templates/story-spec.schema.json` and `DungeonStory` in `types.d.ts`.
408
+
409
+ ```js
410
+ export const stories = [
411
+ {
412
+ id: "H1-power-buyers",
413
+ hook: "H1",
414
+ archetype: "frequency-sweet-spot",
415
+ narrative: "Users with 15+ browse days buy 3x as often as light browsers.",
416
+ mixpanelReport: { type: "Insights", event: "Purchase", breakdown: "per-user count of Browse" },
417
+ assertions: [{
418
+ breakdown: { type: "frequency", event: "Purchase", cohortEvent: "Browse", bins: [5, 15] },
419
+ select: { hi: { where: { bin: ">=15" } }, lo: { where: { bin: "<5" } } },
420
+ expect: { metric: "hi.avg / lo.avg", op: ">=", target: POWER_BUYER_MULT, floor: POWER_BUYER_MULT * 0.8 },
421
+ minCohort: 200,
422
+ }],
423
+ },
424
+ ];
425
+ ```
426
+
427
+ Rules:
428
+
429
+ - **Thresholds derive from the knob you just wrote.** Export the hook's knob
430
+ constants (`const POWER_BUYER_MULT = 3`) and compute `target` from them —
431
+ never paste the number twice. If the read is diluted (organic mixing),
432
+ derive the dilution too and say so in a comment.
433
+ - `floor` must itself be derived (e.g. `target * 0.8`) — never hand-tuned to
434
+ a run. A missed assertion means fixing the hook or the derivation, never
435
+ relaxing the number to match output.
436
+ - Set `minCohort` from the cohort math (share × numUsers × ~0.5 safety) so
437
+ reduced-scale runs cap at WEAK instead of passing on noise.
438
+ - One story per pattern; story `hook` matches the doc-block numbering (`H3`).
439
+ - The `assert` function escape hatch is discouraged — each use needs a
440
+ comment saying why the declarative `expect` grammar can't express it.
441
+ - Stories are JS-dungeon-only (`dungeon-to-json` drops them).
442
+
321
443
  ## Hook Ordering Within `everything`
322
444
 
323
445
  The order of operations inside the everything hook matters when hooks interact:
@@ -466,22 +588,37 @@ cohort.
466
588
  2. Translate the user's story description into 3–5 engineered patterns.
467
589
  Consult `HOOKS.md` for recipe ideas that match the user's story. Each recipe
468
590
  includes the hook type, code snippet, and Mixpanel report format.
469
- 3. For each pattern:
591
+ 3. **Calibrate thresholds against the real distribution.** Before choosing
592
+ any "N+ events" gate or cohort cutoff, generate a small run and query the
593
+ actual per-user distribution (see "Threshold Calibration" above for the
594
+ query). Set gates at ~the 80th percentile of what the data shows — a
595
+ threshold picked from intuition is the most common cause of empty cohorts.
596
+ ```bash
597
+ node scripts/verify-runner.mjs <dungeon> calib --small
598
+ ```
599
+ 4. For each pattern:
470
600
  - Pick a pattern from `lib/hook-patterns/` if it fits the analysis 1:1.
471
601
  - Otherwise compose atoms from `lib/hook-helpers/`.
472
602
  - Document the pattern in a comment block (Mixpanel report instructions).
473
- 4. Write the `hook` function, importing atoms/patterns at the top of the file.
474
- 5. Smoke-test:
603
+ 5. Write the `hook` function, importing atoms/patterns at the top of the
604
+ file. Finish the doc block with the EXPECTED METRICS SUMMARY table.
605
+ 6. Write the `stories` export — one story per pattern, `target`/`floor`
606
+ derived from the exported knob constants (see "Stories export" above).
607
+ 7. Smoke-test generation, then evaluate the stories:
475
608
  ```bash
476
609
  node scripts/verify-runner.mjs <dungeon> verify-dungeon --small
610
+ node scripts/verify-stories.mjs <dungeon> --data-prefix verify-dungeon
477
611
  ```
478
- Confirm the run completes without errors.
479
- 6. Hand off:
612
+ Reduced-scale runs legitimately cap at WEAK on `minCohort` guards; what
613
+ you're checking here is no NONE/INVERSE and no assertion errors.
614
+ 8. Hand off:
480
615
  ```
481
616
  /verify-dungeon <dungeon>
482
617
  ```
483
- If verify-dungeon returns WEAK, NONE, or INVERSE on any pattern, return to
484
- step 4 and refine. Iterate until all patterns score STRONG or NAILED.
618
+ If verify-dungeon returns WEAK, NONE, or INVERSE on any pattern at full
619
+ fidelity, return to step 4 and refine fix the hook or the derivation,
620
+ never relax a threshold to match output. Iterate until all patterns score
621
+ STRONG or NAILED.
485
622
 
486
623
  ## Stopping condition
487
624
 
@@ -492,5 +629,6 @@ still off in the dungeon's overview comment and report the gap to the user.
492
629
  ## Output
493
630
 
494
631
  Modify the dungeon file in place. Add the `hook` function. Add the imports.
495
- Add the documentation block above the config. Do NOT modify any other file.
632
+ Add the documentation block (with EXPECTED METRICS SUMMARY) above the config.
633
+ Add the `stories` export after the config. Do NOT modify any other file.
496
634
  Tell the user to run `/verify-dungeon <dungeon>` next.