@ak--47/dungeon-master 1.5.4 → 1.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/analyze-soup/SKILL.md +21 -12
- package/.claude/skills/create-dungeon/SKILL.md +65 -5
- package/.claude/skills/create-project/SKILL.md +1 -1
- package/.claude/skills/create-project/provision.mjs +57 -10
- package/.claude/skills/verify-dungeon/SKILL.md +56 -25
- package/.claude/skills/verify-dungeon/references/counting-semantics.md +26 -6
- package/.claude/skills/verify-dungeon/references/report-format.md +14 -8
- package/.claude/skills/write-hooks/SKILL.md +156 -18
- package/CHANGELOG.md +183 -0
- package/HOOKS.md +566 -112
- package/README.md +28 -9
- package/docs/guides/1.3.0-upgrade-guide.md +262 -0
- package/docs/guides/1.3.1-upgrade-guide.md +328 -0
- package/docs/guides/1.4.0-upgrade-guide.md +130 -0
- package/docs/guides/1.4.1-upgrade-guide.md +125 -0
- package/docs/guides/1.4.3-upgrade-guide.md +200 -0
- package/docs/guides/1.4.4-upgrade-guide.md +95 -0
- package/docs/guides/1.5.0-upgrade-guide.md +426 -0
- package/docs/guides/1.5.1-upgrade-guide.md +178 -0
- package/docs/guides/1.5.3-upgrade-guide.md +117 -0
- package/docs/guides/1.6.0-upgrade-guide.md +204 -0
- package/dungeons/technical/stories-verify.js +152 -0
- package/dungeons/vertical/README.md +154 -0
- package/dungeons/vertical/ai-platform/ai-platform.js +1424 -0
- package/dungeons/vertical/ai-platform/ai-platform.sql +184 -0
- package/dungeons/vertical/ai-platform/ai-platform.verify.mjs +71 -0
- package/dungeons/vertical/community/community.js +1589 -0
- package/dungeons/vertical/community/community.sql +216 -0
- package/dungeons/vertical/community/community.verify.mjs +72 -0
- package/dungeons/vertical/crypto/crypto.js +1483 -0
- package/dungeons/vertical/crypto/crypto.sql +194 -0
- package/dungeons/vertical/crypto/crypto.verify.mjs +78 -0
- package/dungeons/vertical/dating/dating.js +1540 -0
- package/dungeons/vertical/dating/dating.sql +288 -0
- package/dungeons/vertical/dating/dating.verify.mjs +72 -0
- package/dungeons/vertical/devtools/devtools.js +1524 -0
- package/dungeons/vertical/devtools/devtools.sql +161 -0
- package/dungeons/vertical/devtools/devtools.verify.mjs +73 -0
- package/dungeons/vertical/ecommerce/ecommerce.js +1480 -0
- package/dungeons/vertical/ecommerce/ecommerce.sql +218 -0
- package/dungeons/vertical/ecommerce/ecommerce.verify.mjs +70 -0
- package/dungeons/vertical/education/education.js +1763 -0
- package/dungeons/vertical/education/education.sql +243 -0
- package/dungeons/vertical/education/education.verify.mjs +75 -0
- package/dungeons/vertical/fintech/fintech.js +1410 -0
- package/dungeons/vertical/fintech/fintech.sql +175 -0
- package/dungeons/vertical/fintech/fintech.verify.mjs +70 -0
- package/dungeons/vertical/fitness/fitness.js +1495 -0
- package/dungeons/vertical/fitness/fitness.sql +206 -0
- package/dungeons/vertical/fitness/fitness.verify.mjs +71 -0
- package/dungeons/vertical/food-delivery/food-delivery.js +1641 -0
- package/dungeons/vertical/food-delivery/food-delivery.sql +183 -0
- package/dungeons/vertical/food-delivery/food-delivery.verify.mjs +75 -0
- package/dungeons/vertical/gaming/gaming.js +1741 -0
- package/dungeons/vertical/gaming/gaming.sql +249 -0
- package/dungeons/vertical/gaming/gaming.verify.mjs +70 -0
- package/dungeons/vertical/healthcare/healthcare.js +1488 -0
- package/dungeons/vertical/healthcare/healthcare.sql +193 -0
- package/dungeons/vertical/healthcare/healthcare.verify.mjs +72 -0
- package/dungeons/vertical/insurance-application/insurance-application.js +1852 -0
- package/dungeons/vertical/insurance-application/insurance-application.sql +174 -0
- package/dungeons/vertical/insurance-application/insurance-application.verify.mjs +74 -0
- package/dungeons/vertical/logistics/logistics.js +1541 -0
- package/dungeons/vertical/logistics/logistics.sql +205 -0
- package/dungeons/vertical/logistics/logistics.verify.mjs +73 -0
- package/dungeons/vertical/marketplace/marketplace.js +1438 -0
- package/dungeons/vertical/marketplace/marketplace.sql +148 -0
- package/dungeons/vertical/marketplace/marketplace.verify.mjs +74 -0
- package/dungeons/vertical/{media.js → media/media.js} +557 -58
- package/dungeons/vertical/media/media.sql +173 -0
- package/dungeons/vertical/media/media.verify.mjs +74 -0
- package/dungeons/vertical/real-estate/real-estate.js +1629 -0
- package/dungeons/vertical/real-estate/real-estate.sql +167 -0
- package/dungeons/vertical/real-estate/real-estate.verify.mjs +75 -0
- package/dungeons/vertical/sass/sass.js +1733 -0
- package/dungeons/vertical/sass/sass.sql +173 -0
- package/dungeons/vertical/sass/sass.verify.mjs +74 -0
- package/dungeons/vertical/social/social.js +1456 -0
- package/dungeons/vertical/social/social.sql +160 -0
- package/dungeons/vertical/social/social.verify.mjs +76 -0
- package/dungeons/vertical/streaming/streaming.js +677 -0
- package/dungeons/vertical/streaming/streaming.sql +101 -0
- package/dungeons/vertical/streaming/streaming.verify.mjs +75 -0
- package/dungeons/vertical/support-desk/support-desk.js +741 -0
- package/dungeons/vertical/support-desk/support-desk.sql +137 -0
- package/dungeons/vertical/support-desk/support-desk.verify.mjs +75 -0
- package/dungeons/vertical/travel/travel.js +1603 -0
- package/dungeons/vertical/travel/travel.sql +236 -0
- package/dungeons/vertical/travel/travel.verify.mjs +72 -0
- package/index.js +2 -2
- package/lib/core/config-validator.js +37 -2
- package/lib/core/context.js +4 -2
- package/lib/core/dungeon-loader.js +20 -15
- package/lib/core/dungeon-to-json.js +22 -4
- package/lib/core/storage.js +3 -3
- package/lib/generators/funnels.js +36 -9
- package/lib/hook-helpers/cohort.js +53 -0
- package/lib/hook-helpers/index.js +8 -0
- package/lib/hook-helpers/mutate.js +1 -1
- package/lib/hook-helpers/shape.js +319 -0
- package/lib/hook-patterns/aggregate-per-user-by-bin.js +13 -5
- package/lib/hook-patterns/attributed-by-source.js +60 -58
- package/lib/hook-patterns/frequency-by-frequency.js +14 -4
- package/lib/hook-patterns/funnel-frequency-breakdown.js +24 -12
- package/lib/hook-patterns/index.js +1 -1
- package/lib/hook-patterns/time-to-convert-by-segment.js +75 -13
- package/lib/orchestrators/user-loop.js +41 -0
- package/lib/templates/story-spec.schema.json +175 -0
- package/lib/utils/json-evaluator.js +1 -1
- package/lib/utils/utils.js +5 -4
- package/lib/verify/coerce.js +186 -0
- package/lib/verify/counting.js +270 -32
- package/lib/verify/emulate-breakdown.js +1285 -203
- package/lib/verify/first-time.js +89 -0
- package/lib/verify/flows.js +679 -0
- package/lib/verify/formula.js +259 -0
- package/lib/verify/funnel-engine.js +697 -113
- package/lib/verify/index.js +32 -4
- package/lib/verify/sessionize.js +275 -0
- package/lib/verify/story-runner.js +483 -0
- package/lib/verify/verify-dungeon.js +59 -37
- package/package.json +2 -1
- package/scripts/extract-dungeon-schema.mjs +2 -2
- package/scripts/run-many.mjs +2 -2
- package/scripts/verify-runner.mjs +2 -2
- package/scripts/verify-stories.mjs +244 -0
- package/types.d.ts +296 -30
- package/dungeons/vertical/ai-platform.js +0 -818
- package/dungeons/vertical/community.js +0 -966
- package/dungeons/vertical/crypto.js +0 -895
- package/dungeons/vertical/dating.js +0 -830
- package/dungeons/vertical/devtools.js +0 -1050
- package/dungeons/vertical/ecommerce.js +0 -811
- package/dungeons/vertical/education.js +0 -964
- package/dungeons/vertical/fintech.js +0 -946
- package/dungeons/vertical/fitness.js +0 -935
- package/dungeons/vertical/food-delivery.js +0 -934
- package/dungeons/vertical/gaming.js +0 -1120
- package/dungeons/vertical/healthcare.js +0 -945
- package/dungeons/vertical/insurance-application.js +0 -1039
- package/dungeons/vertical/logistics.js +0 -986
- package/dungeons/vertical/marketplace.js +0 -1019
- package/dungeons/vertical/real-estate.js +0 -872
- package/dungeons/vertical/sass.js +0 -995
- package/dungeons/vertical/social.js +0 -904
- package/dungeons/vertical/travel.js +0 -930
|
@@ -4,13 +4,17 @@ Templates and conventions for writing `hook-results.md` and per-dungeon verifica
|
|
|
4
4
|
|
|
5
5
|
## Verdict criteria (5-tier)
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
- **STRONG** — Within 25% of expected. Direction correct, clearly visible. An analyst would find this pattern immediately.
|
|
9
|
-
- **WEAK** — Within 50% of expected. Directionally correct but magnitude is off, OR sample size is too small to be conclusive.
|
|
10
|
-
- **NONE** — No statistically meaningful difference between cohorts. The hook has no observable effect.
|
|
11
|
-
- **INVERSE** — Effect goes the opposite direction from intended. The story is backwards.
|
|
7
|
+
Verdicts are **mechanical** — computed by `scripts/verify-stories.mjs` from each story assertion's declared `target` / `floor` / `minCohort` (see the `StoryVerdict` typedef in `types.d.ts`), not eyeballed percentages:
|
|
12
8
|
|
|
13
|
-
NAILED
|
|
9
|
+
- **NAILED** — observed within ±10% of `target`.
|
|
10
|
+
- **STRONG** — passes `floor` (or `target` when no floor is declared).
|
|
11
|
+
- **WEAK** — fails `floor` but effect direction is correct, **or** the selected cohort is smaller than `minCohort`. The population floor is a hard cap: a 12-user cohort can never score NAILED, no matter how clean its ratio.
|
|
12
|
+
- **NONE** — no measurable effect, or the selection matched no rows.
|
|
13
|
+
- **INVERSE** — effect direction is opposite the assertion.
|
|
14
|
+
|
|
15
|
+
Story verdict = worst assertion verdict. NAILED and STRONG are passing; WEAK, NONE, and INVERSE fail and require investigation.
|
|
16
|
+
|
|
17
|
+
Hand-assigned verdicts appear only in the legacy no-stories fallback and MUST follow the same definitions: derive a target from the hook's knob constants, compute the band the observed value lands in, and state the derivation in the detail block — never assign a tier by feel.
|
|
14
18
|
|
|
15
19
|
## Ordering: failures first
|
|
16
20
|
|
|
@@ -25,6 +29,8 @@ The summary table should also be sorted this way (INVERSE → NONE → WEAK →
|
|
|
25
29
|
|
|
26
30
|
## Single-dungeon report structure
|
|
27
31
|
|
|
32
|
+
For story-backed dungeons, `hook-results.md` **renders the runner's JSON**: run `verify-stories.mjs --json` and build the Hook Summary table directly from its per-story records (story id, hook number, archetype, observed vs target per assertion, computed verdict). Do not recompute verdicts the runner already settled. Detailed Results blocks exist only for stories below STRONG, `duckdb`-type assertions, and legacy no-stories hooks.
|
|
33
|
+
|
|
28
34
|
```markdown
|
|
29
35
|
# Dungeon Verification Report
|
|
30
36
|
|
|
@@ -191,7 +197,7 @@ grep "^DUNGEON:" research/hook-query-log.txt # list of dungeons querie
|
|
|
191
197
|
|
|
192
198
|
When verifying a dungeon in `dungeons/user/`, write a standalone DuckDB SQL file alongside the dungeon in its folder at `dungeons/user/<name>/<name>-verifications.sql`. This file is the reproducible verification artifact — anyone can re-run it against fresh data.
|
|
193
199
|
|
|
194
|
-
Follow the format in `
|
|
200
|
+
Follow the format in `dungeons/vertical/<name>/`:
|
|
195
201
|
|
|
196
202
|
```sql
|
|
197
203
|
-- ============================================================================
|
|
@@ -211,4 +217,4 @@ Follow the format in `verification/verticals/`:
|
|
|
211
217
|
|
|
212
218
|
Each query block includes the pattern description, observed result, and verdict as SQL comments. This makes the file self-documenting and grep-friendly.
|
|
213
219
|
|
|
214
|
-
**This step is mandatory for user dungeons.** Vertical dungeons already have their SQL
|
|
220
|
+
**This step is mandatory for user dungeons.** Vertical dungeons already have their SQL co-located at `dungeons/vertical/<name>/<name>.sql`. User dungeons keep theirs co-located with the dungeon file.
|
|
@@ -20,7 +20,11 @@ already complete (produced by `create-dungeon`). After writing, hand off to
|
|
|
20
20
|
In scope:
|
|
21
21
|
- The `hook: function(record, type, meta) { ... }` body
|
|
22
22
|
- Documentation comments above the hook explaining each engineered pattern,
|
|
23
|
-
including a reference Mixpanel report block per pattern
|
|
23
|
+
including a reference Mixpanel report block per pattern and the mandatory
|
|
24
|
+
EXPECTED METRICS SUMMARY table
|
|
25
|
+
- The `stories` named export — one machine-checkable story per engineered
|
|
26
|
+
pattern (see "Stories export" below). A hook without stories is
|
|
27
|
+
unverifiable; this skill is not done until the stories exist.
|
|
24
28
|
|
|
25
29
|
Out of scope:
|
|
26
30
|
- Schema changes (events, properties, funnels, superProps, userProps).
|
|
@@ -29,6 +33,18 @@ Out of scope:
|
|
|
29
33
|
- New top-level config knobs.
|
|
30
34
|
- Removing the `hook: function...` body to start over with a new schema.
|
|
31
35
|
|
|
36
|
+
**Before writing a hook, ask: is this trend structural?** Between-path
|
|
37
|
+
comparisons (path X converts worse / slower than path Y, detour-takers drop
|
|
38
|
+
off, mix shift drags the blended rate) are better architected as initial
|
|
39
|
+
conditions — duplicate funnels with swapped steps/props/`conversionRate`/
|
|
40
|
+
`timeToConvert`/`weight` (see the "Structural trend engineering" section in
|
|
41
|
+
`create-dungeon`). If a story reduces to structure, recommend the funnel
|
|
42
|
+
change back to the schema instead of writing a hook to fight the engine —
|
|
43
|
+
the knob IS the expected value, which makes the story band knob-derivable
|
|
44
|
+
(NAILED-capable) instead of confounded (STRONG-capped). Hooks are for
|
|
45
|
+
within-cohort behavior: segments doing more/less of something over time,
|
|
46
|
+
property values that differ by cohort, injected bursts, lifecycle waves.
|
|
47
|
+
|
|
32
48
|
## Reference reading
|
|
33
49
|
|
|
34
50
|
- `lib/hook-helpers/index.js` — atoms (cohort, mutate, timing, inject,
|
|
@@ -36,8 +52,11 @@ Out of scope:
|
|
|
36
52
|
- `lib/hook-patterns/index.js` — high-level recipes (one per Mixpanel
|
|
37
53
|
analysis type).
|
|
38
54
|
- `lib/verify/emulate-breakdown.js` — what `verify-dungeon` will check.
|
|
39
|
-
- `
|
|
40
|
-
|
|
55
|
+
- `lib/templates/story-spec.schema.json` + `DungeonStory` in `types.d.ts` —
|
|
56
|
+
the story-spec grammar the `stories` export must follow.
|
|
57
|
+
- `dungeons/vertical/ecommerce/ecommerce.js` — reference dungeon using a mix
|
|
58
|
+
of atoms and hand-rolled logic, with the mandatory EXPECTED METRICS SUMMARY
|
|
59
|
+
table and a full `stories` export.
|
|
41
60
|
- `dungeons/technical/pattern-*.js` — five minimal pattern fixtures, one per
|
|
42
61
|
recipe.
|
|
43
62
|
- `HOOKS.md` — encyclopedia of hook recipes organized by story pattern. Contains
|
|
@@ -134,6 +153,10 @@ if (type === 'funnel-post' && meta.experiment) {
|
|
|
134
153
|
| inject | `injectOnNewDays(events, eventName, targetDistinctDays)` | **Cohort-only.** Spreads injections across previously-empty days. Use for cohort-conditional active-day boosts; for global active-day shape, use `Dungeon.avgActiveDaysPerUser` config knob. |
|
|
135
154
|
| identity | `isPreAuthEvent(event, authTime)` | Standalone variant of meta.isPreAuth |
|
|
136
155
|
| identity | `splitByAuth(events, authTime)` | { preAuth, postAuth, stitch } partition |
|
|
156
|
+
| cohort | `hashCohort(id, pct)` | Deterministic pct% cohort (0–100 scale). **Use this first for hidden cohorts** — replaces ad-hoc `charCodeAt % N`. When one dungeon needs several NON-overlapping cohorts, gate on disjoint `hashFloat(uid)` bands instead (e.g. `[0, 0.45)`, `[0.45, 0.70)`) |
|
|
157
|
+
| shape | `applyLifecycleWave(events, uid, opts)` | Dormancy window + resurrection burst for Lifecycle reports. When-to-use: the story is "users go quiet, then come back". Gap discipline: ONE stray value moment inside the window destroys the Resurrected read — size `dormantDays` to cover ≥2 whole lifecycle periods, keep the window inside the user's lifespan |
|
|
158
|
+
| shape | `applyPathBias(events, uid, opts)` | Inject a Flows path after the user's FIRST anchor occurrence. When-to-use: the story is "X% of users take this route". `share` is a 0–1 FRACTION (not `hashCohort`'s pct scale); needs ~≥0.20–0.25 to survive Sankey top-3-per-level pruning; per-step gaps clamped ≥1s so ordering survives |
|
|
159
|
+
| shape | `applySessionShape(events, uid, opts)` | Retime the whole stream into `sessionsPerWeek` clusters of `sessionMinutes`. When-to-use: session-duration/cadence stories (sessionMetrics reads). Retiming ONLY — no adds/drops; intra-session gaps stay <28min, inter-session >30min, no cluster crosses UTC midnight. Combine with `hashCohort` for per-role shapes; call BEFORE `applyPathBias` so injected paths keep their own tight gaps |
|
|
137
160
|
|
|
138
161
|
### Hook anti-patterns
|
|
139
162
|
|
|
@@ -214,17 +237,51 @@ on undeclared entries.
|
|
|
214
237
|
Higher-level recipes. Each maps to ONE Mixpanel analysis the verify-dungeon
|
|
215
238
|
emulator can re-derive.
|
|
216
239
|
|
|
217
|
-
| Pattern | Mixpanel analysis | Hook type |
|
|
218
|
-
|
|
219
|
-
| `applyFrequencyByFrequency` | Insights — count(A) by per-user count(B) | everything |
|
|
220
|
-
| `applyFunnelFrequencyBreakdown` | Funnels — completion by per-user count(X) | funnel-post |
|
|
221
|
-
| `applyAggregateByBin` | Insights — avg(prop X) by per-user count(B) | everything |
|
|
222
|
-
| `
|
|
223
|
-
| `applyAttributedBySource` | Conversions by Source (first/last touch) | everything |
|
|
240
|
+
| Pattern | Mixpanel analysis | Hook type | Caveat (HOOKS.md) |
|
|
241
|
+
|---------|-------------------|-----------|-------------------|
|
|
242
|
+
| `applyFrequencyByFrequency` | Insights — count(A) by per-user count(B) | everything | `binBy` defaults to `'distinctDays'` (v1.6) — bins match Mixpanel's per-user distinct-day counting, not raw event totals |
|
|
243
|
+
| `applyFunnelFrequencyBreakdown` | Funnels — completion by per-user count(X) | funnel-post | When funnels share a step prefix, restrict scaling to the target funnel — scaling every instance lets first-occurrence funnel evaluation assemble chains across unscaled instances and the ratio never reaches the report |
|
|
244
|
+
| `applyAggregateByBin` | Insights — avg(prop X) by per-user count(B) | everything | Same `binBy: 'distinctDays'` default as above |
|
|
245
|
+
| `applyTTCBySegmentV2` | Funnel TTC — broken down by user-property segment | everything | v1 (`applyTTCBySegment`, funnel-post) is **deprecated**: Mixpanel TTC reads each step's FIRST occurrence per user, so per-run gap scaling only reaches the report on `isFirstFunnel` runs. V2 finds the greedy first sequence (`findFirstSequence`) and scales that |
|
|
246
|
+
| `applyAttributedBySource` | Conversions by Source (first/last touch) | everything | OVERWRITES the engine-stamped touch the chosen model reads; never stamps UTMs onto unstamped events (would blow the `maxTouchpointsPerUser` cap and land outside the last-10 lookback) |
|
|
224
247
|
|
|
225
248
|
Use a pattern when the trend matches its analysis 1:1. Drop down to atoms when
|
|
226
249
|
the trend is bespoke or composite.
|
|
227
250
|
|
|
251
|
+
### New story archetypes (v1.6) — design rules per report family
|
|
252
|
+
|
|
253
|
+
Four archetypes joined the story-spec enum in v1.6. Each has a
|
|
254
|
+
non-negotiable design rule learned the hard way:
|
|
255
|
+
|
|
256
|
+
- **`lifecycle-wave`** (`applyLifecycleWave`) — GAP DISCIPLINE. Mixpanel's
|
|
257
|
+
"dormant" state is an `EqualTo 0` filter over the whole period: one stray
|
|
258
|
+
value-moment event inside the dormancy window (including events OTHER hooks
|
|
259
|
+
injected earlier in the same `everything` pass) flips the user out of
|
|
260
|
+
Resurrected. Run the wave AFTER every injecting hook, size `dormantDays` to
|
|
261
|
+
≥2 lifecycle periods, and keep `dormantFromDay + dormantDays` inside the
|
|
262
|
+
user's lifespan (the future-time guard eats bursts past dataset end).
|
|
263
|
+
- **`path-share`** (`applyPathBias`) — FIRST-FLOW ANCHORING. Flows' unique
|
|
264
|
+
counting reads only each user's FIRST flow past the anchor, so the injected
|
|
265
|
+
path must follow the FIRST anchor occurrence (the atom does this; don't
|
|
266
|
+
hand-roll a later anchor). Sankey prunes to the top ~3 branches per level:
|
|
267
|
+
an engineered branch below ~20–25% share silently disappears from the
|
|
268
|
+
visualization even though the data is there. Label-only path reads can
|
|
269
|
+
INVERT when a busier cohort glues extra visible events between path steps —
|
|
270
|
+
assert the share on the cohort you engineered, not globally.
|
|
271
|
+
- **`session-shape`** (`applySessionShape`) — 30-MIN STRADDLING + MIDNIGHT
|
|
272
|
+
RULE. Mixpanel derives sessions with a 30-min idle timeout and splits at
|
|
273
|
+
UTC midnight. Engineered cadences must keep intra-session gaps clearly
|
|
274
|
+
UNDER 30min and inter-session gaps clearly OVER it — a gap that straddles
|
|
275
|
+
the timeout makes session counts jitter across runs. Never let an
|
|
276
|
+
engineered session cross UTC midnight (the day split cuts it in two). The
|
|
277
|
+
atom guarantees all three; hand-rolled retiming must too.
|
|
278
|
+
- **`composition-drift`** — the breakdown's SHARE of a segment moves over
|
|
279
|
+
time while totals stay flat (e.g. plan-mix shifts toward premium). Engineer
|
|
280
|
+
by flipping an existing property value on a date-gated cohort, never by
|
|
281
|
+
changing volumes — volume changes read as `temporal-inflection` instead.
|
|
282
|
+
Assert with a `timeBucket` breakdown comparing first-window vs last-window
|
|
283
|
+
share.
|
|
284
|
+
|
|
228
285
|
## Anti-flag-stamping rule (HARD WALL)
|
|
229
286
|
|
|
230
287
|
Hooks MUST NOT add new properties to records. The schema (config) defines what
|
|
@@ -318,6 +375,71 @@ checks against and what consumers read to understand the dataset.
|
|
|
318
375
|
* Expected ratio: bin>=15 / bin<5 ≈ 3x (within ±15%)
|
|
319
376
|
```
|
|
320
377
|
|
|
378
|
+
### EXPECTED METRICS SUMMARY table (MANDATORY)
|
|
379
|
+
|
|
380
|
+
The doc block MUST end with an EXPECTED METRICS SUMMARY table — one row per
|
|
381
|
+
verifiable read, with the DERIVATION of each expected number from the hook's
|
|
382
|
+
knob constants (never a bare number someone has to trust). Follow the style
|
|
383
|
+
in `dungeons/vertical/ecommerce/ecommerce.js`:
|
|
384
|
+
|
|
385
|
+
```
|
|
386
|
+
* EXPECTED METRICS SUMMARY
|
|
387
|
+
* ============================================================================
|
|
388
|
+
*
|
|
389
|
+
* Hook | Metric | Derivation | Expected | Measured (full)
|
|
390
|
+
* -----|---------------------------------|---------------------|----------|----------------
|
|
391
|
+
* H2 | avg watchTimeSec post/pre | 1.52/0.48 | 3.17x | 3.19x
|
|
392
|
+
* H6 | sweet/over avg cart item amount | SWEET_CART_BOOST | 1.25x | 1.233x
|
|
393
|
+
* H9 | dark/light checkouts per user | 20/13 diluted | ~1.48x | 1.412x
|
|
394
|
+
* ============================================================================
|
|
395
|
+
```
|
|
396
|
+
|
|
397
|
+
Fill the Measured column from the verification run (a reduced-scale iteration
|
|
398
|
+
number is fine if labeled). "Diluted" derivations must say WHAT dilutes
|
|
399
|
+
(organic events, cohort mixing) — the Derivation column is the anchor
|
|
400
|
+
`/verify-dungeon` uses when a read misses.
|
|
401
|
+
|
|
402
|
+
## Stories export (MANDATORY — the machine contract)
|
|
403
|
+
|
|
404
|
+
Every engineered pattern ships with one story in a `stories` named export —
|
|
405
|
+
the machine-checkable form of the doc block. `scripts/verify-stories.mjs`
|
|
406
|
+
evaluates them; `/verify-dungeon` runs it as step 1. Grammar:
|
|
407
|
+
`lib/templates/story-spec.schema.json` and `DungeonStory` in `types.d.ts`.
|
|
408
|
+
|
|
409
|
+
```js
|
|
410
|
+
export const stories = [
|
|
411
|
+
{
|
|
412
|
+
id: "H1-power-buyers",
|
|
413
|
+
hook: "H1",
|
|
414
|
+
archetype: "frequency-sweet-spot",
|
|
415
|
+
narrative: "Users with 15+ browse days buy 3x as often as light browsers.",
|
|
416
|
+
mixpanelReport: { type: "Insights", event: "Purchase", breakdown: "per-user count of Browse" },
|
|
417
|
+
assertions: [{
|
|
418
|
+
breakdown: { type: "frequency", event: "Purchase", cohortEvent: "Browse", bins: [5, 15] },
|
|
419
|
+
select: { hi: { where: { bin: ">=15" } }, lo: { where: { bin: "<5" } } },
|
|
420
|
+
expect: { metric: "hi.avg / lo.avg", op: ">=", target: POWER_BUYER_MULT, floor: POWER_BUYER_MULT * 0.8 },
|
|
421
|
+
minCohort: 200,
|
|
422
|
+
}],
|
|
423
|
+
},
|
|
424
|
+
];
|
|
425
|
+
```
|
|
426
|
+
|
|
427
|
+
Rules:
|
|
428
|
+
|
|
429
|
+
- **Thresholds derive from the knob you just wrote.** Export the hook's knob
|
|
430
|
+
constants (`const POWER_BUYER_MULT = 3`) and compute `target` from them —
|
|
431
|
+
never paste the number twice. If the read is diluted (organic mixing),
|
|
432
|
+
derive the dilution too and say so in a comment.
|
|
433
|
+
- `floor` must itself be derived (e.g. `target * 0.8`) — never hand-tuned to
|
|
434
|
+
a run. A missed assertion means fixing the hook or the derivation, never
|
|
435
|
+
relaxing the number to match output.
|
|
436
|
+
- Set `minCohort` from the cohort math (share × numUsers × ~0.5 safety) so
|
|
437
|
+
reduced-scale runs cap at WEAK instead of passing on noise.
|
|
438
|
+
- One story per pattern; story `hook` matches the doc-block numbering (`H3`).
|
|
439
|
+
- The `assert` function escape hatch is discouraged — each use needs a
|
|
440
|
+
comment saying why the declarative `expect` grammar can't express it.
|
|
441
|
+
- Stories are JS-dungeon-only (`dungeon-to-json` drops them).
|
|
442
|
+
|
|
321
443
|
## Hook Ordering Within `everything`
|
|
322
444
|
|
|
323
445
|
The order of operations inside the everything hook matters when hooks interact:
|
|
@@ -466,22 +588,37 @@ cohort.
|
|
|
466
588
|
2. Translate the user's story description into 3–5 engineered patterns.
|
|
467
589
|
Consult `HOOKS.md` for recipe ideas that match the user's story. Each recipe
|
|
468
590
|
includes the hook type, code snippet, and Mixpanel report format.
|
|
469
|
-
3.
|
|
591
|
+
3. **Calibrate thresholds against the real distribution.** Before choosing
|
|
592
|
+
any "N+ events" gate or cohort cutoff, generate a small run and query the
|
|
593
|
+
actual per-user distribution (see "Threshold Calibration" above for the
|
|
594
|
+
query). Set gates at ~the 80th percentile of what the data shows — a
|
|
595
|
+
threshold picked from intuition is the most common cause of empty cohorts.
|
|
596
|
+
```bash
|
|
597
|
+
node scripts/verify-runner.mjs <dungeon> calib --small
|
|
598
|
+
```
|
|
599
|
+
4. For each pattern:
|
|
470
600
|
- Pick a pattern from `lib/hook-patterns/` if it fits the analysis 1:1.
|
|
471
601
|
- Otherwise compose atoms from `lib/hook-helpers/`.
|
|
472
602
|
- Document the pattern in a comment block (Mixpanel report instructions).
|
|
473
|
-
|
|
474
|
-
|
|
603
|
+
5. Write the `hook` function, importing atoms/patterns at the top of the
|
|
604
|
+
file. Finish the doc block with the EXPECTED METRICS SUMMARY table.
|
|
605
|
+
6. Write the `stories` export — one story per pattern, `target`/`floor`
|
|
606
|
+
derived from the exported knob constants (see "Stories export" above).
|
|
607
|
+
7. Smoke-test generation, then evaluate the stories:
|
|
475
608
|
```bash
|
|
476
609
|
node scripts/verify-runner.mjs <dungeon> verify-dungeon --small
|
|
610
|
+
node scripts/verify-stories.mjs <dungeon> --data-prefix verify-dungeon
|
|
477
611
|
```
|
|
478
|
-
|
|
479
|
-
|
|
612
|
+
Reduced-scale runs legitimately cap at WEAK on `minCohort` guards; what
|
|
613
|
+
you're checking here is no NONE/INVERSE and no assertion errors.
|
|
614
|
+
8. Hand off:
|
|
480
615
|
```
|
|
481
616
|
/verify-dungeon <dungeon>
|
|
482
617
|
```
|
|
483
|
-
If verify-dungeon returns WEAK, NONE, or INVERSE on any pattern
|
|
484
|
-
step 4 and refine
|
|
618
|
+
If verify-dungeon returns WEAK, NONE, or INVERSE on any pattern at full
|
|
619
|
+
fidelity, return to step 4 and refine — fix the hook or the derivation,
|
|
620
|
+
never relax a threshold to match output. Iterate until all patterns score
|
|
621
|
+
STRONG or NAILED.
|
|
485
622
|
|
|
486
623
|
## Stopping condition
|
|
487
624
|
|
|
@@ -492,5 +629,6 @@ still off in the dungeon's overview comment and report the gap to the user.
|
|
|
492
629
|
## Output
|
|
493
630
|
|
|
494
631
|
Modify the dungeon file in place. Add the `hook` function. Add the imports.
|
|
495
|
-
Add the documentation block
|
|
632
|
+
Add the documentation block (with EXPECTED METRICS SUMMARY) above the config.
|
|
633
|
+
Add the `stories` export after the config. Do NOT modify any other file.
|
|
496
634
|
Tell the user to run `/verify-dungeon <dungeon>` next.
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,189 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to `@ak--47/dungeon-master`.
|
|
4
4
|
|
|
5
|
+
## 1.6.0 — 2026-07-04
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- **Emulator: five new analysis types + retention completion**
|
|
10
|
+
(`emulateBreakdown`, all ARB-cited):
|
|
11
|
+
- `eventBreakdown` — Insights "Total" broken down by a property, with
|
|
12
|
+
Mixpanel's exact segment coercion (list fan-out, `$empty_list`,
|
|
13
|
+
`undefined` bucket, case-sensitive type-tagged segments, topN 250);
|
|
14
|
+
`countType: 'unique' | 'sessions'`, `firstTimeOnly` compose;
|
|
15
|
+
unrecognized `countType` values throw (same strict-option rule as
|
|
16
|
+
retention keys).
|
|
17
|
+
- `uniques` — per-interval independent dedup, rolling XAU windows,
|
|
18
|
+
cumulative running distinct; `countType: 'sessions'`, `firstTimeOnly`.
|
|
19
|
+
- `lifecycle` — Lifecycle Cohort Analysis board-template classification
|
|
20
|
+
(new / retained / resurrected / dormant) on a value-moment event, 7- or
|
|
21
|
+
30-day periods.
|
|
22
|
+
- `topPaths` — Flows: next-anchor-only matching, forward/reverse capacity
|
|
23
|
+
rings, per-level top-N pruning into `$mp_uncommon_flows_events`,
|
|
24
|
+
`hiddenEvents` / `visibleEvents`, `countType: 'general' | 'unique' |
|
|
25
|
+
'sessions'`.
|
|
26
|
+
- `distinctCount` — distinct values of a property + top-N value counts.
|
|
27
|
+
- `retention` completion — `compounded`, `birthCanRetain`,
|
|
28
|
+
`carryForward` / `carryBack` / `consecutiveForward`, `calendarStart`,
|
|
29
|
+
`cohortWindow`, `segmentOn: 'return'`, internal-event ignore list.
|
|
30
|
+
- **Funnel evaluator upgrades**: session-count conversion windows,
|
|
31
|
+
`countMode: 'sessions'`, ARB-exact exclusion handling, any-order step
|
|
32
|
+
blocks, step-0-anchored trends under `timeBucket`.
|
|
33
|
+
- **New verify primitives**: `sessionize()` (query-time sessions — 30-min
|
|
34
|
+
gap / 24h max / UTC-day triggers, synthetic `$session_start`/`$session_end`),
|
|
35
|
+
`filterFirstTimeEver()`, `evaluateFormula()` (ARB formula grammar),
|
|
36
|
+
`extractFlows` / `aggregateFlows`, breakdown-key coercion
|
|
37
|
+
(`lib/verify/coerce.js`), `frequencyHistogram`, null-aware avg/sum
|
|
38
|
+
`{ flatten: true }`, `attributedBy` per-conversion output.
|
|
39
|
+
- **Hook atoms**: `hashCohort` (seed-stable cohort assignment),
|
|
40
|
+
`applyLifecycleWave`, `applyPathBias`, `applySessionShape`; pattern
|
|
41
|
+
`applyTTCBySegmentV2` (see Deprecated).
|
|
42
|
+
- **Experiments: `sticky` knob** (`ExperimentConfig.sticky`, default
|
|
43
|
+
`true`). Sticky bucketing — the pre-1.6 per-user hash — is now explicit
|
|
44
|
+
and opt-out-able: `sticky: false` re-rolls the variant on every funnel
|
|
45
|
+
pass via the seeded RNG. Default preserves byte-identical output for
|
|
46
|
+
existing dungeons.
|
|
47
|
+
- **Story layer**: `stories` named export on dungeons — one machine-checkable
|
|
48
|
+
story per hook (`DungeonStory` typedef,
|
|
49
|
+
`lib/templates/story-spec.schema.json`) — and the
|
|
50
|
+
`scripts/verify-stories.mjs` runner: mechanical five-tier verdicts
|
|
51
|
+
(NAILED / STRONG / WEAK / NONE / INVERSE), population floors (`minCohort`),
|
|
52
|
+
hook-coverage discipline, disk + in-memory modes, `--json`.
|
|
53
|
+
- **Verticals**: `dungeons/vertical/` restructured to one folder per vertical
|
|
54
|
+
(`<name>/<name>.js` + `<name>.verify.mjs` + `<name>.sql`); `stories`
|
|
55
|
+
exports and rebuilt hooks across all verticals; two new showcase dungeons —
|
|
56
|
+
`streaming` (lifecycle) and `support-desk` (flows + sessions).
|
|
57
|
+
- **Skills**: `/write-hooks` authors the stories export; `/verify-dungeon`
|
|
58
|
+
runs the story runner first and investigates only failures;
|
|
59
|
+
`/create-dungeon` designs analysis-friendly vocabularies (session
|
|
60
|
+
fan-out, value moment, hidden-event hygiene); `/analyze-soup` queries in
|
|
61
|
+
UTC; `/create-project` builds business context from the stories export.
|
|
62
|
+
- **Docs**: HOOKS.md §2.12–2.17 (event breakdown coercion, uniques/XAU,
|
|
63
|
+
formulas, first-time-ever, lifecycle, flows, sessions), recipes 4.29–4.31,
|
|
64
|
+
atom/helper reference sections.
|
|
65
|
+
|
|
66
|
+
### Behavior changes
|
|
67
|
+
|
|
68
|
+
- **Retention option keys are strict** (P1.5). Unknown keys in a `retention`
|
|
69
|
+
emulator config now throw instead of being silently ignored — a typo'd
|
|
70
|
+
option previously ran with defaults and produced plausible-but-wrong
|
|
71
|
+
numbers. `carry_forward: true` is kept as a deprecated alias for
|
|
72
|
+
`unbounded: 'carryForward'`.
|
|
73
|
+
- **Funnel exclusions no longer fire before step 0** (P1.6.4).
|
|
74
|
+
`evaluateFunnel`'s `exclusionSteps` previously defaulted `afterStep` to
|
|
75
|
+
−Infinity, so an exclusion event could condemn an attempt before the first
|
|
76
|
+
step was ever reached. ARB has no exclusion gaps before the first step: a
|
|
77
|
+
pre-step-0 exclusion event now only matters inside the 2-second grace rule
|
|
78
|
+
at step 0 (condemns with `excludedAtStep`), otherwise the attempt proceeds.
|
|
79
|
+
- **Non-sequential funnel orders verify with full ARB semantics** (P1.6.6).
|
|
80
|
+
`first-fixed` / `last-fixed` / `first-and-last-fixed` / `outside-in` /
|
|
81
|
+
`random` previously verified via set-membership ("fired all step events,
|
|
82
|
+
any order", `verificationKind: 'partial'`); they now route through
|
|
83
|
+
any-order step blocks with full conversion-window / 2-second-rule /
|
|
84
|
+
exclusion / anchor-ordering semantics. Users that passed the loose check
|
|
85
|
+
but violate window or anchor ordering no longer convert. `middle-fixed`
|
|
86
|
+
keeps set-membership (its scrambled slots are non-contiguous).
|
|
87
|
+
- **`sessionMetrics` defaults to query-time derived sessions** (P1.7.2). New
|
|
88
|
+
`source: 'derived' | 'stamped'` option, default `'derived'`: sessions are
|
|
89
|
+
re-derived from raw timestamps via `sessionize()` — what Mixpanel actually
|
|
90
|
+
computes — instead of reading the generator's pre-stamped `session_id`.
|
|
91
|
+
The stamped path remains via `source: 'stamped'`, and the per-row
|
|
92
|
+
`stampedDivergence` count audits the gap between the two.
|
|
93
|
+
- **`$experiment_started` is pinned to funnel-pass start** (P4.2 engine fix,
|
|
94
|
+
pre-existing since 1.4.0). For experiment funnels with a non-`sequential`
|
|
95
|
+
`order` (`last-fixed`, `random`, `first-fixed`, ...), `applyOrderingStrategy`
|
|
96
|
+
shuffled the synthetic exposure event into the funnel body — the exposure
|
|
97
|
+
landed mid-pass at a uniform position, so exposure→conversion TTC read ~58%
|
|
98
|
+
of `timeToConvert`, and any exposure-anchored conversion measurement (the
|
|
99
|
+
Mixpanel Experiments report, ordered-funnel pairing from
|
|
100
|
+
`$experiment_started`) undercounted variant lift. The ordering strategy now
|
|
101
|
+
shuffles only the real steps; `$experiment_started` stays at execution index
|
|
102
|
+
0 (offset 0), and `first-fixed`/`first-and-last-fixed` pin the true first
|
|
103
|
+
step instead of the exposure marker. Output changes (event order + RNG
|
|
104
|
+
stream) for experiment funnels with shuffle orders; `sequential` experiment
|
|
105
|
+
funnels are unaffected.
|
|
106
|
+
- **Session IDs are re-derived after the `everything` hook** (P2.1). The first
|
|
107
|
+
`assignSessionIds` pass still runs before hooks (hooks may read
|
|
108
|
+
`session_id`), but a second pass now relabels on the FINAL event set — after
|
|
109
|
+
the `everything` hook, auto-sort, and the future-time guard. Time-mutating
|
|
110
|
+
hooks (TTC scaling, injected bursts) previously left stale session ids that
|
|
111
|
+
disagreed with what Mixpanel derives from timestamps at query time. Session
|
|
112
|
+
ids hash from (user key + first event time of the session), so sessions
|
|
113
|
+
whose events did not move keep their exact ids. The per-session sticky-device
|
|
114
|
+
rewrite is NOT re-run — relabeling never mutates identity fields. Behavior
|
|
115
|
+
change only for dungeons whose hooks mutate event times; their stamped
|
|
116
|
+
`session_id` values now match query-time derivation
|
|
117
|
+
(`stampedDivergence === 0`).
|
|
118
|
+
- **Churn is now a hard activity boundary** (P2.2). `isChurnEvent` broke the
|
|
119
|
+
budget loop (stopping generation), but already-generated events carry
|
|
120
|
+
independent timestamps — uniform TimeSoup draws on the legacy path, a
|
|
121
|
+
shuffled active-day plan under `avgActiveDaysPerUser`/`retentionCurve` — so
|
|
122
|
+
churned users kept events DATED after their churn event. Churned users'
|
|
123
|
+
events are now truncated at the churn event's timestamp (the churn event
|
|
124
|
+
itself survives). Affects only dungeons using `isChurnEvent`; users who
|
|
125
|
+
return (`returnLikelihood` roll succeeds) are untouched. `simplest.js` has
|
|
126
|
+
no churn events, so the engine-shape canary and sweep are unaffected.
|
|
127
|
+
- **Bin-based patterns bin by distinct days by default** (P2.4).
|
|
128
|
+
`applyFrequencyByFrequency`, `applyFunnelFrequencyBreakdown`, and
|
|
129
|
+
`applyAggregateByBin` gain `binBy: 'events' | 'distinctDays'` (default
|
|
130
|
+
`'distinctDays'`, via `binByDistinctPeriods`). Mixpanel's frequency reports
|
|
131
|
+
— and the local emulator — bucket users by distinct calendar days, so the
|
|
132
|
+
old total-event-count axis could put a user in a different cohort than the
|
|
133
|
+
report bucket their data lands in, diluting engineered signal. Pass
|
|
134
|
+
`binBy: 'events'` to restore the pre-1.6 axis (also the right choice for
|
|
135
|
+
`applyFunnelFrequencyBreakdown`'s funnelEvents fallback, where one funnel
|
|
136
|
+
run rarely spans two days).
|
|
137
|
+
- **`applyAttributedBySource` rewritten to overwrite engine-stamped touches**
|
|
138
|
+
(P2.4, HOOKS.md recipe 4.26 as code). New opts:
|
|
139
|
+
`{ weights, property = 'utm_source', model = 'firstTouch'|'lastTouch'|'both' }`;
|
|
140
|
+
returns `{ overwritten, touches }`. The old copy-source-to-conversion
|
|
141
|
+
mechanism stamped fresh values, which under the v1.5 touchpoint cap land
|
|
142
|
+
outside Mixpanel's lookback and never move the attribution report. The
|
|
143
|
+
pattern now overwrites the value on the touch the chosen model reads and
|
|
144
|
+
never adds the property to unstamped events.
|
|
145
|
+
|
|
146
|
+
### Changed
|
|
147
|
+
|
|
148
|
+
- **Shipped vertical dungeons: hook fixes that change generated output**
|
|
149
|
+
(P4.2 rebuild — same seeds, different data where noted):
|
|
150
|
+
- **media**: H10 applied the plan-tier factor to `watch_duration_min` in
|
|
151
|
+
two separate blocks — the engineered free/premium ratio compounded to
|
|
152
|
+
~4.4x instead of the documented 2.09x. Single application now; the
|
|
153
|
+
duplicate block is deleted (no RNG-stream impact).
|
|
154
|
+
- **marketplace**: H9 funnel-post TTC scaling is restricted to the
|
|
155
|
+
Browse-to-Purchase funnel (it previously scaled all five; Buyer
|
|
156
|
+
Onboarding shares the search→view→cart prefix, so first-occurrence
|
|
157
|
+
funnel evaluation assembled chains across unscaled instances and the
|
|
158
|
+
engineered ratio never reached the report). H10 redesigned from a
|
|
159
|
+
windowed message-cohort purchase-drop to a total-message-count cohort
|
|
160
|
+
with property-only `offer_amount` effects.
|
|
161
|
+
- **sass**: H9 TTC scaling moved from one stitched whole-history
|
|
162
|
+
sequence (everything hook) to per-instance funnel-post gap scaling
|
|
163
|
+
gated on the `alert triggered` funnel — the old single scaled sequence
|
|
164
|
+
was diluted by the user's unscaled instances and never survived to the
|
|
165
|
+
funnel report.
|
|
166
|
+
- **crypto**: all hook day-boundary math converted from local-time dayjs
|
|
167
|
+
to UTC (dataset timestamps are UTC; boundaries previously shifted by
|
|
168
|
+
the host's UTC offset). H9 TTC scaling restricted to the onboarding
|
|
169
|
+
funnel (same cross-instance dilution class as marketplace). H6 churn
|
|
170
|
+
no longer erases a user's first 24 hours — the old absolute-day cutoff
|
|
171
|
+
shredded late-born users' signup/onboarding/auth events under the
|
|
172
|
+
growth macro.
|
|
173
|
+
|
|
174
|
+
### Deprecated
|
|
175
|
+
|
|
176
|
+
- **`applyTTCBySegment`** (P2.4) — the funnel-post variant scales one run's
|
|
177
|
+
internal gaps, but Mixpanel's TTC measures the FIRST occurrence of each
|
|
178
|
+
step per user, so the scaling only reaches the report for `isFirstFunnel`
|
|
179
|
+
runs. Still functional; warns once. Use **`applyTTCBySegmentV2`** (new,
|
|
180
|
+
`everything` hook) — finds the greedy first sequence via
|
|
181
|
+
`findFirstSequence` and scales it with `scaleFunnelTTC`.
|
|
182
|
+
- **`Persona.churnRate`, `Persona.activeWindow`, `Persona.soupOverride`**
|
|
183
|
+
(P2.5) — declared config surface that was never implemented: nothing in
|
|
184
|
+
lib/ reads them after validation. Marked `@deprecated` in types.d.ts; the
|
|
185
|
+
validator warns once per process when a dungeon sets any of them. Not
|
|
186
|
+
removed (declared surface) and not implemented (config-shape freeze).
|
|
187
|
+
|
|
5
188
|
## 1.5.4 — 2026-06-04
|
|
6
189
|
|
|
7
190
|
Patch. Import-phase progress now reaches `onProgress` consumers.
|