@ak--47/dungeon-master 1.2.3 → 1.3.0

package/CHANGELOG.md

@@ -0,0 +1,42 @@
+# Changelog
+
+All notable changes to `@ak--47/dungeon-master`.
+
+## 1.3.0 — 2026-04-28
+
+The "no more end-of-dataset blowup" release. Rewrote how per-user event budgets and big-picture trends are decided so the default chart looks flat with a weekly cycle, instead of meteoric ramp + cliff in the final ~14 days.
+
+### Added
+
+- **`avgEventsPerUserPerDay` (canonical event-volume primitive).** Per-user event budgets now scale with each user's active days (`rate × user_active_days`), so users born late in the dataset don't compress their entire event budget into a tiny window. `numEvents` still works as a fallback target — see [`research/1.3.0-upgrade-guide.md`](research/1.3.0-upgrade-guide.md).
+- **`macro` preset system.** Big-picture trend across the whole window. Default: `"flat"`. Other presets: `"steady"`, `"growth"`, `"viral"`, `"decline"`. Accepts a string, `{ preset, ...overrides }`, or a fully custom object.
+- **`preExistingSpread` config field.** `"uniform"` (default in `flat`) spreads pre-existing users' first event time across `[FIXED_BEGIN-30d, FIXED_BEGIN]`. `"pinned"` (legacy, used by `growth`/`viral`) stacks them all at `FIXED_BEGIN`.
+- **Future-event guard.** End of `lib/orchestrators/user-loop.js` drops any event whose timestamp landed past `MAX_TIME` after the everything hook runs. Mixpanel rejects future-dated events; this catches hook-injected duplicates that overflow.
+- **Public type aliases.** `Soup`, `Macro`, `ResolvedMacro`, plus per-hook-type meta interfaces (`HookMetaEvent`, `HookMetaUser`, `HookMetaEverything`, etc.) for users who want to narrow inside their hooks.
+- **Validator hardening.** Rejects `numUsers <= 0` and explicit `numDays <= 0`; clamps `bornRecentBias` to `[-1, 1]`; coerces non-finite `bornRecentBias` to `0`.
+- **`engines.node: ">=18.0.0"`** declared in `package.json`.
+- **`tests/macro-and-rate.test.js`** and **`tests/integration-1.3.0.test.js`** — 34 new tests covering the rate primitive, macro resolution, validator guards, future-event guard, per-user budget, and `preExistingSpread`.
+
+### Changed
+
+- **Default trend is now `flat`.** Previously the implicit trend (via the `growth` soup preset) used `bornRecentBias: 0.3` and `percentUsersBornInDataset: 15`. New default produces a level chart with weekly oscillation. To recover the old shape, set `macro: "growth"`.
+- **Soup presets are now intra-week / intra-day only.** `bornRecentBias` and `percentUsersBornInDataset` were removed from soup presets and live on `macro` instead. Soup preset names are unchanged.
+- **`dungeons/vertical/questforge.js` → `dungeons/vertical/gaming.js`.** File names follow the vertical convention; the product/app name lives inside file comments.
+- **Auto-batch ordering bug fixed.** The `numEvents >= 2_000_000` auto-batch check in `config-validator.js` now runs AFTER rate→numEvents resolution, so dungeons that set only `avgEventsPerUserPerDay` correctly trigger batch mode.
+- **Validator no longer mutates the caller's config.** Macro-resolved values (`bornRecentBias`, `percentUsersBornInDataset`, `preExistingSpread`) are stored in local vars and added to the output, instead of being written back onto the input object.
+- **All 36 dungeons migrated** via `scripts/experiments/migrate-dungeons.mjs` to use `avgEventsPerUserPerDay`. Hook bugs surfaced by `/verify-hooks` (13 failures) fixed across `dating`, `social`, `travel`, `community`, `logistics`, `media`, `fintech`, `food-delivery`, `education`, `real-estate`, `devtools`, and `marketplace`.
+- **npm publish surface trimmed.** `dungeons/user/` and `scripts/experiments/` excluded from the published package.
+
+### Documentation
+
+- **`research/1.3.0-upgrade-guide.md`** — full upgrade guide for AI agents, including hook-writing learnings from the verify-hooks pass.
+- **`research/end-bunchiness.md`** — full diagnosis, experiment data, before/after numbers.
+- **`research/hook-results.md`** — consolidated PASS/WEAK/FAIL report for ~160 hooks across 20 vertical dungeons.
+- **`CLAUDE.md`** and **`.claude/skills/create-dungeon/SKILL.md`** updated with the new two-tier preset API and hook-writing rules.
+
+### Backward compatibility
+
+- Dungeons that set only `numEvents` continue to work — the per-day rate is derived. They render with the new `flat` macro defaults; add `macro: "growth"` to restore the old growth-bias.
+- Dungeons that set `bornRecentBias` / `percentUsersBornInDataset` directly continue to work — those values override the macro preset.
+- Soup preset names are unchanged. Hook semantics are unchanged. Output file naming is unchanged.
+- `numEvents` is **not deprecated** — it remains the right primitive for fixed total-volume targets.

package/README.md

@@ -272,24 +272,54 @@ hook: function(record, type, meta) {
 4. to drop events, use the `everything` hook and return a filtered array. don't return `{}` from event hooks (creates broken events)
 5. the `everything` hook is the most powerful. it sees all events for one user, has access to `meta.profile`, and can correlate across event types
 
-## timesoup
+## time shape — macro and soup
 
-timesoup controls how events are distributed across time. it uses gaussian cluster sampling layered with day-of-week and hour-of-day weighting derived from... i won't tell you. a prize goes to whoever can guess. the result is realistic temporal patterns: weekday peaks, weekend valleys, morning surges, afternoon dips.
+two orthogonal axes shape how events are distributed in time:
+
+- **`macro`** — big-picture trend across the whole window (births, growth, decline). default: `"flat"`.
+- **`soup`** — intra-week and intra-day rhythm (DOW/HOD weights, peak count, deviation). default: `"growth"`.
+
+mix and match. most dungeons want `macro: "flat"` (the chart doesn't blow up at the right edge) plus a soup that gives the desired weekly/daily texture.
+
+### per-user-per-day rate
+
+`avgEventsPerUserPerDay` is the canonical event-volume primitive. born-late users get `rate × remaining_days`, not a full per-user budget compressed into a small window — that's what prevents the meteoric ramp at the right edge of the chart. `numEvents` still works as a fallback (the rate is derived from `numEvents / numUsers / numDays`), but new dungeons should set the rate directly.
+
+### macro presets
+
+| preset | trend shape | use case |
+|--------|-------------|----------|
+| `"flat"` (default) | pure weekly oscillation, no net trend | mature product; let hooks supply the story |
+| `"steady"` | slight uptrend | lightly-growing saas |
+| `"growth"` | visible acquisition story (no spike) | startup acquisition narrative |
+| `"viral"` | hockey-stick acquisition | pair with persona / feature hooks |
+| `"decline"` | sunsetting product | pair with churn hooks |
+
+```javascript
+macro: 'flat'                                          // default
+macro: 'growth'                                        // preset string
+macro: { preset: 'growth', percentUsersBornInDataset: 40 }  // preset + override
+macro: { bornRecentBias: 0, percentUsersBornInDataset: 15, preExistingSpread: 'uniform' }  // fully custom
+```
+
+## timesoup (intra-week / intra-day rhythm)
+
+timesoup controls the texture of events inside the macro trend. it uses gaussian cluster sampling layered with day-of-week and hour-of-day weighting derived from... i won't tell you. a prize goes to whoever can guess. the result is realistic temporal patterns: weekday peaks, weekend valleys, morning surges, afternoon dips.
 
 ### presets
 
 ```javascript
-soup: 'growth'    // default. gradual uptrend with weekly cycles
+soup: 'growth'    // default. real-world weekly + daily rhythm
 ```
 
 | preset | pattern | use case |
 |--------|---------|----------|
-| `"steady"` | flat, minimal variation | mature saas, utility apps |
-| `"growth"` | gradual uptrend + weekly cycle | general purpose (default) |
+| `"steady"` | tighter clustering | mature saas, utility apps |
+| `"growth"` | standard intra-week rhythm | general purpose (default) |
 | `"spiky"` | dramatic peaks and valleys | gaming, social, viral products |
 | `"seasonal"` | 3-4 major waves | ecommerce, education |
 | `"global"` | flat DOW + flat HOD | global saas, infrastructure |
-| `"churny"` | flat, no growth trend | declining products (pair with churn hooks) |
+| `"churny"` | standard rhythm | pair with `macro: "decline"` for declining shape |
 | `"chaotic"` | wild variation | anomaly detection, incident response |
 
 ### custom configuration
@@ -401,13 +431,18 @@ ordering strategies: `sequential`, `random`, `first-fixed`, `last-fixed`, `first
 
 ## user generation
 
-users are generated with configurable birth distributions. `percentUsersBornInDataset` controls how many users were "created" during the dataset window vs. pre-existing. `bornRecentBias` skews new user creation toward recent dates (0 = uniform, 1 = heavily recent).
+users are generated with configurable birth distributions, normally controlled via the `macro` preset (see "time shape" above). these three knobs can also be set directly on the dungeon config — they override the preset's values.
+
+| knob | range | effect |
+|------|-------|--------|
+| `percentUsersBornInDataset` | 0..100 | share of users created inside the window vs pre-existing |
+| `bornRecentBias` | -1..1 | birth-date skew. negative = early, 0 = uniform, positive = recent |
+| `preExistingSpread` | `"uniform"` \| `"pinned"` | placement of pre-existing users' first event time |
 
 ```javascript
 {
   numUsers: 10_000,
-  percentUsersBornInDataset: 25,  // 25% of users sign up during the time window
-  bornRecentBias: 0.5             // new users skew toward recent dates
+  macro: { preset: 'growth', percentUsersBornInDataset: 40, bornRecentBias: 0.5 }
 }
 ```
 
@@ -482,7 +517,8 @@ see [types.d.ts](types.d.ts) for the complete `Dungeon` interface. here are the
 | property | type | default | description |
 |----------|------|---------|-------------|
 | `numUsers` | number | 1000 | number of users to generate |
-| `numEvents` | number | 100000 | target event count |
+| `numEvents` | number | 100000 | target event count (legacy fallback; derived from `avgEventsPerUserPerDay` when set) |
+| `avgEventsPerUserPerDay` | number | derived | per-user-per-day rate (canonical event-volume primitive) |
 | `numDays` | number | 30 | days the dataset spans |
 | `seed` | string | random | RNG seed for reproducibility |
 | `format` | string | `'csv'` | output format (csv, json, parquet) |
@@ -494,9 +530,11 @@ see [types.d.ts](types.d.ts) for the complete `Dungeon` interface. here are the
 | `strictEventCount` | boolean | false | stop at exact numEvents |
 | `batchSize` | number | 2500000 | records before auto-flush |
 | `concurrency` | number | 1 | parallel user generation |
-| `soup` | string/object | `'growth'` | time distribution preset |
-| `bornRecentBias` | number | 0.3 | user birth date skew (0-1) |
-| `percentUsersBornInDataset` | number | 15 | % of users born in time window |
+| `macro` | string/object | `'flat'` | big-picture trend preset (flat/steady/growth/viral/decline) |
+| `soup` | string/object | `'growth'` | intra-week / intra-day rhythm preset |
+| `bornRecentBias` | number | 0 (from macro `flat`) | user birth date skew (-1..1) |
+| `percentUsersBornInDataset` | number | 15 (from macro `flat`) | % of users born in time window |
+| `preExistingSpread` | string | `'uniform'` (from macro `flat`) | placement of pre-existing users' first event |
 | `hook` | function/string | passthrough | data transformation function |
 | `hasLocation` | boolean | false | include geo properties |
 | `hasCampaigns` | boolean | false | include UTM properties |

package/dungeons/technical/ad-spend.js

@@ -2,7 +2,7 @@
 const SEED = "ad spend test";
 const num_days = 90;
 const num_users = 1_000;
-const avg_events_per_user = 50;
+const avg_events_per_user_per_day = 0.56;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -39,7 +39,7 @@ const config = {
 	seed: SEED,
 	name: "ad-spend",
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json',
 	region: "US",

package/dungeons/technical/anonymous-users.js

@@ -2,7 +2,7 @@
 const SEED = "anonymous-users";
 const num_days = 180;
 const num_users = 1_000;
-const avg_events_per_user = 50;
+const avg_events_per_user_per_day = 0.28;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -30,7 +30,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: "json",
 	region: "US",

package/dungeons/technical/array-of-object-lookup.js

@@ -2,7 +2,7 @@
 const SEED = "dm4-array-of-object-lookup";
 const num_days = 60;
 const num_users = 1_000;
-const avg_events_per_user = 100;
+const avg_events_per_user_per_day = 1.67;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -54,7 +54,7 @@ const config = {
 	seed: SEED,
 	name: "array-of-object-lookup",
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json', //csv or json
 	region: "US",

package/dungeons/technical/experiments.js

@@ -2,7 +2,7 @@
 const SEED = "experiments";
 const num_days = 60;
 const num_users = 2_000;
-const avg_events_per_user = 50;
+const avg_events_per_user_per_day = 0.83;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -30,7 +30,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: "json",
 	region: "US",

package/dungeons/technical/foobar.js

@@ -2,7 +2,7 @@
 const SEED = "dm4-foobar";
 const num_days = 30;
 const num_users = 4_000_000;
-const avg_events_per_user = 500;
+const avg_events_per_user_per_day = 16.67;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -166,7 +166,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json', //csv or json
 	region: "US",

package/dungeons/technical/group-analytics.js

@@ -2,7 +2,7 @@
 const SEED = "group-analytics";
 const num_days = 60;
 const num_users = 500;
-const avg_events_per_user = 60;
+const avg_events_per_user_per_day = 1;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -30,7 +30,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: "json",
 	region: "US",

package/dungeons/technical/mirror-strategies.js

@@ -2,7 +2,7 @@
 const SEED = "mirror-strategies";
 const num_days = 30;
 const num_users = 500;
-const avg_events_per_user = 60;
+const avg_events_per_user_per_day = 2;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -29,7 +29,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: "json",
 	region: "US",

package/dungeons/technical/nested-objects.js

@@ -2,7 +2,7 @@
 const SEED = "dm4-nested-objects";
 const num_days = 30;
 const num_users = 500;
-const avg_events_per_user = 60;
+const avg_events_per_user_per_day = 2;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -44,7 +44,7 @@ const config = {
 	seed: SEED,
 	name: "nested-objects",
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json',
 	region: "US",

package/dungeons/technical/retention-cadence.js

@@ -2,7 +2,7 @@
 const SEED = "kurby-retention";
 const num_days = 360;
 const num_users = 100;
-const avg_events_per_user = 50;
+const avg_events_per_user_per_day = 0.14;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -110,7 +110,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	hasAnonIds: false,
 	hasSessionIds: false,
@@ -127,7 +127,6 @@ const config = {
 	hasAvatar: false,
 	concurrency: 1,
 	writeToDisk: false,
-	percentUsersBornInDataset: 100,
 
 	funnels: [],
 

package/dungeons/technical/sanity.js

@@ -2,7 +2,7 @@
 const SEED = "foo bar";
 const num_days = 90;
 const num_users = 500;
-const avg_events_per_user = 100;
+const avg_events_per_user_per_day = 1.11;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -43,7 +43,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json', //csv or json
 	region: "US",

package/dungeons/technical/scale-test.js

@@ -2,7 +2,7 @@
 const SEED = "scale test";
 const num_days = 365;
 const num_users = 10_000;
-const avg_events_per_user = 50;
+const avg_events_per_user_per_day = 0.14;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -38,7 +38,7 @@ const config = {
 	seed: SEED,
 	name: "scale-test",
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json',
 	gzip: true,

package/dungeons/technical/scd.js

@@ -2,7 +2,7 @@
 const SEED = "simple is best";
 const num_days = 30;
 const num_users = 500;
-const avg_events_per_user = 100;
+const avg_events_per_user_per_day = 3.33;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -64,7 +64,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'csv', //csv or json
 	region: "US",

package/dungeons/technical/simple.js

@@ -2,7 +2,7 @@
 const SEED = "simple is best";
 const num_days = 100;
 const num_users = 5_000;
-const avg_events_per_user = 100;
+const avg_events_per_user_per_day = 1;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -73,7 +73,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json', //csv or json
 	region: "US",

package/dungeons/technical/simplest.js

@@ -2,7 +2,7 @@
 const SEED = "simple is best";
 const num_days = 100;
 const num_users = 2_500;
-const avg_events_per_user = 100;
+const avg_events_per_user_per_day = 1;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -47,7 +47,7 @@ const config = {
 	token,
 	seed: SEED,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	format: 'json', //csv or json
 	region: "US",

package/dungeons/technical/text-generation.js

@@ -2,7 +2,7 @@
 const SEED = "SUPER DUPER DANGEROUS BROOO";
 const num_days = 92;
 const num_users = 8_000;
-const avg_events_per_user = 120;
+const avg_events_per_user_per_day = 1.3;
 let token = "your-mixpanel-token";
 
 // ── env overrides ──
@@ -407,7 +407,7 @@ const dungeon = {
 	seed: SEED,
 	token,
 	numDays: num_days,
-	numEvents: num_users * avg_events_per_user,
+	avgEventsPerUserPerDay: avg_events_per_user_per_day,
 	numUsers: num_users,
 	hasAnonIds: false,
 	hasSessionIds: true, // Enable for chat flow tracking