@ak--47/dungeon-master 1.5.4 → 1.6.0

package/docs/guides/1.5.0-upgrade-guide.md

@@ -0,0 +1,426 @@
+# 1.5.0 Upgrade Guide
+
+**TL;DR: No breaking API changes.** Existing dungeons run without modification. Six behavioral changes affect generated output — all produce more correct data for Mixpanel analysis.
+
+## What Changed
+
+### 1. `bunchIntoSessions` Removed
+
+**What**: The function that rewrote all event timestamps into synthetic session clusters is deleted. Events now use their natural TimeSoup-generated timestamps. `assignSessionIds` (which was already running) provides session IDs based on 30-minute gaps.
+
+**Why**: `bunchIntoSessions` overwrote the exact timestamps that funnels depended on. A funnel generating events in sequence (step A → step B → step C) would have its ordering scrambled when timestamps were reassigned into session clusters. This made greedy single-pass funnel evaluation (Mixpanel's actual algorithm) unreliable.
+
+**Impact on your data**:
+- Event timestamps are more spread out (follow TimeSoup's Gaussian distribution instead of tight session clusters)
+- Funnel step ordering is now temporally correct — step A always occurs before step B
+- Session IDs still exist when `hasSessionIds: true`; they're computed from natural 30-minute gaps instead of artificial clusters
+- If your hooks or downstream analysis relied on events being tightly clustered in ~30min sessions, you'll see looser temporal distribution
+
+**Action needed**: None. The new behavior is strictly more correct for Mixpanel analysis.
+
+### 2. `isStrictEvent` Auto-Promote
+
+**What**: The config validator now detects events that appear in both `events[]` and any `funnels[].sequence` and auto-promotes them to `isStrictEvent: true`. This prevents standalone instances of funnel-step events from confounding greedy funnel counting.
+
+**Why**: Mixpanel's funnel engine uses greedy single-pass matching. If a standalone "sign up" event appears between two funnel runs, the greedy matcher counts it as a funnel conversion — producing phantom conversions that don't correspond to actual user journeys.
+
+**Impact on your data**:
+- Events that are funnel steps will only appear inside funnel sequences, not as standalone events
+- You'll see auto-promote warnings in stderr (these are informational)
+- Total event count may decrease if you had standalone instances of funnel-step events
+- If your hooks count standalone "sign up" events outside funnel context, those events won't exist anymore
+
+**Action needed**: If you need standalone instances of a funnel-step event, add `isStrictEvent: false` on that event's config:
+
+```javascript
+events: [
+  { event: 'sign up', weight: 3, isStrictEvent: false },  // keep standalone instances
+  { event: 'purchase', weight: 5 },  // auto-promoted if in a funnel
+]
+```
+
+### 3. Touchpoint Cap (10 per user)
+
+**What**: UTM stamping now samples up to `maxTouchpointsPerUser` (default 10) eligible events per user, matching Mixpanel's `attributed_value_reader.cpp` 10-touchpoint cap. Replaces the old behavior of stamping ~25% of all events inline.
+
+**Why**: Mixpanel's attribution engine ignores touchpoints beyond the first 10. Generating 25% of all events with UTMs produced unrealistic attribution distributions that couldn't be reproduced in Mixpanel queries.
+
+**Impact on your data**:
+- Fewer events have UTM parameters (~10 per user instead of ~25% of all events)
+- Attribution analysis in Mixpanel will match what `emulateBreakdown` computes
+- Events without `isAttributionEvent: true` are eligible for touchpoint selection
+
+**Action needed**: None for most dungeons. If your hooks depend on a specific density of UTM-stamped events, set `maxTouchpointsPerUser` higher:
+
+```javascript
+{ maxTouchpointsPerUser: 50 }  // more touchpoints for attribution-heavy analysis
+```
+
+### 4. Auto-Sort After `everything` Hook
+
+**What**: After the `everything` hook fires, the engine auto-sorts all user events by timestamp (default `autoSortAfterEverything: true`).
+
+**Why**: Hooks that inject events (e.g., `injectBetween`, `injectBurst`, event cloning) often produce out-of-order timestamps. The greedy funnel engine requires strictly ordered events. Auto-sort guarantees correctness without requiring hook authors to manually sort.
+
+**Impact on your data**:
+- Events are always in chronological order per user
+- Hooks no longer need `events.sort((a,b) => ...)` at the end
+- If a hook intentionally produces non-chronological order (rare), set `autoSortAfterEverything: false`
+
+**Action needed**: None. Remove manual sort calls from `everything` hooks if you had them.
+
+### 5. Conversion Window on Funnels
+
+**What**: New `conversionWindowDays` field on funnels (default 30, hard cap 180). Validator auto-bumps when `timeToConvert` exceeds the default. Generator caps step-to-step time spans to the window.
+
+**Why**: Mixpanel applies a conversion window (`conversion_window.cpp`) with strict `<` comparison. Events outside the window don't count as conversions. The generator must respect this so that verification and actual Mixpanel reports agree.
+
+**Impact on your data**:
+- Funnels with `timeToConvert` > 30 days will have `conversionWindowDays` auto-set to `ceil(TTC * 1.5)` (capped at 180)
+- You'll see a warning: `"Funnel X: timeToConvert exceeds default 30d conversion window. Auto-set conversionWindowDays=Y."`
+- Generated funnel events are guaranteed to complete within the conversion window
+
+**Action needed**: None for most dungeons. To silence the warning, set `conversionWindowDays` explicitly:
+
+```javascript
+funnels: [{
+  sequence: ['trial start', 'trial end', 'subscribe'],
+  timeToConvert: 45,           // 45 days between steps
+  conversionWindowDays: 60,    // explicit window
+}]
+```
+
+### 6. Sessions Split on UTC Day Boundary
+
+**What**: `assignSessionIds` now ends a session whenever the UTC day index changes between consecutive events. Three reset triggers: timeout gap > 30 min, max session > 24h, OR day-index change.
+
+**Why**: Mixpanel `session_query.cpp:828-830, 911` terminates sessions on day boundary in query timezone. Without this, a session crossing midnight would be ONE session in our generator but TWO in Mixpanel — verification numbers would diverge.
+
+**Impact on your data**:
+- Sessions per user may be HIGHER than 1.4 baselines (sessions spanning midnight now split)
+- Session duration aggregates shift (no more multi-day sessions)
+- Funnel evaluation with `sessionScoped: true` sees more session boundaries
+
+**Action needed**: None. Update any test expectations that asserted exact session counts under the old behavior.
+
+### 7. Engine Validation Strict Clamps (post-eval ship gate)
+
+**What**: 1.5.0 added a 194-combo engine-validation sweep that proves the no-hook baseline produces in-band charts across the param space. It also added 7 validator strict-clamps that override pathological knob combinations at config time with a `console.warn` explaining what changed.
+
+**Why**: Earlier 1.5 builds could produce nosedive (last-day cliff) on funnel-heavy dungeons OR right-edge explosion on `percentUsersBornInDataset: 100` + `bornRecentBias: 0.6` configs. Both look broken in Mixpanel charts. The engine fix (`FUNNEL_DEAD_ZONE_CAP_SEC = 0`) eliminates the cliff; the clamps catch the explosion at config time before it ships.
+
+**Engine fix**: `lib/orchestrators/user-loop.js` removed the 1-day "dead zone" before `FIXED_NOW` for funnel step-1 anchors. Earlier defense-in-depth that became unnecessary once the cursor-accumulation bug was fixed in round 1; the future-time guard at storage step 14 catches anything past `FIXED_NOW`. Funnels can now anchor right up to `FN`. **Verified: `futureEvents == 0` across the entire 194-combo matrix.**
+
+**Validator clamps** (each emits one `console.warn` per validation pass):
+
+| # | Clamp | When it fires |
+|---|-------|--------------|
+| 1 | `percentUsersBornInDataset` ∈ `[0, 100]` | Always (data sanity) |
+| 2 | Per-macro born cap (flat=12, steady=12, growth=30, viral=55, decline=5) | User explicitly sets BOTH `macro` AND `percentUsersBornInDataset` |
+| 3 | `bornRecentBias` ∈ `[-0.5, 0.5]` | User-explicit (top-level OR `macro: { ..., bornRecentBias }`) |
+| 4 | Compound `born > 60 && bias > 0.4` → bias=0.3 | Either user-explicit |
+| 5 | `bornRecentBias` ∈ `[-1, 1]` (`Math.pow` guard) | Always |
+| 6 | `avgEventsPerUserPerDay > 50` → 50 | Always |
+| 7 | `avgActiveDaysPerUser > numDays * 0.5` → `floor(numDays * 0.5)` | Always |
+
+Plus a warning-only check for `numDays < 14`.
+
+**Impact on your data**:
+- **None for 1.5.0 and earlier vertical dungeons** — none of them set `percentUsersBornInDataset`, `bornRecentBias`, or `avgEventsPerUserPerDay > 50` explicitly. All clamps are no-ops on existing dungeons.
+- **Macro presets are exempt from clamps 2-4** — `macro: 'viral'` uses bias=0.6 + born=55 by design and continues to work unchanged.
+- **New dungeons that set `macro: 'flat'` + `percentUsersBornInDataset: 100`** will see born clamped to 12 with a warning. Switch to `macro: 'growth'` or `macro: 'viral'` to genuinely run with high born%.
+- **Dungeons that set `macro: { preset: 'growth', bornRecentBias: 0.7 }` (object override form)** were previously bypassing clamps; v1.5 final closes this backdoor — bias clamps to 0.5.
+
+**Action needed**: Watch stderr for `⚠️ ... clamped to ... To suppress, fix the config.` lines on first run. Adjust config or accept clamped values.
+
+**The 6 strict-bar conditions** — what counts as "in-band" output (per macro):
+
+| Macro   | tail band     | spike cap | l7c min |
+|---------|---------------|-----------|---------|
+| flat    | `[0.85, 1.5]` | 2.5       | 0.5     |
+| steady  | `[0.85, 1.7]` | 2.5       | 0.5     |
+| growth  | `[0.85, 2.5]` | 3.5       | 0.45    |
+| viral   | `[0.5, 5.0]`  | 7.0       | 0.3     |
+| decline | `[0.4, 2.0]`  | 3.0       | 0.3     |
+
+Plus `lastDay >= 0.7 * sameDowPrev` (DOW-fair comparison; relaxed to 0.6 in `avgActiveDaysPerUser` mode), `futureEvents == 0`, and a signup-floor check (bypassed for `mean < 5/day` or `macro === 'decline'`).
+
+**Engine guarantees apply to no-hook configs only.** Hooks own their shape — engagementDecay churn cohorts can produce tail < 0.4, viral hooks with persona-driven late-cohort lift can exceed the spike cap. Document intentional deviations in your dungeon's overview JSDoc.
+
+**Reference**: `plans/ENGINE-VALIDATION/PLAN.md` (spec), `plans/ENGINE-VALIDATION/FIX.md` (sweep evidence), `tests/engine/sweep-engine.mjs` (harness), `tests/unit/engine-shape-canary.test.js` (commit-time gate), `tests/e2e/engine-shape-full-sweep.test.js` (`RUN_FULL_SWEEP=1` pre-release gate).
+
+## New Features (Opt-In)
+
+### `avgActiveDaysPerUser`
+
+Concentrates events onto fewer distinct UTC days per user. A user with `avgActiveDaysPerUser: 5` in a 30-day window will have events on ~5 distinct days (normal distribution, sd ≈ target/3).
+
+```javascript
+{
+  avgActiveDaysPerUser: 5,
+  avgEventsPerUserPerDay: 4,  // 4 events per ACTIVE day, not per calendar day
+}
+```
+
+Interacts with:
+- **`engagementDecay`**: Decay protects the last event on each picked active day from being dropped
+- **`soup.dayOfWeekWeights`**: Active days are picked using DOW weights (weekday-biased by default)
+- **Funnel generation**: First funnel event for a user lands on the first picked active day
+- **`everything` hooks**: Events are already concentrated when the hook fires
+
+### `maxTouchpointsPerUser`
+
+Control the touchpoint cap per user. Default 10 matches Mixpanel.
+
+```javascript
+{ maxTouchpointsPerUser: 20 }  // more attribution data
+```
+
+### `autoSortAfterEverything`
+
+Disable auto-sort if your hook intentionally produces non-chronological event sequences.
+
+```javascript
+{ autoSortAfterEverything: false }  // hook manages its own ordering
+```
+
+## Migration Checklist
+
+For most dungeons: **nothing to do.** Upgrade and run.
+
+If you want to fine-tune:
+
+1. **Run your dungeon and check for auto-promote warnings.** Add `isStrictEvent: false` where you need standalone instances of funnel-step events.
+2. **Review hooks that count UTM events.** Touchpoint cap means fewer UTM events per user.
+3. **Remove manual sort calls** from `everything` hooks. Auto-sort handles it.
+4. **Add `avgActiveDaysPerUser`** if you want realistic usage patterns (most real products have users active on 3-10 days per month, not every day).
+5. **Run `/verify-dungeon`** to confirm hooks still produce expected patterns under v1.5 engine behavior.
+
+## Verifier Updates
+
+`emulateBreakdown` now:
+- Auto-applies `conversionWindowDays` from matched funnel config
+- Auto-dispatches `evaluateAnyOrderCompletion` for non-sequential funnel modes
+- Passes `funnelOrder` through to `funnelFrequency` and `timeToConvert` analyses
+- **Auto-builds identity map** when `profiles` are passed — pre-auth `device_id`
+  events resolve to the same canonical user as post-auth `user_id` events.
+  Apply this to ANY identity-model dungeon (`avgDevicePerUser > 0` or
+  `hasAnonIds: true`). The map is built ONCE and reused across `timeBucket`
+  recursion (no per-bucket rebuild).
+- **Threads new funnel options** (`reentry`, `exclusionSteps`,
+  `trackStepProperties`, `sessionScoped`) for `funnelFrequency` and
+  `timeToConvert`.
+- **Applies funnel-level v1.5.0 hints** when matching (verify-dungeon):
+  `funnel.reentry`, `funnel.exclusionEvents`, `funnel.stepFilters`.
+- **New types** — `'sessionMetrics'` and `'retention'` breakdowns; cross-cutting
+  `timeBucket: 'day' | 'week' | 'month'` on every type.
+
+### Complete module exports
+
+`@ak--47/dungeon-master/verify` (resolved from `lib/verify/index.js`):
+
+```javascript
+import {
+  // top-level orchestration
+  emulateBreakdown,            // dispatch on { type: 'frequencyByFrequency' | 'funnelFrequency'
+                               //              | 'aggregatePerUser' | 'timeToConvert' | 'attributedBy'
+                               //              | 'sessionMetrics' | 'retention' }
+  verifyDungeon,               // run dungeon + run check[] against output, return structured report
+
+  // schema validation
+  deriveExpectedSchema,        // (Dungeon) → Map<eventType, Set<column>>
+  validateSchema,              // (events, Dungeon) → SchemaReport (catches flag stamping)
+
+  // funnel engine + extensions
+  evaluateFunnel,              // greedy single-pass + reentry/exclusion/step-filter/sessionScoped/totals
+  evaluateFunnelHPC,           // Hold Property Constant — parallel sub-funnels per holdProperty value
+  resolveFunnelSegment,        // (FunnelResult, 'first'|'last'|{step:N}) → step properties
+  evaluateAnyOrderCompletion,  // set-membership check for non-sequential funnel modes
+  normalizeStep,               // FunnelStep → { event, where? }
+  matchesStepFilter,           // (event, where?) → bool — step where-clause check
+
+  // funnel internals (also exported for advanced use)
+  timestampComesAfter,         // (t1, t2, graceperiod=true) — Mixpanel 2s grace rule
+  withinConversionWindow,      // (eventTime, step0Time, windowMs) — strict <
+
+  // identity resolution
+  buildIdentityMap,            // (profiles[]) → Map<device_id, canonical_user_id>
+  resolveUserId,               // (event, identityMap?) → canonical id (distinct_id wins)
+
+  // counting + aggregation primitives
+  countDistinctPeriods,        // (events, eventName, unit='day', { algorithm }) — Mixpanel addiction_query parity
+  binByDistinctPeriods,        // bin a user by distinct-period count
+  nullAwareAvg,                // skips null/NaN/non-numeric (normal_query parity)
+  nullAwareSum,
+  nullAwareExtreme,            // (values, 'min'|'max')
+
+  // time-bucketing
+  partitionByTimeBucket,       // (events, 'day'|'week'|'month', { from, to }?) — empty-backfill optional
+} from '@ak--47/dungeon-master/verify';
+```
+
+All of these are stable v1.5.0 surface. JSDoc on each function points to the
+specific Mixpanel C++ source file it mirrors.
+
+### New `Funnel` config fields (all optional, opt-in)
+
+```javascript
+funnels: [{
+  sequence: ['land', 'sign_up'],
+  conversionRate: 30,
+  exclusionEvents: ['rage_click'],          // generator + verifier
+  reentry: true,                            // verifier-only hint
+  stepFilters: { 1: { prop: 'plan', op: 'eq', value: 'pro' } },  // verifier-only
+}]
+```
+
+> **Note:** `Funnel.holdPropertyConstant` was considered and dropped during the
+> Mixpanel parity audit. HPC requires a different report shape than
+> `funnelFrequency` (one row per `step × hpc_value` instead of per `step ×
+> cohort`), so it cannot be auto-routed. Call `evaluateFunnelHPC` directly
+> inside your `verifyDungeon` check's `assert` callback when you need it.
+
+### Identity-aware verification (REQUIRED for identity-model dungeons)
+
+If you set `hasAnonIds: true` or `avgDevicePerUser > 0`, ALWAYS pass
+`profiles` to `emulateBreakdown`:
+
+```javascript
+const result = await DUNGEON_MASTER(config);
+const events = Array.from(result.eventData);
+const profiles = Array.from(result.userProfilesData);
+
+emulateBreakdown(events, {
+  type: 'funnelFrequency',
+  steps: ['visit_landing', 'sign_up', 'first_action'],
+  breakdownByFrequencyOf: 'visit_landing',
+  profiles,                                 // ← required for correct grouping
+});
+```
+
+Without `profiles`, pre-auth `device_id` events bucket as separate "users";
+funnel completion drops, retention deflates, attribution mis-routes.
+
+## Mixpanel Parity Semantics (post-audit)
+
+The verifier was audited against `mixpanel/analytics` C++ source after the
+initial implementation. These are non-obvious behaviors callers should know:
+
+### Retention buckets are ms-delta from birth (not calendar-day diff)
+
+Reference: `retention_query.cpp:1227-1231`. Bucket = `floor((return_ms -
+birth_ms) / DAY_MS)`. A return 23h after a birth at 6pm UTC lands in **bucket 0**
+(within 24h); a return 25h later lands in **bucket 1** — even when both fall
+on the next UTC calendar day.
+
+```javascript
+emulateBreakdown(events, {
+  type: 'retention',
+  cohortEvent: 'Sign Up',
+  returnEvent: 'Login',
+  dayBuckets: [0, 1, 7, 14, 30],  // bucket 0 = within 24h
+  birthCanRetain: false,           // (default) excludes returns at exact birth ms
+});
+```
+
+Naive intuition ("day 1 = next calendar day") will give the wrong answer.
+
+### `aggregatePerUser` cohort-level columns
+
+When `agg: 'sum'`, `'min'`, or `'max'`, the result row includes BOTH:
+- `avg_aggregate` — mean of per-user aggregates within the cohort
+- `cohort_sum` / `cohort_min` / `cohort_max` — the same `agg` applied across users
+
+Pick the column matching the Mixpanel report you're verifying. Mixpanel's
+"Aggregate per user → SUM" report shows the cohort sum, not the mean of sums.
+
+### `timeBucket` empty-bucket markers
+
+`timeBucketRange: { from, to }` enumerates every bucket in the range and emits
+`{ period, _empty: true }` for buckets with no events (Mixpanel
+`normal_query.cpp:352-356` parity). **Consumers MUST filter `r._empty`**
+before any numerical aggregation:
+
+```javascript
+const rows = emulateBreakdown(events, {
+  type: 'frequencyByFrequency',
+  metricEvent: 'Purchase',
+  breakdownByFrequencyOf: 'Browse',
+  timeBucket: 'day',
+  timeBucketRange: { from: '2026-01-01', to: '2026-01-31' },
+});
+const dataRows = rows.filter(r => !r._empty);  // skip backfilled markers
+```
+
+### `countMode: 'totals'` semantics
+
+`evaluateFunnel({ countMode: 'totals' })` returns `FunnelResult[]` of ALL
+attempts including INCOMPLETE drop-offs (`funnel_query.cpp:1747` aggregates
+`reached >= 0`, not "completed"). Without `reentry: true`, returns a
+single-attempt array. The "totals requires reentry" constraint that existed
+in the initial impl was removed during the audit.
+
+### Identity resolver order
+
+`resolveUserId(event, identityMap)` priority:
+1. `event.distinct_id` (Mixpanel canonical post-merge id; never demoted)
+2. `identityMap.get(event.device_id)` (device→user merge from profile inversion)
+3. `event.user_id`
+4. `event.device_id`
+
+The v1.5 generator does NOT stamp `distinct_id` on raw events — that branch
+exists for callers feeding in already-stitched data from external pipelines.
+If a hook accidentally writes `distinct_id` onto event records, identity
+merge breaks; strip it.
+
+### Funnel exclusion `afterStep` / `beforeStep`
+
+`exclusionSteps: [{event, afterStep, beforeStep}]` — `afterStep` is the index
+of the step that must HAVE BEEN reached; `beforeStep` is the step that must NOT
+have been reached yet. Range check: `reached >= afterStep && reached < beforeStep`.
+
+Defaults: `afterStep = -Infinity`, `beforeStep = steps.length`. With these, the
+exclusion fires anywhere in the attempt — used by the simple shape that
+`Funnel.exclusionEvents: ['X']` produces. For "exclusion only between step 1
+and step 2," set explicit `{event: 'X', afterStep: 1, beforeStep: 2}`.
+
+### Generator: exclusion event injection is schema-clean
+
+When the generator stamps an exclusion event for a non-converter, it copies
+ONLY identity (`user_id`, `device_id`, `distinct_id`, `session_id`,
+`insert_id`) + super props + group keys + props declared on the exclusion
+event's own config. Source-event-specific props (e.g., `cart_value` on `Add
+to Cart`) WILL NOT bleed onto a different event type. Schema validator
+passes cleanly.
+
+## Documented divergences (intentional v1.5.0 scope)
+
+These were considered and explicitly out of scope:
+
+- **HPC list-property values** — scalar only. Mixpanel
+  `aggregate_hash_get_key_cursor` explodes list values into N sub-funnels per
+  event; we don't.
+- **`sessionScoped` funnel** + **`sessionMetrics({ event })` filter** —
+  verifier-only conveniences; not directly reproducible in Mixpanel UI.
+- **Retention COMPOUNDED, CARRY_BACK, CONSECUTIVE_FORWARD, CALENDAR_START,
+  segment_event=SECOND, cohort window, week/month bucket units** — out of
+  scope for v1.5.0.
+- **Timezone** — verifier uses UTC; Mixpanel uses query timezone (qtz).
+  Bucket boundaries shift by hours for non-UTC accounts.
+- **Percentiles** — `sessionMetrics` p50/p90 use linear interpolation;
+  Mixpanel uses TDigest (single-digit % divergence at p90 on small N).
+- **Selector grammar** — eq/neq/gt/lt/gte/lte/contains/not_contains only;
+  no is_set/between/regex/contains_ci.
+
+## Test Suite
+
+Full suite after v1.5.0 (engine + verifier + audit fixes): **46 files,
+1108 tests, 1 skipped** (~9s wall time). New files cover:
+- `tests/unit/identity-resolution.test.js` (14)
+- `tests/unit/session-metrics.test.js` (11) — includes generator UTC day-boundary split parity
+- `tests/unit/retention.test.js` (9) — ms-delta bucketing + birthCanRetain
+- `tests/unit/time-bucketed.test.js` (11) — day/week/month + empty backfill
+- `tests/unit/funnel-engine.test.js` extended to 51 tests (was 18) — reentry, exclusion (default + explicit), totals (with incomplete attempts), HPC, step filters, step properties, segment modes, sessionScoped. Includes 5+ ported fixtures from `mixpanel/analytics test_qt_funnel.py`.
+- `tests/integration/hook-patterns-emulator.test.js` extended (+7) — funnel option threading + new breakdown types + `aggregatePerUser` SUM/MAX cohort columns
+- `tests/integration/features.test.js` extended (+4) — `exclusionEvents` validator + injection + schema-clean clone roundtrip
+- `tests/integration/identity-model.test.js` extended (+1) — `emulateBreakdown` profile-merge round-trip

package/docs/guides/1.5.1-upgrade-guide.md

@@ -0,0 +1,178 @@
+# 1.5.1 Upgrade Guide
+
+**TL;DR: No breaking API changes.** Old top-level config keys keep working through 1.5.x. Most dungeons run without modification. Two behavioral changes are worth knowing about: `numEvents` is now accurate (was over-shooting by 1.6-2x), and anonymous non-converters get `_drop: true` stamped on their profile.
+
+## What Changed
+
+### 1. Config Sub-Object API (`credentials` / `switches` / `identity`)
+
+**What**: Three new optional config sub-objects group related keys:
+
+```js
+const config = {
+    seed: SEED,
+    numUsers: 10_000,
+    datasetStart: "2026-01-01T00:00:00Z",
+    datasetEnd: "2026-05-01T23:59:59Z",
+    avgEventsPerUserPerDay: 2,
+
+    // NEW shape (recommended)
+    credentials: {
+        token,
+        region: "US",
+        serviceAccount,
+        serviceSecret,
+        projectId,
+    },
+    switches: {
+        hasLocation: true,
+        hasCampaigns: true,
+        hasSessionIds: true,
+        hasIOSDevices: true,
+        hasAndroidDevices: true,
+        // ...
+    },
+    identity: {
+        avgDevicePerUser: 2,
+    },
+
+    events: [...],
+    funnels: [...],
+};
+```
+
+**Why**: Flat configs with 20+ keys at the top level became hard to scan. Grouped sub-objects make the dungeon's intent clearer at a glance.
+
+**Action needed**: None. The old flat shape keeps working. `mergeConfigSubObjects` hoists sub-object values into the top-level keys at validation time. When verbose, the validator emits a one-time warning per dungeon nudging migration. All 48 shipped dungeons have already moved.
+
+### 2. `hasAnonIds` Deprecated
+
+**What**: `hasAnonIds: true` is a deprecated alias. Use `identity.avgDevicePerUser: 1` (or higher) instead.
+
+```js
+// Before
+{ hasAnonIds: true }
+
+// After
+{ identity: { avgDevicePerUser: 1 } }
+```
+
+**Why**: `hasAnonIds` was a binary flag that conflated "users have a device pool" with "device pool size = 1." The new shape exposes the pool size directly.
+
+**Action needed**: Optional — `hasAnonIds: true` still maps to `avgDevicePerUser: 1` at validation. The validator emits a deprecation note in verbose mode.
+
+### 3. Anonymous Non-Converters Get `_drop: true`
+
+**What**: Born-in-dataset users who never reach an `isAuthEvent` step in their first funnel are anonymous. Their profile object exists in `result.userProfilesData` but carries `_drop: true`, and the Mixpanel sender filters those out before pushing to `/engage`. `result.profilesPushed` reports the count actually pushed.
+
+**Why**: Mirrors real Mixpanel `$identify` semantics — profiles only exist for users who actually identified. Pre-existing 1.5.0 behavior pushed phantom profiles for every born-in user, including anonymous ones who never called `identify()`.
+
+**Impact on your data**:
+- `userProfilesData` size is unchanged (still includes every profile).
+- Profile push count drops by the anonymous-non-converter rate (typically 30-70% of born-in users at default first-funnel conversion rates).
+- Pre-existing users (born outside the window) never get `_drop` — they're considered already-identified per our model.
+- Anonymous users' EVENTS still flow normally (tied to `device_id`).
+
+**Action needed**: If your downstream cares about every profile object regardless of identification state, read `userProfilesData` directly instead of relying on Mixpanel's `/engage` count. The `everything` hook can rescue a profile by `delete meta.profile._drop`.
+
+### 4. `numEvents` More Accurate (40-60% Fewer Events at Same Rate)
+
+**What**: Per-user event budget computation cleaned up. Removed `0.714` magic dampening + random dice rolls; replaced with a clean `chance.normal(mean=budget, dev=budget/3)`.
+
+**Why**: Old behavior overshot the target by 1.6-2x. New behavior matches the configured rate within ±3% across all 5 macros at 50K target.
+
+**Impact on your data**: If you previously tuned `avgEventsPerUserPerDay` around the overshoot, expect **~40-60% fewer events at the same rate**. Recompute targets.
+
+**Action needed**: Bump `avgEventsPerUserPerDay` to the actual rate you want. The old config that produced 2M events at `rate=1.0` will produce ~800K-1.2M events at `rate=1.0` in 1.5.1. Set `rate=2.0` (or whatever) to hit your old volume.
+
+### 5. Default `Platform` Property Removed
+
+**What**: 59 `Platform:` entries removed from `lib/templates/defaults.js`. `os` already carries the platform signal (`iOS`, `Android`, `Windows`, `macOS`, `Linux`, etc.).
+
+**Why**: Two competing properties (`Platform` and `os`) carrying the same signal led to inconsistent breakdowns and downstream confusion.
+
+**Impact on your data**: Events generated by 1.5.1 will not have a `Platform` property by default. Hooks or downstream queries that read `Platform` will see undefined.
+
+**Action needed**: If you need `Platform` for downstream compatibility, define it explicitly in your event properties:
+
+```js
+events: [
+    {
+        event: "page view",
+        properties: {
+            Platform: ["iOS", "Android", "Windows", "macOS", "Linux"],
+            // ...
+        },
+    },
+]
+```
+
+### 6. `retentionCurve` Config Knob
+
+**What**: New top-level config knob that shapes retention via log-linear interpolation. Independent of `engagementDecay`.
+
+```js
+const config = {
+    // ...
+    retentionCurve: [
+        { day: 0,  retention: 1.0 },
+        { day: 1,  retention: 0.80 },
+        { day: 7,  retention: 0.50 },
+        { day: 30, retention: 0.20 },
+    ],
+};
+```
+
+**Why**: Hand-rolling retention shapes via `engagementDecay` hooks is verbose. `retentionCurve` lets you express the intent declaratively at config level.
+
+**Action needed**: Opt-in only. No change for existing dungeons.
+
+### 7. Concurrent `generate()` Now Safe In-Process
+
+**What**: `DATASET_NOW` / `DATASET_BEGIN` scoped via `AsyncLocalStorage`. Two parallel `generate()` calls with different `datasetStart`/`datasetEnd` windows now produce isolated, in-window output.
+
+**Why**: Pre-1.5.1, those values lived as module-level mutables. Concurrent calls would race and leak each other's windows.
+
+**Action needed**: None for normal usage. If you need explicit scoping (rare), import `runWithDataset(begin, now, fn)` from `lib/utils/dataset-context.js`.
+
+### 8. Quiet by Default
+
+**What**: `verbose: false` (default) now gates all warnings, info logs, and dataset-context messages.
+
+**Why**: Test output and CI logs were dominated by harmless validator chatter. Pinning the default to quiet lets actual errors surface.
+
+**Action needed**: Set `verbose: true` to opt back into the chatty output.
+
+### 9. UTC Bare-Date Parsing
+
+**What**: `datasetStart: "2026-01-01"` now parses as `2026-01-01T00:00:00Z` (not local midnight, which shifted the window by UTC offset).
+
+**Why**: Local-midnight parsing made `datasetStart` ambiguous across timezones — the same config produced different windows depending on where it ran.
+
+**Action needed**: None unless you have a dungeon that relied on the old local-midnight behavior (none of the shipped dungeons did).
+
+### 10. Bug Fixes
+
+- **Standalone events now stamp `config.superProps`.** Was silently `{}` before — masked by the validator's auto-funnel pre-fix in 1.5.0. If your hooks/downstream noticed missing superProps on standalone events, that's fixed.
+- **Born-late funnel auth events past `FIXED_NOW` no longer set `userAuthTimeMs`.** Affected ~1-2% of users — engine dropped the event at storage time but still marked the user as authed. Anonymous-non-converter test now strictly asserts every kept profile has a sign_up event.
+- **Pre-existing user events strict-clamp at `FIXED_BEGIN`.** Born-outside-window users no longer leak events into the pre-dataset window via TimeSoup's sub-window distribution.
+
+## Migration Checklist
+
+For most users:
+
+- [ ] Read your `numEvents` output — likely ~40-60% lower than 1.5.0. Bump `avgEventsPerUserPerDay` if you want the old volume.
+- [ ] If you read `Platform` in hooks or downstream, declare it explicitly in your event properties.
+- [ ] If you push profiles to Mixpanel, note that `result.profilesPushed` may be lower than `userProfilesData.size` — `_drop:true` profiles are filtered.
+- [ ] (Optional) Migrate config keys to `credentials`/`switches`/`identity` sub-objects for cleaner files.
+- [ ] (Optional) Replace `hasAnonIds: true` with `identity.avgDevicePerUser: 1`.
+
+## What's NOT in 1.5.1
+
+Deferred to 1.6:
+
+- New HOOKS.md recipes for active-day, conversion window, and touchpoint budget.
+- RNG scoping via AsyncLocalStorage (same pattern as DATASET_NOW).
+- dayjs performance optimization (primary bottleneck).
+- Sweep harness simplification (replace `child_process.fork` with `Promise.all`).
+- Engine-level rate derivation that accounts for born-late undershoot.