@ak--47/dungeon-master 1.6.2 → 1.6.4

package/HOOKS.md

@@ -88,6 +88,58 @@ users between frequency bins. Both `injectOnNewDays` and the default
 `countDistinctPeriods` algorithm use calendar-bucket math, so they agree
 at boundaries.
 
+One second is also below Mixpanel's 30-minute session gap, so the default
+spread cannot create a new session either. The full list of metrics a default
+`scaleEventCount` call **cannot** move: active days per user, DAU, stickiness
+(DAU÷MAU), sessions per user, frequency bins, and retention. It moves event
+volume, and nothing else.
+
+As of v1.6.4 you can pass `{ spreadDays: N }` to scatter the clones across the
+next N days instead:
+
+```js
+// volume only — same day, same session
+scaleEventCount(record, "commit pushed", 3);
+
+// volume AND active days AND sessions
+scaleEventCount(record, "commit pushed", 3, { spreadDays: 7 });
+```
+
+`injectOnNewDays` remains the better tool when you want a specific target
+day count rather than a multiplier.
+
+### 2.1.1 Cloned events MUST carry a fresh `insert_id`
+
+Mixpanel deduplicates on `insert_id` at ingest. A clone that keeps its source's
+`insert_id` is silently discarded — the surge you engineered never appears in the
+project, no error is raised, and local verification does not catch it because
+`emulateBreakdown` never inspects `insert_id`.
+
+Leaving `insert_id` blank is not safe either. The importer content-hashes events
+that lack one, so identical clones hash to the same value and collide the same way.
+
+```js
+// PREFERRED — cloneEvent stamps a fresh insert_id
+record.push(cloneEvent(sourceEvent, { time: newTime }));
+
+// ALSO FINE — the engine re-stamps the duplicate id
+record.push({ ...sourceEvent, time: newTime });
+
+// HISTORICALLY BROKEN — pre-1.6.3 this deduped the whole surge away
+const clone = JSON.parse(JSON.stringify(sourceEvent));
+clone.time = newTime;
+record.push(clone);
+```
+
+Since v1.6.3 the engine re-stamps any duplicate or missing `insert_id` across each
+user's final stream ([user-loop.js:753-776](lib/orchestrators/user-loop.js#L753-L776)),
+so all three shapes now survive ingest. Prefer
+[`cloneEvent`](lib/hook-helpers/mutate.js) anyway: the engine pass is a per-user
+last resort, and it cannot help a clone that a hook moves onto a different user.
+
+If you generated data with a hand-rolled deep-copy clone on **1.6.2 or earlier**,
+that data is wrong in the project. Regenerate and re-import.
+
 ### 2.2 Funnels are GREEDY single-pass with a 2-second grace
 
 Mixpanel processes events in chronological order, single pass. Each event is
@@ -217,6 +269,15 @@ count BELOW the configured target. Pick one. If you need both effects, set
 `avgActiveDaysPerUser` and write decay logic in an `everything` hook scoped
 to specific cohorts (gives explicit control over the interaction).
 
+As of v1.6.4 the validator warns **unconditionally** when both are set — the
+warning is not gated behind `verbose`, because the combination silently returns
+an active-day count the author did not ask for.
+
+**Precedence with `retentionCurve`:** when `retentionCurve` is set, it wins.
+The active-day planner runs from the curve and `avgActiveDaysPerUser` is ignored
+([user-loop.js:326-337](lib/orchestrators/user-loop.js#L326-L337)). Set one or
+the other, not both.
+
 ### 2.6 Sessions are query-time computed (30-min gap, 24h max, day-boundary split)
 
 Reference: `backend/arb/reader/queries/session_query.cpp:828-830, 905-928`.
@@ -666,10 +727,12 @@ Reference: `flows_query.cpp:988-994` (next-anchor-only), `flows.cpp:680-717`
     still correct.
 
 22. **`scaleEventCount` does not move users between frequency bins.** Cloning
-    Buy events at sub-second offsets places them on the same calendar day, so
-    the user's distinct-day count is unchanged. To shift frequency bins use
+    Buy events at sub-second offsets places them on the same calendar day and
+    inside the same 30-minute session window, so distinct days, sessions, DAU,
+    stickiness, and retention are all unchanged. It moves event volume only. To
+    shift any of the others, pass `{ spreadDays: N }` (v1.6.4) or use
     [`injectOnNewDays`](lib/hook-helpers/inject.js), which spreads injections
-    across previously empty days within the user's active window.
+    across previously empty days within the user's active window. See §2.1.
 
 23. **Out-of-order injected events get consumed by the funnel engine.** Adding
     a "step C" event before "step B" in the stream causes Mixpanel's greedy
@@ -1705,9 +1768,9 @@ Import from `@ak--47/dungeon-master/hook-helpers`:
 | `userInProfileSegment` | cohort | `(profile, key, values) -> boolean` | Profile property match |
 | **`hashFloat`** | cohort | `(id) -> number` | FNV-1a over the FULL id string → [0,1). Deterministic bucketing primitive (v1.6) — replaces `charCodeAt(0) % N` idioms, which bias cohort rates on hex-ish id alphabets |
 | **`hashCohort`** | cohort | `(id, pct) -> boolean` | True for ~`pct`% of ids (pct on a 0–100 scale). Membership nests: `pct=5` ⊂ `pct=20` |
-| `cloneEvent` | mutate | `(template, overrides?) -> event` | Shallow clone with overrides |
+| `cloneEvent` | mutate | `(template, overrides?) -> event` | Shallow clone with overrides **and a fresh `insert_id`** — never hand-roll this (see Section 2.1) |
 | `dropEventsWhere` | mutate | `(events, predicate) -> number` | Remove matching events in-place |
-| `scaleEventCount` | mutate | `(events, eventName, factor) -> number` | Scale total count via clones at sub-second offsets (does NOT move frequency-distribution bins — see Section 2.1) |
+| `scaleEventCount` | mutate | `(events, eventName, factor, options?) -> number` | Scale total count via clones. Default 1s offsets do NOT move frequency, session, active-day, or retention metrics — see Section 2.1. Pass `{ spreadDays: N }` (v1.6.4) or use `injectOnNewDays` |
 | `scalePropertyValue` | mutate | `(events, predicate, prop, factor) -> number` | Multiply numeric property; null-aware safe |
 | `shiftEventTime` | mutate | `(event, deltaMs) -> event` | Shift one timestamp |
 | `scaleTimingBetween` | timing | `(events, eventA, eventB, factor) -> boolean` | Scale gap between first A and first B |

package/README.md

@@ -584,6 +584,15 @@ all randomness is seeded. same seed + same config + concurrency=1 = identical ou
 }
 ```
 
+pin `datasetStart` and `datasetEnd` too, or the dataset window moves with the
+calendar and every timestamp shifts.
+
+**one exception: `insert_id`.** since 1.4.0 it is a `randomUUID()`, so it differs
+on every run by design — that is what keeps Mixpanel from deduping re-imports of
+the same dataset. strip `insert_id` before diffing two runs. everything else
+(event count, order, timestamps, every property, profiles, groups) is
+byte-identical.
+
 ## what gets generated
 
 the result object contains everything:
@@ -689,6 +698,53 @@ engine tests are NOT shipped in the npm package and NOT run as part of `npm test
 
 ## config reference
 
+### one config surface, not two
+
+three groups of keys accept both a nested sub-object and a flat top-level form:
+
+| sub-object | keys it groups |
+|---|---|
+| `credentials` | `token`, `region`, `serviceAccount`, `serviceSecret`, `projectId` |
+| `switches` | `hasLocation`, `hasCampaigns`, `hasAdSpend`, `hasSessionIds`, `hasAvatar`, `hasIOSDevices`, `hasAndroidDevices`, `hasDesktopDevices`, `hasBrowser`, `isAnonymous`, `alsoInferFunnels` |
+| `identity` | `avgDevicePerUser`, `sessionTimeout` |
+
+**the sub-object form is canonical.** the flat top-level keys are a back-compat
+alias and stay supported. emit one form or the other — never both. when a key is
+set in both places the **top-level value wins**, with a `verbose`-gated warning
+you will not see unless `verbose: true`.
+
+```javascript
+// canonical
+{ credentials: { token: process.env.MIXPANEL_TOKEN, region: 'US' },
+  switches:    { hasCampaigns: true, hasAdSpend: true },
+  identity:    { avgDevicePerUser: 2 } }
+
+// back-compat alias — still works
+{ token: process.env.MIXPANEL_TOKEN, region: 'US',
+  hasCampaigns: true, hasAdSpend: true, avgDevicePerUser: 2 }
+```
+
+`hasAttributionFlags` is **not** a switch. the validator derives it from
+`events[].isAttributionEvent`; setting it has no effect.
+
+### group keys
+
+`groupKeys` accepts a positional tuple or a named object. both normalize to the
+tuple internally, so hooks and the verifier see one shape:
+
+```javascript
+groupKeys: [
+  ['company_id', 50],                                  // tuple
+  ['team_id', 200, ['Deploy', 'Merge PR']],            // tuple + scoped events
+  { key: 'org_id', cardinality: 25 },                  // named (v1.6.4)
+  { key: 'workspace_id', cardinality: 80, events: ['Save'] },
+]
+```
+
+an omitted or empty `events` list means every event carries that group key.
+
+### commonly used properties
+
 see [types.d.ts](types.d.ts) for the complete `Dungeon` interface. here are the most commonly used properties:
 
 | property | type | default | description |
@@ -702,7 +758,7 @@ see [types.d.ts](types.d.ts) for the complete `Dungeon` interface. here are the
 | `seed` | string | random | RNG seed for reproducibility |
 | `format` | string | `'csv'` | output format (csv, json, parquet) |
 | `token` | string | null | mixpanel project token (triggers import) |
-| `region` | string | `'US'` | mixpanel data residency |
+| `region` | string | `'US'` | mixpanel data residency (`US` / `EU` / `IN`) |
 | `writeToDisk` | boolean/string | false | write files to ./data/ or a gs:// path |
 | `gzip` | boolean | false | compress output files |
 | `verbose` | boolean | false | print progress |
@@ -714,7 +770,8 @@ see [types.d.ts](types.d.ts) for the complete `Dungeon` interface. here are the
 | `bornRecentBias` | number | 0 (from macro `flat`) | user birth date skew (safe range [-0.5, 0.5]; user-explicit values outside the band are clamped) |
 | `percentUsersBornInDataset` | number | 12 (from macro `flat`) | % of users born in window (clamped per-macro when both `macro` and this field are explicit) |
 | `preExistingSpread` | string | `'uniform'` (from macro `flat`) | placement of pre-existing users' first event |
-| `avgActiveDaysPerUser` | number | undefined | concentrate events onto N distinct UTC days per user (preserves total event count) |
+| `avgActiveDaysPerUser` | number | undefined | concentrate events onto N distinct UTC days per user (preserves total event count). ignored when `retentionCurve` is set; warns when combined with `engagementDecay` |
+| `retentionCurve` | object | undefined | per-day return probabilities. **wins over `avgActiveDaysPerUser`** when both are set |
 | `maxTouchpointsPerUser` | number | 10 | UTM stamping cap per user (Mixpanel `TOUCHPOINTS_LIMIT` parity) |
 | `autoSortAfterEverything` | boolean | true | sort events by time after `everything` hook (defends greedy funnel engine) |
 | `hook` | function/string | passthrough | data transformation function |

package/index.js

@@ -347,7 +347,10 @@ async function generateAdSpendData(context) {
  */
 async function generateGroupProfiles(context) {
 	const { config, storage } = context;
-	const { groupKeys, groupProps = {} } = config;
+	// `config` here is the VALIDATED config, so `normalizeGroupKeys` has already
+	// converted any `{ key, cardinality }` entries to positional tuples.
+	const groupKeys = /** @type {import('./types').GroupKeyTuple[]} */ (config.groupKeys);
+	const { groupProps = {} } = config;
 
 	if (config.verbose) {
 		logger.info('Generating group profiles...');
@@ -430,7 +433,9 @@ async function generateLookupTables(context) {
  */
 async function generateGroupSCDs(context) {
 	const { config, storage } = context;
-	const { scdProps, groupKeys } = config;
+	const { scdProps } = config;
+	// Validated config: `normalizeGroupKeys` has already flattened named entries.
+	const groupKeys = /** @type {import('./types').GroupKeyTuple[]} */ (config.groupKeys);
 
 	if (config.verbose) {
 		logger.info('Generating group SCDs...');

package/lib/core/config-validator.js

@@ -181,7 +181,11 @@ const KILLED_CONFIG_KEYS = ['subscription', 'attribution', 'geo', 'features', 'a
 // v1.5.1 — migration is gradual.
 const CONFIG_SUBOBJECTS = {
 	credentials: ['token', 'region', 'serviceAccount', 'serviceSecret', 'projectId'],
-	switches: ['hasLocation', 'hasCampaigns', 'hasAdSpend', 'hasSessionIds', 'hasAvatar', 'hasIOSDevices', 'hasAndroidDevices', 'hasDesktopDevices', 'hasBrowser', 'isAnonymous', 'alsoInferFunnels', 'hasAttributionFlags'],
+	// v1.6.4: `hasAttributionFlags` removed from this list. It is derived, not
+	// settable — line ~942 unconditionally overwrites it with
+	// `validatedEvents.some(e => e.isAttributionEvent)`. Listing it here presented
+	// a knob that never did anything.
+	switches: ['hasLocation', 'hasCampaigns', 'hasAdSpend', 'hasSessionIds', 'hasAvatar', 'hasIOSDevices', 'hasAndroidDevices', 'hasDesktopDevices', 'hasBrowser', 'isAnonymous', 'alsoInferFunnels'],
 	identity: ['avgDevicePerUser', 'sessionTimeout'],
 };
 
@@ -744,6 +748,23 @@ export function validateDungeonConfig(config) {
 	if (verbose && numDays < 14) {
 		console.warn(`⚠️  numDays=${numDays} < 14. Strict-bar engine-validation metrics use 14-day windows; results are noisy below 14 days. Consider increasing numDays.`);
 	}
+
+	// v1.6.4 (P3-9): warn when `avgActiveDaysPerUser` and `engagementDecay` are both
+	// set. The active-day planner picks a fixed number of distinct days; decay then
+	// filters events off the late ones. v1.5 Fix #1 protects the last surviving event
+	// on each picked day, so the loss is bounded — but the realized distinct-day count
+	// still lands at or below the configured value, never above. This warning is NOT
+	// verbose-gated: the combination quietly returns a number the author did not ask for.
+	if (
+		avgActiveDaysPerUser !== undefined && avgActiveDaysPerUser !== null &&
+		Number.isFinite(avgActiveDaysPerUser) && config.engagementDecay
+	) {
+		console.warn(
+			`⚠️  [dungeon-master] avgActiveDaysPerUser=${avgActiveDaysPerUser} is set together with engagementDecay. ` +
+			`Decay erodes the effective active-day count, so the realized value will be at or below ` +
+			`${avgActiveDaysPerUser}, never above. Prefer one knob or the other. See HOOKS.md §2.5.`
+		);
+	}
 	// ──────────────────────────────────────────────────────────────────────
 
 	// Use provided name if non-empty string, otherwise generate one
@@ -909,6 +930,9 @@ export function validateDungeonConfig(config) {
 		dataQuality = validateDataQuality(dataQuality);
 	}
 
+	// v1.6.4: accept the named-object groupKeys form alongside the legacy tuple.
+	const normalizedGroupKeys = normalizeGroupKeys(groupKeys);
+
 	// Phase 1: validate Funnel.attempts on every funnel (additive — most have none).
 	validateAttempts(funnels);
 
@@ -971,7 +995,7 @@ export function validateDungeonConfig(config) {
 		userProps,
 		scdProps,
 		mirrorProps,
-		groupKeys,
+		groupKeys: normalizedGroupKeys,
 		groupProps,
 		lookupTables,
 		hasAnonIds: hasAnonIdsResolved,
@@ -1104,6 +1128,43 @@ function transformSCDPropsWithoutCredentials(config) {
 	if (config.verbose === true) console.log('\u2713 SCD properties converted to static properties\n');
 }
 
+/**
+ * v1.6.4: normalize `groupKeys` to the positional tuple form the generators,
+ * hooks, and verifier all expect.
+ *
+ * Accepts either form, mixed freely in one array:
+ *   - `["company_id", 50]`                       → unchanged
+ *   - `["company_id", 50, ["Purchase"]]`         → unchanged
+ *   - `{ key: "company_id", cardinality: 50 }`   → `["company_id", 50]`
+ *   - `{ key, cardinality, events: [...] }`      → `[key, cardinality, events]`
+ *
+ * The named form exists so a form-driven generator never has to emit an untyped
+ * positional tuple. Nothing downstream needs to know which form the author used.
+ *
+ * @param {Array} groupKeys
+ * @returns {Array} tuple-form group keys
+ */
+function normalizeGroupKeys(groupKeys) {
+	if (!Array.isArray(groupKeys)) return [];
+	return groupKeys.map((entry, i) => {
+		if (Array.isArray(entry)) return entry;
+		if (entry && typeof entry === 'object') {
+			const { key, cardinality, events } = entry;
+			if (typeof key !== 'string' || !key) {
+				throw new Error(`groupKeys[${i}]: named form requires a non-empty string "key" (got ${JSON.stringify(key)})`);
+			}
+			if (!Number.isFinite(cardinality) || cardinality < 1) {
+				throw new Error(`groupKeys[${i}] ("${key}"): named form requires a "cardinality" >= 1 (got ${JSON.stringify(cardinality)})`);
+			}
+			if (events !== undefined && !Array.isArray(events)) {
+				throw new Error(`groupKeys[${i}] ("${key}"): "events" must be an array of event names when present (got ${typeof events})`);
+			}
+			return events && events.length ? [key, cardinality, events] : [key, cardinality];
+		}
+		throw new Error(`groupKeys[${i}]: expected a [key, cardinality] tuple or a { key, cardinality } object (got ${typeof entry})`);
+	});
+}
+
 // ── Advanced Feature Validation Functions ──
 
 // P2.5 (v1.6): churnRate / activeWindow / soupOverride are declared Persona

package/lib/generators/funnels.js

@@ -376,7 +376,10 @@ export async function makeFunnel(context, funnel, user, firstEventTime, profile
 		if (Number.isFinite(lastTimeMs)) {
 			const IDENTITY_KEYS = ['user_id', 'device_id', 'distinct_id', 'session_id', 'insert_id'];
 			const superPropKeys = Object.keys(superProps || {});
-			const groupKeyNames = (groupKeys || []).map(gk => Array.isArray(gk) ? gk[0] : gk).filter(Boolean);
+			// Validated config: group keys are always positional tuples here.
+			const groupKeyNames = /** @type {string[]} */ (
+				(groupKeys || []).map(gk => Array.isArray(gk) ? gk[0] : gk).filter(gk => typeof gk === 'string')
+			);
 			const numToInject = chance.integer({ min: 1, max: 2 });
 			for (let i = 0; i < numToInject; i++) {
 				const excName = chance.pickone(funnel.exclusionEvents);

package/lib/hook-helpers/_internal.js

@@ -1,3 +1,48 @@
+import { randomUUID } from 'node:crypto';
+
+/**
+ * Give a cloned event its own `insert_id`.
+ *
+ * Clones MUST NOT inherit their template's id, and stripping the id is not
+ * enough either: mixpanel-import runs with `fixData: true`, which synthesizes a
+ * missing id by content-hashing the record — so two clones that differ only in
+ * ways the hash ignores collapse into one on ingest. Mixpanel then dedupes them
+ * server-side and the engineered volume silently disappears from the project
+ * while local verification (which never looks at `insert_id`) still passes.
+ *
+ * `insert_id` is already assigned via `randomUUID()` in the event generator
+ * (`lib/generators/events.js`), so it sits outside the seeded
+ * byte-identical-output guarantee; using it here keeps clone ids consistent with
+ * engine-generated ones.
+ *
+ * @template {{insert_id?: string}} T
+ * @param {T} clone
+ * @returns {T} the same object, with a fresh `insert_id`
+ */
+export function stampFreshInsertId(clone) {
+	clone.insert_id = randomUUID();
+	return clone;
+}
+
+/**
+ * Spread a template into a new event, merge `overrides`, and give it a fresh
+ * `insert_id` — unless the caller pinned one explicitly.
+ *
+ * Every clone site should go through this so the override contract is uniform:
+ * a caller that passes `insert_id` in `overrides` keeps it, and everyone else
+ * gets a unique id rather than a duplicate of the template's.
+ *
+ * @template {Record<string, any>} T
+ * @param {T} template - Event to clone from.
+ * @param {Record<string, any>} [overrides] - Fields to merge on top.
+ * @returns {T} the new event
+ */
+export function cloneWithFreshId(template, overrides = {}) {
+	const clone = /** @type {T} */ ({ ...template, ...overrides });
+	if (!(overrides && 'insert_id' in overrides)) stampFreshInsertId(clone);
+	return clone;
+}
+
 export function toMs(t) {
 	if (typeof t === 'number') return t > 1e12 ? t : t > 1e9 ? t * 1000 : t;
 	return Date.parse(t);

package/lib/hook-helpers/inject.js

@@ -10,7 +10,7 @@
  */
 
 import { getChance } from '../utils/utils.js';
-import { toMs, writeTime } from './_internal.js';
+import { toMs, writeTime, stampFreshInsertId, cloneWithFreshId } from './_internal.js';
 
 /**
  * Splice a cloned event into `events` immediately after `sourceEvent`. Time is
@@ -29,7 +29,7 @@ export function injectAfterEvent(events, sourceEvent, templateEvent, gapMs, over
 	if (!events || !sourceEvent || !templateEvent) return null;
 	const baseT = toMs(sourceEvent.time);
 	if (!Number.isFinite(baseT)) return null;
-	const newEv = { ...templateEvent, ...overrides };
+	const newEv = cloneWithFreshId(templateEvent, overrides);
 	writeTime(newEv, baseT + gapMs);
 	const idx = events.indexOf(sourceEvent);
 	if (idx >= 0) events.splice(idx + 1, 0, newEv);
@@ -60,7 +60,7 @@ export function injectBetween(events, eventA, eventB, templateEvent, overrides =
 	const aT = toMs(a.time);
 	const bT = toMs(b.time);
 	if (!Number.isFinite(aT) || !Number.isFinite(bT)) return null;
-	const newEv = { ...templateEvent, ...overrides };
+	const newEv = cloneWithFreshId(templateEvent, overrides);
 	writeTime(newEv, (aT + bT) / 2);
 	const bIdxOrig = events.indexOf(b);
 	if (bIdxOrig >= 0) events.splice(bIdxOrig, 0, newEv);
@@ -89,7 +89,7 @@ export function injectBurst(events, templateEvent, count, anchorTime, spreadMs,
 	const created = [];
 	for (let i = 0; i < count; i++) {
 		const offset = chance.floating({ min: -spreadMs, max: spreadMs });
-		const newEv = { ...templateEvent, ...overrides };
+		const newEv = cloneWithFreshId(templateEvent, overrides);
 		writeTime(newEv, anchorMs + offset);
 		events.push(newEv);
 		created.push(newEv);
@@ -119,7 +119,8 @@ const DAY_MS = 86400000;
  *   5. Find a template event of `eventName`; if none exist for this user,
  *      return unchanged (we honor schema-first: don't fabricate events).
  *   6. Clone the template onto each picked day at a random hour within the
- *      day. `insert_id` is stripped (Mixpanel re-deduplicates on import).
+ *      day. Each clone gets a FRESH `insert_id` (inheriting or omitting it lets
+ *      Mixpanel dedupe the clones away on ingest).
  *   7. Append clones to the array. Caller's downstream sort handles ordering.
  *
  * @param {Object[]} events - Full user event array (from `everything` hook).
@@ -183,9 +184,8 @@ export function injectOnNewDays(events, eventName, targetDays, options = {}) {
 		const lo = Math.max(dayStart, minMs);
 		const hi = Math.min(dayEnd, maxMs);
 		const newMs = lo >= hi ? lo : chance.integer({ min: lo, max: hi });
-		const clone = { ...template, ...overrides };
+		const clone = cloneWithFreshId(template, overrides);
 		writeTime(clone, newMs);
-		delete clone.insert_id;
 		events.push(clone);
 	}
 

package/lib/hook-helpers/mutate.js

@@ -8,7 +8,7 @@
  * `Math.random()` — so dungeon runs stay reproducible.
  */
 
-import { toMs } from './_internal.js';
+import { toMs, stampFreshInsertId, cloneWithFreshId } from './_internal.js';
 
 import { getChance } from '../utils/utils.js';
 
@@ -25,7 +25,9 @@ import { getChance } from '../utils/utils.js';
  */
 export function cloneEvent(template, overrides = {}) {
 	if (!template) throw new Error('cloneEvent: template is required');
-	return /** @type {T} */ ({ ...template, ...overrides });
+	// A clone that keeps its template's insert_id is deduped away by Mixpanel on
+	// ingest. Callers may still pin one explicitly via overrides.
+	return /** @type {T} */ (cloneWithFreshId(template, overrides));
 }
 
 /**
@@ -51,35 +53,69 @@ export function dropEventsWhere(events, predicate) {
  * Scale the count of events with name `eventName` in `events` by `factor`. Mutates
  * `events` in place.
  *
- * - factor > 1: clones existing matches with small monotonic time offsets (1s steps)
- *   so duplicates land just after their source. Returns positive integer = clones added.
+ * - factor > 1: clones existing matches. Returns positive integer = clones added.
  * - factor < 1: drops matches at random using the seeded RNG. Returns negative
  *   integer = -dropped.
  * - factor === 1 or no matches: no-op, returns 0.
  *
- * Note: the `insert_id` of cloned events is removed so a downstream pass can
- * regenerate it (otherwise Mixpanel will dedupe on import).
+ * ## WARNING — the default spread cannot move frequency or session metrics
+ *
+ * By default clones land 1 second after their source (monotonic 1s steps). One
+ * second is not enough to create a new distinct active day, and not enough to
+ * open a new session under Mixpanel's 30-minute inactivity gap. So a default
+ * `scaleEventCount` call raises **event volume only**. It does NOT raise:
+ *
+ *   - active days per user / DAU / stickiness (DAU÷MAU)
+ *   - sessions per user or session count
+ *   - "days active" or frequency-of-use reports
+ *   - retention, which is computed on distinct days
+ *
+ * If your story claims any of those, pass `{ spreadDays: N }` to scatter clones
+ * across the following N days, or use `injectOnNewDays` — the helper built for
+ * exactly this.
+ *
+ * Note: cloned events get a FRESH `insert_id` — they must not inherit the
+ * source's, and leaving it blank is not safe either (the importer content-hashes
+ * missing ids, so identical clones collide and Mixpanel dedupes them away).
  *
  * @param {Array<{event:string,time:string|number,insert_id?:string}>} events
  * @param {string} eventName
  * @param {number} factor
+ * @param {{spreadDays?: number}} [options] - `spreadDays`: scatter clones uniformly
+ *   over `[source_time, source_time + spreadDays]` instead of stepping 1s at a time.
+ *   Use this when the clones must land on new calendar days or in new sessions.
+ *   Clones that land past `FIXED_NOW` are dropped by the engine's future-time guard,
+ *   so a large `spreadDays` on late-window events yields fewer surviving clones than
+ *   the return value reports.
  * @returns {number}
  */
-export function scaleEventCount(events, eventName, factor) {
+export function scaleEventCount(events, eventName, factor, options = {}) {
 	if (!events || !eventName || typeof factor !== 'number' || factor === 1) return 0;
+	const { spreadDays } = options || {};
+	const spreadMs = (Number.isFinite(spreadDays) && spreadDays > 0)
+		? spreadDays * 86_400_000
+		: 0;
 	if (factor > 1) {
 		const matches = events.filter(e => e && e.event === eventName);
 		if (!matches.length) return 0;
 		const additionalNeeded = Math.round(matches.length * (factor - 1));
+		const chance = spreadMs ? getChance() : null;
 		let added = 0;
 		for (let i = 0; i < additionalNeeded; i++) {
 			const src = matches[i % matches.length];
 			const baseMs = toMs(src.time);
-			const newTime = Number.isFinite(baseMs)
-				? new Date(baseMs + (i + 1) * 1000).toISOString()
-				: src.time;
-			const clone = { ...src, time: newTime };
-			delete clone.insert_id;
+			let newTime;
+			if (!Number.isFinite(baseMs)) {
+				newTime = src.time;
+			} else if (spreadMs) {
+				// Seeded uniform offset across the spread window. Floor at 1s so a
+				// clone never collides exactly with its source's timestamp.
+				const offset = Math.max(1000, Math.round(chance.floating({ min: 0, max: spreadMs })));
+				newTime = new Date(baseMs + offset).toISOString();
+			} else {
+				newTime = new Date(baseMs + (i + 1) * 1000).toISOString();
+			}
+			const clone = stampFreshInsertId({ ...src, time: newTime });
 			events.push(clone);
 			added++;
 		}

package/lib/hook-helpers/shape.js

@@ -6,13 +6,13 @@
  * (HOOKS.md §2.16), a biased Flows path branch (§2.17), and a deterministic
  * session cadence (§2.13). All three are `everything`-hook-only — they need
  * the whole stream — and obey the schema-first rules: clones only (spread
- * from the user's own events, `insert_id` stripped), no fabricated events,
+ * from the user's own events, each given a fresh `insert_id`), no fabricated events,
  * seeded `chance` for all randomness. Timestamp rewrites are safe because
  * the engine re-derives `session_id` on the final event set (v1.6 P2.1).
  */
 
 import { getChance } from '../utils/utils.js';
-import { toMs, writeTime } from './_internal.js';
+import { toMs, writeTime, stampFreshInsertId } from './_internal.js';
 import { hashFloat } from './cohort.js';
 
 const DAY_MS = 86400000;
@@ -100,7 +100,7 @@ export function applyLifecycleWave(events, uid, opts) {
 	for (let i = 0; i < resurrectBurst; i++) {
 		const clone = { ...template };
 		writeTime(clone, t);
-		delete clone.insert_id;
+		stampFreshInsertId(clone);
 		if (!clone.user_id && uid) clone.user_id = uid;
 		kept.push(clone);
 		t += chance.integer({ min: MIN_MS, max: 10 * MIN_MS });
@@ -168,7 +168,7 @@ export function applyPathBias(events, uid, opts) {
 		t += chance.integer({ min: lo, max: hi }) * 1000;
 		const clone = { ...tpl };
 		writeTime(clone, t);
-		delete clone.insert_id;
+		stampFreshInsertId(clone);
 		if (!clone.user_id && uid) clone.user_id = uid;
 		events.push(clone);
 	}

package/lib/hook-patterns/frequency-by-frequency.js

@@ -16,8 +16,8 @@
  * - Operates on the user's full event stream — call from the `everything` hook.
  * - Does NOT add new properties; uses existing event names defined in the dungeon
  *   schema.
- * - Cloned events have their `insert_id` stripped (mutate.scaleEventCount handles
- *   that), so the engine's batch writer can re-stamp them downstream.
+ * - Cloned events get a fresh `insert_id` (mutate.scaleEventCount handles that),
+ *   so Mixpanel does not dedupe them away on ingest.
  */
 
 import { binUsersByEventCount } from '../hook-helpers/cohort.js';

package/lib/orchestrators/user-loop.js

@@ -750,6 +750,33 @@ export async function userLoop(context) {
 				});
 			}
 
+			// v1.6.3: guarantee unique `insert_id` across the user's final stream.
+			//
+			// Hooks clone events by spreading an existing one — the documented way
+			// to inject — and a spread copies `insert_id` along with everything
+			// else. Mixpanel deduplicates on `insert_id` at ingest, so those clones
+			// are accepted, reported as successful, and then silently dropped: the
+			// engineered volume never appears in the project. Nothing downstream
+			// catches it, because local verification never inspects `insert_id`.
+			//
+			// The hook-helper clone atoms stamp fresh ids themselves, but hooks are
+			// free to hand-roll a spread (and many shipped dungeons do), so the
+			// only reliable place to enforce this is here, over the finished set.
+			// A legitimate stream never contains two events with the same id, so
+			// any collision at this point is a clone that needs its own id.
+			// `> 0`, not `> 1`: a lone event cannot collide, but it can still be
+			// missing an id if a hook replaced it with a constructed object.
+			if (usersEvents.length > 0) {
+				const seenInsertIds = new Set();
+				for (const ev of usersEvents) {
+					if (!ev) continue;
+					if (!ev.insert_id || seenInsertIds.has(ev.insert_id)) {
+						ev.insert_id = randomUUID();
+					}
+					seenInsertIds.add(ev.insert_id);
+				}
+			}
+
 			// Defensive guard: drop any events whose timestamp landed past the
 			// configured dataset end. Hooks that duplicate events with time offsets
 			// (weekend surges, viral spreads) can leak a few past the boundary.