@ak--47/dungeon-master 1.8.1 → 1.8.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,5 +1,31 @@
1
1
  # 1.8.1 verification contract
2
2
 
3
+ ## 1.8.2 live verification update
4
+
5
+ The [1.8.2 guide](../../../../docs/guides/1.8.2-upgrade-guide.md) and repository
6
+ `tests/alignment/live/REPORT.md` extend this contract with retained live comparisons.
7
+ The 1.8.1 audit below remains historical. It used selected source-derived contracts;
8
+ 1.8.2 also imports isolated datasets and compares native Mixpanel query results.
9
+ Public options and defaults remain unchanged.
10
+
11
+ Automatic verifier identity now derives from emitted both-ID events. Explicit
12
+ `buildIdentityMap(profiles)` overrides retain their compatibility behavior.
13
+ List-valued held-property expansion and supported HPC/session report combinations
14
+ have source and live fixtures. Non-UTC/DST and custom session-exclusion variants
15
+ still need separate verification.
16
+
17
+ Import success is not query readiness. Reconcile totals and identity before
18
+ accepting comparisons. Historical anonymous events can join later without another
19
+ import. Every live test must use a run property and disjoint run identities.
20
+ Retain incomplete responses and do other work before a bounded retry.
21
+
22
+ Distinguish raw per-user count histograms, UTC calendar activity, and rolling
23
+ Frequency/Addiction. The default local calendar algorithm is not a universal
24
+ Mixpanel frequency rule. Lifetime attribution endpoints require conversion/lookback
25
+ eligibility; conflicting values with tied timestamps are not stable across ingest.
26
+ `minCohort` now enforces conservative independent-user lower bounds, including
27
+ custom callbacks. Unknown evidence caps a passing verdict instead of counting rows.
28
+
3
29
  Use this contract when authoring, verifying, provisioning, or presenting a dungeon.
4
30
  The [1.8.1 guide](../../../../docs/guides/1.8.1-upgrade-guide.md) describes the
5
31
  release. The local repository's `tests/alignment/REPORT.md` and
@@ -17,8 +43,8 @@ a numeric property, a mean with a median, or conversion with event volume to
17
43
  rescue a failed story. Supplementary diagnostics must keep their own labels.
18
44
 
19
45
  Local checks prove selected source-derived contracts, not universal Mixpanel
20
- parity. The audit read analytics source without compiling or executing it. It
21
- did not run live differential queries. Macro/soup interactions, table surfaces,
46
+ parity. The 1.8.1 audit read analytics source without compiling or executing it.
47
+ That audit did not run live differential queries. Macro/soup interactions, table surfaces,
22
48
  arbitrary hooks, parallel execution, and several counting variants remain gaps.
23
49
 
24
50
  ## Separate a measured effect from enough evidence
@@ -52,8 +78,8 @@ and preserve its original failed evidence.
52
78
  - Derive sessions from the full resolved user stream before HPC partitioning.
53
79
  Local defaults are UTC, a 30-minute idle timeout, and a 24-hour maximum.
54
80
  `session_id` is a diagnostic or explicit legacy mode, not the default truth.
55
- Non-UTC projects, list-valued HPC, and project-specific hidden/excluded session
56
- events remain outside this proof.
81
+ Non-UTC projects and project-specific hidden/excluded session events remain
82
+ outside the live proof. List-valued HPC has 1.8.2 source and live fixtures.
57
83
  - Profile device pools alone establish no identity mapping evidence. The public
58
84
  `buildIdentityMap(profiles)` helper retains profile-based compatibility behavior.
59
85
  For emitted-identity proof, derive links from valid ordinary both-ID events,
@@ -210,9 +210,11 @@ emulateBreakdown(events, {
210
210
  });
211
211
  ```
212
212
 
213
- Without an explicit map, the compatibility path can call `buildIdentityMap(profiles)`
214
- (reads `device_ids`, then `anonymousIds`). Label that profile-derived assumption;
215
- it cannot prove that an identity link survived generation or ingest.
213
+ In 1.8.2, automatic mapping uses emitted both-ID records before report filtering.
214
+ The public `buildIdentityMap(profiles)` helper still reads profile pools when a
215
+ caller explicitly requests that override. It cannot prove that a link survived
216
+ generation or ingest. Imported historical identity can become query-visible later;
217
+ verify readiness before accepting a local/live comparison.
216
218
 
217
219
  ## Time-series verification (timeBucket)
218
220
 
package/CHANGELOG.md CHANGED
@@ -2,6 +2,47 @@
2
2
 
3
3
  All notable changes to `@ak--47/dungeon-master`.
4
4
 
5
+ ## 1.8.2 - 2026-09-13
6
+
7
+ ### Fixed
8
+
9
+ - Build automatic verifier identity maps from emitted both-ID events. Profile
10
+ device pools no longer create links by themselves. Explicit identity maps
11
+ retain their existing behavior; conflicting authenticated events retain their user.
12
+ - Pass supported held-property session funnels through the report wrapper. Keep
13
+ implicit one-session windows when applying matching dungeon funnel defaults.
14
+ - Expand list-valued held properties into separate funnel histories, including
15
+ repeated cursor visits and full-stream session boundaries.
16
+ - Preserve prior attribution touches across time buckets, ignore unstamped
17
+ touches, and retain conversions with no matching touch in the unknown segment.
18
+ - Make `minCohort` use conservative independent-user evidence. Repeated period
19
+ rows cannot inflate populations, and custom assertions receive the same guard.
20
+ - Include declared global properties on synthetic experiment exposure events,
21
+ so run and segment filters retain both exposures and outcomes.
22
+ - Make funnel-frequency drop decisions independent of random insertion IDs.
23
+
24
+ ### Verified
25
+
26
+ - Add opt-in live comparisons through the actual importer and Mixpanel query
27
+ layer, with run-isolated identities, import budgets, retained query payloads,
28
+ exact local/live comparisons, fixed seeds, and neutral controls.
29
+ - Verify selected generated conversion, TTC, retention, volume, weight, incident,
30
+ frequency, aggregate, attribution, session, and Flows effects against live reports.
31
+ - Separate ingestion readiness from import acceptance. Historical identity links
32
+ can become visible later; one retained fixture converged without another import.
33
+
34
+ ### Compatibility
35
+
36
+ Public exports, signatures, options, and defaults remain unchanged. Corrected
37
+ counts and seeded hook outcomes can differ from 1.8.1. Explicit profile-map
38
+ overrides and totals' `reentry: false` default remain supported. Calendar-day
39
+ activity, rolling frequency, and raw event-count histograms are separate reports.
40
+ Finite-lookback attribution, ambiguous tied touches, non-UTC project settings,
41
+ and arbitrary combinations remain subject to the documented limits.
42
+
43
+ See the [upgrade guide](docs/guides/1.8.2-upgrade-guide.md) and the repository's
44
+ live alignment report for measured scope. This entry does not indicate npm publication.
45
+
5
46
  ## 1.8.1 — 2026-09-12
6
47
 
7
48
  ### Fixed
package/HOOKS.md CHANGED
@@ -1,9 +1,9 @@
1
1
  # HOOKS.md -- Hook Encyclopedia
2
2
 
3
- Hook reference and recipe catalog for dungeon-master. Every recipe is calibrated
4
- against Mixpanel's actual counting semantics (greedy single-pass funnels,
5
- distinct-period frequency, null-aware aggregation, capped attribution) see
6
- [Section 2](#2-how-mixpanel-counts-things) before adapting any pattern.
3
+ Hook reference and recipe catalog for dungeon-master. Recipes have different
4
+ counting and evidence requirements. Read [Section 2](#2-how-mixpanel-counts-things)
5
+ before adapting a pattern, then verify the exact report on the emitted dataset.
6
+ The [1.8.2 guide](docs/guides/1.8.2-upgrade-guide.md) records live-tested scope and limits.
7
7
 
8
8
  ---
9
9
 
@@ -172,20 +172,28 @@ verification emulator (`@ak--47/dungeon-master/verify`) now matches these
172
172
  rules; old recipes that ignored them will look correct on the dataset but
173
173
  fail when verified or queried in Mixpanel.
174
174
 
175
- ### 2.1 Frequency reports count DISTINCT PERIODS, not total events
175
+ ### 2.1 Frequency depends on the selected report
176
176
 
177
- Mixpanel's frequency distribution / cohort-by-event-count reports count
178
- **distinct time periods** (default: days) on which the user fired the
179
- event. Two purchases on the same day = frequency **1**, not 2.
177
+ Three measurements require different checks:
178
+
179
+ - **Raw per-user event count:** two purchases on one day count as two. Insights
180
+ per-user count histograms and frequency breakdowns using total event count use
181
+ this axis. `applyFrequencyByFrequency` scales this target count.
182
+ - **Distinct calendar-day activity:** those purchases count as one active day.
183
+ `countDistinctPeriods` defaults to UTC calendar buckets (`ui-bucket`), and
184
+ `binBy: 'distinctDays'` uses the same rule. This default is retained for
185
+ compatibility; it is not a universal Mixpanel frequency-report rule.
186
+ - **Rolling Frequency/Addiction:** an event counts when it is at least one
187
+ selected unit after the last counted event. Histories reset per report interval.
188
+ Two events around midnight can occupy two calendar days but one rolling period.
180
189
 
181
190
  Two related rules exist (v1.6 names them for what they are; the old
182
191
  `'calendar'` / `'rolling'` names remain as silent aliases, unknown names
183
192
  now throw):
184
193
 
185
194
  - **`algorithm: 'ui-bucket'`** (default in our verifier):
186
- `COUNT(DISTINCT date_trunc(unit, time))` in UTC. Matches what the Mixpanel
187
- UI shows and what [`injectOnNewDays`](lib/hook-helpers/inject.js) uses
188
- internally.
195
+ `COUNT(DISTINCT date_trunc(unit, time))` in UTC. This is also what
196
+ [`injectOnNewDays`](lib/hook-helpers/inject.js) uses internally.
189
197
  - **`algorithm: 'mixpanel-rolling'`**: the C++
190
198
  `addiction_query.cpp` rule `qtz_time >= last_counted + seconds_for_unit`
191
199
  (`addiction_query_update_history`, `addiction_query.cpp:363-374`) — what
@@ -193,8 +201,8 @@ now throw):
193
201
  boundaries (events at 23:59 + 00:01 next day = 1 rolling period, 2
194
202
  calendar periods).
195
203
 
196
- Use the default (`ui-bucket`) for hooks. Use `'mixpanel-rolling'` only
197
- when verifying behavior that explicitly depends on the C++ implementation.
204
+ Choose the algorithm from the requested report. Do not replace a raw-count
205
+ histogram with active-day counts, or use calendar days to claim rolling parity.
198
206
 
199
207
  **The actual Frequency report shape** is `frequencyHistogram(events,
200
208
  { event, unit, intervalDays, profiles })` (v1.6): per report interval, a
@@ -205,19 +213,19 @@ zero bucket (`addiction_query.cpp:546-573`). Array length is
205
213
  `ceil(interval / unit)` (`unit.c:108-113`). Use it when a dungeon targets
206
214
  the Frequency report itself rather than a frequency-derived cohort.
207
215
 
208
- **Implication for hooks:** `scaleEventCount(record, "Buy", 3)` clones 3x as
209
- many Buy events at sub-second offsets they all land on the same calendar
210
- day, so the user moves **zero bins** in Mixpanel's frequency report. Use
216
+ **Implication for hooks:** `scaleEventCount(record, "Buy", 3)` targets 3x as
217
+ many Buy events at one-second offsets. This changes raw-count histogram bins.
218
+ It generally does not add active days or rolling periods. Use
211
219
  [`injectOnNewDays`](lib/hook-helpers/inject.js) when the goal is to move
212
- users between frequency bins. Both `injectOnNewDays` and the default
220
+ users between calendar-day activity bins. Both `injectOnNewDays` and the default
213
221
  `countDistinctPeriods` algorithm use calendar-bucket math, so they agree
214
222
  at boundaries.
215
223
 
216
224
  One second is also below Mixpanel's 30-minute session gap, so the default
217
225
  spread cannot create a new session either. The full list of metrics a default
218
- `scaleEventCount` call **cannot** move: active days per user, DAU, stickiness
219
- (DAU÷MAU), sessions per user, frequency bins, and retention. It moves event
220
- volume, and nothing else.
226
+ `scaleEventCount` call does not reliably move: active days per user, DAU,
227
+ stickiness (DAU/MAU), sessions per user, rolling frequency, and retention.
228
+ At dataset edges, clones can be clipped. Verify surviving counts separately.
221
229
 
222
230
  As of v1.6.4 you can pass `{ spreadDays: N }` to scatter the clones across the
223
231
  next N days instead:
@@ -363,7 +371,7 @@ sampling.
363
371
 
364
372
  ### 2.5 Active-day distribution is config-first
365
373
 
366
- Mixpanel frequency reports count distinct days (§2.1). The v1.5 engine
374
+ Distinct-day activity is a separate measurement from raw counts (§2.1). The engine
367
375
  exposes `Dungeon.avgActiveDaysPerUser` as the canonical primitive for this
368
376
  shape. Set it at the config level and the engine concentrates each user's
369
377
  events onto a sampled subset of days drawn from `normal(mean=N, sd=N/3)`,
@@ -876,15 +884,12 @@ Reference: `flows_query.cpp:988-994` (next-anchor-only), `flows.cpp:680-717`
876
884
 
877
885
  ### New principles from the emulator alignment
878
886
 
879
- 21. **Distinct-day vs total-event binning.** Frequency-distribution reports in
880
- Mixpanel count distinct days (Section 2.1). For any hook whose verification
881
- target is a frequency report, use [`binByDistinctPeriods`](lib/verify/counting.js)
882
- instead of `binUsersByEventCount`. For hooks targeting raw event counts
883
- (Insights `total events`, `events per user`), `binUsersByEventCount` is
884
- still correct.
887
+ 21. **Match the bin to the report.** Use `binUsersByEventCount` for raw event
888
+ counts and `binByDistinctPeriods` for calendar-day activity. Use rolling
889
+ counting for the Frequency/Addiction report. Section 2.1 distinguishes them.
885
890
 
886
- 22. **`scaleEventCount` does not move users between frequency bins.** Cloning
887
- Buy events at sub-second offsets places them on the same calendar day and
891
+ 22. **`scaleEventCount` changes raw-count bins.** Cloning
892
+ Buy events at one-second offsets usually places them on the same calendar day and
888
893
  inside the same 30-minute session window, so distinct days, sessions, DAU,
889
894
  stickiness, and retention are all unchanged. It moves event volume only. To
890
895
  shift any of the others, pass `{ spreadDays: N }` (v1.6.4) or use
@@ -898,11 +903,10 @@ Reference: `flows_query.cpp:988-994` (next-anchor-only), `flows.cpp:680-717`
898
903
  after the prior step's timestamp (with margin > 2 seconds for the grace
899
904
  window).
900
905
 
901
- 24. **Attribution stamping is capped at 10 touchpoints.** When biasing
902
- `firstTouch` attribution by stamping touchpoint events, ≤10 touches per
903
- user enter the candidate pool. Stamping 50 weighted Touch events per
904
- user gives the same answer as stamping 10. Aim for sparse, distinct
905
- touches with deterministic weight ratios.
906
+ 24. **Separate stamping caps from attribution reads.** The generator defaults
907
+ to ten stamped touches per user. Mixpanel first/last reads are uncapped
908
+ within their conversion-bounded lookback. Distinct touch timestamps avoid
909
+ ambiguous attribution when different values tie at the first or last time.
906
910
 
907
911
  25. **Null-aware aggregation removes the need to "fill" defaults.** Don't
908
912
  coalesce missing numeric properties to 0 to keep AVG sane — Mixpanel
@@ -1741,6 +1745,13 @@ touch with a seeded weighted pick (models: `firstTouch`, `lastTouch`,
1741
1745
  `both`). It never stamps unstamped events, so total touch count is
1742
1746
  unchanged.
1743
1747
 
1748
+ The helper selects lifetime endpoints. A report reads eligible touches before
1749
+ each conversion and inside its lookback. The lifetime-last touch can occur after
1750
+ the conversion; the lifetime-first can expire. Neither endpoint is a general
1751
+ conversion-aware guarantee. Pass a deliberately eligible stream to the existing
1752
+ helper or implement that selection in the hook, then verify the requested report.
1753
+ Conflicting sources at an equal timestamp have no stable cross-ingestion order.
1754
+
1744
1755
  ---
1745
1756
 
1746
1757
  #### 4.27 Active-Day Cohort Engineering (v1.5)
package/README.md CHANGED
@@ -18,6 +18,10 @@ i built this because i needed it. and after using it across hundreds of customer
18
18
  npm install @ak--47/dungeon-master
19
19
  ```
20
20
 
21
+ 1.8.2 adds live counting verification and fixes identity, session-funnel,
22
+ attribution, and experiment-filter defects. See the
23
+ [upgrade guide](docs/guides/1.8.2-upgrade-guide.md) for compatibility and tested limits.
24
+
21
25
  ## quick start
22
26
 
23
27
  ```javascript
@@ -0,0 +1,110 @@
1
+ # 1.8.2 upgrade guide
2
+
3
+ 1.8.2 fixes verifier counting and experiment-filter defects found while importing
4
+ synthetic datasets and querying them in Mixpanel. Public exports, signatures,
5
+ options, and defaults remain unchanged. No new runtime dependency is required.
6
+
7
+ After publication:
8
+
9
+ ```sh
10
+ npm install @ak--47/dungeon-master@1.8.2
11
+ ```
12
+
13
+ This guide describes a release candidate. A branch, PR, or passing test does not
14
+ publish a package.
15
+
16
+ ## identity requires emitted evidence
17
+
18
+ Automatic verification links a device only when an emitted event contains both
19
+ `device_id` and `user_id`. A profile device pool alone cannot create a link.
20
+ Ordinary event names work; linking is not restricted to an auth-named event.
21
+
22
+ `buildIdentityMap(profiles)` remains unchanged for explicit caller overrides.
23
+ Pass that map only when it represents trusted identity knowledge. Automatic
24
+ mapping preserves an explicit authenticated user on a conflicting device event.
25
+ Original-merge projects, B2B identity rules, external mapping history, and all
26
+ ingestion validation policies are not fully emulated.
27
+
28
+ Historical anonymous events may join their user after the import receipt returns.
29
+ The live verification run observed that delay and later verified the same fixture
30
+ without sending it again. Check event totals and identity readiness before accepting
31
+ a funnel comparison. No sender ordering change or fixed sleep claims to solve this.
32
+
33
+ ## report wrappers use the supported counting code
34
+
35
+ Held-property funnels now support session counting through `funnelFrequency`.
36
+ Matching dungeon defaults no longer inject a millisecond conversion window into
37
+ an implicit one-session report. List-valued held properties expand into per-value
38
+ histories while preserving full-stream session boundaries.
39
+
40
+ Scalar-only held-property keys retain their existing behavior. Mixed list/scalar
41
+ keys use the source-derived string representation. Explicit list-only mode,
42
+ extreme value formatting, and engine cardinality caps remain outside this API.
43
+
44
+ Totals still defaults to `reentry: false`. Set `reentry: true` when comparing
45
+ repeated histories with Mixpanel general totals. Session tests use UTC and the
46
+ default thirty-minute inactivity timeout. Project exclusions and non-UTC/DST
47
+ variants require separate proof.
48
+
49
+ ## passing stories need enough independent users
50
+
51
+ `minCohort` no longer sums overlapping period populations or treats arbitrary rows
52
+ as independent users. It uses a conservative lower bound and applies to custom
53
+ callbacks too. Missing denominator evidence caps an otherwise passing result at
54
+ `WEAK`, with an explanation. Assertions without `minCohort` retain their behavior.
55
+
56
+ This can expose previously optimistic story verdicts. Inspect the selected users,
57
+ entrants, converters, or mature retention cohorts. Do not lower the threshold just
58
+ to restore a green verdict.
59
+
60
+ ## exposure events retain global filters
61
+
62
+ Synthetic `$experiment_started` events now include declared `superProps`, including
63
+ context-aware values. Experiment name and variant fields remain authoritative.
64
+ Run-isolated experiment queries therefore retain the exposure and outcome rows.
65
+ This can change seeded output when global property generation consumes randomness.
66
+ Same-version deterministic runs still exclude only fresh `insert_id` values.
67
+
68
+ Funnel-frequency keep/drop decisions now use stable event fields instead of random
69
+ insertion IDs. Their outcome can change from 1.8.1 but is reproducible on rerun.
70
+
71
+ ## frequency and attribution have distinct contracts
72
+
73
+ Raw event-count histograms, calendar-day activity, and rolling Frequency/Addiction
74
+ reports are separate measurements. `applyFrequencyByFrequency` scales raw target
75
+ event counts. Its existing `binBy: 'distinctDays'` default only selects the cohort
76
+ axis; it does not make clones add new active days. Use explicit report settings.
77
+
78
+ `attributedBy` now ignores null or absent touch values, retains touchless conversions
79
+ as `unknown`, and preserves earlier touches across conversion time buckets.
80
+ The public default still selects the first conversion per user; `perConversion:
81
+ 'all'` evaluates every conversion. The API has no finite-lookback argument.
82
+
83
+ `applyAttributedBySource` retains lifetime endpoint selection. Verify that the
84
+ selected endpoint is inside each intended conversion's lookback. Distinct touch
85
+ timestamps are required for reproducible conflicting source order. Same-time
86
+ ties and backend transition compression can differ from raw event-array order.
87
+
88
+ ## verified scope and remaining work
89
+
90
+ The repository's `tests/alignment/live/REPORT.md` records source revision, run IDs,
91
+ queries, counts, practical effects, neutral controls, and historical failures.
92
+ The local analytics checkout supplied counting contracts; live Mixpanel queries
93
+ tested their behavior on imported data. The C++ engine was not built locally.
94
+
95
+ Evidence covers selected generated conversion, TTC, retention, volume, weights,
96
+ incidents, raw-count frequency, numeric aggregates, attribution, sessions, and
97
+ Flows. It does not establish every configuration cross-product, literal retention
98
+ probability calibration, arbitrary hook behavior, warehouse deployment, or every
99
+ project setting. Capacity and ambiguous-order limits remain explicit.
100
+
101
+ Repository gates preserve data and deny network access:
102
+
103
+ ```sh
104
+ node tests/alignment/run.mjs
105
+ node tests/alignment/run.mjs --sweep --timeout-ms=600000
106
+ ```
107
+
108
+ Live scripts are opt-in and require project-specific authorization and credentials.
109
+ They are excluded from the npm package. Do not use the default pruning test setup
110
+ while retained live datasets are still needed.
@@ -520,6 +520,7 @@ function buildFunnelEvents(context, sequence, chosenFunnelProps, bindPropsIndex,
520
520
  return {
521
521
  event: "$experiment_started",
522
522
  properties: {
523
+ ...config.superProps,
523
524
  "Experiment name": experimentName,
524
525
  "Variant name": experimentVariant
525
526
  }
@@ -43,9 +43,10 @@ import { getChance } from '../utils/utils.js';
43
43
  * last touch BEFORE THE CONVERSION EVENT (lookback bounded by the
44
44
  * conversion timestamp — get_last_value, whoval/read.cpp:643-655). If the
45
45
  * user has stamped touches AFTER their conversion, the touch this pattern
46
- * biases is not the one the report reads. `firstTouch` (the default) has
47
- * no such gap: the lifetime-first touch is exactly what FIRST reads.
48
- * Conversion-aware target selection is planned for 1.6.1.
46
+ * biases is not the one the report reads. `firstTouch` also requires the
47
+ * lifetime-first touch to remain inside the report's lookback. Callers must
48
+ * select eligible history when a conversion-bounded report is intended.
49
+ * Equal-time conflicting values have no stable cross-ingestion ordering.
49
50
  * @returns {{ overwritten: number, touches: number }} `touches` = stamped
50
51
  * events found; `overwritten` = touches whose value was replaced.
51
52
  */
@@ -66,14 +66,11 @@ export function applyFunnelFrequencyBreakdown(allUserEvents, _profile, funnelEve
66
66
  }
67
67
  if (!stepName) return { bin, droppedFinal: false };
68
68
 
69
- // Deterministic drop decision without consuming the seeded RNG stream:
70
- // hash the funnel's first insert_id (fallback: time) through
71
- // simpleHashFloat — FNV-1a over the full string, quantized to 1/1000 —
72
- // and drop when the hash lands under (1 - keepRate). Per-funnel stable,
73
- // so reruns with the same seed drop the same funnels.
74
69
  const dropProb = 1 - keepRate;
75
- const seed = funnelEvents[0] && (funnelEvents[0].insert_id || funnelEvents[0].time) || '';
76
- const det = simpleHashFloat(String(seed));
70
+ const anchor = funnelEvents[0] || {};
71
+ const seed = JSON.stringify([anchor.user_id || anchor.device_id || anchor.distinct_id || '',
72
+ anchor.event || '', anchor.time || '', stepName]);
73
+ const det = simpleHashFloat(seed);
77
74
  if (det < dropProb) {
78
75
  const before = funnelEvents.length;
79
76
  dropEventsWhere(funnelEvents, e => e && e.event === stepName);
@@ -12,7 +12,7 @@
12
12
 
13
13
  import { toMs } from '../hook-helpers/_internal.js';
14
14
  import { matchesWhere } from './coerce.js';
15
- import { buildIdentityMap, resolveUserId } from './identity.js';
15
+ import { buildEventIdentityMap, resolveUserId } from './identity.js';
16
16
 
17
17
  const SECONDS_PER_UNIT = {
18
18
  hour: 3600,
@@ -27,8 +27,8 @@ const SECONDS_PER_UNIT = {
27
27
  * them for what they are:
28
28
  *
29
29
  * 1. **`'ui-bucket'`** (default): `COUNT(DISTINCT date_trunc(unit, time))`
30
- * in UTC calendar-bucket counting, the shape the Mixpanel UI presents
31
- * in frequency distribution charts.
30
+ * in UTC. This measures calendar activity and remains the compatibility
31
+ * default; raw-count histograms and rolling frequency use other rules.
32
32
  *
33
33
  * 2. **`'mixpanel-rolling'`**: the addiction_query.cpp rule,
34
34
  * `qtz_time >= interval->last_counted + seconds_for_unit(unit)`
@@ -39,8 +39,7 @@ const SECONDS_PER_UNIT = {
39
39
  * calendar days.
40
40
  *
41
41
  * **The default does NOT match addiction_query.cpp's rolling rule.** It is
42
- * kept because it matches what users see in report buckets, and because it
43
- * aligns with `injectOnNewDays` (which classifies days by
42
+ * retained for compatibility with `injectOnNewDays` (which classifies days by
44
43
  * `Math.floor(t / DAY_MS)`); mixing the two algorithms makes the atom and
45
44
  * verifier disagree at boundaries. For the actual Frequency report output
46
45
  * shape (per-interval rolling counters + histogram), use
@@ -69,8 +68,6 @@ export function countDistinctPeriods(events, eventName, unit = 'day', options =
69
68
  const unitMs = seconds * 1000;
70
69
 
71
70
  if (algorithm === 'ui-bucket') {
72
- // Calendar bucket — UTC floor by unit. Matches what Mixpanel's UI
73
- // shows and what `injectOnNewDays` uses internally.
74
71
  const buckets = new Set();
75
72
  for (const t of matches) buckets.add(Math.floor(t / unitMs));
76
73
  return buckets.size;
@@ -500,13 +497,13 @@ export function binByDistinctPeriods(events, eventName, bins, unit = 'day', opti
500
497
  * @param {string} [options.event] - Event name to count. Required at runtime — throws if missing.
501
498
  * @param {('hour'|'day'|'week')} [options.unit='day'] - Rolling unit.
502
499
  * @param {number} [options.intervalDays] - Report interval length in days (positive integer). Required at runtime.
503
- * @param {Object[]} [options.profiles] - Profiles for device→user identity resolution.
500
+ * @param {Object[]} [options.profiles] - Accepted for compatibility; identity uses emitted both-ID events.
504
501
  * @returns {Array<{ interval: string, histogram: number[] }>} One row per
505
502
  * interval (label = ISO date of the interval start); `histogram[i]` =
506
503
  * number of users with rolling count `i + 1` in that interval.
507
504
  */
508
505
  export function frequencyHistogram(events, options = {}) {
509
- const { event, unit = 'day', intervalDays, profiles } = options;
506
+ const { event, unit = 'day', intervalDays } = options;
510
507
  if (typeof event !== 'string' || !event) {
511
508
  throw new Error('frequencyHistogram: event is required');
512
509
  }
@@ -517,7 +514,7 @@ export function frequencyHistogram(events, options = {}) {
517
514
  }
518
515
  if (!Array.isArray(events) || !events.length) return [];
519
516
 
520
- const identityMap = profiles ? buildIdentityMap(profiles) : undefined;
517
+ const identityMap = buildEventIdentityMap(events);
521
518
  let minMs = Infinity, maxMs = -Infinity;
522
519
  const matches = [];
523
520
  for (const e of events) {
@@ -23,7 +23,7 @@
23
23
 
24
24
  import { toMs } from '../hook-helpers/_internal.js';
25
25
  import { evaluateFunnel, evaluateFunnelHPC, evaluateAnyOrderCompletion } from './funnel-engine.js';
26
- import { buildIdentityMap, resolveUserId } from './identity.js';
26
+ import { buildEventIdentityMap, resolveUserId } from './identity.js';
27
27
  import { coerceToBreakdownKey, breakdownSegmentKey, matchesWhere } from './coerce.js';
28
28
  import { filterFirstTimeEver } from './first-time.js';
29
29
  import { sessionize } from './sessionize.js';
@@ -277,18 +277,7 @@ export function emulateBreakdown(events, config) {
277
277
  if (!Array.isArray(events)) throw new Error('emulateBreakdown: events must be an array');
278
278
  if (!config || !config.type) throw new Error('emulateBreakdown: config.type is required');
279
279
 
280
- // Auto-build identity map ONCE when profiles supplied. Threads through every
281
- // breakdown type AND every time-bucket recursive call so pre-auth (device_id
282
- // only) events resolve to the same canonical user as post-auth (user_id)
283
- // events. Hoisted above the timeBucket dispatch to avoid rebuilding the
284
- // map per-bucket on large datasets.
285
- const identityMap = config.identityMap
286
- || (Array.isArray(config.profiles)
287
- && config.profiles.some(p =>
288
- p && ((Array.isArray(p.device_ids) && p.device_ids.length)
289
- || (Array.isArray(p.anonymousIds) && p.anonymousIds.length)))
290
- ? buildIdentityMap(config.profiles)
291
- : undefined);
280
+ const identityMap = config.identityMap || buildEventIdentityMap(events);
292
281
 
293
282
  // v1.5: time-bucketed wrapper. Partition events by UTC bucket, run the
294
283
  // underlying breakdown per partition, tag rows with `period`.
@@ -318,12 +307,35 @@ export function emulateBreakdown(events, config) {
318
307
  throw new Error('emulateBreakdown: type "topPaths" does not compose with timeBucket — flows aggregate one path universe over the range');
319
308
  }
320
309
  const range = config.timeBucketRange || {};
321
- const buckets = partitionByTimeBucket(events, config.timeBucket, range);
310
+ const bucketEvents = config.type === 'attributedBy'
311
+ ? events.filter(event => event && event.event === config.conversionEvent)
312
+ : events;
313
+ const buckets = partitionByTimeBucket(bucketEvents, config.timeBucket, range);
322
314
  // Pass the pre-built identityMap into recursive calls so the auto-build
323
315
  // branch above is a no-op per bucket (would otherwise rebuild N times).
324
316
  const inner = { ...config, timeBucket: undefined, timeBucketRange: undefined, identityMap };
325
317
  const out = [];
326
318
 
319
+ if (config.type === 'attributedBy') {
320
+ attributedBy([], {
321
+ conversionEvent: config.conversionEvent,
322
+ attributionEvent: config.attributionEvent,
323
+ attributionProperty: config.attributionProperty,
324
+ model: config.model,
325
+ perConversion: config.perConversion,
326
+ identityMap,
327
+ });
328
+ for (const { period } of buckets) {
329
+ const rows = attributedBy(events, /** @type {*} */ (inner), bucketBoundsMs(period, config.timeBucket));
330
+ if (rows.length) {
331
+ for (const row of rows) out.push({ period, ...row });
332
+ } else {
333
+ out.push({ period, _empty: true });
334
+ }
335
+ }
336
+ return out;
337
+ }
338
+
327
339
  // v1.6.0 (P1.6.5): step-0-anchored trend types. Mixpanel evaluates each
328
340
  // trend interval as "step 0 in [start, stop); steps 1+ in
329
341
  // [start, stop + conversion window)" (funnel_query.cpp:1398-1401), and
@@ -483,13 +495,6 @@ function funnelFrequency(events, { steps, breakdownByFrequencyOf, conversionWind
483
495
  if ((isTotalsMode || holdPropertyConstant) && !isSequentialOrder) {
484
496
  throw new Error(`funnelFrequency: countMode '${countMode}' / holdPropertyConstant require a sequential funnel order (got '${funnelOrder}')`);
485
497
  }
486
- // HPC buckets events by property value BEFORE evaluation (evaluateFunnelHPC),
487
- // but Mixpanel derives session boundaries from the user's FULL event stream —
488
- // sessionizing each bucket independently would merge across the gaps left by
489
- // removed events and produce wrong ordinals. Refuse rather than mis-count.
490
- if (holdPropertyConstant && (countMode === 'sessions' || conversionWindow)) {
491
- throw new Error('funnelFrequency: holdPropertyConstant cannot combine with session-count conversion windows — session boundaries derive from the full event stream, but HPC evaluates per-property-value event subsets');
492
- }
493
498
 
494
499
  const userEvents = groupByUser(events, identityMap);
495
500
  const result = [];
@@ -787,29 +792,43 @@ function attributedBy(events, {
787
792
  model = 'firstTouch',
788
793
  perConversion = 'first',
789
794
  identityMap,
790
- }) {
795
+ }, conversionRange = null) {
791
796
  if (!conversionEvent || !attributionEvent || !attributionProperty) {
792
797
  throw new Error('attributedBy requires conversionEvent, attributionEvent, attributionProperty');
793
798
  }
794
799
  if (perConversion !== 'first' && perConversion !== 'all') {
795
800
  throw new Error(`attributedBy: unknown perConversion "${perConversion}" — use 'first' or 'all'`);
796
801
  }
802
+ // Attribution is unbounded over supplied history; this API has no finite
803
+ // lookback. The private bucket interval limits conversions, never touches.
804
+ // Backend transitions also compress case-insensitive equal values within
805
+ // 30 minutes (value/transitions.cpp, whoval/util.cpp). This raw-value model
806
+ // does not emulate that compression, which can retain earlier string casing.
797
807
  const userEvents = groupByUser(events, identityMap);
798
808
  const counts = new Map();
799
809
  for (const [, evs] of userEvents) {
800
810
  const sorted = sortByTime(evs);
811
+ const inRange = sorted.filter(event => {
812
+ if (!event || event.event !== conversionEvent) return false;
813
+ const time = toMs(event.time);
814
+ return !conversionRange || (time >= conversionRange.startMs && time < conversionRange.endMs);
815
+ });
801
816
  const conversions = perConversion === 'all'
802
- ? sorted.filter(e => e && e.event === conversionEvent)
803
- : sorted.filter(e => e && e.event === conversionEvent).slice(0, 1);
817
+ ? inRange
818
+ : inRange.slice(0, 1);
804
819
  for (const conversion of conversions) {
805
820
  const conversionTime = toMs(conversion.time);
806
- const allTouches = sorted.filter(e =>
807
- e && e.event === attributionEvent && toMs(e.time) <= conversionTime
821
+ // attribution.py's defined-property filter excludes null and undefined;
822
+ // whoval/read.cpp first/last include the conversion timestamp.
823
+ const allTouches = sorted.filter(event =>
824
+ event && event.event === attributionEvent && event[attributionProperty] != null
825
+ && toMs(event.time) <= conversionTime
808
826
  );
809
- if (!allTouches.length) continue;
810
827
  const touch = model === 'lastTouch' ? allTouches[allTouches.length - 1] : allTouches[0];
811
- const v = touch[attributionProperty] ?? 'unknown';
812
- counts.set(v, (counts.get(v) || 0) + 1);
828
+ // Backend missing attribution is undefined. Preserve this API's existing
829
+ // 'unknown' label, not coerceToBreakdownKey's general 'undefined' label.
830
+ const value = touch?.[attributionProperty] ?? 'unknown';
831
+ counts.set(value, (counts.get(value) || 0) + 1);
813
832
  }
814
833
  }
815
834
  return [...counts.entries()].map(([source, count]) => ({
@@ -187,7 +187,7 @@
187
187
 
188
188
  import { toMs } from '../hook-helpers/_internal.js';
189
189
  import { sessionOrdinals } from './sessionize.js';
190
- import { matchesWhere } from './coerce.js';
190
+ import { coerceToBreakdownKey, matchesWhere } from './coerce.js';
191
191
 
192
192
  const OUT_OF_ORDER_MS = 2000;
193
193
  const DAY_MS = 86400 * 1000;
@@ -1016,13 +1016,21 @@ function evaluateFunnelWithContext(events, steps, options, fullStreamOrdinals) {
1016
1016
  * group and drop off in another simultaneously). Session windows derive
1017
1017
  * ordinals from the full user stream before routing events to HPC buckets.
1018
1018
  *
1019
- * Reference: `funnel_query.cpp` lines 749-784 (`aggregate_hash_get_key_cursor`).
1019
+ * Analytics 717286d2: backend/arb/reader/queries/funnel_query.cpp:749
1020
+ * processes each key from backend/libquery/aggregate.cpp:36,113. Ordinary
1021
+ * HPC expands lists one level and accepts scalars. Both use value_to_string
1022
+ * (backend/libquery/value/value.c:184), not typed breakdown/filter equality.
1023
+ * Empty lists emit no visits; null/missing items use the "undefined" label.
1024
+ * Duplicate labels share a history but retain cursor visits, which can
1025
+ * advance repeated steps (funnel_query.cpp:1257; funnels/history.cpp:387).
1020
1026
  *
1021
- * **Limitation (v1.5.0):** scalar HPC values only. Mixpanel's
1022
- * `aggregate_hash_get_key_cursor` iterates *each value* of a list-valued
1023
- * property, exploding into N sub-funnels per event. List-valued HPC keys are
1024
- * not supported here events with non-scalar `holdProperty` values will
1025
- * stringify and bucket incorrectly.
1027
+ * Compatibility: all-scalar streams retain raw keys and omit null/missing
1028
+ * values. List-bearing streams use string labels for all routed values.
1029
+ * This property-name API does not expose the separate explicit-list mode,
1030
+ * which rejects non-lists (aggregate.cpp:126). Events and their copy-data
1031
+ * properties remain unchanged; only the routing key is normalized.
1032
+ * The shared label helper does not reproduce ARB's numeric %.16g fallback,
1033
+ * structured-value serialization/truncation, or aggregate cardinality cap.
1026
1034
  *
1027
1035
  * @param {Array<Object>} events
1028
1036
  * @param {FunnelStep[]} steps
@@ -1044,6 +1052,11 @@ export function evaluateFunnelHPC(events, steps, holdProperty, options = {}) {
1044
1052
  const fullStreamOrdinals = options.countMode === 'sessions' || options.conversionWindow?.unit === 'sessions'
1045
1053
  ? sessionOrdinals((events || []).filter(ev => ev && typeof ev.event === 'string'))
1046
1054
  : undefined;
1055
+ const hasListValues = (events || []).some(ev => ev && typeof ev.event === 'string' && Array.isArray(ev[holdProperty]));
1056
+ const heldValues = (value) => {
1057
+ if (!hasListValues) return value == null ? [] : [value];
1058
+ return (Array.isArray(value) ? value : [value]).map(coerceToBreakdownKey);
1059
+ };
1047
1060
 
1048
1061
  // Bucket events by HPC value. The step-0 events define the universe of
1049
1062
  // HPC values for this user; later events only populate buckets whose
@@ -1053,18 +1066,18 @@ export function evaluateFunnelHPC(events, steps, holdProperty, options = {}) {
1053
1066
  if (!ev || typeof ev.event !== 'string') continue;
1054
1067
  // Step-0 events seed the bucket on their own value.
1055
1068
  if (ev.event === step0Name) {
1056
- const v = ev[holdProperty];
1057
- if (v === undefined || v === null) continue;
1058
- if (!valueBuckets.has(v)) valueBuckets.set(v, []);
1059
- valueBuckets.get(v).push(ev);
1069
+ for (const value of heldValues(ev[holdProperty])) {
1070
+ if (!valueBuckets.has(value)) valueBuckets.set(value, []);
1071
+ valueBuckets.get(value).push(ev);
1072
+ }
1060
1073
  }
1061
1074
  }
1062
1075
  // Now route every event with a known HPC value into its bucket.
1063
1076
  for (const ev of events || []) {
1064
1077
  if (!ev || typeof ev.event !== 'string' || ev.event === step0Name) continue;
1065
- const v = ev[holdProperty];
1066
- if (v === undefined || v === null) continue;
1067
- if (valueBuckets.has(v)) valueBuckets.get(v).push(ev);
1078
+ for (const value of heldValues(ev[holdProperty])) {
1079
+ if (valueBuckets.has(value)) valueBuckets.get(value).push(ev);
1080
+ }
1068
1081
  }
1069
1082
 
1070
1083
  const out = new Map();
@@ -1,15 +1,13 @@
1
1
  /**
2
- * Identity resolution for verifier builds a device→user map from profiles
3
- * and resolves a canonical user id per event. Mirrors Mixpanel's ID merge
4
- * semantics: pre-auth events stamped with `device_id` and post-auth events
5
- * stamped with `user_id` belong to the same canonical identity.
2
+ * Identity resolution for verifier. Automatic links come from emitted
3
+ * both-ID events; profile pools are only an explicit caller override.
6
4
  *
7
- * Reference: `mixpanel/analytics` identity merge / profiles `device_ids`
8
- * inversion. We invert each profile's `device_ids` array into a flat
9
- * `Map<device_id, canonical_user_id>` so query-time event grouping resolves
10
- * pre-auth touches alongside post-auth events.
5
+ * Reference: analytics identity-manager v3 lookup_and_update_handler.go
6
+ * and identity/device_and_user_records.go at 717286d2d3ed03e9e3f9cb4346e4c6b2e561fb9a.
11
7
  */
12
8
 
9
+ const emittedIdentityMap = Symbol('emittedIdentityMap');
10
+
13
11
  /**
14
12
  * Build a `Map<device_id, canonical_user_id>` by inverting each profile's
15
13
  * device-pool array. Reads `device_ids` first, falling back to the legacy
@@ -41,12 +39,36 @@ export function buildIdentityMap(profiles) {
41
39
  return map;
42
40
  }
43
41
 
42
+ /**
43
+ * Build automatic device-to-user links from the full emitted stream.
44
+ * Event names do not gate linking. The first valid pair for a device wins;
45
+ * v3 does not reassign an already-linked device to a conflicting user.
46
+ * User IDs with the reserved $device: prefix cannot establish a link.
47
+ *
48
+ * @param {Array<Object>} events
49
+ * @returns {Map<string, string>}
50
+ */
51
+ export function buildEventIdentityMap(events) {
52
+ const map = new Map();
53
+ Object.defineProperty(map, emittedIdentityMap, { value: true });
54
+ if (!Array.isArray(events)) return map;
55
+ for (const event of events) {
56
+ const deviceId = event?.device_id;
57
+ const userId = event?.user_id;
58
+ if (typeof deviceId !== 'string' || !deviceId.trim()
59
+ || typeof userId !== 'string' || !userId.trim()
60
+ || userId.startsWith('$device:') || map.has(deviceId)) continue;
61
+ map.set(deviceId, userId);
62
+ }
63
+ return map;
64
+ }
65
+
44
66
  /**
45
67
  * Resolve the canonical user id for an event. Lookup order:
46
68
  * 1. `event.distinct_id` — Mixpanel's canonical post-merge identifier. When
47
69
  * a downstream pipeline has already stitched the cluster, this is the
48
70
  * ground truth; never override it.
49
- * 2. `identityMap.get(event.device_id)` — device→user merge from profile inversion.
71
+ * 2. `identityMap.get(event.device_id)` — emitted link or explicit override.
50
72
  * 3. `event.user_id` — already authed (pre-merge analog of distinct_id).
51
73
  * 4. `event.device_id` — anonymous fallback.
52
74
  *
@@ -70,6 +92,7 @@ export function buildIdentityMap(profiles) {
70
92
  export function resolveUserId(event, identityMap) {
71
93
  if (!event) return undefined;
72
94
  if (event.distinct_id) return event.distinct_id;
95
+ if (event.user_id && identityMap && Object.getOwnPropertyDescriptor(identityMap, emittedIdentityMap)) return event.user_id;
73
96
  if (identityMap && event.device_id) {
74
97
  const merged = identityMap.get(event.device_id);
75
98
  if (merged) return merged;
@@ -20,9 +20,9 @@
20
20
  * neutral. For 'between' the wanted side is where the band's midpoint sits
21
21
  * relative to the neutral point.
22
22
  *
23
- * Cohort size per named selection: sum of `user_count` over the selected rows
24
- * when any row carries one, else the row count. The SMALLEST selection is
25
- * compared against `minCohort`.
23
+ * Cohort evidence per selection is an independent-user lower bound: sum only
24
+ * proven disjoint bins within a period, then take the largest period. Other
25
+ * user counts use the largest row. Missing evidence caps passes at WEAK.
26
26
  */
27
27
 
28
28
  import { emulateBreakdown } from './emulate-breakdown.js';
@@ -166,11 +166,59 @@ function resolveRef(ref, selected) {
166
166
  return sum;
167
167
  }
168
168
 
169
- function cohortOf(rows) {
170
- if (rows.some(r => typeof r.user_count === 'number')) {
171
- return rows.reduce((s, r) => s + (typeof r.user_count === 'number' ? r.user_count : 0), 0);
169
+ function cohortOf(rows, breakdown, sourceRows) {
170
+ if (!Array.isArray(rows)) return null;
171
+ let evidenceRows = (rows || []).filter(row => row && !row._empty);
172
+ if (!evidenceRows.length) return null;
173
+ const isCount = value => Number.isSafeInteger(value) && value >= 0;
174
+ const hasPeriod = row => typeof row.period === 'string' && row.period.length > 0;
175
+ let column = 'user_count';
176
+ let binColumns = [];
177
+ if (breakdown.type === 'funnelFrequency') {
178
+ if (breakdown.countMode !== undefined && breakdown.countMode !== 'uniques') return null;
179
+ const periodRequired = breakdown.timeBucket !== undefined || evidenceRows.some(row => row.period !== undefined);
180
+ if (evidenceRows.some(row => !isCount(row.step_index) || !isCount(row.breakdown_freq)
181
+ || (periodRequired && !hasPeriod(row)))) return null;
182
+ const binKey = row => JSON.stringify([row.period, row.breakdown_freq]);
183
+ const selectedBins = new Set(evidenceRows.map(binKey));
184
+ evidenceRows = (sourceRows || []).filter(row => row && !row._empty && row.step_index === 0
185
+ && isCount(row.breakdown_freq) && (row.period === undefined || hasPeriod(row))
186
+ && selectedBins.has(binKey(row)));
187
+ if (new Set(evidenceRows.map(binKey)).size !== selectedBins.size) return null;
188
+ column = 'conversions';
189
+ binColumns = ['breakdown_freq'];
190
+ } else if (breakdown.type === 'eventBreakdown') {
191
+ column = 'total_users';
192
+ } else if (breakdown.type === 'uniques') {
193
+ if (breakdown.countType !== undefined && breakdown.countType !== 'unique') return null;
194
+ column = 'uniques';
195
+ } else if (breakdown.type === 'retention') {
196
+ column = 'cohort_size';
197
+ } else if (breakdown.type === 'frequencyByFrequency') {
198
+ binColumns = ['metric_freq', 'breakdown_freq'];
199
+ } else if (breakdown.type === 'aggregatePerUser') {
200
+ binColumns = ['breakdown_freq'];
172
201
  }
173
- return rows.length;
202
+ if (evidenceRows.some(row => !isCount(row[column]))) return null;
203
+ const largestRow = evidenceRows.reduce((largest, row) => Math.max(largest, row[column]), 0);
204
+ const periodRequired = breakdown.timeBucket !== undefined || evidenceRows.some(row => row.period !== undefined);
205
+ if (!binColumns.length || evidenceRows.some(row => binColumns.some(bin => !isCount(row[bin]))
206
+ || (periodRequired && !hasPeriod(row)))) return largestRow;
207
+ const periods = new Map();
208
+ for (const row of evidenceRows) {
209
+ if (!periods.has(row.period)) periods.set(row.period, new Map());
210
+ const bins = periods.get(row.period);
211
+ const key = JSON.stringify(binColumns.map(bin => row[bin]));
212
+ bins.set(key, Math.max(bins.get(key) || 0, row[column]));
213
+ }
214
+ let largestPeriod = 0;
215
+ for (const bins of periods.values()) {
216
+ let total = 0;
217
+ for (const count of bins.values()) total += count;
218
+ if (!Number.isSafeInteger(total)) return largestRow;
219
+ largestPeriod = Math.max(largestPeriod, total);
220
+ }
221
+ return largestPeriod;
174
222
  }
175
223
 
176
224
  // ── verdicts ────────────────────────────────────────────────────────────────
@@ -243,40 +291,55 @@ const fmt = (n) => (typeof n === 'number' && Number.isFinite(n)) ? (Math.abs(n)
243
291
  */
244
292
  export function evaluateAssertion(rows, assertion, ctx) {
245
293
  try {
294
+ const minCohort = assertion.minCohort;
295
+ let result, selected, cohorts;
246
296
  if (typeof assertion.assert === 'function') {
297
+ if (typeof minCohort === 'number') {
298
+ selected = assertion.select === undefined ? { all: rows } : selectRows(rows, assertion.select);
299
+ cohorts = Object.values(selected).map(selection => cohortOf(selection, assertion.breakdown, rows));
300
+ }
247
301
  /** @type {{ pass?: boolean, detail?: string, verdict?: import('../../types').StoryVerdict }} */
248
302
  const res = assertion.assert(rows, ctx) || {};
249
303
  if (res.verdict && VERDICT_RANK[res.verdict] !== undefined) {
250
- return { verdict: res.verdict, observed: null, detail: res.detail || 'custom assert' };
304
+ result = { verdict: res.verdict, observed: null, detail: res.detail || 'custom assert' };
305
+ } else {
306
+ result = res.pass
307
+ ? { verdict: 'STRONG', observed: null, detail: res.detail || 'custom assert passed' }
308
+ : { verdict: 'NONE', observed: null, detail: res.detail || 'custom assert failed' };
251
309
  }
252
- return res.pass
253
- ? { verdict: 'STRONG', observed: null, detail: res.detail || 'custom assert passed' }
254
- : { verdict: 'NONE', observed: null, detail: res.detail || 'custom assert failed' };
255
- }
256
- const parsed = parseMetric(assertion.expect.metric);
257
- const selected = selectRows(rows, assertion.select);
258
- const left = resolveRef(parsed.left, selected);
259
- let observed, neutral;
260
- if (parsed.kind === 'single') {
261
- observed = left; neutral = null;
262
310
  } else {
263
- const right = resolveRef(parsed.right, selected);
264
- if (parsed.kind === 'ratio') {
265
- if (right === 0) throw new Error('ratio denominator is 0');
266
- observed = left / right; neutral = 1;
311
+ const parsed = parseMetric(assertion.expect.metric);
312
+ selected = selectRows(rows, assertion.select);
313
+ const left = resolveRef(parsed.left, selected);
314
+ let observed, neutral;
315
+ if (parsed.kind === 'single') {
316
+ observed = left; neutral = null;
267
317
  } else {
268
- observed = left - right; neutral = 0;
318
+ const right = resolveRef(parsed.right, selected);
319
+ if (parsed.kind === 'ratio') {
320
+ if (right === 0) throw new Error('ratio denominator is 0');
321
+ observed = left / right; neutral = 1;
322
+ } else {
323
+ observed = left - right; neutral = 0;
324
+ }
269
325
  }
326
+ const { verdict, detail } = verdictFor(observed, assertion.expect, neutral);
327
+ result = { verdict, observed, detail };
270
328
  }
271
- let { verdict, detail } = verdictFor(observed, assertion.expect, neutral);
272
- if (typeof assertion.minCohort === 'number' && VERDICT_RANK[verdict] > VERDICT_RANK.WEAK) {
273
- const smallest = Math.min(...Object.values(selected).map(cohortOf));
274
- if (smallest < assertion.minCohort) {
275
- verdict = 'WEAK';
276
- detail += ` — capped: smallest cohort ${smallest} < minCohort ${assertion.minCohort}`;
329
+ if (typeof minCohort === 'number' && VERDICT_RANK[result.verdict] > VERDICT_RANK.WEAK) {
330
+ cohorts ??= Object.values(selected).map(selection => cohortOf(selection, assertion.breakdown, rows));
331
+ if (!cohorts.length || cohorts.some(cohort => cohort === null)) {
332
+ result.verdict = 'WEAK';
333
+ result.detail += ` - capped: insufficient evidence for minCohort ${minCohort}; independent-user denominator unavailable`;
334
+ } else {
335
+ const smallest = Math.min(...cohorts);
336
+ if (smallest < minCohort) {
337
+ result.verdict = 'WEAK';
338
+ result.detail += ` - capped: insufficient evidence; smallest independent-user lower bound ${smallest} < minCohort ${minCohort}`;
339
+ }
277
340
  }
278
341
  }
279
- return { verdict, observed, detail };
342
+ return result;
280
343
  } catch (err) {
281
344
  return { verdict: 'NONE', observed: null, detail: `error: ${err.message}` };
282
345
  }
@@ -21,6 +21,7 @@
21
21
 
22
22
  import DUNGEON_MASTER from '../../index.js';
23
23
  import { emulateBreakdown } from './emulate-breakdown.js';
24
+ import { buildEventIdentityMap } from './identity.js';
24
25
  import { validateSchema } from './schema-validator.js';
25
26
 
26
27
  /**
@@ -83,7 +84,7 @@ export function applyFunnelDefaults(breakdownArgs, funnels, profiles) {
83
84
  // is the author choosing a window — injecting conversionWindowMs on top
84
85
  // would trip evaluateFunnel's mutual-exclusion throw. Only inject when
85
86
  // NEITHER window form is present.
86
- if (args.conversionWindowMs === undefined && args.conversionWindow === undefined && Number.isFinite(matched.conversionWindowDays)) {
87
+ if (args.countMode !== 'sessions' && args.conversionWindowMs === undefined && args.conversionWindow === undefined && Number.isFinite(matched.conversionWindowDays)) {
87
88
  args.conversionWindowMs = matched.conversionWindowDays * 86400000;
88
89
  }
89
90
  if (args.funnelOrder === undefined && matched.order) {
@@ -137,6 +138,7 @@ export async function verifyDungeon(config, checks, overrides) {
137
138
  }
138
139
  const events = Array.isArray(result.eventData) ? result.eventData : Array.from(result.eventData);
139
140
  const profiles = Array.isArray(result.userProfilesData) ? result.userProfilesData : Array.from(result.userProfilesData);
141
+ const identityMap = buildEventIdentityMap(events);
140
142
  // v1.6.2: schema-check against the config the run actually used. `config` may be a
141
143
  // PATH STRING (no fields at all → every property reads as unexpected), and even for
142
144
  // an object input a v1.5.1 dungeon keeps `hasAndroidDevices` / `hasBrowser` under
@@ -155,6 +157,7 @@ export async function verifyDungeon(config, checks, overrides) {
155
157
  for (const check of checks) {
156
158
  try {
157
159
  const breakdownArgs = applyFunnelDefaults(check.breakdown, validatedFunnels, profiles);
160
+ if (breakdownArgs.identityMap === undefined) breakdownArgs.identityMap = identityMap;
158
161
  const rows = emulateBreakdown(events, breakdownArgs);
159
162
  const verdict = check.assert(rows, ctx);
160
163
  results.push({ name: check.name, pass: !!verdict.pass, detail: verdict.detail, rows });
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@ak--47/dungeon-master",
3
- "version": "1.8.1",
3
+ "version": "1.8.2",
4
4
  "description": "generate fancy datasets",
5
5
  "type": "module",
6
6
  "main": "index.js",
@@ -38,8 +38,8 @@ import { parse as parseCsv } from 'csv-parse';
38
38
  import generate from '../index.js';
39
39
  import { extractComments } from '../lib/core/extract-comments.js';
40
40
  import { validateDungeonConfig } from '../lib/core/config-validator.js';
41
+ import { buildEventIdentityMap } from '../lib/verify/identity.js';
41
42
  import {
42
- buildIdentityMap,
43
43
  VERDICT_RANK,
44
44
  validateStories,
45
45
  validateSchema,
@@ -149,7 +149,7 @@ if (inMemory) {
149
149
  ? await evaluateStories(stories, events, {
150
150
  profiles,
151
151
  funnels: Array.isArray(validated.funnels) ? validated.funnels : [],
152
- identityMap: buildIdentityMap(profiles),
152
+ identityMap: buildEventIdentityMap(events),
153
153
  warehouseRows,
154
154
  warehouseSpecs,
155
155
  datasetStart: validated.datasetStart,
@@ -187,7 +187,7 @@ if (inMemory) {
187
187
  // here and use the RETURN value — as of v1.6.2 validateDungeonConfig does not
188
188
  // enrich the object you hand it.
189
189
  const validated = validateDungeonConfig({ ...config, token: '' });
190
- const identityMap = buildIdentityMap(profiles);
190
+ const identityMap = buildEventIdentityMap(events);
191
191
  schemaPass = !!validateSchema(events, validated)?.pass;
192
192
  const warehouseSpecs = Object.fromEntries((validated.warehouseMetrics || []).map((spec) => [spec.name, spec]));
193
193
  const warehouseManifest = loadWarehouseManifest(prefixPath);