@ak--47/dungeon-master 1.8.1 → 1.8.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/verify-dungeon/references/alignment-contract.md +30 -4
- package/.claude/skills/verify-dungeon/references/counting-semantics.md +5 -3
- package/CHANGELOG.md +41 -0
- package/HOOKS.md +45 -34
- package/README.md +4 -0
- package/docs/guides/1.8.2-upgrade-guide.md +110 -0
- package/lib/generators/funnels.js +1 -0
- package/lib/hook-patterns/attributed-by-source.js +4 -3
- package/lib/hook-patterns/funnel-frequency-breakdown.js +4 -7
- package/lib/verify/counting.js +7 -10
- package/lib/verify/emulate-breakdown.js +48 -29
- package/lib/verify/funnel-engine.js +27 -14
- package/lib/verify/identity.js +32 -9
- package/lib/verify/story-runner.js +93 -30
- package/lib/verify/verify-dungeon.js +4 -1
- package/package.json +1 -1
- package/scripts/verify-stories.mjs +3 -3
|
@@ -1,5 +1,31 @@
|
|
|
1
1
|
# 1.8.1 verification contract
|
|
2
2
|
|
|
3
|
+
## 1.8.2 live verification update
|
|
4
|
+
|
|
5
|
+
The [1.8.2 guide](../../../../docs/guides/1.8.2-upgrade-guide.md) and repository
|
|
6
|
+
`tests/alignment/live/REPORT.md` extend this contract with retained live comparisons.
|
|
7
|
+
The 1.8.1 audit below remains historical. It used selected source-derived contracts;
|
|
8
|
+
1.8.2 also imports isolated datasets and compares native Mixpanel query results.
|
|
9
|
+
Public options and defaults remain unchanged.
|
|
10
|
+
|
|
11
|
+
Automatic verifier identity now derives from emitted both-ID events. Explicit
|
|
12
|
+
`buildIdentityMap(profiles)` overrides retain their compatibility behavior.
|
|
13
|
+
List-valued held-property expansion and supported HPC/session report combinations
|
|
14
|
+
have source and live fixtures. Non-UTC/DST and custom session-exclusion variants
|
|
15
|
+
still need separate verification.
|
|
16
|
+
|
|
17
|
+
Import success is not query readiness. Reconcile totals and identity before
|
|
18
|
+
accepting comparisons. Historical anonymous events can join later without another
|
|
19
|
+
import. Every live test must use a run property and disjoint run identities.
|
|
20
|
+
Retain incomplete responses and do other work before a bounded retry.
|
|
21
|
+
|
|
22
|
+
Distinguish raw per-user count histograms, UTC calendar activity, and rolling
|
|
23
|
+
Frequency/Addiction. The default local calendar algorithm is not a universal
|
|
24
|
+
Mixpanel frequency rule. Lifetime attribution endpoints require conversion/lookback
|
|
25
|
+
eligibility; conflicting values with tied timestamps are not stable across ingest.
|
|
26
|
+
`minCohort` now enforces conservative independent-user lower bounds, including
|
|
27
|
+
custom callbacks. Unknown evidence caps a passing verdict instead of counting rows.
|
|
28
|
+
|
|
3
29
|
Use this contract when authoring, verifying, provisioning, or presenting a dungeon.
|
|
4
30
|
The [1.8.1 guide](../../../../docs/guides/1.8.1-upgrade-guide.md) describes the
|
|
5
31
|
release. The local repository's `tests/alignment/REPORT.md` and
|
|
@@ -17,8 +43,8 @@ a numeric property, a mean with a median, or conversion with event volume to
|
|
|
17
43
|
rescue a failed story. Supplementary diagnostics must keep their own labels.
|
|
18
44
|
|
|
19
45
|
Local checks prove selected source-derived contracts, not universal Mixpanel
|
|
20
|
-
parity. The audit read analytics source without compiling or executing it.
|
|
21
|
-
did not run live differential queries. Macro/soup interactions, table surfaces,
|
|
46
|
+
parity. The 1.8.1 audit read analytics source without compiling or executing it.
|
|
47
|
+
That audit did not run live differential queries. Macro/soup interactions, table surfaces,
|
|
22
48
|
arbitrary hooks, parallel execution, and several counting variants remain gaps.
|
|
23
49
|
|
|
24
50
|
## Separate a measured effect from enough evidence
|
|
@@ -52,8 +78,8 @@ and preserve its original failed evidence.
|
|
|
52
78
|
- Derive sessions from the full resolved user stream before HPC partitioning.
|
|
53
79
|
Local defaults are UTC, a 30-minute idle timeout, and a 24-hour maximum.
|
|
54
80
|
`session_id` is a diagnostic or explicit legacy mode, not the default truth.
|
|
55
|
-
Non-UTC projects
|
|
56
|
-
|
|
81
|
+
Non-UTC projects and project-specific hidden/excluded session events remain
|
|
82
|
+
outside the live proof. List-valued HPC has 1.8.2 source and live fixtures.
|
|
57
83
|
- Profile device pools alone establish no identity mapping evidence. The public
|
|
58
84
|
`buildIdentityMap(profiles)` helper retains profile-based compatibility behavior.
|
|
59
85
|
For emitted-identity proof, derive links from valid ordinary both-ID events,
|
|
@@ -210,9 +210,11 @@ emulateBreakdown(events, {
|
|
|
210
210
|
});
|
|
211
211
|
```
|
|
212
212
|
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
213
|
+
In 1.8.2, automatic mapping uses emitted both-ID records before report filtering.
|
|
214
|
+
The public `buildIdentityMap(profiles)` helper still reads profile pools when a
|
|
215
|
+
caller explicitly requests that override. It cannot prove that a link survived
|
|
216
|
+
generation or ingest. Imported historical identity can become query-visible later;
|
|
217
|
+
verify readiness before accepting a local/live comparison.
|
|
216
218
|
|
|
217
219
|
## Time-series verification (timeBucket)
|
|
218
220
|
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,47 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to `@ak--47/dungeon-master`.
|
|
4
4
|
|
|
5
|
+
## 1.8.2 - 2026-09-13
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- Build automatic verifier identity maps from emitted both-ID events. Profile
|
|
10
|
+
device pools no longer create links by themselves. Explicit identity maps
|
|
11
|
+
retain their existing behavior; conflicting authenticated events retain their user.
|
|
12
|
+
- Pass supported held-property session funnels through the report wrapper. Keep
|
|
13
|
+
implicit one-session windows when applying matching dungeon funnel defaults.
|
|
14
|
+
- Expand list-valued held properties into separate funnel histories, including
|
|
15
|
+
repeated cursor visits and full-stream session boundaries.
|
|
16
|
+
- Preserve prior attribution touches across time buckets, ignore unstamped
|
|
17
|
+
touches, and retain conversions with no matching touch in the unknown segment.
|
|
18
|
+
- Make `minCohort` use conservative independent-user evidence. Repeated period
|
|
19
|
+
rows cannot inflate populations, and custom assertions receive the same guard.
|
|
20
|
+
- Include declared global properties on synthetic experiment exposure events,
|
|
21
|
+
so run and segment filters retain both exposures and outcomes.
|
|
22
|
+
- Make funnel-frequency drop decisions independent of random insertion IDs.
|
|
23
|
+
|
|
24
|
+
### Verified
|
|
25
|
+
|
|
26
|
+
- Add opt-in live comparisons through the actual importer and Mixpanel query
|
|
27
|
+
layer, with run-isolated identities, import budgets, retained query payloads,
|
|
28
|
+
exact local/live comparisons, fixed seeds, and neutral controls.
|
|
29
|
+
- Verify selected generated conversion, TTC, retention, volume, weight, incident,
|
|
30
|
+
frequency, aggregate, attribution, session, and Flows effects against live reports.
|
|
31
|
+
- Separate ingestion readiness from import acceptance. Historical identity links
|
|
32
|
+
can become visible later; one retained fixture converged without another import.
|
|
33
|
+
|
|
34
|
+
### Compatibility
|
|
35
|
+
|
|
36
|
+
Public exports, signatures, options, and defaults remain unchanged. Corrected
|
|
37
|
+
counts and seeded hook outcomes can differ from 1.8.1. Explicit profile-map
|
|
38
|
+
overrides and totals' `reentry: false` default remain supported. Calendar-day
|
|
39
|
+
activity, rolling frequency, and raw event-count histograms are separate reports.
|
|
40
|
+
Finite-lookback attribution, ambiguous tied touches, non-UTC project settings,
|
|
41
|
+
and arbitrary combinations remain subject to the documented limits.
|
|
42
|
+
|
|
43
|
+
See the [upgrade guide](docs/guides/1.8.2-upgrade-guide.md) and the repository's
|
|
44
|
+
live alignment report for measured scope. This entry does not indicate npm publication.
|
|
45
|
+
|
|
5
46
|
## 1.8.1 — 2026-09-12
|
|
6
47
|
|
|
7
48
|
### Fixed
|
package/HOOKS.md
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
# HOOKS.md -- Hook Encyclopedia
|
|
2
2
|
|
|
3
|
-
Hook reference and recipe catalog for dungeon-master.
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
[
|
|
3
|
+
Hook reference and recipe catalog for dungeon-master. Recipes have different
|
|
4
|
+
counting and evidence requirements. Read [Section 2](#2-how-mixpanel-counts-things)
|
|
5
|
+
before adapting a pattern, then verify the exact report on the emitted dataset.
|
|
6
|
+
The [1.8.2 guide](docs/guides/1.8.2-upgrade-guide.md) records live-tested scope and limits.
|
|
7
7
|
|
|
8
8
|
---
|
|
9
9
|
|
|
@@ -172,20 +172,28 @@ verification emulator (`@ak--47/dungeon-master/verify`) now matches these
|
|
|
172
172
|
rules; old recipes that ignored them will look correct on the dataset but
|
|
173
173
|
fail when verified or queried in Mixpanel.
|
|
174
174
|
|
|
175
|
-
### 2.1 Frequency
|
|
175
|
+
### 2.1 Frequency depends on the selected report
|
|
176
176
|
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
event
|
|
177
|
+
Three measurements require different checks:
|
|
178
|
+
|
|
179
|
+
- **Raw per-user event count:** two purchases on one day count as two. Insights
|
|
180
|
+
per-user count histograms and frequency breakdowns using total event count use
|
|
181
|
+
this axis. `applyFrequencyByFrequency` scales this target count.
|
|
182
|
+
- **Distinct calendar-day activity:** those purchases count as one active day.
|
|
183
|
+
`countDistinctPeriods` defaults to UTC calendar buckets (`ui-bucket`), and
|
|
184
|
+
`binBy: 'distinctDays'` uses the same rule. This default is retained for
|
|
185
|
+
compatibility; it is not a universal Mixpanel frequency-report rule.
|
|
186
|
+
- **Rolling Frequency/Addiction:** an event counts when it is at least one
|
|
187
|
+
selected unit after the last counted event. Histories reset per report interval.
|
|
188
|
+
Two events around midnight can occupy two calendar days but one rolling period.
|
|
180
189
|
|
|
181
190
|
Two related rules exist (v1.6 names them for what they are; the old
|
|
182
191
|
`'calendar'` / `'rolling'` names remain as silent aliases, unknown names
|
|
183
192
|
now throw):
|
|
184
193
|
|
|
185
194
|
- **`algorithm: 'ui-bucket'`** (default in our verifier):
|
|
186
|
-
`COUNT(DISTINCT date_trunc(unit, time))` in UTC.
|
|
187
|
-
|
|
188
|
-
internally.
|
|
195
|
+
`COUNT(DISTINCT date_trunc(unit, time))` in UTC. This is also what
|
|
196
|
+
[`injectOnNewDays`](lib/hook-helpers/inject.js) uses internally.
|
|
189
197
|
- **`algorithm: 'mixpanel-rolling'`**: the C++
|
|
190
198
|
`addiction_query.cpp` rule `qtz_time >= last_counted + seconds_for_unit`
|
|
191
199
|
(`addiction_query_update_history`, `addiction_query.cpp:363-374`) — what
|
|
@@ -193,8 +201,8 @@ now throw):
|
|
|
193
201
|
boundaries (events at 23:59 + 00:01 next day = 1 rolling period, 2
|
|
194
202
|
calendar periods).
|
|
195
203
|
|
|
196
|
-
|
|
197
|
-
|
|
204
|
+
Choose the algorithm from the requested report. Do not replace a raw-count
|
|
205
|
+
histogram with active-day counts, or use calendar days to claim rolling parity.
|
|
198
206
|
|
|
199
207
|
**The actual Frequency report shape** is `frequencyHistogram(events,
|
|
200
208
|
{ event, unit, intervalDays, profiles })` (v1.6): per report interval, a
|
|
@@ -205,19 +213,19 @@ zero bucket (`addiction_query.cpp:546-573`). Array length is
|
|
|
205
213
|
`ceil(interval / unit)` (`unit.c:108-113`). Use it when a dungeon targets
|
|
206
214
|
the Frequency report itself rather than a frequency-derived cohort.
|
|
207
215
|
|
|
208
|
-
**Implication for hooks:** `scaleEventCount(record, "Buy", 3)`
|
|
209
|
-
many Buy events at
|
|
210
|
-
|
|
216
|
+
**Implication for hooks:** `scaleEventCount(record, "Buy", 3)` targets 3x as
|
|
217
|
+
many Buy events at one-second offsets. This changes raw-count histogram bins.
|
|
218
|
+
It generally does not add active days or rolling periods. Use
|
|
211
219
|
[`injectOnNewDays`](lib/hook-helpers/inject.js) when the goal is to move
|
|
212
|
-
users between
|
|
220
|
+
users between calendar-day activity bins. Both `injectOnNewDays` and the default
|
|
213
221
|
`countDistinctPeriods` algorithm use calendar-bucket math, so they agree
|
|
214
222
|
at boundaries.
|
|
215
223
|
|
|
216
224
|
One second is also below Mixpanel's 30-minute session gap, so the default
|
|
217
225
|
spread cannot create a new session either. The full list of metrics a default
|
|
218
|
-
`scaleEventCount` call
|
|
219
|
-
(
|
|
220
|
-
|
|
226
|
+
`scaleEventCount` call does not reliably move: active days per user, DAU,
|
|
227
|
+
stickiness (DAU/MAU), sessions per user, rolling frequency, and retention.
|
|
228
|
+
At dataset edges, clones can be clipped. Verify surviving counts separately.
|
|
221
229
|
|
|
222
230
|
As of v1.6.4 you can pass `{ spreadDays: N }` to scatter the clones across the
|
|
223
231
|
next N days instead:
|
|
@@ -363,7 +371,7 @@ sampling.
|
|
|
363
371
|
|
|
364
372
|
### 2.5 Active-day distribution is config-first
|
|
365
373
|
|
|
366
|
-
|
|
374
|
+
Distinct-day activity is a separate measurement from raw counts (§2.1). The engine
|
|
367
375
|
exposes `Dungeon.avgActiveDaysPerUser` as the canonical primitive for this
|
|
368
376
|
shape. Set it at the config level and the engine concentrates each user's
|
|
369
377
|
events onto a sampled subset of days drawn from `normal(mean=N, sd=N/3)`,
|
|
@@ -876,15 +884,12 @@ Reference: `flows_query.cpp:988-994` (next-anchor-only), `flows.cpp:680-717`
|
|
|
876
884
|
|
|
877
885
|
### New principles from the emulator alignment
|
|
878
886
|
|
|
879
|
-
21. **
|
|
880
|
-
|
|
881
|
-
|
|
882
|
-
instead of `binUsersByEventCount`. For hooks targeting raw event counts
|
|
883
|
-
(Insights `total events`, `events per user`), `binUsersByEventCount` is
|
|
884
|
-
still correct.
|
|
887
|
+
21. **Match the bin to the report.** Use `binUsersByEventCount` for raw event
|
|
888
|
+
counts and `binByDistinctPeriods` for calendar-day activity. Use rolling
|
|
889
|
+
counting for the Frequency/Addiction report. Section 2.1 distinguishes them.
|
|
885
890
|
|
|
886
|
-
22. **`scaleEventCount`
|
|
887
|
-
|
|
891
|
+
22. **`scaleEventCount` changes raw-count bins.** Cloning
|
|
892
|
+
Buy events at one-second offsets usually places them on the same calendar day and
|
|
888
893
|
inside the same 30-minute session window, so distinct days, sessions, DAU,
|
|
889
894
|
stickiness, and retention are all unchanged. It moves event volume only. To
|
|
890
895
|
shift any of the others, pass `{ spreadDays: N }` (v1.6.4) or use
|
|
@@ -898,11 +903,10 @@ Reference: `flows_query.cpp:988-994` (next-anchor-only), `flows.cpp:680-717`
|
|
|
898
903
|
after the prior step's timestamp (with margin > 2 seconds for the grace
|
|
899
904
|
window).
|
|
900
905
|
|
|
901
|
-
24. **
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
touches with deterministic weight ratios.
|
|
906
|
+
24. **Separate stamping caps from attribution reads.** The generator defaults
|
|
907
|
+
to ten stamped touches per user. Mixpanel first/last reads are uncapped
|
|
908
|
+
within their conversion-bounded lookback. Distinct touch timestamps avoid
|
|
909
|
+
ambiguous attribution when different values tie at the first or last time.
|
|
906
910
|
|
|
907
911
|
25. **Null-aware aggregation removes the need to "fill" defaults.** Don't
|
|
908
912
|
coalesce missing numeric properties to 0 to keep AVG sane — Mixpanel
|
|
@@ -1741,6 +1745,13 @@ touch with a seeded weighted pick (models: `firstTouch`, `lastTouch`,
|
|
|
1741
1745
|
`both`). It never stamps unstamped events, so total touch count is
|
|
1742
1746
|
unchanged.
|
|
1743
1747
|
|
|
1748
|
+
The helper selects lifetime endpoints. A report reads eligible touches before
|
|
1749
|
+
each conversion and inside its lookback. The lifetime-last touch can occur after
|
|
1750
|
+
the conversion; the lifetime-first can expire. Neither endpoint is a general
|
|
1751
|
+
conversion-aware guarantee. Pass a deliberately eligible stream to the existing
|
|
1752
|
+
helper or implement that selection in the hook, then verify the requested report.
|
|
1753
|
+
Conflicting sources at an equal timestamp have no stable cross-ingestion order.
|
|
1754
|
+
|
|
1744
1755
|
---
|
|
1745
1756
|
|
|
1746
1757
|
#### 4.27 Active-Day Cohort Engineering (v1.5)
|
package/README.md
CHANGED
|
@@ -18,6 +18,10 @@ i built this because i needed it. and after using it across hundreds of customer
|
|
|
18
18
|
npm install @ak--47/dungeon-master
|
|
19
19
|
```
|
|
20
20
|
|
|
21
|
+
1.8.2 adds live counting verification and fixes identity, session-funnel,
|
|
22
|
+
attribution, and experiment-filter defects. See the
|
|
23
|
+
[upgrade guide](docs/guides/1.8.2-upgrade-guide.md) for compatibility and tested limits.
|
|
24
|
+
|
|
21
25
|
## quick start
|
|
22
26
|
|
|
23
27
|
```javascript
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# 1.8.2 upgrade guide
|
|
2
|
+
|
|
3
|
+
1.8.2 fixes verifier counting and experiment-filter defects found while importing
|
|
4
|
+
synthetic datasets and querying them in Mixpanel. Public exports, signatures,
|
|
5
|
+
options, and defaults remain unchanged. No new runtime dependency is required.
|
|
6
|
+
|
|
7
|
+
After publication:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
npm install @ak--47/dungeon-master@1.8.2
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
This guide describes a release candidate. A branch, PR, or passing test does not
|
|
14
|
+
publish a package.
|
|
15
|
+
|
|
16
|
+
## identity requires emitted evidence
|
|
17
|
+
|
|
18
|
+
Automatic verification links a device only when an emitted event contains both
|
|
19
|
+
`device_id` and `user_id`. A profile device pool alone cannot create a link.
|
|
20
|
+
Ordinary event names work; linking is not restricted to an auth-named event.
|
|
21
|
+
|
|
22
|
+
`buildIdentityMap(profiles)` remains unchanged for explicit caller overrides.
|
|
23
|
+
Pass that map only when it represents trusted identity knowledge. Automatic
|
|
24
|
+
mapping preserves an explicit authenticated user on a conflicting device event.
|
|
25
|
+
Original-merge projects, B2B identity rules, external mapping history, and all
|
|
26
|
+
ingestion validation policies are not fully emulated.
|
|
27
|
+
|
|
28
|
+
Historical anonymous events may join their user after the import receipt returns.
|
|
29
|
+
The live verification run observed that delay and later verified the same fixture
|
|
30
|
+
without sending it again. Check event totals and identity readiness before accepting
|
|
31
|
+
a funnel comparison. No sender ordering change or fixed sleep claims to solve this.
|
|
32
|
+
|
|
33
|
+
## report wrappers use the supported counting code
|
|
34
|
+
|
|
35
|
+
Held-property funnels now support session counting through `funnelFrequency`.
|
|
36
|
+
Matching dungeon defaults no longer inject a millisecond conversion window into
|
|
37
|
+
an implicit one-session report. List-valued held properties expand into per-value
|
|
38
|
+
histories while preserving full-stream session boundaries.
|
|
39
|
+
|
|
40
|
+
Scalar-only held-property keys retain their existing behavior. Mixed list/scalar
|
|
41
|
+
keys use the source-derived string representation. Explicit list-only mode,
|
|
42
|
+
extreme value formatting, and engine cardinality caps remain outside this API.
|
|
43
|
+
|
|
44
|
+
Totals still defaults to `reentry: false`. Set `reentry: true` when comparing
|
|
45
|
+
repeated histories with Mixpanel general totals. Session tests use UTC and the
|
|
46
|
+
default thirty-minute inactivity timeout. Project exclusions and non-UTC/DST
|
|
47
|
+
variants require separate proof.
|
|
48
|
+
|
|
49
|
+
## passing stories need enough independent users
|
|
50
|
+
|
|
51
|
+
`minCohort` no longer sums overlapping period populations or treats arbitrary rows
|
|
52
|
+
as independent users. It uses a conservative lower bound and applies to custom
|
|
53
|
+
callbacks too. Missing denominator evidence caps an otherwise passing result at
|
|
54
|
+
`WEAK`, with an explanation. Assertions without `minCohort` retain their behavior.
|
|
55
|
+
|
|
56
|
+
This can expose previously optimistic story verdicts. Inspect the selected users,
|
|
57
|
+
entrants, converters, or mature retention cohorts. Do not lower the threshold just
|
|
58
|
+
to restore a green verdict.
|
|
59
|
+
|
|
60
|
+
## exposure events retain global filters
|
|
61
|
+
|
|
62
|
+
Synthetic `$experiment_started` events now include declared `superProps`, including
|
|
63
|
+
context-aware values. Experiment name and variant fields remain authoritative.
|
|
64
|
+
Run-isolated experiment queries therefore retain the exposure and outcome rows.
|
|
65
|
+
This can change seeded output when global property generation consumes randomness.
|
|
66
|
+
Same-version deterministic runs still exclude only fresh `insert_id` values.
|
|
67
|
+
|
|
68
|
+
Funnel-frequency keep/drop decisions now use stable event fields instead of random
|
|
69
|
+
insertion IDs. Their outcome can change from 1.8.1 but is reproducible on rerun.
|
|
70
|
+
|
|
71
|
+
## frequency and attribution have distinct contracts
|
|
72
|
+
|
|
73
|
+
Raw event-count histograms, calendar-day activity, and rolling Frequency/Addiction
|
|
74
|
+
reports are separate measurements. `applyFrequencyByFrequency` scales raw target
|
|
75
|
+
event counts. Its existing `binBy: 'distinctDays'` default only selects the cohort
|
|
76
|
+
axis; it does not make clones add new active days. Use explicit report settings.
|
|
77
|
+
|
|
78
|
+
`attributedBy` now ignores null or absent touch values, retains touchless conversions
|
|
79
|
+
as `unknown`, and preserves earlier touches across conversion time buckets.
|
|
80
|
+
The public default still selects the first conversion per user; `perConversion:
|
|
81
|
+
'all'` evaluates every conversion. The API has no finite-lookback argument.
|
|
82
|
+
|
|
83
|
+
`applyAttributedBySource` retains lifetime endpoint selection. Verify that the
|
|
84
|
+
selected endpoint is inside each intended conversion's lookback. Distinct touch
|
|
85
|
+
timestamps are required for reproducible conflicting source order. Same-time
|
|
86
|
+
ties and backend transition compression can differ from raw event-array order.
|
|
87
|
+
|
|
88
|
+
## verified scope and remaining work
|
|
89
|
+
|
|
90
|
+
The repository's `tests/alignment/live/REPORT.md` records source revision, run IDs,
|
|
91
|
+
queries, counts, practical effects, neutral controls, and historical failures.
|
|
92
|
+
The local analytics checkout supplied counting contracts; live Mixpanel queries
|
|
93
|
+
tested their behavior on imported data. The C++ engine was not built locally.
|
|
94
|
+
|
|
95
|
+
Evidence covers selected generated conversion, TTC, retention, volume, weights,
|
|
96
|
+
incidents, raw-count frequency, numeric aggregates, attribution, sessions, and
|
|
97
|
+
Flows. It does not establish every configuration cross-product, literal retention
|
|
98
|
+
probability calibration, arbitrary hook behavior, warehouse deployment, or every
|
|
99
|
+
project setting. Capacity and ambiguous-order limits remain explicit.
|
|
100
|
+
|
|
101
|
+
Repository gates preserve data and deny network access:
|
|
102
|
+
|
|
103
|
+
```sh
|
|
104
|
+
node tests/alignment/run.mjs
|
|
105
|
+
node tests/alignment/run.mjs --sweep --timeout-ms=600000
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Live scripts are opt-in and require project-specific authorization and credentials.
|
|
109
|
+
They are excluded from the npm package. Do not use the default pruning test setup
|
|
110
|
+
while retained live datasets are still needed.
|
|
@@ -520,6 +520,7 @@ function buildFunnelEvents(context, sequence, chosenFunnelProps, bindPropsIndex,
|
|
|
520
520
|
return {
|
|
521
521
|
event: "$experiment_started",
|
|
522
522
|
properties: {
|
|
523
|
+
...config.superProps,
|
|
523
524
|
"Experiment name": experimentName,
|
|
524
525
|
"Variant name": experimentVariant
|
|
525
526
|
}
|
|
@@ -43,9 +43,10 @@ import { getChance } from '../utils/utils.js';
|
|
|
43
43
|
* last touch BEFORE THE CONVERSION EVENT (lookback bounded by the
|
|
44
44
|
* conversion timestamp — get_last_value, whoval/read.cpp:643-655). If the
|
|
45
45
|
* user has stamped touches AFTER their conversion, the touch this pattern
|
|
46
|
-
* biases is not the one the report reads. `firstTouch`
|
|
47
|
-
*
|
|
48
|
-
*
|
|
46
|
+
* biases is not the one the report reads. `firstTouch` also requires the
|
|
47
|
+
* lifetime-first touch to remain inside the report's lookback. Callers must
|
|
48
|
+
* select eligible history when a conversion-bounded report is intended.
|
|
49
|
+
* Equal-time conflicting values have no stable cross-ingestion ordering.
|
|
49
50
|
* @returns {{ overwritten: number, touches: number }} `touches` = stamped
|
|
50
51
|
* events found; `overwritten` = touches whose value was replaced.
|
|
51
52
|
*/
|
|
@@ -66,14 +66,11 @@ export function applyFunnelFrequencyBreakdown(allUserEvents, _profile, funnelEve
|
|
|
66
66
|
}
|
|
67
67
|
if (!stepName) return { bin, droppedFinal: false };
|
|
68
68
|
|
|
69
|
-
// Deterministic drop decision without consuming the seeded RNG stream:
|
|
70
|
-
// hash the funnel's first insert_id (fallback: time) through
|
|
71
|
-
// simpleHashFloat — FNV-1a over the full string, quantized to 1/1000 —
|
|
72
|
-
// and drop when the hash lands under (1 - keepRate). Per-funnel stable,
|
|
73
|
-
// so reruns with the same seed drop the same funnels.
|
|
74
69
|
const dropProb = 1 - keepRate;
|
|
75
|
-
const
|
|
76
|
-
const
|
|
70
|
+
const anchor = funnelEvents[0] || {};
|
|
71
|
+
const seed = JSON.stringify([anchor.user_id || anchor.device_id || anchor.distinct_id || '',
|
|
72
|
+
anchor.event || '', anchor.time || '', stepName]);
|
|
73
|
+
const det = simpleHashFloat(seed);
|
|
77
74
|
if (det < dropProb) {
|
|
78
75
|
const before = funnelEvents.length;
|
|
79
76
|
dropEventsWhere(funnelEvents, e => e && e.event === stepName);
|
package/lib/verify/counting.js
CHANGED
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
|
|
13
13
|
import { toMs } from '../hook-helpers/_internal.js';
|
|
14
14
|
import { matchesWhere } from './coerce.js';
|
|
15
|
-
import {
|
|
15
|
+
import { buildEventIdentityMap, resolveUserId } from './identity.js';
|
|
16
16
|
|
|
17
17
|
const SECONDS_PER_UNIT = {
|
|
18
18
|
hour: 3600,
|
|
@@ -27,8 +27,8 @@ const SECONDS_PER_UNIT = {
|
|
|
27
27
|
* them for what they are:
|
|
28
28
|
*
|
|
29
29
|
* 1. **`'ui-bucket'`** (default): `COUNT(DISTINCT date_trunc(unit, time))`
|
|
30
|
-
* in UTC
|
|
31
|
-
*
|
|
30
|
+
* in UTC. This measures calendar activity and remains the compatibility
|
|
31
|
+
* default; raw-count histograms and rolling frequency use other rules.
|
|
32
32
|
*
|
|
33
33
|
* 2. **`'mixpanel-rolling'`**: the addiction_query.cpp rule,
|
|
34
34
|
* `qtz_time >= interval->last_counted + seconds_for_unit(unit)`
|
|
@@ -39,8 +39,7 @@ const SECONDS_PER_UNIT = {
|
|
|
39
39
|
* calendar days.
|
|
40
40
|
*
|
|
41
41
|
* **The default does NOT match addiction_query.cpp's rolling rule.** It is
|
|
42
|
-
*
|
|
43
|
-
* aligns with `injectOnNewDays` (which classifies days by
|
|
42
|
+
* retained for compatibility with `injectOnNewDays` (which classifies days by
|
|
44
43
|
* `Math.floor(t / DAY_MS)`); mixing the two algorithms makes the atom and
|
|
45
44
|
* verifier disagree at boundaries. For the actual Frequency report output
|
|
46
45
|
* shape (per-interval rolling counters + histogram), use
|
|
@@ -69,8 +68,6 @@ export function countDistinctPeriods(events, eventName, unit = 'day', options =
|
|
|
69
68
|
const unitMs = seconds * 1000;
|
|
70
69
|
|
|
71
70
|
if (algorithm === 'ui-bucket') {
|
|
72
|
-
// Calendar bucket — UTC floor by unit. Matches what Mixpanel's UI
|
|
73
|
-
// shows and what `injectOnNewDays` uses internally.
|
|
74
71
|
const buckets = new Set();
|
|
75
72
|
for (const t of matches) buckets.add(Math.floor(t / unitMs));
|
|
76
73
|
return buckets.size;
|
|
@@ -500,13 +497,13 @@ export function binByDistinctPeriods(events, eventName, bins, unit = 'day', opti
|
|
|
500
497
|
* @param {string} [options.event] - Event name to count. Required at runtime — throws if missing.
|
|
501
498
|
* @param {('hour'|'day'|'week')} [options.unit='day'] - Rolling unit.
|
|
502
499
|
* @param {number} [options.intervalDays] - Report interval length in days (positive integer). Required at runtime.
|
|
503
|
-
* @param {Object[]} [options.profiles] -
|
|
500
|
+
* @param {Object[]} [options.profiles] - Accepted for compatibility; identity uses emitted both-ID events.
|
|
504
501
|
* @returns {Array<{ interval: string, histogram: number[] }>} One row per
|
|
505
502
|
* interval (label = ISO date of the interval start); `histogram[i]` =
|
|
506
503
|
* number of users with rolling count `i + 1` in that interval.
|
|
507
504
|
*/
|
|
508
505
|
export function frequencyHistogram(events, options = {}) {
|
|
509
|
-
const { event, unit = 'day', intervalDays
|
|
506
|
+
const { event, unit = 'day', intervalDays } = options;
|
|
510
507
|
if (typeof event !== 'string' || !event) {
|
|
511
508
|
throw new Error('frequencyHistogram: event is required');
|
|
512
509
|
}
|
|
@@ -517,7 +514,7 @@ export function frequencyHistogram(events, options = {}) {
|
|
|
517
514
|
}
|
|
518
515
|
if (!Array.isArray(events) || !events.length) return [];
|
|
519
516
|
|
|
520
|
-
const identityMap =
|
|
517
|
+
const identityMap = buildEventIdentityMap(events);
|
|
521
518
|
let minMs = Infinity, maxMs = -Infinity;
|
|
522
519
|
const matches = [];
|
|
523
520
|
for (const e of events) {
|
|
@@ -23,7 +23,7 @@
|
|
|
23
23
|
|
|
24
24
|
import { toMs } from '../hook-helpers/_internal.js';
|
|
25
25
|
import { evaluateFunnel, evaluateFunnelHPC, evaluateAnyOrderCompletion } from './funnel-engine.js';
|
|
26
|
-
import {
|
|
26
|
+
import { buildEventIdentityMap, resolveUserId } from './identity.js';
|
|
27
27
|
import { coerceToBreakdownKey, breakdownSegmentKey, matchesWhere } from './coerce.js';
|
|
28
28
|
import { filterFirstTimeEver } from './first-time.js';
|
|
29
29
|
import { sessionize } from './sessionize.js';
|
|
@@ -277,18 +277,7 @@ export function emulateBreakdown(events, config) {
|
|
|
277
277
|
if (!Array.isArray(events)) throw new Error('emulateBreakdown: events must be an array');
|
|
278
278
|
if (!config || !config.type) throw new Error('emulateBreakdown: config.type is required');
|
|
279
279
|
|
|
280
|
-
|
|
281
|
-
// breakdown type AND every time-bucket recursive call so pre-auth (device_id
|
|
282
|
-
// only) events resolve to the same canonical user as post-auth (user_id)
|
|
283
|
-
// events. Hoisted above the timeBucket dispatch to avoid rebuilding the
|
|
284
|
-
// map per-bucket on large datasets.
|
|
285
|
-
const identityMap = config.identityMap
|
|
286
|
-
|| (Array.isArray(config.profiles)
|
|
287
|
-
&& config.profiles.some(p =>
|
|
288
|
-
p && ((Array.isArray(p.device_ids) && p.device_ids.length)
|
|
289
|
-
|| (Array.isArray(p.anonymousIds) && p.anonymousIds.length)))
|
|
290
|
-
? buildIdentityMap(config.profiles)
|
|
291
|
-
: undefined);
|
|
280
|
+
const identityMap = config.identityMap || buildEventIdentityMap(events);
|
|
292
281
|
|
|
293
282
|
// v1.5: time-bucketed wrapper. Partition events by UTC bucket, run the
|
|
294
283
|
// underlying breakdown per partition, tag rows with `period`.
|
|
@@ -318,12 +307,35 @@ export function emulateBreakdown(events, config) {
|
|
|
318
307
|
throw new Error('emulateBreakdown: type "topPaths" does not compose with timeBucket — flows aggregate one path universe over the range');
|
|
319
308
|
}
|
|
320
309
|
const range = config.timeBucketRange || {};
|
|
321
|
-
const
|
|
310
|
+
const bucketEvents = config.type === 'attributedBy'
|
|
311
|
+
? events.filter(event => event && event.event === config.conversionEvent)
|
|
312
|
+
: events;
|
|
313
|
+
const buckets = partitionByTimeBucket(bucketEvents, config.timeBucket, range);
|
|
322
314
|
// Pass the pre-built identityMap into recursive calls so the auto-build
|
|
323
315
|
// branch above is a no-op per bucket (would otherwise rebuild N times).
|
|
324
316
|
const inner = { ...config, timeBucket: undefined, timeBucketRange: undefined, identityMap };
|
|
325
317
|
const out = [];
|
|
326
318
|
|
|
319
|
+
if (config.type === 'attributedBy') {
|
|
320
|
+
attributedBy([], {
|
|
321
|
+
conversionEvent: config.conversionEvent,
|
|
322
|
+
attributionEvent: config.attributionEvent,
|
|
323
|
+
attributionProperty: config.attributionProperty,
|
|
324
|
+
model: config.model,
|
|
325
|
+
perConversion: config.perConversion,
|
|
326
|
+
identityMap,
|
|
327
|
+
});
|
|
328
|
+
for (const { period } of buckets) {
|
|
329
|
+
const rows = attributedBy(events, /** @type {*} */ (inner), bucketBoundsMs(period, config.timeBucket));
|
|
330
|
+
if (rows.length) {
|
|
331
|
+
for (const row of rows) out.push({ period, ...row });
|
|
332
|
+
} else {
|
|
333
|
+
out.push({ period, _empty: true });
|
|
334
|
+
}
|
|
335
|
+
}
|
|
336
|
+
return out;
|
|
337
|
+
}
|
|
338
|
+
|
|
327
339
|
// v1.6.0 (P1.6.5): step-0-anchored trend types. Mixpanel evaluates each
|
|
328
340
|
// trend interval as "step 0 in [start, stop); steps 1+ in
|
|
329
341
|
// [start, stop + conversion window)" (funnel_query.cpp:1398-1401), and
|
|
@@ -483,13 +495,6 @@ function funnelFrequency(events, { steps, breakdownByFrequencyOf, conversionWind
|
|
|
483
495
|
if ((isTotalsMode || holdPropertyConstant) && !isSequentialOrder) {
|
|
484
496
|
throw new Error(`funnelFrequency: countMode '${countMode}' / holdPropertyConstant require a sequential funnel order (got '${funnelOrder}')`);
|
|
485
497
|
}
|
|
486
|
-
// HPC buckets events by property value BEFORE evaluation (evaluateFunnelHPC),
|
|
487
|
-
// but Mixpanel derives session boundaries from the user's FULL event stream —
|
|
488
|
-
// sessionizing each bucket independently would merge across the gaps left by
|
|
489
|
-
// removed events and produce wrong ordinals. Refuse rather than mis-count.
|
|
490
|
-
if (holdPropertyConstant && (countMode === 'sessions' || conversionWindow)) {
|
|
491
|
-
throw new Error('funnelFrequency: holdPropertyConstant cannot combine with session-count conversion windows — session boundaries derive from the full event stream, but HPC evaluates per-property-value event subsets');
|
|
492
|
-
}
|
|
493
498
|
|
|
494
499
|
const userEvents = groupByUser(events, identityMap);
|
|
495
500
|
const result = [];
|
|
@@ -787,29 +792,43 @@ function attributedBy(events, {
|
|
|
787
792
|
model = 'firstTouch',
|
|
788
793
|
perConversion = 'first',
|
|
789
794
|
identityMap,
|
|
790
|
-
}) {
|
|
795
|
+
}, conversionRange = null) {
|
|
791
796
|
if (!conversionEvent || !attributionEvent || !attributionProperty) {
|
|
792
797
|
throw new Error('attributedBy requires conversionEvent, attributionEvent, attributionProperty');
|
|
793
798
|
}
|
|
794
799
|
if (perConversion !== 'first' && perConversion !== 'all') {
|
|
795
800
|
throw new Error(`attributedBy: unknown perConversion "${perConversion}" — use 'first' or 'all'`);
|
|
796
801
|
}
|
|
802
|
+
// Attribution is unbounded over supplied history; this API has no finite
|
|
803
|
+
// lookback. The private bucket interval limits conversions, never touches.
|
|
804
|
+
// Backend transitions also compress case-insensitive equal values within
|
|
805
|
+
// 30 minutes (value/transitions.cpp, whoval/util.cpp). This raw-value model
|
|
806
|
+
// does not emulate that compression, which can retain earlier string casing.
|
|
797
807
|
const userEvents = groupByUser(events, identityMap);
|
|
798
808
|
const counts = new Map();
|
|
799
809
|
for (const [, evs] of userEvents) {
|
|
800
810
|
const sorted = sortByTime(evs);
|
|
811
|
+
const inRange = sorted.filter(event => {
|
|
812
|
+
if (!event || event.event !== conversionEvent) return false;
|
|
813
|
+
const time = toMs(event.time);
|
|
814
|
+
return !conversionRange || (time >= conversionRange.startMs && time < conversionRange.endMs);
|
|
815
|
+
});
|
|
801
816
|
const conversions = perConversion === 'all'
|
|
802
|
-
?
|
|
803
|
-
:
|
|
817
|
+
? inRange
|
|
818
|
+
: inRange.slice(0, 1);
|
|
804
819
|
for (const conversion of conversions) {
|
|
805
820
|
const conversionTime = toMs(conversion.time);
|
|
806
|
-
|
|
807
|
-
|
|
821
|
+
// attribution.py's defined-property filter excludes null and undefined;
|
|
822
|
+
// whoval/read.cpp first/last include the conversion timestamp.
|
|
823
|
+
const allTouches = sorted.filter(event =>
|
|
824
|
+
event && event.event === attributionEvent && event[attributionProperty] != null
|
|
825
|
+
&& toMs(event.time) <= conversionTime
|
|
808
826
|
);
|
|
809
|
-
if (!allTouches.length) continue;
|
|
810
827
|
const touch = model === 'lastTouch' ? allTouches[allTouches.length - 1] : allTouches[0];
|
|
811
|
-
|
|
812
|
-
|
|
828
|
+
// Backend missing attribution is undefined. Preserve this API's existing
|
|
829
|
+
// 'unknown' label, not coerceToBreakdownKey's general 'undefined' label.
|
|
830
|
+
const value = touch?.[attributionProperty] ?? 'unknown';
|
|
831
|
+
counts.set(value, (counts.get(value) || 0) + 1);
|
|
813
832
|
}
|
|
814
833
|
}
|
|
815
834
|
return [...counts.entries()].map(([source, count]) => ({
|
|
@@ -187,7 +187,7 @@
|
|
|
187
187
|
|
|
188
188
|
import { toMs } from '../hook-helpers/_internal.js';
|
|
189
189
|
import { sessionOrdinals } from './sessionize.js';
|
|
190
|
-
import { matchesWhere } from './coerce.js';
|
|
190
|
+
import { coerceToBreakdownKey, matchesWhere } from './coerce.js';
|
|
191
191
|
|
|
192
192
|
const OUT_OF_ORDER_MS = 2000;
|
|
193
193
|
const DAY_MS = 86400 * 1000;
|
|
@@ -1016,13 +1016,21 @@ function evaluateFunnelWithContext(events, steps, options, fullStreamOrdinals) {
|
|
|
1016
1016
|
* group and drop off in another simultaneously). Session windows derive
|
|
1017
1017
|
* ordinals from the full user stream before routing events to HPC buckets.
|
|
1018
1018
|
*
|
|
1019
|
-
*
|
|
1019
|
+
* Analytics 717286d2: backend/arb/reader/queries/funnel_query.cpp:749
|
|
1020
|
+
* processes each key from backend/libquery/aggregate.cpp:36,113. Ordinary
|
|
1021
|
+
* HPC expands lists one level and accepts scalars. Both use value_to_string
|
|
1022
|
+
* (backend/libquery/value/value.c:184), not typed breakdown/filter equality.
|
|
1023
|
+
* Empty lists emit no visits; null/missing items use the "undefined" label.
|
|
1024
|
+
* Duplicate labels share a history but retain cursor visits, which can
|
|
1025
|
+
* advance repeated steps (funnel_query.cpp:1257; funnels/history.cpp:387).
|
|
1020
1026
|
*
|
|
1021
|
-
*
|
|
1022
|
-
*
|
|
1023
|
-
* property
|
|
1024
|
-
*
|
|
1025
|
-
*
|
|
1027
|
+
* Compatibility: all-scalar streams retain raw keys and omit null/missing
|
|
1028
|
+
* values. List-bearing streams use string labels for all routed values.
|
|
1029
|
+
* This property-name API does not expose the separate explicit-list mode,
|
|
1030
|
+
* which rejects non-lists (aggregate.cpp:126). Events and their copy-data
|
|
1031
|
+
* properties remain unchanged; only the routing key is normalized.
|
|
1032
|
+
* The shared label helper does not reproduce ARB's numeric %.16g fallback,
|
|
1033
|
+
* structured-value serialization/truncation, or aggregate cardinality cap.
|
|
1026
1034
|
*
|
|
1027
1035
|
* @param {Array<Object>} events
|
|
1028
1036
|
* @param {FunnelStep[]} steps
|
|
@@ -1044,6 +1052,11 @@ export function evaluateFunnelHPC(events, steps, holdProperty, options = {}) {
|
|
|
1044
1052
|
const fullStreamOrdinals = options.countMode === 'sessions' || options.conversionWindow?.unit === 'sessions'
|
|
1045
1053
|
? sessionOrdinals((events || []).filter(ev => ev && typeof ev.event === 'string'))
|
|
1046
1054
|
: undefined;
|
|
1055
|
+
const hasListValues = (events || []).some(ev => ev && typeof ev.event === 'string' && Array.isArray(ev[holdProperty]));
|
|
1056
|
+
const heldValues = (value) => {
|
|
1057
|
+
if (!hasListValues) return value == null ? [] : [value];
|
|
1058
|
+
return (Array.isArray(value) ? value : [value]).map(coerceToBreakdownKey);
|
|
1059
|
+
};
|
|
1047
1060
|
|
|
1048
1061
|
// Bucket events by HPC value. The step-0 events define the universe of
|
|
1049
1062
|
// HPC values for this user; later events only populate buckets whose
|
|
@@ -1053,18 +1066,18 @@ export function evaluateFunnelHPC(events, steps, holdProperty, options = {}) {
|
|
|
1053
1066
|
if (!ev || typeof ev.event !== 'string') continue;
|
|
1054
1067
|
// Step-0 events seed the bucket on their own value.
|
|
1055
1068
|
if (ev.event === step0Name) {
|
|
1056
|
-
const
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
|
|
1069
|
+
for (const value of heldValues(ev[holdProperty])) {
|
|
1070
|
+
if (!valueBuckets.has(value)) valueBuckets.set(value, []);
|
|
1071
|
+
valueBuckets.get(value).push(ev);
|
|
1072
|
+
}
|
|
1060
1073
|
}
|
|
1061
1074
|
}
|
|
1062
1075
|
// Now route every event with a known HPC value into its bucket.
|
|
1063
1076
|
for (const ev of events || []) {
|
|
1064
1077
|
if (!ev || typeof ev.event !== 'string' || ev.event === step0Name) continue;
|
|
1065
|
-
const
|
|
1066
|
-
|
|
1067
|
-
|
|
1078
|
+
for (const value of heldValues(ev[holdProperty])) {
|
|
1079
|
+
if (valueBuckets.has(value)) valueBuckets.get(value).push(ev);
|
|
1080
|
+
}
|
|
1068
1081
|
}
|
|
1069
1082
|
|
|
1070
1083
|
const out = new Map();
|
package/lib/verify/identity.js
CHANGED
|
@@ -1,15 +1,13 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* Identity resolution for verifier
|
|
3
|
-
*
|
|
4
|
-
* semantics: pre-auth events stamped with `device_id` and post-auth events
|
|
5
|
-
* stamped with `user_id` belong to the same canonical identity.
|
|
2
|
+
* Identity resolution for verifier. Automatic links come from emitted
|
|
3
|
+
* both-ID events; profile pools are only an explicit caller override.
|
|
6
4
|
*
|
|
7
|
-
* Reference:
|
|
8
|
-
*
|
|
9
|
-
* `Map<device_id, canonical_user_id>` so query-time event grouping resolves
|
|
10
|
-
* pre-auth touches alongside post-auth events.
|
|
5
|
+
* Reference: analytics identity-manager v3 lookup_and_update_handler.go
|
|
6
|
+
* and identity/device_and_user_records.go at 717286d2d3ed03e9e3f9cb4346e4c6b2e561fb9a.
|
|
11
7
|
*/
|
|
12
8
|
|
|
9
|
+
const emittedIdentityMap = Symbol('emittedIdentityMap');
|
|
10
|
+
|
|
13
11
|
/**
|
|
14
12
|
* Build a `Map<device_id, canonical_user_id>` by inverting each profile's
|
|
15
13
|
* device-pool array. Reads `device_ids` first, falling back to the legacy
|
|
@@ -41,12 +39,36 @@ export function buildIdentityMap(profiles) {
|
|
|
41
39
|
return map;
|
|
42
40
|
}
|
|
43
41
|
|
|
42
|
+
/**
|
|
43
|
+
* Build automatic device-to-user links from the full emitted stream.
|
|
44
|
+
* Event names do not gate linking. The first valid pair for a device wins;
|
|
45
|
+
* v3 does not reassign an already-linked device to a conflicting user.
|
|
46
|
+
* User IDs with the reserved $device: prefix cannot establish a link.
|
|
47
|
+
*
|
|
48
|
+
* @param {Array<Object>} events
|
|
49
|
+
* @returns {Map<string, string>}
|
|
50
|
+
*/
|
|
51
|
+
export function buildEventIdentityMap(events) {
|
|
52
|
+
const map = new Map();
|
|
53
|
+
Object.defineProperty(map, emittedIdentityMap, { value: true });
|
|
54
|
+
if (!Array.isArray(events)) return map;
|
|
55
|
+
for (const event of events) {
|
|
56
|
+
const deviceId = event?.device_id;
|
|
57
|
+
const userId = event?.user_id;
|
|
58
|
+
if (typeof deviceId !== 'string' || !deviceId.trim()
|
|
59
|
+
|| typeof userId !== 'string' || !userId.trim()
|
|
60
|
+
|| userId.startsWith('$device:') || map.has(deviceId)) continue;
|
|
61
|
+
map.set(deviceId, userId);
|
|
62
|
+
}
|
|
63
|
+
return map;
|
|
64
|
+
}
|
|
65
|
+
|
|
44
66
|
/**
|
|
45
67
|
* Resolve the canonical user id for an event. Lookup order:
|
|
46
68
|
* 1. `event.distinct_id` — Mixpanel's canonical post-merge identifier. When
|
|
47
69
|
* a downstream pipeline has already stitched the cluster, this is the
|
|
48
70
|
* ground truth; never override it.
|
|
49
|
-
* 2. `identityMap.get(event.device_id)` —
|
|
71
|
+
* 2. `identityMap.get(event.device_id)` — emitted link or explicit override.
|
|
50
72
|
* 3. `event.user_id` — already authed (pre-merge analog of distinct_id).
|
|
51
73
|
* 4. `event.device_id` — anonymous fallback.
|
|
52
74
|
*
|
|
@@ -70,6 +92,7 @@ export function buildIdentityMap(profiles) {
|
|
|
70
92
|
export function resolveUserId(event, identityMap) {
|
|
71
93
|
if (!event) return undefined;
|
|
72
94
|
if (event.distinct_id) return event.distinct_id;
|
|
95
|
+
if (event.user_id && identityMap && Object.getOwnPropertyDescriptor(identityMap, emittedIdentityMap)) return event.user_id;
|
|
73
96
|
if (identityMap && event.device_id) {
|
|
74
97
|
const merged = identityMap.get(event.device_id);
|
|
75
98
|
if (merged) return merged;
|
|
@@ -20,9 +20,9 @@
|
|
|
20
20
|
* neutral. For 'between' the wanted side is where the band's midpoint sits
|
|
21
21
|
* relative to the neutral point.
|
|
22
22
|
*
|
|
23
|
-
* Cohort
|
|
24
|
-
*
|
|
25
|
-
*
|
|
23
|
+
* Cohort evidence per selection is an independent-user lower bound: sum only
|
|
24
|
+
* proven disjoint bins within a period, then take the largest period. Other
|
|
25
|
+
* user counts use the largest row. Missing evidence caps passes at WEAK.
|
|
26
26
|
*/
|
|
27
27
|
|
|
28
28
|
import { emulateBreakdown } from './emulate-breakdown.js';
|
|
@@ -166,11 +166,59 @@ function resolveRef(ref, selected) {
|
|
|
166
166
|
return sum;
|
|
167
167
|
}
|
|
168
168
|
|
|
169
|
-
function cohortOf(rows) {
|
|
170
|
-
if (
|
|
171
|
-
|
|
169
|
+
function cohortOf(rows, breakdown, sourceRows) {
|
|
170
|
+
if (!Array.isArray(rows)) return null;
|
|
171
|
+
let evidenceRows = (rows || []).filter(row => row && !row._empty);
|
|
172
|
+
if (!evidenceRows.length) return null;
|
|
173
|
+
const isCount = value => Number.isSafeInteger(value) && value >= 0;
|
|
174
|
+
const hasPeriod = row => typeof row.period === 'string' && row.period.length > 0;
|
|
175
|
+
let column = 'user_count';
|
|
176
|
+
let binColumns = [];
|
|
177
|
+
if (breakdown.type === 'funnelFrequency') {
|
|
178
|
+
if (breakdown.countMode !== undefined && breakdown.countMode !== 'uniques') return null;
|
|
179
|
+
const periodRequired = breakdown.timeBucket !== undefined || evidenceRows.some(row => row.period !== undefined);
|
|
180
|
+
if (evidenceRows.some(row => !isCount(row.step_index) || !isCount(row.breakdown_freq)
|
|
181
|
+
|| (periodRequired && !hasPeriod(row)))) return null;
|
|
182
|
+
const binKey = row => JSON.stringify([row.period, row.breakdown_freq]);
|
|
183
|
+
const selectedBins = new Set(evidenceRows.map(binKey));
|
|
184
|
+
evidenceRows = (sourceRows || []).filter(row => row && !row._empty && row.step_index === 0
|
|
185
|
+
&& isCount(row.breakdown_freq) && (row.period === undefined || hasPeriod(row))
|
|
186
|
+
&& selectedBins.has(binKey(row)));
|
|
187
|
+
if (new Set(evidenceRows.map(binKey)).size !== selectedBins.size) return null;
|
|
188
|
+
column = 'conversions';
|
|
189
|
+
binColumns = ['breakdown_freq'];
|
|
190
|
+
} else if (breakdown.type === 'eventBreakdown') {
|
|
191
|
+
column = 'total_users';
|
|
192
|
+
} else if (breakdown.type === 'uniques') {
|
|
193
|
+
if (breakdown.countType !== undefined && breakdown.countType !== 'unique') return null;
|
|
194
|
+
column = 'uniques';
|
|
195
|
+
} else if (breakdown.type === 'retention') {
|
|
196
|
+
column = 'cohort_size';
|
|
197
|
+
} else if (breakdown.type === 'frequencyByFrequency') {
|
|
198
|
+
binColumns = ['metric_freq', 'breakdown_freq'];
|
|
199
|
+
} else if (breakdown.type === 'aggregatePerUser') {
|
|
200
|
+
binColumns = ['breakdown_freq'];
|
|
172
201
|
}
|
|
173
|
-
return
|
|
202
|
+
if (evidenceRows.some(row => !isCount(row[column]))) return null;
|
|
203
|
+
const largestRow = evidenceRows.reduce((largest, row) => Math.max(largest, row[column]), 0);
|
|
204
|
+
const periodRequired = breakdown.timeBucket !== undefined || evidenceRows.some(row => row.period !== undefined);
|
|
205
|
+
if (!binColumns.length || evidenceRows.some(row => binColumns.some(bin => !isCount(row[bin]))
|
|
206
|
+
|| (periodRequired && !hasPeriod(row)))) return largestRow;
|
|
207
|
+
const periods = new Map();
|
|
208
|
+
for (const row of evidenceRows) {
|
|
209
|
+
if (!periods.has(row.period)) periods.set(row.period, new Map());
|
|
210
|
+
const bins = periods.get(row.period);
|
|
211
|
+
const key = JSON.stringify(binColumns.map(bin => row[bin]));
|
|
212
|
+
bins.set(key, Math.max(bins.get(key) || 0, row[column]));
|
|
213
|
+
}
|
|
214
|
+
let largestPeriod = 0;
|
|
215
|
+
for (const bins of periods.values()) {
|
|
216
|
+
let total = 0;
|
|
217
|
+
for (const count of bins.values()) total += count;
|
|
218
|
+
if (!Number.isSafeInteger(total)) return largestRow;
|
|
219
|
+
largestPeriod = Math.max(largestPeriod, total);
|
|
220
|
+
}
|
|
221
|
+
return largestPeriod;
|
|
174
222
|
}
|
|
175
223
|
|
|
176
224
|
// ── verdicts ────────────────────────────────────────────────────────────────
|
|
@@ -243,40 +291,55 @@ const fmt = (n) => (typeof n === 'number' && Number.isFinite(n)) ? (Math.abs(n)
|
|
|
243
291
|
*/
|
|
244
292
|
export function evaluateAssertion(rows, assertion, ctx) {
|
|
245
293
|
try {
|
|
294
|
+
const minCohort = assertion.minCohort;
|
|
295
|
+
let result, selected, cohorts;
|
|
246
296
|
if (typeof assertion.assert === 'function') {
|
|
297
|
+
if (typeof minCohort === 'number') {
|
|
298
|
+
selected = assertion.select === undefined ? { all: rows } : selectRows(rows, assertion.select);
|
|
299
|
+
cohorts = Object.values(selected).map(selection => cohortOf(selection, assertion.breakdown, rows));
|
|
300
|
+
}
|
|
247
301
|
/** @type {{ pass?: boolean, detail?: string, verdict?: import('../../types').StoryVerdict }} */
|
|
248
302
|
const res = assertion.assert(rows, ctx) || {};
|
|
249
303
|
if (res.verdict && VERDICT_RANK[res.verdict] !== undefined) {
|
|
250
|
-
|
|
304
|
+
result = { verdict: res.verdict, observed: null, detail: res.detail || 'custom assert' };
|
|
305
|
+
} else {
|
|
306
|
+
result = res.pass
|
|
307
|
+
? { verdict: 'STRONG', observed: null, detail: res.detail || 'custom assert passed' }
|
|
308
|
+
: { verdict: 'NONE', observed: null, detail: res.detail || 'custom assert failed' };
|
|
251
309
|
}
|
|
252
|
-
return res.pass
|
|
253
|
-
? { verdict: 'STRONG', observed: null, detail: res.detail || 'custom assert passed' }
|
|
254
|
-
: { verdict: 'NONE', observed: null, detail: res.detail || 'custom assert failed' };
|
|
255
|
-
}
|
|
256
|
-
const parsed = parseMetric(assertion.expect.metric);
|
|
257
|
-
const selected = selectRows(rows, assertion.select);
|
|
258
|
-
const left = resolveRef(parsed.left, selected);
|
|
259
|
-
let observed, neutral;
|
|
260
|
-
if (parsed.kind === 'single') {
|
|
261
|
-
observed = left; neutral = null;
|
|
262
310
|
} else {
|
|
263
|
-
const
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
311
|
+
const parsed = parseMetric(assertion.expect.metric);
|
|
312
|
+
selected = selectRows(rows, assertion.select);
|
|
313
|
+
const left = resolveRef(parsed.left, selected);
|
|
314
|
+
let observed, neutral;
|
|
315
|
+
if (parsed.kind === 'single') {
|
|
316
|
+
observed = left; neutral = null;
|
|
267
317
|
} else {
|
|
268
|
-
|
|
318
|
+
const right = resolveRef(parsed.right, selected);
|
|
319
|
+
if (parsed.kind === 'ratio') {
|
|
320
|
+
if (right === 0) throw new Error('ratio denominator is 0');
|
|
321
|
+
observed = left / right; neutral = 1;
|
|
322
|
+
} else {
|
|
323
|
+
observed = left - right; neutral = 0;
|
|
324
|
+
}
|
|
269
325
|
}
|
|
326
|
+
const { verdict, detail } = verdictFor(observed, assertion.expect, neutral);
|
|
327
|
+
result = { verdict, observed, detail };
|
|
270
328
|
}
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
329
|
+
if (typeof minCohort === 'number' && VERDICT_RANK[result.verdict] > VERDICT_RANK.WEAK) {
|
|
330
|
+
cohorts ??= Object.values(selected).map(selection => cohortOf(selection, assertion.breakdown, rows));
|
|
331
|
+
if (!cohorts.length || cohorts.some(cohort => cohort === null)) {
|
|
332
|
+
result.verdict = 'WEAK';
|
|
333
|
+
result.detail += ` - capped: insufficient evidence for minCohort ${minCohort}; independent-user denominator unavailable`;
|
|
334
|
+
} else {
|
|
335
|
+
const smallest = Math.min(...cohorts);
|
|
336
|
+
if (smallest < minCohort) {
|
|
337
|
+
result.verdict = 'WEAK';
|
|
338
|
+
result.detail += ` - capped: insufficient evidence; smallest independent-user lower bound ${smallest} < minCohort ${minCohort}`;
|
|
339
|
+
}
|
|
277
340
|
}
|
|
278
341
|
}
|
|
279
|
-
return
|
|
342
|
+
return result;
|
|
280
343
|
} catch (err) {
|
|
281
344
|
return { verdict: 'NONE', observed: null, detail: `error: ${err.message}` };
|
|
282
345
|
}
|
|
@@ -21,6 +21,7 @@
|
|
|
21
21
|
|
|
22
22
|
import DUNGEON_MASTER from '../../index.js';
|
|
23
23
|
import { emulateBreakdown } from './emulate-breakdown.js';
|
|
24
|
+
import { buildEventIdentityMap } from './identity.js';
|
|
24
25
|
import { validateSchema } from './schema-validator.js';
|
|
25
26
|
|
|
26
27
|
/**
|
|
@@ -83,7 +84,7 @@ export function applyFunnelDefaults(breakdownArgs, funnels, profiles) {
|
|
|
83
84
|
// is the author choosing a window — injecting conversionWindowMs on top
|
|
84
85
|
// would trip evaluateFunnel's mutual-exclusion throw. Only inject when
|
|
85
86
|
// NEITHER window form is present.
|
|
86
|
-
if (args.conversionWindowMs === undefined && args.conversionWindow === undefined && Number.isFinite(matched.conversionWindowDays)) {
|
|
87
|
+
if (args.countMode !== 'sessions' && args.conversionWindowMs === undefined && args.conversionWindow === undefined && Number.isFinite(matched.conversionWindowDays)) {
|
|
87
88
|
args.conversionWindowMs = matched.conversionWindowDays * 86400000;
|
|
88
89
|
}
|
|
89
90
|
if (args.funnelOrder === undefined && matched.order) {
|
|
@@ -137,6 +138,7 @@ export async function verifyDungeon(config, checks, overrides) {
|
|
|
137
138
|
}
|
|
138
139
|
const events = Array.isArray(result.eventData) ? result.eventData : Array.from(result.eventData);
|
|
139
140
|
const profiles = Array.isArray(result.userProfilesData) ? result.userProfilesData : Array.from(result.userProfilesData);
|
|
141
|
+
const identityMap = buildEventIdentityMap(events);
|
|
140
142
|
// v1.6.2: schema-check against the config the run actually used. `config` may be a
|
|
141
143
|
// PATH STRING (no fields at all → every property reads as unexpected), and even for
|
|
142
144
|
// an object input a v1.5.1 dungeon keeps `hasAndroidDevices` / `hasBrowser` under
|
|
@@ -155,6 +157,7 @@ export async function verifyDungeon(config, checks, overrides) {
|
|
|
155
157
|
for (const check of checks) {
|
|
156
158
|
try {
|
|
157
159
|
const breakdownArgs = applyFunnelDefaults(check.breakdown, validatedFunnels, profiles);
|
|
160
|
+
if (breakdownArgs.identityMap === undefined) breakdownArgs.identityMap = identityMap;
|
|
158
161
|
const rows = emulateBreakdown(events, breakdownArgs);
|
|
159
162
|
const verdict = check.assert(rows, ctx);
|
|
160
163
|
results.push({ name: check.name, pass: !!verdict.pass, detail: verdict.detail, rows });
|
package/package.json
CHANGED
|
@@ -38,8 +38,8 @@ import { parse as parseCsv } from 'csv-parse';
|
|
|
38
38
|
import generate from '../index.js';
|
|
39
39
|
import { extractComments } from '../lib/core/extract-comments.js';
|
|
40
40
|
import { validateDungeonConfig } from '../lib/core/config-validator.js';
|
|
41
|
+
import { buildEventIdentityMap } from '../lib/verify/identity.js';
|
|
41
42
|
import {
|
|
42
|
-
buildIdentityMap,
|
|
43
43
|
VERDICT_RANK,
|
|
44
44
|
validateStories,
|
|
45
45
|
validateSchema,
|
|
@@ -149,7 +149,7 @@ if (inMemory) {
|
|
|
149
149
|
? await evaluateStories(stories, events, {
|
|
150
150
|
profiles,
|
|
151
151
|
funnels: Array.isArray(validated.funnels) ? validated.funnels : [],
|
|
152
|
-
identityMap:
|
|
152
|
+
identityMap: buildEventIdentityMap(events),
|
|
153
153
|
warehouseRows,
|
|
154
154
|
warehouseSpecs,
|
|
155
155
|
datasetStart: validated.datasetStart,
|
|
@@ -187,7 +187,7 @@ if (inMemory) {
|
|
|
187
187
|
// here and use the RETURN value — as of v1.6.2 validateDungeonConfig does not
|
|
188
188
|
// enrich the object you hand it.
|
|
189
189
|
const validated = validateDungeonConfig({ ...config, token: '' });
|
|
190
|
-
const identityMap =
|
|
190
|
+
const identityMap = buildEventIdentityMap(events);
|
|
191
191
|
schemaPass = !!validateSchema(events, validated)?.pass;
|
|
192
192
|
const warehouseSpecs = Object.fromEntries((validated.warehouseMetrics || []).map((spec) => [spec.name, spec]));
|
|
193
193
|
const warehouseManifest = loadWarehouseManifest(prefixPath);
|