@ak--47/dungeon-master 1.6.5 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/.claude/skills/analyze-soup/SKILL.md +21 -11
  2. package/.claude/skills/create-dungeon/SKILL.md +49 -10
  3. package/.claude/skills/create-project/SKILL.md +22 -3
  4. package/.claude/skills/create-project/context.mjs +89 -0
  5. package/.claude/skills/create-project/provision.mjs +1 -60
  6. package/.claude/skills/headless-build/SKILL.md +18 -1
  7. package/.claude/skills/powertools/SKILL.md +20 -1
  8. package/.claude/skills/release-check/SKILL.md +99 -0
  9. package/.claude/skills/verify-dungeon/SKILL.md +71 -16
  10. package/.claude/skills/verify-dungeon/references/counting-semantics.md +14 -0
  11. package/.claude/skills/verify-dungeon/references/report-format.md +18 -1
  12. package/.claude/skills/verify-dungeon/references/sql-recipes.md +36 -1
  13. package/.claude/skills/warehouse-metrics/GAPS-template.md +34 -0
  14. package/.claude/skills/warehouse-metrics/SKILL.md +105 -0
  15. package/.claude/skills/warehouse-metrics/deploy.mjs +651 -0
  16. package/.claude/skills/write-hooks/SKILL.md +33 -3
  17. package/CHANGELOG.md +331 -0
  18. package/HOOKS.md +154 -5
  19. package/README.md +357 -8
  20. package/docs/guides/1.7.0-upgrade-guide.md +154 -0
  21. package/docs/guides/1.8.0-upgrade-guide.md +151 -0
  22. package/dungeons/technical/warehouse.js +187 -0
  23. package/index.js +131 -2
  24. package/lib/core/config-validator.js +264 -13
  25. package/lib/core/context.js +39 -0
  26. package/lib/core/dungeon-loader.js +5 -2
  27. package/lib/core/storage.js +51 -3
  28. package/lib/generators/events.js +53 -7
  29. package/lib/generators/funnels.js +85 -11
  30. package/lib/generators/profiles.js +9 -4
  31. package/lib/generators/standalone.js +248 -0
  32. package/lib/generators/warehouse.js +828 -0
  33. package/lib/orchestrators/mixpanel-sender.js +39 -3
  34. package/lib/orchestrators/user-loop.js +240 -9
  35. package/lib/templates/story-spec.schema.json +41 -16
  36. package/lib/utils/conditions.js +62 -0
  37. package/lib/utils/json-evaluator.js +12 -2
  38. package/lib/utils/utils.js +115 -19
  39. package/lib/verify/index.js +1 -0
  40. package/lib/verify/schema-validator.js +8 -0
  41. package/lib/verify/story-runner.js +71 -8
  42. package/lib/verify/warehouse.js +683 -0
  43. package/package.json +5 -11
  44. package/scripts/verify-stories.mjs +150 -44
  45. package/types.d.ts +606 -38
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: write-hooks
3
3
  description: Use when an existing dungeon needs engineered story trends or "magic number" patterns — writes the `hook` function using atom helpers and high-level patterns. Adds no new event flags; never mutates the schema.
4
- argument-hint: [path/to/dungeon.js] [free-text story / trend description]
4
+ argument-hint: '[path/to/dungeon.js] [free-text story / trend description]'
5
5
  model: claude-opus-4-6
6
6
  effort: max
7
7
  ---
@@ -68,7 +68,7 @@ property values that differ by cohort, injected bursts, lifecycle waves.
68
68
  Hooks fire in this order per user (see `CLAUDE.md` for the canonical reference):
69
69
 
70
70
  1. `"user"` — profile created. Mutate in place; return ignored.
71
- 2. `"scd-pre"` — SCD entries created. Mutate in place OR return new array.
71
+ 2. `"scd-pre"` — SCD entries created. Mutate in place; return ignored.
72
72
  3. For each funnel: `"funnel-pre"` → `"event"` (per step) → `"funnel-post"`.
73
73
 
74
74
  **`funnel-pre` is now reliable for temporal patterns.** Usage funnels advance a
@@ -76,7 +76,7 @@ cursor after each run, so successive `meta.firstEventTime` values spread across
76
76
  the user's active window. Persona and world-event modifiers apply BEFORE the
77
77
  hook — the hook has final authority on `conversionRate`, `timeToConvert`, and
78
78
  `props`.
79
- 4. `"event"` — for non-funnel standalone events. Return value REPLACES the event.
79
+ 4. `"event"` — for non-funnel user events from `events[]`. Return value REPLACES the event.
80
80
  5. `"everything"` — array of ALL the user's events. Return array to replace.
81
81
 
82
82
  **Most engineered trends belong in `everything`.** It sees the full user stream,
@@ -86,6 +86,36 @@ and you can mutate freely.
86
86
  Storage-only hooks (`ad-spend`, `group`, `mirror`, `lookup`) fire later in the
87
87
  pipeline and don't see the same `meta` shape.
88
88
 
89
+ ### Cadence streams and warehouse rows (v1.8.0)
90
+
91
+ These hooks sit outside the per-user sequence and never enter `everything`.
92
+ Neither has person metadata (`meta.profile`, auth state, sessions, or SCDs).
93
+
94
+ - `standaloneEvents`: `type === 'standalone'` fires on storage push before the
95
+ user loop. Read `meta.spec` and `meta.config` to identify the stream. Return
96
+ the record object or an array of records. Returning `undefined` drops it.
97
+ Mutating without returning is therefore insufficient. Preserve required keys
98
+ and use fresh `insert_id` values for clones. Its synthetic `distinct_id` is a
99
+ series identifier, never a person or a retention cohort.
100
+ - `warehouseMetrics`: `type === 'warehouse'` fires after user generation, once
101
+ per materialized row. Mutate the row in place; its return value is ignored.
102
+ Keep the time column and group keys stable. Modify only the value column and
103
+ declared extra columns. Meta includes `spec`, `config`, `metricName`,
104
+ `bucketIndex`, `bucketCount`, `grain`, `seriesKey`, `isBackfill`, and `raw`.
105
+ `raw` describes plus/minus source aggregates before scaling, noise, and carry.
106
+
107
+ Do not add either schema here. Send missing `standaloneEvents` properties or
108
+ warehouse `columns` back to `/create-dungeon`. Warehouse sources consume user
109
+ `events[]` only, including both plus and minus legs; they cannot consume cadence
110
+ streams. Standalone value functions use tick context, warehouse column functions
111
+ use bucket context; neither supplies a user profile.
112
+
113
+ Verify standalone stories with disk-backed `duckdb` assertions against
114
+ `{{PREFIX}}-STANDALONE*.json`. The user-event emulator and `--in-memory` CLI mode
115
+ do not evaluate this stream. Warehouse stories can use `warehouse` assertions
116
+ or `warehouse-stats` assertions; automatic warehouse audits also run without
117
+ stories. Hand off to `/verify-dungeon` with an explicit artifact prefix.
118
+
89
119
  ## Hook meta — identity context
90
120
 
91
121
  Inside `funnel-pre` and `funnel-post`:
package/CHANGELOG.md CHANGED
@@ -2,6 +2,337 @@
2
2
 
3
3
  All notable changes to `@ak--47/dungeon-master`.
4
4
 
5
+ ## 1.8.0 — 2026-09-10
6
+
7
+ ### Added — `standaloneEvents`: identity-less metric snapshots
8
+
9
+ A new top-level config key that generates records describing a **system, not a
10
+ person**. They carry no `user_id` and no `device_id`. Before 1.8.0 the only
11
+ identity-less stream the engine could produce was `$ad_spend` via `hasAdSpend`,
12
+ which is hard-coded to one shape, one cadence, and a Mixpanel reserved event
13
+ name. `standaloneEvents` is the general form.
14
+
15
+ ```js
16
+ standaloneEvents: [{
17
+ event: 'cdn_egress',
18
+ cadence: 'day', // 'hour' | 'day' | 'week', default 'day'
19
+ dimensions: { region: ['us-east', 'us-west', 'eu'] }, // cross-producted
20
+ distinctIdFrom: 'region', // synthetic id, never a person
21
+ properties: {
22
+ gb_out: (ctx) => 400 + ctx.tickIndex * 3,
23
+ cost_usd: (ctx) => (400 + ctx.tickIndex * 3) * 0.085,
24
+ p95_ms: [120, 140, 160],
25
+ },
26
+ }]
27
+ ```
28
+
29
+ - One record per cadence tick per dimension cross-product row.
30
+ - Ticks start at the dataset start and step by the cadence. The last tick is the
31
+ final one at or before the dataset end, so nothing lands in the future.
32
+ - Each record carries `event`, `time`, `insert_id`, `distinct_id`, every
33
+ dimension as a flat property, and every resolved entry in `properties`.
34
+ - `distinct_id` is the value of the dimension named by `distinctIdFrom`, else the
35
+ event name. It exists so Mixpanel accepts the record; it never maps to a person.
36
+ - Property value functions receive a `StandaloneValueContext`:
37
+ `{ time, config, dimensions, tickIndex, tickCount, cadence, event }`.
38
+ `tickIndex / (tickCount - 1)` is window progress — use it to shape a trend.
39
+ - New hook type `standalone` (storage-only). Return the record or an array of
40
+ records; returning nothing drops the record. `meta.spec` carries the resolved
41
+ stream config. The `warehouse` hook instead mutates its row and ignores returns.
42
+ - Lands in `result.standaloneEventData`, writes to a `-STANDALONE` file shard,
43
+ and imports to Mixpanel as its own event stream.
44
+ - Validation **throws** on a malformed entry rather than skipping it. A silent
45
+ skip would drop a whole data stream without the author noticing.
46
+
47
+ New types: `StandaloneEventConfig`, `ResolvedStandaloneEventConfig`,
48
+ `StandaloneValueContext`, `HookMetaStandalone`. `WritePaths` gains
49
+ `standaloneFiles`; `Result` gains `standaloneEventData`; `hookTypes` gains
50
+ `"standalone"`.
51
+
52
+ **Output compatibility.** Additive only. A config without `standaloneEvents` is
53
+ byte-identical to 1.7.0 — the generation pass is gated on the key being present,
54
+ so the seeded RNG stream is untouched. `config.standaloneEvents` normalizes to
55
+ `[]` when absent. Event determinism comparisons exclude the fresh `insert_id`.
56
+
57
+ New tests: `tests/unit/standalone-events.test.js` (25),
58
+ `tests/integration/standalone-events.test.js` (15).
59
+
60
+ ### Added — `warehouseMetrics`: manifest-driven warehouse source tables
61
+
62
+ A new top-level config key that materializes warehouse-ready tables from the
63
+ run's own events after generation completes. This is the local source-table side
64
+ of a warehouse metric demo: bookings rollups, active subscription levels, ARR
65
+ snapshots, and other time-series tables that should read like a real warehouse.
66
+
67
+ ```js
68
+ warehouseMetrics: [{
69
+ name: 'daily_new_bookings',
70
+ source: { event: 'new_booking', measure: 'sum', property: 'booking_value' },
71
+ valueColumn: 'bookings',
72
+ }]
73
+ ```
74
+
75
+ - Supports additive and point-in-time metrics.
76
+ - Grain: `day`, `week`, `month`.
77
+ - Supports subtractive `minus` legs, `groupBy` on up to two declared keys,
78
+ optional `history` backfill, sparse point-in-time emission, seeded `noise`,
79
+ `scale`, and derived `columns`.
80
+ - Lands in `result.warehouseMetricData` keyed by metric name and emits
81
+ `result.warehouseManifest` with table schemas, SQL, and recommended
82
+ aggregation.
83
+ - Writes `<name>-WAREHOUSE-<table>.csv|json` plus
84
+ `<name>-WAREHOUSE-MANIFEST.json` when `writeToDisk` is enabled.
85
+ - Never imports through `token`. Warehouse deploy is a separate flow.
86
+ - New hook type `warehouse` fires once per materialized row with
87
+ `metricName`, `bucketIndex`, `bucketCount`, `grain`, `seriesKey`,
88
+ `isBackfill`, and `raw` bucket stats.
89
+ - Warehouse verification adds `warehouse` / `warehouse-stats` story breakdowns
90
+ plus automatic audits over declared columns, gaps, monotonic time, empty numeric
91
+ cells, and sparse first-bucket coverage. Source correlation is available to story
92
+ assertions; it is not an automatic pass/fail gate.
93
+ - Sparse point-in-time comparison carries emitted levels across missing buckets,
94
+ excludes history from correlation, and compares the first live delta to the scaled
95
+ baseline. Warehouse generation preserves the user event stream, including with noise.
96
+ - Disk verification parses quoted multiline CSV records with `csv-parse` and keeps
97
+ disk and in-memory audit results consistent.
98
+
99
+ ### Added — `/warehouse-metrics`: BigQuery load + warehouse metric save flow
100
+
101
+ The shipped skill at `.claude/skills/warehouse-metrics/` loads the generated
102
+ warehouse tables into BigQuery, connects that dataset to Mixpanel with the
103
+ existing powertools macro, previews each metric SQL, and saves new metrics when
104
+ the CRUD endpoints are available.
105
+
106
+ - Uses the emitted warehouse manifest as the contract.
107
+ - Maps manifest `recommendedAggregation: 'last value'` to the API's
108
+ `aggregation: 'last_value'`.
109
+ - If `GET /crud/getWarehouseMetrics` returns 404, the script still completes the
110
+ BigQuery load and source setup, then writes `warehouse/GAPS.md` for manual
111
+ metric creation.
112
+ - Preview fails fast on the raw substring block (`CREATE`, `UPDATE`, etc.), so
113
+ identifiers like `created_at` and `updated_at` are a real deploy-time trap.
114
+ - The canonical skill name is `/warehouse-metrics`. Bundled commands and handoffs
115
+ use `.claude/skills/warehouse-metrics/`.
116
+ - Uses the shipped Powertools warehouse CRUD and IAM setup macro. Runtime IAM is
117
+ configured; permission failures still stop deployment with the original error.
118
+ - Dry-run works before project provisioning, using placeholders without credentials.
119
+ Live table replacement requires explicit operator consent; the script does not prompt.
120
+
121
+ ### Changed — skill workflow and provisioning context
122
+
123
+ - All nine bundled skills have parsed string argument hints and matching folder names.
124
+ Release tests check frontmatter and warehouse handoffs.
125
+ - Authoring, hooks, and verification distinguish person events, identity-less cadence
126
+ events, and identity-free warehouse rows. Synthetic IDs never count as people.
127
+ - Verification preserves the explicit run prefix and local uncompressed warehouse
128
+ artifacts for deployment. Soup analysis remains scoped to user-event timestamps.
129
+ - Project business context includes separate cadence and warehouse summaries without
130
+ evaluating property functions or including credentials.
131
+ - Headless builds preserve warehouse history and use query preview for fresh results;
132
+ warehouse refresh only invalidates the saved metric cache.
133
+
134
+ ### Added - `/release-check`
135
+
136
+ The new `/release-check` skill audits tests, determinism, documentation, package
137
+ contents, and release handoffs. `.agents/skills` and `.github/skills` link to the
138
+ canonical `.claude/skills` directory for shared agent discovery. Publishing remains
139
+ an explicitly authorized operator action.
140
+
141
+ ### Changed — `streamCSV` preserves falsy cells
142
+
143
+ CSV serialization now writes `0` and `false` as literal cell values instead of
144
+ empty strings. If downstream warehouse SQL or fixtures were treating blank cells
145
+ as zero or false, update them to read the actual value.
146
+
147
+ ## 1.7.0 — 2026-09-03
148
+
149
+ The engine round for DM4 v5. Executes the 1.6.4 "Deferred to 1.7.0" table plus
150
+ the five round-two items (`dm-engine-round-two.md`) and the three round-one items
151
+ that table missed (P0-3, P1-5, P1-6). Every behavior change below carries the
152
+ number that proved the gap and the number after the fix, both from real
153
+ generation passes (400 users × 60 days unless stated; scripts in the session
154
+ scratchpad, assertions pinned in `tests/integration/v170-engine-requests.test.js`).
155
+
156
+ **Output compatibility.** Same seed, same config, `concurrency: 1`, pinned window:
157
+ 1.7.0 and 1.6.5 produce byte-identical **events** (modulo `insert_id`) on every
158
+ technical fixture that does not set `hasLocation` — `simplest`, `datagen-v15-verify`,
159
+ `experiments`, `group-analytics`, `mirror-strategies`, `ad-spend`, `anonymous-users`
160
+ (the funnel-step time pin below still runs TimeSoup, so the RNG stream is
161
+ unchanged). Profiles and groups are identical where the fixture is deterministic
162
+ (`datagen-v15-verify`, `ad-spend`, `anonymous-users`); `simplest`, `experiments`,
163
+ `group-analytics` and `scd` build some profile/group props from their own unseeded
164
+ `new Chance()` and were never run-to-run stable. Three changes alter output on
165
+ purpose, each gated on a feature you would know you are using — see **Behavior
166
+ changes** (B1–B3). The 10-test engine-shape canary passes; `smoke-test-all` runs
167
+ 22/22 shipped dungeons clean. The full 194-combo strict-bar sweep
168
+ (`RUN_FULL_SWEEP=1`, 2026-09-04, window pinned to Wednesday 2026-09-02) passes
169
+ 191/194. The 3 failures are one config three times — `growth/365d/r1.2` with
170
+ born `-`/`30`/`100`, which the growth cap resolves to the same 30 — failing the
171
+ last-day bar at ratio 0.66 vs 0.70. Running the same `long` tier on 1.6.5 (`main`)
172
+ produces the identical 3 failures with identical numbers, so this is a
173
+ pre-existing, calendar-window-dependent marginal dip on one 365-day config, not a
174
+ 1.7.0 regression. Tracked as a follow-up; not a release blocker.
175
+
176
+ **For DM4: tripwires that now flip.** `tests/integration/v5-engine.test.js` pins
177
+ several of the old behaviors; when these assertions fail on 1.7.0 that is the fix
178
+ landing, and the workaround it guards can go:
179
+ - the spike hook for The Moment (`volumeMultiplier` amplifies — P0-3)
180
+ - the flat-only born-share restriction (`macro: { bornRecentBias, percentUsersBornInDataset }` with no `preset` is uncapped — R2-1)
181
+ - the 19-country `SINGLE_COUNTRY_NAMES` enum in `tests/v5-render.test.js` (`singleCountry: 'US'` works; a miss throws — R2-2)
182
+ - the `importResults.users.success` overwrite (the receipt reconciles — R2-5)
183
+ - the repeated-value weighting idiom (`{ __weights }` — P2-1)
184
+ - the "fewer than two non-funnel events" refusal on Persona Difference — its premise was wrong; see P1-5 below
185
+
186
+ ### Tier 1 — silent lies fixed
187
+
188
+ - **R2-1 `MacroConfig` object overrides.** Measured before: `macro: { preset: 'flat',
189
+ percentUsersBornInDataset: 50 }` → 10.3% born; `macro: { percentUsersBornInDataset: 50 }`
190
+ (no preset) → 10.0%; only the top-level key with no `macro` at all gave 52.3%.
191
+ Cause: the born% cap keyed the preset-less object to `flat` (12) and the warning was
192
+ `verbose`-gated. Now: a NAMED preset (string or `{ preset }`) is a shape contract and
193
+ still clamps — 12.3% after, with the clamp in `result.warnings`
194
+ (`{ key: 'percentUsersBornInDataset', requested: 50, applied: 12 }`); an object
195
+ WITHOUT `preset` is a custom macro and is honored as written — 55.0% after for
196
+ `{ percentUsersBornInDataset: 50 }`, 48.8% for `{ bornRecentBias: 0.3, percentUsersBornInDataset: 50 }`.
197
+ Canonical spelling: `macro: { preset, ...overrides }`; the top-level keys are a legacy
198
+ alias that wins over the object. Documented in README, `MacroConfig` JSDoc, CLAUDE.md.
199
+ - **R2-2 `singleCountry`.** Accepts the ISO code or the full name, case-insensitive
200
+ (`resolveSingleCountry`, also hoisted from `switches`). A value matching no country
201
+ THROWS with the list of valid values. Measured before: `'US'` and `'Narnia'` both
202
+ silently deleted every geo property from events and profiles; after: `'US'` → 100%
203
+ `country_code: US` on events and profiles.
204
+ - **R2-3 `strictEventCount` is exact.** Before: stopped on the GENERATED count (drops
205
+ included) and never topped up — 4,987 of 5,000 with ~4x headroom (DM4 measured 4,705
206
+ on its config). Now: the bailout reads the STORED count; a per-user budget controller
207
+ scales the remaining users' budgets by (events still needed ÷ expected remaining
208
+ delivery at the realized yield), clamped to [0.25, 4], aiming slightly high; the
209
+ final user's stream is trimmed with a seeded uniform sample so the count never
210
+ exceeds the target. Measured: 5,000 of 5,000; 30,000 of 30,000; test pins 3,000 of
211
+ 3,000. When capacity cannot reach the target (e.g. a churn event ends every user
212
+ early) the run stops short and `result.warnings` carries
213
+ `{ key: 'numEvents', requested, applied }`. User creation still stops once the target
214
+ is met (legacy). Only under the flag — the default path is untouched.
215
+ - **R2-5 profile receipt.** `importResults.users` now carries `generated` (profiles the
216
+ engine pushed to storage, bots included) and `dropped_anonymous` (`_drop`-flagged
217
+ anonymous non-converters never sent to `/engage`) alongside mixpanel-import's
218
+ `success` / `failed`, so `generated - dropped_anonymous - failed === success` is
219
+ checkable. Counters tick at push time, so they hold in batch mode. The sender logs
220
+ a line when the receipt does not reconcile. DM4's "100 generated, 45 reported" is
221
+ now decidable from the result object.
222
+
223
+ ### Tier 2 — the deferred 1.7.0 feature set
224
+
225
+ - **P0-1 `funnels[].conditions`: operators, validation, docs, tests.** Operator maps
226
+ `eq`, `neq`, `in`, `nin`, `gt`, `gte`, `lt`, `lte` (AND within a key, AND across keys;
227
+ no `or`); the scalar shorthand is unchanged. Matching moved to
228
+ `lib/utils/conditions.js` (re-exported from user-loop). The validator THROWS on
229
+ shapes that silently never matched — function values, bare arrays (points at
230
+ `{ in: [...] }`), unknown operators, `in`/`nin` without an array — and warns into
231
+ `result.warnings` when a condition key is declared nowhere the profile is built from.
232
+ Users who satisfy none of the author's funnels (the engine catch-all excluded) are
233
+ counted and reported once per run (`key: 'funnels.conditions'`). Measured: iOS 80.2%
234
+ vs Android 39.8% purchased-per-viewed on the duplicate-funnel idiom; `{ gte: 10 }` 89.7%
235
+ vs `{ lt: 10 }` 20.0%. README "segmented funnels", typed `FunnelConditions`, 30 unit
236
+ cases + integration coverage of the filter branch that had none.
237
+ - **P0-2 experiment variant on the profile.** Every exposed user carries
238
+ `"Experiment: <name>": "<variant>"`, stamped lazily at first exposure (so
239
+ `startDaysBeforeEnd` is respected and never-exposed users carry nothing), before the
240
+ `everything` hook. `experiment.stampProfile` (default `true`) opts out; `sticky: false`
241
+ implies off. Not stamped on step events (would be undeclared columns). Measured: 0
242
+ mismatches against `Variant name` on 13,005 exposure events.
243
+ - **P1-1 `(ctx) => value`.** `choose(value, ctx)` passes
244
+ `{ profile?, event?, time?, config }` to every property function. Zero-arity functions
245
+ are untouched; a function that declares a parameter is context-aware and skips the
246
+ source-string cache (which would otherwise freeze its first result). Bound natives
247
+ (`chance.animal.bind(chance)`) are still called with no argument. Funnel steps after
248
+ the first now know their final time before properties resolve (`fixedTimeMs`, fed by
249
+ a synchronous side channel from step 0) — TimeSoup still runs so the RNG stream is
250
+ unchanged; context-aware step properties defer from `buildFunnelEvents` into
251
+ `makeEvent`. `json-evaluator` emits `(ctx) => body` for expression bodies and passes
252
+ whole-function bodies through unwrapped. Measured: 203 of 203 `pro` users got
253
+ `revenue: 100` from `(ctx) => ctx.profile.plan === 'pro' ? 100 : 0`; 0 events without
254
+ profile context.
255
+ - **P1-2 `stickyEventProps` + stable location (B2).** `stickyEventProps: ['plan_tier']`
256
+ copies the profile value onto every event after `superProps`, before the `event`
257
+ hook; keys declared only in `superProps` resolve once per user. Schema-first: undeclared
258
+ keys throw; `lib/verify/schema-validator.js` treats sticky keys as legal on every event.
259
+ Measured: 67,355 of 67,355 events matched the profile. **B2:** with `hasLocation: true`
260
+ a user's events now share the user's location — `featureCtx.userLocation` was computed
261
+ and never read; measured 0.8% of events matched their profile city before, 100% after.
262
+ - **P1-3 `personas[].ttcModifier`.** Multiplies `timeToConvert` after the experiment
263
+ `ttcMultiplier`, before `funnel-pre`. Measured median TTC 0.50h vs 1.96h for 0.25 vs 1.
264
+ **B3:** `churnRate`, `activeWindow`, `soupOverride` removed from the `Persona` type
265
+ (never implemented); the validator still accepts and warns on them, and no longer
266
+ defaults `churnRate`.
267
+ - **P1-4 `campaignPerUser`.** One campaign template per user at birth; UTMs stamped on
268
+ the profile and reused on every touchpoint. Profile UTM keys already present (persona
269
+ `properties`, `user` hook) win over the draw, so a persona can own a channel. Measured:
270
+ 0 of 399 users with more than one `utm_source` (400 of 400 before). Ad spend derived
271
+ from acquisitions is deferred to 1.8.0.
272
+ - **P2-1 `{ __weights }` + `autoPowerLaw`.** `{ __weights: { free: 60, pro: 30,
273
+ enterprise: 10 } }` draws exactly those shares (measured 239/123/38 over 400 users);
274
+ `autoPowerLaw: false` turns the implicit 45/25/15 draw off for the run (module flag set
275
+ per run, reset with the value caches). Both round-trip through `dungeon-to-json`.
276
+ - **P2-2 `result.warnings[]`.** Always present. Validator clamps (`percentUsersBornInDataset`,
277
+ `bornRecentBias`, compound bias, `avgEventsPerUserPerDay`, `avgActiveDaysPerUser`,
278
+ `numDays < 14`, `engagementDecay` + active days, auto-set `conversionWindowDays`) plus
279
+ runtime aggregates via `context.addWarning` (one entry per key with `count`). Console
280
+ output stays `verbose`-gated. `EngineWarning` type.
281
+ - **P2-4 `conversionRate` saturation.** Every engine clamp of a modified rate above 100
282
+ — experiment variant, persona, world event, or whatever a `funnel-pre` hook left behind
283
+ — is reported once per funnel and source
284
+ (`funnels[Buy].conversionRate:persona "whale" conversionModifier`, requested 195,
285
+ applied 100). The engine cannot see a hook's own `Math.min(95, rate * 3)`; HOOKS.md
286
+ says so.
287
+
288
+ ### Tier 3 — open items outside the deferred table
289
+
290
+ - **P0-3 `worldEvents[].volumeMultiplier > 1` amplifies.** New per-user pass
291
+ `amplifyWorldEvents` clones affected in-window events — `floor(m − 1)` copies plus one
292
+ with probability `frac(m)` — each with a fresh `insert_id` and a timestamp spread
293
+ uniformly across the window (never past the dataset end), after the churn cut and
294
+ before decay and hooks. Measured 3x on a 4-day window: **1.08x before, 3.06x after**,
295
+ all `insert_id`s unique, clones on every window day; 1.5x lands in [1.3, 1.7].
296
+ `aftermath.volumeMultiplier` follows the same rule. Validation rejects negative or
297
+ non-finite multipliers.
298
+ - **P1-5 `eventMultiplier` and funnels — the premise was wrong.** The multiplier scales
299
+ the whole per-user budget, which drives funnel passes too: measured **3.19x / 2.96x**
300
+ for an asked 3x with every event a funnel step. DM4's 0.96x came from its fixture's
301
+ `isChurnEvent` (`Churned`, weight 1, `returnLikelihood: 0.15`): a churn event in the
302
+ standalone pool ends every user after roughly the same number of events regardless of
303
+ budget — measured **1.04x with the churn event, 2.90x without**, same config. The
304
+ engine now reports it (`key: 'personas.eventMultiplier'`) when more than half the
305
+ users churn while a persona multiplier is in play; `eventMultiplier` and `isChurnEvent`
306
+ docs state the cap. DM4 should replace its "fewer than two non-funnel events" check
307
+ with a churn-event check.
308
+ - **P1-6 day-1 retention floor — documented (option 2).** `retentionCurve` picks session
309
+ days; retention counts events; birth-day funnels spill into day 1 regardless of the
310
+ day plan, so day 1 sits near 0.85 (DM4 measured 0.885 for an asked 0.15; days 7 and 30
311
+ follow the curve). Stated in the `retentionCurve` JSDoc, README config table, and
312
+ HOOKS.md §2.7. Verify from day 7 on.
313
+
314
+ ### Behavior changes (not purely additive)
315
+
316
+ - **B1** — `conditions` values that are functions or bare arrays now THROW at validation.
317
+ Any dungeon relying on them was already producing an empty funnel.
318
+ - **B2** — `hasLocation: true` now yields one stable location per user on events instead
319
+ of a fresh random city per event. Event geo distributions change; `scd.js` (the one
320
+ technical fixture with `hasLocation`) is the reference.
321
+ - **B3** — `churnRate`, `activeWindow`, `soupOverride` are gone from the `Persona` type.
322
+ Runtime still accepts them with the existing once-per-process warning.
323
+ - World-event windows on funnel steps after the first now test the step's FINAL time
324
+ (previously the pre-offset TimeSoup time). Only dungeons combining `worldEvents` with
325
+ multi-step funnels see different `_drop` / `injectProps` decisions; the fix is what the
326
+ docs always described.
327
+
328
+ ### Not built (by request)
329
+
330
+ Session replay, per-funnel `soup`, `or` conditions, anything in `stories` / `verify` /
331
+ `emulateBreakdown`, and ad spend derived from acquisitions (P1-4 item 3, 1.8.0). R2-4
332
+ (the verticals as DM4 templates) is deferred to its own sprint per AK: each vertical
333
+ should demonstrate a different declarative trend type so the template gallery doubles
334
+ as a catalog demo; the R2-4 audit stands as that sprint's punch list.
335
+
5
336
  ## 1.6.5 — 2026-09-02
6
337
 
7
338
  ### Changed
package/HOOKS.md CHANGED
@@ -21,10 +21,12 @@ hook: function (record, type, meta) { ... return record; }
21
21
  | `event` | `events.js:176` | Single event (flat props) | **Used** (replaces event) | `user: { distinct_id }`, `config`, `datasetStart`, `datasetEnd` |
22
22
  | `funnel-post` | `funnels.js:153` | Array of funnel events | Ignored (mutate in-place) | `user`, `profile`, `scd`, `funnel`, `config`, `experiment` |
23
23
  | `everything` | `user-loop.js:280` | Array of ALL user events | **Used** if array returned | `profile`, `scd`, `config`, `datasetStart`, `datasetEnd`, `userIsBornInDataset`, `authTime`, `isPreAuth`, `persona` |
24
- | `ad-spend` | `storage.js` | Ad spend event | Ignored | -- |
25
- | `group` | `storage.js` | Group profile | Ignored | -- |
26
- | `mirror` | `storage.js` | Mirror data point | Ignored | -- |
27
- | `lookup` | `storage.js` | Lookup table entry | Ignored | -- |
24
+ | `ad-spend` | `storage.js` | Ad spend event | Object or array used | -- |
25
+ | `group` | `storage.js` | Group profile | Object or array used | -- |
26
+ | `mirror` | `storage.js` | Mirror data point | Object or array used | -- |
27
+ | `lookup` | `storage.js` | Lookup table entry | Object or array used | -- |
28
+ | `standalone` | `storage.js`, before user loop | Identity-less cadence event | Object or array used; `undefined` drops | `spec`, `config`, `datasetStart`, `datasetEnd` |
29
+ | `warehouse` | `storage.js` | One materialized warehouse row | Ignored | `spec`, `config`, `metricName`, `bucketIndex`, `bucketCount`, `grain`, `seriesKey`, `isBackfill`, `raw`, `datasetStart`, `datasetEnd` |
28
30
 
29
31
  **Per-user execution order:** `user` -> `scd-pre` -> `funnel-pre` -> `event` -> `funnel-post` -> `everything`
30
32
 
@@ -36,7 +38,130 @@ double-fire mutations.
36
38
  **Return rules:**
37
39
  - `event`: return the (possibly replaced) event object.
38
40
  - `everything`: return the (possibly modified) array. Filtered array removes events.
39
- - All other types: mutate `record` in-place. Return value is ignored.
41
+ - `ad-spend`, `group`, `mirror`, `lookup`, `standalone`: return the record or an
42
+ array of records. Returning `undefined` drops the record.
43
+ - `user`, `scd-pre`, `funnel-pre`, `funnel-post`, `warehouse`: mutate `record`
44
+ in place. Return value is ignored.
45
+
46
+ `standaloneEvents` runs before the user loop; `warehouseMetrics` materializes
47
+ after it. Neither hook receives person metadata or enters `everything`.
48
+ Standalone synthetic `distinct_id` values identify series, never people. Use
49
+ disk-backed `duckdb` assertions on `{{PREFIX}}-STANDALONE*.json` for cadence
50
+ stories. Warehouse stories support `warehouse` and `warehouse-stats`
51
+ assertions, with automatic table audits even when no stories are exported.
52
+
53
+ ### 1.1 Warehouse rows (`type === 'warehouse'`)
54
+
55
+ `warehouse` fires once per materialized row, after the user loop and before the
56
+ rows are written to disk. the row already matches the manifest contract:
57
+
58
+ - `timeColumn`
59
+ - every `source.groupBy` key, in order
60
+ - `valueColumn`
61
+ - every declared key in `columns`
62
+
63
+ declare every key up front. warehouse containers are created with a fixed column
64
+ list, and the manifest is built from that same list. an undeclared key is not
65
+ part of the output contract even if it exists briefly in memory.
66
+
67
+ `meta.seriesKey` is the joined group tuple in `source.groupBy` order, separated
68
+ by `|`. examples:
69
+
70
+ - no `groupBy` → `''`
71
+ - `groupBy: ['region']` and `row.region === 'us'` → `'us'`
72
+ - `groupBy: ['region', 'plan_tier']` and `row.region === 'us'`, `row.plan_tier === 'enterprise'` → `'us|enterprise'`
73
+
74
+ `meta.bucketIndex` and `meta.bucketCount` are chronological and include history
75
+ buckets even when `sparse: true` skips repeated rows. `meta.isBackfill` is true
76
+ for the synthetic buckets created by `history`. `meta.raw` is the bucketed
77
+ source truth before `scale`, `noise`, and point-in-time carry-forward.
78
+
79
+ Treat the time axis as immutable. `row[spec.timeColumn]` drives ordering,
80
+ manifest SQL, and warehouse verification. mutate the value column or declared
81
+ extra columns instead.
82
+
83
+ Recipe: scale a point-in-time level for an in-window story slice
84
+
85
+ ```js
86
+ warehouseMetrics: [{
87
+ name: 'daily_active_subscriptions',
88
+ type: 'point-in-time',
89
+ source: { event: 'subscription_started', minus: 'subscription_cancelled', groupBy: 'region' },
90
+ baseline: 40,
91
+ timeColumn: 'date',
92
+ valueColumn: 'active_subscriptions',
93
+ columns: { lifted: false },
94
+ }],
95
+
96
+ hook: (row, type, meta) => {
97
+ if (type !== 'warehouse') return row;
98
+ if (meta.metricName !== 'daily_active_subscriptions') return row;
99
+ if (meta.isBackfill) return row;
100
+
101
+ const liveIndex = meta.bucketIndex - meta.spec.history;
102
+ if (meta.seriesKey === 'us' && liveIndex >= 7 && liveIndex < 14) {
103
+ row.active_subscriptions = Math.round(row.active_subscriptions * 1.2);
104
+ row.lifted = true;
105
+ }
106
+ return row;
107
+ }
108
+ ```
109
+
110
+ Recipe: apply an incident dip to one series only
111
+
112
+ ```js
113
+ warehouseMetrics: [{
114
+ name: 'daily_new_bookings',
115
+ source: {
116
+ event: 'new_booking',
117
+ measure: 'sum',
118
+ property: 'booking_value',
119
+ groupBy: ['region', 'plan_tier'],
120
+ },
121
+ timeColumn: 'date',
122
+ valueColumn: 'bookings',
123
+ columns: { incident: false },
124
+ }],
125
+
126
+ hook: (row, type, meta) => {
127
+ if (type !== 'warehouse') return row;
128
+ if (meta.metricName !== 'daily_new_bookings') return row;
129
+ if (meta.isBackfill) return row;
130
+ if (meta.seriesKey !== 'us|enterprise') return row;
131
+
132
+ const liveIndex = meta.bucketIndex - meta.spec.history;
133
+ if (liveIndex >= 14 && liveIndex <= 16) {
134
+ row.bookings = Math.round(row.bookings * 0.35);
135
+ row.incident = true;
136
+ }
137
+ return row;
138
+ }
139
+ ```
140
+
141
+ **What 1.7.0 changed for hooks.** No hook signature, `meta` field, or firing
142
+ order changed, and the hook-helper atoms and patterns are untouched. What a hook
143
+ SEES did change in five places:
144
+
145
+ 1. `event` hook on funnel steps after the first: `record.time` is now the step's
146
+ FINAL time. Before 1.7.0 it was the pre-offset TimeSoup time, overwritten after
147
+ the hook returned — a hook that derived a property from `record.time` on step 2+
148
+ was reading the wrong timestamp.
149
+ 2. `everything` hook: the array already contains world-event clones
150
+ (`volumeMultiplier > 1`), `stickyEventProps` values on every event, and one
151
+ stable location per user under `hasLocation`. `meta.profile` carries
152
+ `"Experiment: <name>"` for exposed users (stamped before `everything`, after `user`).
153
+ 3. `funnel-pre`: `record.timeToConvert` already includes the persona `ttcModifier`
154
+ (like `conversionModifier` today). The hook stays the final authority. A rate the
155
+ hook leaves above 100 is clamped downstream exactly as before and now shows up in
156
+ `result.warnings`.
157
+ 4. `user` hook: under `campaignPerUser` the profile already carries the drawn
158
+ `utm_*` keys; a hook that overwrites them wins, and the touchpoint pass reads the
159
+ final values. `stickyEventProps` values are read from the profile AFTER the hook.
160
+ 5. Hooks may not add properties (rule 1) — `stickyEventProps`, the experiment
161
+ profile key, and world-event clones are engine-stamped and declared, so the
162
+ schema validator accepts them. Prefer these declarative knobs over a hook when
163
+ they express the story (README "segmented funnels", "sticky event properties",
164
+ "campaigns per user").
40
165
 
41
166
  ---
42
167
 
@@ -399,6 +524,17 @@ All items on the v1.5.0 "documented gaps" list closed in 1.6.0. Unrecognized
399
524
  retention option keys now THROW — kills the silent-ignore class of bug where a
400
525
  typo'd `compounded: true` was dropped without effect.
401
526
 
527
+ **`retentionCurve` cannot move day 1 (v1.7.0 doc).** `buildActiveDayPlan`
528
+ picks which UTC days a user gets a SESSION; retention counts EVENTS. A funnel
529
+ opened on the birth day spills its later steps across the following
530
+ `timeToConvert` hours regardless of the day plan, so day 1 sits on a floor
531
+ near 0.85 for funnel-driven dungeons no matter what `day1` asks for. Measured
532
+ (2,000 users, 60 days): `{ day1: 0.15, day7: 0.06, day30: 0.02 }` delivered
533
+ day 1 = 0.885, day 7 = 0.151, day 30 = 0.060 — the curve governs from day 7
534
+ on and over-delivers by a consistent ~2.5x there. Verify retention stories from
535
+ day 7 onward. To lower day 1, shorten `timeToConvert` on the funnels users
536
+ enter on birth, or drop next-day spill in an `everything` hook.
537
+
402
538
  ### 2.8 Funnel reentry: state machine resets after completion
403
539
 
404
540
  Reference: `history.cpp` (`last_step_starts_next_funnel`). With reentry
@@ -813,6 +949,19 @@ if (type === "funnel-pre") {
813
949
  Greedy funnel engine (Section 2.2) applies after — keep `conversionRate`
814
950
  adjustments modest (1.2x is comfortable; 3x can saturate at the 95% cap).
815
951
 
952
+ **Saturation is reported, but only the engine's own clamp (v1.7.0, P2-4).** When a
953
+ persona `conversionModifier`, an experiment variant, a world event, or the value a
954
+ `funnel-pre` hook leaves behind pushes `conversionRate` above 100, the engine
955
+ clamps to 100 and adds one aggregated entry per funnel and source to
956
+ `result.warnings` (`funnels[Checkout].conversionRate:persona "whale"
957
+ conversionModifier`, `requested: 195`, `applied: 100`). It cannot see a hook's own
958
+ cap: `record.conversionRate = Math.min(95, rate * 3)` on a base of 65 yields 95 — a
959
+ 1.46x lift, not 3x — and the engine never learns the intended 195. To get a true
960
+ multiple, read the base rate and solve for it, or lower the base rate so the multiple
961
+ fits under the cap. For "one segment converts differently on one funnel" prefer the
962
+ declarative `funnels[].conditions` (README "segmented funnels") over a `funnel-pre`
963
+ hook — it needs no cap arithmetic.
964
+
816
965
  ---
817
966
 
818
967
  #### 4.2 Feature Launch Inflection