@ak--47/dungeon-master 1.7.0 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/.claude/skills/analyze-soup/SKILL.md +21 -11
  2. package/.claude/skills/create-dungeon/SKILL.md +49 -10
  3. package/.claude/skills/create-project/SKILL.md +22 -3
  4. package/.claude/skills/create-project/context.mjs +89 -0
  5. package/.claude/skills/create-project/provision.mjs +1 -60
  6. package/.claude/skills/headless-build/SKILL.md +18 -1
  7. package/.claude/skills/powertools/SKILL.md +20 -1
  8. package/.claude/skills/release-check/SKILL.md +99 -0
  9. package/.claude/skills/verify-dungeon/SKILL.md +71 -16
  10. package/.claude/skills/verify-dungeon/references/counting-semantics.md +14 -0
  11. package/.claude/skills/verify-dungeon/references/report-format.md +18 -1
  12. package/.claude/skills/verify-dungeon/references/sql-recipes.md +36 -1
  13. package/.claude/skills/warehouse-metrics/GAPS-template.md +34 -0
  14. package/.claude/skills/warehouse-metrics/SKILL.md +105 -0
  15. package/.claude/skills/warehouse-metrics/deploy.mjs +651 -0
  16. package/.claude/skills/write-hooks/SKILL.md +33 -3
  17. package/CHANGELOG.md +142 -0
  18. package/HOOKS.md +105 -5
  19. package/README.md +228 -0
  20. package/docs/guides/1.8.0-upgrade-guide.md +151 -0
  21. package/dungeons/technical/warehouse.js +187 -0
  22. package/index.js +116 -2
  23. package/lib/core/config-validator.js +21 -0
  24. package/lib/core/dungeon-loader.js +1 -1
  25. package/lib/core/storage.js +51 -3
  26. package/lib/generators/standalone.js +248 -0
  27. package/lib/generators/warehouse.js +828 -0
  28. package/lib/orchestrators/mixpanel-sender.js +27 -2
  29. package/lib/orchestrators/user-loop.js +1 -0
  30. package/lib/templates/story-spec.schema.json +41 -16
  31. package/lib/utils/utils.js +37 -12
  32. package/lib/verify/index.js +1 -0
  33. package/lib/verify/story-runner.js +71 -8
  34. package/lib/verify/warehouse.js +683 -0
  35. package/package.json +4 -2
  36. package/scripts/verify-stories.mjs +150 -44
  37. package/types.d.ts +303 -4
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: write-hooks
3
3
  description: Use when an existing dungeon needs engineered story trends or "magic number" patterns — writes the `hook` function using atom helpers and high-level patterns. Adds no new event flags; never mutates the schema.
4
- argument-hint: [path/to/dungeon.js] [free-text story / trend description]
4
+ argument-hint: '[path/to/dungeon.js] [free-text story / trend description]'
5
5
  model: claude-opus-4-6
6
6
  effort: max
7
7
  ---
@@ -68,7 +68,7 @@ property values that differ by cohort, injected bursts, lifecycle waves.
68
68
  Hooks fire in this order per user (see `CLAUDE.md` for the canonical reference):
69
69
 
70
70
  1. `"user"` — profile created. Mutate in place; return ignored.
71
- 2. `"scd-pre"` — SCD entries created. Mutate in place OR return new array.
71
+ 2. `"scd-pre"` — SCD entries created. Mutate in place; return ignored.
72
72
  3. For each funnel: `"funnel-pre"` → `"event"` (per step) → `"funnel-post"`.
73
73
 
74
74
  **`funnel-pre` is now reliable for temporal patterns.** Usage funnels advance a
@@ -76,7 +76,7 @@ cursor after each run, so successive `meta.firstEventTime` values spread across
76
76
  the user's active window. Persona and world-event modifiers apply BEFORE the
77
77
  hook — the hook has final authority on `conversionRate`, `timeToConvert`, and
78
78
  `props`.
79
- 4. `"event"` — for non-funnel standalone events. Return value REPLACES the event.
79
+ 4. `"event"` — for non-funnel user events from `events[]`. Return value REPLACES the event.
80
80
  5. `"everything"` — array of ALL the user's events. Return array to replace.
81
81
 
82
82
  **Most engineered trends belong in `everything`.** It sees the full user stream,
@@ -86,6 +86,36 @@ and you can mutate freely.
86
86
  Storage-only hooks (`ad-spend`, `group`, `mirror`, `lookup`) fire later in the
87
87
  pipeline and don't see the same `meta` shape.
88
88
 
89
+ ### Cadence streams and warehouse rows (v1.8.0)
90
+
91
+ These hooks sit outside the per-user sequence and never enter `everything`.
92
+ Neither has person metadata (`meta.profile`, auth state, sessions, or SCDs).
93
+
94
+ - `standaloneEvents`: `type === 'standalone'` fires on storage push before the
95
+ user loop. Read `meta.spec` and `meta.config` to identify the stream. Return
96
+ the record object or an array of records. Returning `undefined` drops it.
97
+ Mutating without returning is therefore insufficient. Preserve required keys
98
+ and use fresh `insert_id` values for clones. Its synthetic `distinct_id` is a
99
+ series identifier, never a person or a retention cohort.
100
+ - `warehouseMetrics`: `type === 'warehouse'` fires after user generation, once
101
+ per materialized row. Mutate the row in place; its return value is ignored.
102
+ Keep the time column and group keys stable. Modify only the value column and
103
+ declared extra columns. Meta includes `spec`, `config`, `metricName`,
104
+ `bucketIndex`, `bucketCount`, `grain`, `seriesKey`, `isBackfill`, and `raw`.
105
+ `raw` describes plus/minus source aggregates before scaling, noise, and carry.
106
+
107
+ Do not add either schema here. Send missing `standaloneEvents` properties or
108
+ warehouse `columns` back to `/create-dungeon`. Warehouse sources consume user
109
+ `events[]` only, including both plus and minus legs; they cannot consume cadence
110
+ streams. Standalone value functions use tick context, warehouse column functions
111
+ use bucket context; neither supplies a user profile.
112
+
113
+ Verify standalone stories with disk-backed `duckdb` assertions against
114
+ `{{PREFIX}}-STANDALONE*.json`. The user-event emulator and `--in-memory` CLI mode
115
+ do not evaluate this stream. Warehouse stories can use `warehouse` assertions
116
+ or `warehouse-stats` assertions; automatic warehouse audits also run without
117
+ stories. Hand off to `/verify-dungeon` with an explicit artifact prefix.
118
+
89
119
  ## Hook meta — identity context
90
120
 
91
121
  Inside `funnel-pre` and `funnel-post`:
package/CHANGELOG.md CHANGED
@@ -2,6 +2,148 @@
2
2
 
3
3
  All notable changes to `@ak--47/dungeon-master`.
4
4
 
5
+ ## 1.8.0 — 2026-09-10
6
+
7
+ ### Added — `standaloneEvents`: identity-less metric snapshots
8
+
9
+ A new top-level config key that generates records describing a **system, not a
10
+ person**. They carry no `user_id` and no `device_id`. Before 1.8.0 the only
11
+ identity-less stream the engine could produce was `$ad_spend` via `hasAdSpend`,
12
+ which is hard-coded to one shape, one cadence, and a Mixpanel reserved event
13
+ name. `standaloneEvents` is the general form.
14
+
15
+ ```js
16
+ standaloneEvents: [{
17
+ event: 'cdn_egress',
18
+ cadence: 'day', // 'hour' | 'day' | 'week', default 'day'
19
+ dimensions: { region: ['us-east', 'us-west', 'eu'] }, // cross-producted
20
+ distinctIdFrom: 'region', // synthetic id, never a person
21
+ properties: {
22
+ gb_out: (ctx) => 400 + ctx.tickIndex * 3,
23
+ cost_usd: (ctx) => (400 + ctx.tickIndex * 3) * 0.085,
24
+ p95_ms: [120, 140, 160],
25
+ },
26
+ }]
27
+ ```
28
+
29
+ - One record per cadence tick per dimension cross-product row.
30
+ - Ticks start at the dataset start and step by the cadence. The last tick is the
31
+ final one at or before the dataset end, so nothing lands in the future.
32
+ - Each record carries `event`, `time`, `insert_id`, `distinct_id`, every
33
+ dimension as a flat property, and every resolved entry in `properties`.
34
+ - `distinct_id` is the value of the dimension named by `distinctIdFrom`, else the
35
+ event name. It exists so Mixpanel accepts the record; it never maps to a person.
36
+ - Property value functions receive a `StandaloneValueContext`:
37
+ `{ time, config, dimensions, tickIndex, tickCount, cadence, event }`.
38
+ `tickIndex / (tickCount - 1)` is window progress — use it to shape a trend.
39
+ - New hook type `standalone` (storage-only). Return the record or an array of
40
+ records; returning nothing drops the record. `meta.spec` carries the resolved
41
+ stream config. The `warehouse` hook instead mutates its row and ignores returns.
42
+ - Lands in `result.standaloneEventData`, writes to a `-STANDALONE` file shard,
43
+ and imports to Mixpanel as its own event stream.
44
+ - Validation **throws** on a malformed entry rather than skipping it. A silent
45
+ skip would drop a whole data stream without the author noticing.
46
+
47
+ New types: `StandaloneEventConfig`, `ResolvedStandaloneEventConfig`,
48
+ `StandaloneValueContext`, `HookMetaStandalone`. `WritePaths` gains
49
+ `standaloneFiles`; `Result` gains `standaloneEventData`; `hookTypes` gains
50
+ `"standalone"`.
51
+
52
+ **Output compatibility.** Additive only. A config without `standaloneEvents` is
53
+ byte-identical to 1.7.0 — the generation pass is gated on the key being present,
54
+ so the seeded RNG stream is untouched. `config.standaloneEvents` normalizes to
55
+ `[]` when absent. Event determinism comparisons exclude the fresh `insert_id`.
56
+
57
+ New tests: `tests/unit/standalone-events.test.js` (25),
58
+ `tests/integration/standalone-events.test.js` (15).
59
+
60
+ ### Added — `warehouseMetrics`: manifest-driven warehouse source tables
61
+
62
+ A new top-level config key that materializes warehouse-ready tables from the
63
+ run's own events after generation completes. This is the local source-table side
64
+ of a warehouse metric demo: bookings rollups, active subscription levels, ARR
65
+ snapshots, and other time-series tables that should read like a real warehouse.
66
+
67
+ ```js
68
+ warehouseMetrics: [{
69
+ name: 'daily_new_bookings',
70
+ source: { event: 'new_booking', measure: 'sum', property: 'booking_value' },
71
+ valueColumn: 'bookings',
72
+ }]
73
+ ```
74
+
75
+ - Supports additive and point-in-time metrics.
76
+ - Grain: `day`, `week`, `month`.
77
+ - Supports subtractive `minus` legs, `groupBy` on up to two declared keys,
78
+ optional `history` backfill, sparse point-in-time emission, seeded `noise`,
79
+ `scale`, and derived `columns`.
80
+ - Lands in `result.warehouseMetricData` keyed by metric name and emits
81
+ `result.warehouseManifest` with table schemas, SQL, and recommended
82
+ aggregation.
83
+ - Writes `<name>-WAREHOUSE-<table>.csv|json` plus
84
+ `<name>-WAREHOUSE-MANIFEST.json` when `writeToDisk` is enabled.
85
+ - Never imports through `token`. Warehouse deploy is a separate flow.
86
+ - New hook type `warehouse` fires once per materialized row with
87
+ `metricName`, `bucketIndex`, `bucketCount`, `grain`, `seriesKey`,
88
+ `isBackfill`, and `raw` bucket stats.
89
+ - Warehouse verification adds `warehouse` / `warehouse-stats` story breakdowns
90
+ plus automatic audits over declared columns, gaps, monotonic time, empty numeric
91
+ cells, and sparse first-bucket coverage. Source correlation is available to story
92
+ assertions; it is not an automatic pass/fail gate.
93
+ - Sparse point-in-time comparison carries emitted levels across missing buckets,
94
+ excludes history from correlation, and compares the first live delta to the scaled
95
+ baseline. Warehouse generation preserves the user event stream, including with noise.
96
+ - Disk verification parses quoted multiline CSV records with `csv-parse` and keeps
97
+ disk and in-memory audit results consistent.
98
+
99
+ ### Added — `/warehouse-metrics`: BigQuery load + warehouse metric save flow
100
+
101
+ The shipped skill at `.claude/skills/warehouse-metrics/` loads the generated
102
+ warehouse tables into BigQuery, connects that dataset to Mixpanel with the
103
+ existing powertools macro, previews each metric SQL, and saves new metrics when
104
+ the CRUD endpoints are available.
105
+
106
+ - Uses the emitted warehouse manifest as the contract.
107
+ - Maps manifest `recommendedAggregation: 'last value'` to the API's
108
+ `aggregation: 'last_value'`.
109
+ - If `GET /crud/getWarehouseMetrics` returns 404, the script still completes the
110
+ BigQuery load and source setup, then writes `warehouse/GAPS.md` for manual
111
+ metric creation.
112
+ - Preview fails fast on the raw substring block (`CREATE`, `UPDATE`, etc.), so
113
+ identifiers like `created_at` and `updated_at` are a real deploy-time trap.
114
+ - The canonical skill name is `/warehouse-metrics`. Bundled commands and handoffs
115
+ use `.claude/skills/warehouse-metrics/`.
116
+ - Uses the shipped Powertools warehouse CRUD and IAM setup macro. Runtime IAM is
117
+ configured; permission failures still stop deployment with the original error.
118
+ - Dry-run works before project provisioning, using placeholders without credentials.
119
+ Live table replacement requires explicit operator consent; the script does not prompt.
120
+
121
+ ### Changed — skill workflow and provisioning context
122
+
123
+ - All nine bundled skills have parsed string argument hints and matching folder names.
124
+ Release tests check frontmatter and warehouse handoffs.
125
+ - Authoring, hooks, and verification distinguish person events, identity-less cadence
126
+ events, and identity-free warehouse rows. Synthetic IDs never count as people.
127
+ - Verification preserves the explicit run prefix and local uncompressed warehouse
128
+ artifacts for deployment. Soup analysis remains scoped to user-event timestamps.
129
+ - Project business context includes separate cadence and warehouse summaries without
130
+ evaluating property functions or including credentials.
131
+ - Headless builds preserve warehouse history and use query preview for fresh results;
132
+ warehouse refresh only invalidates the saved metric cache.
133
+
134
+ ### Added - `/release-check`
135
+
136
+ The new `/release-check` skill audits tests, determinism, documentation, package
137
+ contents, and release handoffs. `.agents/skills` and `.github/skills` link to the
138
+ canonical `.claude/skills` directory for shared agent discovery. Publishing remains
139
+ an explicitly authorized operator action.
140
+
141
+ ### Changed — `streamCSV` preserves falsy cells
142
+
143
+ CSV serialization now writes `0` and `false` as literal cell values instead of
144
+ empty strings. If downstream warehouse SQL or fixtures were treating blank cells
145
+ as zero or false, update them to read the actual value.
146
+
5
147
  ## 1.7.0 — 2026-09-03
6
148
 
7
149
  The engine round for DM4 v5. Executes the 1.6.4 "Deferred to 1.7.0" table plus
package/HOOKS.md CHANGED
@@ -21,10 +21,12 @@ hook: function (record, type, meta) { ... return record; }
21
21
  | `event` | `events.js:176` | Single event (flat props) | **Used** (replaces event) | `user: { distinct_id }`, `config`, `datasetStart`, `datasetEnd` |
22
22
  | `funnel-post` | `funnels.js:153` | Array of funnel events | Ignored (mutate in-place) | `user`, `profile`, `scd`, `funnel`, `config`, `experiment` |
23
23
  | `everything` | `user-loop.js:280` | Array of ALL user events | **Used** if array returned | `profile`, `scd`, `config`, `datasetStart`, `datasetEnd`, `userIsBornInDataset`, `authTime`, `isPreAuth`, `persona` |
24
- | `ad-spend` | `storage.js` | Ad spend event | Ignored | -- |
25
- | `group` | `storage.js` | Group profile | Ignored | -- |
26
- | `mirror` | `storage.js` | Mirror data point | Ignored | -- |
27
- | `lookup` | `storage.js` | Lookup table entry | Ignored | -- |
24
+ | `ad-spend` | `storage.js` | Ad spend event | Object or array used | -- |
25
+ | `group` | `storage.js` | Group profile | Object or array used | -- |
26
+ | `mirror` | `storage.js` | Mirror data point | Object or array used | -- |
27
+ | `lookup` | `storage.js` | Lookup table entry | Object or array used | -- |
28
+ | `standalone` | `storage.js`, before user loop | Identity-less cadence event | Object or array used; `undefined` drops | `spec`, `config`, `datasetStart`, `datasetEnd` |
29
+ | `warehouse` | `storage.js` | One materialized warehouse row | Ignored | `spec`, `config`, `metricName`, `bucketIndex`, `bucketCount`, `grain`, `seriesKey`, `isBackfill`, `raw`, `datasetStart`, `datasetEnd` |
28
30
 
29
31
  **Per-user execution order:** `user` -> `scd-pre` -> `funnel-pre` -> `event` -> `funnel-post` -> `everything`
30
32
 
@@ -36,7 +38,105 @@ double-fire mutations.
36
38
  **Return rules:**
37
39
  - `event`: return the (possibly replaced) event object.
38
40
  - `everything`: return the (possibly modified) array. Filtered array removes events.
39
- - All other types: mutate `record` in-place. Return value is ignored.
41
+ - `ad-spend`, `group`, `mirror`, `lookup`, `standalone`: return the record or an
42
+ array of records. Returning `undefined` drops the record.
43
+ - `user`, `scd-pre`, `funnel-pre`, `funnel-post`, `warehouse`: mutate `record`
44
+ in place. Return value is ignored.
45
+
46
+ `standaloneEvents` runs before the user loop; `warehouseMetrics` materializes
47
+ after it. Neither hook receives person metadata or enters `everything`.
48
+ Standalone synthetic `distinct_id` values identify series, never people. Use
49
+ disk-backed `duckdb` assertions on `{{PREFIX}}-STANDALONE*.json` for cadence
50
+ stories. Warehouse stories support `warehouse` and `warehouse-stats`
51
+ assertions, with automatic table audits even when no stories are exported.
52
+
53
+ ### 1.1 Warehouse rows (`type === 'warehouse'`)
54
+
55
+ `warehouse` fires once per materialized row, after the user loop and before the
56
+ rows are written to disk. the row already matches the manifest contract:
57
+
58
+ - `timeColumn`
59
+ - every `source.groupBy` key, in order
60
+ - `valueColumn`
61
+ - every declared key in `columns`
62
+
63
+ declare every key up front. warehouse containers are created with a fixed column
64
+ list, and the manifest is built from that same list. an undeclared key is not
65
+ part of the output contract even if it exists briefly in memory.
66
+
67
+ `meta.seriesKey` is the joined group tuple in `source.groupBy` order, separated
68
+ by `|`. examples:
69
+
70
+ - no `groupBy` → `''`
71
+ - `groupBy: ['region']` and `row.region === 'us'` → `'us'`
72
+ - `groupBy: ['region', 'plan_tier']` and `row.region === 'us'`, `row.plan_tier === 'enterprise'` → `'us|enterprise'`
73
+
74
+ `meta.bucketIndex` and `meta.bucketCount` are chronological and include history
75
+ buckets even when `sparse: true` skips repeated rows. `meta.isBackfill` is true
76
+ for the synthetic buckets created by `history`. `meta.raw` is the bucketed
77
+ source truth before `scale`, `noise`, and point-in-time carry-forward.
78
+
79
+ Treat the time axis as immutable. `row[spec.timeColumn]` drives ordering,
80
+ manifest SQL, and warehouse verification. mutate the value column or declared
81
+ extra columns instead.
82
+
83
+ Recipe: scale a point-in-time level for an in-window story slice
84
+
85
+ ```js
86
+ warehouseMetrics: [{
87
+ name: 'daily_active_subscriptions',
88
+ type: 'point-in-time',
89
+ source: { event: 'subscription_started', minus: 'subscription_cancelled', groupBy: 'region' },
90
+ baseline: 40,
91
+ timeColumn: 'date',
92
+ valueColumn: 'active_subscriptions',
93
+ columns: { lifted: false },
94
+ }],
95
+
96
+ hook: (row, type, meta) => {
97
+ if (type !== 'warehouse') return row;
98
+ if (meta.metricName !== 'daily_active_subscriptions') return row;
99
+ if (meta.isBackfill) return row;
100
+
101
+ const liveIndex = meta.bucketIndex - meta.spec.history;
102
+ if (meta.seriesKey === 'us' && liveIndex >= 7 && liveIndex < 14) {
103
+ row.active_subscriptions = Math.round(row.active_subscriptions * 1.2);
104
+ row.lifted = true;
105
+ }
106
+ return row;
107
+ }
108
+ ```
109
+
110
+ Recipe: apply an incident dip to one series only
111
+
112
+ ```js
113
+ warehouseMetrics: [{
114
+ name: 'daily_new_bookings',
115
+ source: {
116
+ event: 'new_booking',
117
+ measure: 'sum',
118
+ property: 'booking_value',
119
+ groupBy: ['region', 'plan_tier'],
120
+ },
121
+ timeColumn: 'date',
122
+ valueColumn: 'bookings',
123
+ columns: { incident: false },
124
+ }],
125
+
126
+ hook: (row, type, meta) => {
127
+ if (type !== 'warehouse') return row;
128
+ if (meta.metricName !== 'daily_new_bookings') return row;
129
+ if (meta.isBackfill) return row;
130
+ if (meta.seriesKey !== 'us|enterprise') return row;
131
+
132
+ const liveIndex = meta.bucketIndex - meta.spec.history;
133
+ if (liveIndex >= 14 && liveIndex <= 16) {
134
+ row.bookings = Math.round(row.bookings * 0.35);
135
+ row.incident = true;
136
+ }
137
+ return row;
138
+ }
139
+ ```
40
140
 
41
141
  **What 1.7.0 changed for hooks.** No hook signature, `meta` field, or firing
42
142
  order changed, and the hook-helper atoms and patterns are untouched. What a hook
package/README.md CHANGED
@@ -91,6 +91,11 @@ const result = await DUNGEON_MASTER({
91
91
  console.log(result.importResults);
92
92
  ```
93
93
 
94
+ `token` imports event-shaped streams only: events, users, groups, ad spend, and
95
+ `standaloneEvents`. `warehouseMetrics` does **not** import through this path.
96
+ warehouse tables are materialized locally and need a separate warehouse deploy
97
+ step after the run.
98
+
94
99
  ## dungeons
95
100
 
96
101
  a dungeon is a javascript file that exports a configuration object. it defines your entire data model: events, funnels, user properties, group analytics, SCDs, and a hook function that engineers discoverable patterns into the data.
@@ -537,6 +542,8 @@ dungeon-master generates multiple data types that mirror a real analytics implem
537
542
  | SCDs | `scdProps` | slowly changing dimensions (subscription tier over time) |
538
543
  | lookup tables | `lookupTables` | dimension tables (product catalog, region mapping) |
539
544
  | ad spend | `hasAdSpend` | daily ad spend with impressions, clicks, cost metrics |
545
+ | standalone events | `standaloneEvents` | identity-less metric snapshots on a cadence (infrastructure, finance, ops) |
546
+ | warehouse metrics | `warehouseMetrics` | warehouse source tables derived from generated events, with a manifest for downstream deploy |
540
547
  | mirror datasets | `mirrorProps` | transformed copies of event data (A/B versions) |
541
548
  | organic text | `createTextGenerator` | reviews, support tickets, search queries, chat messages |
542
549
 
@@ -606,6 +613,222 @@ variant assignment is **sticky by default**: a deterministic hash of `user_id` +
606
613
 
607
614
  **the variant lands on the user profile (1.7.0).** every exposed user carries `"Experiment: <name>": "<variant>"` (e.g. `"Experiment: Checkout Redesign": "New Checkout"`), so the funnel breaks down by variant in Mixpanel with a user-property breakdown — no cohort built from the exposure event. stamped when the user is first exposed (respects `startDaysBeforeEnd`); never-exposed users carry nothing; the `user` hook fires before exposure and does not see it, the `everything` hook does. `stampProfile: false` turns it off; `sticky: false` implies off. measured: 0 mismatches between the profile value and the `Variant name` on 13,005 exposure events.
608
615
 
616
+ ## standalone events (identity-less metric snapshots)
617
+
618
+ `standaloneEvents` generates records that describe a **system, not a person**. they carry
619
+ no `user_id` and no `device_id`. use them for infrastructure, finance, and ops telemetry:
620
+ daily CDN egress per region, weekly billing rollups per plan tier, hourly queue depth per
621
+ cluster. `hasAdSpend` is the same idea hard-coded to `$ad_spend`; this is the general form
622
+ and it does not use a Mixpanel reserved event name.
623
+
624
+ ```javascript
625
+ standaloneEvents: [
626
+ {
627
+ event: 'cdn_egress',
628
+ cadence: 'day', // 'hour' | 'day' | 'week' (default 'day')
629
+ dimensions: { region: ['us-east', 'us-west', 'eu'] }, // cross-producted
630
+ distinctIdFrom: 'region', // synthetic id, never a person
631
+ properties: {
632
+ gb_out: (ctx) => 400 + ctx.tickIndex * 3, // shape a trend across the window
633
+ cost_usd: (ctx) => (400 + ctx.tickIndex * 3) * 0.085,
634
+ p95_ms: [120, 140, 160], // same ValueValid forms as event props
635
+ },
636
+ },
637
+ {
638
+ event: 'billing_rollup',
639
+ cadence: 'week',
640
+ dimensions: { tier: ['free', 'pro', 'max'] },
641
+ properties: { mrr_usd: (ctx) => ..., churn_usd: (ctx) => ... },
642
+ },
643
+ ]
644
+ ```
645
+
646
+ the engine emits **one record per cadence tick per dimension cross-product row**. the
647
+ example above produces 3 records per day (`cdn_egress`) plus 3 records per week
648
+ (`billing_rollup`). ticks start at the dataset start and step by the cadence; the last tick
649
+ is the final one at or before the dataset end, so nothing lands in the future.
650
+
651
+ each record carries `event`, `time`, `insert_id`, `distinct_id`, every dimension as a flat
652
+ property, and every resolved entry in `properties`.
653
+
654
+ | field | behavior |
655
+ |---|---|
656
+ | `event` | required, unique across `standaloneEvents` |
657
+ | `cadence` | `'hour'`, `'day'`, or `'week'`. default `'day'` |
658
+ | `dimensions` | object of non-empty arrays, cross-producted. omit for one record per tick |
659
+ | `distinctIdFrom` | must name a declared dimension. omitted → `distinct_id` is the event name |
660
+ | `properties` | keys may not collide with a dimension or with `event`/`time`/`insert_id`/`distinct_id`/`user_id`/`device_id` |
661
+
662
+ property value functions receive a `StandaloneValueContext`: `{ time, config, dimensions,
663
+ tickIndex, tickCount, cadence, event }`. `tickIndex / (tickCount - 1)` is window progress —
664
+ use it to shape growth, a dip, or a spike, guarding `tickCount <= 1` before division.
665
+
666
+ the stream lands in `result.standaloneEventData`, writes to its own `-STANDALONE` file
667
+ shard, and imports to Mixpanel as its own event stream. hooks fire with type
668
+ `"standalone"`; `meta.spec` carries the resolved config so a hook can tell streams apart.
669
+ the hook runs before the user loop. return the record object or an array of records;
670
+ returning `undefined` drops the record. it has no person metadata and never enters
671
+ `everything`. warehouse hooks have a different contract: mutate the row in place;
672
+ their return values are ignored.
673
+
674
+ ```javascript
675
+ hook: (record, type, meta) => {
676
+ if (type === 'standalone' && meta.spec.event === 'cdn_egress' && record.region === 'us-east') {
677
+ record.p95_ms *= 40;
678
+ }
679
+ return record;
680
+ }
681
+ ```
682
+
683
+ validation throws rather than skipping. a malformed entry would silently drop a whole data
684
+ stream, and you would not notice until the charts were wrong.
685
+
686
+ ## warehouse metrics (local source tables)
687
+
688
+ `warehouseMetrics` materializes warehouse-ready tables from the run's own event
689
+ stream after user generation completes. use it when you need a bookings table, a
690
+ subscription level snapshot, or an ARR table that reads like a real warehouse
691
+ source. these rows land in `result.warehouseMetricData`, write to
692
+ `<name>-WAREHOUSE-<table>.csv|json`, and emit one manifest at
693
+ `<name>-WAREHOUSE-MANIFEST.json`.
694
+
695
+ they are **not** imported by `token`. that is deliberate. the live path is:
696
+
697
+ 1. run the dungeon
698
+ 2. review `/warehouse-metrics` in dry-run mode
699
+ 3. obtain explicit operator consent for live execution
700
+ 4. load the tables to bigquery and save the metrics there
701
+
702
+ live deploy uses `bq load --replace`, so it overwrites the destination warehouse
703
+ tables. the shipped script does not prompt on its own, so the operator or agent
704
+ must obtain explicit consent before running it in live mode. if the
705
+ warehouse CRUD docs route returns 404, the deploy still loads tables and connects
706
+ the source, then writes `warehouse/GAPS.md` for manual metric creation.
707
+
708
+ the manifest carries `recommendedAggregation: 'sum' | 'last value'`. the
709
+ Mixpanel warehouse metric API spells that second value as `last_value`; the
710
+ deploy flow maps it for you.
711
+
712
+ there is one real preview trap: `previewWarehouseMetric` rejects raw SQL
713
+ containing `DROP`, `DELETE`, `TRUNCATE`, `ALTER`, `CREATE`, `INSERT`, or
714
+ `UPDATE` as plain substrings. `created_at` trips `CREATE`; `updated_at` trips
715
+ `UPDATE`. aliasing only helps if the blocked text disappears from the query
716
+ entirely.
717
+
718
+ ### canonical shapes
719
+
720
+ additive daily bookings:
721
+
722
+ ```javascript
723
+ warehouseMetrics: [{
724
+ name: 'daily_new_bookings',
725
+ source: {
726
+ event: 'new_booking',
727
+ measure: 'sum',
728
+ property: 'booking_value',
729
+ },
730
+ timeColumn: 'date',
731
+ valueColumn: 'bookings',
732
+ }]
733
+ ```
734
+
735
+ point-in-time daily active subscriptions:
736
+
737
+ ```javascript
738
+ warehouseMetrics: [{
739
+ name: 'daily_active_subscriptions',
740
+ type: 'point-in-time',
741
+ source: {
742
+ event: 'subscription_started',
743
+ minus: 'subscription_cancelled',
744
+ measure: 'count',
745
+ },
746
+ baseline: 40,
747
+ timeColumn: 'date',
748
+ valueColumn: 'active_subscriptions',
749
+ }]
750
+ ```
751
+
752
+ sparse monthly ARR with backfill:
753
+
754
+ ```javascript
755
+ warehouseMetrics: [{
756
+ name: 'monthly_arr_snapshot',
757
+ type: 'point-in-time',
758
+ grain: 'month',
759
+ sparse: true,
760
+ history: 18,
761
+ source: {
762
+ event: 'subscription_started',
763
+ minus: 'subscription_cancelled',
764
+ measure: 'sum',
765
+ property: 'monthly_value',
766
+ },
767
+ baseline: 24000,
768
+ scale: 12,
769
+ timeColumn: 'month',
770
+ valueColumn: 'arr_usd',
771
+ }]
772
+ ```
773
+
774
+ the shipped technical fixture uses a 60-day live window plus 18 monthly backfill
775
+ buckets. sample row counts are illustrative only. `grain`, `history`, `sparse`,
776
+ and `groupBy` all change how many rows a table emits.
777
+
778
+ ### config surface
779
+
780
+ | key | default | range / contract |
781
+ |---|---|---|
782
+ | `name` | required | unique table / metric name, `/^[a-z][a-z0-9_]{0,63}$/` |
783
+ | `type` | `'additive'` | `'additive'` or `'point-in-time'` |
784
+ | `grain` | `'day'` | `'day'`, `'week'`, `'month'` |
785
+ | `sparse` | `false` | boolean, valid only with `type: 'point-in-time'` |
786
+ | `source.event` | required | string or string[] of declared source events |
787
+ | `source.minus` | `[]` | string or string[] of declared subtractive events |
788
+ | `source.measure` | `'count'` | `'count'`, `'sum'`, `'avg'`, `'dau'`, `'users'`; point-in-time forbids `'avg'` and `'dau'` |
789
+ | `source.property` | `null` | required for `'sum'` and `'avg'`; must be declared on every source event or in `superProps` |
790
+ | `source.where` | `null` | optional function over flat event rows |
791
+ | `source.groupBy` | `[]` | up to 2 keys, each declared on every source event or in `superProps`; observed cardinality above 50 warns |
792
+ | `timeColumn` | `'date'` | valid JS identifier; becomes the ordered time axis in rows and manifest |
793
+ | `valueColumn` | `'value'` | valid JS identifier |
794
+ | `baseline` | `0` | number `>= 0`; used only for point-in-time metrics, ignored on additive |
795
+ | `scale` | `1` | finite number `> 0`, applied after bucket aggregation |
796
+ | `noise` | `0` | finite number, clamped to `[0, 0.5]` with a warning |
797
+ | `history` | `0` | integer `>= 0`; warns above roughly 3 years at each grain (`1095` day, `156` week, `36` month) |
798
+ | `columns` | `{}` | extra declared output columns; keys must be valid identifiers and cannot collide with time/value/groupBy columns |
799
+ | `format` | dungeon `format`, else `'csv'` | `'csv'` or `'json'` |
800
+
801
+ materialized tables are deterministic at the same seed and do not perturb the
802
+ event stream. the warehouse pass runs after the user loop, so seeded noise and
803
+ derived columns never change generated events.
804
+
805
+ ### result and manifest
806
+
807
+ ```javascript
808
+ const result = await DUNGEON_MASTER(config);
809
+
810
+ result.warehouseMetricData.daily_new_bookings
811
+ result.warehouseManifest.tables
812
+ result.files
813
+ ```
814
+
815
+ each manifest table includes:
816
+
817
+ | field | meaning |
818
+ |---|---|
819
+ | `table` | warehouse table name |
820
+ | `file` | file prefix without extension |
821
+ | `format` | `'csv'` or `'json'` |
822
+ | `grain` | bucket grain |
823
+ | `type` | additive vs point-in-time |
824
+ | `timeColumn` | date axis column |
825
+ | `valueColumn` | numeric value column |
826
+ | `dimensionColumns` | copied `groupBy` keys |
827
+ | `columns` | ordered BigQuery schema (`DATE`, `FLOAT64`, `BOOL`, `STRING`) |
828
+ | `recommendedAggregation` | `'sum'` or `'last value'` |
829
+ | `sql` | `SELECT * FROM \`{{DATASET}}.<table>\` ORDER BY <timeColumn>` |
830
+ | `refreshHint` | currently `'hourly'` |
831
+
609
832
  ## user generation
610
833
 
611
834
  users are generated with configurable birth distributions, normally controlled via the `macro` preset (see "time shape" above). these three knobs can also be set directly on the dungeon config — they override the preset's values.
@@ -697,8 +920,11 @@ result.userProfilesData // user profiles
697
920
  result.scdTableData // SCD mutations
698
921
  result.groupProfilesData // group profiles
699
922
  result.adSpendData // ad spend data
923
+ result.standaloneEventData // identity-less event snapshots
700
924
  result.lookupTableData // lookup table entries
701
925
  result.mirrorEventData // mirror dataset
926
+ result.warehouseMetricData // warehouse tables keyed by metric name
927
+ result.warehouseManifest // warehouse table manifest
702
928
 
703
929
  result.eventCount // total event count
704
930
  result.userCount // total user count
@@ -899,6 +1125,8 @@ see [types.d.ts](types.d.ts) for the complete `Dungeon` interface. here are the
899
1125
  | `hasLocation` | boolean | false | include geo properties |
900
1126
  | `hasCampaigns` | boolean | false | include UTM properties |
901
1127
  | `hasAdSpend` | boolean | false | generate ad spend data |
1128
+ | `standaloneEvents` | array | `[]` | identity-less cadence streams that import as events |
1129
+ | `warehouseMetrics` | array | `[]` | local warehouse source tables + manifest, derived from generated events |
902
1130
  | `hasAnonIds` | boolean | false | generate anonymous IDs |
903
1131
  | `hasSessionIds` | boolean | false | generate session IDs |
904
1132
  | `alsoInferFunnels` | boolean | false | auto-generate funnels from events |