@ak--47/dungeon-master 1.7.0 → 1.8.1

package/HOOKS.md

@@ -21,10 +21,12 @@ hook: function (record, type, meta) { ... return record; }
 | `event` | `events.js:176` | Single event (flat props) | **Used** (replaces event) | `user: { distinct_id }`, `config`, `datasetStart`, `datasetEnd` |
 | `funnel-post` | `funnels.js:153` | Array of funnel events | Ignored (mutate in-place) | `user`, `profile`, `scd`, `funnel`, `config`, `experiment` |
 | `everything` | `user-loop.js:280` | Array of ALL user events | **Used** if array returned | `profile`, `scd`, `config`, `datasetStart`, `datasetEnd`, `userIsBornInDataset`, `authTime`, `isPreAuth`, `persona` |
-| `ad-spend` | `storage.js` | Ad spend event | Ignored | -- |
-| `group` | `storage.js` | Group profile | Ignored | -- |
-| `mirror` | `storage.js` | Mirror data point | Ignored | -- |
-| `lookup` | `storage.js` | Lookup table entry | Ignored | -- |
+| `ad-spend` | `storage.js` | Ad spend event | Object or array used | -- |
+| `group` | `storage.js` | Group profile | Object or array used | -- |
+| `mirror` | `storage.js` | Mirror data point | Object or array used | -- |
+| `lookup` | `storage.js` | Lookup table entry | Object or array used | -- |
+| `standalone` | `storage.js`, before user loop | Identity-less cadence event | Object or array used; `undefined` drops | `spec`, `config`, `datasetStart`, `datasetEnd` |
+| `warehouse` | `storage.js` | One materialized warehouse row | Ignored | `spec`, `config`, `metricName`, `bucketIndex`, `bucketCount`, `grain`, `seriesKey`, `isBackfill`, `raw`, `datasetStart`, `datasetEnd` |
 
 **Per-user execution order:** `user` -> `scd-pre` -> `funnel-pre` -> `event` -> `funnel-post` -> `everything`
 
@@ -36,7 +38,105 @@ double-fire mutations.
 **Return rules:**
 - `event`: return the (possibly replaced) event object.
 - `everything`: return the (possibly modified) array. Filtered array removes events.
-- All other types: mutate `record` in-place. Return value is ignored.
+- `ad-spend`, `group`, `mirror`, `lookup`, `standalone`: return the record or an
+  array of records. Returning `undefined` drops the record.
+- `user`, `scd-pre`, `funnel-pre`, `funnel-post`, `warehouse`: mutate `record`
+  in place. Return value is ignored.
+
+`standaloneEvents` runs before the user loop; `warehouseMetrics` materializes
+after it. Neither hook receives person metadata or enters `everything`.
+Standalone synthetic `distinct_id` values identify series, never people. Use
+disk-backed `duckdb` assertions on `{{PREFIX}}-STANDALONE*.json` for cadence
+stories. Warehouse stories support `warehouse` and `warehouse-stats`
+assertions, with automatic table audits even when no stories are exported.
+
+### 1.1 Warehouse rows (`type === 'warehouse'`)
+
+`warehouse` fires once per materialized row, after the user loop and before the
+rows are written to disk. the row already matches the manifest contract:
+
+- `timeColumn`
+- every `source.groupBy` key, in order
+- `valueColumn`
+- every declared key in `columns`
+
+declare every key up front. warehouse containers are created with a fixed column
+list, and the manifest is built from that same list. an undeclared key is not
+part of the output contract even if it exists briefly in memory.
+
+`meta.seriesKey` is the joined group tuple in `source.groupBy` order, separated
+by `|`. examples:
+
+- no `groupBy` → `''`
+- `groupBy: ['region']` and `row.region === 'us'` → `'us'`
+- `groupBy: ['region', 'plan_tier']` and `row.region === 'us'`, `row.plan_tier === 'enterprise'` → `'us|enterprise'`
+
+`meta.bucketIndex` and `meta.bucketCount` are chronological and include history
+buckets even when `sparse: true` skips repeated rows. `meta.isBackfill` is true
+for the synthetic buckets created by `history`. `meta.raw` is the bucketed
+source truth before `scale`, `noise`, and point-in-time carry-forward.
+
+Treat the time axis as immutable. `row[spec.timeColumn]` drives ordering,
+manifest SQL, and warehouse verification. mutate the value column or declared
+extra columns instead.
+
+Recipe: scale a point-in-time level for an in-window story slice
+
+```js
+warehouseMetrics: [{
+  name: 'daily_active_subscriptions',
+  type: 'point-in-time',
+  source: { event: 'subscription_started', minus: 'subscription_cancelled', groupBy: 'region' },
+  baseline: 40,
+  timeColumn: 'date',
+  valueColumn: 'active_subscriptions',
+  columns: { lifted: false },
+}],
+
+hook: (row, type, meta) => {
+  if (type !== 'warehouse') return row;
+  if (meta.metricName !== 'daily_active_subscriptions') return row;
+  if (meta.isBackfill) return row;
+
+  const liveIndex = meta.bucketIndex - meta.spec.history;
+  if (meta.seriesKey === 'us' && liveIndex >= 7 && liveIndex < 14) {
+    row.active_subscriptions = Math.round(row.active_subscriptions * 1.2);
+    row.lifted = true;
+  }
+  return row;
+}
+```
+
+Recipe: apply an incident dip to one series only
+
+```js
+warehouseMetrics: [{
+  name: 'daily_new_bookings',
+  source: {
+    event: 'new_booking',
+    measure: 'sum',
+    property: 'booking_value',
+    groupBy: ['region', 'plan_tier'],
+  },
+  timeColumn: 'date',
+  valueColumn: 'bookings',
+  columns: { incident: false },
+}],
+
+hook: (row, type, meta) => {
+  if (type !== 'warehouse') return row;
+  if (meta.metricName !== 'daily_new_bookings') return row;
+  if (meta.isBackfill) return row;
+  if (meta.seriesKey !== 'us|enterprise') return row;
+
+  const liveIndex = meta.bucketIndex - meta.spec.history;
+  if (liveIndex >= 14 && liveIndex <= 16) {
+    row.bookings = Math.round(row.bookings * 0.35);
+    row.incident = true;
+  }
+  return row;
+}
+```
 
 **What 1.7.0 changed for hooks.** No hook signature, `meta` field, or firing
 order changed, and the hook-helper atoms and patterns are untouched. What a hook
@@ -437,12 +537,22 @@ enter on birth, or drop next-day spill in an `everything` hook.
 
 ### 2.8 Funnel reentry: state machine resets after completion
 
-Reference: `history.cpp` (`last_step_starts_next_funnel`). With reentry
-enabled, after the state machine reaches the final step the engine resets to
-step 0 and continues scanning. `result.completions` reports the total. In
-`countMode: 'totals'` the engine returns one `FunnelResult` per completion
-(simultaneous histories — one user, many funnel completions). Without
-reentry the funnel runs once per user.
+Reference: `history.cpp` (`history_is_mutable`) and `funnel_query.cpp`
+(shared first/last step handling). With `reentry: true`, completion absorbs
+events through the inclusive 2-second grace period. The next event beyond
+grace starts the next scan. Conversion-window expiry can restart earlier.
+`graceperiod: false` disables the completion wait.
+
+When an event records the ordered last step and also matches the ordered
+first step, it closes one attempt and anchors the next immediately. Both
+selectors must match. Any-order edges do not use this exception.
+`woRepeat` still restarts only at window expiry.
+
+`result.completions` reports repeat completions in uniques mode. In
+`countMode: 'totals'`, the engine returns one `FunnelResult` per attempt,
+including partial attempts. **Compatibility unchanged:** `reentry` defaults
+to `false`, even for totals. Totals alone does not enable analytics general
+counting's repeat-history behavior.
 
 ### 2.9 HPC (Hold Property Constant) — parallel sub-funnels
 
@@ -455,13 +565,24 @@ directly, or (v1.6) pass `holdPropertyConstant: '<prop>'` to the
 `funnelFrequency` emulator — it routes through the HPC engine and reports
 per-held-value sub-funnel counts.
 
+Session windows derive ordinals from the full user stream before HPC
+partitioning. Events with another held value can bridge a session but cannot
+fill steps in the current bucket. This applies to explicit session windows
+and `countMode: 'sessions'`. The local session defaults remain a 30-minute
+timeout, 24-hour maximum, and UTC day boundaries.
+
 ### 2.10 Funnel segment modes (FIRST_TOUCH / LAST_TOUCH / STEP)
 
 Reference: `options.hpp` `funnel_segment_mode`; `history.cpp`
 `property_set_buffer`. The engine snapshots the matched event's properties
-at every funnel step. Segmentation chooses which step's properties to use:
-FIRST_TOUCH (step 0), LAST_TOUCH (last reached), or STEP N (specific index).
-Enable with `evaluateFunnel({ trackStepProperties: true })`, then pick with
+at every reached position. FIRST_TOUCH and LAST_TOUCH merge those snapshots
+in recorded path order, including partial and any-order paths. The first
+or last defined non-null value wins, respectively. Undefined never replaces
+a defined value; null never replaces a defined non-null value. If only null
+and undefined are present, null wins. Snapshots remain unchanged.
+
+STEP N selects one reached position without merging fallback values.
+Enable with `evaluateFunnel(events, steps, { trackStepProperties: true })`, then pick with
 `resolveFunnelSegment(result, 'first' | 'last' | { step: N })`.
 
 ### 2.11 Engine-validation guarantees (v1.5+)
@@ -1743,8 +1864,9 @@ event set rather than one value moment.
 **Hook:** `everything`
 **Mixpanel report:** Flows — top paths after the anchor event show the engineered branch (Section 2.17)
 
-**In Mixpanel:** ~30% of users who view an item proceed straight down
-`add to cart → begin checkout`, making it the dominant Sankey branch.
+**In Mixpanel:** Bias the first branch toward `add to cart → begin checkout`.
+The helper selects ~30% of users for append-only injection. Existing traffic
+can interrupt the branch; the final branch share is not guaranteed to be 30%.
 
 ```js
 import { applyPathBias } from "@ak--47/dungeon-master/hook-helpers";
@@ -1789,6 +1911,8 @@ if (type === "everything") {
     sessionsPerWeek: 3,
     eventsPerSession: 5,
     sessionMinutes: 25,
+    datasetStart: meta.datasetStart,
+    datasetEnd: meta.datasetEnd,
   });
 }
 ```
@@ -1797,12 +1921,35 @@ if (type === "everything") {
 event set (after the `everything` hook), so wholesale timestamp rewrites no
 longer leave stale session labels. The atom keeps intra-session gaps well
 under the 30-min timeout (spacing capped at 20min + bounded jitter), keeps
-inter-session gaps well over it, and never crosses UTC midnight inside one
+inter-session gaps over it when explicit bounds are supplied, and never crosses UTC midnight inside one
 engineered session (the day-boundary split would cut it). Retiming only — no
 events are added or dropped, so total counts and event mixes are untouched.
 Session count follows `min(sessionsPerWeek × weeks, ceil(N /
 eventsPerSession))`: scarce users get fewer sessions, not fabricated events.
 
+`datasetStart` and `datasetEnd` are additive, optional arguments. Existing
+calls with neither bound keep the original full-UTC-day placement between
+the user's first and last active days. A two-event stream at 12:00/12:20 can
+still request two sessions with `eventsPerSession: 1` and `sessionMinutes: 5`.
+Legacy overfull requests also keep their old behavior: they do not throw,
+but their clusters can merge under the 30-minute timeout.
+
+The bounds accept ISO strings, unix seconds, or unix milliseconds. Hook
+metadata uses unix seconds and can be passed directly, as in the example.
+Either bound enables constrained placement; an omitted side uses the start
+of the first active UTC day or the end of the last active UTC day. The helper
+cannot infer `datasetEnd` from the last event. Pass known bounds when the
+dataset ends partway through an active day, including an inclusive midnight
+endpoint, to prevent later engine clipping.
+
+With explicit bounds, partial days compress clusters, including zero-duration
+clusters at midnight. If the requested sessions cannot fit inside a week's
+available day slices, the helper throws `RangeError` before changing any
+events. Invalid bounds also throw. It never silently reduces the target or
+drops records. Widen the allowed window or reduce the session target. Exact
+session separation assumes UTC and the default 30-minute timeout, without
+a maximum session duration.
+
 ---
 
 ## 5. Phase 3 Atom Reference
@@ -1833,7 +1980,7 @@ Import from `@ak--47/dungeon-master/hook-helpers`:
 | `splitByAuth` | identity | `(events, authTime) -> { preAuth, postAuth, stitch }` | Partition by auth boundary |
 | **`applyLifecycleWave`** | shape | `(events, uid, { dormantFromDay, dormantDays, resurrectBurst?, valueMomentEvent, dropAll? }) -> events[]` | Clean dormancy gap + resurrection burst; sweeps the ENTIRE window by timestamp (v1.6, recipe 4.29). Returns a NEW array |
 | **`applyPathBias`** | shape | `(events, uid, { anchor, path, share, gapSeconds? }) -> events[]` | Inject a Flows path after the user's first anchor for ~`share` (fraction) of users; skips users missing any step template (v1.6, recipe 4.30) |
-| **`applySessionShape`** | shape | `(events, uid, { sessionsPerWeek, eventsPerSession, sessionMinutes }) -> events[]` | Retime the stream into deterministic session clusters — intra-gaps ≪ 30min, inter-gaps ≫ 30min, never crosses UTC midnight (v1.6, recipe 4.31) |
+| **`applySessionShape`** | shape | `(events, uid, { sessionsPerWeek, eventsPerSession, sessionMinutes, datasetStart?, datasetEnd? }) -> events[]` | Preserve records; default legacy full-UTC-day placement. Optional bounds constrain placement and throw atomically on insufficient capacity (recipe 4.31) |
 
 **Inject atoms + v1.5:** the engine auto-sorts events by time after the
 `everything` hook (`autoSortAfterEverything: true` default — see Principle

package/README.md

@@ -91,6 +91,11 @@ const result = await DUNGEON_MASTER({
 console.log(result.importResults);
 ```
 
+`token` imports event-shaped streams only: events, users, groups, ad spend, and
+`standaloneEvents`. `warehouseMetrics` does **not** import through this path.
+warehouse tables are materialized locally and need a separate warehouse deploy
+step after the run.
+
 ## dungeons
 
 a dungeon is a javascript file that exports a configuration object. it defines your entire data model: events, funnels, user properties, group analytics, SCDs, and a hook function that engineers discoverable patterns into the data.
@@ -537,6 +542,8 @@ dungeon-master generates multiple data types that mirror a real analytics implem
 | SCDs | `scdProps` | slowly changing dimensions (subscription tier over time) |
 | lookup tables | `lookupTables` | dimension tables (product catalog, region mapping) |
 | ad spend | `hasAdSpend` | daily ad spend with impressions, clicks, cost metrics |
+| standalone events | `standaloneEvents` | identity-less metric snapshots on a cadence (infrastructure, finance, ops) |
+| warehouse metrics | `warehouseMetrics` | warehouse source tables derived from generated events, with a manifest for downstream deploy |
 | mirror datasets | `mirrorProps` | transformed copies of event data (A/B versions) |
 | organic text | `createTextGenerator` | reviews, support tickets, search queries, chat messages |
 
@@ -606,6 +613,222 @@ variant assignment is **sticky by default**: a deterministic hash of `user_id` +
 
 **the variant lands on the user profile (1.7.0).** every exposed user carries `"Experiment: <name>": "<variant>"` (e.g. `"Experiment: Checkout Redesign": "New Checkout"`), so the funnel breaks down by variant in Mixpanel with a user-property breakdown — no cohort built from the exposure event. stamped when the user is first exposed (respects `startDaysBeforeEnd`); never-exposed users carry nothing; the `user` hook fires before exposure and does not see it, the `everything` hook does. `stampProfile: false` turns it off; `sticky: false` implies off. measured: 0 mismatches between the profile value and the `Variant name` on 13,005 exposure events.
 
+## standalone events (identity-less metric snapshots)
+
+`standaloneEvents` generates records that describe a **system, not a person**. they carry
+no `user_id` and no `device_id`. use them for infrastructure, finance, and ops telemetry:
+daily CDN egress per region, weekly billing rollups per plan tier, hourly queue depth per
+cluster. `hasAdSpend` is the same idea hard-coded to `$ad_spend`; this is the general form
+and it does not use a Mixpanel reserved event name.
+
+```javascript
+standaloneEvents: [
+  {
+    event: 'cdn_egress',
+    cadence: 'day',                                       // 'hour' | 'day' | 'week' (default 'day')
+    dimensions: { region: ['us-east', 'us-west', 'eu'] }, // cross-producted
+    distinctIdFrom: 'region',                             // synthetic id, never a person
+    properties: {
+      gb_out:   (ctx) => 400 + ctx.tickIndex * 3,         // shape a trend across the window
+      cost_usd: (ctx) => (400 + ctx.tickIndex * 3) * 0.085,
+      p95_ms:   [120, 140, 160],                          // same ValueValid forms as event props
+    },
+  },
+  {
+    event: 'billing_rollup',
+    cadence: 'week',
+    dimensions: { tier: ['free', 'pro', 'max'] },
+    properties: { mrr_usd: (ctx) => ..., churn_usd: (ctx) => ... },
+  },
+]
+```
+
+the engine emits **one record per cadence tick per dimension cross-product row**. the
+example above produces 3 records per day (`cdn_egress`) plus 3 records per week
+(`billing_rollup`). ticks start at the dataset start and step by the cadence; the last tick
+is the final one at or before the dataset end, so nothing lands in the future.
+
+each record carries `event`, `time`, `insert_id`, `distinct_id`, every dimension as a flat
+property, and every resolved entry in `properties`.
+
+| field | behavior |
+|---|---|
+| `event` | required, unique across `standaloneEvents` |
+| `cadence` | `'hour'`, `'day'`, or `'week'`. default `'day'` |
+| `dimensions` | object of non-empty arrays, cross-producted. omit for one record per tick |
+| `distinctIdFrom` | must name a declared dimension. omitted → `distinct_id` is the event name |
+| `properties` | keys may not collide with a dimension or with `event`/`time`/`insert_id`/`distinct_id`/`user_id`/`device_id` |
+
+property value functions receive a `StandaloneValueContext`: `{ time, config, dimensions,
+tickIndex, tickCount, cadence, event }`. `tickIndex / (tickCount - 1)` is window progress —
+use it to shape growth, a dip, or a spike, guarding `tickCount <= 1` before division.
+
+the stream lands in `result.standaloneEventData`, writes to its own `-STANDALONE` file
+shard, and imports to Mixpanel as its own event stream. hooks fire with type
+`"standalone"`; `meta.spec` carries the resolved config so a hook can tell streams apart.
+the hook runs before the user loop. return the record object or an array of records;
+returning `undefined` drops the record. it has no person metadata and never enters
+`everything`. warehouse hooks have a different contract: mutate the row in place;
+their return values are ignored.
+
+```javascript
+hook: (record, type, meta) => {
+  if (type === 'standalone' && meta.spec.event === 'cdn_egress' && record.region === 'us-east') {
+    record.p95_ms *= 40;
+  }
+  return record;
+}
+```
+
+validation throws rather than skipping. a malformed entry would silently drop a whole data
+stream, and you would not notice until the charts were wrong.
+
+## warehouse metrics (local source tables)
+
+`warehouseMetrics` materializes warehouse-ready tables from the run's own event
+stream after user generation completes. use it when you need a bookings table, a
+subscription level snapshot, or an ARR table that reads like a real warehouse
+source. these rows land in `result.warehouseMetricData`, write to
+`<name>-WAREHOUSE-<table>.csv|json`, and emit one manifest at
+`<name>-WAREHOUSE-MANIFEST.json`.
+
+they are **not** imported by `token`. that is deliberate. the live path is:
+
+1. run the dungeon
+2. review `/warehouse-metrics` in dry-run mode
+3. obtain explicit operator consent for live execution
+4. load the tables to bigquery and save the metrics there
+
+live deploy uses `bq load --replace`, so it overwrites the destination warehouse
+tables. the shipped script does not prompt on its own, so the operator or agent
+must obtain explicit consent before running it in live mode. if the
+warehouse CRUD docs route returns 404, the deploy still loads tables and connects
+the source, then writes `warehouse/GAPS.md` for manual metric creation.
+
+the manifest carries `recommendedAggregation: 'sum' | 'last value'`. the
+Mixpanel warehouse metric API spells that second value as `last_value`; the
+deploy flow maps it for you.
+
+there is one real preview trap: `previewWarehouseMetric` rejects raw SQL
+containing `DROP`, `DELETE`, `TRUNCATE`, `ALTER`, `CREATE`, `INSERT`, or
+`UPDATE` as plain substrings. `created_at` trips `CREATE`; `updated_at` trips
+`UPDATE`. aliasing only helps if the blocked text disappears from the query
+entirely.
+
+### canonical shapes
+
+additive daily bookings:
+
+```javascript
+warehouseMetrics: [{
+  name: 'daily_new_bookings',
+  source: {
+    event: 'new_booking',
+    measure: 'sum',
+    property: 'booking_value',
+  },
+  timeColumn: 'date',
+  valueColumn: 'bookings',
+}]
+```
+
+point-in-time daily active subscriptions:
+
+```javascript
+warehouseMetrics: [{
+  name: 'daily_active_subscriptions',
+  type: 'point-in-time',
+  source: {
+    event: 'subscription_started',
+    minus: 'subscription_cancelled',
+    measure: 'count',
+  },
+  baseline: 40,
+  timeColumn: 'date',
+  valueColumn: 'active_subscriptions',
+}]
+```
+
+sparse monthly ARR with backfill:
+
+```javascript
+warehouseMetrics: [{
+  name: 'monthly_arr_snapshot',
+  type: 'point-in-time',
+  grain: 'month',
+  sparse: true,
+  history: 18,
+  source: {
+    event: 'subscription_started',
+    minus: 'subscription_cancelled',
+    measure: 'sum',
+    property: 'monthly_value',
+  },
+  baseline: 24000,
+  scale: 12,
+  timeColumn: 'month',
+  valueColumn: 'arr_usd',
+}]
+```
+
+the shipped technical fixture uses a 60-day live window plus 18 monthly backfill
+buckets. sample row counts are illustrative only. `grain`, `history`, `sparse`,
+and `groupBy` all change how many rows a table emits.
+
+### config surface
+
+| key | default | range / contract |
+|---|---|---|
+| `name` | required | unique table / metric name, `/^[a-z][a-z0-9_]{0,63}$/` |
+| `type` | `'additive'` | `'additive'` or `'point-in-time'` |
+| `grain` | `'day'` | `'day'`, `'week'`, `'month'` |
+| `sparse` | `false` | boolean, valid only with `type: 'point-in-time'` |
+| `source.event` | required | string or string[] of declared source events |
+| `source.minus` | `[]` | string or string[] of declared subtractive events |
+| `source.measure` | `'count'` | `'count'`, `'sum'`, `'avg'`, `'dau'`, `'users'`; point-in-time forbids `'avg'` and `'dau'` |
+| `source.property` | `null` | required for `'sum'` and `'avg'`; must be declared on every source event or in `superProps` |
+| `source.where` | `null` | optional function over flat event rows |
+| `source.groupBy` | `[]` | up to 2 keys, each declared on every source event or in `superProps`; observed cardinality above 50 warns |
+| `timeColumn` | `'date'` | valid JS identifier; becomes the ordered time axis in rows and manifest |
+| `valueColumn` | `'value'` | valid JS identifier |
+| `baseline` | `0` | number `>= 0`; used only for point-in-time metrics, ignored on additive |
+| `scale` | `1` | finite number `> 0`, applied after bucket aggregation |
+| `noise` | `0` | finite number, clamped to `[0, 0.5]` with a warning |
+| `history` | `0` | integer `>= 0`; warns above roughly 3 years at each grain (`1095` day, `156` week, `36` month) |
+| `columns` | `{}` | extra declared output columns; keys must be valid identifiers and cannot collide with time/value/groupBy columns |
+| `format` | dungeon `format`, else `'csv'` | `'csv'` or `'json'` |
+
+materialized tables are deterministic at the same seed and do not perturb the
+event stream. the warehouse pass runs after the user loop, so seeded noise and
+derived columns never change generated events.
+
+### result and manifest
+
+```javascript
+const result = await DUNGEON_MASTER(config);
+
+result.warehouseMetricData.daily_new_bookings
+result.warehouseManifest.tables
+result.files
+```
+
+each manifest table includes:
+
+| field | meaning |
+|---|---|
+| `table` | warehouse table name |
+| `file` | file prefix without extension |
+| `format` | `'csv'` or `'json'` |
+| `grain` | bucket grain |
+| `type` | additive vs point-in-time |
+| `timeColumn` | date axis column |
+| `valueColumn` | numeric value column |
+| `dimensionColumns` | copied `groupBy` keys |
+| `columns` | ordered BigQuery schema (`DATE`, `FLOAT64`, `BOOL`, `STRING`) |
+| `recommendedAggregation` | `'sum'` or `'last value'` |
+| `sql` | `SELECT * FROM \`{{DATASET}}.<table>\` ORDER BY <timeColumn>` |
+| `refreshHint` | currently `'hourly'` |
+
 ## user generation
 
 users are generated with configurable birth distributions, normally controlled via the `macro` preset (see "time shape" above). these three knobs can also be set directly on the dungeon config — they override the preset's values.
@@ -697,8 +920,11 @@ result.userProfilesData  // user profiles
 result.scdTableData      // SCD mutations
 result.groupProfilesData // group profiles
 result.adSpendData       // ad spend data
+result.standaloneEventData // identity-less event snapshots
 result.lookupTableData   // lookup table entries
 result.mirrorEventData   // mirror dataset
+result.warehouseMetricData // warehouse tables keyed by metric name
+result.warehouseManifest // warehouse table manifest
 
 result.eventCount        // total event count
 result.userCount         // total user count
@@ -731,7 +957,7 @@ styles: `support`, `review`, `search`, `feedback`, `chat`, `email`, `forum`, `co
 ## scripts
 
 ```bash
-npm test                      # full vitest test suite
+npm test                      # default unit/integration/e2e suite; prunes data/tmp
 npm run typecheck             # typescript check
 npm run dungeon:run           # run a dungeon file locally
 npm run dungeon:to-json       # convert JS dungeon to JSON (for UI import)
@@ -752,6 +978,8 @@ node scripts/verify-runner.mjs <path> [prefix]   # generate at full fidelity for
 
 ## tests
 
+1.8.1 compatibility and output changes: [upgrade guide](docs/guides/1.8.1-upgrade-guide.md).
+
 vitest tests live under `tests/` in three tiers:
 
 | dir | scope | wall time |
@@ -773,6 +1001,40 @@ npx vitest tests/unit                                       # watch mode
 
 `tests/e2e/engine-shape-full-sweep.test.js` skips itself unless `RUN_FULL_SWEEP=1` is set (it wraps the long-running 194-combo engine sweep).
 
+### editor and offline alignment tests
+
+VS Code discovers unit, integration, E2E, and alignment tests through one serial
+`vitest.editor.config.js`. The workspace disables Go test
+discovery and ignores the overlapping diagnostic Vitest configs. After changing
+these settings, run **Developer: Reload Window** if stale providers or test runs
+remain in the Testing panel. Editor runs omit the pruning setup, but they are not
+OS-sandboxed.
+
+Use **Tasks: Run Test Task** for `test: regression (offline, macOS)`, or choose
+the named alignment and sweep tasks from **Tasks: Run Task**. These test tasks
+never invoke the prune or dungeon-run tasks. Existing dungeon-run cleanup is
+unchanged and remains separate from testing.
+
+The `test: engine matrix`, `test: engine short sweep`, and `test: engine full sweep`
+tasks expose the direct-run engine checks. They are opt-in and use OS network denial.
+The full engine wrapper is visible under E2E but skipped until `RUN_FULL_SWEEP=1`.
+Direct-run `.mjs` scripts are not Vitest tests, so they do not get a separate
+`engine` folder in the Testing tree. The legacy engine sweeps do not use the
+alignment runner's ten-minute deadline. E2E tests may write files or perform
+network operations when run directly in the editor; editor execution is not an
+offline guarantee.
+
+```sh
+node tests/alignment/run.mjs                               # offline alignment gate
+node tests/alignment/run.mjs --sweep --timeout-ms=600000    # opt-in bounded sweep
+```
+
+Alignment is a separate test family, excluded from `npm test` but visible in the editor.
+Its runner enforces OS network denial on macOS, fails closed elsewhere, and kills
+workers at the ten-minute deadline. Tests and reports live in the source checkout;
+they are not included in the npm package. The default `npm test` and direct root
+Vitest commands still prune `data` and `tmp` through their global setup.
+
 ### engine tests (direct-run, NOT vitest)
 
 `tests/engine/` houses direct-run regression tests at scale. these are NOT vitest-compatible — invoke with `node` directly. used to catch engine regressions across a wide variety of dungeon configurations and for ad-hoc chart inspection. outputs land in `./tmp/` (gitignored).
@@ -899,6 +1161,8 @@ see [types.d.ts](types.d.ts) for the complete `Dungeon` interface. here are the
 | `hasLocation` | boolean | false | include geo properties |
 | `hasCampaigns` | boolean | false | include UTM properties |
 | `hasAdSpend` | boolean | false | generate ad spend data |
+| `standaloneEvents` | array | `[]` | identity-less cadence streams that import as events |
+| `warehouseMetrics` | array | `[]` | local warehouse source tables + manifest, derived from generated events |
 | `hasAnonIds` | boolean | false | generate anonymous IDs |
 | `hasSessionIds` | boolean | false | generate session IDs |
 | `alsoInferFunnels` | boolean | false | auto-generate funnels from events |