@ak--47/dungeon-master 1.8.0 → 1.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -169,6 +169,15 @@ its folder. Otherwise write it to the project root. (The generated
169
169
  6. **Assessment**: Overall quality judgment and recommendations
170
170
 
171
171
  ### Quality Criteria
172
+
173
+ These thresholds are diagnostic heuristics, not alignment acceptance criteria.
174
+ Interpret them against the configured macro, soup, resolved window, and intentional
175
+ hook effects. Read the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md)
176
+ for proof scope. Report insufficient observations separately from shape failures.
177
+ Include empty buckets and identify partial buckets before computing distribution
178
+ statistics; the SQL above reports observed buckets only. Sparse or partial buckets
179
+ alone do not establish a distribution defect.
180
+
172
181
  - **Daily CV**: 0.2-0.6 is ideal (some variation, not flat or spiky)
173
182
  - **Max-to-avg ratio**: < 2.0 at daily level, < 3.0 at hourly level
174
183
  - **Last day spike**: < 1.5x average = PASS, < 2x = WARN, > 2x = FAIL
@@ -35,11 +35,12 @@ In scope here:
35
35
  - `credentials: { token, region, serviceAccount, serviceSecret, projectId }`
36
36
  - `switches: { hasLocation, hasCampaigns, hasSessionIds, hasAvatar,
37
37
  hasIOSDevices, hasAndroidDevices, hasDesktopDevices, hasBrowser,
38
- isAnonymous, alsoInferFunnels, hasAdSpend, hasAttributionFlags }`
38
+ isAnonymous, alsoInferFunnels, hasAdSpend }`
39
39
  - `identity: { avgDevicePerUser, sessionTimeout }`
40
40
 
41
41
  Old top-level keys keep working (verbose warn nudges migration), but new
42
42
  dungeons should ship the sub-object shape.
43
+ `hasAttributionFlags` is derived from `events[].isAttributionEvent`; do not set it.
43
44
  - Surviving advanced entities: `personas`, `worldEvents`, `engagementDecay`,
44
45
  `dataQuality` — use sparingly
45
46
 
@@ -64,6 +65,7 @@ see `HOOKS.md` at the project root.
64
65
 
65
66
  Before writing any code, scan:
66
67
 
68
+ - [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md) - report definitions, neutral controls, and proof limits.
67
69
  - `types.d.ts` — the complete API reference. Every Dungeon field, EventConfig
68
70
  flag, Funnel option, AttemptsConfig, and Hook meta interface is documented
69
71
  with full JSDoc. **Treat this as the source of truth.**
@@ -234,8 +236,8 @@ When using experiments, include `$experiment_started` in the events array with
234
236
 
235
237
  - First funnel: includes `isFirstEvent` AND has `isAuthEvent: true` on the
236
238
  identity-transition step.
237
- - Usage funnels: ordinary sequences without `isFirstFunnel`. Optionally use
238
- `attempts` for repeat-usage modeling (abandon-cart pattern).
239
+ - Usage funnels: ordinary sequences without `isFirstFunnel`. Volume and funnel
240
+ selection control repetitions; `attempts` applies only to born-user first funnels.
239
241
  - Pick `conversionRate` between 30 and 80; `timeToConvert` in hours.
240
242
 
241
243
  Funnel `props` stamp constant properties on all events in that funnel run.
@@ -267,9 +269,9 @@ consumed standalone instances as funnel matches. Set `isStrictEvent: false`
267
269
  explicitly only when you intend mixed funnel/standalone semantics for that
268
270
  event.
269
271
 
270
- **Mark hook-readable funnel-step events with `isStrictEvent: false`.** When a hook reads `event === 'X'` and `X` is also a funnel step, the validator's auto-promote turns it into `isStrictEvent: true` and the engine stops emitting standalone occurrences — cohort goes empty. Identify these candidates at schema time so `write-hooks` doesn't have to re-thread the schema. Common candidates: login, page view, search, add to cart, swap, deposit — anything that's both a funnel step AND a recurring user behavior hooks will likely cohort on.
272
+ **Use `isStrictEvent: false` when a cohort needs standalone occurrences.** Auto-promotion removes standalone traffic but leaves funnel-generated occurrences readable. Declare the opt-out at schema time only when that extra traffic is part of the intended behavior; a hook reading an event name alone does not require it.
271
273
 
272
- **Funnels representing loops need `reentry: true`.** Any funnel named "X loop" / "X cycle" / "session" / per-instance recurring behavior must declare `reentry: true`. Without it, the engine emits one sequence per user and downstream "power user" / "daily active" cohorts have no behavioral signal to bin on.
274
+ **`reentry` is verifier-only.** Usage volume, funnel selection, and the event budget control generated repetitions. Set `reentry: true` only when the intended report counts repeated histories. It does not generate loops; local totals also default to `reentry: false`.
273
275
 
274
276
  **Structural trend engineering — duplicate funnels instead of reaching for
275
277
  hooks.** Not every trend needs a hook: initial conditions can raise or lower
@@ -294,20 +296,19 @@ to architect a comparison directly into the schema:
294
296
  Prefer structure when the story is a *between-path comparison* (this path
295
297
  converts worse / slower than that one). Reach for hooks when the story is a
296
298
  *within-cohort behavior* (these users do more of X over time, this segment's
297
- values differ). Structural trends are cheaper to verify — the knob IS the
298
- expected value.
299
+ values differ). Structural trends still need report-level proof: competing
300
+ histories, saturation, and finite budgets can change the observed effect.
299
301
 
300
302
  ### 3. SuperProps (2–3)
301
303
 
302
304
  Properties present on EVERY event. Common picks: `Plan`, `Region`, `Platform`,
303
- `App Version`. Values must come from same enumerations used in `userProps` for
304
- consistency.
305
+ `App Version`. Matching `userProps` enumerations only shares a value domain.
306
+ Use `stickyEventProps` for values that must match each user's profile.
305
307
 
306
308
  ### 4. UserProps (4–8)
307
309
 
308
- User profile properties. Set once per user. Use enumerations whose values
309
- match `superProps` for any overlapping keys (so per-event Region matches per-user
310
- Region).
310
+ User profile properties. Set once per user. Declare overlapping event keys in
311
+ `superProps` and use `stickyEventProps` when profile/event equality is required.
311
312
 
312
313
  ### 5. Groups (0–2)
313
314
 
@@ -417,9 +418,9 @@ Typical ranges:
417
418
 
418
419
  - Direct-acquisition first funnels: `{ min: 0, max: 0 }` (single attempt) or omit
419
420
  - Shared-link / friction-heavy onboarding: `{ min: 0, max: 2 }` (some retry)
420
- - Re-engagement / abandon-cart usage funnels: `{ min: 0, max: 3 }`
421
+ - Usage funnels do not consume this retry plan; model their repetitions through usage selection and volume.
421
422
 
422
- When set, `attempts.conversionRate` (optional) overrides `funnel.conversionRate`
423
+ For a born user's first funnel, `attempts.conversionRate` (optional) overrides `funnel.conversionRate`
423
424
  on the FINAL attempt. Failed prior attempts truncate before the first
424
425
  `isAuthEvent` step (no stitch fires for those attempts).
425
426
 
@@ -504,12 +505,12 @@ intra-day rhythm). Only override when you have a specific reason:
504
505
 
505
506
  ### Macro × born% / bias compatibility (strict clamps)
506
507
 
507
- When you set `macro` AND `percentUsersBornInDataset` explicitly, the validator
508
- clamps born% to the macro's preset value: flat=12, steady=12, growth=30,
509
- viral=55, decline=5. Same for `bornRecentBias` outside `[-0.5, 0.5]`. If you
510
- need higher born% (e.g., "this app launched mid-window every user is in the
511
- dataset"), switch macros first (flat growth viral) instead of pushing the
512
- preset's value. Setting born% without a macro keeps legacy behavior (no clamp).
508
+ The per-preset born% cap applies only to a named preset (`macro: 'growth'` or
509
+ `macro: { preset: 'growth', ... }`) with explicit `percentUsersBornInDataset`:
510
+ flat=12, steady=12, growth=30, viral=55, decline=5. A custom macro object without
511
+ `preset` has no per-preset cap; own and verify its shape. The global born% clamp
512
+ to [0,100] still applies. Explicit `bornRecentBias` is separately clamped to
513
+ [-0.5,0.5], with compound protection for high born% and bias. Preset bias is exempt.
513
514
 
514
515
  The clamp warning explains why and points to safe alternatives — read it.
515
516
 
@@ -574,18 +575,19 @@ Top-level optional knob. Shape retention via log-linear interpolation
574
575
  between waypoints. Independent of `engagementDecay`.
575
576
 
576
577
  ```js
577
- retentionCurve: [
578
- { day: 0, retention: 1.0 },
579
- { day: 1, retention: 0.80 },
580
- { day: 7, retention: 0.50 },
581
- { day: 30, retention: 0.20 },
582
- ]
578
+ retentionCurve: {
579
+ type: 'logarithmic',
580
+ day1: 0.80,
581
+ day7: 0.50,
582
+ day30: 0.20,
583
+ }
583
584
  ```
584
585
 
585
- Each born-in-dataset user's events get filtered based on the interpolated
586
- retention at the event's age-from-first-event-day. Use when you want a
587
- declarative retention shape at config level (analytical-style D1/D7/D30
588
- targets) instead of writing hook logic.
586
+ Use `type: 'logarithmic'` (default) or `'linear'`, with `dayN` active-day
587
+ weights; day 0 is implicitly 1. The curve weights the active-day plan and takes
588
+ precedence over `avgActiveDaysPerUser`. It does not directly filter each event
589
+ or guarantee report D1/D7/D30 percentages. Verify mature cohorts and account for
590
+ funnel spill and finite event budgets.
589
591
 
590
592
  ### `userSeed` (separate distinct_id RNG seed, v1.5+)
591
593
 
@@ -601,16 +603,15 @@ userSeed: "users-v1", // user-pool RNG (stable across versions)
601
603
 
602
604
  ## SuperProp consistency rule
603
605
 
604
- If `superProps` and `userProps` both define a property like `Plan`, the
605
- enumeration must match exactly. Otherwise users with `userProps.Plan = 'pro'`
606
- will fire events with `superProps.Plan = 'free'` — Mixpanel will see broken
607
- breakdowns.
606
+ Matching enumerations do not guarantee profile/event equality: independent draws
607
+ can differ. Declare both keys and project the profile values with `stickyEventProps`.
608
608
 
609
609
  ```js
610
610
  const PLANS = ["Free", "Free", "Free", "Pro", "Pro", "Enterprise"];
611
611
  // ...
612
612
  superProps: { Plan: PLANS, Region: REGIONS },
613
613
  userProps: { Plan: PLANS, Region: REGIONS, Role: ROLES, ... },
614
+ stickyEventProps: ['Plan', 'Region'],
614
615
  ```
615
616
 
616
617
  ## Verification
@@ -16,6 +16,12 @@ This is the step after a dungeon is authored, reviewed, and tweaked. It turns a
16
16
  local dungeon into one you can actually send to Mixpanel by creating the project
17
17
  and stamping `credentials` back into the file.
18
18
 
19
+ Follow the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md)
20
+ when carrying verification claims into business context. Preserve the actual report
21
+ definition and explicit counting options. Local evidence is selected source-derived
22
+ verification, not live Mixpanel parity. Provisioning is a separately authorized
23
+ handoff; never run it automatically during offline verification.
24
+
19
25
  ## What it does
20
26
 
21
27
  All work runs through the orchestrator `provision.mjs` (this skill's directory),
@@ -37,10 +37,14 @@ saved cohorts and custom properties, saved behaviors/metrics/formulas, and
37
37
  annotations that explain the engineered moments. Then it re-measures the hook
38
38
  stories **against the live project** and fails if they no longer read.
39
39
 
40
- That last step is the point. Local story verification passes on the generated
41
- array in memory; it never sees what survived ingest. A build can render five
40
+ Local story verification uses generated records in memory or retained disk
41
+ artifacts. It does not establish what survived ingest. A build can render five
42
42
  perfect dashboards on top of a story that silently collapsed on the way in.
43
43
 
44
+ Apply the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md)
45
+ when translating local stories into live reports. This build requires authorized
46
+ online work; it is never an automatic step of offline verification.
47
+
44
48
  ## Scope
45
49
 
46
50
  In scope: dashboards + charts, Lexicon enrichment, cohorts, custom properties,
@@ -182,13 +186,14 @@ size, require each bucket to be observed END TO END (`(b+1)*unit - 1` days of hi
182
186
  not `b*unit`), and cap the cohort date. Put both numbers on the board — the diluted
183
187
  one is what the chart shows, and naming why is a better demo than hiding it.
184
188
 
185
- **Time-to-convert is a MEAN, and means are tail-dominated.** Funnel frames carry
189
+ **These funnel response fields report means.** Funnel frames carry
186
190
  `avg_time` and `avg_time_from_start` (seconds; not monotonic across steps — each is
187
191
  over that step's own survivors). A dungeon knob expressed as a median ratio will not
188
192
  reproduce: Square's designed 13x median gap measured 1.95x as a mean over a 30-day
189
- window. Use the **speed curve** instead run the same funnel at 1/3/7/14-day
193
+ window. A **speed curve** can supplement the report: run the same funnel at 1/3/7/14-day
190
194
  conversion windows and read what share of each segment's eventual conversions had
191
- landed by then. Same effect, expressed in a statistic Mixpanel actually computes.
195
+ landed by then. Preserve the requested real report and its statistic. A speed
196
+ curve cannot silently replace its acceptance check.
192
197
 
193
198
  ### 3. Auto-detect the data window
194
199
 
@@ -260,10 +265,14 @@ filters have to move into `show[i].behavior.filters`.
260
265
  - **structure** — every registered entity still exists.
261
266
  - **stories** — re-measure each hook effect live and compare to the knob.
262
267
 
263
- Report MATCH / DIRECTIONAL / MISS per story and exit non-zero on any MISS.
264
- Use a wider tolerance than the dungeon's own ±10% bar (~25%): Mixpanel's cohort
265
- membership is computed over the whole window, not the dungeon's internal binning,
266
- so depth-band style cohorts will not line up exactly.
268
+ Report MATCH / DIRECTIONAL / MISS / INSUFFICIENT_EVIDENCE per story. Exit non-zero
269
+ on a MISS or unresolved evidence gap; insufficient eligible users or converters
270
+ are neither a pass nor a measured miss. Use each story's declared tolerance.
271
+ Document report-semantic differences separately rather than widening tolerances.
272
+ Preserve explicit counting options: local totals default to `reentry: false`.
273
+ Derive local sessions from the full user stream before HPC partitioning; local
274
+ defaults are UTC, 30-minute idle timeout, and 24-hour maximum. Project exclusions
275
+ and timezone variants remain outside the source-derived proof.
267
276
 
268
277
  ### 7. Report
269
278
 
@@ -313,8 +322,9 @@ every build. Prefer one report with N metrics wherever the chart allows it.
313
322
  **`ws._api_client` is lazy** — None immediately after construction. Force it via
314
323
  `ws._get_api_client()` before patching request headers.
315
324
 
316
- **Idempotency.** Dashboards: delete-by-title then recreate. Everything else:
317
- look up by name and reuse. Re-running must never duplicate.
325
+ **Idempotency.** Dashboards: create the replacement first, then retire only the
326
+ recorded old ids after confirmation. Everything else: look up by name and reuse.
327
+ Re-running must never duplicate.
318
328
 
319
329
  ## Commands
320
330
 
@@ -10,7 +10,9 @@ Base URL: `https://mixpanel-power-tools-api-lmozz6xkha-uc.a.run.app`
10
10
 
11
11
  ## Auth model — read this first
12
12
 
13
- - **GET any endpoint path = documentation, no auth.** Always `curl -s GET <base><path>` before first use of an unfamiliar endpoint — docs include exact body params and response shapes.
13
+ - **GET any endpoint path = documentation, no auth.** For authorized online work,
14
+ GET an unfamiliar endpoint's docs before use. During offline verification, use
15
+ local references and report unavailable documentation; make no network calls.
14
16
  - **POST = execute.** `Authorization: Bearer <oauth-token>` (employee OAuth from repo `.env` `BEARER_TOKEN`, or a customer's OAuth token) or `Basic base64(service_acct:secret)`.
15
17
  - **Customer OAuth tokens are accepted** (verified 2026-07-06): `/auth` and `/macro/get-schema` work with a customer token on projects that token can access. `ai_endpoints_allowed: false` for non-employees — the `ai-*` family stays employee-only.
16
18
  - **Every POST body** should include `client_id: "dungeon-master"` and `region` (`US` default).
@@ -62,7 +64,9 @@ warehouse skill's manual-setup fallback; other errors must surface.
62
64
  Its synthetic ids are series keys, so schema snapshots and dashboard counts must
63
65
  distinguish cadence telemetry from user `events[]`. It is not a warehouse source.
64
66
 
65
- GET the path for full docs. Full list: GET `/` and GET `/macro`.
67
+ During authorized online work, GET the path for full docs. Full list: GET `/`
68
+ and GET `/macro`. Preserve explicit report options when comparing local and live
69
+ queries; see the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md).
66
70
 
67
71
  **crud** — 187 endpoints. GET `/crud` for the full list.
68
72
 
@@ -44,6 +44,25 @@ a later success. Record exit codes, pass/skip counts, and the tested commit/diff
44
44
  The smoke script discovers vertical dungeons only. Verify changed technical or
45
45
  customer fixtures separately with the existing verification runners.
46
46
 
47
+ `npm test` excludes alignment. Run its independent gates separately:
48
+
49
+ ```bash
50
+ node tests/alignment/run.mjs
51
+ node tests/alignment/run.mjs --sweep --timeout-ms=600000
52
+ ```
53
+
54
+ Run the sweep when required by release scope. Both commands enforce macOS network
55
+ denial and a ten-minute maximum. If editor tasks are unavailable, use these
56
+ repository commands. If the runner or platform is unavailable, report NOT RUN;
57
+ do not substitute unsandboxed execution. Offline checks must use installed
58
+ dependencies, OS network denial, and empty global/setup-file lists. The default
59
+ suite above is not the offline audit route: do not run pruning setup, customer
60
+ evaluations, imports, or industry-generation scripts during an offline docs audit.
61
+ For a skills-only audit, run only `tests/e2e/skills-contract.test.js` with
62
+ `configFile: false`, that exact include, `globals: true`, `globalSetup: []`,
63
+ `setupFiles: []`, `fileParallelism: false`, and `sequence.concurrent: false`
64
+ through programmatic Vitest under `sandbox-exec` network denial.
65
+
47
66
  For generation changes, require the event-stream determinism test. Compare runs
48
67
  with the same seed, pinned window, and `concurrency: 1`; strip only `insert_id`.
49
68
  Use sequential test cases because the RNG is shared. Include warehouse noise in
@@ -59,12 +78,15 @@ warehouse determinism checks. Never accept an unrun or failing determinism gate.
59
78
  - Check README, HOOKS, type comments, changelog, and guide against actual code.
60
79
  Do not carry stale test counts, old skill paths, or historical operational
61
80
  failures forward as current release claims.
62
- - Follow the complete handoff: author, optional hooks, verify, provision,
81
+ - Audit the documented order of the complete handoff: author, optional hooks, verify, provision,
63
82
  generate/import, optional `/warehouse-metrics`, then headless build.
83
+ Offline release verification does not execute provisioning, imports, warehouse
84
+ deployment, or headless builds.
64
85
  - Distinguish user events, identity-less `standaloneEvents`, and warehouse rows.
65
86
  Synthetic IDs are not people. Standalone hooks return retained records;
66
87
  warehouse hooks mutate rows and ignore returns. Preserve deployment artifacts.
67
- - If API integrations changed, read the current endpoint docs and local handback.
88
+ - If API integrations changed, read local endpoint docs and handback evidence
89
+ during offline checks. Fetch endpoint docs only during authorized online work.
68
90
  Distinguish docs discovery, mocked execution, dry-run, and live verification.
69
91
  Live writes require explicit authorization. Never claim an endpoint GET proves
70
92
  authenticated write permission or end-to-end deployment.
@@ -85,6 +107,9 @@ whole under `plans/archived/`; leave active or ambiguous work in place.
85
107
 
86
108
  Report blockers first, then evidence, skipped checks, compatibility changes,
87
109
  and remaining operational limits. Do not fix unrelated failures or weaken tests.
110
+ Apply the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md):
111
+ report source-derived scope and remaining gaps, separate insufficient evidence
112
+ from passes and measured failures, and preserve actual deployment reports.
88
113
 
89
114
  Only when explicitly requested: inspect all staged files for secrets, commit the
90
115
  authorized changes, push the feature branch, create or reuse its PR, and inspect
@@ -8,7 +8,7 @@ effort: max
8
8
 
9
9
  # Verify Dungeon
10
10
 
11
- Verify a dungeon at full scale: run the story runner (`scripts/verify-stories.mjs`) as the primary mechanical check, validate schema integrity, investigate only what the runner can't settle (failures + `duckdb`-type assertions), write a single consolidated `hook-results.md`.
11
+ Verify a dungeon at its configured scale: run `scripts/verify-stories.mjs`, validate schema, and review report semantics and evidence sufficiency for every story. Investigate misses, bespoke SQL, and semantic gaps, then write one consolidated `hook-results.md`.
12
12
 
13
13
  **Dungeon file(s):** `$ARGUMENTS` — single path, multiple space-separated paths, or glob pattern. In batch mode, process each dungeon sequentially through Steps 1–5, then write one consolidated report in Step 7. Use a unique `name` prefix per dungeon (e.g., `verify-fintech`, `verify-gaming`) so output files don't collide.
14
14
 
@@ -16,6 +16,7 @@ Verify a dungeon at full scale: run the story runner (`scripts/verify-stories.mj
16
16
 
17
17
  Load these on demand:
18
18
 
19
+ - [references/alignment-contract.md](references/alignment-contract.md) - required 1.8.1 proof contract: independent report specification, neutral controls, sufficient populations, and source-derived limits.
19
20
  - [references/counting-semantics.md](references/counting-semantics.md) — Mixpanel counting rules, when to use the emulator vs DuckDB, emulator analysis types, identity-model verification, time-series, common gotchas.
20
21
  - [references/sql-recipes.md](references/sql-recipes.md) — every DuckDB query template (schema validation, identity / experiment invariants, hook archetype queries, pitfalls, TTC verification, dataset-window computation).
21
22
  - [references/report-format.md](references/report-format.md) — single + multi-dungeon report templates, per-hook detail block, verdict criteria, query log format, mandatory verification SQL file for user dungeons.
@@ -41,11 +42,18 @@ For each hook/pattern, catalog:
41
42
  - Which output file the signal lives in (user events, users, groups, standalone cadence shards, warehouse tables)
42
43
  - Mixpanel report instructions — flag missing/vague ones for the report
43
44
 
45
+ Before running, write the independent report specification from the intended
46
+ analysis, then compare the assertion's actual options and defaults with it.
47
+ Plan paired baselines and neutral controls using the shared contract.
48
+
44
49
  ### Step 2: Run the dungeon
45
50
 
46
51
  The runner already exists at `scripts/verify-runner.mjs`. Use it — do NOT recreate.
47
52
 
48
- **ALWAYS run at full fidelity. Never use `--small` for verification.** `--small` runs (1K users, 100K events) compress per-cohort populations and shift ratios within ±25%, hiding real bugs and flagging fake ones. They exist in the runner only as a developer-troubleshooting escape hatch.
53
+ **Run at full fidelity for acceptance.** `--small` is for troubleshooting only.
54
+ Configured scale does not guarantee sufficient eligible users or converters.
55
+ Report `INSUFFICIENT_EVIDENCE` when actual independent populations or observation
56
+ horizons cannot support the claim; do not classify sparse data as a measured miss.
49
57
 
50
58
  ```bash
51
59
  node scripts/verify-runner.mjs <dungeon-path> <run-name>
@@ -106,14 +114,19 @@ without those shards, run the same SQL directly and report that CLI limitation.
106
114
  A standalone-only config without stories does not receive an automatic cadence
107
115
  audit; perform the explicit schema, tick, dimension, and identity checks below.
108
116
 
109
- **Verdicts are computed, not judged.** They include the population floor: a cohort smaller than the assertion's `minCohort` caps at WEAK — a 12-user cohort can no longer score NAILED regardless of how clean its ratio looks. See [report-format.md "Verdict criteria"](references/report-format.md#verdict-criteria-5-tier) for the mechanical definitions.
117
+ **Verdicts are computed, not judged.** `minCohort` caps verdicts when supported
118
+ selected-row population fields fall below the floor. It does not validate every
119
+ denominator or establish full proof. Check actual eligible users/converters and
120
+ mature observation windows separately. See [report-format.md "Verdict criteria"](references/report-format.md#verdict-criteria-5-tier).
110
121
 
111
- **What the LLM investigates after this step — and nothing else:**
112
- 1. **Stories below STRONG** (WEAK / NONE / INVERSE) root-cause via Step 5's decision table. A miss means fixing the hook or the assertion's derivation, never relaxing the number to match output.
113
- 2. **`duckdb`-type assertions** the runner executes them but can't interpret bespoke shapes; sanity-check their output against the story narrative.
114
- 3. **Dungeons without a `stories` export** legacy fallback: full per-hook flow (Step 5) for every documented hook.
122
+ **What to review after this step:**
123
+ 1. **Every story, including passing targets** - check its options against the independent report specification and inspect neutral controls and population sufficiency. An inherited default can verify the wrong report.
124
+ 2. **Stories below STRONG** (WEAK / NONE / INVERSE) - root-cause via Step 5. Distinguish insufficient evidence from an effect failure; never relax a target to match output.
125
+ 3. **`duckdb`-type assertions** - the runner executes them but cannot validate the bespoke query's report semantics.
126
+ 4. **Dungeons without a `stories` export** - legacy fallback: Step 5 for every documented hook.
115
127
 
116
- Do NOT re-derive verdicts the runner already computed as passing. `hook-results.md` (Step 7) renders the runner's JSON.
128
+ Preserve computed verdicts in `hook-results.md`. Add separate semantic and evidence
129
+ statuses; a passing runner verdict alone does not establish acceptance.
117
130
 
118
131
  ### Step 4: Validate schema (BEFORE per-hook checks)
119
132
 
@@ -129,14 +142,18 @@ group keys, value column, and declared extras. Never apply user superProp or
129
142
  identity requirements to cadence records or warehouse rows.
130
143
 
131
144
  **Schema verdicts:**
132
- - **SCHEMA-PASS** added column appears on 100% of events of this type (uniform enrichment, acceptable)
133
- - **SCHEMA-FAIL** added column appears on <100% (flag stamping; conditional property creates inconsistent schema)
145
+ - **SCHEMA-PASS** - every output property is declared or a recognized enabled engine/SDK field for this stream.
146
+ - **SCHEMA-FAIL** - any undeclared property, even at 100% coverage. Uniform flag stamping still violates schema-first authorship.
147
+
148
+ If the runtime validator permits uniform enrichment, record that result separately
149
+ and enforce this stricter authorship gate explicitly.
134
150
 
135
151
  If any event type has SCHEMA-FAIL, flag it prominently in the report header with specific remediation: which hook line adds the property and how to remove it while preserving the intended pattern.
136
152
 
137
153
  ### Step 5: Investigate failures (and legacy no-stories dungeons)
138
154
 
139
- Applies only to the investigation targets from Step 3 failing stories, `duckdb`-type assertions, and dungeons with no `stories` export.
155
+ Applies to Step 3's targets: failed stories, semantic or evidence gaps (including
156
+ passing targets), bespoke SQL, and dungeons with no `stories` export.
140
157
 
141
158
  **Decision: emulator vs DuckDB**
142
159
 
@@ -167,7 +184,9 @@ The emulator now covers lifecycle, flows, sessions, event breakdowns, formulas,
167
184
 
168
185
  **Hand-written DuckDB funnel SQL diverges from Mixpanel — never hand-roll.** If you find yourself writing `WITH step1 AS ..., step2 AS ...` for a funnel, STOP — use `emulateBreakdown` with `funnelFrequency` instead.
169
186
 
170
- For emulator details, identity-model dungeons (must pass `profiles`), and time-series breakdown via `timeBucket`, see [counting-semantics.md](references/counting-semantics.md). For DuckDB query templates by hook archetype, pitfalls, and standard checks, see [sql-recipes.md](references/sql-recipes.md).
187
+ For emulator options, emitted-evidence identity maps, profile segments, and
188
+ time-series breakdown via `timeBucket`, see [counting-semantics.md](references/counting-semantics.md).
189
+ For DuckDB diagnostics and standard checks, see [sql-recipes.md](references/sql-recipes.md).
171
190
 
172
191
  **Always run for every dungeon** (before per-hook checks):
173
192
  - Standard identity-model invariants (stitch counts, pre-existing user stamping) when the dungeon uses the identity model
@@ -200,7 +219,7 @@ Format and conventions: see [report-format.md "Query log format"](references/rep
200
219
 
201
220
  ### Step 7: Write `hook-results.md`
202
221
 
203
- Write to `dungeons/user/<name>/hook-results.md` for a user dungeon, else `./research/hook-results.md`. For story-backed dungeons, **the report renders the runner's JSON** (`verify-stories.mjs --json`): the hook summary table comes straight from the runner's per-story verdicts (story id, hook, archetype, observed vs target, verdict), and detailed blocks exist only for the Step-3 investigation targets. Use the templates in [report-format.md](references/report-format.md):
222
+ Write to `dungeons/user/<name>/hook-results.md` for a user dungeon, else `./research/hook-results.md`. Preserve the runner's JSON verdicts alongside independent semantic and evidence statuses, eligible populations, neutral controls, and source-derived scope. Add details for Step 3's investigation targets. Use [report-format.md](references/report-format.md):
204
223
  - Single-dungeon report structure
205
224
  - Multi-dungeon report structure (when batch mode)
206
225
  - Per-hook detail block
@@ -0,0 +1,84 @@
1
+ # 1.8.1 verification contract
2
+
3
+ Use this contract when authoring, verifying, provisioning, or presenting a dungeon.
4
+ The [1.8.1 guide](../../../../docs/guides/1.8.1-upgrade-guide.md) describes the
5
+ release. The local repository's `tests/alignment/REPORT.md` and
6
+ `tests/alignment/API-COMPATIBILITY.md` retain the audit evidence; those test
7
+ artifacts are not shipped in npm.
8
+
9
+ ## Define the report before measuring
10
+
11
+ Write an independent report specification before choosing assertions. Record
12
+ events, filters, identity rules, cohort definition, denominator, statistic,
13
+ window, timezone, ordering, reentry, session settings, and property selection.
14
+ Compare every assertion with that specification, including passing assertions.
15
+ An inherited default can pass the wrong report. Do not replace funnel TTC with
16
+ a numeric property, a mean with a median, or conversion with event volume to
17
+ rescue a failed story. Supplementary diagnostics must keep their own labels.
18
+
19
+ Local checks prove selected source-derived contracts, not universal Mixpanel
20
+ parity. The audit read analytics source without compiling or executing it. It
21
+ did not run live differential queries. Macro/soup interactions, table surfaces,
22
+ arbitrary hooks, parallel execution, and several counting variants remain gaps.
23
+
24
+ ## Separate a measured effect from enough evidence
25
+
26
+ Retain the story runner's computed verdict. Report semantic correctness and
27
+ evidence sufficiency separately. Full-fidelity scale does not guarantee enough
28
+ independent eligible users, mature retention cohorts, or converters. Inspect the
29
+ actual denominator on each side; event count is not independent user count.
30
+ `minCohort` checks supported selected-row population fields, not every bespoke
31
+ denominator. Meeting it alone is not full proof. Label unresolved acceptance
32
+ `INSUFFICIENT_EVIDENCE`; do not count it as a pass or a measured effect failure.
33
+
34
+ For engineered effects, measure a paired baseline with the same seed, window,
35
+ and report options. Add a neutral control (factor one, zero injection, or hook
36
+ disabled) and an unaffected control cohort where applicable. A factor-one
37
+ control must be measured, not inferred from two differently modified groups.
38
+ Report baseline, treatment, eligible users/converters, and paired lift. Keep
39
+ targets and tolerances fixed before the run; disclose any revised criterion
40
+ and preserve its original failed evidence.
41
+
42
+ ## Preserve the local counting boundaries
43
+
44
+ - `Funnel.reentry` is verifier-only. Generated repetitions depend on usage
45
+ selection and event budget. Local `reentry` defaults to `false`, including
46
+ `countMode: 'totals'`; repeated histories require explicit `reentry: true`.
47
+ - Funnel histories include ordered restart/shared-edge behavior and inclusive
48
+ 2000ms completion grace. TTC uses completed histories under the requested
49
+ options, not independent first occurrences of each event. First/last segment
50
+ selection merges reached-step properties with defined fallback values;
51
+ explicit-step selection is separate.
52
+ - Derive sessions from the full resolved user stream before HPC partitioning.
53
+ Local defaults are UTC, a 30-minute idle timeout, and a 24-hour maximum.
54
+ `session_id` is a diagnostic or explicit legacy mode, not the default truth.
55
+ Non-UTC projects, list-valued HPC, and project-specific hidden/excluded session
56
+ events remain outside this proof.
57
+ - Profile device pools alone establish no identity mapping evidence. The public
58
+ `buildIdentityMap(profiles)` helper retains profile-based compatibility behavior.
59
+ For emitted-identity proof, derive links from valid ordinary both-ID events,
60
+ including later Login events, and pass that explicit `identityMap` to the
61
+ verifier. Earlier device-only rows can resolve retrospectively. Report ID
62
+ conflicts and validation limits; generator auth policy is not ingestion parity.
63
+ - `applyPathBias` is append-only. `share` selects eligible injection recipients;
64
+ it does not promise exact visible branch share. Measure appended payloads,
65
+ competing events, observed path share, and paired lift separately.
66
+ - `applySessionShape` preserves records. Pass known `datasetStart`/`datasetEnd`
67
+ bounds when clipping matters. Explicit bounds reject insufficient capacity
68
+ before mutation; unbounded placement can cross an unknown dataset end.
69
+ - `retentionCurve` weights active-day plans. Budget, funnel spill, and observation
70
+ horizon can prevent literal requested retention percentages. Usage anchors do
71
+ not accumulate previous funnel TTC; retries belong to born-user first funnels.
72
+
73
+ ## Keep schema and operational gates independent
74
+
75
+ Undeclared output properties fail schema review even at 100% coverage. Accept
76
+ only config-declared properties or recognized engine/SDK fields enabled for that
77
+ stream. A permissive runtime schema summary cannot override this authorship rule.
78
+ Keep user events, standalone cadence, and warehouse schemas separate.
79
+
80
+ Offline verification stops at local evidence and a deployment handoff. Do not
81
+ automatically provision, import, probe endpoints, deploy warehouse tables, or run
82
+ headless builds. Retain exact artifact prefixes and actual deployment reports.
83
+ Dry-run output is a plan, not evidence of a live outcome; preserve existing reports
84
+ before a dry run writes local files. Live work requires separate authorization.