@ak--47/dungeon-master 1.7.0 → 1.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/analyze-soup/SKILL.md +30 -11
- package/.claude/skills/create-dungeon/SKILL.md +84 -44
- package/.claude/skills/create-project/SKILL.md +28 -3
- package/.claude/skills/create-project/context.mjs +89 -0
- package/.claude/skills/create-project/provision.mjs +1 -60
- package/.claude/skills/headless-build/SKILL.md +39 -12
- package/.claude/skills/powertools/SKILL.md +26 -3
- package/.claude/skills/release-check/SKILL.md +124 -0
- package/.claude/skills/verify-dungeon/SKILL.md +103 -29
- package/.claude/skills/verify-dungeon/references/alignment-contract.md +84 -0
- package/.claude/skills/verify-dungeon/references/counting-semantics.md +41 -16
- package/.claude/skills/verify-dungeon/references/report-format.md +41 -10
- package/.claude/skills/verify-dungeon/references/sql-recipes.md +171 -226
- package/.claude/skills/warehouse-metrics/GAPS-template.md +34 -0
- package/.claude/skills/warehouse-metrics/SKILL.md +111 -0
- package/.claude/skills/warehouse-metrics/deploy.mjs +651 -0
- package/.claude/skills/write-hooks/SKILL.md +94 -51
- package/CHANGELOG.md +183 -0
- package/HOOKS.md +165 -18
- package/README.md +265 -1
- package/docs/guides/1.8.0-upgrade-guide.md +151 -0
- package/docs/guides/1.8.1-upgrade-guide.md +153 -0
- package/dungeons/technical/warehouse.js +187 -0
- package/index.js +116 -2
- package/lib/core/config-validator.js +21 -0
- package/lib/core/dungeon-loader.js +1 -1
- package/lib/core/storage.js +51 -3
- package/lib/generators/events.js +6 -0
- package/lib/generators/funnels.js +15 -0
- package/lib/generators/standalone.js +248 -0
- package/lib/generators/warehouse.js +828 -0
- package/lib/hook-helpers/shape.js +73 -17
- package/lib/orchestrators/mixpanel-sender.js +27 -2
- package/lib/orchestrators/user-loop.js +83 -15
- package/lib/templates/story-spec.schema.json +41 -16
- package/lib/utils/utils.js +37 -12
- package/lib/verify/funnel-engine.js +66 -26
- package/lib/verify/index.js +1 -0
- package/lib/verify/story-runner.js +71 -8
- package/lib/verify/warehouse.js +683 -0
- package/package.json +4 -2
- package/scripts/verify-stories.mjs +150 -44
- package/types.d.ts +312 -9
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: headless-build
|
|
3
3
|
description: Use after a dungeon's data is loaded into a real Mixpanel project — builds the full demoable environment with mixpanel-headless (dashboards, charts, Lexicon, cohorts, custom properties, behaviors/metrics/formulas, annotations) targeted at that dungeon's engineered stories, then verifies the stories still read live. Final step after create-dungeon / write-hooks / verify-dungeon / create-project.
|
|
4
|
-
argument-hint: [dungeon path, e.g. dungeons/user/nyc-dcp/nyc-dcp.js]
|
|
4
|
+
argument-hint: '[dungeon path, e.g. dungeons/user/nyc-dcp/nyc-dcp.js]'
|
|
5
5
|
model: claude-opus-4-6
|
|
6
6
|
effort: max
|
|
7
7
|
---
|
|
@@ -14,16 +14,37 @@ Last step of the pipeline:
|
|
|
14
14
|
/create-dungeon → /write-hooks → /verify-dungeon → /create-project → /headless-build
|
|
15
15
|
```
|
|
16
16
|
|
|
17
|
+
For `warehouseMetrics`, insert `/warehouse-metrics` after verification and project
|
|
18
|
+
provisioning, before this build. Use the verified local disk artifact prefix.
|
|
19
|
+
Ordinary event import loads `standaloneEvents` but does not deploy warehouse tables.
|
|
20
|
+
|
|
21
|
+
Keep cadence and warehouse metrics separate from people analyses. Synthetic
|
|
22
|
+
standalone `distinct_id` values identify series; never count them as users or
|
|
23
|
+
include them in funnels, retention, or identity stitching. Build aggregate reports
|
|
24
|
+
from their declared numeric properties and dimensions.
|
|
25
|
+
|
|
26
|
+
For warehouse-backed charts, read the manifest and saved metric ids from the
|
|
27
|
+
warehouse handoff. `history` may extend the metric window before the user-event
|
|
28
|
+
window; preserve that range and label synthetic backfill. Check saved definitions
|
|
29
|
+
and `previewWarehouseMetric` results before narrating values. A successful
|
|
30
|
+
`refreshWarehouseMetric` only invalidates cache; it does not execute the query.
|
|
31
|
+
Use the existing `/macro/setup-bq-warehouse` flow through `/warehouse-metrics`
|
|
32
|
+
for source setup. Do not reimplement source creation or IAM grants in build code.
|
|
33
|
+
|
|
17
34
|
The project already has data. This skill builds everything a human sees: themed
|
|
18
35
|
dashboards whose narrative is computed from the live data, an annotated Lexicon,
|
|
19
36
|
saved cohorts and custom properties, saved behaviors/metrics/formulas, and
|
|
20
37
|
annotations that explain the engineered moments. Then it re-measures the hook
|
|
21
38
|
stories **against the live project** and fails if they no longer read.
|
|
22
39
|
|
|
23
|
-
|
|
24
|
-
|
|
40
|
+
Local story verification uses generated records in memory or retained disk
|
|
41
|
+
artifacts. It does not establish what survived ingest. A build can render five
|
|
25
42
|
perfect dashboards on top of a story that silently collapsed on the way in.
|
|
26
43
|
|
|
44
|
+
Apply the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md)
|
|
45
|
+
when translating local stories into live reports. This build requires authorized
|
|
46
|
+
online work; it is never an automatic step of offline verification.
|
|
47
|
+
|
|
27
48
|
## Scope
|
|
28
49
|
|
|
29
50
|
In scope: dashboards + charts, Lexicon enrichment, cohorts, custom properties,
|
|
@@ -165,13 +186,14 @@ size, require each bucket to be observed END TO END (`(b+1)*unit - 1` days of hi
|
|
|
165
186
|
not `b*unit`), and cap the cohort date. Put both numbers on the board — the diluted
|
|
166
187
|
one is what the chart shows, and naming why is a better demo than hiding it.
|
|
167
188
|
|
|
168
|
-
**
|
|
189
|
+
**These funnel response fields report means.** Funnel frames carry
|
|
169
190
|
`avg_time` and `avg_time_from_start` (seconds; not monotonic across steps — each is
|
|
170
191
|
over that step's own survivors). A dungeon knob expressed as a median ratio will not
|
|
171
192
|
reproduce: Square's designed 13x median gap measured 1.95x as a mean over a 30-day
|
|
172
|
-
window.
|
|
193
|
+
window. A **speed curve** can supplement the report: run the same funnel at 1/3/7/14-day
|
|
173
194
|
conversion windows and read what share of each segment's eventual conversions had
|
|
174
|
-
landed by then.
|
|
195
|
+
landed by then. Preserve the requested real report and its statistic. A speed
|
|
196
|
+
curve cannot silently replace its acceptance check.
|
|
175
197
|
|
|
176
198
|
### 3. Auto-detect the data window
|
|
177
199
|
|
|
@@ -243,10 +265,14 @@ filters have to move into `show[i].behavior.filters`.
|
|
|
243
265
|
- **structure** — every registered entity still exists.
|
|
244
266
|
- **stories** — re-measure each hook effect live and compare to the knob.
|
|
245
267
|
|
|
246
|
-
Report MATCH / DIRECTIONAL / MISS per story
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
268
|
+
Report MATCH / DIRECTIONAL / MISS / INSUFFICIENT_EVIDENCE per story. Exit non-zero
|
|
269
|
+
on a MISS or unresolved evidence gap; insufficient eligible users or converters
|
|
270
|
+
are neither a pass nor a measured miss. Use each story's declared tolerance.
|
|
271
|
+
Document report-semantic differences separately rather than widening tolerances.
|
|
272
|
+
Preserve explicit counting options: local totals default to `reentry: false`.
|
|
273
|
+
Derive local sessions from the full user stream before HPC partitioning; local
|
|
274
|
+
defaults are UTC, 30-minute idle timeout, and 24-hour maximum. Project exclusions
|
|
275
|
+
and timezone variants remain outside the source-derived proof.
|
|
250
276
|
|
|
251
277
|
### 7. Report
|
|
252
278
|
|
|
@@ -296,8 +322,9 @@ every build. Prefer one report with N metrics wherever the chart allows it.
|
|
|
296
322
|
**`ws._api_client` is lazy** — None immediately after construction. Force it via
|
|
297
323
|
`ws._get_api_client()` before patching request headers.
|
|
298
324
|
|
|
299
|
-
**Idempotency.** Dashboards:
|
|
300
|
-
look up by name and reuse.
|
|
325
|
+
**Idempotency.** Dashboards: create the replacement first, then retire only the
|
|
326
|
+
recorded old ids after confirmation. Everything else: look up by name and reuse.
|
|
327
|
+
Re-running must never duplicate.
|
|
301
328
|
|
|
302
329
|
## Commands
|
|
303
330
|
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: powertools
|
|
3
3
|
description: Use when any task needs the Mixpanel Power Tools API ("use powertools") — schema export (get-schema), event volumes, project CRUD, query methods, macros, or snapshotting a prod project's schema to copy it into a dungeon. Companion to create-project (which handles provisioning specifically).
|
|
4
|
-
argument-hint: [what to do, e.g. "get schema for project 12345" or "copy project 12345 into a dungeon"]
|
|
4
|
+
argument-hint: '[what to do, e.g. "get schema for project 12345" or "copy project 12345 into a dungeon"]'
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# Power Tools API
|
|
@@ -10,7 +10,9 @@ Base URL: `https://mixpanel-power-tools-api-lmozz6xkha-uc.a.run.app`
|
|
|
10
10
|
|
|
11
11
|
## Auth model — read this first
|
|
12
12
|
|
|
13
|
-
- **GET any endpoint path = documentation, no auth.**
|
|
13
|
+
- **GET any endpoint path = documentation, no auth.** For authorized online work,
|
|
14
|
+
GET an unfamiliar endpoint's docs before use. During offline verification, use
|
|
15
|
+
local references and report unavailable documentation; make no network calls.
|
|
14
16
|
- **POST = execute.** `Authorization: Bearer <oauth-token>` (employee OAuth from repo `.env` `BEARER_TOKEN`, or a customer's OAuth token) or `Basic base64(service_acct:secret)`.
|
|
15
17
|
- **Customer OAuth tokens are accepted** (verified 2026-07-06): `/auth` and `/macro/get-schema` work with a customer token on projects that token can access. `ai_endpoints_allowed: false` for non-employees — the `ai-*` family stays employee-only.
|
|
16
18
|
- **Every POST body** should include `client_id: "dungeon-master"` and `region` (`US` default).
|
|
@@ -43,7 +45,28 @@ node .claude/skills/powertools/snapshot-project.mjs <project_id> --bearer <token
|
|
|
43
45
|
|
|
44
46
|
## Endpoint catalog (the useful subset)
|
|
45
47
|
|
|
46
|
-
|
|
48
|
+
### Warehouse handoff (v1.8.0)
|
|
49
|
+
|
|
50
|
+
For a dungeon with `warehouseMetrics`, use `/warehouse-metrics` after local
|
|
51
|
+
verification produces disk tables and a manifest, and `/create-project` writes
|
|
52
|
+
the project id. Pass the explicit verified `--data-prefix`; preserve those files.
|
|
53
|
+
That skill owns load, source setup, SQL preview, and metric create-or-skip.
|
|
54
|
+
Reuse `/macro/setup-bq-warehouse` through `pt.mjs`; do not reimplement its
|
|
55
|
+
source creation, GCP IAM grant, or dataset ACL handling here.
|
|
56
|
+
|
|
57
|
+
Warehouse metric endpoints include `getWarehouseMetrics`, `previewWarehouseMetric`,
|
|
58
|
+
`createWarehouseMetric`, and `refreshWarehouseMetric` in the `/crud` family.
|
|
59
|
+
Read endpoint docs before live use. Preview executes the read query; create does
|
|
60
|
+
not validate SQL, and refresh only invalidates cache. A docs-route 404 uses the
|
|
61
|
+
warehouse skill's manual-setup fallback; other errors must surface.
|
|
62
|
+
|
|
63
|
+
`standaloneEvents` imports as a separate event stream through the ordinary sender.
|
|
64
|
+
Its synthetic ids are series keys, so schema snapshots and dashboard counts must
|
|
65
|
+
distinguish cadence telemetry from user `events[]`. It is not a warehouse source.
|
|
66
|
+
|
|
67
|
+
During authorized online work, GET the path for full docs. Full list: GET `/`
|
|
68
|
+
and GET `/macro`. Preserve explicit report options when comparing local and live
|
|
69
|
+
queries; see the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md).
|
|
47
70
|
|
|
48
71
|
**crud** — 187 endpoints. GET `/crud` for the full list.
|
|
49
72
|
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: release-check
|
|
3
|
+
description: "Use before a dungeon-master release to audit versioned docs, skills, tests, determinism, package contents, and Git state; report release blockers and prepare an explicitly authorized PR and merge handoff."
|
|
4
|
+
argument-hint: "[version, e.g. 1.8.0] [optional base branch]"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Release check
|
|
8
|
+
|
|
9
|
+
Check the current checkout before release. Read `AGENTS.md`, `package.json`,
|
|
10
|
+
the target changelog entry, and its upgrade guide. Default to validation only.
|
|
11
|
+
Never publish to npm unless the user gives explicit authorization for npm publishing.
|
|
12
|
+
An instruction to merge a PR is not permission to publish a package.
|
|
13
|
+
|
|
14
|
+
## Establish the release scope
|
|
15
|
+
|
|
16
|
+
1. Inspect `git status --short`, the current branch, remotes, and recent history.
|
|
17
|
+
Preserve unrelated changes. Never reset or clean the worktree to make checks pass.
|
|
18
|
+
2. Confirm the requested version matches `package.json`, the lockfile, changelog,
|
|
19
|
+
and guide. Do not invent a release date or bump a version without authorization.
|
|
20
|
+
3. Compare the release diff against the base branch. Check exports, dependencies,
|
|
21
|
+
config defaults, compatibility notes, examples, and migration instructions.
|
|
22
|
+
4. Inspect test setup before running it. This repo's suite prunes `data/` and `tmp/`.
|
|
23
|
+
If pending deployment artifacts exist, preserve them outside those directories
|
|
24
|
+
or obtain consent before running the suite. Do not silently delete verified inputs.
|
|
25
|
+
|
|
26
|
+
## Run executable gates
|
|
27
|
+
|
|
28
|
+
Use `set -o pipefail` for every piped command. Keep all Vitest output behind
|
|
29
|
+
`2>&1 | tail -50`; a successful tail command alone is not passing evidence.
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
set -o pipefail
|
|
33
|
+
npm test 2>&1 | tail -50
|
|
34
|
+
npm run typecheck
|
|
35
|
+
git diff --check
|
|
36
|
+
npx vitest run tests/unit/engine-shape-canary.test.js 2>&1 | tail -50
|
|
37
|
+
RUN_FULL_SWEEP=1 npx vitest run tests/e2e/engine-shape-full-sweep.test.js 2>&1 | tail -50
|
|
38
|
+
node tests/engine/smoke-test-all.mjs
|
|
39
|
+
npm pack --dry-run --json
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Run commands separately or chain with `&&` so a failure cannot disappear behind
|
|
43
|
+
a later success. Record exit codes, pass/skip counts, and the tested commit/diff.
|
|
44
|
+
The smoke script discovers vertical dungeons only. Verify changed technical or
|
|
45
|
+
customer fixtures separately with the existing verification runners.
|
|
46
|
+
|
|
47
|
+
`npm test` excludes alignment. Run its independent gates separately:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
node tests/alignment/run.mjs
|
|
51
|
+
node tests/alignment/run.mjs --sweep --timeout-ms=600000
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Run the sweep when required by release scope. Both commands enforce macOS network
|
|
55
|
+
denial and a ten-minute maximum. If editor tasks are unavailable, use these
|
|
56
|
+
repository commands. If the runner or platform is unavailable, report NOT RUN;
|
|
57
|
+
do not substitute unsandboxed execution. Offline checks must use installed
|
|
58
|
+
dependencies, OS network denial, and empty global/setup-file lists. The default
|
|
59
|
+
suite above is not the offline audit route: do not run pruning setup, customer
|
|
60
|
+
evaluations, imports, or industry-generation scripts during an offline docs audit.
|
|
61
|
+
For a skills-only audit, run only `tests/e2e/skills-contract.test.js` with
|
|
62
|
+
`configFile: false`, that exact include, `globals: true`, `globalSetup: []`,
|
|
63
|
+
`setupFiles: []`, `fileParallelism: false`, and `sequence.concurrent: false`
|
|
64
|
+
through programmatic Vitest under `sandbox-exec` network denial.
|
|
65
|
+
|
|
66
|
+
For generation changes, require the event-stream determinism test. Compare runs
|
|
67
|
+
with the same seed, pinned window, and `concurrency: 1`; strip only `insert_id`.
|
|
68
|
+
Use sequential test cases because the RNG is shared. Include warehouse noise in
|
|
69
|
+
warehouse determinism checks. Never accept an unrun or failing determinism gate.
|
|
70
|
+
|
|
71
|
+
## Audit docs and skills
|
|
72
|
+
|
|
73
|
+
- Run `tests/e2e/skills-contract.test.js`. Parse YAML frontmatter rather than
|
|
74
|
+
guessing from appearance: names match folders, descriptions are strings, and
|
|
75
|
+
`argument-hint` is a quoted string.
|
|
76
|
+
- The canonical skills live in `.claude/skills`. Verify `.agents/skills` and
|
|
77
|
+
`.github/skills` resolve to that directory. Do not duplicate skill content.
|
|
78
|
+
- Check README, HOOKS, type comments, changelog, and guide against actual code.
|
|
79
|
+
Do not carry stale test counts, old skill paths, or historical operational
|
|
80
|
+
failures forward as current release claims.
|
|
81
|
+
- Audit the documented order of the complete handoff: author, optional hooks, verify, provision,
|
|
82
|
+
generate/import, optional `/warehouse-metrics`, then headless build.
|
|
83
|
+
Offline release verification does not execute provisioning, imports, warehouse
|
|
84
|
+
deployment, or headless builds.
|
|
85
|
+
- Distinguish user events, identity-less `standaloneEvents`, and warehouse rows.
|
|
86
|
+
Synthetic IDs are not people. Standalone hooks return retained records;
|
|
87
|
+
warehouse hooks mutate rows and ignore returns. Preserve deployment artifacts.
|
|
88
|
+
- If API integrations changed, read local endpoint docs and handback evidence
|
|
89
|
+
during offline checks. Fetch endpoint docs only during authorized online work.
|
|
90
|
+
Distinguish docs discovery, mocked execution, dry-run, and live verification.
|
|
91
|
+
Live writes require explicit authorization. Never claim an endpoint GET proves
|
|
92
|
+
authenticated write permission or end-to-end deployment.
|
|
93
|
+
|
|
94
|
+
## Inspect the package and internal files
|
|
95
|
+
|
|
96
|
+
Read the file list from `npm pack --dry-run --json`. Confirm new runtime modules,
|
|
97
|
+
dependencies, scripts, and canonical skill files ship. Reject credentials,
|
|
98
|
+
customer data, `plans/`, `research/`, `.superpowers/`, or obsolete skill paths.
|
|
99
|
+
Check symlink handling in the package separately from workspace discovery.
|
|
100
|
+
|
|
101
|
+
Verify `git ls-files .superpowers` is empty and `.gitignore` covers the directory.
|
|
102
|
+
Ignoring an already tracked file does not untrack it. Obtain authorization before
|
|
103
|
+
removing index entries; preserve local reports. Archive completed plan folders
|
|
104
|
+
whole under `plans/archived/`; leave active or ambiguous work in place.
|
|
105
|
+
|
|
106
|
+
## Report and optional shipping handoff
|
|
107
|
+
|
|
108
|
+
Report blockers first, then evidence, skipped checks, compatibility changes,
|
|
109
|
+
and remaining operational limits. Do not fix unrelated failures or weaken tests.
|
|
110
|
+
Apply the [1.8.1 verification contract](../verify-dungeon/references/alignment-contract.md):
|
|
111
|
+
report source-derived scope and remaining gaps, separate insufficient evidence
|
|
112
|
+
from passes and measured failures, and preserve actual deployment reports.
|
|
113
|
+
|
|
114
|
+
Only when explicitly requested: inspect all staged files for secrets, commit the
|
|
115
|
+
authorized changes, push the feature branch, create or reuse its PR, and inspect
|
|
116
|
+
CI and mergeability. Require successful completed checks for the current PR head
|
|
117
|
+
before squash merge. If no CI checks are configured, report that explicitly and
|
|
118
|
+
use recorded local release gates; do not describe absent CI as passing. A failed
|
|
119
|
+
release gate requires investigation and an explicit operator decision before shipping.
|
|
120
|
+
Never bypass protections or use admin merge
|
|
121
|
+
to hide a failing check. Confirm the PR is merged before switching local branches.
|
|
122
|
+
Fetch, switch to `main`, and pull with `--ff-only`; stop rather than discard local
|
|
123
|
+
changes or divergent commits. Report the PR URL, merge SHA, current branch, and
|
|
124
|
+
working-tree status. Leave npm publishing to the operator unless separately authorized.
|
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: verify-dungeon
|
|
3
|
-
description: Use when a dungeon
|
|
4
|
-
argument-hint: [dungeon path(s), e.g. dungeons/vertical/gaming/gaming.js or dungeons/vertical/fintech/fintech.js]
|
|
3
|
+
description: Use when a dungeon needs verification before import or warehouse deployment, including standaloneEvents cadence streams and warehouseMetrics tables. Runs full-scale story checks, schema validation, and automatic warehouse audits even without stories. Reports per-hook NAILED/STRONG/WEAK/NONE/INVERSE verdicts; investigates failures and duckdb assertions.
|
|
4
|
+
argument-hint: '[dungeon path(s), e.g. dungeons/vertical/gaming/gaming.js or dungeons/vertical/fintech/fintech.js]'
|
|
5
5
|
model: claude-opus-4-6
|
|
6
6
|
effort: max
|
|
7
7
|
---
|
|
8
8
|
|
|
9
9
|
# Verify Dungeon
|
|
10
10
|
|
|
11
|
-
Verify a dungeon at
|
|
11
|
+
Verify a dungeon at its configured scale: run `scripts/verify-stories.mjs`, validate schema, and review report semantics and evidence sufficiency for every story. Investigate misses, bespoke SQL, and semantic gaps, then write one consolidated `hook-results.md`.
|
|
12
12
|
|
|
13
13
|
**Dungeon file(s):** `$ARGUMENTS` — single path, multiple space-separated paths, or glob pattern. In batch mode, process each dungeon sequentially through Steps 1–5, then write one consolidated report in Step 7. Use a unique `name` prefix per dungeon (e.g., `verify-fintech`, `verify-gaming`) so output files don't collide.
|
|
14
14
|
|
|
@@ -16,6 +16,7 @@ Verify a dungeon at full scale: run the story runner (`scripts/verify-stories.mj
|
|
|
16
16
|
|
|
17
17
|
Load these on demand:
|
|
18
18
|
|
|
19
|
+
- [references/alignment-contract.md](references/alignment-contract.md) - required 1.8.1 proof contract: independent report specification, neutral controls, sufficient populations, and source-derived limits.
|
|
19
20
|
- [references/counting-semantics.md](references/counting-semantics.md) — Mixpanel counting rules, when to use the emulator vs DuckDB, emulator analysis types, identity-model verification, time-series, common gotchas.
|
|
20
21
|
- [references/sql-recipes.md](references/sql-recipes.md) — every DuckDB query template (schema validation, identity / experiment invariants, hook archetype queries, pitfalls, TTC verification, dataset-window computation).
|
|
21
22
|
- [references/report-format.md](references/report-format.md) — single + multi-dungeon report templates, per-hook detail block, verdict criteria, query log format, mandatory verification SQL file for user dungeons.
|
|
@@ -35,17 +36,24 @@ Read the dungeon at `$ARGUMENTS`. If it's a bare filename (no `/`), check `dunge
|
|
|
35
36
|
|
|
36
37
|
For each hook/pattern, catalog:
|
|
37
38
|
- Hook number and name (e.g., "Hook #1: Compass users have 3x quest completion")
|
|
38
|
-
- Hook type (`event`, `everything`, `funnel-pre`, `funnel-post`, `user`, `scd-pre`)
|
|
39
|
+
- Hook type (`event`, `everything`, `funnel-pre`, `funnel-post`, `user`, `scd-pre`, `standalone`, `warehouse`)
|
|
39
40
|
- Mechanism — what the code actually does
|
|
40
41
|
- Expected signal — specific, measurable outcome (e.g., "compass_user=true events should have ~1.5x reward_gold")
|
|
41
|
-
- Which output file the signal lives in (events, users, groups)
|
|
42
|
+
- Which output file the signal lives in (user events, users, groups, standalone cadence shards, warehouse tables)
|
|
42
43
|
- Mixpanel report instructions — flag missing/vague ones for the report
|
|
43
44
|
|
|
45
|
+
Before running, write the independent report specification from the intended
|
|
46
|
+
analysis, then compare the assertion's actual options and defaults with it.
|
|
47
|
+
Plan paired baselines and neutral controls using the shared contract.
|
|
48
|
+
|
|
44
49
|
### Step 2: Run the dungeon
|
|
45
50
|
|
|
46
51
|
The runner already exists at `scripts/verify-runner.mjs`. Use it — do NOT recreate.
|
|
47
52
|
|
|
48
|
-
**
|
|
53
|
+
**Run at full fidelity for acceptance.** `--small` is for troubleshooting only.
|
|
54
|
+
Configured scale does not guarantee sufficient eligible users or converters.
|
|
55
|
+
Report `INSUFFICIENT_EVIDENCE` when actual independent populations or observation
|
|
56
|
+
horizons cannot support the claim; do not classify sparse data as a measured miss.
|
|
49
57
|
|
|
50
58
|
```bash
|
|
51
59
|
node scripts/verify-runner.mjs <dungeon-path> <run-name>
|
|
@@ -59,16 +67,25 @@ node scripts/verify-runner.mjs dungeons/vertical/gaming/gaming.js verify-gaming
|
|
|
59
67
|
Full-fidelity runs can take minutes (50K+ user dungeons). Plan accordingly — kick off the run, do other reading, return when the file lands. If a run takes longer than your budget, report that as a finding ("dungeon too large to verify in current session") rather than falling back to `--small`.
|
|
60
68
|
|
|
61
69
|
**Expected output files** (in `./data/`, prefixed by `<run-name>`):
|
|
62
|
-
- `<run-name>-EVENTS.json` —
|
|
70
|
+
- `<run-name>-EVENTS.json` — user events only (JSONL)
|
|
63
71
|
- `<run-name>-USERS.json` — user profiles
|
|
64
72
|
- `<run-name>-*-GROUPS.json` — group profiles (if dungeon has groups)
|
|
65
73
|
- `<run-name>-*-SCD.json` — SCD data (if dungeon has SCDs)
|
|
74
|
+
- `<run-name>-STANDALONE*.json` — `standaloneEvents` cadence records, separate from users
|
|
75
|
+
- `<run-name>-WAREHOUSE-<metric>.*` — `warehouseMetrics` tables; use manifest paths and formats
|
|
76
|
+
- `<run-name>-WAREHOUSE-MANIFEST.json` — table columns, grain, paths, SQL, and deployment metadata
|
|
77
|
+
|
|
78
|
+
Use local disk output, `format: 'json'`, `gzip: false`, and sending disabled
|
|
79
|
+
(`token: ''`) for this verification path. The existing runner supplies these
|
|
80
|
+
overrides. Record the exact prefix, for example `data/verify-gaming`, and pass
|
|
81
|
+
it explicitly to every downstream command. Preserve table files and their
|
|
82
|
+
matching manifest for `/warehouse-metrics`; do not regenerate silently.
|
|
66
83
|
|
|
67
84
|
For batched output (>2M events), the runner writes `<run-name>-EVENTS-part-*.json`. See [sql-recipes.md "Multi-part EVENTS files"](references/sql-recipes.md#multi-part-events-files-batch-mode).
|
|
68
85
|
|
|
69
86
|
### Step 3: Run the story runner (primary mechanical check)
|
|
70
87
|
|
|
71
|
-
|
|
88
|
+
Run this for any dungeon with a `stories` export or `warehouseMetrics`:
|
|
72
89
|
|
|
73
90
|
```bash
|
|
74
91
|
node scripts/verify-stories.mjs <dungeon-path> --data-prefix <run-name>
|
|
@@ -77,28 +94,66 @@ node scripts/verify-stories.mjs <dungeon-path> --data-prefix <run-name> --json
|
|
|
77
94
|
|
|
78
95
|
The runner streams the shards from Step 2, evaluates every assertion against its declared `target` / `floor` / `minCohort`, substitutes `{{PREFIX}}` into `duckdb`-type assertions and shells them out, enforces hook coverage (every numbered hook in the HOOK STORIES comment block must be targeted by at least one story), and prints a five-tier verdict table. Exit code is non-zero when any story lands WEAK / NONE / INVERSE or coverage is incomplete.
|
|
79
96
|
|
|
80
|
-
|
|
97
|
+
When the dungeon declares `warehouseMetrics`, the runner also performs an automatic warehouse shape audit even if the dungeon exports no `stories`: it checks declared-column integrity, dense-gap absence, monotonic time ordering, empty numeric cells, sparse repeated-value suppression, and row-count sanity against the dataset window. Audit failures are reported alongside story verdicts and fail the CLI.
|
|
98
|
+
|
|
99
|
+
Warehouse stories support `warehouse` row assertions and `warehouse-stats`
|
|
100
|
+
summary assertions in the story schema. Automatic audits do not replace engineered story
|
|
101
|
+
assertions, and an audit-only dungeon must not be reported as having passing stories.
|
|
102
|
+
|
|
103
|
+
If a request calls these checks `rawStats`, translate that intent into supported
|
|
104
|
+
`warehouse-stats` fields or disk DuckDB. There is no `rawStats` breakdown type
|
|
105
|
+
in the local story schema or dispatcher. Inspect `lib/verify/warehouse.js` and
|
|
106
|
+
`lib/verify/story-runner.js` for the available summary fields; never invent one.
|
|
107
|
+
|
|
108
|
+
For `standaloneEvents`, use disk-backed `duckdb` assertions with
|
|
109
|
+
`read_json_auto('{{PREFIX}}-STANDALONE*.json', union_by_name=true, sample_size=-1)`.
|
|
110
|
+
The CLI's emulator input contains user EVENTS shards only. `--in-memory` skips
|
|
111
|
+
disk-only DuckDB assertions; it cannot prove standalone stories. The disk CLI
|
|
112
|
+
also requires nonempty user EVENTS shards. For a standalone-only artifact set
|
|
113
|
+
without those shards, run the same SQL directly and report that CLI limitation.
|
|
114
|
+
A standalone-only config without stories does not receive an automatic cadence
|
|
115
|
+
audit; perform the explicit schema, tick, dimension, and identity checks below.
|
|
116
|
+
|
|
117
|
+
**Verdicts are computed, not judged.** `minCohort` caps verdicts when supported
|
|
118
|
+
selected-row population fields fall below the floor. It does not validate every
|
|
119
|
+
denominator or establish full proof. Check actual eligible users/converters and
|
|
120
|
+
mature observation windows separately. See [report-format.md "Verdict criteria"](references/report-format.md#verdict-criteria-5-tier).
|
|
81
121
|
|
|
82
|
-
**What
|
|
83
|
-
1. **
|
|
84
|
-
2.
|
|
85
|
-
3. **
|
|
122
|
+
**What to review after this step:**
|
|
123
|
+
1. **Every story, including passing targets** - check its options against the independent report specification and inspect neutral controls and population sufficiency. An inherited default can verify the wrong report.
|
|
124
|
+
2. **Stories below STRONG** (WEAK / NONE / INVERSE) - root-cause via Step 5. Distinguish insufficient evidence from an effect failure; never relax a target to match output.
|
|
125
|
+
3. **`duckdb`-type assertions** - the runner executes them but cannot validate the bespoke query's report semantics.
|
|
126
|
+
4. **Dungeons without a `stories` export** - legacy fallback: Step 5 for every documented hook.
|
|
86
127
|
|
|
87
|
-
|
|
128
|
+
Preserve computed verdicts in `hook-results.md`. Add separate semantic and evidence
|
|
129
|
+
statuses; a passing runner verdict alone does not establish acceptance.
|
|
88
130
|
|
|
89
131
|
### Step 4: Validate schema (BEFORE per-hook checks)
|
|
90
132
|
|
|
91
133
|
Catches hooks that introduce undeclared columns (flag stamping). For each unique event type, compare actual columns against config-declared properties. See [sql-recipes.md "Schema validation queries"](references/sql-recipes.md#schema-validation-queries) for the SQL and the expected-schema source table.
|
|
92
134
|
|
|
135
|
+
Keep schemas separate. User-event checks use `events[]`, superProps, and enabled
|
|
136
|
+
SDK fields. Standalone checks use only core `event`, `time`, `insert_id`,
|
|
137
|
+
`distinct_id`, plus that spec's dimensions and properties. Require no `user_id`
|
|
138
|
+
or `device_id`; synthetic ids never count as people. Check one row per cadence
|
|
139
|
+
tick and dimension tuple unless a documented hook intentionally changes it.
|
|
140
|
+
Warehouse checks use the manifest's fixed columns and the metric's time column,
|
|
141
|
+
group keys, value column, and declared extras. Never apply user superProp or
|
|
142
|
+
identity requirements to cadence records or warehouse rows.
|
|
143
|
+
|
|
93
144
|
**Schema verdicts:**
|
|
94
|
-
- **SCHEMA-PASS**
|
|
95
|
-
- **SCHEMA-FAIL**
|
|
145
|
+
- **SCHEMA-PASS** - every output property is declared or a recognized enabled engine/SDK field for this stream.
|
|
146
|
+
- **SCHEMA-FAIL** - any undeclared property, even at 100% coverage. Uniform flag stamping still violates schema-first authorship.
|
|
147
|
+
|
|
148
|
+
If the runtime validator permits uniform enrichment, record that result separately
|
|
149
|
+
and enforce this stricter authorship gate explicitly.
|
|
96
150
|
|
|
97
151
|
If any event type has SCHEMA-FAIL, flag it prominently in the report header with specific remediation: which hook line adds the property and how to remove it while preserving the intended pattern.
|
|
98
152
|
|
|
99
153
|
### Step 5: Investigate failures (and legacy no-stories dungeons)
|
|
100
154
|
|
|
101
|
-
Applies
|
|
155
|
+
Applies to Step 3's targets: failed stories, semantic or evidence gaps (including
|
|
156
|
+
passing targets), bespoke SQL, and dungeons with no `stories` export.
|
|
102
157
|
|
|
103
158
|
**Decision: emulator vs DuckDB**
|
|
104
159
|
|
|
@@ -121,12 +176,17 @@ Applies only to the investigation targets from Step 3 — failing stories, `duck
|
|
|
121
176
|
| Identity-model invariants | DuckDB |
|
|
122
177
|
| Experiment invariants | DuckDB |
|
|
123
178
|
| True bespokes (no emulator analysis fits) | DuckDB |
|
|
179
|
+
| Standalone cadence values, dimensions, tick counts | Disk DuckDB on `{{PREFIX}}-STANDALONE*.json` |
|
|
180
|
+
| Warehouse value stories | `warehouse` or `warehouse-stats` assertion |
|
|
181
|
+
| Warehouse schema, gaps, ordering, numeric cells | Automatic warehouse audit, with or without stories |
|
|
124
182
|
|
|
125
183
|
The emulator now covers lifecycle, flows, sessions, event breakdowns, formulas, and compounded retention — DuckDB's remit is schema / identity / experiment invariants plus true bespoke shapes. If a "bespoke" check is really a funnel, frequency, path, or breakdown in disguise, it belongs in the emulator.
|
|
126
184
|
|
|
127
185
|
**Hand-written DuckDB funnel SQL diverges from Mixpanel — never hand-roll.** If you find yourself writing `WITH step1 AS ..., step2 AS ...` for a funnel, STOP — use `emulateBreakdown` with `funnelFrequency` instead.
|
|
128
186
|
|
|
129
|
-
For emulator
|
|
187
|
+
For emulator options, emitted-evidence identity maps, profile segments, and
|
|
188
|
+
time-series breakdown via `timeBucket`, see [counting-semantics.md](references/counting-semantics.md).
|
|
189
|
+
For DuckDB diagnostics and standard checks, see [sql-recipes.md](references/sql-recipes.md).
|
|
130
190
|
|
|
131
191
|
**Always run for every dungeon** (before per-hook checks):
|
|
132
192
|
- Standard identity-model invariants (stitch counts, pre-existing user stamping) when the dungeon uses the identity model
|
|
@@ -142,9 +202,9 @@ generated artifacts into `dungeons/user/<name>/`:
|
|
|
142
202
|
- `hook-query-log.txt` (Step 6)
|
|
143
203
|
- `<name>-verifications.sql` (Step 7b)
|
|
144
204
|
|
|
145
|
-
The
|
|
146
|
-
|
|
147
|
-
|
|
205
|
+
The exception is run data under `./data/`, including cadence shards, warehouse
|
|
206
|
+
tables, and the manifest. Keep it under its explicit prefix until verification
|
|
207
|
+
and deployment finish. Step 8 requires consent before any scoped cleanup.
|
|
148
208
|
|
|
149
209
|
For non-user dungeons (technical/vertical) or batch runs across many dungeons,
|
|
150
210
|
fall back to `./research/` for `hook-results.md` / `hook-query-log.txt`.
|
|
@@ -159,7 +219,7 @@ Format and conventions: see [report-format.md "Query log format"](references/rep
|
|
|
159
219
|
|
|
160
220
|
### Step 7: Write `hook-results.md`
|
|
161
221
|
|
|
162
|
-
Write to `dungeons/user/<name>/hook-results.md` for a user dungeon, else `./research/hook-results.md`.
|
|
222
|
+
Write to `dungeons/user/<name>/hook-results.md` for a user dungeon, else `./research/hook-results.md`. Preserve the runner's JSON verdicts alongside independent semantic and evidence statuses, eligible populations, neutral controls, and source-derived scope. Add details for Step 3's investigation targets. Use [report-format.md](references/report-format.md):
|
|
163
223
|
- Single-dungeon report structure
|
|
164
224
|
- Multi-dungeon report structure (when batch mode)
|
|
165
225
|
- Per-hook detail block
|
|
@@ -171,22 +231,25 @@ Write to `dungeons/user/<name>/hook-results.md` for a user dungeon, else `./rese
|
|
|
171
231
|
|
|
172
232
|
When verifying a dungeon in `dungeons/user/`, also write a standalone DuckDB SQL file alongside the dungeon in its folder at `dungeons/user/<name>/<name>-verifications.sql`. Vertical dungeons already have their SQL co-located at `dungeons/vertical/<name>/<name>.sql`. Format: see [report-format.md "Verification SQL file"](references/report-format.md#verification-sql-file-mandatory-for-user-dungeons).
|
|
173
233
|
|
|
174
|
-
### Step 8:
|
|
234
|
+
### Step 8: Preserve artifacts and hand off
|
|
175
235
|
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
236
|
+
Record the exact verified prefix and retained files in the report. If warehouse
|
|
237
|
+
tables are present, provision with `/create-project`, then hand off to
|
|
238
|
+
`/warehouse-metrics` using `--data-prefix <verified-prefix>`. Provisioning and
|
|
239
|
+
ordinary event import do not deploy these tables.
|
|
179
240
|
|
|
180
|
-
|
|
241
|
+
Do not run blanket prune or delete `verify-*` globs. Only after deployment is
|
|
242
|
+
complete, list files for this exact run and obtain user consent for scoped
|
|
243
|
+
cleanup. Keep reports and reproducible SQL. Do not delete files during an audit.
|
|
181
244
|
|
|
182
245
|
## Hook execution model
|
|
183
246
|
|
|
184
247
|
Per user, hooks fire in this order:
|
|
185
248
|
|
|
186
249
|
1. `"user"` — profile created (mutate in-place; return ignored)
|
|
187
|
-
2. `"scd-pre"` — SCD entries created (mutate in-place
|
|
250
|
+
2. `"scd-pre"` — SCD entries created (mutate in-place; return ignored)
|
|
188
251
|
3. For each funnel: `"funnel-pre"` → `"event"` (per step) → `"funnel-post"`
|
|
189
|
-
4. `"event"` — for non-funnel
|
|
252
|
+
4. `"event"` — for non-funnel user events from `events[]` (return value REPLACES the event)
|
|
190
253
|
5. `"everything"` — array of ALL the user's events (return array to replace)
|
|
191
254
|
6. **Storage phase** — data written to disk. Hooks for `event`, `user`, `scd` do NOT re-fire (already applied above). Hooks for `mirror`, `ad-spend`, `group`, `lookup` fire only in storage.
|
|
192
255
|
|
|
@@ -195,6 +258,15 @@ Return-value behavior:
|
|
|
195
258
|
- `everything` hook: return value IS used if it's an array (replaces event list)
|
|
196
259
|
- `user`, `scd-pre`, `funnel-post`: return value IGNORED — only in-place mutations work
|
|
197
260
|
- `funnel-pre`: return value IGNORED — mutate the `record` object in-place (e.g., `record.conversionRate = 0.9`)
|
|
261
|
+
- `standalone`: runs before the user loop. Return an object or array; `undefined`
|
|
262
|
+
drops the record. `meta.spec` and `meta.config` describe the cadence stream.
|
|
263
|
+
- `warehouse`: runs after the user loop. Mutate the row; return value is ignored.
|
|
264
|
+
Meta includes `spec`, `config`, `metricName`, bucket fields, `grain`,
|
|
265
|
+
`seriesKey`, `isBackfill`, and `raw` source aggregates.
|
|
266
|
+
|
|
267
|
+
Neither `standalone` nor `warehouse` receives person metadata or passes through
|
|
268
|
+
`everything`. Their generation phases differ even though both hooks fire on
|
|
269
|
+
storage push. Other storage-only types return objects or arrays to retain rows.
|
|
198
270
|
|
|
199
271
|
## Final output
|
|
200
272
|
|
|
@@ -204,5 +276,7 @@ Tell the user:
|
|
|
204
276
|
3. Query log path (if written): alongside the report (`dungeons/user/<name>/hook-query-log.txt`, else `./research/hook-query-log.txt`)
|
|
205
277
|
4. Verdict counts from the story runner (per dungeon if batch mode), plus which stories needed LLM investigation
|
|
206
278
|
5. One-line summary of the most interesting finding
|
|
279
|
+
6. Separate standalone checks and `warehouseAudits`, including audit-only runs,
|
|
280
|
+
skipped assertions, retained artifact prefix, and warehouse deployment handoff
|
|
207
281
|
|
|
208
282
|
If hooks failed, note that `hook-results.md` can be used as context for fixing them: "read hook-results.md and fix the failing hooks in <dungeon-file>".
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# 1.8.1 verification contract
|
|
2
|
+
|
|
3
|
+
Use this contract when authoring, verifying, provisioning, or presenting a dungeon.
|
|
4
|
+
The [1.8.1 guide](../../../../docs/guides/1.8.1-upgrade-guide.md) describes the
|
|
5
|
+
release. The local repository's `tests/alignment/REPORT.md` and
|
|
6
|
+
`tests/alignment/API-COMPATIBILITY.md` retain the audit evidence; those test
|
|
7
|
+
artifacts are not shipped in npm.
|
|
8
|
+
|
|
9
|
+
## Define the report before measuring
|
|
10
|
+
|
|
11
|
+
Write an independent report specification before choosing assertions. Record
|
|
12
|
+
events, filters, identity rules, cohort definition, denominator, statistic,
|
|
13
|
+
window, timezone, ordering, reentry, session settings, and property selection.
|
|
14
|
+
Compare every assertion with that specification, including passing assertions.
|
|
15
|
+
An inherited default can pass the wrong report. Do not replace funnel TTC with
|
|
16
|
+
a numeric property, a mean with a median, or conversion with event volume to
|
|
17
|
+
rescue a failed story. Supplementary diagnostics must keep their own labels.
|
|
18
|
+
|
|
19
|
+
Local checks prove selected source-derived contracts, not universal Mixpanel
|
|
20
|
+
parity. The audit read analytics source without compiling or executing it. It
|
|
21
|
+
did not run live differential queries. Macro/soup interactions, table surfaces,
|
|
22
|
+
arbitrary hooks, parallel execution, and several counting variants remain gaps.
|
|
23
|
+
|
|
24
|
+
## Separate a measured effect from enough evidence
|
|
25
|
+
|
|
26
|
+
Retain the story runner's computed verdict. Report semantic correctness and
|
|
27
|
+
evidence sufficiency separately. Full-fidelity scale does not guarantee enough
|
|
28
|
+
independent eligible users, mature retention cohorts, or converters. Inspect the
|
|
29
|
+
actual denominator on each side; event count is not independent user count.
|
|
30
|
+
`minCohort` checks supported selected-row population fields, not every bespoke
|
|
31
|
+
denominator. Meeting it alone is not full proof. Label unresolved acceptance
|
|
32
|
+
`INSUFFICIENT_EVIDENCE`; do not count it as a pass or a measured effect failure.
|
|
33
|
+
|
|
34
|
+
For engineered effects, measure a paired baseline with the same seed, window,
|
|
35
|
+
and report options. Add a neutral control (factor one, zero injection, or hook
|
|
36
|
+
disabled) and an unaffected control cohort where applicable. A factor-one
|
|
37
|
+
control must be measured, not inferred from two differently modified groups.
|
|
38
|
+
Report baseline, treatment, eligible users/converters, and paired lift. Keep
|
|
39
|
+
targets and tolerances fixed before the run; disclose any revised criterion
|
|
40
|
+
and preserve its original failed evidence.
|
|
41
|
+
|
|
42
|
+
## Preserve the local counting boundaries
|
|
43
|
+
|
|
44
|
+
- `Funnel.reentry` is verifier-only. Generated repetitions depend on usage
|
|
45
|
+
selection and event budget. Local `reentry` defaults to `false`, including
|
|
46
|
+
`countMode: 'totals'`; repeated histories require explicit `reentry: true`.
|
|
47
|
+
- Funnel histories include ordered restart/shared-edge behavior and inclusive
|
|
48
|
+
2000ms completion grace. TTC uses completed histories under the requested
|
|
49
|
+
options, not independent first occurrences of each event. First/last segment
|
|
50
|
+
selection merges reached-step properties with defined fallback values;
|
|
51
|
+
explicit-step selection is separate.
|
|
52
|
+
- Derive sessions from the full resolved user stream before HPC partitioning.
|
|
53
|
+
Local defaults are UTC, a 30-minute idle timeout, and a 24-hour maximum.
|
|
54
|
+
`session_id` is a diagnostic or explicit legacy mode, not the default truth.
|
|
55
|
+
Non-UTC projects, list-valued HPC, and project-specific hidden/excluded session
|
|
56
|
+
events remain outside this proof.
|
|
57
|
+
- Profile device pools alone establish no identity mapping evidence. The public
|
|
58
|
+
`buildIdentityMap(profiles)` helper retains profile-based compatibility behavior.
|
|
59
|
+
For emitted-identity proof, derive links from valid ordinary both-ID events,
|
|
60
|
+
including later Login events, and pass that explicit `identityMap` to the
|
|
61
|
+
verifier. Earlier device-only rows can resolve retrospectively. Report ID
|
|
62
|
+
conflicts and validation limits; generator auth policy is not ingestion parity.
|
|
63
|
+
- `applyPathBias` is append-only. `share` selects eligible injection recipients;
|
|
64
|
+
it does not promise exact visible branch share. Measure appended payloads,
|
|
65
|
+
competing events, observed path share, and paired lift separately.
|
|
66
|
+
- `applySessionShape` preserves records. Pass known `datasetStart`/`datasetEnd`
|
|
67
|
+
bounds when clipping matters. Explicit bounds reject insufficient capacity
|
|
68
|
+
before mutation; unbounded placement can cross an unknown dataset end.
|
|
69
|
+
- `retentionCurve` weights active-day plans. Budget, funnel spill, and observation
|
|
70
|
+
horizon can prevent literal requested retention percentages. Usage anchors do
|
|
71
|
+
not accumulate previous funnel TTC; retries belong to born-user first funnels.
|
|
72
|
+
|
|
73
|
+
## Keep schema and operational gates independent
|
|
74
|
+
|
|
75
|
+
Undeclared output properties fail schema review even at 100% coverage. Accept
|
|
76
|
+
only config-declared properties or recognized engine/SDK fields enabled for that
|
|
77
|
+
stream. A permissive runtime schema summary cannot override this authorship rule.
|
|
78
|
+
Keep user events, standalone cadence, and warehouse schemas separate.
|
|
79
|
+
|
|
80
|
+
Offline verification stops at local evidence and a deployment handoff. Do not
|
|
81
|
+
automatically provision, import, probe endpoints, deploy warehouse tables, or run
|
|
82
|
+
headless builds. Retain exact artifact prefixes and actual deployment reports.
|
|
83
|
+
Dry-run output is a plan, not evidence of a live outcome; preserve existing reports
|
|
84
|
+
before a dry run writes local files. Live work requires separate authorization.
|