@wrangled/cli 0.3.166 → 0.3.168
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/guide.js +5 -4
- package/dist/main.js +52 -7
- package/dist/map.js +14 -2
- package/dist/playbooks.js +6 -5
- package/package.json +1 -1
package/dist/guide.js
CHANGED
|
@@ -27,7 +27,8 @@ export const funnelTestExample = {
|
|
|
27
27
|
min_lift: 0.1,
|
|
28
28
|
people_per_arm: 400,
|
|
29
29
|
decision_rule: "Scale the video if sign-ups per visitor rise at least 10% with cost per sale no worse.",
|
|
30
|
-
|
|
30
|
+
category: "design",
|
|
31
|
+
subcategory: "hero_image",
|
|
31
32
|
levers: ["clarity"],
|
|
32
33
|
control: { label: "letter at the top", content: "A letter from the founder: why this webinar exists.", url: "https://example.com/signup" },
|
|
33
34
|
variants: [{ label: "video at the top", content: "A 90-second video of the founder saying what the webinar covers.", url: "https://example.com/signup-video" }],
|
|
@@ -117,9 +118,9 @@ The \`weekly-loop\` playbook (\`${cli} skill install weekly-loop\` installs it a
|
|
|
117
118
|
|
|
118
119
|
## Trying one change: tests
|
|
119
120
|
|
|
120
|
-
A test writes down one change to a page or funnel step, tried against what runs now. Each test is kept in the workspace's warehouse, in \`cro_tests\` (one row per test), \`cro_test_arms\` (its sides), \`cro_test_events\` (its history), \`cro_test_results_daily\` (each side's numbers per day) and \`cro_test_reported\` (numbers other tools counted), so you can join tests with visits, sign-ups and sales in one query: \`${cli} warehouse query "SELECT test_id, status,
|
|
121
|
+
A test writes down one change to a page or funnel step, tried against what runs now. Each test is kept in the workspace's warehouse, in \`cro_tests\` (one row per test), \`cro_test_arms\` (its sides), \`cro_test_events\` (its history), \`cro_test_results_daily\` (each side's numbers per day) and \`cro_test_reported\` (numbers other tools counted), so you can join tests with visits, sign-ups and sales in one query: \`${cli} warehouse query "SELECT test_id, status, category, subcategory FROM cro_tests"\`.
|
|
121
122
|
|
|
122
|
-
A test moves through these stages: \`idea\` (only the one change and where are needed), \`planned\` (the full plan is in and the owner said yes), \`running\` (its plan is frozen), then \`decided\`, \`abandoned\` or \`invalid\` (the tracking broke or traffic did not split as planned). Where is a funnel step (\`funnel_key\` and \`step_key\` from the funnels map), a page by its address (\`page_url\`, even one not on the map yet), or both. The full plan is: \`because\` (what led to the idea), \`audience\`, \`expected_lift\` (relative, 0.18 for 18%), \`prediction_confidence\` (low, medium or high), \`disproved_if\`, \`design_type\` (how traffic is split: ad_split_randomized, Meta's own A/B test; ad_split_duplicated, two copies of a campaign, one to each page, which is not a true random split; tool_split, inside a page builder or webinar tool named in a \`design_note\`; link_split; before_after; or adaptive), \`traffic\` (\`{campaigns, optimization_goal, note}\`: which campaigns feed it and what they optimize for, frozen with the plan), \`metric\` (step_conversion, which needs a step, sales_per_visitor, revenue_per_visitor, cost_per_sale or cost_per_lead, spend per sign-up) and \`unit\` (people or sessions), \`guardrails\` (numbers that must not get worse), \`baseline_rate\` with \`baseline_from\` and \`baseline_to\`, \`min_lift\`, \`people_per_arm\`, \`min_days\` and \`max_days\` (14 and 56 unless set), \`decision_rule\`, \`concurrent_changes\`, and the control and 1 to 3 \`variants\`, each with the exact \`content\` it shows and, when a tool runs the split, its \`platform_ref\` (the tool's own id for the side).
|
|
123
|
+
A test moves through these stages: \`idea\` (only the one change and where are needed), \`planned\` (the full plan is in and the owner said yes), \`running\` (its plan is frozen), then \`decided\`, \`abandoned\` or \`invalid\` (the tracking broke or traffic did not split as planned). Where is a funnel step (\`funnel_key\` and \`step_key\` from the funnels map), a page by its address (\`page_url\`, even one not on the map yet), or both. The full plan is: \`because\` (what led to the idea), \`audience\`, \`expected_lift\` (relative, 0.18 for 18%), \`prediction_confidence\` (low, medium or high), \`disproved_if\`, \`design_type\` (how traffic is split: ad_split_randomized, Meta's own A/B test; ad_split_duplicated, two copies of a campaign, one to each page, which is not a true random split; tool_split, inside a page builder or webinar tool named in a \`design_note\`; link_split; before_after; or adaptive), \`traffic\` (\`{campaigns, optimization_goal, note}\`: which campaigns feed it and what they optimize for, frozen with the plan), \`metric\` (step_conversion, which needs a step, sales_per_visitor, revenue_per_visitor, cost_per_sale or cost_per_lead, spend per sign-up) and \`unit\` (people or sessions), \`guardrails\` (numbers that must not get worse), \`baseline_rate\` with \`baseline_from\` and \`baseline_to\`, \`min_lift\`, \`people_per_arm\`, \`min_days\` and \`max_days\` (14 and 56 unless set), \`decision_rule\`, \`concurrent_changes\`, and the control and 1 to 3 \`variants\`, each with the exact \`content\` it shows and, when a tool runs the split, its \`platform_ref\` (the tool's own id for the side). A page builder that serves every side at one address (ClickFunnels' first step, Unbounce) is counted from the site's own visits when each side gives the \`version_label\` its page declares through the visit tag (\`data-${brand.slug}-version\`) and the shared \`page\` or \`url\`; a visitor who saw two versions counts on neither side. GoHighLevel sends each side to its own path, so each side's \`url\` is enough. What a test is mainly about is a \`category\` and its \`subcategory\`, from a fixed list: copy (headline, subheadline, body, call_to_action, bullets, story, other), offer (price, bonus, guarantee, payment_plan, order_bump, upsell, deadline_or_scarcity, other), proof (testimonials, case_study, numbers_and_results, logos_or_press, reviews, other), design (layout, hero_image, video, colors, page_length, mobile, other), form_checkout (form_fields, number_of_steps, checkout_page, other), webinar_event (length, presenter, offer_timing, replay, reminders, other), follow_up (email, text_message, thank_you_page, other), audience_traffic (who_sees_it, ad_angle, optimization_goal, other), tool_platform (webinar_platform, page_builder, checkout_tool, other), and custom, which has no subcategory. Subcategory \`other\` and category \`custom\` need a \`label\` naming what was tested, such as "exit pop-up", kept in lower case. \`also_changed\` lists anything else the test changed, each \`copy.subheadline\`, \`offer.other:<label>\` or \`custom:<label>\`. \`${cli} tests categories\` prints the list and the labels this workspace already uses: reuse one that fits before writing a new one. \`${cli} tests draft\` and \`${cli} tests edit\` take them as flags too: \`--category copy --subcategory headline --also copy.subheadline\`. \`levers\` tag why it should work, from a fixed list: clarity, relevance, trust, value, urgency, less_effort, motivation, other. \`ice_impact\`, \`ice_confidence\` and \`ice_ease\` (1 to 10) sort the idea list. \`evidence\` quotes the proof, each \`{kind, ref, note}\`, kind one of leak, lesson, customer_words, page, ad or other. A test changes one thing: a \`change\` naming two is refused.
|
|
123
124
|
|
|
124
125
|
Write it as JSON, such as \`${JSON.stringify(funnelTestExample)}\`, and run \`${cli} tests draft --file test.json\`. It gets Adzara-style labels from the change and this month (\`hyp-<slug>\` and \`test-<yyyy-mm>-<slug>\`); give \`hypothesis_id\` and \`test_id\` to reuse Adzara's own, so the results of the ads Adzara published for the test (\`ad_test_results_daily\`, \`ad_hypothesis_results_daily\`) join on the same label. \`${cli} tests edit <test_id> --file changes.json\` changes some fields (null clears one). ${brand.name} shows no approval screen: ask the owner in your own session, then record the yes with \`${cli} tests plan <test_id> --approved-by "<owner>"\`, which refuses a plan with a gap and names every missing field. \`${cli} tests start <test_id>\` marks a planned test as running from today and freezes its plan; an edit after that is still saved, and its history and results say so. Starting changes nothing live: the page still goes live through its own approval, and the ads through ${brand.name}'s Meta commands. Until #2202, an ad split test can be measured only on ads Adzara made before the switch, because its sides are counted from Adzara's labels; a \`before_after\` test still works. \`${cli} tests abandon <test_id> --note "<why>"\` drops one that has not ended, \`${cli} tests invalidate <test_id> --note "<what went wrong>"\` ends a running one whose numbers cannot be trusted, and \`${cli} tests note <test_id> --note "<note>"\` adds to its history. \`${cli} tests [--status <status>] [--funnel <funnel_key>] [--limit <n>]\` lists them, newest first (50 unless \`--limit\`, at most 200), and \`${cli} tests show <test_id>\` shows one with its history.
|
|
125
126
|
|
|
@@ -591,7 +592,7 @@ ${cli} number-alerts [--since <date>] the key numbers that swung far from norm
|
|
|
591
592
|
${cli} facts propose --file <fact.json> suggest a fact for a person to confirm
|
|
592
593
|
${cli} lessons [--status active|retired] [--about ads|pages|funnel|messages|cross_channel] [--dimension <d>] | lessons add --file <lesson.json> | lessons retire <id> --note "<why>" lessons from winners and flops, writing one, retiring one
|
|
593
594
|
${cli} tests p1 this brand's P1 test ads by verdict, judged on its own sales against its P1 thresholds, with Meta's count beside
|
|
594
|
-
${cli} tests [--status <status>] [--funnel <key>] [--limit <n>] | tests show <id> | tests draft --file <test.json> | tests start <id> | tests abandon <id> [--note "<why>"] | tests results <id> | tests decide <id> --decision <d> [--note "<why>"] | tests report <id> --file <counts.json> funnel tests, one change each
|
|
595
|
+
${cli} tests [--status <status>] [--funnel <key>] [--limit <n>] | tests show <id> | tests categories | tests draft --file <test.json> | tests start <id> | tests abandon <id> [--note "<why>"] | tests results <id> | tests decide <id> --decision <d> [--note "<why>"] | tests report <id> --file <counts.json> funnel tests, one change each
|
|
595
596
|
${cli} ads tags [--ad <id>] [--status <status>] [--limit <n>] | ads tag --file <tags.json> the marketing tags of ads not made here, and suggesting them
|
|
596
597
|
${cli} ads names [--unparsed] Meta ad names and the tags each gives; --unparsed lists those that fit no pattern
|
|
597
598
|
${cli} ads tag-formats tag every untagged ad's format (image, video or carousel) from Meta's records, free; run it before ads autotag
|
package/dist/main.js
CHANGED
|
@@ -41,6 +41,7 @@ const AD_CHANGE_COLUMNS = ["platform", ...GOOGLE_ADS_CHANGE_COLUMNS];
|
|
|
41
41
|
const SERVER_WAIT_CAP = 50;
|
|
42
42
|
const REVIEW_WAIT_SECONDS = 1800;
|
|
43
43
|
const PROFILE_MESSAGING_USAGE = `profile messaging [--section ${PROFILE_MESSAGING_KINDS.join("|")}] [--offer <offer key>] [--json]`;
|
|
44
|
+
const TEST_CHANGE_FLAGS = '[--category <category> [--subcategory <subcategory>] [--label "<words>"]] [--also <change>]...';
|
|
44
45
|
const PROFILE_ANSWERS_USAGE = "profile answers (--category <category> | --ids <id,id,...>) [--json]";
|
|
45
46
|
const RULES_ADD_USAGE = 'rules add banned_claims|words_to_avoid|required_disclaimers "<text>" [--note <why>]';
|
|
46
47
|
const PROFILE_CHECK_USAGE = "profile check --file ad.json | --input '<json>' | < ad.json [--json]";
|
|
@@ -3230,23 +3231,34 @@ const commands = {
|
|
|
3230
3231
|
printTest(shown.test, shown.events);
|
|
3231
3232
|
},
|
|
3232
3233
|
},
|
|
3234
|
+
"tests categories": {
|
|
3235
|
+
usage: "tests categories what a test can be about: each category and its subcategories, and the labels this workspace's tests already use",
|
|
3236
|
+
async run(args, options) {
|
|
3237
|
+
if (args.length > 0)
|
|
3238
|
+
throw new CliError("Usage: tests categories", exitCodes.usage);
|
|
3239
|
+
const list = await api(await resolveConnection(options), `${TESTS_PATH}/categories`);
|
|
3240
|
+
if (options.format === "json")
|
|
3241
|
+
return printJson(list);
|
|
3242
|
+
printTestCategories(list);
|
|
3243
|
+
},
|
|
3244
|
+
},
|
|
3233
3245
|
"tests draft": {
|
|
3234
|
-
usage:
|
|
3246
|
+
usage: `tests draft --file <test.json> | < test.json ${TEST_CHANGE_FLAGS} write down a test idea: the one change and where (a funnel step or a page); the rest of the plan may come later with tests edit`,
|
|
3235
3247
|
async run(args, options) {
|
|
3236
3248
|
if (args.length > 0)
|
|
3237
|
-
throw new CliError(
|
|
3238
|
-
const { test } = await api(await resolveConnection(options), TESTS_PATH, await readToolInput(options));
|
|
3249
|
+
throw new CliError(`Usage: tests draft --file <test.json> | < test.json ${TEST_CHANGE_FLAGS}`, exitCodes.usage);
|
|
3250
|
+
const { test } = await api(await resolveConnection(options), TESTS_PATH, withTestChange(await readToolInput(options), options));
|
|
3239
3251
|
if (options.format === "json")
|
|
3240
3252
|
return printJson({ test });
|
|
3241
3253
|
console.log(`Wrote down test ${test.test_id} as an idea.`);
|
|
3242
3254
|
},
|
|
3243
3255
|
},
|
|
3244
3256
|
"tests edit": {
|
|
3245
|
-
usage:
|
|
3257
|
+
usage: `tests edit <test_id> [--file <changes.json> | < changes.json] ${TEST_CHANGE_FLAGS} change some of a test's fields (null clears one); a change to the plan after the start is kept and marked in its history`,
|
|
3246
3258
|
async run(args, options) {
|
|
3247
3259
|
if (args.length !== 1)
|
|
3248
|
-
throw new CliError(
|
|
3249
|
-
const { test } = await api(await resolveConnection(options), testPath(args[0]), await readToolInput(options), "PATCH");
|
|
3260
|
+
throw new CliError(`Usage: tests edit <test_id> [--file <changes.json> | < changes.json] ${TEST_CHANGE_FLAGS}`, exitCodes.usage);
|
|
3261
|
+
const { test } = await api(await resolveConnection(options), testPath(args[0]), withTestChange(await readToolInput(options), options), "PATCH");
|
|
3250
3262
|
if (options.format === "json")
|
|
3251
3263
|
return printJson({ test });
|
|
3252
3264
|
console.log(`Changed test ${test.test_id} (${test.status}).`);
|
|
@@ -4810,6 +4822,27 @@ const VOICE_SUGGESTIONS_PATH = "/api/v1/voice-lists/suggestions";
|
|
|
4810
4822
|
const LESSONS_PATH = "/api/v1/lessons";
|
|
4811
4823
|
const TESTS_PATH = "/api/v1/funnel-tests";
|
|
4812
4824
|
const TEST_COLUMNS = ["test_id", "status", "where", "change", "metric"];
|
|
4825
|
+
/** The test body with the change flags on top: --category, --subcategory and --label give the main change, each --also one more. */
|
|
4826
|
+
function withTestChange(body, options) {
|
|
4827
|
+
const flags = {
|
|
4828
|
+
...(options.category === undefined ? {} : { category: options.category }),
|
|
4829
|
+
...(options.subcategory === undefined ? {} : { subcategory: options.subcategory }),
|
|
4830
|
+
...(options.label === undefined ? {} : { label: options.label }),
|
|
4831
|
+
...(options.also === undefined ? {} : { also_changed: options.also }),
|
|
4832
|
+
};
|
|
4833
|
+
if (Object.keys(flags).length === 0)
|
|
4834
|
+
return body;
|
|
4835
|
+
if (body === null || typeof body !== "object" || Array.isArray(body))
|
|
4836
|
+
throw new CliError("The test file must hold a JSON object to add --category, --subcategory, --label or --also to.", exitCodes.usage);
|
|
4837
|
+
return { ...body, ...flags };
|
|
4838
|
+
}
|
|
4839
|
+
/** A change as the API returns it, written as also_changed takes it: copy.headline, offer.other:<label> or custom:<label>. */
|
|
4840
|
+
const changeText = (c) => (c.category === "custom" ? `custom:${c.label}` : c.label === undefined ? `${c.category}.${c.subcategory}` : `${c.category}.other:${c.label}`);
|
|
4841
|
+
function printTestCategories(list) {
|
|
4842
|
+
const lines = list.categories.map((c) => `${c.category} (${c.words}): ${c.subcategories.map((s) => (s.words === s.subcategory.replaceAll("_", " ") ? s.subcategory : `${s.subcategory} (${s.words})`)).join(", ") || "no subcategories; give --label"}`);
|
|
4843
|
+
console.log(lines.join("\n"));
|
|
4844
|
+
console.log(list.labels.length === 0 ? "\nNo labels used yet." : `\nLabels already used (reuse one that fits):\n${renderTable(["category", "label", "tests"], list.labels)}`);
|
|
4845
|
+
}
|
|
4813
4846
|
const testPath = (ref) => `${TESTS_PATH}/${encodeURIComponent(ref)}`;
|
|
4814
4847
|
function testRow(t) {
|
|
4815
4848
|
return { test_id: t.test_id, status: t.status, where: testWhere(t), change: excerpt(t.change, 50), metric: t.metric ?? "" };
|
|
@@ -4822,6 +4855,7 @@ function printTest(t, events) {
|
|
|
4822
4855
|
a.item_id && `item ${a.item_id}`,
|
|
4823
4856
|
a.asset_version && `version ${a.asset_version}`,
|
|
4824
4857
|
a.platform_ref && `tool id ${a.platform_ref}`,
|
|
4858
|
+
a.version_label && `version label ${a.version_label}`,
|
|
4825
4859
|
a.weight !== null && `${Math.round(a.weight * 100)}% of traffic`,
|
|
4826
4860
|
]
|
|
4827
4861
|
.filter(Boolean)
|
|
@@ -4849,7 +4883,9 @@ function printTest(t, events) {
|
|
|
4849
4883
|
...line("Decision rule", t.decision_rule),
|
|
4850
4884
|
...line("Also changing", t.concurrent_changes),
|
|
4851
4885
|
...line("Traffic", t.traffic && [t.traffic.campaigns?.length && `campaigns ${t.traffic.campaigns.join(", ")}`, t.traffic.optimization_goal && `optimized for ${t.traffic.optimization_goal}`, t.traffic.note].filter(Boolean).join("; ")),
|
|
4852
|
-
...line("
|
|
4886
|
+
...line("Category", t.main_change && changeText(t.main_change)),
|
|
4887
|
+
...line("The test also changes", t.also_changed.map(changeText).join(", ")),
|
|
4888
|
+
...line("Tags", t.levers.join(", ")),
|
|
4853
4889
|
...line("ICE", t.ice_impact !== null || t.ice_confidence !== null || t.ice_ease !== null ? `${t.ice_impact ?? "-"} / ${t.ice_confidence ?? "-"} / ${t.ice_ease ?? "-"}` : null),
|
|
4854
4890
|
...(t.evidence.length > 0 ? ["Evidence:", ...t.evidence.map((e) => ` ${e.kind}: ${e.ref}${e.note ? ` (${e.note})` : ""}`)] : []),
|
|
4855
4891
|
...line("Started", t.started_on && `${t.started_on}${t.ended_on ? `, ended ${t.ended_on}` : ""}`),
|
|
@@ -5338,6 +5374,11 @@ Options:
|
|
|
5338
5374
|
--approved-by <name> tests plan: who said yes to the test's plan, as the owner gave their name in your session
|
|
5339
5375
|
--outcome <o> tests decide: what the test found, won, lost, inconclusive or invalid; needed with scale, iterate and stop
|
|
5340
5376
|
--learning <text> tests decide: what the test taught, in a sentence, kept on the test
|
|
5377
|
+
--category <c> tests draft and edit: what the test is mainly about, such as copy (tests categories lists them)
|
|
5378
|
+
--subcategory <s> tests draft and edit: the part of the category it changes, such as headline
|
|
5379
|
+
--label <words> tests draft and edit: for subcategory other or category custom, what was tested, such as "exit pop-up"
|
|
5380
|
+
--also <change> tests draft and edit: something else the test changed, such as copy.subheadline,
|
|
5381
|
+
offer.other:<label> or custom:<label>; repeat for several
|
|
5341
5382
|
--about <a> with lessons, only the lessons with a proof about ads, pages, funnel (steps and tests) or messages
|
|
5342
5383
|
(emails, texts and sequences), or cross_channel: proven on two or more of ads, pages and messages
|
|
5343
5384
|
--dimension <d> with lessons, only the lessons along that dimension (hook, angle, format, offer, persona, page, funnel_step, follow_up, other); repeat for several
|
|
@@ -5625,6 +5666,8 @@ function parseOptions(argv) {
|
|
|
5625
5666
|
awareness: { type: "string" },
|
|
5626
5667
|
confidence: { type: "string" },
|
|
5627
5668
|
category: { type: "string" },
|
|
5669
|
+
subcategory: { type: "string" },
|
|
5670
|
+
also: { type: "string", multiple: true },
|
|
5628
5671
|
ids: { type: "string" },
|
|
5629
5672
|
section: { type: "string" },
|
|
5630
5673
|
offer: { type: "string" },
|
|
@@ -5854,6 +5897,8 @@ function parseOptions(argv) {
|
|
|
5854
5897
|
entryTags: Object.fromEntries(Object.entries(entryTagFlags).flatMap(([field, flag]) => (values[flag] === undefined ? [] : [[field, values[flag]]]))),
|
|
5855
5898
|
confidence: values.confidence,
|
|
5856
5899
|
category: values.category,
|
|
5900
|
+
subcategory: values.subcategory,
|
|
5901
|
+
also: values.also,
|
|
5857
5902
|
ids: values.ids,
|
|
5858
5903
|
section: values.section,
|
|
5859
5904
|
offer: values.offer,
|
package/dist/map.js
CHANGED
|
@@ -83,8 +83,20 @@ export function renderQuestions(map) {
|
|
|
83
83
|
const note = unmeasured ? [" Worked out from the nightly counts, so every question shows as not yet until the first nightly run."] : [];
|
|
84
84
|
return `${["Questions", ...note, ...map.questions.flatMap(questionLines)].join("\n")}\n`;
|
|
85
85
|
}
|
|
86
|
+
/** Wrangled's own tables and views, grouped by owner, one line each with its kinds. */
|
|
87
|
+
export function renderOwn(own) {
|
|
88
|
+
const lines = [`${brand.name}'s own data (not counted yet)`];
|
|
89
|
+
own.forEach((row, i) => {
|
|
90
|
+
if (row.owner !== own[i - 1]?.owner)
|
|
91
|
+
lines.push(` ${row.owner}`);
|
|
92
|
+
lines.push(` ${row.table} ${row.kinds.join(", ")}`);
|
|
93
|
+
});
|
|
94
|
+
return `${lines.join("\n")}\n`;
|
|
95
|
+
}
|
|
86
96
|
export function renderDataMap(map) {
|
|
97
|
+
// A server from before the own section sends none.
|
|
98
|
+
const own = map.own?.length ? [renderOwn(map.own)] : [];
|
|
87
99
|
if (map.sources.length === 0)
|
|
88
|
-
return "No sources connected.\n";
|
|
89
|
-
return `${map.sources.map((source) => sourceLines(source).join("\n")).join("\n\n")}\n
|
|
100
|
+
return ["No sources connected.\n", ...own].join("\n");
|
|
101
|
+
return [`${map.sources.map((source) => sourceLines(source).join("\n")).join("\n\n")}\n`, ...own, renderQuestions(map)].join("\n");
|
|
90
102
|
}
|
package/dist/playbooks.js
CHANGED
|
@@ -2017,7 +2017,8 @@ export const croTestExample = {
|
|
|
2017
2017
|
min_lift: 0.1,
|
|
2018
2018
|
people_per_arm: 300,
|
|
2019
2019
|
decision_rule: "Keep the text if attendance rises at least 10% with unsubscribes no higher.",
|
|
2020
|
-
|
|
2020
|
+
category: "follow_up",
|
|
2021
|
+
subcategory: "text_message",
|
|
2021
2022
|
levers: ["urgency"],
|
|
2022
2023
|
control: { label: "today's reminders: an email the day before and an hour before", content: "Email the day before and an hour before; no text." },
|
|
2023
2024
|
variants: [{ label: "the same reminders plus a text 15 minutes before", content: "We're live in 15 minutes: {link}" }],
|
|
@@ -2079,10 +2080,10 @@ ${JSON.stringify(croTestExample)}
|
|
|
2079
2080
|
\`\`\`
|
|
2080
2081
|
|
|
2081
2082
|
- \`change\` is the one change. \`because\` is what led to it, \`audience\` who sees it, \`expected_lift\` how much it should move (0.15 for 15%) and \`prediction_confidence\` how sure you are; \`disproved_if\` says what result would prove it wrong.
|
|
2082
|
-
- \`control\` is what runs now; \`variants\` is the changed version, each \`{label, content, page?, url?, item_id?, asset_version?}\`. \`content\` is the exact words or change each side shows.
|
|
2083
|
+
- \`control\` is what runs now; \`variants\` is the changed version, each \`{label, content, page?, url?, item_id?, asset_version?, platform_ref?, version_label?}\`. \`content\` is the exact words or change each side shows.
|
|
2083
2084
|
- \`metric\` is the number that should move: \`step_conversion\` (only with a \`step_key\`), \`sales_per_visitor\`, \`revenue_per_visitor\`, \`cost_per_sale\` or \`cost_per_lead\` (spend per sign-up), counted by \`unit\` (people or sessions). \`baseline_rate\` is today's rate over \`baseline_from\` to \`baseline_to\`, \`min_lift\` the smallest lift worth finding, \`people_per_arm\` how many each side needs, \`guardrails\` the numbers that must not get worse, and \`decision_rule\` how you will call it. \`min_days\` and \`max_days\` are 14 and 56 unless set.
|
|
2084
|
-
- \`design_type\` is how traffic is split: \`ad_split_randomized\` (Meta's own A/B test), \`ad_split_duplicated\` (two copies of a campaign, one to each page; not a true random split), \`tool_split\` (inside a page builder or webinar tool, named in \`design_note\`), \`link_split\`, \`before_after\` or \`adaptive\`. \`traffic\` says which campaigns feed it and what they optimize for. Until #2202, an ad split test is counted only from Adzara's labels, so it can be measured only on ads Adzara made before the switch; for a new test, use \`before_after\`.
|
|
2085
|
-
- \`
|
|
2085
|
+
- \`design_type\` is how traffic is split: \`ad_split_randomized\` (Meta's own A/B test), \`ad_split_duplicated\` (two copies of a campaign, one to each page; not a true random split), \`tool_split\` (inside a page builder or webinar tool, named in \`design_note\`; for GoHighLevel, which sends each side to its own path, give each side's \`url\`; for a builder that serves every side at one address, such as ClickFunnels' first step, give each side the shared \`url\` and the \`version_label\` its page declares with \`data-${cli}-version\`, and the visits count it), \`link_split\`, \`before_after\` or \`adaptive\`. \`traffic\` says which campaigns feed it and what they optimize for. Until #2202, an ad split test is counted only from Adzara's labels, so it can be measured only on ads Adzara made before the switch; for a new test, use \`before_after\`.
|
|
2086
|
+
- \`category\` and \`subcategory\` say what the test is mainly about, and \`also_changed\` anything else it changed: run \`${cli} tests categories\` for the lists and the labels already used, and reuse a label before writing a new one. \`levers\` tag why it should work, from \`${cli} guide\`'s fixed list.
|
|
2086
2087
|
- \`evidence\` quotes the proof, each \`{kind, ref, note}\` with kind \`leak\`, \`lesson\`, \`customer_words\`, \`page\`, \`ad\` or \`other\`. Start with the leak itself.
|
|
2087
2088
|
- Give \`hypothesis_id\` and \`test_id\` only to reuse Adzara's own labels; left out, they are made for you.
|
|
2088
2089
|
|
|
@@ -2139,7 +2140,7 @@ Without a shell: MCP \`get_funnel_test_results\` with \`{"id": "<test_id>"}\`.
|
|
|
2139
2140
|
- **Every call is \`too_early\` or \`not_enough_yet\`:** leave the test running; that needs no command.
|
|
2140
2141
|
- **A call is \`split_broken\`:** people did not split between the sides as planned. Tell the owner the result cannot be trusted, and suggest marking the test invalid (\`${cli} tests invalidate <test_id> --note "<what went wrong>"\`) once the split is checked.
|
|
2141
2142
|
- **A call is directional:** it is a hunch, not a win. Suggest keeping the test running to its most days, or \`iterate\` to retest a sharper version; never \`scale\` on it. Only when something outside the test got in the way (the page broke, a price changed, the ads stopped), suggest \`hold\` to the owner.
|
|
2142
|
-
- **The split runs in another tool** (a \`tool_split\`
|
|
2143
|
+
- **The split runs in another tool and no side was counted here** (a \`tool_split\` with no side's page or version label counted, a Meta A/B test): read each side's numbers in that tool and enter them with \`${cli} tests report <test_id> --file counts.json\` (MCP \`report_test_numbers\`), then read the results again; their words start "Counted by <tool>". Never decide a test judged on sales on Meta's purchase count alone.
|
|
2143
2144
|
- **A call is ready** (a win, or no difference found): suggest \`scale\` (the new version won), \`stop\` (it lost or made no difference) or \`iterate\` (promising; it writes down a sharper version). \`iterate\` writes a new test, so suggest it only while the week is under its 3; otherwise suggest \`stop\` and draft the sharper version next week.
|
|
2144
2145
|
|
|
2145
2146
|
Put each suggestion to the owner with the ideas from step 4, then record the owner's decision:
|