@officexapp/vidfarm-devcli 0.21.38 → 0.21.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -284,7 +284,7 @@ The licensed harness also carries the **generative build workflow** guidance (ch
|
|
|
284
284
|
| `vidfarm handoff image --theme "<what>" [--items "a,b,c"] [--style …] [--single]` | local (pure text) | **The interactive-mode IMAGE brief.** Prints the exact prompt, the numbered steps, the free tools (meta.ai / ChatGPT / Gemini / HF Spaces) and the follow-up command. Defaults to a **sticker pack**: ONE sheet holding every item on a chroma plate → `vidfarm sticker-pack` splits it for $0. Picks a plate the art won't collide with (green art → magenta plate), spells out what the local keyer actually needs (a crisp silhouette in a different color from the plate, sealed shapes, clear gaps between items — hollow art and plate-colored detail INSIDE a shape are fine now), and carries that `--key-color` into the follow-up. `--single` for one subject. `--zoned` asks instead for a color-block sheet (one panel colour per item) and hands back a `--zones RxC` follow-up — worth it when the pack's own colors fight one plate, but leave it off for a free consumer tool that may not follow a grid. |
|
|
285
285
|
| `vidfarm handoff raws --keywords "a,b" [--platforms tiktok,youtube] [--count N] [--purpose "…"]` | local (pure text) | **The interactive-mode CLIP-SOURCING brief** — the bottom rung of the sourcing ladder (browser control → `clipper`/`raws scan --cloud` → public raws → the human). Prints what to search, how to download (a Google *search* for a downloader, never a link that rots), and the import command for when the folder is ready. |
|
|
286
286
|
| `vidfarm storyboard [dir] [--init] [--frames "Title\|scene,…"] [--json]` | local (`STORYBOARD.md` / `SCRIPT.md`) | **The plan pass, and a core part of the composition format.** Scaffolds/reads the project's `STORYBOARD.md`: ordered frames with `duration`/`status`/`src`/`scene`/`voiceover`. The Vidfarm editor renders this file in its **Storyboard** view (contact sheet + per-frame comments + `outline → built → animated` progress), so it's the cheapest place to get a director's approval before building. Not to be confused with `vidfarm sequence` (which GENERATES storyboard images for the pure-videogen pipeline). Alias: `plan`. |
|
|
287
|
-
| `vidfarm experiment [dir] [--init] [round …] [log <video> …] [--json]` | local (`EXPERIMENTS_DIARY.md`) | **Ad testing over weeks, not one video.** Owns the campaign ledger and ONLY that: sizes each round (`videos ÷
|
|
287
|
+
| `vidfarm experiment [dir] [--init] [round …] [log <video> …] [--json]` | local (`EXPERIMENTS_DIARY.md`) | **Ad testing over weeks, not one video.** Owns the campaign ledger and ONLY that: sizes each round (`videos ÷ capacity = epochs`, where capacity is the SUM of per-channel posting rates — `--channels "tiktok_a x2, li_a 3/week, fb_a paused"` — and epoch slots are dealt out in proportion, warning on `channel-overposted`), ranks the north-star metric, flags outliers vs the median, and lints the method — two variables in one structured round, a winner promoted off one post, results read at mixed ages, a structured round handed to gigworkers, unspent capacity. Feedback, not a gate (exits 0). Also carries the FORMAT decision into planning: `--init` prints the copywriting-led menu (b-roll / talking head / process / loop background / satisfying / lifestyle / POV quote) and records the pick in Setup, which every round inherits. Two writes: `round --videos N --variable angle …` and `log <video> --comments N --channel <acct> --source flockposter --age 48h` (ALWAYS pass `--channel`: it keeps a per-account median so each video is ranked against its own account, not the fleet — account health moves numbers by multiples — and a second account's reading counts as the retest that clears `account-health-confound`) (or `log <video> --posted --channel <id>`, which only RECORDS a post). It deliberately does not re-wrap `channels` (capacity), `harness`/`qa` (constants), `handoff` (briefs), `dedupe` (per-channel copies) or `approve`+`schedule` (posting). Method: <https://vidfarm.cc/experiments.md>. Alias: `experiments`. |
|
|
288
288
|
| `vidfarm wallet [--job <id>\|--tracer <t>] [--limit <n>]` | `GET /api/v1/user/me/wallet` | cost log: balance + lifetime spend + recent charges. `--job <renderJobId>` prints **what that one video cost** (sums its charges); `--tracer <t>` sums a tracer. Cloud-only; readable on the free plan too (shows $0.00). Aliases: `spend`, `costs` |
|
|
289
289
|
| `vidfarm provider-keys` / `vidfarm add-provider-key <p> <secret>` | `GET`·`POST /api/v1/user/me/provider-keys` | manage AI keys |
|
|
290
290
|
| `vidfarm upload <file> [--folder <path>]` | presign → S3 PUT → finalize (`.../temporary-files/presign` + `.../temporary-files`) | upload → durable URL (ephemeral, 30-day TTL; prefer `--folder temp` for scratch). Goes direct to S3, so large files (up to **200 MB**) bypass the ~6 MB Lambda body limit |
|
package/SKILL.md
CHANGED
|
@@ -103,7 +103,7 @@ Otherwise, pull it over the network and load only the reference you need:
|
|
|
103
103
|
- `references/rest-api.md` — complete REST map (load only for direct HTTP integration)
|
|
104
104
|
- `recipes/` — `find-and-fork-template.md` · `retheme-template.md` · `local-edit-render-approve.md` · `onboard-a-new-director.md` · `bulk-scripting-with-a-harness.md` (volume/N-variant work) · `cutout-graphics-for-explainers.md` (sticker-art explainers)
|
|
105
105
|
- `harnesses/` — `README.md` (**start here for anything harness-shaped**: the three director phrasings, the format, the DNA strands) plus the bundled bases `short-form` · `hooks` · `ugc-testimonial` · `explainer` · `product-demo`, each as `<name>.HARNESS.md`
|
|
106
|
-
- **Running ad experiments (finding the winning ad):** `https://vidfarm.cc/
|
|
106
|
+
- **Running ad experiments (finding the winning ad):** `https://vidfarm.cc/experiments.md` — **fetch this whenever the ask is about performance rather than one video**: "which ad should I run", "test these angles", "what's working", "we post daily on N channels", or any campaign that runs for weeks. It primes you as a short-form ad scientist: the KPI interview (default north star = **comments**), channel count → testing capacity, **epochs** and **rounds**, **Creative Mode** (N free-form videos, the default, ideal for gigworkers) vs **Structured Mode** (one variable, everything else pinned by a harness, your own agents only), the composition params, and the standalone `EXPERIMENTS_DIARY.md` ledger. Plan → user approval → one static handoff document per video.
|
|
107
107
|
- **Crowdsourcing the editing:** `https://vidfarm.cc/crowdsourcing.md` — the client-vs-gigworker router for agentic clipper loops (a Dollar Platoon vending machine, recruiting editors, claiming tasks, proofs and payouts). This is how a Creative Mode batch gets produced in bulk.
|
|
108
108
|
- **Updating Vidfarm:** `https://vidfarm.cc/update.md` — **fetch this file whenever the user asks to update/upgrade Vidfarm**, says their skill or devcli is out of date, or hits a command/route that "used to work". It's the clean uninstall→reinstall runbook for BOTH halves (the `vidfarm` skill pack and `@officexapp/vidfarm-devcli`), and it carries the current breaking-change notes — read it live rather than improvising an `npm i -g`, because the notes are exactly what a blind reinstall misses.
|
|
109
109
|
- **Skill pack index (what else is installable):** `https://vidfarm.cc/skill-pack/index.json`
|
|
@@ -125,7 +125,7 @@ For composition *authoring* craft (motion, keyframes, scene design), Vidfarm shi
|
|
|
125
125
|
5. "Script / batch / render loop" → `references/automation-and-local-dev.md`
|
|
126
126
|
5b. **"Explainer with cutout/sticker art"** (flat illustrations on a stage, sticker sheets, "like those animated explainer videos") → `recipes/cutout-graphics-for-explainers.md`
|
|
127
127
|
6. "I need TTS / music / captions / background removal" → `references/primitives.md`
|
|
128
|
-
6b. **"Which ad wins?" / "test these angles" / "we post every day and want results"** → fetch <https://vidfarm.cc/
|
|
128
|
+
6b. **"Which ad wins?" / "test these angles" / "we post every day and want results"** → fetch <https://vidfarm.cc/experiments.md> and run the experimentation method (capacity → epochs/rounds → creative vs structured → `EXPERIMENTS_DIARY.md`). Do not improvise a testing plan; a round with two variables teaches nothing. To produce a bulk batch with gigworkers, pair it with <https://vidfarm.cc/crowdsourcing.md>.
|
|
129
129
|
7. **"Update / upgrade vidfarm"** (or anything that smells like a stale install — a missing command, a 404 on a documented route, a version mismatch) → fetch <https://vidfarm.cc/update.md> and follow it. Update the skill pack and the devcli **together**; updating one alone is the usual cause of "the skill says to do X but it fails."
|
|
130
130
|
|
|
131
131
|
## Non-negotiables
|
package/dist/src/cli.js
CHANGED
|
@@ -38,7 +38,7 @@ import { CostModeBlockedError, assertBilledAllowed, clearStoredCostMode, costMod
|
|
|
38
38
|
import { INTERACTION_MODES, INTERACTION_MODE_BLURB, clearStoredInteractionMode, interactionModeExplainer, interactionModeSummaryLine, interactiveOfferLine, normalizeInteractionMode, resolveInteractionMode, writeStoredInteractionMode } from "./devcli/interaction-mode.js";
|
|
39
39
|
import { buildImageHandoff, buildRawsHandoff, formatHandoff } from "./devcli/handoff.js";
|
|
40
40
|
import { STORYBOARD_FILENAME, readStoryboard, renderStoryboardScaffold, storyboardStage } from "./devcli/storyboard.js";
|
|
41
|
-
import { CORE_METRICS, DEFAULT_METRIC, EXPERIMENTS_FILENAME, EXPERIMENT_MODES, analyzeRound, appendResult, appendRound, capacityOf, epochsNeeded, findRoundIndex, lintDiary, markPosted, readDiary, renderDiaryScaffold, writeDiary } from "./devcli/experiments.js";
|
|
41
|
+
import { CORE_METRICS, DEFAULT_FORMAT, DEFAULT_METRIC, EASY_FORMATS, EXPERIMENTS_FILENAME, EXPERIMENT_MODES, analyzeRound, appendResult, appendRound, capacityOf, epochsNeeded, findRoundIndex, lintDiary, markPosted, readDiary, renderDiaryScaffold, writeDiary } from "./devcli/experiments.js";
|
|
42
42
|
// vidfarm-devcli — command-line bridge for the Vidfarm video studio. The
|
|
43
43
|
// `serve` command boots the FULL editor locally (single origin, disk-backed
|
|
44
44
|
// records + storage) so power users edit compositions on disk while a browser
|
|
@@ -132,21 +132,33 @@ Account (persisted login — points the CLI + \`serve\` at cloud prod):
|
|
|
132
132
|
--title/--format/--message/--arc/--audience Frontmatter for --init
|
|
133
133
|
--json The parsed manifest (frames, statuses, warnings)
|
|
134
134
|
experiment [dir] AD TESTING — read the campaign's EXPERIMENTS_DIARY.md,
|
|
135
|
-
size each round (videos ÷
|
|
135
|
+
size each round (videos ÷ capacity = epochs), rank the
|
|
136
136
|
north-star metric, flag outliers vs the median, and lint
|
|
137
137
|
the method (two variables in one structured round, a
|
|
138
138
|
winner promoted off one post, results read at mixed ages,
|
|
139
139
|
a structured round handed to gigworkers, unspent capacity).
|
|
140
|
-
Feedback, not a gate. Method: vidfarm.cc/
|
|
140
|
+
Feedback, not a gate. Method: vidfarm.cc/experiments.md
|
|
141
141
|
(alias: experiments)
|
|
142
|
-
--init Scaffold the diary
|
|
143
|
-
|
|
142
|
+
--init Scaffold the diary. --channels takes PER-CHANNEL RATES —
|
|
143
|
+
"tiktok_a x2, tiktok_b, yt_a 1/day, li_a 3/week, fb_a paused"
|
|
144
|
+
— and capacity becomes the SUM of them, so round slots are
|
|
145
|
+
dealt out in proportion instead of round-robin.
|
|
146
|
+
(--metric/--channels/--mode/--product/
|
|
147
|
+
--format/--baseline/--editors · --force overwrites).
|
|
148
|
+
Prints the copywriting-led FORMAT MENU (b-roll · talking
|
|
149
|
+
head · process · loop background · satisfying · lifestyle ·
|
|
150
|
+
POV quote) when --format is omitted — the format is a
|
|
151
|
+
planning decision, not a build-time discovery.
|
|
144
152
|
round --videos <n> Append a planned round + its epoch slot tables
|
|
145
153
|
[--variable angle] [--constants "format,hooks"] [--mode structured|creative]
|
|
146
|
-
[--slots "v001|angle,…"] [--why "…"] [--win "…"] [--title "…"]
|
|
154
|
+
[--slots "v001|angle,…"] [--format "…"] [--why "…"] [--win "…"] [--title "…"]
|
|
147
155
|
log <video> Record what happened: --views/--comments/--clicks/--buys
|
|
148
|
-
[--source flockposter] [--age 48h] [--note "…"]
|
|
149
|
-
or --posted [--channel <id>] to mark a slot live.
|
|
156
|
+
--channel <id> [--source flockposter] [--age 48h] [--note "…"]
|
|
157
|
+
[--round N], or --posted [--channel <id>] to mark a slot live.
|
|
158
|
+
ALWAYS pass --channel: account health moves numbers by
|
|
159
|
+
multiples, so each video is ranked against ITS OWN account's
|
|
160
|
+
median (1.3×acct), and a reading from a second account counts
|
|
161
|
+
as the retest that clears account-health-confound.
|
|
150
162
|
--json Parsed setup + per-round analysis + findings
|
|
151
163
|
It does NOT re-wrap what already exists: capacity comes from
|
|
152
164
|
"vidfarm channels", constants from "vidfarm harness", briefs
|
|
@@ -9175,7 +9187,7 @@ async function runStoryboardCommand(argv) {
|
|
|
9175
9187
|
}
|
|
9176
9188
|
// `vidfarm experiment [dir]` — the ad-testing ledger, and only the ledger.
|
|
9177
9189
|
//
|
|
9178
|
-
// The method is https://vidfarm.cc/
|
|
9190
|
+
// The method is https://vidfarm.cc/experiments.md: rounds of videos that vary one
|
|
9179
9191
|
// composition param (structured) or vary everything (creative), posted across the
|
|
9180
9192
|
// channels the director holds, read back against one north-star metric. This
|
|
9181
9193
|
// command owns the three things nothing else in the CLI owns — the
|
|
@@ -9198,6 +9210,7 @@ async function runExperimentCommand(argv) {
|
|
|
9198
9210
|
channels: { type: "string" },
|
|
9199
9211
|
mode: { type: "string" },
|
|
9200
9212
|
product: { type: "string" },
|
|
9213
|
+
format: { type: "string" },
|
|
9201
9214
|
baseline: { type: "string" },
|
|
9202
9215
|
editors: { type: "string" },
|
|
9203
9216
|
// round
|
|
@@ -9245,6 +9258,7 @@ async function runExperimentCommand(argv) {
|
|
|
9245
9258
|
const body = renderDiaryScaffold({
|
|
9246
9259
|
product: parsed.values.product,
|
|
9247
9260
|
metric: parsed.values.metric?.toLowerCase(),
|
|
9261
|
+
format: parsed.values.format,
|
|
9248
9262
|
channels: asList(parsed.values.channels) ?? [],
|
|
9249
9263
|
mode: mode,
|
|
9250
9264
|
baseline: parsed.values.baseline,
|
|
@@ -9256,6 +9270,13 @@ async function runExperimentCommand(argv) {
|
|
|
9256
9270
|
return printJson({ ok: true, created: target });
|
|
9257
9271
|
console.log(`${GREEN}Diary created:${RESET} ${target}`);
|
|
9258
9272
|
console.log(`${DIM}Fill the channel list from ${BOLD}vidfarm channels${RESET}${DIM} — the channel count IS your testing capacity per epoch.${RESET}`);
|
|
9273
|
+
if (!parsed.values.format) {
|
|
9274
|
+
// The format is a PLANNING decision. Print the menu now rather than let it
|
|
9275
|
+
// get discovered at build time, one drifting video at a time.
|
|
9276
|
+
console.log(`${DIM}Format defaulted to ${BOLD}${DEFAULT_FORMAT}${RESET}${DIM}. Offer the director the menu (--format sets it):${RESET}`);
|
|
9277
|
+
for (const f of EASY_FORMATS)
|
|
9278
|
+
console.log(` ${DIM}·${RESET} ${f.label.padEnd(38)} ${DIM}${f.note}${RESET}`);
|
|
9279
|
+
}
|
|
9259
9280
|
console.log(`${DIM}Then plan the first round: ${BOLD}vidfarm experiment round --variable angle --videos 8 --why "awareness level is the largest unknown"${RESET}`);
|
|
9260
9281
|
return;
|
|
9261
9282
|
}
|
|
@@ -9282,6 +9303,7 @@ async function runExperimentCommand(argv) {
|
|
|
9282
9303
|
mode: mode,
|
|
9283
9304
|
variable: parsed.values.variable,
|
|
9284
9305
|
constants: asList(parsed.values.constants),
|
|
9306
|
+
format: parsed.values.format ?? read.manifest.setup.format,
|
|
9285
9307
|
justification: parsed.values.why,
|
|
9286
9308
|
win: parsed.values.win,
|
|
9287
9309
|
videos,
|
|
@@ -9289,6 +9311,7 @@ async function runExperimentCommand(argv) {
|
|
|
9289
9311
|
capacity,
|
|
9290
9312
|
slots,
|
|
9291
9313
|
channels: read.manifest.setup.channels,
|
|
9314
|
+
channelPlans: read.manifest.setup.channelPlans,
|
|
9292
9315
|
date: today
|
|
9293
9316
|
};
|
|
9294
9317
|
const next = appendRound(readFileSync(read.absPath, "utf8"), input);
|
|
@@ -9298,7 +9321,7 @@ async function runExperimentCommand(argv) {
|
|
|
9298
9321
|
return printJson({ ok: true, round: number, videos, capacity, epochs: Number.isFinite(epochs) ? epochs : null, path: read.absPath });
|
|
9299
9322
|
console.log(`${GREEN}Round ${number} planned${RESET} ${DIM}${read.absPath}${RESET}`);
|
|
9300
9323
|
console.log(` ${DIM}mode${RESET} ${mode}${input.variable ? ` ${DIM}variable${RESET} ${input.variable}` : ""}`);
|
|
9301
|
-
console.log(` ${DIM}sizing${RESET} ${videos} videos ÷ ${capacity || "?"}
|
|
9324
|
+
console.log(` ${DIM}sizing${RESET} ${videos} videos ÷ ${capacity || "?"} slots/epoch = ${BOLD}${Number.isFinite(epochs) ? epochs : "?"} epoch(s)${RESET}`);
|
|
9302
9325
|
if (mode === "structured") {
|
|
9303
9326
|
const rel = path.relative(process.cwd(), dir);
|
|
9304
9327
|
const shown = !rel ? "." : rel.startsWith("..") ? dir : rel;
|
|
@@ -9342,6 +9365,7 @@ async function runExperimentCommand(argv) {
|
|
|
9342
9365
|
const written = appendResult(source, read.manifest, roundIndex, {
|
|
9343
9366
|
video,
|
|
9344
9367
|
metrics,
|
|
9368
|
+
channel: parsed.values.channel,
|
|
9345
9369
|
note: parsed.values.note,
|
|
9346
9370
|
source: parsed.values.source ?? read.manifest.setup.source,
|
|
9347
9371
|
age: parsed.values.age,
|
|
@@ -9374,6 +9398,7 @@ async function runExperimentCommand(argv) {
|
|
|
9374
9398
|
mode: a.round.mode ?? read.manifest.setup.mode,
|
|
9375
9399
|
variable: a.round.variable,
|
|
9376
9400
|
constants: a.round.constants,
|
|
9401
|
+
format: a.round.format ?? read.manifest.setup.format ?? null,
|
|
9377
9402
|
metric: a.metric,
|
|
9378
9403
|
videosPlanned: a.videosPlanned,
|
|
9379
9404
|
epochsNeeded: Number.isFinite(a.epochsNeeded ?? NaN) ? a.epochsNeeded : null,
|
|
@@ -9382,6 +9407,8 @@ async function runExperimentCommand(argv) {
|
|
|
9382
9407
|
unposted: a.unposted,
|
|
9383
9408
|
awaiting: a.awaiting,
|
|
9384
9409
|
median: a.median ?? null,
|
|
9410
|
+
channelMedians: a.channelMedians,
|
|
9411
|
+
retested: a.retested,
|
|
9385
9412
|
scored: a.scored,
|
|
9386
9413
|
outliers: a.outliers,
|
|
9387
9414
|
finding: a.round.finding,
|
|
@@ -9395,15 +9422,20 @@ async function runExperimentCommand(argv) {
|
|
|
9395
9422
|
console.log(`${YELLOW}No ${EXPERIMENTS_FILENAME} in ${dir}.${RESET}`);
|
|
9396
9423
|
console.log(`${DIM}The diary is the source of truth for a campaign that runs for weeks — start it before the first post:${RESET}`);
|
|
9397
9424
|
console.log(` ${BOLD}vidfarm experiment --init --metric comments --channels "$(vidfarm channels --json | jq -r '.channels[].destination_id' | paste -sd,)"${RESET}`);
|
|
9398
|
-
console.log(`${DIM}Method: https://vidfarm.cc/
|
|
9425
|
+
console.log(`${DIM}Method: https://vidfarm.cc/experiments.md${RESET}`);
|
|
9399
9426
|
return;
|
|
9400
9427
|
}
|
|
9401
9428
|
const setup = read.manifest.setup;
|
|
9402
9429
|
const capacity = capacityOf(setup);
|
|
9403
9430
|
console.log(`${BOLD}Experiments${RESET} ${DIM}${read.absPath}${RESET}`);
|
|
9404
9431
|
console.log(` ${DIM}north star${RESET} ${setup.metric ?? `${DEFAULT_METRIC} (assumed)`}`);
|
|
9405
|
-
|
|
9432
|
+
// Echo what the diary literally says ("li_a 3/week"), never the derived float.
|
|
9433
|
+
const rates = setup.channelPlans.length
|
|
9434
|
+
? setup.channelPlans.map((p) => p.raw.trim()).join(", ")
|
|
9435
|
+
: setup.channels.join(", ");
|
|
9436
|
+
console.log(` ${DIM}capacity${RESET} ${capacity || "?"} video(s)/epoch ${DIM}${rates}${RESET}`);
|
|
9406
9437
|
console.log(` ${DIM}mode${RESET} ${setup.mode ?? "creative (assumed)"}${setup.editors ? ` ${DIM}editors${RESET} ${setup.editors}` : ""}`);
|
|
9438
|
+
console.log(` ${DIM}format${RESET} ${setup.format ?? `${DEFAULT_FORMAT} (assumed)`}`);
|
|
9407
9439
|
if (setup.baseline)
|
|
9408
9440
|
console.log(` ${DIM}baseline${RESET} ${setup.baseline}`);
|
|
9409
9441
|
for (const a of analyses) {
|
|
@@ -9411,17 +9443,22 @@ async function runExperimentCommand(argv) {
|
|
|
9411
9443
|
console.log("");
|
|
9412
9444
|
console.log(` ${BOLD}Round ${n}${a.round.title ? ` — ${a.round.title}` : ""}${RESET} ${DIM}${a.round.mode ?? setup.mode ?? ""}${a.round.variable ? ` · variable: ${a.round.variable}` : ""}${RESET}`);
|
|
9413
9445
|
if (a.videosPlanned !== undefined) {
|
|
9414
|
-
console.log(` ${DIM}sizing${RESET} ${a.videosPlanned} videos ÷ ${a.capacity || "?"} = ${Number.isFinite(a.epochsNeeded ?? NaN) ? a.epochsNeeded : "?"} epoch(s) · ${a.epochsRun} run · ${a.postedCount} posted`);
|
|
9446
|
+
console.log(` ${DIM}sizing${RESET} ${a.videosPlanned} videos ÷ ${a.capacity || "?"} slots/epoch = ${Number.isFinite(a.epochsNeeded ?? NaN) ? a.epochsNeeded : "?"} epoch(s) · ${a.epochsRun} run · ${a.postedCount} posted`);
|
|
9415
9447
|
}
|
|
9416
9448
|
if (a.scored.length > 0) {
|
|
9417
|
-
|
|
9449
|
+
const perAccount = Object.entries(a.channelMedians);
|
|
9450
|
+
console.log(` ${DIM}${a.metric} — round median ${a.median ?? "?"}${perAccount.length ? ` · per-account: ${perAccount.map(([c, m]) => `${c} ${m}`).join(" · ")}` : ""}${RESET}`);
|
|
9418
9451
|
for (const s of a.scored) {
|
|
9419
9452
|
const mark = a.outliers.includes(s) ? `${GREEN}★${RESET}` : s === a.weakest ? `${DIM}·${RESET}` : " ";
|
|
9420
9453
|
const ratio = s.ratio !== undefined ? `${s.ratio.toFixed(1)}×` : "";
|
|
9421
9454
|
// When a round was read at more than one age, the same video appears once
|
|
9422
9455
|
// per reading — label them, or the list looks like a duplicate row.
|
|
9423
9456
|
const age = a.readContexts.length > 1 && s.age ? ` @${s.age}` : "";
|
|
9424
|
-
|
|
9457
|
+
// The account-normalized ratio is the trustworthy one — mark it so nobody
|
|
9458
|
+
// reads a cross-account raw ranking as a result.
|
|
9459
|
+
const shown = s.channelRatio !== undefined ? `${s.channelRatio.toFixed(1)}×acct` : ratio;
|
|
9460
|
+
const chan = s.channel ? ` ${s.channel}` : "";
|
|
9461
|
+
console.log(` ${mark} ${s.video.padEnd(8)} ${String(s.value).padStart(7)} ${DIM}${shown.padStart(9)}${age}${chan}${s.variant ? ` ${s.variant}` : ""}${RESET}`);
|
|
9425
9462
|
}
|
|
9426
9463
|
}
|
|
9427
9464
|
if (a.unposted.length > 0)
|
|
@@ -9443,7 +9480,7 @@ async function runExperimentCommand(argv) {
|
|
|
9443
9480
|
for (const w of read.manifest.warnings)
|
|
9444
9481
|
console.log(` ${YELLOW}warn${RESET} ${DIM}${w.message}${w.line ? ` (line ${w.line})` : ""}${RESET}`);
|
|
9445
9482
|
console.log("");
|
|
9446
|
-
console.log(`${DIM}Feedback, not a gate. Method: https://vidfarm.cc/
|
|
9483
|
+
console.log(`${DIM}Feedback, not a gate. Method: https://vidfarm.cc/experiments.md${RESET}`);
|
|
9447
9484
|
}
|
|
9448
9485
|
// `vidfarm mode [interactive|autonomous]` — show or set whether the HUMAN is in
|
|
9449
9486
|
// the loop. Independent of cost mode: any cost mode runs either way, and the
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
// An ad campaign is an evolution, not a delivery: rounds of videos that vary ONE
|
|
4
4
|
// composition param (structured) or vary everything (creative), posted across
|
|
5
5
|
// however many channels the director holds, then read back against one north-star
|
|
6
|
-
// metric. The method lives at `https://vidfarm.cc/
|
|
6
|
+
// metric. The method lives at `https://vidfarm.cc/experiments.md`; the ledger lives
|
|
7
7
|
// at the work root as one markdown file, `EXPERIMENTS_DIARY.md`.
|
|
8
8
|
//
|
|
9
9
|
// This module owns ONLY the parts nothing else in the CLI owns:
|
|
@@ -25,6 +25,22 @@ export const EXPERIMENT_MODES = ["creative", "structured"];
|
|
|
25
25
|
/** The four core metrics, in funnel order. Any of them can be the north star. */
|
|
26
26
|
export const CORE_METRICS = ["views", "comments", "clicks", "buys"];
|
|
27
27
|
export const DEFAULT_METRIC = "comments";
|
|
28
|
+
/** The starting format menu, offered during planning — NOT discovered at build
|
|
29
|
+
* time. Deliberately copywriting-led: in every one of these the words carry the
|
|
30
|
+
* persuasion and the footage only has to hold attention, which is what makes
|
|
31
|
+
* them fast, cheap and repeatable. All seven are sourceable for ~$0 from
|
|
32
|
+
* `vidfarm public-raws --categories`. */
|
|
33
|
+
export const EASY_FORMATS = [
|
|
34
|
+
{ key: "b-roll", label: "kinetic captions over b-roll footage", note: "the workhorse — start here" },
|
|
35
|
+
{ key: "talking-head", label: "talking head", note: "only if the user will film themselves; strongest trust signal" },
|
|
36
|
+
{ key: "process", label: "process footage", note: "high watch-through, needs no narration" },
|
|
37
|
+
{ key: "loop-background", label: "loop background footage", note: "cheapest of all; the copy is the entire video" },
|
|
38
|
+
{ key: "satisfying", label: "satisfying footage", note: "strong retention, weak topical fit" },
|
|
39
|
+
{ key: "lifestyle", label: "lifestyle footage", note: "best for identity and status angles" },
|
|
40
|
+
{ key: "pov-quote", label: "POV quote aesthetic", note: "pure copywriting; a natural fit for hook tests" }
|
|
41
|
+
];
|
|
42
|
+
/** What to take when the director has no opinion — never stall on this choice. */
|
|
43
|
+
export const DEFAULT_FORMAT = EASY_FORMATS[0].label;
|
|
28
44
|
/** A variant only counts as tested once it has this many posts behind it. One
|
|
29
45
|
* post is noise in short form, so nothing is promoted off a single result. */
|
|
30
46
|
export const MIN_POSTS_PER_VARIANT = 2;
|
|
@@ -73,6 +89,26 @@ function parseMetricNumber(raw) {
|
|
|
73
89
|
const n = Number.parseFloat(t.replace(/[^0-9.eE+-]/g, ""));
|
|
74
90
|
return Number.isFinite(n) ? n : undefined;
|
|
75
91
|
}
|
|
92
|
+
/** Posting frequency written next to a channel. Accepts the shapes a director
|
|
93
|
+
* actually types: "tiktok_a x2", "tiktok_a ×2", "yt_a 1/day", "li_a 3/week",
|
|
94
|
+
* "fb_a 2/month", and a bare "ig_a" (= once per epoch). "paused" or a 0 rate
|
|
95
|
+
* contributes no capacity but stays listed. */
|
|
96
|
+
const CHANNEL_RATE_RE = /^(.*?)[\s(]*(?:[x×]\s*(\d+(?:\.\d+)?)|(\d+(?:\.\d+)?)\s*\/\s*(day|epoch|wk|week|mo|month)|(paused|off))\)?\s*$/i;
|
|
97
|
+
export function parseChannelPlan(entry) {
|
|
98
|
+
const raw = entry.trim();
|
|
99
|
+
const m = CHANNEL_RATE_RE.exec(raw);
|
|
100
|
+
if (!m || (!m[2] && !m[3] && !m[5]))
|
|
101
|
+
return { id: raw, perEpoch: 1, raw };
|
|
102
|
+
const id = (m[1] ?? "").trim() || raw;
|
|
103
|
+
if (m[5])
|
|
104
|
+
return { id, perEpoch: 0, raw };
|
|
105
|
+
if (m[2])
|
|
106
|
+
return { id, perEpoch: Number.parseFloat(m[2]), raw };
|
|
107
|
+
const n = Number.parseFloat(m[3] ?? "1");
|
|
108
|
+
const unit = (m[4] ?? "day").toLowerCase();
|
|
109
|
+
const perEpoch = unit.startsWith("w") ? n / 7 : unit.startsWith("m") ? n / 30 : n;
|
|
110
|
+
return { id, perEpoch, raw };
|
|
111
|
+
}
|
|
76
112
|
function isMode(value) {
|
|
77
113
|
return EXPERIMENT_MODES.includes(value);
|
|
78
114
|
}
|
|
@@ -84,7 +120,7 @@ function isMode(value) {
|
|
|
84
120
|
export function parseDiary(source) {
|
|
85
121
|
const warnings = [];
|
|
86
122
|
const lines = source.split(/\r?\n/);
|
|
87
|
-
const setup = { secondary: [], channels: [], extra: {} };
|
|
123
|
+
const setup = { secondary: [], channels: [], channelPlans: [], extra: {} };
|
|
88
124
|
const rounds = [];
|
|
89
125
|
let inSetup = false;
|
|
90
126
|
let round = null;
|
|
@@ -214,14 +250,14 @@ export function parseDiary(source) {
|
|
|
214
250
|
const video = (get("video", "id", "clip") ?? "").trim();
|
|
215
251
|
const metrics = {};
|
|
216
252
|
tableHeader.forEach((name, idx) => {
|
|
217
|
-
if (!name || name === "video" || name === "id" || name === "clip" || name === "note" || name === "notes")
|
|
253
|
+
if (!name || name === "video" || name === "id" || name === "clip" || name === "note" || name === "notes" || name === "channel" || name === "account")
|
|
218
254
|
return;
|
|
219
255
|
const value = parseMetricNumber(row[idx] ?? "");
|
|
220
256
|
if (value !== undefined)
|
|
221
257
|
metrics[name] = value;
|
|
222
258
|
});
|
|
223
259
|
if (video) {
|
|
224
|
-
results.rows.push({ video, metrics, note: get("note", "notes"), line: lineNo });
|
|
260
|
+
results.rows.push({ video, metrics, channel: get("channel", "account")?.trim() || undefined, note: get("note", "notes"), line: lineNo });
|
|
225
261
|
}
|
|
226
262
|
else {
|
|
227
263
|
warnings.push({ message: "Result row with no video id — skipped.", line: lineNo });
|
|
@@ -255,9 +291,11 @@ export function parseDiary(source) {
|
|
|
255
291
|
}
|
|
256
292
|
else if (key === "channels" || key.startsWith("channels")) {
|
|
257
293
|
const inner = /\(([^)]*)\)/.exec(value);
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
294
|
+
// Strip only a LEADING "(capacity N/epoch)" label — a per-channel rate
|
|
295
|
+
// in parentheses further along is data, not a label.
|
|
296
|
+
const entries = splitList(value.replace(/^\s*\([^)]*\)\s*/, ""));
|
|
297
|
+
setup.channelPlans = (entries.length ? entries : splitList(inner?.[1] ?? "")).map(parseChannelPlan);
|
|
298
|
+
setup.channels = setup.channelPlans.map((p) => p.id);
|
|
261
299
|
}
|
|
262
300
|
else if (key === "mode") {
|
|
263
301
|
const normalized = value.toLowerCase();
|
|
@@ -268,6 +306,8 @@ export function parseDiary(source) {
|
|
|
268
306
|
warnings.push({ message: `Unknown mode "${value}" — expected creative or structured.`, line: lineNo });
|
|
269
307
|
}
|
|
270
308
|
}
|
|
309
|
+
else if (key === "format" || key === "video_format")
|
|
310
|
+
setup.format = value;
|
|
271
311
|
else if (key === "product")
|
|
272
312
|
setup.product = value;
|
|
273
313
|
else if (key.startsWith("baseline"))
|
|
@@ -299,6 +339,8 @@ export function parseDiary(source) {
|
|
|
299
339
|
round.variable = v;
|
|
300
340
|
else if (k === "constants")
|
|
301
341
|
round.constants = splitList(v);
|
|
342
|
+
else if (k === "format")
|
|
343
|
+
round.format = v;
|
|
302
344
|
else
|
|
303
345
|
round.extra[k] = v;
|
|
304
346
|
}
|
|
@@ -353,7 +395,7 @@ export function readDiary(dir) {
|
|
|
353
395
|
exists: false,
|
|
354
396
|
path: EXPERIMENTS_FILENAME,
|
|
355
397
|
absPath,
|
|
356
|
-
manifest: { setup: { secondary: [], channels: [], extra: {} }, rounds: [], warnings: [] }
|
|
398
|
+
manifest: { setup: { secondary: [], channels: [], channelPlans: [], extra: {} }, rounds: [], warnings: [] }
|
|
357
399
|
};
|
|
358
400
|
}
|
|
359
401
|
return { exists: true, path: EXPERIMENTS_FILENAME, absPath, manifest: parseDiary(readFileSync(absPath, "utf8")) };
|
|
@@ -364,7 +406,11 @@ export function readDiary(dir) {
|
|
|
364
406
|
/** Videos per epoch = channels held. One channel carries about one test post a
|
|
365
407
|
* day before it reads as spam. */
|
|
366
408
|
export function capacityOf(setup) {
|
|
367
|
-
|
|
409
|
+
if (setup.channelPlans.length === 0)
|
|
410
|
+
return setup.channels.length;
|
|
411
|
+
const total = setup.channelPlans.reduce((sum, p) => sum + (Number.isFinite(p.perEpoch) ? p.perEpoch : 0), 0);
|
|
412
|
+
// Round to 2dp so "3/week" style fractions don't print as 0.4285714285714286.
|
|
413
|
+
return Math.round(total * 100) / 100;
|
|
368
414
|
}
|
|
369
415
|
export function epochsNeeded(videos, capacity) {
|
|
370
416
|
if (!Number.isFinite(videos) || videos <= 0)
|
|
@@ -373,6 +419,36 @@ export function epochsNeeded(videos, capacity) {
|
|
|
373
419
|
return Number.POSITIVE_INFINITY;
|
|
374
420
|
return Math.ceil(videos / capacity);
|
|
375
421
|
}
|
|
422
|
+
/** Deal N video slots out across channels, epoch by epoch, respecting each
|
|
423
|
+
* channel's posting frequency.
|
|
424
|
+
*
|
|
425
|
+
* Fractional rates (a channel that can only take 3 posts a week) are handled by
|
|
426
|
+
* carrying credit forward: the channel accrues `perEpoch` each epoch and takes a
|
|
427
|
+
* slot whenever its credit reaches 1. That keeps a 3/week channel on a real
|
|
428
|
+
* every-other-day cadence instead of either over-posting it daily or dropping it.
|
|
429
|
+
* Deterministic — same inputs, same schedule. */
|
|
430
|
+
export function allocateSlots(plans, videos) {
|
|
431
|
+
const active = plans.filter((p) => Number.isFinite(p.perEpoch) && p.perEpoch > 0);
|
|
432
|
+
if (active.length === 0 || videos <= 0)
|
|
433
|
+
return [];
|
|
434
|
+
const credit = new Map(active.map((p) => [p.id, 0]));
|
|
435
|
+
const out = [];
|
|
436
|
+
// Bound the loop: even the slowest channel mix can't need more epochs than
|
|
437
|
+
// videos / smallest-rate, and the +2 covers the first accrual.
|
|
438
|
+
const slowest = Math.min(...active.map((p) => p.perEpoch));
|
|
439
|
+
const maxEpochs = Math.ceil(videos / slowest) + 2;
|
|
440
|
+
for (let epoch = 1; epoch <= maxEpochs && out.length < videos; epoch++) {
|
|
441
|
+
for (const plan of active) {
|
|
442
|
+
let c = (credit.get(plan.id) ?? 0) + plan.perEpoch;
|
|
443
|
+
while (c >= 1 && out.length < videos) {
|
|
444
|
+
out.push({ epoch, channel: plan.id });
|
|
445
|
+
c -= 1;
|
|
446
|
+
}
|
|
447
|
+
credit.set(plan.id, c);
|
|
448
|
+
}
|
|
449
|
+
}
|
|
450
|
+
return out;
|
|
451
|
+
}
|
|
376
452
|
export function median(values) {
|
|
377
453
|
const sorted = values.filter((v) => Number.isFinite(v)).sort((a, b) => a - b);
|
|
378
454
|
if (sorted.length === 0)
|
|
@@ -387,6 +463,8 @@ export function analyzeRound(round, setup, opts) {
|
|
|
387
463
|
const slotRows = round.epochs.flatMap((e) => e.rows);
|
|
388
464
|
const variantOf = new Map();
|
|
389
465
|
const postsOf = new Map();
|
|
466
|
+
// A video can occupy more than one slot — that IS the cross-account retest.
|
|
467
|
+
const channelsOf = new Map();
|
|
390
468
|
for (const row of slotRows) {
|
|
391
469
|
if (!row.video)
|
|
392
470
|
continue;
|
|
@@ -394,6 +472,11 @@ export function analyzeRound(round, setup, opts) {
|
|
|
394
472
|
variantOf.set(row.video, row.variant);
|
|
395
473
|
if (row.posted)
|
|
396
474
|
postsOf.set(row.video, (postsOf.get(row.video) ?? 0) + 1);
|
|
475
|
+
if (row.channel) {
|
|
476
|
+
const set = channelsOf.get(row.video) ?? new Set();
|
|
477
|
+
set.add(row.channel);
|
|
478
|
+
channelsOf.set(row.video, set);
|
|
479
|
+
}
|
|
397
480
|
}
|
|
398
481
|
const resultRows = round.results.flatMap((block) => block.rows.map((row) => ({ row, block })));
|
|
399
482
|
const scored = [];
|
|
@@ -404,15 +487,59 @@ export function analyzeRound(round, setup, opts) {
|
|
|
404
487
|
scored.push({
|
|
405
488
|
video: row.video,
|
|
406
489
|
value,
|
|
490
|
+
channel: row.channel ?? (channelsOf.get(row.video)?.size === 1 ? Array.from(channelsOf.get(row.video) ?? [])[0] : undefined),
|
|
407
491
|
variant: variantOf.get(row.video),
|
|
408
492
|
posts: postsOf.get(row.video) ?? 0,
|
|
409
493
|
age: block.age,
|
|
410
494
|
source: block.source
|
|
411
495
|
});
|
|
412
496
|
}
|
|
497
|
+
// A RETEST is the same video MEASURED on two accounts — not a video whose slot
|
|
498
|
+
// was planned for one account and whose result came back from another (that is
|
|
499
|
+
// a plan change, and counting it would clear the confound warning for free).
|
|
500
|
+
// So: prefer distinct channels across RESULT rows; fall back to distinct
|
|
501
|
+
// channels across slots that were actually marked posted.
|
|
502
|
+
const measuredOn = new Map();
|
|
503
|
+
for (const s of scored) {
|
|
504
|
+
if (!s.channel)
|
|
505
|
+
continue;
|
|
506
|
+
const set = measuredOn.get(s.video) ?? new Set();
|
|
507
|
+
set.add(s.channel);
|
|
508
|
+
measuredOn.set(s.video, set);
|
|
509
|
+
}
|
|
510
|
+
for (const row of slotRows) {
|
|
511
|
+
if (!row.video || !row.channel || !row.posted)
|
|
512
|
+
continue;
|
|
513
|
+
if (resultRows.some(({ row: r }) => r.video === row.video && r.channel))
|
|
514
|
+
continue;
|
|
515
|
+
const set = measuredOn.get(row.video) ?? new Set();
|
|
516
|
+
set.add(row.channel);
|
|
517
|
+
measuredOn.set(row.video, set);
|
|
518
|
+
}
|
|
413
519
|
const med = median(scored.map((s) => s.value));
|
|
414
520
|
for (const s of scored)
|
|
415
521
|
s.ratio = med && med > 0 ? s.value / med : undefined;
|
|
522
|
+
// Normalize per ACCOUNT. Account health moves numbers by multiples, so a video
|
|
523
|
+
// is only fairly judged against its own channel's median — and a channel needs
|
|
524
|
+
// at least two results before it has one.
|
|
525
|
+
const byChannel = new Map();
|
|
526
|
+
for (const s of scored) {
|
|
527
|
+
if (!s.channel)
|
|
528
|
+
continue;
|
|
529
|
+
byChannel.set(s.channel, [...(byChannel.get(s.channel) ?? []), s.value]);
|
|
530
|
+
}
|
|
531
|
+
const channelMedians = {};
|
|
532
|
+
for (const [channel, values] of byChannel) {
|
|
533
|
+
if (values.length < 2)
|
|
534
|
+
continue;
|
|
535
|
+
const m = median(values);
|
|
536
|
+
if (m !== undefined && m > 0)
|
|
537
|
+
channelMedians[channel] = m;
|
|
538
|
+
}
|
|
539
|
+
for (const s of scored) {
|
|
540
|
+
const m = s.channel ? channelMedians[s.channel] : undefined;
|
|
541
|
+
s.channelRatio = m ? s.value / m : undefined;
|
|
542
|
+
}
|
|
416
543
|
const ranked = [...scored].sort((a, b) => b.value - a.value);
|
|
417
544
|
const reported = new Set(resultRows.map(({ row }) => row.video));
|
|
418
545
|
return {
|
|
@@ -428,7 +555,11 @@ export function analyzeRound(round, setup, opts) {
|
|
|
428
555
|
awaiting: slotRows.filter((r) => r.posted && r.video && !reported.has(r.video)).map((r) => r.video),
|
|
429
556
|
scored: ranked,
|
|
430
557
|
median: med,
|
|
431
|
-
|
|
558
|
+
channelMedians,
|
|
559
|
+
retested: Array.from(measuredOn.entries()).filter(([, set]) => set.size > 1).map(([video]) => video),
|
|
560
|
+
// Prefer the account-normalized ratio when the channel has a baseline; fall
|
|
561
|
+
// back to the round median otherwise.
|
|
562
|
+
outliers: med && med > 0 ? ranked.filter((s) => (s.channelRatio ?? s.ratio ?? 0) >= outlierRatio) : [],
|
|
432
563
|
weakest: ranked[ranked.length - 1],
|
|
433
564
|
readContexts: Array.from(new Set(round.results.map((b) => `${b.source ?? "?"}@${b.age ?? "?"}`)))
|
|
434
565
|
};
|
|
@@ -445,6 +576,14 @@ export function lintDiary(manifest, analyses) {
|
|
|
445
576
|
fix: `Add "- North-star metric: ${DEFAULT_METRIC}" (comments is the richest early-stage intel).`
|
|
446
577
|
});
|
|
447
578
|
}
|
|
579
|
+
if (!setup.format) {
|
|
580
|
+
findings.push({
|
|
581
|
+
level: "warn",
|
|
582
|
+
code: "no-format",
|
|
583
|
+
message: "No starting video format in Setup — the format is a PLANNING decision, not something to discover while editing.",
|
|
584
|
+
fix: `Pick one in the interview and write it down: ${EASY_FORMATS.map((f) => f.label).join(" · ")}. No opinion? Take "${DEFAULT_FORMAT}" and move — don't stall.`
|
|
585
|
+
});
|
|
586
|
+
}
|
|
448
587
|
if (capacity === 0) {
|
|
449
588
|
findings.push({
|
|
450
589
|
level: "warn",
|
|
@@ -483,6 +622,17 @@ export function lintDiary(manifest, analyses) {
|
|
|
483
622
|
if (!round.variable) {
|
|
484
623
|
findings.push({ ...at, level: "error", code: "no-variable", message: `Round ${at.round} is structured but names no variable.`, fix: "Add \"- Variable: <param>\"." });
|
|
485
624
|
}
|
|
625
|
+
// Format is a constant in most rounds. If the round neither pins one nor
|
|
626
|
+
// inherits one from Setup, its "constants" were never actually constant.
|
|
627
|
+
if (!round.format && !setup.format && !/format/i.test(round.variable ?? "")) {
|
|
628
|
+
findings.push({
|
|
629
|
+
...at,
|
|
630
|
+
level: "warn",
|
|
631
|
+
code: "round-no-format",
|
|
632
|
+
message: `Round ${at.round} names no video format, so every video in it can drift to a different one.`,
|
|
633
|
+
fix: `Add "- Format: <choice>" to the round (or to Setup, which it inherits). Default: "${DEFAULT_FORMAT}".`
|
|
634
|
+
});
|
|
635
|
+
}
|
|
486
636
|
if (round.constants.length === 0) {
|
|
487
637
|
findings.push({
|
|
488
638
|
...at,
|
|
@@ -515,6 +665,22 @@ export function lintDiary(manifest, analyses) {
|
|
|
515
665
|
fix: "Re-read the whole round at one age, or compare only within a single results block."
|
|
516
666
|
});
|
|
517
667
|
}
|
|
668
|
+
// Account health is the biggest confounder in the method: it moves numbers by
|
|
669
|
+
// multiples, often more than the variable under test. A round whose videos each
|
|
670
|
+
// sat on a different account has measured the accounts as much as the videos.
|
|
671
|
+
const channelsUsed = new Set(analysis.scored.map((s) => s.channel).filter(Boolean));
|
|
672
|
+
if (channelsUsed.size > 1 && analysis.retested.length === 0 && analysis.scored.length > 1) {
|
|
673
|
+
const normalized = Object.keys(analysis.channelMedians).length;
|
|
674
|
+
findings.push({
|
|
675
|
+
...at,
|
|
676
|
+
level: normalized > 0 ? "warn" : "error",
|
|
677
|
+
code: "account-health-confound",
|
|
678
|
+
message: `Round ${at.round} ranks ${analysis.scored.length} videos across ${channelsUsed.size} accounts with no variant retested on a second account — account health can outweigh the variable you are testing.`,
|
|
679
|
+
fix: normalized > 0
|
|
680
|
+
? "Per-account medians exist for some channels — judge by those, not the round median. Then retest the leader on a different account to confirm the ordering holds."
|
|
681
|
+
: "Re-post the apparent winner on a DIFFERENT account (deduped) and the apparent loser on the winner's account. If the ordering survives the swap, the effect is real; if it flips, you measured account health."
|
|
682
|
+
});
|
|
683
|
+
}
|
|
518
684
|
for (const outlier of analysis.outliers) {
|
|
519
685
|
if (outlier.posts < MIN_POSTS_PER_VARIANT) {
|
|
520
686
|
findings.push({
|
|
@@ -529,16 +695,50 @@ export function lintDiary(manifest, analyses) {
|
|
|
529
695
|
if (capacity > 0 && analysis.epochsRun > 0) {
|
|
530
696
|
const latest = round.epochs[round.epochs.length - 1];
|
|
531
697
|
const used = latest?.rows.length ?? 0;
|
|
532
|
-
|
|
698
|
+
// Only whole slots are postable, and fractional capacity ("3/week") leaves a
|
|
699
|
+
// remainder that isn't a usable slot — don't report 0.43 free.
|
|
700
|
+
const free = Math.floor(capacity - used);
|
|
701
|
+
if (used > 0 && free >= 1) {
|
|
533
702
|
findings.push({
|
|
534
703
|
...at,
|
|
535
704
|
level: "info",
|
|
536
705
|
code: "unspent-capacity",
|
|
537
|
-
message: `Latest epoch of round ${at.round} uses ${used} of ${capacity} slots — ${
|
|
706
|
+
message: `Latest epoch of round ${at.round} uses ${used} of ${capacity} slots — ${free} free.`,
|
|
538
707
|
fix: "Fill them: more samples of this variable, a parallel round on another param (formats or hooks are the usual next), or a one-off theory logged as a one-off."
|
|
539
708
|
});
|
|
540
709
|
}
|
|
541
710
|
}
|
|
711
|
+
// A channel scheduled beyond its own stated frequency is how an account gets
|
|
712
|
+
// throttled or flagged — and a throttled account poisons every number on it.
|
|
713
|
+
for (const epoch of round.epochs) {
|
|
714
|
+
const counts = new Map();
|
|
715
|
+
for (const row of epoch.rows) {
|
|
716
|
+
if (!row.channel)
|
|
717
|
+
continue;
|
|
718
|
+
counts.set(row.channel, (counts.get(row.channel) ?? 0) + 1);
|
|
719
|
+
}
|
|
720
|
+
for (const [channel, used] of counts) {
|
|
721
|
+
const plan = setup.channelPlans.find((p) => p.id === channel);
|
|
722
|
+
if (!plan)
|
|
723
|
+
continue;
|
|
724
|
+
const allowed = Math.max(1, Math.ceil(plan.perEpoch));
|
|
725
|
+
if (plan.perEpoch === 0) {
|
|
726
|
+
findings.push({ ...at, level: "warn", code: "paused-channel-scheduled", message: `Round ${at.round}, epoch ${epoch.label ?? "?"}: ${channel} is marked paused but has ${used} slot(s).`, fix: "Un-pause it in Setup, or move those slots to an active channel." });
|
|
727
|
+
}
|
|
728
|
+
else if (used > allowed) {
|
|
729
|
+
// A bare channel name carries no explicit rate — say "the default" rather
|
|
730
|
+
// than echoing the id back as if it were a frequency.
|
|
731
|
+
const stated = plan.raw.trim() === plan.id ? "once per epoch (the default)" : `${plan.raw.trim()} (~${allowed}/epoch)`;
|
|
732
|
+
findings.push({
|
|
733
|
+
...at,
|
|
734
|
+
level: "warn",
|
|
735
|
+
code: "channel-overposted",
|
|
736
|
+
message: `Round ${at.round}, epoch ${epoch.label ?? "?"}: ${channel} has ${used} slots but its stated frequency is ${stated}.`,
|
|
737
|
+
fix: "Spread them over more epochs, or raise the channel's frequency in Setup if it really can take that volume."
|
|
738
|
+
});
|
|
739
|
+
}
|
|
740
|
+
}
|
|
741
|
+
}
|
|
542
742
|
if (analysis.awaiting.length > 0) {
|
|
543
743
|
findings.push({ ...at, level: "info", code: "awaiting-results", message: `Round ${at.round}: ${analysis.awaiting.length} posted video(s) have no results logged (${analysis.awaiting.join(", ")}).`, fix: "Log them: `vidfarm experiment log <video> --views N --comments N --source <where> --age 48h`." });
|
|
544
744
|
}
|
|
@@ -552,22 +752,34 @@ export function lintDiary(manifest, analyses) {
|
|
|
552
752
|
return findings;
|
|
553
753
|
}
|
|
554
754
|
export function renderDiaryScaffold(input) {
|
|
555
|
-
const
|
|
755
|
+
const plans = input.channelPlans?.length
|
|
756
|
+
? input.channelPlans
|
|
757
|
+
: (input.channels ?? []).map((id) => parseChannelPlan(id));
|
|
758
|
+
const channelList = plans.map((p) => p.raw.trim()).join(", ");
|
|
759
|
+
const capacity = plans.length
|
|
760
|
+
? Math.round(plans.reduce((sum, p) => sum + p.perEpoch, 0) * 100) / 100
|
|
761
|
+
: 0;
|
|
556
762
|
const metric = input.metric ?? DEFAULT_METRIC;
|
|
557
763
|
const secondary = input.secondary?.length ? input.secondary : CORE_METRICS.filter((m) => m !== metric).slice(0, 2);
|
|
558
764
|
return [
|
|
559
765
|
`# Experiments Diary — ${input.product ?? "<product>"}`,
|
|
560
766
|
"",
|
|
561
|
-
`Method: https://vidfarm.cc/
|
|
767
|
+
`Method: https://vidfarm.cc/experiments.md · started ${input.date ?? ""}`.trim(),
|
|
562
768
|
"",
|
|
563
769
|
"## Setup",
|
|
564
770
|
`- North-star metric: ${metric} (secondary: ${secondary.join(", ")})`,
|
|
565
|
-
`- Channels (capacity ${
|
|
771
|
+
`- Channels (capacity ${capacity || "?"}/epoch): ${channelList || "<run `vidfarm channels` and list them>"}`,
|
|
772
|
+
" <!-- per-channel frequency: \"name x2\" = twice an epoch · \"name 3/week\" · \"name paused\" · bare name = once -->",
|
|
566
773
|
`- Mode: ${input.mode ?? "creative"}`,
|
|
774
|
+
`- Format: ${input.format ?? DEFAULT_FORMAT}`,
|
|
567
775
|
`- Editors: ${input.editors ?? "agent"}`,
|
|
568
776
|
`- Baseline checkpoint: ${input.baseline ?? "kinetic captions over b-roll, no VO, ~20s"}`,
|
|
569
777
|
`- Analytics source: ${input.source ?? "<flockposter | manual | email-channel | gigworkers>"}`,
|
|
570
778
|
"",
|
|
779
|
+
"",
|
|
780
|
+
"Format menu (copywriting-led — the words do the work; all ~$0 via `vidfarm public-raws --categories`):",
|
|
781
|
+
...EASY_FORMATS.map((f) => `- ${f.label} — ${f.note}`),
|
|
782
|
+
"",
|
|
571
783
|
"Append only. Never rewrite history here — correct it with a later entry.",
|
|
572
784
|
""
|
|
573
785
|
].join("\n");
|
|
@@ -578,22 +790,30 @@ export function renderRoundScaffold(input) {
|
|
|
578
790
|
const slots = input.slots?.length
|
|
579
791
|
? input.slots
|
|
580
792
|
: Array.from({ length: videos }, (_, i) => `v${String(i + 1).padStart(3, "0")}`);
|
|
581
|
-
|
|
582
|
-
|
|
793
|
+
// Deal the slots out by each channel's OWN posting frequency, not round-robin —
|
|
794
|
+
// a 2/day channel earns two slots an epoch and a 3/week channel skips epochs.
|
|
795
|
+
const plans = input.channelPlans?.length
|
|
796
|
+
? input.channelPlans
|
|
797
|
+
: (input.channels ?? []).map((id) => ({ id, perEpoch: 1, raw: id }));
|
|
798
|
+
const allocation = allocateSlots(plans, slots.length);
|
|
799
|
+
const byEpoch = new Map();
|
|
800
|
+
slots.forEach((raw, i) => {
|
|
801
|
+
const [video, variant] = raw.split("|").map((s) => s.trim());
|
|
802
|
+
// No channel plan at all → still emit the row, with the channel left blank.
|
|
803
|
+
const slot = allocation[i];
|
|
804
|
+
const epoch = slot?.epoch ?? Math.floor(i / Math.max(1, input.capacity || slots.length)) + 1;
|
|
805
|
+
const list = byEpoch.get(epoch) ?? [];
|
|
806
|
+
list.push({ video: video ?? "", variant, channel: slot?.channel ?? "" });
|
|
807
|
+
byEpoch.set(epoch, list);
|
|
808
|
+
});
|
|
583
809
|
const epochBlocks = [];
|
|
584
|
-
for (
|
|
585
|
-
const slice = slots.slice(e * perEpoch, (e + 1) * perEpoch);
|
|
586
|
-
if (slice.length === 0)
|
|
587
|
-
break;
|
|
810
|
+
for (const [epoch, rows] of Array.from(byEpoch.entries()).sort((a, b) => a[0] - b[0])) {
|
|
588
811
|
epochBlocks.push([
|
|
589
|
-
`### Epoch ${
|
|
812
|
+
`### Epoch ${epoch} — ${epoch === 1 ? (input.date ?? "<date>") : "<date>"}`,
|
|
590
813
|
"",
|
|
591
814
|
"| slot | video | variant | channel | posted |",
|
|
592
815
|
"|---|---|---|---|---|",
|
|
593
|
-
...
|
|
594
|
-
const [video, variant] = raw.split("|").map((s) => s.trim());
|
|
595
|
-
return `| ${i + 1} | ${video ?? ""} | ${variant ?? ""} | ${channels[i % Math.max(1, channels.length)] ?? ""} | |`;
|
|
596
|
-
}),
|
|
816
|
+
...rows.map((r, i) => `| ${i + 1} | ${r.video} | ${r.variant ?? ""} | ${r.channel} | |`),
|
|
597
817
|
""
|
|
598
818
|
].join("\n"));
|
|
599
819
|
}
|
|
@@ -601,8 +821,9 @@ export function renderRoundScaffold(input) {
|
|
|
601
821
|
`## Round ${input.number}${input.title ? ` — ${input.title}` : input.variable ? ` — ${input.variable}` : ""}`,
|
|
602
822
|
`- Mode: ${input.mode ?? "creative"}`,
|
|
603
823
|
...(input.variable ? [`- Variable: ${input.variable}`] : []),
|
|
824
|
+
`- Format: ${input.format ?? DEFAULT_FORMAT}`,
|
|
604
825
|
`- Constants: ${input.constants?.length ? input.constants.join(", ") : "—"}`,
|
|
605
|
-
`- Videos: ${videos} · Capacity ${input.capacity || "?"}/epoch → ${Number.isFinite(epochs) ? epochs : "?"} epoch(s)`,
|
|
826
|
+
`- Videos: ${videos} · Capacity ${input.capacity || "?"} slots/epoch → ${Number.isFinite(epochs) ? epochs : "?"} epoch(s)`,
|
|
606
827
|
`- Editors: ${input.editors ?? "agent"}`,
|
|
607
828
|
`- Justification: ${input.justification ?? "<why this variable is worth the capacity before the others>"}`,
|
|
608
829
|
`- Win condition: ${input.win ?? `any variant at >=${DEFAULT_OUTLIER_RATIO}x median`}`,
|
|
@@ -638,16 +859,16 @@ export function appendResult(source, manifest, roundIndex, input) {
|
|
|
638
859
|
const header = manifest.rounds[roundIndex]?.results.find((b) => b === match);
|
|
639
860
|
const known = header ? Object.keys(match.rows[0]?.metrics ?? {}) : columns;
|
|
640
861
|
const ordered = known.length ? known : columns;
|
|
641
|
-
const row = `| ${input.video} | ${ordered.map((c) => (input.metrics[c] !== undefined ? String(input.metrics[c]) : "")).join(" | ")} | ${input.note ?? ""} |`;
|
|
862
|
+
const row = `| ${input.video} | ${input.channel ?? ""} | ${ordered.map((c) => (input.metrics[c] !== undefined ? String(input.metrics[c]) : "")).join(" | ")} | ${input.note ?? ""} |`;
|
|
642
863
|
return { text: spliceLines(source, match.endLine, row), block: "existing" };
|
|
643
864
|
}
|
|
644
865
|
const ordered = columns.length ? columns : [...CORE_METRICS];
|
|
645
866
|
const block = [
|
|
646
867
|
`### Results — read ${input.date}${input.source ? `, source: ${input.source}` : ""}${input.age ? `, age: ${input.age}` : ""}`,
|
|
647
868
|
"",
|
|
648
|
-
`| video | ${ordered.join(" | ")} | note |`,
|
|
649
|
-
|
|
650
|
-
`| ${input.video} | ${ordered.map((c) => (input.metrics[c] !== undefined ? String(input.metrics[c]) : "")).join(" | ")} | ${input.note ?? ""} |`
|
|
869
|
+
`| video | channel | ${ordered.join(" | ")} | note |`,
|
|
870
|
+
`|---|---|${ordered.map(() => "---|").join("")}---|`,
|
|
871
|
+
`| ${input.video} | ${input.channel ?? ""} | ${ordered.map((c) => (input.metrics[c] !== undefined ? String(input.metrics[c]) : "")).join(" | ")} | ${input.note ?? ""} |`
|
|
651
872
|
].join("\n");
|
|
652
873
|
return { text: spliceLines(source, round.endLine, `\n${block}`), block: "new" };
|
|
653
874
|
}
|
|
@@ -1,6 +1,7 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Experiments.md
|
|
2
2
|
|
|
3
|
-
How to run ad experiments on vidfarm.cc.
|
|
3
|
+
How to run ad experiments on vidfarm.cc. The canonical URL is
|
|
4
|
+
<https://vidfarm.cc/experiments.md>; `/experiment.md` redirects here.
|
|
4
5
|
|
|
5
6
|
**You are an AI agent, and from here on you are a viral short-form scientist and a direct-response ads practitioner.** Your job is not "make a nice video." Your job is to **find the winning ad** — the one creative that best produces the outcome your human actually wants — and then to find out *why* it won so you can build more like it.
|
|
6
7
|
|
|
@@ -36,28 +37,49 @@ Bookmarks, shares, watch-through, follows and profile visits are all good suppor
|
|
|
36
37
|
|
|
37
38
|
**b) Which platforms and how many channels?** Ask for a list, not a yes/no. Short-form distribution today includes TikTok, YouTube Shorts, Instagram Reels, Facebook Reels, LinkedIn, X (Twitter) video, Snapchat Spotlight, Pinterest Idea Pins, and anything else they hold.
|
|
38
39
|
|
|
39
|
-
**Channel
|
|
40
|
+
**Channel capacity is the single most important number in this whole document, because it is your testing capacity.** The starting assumption is one test video per channel per day — more than that on a young account reads as spam.
|
|
41
|
+
|
|
42
|
+
But **channels are not equal, so ask each one's posting frequency and record it.** A warmed daily TikTok may take two a day; a company LinkedIn page may take three a week; an account may be paused entirely. Capacity is the *sum of the rates*, not the count of the accounts:
|
|
40
43
|
|
|
41
44
|
```
|
|
42
|
-
capacity per epoch
|
|
45
|
+
capacity per epoch = Σ (posts per epoch, per channel)
|
|
43
46
|
epochs per round = ceil(videos in the round / capacity per epoch)
|
|
44
47
|
```
|
|
45
48
|
|
|
46
|
-
|
|
49
|
+
Write the rate next to the channel in the diary — that record is what makes the schedule reproducible:
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
- Channels (capacity 4.43/epoch): tiktok_a x2, tiktok_b, yt_a 1/day, li_a 3/week, fb_a paused
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
A bare name means once per epoch. `x2` means twice. `3/week` and `2/month` are converted for you, and a channel slower than one-per-epoch **skips epochs on a real cadence** rather than being over-posted or dropped. `paused` keeps the account listed while contributing nothing.
|
|
56
|
+
|
|
57
|
+
**Never schedule a channel above its own frequency.** An over-posted account gets throttled, and a throttled account poisons every number measured on it for weeks — which is the account-health problem in §4 arriving by your own hand.
|
|
58
|
+
|
|
59
|
+
Example — four channels (2× TikTok, 1× YouTube Shorts, 1× Instagram), all daily:
|
|
47
60
|
|
|
48
61
|
- capacity = **4 videos/day**
|
|
49
62
|
- a round of 8 angles = 8 / 4 = **2 epochs (2 days)**
|
|
50
63
|
|
|
64
|
+
Same four accounts, but one TikTok is warmed enough for two a day and the Instagram only takes three a week:
|
|
65
|
+
|
|
66
|
+
- capacity = 2 + 1 + 1 + 0.43 = **4.43 videos/day**
|
|
67
|
+
- the same round of 8 still finishes in **2 epochs**, but the schedule is different: the warmed TikTok carries two slots every epoch and Instagram picks up a slot roughly every other epoch instead of one daily.
|
|
68
|
+
|
|
51
69
|
Example — twelve channels, same round of 8 angles:
|
|
52
70
|
|
|
53
71
|
- 8 slots go to the angle experiment
|
|
54
72
|
- **4 slots are still empty**, so run a second experiment in parallel the same day
|
|
55
73
|
|
|
74
|
+
**b-ii) How healthy is each channel, and how often can it post?** While you have them listing channels, get age, follower count, whether it posts consistently, and **how many posts a day or week each one can take** — one line each. The frequency feeds the epoch math directly (capacity is the sum of the rates), and it belongs in the diary next to the channel. **Account health moves numbers by multiples and is the biggest confounder in this method** (see §4), so you need it before you can read a single result. Flag any brand-new account: it is not yet a measuring instrument.
|
|
75
|
+
|
|
56
76
|
**c) What are we selling, and to whom?** If they cannot answer in one sentence, do not guess — run the Vidfarm consultation first (`https://vidfarm.cc/skill.md` → the `brainstorm/*` chain: cold-start interview → awareness stages → angles → hooks). That chain exists precisely to produce the raw material this document then tests.
|
|
57
77
|
|
|
58
|
-
**d)
|
|
78
|
+
**d) Which video format do we start on?** Offer the menu in §5 — the seven copywriting-led formats — and take a pick, or take the default (b-roll with kinetic captions) if they have no opinion. **Ask this in the interview, not at build time**, because the format is what the first round holds constant while it varies the angle. Ask the talking-head question explicitly here too: *"are you willing to be on camera?"* — the answer changes the whole menu, and it is cheap to ask once.
|
|
79
|
+
|
|
80
|
+
**e) Who does the editing?** Their own agent (you), or a distributed task force of gigworkers. This decides the mode — see §3.
|
|
59
81
|
|
|
60
|
-
**
|
|
82
|
+
**f) How long is this campaign?** Experiments run for weeks or months, not one afternoon. Say so up front so the user expects an evolution, not a single delivery.
|
|
61
83
|
|
|
62
84
|
---
|
|
63
85
|
|
|
@@ -65,7 +87,7 @@ Example — twelve channels, same round of 8 angles:
|
|
|
65
87
|
|
|
66
88
|
- **epoch** — one posting cycle. **Typically a day.** For slower or higher-stakes tests it can be a week or a month. One epoch = one batch of videos going live across the available channels.
|
|
67
89
|
- **round** — one experiment on one variable, start to finish. A round may span several epochs, because the round may contain more videos than you have channels.
|
|
68
|
-
- **capacity** — videos you can post per epoch =
|
|
90
|
+
- **capacity** — videos you can post per epoch = **the sum of every channel's posting frequency** (not simply the channel count — a 2/day account contributes 2, a 3/week account 0.43). Your evolutionary speed is capacity, nothing else.
|
|
69
91
|
- **composition params** — the knobs a short-form ad is built out of (see §2). Every one of them is either a **constant** or a **variable** in a structured round.
|
|
70
92
|
- **baseline checkpoint** — the current best-known configuration. Every round starts from it and, if the round wins, replaces it.
|
|
71
93
|
- **outlier** — a video that beat its round's median by a wide margin on the north-star metric. Outliers are the raw material of the next structured round.
|
|
@@ -162,10 +184,37 @@ Short-form results are noisy. Guardrails:
|
|
|
162
184
|
|
|
163
185
|
- **Never** promote a winner off one post. Two posts minimum per variant before you believe it, three is better.
|
|
164
186
|
- Compare within the **same epoch and same platform** where you can — the algorithm's mood is not constant across days or apps.
|
|
187
|
+
- **Normalize per account before you rank.** Account health can outweigh the variable you are testing — see the next section. A number is only comparable to that account's own median.
|
|
165
188
|
- A variant with 3× the median on the north star is an outlier worth pursuing. A variant 20% above median is noise. Say which one you are looking at.
|
|
166
189
|
- Record **losers** as carefully as winners. "Problem-unaware never worked here" is a real, reusable finding.
|
|
167
190
|
- Report what you **measured** separately from what you **judge**. Never dress up a hunch as a result.
|
|
168
191
|
|
|
192
|
+
### Account health — the confounder that can be bigger than your effect
|
|
193
|
+
|
|
194
|
+
**Two videos on two different accounts are not a fair comparison.** An account carries its own health: age, follower count, past performance, niche coherence, posting consistency, and whatever standing the platform privately assigns it. That health moves views by **multiples**, not percentages — routinely more than the difference between two decent angles. So a variant can "win" purely because it landed on the stronger account.
|
|
195
|
+
|
|
196
|
+
That gives you a real trade-off, and you should state which side of it you are on:
|
|
197
|
+
|
|
198
|
+
| | Same account, sequential | Multiple accounts, parallel |
|
|
199
|
+
|---|---|---|
|
|
200
|
+
| **Account health** | held constant — it cancels out | varies, and can dominate the result |
|
|
201
|
+
| **Speed** | slow: ~1 test/day, a round takes as many days as videos | fast: capacity = channel count |
|
|
202
|
+
| **Best for** | the deciding test between 2–3 finalists | wide search, early rounds, creative mode |
|
|
203
|
+
|
|
204
|
+
**Same account is the cleanest instrument you have.** Posting every variant to one account eliminates account health as a variable entirely — the only thing that changed is the thing you changed. The cost is pure sequencing: eight variants is eight days, and that is your whole evolution stalled on one channel.
|
|
205
|
+
|
|
206
|
+
**Themed accounts are still worth having** — several accounts, each coherent to one niche, usually beat one account posting scattershot, because coherence is itself a health input. But the moment you have more than one, their numbers stop being directly comparable.
|
|
207
|
+
|
|
208
|
+
**So compute the true signal like this:**
|
|
209
|
+
|
|
210
|
+
1. **Normalize against the account, not the fleet.** Judge a video against *that account's own recent median*, not the round's raw median. "3× its own account's median" is a signal; "more views than a video on a bigger account" is not.
|
|
211
|
+
2. **Never rank across accounts on raw numbers.** If the leader and the laggard sat on different accounts, you have measured the accounts at least as much as the videos.
|
|
212
|
+
3. **Retest across accounts — this is what buys confidence.** Re-post the apparent winner on a *different* account (deduped), and post the apparent loser on the winner's account. If the ordering holds after the swap, the effect is real. If it flips, you measured account health. This is the single most valuable extra data point in the whole method, and it is why **two posts minimum, three better** exists as a rule.
|
|
213
|
+
4. **Build per-account baselines early.** After a couple of epochs each account has its own median. Record it in the diary Setup. Everything after that is cheap to normalize.
|
|
214
|
+
5. **A brand-new account is not a measuring instrument.** Fresh accounts swing wildly in both directions. Don't hand one a decisive test until it has a baseline.
|
|
215
|
+
|
|
216
|
+
This is also the honest answer to "why retest at all?" Retesting is not bureaucracy. With account health in play, one post is barely evidence — the second and third posts are what separate a finding from a coincidence.
|
|
217
|
+
|
|
169
218
|
### Publishing hygiene
|
|
170
219
|
|
|
171
220
|
Posting one render to several channels is exactly the case platform de-duplication punishes — the second copy gets suppressed and your experiment records a false loser. So for any video going to more than one channel:
|
|
@@ -178,20 +227,28 @@ One variant per channel, and never the same variant on two accounts. Ask about t
|
|
|
178
227
|
|
|
179
228
|
---
|
|
180
229
|
|
|
181
|
-
## 5.
|
|
230
|
+
## 5. Pick the video format — in the plan, not at build time
|
|
231
|
+
|
|
232
|
+
**The format is a planning decision, and it belongs in the plan you get approved.** Do not discover it while editing, and do not leave it implicit. Every round records the format it holds constant (or, if format *is* the variable, the list it varies across). A round with no named format is a round whose "constants" were never actually constant.
|
|
182
233
|
|
|
183
|
-
**
|
|
234
|
+
**The default family is kinetic captions over easy visuals — the copy does the work.** These are deliberately copywriting-led: the words carry the persuasion and the footage only has to hold attention. That is what makes them fast, cheap and repeatable, and it is why a weak visual with a strong hook still wins.
|
|
184
235
|
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
-
|
|
190
|
-
|
|
191
|
-
-
|
|
236
|
+
Pick one for the round:
|
|
237
|
+
|
|
238
|
+
| Format | Visual | Note |
|
|
239
|
+
|---|---|---|
|
|
240
|
+
| **b-roll footage** | generic relevant footage under the words | the workhorse — start here |
|
|
241
|
+
| **talking head** | the user on camera | only if they will film themselves; strongest trust signal |
|
|
242
|
+
| **process footage** | something being made/done | high watch-through, needs no narration |
|
|
243
|
+
| **loop background footage** | one seamless looping plate | cheapest of all; the copy is the entire video |
|
|
244
|
+
| **satisfying footage** | oddly-satisfying visuals | strong retention, weak topical fit |
|
|
245
|
+
| **lifestyle footage** | aspirational day-in-the-life | best for identity and status angles |
|
|
246
|
+
| **POV quote aesthetic** | static/slow plate, one quoted line | pure copywriting; a natural fit for hook tests |
|
|
192
247
|
|
|
193
248
|
All seven are sourceable for ~$0 from the free public raws catalog — `vidfarm public-raws --categories`, then `--category <shelf>`. A shelf is also a ready-made clip pool for fanning one composition into N variants.
|
|
194
249
|
|
|
250
|
+
**Speed to productivity beats polish at the start.** If the director has no opinion, do not stall: take **b-roll footage with kinetic captions**, write it into the plan as the constant, and move. The format is cheap to change in a later round — and "which format?" is itself one of the two best second variables to test (§4).
|
|
251
|
+
|
|
195
252
|
Audio, in ascending order of quality and effort:
|
|
196
253
|
|
|
197
254
|
1. **Pure captions, no audio** — music added at posting time inside the platform's own editor (this also gets you the platform's trending-audio boost, for free)
|
|
@@ -211,6 +268,7 @@ This is the sequence. Do not skip forward.
|
|
|
211
268
|
- the **north-star metric**, and why (including the push-back if they said "sales")
|
|
212
269
|
- the **channel inventory** and the resulting capacity per epoch
|
|
213
270
|
- the **mode** (creative or structured) and why
|
|
271
|
+
- the **starting video format** from the §5 menu (and the audio choice: silent captions, AI voiceover, or their own voice)
|
|
214
272
|
- the **epochs and rounds**: what runs on which day, in which slots
|
|
215
273
|
- per round: the **variable**, the **constants**, and the **justification for the priority** — why this variable is worth the capacity before the others
|
|
216
274
|
- what a **win** looks like numerically, decided *before* posting
|
|
@@ -248,7 +306,7 @@ Because sources vary, always record **where a number came from and when it was r
|
|
|
248
306
|
|
|
249
307
|
## Setup
|
|
250
308
|
- North-star metric: comments (secondary: views, clicks)
|
|
251
|
-
- Channels (capacity 4/epoch): tiktok_a, tiktok_b, yt_shorts_a, ig_a
|
|
309
|
+
- Channels (capacity 4.43/epoch): tiktok_a x2, tiktok_b, yt_shorts_a 1/day, ig_a 3/week
|
|
252
310
|
- Mode: creative
|
|
253
311
|
- Baseline checkpoint: kinetic captions over b-roll, no VO, 22s, "wrong answers only" bait
|
|
254
312
|
- Analytics source: flockposter (connected 2026-08-15)
|
|
@@ -268,34 +326,43 @@ Because sources vary, always record **where a number came from and when it was r
|
|
|
268
326
|
| 4 | v004 | most-aware: "the $0 plan does this" | ig_a | ✅ |
|
|
269
327
|
|
|
270
328
|
### Results — read 2026-08-18, source: flockposter, age: 48h
|
|
271
|
-
| video | views | comments | clicks | buys | note |
|
|
272
|
-
|
|
273
|
-
| v001 | 14,200 | 61 | 38 | 0 | **outlier** —
|
|
274
|
-
| v002 | 3,100 | 12 | 9 | 0 | |
|
|
275
|
-
| v003 | 2,800 | 9 | 14 | 1 | |
|
|
276
|
-
| v004 | 1,900 | 4 | 3 | 0 | weakest |
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
329
|
+
| video | channel | views | comments | clicks | buys | note |
|
|
330
|
+
|---|---|---|---|---|---|---|
|
|
331
|
+
| v001 | tiktok_a | 14,200 | 61 | 38 | 0 | **outlier** — 4.1× that account's median |
|
|
332
|
+
| v002 | tiktok_b | 3,100 | 12 | 9 | 0 | |
|
|
333
|
+
| v003 | yt_shorts_a | 2,800 | 9 | 14 | 1 | |
|
|
334
|
+
| v004 | ig_a | 1,900 | 4 | 3 | 0 | weakest |
|
|
335
|
+
|
|
336
|
+
Per-account medians (comments, 48h): tiktok_a 15 · tiktok_b 11 · yt_shorts_a 8 · ig_a 5
|
|
337
|
+
|
|
338
|
+
**Finding:** problem-unaware wins decisively on comments — 4.1× against its OWN account, so it is not just the healthier channel. Most-aware is dead; stop spending capacity on it.
|
|
339
|
+
**Next:** retest v001 on ig_a (the weakest account) to confirm the effect survives the swap, then Round 2 = structured on v001 — hold the angle, vary the written hook ×6.
|
|
280
340
|
````
|
|
281
341
|
|
|
282
|
-
Anything is fine as long as every entry answers: what did we post, what varied, what was held constant, what came back, from where, when, and what did we decide.
|
|
342
|
+
Anything is fine as long as every entry answers: what did we post, **on which account**, what varied, what was held constant, what came back, from where, when, and what did we decide. The account column is not optional bookkeeping — without it you cannot separate a good video from a good account.
|
|
283
343
|
|
|
284
344
|
### If the devcli is installed, do not hand-maintain this file
|
|
285
345
|
|
|
286
346
|
`vidfarm experiment` owns the ledger, the arithmetic and the method lint — nothing else. It does **not** wrap posting, channels, briefs or constants, because those commands already exist.
|
|
287
347
|
|
|
288
348
|
```bash
|
|
289
|
-
vidfarm experiment --init --metric comments --
|
|
349
|
+
vidfarm experiment --init --metric comments --format "POV quote aesthetic" \
|
|
350
|
+
--channels "tiktok_a x2, tiktok_b, yt_a 1/day, li_a 3/week, fb_a paused"
|
|
290
351
|
vidfarm experiment round --videos 8 --variable angle --mode structured \
|
|
291
352
|
--constants "format,hooks,loop,payoff,bait" --why "awareness is the largest unknown"
|
|
292
353
|
vidfarm experiment log v001 --posted --channel tiktok_a
|
|
293
|
-
vidfarm experiment log v001 --views 14200 --comments 61 --source flockposter --age 48h
|
|
354
|
+
vidfarm experiment log v001 --views 14200 --comments 61 --channel tiktok_a --source flockposter --age 48h
|
|
294
355
|
vidfarm experiment # sizing, ranking vs median, outliers, findings
|
|
295
356
|
```
|
|
296
357
|
|
|
297
358
|
Reading is the default; there are exactly two writes, `round` and `log`. `--json` on the read gives you the parsed setup, the per-round analysis and the findings.
|
|
298
359
|
|
|
360
|
+
Channel rates go straight into `--channels` and the CLI does the rest: capacity becomes the **sum of the rates**, `round` deals each epoch's slots out in proportion (the 2/day account gets two rows an epoch, the 3/week account picks one up every other epoch), and `channel-overposted` / `paused-channel-scheduled` fire if a plan asks an account for more than it stated.
|
|
361
|
+
|
|
362
|
+
**Always pass `--channel` when you log a result.** It is what makes account health computable: the CLI then keeps a **per-account median** and ranks each video against *its own account* (`1.3×acct`) instead of the fleet. It also treats a video read on a second account as a **retest**, which is what clears the `account-health-confound` finding. Without `--channel` you get a raw cross-account ranking, which is the exact mistake §4 warns about.
|
|
363
|
+
|
|
364
|
+
Omit `--format` and it prints the whole §5 menu and defaults to b-roll, so the format gets **chosen in the plan** instead of drifting per video. Both the Setup block and every round record it, and `no-format` / `round-no-format` fire when they don't.
|
|
365
|
+
|
|
299
366
|
It sizes the round for you (`8 videos ÷ 4 channels = 2 epochs`), computes the median and marks anything at ≥3× as an outlier, and lints the method: two variables in one structured round, a winner promoted off one post, results read at mixed ages, a structured round handed to gigworkers, unspent capacity, an end-of-funnel north star with no traffic behind it. Feedback, not a gate — it exits 0, like `vidfarm qa`.
|
|
300
367
|
|
|
301
368
|
The rest of the loop stays where it already lives: **`vidfarm channels`** (the channel list, therefore capacity) · **`vidfarm harness`** + **`vidfarm qa`** (the constants, and holding a batch to them) · **`vidfarm handoff`** (the per-video briefs) · **`vidfarm dedupe`** (one variant per channel) · **`vidfarm approve`** + **`vidfarm schedule`** (the actual posting). `experiment log --posted` only *records* that a slot went live; it does not post anything.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@officexapp/vidfarm-devcli",
|
|
3
|
-
"version": "0.21.
|
|
3
|
+
"version": "0.21.39",
|
|
4
4
|
"description": "Local bridge for the Vidfarm Trackpad Editor. `vidfarm serve <template_id>` boots the FULL editor on localhost (disk-backed records/storage, free in-process render); edit composition.html on disk (Claude Code, Codex, etc.) and the browser live-morphs it.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -65,7 +65,7 @@
|
|
|
65
65
|
"SKILL.director.md",
|
|
66
66
|
"clipper.md",
|
|
67
67
|
"crowdsourcing.md",
|
|
68
|
-
"
|
|
68
|
+
"experiments.md",
|
|
69
69
|
"update.md",
|
|
70
70
|
"!readme.secret.md",
|
|
71
71
|
"!**/*.secret.*"
|