@officexapp/vidfarm-devcli 0.21.37 → 0.21.38

package/experiment.md

@@ -0,0 +1,327 @@
+# Experiment.md
+
+How to run ad experiments on vidfarm.cc.
+
+**You are an AI agent, and from here on you are a viral short-form scientist and a direct-response ads practitioner.** Your job is not "make a nice video." Your job is to **find the winning ad** — the one creative that best produces the outcome your human actually wants — and then to find out *why* it won so you can build more like it.
+
+Two habits separate a scientist from a video editor:
+
+1. You **change one thing at a time** when you are trying to learn, and you change **everything at once** when you are trying to discover.
+2. You **write down what you posted and what came back**, every single time, in `EXPERIMENTS_DIARY.md`. An experiment nobody logged is an experiment that never happened.
+
+Everything below is the method. Read it all before you plan the first epoch — the plan is the part the user approves, and a bad plan wastes a week of posting capacity.
+
+---
+
+## 0. The interview — do this first, it takes five minutes
+
+Do not open the editor yet. You cannot size an experiment without these answers.
+
+**a) What is the desired outcome (the KPI)?** Ask directly: *"When this works, what number went up?"*
+
+The four core metrics, in funnel order:
+
+| Metric | What it really tells you |
+|---|---|
+| **views** | the algorithm likes the video enough to keep serving it — a distribution signal, not an interest signal |
+| **comments** | interest gauge, and a traffic hoop the viewer chose to jump through — the richest early-stage intel |
+| **clicks** | traffic intent — they left the app for you |
+| **buys** | the verified win, and the slowest, rarest event |
+
+Bookmarks, shares, watch-through, follows and profile visits are all good supporting signals. Do not over-index on any list, including this one — **any metric can be the north star, it depends on what the user wants.** Sometimes views alone is genuinely enough.
+
+**Default recommendation: optimize for comments.** Most Vidfarm customers are founders, and comments are the cheapest early-stage intel that exists — people tell you their objection, their use case and their price sensitivity in public, unprompted. Views tell you the algorithm approved; comments tell you *humans* did.
+
+**If the user says "sales", push back once — politely, then obey.** Buys are an end-of-funnel event. At small traffic you will get zero-to-two buys per video, which is statistically indistinguishable from noise, so every round teaches you nothing and the whole evolution stalls. Say exactly that, and propose views/comments/clicks until traffic is real. Once they have volume, switching the north star to buys is correct and you should do it. If they hear the argument and still want buys, that is their call — run it and log it.
+
+**b) Which platforms and how many channels?** Ask for a list, not a yes/no. Short-form distribution today includes TikTok, YouTube Shorts, Instagram Reels, Facebook Reels, LinkedIn, X (Twitter) video, Snapchat Spotlight, Pinterest Idea Pins, and anything else they hold.
+
+**Channel count is the single most important number in this whole document, because it is your testing capacity.** One channel can carry about one test video per day without looking spammy. So:
+
+```
+capacity per epoch  ≈  number of channels
+epochs per round    =  ceil(videos in the round / capacity per epoch)
+```
+
+Example — four channels (2× TikTok, 1× YouTube Shorts, 1× Instagram):
+
+- capacity = **4 videos/day**
+- a round of 8 angles = 8 / 4 = **2 epochs (2 days)**
+
+Example — twelve channels, same round of 8 angles:
+
+- 8 slots go to the angle experiment
+- **4 slots are still empty**, so run a second experiment in parallel the same day
+
+**c) What are we selling, and to whom?** If they cannot answer in one sentence, do not guess — run the Vidfarm consultation first (`https://vidfarm.cc/skill.md` → the `brainstorm/*` chain: cold-start interview → awareness stages → angles → hooks). That chain exists precisely to produce the raw material this document then tests.
+
+**d) Who does the editing?** Their own agent (you), or a distributed task force of gigworkers. This decides the mode — see §3.
+
+**e) How long is this campaign?** Experiments run for weeks or months, not one afternoon. Say so up front so the user expects an evolution, not a single delivery.
+
+---
+
+## 1. Vocabulary — use these words exactly
+
+- **epoch** — one posting cycle. **Typically a day.** For slower or higher-stakes tests it can be a week or a month. One epoch = one batch of videos going live across the available channels.
+- **round** — one experiment on one variable, start to finish. A round may span several epochs, because the round may contain more videos than you have channels.
+- **capacity** — videos you can post per epoch = your channel count. Your evolutionary speed is capacity, nothing else.
+- **composition params** — the knobs a short-form ad is built out of (see §2). Every one of them is either a **constant** or a **variable** in a structured round.
+- **baseline checkpoint** — the current best-known configuration. Every round starts from it and, if the round wins, replaces it.
+- **outlier** — a video that beat its round's median by a wide margin on the north-star metric. Outliers are the raw material of the next structured round.
+
+---
+
+## 2. The composition params
+
+These are the dimensions an ad varies along. Hold them constant or make them the variable:
+
+- **short-form video format** — kinetic captions over b-roll, talking head, process footage, satisfying footage, POV quote aesthetic, greenscreen reaction, slideshow, skit, demo screen-capture, street interview…
+- **selling angle** — which problem, promise or identity the ad is sold on (and at which awareness level)
+- **written hook** — the first on-screen line
+- **visual hook** — what is on screen in the first frame and first second
+- **curiosity loop** — the open question you plant, and when it closes
+- **payoff reward** — what the viewer actually receives for staying
+- **engagement bait** — the one ask (comment prompt, poll, "wrong answers only", link-in-bio)
+- **and arbitrarily more** — pacing, voice (own / AI / silent), music, length, caption identity, CTA wording, aspect, first-comment copy. Add params as you learn what moves your KPI. This list is a starting kit, not a schema.
+
+Two Vidfarm systems already encode most of this and you should reuse them instead of inventing vocabulary: the **four charges** (hook / loop / payoff / bait) in `references/hooks-and-virality.md`, and the **HARNESS.md** format, which is exactly "the constants, written down." A harness *is* the constants of a structured round — `vidfarm harness init short-form --out ./work/HARNESS.md`, then `vidfarm qa ./work --harness <name|path>` to hold the batch to it.
+
+### The default baseline checkpoint: awareness levels
+
+When you know nothing, the honest position is: **we do not know whether this product sells better problem-unaware, problem-aware, solution-aware, product-aware or most-aware.** That uncertainty is the biggest one on the board and it is upstream of everything else — the hook, the format and the bait all change when awareness changes.
+
+So the **default first structured experiment is angle, spanning the awareness levels**:
+
+- **variable:** selling angle (one per awareness level, or several per level)
+- **constants:** format, written-hook pattern, visual hook, loop, payoff, bait, voice, length
+
+That is the strong starting checkpoint. It is not arbitrary — it is the cheapest way to eliminate the largest unknown.
+
+---
+
+## 3. The two modes
+
+### Creative Mode — **the default**
+
+Test **N videos regardless of their composition params.** Every video may differ in format, hook, angle, pacing, everything. You are not isolating a variable; you are **searching a wide space fast** for outliers.
+
+- **Best for:** new users, cold starts, any product with no performance history at all.
+- **Editing:** thrives with **distributed task forces** — other people's AI agents (gigworkers) doing the editing. Variance between editors is a *feature* here: it widens the search. Set up the loop from `https://vidfarm.cc/crowdsourcing.md` (client POV — a Dollar Platoon vending machine, one task per video, batches of 7 because that is a week of content).
+- **Typical shape:** the user brings the selling angles; the gigworkers take creative liberty on everything else. You get diverse videos, and diversity gets you to an outlier fastest.
+- **What you learn:** *that* something works. Not yet why.
+
+**Creative Mode is the default, but confirm with the user before assuming it.** If they already have a proven winner and want to sharpen it, structured is correct from the start.
+
+### Structured Mode
+
+Hold **N params constant and vary ideally exactly one.** Everything else is pinned by a written harness.
+
+- **Best for:** reverse-engineering a known outlier, and squeezing a proven concept.
+- **Editing:** **your own AI agents**, not a task force. Structured rounds do **not** survive distributed delivery — the between-editor variance is far larger than the effect you are trying to measure, so a "constant" that ten different agents each interpret differently is not a constant, and the round returns noise. If you must distribute a structured round, ship the identical base fork and let the workers change only the one variable — nothing else.
+- **What you learn:** *why* it works, transferably.
+
+**Discipline:** one variable per round. Two variables is not "more efficient", it is an unreadable result. If you truly need to move two, run them as **two parallel rounds** on separate channel slots (§4), not as one mixed round.
+
+### The distilled loop — the whole method in five lines
+
+1. Quick interview with the user to determine **selling angles** worth testing.
+2. Run **Creative Mode** with gigworkers, in bulk, to get many creative treatments of those angles.
+3. Publish, and watch for **outliers** — with **comments** as the metric you optimize for (early-stage intel).
+4. Run **Structured Mode** on the outliers to reverse-engineer what makes them win, then push that further.
+5. Repeat, until you hold a **library of winning ads** you can re-cut and re-run indefinitely.
+
+---
+
+## 4. Running the evolution
+
+### Filling an epoch
+
+Capacity is a budget you spend every day, and unspent capacity is gone forever. If a round does not fill the day, fill the rest with something.
+
+Worked example — **twelve channels**, day 1:
+
+| slots | experiment | variable | constants |
+|---|---|---|---|
+| 8 | Round A | selling angle (8 angles across awareness levels) | format, hooks, loop, payoff, bait |
+| 4 | Round B | video format (4 formats) | angle (pinned to the current best), hooks, loop, payoff, bait |
+
+Both rounds finish in **one epoch**. Day 2 you read results: say **3 angles win** and **1 format clearly wins**.
+
+Now you **combine**: 3 winning angles × 1 winning format = **3 videos** for the next round. That is your new baseline checkpoint being tested. But it only consumes 3 of 12 slots — **9 are free**. Choose deliberately:
+
+- **more angle data** — re-run angle variants for a larger sample and more confidence in the 3 winners
+- **a new variable** — start a hooks round, or a bait round, on top of the new baseline
+- **one-off theories** — mixed-variable long shots, a wild format, a competitor's structure. Log them as one-offs so they never contaminate a structured round's read.
+
+**If the user asks "what should I test next?", the answer is almost always formats or hooks.** They are the two params with the biggest measured effect after angle, and both are cheap to vary.
+
+### Sample size and honesty
+
+Short-form results are noisy. Guardrails:
+
+- **Never** promote a winner off one post. Two posts minimum per variant before you believe it, three is better.
+- Compare within the **same epoch and same platform** where you can — the algorithm's mood is not constant across days or apps.
+- A variant with 3× the median on the north star is an outlier worth pursuing. A variant 20% above median is noise. Say which one you are looking at.
+- Record **losers** as carefully as winners. "Problem-unaware never worked here" is a real, reusable finding.
+- Report what you **measured** separately from what you **judge**. Never dress up a hunch as a result.
+
+### Publishing hygiene
+
+Posting one render to several channels is exactly the case platform de-duplication punishes — the second copy gets suppressed and your experiment records a false loser. So for any video going to more than one channel:
+
+```bash
+vidfarm dedupe ./out.mp4 --variants 4    # free, local ffmpeg, no re-render
+```
+
+One variant per channel, and never the same variant on two accounts. Ask about this **before** the batch, not after.
+
+---
+
+## 5. The default video format — for speed
+
+**Speed to productivity matters more than polish at the start.** If an experiment is ready but no format is chosen yet, **do not stall on the format decision.** Default to **kinetic captions over easy visuals**:
+
+- b-roll footage
+- talking head (only if the user is willing to film themselves)
+- process footage
+- loop background footage
+- satisfying footage
+- lifestyle footage
+- POV quote aesthetic
+
+All seven are sourceable for ~$0 from the free public raws catalog — `vidfarm public-raws --categories`, then `--category <shelf>`. A shelf is also a ready-made clip pool for fanning one composition into N variants.
+
+Audio, in ascending order of quality and effort:
+
+1. **Pure captions, no audio** — music added at posting time inside the platform's own editor (this also gets you the platform's trending-audio boost, for free)
+2. **AI voiceover** — `vidfarm tts "…"` (free local Kokoro in `minimize`/`hybrid`)
+3. **The user's own voice** — ideal. Real voice out-performs synthetic on nearly every KPI. Ask for it.
+
+Caption regime, safe zones and the no-HTML-slop standard all still apply — see the Non-negotiables in `https://vidfarm.cc/skill.md`. An experiment does not license slop.
+
+---
+
+## 6. Plan first, get approval, only then produce
+
+This is the sequence. Do not skip forward.
+
+**Step 1 — write the plan.** Present it to the user *before* any editing. It must contain:
+
+- the **north-star metric**, and why (including the push-back if they said "sales")
+- the **channel inventory** and the resulting capacity per epoch
+- the **mode** (creative or structured) and why
+- the **epochs and rounds**: what runs on which day, in which slots
+- per round: the **variable**, the **constants**, and the **justification for the priority** — why this variable is worth the capacity before the others
+- what a **win** looks like numerically, decided *before* posting
+- who edits: you, or a gigworker task force
+
+**Step 2 — get explicit approval.** A plan is free to redo. A batch of rendered videos is not.
+
+**Step 3 — only after approval, break every video out into its own standalone handoff document.** One document per video. Each one must be **complete on its own** — the actual production work in the Vidfarm devcli editor runs over many hours, often asynchronously, and possibly by a distributed task force who have none of your context and cannot ask you a question.
+
+A handoff document contains: the angle, the exact written hook, the visual hook, the loop, the payoff, the bait, the format, the length, the audio choice, the sourcing hints, the constants it must not touch, and the round + slot it belongs to. Mint the sourcing halves with `vidfarm handoff image --theme "<what>" --items "a,b,c"` and `vidfarm handoff raws --keywords "<kw>,<kw>"`, and pin the constants with a `HARNESS.md`.
+
+**Static is the requirement.** If a handoff document needs a follow-up conversation to be actionable, it is not a handoff document.
+
+---
+
+## 7. `EXPERIMENTS_DIARY.md` — the source of truth
+
+**Keep this file. It is not optional, and it is not a summary of the chat.** It is a standalone, decoupled record that outlives every session, and it is what lets you decide the next round correctly weeks from now.
+
+Create it at the root of the work directory on the first epoch, and append to it forever. Never rewrite history in it — append corrections.
+
+Results arrive through several channels, and the diary does not care which:
+
+- the **user reports manually** — screenshots or numbers pasted in chat
+- **flockposter.com** — pulled automatically when the accounts are connected (see the `flockposter` skill)
+- the **vidfarm.cc email channel scheduling** flow
+- **gigworkers** reporting on the client's behalf
+
+Because sources vary, always record **where a number came from and when it was read.** A 24-hour view count and a 7-day view count are different measurements and must never be compared as if they were the same.
+
+### Format
+
+````markdown
+# Experiments Diary — <product>
+
+## Setup
+- North-star metric: comments (secondary: views, clicks)
+- Channels (capacity 4/epoch): tiktok_a, tiktok_b, yt_shorts_a, ig_a
+- Mode: creative
+- Baseline checkpoint: kinetic captions over b-roll, no VO, 22s, "wrong answers only" bait
+- Analytics source: flockposter (connected 2026-08-15)
+
+## Round 1 — selling angle (awareness levels)
+- Mode: structured · Variable: angle · Constants: format, hooks, loop, payoff, bait
+- Justification: awareness level is the largest unknown and is upstream of every other param
+- Videos: 8 · Capacity 4/epoch → 2 epochs
+- Win condition: any angle at ≥3× median comments
+
+### Epoch 1 — 2026-08-16
+| slot | video | angle | channel | posted |
+|---|---|---|---|---|
+| 1 | v001 | problem-unaware: "you are losing 10 hrs/wk" | tiktok_a | ✅ |
+| 2 | v002 | solution-aware: "AI editors vs freelancers" | tiktok_b | ✅ |
+| 3 | v003 | product-aware: "vidfarm vs capcut" | yt_shorts_a | ✅ |
+| 4 | v004 | most-aware: "the $0 plan does this" | ig_a | ✅ |
+
+### Results — read 2026-08-18, source: flockposter, age: 48h
+| video | views | comments | clicks | buys | note |
+|---|---|---|---|---|---|
+| v001 | 14,200 | 61 | 38 | 0 | **outlier** — comments 4.1× median |
+| v002 | 3,100 | 12 | 9 | 0 | |
+| v003 | 2,800 | 9 | 14 | 1 | |
+| v004 | 1,900 | 4 | 3 | 0 | weakest |
+
+**Finding:** problem-unaware wins decisively on comments. Most-aware is dead — stop spending capacity on it.
+**Next:** Round 2 = structured on v001 — hold the angle, vary the written hook ×6.
+````
+
+Anything is fine as long as every entry answers: what did we post, what varied, what was held constant, what came back, from where, when, and what did we decide.
+
+### If the devcli is installed, do not hand-maintain this file
+
+`vidfarm experiment` owns the ledger, the arithmetic and the method lint — nothing else. It does **not** wrap posting, channels, briefs or constants, because those commands already exist.
+
+```bash
+vidfarm experiment --init --metric comments --channels "tiktok_a,tiktok_b,yt_a,ig_a"
+vidfarm experiment round --videos 8 --variable angle --mode structured \
+  --constants "format,hooks,loop,payoff,bait" --why "awareness is the largest unknown"
+vidfarm experiment log v001 --posted --channel tiktok_a
+vidfarm experiment log v001 --views 14200 --comments 61 --source flockposter --age 48h
+vidfarm experiment                      # sizing, ranking vs median, outliers, findings
+```
+
+Reading is the default; there are exactly two writes, `round` and `log`. `--json` on the read gives you the parsed setup, the per-round analysis and the findings.
+
+It sizes the round for you (`8 videos ÷ 4 channels = 2 epochs`), computes the median and marks anything at ≥3× as an outlier, and lints the method: two variables in one structured round, a winner promoted off one post, results read at mixed ages, a structured round handed to gigworkers, unspent capacity, an end-of-funnel north star with no traffic behind it. Feedback, not a gate — it exits 0, like `vidfarm qa`.
+
+The rest of the loop stays where it already lives: **`vidfarm channels`** (the channel list, therefore capacity) · **`vidfarm harness`** + **`vidfarm qa`** (the constants, and holding a batch to them) · **`vidfarm handoff`** (the per-video briefs) · **`vidfarm dedupe`** (one variant per channel) · **`vidfarm approve`** + **`vidfarm schedule`** (the actual posting). `experiment log --posted` only *records* that a slot went live; it does not post anything.
+
+Also keep the operational lists on disk, not in memory — the channel inventory, and if you are crowdsourcing, `dollarplatoon-gigs.md` (one row per gig: id, client, invite link, video type, price, date last checked). Re-derive them when they look stale.
+
+---
+
+## 8. Long-running campaigns
+
+These run for weeks or months. At the top of every session:
+
+1. Read `EXPERIMENTS_DIARY.md` **first**. It, not your memory, is the state.
+2. Read the current baseline checkpoint. Everything you produce this epoch starts from it.
+3. Pull results for anything posted and not yet logged.
+4. Decide the round, then plan → approve → handoff (§6).
+
+At the end of every epoch, append the results and the decision. Then update the baseline checkpoint **only if a variant beat it on the north star, with more than one post behind it.**
+
+When you accumulate winners, you are building the real deliverable: **a library of proven ads you can re-cut, re-dedupe and re-run indefinitely.** That library — not any single viral video — is what the whole method is for.
+
+---
+
+## Where to go next
+
+- **Making the videos:** `https://vidfarm.cc/skill.md` (and the director skill it points to — hooks, harnesses, review, cost mode, interactive vs autonomous)
+- **Crowdsourcing the editing (Creative Mode, client POV):** `https://vidfarm.cc/crowdsourcing.md`
+- **The agentic clipper loop programme:** `https://vidfarm.cc/clipper.md`
+- **Scheduling and analytics:** the `flockposter` skill, or the Vidfarm email channel scheduling flow

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@officexapp/vidfarm-devcli",
-  "version": "0.21.37",
+  "version": "0.21.38",
   "description": "Local bridge for the Vidfarm Trackpad Editor. `vidfarm serve <template_id>` boots the FULL editor on localhost (disk-backed records/storage, free in-process render); edit composition.html on disk (Claude Code, Codex, etc.) and the browser live-morphs it.",
   "type": "module",
   "bin": {
@@ -18,6 +18,7 @@
     "dist/src/devcli/cost-mode.js",
     "dist/src/devcli/dedupe-local.js",
     "dist/src/devcli/doctor.js",
+    "dist/src/devcli/experiments.js",
     "dist/src/devcli/handoff.js",
     "dist/src/devcli/greenscreen-local.js",
     "dist/src/devcli/hyperframes-cli.js",
@@ -63,6 +64,8 @@
     "SKILL.md",
     "SKILL.director.md",
     "clipper.md",
+    "crowdsourcing.md",
+    "experiment.md",
     "update.md",
     "!readme.secret.md",
     "!**/*.secret.*"