@officexapp/vidfarm-devcli 0.21.59 → 0.21.61

package/experimental/meme-recaption.md

@@ -54,7 +54,7 @@ Two ingredients, no more:
 
 | Ingredient | What it is | Where it comes from |
 |---|---|---|
-| 🎬 **The meme** | A borrowed clip everyone already half-recognises, doing one absurd physical thing | `vidfarm.cc/explore` → public raws, `sourceType: MemeScreens` (851 of them, once you page past what the CLI shows you) |
+| 🎬 **The meme** | A borrowed clip everyone already half-recognises, doing one absurd physical thing | `vidfarm.cc/explore` → public raws, `sourceType: MemeScreens` (851 of them — reached with `vector_q`, **not** with the CLI, which shows you 200 and does not say so) |
 | ✍️ **The caption** | One sentence naming a specific lived experience, in the second person or the first | You. This is the only original work in the video, and it is 90% of the outcome |
 
 **The offer is named on the video.** First choice: inside the caption, as a character in the setup
@@ -446,43 +446,201 @@ it is the cheapest route out of a caption that got safe.
 
 ## Casting the meme
 
-### Browse the shelf. Do not search it.
+### Two channels: the slug is the IDENTITY, the vector is the BODY
+
+The shelf answers two completely different questions and it answers them through two different
+fields. Casting badly is almost always the result of asking one of them the other's question.
+
+| You want to know | Ask | Why |
+|---|---|---|
+| **which meme is this** | the `slugId` | The AI descriptions are blind to the meme's identity — see the warning below |
+| **what does the body DO** | `vector_q`, semantic search | The embedding is built from the description **plus the action / emotion / motion / subject tags**, which is exactly the verb axis this format casts on |
 
 > ⚠️ **The descriptions on this shelf were written by a model that has never heard of the meme.**
-> This is not a small problem — it is the defining fact of sourcing this format.
->
 > `my-brain-every-10-seconds` is described as *"A man in a suit gestures and looks at the camera
 > with a neutral expression."* `jonah-jameson-laugh-you-serious` is described as *"A solid green
-> screen background."* A semantic query for `hungry food eating` returns **zero results** on a shelf
-> that contains a dozen clips of people eating.
+> screen background."*
 >
-> **The `slugId` is the index. The description is noise.** Pull the whole shelf as JSON, read the
-> slugs, and only look at descriptions to break a tie.
+> **The `slugId` is the index of identity. The description is noise about identity.** But that same
+> description says *what the body is doing*, and **that is the only thing this format casts on.** A
+> model that does not know it is looking at J. Jonah Jameson can still tell you the man is laughing
+> in your face — so the descriptions are worthless for the question you were not going to ask and
+> useful for the one you were.
+
+#### 🚨 `--query` is LEXICAL. Semantic search is a different parameter, and the CLI has no flag for it
+
+This is the single most expensive mistake in sourcing this format, and an earlier draft of this file
+made it and drew the wrong conclusion from it.
+
+`vidfarm public-raws --query "…"` sends `?q=`, and `q` is a **substring filter that runs first**. It
+throws away every clip whose text does not literally contain your words, and only *then* does the
+server rank whatever survived. On a shelf whose descriptions never use your vocabulary, the filter
+empties the set before the ranking gets to see it — so a perfectly good semantic query returns
+nothing, and it looks like semantic search is broken. It is not. It never ran.
 
 ```bash
-vidfarm public-raws --categories                       # the shelves + live counts
-vidfarm public-raws --category greenscreen --limit 200 # 193 of these 200 are MemeScreens
-                                                       # ...and this is only the FIRST 200. See below
-vidfarm public-raws --category text-graphics --limit 200
-vidfarm public-raws --category lifestyle --limit 200   # also almost entirely MemeScreens
+# ❌ lexical filter. Measured: 0 results, on a shelf full of blank stares
+vidfarm api GET "/api/v1/public-raws?category=greenscreen&limit=8&q=blank%20stare%20confused"
+
+# ✅ semantic. Same shelf, same words, ranked by MEANING
+vidfarm api GET "/api/v1/public-raws?category=greenscreen&limit=8&vector_q=blank%20stare%20confused%20no%20comprehension"
+#   cat-staring · mrbean-wtf-greenscreen · james-franco-shocked · spongebob-panting …
 ```
 
+> **Pass `vector_q` and do NOT pass `q`.** The server defaults `vector_q` to `q` when you omit it,
+> which means the CLI's `--query` gets you the lexical filter *and* the ranking — and the filter has
+> already deleted the answer. Sending only `vector_q` skips the filter and ranks the whole shelf.
+>
+> **`vidfarm public-raws` cannot do this.** It maps `--query` to `q` and has no vector flag. Every
+> semantic cast in this file goes through `vidfarm api GET`.
+
+**It needs a login and an embedding key, and it fails SILENTLY without one.** The ranking requires a
+logged-in customer with a saved gemini or openai provider key. With no key — or logged out — the
+server skips the ranking, returns **HTTP 200 with a full page of results**, and says nothing. You
+get the shelf in recency order wearing the costume of a search result.
+
+```bash
+vidfarm whoami                       # must show a login
+vidfarm api GET /api/v1/user/me/provider-keys | grep -iE "gemini|openai"
+```
+
+**Confirm the ranking is live before you trust a cast:** query `arms up jumping celebration confetti`
+and query `shoulders drop giving up defeated sigh`. If the top result is the same clip for both, the
+embedder is not running and you are reading a recency list.
+
+#### Semantic search reaches the whole shelf. The CLI reaches a quarter of it
+
 > 🚨 **`vidfarm public-raws` shows you 200 clips and stops, and it does not tell you.** The server
 > caps a shelf at 200 per call and returns a `next_cursor` — but the CLI has **no `--cursor` flag**,
-> so raising `--limit` to 1200 still returns exactly 200. Paging the same shelves through
-> `vidfarm api GET` instead turns **200 clips into 959** (851 of them MemeScreens). Cast from the
-> first 200 and you are picking from under a quarter of the library, with nothing anywhere saying so.
->
-> **Build the full index once (Appendix C) and cast from the TSV, not from the CLI.**
+> so raising `--limit` to 1200 still returns exactly 200.
+
+`vector_q` on a **curated shelf** (`greenscreen`, `reaction`, `lifestyle`, `text-graphics`, …) is
+not a rerank of that 200. The server scans **1500 records** for a curated category and ranks all of
+them, so one call sees essentially the entire MemeScreens library and hands you its top N.
+
+Measured on the `greenscreen` shelf, ten verb queries against one plain `--limit 200` page:
+
+| | |
+|---|---|
+| Distinct clips the ten queries returned | **143** |
+| Of those, **not present** in the CLI's first 200 | **93 — 65%** |
+| Of those, hallucinated / not really on the shelf | **0.** All 143 verified against the fully-paged 693 |
+
+**Two thirds of what semantic search finds is invisible to the CLI**, and the two thirds it hides
+are not the boring ones — they are the animal and anonymous clips (`cat-staring`, `cat-sleeping`,
+`hangover-dog`) that are 🟢 **safe tier**, which is the tier client work is supposed to default to.
+Casting from the first 200 quietly pushes you toward the celebrity clips you are not allowed to use.
+
+```bash
+vidfarm public-raws --categories                       # the shelves + live counts. Start here
+vidfarm public-raws --category greenscreen --limit 200 # the first 200 only — for a browse, not a cast
+```
 
 **Every shelf the CLI exposes is dominated by MemeScreens** — `b-roll`, `lifestyle` and
 `text-graphics` all come back as greenscreen memes. Good news for this format, bad news for
 backgrounds (see below).
 
-> **Delivery work: open your delivery log in the same breath as the shelf.** Anything you already
+### The diversity sweep — one query per verb, then dedupe
+
+> **A single vector query is a relevance tool. It is the OPPOSITE of a diversity tool.** It returns
+> the twenty clips nearest one point, and near-neighbours are near-duplicates: four `cat-sitting`
+> variants of the same cat. Cast ten videos off one query and you will ship ten videos that look
+> like one video.
+
+Diversity comes from **querying ten different points and taking the union.** The ten points are
+already written down — they are the ten verb tokens in the table below, which is why the list is
+closed. One query per verb, top 20 each, deduped, is a 143-clip casting pool spread across the whole
+shelf by meaning instead of by upload date.
+
+```python
+#!/usr/bin/env python3
+"""meme-sweep.py — one vector query per verb token -> a deduped casting pool.
+
+Needs a vidfarm login + a saved gemini/openai key, or the ranking silently
+does not happen (see above). Run it once per session; it is ~10 API calls.
+"""
+import json, subprocess, urllib.parse
+
+VERBS = {
+    "deflate":    "shoulders drop eyes go flat accepting a bad outcome without a fight",
+    "blank":      "total non-comprehension head tilt dead eyes nothing processing",
+    "loop":       "the same thought going round again mechanically forever repetitive",
+    "smug":       "arms folded self-satisfied already solved it insufferable",
+    "overreact":  "full body screaming sobbing flailing at something trivial",
+    "correct":    "turning to camera to tell you patiently that you are wrong",
+    "unbothered": "completely relaxed doing nothing while chaos happens calm",
+    "celebrate":  "arms up jumping celebration confetti scale joy",
+    "stare":      "holding eye contact with the camera saying nothing silent",
+    "wait":       "still waiting visibly longer than is reasonable impatient",
+}
+SHELVES = ("greenscreen", "reaction", "lifestyle", "text-graphics")
+
+def get(path):
+    out = subprocess.run(["vidfarm", "api", "GET", path], capture_output=True, text=True).stdout
+    brace = out.find("{")           # first line is a coloured "GET … → 200" status line
+    return json.JSONDecoder().raw_decode(out[brace:])[0] if brace >= 0 else {}
+
+pool, tops = {}, {}
+for shelf in SHELVES:
+    for verb, q in VERBS.items():
+        d = get(f"/api/v1/public-raws?category={shelf}&limit=20"
+                f"&vector_q={urllib.parse.quote(q)}")          # vector_q ONLY. Never q
+        raws = d.get("raws", [])
+        if raws:
+            tops.setdefault(shelf, []).append(raws[0]["rawId"])
+        for rank, raw in enumerate(raws):
+            slug = raw.get("slugId") or raw["rawId"]
+            pool.setdefault(slug, {"verb": verb, "rank": rank, "id": raw["rawId"],
+                                   "desc": (raw.get("description") or "")[:90]})
+
+# The ranking fails SILENTLY (no login / no embedding key) and still returns a
+# full page. The only reliable tell is differential: ten different queries that
+# rank must not all return the same clip first. Do NOT test this by looking for
+# a slugId — several shelves legitimately return bare UUIDs.
+for shelf, first in tops.items():
+    if len(set(first)) < len(first) / 2:
+        raise SystemExit(f"{shelf}: {len(set(first))} distinct top hits from {len(first)} queries — "
+                         "the embedder is not running. Check `vidfarm whoami` and your gemini/openai key.")
+
+with open("casting-pool.tsv", "w") as f:
+    f.write("slug\tverb\trank\trawId\tdescription\n")
+    for slug, v in sorted(pool.items(), key=lambda kv: (kv[1]["verb"], kv[1]["rank"])):
+        f.write(f"{slug}\t{v['verb']}\t{v['rank']}\t{v['id']}\t{v['desc']}\n")
+print(len(pool), "distinct clips -> casting-pool.tsv")
+```
+
+**Then cast off the TSV, and cast off a verb you have not shipped this week:**
+
+```bash
+awk -F'\t' '$2=="unbothered"' casting-pool.tsv | cut -f1,5    # the whole neighbourhood, one verb
+cut -f2 casting-pool.tsv | sort | uniq -c                     # how deep each verb actually goes
+```
+
+**Three rules for keeping the pool diverse, all of them cheap:**
+
+1. **Drop the near-duplicate families.** `cat-sitting-5e58…` and `cat-sitting-8edd…` are one casting
+   choice, not two. Collapse on the slug stem (the slug minus its trailing hex) before you count
+   your options, or you will believe you have 143 and be choosing among about 110.
+2. **Take the top ~20 per verb, never the top 60 of one.** The tail of one query is further from
+   that verb than the head of the next query is — depth on one point buys worse clips, breadth
+   across points buys better ones.
+3. **Subtract the delivery log before you look, not after.** `join` the pool against
+   `DELIVERED.tsv` so a retired clip is never on the sheet you are choosing from. A clip you have
+   already sent is not an option you rejected, it is an option you should not have seen.
+
+```bash
+cut -f5 CLIPPER/DELIVERED.tsv | sort -u > /tmp/used.txt
+grep -vFf /tmp/used.txt casting-pool.tsv > casting-pool.fresh.tsv   # cast from THIS
+```
+
+> **The sweep is how "there was nothing else" stops being sayable.** Rule 11 asks you not to reuse a
+> clip you have already delivered; the honest objection has always been that finding a fresh one
+> meant paging a 693-clip shelf by hand. It is now ten API calls and a TSV.
+
+> **Delivery work: open your delivery log in the same breath as the sweep.** Anything you already
 > sent this client is out (Rule 11), and anything that has been passed on twice is out for everybody.
 > **A clip you have already used is not a shortcut, it is the shelf you did not read** — and the
-> shelf is 851 clips deep, so "there was nothing else" is almost never the real reason.
+> sweep hands you 143 candidates, so "there was nothing else" is almost never the real reason.
 >
 > ```bash
 > cut -f1,5,8 CLIPPER/DELIVERED.tsv    # client · clip · verdict — read it BEFORE the contact sheet
@@ -510,14 +668,26 @@ string equality instead of a judgement, which is the only form of it a gate can
 | `overreact` | a full-body scream, sob or flail at something trivial | `neon-screaming`, `jontron-nooooooo`, `girl-fake-crying` |
 | `correct` | turning to camera to tell you, patiently, that you are wrong | `jonathan-frakes-telling-you-youre-wrong-for-47-seconds`, `women-mocking` |
 | `unbothered` | the room is on fire and the body has not moved | `my-favorite-thing-in-the-world-to-do-is-nothing`, `boss-baby-chilling` |
-| `celebrate` | arms up, jumping, confetti-scale joy | *the natural win-pole verb — cast it off the shelf and confirm on the sheet* |
-| `stare` | holding eye contact with the camera and saying nothing | *the `<Entity> watching me …` verb (Rule 1, tier 1)* |
-| `wait` | still present, still waiting, visibly longer than is reasonable | *for captions whose collapse is elapsed time* |
+| `celebrate` | arms up, jumping, confetti-scale joy | 🔎 `celebration-dance`, `girl-dancing`, `jojo-siwa-dancing` — the natural win-pole verb |
+| `stare` | holding eye contact with the camera and saying nothing | 🔎 `cat-staring` (🟢), `mortal-kombat-1-shang-tsung` (🔴) — the `<Entity> watching me …` verb (Rule 1, tier 1) |
+| `wait` | still present, still waiting, visibly longer than is reasonable | 🔎 `wait-hold-on-wait` — for captions whose collapse is elapsed time |
 
-The last three have no verified slug in this file — nobody has cast them here yet. Find one on the
-shelf and confirm it on the contact sheet; **do not add an eleventh token** to avoid the work of
-choosing among ten. If a clip genuinely does something none of the ten covers, that is a real
-finding: add the token here, with its body line, before you use it.
+**The queries that produce this table are the `VERBS` dict in `meme-sweep.py` above** — the table is
+its cached output, not a separate list to maintain. Re-run the sweep and you regenerate it.
+
+> 🔎 **marks a slug found by the vector sweep and NOT yet confirmed on a contact sheet.** The first
+> seven rows were cast by hand and shipped; the last three were empty until the sweep filled them,
+> and the sweep ranks descriptions, not footage. **Build the sheet before you use one** — that rule
+> does not relax because a machine agreed with you.
+
+**Do not add an eleventh token** to avoid the work of choosing among ten. If a clip genuinely does
+something none of the ten covers, that is a real finding: add the token here, **with its body line
+and its vector query**, before you use it.
+
+> **The sweep also grades the token list.** `deflate` and `celebrate` came back with 20 and 19 clips
+> the other nine queries had not already found; `stare` and `wait` came back with 9 each. A verb
+> whose neighbourhood is that thin is a verb the shelf cannot really cast — treat a low count as a
+> warning that you are about to force a clip, not as a challenge.
 
 **Then confirm by eye.** Build a contact sheet before you commit — a slug promises an action the
 clip does not always deliver, and half these clips are 16:9 with the subject small at the bottom:
@@ -646,21 +816,68 @@ class before you write a single ffmpeg command.**
 | Where the caption goes | On the background, above the subject | On a plate above the clip, outside the picture |
 | Share of the shelf | The large majority | A minority — check for burned-in text on the contact sheet |
 
-**Check the plate colour before you key anything.** These clips are re-encodes and the plate is not
-always what it looks like:
+### Decide the class MECHANICALLY, then read the plate
 
-```bash
-python3 -c "
-import subprocess,numpy as np
-p=subprocess.run(['ffmpeg','-v','error','-i','meme.mp4','-vf','fps=1,crop=200:200:20:20',
-                  '-pix_fmt','rgb24','-f','rawvideo','-'],capture_output=True)
-a=np.frombuffer(p.stdout,np.uint8).reshape(-1,200,200,3)
-m=a.reshape(-1,3).mean(0).astype(int); print('plate rgb',m,'hex %02X%02X%02X'%tuple(m))"
+> **The class is not a judgement call and it must not be one.** "Is there a plate" has to be
+> answered before "what colour is the plate", or you key a Class B clip against a muddy average of
+> somebody's living room and punch holes in it. **Nothing downstream catches that** — the key
+> "works", a subject is found, the render is quietly ruined, and `no_residual_green` passes because
+> there was never any green.
+
+Run this on every clip, before Appendix A. It answers both questions in the right order:
+
+```python
+#!/usr/bin/env python3
+"""plate-check.py <meme.mp4> — Class A or Class B, and the plate if there is one."""
+import subprocess, sys, numpy as np
+
+src, W, H = sys.argv[1], 320, 320
+p = subprocess.run(["ffmpeg", "-v", "error", "-i", src, "-vf", f"fps=2,scale={W}:{H}",
+                    "-pix_fmt", "rgb24", "-f", "rawvideo", "-"], capture_output=True)
+a = np.frombuffer(p.stdout, np.uint8)
+a = a[: len(a) // (W * H * 3) * (W * H * 3)].reshape(-1, H, W, 3).astype(np.int16)
+
+# All four corners, not just the top-left: on a keyed clip every corner is plate,
+# on a baked clip they are four different bits of a room.
+c = np.concatenate([a[:, :24, :24], a[:, :24, -24:],
+                    a[:, -24:, :24], a[:, -24:, -24:]], axis=2).reshape(-1, 3)
+plate = c.mean(0).astype(int)
+flat  = c.std(0).mean()                          # one colour, or a scene
+cover = (np.abs(a - plate).max(3) < 40).mean()   # how much of the frame IS plate
+sat   = int(plate.max() - plate.min())           # letterbox bars read flat too
+
+print(f"plate rgb {plate.tolist()} hex {'%02X%02X%02X' % tuple(plate)}  "
+      f"corner stdev {flat:.1f}  coverage {cover*100:.0f}%  saturation {sat}")
+
+if flat > 12 or cover < 0.25 or sat < 60:
+    sys.exit("CLASS B — no saturated flat plate. Do NOT key. Layout A, crop the burned-in text")
+
+digital = plate[1] > 240 and plate[0] < 40 and plate[2] < 40
+print("CLASS A — " + ("digital plate: chromakey 0.18:0.03, NO despill" if digital
+                      else "filmed plate: chromakey 0.30:0.08, light despill"))
 ```
 
-A pure `00FF00` reading means a digital plate: key tight (`0.18:0.03`) and **skip despill.** A
-reading like `1F D3 1F` means a filmed plate: key looser and despill lightly. Getting this backwards
-is the first defect in the list below.
+**The three thresholds and what each one is for:** `flat > 12` is a scene wearing a plate's
+costume, `cover < 0.25` is a plate too small to be the background, `sat < 60` is black or white
+letterbox bars, which are flat and are not a plate. All three are first-pass numbers measured on one
+shelf — **calibrate them against a clip you already agree about before you let them fail anything.**
+
+**Then read the colour, because these clips are re-encodes and the plate is not always what it looks
+like.** A pure `00FF00` reading means a digital plate: key tight (`0.18:0.03`) and **skip despill.**
+A reading like `1F D3 1F` means a filmed plate: key looser and despill lightly. Getting this
+backwards is the first defect in the list below.
+
+> **The shelf is overwhelmingly green, and the gate only knows green.** `no_residual_green` (gate 2)
+> tests a green hue, so it catches both defects that matter on this shelf — a fringe from a tight
+> key, and a key that never ran at all. `no_unkeyed_plate` sits beside it and is hue-agnostic, for
+> the blue or magenta plate the green test is blind to. **If `plate-check.py` prints a plate that is
+> not green, say so in the build log** — the named check is no longer the one protecting you.
+
+> **Why this harness does not use `vidfarm remove-greenscreen` for the video.** The skill's keying
+> surface (`remove-greenscreen --local` / `--smart`, `cutout`, `sticker-pack`) is the right answer
+> almost everywhere else, and it is a **transparent-output** surface. This format never needs
+> transparency in its output — it composites onto a background and ships an opaque MP4 — and the
+> alpha intermediate is where the free path breaks. See the warning under "Cost-saving mode".
 
 ### Class B — dealing with the burned-in text
 
@@ -1183,8 +1400,13 @@ except you.
 **The shelf is public, which makes this sharper than it looks.** You cast from `vidfarm public-raws`
 and so does every other clipper on the marketplace. A slug that is easy to find is easy for everyone
 to find, so the buyer may have seen it this week from somebody else — and 851 MemeScreens means the
-excuse "there was nothing else" is almost never true. Cast past the first 200 (Appendix C) before
-you reach for a clip you have already used.
+excuse "there was nothing else" is almost never true. **Run the diversity sweep before you reach for
+a clip you have already used**; it costs ten API calls and it deletes the excuse.
+
+**And note what "easy to find" now means.** Every clipper on this marketplace reads the same CLI, and
+the CLI shows everybody the same first 200. If you cast from that page you are competing for 200
+clips while 65% of the shelf sits unread. The sweep is not only how you avoid repeating *yourself* —
+it is how you avoid arriving at the buyer with the same clip as everyone who did not run it.
 
 **The number, if you want one:** the same `clip_slug` in more than about **3 of your last 20
 deliveries** is a signature you did not choose. Either it has the approvals to justify it, or you
@@ -1261,8 +1483,16 @@ interchangeable.
 is one ffmpeg command and the caption layer does not change. Do that before you write a second
 caption — you will learn more from five castings of one joke than from five jokes.
 
-Keep the index (Appendix C) and a `captions.tsv` of rung-3 lines. The pairing is the creative act;
-everything downstream is a loop.
+> ⚠️ **Take the five clips from five DIFFERENT verb neighbourhoods, not from the top five of one
+> query.** This is where a matrix quietly stops being an experiment. The top five of one vector
+> query are five near-neighbours — often the same performer in four crops — so the five videos
+> differ by almost nothing and the five data points are one data point with error bars. Pull one
+> clip per verb from `casting-pool.fresh.tsv` and the five castings actually span the shelf.
+
+Keep three files: the identity index (Appendix C), a fresh `casting-pool.fresh.tsv` from the sweep,
+and a `captions.tsv` of rung-3 lines. **The sweep is the only one of the three that goes stale** —
+re-run it per session, because it is also the file that has your delivery log subtracted from it.
+The pairing is the creative act; everything downstream is a loop.
 
 ### ⚠️ The matrix is YOUR test set. It is not one client's inbox
 
@@ -1347,7 +1577,8 @@ What gate 2 enforces, and the number it uses:
 | Check | Threshold | The defect it exists for |
 |---|---|---|
 | `first_frame_has_subject` / `last_frame_has_subject` | lower-band stdev > 12 | Blank thumbnail / empty tail from un-trimmed plate |
-| `no_residual_green` | < 0.5% green pixels | A key that was too tight |
+| `no_residual_green` | < 0.5% green pixels | A key that was too tight — a fringe on the shelf's usual `00FF00` plate |
+| `no_unkeyed_plate` | largest flat saturated colour < 25% of frame, **or** `--flat-bg-ok` | The key never ran, or it ran on a blue/magenta plate the green test is blind to. Pass `--flat-bg-ok` only on ladder tier 6, where a flat brand plate really is the background |
 | `caption_horizontally_centred` | block centre 50% ± 3% | The flush-left plate with a lake of dead space |
 | `caption_side_margins` | > 4% and < 96% | A plate touching an edge |
 | `caption_vertical_placement` | inside 8–85% | Under the platform UI, or on the subject's face |
@@ -1415,7 +1646,11 @@ Run this **before** you build. An unchecked box is a rewrite, not a fix in the e
 - [ ] **Reusing this clip on a DIFFERENT client?** It has an approval on it, or it is unproven and
       this is its one outing. Passed on twice with no approval = retired
 - [ ] This `clip_slug` is not in more than ~3 of your last 20 deliveries
-- [ ] You read past the first 200 of the shelf (Appendix C) before reaching for a clip you own
+- [ ] **The diversity sweep ran this session** and you cast off `casting-pool.fresh.tsv`, not off
+      the CLI's first 200 and not off memory
+- [ ] The sweep's ranking was **live** — `vidfarm whoami` shows a login, a gemini/openai key is
+      saved, and two different verb queries returned two different top clips. No key = no ranking,
+      silently, with a full page of results to hide it
 - [ ] The style — plate, font, band, grade, handle — is **identical** to the last delivery. That part
       is meant to repeat
 
@@ -1445,6 +1680,8 @@ Run this **before** you build. An unchecked box is a rewrite, not a fix in the e
 
 **The clip**
 - [ ] Rights tier declared. 🟢 for client work, or a 🟢 alternate is built alongside
+- [ ] **`plate-check.py` run, and the class it printed is the class you built** — Class A keys and
+      needs a background, Class B is never keyed and needs the burned-in text cropped
 - [ ] Plate colour sampled, not assumed
 - [ ] Contact sheet built and looked at
 - [ ] The action lands 0.6–1.0s in, and frame 0 has the subject on it (`-vframes 1`, looked at)
@@ -1462,6 +1699,8 @@ Run this **before** you build. An unchecked box is a rewrite, not a fix in the e
 - [ ] 1080×1920, 30fps
 - [ ] The meme is NOT cropped to 9:16
 - [ ] Layout A or B chosen deliberately; in B the subject is bottom-anchored and full width
+- [ ] **Class A: the plate is gone and a background is behind the subject.** No green edge on the
+      hair or shoulders, no green in the gaps between limbs, nothing green in the last frame
 - [ ] Caption **block centred**, words left-aligned, lines hand-broken to near-equal length
 - [ ] Caption fully legible at frame 0, static, plated, clear of the subject's head
 - [ ] Poster layer present, so frame 0 is never bare background
@@ -1573,9 +1812,19 @@ def frames(path, w, h, vf):
     n = len(p.stdout) // (w * h * 3)
     return np.frombuffer(p.stdout, np.uint8)[:n * w * h * 3].reshape(n, h, w, 3).astype(np.int16)
 
-# --- 1. read the plate colour off the top-left corner, which is always plate ---
+# --- 1. confirm there IS a plate, THEN read its colour off the corner ---
+# The order is load-bearing. Handed a Class B clip (its own world, no plate),
+# every line below still runs: it keys against a muddy average of somebody's
+# living room, step 2 finds a "subject", and the render ships with holes punched
+# in it. Nothing catches that downstream — no_residual_green passes, because
+# there was never any green. So refuse here, not at render.
 corner = frames(meme, 200, 200, "fps=1,crop=200:200:20:20")
+flat = corner.reshape(-1, 3).std(0).mean()
 plate = corner.reshape(-1, 3).mean(0).astype(int)
+if flat > 12 or int(plate.max() - plate.min()) < 60:
+    sys.exit(f"corner stdev {flat:.1f}, plate {plate.tolist()} — not a flat saturated "
+             "chroma plate. This is CLASS B: do not key it. Use Layout A and crop "
+             "the burned-in text. (Run plate-check.py for the full read.)")
 digital = bool(plate[1] > 240 and plate[0] < 40 and plate[2] < 40)
 key_hex = "0x%02X%02X%02X" % tuple(plate)
 tol, soft, despill = (0.18, 0.03, False) if digital else (0.30, 0.08, True)
@@ -1651,11 +1900,21 @@ def probe_key(tol_try, soft_try, despill_try, t):
     lum = flat.mean(1)
     lit = flat[lum > max(lum.max() * 0.75, 1)]          # the subject's bright side
     green_left = ((flat[:, 1] - flat[:, 0] > 40) & (flat[:, 1] - flat[:, 2] > 40)).mean()
-    return lit.mean(), green_left
+    # A radius that erased the whole subject leaves NO bright pixels, and
+    # `lit.mean()` on an empty slice is nan, not 0. nan then loses every
+    # comparison below in silence: `nan > 0.88` is False, so the radius is
+    # rejected for the right reason by accident — but if the UN-keyed probe
+    # comes back nan too, every `keep` is nan, `best` stays None, and the run
+    # falls through to the sampled default with only a printed line. Return a
+    # real 0.0 so a dead frame scores like a dead frame.
+    return (float(lit.mean()) if lit.size else 0.0), green_left
 
 t_probe = START + DUR / 2
 # chromakey's similarity floor is 1e-05, not 0 — a literal 0 is a hard filter error
 src_lit, _ = probe_key(1e-5, 0.0, False, t_probe)       # ~nothing keyed = the truth
+if not src_lit:
+    sys.exit(f"the un-keyed probe frame at {t_probe:.2f}s has no bright pixels — "
+             "the clip is dark or the presence window is wrong. Check it by eye")
 best = None
 for tol_try in (0.34, 0.28, 0.22, 0.18, 0.14, 0.10, 0.07):
     r = probe_key(tol_try, soft, despill, t_probe)
@@ -1804,16 +2063,24 @@ ffmpeg -i media/scene.mp4 -vframes 1 -update 1 media/poster.png   # frame 0 of t
 
 ## Appendix C — build the shelf index once
 
-The shelf pages at 200, the CLI cannot follow the cursor, and the slugs are the index — so pull the
-whole thing once through the REST route and grep the file instead of re-querying.
+> **This is the IDENTITY index, and it is the second of the two tools.** `meme-sweep.py` (in
+> *The diversity sweep*, above) is how you **cast** — it ranks the whole shelf by what the body does
+> and hands you a fresh pool in ten calls. This appendix builds the full local mirror you **grep**:
+> for a slug you half-remember, for the rights tier of a name, for the ledger joins in Rule 11, and
+> for working with no login or no embedding key. Sweep to choose a clip. Index to look one up.
+
+The shelf pages at 200, the CLI cannot follow the cursor, and the slugs are the index of identity —
+so pull the whole thing once through the REST route and grep the file instead of re-querying.
 
 ```python
 #!/usr/bin/env python3
 """meme-index.py — pull every MemeScreens raw into one greppable TSV.
 
-The `--query` filter on this endpoint is thin and inconsistent (a search for
-"hungry food eating" returns nothing on a shelf full of people eating), and the
-AI descriptions are blind to the meme's identity. So: pull it all, index by slug.
+Note this pages with `q` unset entirely: the `q` filter is a LEXICAL substring
+match that runs BEFORE ranking, so any query here would silently shrink the
+mirror. Semantic search is a different parameter (`vector_q`) and lives in
+meme-sweep.py. The AI descriptions are blind to the meme's identity, so this
+file is indexed by slug.
 
 Paging note: `vidfarm public-raws` has NO --cursor flag and the server caps a
 shelf at 200 per call, so the CLI alone can only ever show you the first 200 of
@@ -2015,13 +2282,16 @@ and that its answers do not contradict each other. That is a smaller claim than
 "the match is verified", and it is the one this script is entitled to make.
 
 usage: meme-qa.py <render.mp4> <composition.html> [--offer dishcover.io]
-                  [--pole pain|win] [--silent-ok] [--json]
+                  [--pole pain|win] [--silent-ok] [--flat-bg-ok] [--json]
 """
 import json, re, subprocess, sys
 import numpy as np
 
 render, comp = sys.argv[1], sys.argv[2]
 SILENT_OK = "--silent-ok" in sys.argv
+# Ladder tier 6 only: a flat brand plate really is the background. Everywhere
+# else a slab of one saturated colour is a plate that never got keyed.
+FLAT_BG_OK = "--flat-bg-ok" in sys.argv
 OFFER = (sys.argv[sys.argv.index("--offer") + 1].strip().lower()
          if "--offer" in sys.argv else None)
 # The pole changes what the payoff is allowed to do (Rule 2). Default to PAIN:
@@ -2120,7 +2390,21 @@ for label, fr in (("first", first), ("last", last)):
     check(f"{label}_frame_has_subject", band.std() > 12,
           f"lower-band stdev {band.std():.1f} (>12)")
 
-# ------------------------------------------------------ 3. no residual green key
+# -------------------------------------------- 3. the plate is GONE, in any hue
+# Two checks, because these are two different defects. `no_residual_green` is a
+# FRINGE test on the shelf's usual 00FF00 plate — a key that was slightly too
+# tight. `no_unkeyed_plate` is hue-agnostic and catches the bigger, dumber one:
+# the key never ran, or it ran on a blue/magenta plate the green test cannot see.
+#
+# Measured on four fixtures: an unkeyed 00FF00 plate fails both; a photographic
+# background passes no_unkeyed_plate at 15%; a flat brand plate fails it at 79%
+# and needs --flat-bg-ok; an unkeyed MAGENTA plate reads 0.00% green and is
+# caught ONLY by no_unkeyed_plate. That last row is the whole reason it exists.
+#
+# Watch the green check if the joke's world is a lawn, a forest or a pool table:
+# 0.5% is tight, and a genuinely green BACKGROUND can trip it. That is a false
+# positive, not a bad key — confirm by eye, then pick a different world rather
+# than loosening the number.
 mid = probe(render, "scale=270:480", 270, 480, pre=("-ss", str(dur / 2)))
 if mid is not None:
     a = mid[0]
@@ -2128,6 +2412,23 @@ if mid is not None:
     green = ((g - r > 45) & (g - b > 45) & (g > 80)).mean()
     check("no_residual_green", green < 0.005, f"{green*100:.2f}% green pixels (<0.5%)")
 
+    # A quarter of the frame in ONE saturated colour is a plate, not a
+    # background: a background is a photograph, and photographs vary. The
+    # caption plate is white or black, so it is not saturated and not counted.
+    # Ladder tier 6 (a flat brand plate as the background) is a legitimate
+    # exception and the only one — pass --flat-bg-ok, the same way --silent-ok
+    # works, so the exception is a decision somebody made rather than a default.
+    sat_px = (a.max(2) - a.min(2)) > 60
+    if sat_px.any():
+        q = a[sat_px] // 32                                  # 8^3 colour buckets
+        keys = q[:, 0] * 64 + q[:, 1] * 8 + q[:, 2]
+        biggest = float(np.bincount(keys).max()) / a[..., 0].size
+    else:
+        biggest = 0.0
+    check("no_unkeyed_plate", FLAT_BG_OK or biggest < 0.25,
+          f"largest flat saturated colour = {biggest*100:.0f}% of frame (<25%)"
+          + (" [--flat-bg-ok]" if FLAT_BG_OK else ""))
+
 # ------------------------------------- 4. caption block is horizontally CENTRED
 # Find the caption plate in frame 0 by its own darkness against the background,
 # then compare its horizontal centre to the canvas centre.