@officexapp/vidfarm-devcli 0.21.30 → 0.21.32

package/.agents/skills/vidfarm/references/primitives.md

@@ -35,7 +35,7 @@ curl -X POST "$VIDFARM_BASE/api/v1/primitives/images/remove-background" \
 
 - `POST /api/v1/primitives/images/create-overlay` (flat alias: `POST /api/v1/primitives/create-media-overlay`)
 - Body: `{ "tracer": "...", "payload": { "prompt": "a cartoon rocket ship, flat vector illustration", "provider"?: "...", "model"?: "...", "prompt_attachments"?: ["https://..."], "aspect_ratio"?: "1:1", "image_size"?: "1K"|"2K"|"4K", "key_color"?: "#00FF00", "tolerance"?: 0.3, "softness"?: 0.1, "despill"?: true, "output_format"?: "png"|"webp" }, "webhook_url"?: "..." }`
-  - Just describe the **subject** in `prompt` — do NOT describe the background; the primitive appends the flat key-color background requirement for you.
+  - Just describe the **subject** in `prompt` — do NOT describe the background; the primitive appends the flat key-color background requirement for you, **and** the key-safe *artwork* requirement (closed solidly-filled shapes, no outline-only/hollow art, nothing on the subject in the key hue or a near-shade, fully opaque, no glow/translucency). That second clause is what stops the classic failure: outline art whose interior is bare plate keys into a **rim around a transparent hole**. If you build the prompt yourself for `images/generate` instead of using this primitive, write that clause in by hand.
   - `key_color` lets you key against something other than green if your subject is itself green-heavy (e.g. key against `#FF00FF` magenta for a green frog).
   - The chroma-key params (`tolerance`/`softness`/`despill`) behave exactly as in `image_remove_background_greenscreen`.
 - Response: `primary_file_url` is the finished transparent overlay; `greenscreen_source_url` is the raw pre-key frame (kept so you can re-key at a different tolerance without paying for another generation).
@@ -136,6 +136,85 @@ curl -X POST "$VIDFARM_BASE/api/v1/primitives/audio/download" \
   -d '{"tracer": "demo-download-audio", "payload": {"source_url": "https://www.youtube.com/watch?v=example"}}'
 ```
 
+## Primitive: social/recycle
+
+**"Tweet to TikTok" and "Reddit to TikTok" live here.** The text-and-thread sibling of `videos/download`: point it at a Reddit or X/Twitter source and it **decomposes** that source into reusable JSON — the post text, the comment/reply thread, every image/video media URL, the author's profile pic and follower count, and the engagement stats — so YOU decide what to recycle into a new video. It deliberately does **not** summarize, rank, score, or filter. It dumps the data back raw.
+
+**Vidfarm does not hold Reddit/X vendor keys.** This primitive is a thin broker over the two OfficeX apps that already own that integration — **reddit-lead-gen** and **x-lead-gen** — called through their public REST job API. They own the vendor keys, retries, rate-limit budget, auto-pagination, and 90-day result storage; vidfarm owns target resolution, normalization, and the wallet charge. Both are async job APIs, so a recycle is **a job wrapping a job** — but the nesting is invisible: this primitive polls upstream to completion server-side and only resolves when the data is in hand, exactly like the GhostCut leg of `videos/remove-captions`. You poll `GET /api/v1/primitives/jobs/:jobId` like any other primitive.
+
+- `POST /api/v1/primitives/social/recycle` (alias: `POST /api/v1/primitives/recycle-social`)
+- Body: `{ "tracer": "...", "payload": { "source_url": "https://..." }, "webhook_url"?: "..." }`
+
+### The four shapes
+
+| You pass | Mode (auto-detected) | You get |
+| --- | --- | --- |
+| `https://reddit.com/r/<sub>/comments/<id>/...` | `reddit_thread` | the post + its comment tree, flattened in reading order |
+| `https://reddit.com/r/<sub>` (or `r/<sub>`) | `reddit_channel` | that subreddit's threads, paginated |
+| `https://x.com/<user>/status/<id>` | `twitter_thread` | the tweet + the author's self-thread + its replies, paginated |
+| `https://x.com/<user>` (or `@<user>`) | `twitter_channel` | that account's tweets (pinned first), paginated |
+
+A fifth mode, `reddit_user` (`https://reddit.com/user/<name>` or `u/<name>`), returns a redditor's posts and comments. Pass `mode` explicitly to override auto-detection, or `handle` + `network` instead of a URL.
+
+### Payload
+
+| Field | Default | Notes |
+| --- | --- | --- |
+| `source_url` | — | Reddit/X URL. Required unless you pass `handle` + `network`. Aliases: `url`, `thread_url`, `profile_url` |
+| `handle` + `network` | — | `{ "handle": "wallstreetbets", "network": "reddit" }`. Aliases for handle: `username`, `screenname`, `subreddit` |
+| `mode` | auto | Override the detected shape |
+| `max_records` | `100` | Cap on records (1–1000). The lead-gen apps auto-paginate to reach it and reserve credits for `ceil(max_records / 25)` pages — **this is your spend ceiling** |
+| `cursor` | — | Resume token from a previous run's `nextCursor` |
+| `prompt_filter` | — | Hand the lead-gen app a Gemini scoring prompt; adds `match_score` + `ai_notes` per item and costs slightly more per task upstream. We surface it and never act on it |
+| `query` | — | Restrict a channel pull to matching posts (Reddit search / X `from:<user> <query>`) |
+| `filter` / `sort` / `time_filter` | — | Reddit listing controls (`hot`, `new`, `top`, …) |
+| `search_type` | — | X search mode (`Top`, `Latest`, `Photos`, `Videos`) |
+| `include_profile` | `true` | One extra upstream job for the subreddit/account avatar, banner, bio, follower count |
+| `include_comments` | `true` | Thread modes only — set `false` to skip the replies leg (and its cost) |
+| `save_raw` | `true` | Persist the untouched upstream pages as `raw-pages.json` |
+
+### Result
+
+Poll `GET /api/v1/primitives/jobs/:jobId` until `status: "succeeded"`, then read:
+
+- `items[]` — the normalized view. Each entry: `{ network, kind: "post" | "comment", id, title, text, created_at, permalink, link_url, author: { handle, name, avatar_url, banner_url, followers, profile_url, bio, verified }, stats: { … }, media: [{ type: "image"|"video"|"gif", url, thumbnail_url, width, height, duration_ms }] }`. X video media resolves to the **highest-bitrate MP4** variant, not the HLS manifest.
+- `profile` — the subreddit/account itself (avatar, banner, bio, subscriber/follower count).
+- `mediaUrls[]` — every media URL flattened, ready to drop into layers or feed to `videos/download`.
+- `nextCursor` — pass it back as `payload.cursor` to page further. `truncated: true` means there IS more.
+- `counts` — `{ items, posts, comments, media, pages, upstream_jobs, credits_charged }`.
+- `upstreamJobs[]` — one entry per lead-gen job we ran: `{ network, endpoint, job_id, status, tasks, credits_charged, view_url }`. **`view_url` opens a NocoDB spreadsheet of that pull — hand it to the human when they want to browse results visually.**
+- `recycled.json` (also `primary_file_url`) — the **complete** normalized payload. `items` in the job result is trimmed when it would blow the job-record size limit; `itemsTruncatedInline: true` tells you to read the artifact instead.
+- `raw-pages.json` — every untouched upstream page, so you can reach fields this primitive doesn't model.
+
+- Billing: wallet, metered on the **OfficeX credits the lead-gen apps actually consumed** (`officex_social_recycle`), converted at `OFFICEX_CREDIT_USD`. A one-task thread costs a fraction of a 10-page timeline. Paid plans only — free plans get `402`, same as the download routes. If the *upstream* app runs out of OfficeX credits you get a `402` naming that app, which is an operator problem, not a customer one.
+- devcli wrapper: `vidfarm recycle <source>` (aliases: `recycle-social`, `tweet-to-video`, `reddit-to-video`). Flags: `--max-records N`, `--cursor`, `--query`, `--filter hot|new|top`, `--mode`, `--network`, `--no-comments`, `--no-profile`, `--out ./recycled.json` (writes the FULL normalized payload to disk), `--no-wait`. It prints the counts, the upstream NocoDB `view_url`s, and the resume cursor.
+- Legs: a recycle runs 1–3 upstream jobs (profile, feed, replies) **sequentially**, so a failure on the first never pays for the second. The optional profile leg degrades rather than sinking the pull.
+- Timeouts: upstream polling gives up after `SOCIAL_RECYCLE_POLL_TIMEOUT_MS` (default 5 min); the upstream job may still finish and can be browsed in that app.
+
+### Turning a recycle into a video
+
+1. Recycle the source with a small `max_records` (25–50 is usually plenty for one video).
+2. Read `items[]` yourself and pick the beat: the post's hook line, the two funniest comments, the stat everyone reacted to.
+3. Pull any `media[].url` you want as real footage through `videos/download` or straight into a layer; use `profile.avatar_url` for the "who said this" badge.
+4. Script and build as normal. **The recycled text is source material, not a script** — rewrite it to the hook/loop/payoff standard rather than pasting a comment thread on screen.
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/social/recycle" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"tracer": "demo-recycle", "payload": {"source_url": "https://www.reddit.com/r/wallstreetbets/comments/p0esdp/slug/", "max_records": 50}}'
+```
+
+```bash
+# An X account's recent posts, no replies, cheapest possible pull
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/recycle-social" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"tracer": "demo-recycle-x", "payload": {"source_url": "https://x.com/elonmusk", "max_records": 40, "include_profile": true}}'
+```
+
 ## Primitive: video_remove_captions
 
 Remove burned-in captions/subtitles/on-screen text from any video URL (GhostCut-powered — the same pipeline auto-decompose uses for its caption-free mirror, exposed as a standalone reusable primitive). Result is a durable caption-free MP4 stored at a Vidfarm URL.
@@ -158,38 +237,83 @@ curl -X POST "$VIDFARM_BASE/api/v1/primitives/videos/remove-captions" \
 
 ## Primitive: media_dedupe
 
-Apply subtle camouflage transforms (zoom, tilt, rotate, saturation, playback speed, contrast, brightness, hue rotate, blur, horizontal flip, tint overlay) to any source **image or video** URL. Useful when reposting existing media and you want small perceptual differences to reduce duplicate-detection.
+**Deduplication** — make a finished image or video read as a **new upload** to a social platform's duplicate-content detector, while staying invisible to a viewer. Platforms fingerprint every upload; posting the same render twice (across accounts, or again next month) gets the later copy suppressed as duplicate/reused content. This primitive nudges geometry, color, timing and grain by a couple of percent, strips container metadata, and walks the encoder's CRF, so each copy carries a distinct fingerprint.
+
+**Ask the operator before you publish or bulk-produce.** Dedupe runs on the EXPORTED file, so the correct order is *render once → dedupe N times*, never *render N times*. Deciding up front avoids paying for a second render later.
 
 - `POST /api/v1/primitives/media/dedupe`
 - Body: `{ "tracer": "...", "payload": { ...fields... }, "webhook_url"?: "..." }`
   - Note: webhook delivery is not yet active — `webhook_url` is accepted and persisted on the job but never fired. Poll the job endpoints (`GET /api/v1/primitives/jobs/:jobId`) for completion.
-- Response: standard primitive job. Poll to completion, then read `primary_file_url` (also `video.file_url` for MP4 or `image.file_url` for stills)
-- Billing: metered as a Vidfarm render (via `hyperframes_lambda` on prod cloud render; free on a local serve box)
+- Response: standard primitive job. Poll to completion, then read `primary_file_url` (also `video.file_url` for MP4 or `image.file_url` for stills). `output.dedupe` carries the resolved effects, the effective zoom, the CRF and a human-readable `notes` list.
+- Billing: one ffmpeg pass, metered at real compute (much cheaper than a render). Free on a local serve box — and free ANYWHERE via `vidfarm dedupe --local`, which runs the identical filter graph on bundled ffmpeg.
+
+### Presets
+
+`preset` picks a calibrated transformation set. Default `standard`.
 
-Payload fields:
+| preset | skew | zoom | rotate | speed | saturation | when |
+| --- | --- | --- | --- | --- | --- | --- |
+| `none` | — | — | — | — | — | no-op; re-encode only |
+| `light` | 1% | 2% | 0.75° | +1% | +2% | lightly reused footage, or tight framing you can't crop |
+| `standard` | 2% | 3% | 2° | +2% | +4% | **the default** — the house standard |
+| `strong` | 3.5% | 6% | 3° | +5% | +8% | an Nth re-post, or an account that already ran this clip |
+| `legacy` | — | 4% | 3° | +5% | +5% | the pre-ffmpeg composition-renderer defaults |
+
+`light`/`standard`/`strong` also move contrast, brightness, hue and grain. Any individual knob in `effects` overrides the preset.
+
+### Minting N distinct copies
+
+`variant` (1-based) is what makes bulk posting work. Variant 1 is the preset as authored; later variants get deterministically jittered magnitudes and **alternating signs** (a sign flip moves a perceptual hash much further than a magnitude nudge), so N copies differ from the original *and from each other*. Reuse one `seed` across the batch, and post each variant to a different account/slot.
+
+### Payload fields
 
 - `source_media_url` (required, URL) — the image or video to transform
 - `media_type` (`"image" | "video"`, optional) — auto-detected from URL extension if omitted (`.mp4/.mov/.webm/.m4v` → video, else image)
-- `effects` (optional object). All fields optional; defaults camouflage lightly:
-  - `zoom` (default `1.04`) — scale factor
-  - `tilt` (default `3`) — degrees of 3D X-axis tilt (perspective 1000px)
-  - `rotate` (default `3`) — degrees of 2D rotation
-  - `saturation` (default `1.05`)
-  - `speed` (default `1.05`, video only) — playback rate, also compresses output duration accordingly
-  - `horizontal_flip` (default `false`)
-  - `contrast` (default `1.05`)
-  - `brightness` (default `1.05`)
-  - `hue_rotate` (default `0`) — degrees
-  - `blur` (default `0`) — pixels
-- `tint_color` (default `"#FF8C00"`), `tint_opacity` (default `0.08`) — subtle color overlay
-- `width` (default `1080`), `height` (default `1920`) — output canvas
-- `duration_ms` — output duration for video; if omitted, `fallback_duration_ms` (default `5000`) is used
-- `object_fit` (`"cover" | "contain" | "fill" | "none" | "scale-down"`, default `"cover"`)
-- `background_color` (default `"#000000"`) — visible when `object_fit` leaves letterboxing
-- `muted` (default `false`), `volume` (default `1`) — audio pass-through on video
+- `preset` (`"none" | "light" | "standard" | "strong" | "legacy"`, default `"standard"`)
+- `engine` (`"ffmpeg" | "composition"`, default `"ffmpeg"`) — `ffmpeg` is a real pixel/timing transform on the source file (true shear, honest playback speed, grain, metadata strip) and is both cheaper and stronger. `composition` is the legacy HyperFrames-render path, kept only for callers that depend on its exact output.
+- `variant` (int ≥ 1, default `1`), `seed` (string, optional), `jitter` (bool, optional — defaults on for `variant > 1`)
+- `strip_metadata` (default `true`) — drop creation time / encoder / source handler. Several platforms compare that **before** they compare pixels.
+- `effects` (optional object). Every field optional; each one overrides the preset:
+  - `zoom` — scale factor, centre-cropped back (`1.03` = 3% punch-in)
+  - `skew` — horizontal shear as a **percent of frame width** (ffmpeg engine only)
+  - `rotate` — degrees of 2D rotation
+  - `tilt` — degrees of 3D X-axis tilt on the composition engine; folded into the shear budget on ffmpeg
+  - `speed` — playback multiplier, video only; changes duration **and** pitch-preserved audio tempo
+  - `saturation`, `contrast`, `brightness` — multipliers around `1`
+  - `hue_rotate` — degrees
+  - `noise` — film grain `0..100` (ffmpeg engine only). Cheap, invisible, moves a lot of hash.
+  - `blur` — gaussian sigma in px. Usually `0` — blur is the one knob viewers notice.
+  - `volume` — audio gain multiplier
+  - `horizontal_flip` (default `false`) — the strongest single knob, but it visibly reverses on-screen text. Opt in deliberately.
+- `tint_color` (default `"#FF8C00"`), `tint_opacity` (default `0.08`) — flat color wash; set opacity `0` to skip
+- `width` / `height` — **optional on the ffmpeg engine**; omit to keep the source's own frame size (forcing 1080×1920 onto a 16:9 source would squash it). The composition engine falls back to 1080×1920.
+- `crf` — base x264 quality; jittered ±1 per variant so the coded bitstream differs too
 - `output_format` (`"png" | "jpeg" | "webp"`, default `"png"`) — image mode only; video mode always outputs MP4
+- Composition-engine only: `duration_ms` / `fallback_duration_ms` (default `5000`), `object_fit`, `object_position`, `background_color`, `muted`
+
+**A rotate forces a bigger crop than you asked for.** Black corners have to go somewhere, so the primitive raises `zoom` to the smallest value that covers the rotation and says so in `output.dedupe.notes`. On a tall 1080×1920 frame a 2° rotate costs ~6.7% of the frame. If framing matters more than fingerprint distance, pass `effects.rotate: 0`.
 
-Video example (camouflage a reused clip):
+Video example — three copies of one render, one per account:
+
+```bash
+for V in 1 2 3; do
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d "{
+    \"tracer\": \"dedupe-launch-reel-v$V\",
+    \"payload\": {
+      \"source_media_url\": \"https://cdn.example.com/reel.mp4\",
+      \"media_type\": \"video\",
+      \"preset\": \"standard\",
+      \"variant\": $V,
+      \"seed\": \"launch-reel\"
+    }
+  }"
+done
+```
+
+Custom knobs (keep the framing, lean on color and timing instead):
 
 ```bash
 curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
@@ -200,15 +324,7 @@ curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
     "payload": {
       "source_media_url": "https://cdn.example.com/reel.mp4",
       "media_type": "video",
-      "duration_ms": 8500,
-      "effects": {
-        "zoom": 1.05,
-        "tilt": 2,
-        "rotate": -2,
-        "speed": 1.03,
-        "hue_rotate": 4,
-        "horizontal_flip": true
-      },
+      "effects": { "rotate": 0, "skew": 1.5, "zoom": 1.02, "speed": 1.03, "hue_rotate": 6, "noise": 2 },
       "tint_color": "#00A3FF",
       "tint_opacity": 0.06
     }
@@ -226,12 +342,18 @@ curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
     "payload": {
       "source_media_url": "https://cdn.example.com/photo.jpg",
       "media_type": "image",
-      "effects": { "zoom": 1.06, "rotate": -1, "hue_rotate": 8 },
+      "preset": "light",
       "output_format": "webp"
     }
   }'
 ```
 
+Local equivalent (free, offline, identical transforms — prefer this):
+
+```bash
+vidfarm dedupe ./out/final.mp4 --variants 3 --out-dir ./out/posts
+```
+
 ## Primitive: music (text → music)
 
 Generate music (instrumental, songs with lyrics, background beds, jingles, scores) via ElevenLabs. **Default this on freely — music is a core primitive.** `use_wallet_credits` defaults **true**: it runs on vidfarm's platform ElevenLabs key and bills the customer's wallet. Recommend keeping it on; set it false only to save wallet credits or to use the customer's OWN saved ElevenLabs key.