@koda-sl/baker-cli 0.209.0 → 0.211.0-dev.e5398b13c

package/README.md

@@ -2510,6 +2510,8 @@ Re-ingesting the same link returns the existing row with `deduped: true` instead
 | `--context` | What you know that the video cannot show — campaign, who is on camera, intended use. Feeds the analysis, exactly as on `upload`. |
 | `--dry-run` | Preview the resolved route and parameters. |
 
+**The download is uncapped, and takes the best the site will release.** There is no height ceiling: Mux re-encodes every clip into its own renditions, so downscaling is already handled downstream and doing it *during* the download would throw away source pixels the one system that can use them never saw. A 1080p ceiling used to apply here on the reasoning that a 4K master is bytes we re-encode anyway — true, but it made the master permanently 1080p for everything the library is ever asked to do with that clip afterwards. Downscaling is cheap; re-acquiring a master is not. The remaining budgets are duration (3h, refused from metadata before a byte is fetched), size (16 GB), and the download window.
+
 Baker fetches directly first and only pays for a proxied connection when the site refuses it. `--country` pins those paid connections to a specific place — it does not change the first, free attempt. The result names both the exit used (`egress`) and the country, so a geo-targeted fetch is never invisible after the fact. A region failure says so and suggests the flag; other failures don't, because a different country would not fix them.
 
 ### Vimeo needs a signed-in session
@@ -2520,12 +2522,14 @@ Use an account created for this and holding nothing else — the value travels i
 
 An audio-only link (a music or podcast page) is refused up front — there is no picture to store in a video library. A download is also given a time budget scaled to the video's length, capped at 15 minutes, because the turn is blocked while it runs; a video that outruns it says so and points at `upload`.
 
-The download route caps the pull at 1080p — Baker re-encodes for streaming anyway, so fetching a 4K master (what modern YouTube serves by default) only makes the download slower for pixels that get discarded.
-
 **Blocked downloads climb the egress ladder automatically.** The download route goes through the same **direct → datacenter → residential** ladder as every other outbound read (`@baker/proxy`; see [Operations](../../docs/operations.md) → Egress Proxy). There is no proxy flag on any command and nothing to opt into — the unproxied attempt always runs first, so a paid exit costs nothing until a host actually refuses us, and with no `OXYLABS_*` credentials in the environment this is a single plain download.
 
 A rung only opens on evidence that *who we are* was the problem — a 403/429/451, a WAF connection reset, or a bot wall like YouTube's "Sign in to confirm you're not a bot". A private video, a deleted one, a 404 or an unsupported URL fails identically from every address on earth, so it stops after one attempt instead of burning a timeout per tier. That decision is `shouldEscalate`'s, shared with the capture engine and the ad-media downloaders; yt-dlp's stderr is only *parsed* into the signal it takes (`src/lib/ytDlpSignal.ts`), never allowed to judge for itself.
 
+**A YouTube refusal that no exit IP can fix gets a different player, not a different address.** yt-dlp's default client selection currently lands on `android_vr`, whose media URLs YouTube signs and then rejects: extraction succeeds, a format is chosen, and the byte fetch returns `unable to download video data: HTTP Error 403: Forbidden`. Measured from a Runtime, that failure is identical on direct, datacenter *and* residential — so the ladder spends both paid rungs reproducing it and the video never arrives. Only once the ladder has been exhausted on that exact phrase does the download retry with `--extractor-args youtube:player_client=android` (`src/lib/ytDlpPlayerClient.ts`), which resolves a progressive format the same address is allowed to fetch. **That retry repeats the rung it was refused on and opens no new one** — it is a different player, not a different address, and walking the ladder a second time would re-climb through both paid tiers on a 403, which is the spend this exists to stop. A fallback rather than the default because it currently offers only 360p: worth reaching for after the good formats have been refused, worth nothing before that. The page-level bot wall ("Sign in to confirm you're not a bot") is deliberately *not* matched here — a different IP does fix that one, and it belongs to the ladder.
+
+The retry is **automatic and unasked-for; its cost is not**. There is no flag and nothing to opt into — but a download that only completed this way comes back with `reducedQuality: true` and a hint saying the clip is standard quality, for the same reason `egress` names the rung: a price the caller did not choose has to be reported, and a 360p clip is otherwise indistinguishable from a 1080p one until someone watches it. The canvas `ingest` node shares the same downloader and reports it the same way, as `reduced_quality: true` on the asset's metadata plus a line in the run log.
+
 Requires `yt-dlp` on `PATH` (present in the Baker Runtime). Private, age-restricted, or region-locked videos cannot be downloaded — the error carries a `fix` pointing at `baker videos upload <file>`.
 
 **yt-dlp goes stale, and that looks like a broken video.** Large platforms change their players every few weeks and an out-of-date `yt-dlp` then resolves *no* formats at all, failing with "Requested format is not available" or "Only images are available" rather than anything that names the real cause. The error classifies this case separately and says the link is probably fine. The Runtime image installs `yt-dlp` unpinned at build time (`packages/e2b-template/src/template.ts`), so it ages with the image — if platform ingests start failing across the board, rebuild the template before investigating anything else. Direct file URLs don't depend on `yt-dlp` at all and keep working regardless.
@@ -2774,6 +2778,10 @@ baker scheduled-actions update <id-or-temp_sched_id> --run-at "2026-09-02T07:00:
 baker scheduled-actions delete <id-or-temp_sched_id>
 
 baker scheduled-actions trigger <id>
+
+baker scheduled-actions templates --list                  # every recipe this company can run
+baker scheduled-actions templates --category measurement  # one group only
+baker scheduled-actions create --template baker-ads --name "Weekly ad review" --description "..." --cron "0 8 * * MON"
 ```
 
 Rules:
@@ -2787,6 +2795,34 @@ Rules:
 - Scheduled actions spawn agents by default; create with `--no-spawn-agent` or update with `--spawn-agent false` to disable it.
 - `--prompt` maps to the scheduled action's additional spawned-agent instructions (`agentPrompt`), not a replacement for Baker's automatic spawned Work Action reference.
 - `--prompt` requires an agent: it is rejected alongside `--no-spawn-agent` (create) or `--spawn-agent false` (update), since a bypassed agent never reads it.
+- `--template <id>` binds the schedule to a marketplace recipe instead of a prompt you wrote. The row stores the id alone: name, brief and prompt resolve at fire time, so the job improves when the recipe does. `--description` carries what is true of this company. One row per recipe per company; a second `create` for the same id is refused.
+
+#### Marketplace recipes (`templates`)
+
+`templates` lists every recipe a company can switch on — the ones Baker maintains plus any the company wrote — and is also how one of theirs is written. Recipes we maintain take priority: a company recipe can never use one of our ids, and resolution reads ours first.
+
+```bash
+baker scheduled-actions templates --list --drafts        # include theirs not offered yet
+
+# Writing one of theirs, in this order:
+baker scheduled-actions templates --save weekly-posthog-check \
+  --name "Weekly PostHog check" \
+  --tagline "What moved in the funnel, from PostHog rather than GA4." \
+  --recipe-category measurement --frequency weekly \
+  --brief-file ./brief.md
+
+# ...schedule it with --template, trigger it, read what the run produced, then:
+baker scheduled-actions templates --publish weekly-posthog-check
+baker scheduled-actions templates --delete weekly-posthog-check
+```
+
+- **Saving does not offer it.** A saved recipe is invisible and cannot be scheduled until `--publish`. The step between the two is running it: nothing enforces that, and it is the only way to know it produces anything worth reading.
+- **`--publish` refuses what it can check**: a brief too short to run unattended, a cadence or category that does not exist, an id that shadows one of Baker's, and a brief naming no source this company has connected. The problems come back as text addressed to the agent.
+- **Only company-wide and organization-wide connections count.** A run fires unattended, so a tool one person connected for themselves works in the chat where the recipe was written and returns nothing every run after that.
+- **Prefer `--brief-file`** over `--brief`: shell quoting eats apostrophes and newlines, and the brief is the whole recipe.
+- **Publishing stages**, like every other Session effect. The recipe reaches the marketplace when the chat is published and is listed there as a change; discarding the chat leaves the marketplace as it was. `BAKER_CHAT_ID` is required for `--save`, `--publish` and `--delete`.
+- **A company's own starts on Recommends** and offers every cadence. The company changes that from its Marketplace tab.
+- **`--delete` stops any schedule still pointing at it** — the response says how many.
 
 ---
 

package/dist/{chunk-3E33AGNR.js → chunk-2IB3IL4K.js}

@@ -1,6 +1,6 @@
 import {
   debugLogSetting
-} from "./chunk-IRNM3T6B.js";
+} from "./chunk-DZUVUGEP.js";
 
 // src/debugLog.ts
 import { appendFileSync, mkdirSync, renameSync, statSync } from "fs";
@@ -150,4 +150,4 @@ export {
   readBodyForLog,
   installStreamTaps
 };
-//# sourceMappingURL=chunk-3E33AGNR.js.map
+//# sourceMappingURL=chunk-2IB3IL4K.js.map

package/dist/{chunk-CRXSLYEX.js → chunk-3FBQKD7O.js}

@@ -1,10 +1,10 @@
 import {
   debugLogHttp,
   readBodyForLog
-} from "./chunk-3E33AGNR.js";
+} from "./chunk-2IB3IL4K.js";
 import {
   getEnv
-} from "./chunk-IRNM3T6B.js";
+} from "./chunk-DZUVUGEP.js";
 
 // src/client.ts
 var MAX_RATE_LIMIT_RETRIES = 3;
@@ -198,4 +198,4 @@ export {
   apiGet,
   apiPost
 };
-//# sourceMappingURL=chunk-CRXSLYEX.js.map
+//# sourceMappingURL=chunk-3FBQKD7O.js.map

package/dist/{chunk-FUSTYLP5.js → chunk-5I66YT7A.js}

@@ -1,10 +1,10 @@
 import {
   ApiError,
   apiGet
-} from "./chunk-CRXSLYEX.js";
+} from "./chunk-3FBQKD7O.js";
 import {
   getEnv
-} from "./chunk-IRNM3T6B.js";
+} from "./chunk-DZUVUGEP.js";
 
 // src/error-handler.ts
 var REQUEST_CONNECTION_PLATFORM = {
@@ -295,4 +295,4 @@ export {
   writeAdsOutput,
   resolveCustomerId
 };
-//# sourceMappingURL=chunk-FUSTYLP5.js.map
+//# sourceMappingURL=chunk-5I66YT7A.js.map

package/dist/{chunk-IRNM3T6B.js → chunk-DZUVUGEP.js}

@@ -334,6 +334,7 @@ export {
   shouldEscalate,
   isPrivateAddress,
   refuseNonPublicUrl,
+  DIRECT_ROUTE,
   plannedRoutes,
   getEnv,
   debugLogSetting,
@@ -344,4 +345,4 @@ export {
   captureVimeoCookie,
   childEnvWith
 };
-//# sourceMappingURL=chunk-IRNM3T6B.js.map
+//# sourceMappingURL=chunk-DZUVUGEP.js.map

package/dist/{chunk-3WODSPNZ.js → chunk-P6ZNIFAQ.js}

@@ -2,13 +2,13 @@ import {
   handleConnectionError,
   needsConnectionFix,
   writeAdsJson
-} from "./chunk-FUSTYLP5.js";
+} from "./chunk-5I66YT7A.js";
 import {
   ApiError
-} from "./chunk-CRXSLYEX.js";
+} from "./chunk-3FBQKD7O.js";
 import {
   getEnv
-} from "./chunk-IRNM3T6B.js";
+} from "./chunk-DZUVUGEP.js";
 
 // src/commands/ads/meta/shared.ts
 var DAY_MS = 864e5;
@@ -108,4 +108,4 @@ export {
   csvOrJson,
   resolveEffectiveStatus
 };
-//# sourceMappingURL=chunk-3WODSPNZ.js.map
+//# sourceMappingURL=chunk-P6ZNIFAQ.js.map

package/dist/{chunk-E2WYWHJJ.js → chunk-XGIJGOKR.js}

@@ -1,8 +1,9 @@
 import {
   debugLogHttp,
   readBodyForLog
-} from "./chunk-3E33AGNR.js";
+} from "./chunk-2IB3IL4K.js";
 import {
+  DIRECT_ROUTE,
   __commonJS,
   __toESM,
   captureProxyCredentials,
@@ -13,11 +14,11 @@ import {
   plannedRoutes,
   refuseNonPublicUrl,
   shouldEscalate
-} from "./chunk-IRNM3T6B.js";
+} from "./chunk-DZUVUGEP.js";
 
-// ../../node_modules/.pnpm/safe-stable-stringify@2.5.0/node_modules/safe-stable-stringify/index.js
+// ../../.pnpm-store/v10/links/@/safe-stable-stringify/2.5.0/810146e81bae4e3a061fe487864f2fde80c4b03b886877dc0f1fffbc6480b67e/node_modules/safe-stable-stringify/index.js
 var require_safe_stable_stringify = __commonJS({
-  "../../node_modules/.pnpm/safe-stable-stringify@2.5.0/node_modules/safe-stable-stringify/index.js"(exports, module) {
+  "../../.pnpm-store/v10/links/@/safe-stable-stringify/2.5.0/810146e81bae4e3a061fe487864f2fde80c4b03b886877dc0f1fffbc6480b67e/node_modules/safe-stable-stringify/index.js"(exports, module) {
     "use strict";
     var { hasOwnProperty } = Object.prototype;
     var stringify = configure2();
@@ -1091,7 +1092,7 @@ function resolveAdaptFormats(params) {
   return params.formats ?? [];
 }
 
-// ../../node_modules/.pnpm/safe-stable-stringify@2.5.0/node_modules/safe-stable-stringify/esm/wrapper.js
+// ../../.pnpm-store/v10/links/@/safe-stable-stringify/2.5.0/810146e81bae4e3a061fe487864f2fde80c4b03b886877dc0f1fffbc6480b67e/node_modules/safe-stable-stringify/esm/wrapper.js
 var import__ = __toESM(require_safe_stable_stringify(), 1);
 var configure = import__.default.configure;
 var wrapper_default = import__.default;
@@ -5260,6 +5261,23 @@ function isVimeoAuthFailure(detail) {
   return VIMEO_AUTH_SIGNATURES.some((signature) => haystack.includes(signature));
 }
 
+// src/lib/ytDlpPlayerClient.ts
+var YOUTUBE_FALLBACK_CLIENT_ARGS = ["--extractor-args", "youtube:player_client=android"];
+var MEDIA_FETCH_REFUSED = "unable to download video data";
+var YOUTUBE_HOSTS = ["youtube.com", "youtu.be"];
+function isYouTube(url) {
+  let host;
+  try {
+    host = new URL(url).hostname.toLowerCase();
+  } catch {
+    return false;
+  }
+  return YOUTUBE_HOSTS.some((known) => host === known || host.endsWith(`.${known}`));
+}
+function needsPlayerClientRetry(url, stderr) {
+  return isYouTube(url) && stderr.toLowerCase().includes(MEDIA_FETCH_REFUSED);
+}
+
 // src/lib/ytDlpSignal.ts
 var HTTP_ERROR = /HTTP Error (\d{3})/i;
 var CHALLENGE_PHRASES = [
@@ -5312,18 +5330,18 @@ async function runYtDlp(args) {
   if (args.audioOnly) {
     argv.push("-x", "--audio-format", "m4a");
   } else {
-    const format = args.maxHeight ? `bv*[height<=${args.maxHeight}]+ba/b[height<=${args.maxHeight}]/bv*+ba/b` : "bv*+ba/b";
-    argv.push("--format", format, "--merge-output-format", "mp4");
+    argv.push("--format", "bv*+ba/b", "--merge-output-format", "mp4");
   }
   const cookieFile = await writeVimeoCookieFile(args.vimeoCookie, args.workDir);
   if (cookieFile) argv.push("--cookies", cookieFile);
   args.onLog?.(`yt-dlp: downloading ${args.audioOnly ? "audio" : "video"}`);
-  const { code, stderr, tier } = await runViaLadder({
+  const timeoutMs = args.timeoutMs ?? YT_DLP_TIMEOUT_MS;
+  const { code, stderr, tier, reducedQuality } = await downloadWithClientFallback({
     url: args.url,
     argv,
-    timeoutMs: args.timeoutMs ?? YT_DLP_TIMEOUT_MS,
     onLog: args.onLog,
-    country: args.country
+    ladder: (attemptArgv) => runViaLadder({ url: args.url, argv: attemptArgv, timeoutMs, onLog: args.onLog, country: args.country }),
+    repeat: (attemptArgv, route) => spawnAndWait(YT_DLP_BIN, attemptArgv, timeoutMs, { extraEnv: proxyEnvFor(route) })
   });
   if (code !== 0) {
     throw new YtDlpError(`yt-dlp exited ${code}`, tail(stderr, 40));
@@ -5339,7 +5357,7 @@ async function runYtDlp(args) {
     info = JSON.parse(await readFile3(infoPath, "utf-8"));
   } catch {
   }
-  return { filePath: path5.join(args.workDir, downloaded), info, tier };
+  return { filePath: path5.join(args.workDir, downloaded), info, tier, reducedQuality };
 }
 async function probeYtDlp(args) {
   const argv = [
@@ -5402,16 +5420,25 @@ function canonicalUrl(info) {
 }
 async function walkYtDlpLadder(args) {
   const routes = plannedRoutes(args.url, args.credentials, args.country ? { country: args.country } : void 0);
-  let last = { code: -1, stderr: "", tier: "direct" };
+  let last = { code: -1, stderr: "", tier: "direct", route: DIRECT_ROUTE };
   for (const route of routes) {
     const tier = route.kind === "proxy" ? route.tier : "direct";
     if (route.kind === "proxy") args.onLog?.(`yt-dlp: retrying via ${tier}`);
-    last = { ...await args.attempt(route), tier };
+    last = { ...await args.attempt(route), tier, route };
     if (last.code === 0) return last;
     if (!shouldEscalate(ytDlpBlockSignal(last.stderr))) return last;
   }
   return last;
 }
+async function downloadWithClientFallback(args) {
+  const first = await args.ladder(args.argv);
+  if (first.code === 0 || !needsPlayerClientRetry(args.url, first.stderr)) {
+    return { ...first, reducedQuality: false };
+  }
+  args.onLog?.("yt-dlp: retrying with the fallback player client");
+  const again = await args.repeat([...args.argv, ...YOUTUBE_FALLBACK_CLIENT_ARGS], first.route);
+  return { ...again, tier: first.tier, route: first.route, reducedQuality: again.code === 0 };
+}
 function runViaLadder(args) {
   return walkYtDlpLadder({
     url: args.url,
@@ -6103,7 +6130,7 @@ async function execYtDlp(params, ctx) {
   }
   const workDir = await mkdtemp(path6.join(tmpdir(), "ingest-yt-"));
   try {
-    const { filePath, info } = await runYtDlpForNode({
+    const { filePath, info, reducedQuality } = await runYtDlpForNode({
       url: params.url,
       audioOnly: params.expect === "audio",
       workDir,
@@ -6120,6 +6147,10 @@ async function execYtDlp(params, ctx) {
     const kind = params.expect;
     const mime = YT_DLP_MIME[kind];
     const metadata = buildYtDlpMetadata(params.url, info);
+    if (reducedQuality) {
+      metadata.reduced_quality = true;
+      ctx.log(`ingest: ${params.url} was only available at reduced resolution`);
+    }
     return await uploadAndIngest({ bytes, kind, mime, metadata, ctx });
   } finally {
     await rm(workDir, { recursive: true, force: true }).catch((e) => {
@@ -8924,4 +8955,4 @@ export {
   defaultRegistry,
   createEngineFromEnv
 };
-//# sourceMappingURL=chunk-E2WYWHJJ.js.map
+//# sourceMappingURL=chunk-XGIJGOKR.js.map