pi-multimodal-proxy 1.12.0 → 1.12.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
4
4
 
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
6
6
 
7
+ ## [1.12.1] - 2026-08-07
8
+
9
+ ### Fixed
10
+
11
+ - **Downloaded videos now save to the real Downloads folder.** The post-analysis "Save downloaded video?" prompt hardcoded `join(os.homedir(), "Downloads")` and wrapped `copyFile` in a silent `catch {}`. On machines where the Windows Downloads folder is relocated (e.g. `D:\Downloads`) or OneDrive-redirected, the default path doesn't exist, so `copyFile` threw `ENOENT`, the error was swallowed, and the user saw no file and no message after clicking **Yes**. `vision-proxy.ts` now resolves the real Downloads folder via the Windows Shell known-folder API (`FOLDERID_Downloads` / `shell:Downloads`, which respects relocation and OneDrive redirection), creates it on demand (`mkdir -p`), and surfaces any save failure with the target path. The success notification now includes the full saved path. New helpers: `resolveDownloadsDir`, `pathExists`.
12
+
13
+ ### Added
14
+
15
+ - **yt-dlp auth knobs to defeat YouTube 403s.** YouTube increasingly returns `HTTP Error 403: Forbidden` on the media fetch even with the latest `yt-dlp`; the proxy previously passed no credentials. Two opt-in, persisted settings now forward to yt-dlp: `ytdlpCookiesFromBrowser` (`/multimodal-proxy ytdlp cookies <browser|off>`, validated against `chrome|firefox|edge|brave|opera|safari|vivaldi|chromium|whale`) reuses a logged-in browser session, and `ytdlpExtractorArgs` (`/multimodal-proxy ytdlp extractor-args "<text>|off>`) forwards arbitrary `--extractor-args` (e.g. `youtube:player_client=web_safari,web`). A 403 download failure now appends a hint pointing at the cookies knob. Env overrides: `PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER`, `PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS`. New sanitizers + tests: `sanitizeYtdlpCookiesFromBrowser`, `sanitizeYtdlpExtractorArgs`, `YTDLP_COOKIES_BROWSERS`.
16
+
7
17
  ## [1.11.0] - 2026-07-29
8
18
 
9
19
  ### Added
package/README.md CHANGED
@@ -8,6 +8,11 @@ When **video or audio files** are detected, they are routed to a **multimodal mo
8
8
 
9
9
  **YouTube links** are detected too: paste a URL (`youtube.com/watch?v=…`, `youtu.be/…`, `/shorts/…`, etc.) and the video is downloaded with [`yt-dlp`](https://github.com/yt-dlp/yt-dlp) and analyzed exactly like a local file.
10
10
 
11
+ ## What's new in 1.12.1
12
+
13
+ - **Fix: downloaded videos now save to your real Downloads folder.** The "Save downloaded video?" prompt hardcoded `~/Downloads` and wrapped the copy in a silent `catch`. If you relocated Downloads (e.g. to `D:\Downloads`) or use OneDrive redirection, that path doesn't exist — so clicking **Yes** silently failed with no file and no error. The proxy now resolves the real Downloads folder via the Windows Shell known-folder API (`FOLDERID_Downloads`, respects relocation + OneDrive), creates it if missing, and surfaces any save error with the target path. The success toast now shows the full saved path.
14
+ - **Configurable yt-dlp auth (defeats YouTube 403s).** YouTube increasingly rejects unauthenticated media fetches with `HTTP 403`, even on the latest `yt-dlp`. Two opt-in knobs satisfy the check: `/multimodal-proxy ytdlp cookies <browser>` reuses a logged-in browser session (`chrome`, `firefox`, `edge`, `brave`, …), and `/multimodal-proxy ytdlp extractor-args "<args>"` forwards arbitrary `--extractor-args` (e.g. `youtube:player_client=web_safari,web`). A 403 failure now also hints at the cookies knob. Env overrides: `PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER`, `PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS`.
15
+
11
16
  ## What's new in 1.12.0
12
17
 
13
18
  - **YouTube video download** — paste a YouTube URL in your prompt and the extension downloads it via `yt-dlp` and analyzes it like any local video file. Gated by path detection (on by default); requires `yt-dlp` on your PATH. See [YouTube videos](#youtube-videos). After successful analysis you are prompted to optionally keep the downloaded file in your Downloads folder.
@@ -76,6 +81,16 @@ winget install yt-dlp.yt-dlp # Windows
76
81
 
77
82
  `ffmpeg` is also used (for duration probing and stream merging) — it usually ships alongside yt-dlp. Downloads are capped to ≤720p and rejected past the configured size limit (`PI_VISION_PROXY_MAX_VIDEO_BYTES`, default 200 MB). To disable URL auto-download, turn off path detection: `/multimodal-proxy path-detection off`.
78
83
 
84
+ #### 403 Forbidden / download failures
85
+
86
+ YouTube periodically rejects unauthenticated media fetches with `HTTP Error 403: Forbidden`, even on the latest `yt-dlp`. The proxy can't fix YouTube's backend, but you can satisfy the auth check:
87
+
88
+ - **Reuse a logged-in browser session** — `/multimodal-proxy ytdlp cookies chrome` (also `firefox`, `edge`, `brave`, `opera`, `safari`, `vivaldi`, `chromium`, `whale`). yt-dlp reads that browser's YouTube cookies. This fixes most 403s.
89
+ - **Try alternate player clients** — `/multimodal-proxy ytdlp extractor-args "youtube:player_client=web_safari,web"`.
90
+ - Show current settings with `/multimodal-proxy ytdlp`; turn either off with `... off`.
91
+
92
+ Both are persisted in `~/.pi/agent/multimodal-proxy.json` and can be set via env (`PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER`, `PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS`), which override and lock the command.
93
+
79
94
  ## Modes
80
95
 
81
96
  | Mode | Behavior |
@@ -51,6 +51,9 @@ import {
51
51
  isValidNamedRegion,
52
52
  expandLeadingTilde,
53
53
  sanitizeAllowedFolders,
54
+ sanitizeYtdlpCookiesFromBrowser,
55
+ sanitizeYtdlpExtractorArgs,
56
+ YTDLP_COOKIES_BROWSERS,
54
57
  pathAccessFromConfig,
55
58
  MAX_ALLOWED_FOLDERS,
56
59
  LRUCache,
@@ -265,18 +268,18 @@ describe("readEnvOverrides", () => {
265
268
 
266
269
  describe("envFlags", () => {
267
270
  it("reports presence per variable", () => {
268
- assert.deepEqual(envFlags({}), { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false });
271
+ assert.deepEqual(envFlags({}), { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false, ytdlpCookies: false, ytdlpExtractorArgs: false });
269
272
  assert.deepEqual(
270
273
  envFlags({
271
274
  PI_VISION_PROXY_MODE: "x",
272
275
  PI_VISION_PROXY_MODEL: "y",
273
276
  PI_VISION_PROXY_INCLUDE_CONTEXT: "",
274
277
  }),
275
- { mode: true, model: true, context: true, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false },
278
+ { mode: true, model: true, context: true, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false, ytdlpCookies: false, ytdlpExtractorArgs: false },
276
279
  );
277
280
  assert.deepEqual(
278
281
  envFlags({ PI_VISION_PROXY_ALLOW_HOME: "1", PI_VISION_PROXY_ALLOWED_FOLDERS: "/a" }),
279
- { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: true, allowedFolders: true, statusLine: false, pathDetection: false },
282
+ { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: true, allowedFolders: true, statusLine: false, pathDetection: false, ytdlpCookies: false, ytdlpExtractorArgs: false },
280
283
  );
281
284
  assert.equal(envFlags({ PI_VISION_PROXY_ALLOWED_PROVIDERS: "" }).allowedProviders, true);
282
285
  // An unrecognized ALLOW_HOME value is not an override and must not lock the command
@@ -287,6 +290,32 @@ describe("envFlags", () => {
287
290
  // An invalid value is not applied by readEnvOverrides, so it must not
288
291
  // report (and thereby lock) the setting as env-overridden either.
289
292
  assert.equal(envFlags({ PI_VISION_PROXY_STATUS_LINE: "bogus" }).statusLine, false);
293
+ assert.equal(envFlags({ PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER: "chrome" }).ytdlpCookies, true);
294
+ assert.equal(envFlags({ PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS: "youtube:player_client=web" }).ytdlpExtractorArgs, true);
295
+ });
296
+ });
297
+
298
+ describe("yt-dlp config sanitizers", () => {
299
+ it("accepts known browsers case-insensitively, rejects others", () => {
300
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("Chrome"), "chrome");
301
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("FIREFOX"), "firefox");
302
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("edge"), "edge");
303
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("internet explorer"), "");
304
+ assert.equal(sanitizeYtdlpCookiesFromBrowser(""), "");
305
+ assert.equal(sanitizeYtdlpCookiesFromBrowser(undefined), "");
306
+ assert.equal(sanitizeYtdlpCookiesFromBrowser(123 as any), "");
307
+ });
308
+ it("exposes the supported browser set", () => {
309
+ assert.ok(YTDLP_COOKIES_BROWSERS.has("chrome"));
310
+ assert.ok(YTDLP_COOKIES_BROWSERS.has("brave"));
311
+ });
312
+ it("strips control chars and caps extractor-args length", () => {
313
+ assert.equal(sanitizeYtdlpExtractorArgs("youtube:player_client=web"), "youtube:player_client=web");
314
+ assert.equal(sanitizeYtdlpExtractorArgs(" trim me "), "trim me");
315
+ assert.equal(sanitizeYtdlpExtractorArgs("a\x00b\x07c"), "abc");
316
+ assert.equal(sanitizeYtdlpExtractorArgs(""), "");
317
+ const long = "x".repeat(600);
318
+ assert.equal(sanitizeYtdlpExtractorArgs(long).length, 500);
290
319
  });
291
320
  });
292
321
 
@@ -79,6 +79,12 @@ export interface VisionConfig {
79
79
  // (event.images) are always processed; only the convenience path-scan is
80
80
  // gated. Explicit references via /multimodal-proxy describe still work.
81
81
  pathDetection: ToolSetting;
82
+ // 1.12.1 — yt-dlp tuning to defeat YouTube 403s on the media fetch. Both
83
+ // empty by default. cookiesFromBrowser reuses a logged-in browser session
84
+ // (e.g. "chrome"); extractorArgs forwards arbitrary --extractor-args (e.g.
85
+ // "youtube:player_client=web_safari,web").
86
+ ytdlpCookiesFromBrowser: string;
87
+ ytdlpExtractorArgs: string;
82
88
  }
83
89
 
84
90
  export interface ImageMeta {
@@ -704,6 +710,8 @@ export const DEFAULT_CONFIG: VisionConfig = {
704
710
  "google/gemini-2.5-pro": { format: "gemini_normalized_1000" },
705
711
  "google/gemini-3-pro": { format: "gemini_normalized_1000" },
706
712
  },
713
+ ytdlpCookiesFromBrowser: "",
714
+ ytdlpExtractorArgs: "",
707
715
  };
708
716
 
709
717
  // ── Persistent file storage ────────────────────────────────────────────────
@@ -729,6 +737,8 @@ const PERSISTED_CONFIG_KEYS = new Set([
729
737
  "allowedFolders", "allowHome",
730
738
  "statusLine",
731
739
  "pathDetection",
740
+ "ytdlpCookiesFromBrowser",
741
+ "ytdlpExtractorArgs",
732
742
  ]);
733
743
 
734
744
  /** Read config from the persistent file. Returns empty object on any failure. */
@@ -868,10 +878,15 @@ export function readEnvOverrides(env: NodeJS.ProcessEnv = process.env): Partial<
868
878
  if (foldersEnv !== undefined) {
869
879
  overrides.allowedFolders = sanitizeAllowedFolders(foldersEnv.split(delimiter));
870
880
  }
881
+ // 1.12.1 yt-dlp env overrides
882
+ const cookiesEnv = env.PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER;
883
+ if (cookiesEnv !== undefined) overrides.ytdlpCookiesFromBrowser = sanitizeYtdlpCookiesFromBrowser(cookiesEnv);
884
+ const extractorArgsEnv = env.PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS;
885
+ if (extractorArgsEnv !== undefined) overrides.ytdlpExtractorArgs = sanitizeYtdlpExtractorArgs(extractorArgsEnv);
871
886
  return overrides;
872
887
  }
873
888
 
874
- export function envFlags(env: NodeJS.ProcessEnv = process.env): { mode: boolean; model: boolean; context: boolean; tool: boolean; maxImagesPerCall: boolean; maxBatch: boolean; cacheSize: boolean; videoModel: boolean; allowedProviders: boolean; allowHome: boolean; allowedFolders: boolean; statusLine: boolean; pathDetection: boolean } {
889
+ export function envFlags(env: NodeJS.ProcessEnv = process.env): { mode: boolean; model: boolean; context: boolean; tool: boolean; maxImagesPerCall: boolean; maxBatch: boolean; cacheSize: boolean; videoModel: boolean; allowedProviders: boolean; allowHome: boolean; allowedFolders: boolean; statusLine: boolean; pathDetection: boolean; ytdlpCookies: boolean; ytdlpExtractorArgs: boolean } {
875
890
  return {
876
891
  mode: Boolean(env.PI_VISION_PROXY_MODE),
877
892
  model: Boolean(env.PI_VISION_PROXY_MODEL),
@@ -891,6 +906,8 @@ export function envFlags(env: NodeJS.ProcessEnv = process.env): { mode: boolean;
891
906
  statusLine: env.PI_VISION_PROXY_STATUS_LINE === "on" || env.PI_VISION_PROXY_STATUS_LINE === "off",
892
907
  // Same rule: only a recognized value overrides path detection.
893
908
  pathDetection: env.PI_VISION_PROXY_PATH_DETECTION === "on" || env.PI_VISION_PROXY_PATH_DETECTION === "off",
909
+ ytdlpCookies: env.PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER !== undefined,
910
+ ytdlpExtractorArgs: env.PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS !== undefined,
894
911
  };
895
912
  }
896
913
 
@@ -984,6 +1001,33 @@ export function sanitizeAllowedFolders(value: unknown): string[] {
984
1001
  return out;
985
1002
  }
986
1003
 
1004
+ /** Browsers accepted by yt-dlp's --cookies-from-browser (1.12.1). */
1005
+ export const YTDLP_COOKIES_BROWSERS = new Set([
1006
+ "chrome", "firefox", "edge", "brave", "opera", "safari", "vivaldi", "chromium", "whale",
1007
+ ]);
1008
+
1009
+ /** Max length of the freeform --extractor-args value (1.12.1). */
1010
+ export const YTDLP_EXTRACTOR_ARGS_MAX = 500;
1011
+
1012
+ /**
1013
+ * Validate a yt-dlp --cookies-from-browser value (1.12.1). Lowercases and
1014
+ * accepts only known browser names; anything else collapses to "" (off).
1015
+ */
1016
+ export function sanitizeYtdlpCookiesFromBrowser(value: unknown): string {
1017
+ if (typeof value !== "string") return "";
1018
+ const v = value.trim().toLowerCase();
1019
+ return YTDLP_COOKIES_BROWSERS.has(v) ? v : "";
1020
+ }
1021
+
1022
+ /**
1023
+ * Validate a yt-dlp --extractor-args value (1.12.1). Strips control/null bytes
1024
+ * and caps length; returned verbatim (yt-dlp parses the single arg value).
1025
+ */
1026
+ export function sanitizeYtdlpExtractorArgs(value: unknown): string {
1027
+ if (typeof value !== "string") return "";
1028
+ return value.replace(/[\x00-\x1f\x7f]/g, "").trim().slice(0, YTDLP_EXTRACTOR_ARGS_MAX);
1029
+ }
1030
+
987
1031
  export function sanitize(config: VisionConfig): VisionConfig {
988
1032
  const safe: VisionConfig = { ...config };
989
1033
  if (typeof safe.provider === "string") safe.provider = canonicalProvider(safe.provider);
@@ -1052,6 +1096,9 @@ export function sanitize(config: VisionConfig): VisionConfig {
1052
1096
  if (safe.statusLine !== "on" && safe.statusLine !== "off") safe.statusLine = DEFAULT_CONFIG.statusLine;
1053
1097
  // 1.11.0 path-detection field
1054
1098
  if (safe.pathDetection !== "on" && safe.pathDetection !== "off") safe.pathDetection = DEFAULT_CONFIG.pathDetection;
1099
+ // 1.12.1 yt-dlp tuning fields
1100
+ safe.ytdlpCookiesFromBrowser = sanitizeYtdlpCookiesFromBrowser(safe.ytdlpCookiesFromBrowser);
1101
+ safe.ytdlpExtractorArgs = sanitizeYtdlpExtractorArgs(safe.ytdlpExtractorArgs);
1055
1102
  return safe;
1056
1103
  }
1057
1104
 
@@ -37,13 +37,15 @@
37
37
  * PI_VISION_PROXY_MAX_VIDEO_BYTES - positive integer
38
38
  * PI_VISION_PROXY_ALLOWED_PROVIDERS - comma-separated pre-consented providers
39
39
  * PI_VISION_PROXY_STATUS_LINE - "on" | "off"
40
+ * PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER - chrome|firefox|edge|brave|opera|safari|vivaldi|chromium|whale (defeats YouTube 403s)
41
+ * PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS - e.g. "youtube:player_client=web_safari,web"
40
42
  *
41
43
  * Install:
42
44
  * pi install ./packages/pi-multimodal-proxy
43
45
  */
44
46
 
45
47
  import { execFile } from "node:child_process";
46
- import { copyFile, mkdtemp, readFile, readdir, rm } from "node:fs/promises";
48
+ import { access, copyFile, mkdir, mkdtemp, readFile, readdir, rm } from "node:fs/promises";
47
49
  import os from "node:os";
48
50
  import { isAbsolute, join } from "node:path";
49
51
  import { promisify } from "node:util";
@@ -128,6 +130,8 @@ import {
128
130
  expandLeadingTilde,
129
131
  isUncPath,
130
132
  MAX_ALLOWED_FOLDERS,
133
+ sanitizeYtdlpExtractorArgs,
134
+ YTDLP_COOKIES_BROWSERS,
131
135
 
132
136
  readMediaFileWithReason,
133
137
  type ReadMediaReason,
@@ -975,6 +979,7 @@ async function downloadMediaUrl(
975
979
  url: string,
976
980
  signal: AbortSignal | undefined,
977
981
  ctx: ExtensionContext,
982
+ ytdlp?: { cookiesFromBrowser?: string; extractorArgs?: string },
978
983
  ): Promise<DownloadedMedia | null> {
979
984
  if (!(await checkYtDlp())) {
980
985
  ctx.ui.notify(
@@ -991,19 +996,27 @@ async function downloadMediaUrl(
991
996
  // actual download — it can satisfy the print from metadata alone. Adding
992
997
  // an after_move field forces the download (it is only known once the file
993
998
  // is in its final location) and also hands us the exact output path.
999
+ const args = [
1000
+ "--no-playlist",
1001
+ "--no-warnings",
1002
+ "--no-progress",
1003
+ "-f", "best[ext=mp4][height<=720]/best[height<=720]/best",
1004
+ "--merge-output-format", "mp4",
1005
+ "-o", join(tempDir, "%(id)s.%(ext)s"),
1006
+ ];
1007
+ // Optional auth / player-client knobs (1.12.1) — defeat YouTube 403s on
1008
+ // the media fetch by reusing a logged-in browser session and/or alternate
1009
+ // player clients. Both opt-in; absent by default.
1010
+ if (ytdlp?.cookiesFromBrowser) {
1011
+ args.push("--cookies-from-browser", ytdlp.cookiesFromBrowser);
1012
+ }
1013
+ if (ytdlp?.extractorArgs) {
1014
+ args.push("--extractor-args", ytdlp.extractorArgs);
1015
+ }
1016
+ args.push("--print", "%(title)s", "--print", "after_move:filepath", url);
994
1017
  const { stdout } = await execFileAsync(
995
1018
  "yt-dlp",
996
- [
997
- "--no-playlist",
998
- "--no-warnings",
999
- "--no-progress",
1000
- "-f", "best[ext=mp4][height<=720]/best[height<=720]/best",
1001
- "--merge-output-format", "mp4",
1002
- "-o", join(tempDir, "%(id)s.%(ext)s"),
1003
- "--print", "%(title)s",
1004
- "--print", "after_move:filepath",
1005
- url,
1006
- ],
1019
+ args,
1007
1020
  { windowsHide: true, timeout: 240_000, maxBuffer: 4 * 1024 * 1024, signal },
1008
1021
  );
1009
1022
 
@@ -1027,11 +1040,77 @@ async function downloadMediaUrl(
1027
1040
  } catch (err) {
1028
1041
  if (signal?.aborted) return null;
1029
1042
  const msg = err instanceof Error ? err.message : String(err);
1030
- ctx.ui.notify(`[multimodal-proxy] YouTube download failed for ${url}: ${msg}`, "warning");
1043
+ // A 403 on the media fetch usually means YouTube wants a logged-in
1044
+ // session; point the user at the cookies knob if it isn't already set.
1045
+ const hint = /403|forbidden/i.test(msg) && !ytdlp?.cookiesFromBrowser
1046
+ ? " (a 403 often means YouTube wants a logged-in session — try /multimodal-proxy ytdlp cookies <browser>)"
1047
+ : "";
1048
+ ctx.ui.notify(`[multimodal-proxy] YouTube download failed for ${url}: ${msg}${hint}`, "warning");
1031
1049
  return null;
1032
1050
  }
1033
1051
  }
1034
1052
 
1053
+ // ── Downloads folder resolution ─────────────────────────────────────────────
1054
+ //
1055
+ // The Windows default `C:\Users\<user>\Downloads` is frequently NOT the folder
1056
+ // File Explorer shows: OneDrive redirection or a custom relocation (e.g.
1057
+ // `D:\Downloads`) moves it elsewhere. Hardcoding the default then makes
1058
+ // `copyFile` throw ENOENT — and pre-fix that error was swallowed silently, so
1059
+ // the user clicked "Save", nothing appeared, and no error was surfaced.
1060
+ //
1061
+ // We resolve the real folder via the Shell known-folder (FOLDERID_Downloads)
1062
+ // on Windows, cache the result, and fall back to `~/Downloads` elsewhere.
1063
+
1064
+ let downloadsDirCache: string | null = null;
1065
+
1066
+ async function pathExists(p: string): Promise<boolean> {
1067
+ try {
1068
+ await access(p);
1069
+ return true;
1070
+ } catch {
1071
+ return false;
1072
+ }
1073
+ }
1074
+
1075
+ /**
1076
+ * Resolve the user's real Downloads folder, respecting Windows relocation /
1077
+ * OneDrive redirection. Falls back to the default `~/Downloads` (which the
1078
+ * caller creates on demand). Result is cached for the process lifetime.
1079
+ */
1080
+ async function resolveDownloadsDir(): Promise<string> {
1081
+ if (downloadsDirCache) return downloadsDirCache;
1082
+ const defaultDir = join(os.homedir(), "Downloads");
1083
+ if (await pathExists(defaultDir)) {
1084
+ downloadsDirCache = defaultDir;
1085
+ return defaultDir;
1086
+ }
1087
+ // Default is missing (relocated / OneDrive-redirected). On Windows, ask the
1088
+ // Shell for the real Downloads known-folder path; this respects redirection.
1089
+ if (process.platform === "win32") {
1090
+ try {
1091
+ const { stdout } = await execFileAsync(
1092
+ "powershell.exe",
1093
+ [
1094
+ "-NoProfile",
1095
+ "-NonInteractive",
1096
+ "-Command",
1097
+ "(New-Object -ComObject Shell.Application).NameSpace('shell:Downloads').Self.Path",
1098
+ ],
1099
+ { windowsHide: true, timeout: 10_000 },
1100
+ );
1101
+ const resolved = stdout.split(/\r?\n/).map((l) => l.trim()).find((l) => l.length > 0) ?? "";
1102
+ if (resolved && (await pathExists(resolved))) {
1103
+ downloadsDirCache = resolved;
1104
+ return resolved;
1105
+ }
1106
+ } catch {
1107
+ // PowerShell unavailable or failed — fall back to default (created below).
1108
+ }
1109
+ }
1110
+ downloadsDirCache = defaultDir;
1111
+ return defaultDir;
1112
+ }
1113
+
1035
1114
  async function analyzeVideoViaXaiStt(
1036
1115
  mediaFile: { type: "image"; data: string; mimeType: string },
1037
1116
  filename: string,
@@ -1693,7 +1772,10 @@ export default function (pi: ExtensionAPI) {
1693
1772
  ctx,
1694
1773
  () => `Downloading YouTube ${id}…`,
1695
1774
  `Downloading YouTube ${id}…`,
1696
- () => downloadMediaUrl(watch, ctx.signal, ctx),
1775
+ () => downloadMediaUrl(watch, ctx.signal, ctx, {
1776
+ cookiesFromBrowser: config.ytdlpCookiesFromBrowser,
1777
+ extractorArgs: config.ytdlpExtractorArgs,
1778
+ }),
1697
1779
  );
1698
1780
  if (!dl) continue;
1699
1781
  downloadedTempDirs.push(dl.tempDir);
@@ -1813,15 +1895,23 @@ export default function (pi: ExtensionAPI) {
1813
1895
  "Save downloaded video?",
1814
1896
  `"${info.title}"\n\n${sizeMB} MB — save to your Downloads folder?`,
1815
1897
  );
1816
- if (save) {
1817
- const safeTitle = info.title.replace(/[<>:"\/\\|?*]/g, "_").replace(/\s+/g, " ").trim().slice(0, 200);
1818
- const dest = join(os.homedir(), "Downloads", `${safeTitle}.mp4`);
1898
+ if (!save) continue;
1899
+ const safeTitle = info.title.replace(/[<>:"\/\\|?*]/g, "_").replace(/\s+/g, " ").trim().slice(0, 200);
1900
+ const destDir = await resolveDownloadsDir();
1901
+ const dest = join(destDir, `${safeTitle}.mp4`);
1902
+ try {
1903
+ await mkdir(destDir, { recursive: true });
1819
1904
  await copyFile(path, dest);
1820
- ctx.ui.notify(`[multimodal-proxy] ✓ Saved "${safeTitle}.mp4" to Downloads`, "info");
1905
+ ctx.ui.notify(`[multimodal-proxy] ✓ Saved "${safeTitle}.mp4" to ${dest}`, "info");
1906
+ } catch (saveErr) {
1907
+ // Surface save failures (missing dir, permissions, disk) — the
1908
+ // previous silent catch hid them and the video just vanished.
1909
+ const reason = saveErr instanceof Error ? saveErr.message : String(saveErr);
1910
+ ctx.ui.notify(`[multimodal-proxy] ✗ Could not save video to ${dest}: ${reason}`, "warning");
1821
1911
  }
1822
1912
  } catch {
1823
- // Don't block the agent on save errors — temp files are
1824
- // cleaned below regardless.
1913
+ // confirm() dialog itself failed — don't block the agent. Temp
1914
+ // files are cleaned below regardless.
1825
1915
  }
1826
1916
  }
1827
1917
  await cleanupDownloads();
@@ -2522,6 +2612,73 @@ Use "*" or "all" to grant consent for all providers globally.`,
2522
2612
  return;
2523
2613
  }
2524
2614
 
2615
+ // ── yt-dlp cookies / extractor-args (defeat YouTube 403s) ───────
2616
+ if (sub === "ytdlp") {
2617
+ const { sub: ySub, value: yValue } = splitSubcommand(value);
2618
+
2619
+ if (ySub === "cookies") {
2620
+ if (env.ytdlpCookies) {
2621
+ ctx.ui.notify(
2622
+ "[multimodal-proxy] PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER is set - env overrides commands. Unset to change.",
2623
+ "warning",
2624
+ );
2625
+ return;
2626
+ }
2627
+ const trimmed = yValue.trim().toLowerCase();
2628
+ if (!trimmed || trimmed === "off") {
2629
+ writePersisted({ ...persisted, ytdlpCookiesFromBrowser: "" });
2630
+ ctx.ui.notify("[multimodal-proxy] yt-dlp cookies-from-browser: off", "info");
2631
+ return;
2632
+ }
2633
+ if (!YTDLP_COOKIES_BROWSERS.has(trimmed)) {
2634
+ ctx.ui.notify(
2635
+ `[multimodal-proxy] Unknown browser "${trimmed}". Supported: ${[...YTDLP_COOKIES_BROWSERS].join(", ")}`,
2636
+ "warning",
2637
+ );
2638
+ return;
2639
+ }
2640
+ writePersisted({ ...persisted, ytdlpCookiesFromBrowser: trimmed });
2641
+ ctx.ui.notify(
2642
+ `[multimodal-proxy] yt-dlp cookies-from-browser: ${trimmed} (applied on next YouTube download)`,
2643
+ "info",
2644
+ );
2645
+ return;
2646
+ }
2647
+
2648
+ if (ySub === "extractor-args") {
2649
+ if (env.ytdlpExtractorArgs) {
2650
+ ctx.ui.notify(
2651
+ "[multimodal-proxy] PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS is set - env overrides commands. Unset to change.",
2652
+ "warning",
2653
+ );
2654
+ return;
2655
+ }
2656
+ const trimmed = yValue.trim();
2657
+ if (!trimmed || trimmed.toLowerCase() === "off") {
2658
+ writePersisted({ ...persisted, ytdlpExtractorArgs: "" });
2659
+ ctx.ui.notify("[multimodal-proxy] yt-dlp extractor-args: off", "info");
2660
+ return;
2661
+ }
2662
+ const cleaned = sanitizeYtdlpExtractorArgs(trimmed);
2663
+ writePersisted({ ...persisted, ytdlpExtractorArgs: cleaned });
2664
+ ctx.ui.notify(`[multimodal-proxy] yt-dlp extractor-args: ${cleaned}`, "info");
2665
+ return;
2666
+ }
2667
+
2668
+ ctx.ui.notify(
2669
+ "[multimodal-proxy] yt-dlp options:\n" +
2670
+ ` cookies-from-browser: ${effective.ytdlpCookiesFromBrowser || "(off)"}\n` +
2671
+ ` extractor-args: ${effective.ytdlpExtractorArgs || "(off)"}\n` +
2672
+ (env.ytdlpCookies || env.ytdlpExtractorArgs ? " (env override active)\n" : "") +
2673
+ "Usage:\n" +
2674
+ " /multimodal-proxy ytdlp cookies <browser|off> reuse a logged-in YouTube session (fixes most 403s)\n" +
2675
+ ' /multimodal-proxy ytdlp extractor-args <text|off> e.g. youtube:player_client=web_safari,web\n' +
2676
+ `Browsers: ${[...YTDLP_COOKIES_BROWSERS].join(", ")}`,
2677
+ "info",
2678
+ );
2679
+ return;
2680
+ }
2681
+
2525
2682
  // ── max-images-per-call ────────────────────────────
2526
2683
  if (sub === "max-images-per-call") {
2527
2684
  if (env.maxImagesPerCall) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-multimodal-proxy",
3
- "version": "1.12.0",
3
+ "version": "1.12.1",
4
4
  "description": "Automatic image, video and audio description for any model in Pi. Routes media to a multimodal model and injects descriptions into context.",
5
5
  "keywords": [
6
6
  "pi-package"