pi-multimodal-proxy 1.12.0 → 1.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file.
4
4
 
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
6
6
 
7
+ ## [1.13.0] - 2026-08-09
8
+
9
+ ### Fixed
10
+ - Migrated to `ModelRegistry.complete`.
11
+
12
+ ## [1.12.1] - 2026-08-07
13
+
14
+ ### Fixed
15
+
16
+ - **Downloaded videos now save to the real Downloads folder.** The post-analysis "Save downloaded video?" prompt hardcoded `join(os.homedir(), "Downloads")` and wrapped `copyFile` in a silent `catch {}`. On machines where the Windows Downloads folder is relocated (e.g. `D:\Downloads`) or OneDrive-redirected, the default path doesn't exist, so `copyFile` threw `ENOENT`, the error was swallowed, and the user saw no file and no message after clicking **Yes**. `vision-proxy.ts` now resolves the real Downloads folder via the Windows Shell known-folder API (`FOLDERID_Downloads` / `shell:Downloads`, which respects relocation and OneDrive redirection), creates it on demand (`mkdir -p`), and surfaces any save failure with the target path. The success notification now includes the full saved path. New helpers: `resolveDownloadsDir`, `pathExists`.
17
+
18
+ ### Added
19
+
20
+ - **yt-dlp auth knobs to defeat YouTube 403s.** YouTube increasingly returns `HTTP Error 403: Forbidden` on the media fetch even with the latest `yt-dlp`; the proxy previously passed no credentials. Two opt-in, persisted settings now forward to yt-dlp: `ytdlpCookiesFromBrowser` (`/multimodal-proxy ytdlp cookies <browser|off>`, validated against `chrome|firefox|edge|brave|opera|safari|vivaldi|chromium|whale`) reuses a logged-in browser session, and `ytdlpExtractorArgs` (`/multimodal-proxy ytdlp extractor-args "<text>|off>`) forwards arbitrary `--extractor-args` (e.g. `youtube:player_client=web_safari,web`). A 403 download failure now appends a hint pointing at the cookies knob. Env overrides: `PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER`, `PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS`. New sanitizers + tests: `sanitizeYtdlpCookiesFromBrowser`, `sanitizeYtdlpExtractorArgs`, `YTDLP_COOKIES_BROWSERS`.
21
+
7
22
  ## [1.11.0] - 2026-07-29
8
23
 
9
24
  ### Added
package/README.md CHANGED
@@ -8,6 +8,11 @@ When **video or audio files** are detected, they are routed to a **multimodal mo
8
8
 
9
9
  **YouTube links** are detected too: paste a URL (`youtube.com/watch?v=…`, `youtu.be/…`, `/shorts/…`, etc.) and the video is downloaded with [`yt-dlp`](https://github.com/yt-dlp/yt-dlp) and analyzed exactly like a local file.
10
10
 
11
+ ## What's new in 1.12.1
12
+
13
+ - **Fix: downloaded videos now save to your real Downloads folder.** The "Save downloaded video?" prompt hardcoded `~/Downloads` and wrapped the copy in a silent `catch`. If you relocated Downloads (e.g. to `D:\Downloads`) or use OneDrive redirection, that path doesn't exist — so clicking **Yes** silently failed with no file and no error. The proxy now resolves the real Downloads folder via the Windows Shell known-folder API (`FOLDERID_Downloads`, respects relocation + OneDrive), creates it if missing, and surfaces any save error with the target path. The success toast now shows the full saved path.
14
+ - **Configurable yt-dlp auth (defeats YouTube 403s).** YouTube increasingly rejects unauthenticated media fetches with `HTTP 403`, even on the latest `yt-dlp`. Two opt-in knobs satisfy the check: `/multimodal-proxy ytdlp cookies <browser>` reuses a logged-in browser session (`chrome`, `firefox`, `edge`, `brave`, …), and `/multimodal-proxy ytdlp extractor-args "<args>"` forwards arbitrary `--extractor-args` (e.g. `youtube:player_client=web_safari,web`). A 403 failure now also hints at the cookies knob. Env overrides: `PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER`, `PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS`.
15
+
11
16
  ## What's new in 1.12.0
12
17
 
13
18
  - **YouTube video download** — paste a YouTube URL in your prompt and the extension downloads it via `yt-dlp` and analyzes it like any local video file. Gated by path detection (on by default); requires `yt-dlp` on your PATH. See [YouTube videos](#youtube-videos). After successful analysis you are prompted to optionally keep the downloaded file in your Downloads folder.
@@ -76,6 +81,16 @@ winget install yt-dlp.yt-dlp # Windows
76
81
 
77
82
  `ffmpeg` is also used (for duration probing and stream merging) — it usually ships alongside yt-dlp. Downloads are capped to ≤720p and rejected past the configured size limit (`PI_VISION_PROXY_MAX_VIDEO_BYTES`, default 200 MB). To disable URL auto-download, turn off path detection: `/multimodal-proxy path-detection off`.
78
83
 
84
+ #### 403 Forbidden / download failures
85
+
86
+ YouTube periodically rejects unauthenticated media fetches with `HTTP Error 403: Forbidden`, even on the latest `yt-dlp`. The proxy can't fix YouTube's backend, but you can satisfy the auth check:
87
+
88
+ - **Reuse a logged-in browser session** — `/multimodal-proxy ytdlp cookies chrome` (also `firefox`, `edge`, `brave`, `opera`, `safari`, `vivaldi`, `chromium`, `whale`). yt-dlp reads that browser's YouTube cookies. This fixes most 403s.
89
+ - **Try alternate player clients** — `/multimodal-proxy ytdlp extractor-args "youtube:player_client=web_safari,web"`.
90
+ - Show current settings with `/multimodal-proxy ytdlp`; turn either off with `... off`.
91
+
92
+ Both are persisted in `~/.pi/agent/multimodal-proxy.json` and can be set via env (`PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER`, `PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS`), which override and lock the command.
93
+
79
94
  ## Modes
80
95
 
81
96
  | Mode | Behavior |
@@ -299,3 +314,4 @@ For the full security audit see [`SECURITY-REVIEW.md`](./SECURITY-REVIEW.md).
299
314
  ## License
300
315
 
301
316
  MIT
317
+
@@ -51,6 +51,9 @@ import {
51
51
  isValidNamedRegion,
52
52
  expandLeadingTilde,
53
53
  sanitizeAllowedFolders,
54
+ sanitizeYtdlpCookiesFromBrowser,
55
+ sanitizeYtdlpExtractorArgs,
56
+ YTDLP_COOKIES_BROWSERS,
54
57
  pathAccessFromConfig,
55
58
  MAX_ALLOWED_FOLDERS,
56
59
  LRUCache,
@@ -265,18 +268,18 @@ describe("readEnvOverrides", () => {
265
268
 
266
269
  describe("envFlags", () => {
267
270
  it("reports presence per variable", () => {
268
- assert.deepEqual(envFlags({}), { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false });
271
+ assert.deepEqual(envFlags({}), { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false, ytdlpCookies: false, ytdlpExtractorArgs: false });
269
272
  assert.deepEqual(
270
273
  envFlags({
271
274
  PI_VISION_PROXY_MODE: "x",
272
275
  PI_VISION_PROXY_MODEL: "y",
273
276
  PI_VISION_PROXY_INCLUDE_CONTEXT: "",
274
277
  }),
275
- { mode: true, model: true, context: true, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false },
278
+ { mode: true, model: true, context: true, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: false, allowedFolders: false, statusLine: false, pathDetection: false, ytdlpCookies: false, ytdlpExtractorArgs: false },
276
279
  );
277
280
  assert.deepEqual(
278
281
  envFlags({ PI_VISION_PROXY_ALLOW_HOME: "1", PI_VISION_PROXY_ALLOWED_FOLDERS: "/a" }),
279
- { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: true, allowedFolders: true, statusLine: false, pathDetection: false },
282
+ { mode: false, model: false, context: false, tool: false, maxImagesPerCall: false, maxBatch: false, cacheSize: false, videoModel: false, allowedProviders: false, allowHome: true, allowedFolders: true, statusLine: false, pathDetection: false, ytdlpCookies: false, ytdlpExtractorArgs: false },
280
283
  );
281
284
  assert.equal(envFlags({ PI_VISION_PROXY_ALLOWED_PROVIDERS: "" }).allowedProviders, true);
282
285
  // An unrecognized ALLOW_HOME value is not an override and must not lock the command
@@ -287,6 +290,32 @@ describe("envFlags", () => {
287
290
  // An invalid value is not applied by readEnvOverrides, so it must not
288
291
  // report (and thereby lock) the setting as env-overridden either.
289
292
  assert.equal(envFlags({ PI_VISION_PROXY_STATUS_LINE: "bogus" }).statusLine, false);
293
+ assert.equal(envFlags({ PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER: "chrome" }).ytdlpCookies, true);
294
+ assert.equal(envFlags({ PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS: "youtube:player_client=web" }).ytdlpExtractorArgs, true);
295
+ });
296
+ });
297
+
298
+ describe("yt-dlp config sanitizers", () => {
299
+ it("accepts known browsers case-insensitively, rejects others", () => {
300
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("Chrome"), "chrome");
301
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("FIREFOX"), "firefox");
302
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("edge"), "edge");
303
+ assert.equal(sanitizeYtdlpCookiesFromBrowser("internet explorer"), "");
304
+ assert.equal(sanitizeYtdlpCookiesFromBrowser(""), "");
305
+ assert.equal(sanitizeYtdlpCookiesFromBrowser(undefined), "");
306
+ assert.equal(sanitizeYtdlpCookiesFromBrowser(123 as any), "");
307
+ });
308
+ it("exposes the supported browser set", () => {
309
+ assert.ok(YTDLP_COOKIES_BROWSERS.has("chrome"));
310
+ assert.ok(YTDLP_COOKIES_BROWSERS.has("brave"));
311
+ });
312
+ it("strips control chars and caps extractor-args length", () => {
313
+ assert.equal(sanitizeYtdlpExtractorArgs("youtube:player_client=web"), "youtube:player_client=web");
314
+ assert.equal(sanitizeYtdlpExtractorArgs(" trim me "), "trim me");
315
+ assert.equal(sanitizeYtdlpExtractorArgs("a\x00b\x07c"), "abc");
316
+ assert.equal(sanitizeYtdlpExtractorArgs(""), "");
317
+ const long = "x".repeat(600);
318
+ assert.equal(sanitizeYtdlpExtractorArgs(long).length, 500);
290
319
  });
291
320
  });
292
321
 
@@ -79,6 +79,12 @@ export interface VisionConfig {
79
79
  // (event.images) are always processed; only the convenience path-scan is
80
80
  // gated. Explicit references via /multimodal-proxy describe still work.
81
81
  pathDetection: ToolSetting;
82
+ // 1.12.1 — yt-dlp tuning to defeat YouTube 403s on the media fetch. Both
83
+ // empty by default. cookiesFromBrowser reuses a logged-in browser session
84
+ // (e.g. "chrome"); extractorArgs forwards arbitrary --extractor-args (e.g.
85
+ // "youtube:player_client=web_safari,web").
86
+ ytdlpCookiesFromBrowser: string;
87
+ ytdlpExtractorArgs: string;
82
88
  }
83
89
 
84
90
  export interface ImageMeta {
@@ -704,6 +710,8 @@ export const DEFAULT_CONFIG: VisionConfig = {
704
710
  "google/gemini-2.5-pro": { format: "gemini_normalized_1000" },
705
711
  "google/gemini-3-pro": { format: "gemini_normalized_1000" },
706
712
  },
713
+ ytdlpCookiesFromBrowser: "",
714
+ ytdlpExtractorArgs: "",
707
715
  };
708
716
 
709
717
  // ── Persistent file storage ────────────────────────────────────────────────
@@ -729,6 +737,8 @@ const PERSISTED_CONFIG_KEYS = new Set([
729
737
  "allowedFolders", "allowHome",
730
738
  "statusLine",
731
739
  "pathDetection",
740
+ "ytdlpCookiesFromBrowser",
741
+ "ytdlpExtractorArgs",
732
742
  ]);
733
743
 
734
744
  /** Read config from the persistent file. Returns empty object on any failure. */
@@ -868,10 +878,15 @@ export function readEnvOverrides(env: NodeJS.ProcessEnv = process.env): Partial<
868
878
  if (foldersEnv !== undefined) {
869
879
  overrides.allowedFolders = sanitizeAllowedFolders(foldersEnv.split(delimiter));
870
880
  }
881
+ // 1.12.1 yt-dlp env overrides
882
+ const cookiesEnv = env.PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER;
883
+ if (cookiesEnv !== undefined) overrides.ytdlpCookiesFromBrowser = sanitizeYtdlpCookiesFromBrowser(cookiesEnv);
884
+ const extractorArgsEnv = env.PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS;
885
+ if (extractorArgsEnv !== undefined) overrides.ytdlpExtractorArgs = sanitizeYtdlpExtractorArgs(extractorArgsEnv);
871
886
  return overrides;
872
887
  }
873
888
 
874
- export function envFlags(env: NodeJS.ProcessEnv = process.env): { mode: boolean; model: boolean; context: boolean; tool: boolean; maxImagesPerCall: boolean; maxBatch: boolean; cacheSize: boolean; videoModel: boolean; allowedProviders: boolean; allowHome: boolean; allowedFolders: boolean; statusLine: boolean; pathDetection: boolean } {
889
+ export function envFlags(env: NodeJS.ProcessEnv = process.env): { mode: boolean; model: boolean; context: boolean; tool: boolean; maxImagesPerCall: boolean; maxBatch: boolean; cacheSize: boolean; videoModel: boolean; allowedProviders: boolean; allowHome: boolean; allowedFolders: boolean; statusLine: boolean; pathDetection: boolean; ytdlpCookies: boolean; ytdlpExtractorArgs: boolean } {
875
890
  return {
876
891
  mode: Boolean(env.PI_VISION_PROXY_MODE),
877
892
  model: Boolean(env.PI_VISION_PROXY_MODEL),
@@ -891,6 +906,8 @@ export function envFlags(env: NodeJS.ProcessEnv = process.env): { mode: boolean;
891
906
  statusLine: env.PI_VISION_PROXY_STATUS_LINE === "on" || env.PI_VISION_PROXY_STATUS_LINE === "off",
892
907
  // Same rule: only a recognized value overrides path detection.
893
908
  pathDetection: env.PI_VISION_PROXY_PATH_DETECTION === "on" || env.PI_VISION_PROXY_PATH_DETECTION === "off",
909
+ ytdlpCookies: env.PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER !== undefined,
910
+ ytdlpExtractorArgs: env.PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS !== undefined,
894
911
  };
895
912
  }
896
913
 
@@ -984,6 +1001,33 @@ export function sanitizeAllowedFolders(value: unknown): string[] {
984
1001
  return out;
985
1002
  }
986
1003
 
1004
+ /** Browsers accepted by yt-dlp's --cookies-from-browser (1.12.1). */
1005
+ export const YTDLP_COOKIES_BROWSERS = new Set([
1006
+ "chrome", "firefox", "edge", "brave", "opera", "safari", "vivaldi", "chromium", "whale",
1007
+ ]);
1008
+
1009
+ /** Max length of the freeform --extractor-args value (1.12.1). */
1010
+ export const YTDLP_EXTRACTOR_ARGS_MAX = 500;
1011
+
1012
+ /**
1013
+ * Validate a yt-dlp --cookies-from-browser value (1.12.1). Lowercases and
1014
+ * accepts only known browser names; anything else collapses to "" (off).
1015
+ */
1016
+ export function sanitizeYtdlpCookiesFromBrowser(value: unknown): string {
1017
+ if (typeof value !== "string") return "";
1018
+ const v = value.trim().toLowerCase();
1019
+ return YTDLP_COOKIES_BROWSERS.has(v) ? v : "";
1020
+ }
1021
+
1022
+ /**
1023
+ * Validate a yt-dlp --extractor-args value (1.12.1). Strips control/null bytes
1024
+ * and caps length; returned verbatim (yt-dlp parses the single arg value).
1025
+ */
1026
+ export function sanitizeYtdlpExtractorArgs(value: unknown): string {
1027
+ if (typeof value !== "string") return "";
1028
+ return value.replace(/[\x00-\x1f\x7f]/g, "").trim().slice(0, YTDLP_EXTRACTOR_ARGS_MAX);
1029
+ }
1030
+
987
1031
  export function sanitize(config: VisionConfig): VisionConfig {
988
1032
  const safe: VisionConfig = { ...config };
989
1033
  if (typeof safe.provider === "string") safe.provider = canonicalProvider(safe.provider);
@@ -1052,6 +1096,9 @@ export function sanitize(config: VisionConfig): VisionConfig {
1052
1096
  if (safe.statusLine !== "on" && safe.statusLine !== "off") safe.statusLine = DEFAULT_CONFIG.statusLine;
1053
1097
  // 1.11.0 path-detection field
1054
1098
  if (safe.pathDetection !== "on" && safe.pathDetection !== "off") safe.pathDetection = DEFAULT_CONFIG.pathDetection;
1099
+ // 1.12.1 yt-dlp tuning fields
1100
+ safe.ytdlpCookiesFromBrowser = sanitizeYtdlpCookiesFromBrowser(safe.ytdlpCookiesFromBrowser);
1101
+ safe.ytdlpExtractorArgs = sanitizeYtdlpExtractorArgs(safe.ytdlpExtractorArgs);
1055
1102
  return safe;
1056
1103
  }
1057
1104
 
@@ -37,23 +37,55 @@
37
37
  * PI_VISION_PROXY_MAX_VIDEO_BYTES - positive integer
38
38
  * PI_VISION_PROXY_ALLOWED_PROVIDERS - comma-separated pre-consented providers
39
39
  * PI_VISION_PROXY_STATUS_LINE - "on" | "off"
40
+ * PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER - chrome|firefox|edge|brave|opera|safari|vivaldi|chromium|whale (defeats YouTube 403s)
41
+ * PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS - e.g. "youtube:player_client=web_safari,web"
40
42
  *
41
43
  * Install:
42
44
  * pi install ./packages/pi-multimodal-proxy
43
45
  */
44
46
 
45
47
  import { execFile } from "node:child_process";
46
- import { copyFile, mkdtemp, readFile, readdir, rm } from "node:fs/promises";
48
+ import { access, copyFile, mkdir, mkdtemp, readFile, readdir, rm } from "node:fs/promises";
47
49
  import os from "node:os";
48
50
  import { isAbsolute, join } from "node:path";
49
51
  import { promisify } from "node:util";
50
- import { type ImageContent as PiAiImage, complete } from "@earendil-works/pi-ai";
52
+ import { type ImageContent as PiAiImage, type Api, type Model, type Context, type ProviderStreamOptions, type AssistantMessage } from "@earendil-works/pi-ai";
53
+
54
+ type LegacyComplete = <TApi extends Api>(model: Model<TApi>, context: Context, options?: ProviderStreamOptions) => Promise<AssistantMessage>;
55
+
56
+ // move `complete` to @earendil-works/pi-ai/compat
57
+ let legacyCompletePromise: Promise<LegacyComplete> | undefined;
58
+ function loadLegacyComplete(): Promise<LegacyComplete> {
59
+ if (!legacyCompletePromise) {
60
+ const compatSpecifier: string = "@earendil-works/pi-ai/compat";
61
+ legacyCompletePromise = import("@earendil-works/pi-ai").then((mod: any) =>
62
+ typeof mod.complete === "function"
63
+ ? mod.complete
64
+ : import(compatSpecifier).then((compat: any) => compat.complete),
65
+ );
66
+ }
67
+ return legacyCompletePromise;
68
+ }
69
+
70
+ /** Compatibility wrapper: uses new ModelRegistry.complete when available, otherwise falls back to legacyComplete. */
71
+ function hasComplete(mr: ModelRegistry): mr is ModelRegistry & { complete: LegacyComplete } {
72
+ return typeof (mr as any).complete === "function";
73
+ }
74
+
75
+ async function completeCompat<TApi extends Api>(ctx: ExtensionContext, model: Model<TApi>, request: Context, options?: ProviderStreamOptions) {
76
+ if (hasComplete(ctx.modelRegistry)) {
77
+ return ctx.modelRegistry.complete(model, request, options);
78
+ }
79
+ const legacyComplete = await loadLegacyComplete();
80
+ return legacyComplete(model, request, options);
81
+ }
82
+
51
83
  import type {
52
84
  BeforeAgentStartEvent,
53
85
  BeforeAgentStartEventResult,
54
86
  ContextEvent,
55
87
  ExtensionAPI,
56
- ExtensionContext,
88
+ ExtensionContext, ModelRegistry,
57
89
  SessionCompactEvent,
58
90
  SessionEntry,
59
91
  SessionStartEvent,
@@ -128,6 +160,8 @@ import {
128
160
  expandLeadingTilde,
129
161
  isUncPath,
130
162
  MAX_ALLOWED_FOLDERS,
163
+ sanitizeYtdlpExtractorArgs,
164
+ YTDLP_COOKIES_BROWSERS,
131
165
 
132
166
  readMediaFileWithReason,
133
167
  type ReadMediaReason,
@@ -613,7 +647,9 @@ async function analyzeImages(
613
647
  config: VisionConfig,
614
648
  ctx: ExtensionContext,
615
649
  ): Promise<AnalysisResult[] | null> {
650
+
616
651
  const visionModel = ctx.modelRegistry.find(config.provider, config.modelId);
652
+
617
653
  if (!visionModel) {
618
654
  ctx.ui.notify(
619
655
  `[multimodal-proxy] Model "${modelLabel(config)}" not found. Use /multimodal-proxy pick to choose one.`,
@@ -663,7 +699,7 @@ async function analyzeImages(
663
699
  storeImageData(imageData, hash, piAiImage.data, piAiImage.mimeType);
664
700
 
665
701
  try {
666
- const response = await complete(
702
+ const response = await completeCompat(ctx,
667
703
  visionModel,
668
704
  {
669
705
  systemPrompt: config.systemPrompt,
@@ -786,7 +822,7 @@ async function analyzeVideo(
786
822
  : "";
787
823
 
788
824
  try {
789
- const response = await complete(
825
+ const response = await completeCompat(ctx,
790
826
  videoModel,
791
827
  {
792
828
  systemPrompt: config.videoSystemPrompt,
@@ -975,6 +1011,7 @@ async function downloadMediaUrl(
975
1011
  url: string,
976
1012
  signal: AbortSignal | undefined,
977
1013
  ctx: ExtensionContext,
1014
+ ytdlp?: { cookiesFromBrowser?: string; extractorArgs?: string },
978
1015
  ): Promise<DownloadedMedia | null> {
979
1016
  if (!(await checkYtDlp())) {
980
1017
  ctx.ui.notify(
@@ -991,19 +1028,27 @@ async function downloadMediaUrl(
991
1028
  // actual download — it can satisfy the print from metadata alone. Adding
992
1029
  // an after_move field forces the download (it is only known once the file
993
1030
  // is in its final location) and also hands us the exact output path.
1031
+ const args = [
1032
+ "--no-playlist",
1033
+ "--no-warnings",
1034
+ "--no-progress",
1035
+ "-f", "best[ext=mp4][height<=720]/best[height<=720]/best",
1036
+ "--merge-output-format", "mp4",
1037
+ "-o", join(tempDir, "%(id)s.%(ext)s"),
1038
+ ];
1039
+ // Optional auth / player-client knobs (1.12.1) — defeat YouTube 403s on
1040
+ // the media fetch by reusing a logged-in browser session and/or alternate
1041
+ // player clients. Both opt-in; absent by default.
1042
+ if (ytdlp?.cookiesFromBrowser) {
1043
+ args.push("--cookies-from-browser", ytdlp.cookiesFromBrowser);
1044
+ }
1045
+ if (ytdlp?.extractorArgs) {
1046
+ args.push("--extractor-args", ytdlp.extractorArgs);
1047
+ }
1048
+ args.push("--print", "%(title)s", "--print", "after_move:filepath", url);
994
1049
  const { stdout } = await execFileAsync(
995
1050
  "yt-dlp",
996
- [
997
- "--no-playlist",
998
- "--no-warnings",
999
- "--no-progress",
1000
- "-f", "best[ext=mp4][height<=720]/best[height<=720]/best",
1001
- "--merge-output-format", "mp4",
1002
- "-o", join(tempDir, "%(id)s.%(ext)s"),
1003
- "--print", "%(title)s",
1004
- "--print", "after_move:filepath",
1005
- url,
1006
- ],
1051
+ args,
1007
1052
  { windowsHide: true, timeout: 240_000, maxBuffer: 4 * 1024 * 1024, signal },
1008
1053
  );
1009
1054
 
@@ -1027,11 +1072,77 @@ async function downloadMediaUrl(
1027
1072
  } catch (err) {
1028
1073
  if (signal?.aborted) return null;
1029
1074
  const msg = err instanceof Error ? err.message : String(err);
1030
- ctx.ui.notify(`[multimodal-proxy] YouTube download failed for ${url}: ${msg}`, "warning");
1075
+ // A 403 on the media fetch usually means YouTube wants a logged-in
1076
+ // session; point the user at the cookies knob if it isn't already set.
1077
+ const hint = /403|forbidden/i.test(msg) && !ytdlp?.cookiesFromBrowser
1078
+ ? " (a 403 often means YouTube wants a logged-in session — try /multimodal-proxy ytdlp cookies <browser>)"
1079
+ : "";
1080
+ ctx.ui.notify(`[multimodal-proxy] YouTube download failed for ${url}: ${msg}${hint}`, "warning");
1031
1081
  return null;
1032
1082
  }
1033
1083
  }
1034
1084
 
1085
+ // ── Downloads folder resolution ─────────────────────────────────────────────
1086
+ //
1087
+ // The Windows default `C:\Users\<user>\Downloads` is frequently NOT the folder
1088
+ // File Explorer shows: OneDrive redirection or a custom relocation (e.g.
1089
+ // `D:\Downloads`) moves it elsewhere. Hardcoding the default then makes
1090
+ // `copyFile` throw ENOENT — and pre-fix that error was swallowed silently, so
1091
+ // the user clicked "Save", nothing appeared, and no error was surfaced.
1092
+ //
1093
+ // We resolve the real folder via the Shell known-folder (FOLDERID_Downloads)
1094
+ // on Windows, cache the result, and fall back to `~/Downloads` elsewhere.
1095
+
1096
+ let downloadsDirCache: string | null = null;
1097
+
1098
+ async function pathExists(p: string): Promise<boolean> {
1099
+ try {
1100
+ await access(p);
1101
+ return true;
1102
+ } catch {
1103
+ return false;
1104
+ }
1105
+ }
1106
+
1107
+ /**
1108
+ * Resolve the user's real Downloads folder, respecting Windows relocation /
1109
+ * OneDrive redirection. Falls back to the default `~/Downloads` (which the
1110
+ * caller creates on demand). Result is cached for the process lifetime.
1111
+ */
1112
+ async function resolveDownloadsDir(): Promise<string> {
1113
+ if (downloadsDirCache) return downloadsDirCache;
1114
+ const defaultDir = join(os.homedir(), "Downloads");
1115
+ if (await pathExists(defaultDir)) {
1116
+ downloadsDirCache = defaultDir;
1117
+ return defaultDir;
1118
+ }
1119
+ // Default is missing (relocated / OneDrive-redirected). On Windows, ask the
1120
+ // Shell for the real Downloads known-folder path; this respects redirection.
1121
+ if (process.platform === "win32") {
1122
+ try {
1123
+ const { stdout } = await execFileAsync(
1124
+ "powershell.exe",
1125
+ [
1126
+ "-NoProfile",
1127
+ "-NonInteractive",
1128
+ "-Command",
1129
+ "(New-Object -ComObject Shell.Application).NameSpace('shell:Downloads').Self.Path",
1130
+ ],
1131
+ { windowsHide: true, timeout: 10_000 },
1132
+ );
1133
+ const resolved = stdout.split(/\r?\n/).map((l) => l.trim()).find((l) => l.length > 0) ?? "";
1134
+ if (resolved && (await pathExists(resolved))) {
1135
+ downloadsDirCache = resolved;
1136
+ return resolved;
1137
+ }
1138
+ } catch {
1139
+ // PowerShell unavailable or failed — fall back to default (created below).
1140
+ }
1141
+ }
1142
+ downloadsDirCache = defaultDir;
1143
+ return defaultDir;
1144
+ }
1145
+
1035
1146
  async function analyzeVideoViaXaiStt(
1036
1147
  mediaFile: { type: "image"; data: string; mimeType: string },
1037
1148
  filename: string,
@@ -1384,7 +1495,7 @@ async function handleAnalyzeImage(
1384
1495
 
1385
1496
  try {
1386
1497
  const startTime = Date.now();
1387
- const response = await complete(
1498
+ const response = await completeCompat(ctx,
1388
1499
  visionModel,
1389
1500
  {
1390
1501
  systemPrompt,
@@ -1693,7 +1804,10 @@ export default function (pi: ExtensionAPI) {
1693
1804
  ctx,
1694
1805
  () => `Downloading YouTube ${id}…`,
1695
1806
  `Downloading YouTube ${id}…`,
1696
- () => downloadMediaUrl(watch, ctx.signal, ctx),
1807
+ () => downloadMediaUrl(watch, ctx.signal, ctx, {
1808
+ cookiesFromBrowser: config.ytdlpCookiesFromBrowser,
1809
+ extractorArgs: config.ytdlpExtractorArgs,
1810
+ }),
1697
1811
  );
1698
1812
  if (!dl) continue;
1699
1813
  downloadedTempDirs.push(dl.tempDir);
@@ -1813,15 +1927,23 @@ export default function (pi: ExtensionAPI) {
1813
1927
  "Save downloaded video?",
1814
1928
  `"${info.title}"\n\n${sizeMB} MB — save to your Downloads folder?`,
1815
1929
  );
1816
- if (save) {
1817
- const safeTitle = info.title.replace(/[<>:"\/\\|?*]/g, "_").replace(/\s+/g, " ").trim().slice(0, 200);
1818
- const dest = join(os.homedir(), "Downloads", `${safeTitle}.mp4`);
1930
+ if (!save) continue;
1931
+ const safeTitle = info.title.replace(/[<>:"\/\\|?*]/g, "_").replace(/\s+/g, " ").trim().slice(0, 200);
1932
+ const destDir = await resolveDownloadsDir();
1933
+ const dest = join(destDir, `${safeTitle}.mp4`);
1934
+ try {
1935
+ await mkdir(destDir, { recursive: true });
1819
1936
  await copyFile(path, dest);
1820
- ctx.ui.notify(`[multimodal-proxy] ✓ Saved "${safeTitle}.mp4" to Downloads`, "info");
1937
+ ctx.ui.notify(`[multimodal-proxy] ✓ Saved "${safeTitle}.mp4" to ${dest}`, "info");
1938
+ } catch (saveErr) {
1939
+ // Surface save failures (missing dir, permissions, disk) — the
1940
+ // previous silent catch hid them and the video just vanished.
1941
+ const reason = saveErr instanceof Error ? saveErr.message : String(saveErr);
1942
+ ctx.ui.notify(`[multimodal-proxy] ✗ Could not save video to ${dest}: ${reason}`, "warning");
1821
1943
  }
1822
1944
  } catch {
1823
- // Don't block the agent on save errors — temp files are
1824
- // cleaned below regardless.
1945
+ // confirm() dialog itself failed — don't block the agent. Temp
1946
+ // files are cleaned below regardless.
1825
1947
  }
1826
1948
  }
1827
1949
  await cleanupDownloads();
@@ -1939,7 +2061,7 @@ export default function (pi: ExtensionAPI) {
1939
2061
  ...jointImages,
1940
2062
  ];
1941
2063
 
1942
- const jointResponse = await complete(
2064
+ const jointResponse = await completeCompat(ctx,
1943
2065
  jointVisionModel,
1944
2066
  {
1945
2067
  systemPrompt: jointSystemPrompt,
@@ -2522,6 +2644,73 @@ Use "*" or "all" to grant consent for all providers globally.`,
2522
2644
  return;
2523
2645
  }
2524
2646
 
2647
+ // ── yt-dlp cookies / extractor-args (defeat YouTube 403s) ───────
2648
+ if (sub === "ytdlp") {
2649
+ const { sub: ySub, value: yValue } = splitSubcommand(value);
2650
+
2651
+ if (ySub === "cookies") {
2652
+ if (env.ytdlpCookies) {
2653
+ ctx.ui.notify(
2654
+ "[multimodal-proxy] PI_VISION_PROXY_YTDLP_COOKIES_FROM_BROWSER is set - env overrides commands. Unset to change.",
2655
+ "warning",
2656
+ );
2657
+ return;
2658
+ }
2659
+ const trimmed = yValue.trim().toLowerCase();
2660
+ if (!trimmed || trimmed === "off") {
2661
+ writePersisted({ ...persisted, ytdlpCookiesFromBrowser: "" });
2662
+ ctx.ui.notify("[multimodal-proxy] yt-dlp cookies-from-browser: off", "info");
2663
+ return;
2664
+ }
2665
+ if (!YTDLP_COOKIES_BROWSERS.has(trimmed)) {
2666
+ ctx.ui.notify(
2667
+ `[multimodal-proxy] Unknown browser "${trimmed}". Supported: ${[...YTDLP_COOKIES_BROWSERS].join(", ")}`,
2668
+ "warning",
2669
+ );
2670
+ return;
2671
+ }
2672
+ writePersisted({ ...persisted, ytdlpCookiesFromBrowser: trimmed });
2673
+ ctx.ui.notify(
2674
+ `[multimodal-proxy] yt-dlp cookies-from-browser: ${trimmed} (applied on next YouTube download)`,
2675
+ "info",
2676
+ );
2677
+ return;
2678
+ }
2679
+
2680
+ if (ySub === "extractor-args") {
2681
+ if (env.ytdlpExtractorArgs) {
2682
+ ctx.ui.notify(
2683
+ "[multimodal-proxy] PI_VISION_PROXY_YTDLP_EXTRACTOR_ARGS is set - env overrides commands. Unset to change.",
2684
+ "warning",
2685
+ );
2686
+ return;
2687
+ }
2688
+ const trimmed = yValue.trim();
2689
+ if (!trimmed || trimmed.toLowerCase() === "off") {
2690
+ writePersisted({ ...persisted, ytdlpExtractorArgs: "" });
2691
+ ctx.ui.notify("[multimodal-proxy] yt-dlp extractor-args: off", "info");
2692
+ return;
2693
+ }
2694
+ const cleaned = sanitizeYtdlpExtractorArgs(trimmed);
2695
+ writePersisted({ ...persisted, ytdlpExtractorArgs: cleaned });
2696
+ ctx.ui.notify(`[multimodal-proxy] yt-dlp extractor-args: ${cleaned}`, "info");
2697
+ return;
2698
+ }
2699
+
2700
+ ctx.ui.notify(
2701
+ "[multimodal-proxy] yt-dlp options:\n" +
2702
+ ` cookies-from-browser: ${effective.ytdlpCookiesFromBrowser || "(off)"}\n` +
2703
+ ` extractor-args: ${effective.ytdlpExtractorArgs || "(off)"}\n` +
2704
+ (env.ytdlpCookies || env.ytdlpExtractorArgs ? " (env override active)\n" : "") +
2705
+ "Usage:\n" +
2706
+ " /multimodal-proxy ytdlp cookies <browser|off> reuse a logged-in YouTube session (fixes most 403s)\n" +
2707
+ ' /multimodal-proxy ytdlp extractor-args <text|off> e.g. youtube:player_client=web_safari,web\n' +
2708
+ `Browsers: ${[...YTDLP_COOKIES_BROWSERS].join(", ")}`,
2709
+ "info",
2710
+ );
2711
+ return;
2712
+ }
2713
+
2525
2714
  // ── max-images-per-call ────────────────────────────
2526
2715
  if (sub === "max-images-per-call") {
2527
2716
  if (env.maxImagesPerCall) {
@@ -2963,7 +3152,7 @@ Use "*" or "all" to grant consent for all providers globally.`,
2963
3152
 
2964
3153
  try {
2965
3154
  const startTime = Date.now();
2966
- const response = await complete(
3155
+ const response = await completeCompat(ctx,
2967
3156
  descVisionModel,
2968
3157
  {
2969
3158
  systemPrompt,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-multimodal-proxy",
3
- "version": "1.12.0",
3
+ "version": "1.13.0",
4
4
  "description": "Automatic image, video and audio description for any model in Pi. Routes media to a multimodal model and injects descriptions into context.",
5
5
  "keywords": [
6
6
  "pi-package"