open-agents-ai 0.19.0 → 0.20.0

package/README.md

@@ -121,7 +121,11 @@ All proposals are indexed in `.oa/dreams/PROPOSAL-INDEX.md` for easy review.
 
 ## Listen Mode — Live Bidirectional Audio
 
-Listen mode enables real-time voice communication with the agent. Your microphone audio is captured, streamed through Whisper (via `transcribe-cli`), and the transcription is injected directly into the input line — creating a hands-free coding workflow.
+Listen mode enables real-time voice communication with the agent. Your microphone audio is captured, streamed through Whisper, and the transcription is injected directly into the input line — creating a hands-free coding workflow.
+
+Two transcription backends ensure broad platform support:
+- **transcribe-cli** (faster-whisper / ONNX) — used by default, fastest on x86
+- **openai-whisper** (Python venv) — automatic fallback for ARM, linux-arm64, or when ONNX is unavailable. Auto-creates a venv and installs deps on first use.
 
 ```bash
 /listen             # Toggle microphone capture on/off
@@ -143,10 +147,11 @@ Listen mode enables real-time voice communication with the agent. Your microphon
 When combined with `/voice`, you get full bidirectional audio — speak your tasks, hear the agent's progress through TTS, and speak corrections mid-task. The status bar shows a blinking red `● REC` indicator with a countdown timer during auto-mode recording.
 
 **Platform support:**
-- **Linux**: `arecord` (ALSA) or `ffmpeg` (PulseAudio)
+- **Linux x86**: `arecord` (ALSA) or `ffmpeg` (PulseAudio) + transcribe-cli
+- **Linux ARM**: `arecord` or `ffmpeg` + openai-whisper (auto-installed in Python venv)
 - **macOS**: `sox` (CoreAudio) or `ffmpeg` (AVFoundation)
 
-The `transcribe-cli` dependency auto-installs in the background on first use.
+The `transcribe-cli` dependency auto-installs in the background on first use. On ARM or when transcribe-cli fails, the system automatically falls back to `openai-whisper` via a self-managed Python venv (same approach used by Moondream vision).
 
 **File transcription**: Drag-and-drop audio/video files (`.mp3`, `.wav`, `.mp4`, `.mkv`, etc.) onto the terminal to transcribe them. Results are saved to `.oa/transcripts/`.
 

package/dist/index.js

@@ -5407,8 +5407,8 @@ async function loadTranscribeCli() {
   const nvmBase = join13(homedir5(), ".nvm", "versions", "node");
   if (existsSync10(nvmBase)) {
     try {
-      const { readdirSync: readdirSync9 } = await import("node:fs");
-      for (const ver of readdirSync9(nvmBase)) {
+      const { readdirSync: readdirSync10 } = await import("node:fs");
+      for (const ver of readdirSync10(nvmBase)) {
         const tcPath = join13(nvmBase, ver, "lib", "node_modules", "transcribe-cli");
         if (existsSync10(join13(tcPath, "dist", "index.js"))) {
           const { createRequire: createRequire4 } = await import("node:module");
@@ -9857,7 +9857,19 @@ var init_agenticRunner = __esm({
   "packages/orchestrator/dist/agenticRunner.js"() {
     "use strict";
     init_dist();
-    SYSTEM_PROMPT = `You are Open Agent, an expert coding assistant that works iteratively using tools. You solve coding tasks by exploring code, making precise changes, and validating results through test execution.
+    SYSTEM_PROMPT = `You are Open Agent, an autonomous AI agent with full access to the local machine. You can read/write files, execute shell commands, browse the web, control the desktop (open applications, click UI elements, take screenshots), and interact with any software on the system. You solve tasks by using your tools iteratively until complete.
+
+## NEVER REFUSE \u2014 ALWAYS ATTEMPT
+
+You have a comprehensive set of tools. NEVER say "I can't do that" or "I don't have the ability to...". Instead, ALWAYS attempt the task using your tools:
+- Need to open Firefox? Use shell: \`firefox https://example.com &\`
+- Need to click a button? Use desktop_click or shell with xdotool
+- Need to see the screen? Use screenshot or desktop_describe
+- Need to type text? Use shell with xdotool: \`xdotool type "text"\`
+- Need to install software? Use shell: \`sudo apt install ...\`
+- Need to interact with a website? Use web_fetch, or open the browser and use desktop tools
+
+If a tool fails, try a different approach. If you're unsure, explore with your tools first. Do NOT give a text-only response when tools could accomplish the task.
 
 ## Available Tools
 
@@ -9909,14 +9921,27 @@ Use background_run for long-running commands (builds, test suites) so you can co
 Use sub_agent to parallelize independent sub-tasks or explore different approaches simultaneously.
 Check task_status periodically and read task_output when tasks complete.
 
-## Image & Visual Context
+## Desktop Automation & Vision
 
-- image_read: Read an image file (returns base64, dimensions, OCR text)
+- desktop_click: Click a UI element by natural language description. Takes a screenshot, finds the element with vision, clicks it. Example: desktop_click({target: "the Save button"})
+- desktop_describe: Take a screenshot and describe what's on screen (or ask a question about it). Use this to "see" the desktop.
+- vision: Analyze any image with Moondream VLM \u2014 caption, query, detect objects, find click targets
 - screenshot: Capture the screen or active window
+- image_read: Read an image file (returns base64, dimensions, OCR text)
 - ocr: Extract text from an image using OCR (supports region cropping/zoom)
 
-When you receive image context from the user, use ocr to extract text or describe what you see.
-Use screenshot to capture visual state for debugging UI issues.
+### Desktop Interaction Workflow
+
+When asked to interact with desktop applications (open browsers, click buttons, fill forms, etc.):
+1. Use shell to launch applications: \`firefox https://example.com &\`
+2. Use screenshot or desktop_describe to see what's on screen
+3. Use desktop_click to click UI elements: \`desktop_click({target: "Sign Up button"})\`
+4. Use shell with xdotool for keyboard input: \`xdotool type "username"\` and \`xdotool key Return\`
+5. Use shell with xdotool for navigation: \`xdotool key Tab\`, \`xdotool key ctrl+l\`
+6. Take screenshots between steps to verify progress
+
+You CAN open Firefox, Chrome, or any application. You CAN click buttons, fill forms, and navigate websites.
+You CAN use xdotool for keyboard/mouse control. These are real capabilities, not hypothetical.
 
 ### Self-Guided Image Exploration
 
@@ -11712,10 +11737,12 @@ var init_dist5 = __esm({
 
 // packages/cli/dist/tui/listen.js
 import { spawn as spawn7, execSync as execSync12 } from "node:child_process";
-import { existsSync as existsSync13, mkdirSync as mkdirSync5, writeFileSync as writeFileSync5 } from "node:fs";
-import { join as join19 } from "node:path";
+import { existsSync as existsSync13, mkdirSync as mkdirSync5, writeFileSync as writeFileSync5, readdirSync as readdirSync6 } from "node:fs";
+import { join as join19, dirname as dirname6 } from "node:path";
 import { homedir as homedir6 } from "node:os";
+import { fileURLToPath as fileURLToPath3 } from "node:url";
 import { EventEmitter } from "node:events";
+import { createInterface as createInterface2 } from "node:readline";
 function isAudioPath(path) {
   const ext = path.toLowerCase().split(".").pop();
   return ext ? AUDIO_EXTENSIONS.has(`.${ext}`) : false;
@@ -11794,6 +11821,49 @@ function findMicCaptureCommand() {
   }
   return null;
 }
+function findLiveWhisperScript() {
+  const thisDir = dirname6(fileURLToPath3(import.meta.url));
+  const candidates = [
+    join19(thisDir, "../../../../packages/execution/scripts/live-whisper.py"),
+    join19(thisDir, "../../../packages/execution/scripts/live-whisper.py"),
+    join19(thisDir, "../../execution/scripts/live-whisper.py"),
+    // npm install layout — scripts bundled alongside dist
+    join19(thisDir, "../scripts/live-whisper.py"),
+    join19(thisDir, "../../scripts/live-whisper.py")
+  ];
+  for (const p of candidates) {
+    if (existsSync13(p))
+      return p;
+  }
+  try {
+    const globalRoot = execSync12("npm root -g", {
+      encoding: "utf-8",
+      timeout: 5e3,
+      stdio: ["pipe", "pipe", "pipe"]
+    }).trim();
+    const candidates2 = [
+      join19(globalRoot, "open-agents-ai", "dist", "scripts", "live-whisper.py"),
+      join19(globalRoot, "open-agents-ai", "scripts", "live-whisper.py")
+    ];
+    for (const p of candidates2) {
+      if (existsSync13(p))
+        return p;
+    }
+  } catch {
+  }
+  const nvmBase = join19(homedir6(), ".nvm", "versions", "node");
+  if (existsSync13(nvmBase)) {
+    try {
+      for (const ver of readdirSync6(nvmBase)) {
+        const p = join19(nvmBase, ver, "lib", "node_modules", "open-agents-ai", "dist", "scripts", "live-whisper.py");
+        if (existsSync13(p))
+          return p;
+      }
+    } catch {
+    }
+  }
+  return null;
+}
 function ensureTranscribeCliBackground() {
   if (_bgInstallPromise)
     return;
@@ -11827,7 +11897,7 @@ function getListenEngine(config) {
   }
   return _engine;
 }
-var AUDIO_EXTENSIONS, VIDEO_EXTENSIONS, ListenEngine, _bgInstallPromise, _engine;
+var AUDIO_EXTENSIONS, VIDEO_EXTENSIONS, WhisperFallbackTranscriber, ListenEngine, _bgInstallPromise, _engine;
 var init_listen = __esm({
   "packages/cli/dist/tui/listen.js"() {
     "use strict";
@@ -11852,11 +11922,104 @@ var init_listen = __esm({
       ".m4v",
       ".ts"
     ]);
+    WhisperFallbackTranscriber = class extends EventEmitter {
+      model;
+      scriptPath;
+      process = null;
+      _ready = false;
+      constructor(model, scriptPath2) {
+        super();
+        this.model = model;
+        this.scriptPath = scriptPath2;
+      }
+      get ready() {
+        return this._ready;
+      }
+      async start() {
+        return new Promise((resolve19, reject) => {
+          const timeout = setTimeout(() => {
+            reject(new Error("Whisper fallback: model load timeout (5 min). First run downloads the model."));
+          }, 3e5);
+          this.process = spawn7("python3", [
+            this.scriptPath,
+            "--model",
+            this.model,
+            "--chunk-seconds",
+            "3",
+            "--window-seconds",
+            "10"
+          ], {
+            stdio: ["pipe", "pipe", "pipe"],
+            env: { ...process.env }
+          });
+          const rl = createInterface2({ input: this.process.stdout });
+          rl.on("line", (line) => {
+            try {
+              const evt = JSON.parse(line);
+              switch (evt.type) {
+                case "status":
+                  this.emit("status", evt.message);
+                  break;
+                case "ready":
+                  this._ready = true;
+                  clearTimeout(timeout);
+                  this.emit("ready");
+                  resolve19();
+                  break;
+                case "transcript":
+                  this.emit("transcript", {
+                    text: evt.text,
+                    isFinal: evt.isFinal ?? false
+                  });
+                  break;
+                case "error":
+                  this.emit("error", new Error(evt.message));
+                  break;
+              }
+            } catch {
+            }
+          });
+          this.process.stderr?.on("data", (data) => {
+            const text = data.toString().trim();
+            if (text)
+              this.emit("status", text);
+          });
+          this.process.on("error", (err) => {
+            clearTimeout(timeout);
+            reject(err);
+          });
+          this.process.on("close", (code) => {
+            if (!this._ready) {
+              clearTimeout(timeout);
+              reject(new Error(`Whisper worker exited with code ${code} before ready`));
+            }
+          });
+        });
+      }
+      /** Write PCM16 audio data to the worker's stdin. */
+      write(chunk) {
+        if (this.process?.stdin?.writable) {
+          this.process.stdin.write(chunk);
+        }
+      }
+      /** Stop the worker. */
+      stop() {
+        if (this.process) {
+          try {
+            this.process.stdin?.end();
+            this.process.kill("SIGTERM");
+          } catch {
+          }
+          this.process = null;
+        }
+        this._ready = false;
+      }
+    };
     ListenEngine = class extends EventEmitter {
       config;
       micProcess = null;
       liveTranscriber = null;
-      // TranscribeLive from transcribe-cli
+      // TranscribeLive from transcribe-cli or WhisperFallbackTranscriber
       active = false;
       silenceTimer = null;
       countdownInterval = null;
@@ -11939,8 +12102,8 @@ var init_listen = __esm({
         const nvmBase = join19(homedir6(), ".nvm", "versions", "node");
         if (existsSync13(nvmBase)) {
           try {
-            const { readdirSync: readdirSync9 } = await import("node:fs");
-            for (const ver of readdirSync9(nvmBase)) {
+            const { readdirSync: readdirSync10 } = await import("node:fs");
+            for (const ver of readdirSync10(nvmBase)) {
               const tcPath = join19(nvmBase, ver, "lib", "node_modules", "transcribe-cli");
               if (existsSync13(join19(tcPath, "dist", "index.js"))) {
                 const { createRequire: createRequire4 } = await import("node:module");
@@ -11972,7 +12135,6 @@ var init_listen = __esm({
             tc = await this.loadTranscribeCli();
           }
           if (!tc) {
-            this.emit("info", "Installing transcribe-cli...");
             try {
               execSync12("npm i -g transcribe-cli", { stdio: "pipe", timeout: 18e4 });
               this.transcribeCliAvailable = null;
@@ -11980,62 +12142,85 @@ var init_listen = __esm({
             } catch {
             }
           }
-          if (!tc) {
-            return "Failed to install transcribe-cli. Try manually: npm i -g transcribe-cli";
-          }
         }
-        const TranscribeLive = tc.TranscribeLive;
-        if (!TranscribeLive) {
-          return "transcribe-cli does not export TranscribeLive. Try updating: npm i -g transcribe-cli@latest";
-        }
-        try {
-          this.liveTranscriber = new TranscribeLive({
-            model: this.config.model,
-            sampleRate: 16e3,
-            channels: 1,
-            sampleWidth: 2,
-            chunkDuration: 3
-            // 3s chunks for responsive transcription
-          });
-        } catch (err) {
-          const arch = process.arch;
-          const armHint = arch === "arm64" || arch === "arm" ? ` Live transcription may not be supported on ${process.platform}-${arch}.` : "";
-          return `Failed to create live transcriber.${armHint} Error: ${err instanceof Error ? err.message : String(err)}`;
+        let usedFallback = false;
+        let transcribeCliError = null;
+        if (tc) {
+          const TranscribeLive = tc.TranscribeLive;
+          if (TranscribeLive) {
+            try {
+              this.liveTranscriber = new TranscribeLive({
+                model: this.config.model,
+                sampleRate: 16e3,
+                channels: 1,
+                sampleWidth: 2,
+                chunkDuration: 3
+              });
+              this.liveTranscriber.on("transcript", (evt) => {
+                if (!evt.text.trim())
+                  return;
+                this.lastTranscriptTime = Date.now();
+                this.pendingText = evt.text.trim();
+                this.emit("transcript", this.pendingText, evt.isFinal);
+                if (this.config.mode === "auto")
+                  this.resetSilenceTimer();
+              });
+              this.liveTranscriber.on("error", (err) => {
+                this.emit("error", err);
+              });
+              await new Promise((resolve19, reject) => {
+                const timeout = setTimeout(() => reject(new Error("Model load timeout (60s)")), 6e4);
+                this.liveTranscriber.on("ready", () => {
+                  clearTimeout(timeout);
+                  resolve19();
+                });
+                this.liveTranscriber.on("error", (err) => {
+                  clearTimeout(timeout);
+                  reject(err);
+                });
+              });
+            } catch (err) {
+              try {
+                this.liveTranscriber?.stop();
+              } catch {
+              }
+              this.liveTranscriber = null;
+              transcribeCliError = err instanceof Error ? err.message : String(err);
+            }
+          }
         }
-        this.liveTranscriber.on("transcript", (evt) => {
-          if (!evt.text.trim())
-            return;
-          this.lastTranscriptTime = Date.now();
-          this.pendingText = evt.text.trim();
-          this.emit("transcript", this.pendingText, evt.isFinal);
-          if (this.config.mode === "auto") {
-            this.resetSilenceTimer();
+        if (!this.liveTranscriber) {
+          const scriptPath2 = findLiveWhisperScript();
+          if (!scriptPath2) {
+            const hint = transcribeCliError ? ` transcribe-cli error: ${transcribeCliError}` : "";
+            return `No transcription backend available.${hint} live-whisper.py not found.`;
           }
-        });
-        this.liveTranscriber.on("error", (err) => {
-          this.emit("error", err);
-        });
-        try {
-          await new Promise((resolve19, reject) => {
-            const timeout = setTimeout(() => reject(new Error("Model load timeout (60s)")), 6e4);
-            this.liveTranscriber.on("ready", () => {
-              clearTimeout(timeout);
-              resolve19();
+          try {
+            const fallback = new WhisperFallbackTranscriber(this.config.model, scriptPath2);
+            usedFallback = true;
+            fallback.on("status", (msg) => {
+              this.emit("info", msg);
             });
-            this.liveTranscriber.on("error", (err) => {
-              clearTimeout(timeout);
-              reject(err);
+            await fallback.start();
+            fallback.on("transcript", (evt) => {
+              if (!evt.text.trim())
+                return;
+              this.lastTranscriptTime = Date.now();
+              this.pendingText = evt.text.trim();
+              this.emit("transcript", this.pendingText, evt.isFinal);
+              if (this.config.mode === "auto")
+                this.resetSilenceTimer();
             });
-          });
-        } catch (err) {
-          try {
-            this.liveTranscriber.stop();
-          } catch {
+            fallback.on("error", (err) => {
+              this.emit("error", err);
+            });
+            this.liveTranscriber = fallback;
+          } catch (err) {
+            const msg = err instanceof Error ? err.message : String(err);
+            const tcHint = transcribeCliError ? `
+transcribe-cli error: ${transcribeCliError}` : "";
+            return `Failed to start live transcription: ${msg}${tcHint}`;
           }
-          this.liveTranscriber = null;
-          const arch = process.arch;
-          const armHint = arch === "arm64" || arch === "arm" ? ` The whisper model or phonemizer WASM may not support ${process.platform}-${arch}. File transcription (/transcribe <file>) may still work.` : "";
-          return `Failed to start live transcription: ${err instanceof Error ? err.message : String(err)}${armHint}`;
         }
         this.micProcess = spawn7(micCmd.cmd, micCmd.args, {
           stdio: ["pipe", "pipe", "pipe"],
@@ -12067,7 +12252,8 @@ var init_listen = __esm({
         this.emit("started");
         this.emit("recording", true);
         const modeDesc = this.config.mode === "auto" ? `auto (${this.config.silenceTimeoutMs / 1e3}s timeout)` : "confirm (press Enter to submit)";
-        return `Listening with ${this.config.model} model (${modeDesc})`;
+        const backend = usedFallback ? "openai-whisper" : "transcribe-cli";
+        return `Listening with ${this.config.model} model via ${backend} (${modeDesc})`;
       }
       /**
        * Stop listening — cleanup mic, transcriber, timers.
@@ -13317,8 +13503,8 @@ Approach this task thoughtfully:
 });
 
 // packages/prompts/dist/index.js
-import { join as join20, dirname as dirname6 } from "node:path";
-import { fileURLToPath as fileURLToPath3 } from "node:url";
+import { join as join20, dirname as dirname7 } from "node:path";
+import { fileURLToPath as fileURLToPath4 } from "node:url";
 var _dir, _packageRoot;
 var init_dist6 = __esm({
   "packages/prompts/dist/index.js"() {
@@ -13327,13 +13513,13 @@ var init_dist6 = __esm({
     init_render2();
     init_task_templates();
     init_render2();
-    _dir = dirname6(fileURLToPath3(import.meta.url));
+    _dir = dirname7(fileURLToPath4(import.meta.url));
     _packageRoot = join20(_dir, "..");
   }
 });
 
 // packages/cli/dist/tui/oa-directory.js
-import { existsSync as existsSync14, mkdirSync as mkdirSync6, readFileSync as readFileSync12, writeFileSync as writeFileSync6, readdirSync as readdirSync6, statSync as statSync6, unlinkSync as unlinkSync2 } from "node:fs";
+import { existsSync as existsSync14, mkdirSync as mkdirSync6, readFileSync as readFileSync12, writeFileSync as writeFileSync6, readdirSync as readdirSync7, statSync as statSync6, unlinkSync as unlinkSync2 } from "node:fs";
 import { join as join21, relative as relative2, basename as basename5, extname as extname8 } from "node:path";
 import { homedir as homedir7 } from "node:os";
 function initOaDirectory(repoRoot) {
@@ -13514,7 +13700,7 @@ function loadRecentSessions(repoRoot, limit = 5) {
   if (!existsSync14(historyDir))
     return [];
   try {
-    const files = readdirSync6(historyDir).filter((f) => f.endsWith(".json")).map((f) => {
+    const files = readdirSync7(historyDir).filter((f) => f.endsWith(".json")).map((f) => {
       const stat5 = statSync6(join21(historyDir, f));
       return { file: f, mtime: stat5.mtimeMs };
     }).sort((a, b) => b.mtime - a.mtime).slice(0, limit);
@@ -13612,7 +13798,7 @@ function buildDirTree(root, maxDepth, prefix = "", depth = 0) {
     return "";
   let result = "";
   try {
-    const entries = readdirSync6(root, { withFileTypes: true }).filter((e) => !e.name.startsWith(".") || e.name === ".github").filter((e) => !SKIP_DIRS.has(e.name)).sort((a, b) => {
+    const entries = readdirSync7(root, { withFileTypes: true }).filter((e) => !e.name.startsWith(".") || e.name === ".github").filter((e) => !SKIP_DIRS.has(e.name)).sort((a, b) => {
       if (a.isDirectory() && !b.isDirectory())
         return -1;
       if (!a.isDirectory() && b.isDirectory())
@@ -13627,7 +13813,7 @@ function buildDirTree(root, maxDepth, prefix = "", depth = 0) {
       if (entry.isDirectory()) {
         let fileCount = 0;
         try {
-          fileCount = readdirSync6(join21(root, entry.name)).filter((f) => !f.startsWith(".")).length;
+          fileCount = readdirSync7(join21(root, entry.name)).filter((f) => !f.startsWith(".")).length;
         } catch {
         }
         result += `${prefix}${connector}${entry.name}/ (${fileCount})
@@ -14843,11 +15029,11 @@ async function handleUpdate(subcommand, ctx) {
   let currentVersion = "0.0.0";
   try {
     const { createRequire: createRequire4 } = await import("node:module");
-    const { fileURLToPath: fileURLToPath6 } = await import("node:url");
-    const { dirname: dirname9, join: join32 } = await import("node:path");
+    const { fileURLToPath: fileURLToPath7 } = await import("node:url");
+    const { dirname: dirname10, join: join32 } = await import("node:path");
     const { existsSync: existsSync21 } = await import("node:fs");
     const req = createRequire4(import.meta.url);
-    const thisDir = dirname9(fileURLToPath6(import.meta.url));
+    const thisDir = dirname10(fileURLToPath7(import.meta.url));
     const candidates = [
       join32(thisDir, "..", "package.json"),
       join32(thisDir, "..", "..", "package.json"),
@@ -14959,7 +15145,7 @@ var init_commands = __esm({
 });
 
 // packages/cli/dist/tui/project-context.js
-import { existsSync as existsSync16, readFileSync as readFileSync13, readdirSync as readdirSync7 } from "node:fs";
+import { existsSync as existsSync16, readFileSync as readFileSync13, readdirSync as readdirSync8 } from "node:fs";
 import { join as join23, basename as basename6 } from "node:path";
 import { execSync as execSync14 } from "node:child_process";
 import { homedir as homedir9, platform, release } from "node:os";
@@ -15045,7 +15231,7 @@ function loadMemoryDir(memDir, scope) {
     return "";
   const lines = [];
   try {
-    const files = readdirSync7(memDir).filter((f) => f.endsWith(".json"));
+    const files = readdirSync8(memDir).filter((f) => f.endsWith(".json"));
     for (const file of files.slice(0, 10)) {
       try {
         const raw = readFileSync13(join23(memDir, file), "utf-8");
@@ -17246,7 +17432,7 @@ var init_edit_history = __esm({
 });
 
 // packages/cli/dist/tui/dream-engine.js
-import { mkdirSync as mkdirSync10, writeFileSync as writeFileSync9, readFileSync as readFileSync15, existsSync as existsSync18, cpSync, rmSync, readdirSync as readdirSync8 } from "node:fs";
+import { mkdirSync as mkdirSync10, writeFileSync as writeFileSync9, readFileSync as readFileSync15, existsSync as existsSync18, cpSync, rmSync, readdirSync as readdirSync9 } from "node:fs";
 import { join as join26, basename as basename7 } from "node:path";
 import { execSync as execSync16 } from "node:child_process";
 function adaptTool(tool) {
@@ -17784,7 +17970,7 @@ Each proposal includes implementation entrypoints and estimated effort.
       /** Update the master proposal index */
       updateProposalIndex() {
         try {
-          const files = readdirSync8(this.dreamsDir).filter((f) => f.endsWith(".md") && f !== "PROPOSAL-INDEX.md" && f !== "dream-state.json").sort();
+          const files = readdirSync9(this.dreamsDir).filter((f) => f.endsWith(".md") && f !== "PROPOSAL-INDEX.md" && f !== "dream-state.json").sort();
           const index = `# Dream Proposals Index
 
 **Last updated**: ${(/* @__PURE__ */ new Date()).toISOString().split("T")[0]}
@@ -18186,15 +18372,15 @@ var init_status_bar = __esm({
 import * as readline2 from "node:readline";
 import { Writable } from "node:stream";
 import { cwd } from "node:process";
-import { resolve as resolve16, join as join27, dirname as dirname7, extname as extname9 } from "node:path";
+import { resolve as resolve16, join as join27, dirname as dirname8, extname as extname9 } from "node:path";
 import { createRequire as createRequire2 } from "node:module";
-import { fileURLToPath as fileURLToPath4 } from "node:url";
+import { fileURLToPath as fileURLToPath5 } from "node:url";
 import { readFileSync as readFileSync16 } from "node:fs";
 import { existsSync as existsSync19 } from "node:fs";
 function getVersion() {
   try {
     const require2 = createRequire2(import.meta.url);
-    const thisDir = dirname7(fileURLToPath4(import.meta.url));
+    const thisDir = dirname8(fileURLToPath5(import.meta.url));
     const candidates = [
       join27(thisDir, "..", "package.json"),
       join27(thisDir, "..", "..", "package.json"),
@@ -20308,8 +20494,8 @@ init_output();
 init_updater();
 import { parseArgs as nodeParseArgs2 } from "node:util";
 import { createRequire as createRequire3 } from "node:module";
-import { fileURLToPath as fileURLToPath5 } from "node:url";
-import { dirname as dirname8, join as join31 } from "node:path";
+import { fileURLToPath as fileURLToPath6 } from "node:url";
+import { dirname as dirname9, join as join31 } from "node:path";
 
 // packages/cli/dist/cli.js
 import { createInterface } from "node:readline";
@@ -20416,7 +20602,7 @@ init_output();
 function getVersion2() {
   try {
     const require2 = createRequire3(import.meta.url);
-    const pkgPath = join31(dirname8(fileURLToPath5(import.meta.url)), "..", "package.json");
+    const pkgPath = join31(dirname9(fileURLToPath6(import.meta.url)), "..", "package.json");
     const pkg = require2(pkgPath);
     return pkg.version;
   } catch {

package/dist/scripts/live-whisper.py

@@ -0,0 +1,242 @@
+#!/usr/bin/env python3
+"""
+live-whisper.py — Self-contained live transcription worker using openai-whisper.
+
+Fallback for transcribe-cli on platforms where faster-whisper / ONNX fails
+(e.g. linux-arm64). Auto-creates a Python venv and installs openai-whisper.
+
+Protocol:
+  stdin  — raw PCM16 audio (16kHz, mono, 16-bit signed LE)
+  stdout — JSON lines:
+    {"type":"status","message":"Installing dependencies..."}
+    {"type":"status","message":"Loading model..."}
+    {"type":"ready"}
+    {"type":"transcript","text":"hello world","isFinal":false}
+    {"type":"transcript","text":"hello world how are you","isFinal":true}
+    {"type":"error","message":"..."}
+
+Usage:
+  arecord -f S16_LE -r 16000 -c 1 -t raw -q - | python3 live-whisper.py --model base
+
+Based on the proven ARM-compatible approach from hydra/whisper_asr.
+"""
+
+import sys
+import os
+import json
+import subprocess
+import importlib
+import struct
+import time
+import threading
+from pathlib import Path
+
+# ---------------------------------------------------------------------------
+# Configuration
+# ---------------------------------------------------------------------------
+
+SCRIPT_DIR = Path(__file__).resolve().parent
+VENV = SCRIPT_DIR / ".whisper-venv"
+PY = VENV / "bin" / "python"
+PIP = VENV / "bin" / "pip"
+
+SAMPLE_RATE = 16000
+CHANNELS = 1
+SAMPLE_WIDTH = 2  # 16-bit
+CHUNK_SECONDS = 3  # Transcribe every N seconds of audio
+WINDOW_SECONDS = 10  # Transcribe last N seconds (sliding window)
+
+# ---------------------------------------------------------------------------
+# Output helpers (JSON lines to stdout)
+# ---------------------------------------------------------------------------
+
+def emit(event: dict):
+    """Write a JSON event to stdout and flush."""
+    sys.stdout.write(json.dumps(event) + "\n")
+    sys.stdout.flush()
+
+
+def emit_status(msg: str):
+    emit({"type": "status", "message": msg})
+
+
+def emit_error(msg: str):
+    emit({"type": "error", "message": msg})
+
+
+def emit_transcript(text: str, is_final: bool = False):
+    emit({"type": "transcript", "text": text, "isFinal": is_final})
+
+# ---------------------------------------------------------------------------
+# Venv bootstrap (inspired by hydra/whisper_asr)
+# ---------------------------------------------------------------------------
+
+def _in_venv() -> bool:
+    return sys.prefix != sys.base_prefix
+
+
+def _ensure_venv():
+    if VENV.exists():
+        return
+    emit_status("Creating Python venv for Whisper...")
+    import venv
+    venv.EnvBuilder(with_pip=True).create(str(VENV))
+    # Upgrade pip quietly
+    subprocess.check_call(
+        [str(PY), "-m", "pip", "install", "--upgrade", "pip"],
+        stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+    )
+
+
+def _ensure_deps():
+    """Check and install missing dependencies."""
+    need = []
+    try:
+        import numpy
+    except ImportError:
+        need.append("numpy")
+    try:
+        import whisper
+    except ImportError:
+        need.append("openai-whisper")
+
+    if need:
+        emit_status(f"Installing: {', '.join(need)}...")
+        subprocess.check_call(
+            [str(PIP), "install", *need],
+            stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
+        )
+        # Force reimport
+        for mod in ["numpy", "whisper"]:
+            if mod in sys.modules:
+                del sys.modules[mod]
+
+# ---------------------------------------------------------------------------
+# Bootstrap: ensure we're in the venv with deps
+# ---------------------------------------------------------------------------
+
+if not _in_venv():
+    _ensure_venv()
+    # Re-exec this script inside the venv
+    os.execv(str(PY), [str(PY)] + sys.argv)
+
+_ensure_deps()
+
+# Now safe to import
+import numpy as np
+
+# ---------------------------------------------------------------------------
+# Main transcription loop
+# ---------------------------------------------------------------------------
+
+def main():
+    import argparse
+    parser = argparse.ArgumentParser(description="Live Whisper transcription worker")
+    parser.add_argument("--model", default="base", help="Whisper model size (tiny/base/small/medium/large)")
+    parser.add_argument("--chunk-seconds", type=float, default=CHUNK_SECONDS, help="Transcribe interval")
+    parser.add_argument("--window-seconds", type=float, default=WINDOW_SECONDS, help="Sliding window size")
+    parser.add_argument("--language", default=None, help="Language code (e.g. en, es, fr). Auto-detect if omitted.")
+    args = parser.parse_args()
+
+    import whisper
+
+    # Load model
+    emit_status(f"Loading Whisper {args.model} model...")
+    try:
+        device = "cpu"
+        try:
+            import torch
+            if torch.cuda.is_available():
+                device = "cuda"
+        except ImportError:
+            pass
+
+        model = whisper.load_model(args.model, device=device)
+    except Exception as e:
+        emit_error(f"Failed to load model: {e}")
+        sys.exit(1)
+
+    emit({"type": "ready"})
+
+    # Audio buffer — accumulates PCM16 as float32 @ 16kHz
+    audio_buf = np.zeros(0, dtype=np.float32)
+    buf_lock = threading.Lock()
+    chunk_bytes = int(args.chunk_seconds * SAMPLE_RATE * SAMPLE_WIDTH)
+    window_samples = int(args.window_seconds * SAMPLE_RATE)
+    last_text = ""
+    running = True
+
+    def read_stdin():
+        """Read PCM16 from stdin in a background thread."""
+        nonlocal audio_buf, running
+        try:
+            while running:
+                data = sys.stdin.buffer.read(chunk_bytes)
+                if not data:
+                    break  # EOF
+                # Convert PCM16 LE to float32 [-1, 1]
+                samples = np.frombuffer(data, dtype=np.int16).astype(np.float32) / 32768.0
+                with buf_lock:
+                    audio_buf = np.concatenate([audio_buf, samples])
+        except Exception:
+            pass
+        finally:
+            running = False
+
+    # Start stdin reader thread
+    reader = threading.Thread(target=read_stdin, daemon=True)
+    reader.start()
+
+    try:
+        while running:
+            time.sleep(args.chunk_seconds)
+
+            with buf_lock:
+                if len(audio_buf) < SAMPLE_RATE:  # Need at least 1s of audio
+                    continue
+                # Take the last window_seconds of audio
+                window = audio_buf[-window_samples:].copy() if len(audio_buf) > window_samples else audio_buf.copy()
+
+            # Transcribe
+            try:
+                fp16 = (device == "cuda")
+                result = model.transcribe(
+                    window,
+                    fp16=fp16,
+                    language=args.language,
+                    no_speech_threshold=0.6,
+                    condition_on_previous_text=False,
+                )
+                text = result.get("text", "").strip()
+                if text and text != last_text:
+                    last_text = text
+                    emit_transcript(text, is_final=False)
+            except Exception as e:
+                emit_error(f"Transcription error: {e}")
+
+    except KeyboardInterrupt:
+        pass
+
+    # Final transcription of the full buffer
+    with buf_lock:
+        full_audio = audio_buf.copy()
+
+    if len(full_audio) >= SAMPLE_RATE:
+        try:
+            fp16 = (device == "cuda")
+            result = model.transcribe(
+                full_audio,
+                fp16=fp16,
+                language=args.language,
+            )
+            text = result.get("text", "").strip()
+            if text:
+                emit_transcript(text, is_final=True)
+        except Exception:
+            pass
+
+    running = False
+
+
+if __name__ == "__main__":
+    main()

package/dist/scripts/start-moondream.py

@@ -0,0 +1,112 @@
+#!/usr/bin/env python3
+"""
+Headless Moondream Station launcher for open-agents.
+
+Starts the Moondream vision model REST API server on port 2020 without
+the interactive REPL. Designed to be auto-launched by the VisionTool.
+
+Usage:
+  python start-moondream.py [--port 2020] [--host 127.0.0.1]
+
+Environment:
+  HF_TOKEN  - HuggingFace token (optional, for gated models)
+"""
+
+import sys
+import signal
+import time
+import argparse
+
+def main():
+    parser = argparse.ArgumentParser(description="Start Moondream Station REST server")
+    parser.add_argument("--port", type=int, default=2020, help="Server port (default: 2020)")
+    parser.add_argument("--host", default="127.0.0.1", help="Server host (default: 127.0.0.1)")
+    parser.add_argument("--model", default=None, help="Model to use (default: auto-detect, prefers non-gated moondream-2)")
+    args = parser.parse_args()
+
+    try:
+        from moondream_station.core.config import ConfigManager
+        from moondream_station.core.manifest import ManifestManager
+        from moondream_station.core.models import ModelManager
+        from moondream_station.core.service import ServiceManager
+        from moondream_station.core.analytics import Analytics
+        from moondream_station.session import SessionState
+        from moondream_station.ui.display import Display
+    except ImportError:
+        print("ERROR: moondream-station not installed. Install with: pip install moondream-station", file=sys.stderr)
+        sys.exit(1)
+
+    print(f"[moondream] Initializing...", flush=True)
+
+    config = ConfigManager()
+    config.set("service_host", args.host)
+    config.set("service_port", args.port)
+
+    manifest_manager = ManifestManager(config)
+    analytics = Analytics(config, manifest_manager)
+    display = Display()
+    models = ModelManager(config, manifest_manager)
+    session_state = SessionState()
+
+    # Load manifest
+    manifest_url = "https://m87-md-prod-assets.s3.us-west-2.amazonaws.com/station/mds2/production_manifest.json"
+    print(f"[moondream] Loading manifest...", flush=True)
+    try:
+        manifest_manager.load_manifest(manifest_url, analytics, display)
+    except Exception as e:
+        print(f"ERROR: Failed to load manifest: {e}", file=sys.stderr)
+        sys.exit(1)
+
+    # Select model — prefer moondream-2 (non-gated) unless overridden
+    import os
+    model_name = args.model
+    if not model_name:
+        # Prefer moondream-2 (no HF token required) unless user has HF_TOKEN
+        has_hf_token = bool(os.environ.get("HF_TOKEN") or config.get("hf_token"))
+        if has_hf_token:
+            model_name = manifest_manager.get_available_default_model()
+        else:
+            model_name = "moondream-2"
+    if not model_name:
+        model_name = manifest_manager.get_available_default_model()
+    if not model_name:
+        print("ERROR: No model available", file=sys.stderr)
+        sys.exit(1)
+
+    print(f"[moondream] Switching to model: {model_name}", flush=True)
+    if not models.switch_model(model_name, display):
+        print(f"ERROR: Failed to switch to model {model_name}", file=sys.stderr)
+        sys.exit(1)
+
+    # Start REST server
+    service = ServiceManager(config, manifest_manager, session_state, analytics)
+    print(f"[moondream] Starting REST server on {args.host}:{args.port}...", flush=True)
+
+    if not service.start(model_name, args.port):
+        print("ERROR: Failed to start REST server", file=sys.stderr)
+        sys.exit(1)
+
+    print(f"[moondream] Server running at http://{args.host}:{args.port}/v1", flush=True)
+    print(f"[moondream] Endpoints: /v1/caption, /v1/query, /v1/detect, /v1/point", flush=True)
+    print(f"READY", flush=True)
+
+    # Handle shutdown
+    def shutdown(signum, frame):
+        print(f"\n[moondream] Shutting down...", flush=True)
+        service.stop()
+        sys.exit(0)
+
+    signal.signal(signal.SIGTERM, shutdown)
+    signal.signal(signal.SIGINT, shutdown)
+
+    # Keep alive
+    try:
+        while service.is_running():
+            time.sleep(1)
+    except KeyboardInterrupt:
+        shutdown(None, None)
+
+    print("[moondream] Server stopped", flush=True)
+
+if __name__ == "__main__":
+    main()

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-agents-ai",
-  "version": "0.19.0",
+  "version": "0.20.0",
   "description": "AI coding agent powered by open-source models (Ollama/vLLM) — interactive TUI with agentic tool-calling loop",
   "type": "module",
   "main": "./dist/index.js",