ft-scout 8.0.2 → 8.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,7 @@
1
- styles.css,1788837280424,9af505c412e76d8c7ba06682686741a3cf74576fd4c813cc04f76a3e84ee335f
2
1
  ScoutFavicon.png,1786186569805,41a9ca9abf7acd2976e78c72a72285b49cc0e64b2b5eeb2c2b88c69628fc618a
3
- web_cli.html,1788838401645,30d5e62a5d8c1c8bd4df6bad3d149beacfee179854732d44a8851aeb18de4f6a
4
- index.html,1788838160025,0817fa631e6fa85023f78555327ba5566b0d1a0452edfd21988f9d36c2399eb7
5
- paid_plans.html,1788838401644,f57e08585d95ed75ccd174a5a1fe7561ad14a394a92757c072ee526c06a3b5c1
6
- authentication.html,1788838248887,0c18d51a13b2af09c31101ec5dd756ea8d9d536f7788dcfbda8793a8491bf3e3
7
- app.js,1788838401648,4c6b4edabf046f404464a4a6a1655e9e33be045d32740a4d418fc5bd1aa99356
2
+ web_cli.html,1789048513457,75b7648de5366eb0f6565544610c965521e69ceb9248581edff14495dd1d839f
3
+ authentication.html,1789048395369,535f9fc282d8f7417fb51bd04f905184a603bf4677dd1c03c94babf0a66323f6
4
+ styles.css,1789048603188,7ac973be4a0034bc84096400b12c5a6f327c394db9fc189e97d48c1260474906
5
+ paid_plans.html,1789048625479,2f8899f9ec336e90a5cecb5c0ce489ff3aeb628eb87e559c497225a92410b6cc
6
+ index.html,1789048612383,60105d1acd08ea4b160fdff9d0f64cafbe5e4783f48385f140d509a09bcfb30d
7
+ app.js,1789048647291,6e29d4a713c000d6450dfe9528d12c9be6a83a2e2a1178dda3c1c043c11996cf
package/README.md CHANGED
@@ -1,4 +1,4 @@
1
- # scout (FrontTerrain CLI v8.0.2)
1
+ # scout (FrontTerrain CLI v8.0.4)
2
2
 
3
3
  **FrontTerrain (`scout`)** — The Autonomous AI Principal Software Engineer & Codebase Co-Pilot in your terminal.
4
4
 
@@ -32,14 +32,16 @@ scout goal "Refactor database query logic" -y
32
32
  ### Interactive Scout Agent Session (Persistent chat loop with Slash Commands & full ReAct tool execution)
33
33
  scout agent
34
34
 
35
- # Select Interaction Mode: 💬 Text Interaction OR 🎙️ Audio Interaction (Voice STT + TTS)
36
- # Force Voice Mode directly via CLI flag:
37
- scout agent -V
35
+ # Select Interaction Mode: Text Interaction | Audio Interaction (Voice STT + TTS) | Live Screen Share Interaction
36
+ # Launch directly into your preferred mode via CLI flags:
37
+ scout agent -S # Launch with Live Screen Sharing (streams active app window to model in real time)
38
+ scout agent --screen-share
39
+ scout agent -V # Launch with Audio Voice mode (STT + TTS)
38
40
  scout agent --voice
39
41
 
40
42
  # Inside session:
41
- # Agent Prompt: Add dark mode toggle button
42
- # Slash commands: /help /scratchpad /voice /speak /tree /stats /test /explain /export /diff /undo /clear /scan /plan /history /exit
43
+ # Agent Prompt: "Inspect Chrome and automate login form submission"
44
+ # Slash commands: /help /screenshare /scratchpad /voice /speak /tree /stats /test /explain /export /diff /undo /clear /history /exit
43
45
 
44
46
  # Generate and create a new source code file
45
47
  scout agent --create src/utils/math.ts "Create helper functions for sum, average, and array chunking"
@@ -142,11 +144,23 @@ Scout is built for uninterrupted autonomy across multiple LLM providers:
142
144
 
143
145
  ---
144
146
 
145
- ## Dual Interaction Modes (Text vs Audio)
147
+ ## Triple Interaction Modes (Text, Voice, & Live Screen Share)
146
148
 
147
- When starting interactive sessions in `scout agent`, `scout init` (ask mode), or `scout search`, Scout presents a **2-Option Interaction Selector**:
148
- 1. **Text Interaction**: Standard prompt input -> Text response in terminal.
149
+ When starting interactive sessions in `scout agent`, `scout init` (ask mode), or `scout search`, Scout presents a **3-Option Interaction Selector**:
150
+ 1. **Text Interaction**: Standard command/prompt input -> Text response in terminal.
149
151
  2. **Audio Interaction**: Voice command input via microphone Speech-to-Text (STT) -> Text response + Audio response spoken out loud via Text-to-Speech (TTS).
152
+ 3. **Live Screen Share Interaction**: Real-time live screen share stream directly to AI vision models + Target App Window Isolation + Continuous Futuristic Sky-Blue Aura HUD.
153
+
154
+ ---
155
+
156
+ ## Real-Time Live Screen Sharing & Visual Perception Engine
157
+
158
+ Scout features a native in-memory live screen sharing engine designed specifically for multimodal AI co-pilot workflows:
159
+
160
+ - **Direct In-Memory Streaming (0 Files Saved to Disk)**: Completely eliminates saving temporary `.png` screenshot files to disk. Real-time compressed frames stream directly in-memory to AI multimodal vision models (Google Gemini 2.0 Flash, NVIDIA Vision, OpenAI GPT-4o) with zero disk I/O overhead.
161
+ - **Target Application Window Isolation**: When the agent is operating on an external app (such as Google Chrome, Microsoft Edge, VS Code, or Notepad), the capture engine isolates **ONLY that specific application's window rectangle** (`BoundingRectangle`). Background applications, sensitive tabs, and other monitor areas are completely excluded from the stream.
162
+ - **Continuous Sky-Blue Aura HUD Overlay**: While live screen sharing is active, a glowing sky-blue (`#00e5ff`) aura border and corner HUD pulses smoothly across screen edges with an active status badge (`Live Screen Share Active [App: chrome] • Streaming to Model Perception Active`).
163
+ - **Autonomous App Focus Synchronization**: Whenever the agent opens or switches applications (`openApp('chrome')` or `takeoverControl('chrome')`), the live stream viewport automatically retargets and locks onto that application window.
150
164
 
151
165
  ---
152
166
 
@@ -155,6 +169,7 @@ When starting interactive sessions in `scout agent`, `scout init` (ask mode), or
155
169
  Inside an active `scout agent` interactive session, type any slash command to trigger agent actions:
156
170
 
157
171
  - `/help` — Display comprehensive guide to all 18 tools and interactive slash commands.
172
+ - `/screenshare` or `/livescreen` — Toggle real-time in-memory screen share stream to the AI model.
158
173
  - `/scratchpad` — View active working memory plan & step checklist (`.ft/scratchpad.md`).
159
174
  - `/voice` — Activate microphone Speech-to-Text (STT) listener to speak your prompt into CLI.
160
175
  - `/speak <text>` — Read entered text out loud via Text-to-Speech (TTS) voice synthesis.
@@ -171,14 +186,15 @@ Inside an active `scout agent` interactive session, type any slash command to tr
171
186
 
172
187
  ---
173
188
 
174
- ## What's New in v8.0.2
189
+ ## What's New in v8.0.4
175
190
 
176
- - **Nemotron & DeepSeek Fallback Tool-Calling**: Added smart JSON prompt formatting and fallback parser so models lacking native function-calling can reliably trigger tools, desktop automation, and scratchpad updates.
191
+ - **Real-Time Live Screen Sharing Engine**: Replaced static disk screenshots with in-memory video streaming directly to vision models. Zero disk writes, zero garbage files.
192
+ - **Target App Window Isolation**: Selectively captures and streams ONLY the active application window the agent is working in (e.g., Chrome) instead of leaking the entire desktop.
193
+ - **Continuous Sky-Blue Aura HUD**: A pulsing `#00e5ff` HUD border and corner brackets play continuously throughout the live screen sharing session.
194
+ - **Nemotron & DeepSeek Fallback Tool-Calling**: Smart JSON prompt formatting and fallback parser so models lacking native function-calling can reliably trigger tools, desktop automation, and scratchpad updates.
177
195
  - **Extended 90s LLM Timeout**: Boosted NVIDIA NIM client timeout from 35s to 90s to accommodate complex multi-step reasoning models without early cutoff.
178
196
  - **Instant Timeout Failover**: Added timeout detection (`ETIMEDOUT`, `ECONNABORTED`, `408`, `AbortError`) to trigger immediate model fallback instead of burning retries.
179
197
  - **Desktop Takeover & Scratchpad Tasks**: Enhanced cross-platform cursor positioning, mouse clicking, keystroke injection, and external app automation.
180
- - **Zero-Screenshot Privacy Architecture**: Permanently eliminates desktop screenshot captures and vision latency for 100% desktop privacy and instant action execution.
181
- - **Scratchpad Working Memory**: Dynamic step checklists and notes in `.ft/scratchpad.md` synchronized across agent loops.
182
198
 
183
199
  ---
184
200
 
@@ -1 +1 @@
1
- {"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["agent.ts"],"names":[],"mappings":"AA8BA,MAAM,WAAW,YAAY;IACzB,EAAE,EAAE,MAAM,CAAC;IACX,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,GAAG,QAAQ,GAAG,KAAK,CAAC;IACpC,WAAW,EAAE,MAAM,CAAC;CACvB;AAKD,wBAAgB,cAAc,CAAC,MAAM,EAAE,YAAY,EAAE,QAKpD;AAED,wBAAgB,cAAc,IAAI,YAAY,EAAE,CAO/C;AAED,MAAM,WAAW,YAAY;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC5B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAAC;IACpC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,EAAE,GAAG,SAAS,CAAC;IACpC,IAAI,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC3B,GAAG,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,SAAS,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC/B,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC3B,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC9B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC7B,GAAG,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;CAC/B;AAED,wBAAgB,wBAAwB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAsDlE;AAED,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAG3E;AAED,MAAM,WAAW,gBAAgB;IAC7B,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,UAAU,GAAG,QAAQ,GAAG,QAAQ,GAAG,MAAM,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,CAAC;IAC/E,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,WAAW,CAAC,EAAE,MAAM,EAAE,GAAG,SAAS,CAAC;CACtC;AAED,wBAAgB,qBAAqB,CAAC,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,GAAG,gBAAgB,CA2F5F;AAED,wBAAgB,yBAAyB,CACrC,UAAU,EAAE,MAAM,EAClB,aAAa,EAAE,MAAM,GAAG,SAAS,EACjC,SAAS,EAAE,MAAM,EAAE,GACpB,MAAM,EAAE,CAgGV;AAED,wBAAsB,iBAAiB,CAAC,cAAc,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,CAoFxF;AAED,wBAAsB,WAAW,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,YAAY,iBAmkC3E"}
1
+ {"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["agent.ts"],"names":[],"mappings":"AA+BA,MAAM,WAAW,YAAY;IACzB,EAAE,EAAE,MAAM,CAAC;IACX,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,GAAG,QAAQ,GAAG,KAAK,CAAC;IACpC,WAAW,EAAE,MAAM,CAAC;CACvB;AAKD,wBAAgB,cAAc,CAAC,MAAM,EAAE,YAAY,EAAE,QAKpD;AAED,wBAAgB,cAAc,IAAI,YAAY,EAAE,CAO/C;AAED,MAAM,WAAW,YAAY;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC5B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAAC;IACpC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,EAAE,GAAG,SAAS,CAAC;IACpC,IAAI,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC3B,GAAG,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,SAAS,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC/B,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC3B,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC9B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC7B,GAAG,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC5B,WAAW,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;CACrC;AAED,wBAAgB,wBAAwB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAsDlE;AAED,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAG3E;AAED,MAAM,WAAW,gBAAgB;IAC7B,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,UAAU,GAAG,QAAQ,GAAG,QAAQ,GAAG,MAAM,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,CAAC;IAC/E,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,WAAW,CAAC,EAAE,MAAM,EAAE,GAAG,SAAS,CAAC;CACtC;AAED,wBAAgB,qBAAqB,CAAC,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,GAAG,gBAAgB,CA2F5F;AAED,wBAAgB,yBAAyB,CACrC,UAAU,EAAE,MAAM,EAClB,aAAa,EAAE,MAAM,GAAG,SAAS,EACjC,SAAS,EAAE,MAAM,EAAE,GACpB,MAAM,EAAE,CAgGV;AAED,wBAAsB,iBAAiB,CAAC,cAAc,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,CAuLxF;AAED,wBAAsB,WAAW,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,YAAY,iBA2lC3E"}
@@ -14,6 +14,7 @@ import { handleUndo } from './undo.js';
14
14
  import { AgentExecutionLoop, buildDirectoryTree, AGENT_TOOLS } from '../engine/agentEngine.js';
15
15
  import { verifyAndSelfHealFiles, executeSmartCommand, runProjectVerification } from '../engine/verifier.js';
16
16
  import { speakText, listenSpeechToText, promptInteractionMode } from '../engine/voiceEngine.js';
17
+ import { startLiveScreenShare, stopLiveScreenShare, isLiveScreenShareActive } from '../engine/liveScreenEngine.js';
17
18
  import { exec } from 'child_process';
18
19
  import { promisify } from 'util';
19
20
  const execAsync = promisify(exec);
@@ -247,6 +248,7 @@ export async function getAgentGoalInput(placeholderStr) {
247
248
  }
248
249
  return new Promise((resolve) => {
249
250
  let buffer = '';
251
+ let cursorIndex = 0;
250
252
  let isDone = false;
251
253
  const rl = readline.createInterface({
252
254
  input: process.stdin,
@@ -256,7 +258,15 @@ export async function getAgentGoalInput(placeholderStr) {
256
258
  if (process.stdin.setRawMode) {
257
259
  process.stdin.setRawMode(true);
258
260
  }
259
- process.stdout.write(`\n${chalk.cyan('◇')} ${chalk.bold('Agent Prompt:')}\n${chalk.cyan('│')} `);
261
+ const prefix = `${chalk.cyan('│')} `;
262
+ const prefixVisibleLen = 3;
263
+ process.stdout.write(`\n${chalk.cyan('◇')} ${chalk.bold('Agent Prompt:')}\n${prefix}`);
264
+ const renderLine = () => {
265
+ readline.cursorTo(process.stdout, 0);
266
+ readline.clearLine(process.stdout, 0);
267
+ process.stdout.write(prefix + buffer);
268
+ readline.cursorTo(process.stdout, prefixVisibleLen + cursorIndex);
269
+ };
260
270
  const cleanup = () => {
261
271
  process.stdin.removeListener('keypress', onKeypress);
262
272
  if (process.stdin.setRawMode) {
@@ -281,7 +291,7 @@ export async function getAgentGoalInput(placeholderStr) {
281
291
  cleanup();
282
292
  readline.cursorTo(process.stdout, 0);
283
293
  readline.clearLine(process.stdout, 0);
284
- process.stdout.write(`${chalk.cyan('│')} /\n`);
294
+ process.stdout.write(`${prefix}/\n`);
285
295
  resolve('/');
286
296
  return;
287
297
  }
@@ -293,19 +303,106 @@ export async function getAgentGoalInput(placeholderStr) {
293
303
  resolve(buffer.trim());
294
304
  return;
295
305
  }
306
+ // Left Arrow Navigation
307
+ if (key && key.name === 'left') {
308
+ if (key.ctrl || key.meta) {
309
+ let newPos = cursorIndex;
310
+ while (newPos > 0 && buffer[newPos - 1] === ' ')
311
+ newPos--;
312
+ while (newPos > 0 && buffer[newPos - 1] !== ' ')
313
+ newPos--;
314
+ cursorIndex = newPos;
315
+ }
316
+ else if (cursorIndex > 0) {
317
+ cursorIndex--;
318
+ }
319
+ renderLine();
320
+ return;
321
+ }
322
+ // Right Arrow Navigation
323
+ if (key && key.name === 'right') {
324
+ if (key.ctrl || key.meta) {
325
+ let newPos = cursorIndex;
326
+ while (newPos < buffer.length && buffer[newPos] === ' ')
327
+ newPos++;
328
+ while (newPos < buffer.length && buffer[newPos] !== ' ')
329
+ newPos++;
330
+ cursorIndex = newPos;
331
+ }
332
+ else if (cursorIndex < buffer.length) {
333
+ cursorIndex++;
334
+ }
335
+ renderLine();
336
+ return;
337
+ }
338
+ // Home / Ctrl+A
339
+ if ((key && key.name === 'home') || (key && key.ctrl && key.name === 'a')) {
340
+ cursorIndex = 0;
341
+ renderLine();
342
+ return;
343
+ }
344
+ // End / Ctrl+E
345
+ if ((key && key.name === 'end') || (key && key.ctrl && key.name === 'e')) {
346
+ cursorIndex = buffer.length;
347
+ renderLine();
348
+ return;
349
+ }
350
+ // Up / Down arrow keys (ignore to prevent terminal escape sequences from polluting buffer)
351
+ if (key && (key.name === 'up' || key.name === 'down')) {
352
+ return;
353
+ }
354
+ // Clear to start (Ctrl+U)
355
+ if (key && key.ctrl && key.name === 'u') {
356
+ buffer = buffer.slice(cursorIndex);
357
+ cursorIndex = 0;
358
+ renderLine();
359
+ return;
360
+ }
361
+ // Clear to end (Ctrl+K)
362
+ if (key && key.ctrl && key.name === 'k') {
363
+ buffer = buffer.slice(0, cursorIndex);
364
+ renderLine();
365
+ return;
366
+ }
367
+ // Delete word left (Ctrl+W)
368
+ if (key && key.ctrl && key.name === 'w') {
369
+ let newPos = cursorIndex;
370
+ while (newPos > 0 && buffer[newPos - 1] === ' ')
371
+ newPos--;
372
+ while (newPos > 0 && buffer[newPos - 1] !== ' ')
373
+ newPos--;
374
+ buffer = buffer.slice(0, newPos) + buffer.slice(cursorIndex);
375
+ cursorIndex = newPos;
376
+ renderLine();
377
+ return;
378
+ }
379
+ // Backspace
296
380
  const isBackspace = (key && key.name === 'backspace') || str === '\b' || str === '\x7f';
297
381
  if (isBackspace) {
298
- if (buffer.length > 0) {
299
- buffer = buffer.slice(0, -1);
300
- readline.cursorTo(process.stdout, 0);
301
- readline.clearLine(process.stdout, 0);
302
- process.stdout.write(`${chalk.cyan('│')} ${buffer}`);
382
+ if (cursorIndex > 0) {
383
+ buffer = buffer.slice(0, cursorIndex - 1) + buffer.slice(cursorIndex);
384
+ cursorIndex--;
385
+ renderLine();
303
386
  }
304
387
  return;
305
388
  }
389
+ // Delete (Forward delete)
390
+ if (key && key.name === 'delete') {
391
+ if (cursorIndex < buffer.length) {
392
+ buffer = buffer.slice(0, cursorIndex) + buffer.slice(cursorIndex + 1);
393
+ renderLine();
394
+ }
395
+ return;
396
+ }
397
+ // Regular character insertion
306
398
  if (str && !key?.ctrl && !key?.meta && key?.name !== 'escape' && !isEnter && !isBackspace) {
307
- buffer += str;
308
- process.stdout.write(str);
399
+ const cleanStr = str.replace(/[\r\n]/g, '');
400
+ if (cleanStr.length > 0) {
401
+ buffer = buffer.slice(0, cursorIndex) + cleanStr + buffer.slice(cursorIndex);
402
+ cursorIndex += cleanStr.length;
403
+ renderLine();
404
+ }
405
+ return;
309
406
  }
310
407
  };
311
408
  process.stdin.on('keypress', onKeypress);
@@ -345,36 +442,43 @@ export async function handleAgent(goalInput, options) {
345
442
  const projectName = existingContext?.projectName || path.basename(cwd) || 'Codebase';
346
443
  let initialGoal = goalInput || options?.goal;
347
444
  let isFirstRun = true;
445
+ let isAudioMode = options?.voice;
446
+ let isScreenShareMode = options?.screenShare;
447
+ if (isAudioMode === undefined && isScreenShareMode === undefined && !initialGoal && process.stdout.isTTY) {
448
+ const selectedMode = await promptInteractionMode();
449
+ isAudioMode = selectedMode === 'audio';
450
+ isScreenShareMode = selectedMode === 'screen_share';
451
+ }
452
+ if (isScreenShareMode) {
453
+ await startLiveScreenShare();
454
+ }
348
455
  const agentLoop = new AgentExecutionLoop({
349
456
  autoApprove: options?.yes,
350
457
  maxSteps: options?.steps || options?.maxSteps || 30,
351
458
  language: targetLang,
352
459
  projectName,
460
+ screenShare: isScreenShareMode,
353
461
  });
354
462
  const modelInfo = getActiveModelInfo();
355
463
  const fallbackPipeline = modelInfo.fallbackChain.slice(1).join(' ➔ ');
356
- let isAudioMode = options?.voice;
357
- if (isAudioMode === undefined && !initialGoal && process.stdout.isTTY) {
358
- const selectedMode = await promptInteractionMode();
359
- isAudioMode = selectedMode === 'audio';
360
- }
361
464
  const welcomeBanner = `
362
- ${chalk.bold.cyan('Scout Autonomous AI Agent')} ${targetLang ? `(${targetLang})` : ''} [${isAudioMode ? ' Audio Mode' : ' Text Mode'}]
465
+ ${chalk.bold.cyan('Scout Autonomous AI Agent')} ${targetLang ? `(${targetLang})` : ''} [${isScreenShareMode ? 'Live Screen Share Mode' : isAudioMode ? '🎤 Audio Mode' : '📝 Text Mode'}]
363
466
  ${chalk.bold('Active Model:')} ${chalk.bold.green(modelInfo.currentActiveModel)}
364
467
  ${chalk.bold('Fallback Pipeline:')} ${chalk.dim(fallbackPipeline)}
365
468
 
366
469
  ${chalk.dim('Type your goal, or use slash commands:')}
367
- ${chalk.cyan('/help')} - Show agent tools & commands (${AGENT_TOOLS.length} tools)
368
- ${chalk.cyan('/tree')} - View workspace directory tree
369
- ${chalk.cyan('/stats')} - View real-time session & model telemetry
370
- ${chalk.cyan('/test')} - Run automated project test suite
371
- ${chalk.cyan('/diff')} - Show git diff of uncommitted changes
372
- ${chalk.cyan('/export')} - Save session report to .ft/
373
- ${chalk.cyan('/undo')} - Revert last file modifications
374
- ${chalk.cyan('/clear')} - Reset conversation context
375
- ${chalk.cyan('/scan')} - Run vulnerability scan
376
- ${chalk.cyan('/plan')} - Generate implementation roadmap
377
- ${chalk.cyan('/exit')} - Exit session
470
+ ${chalk.cyan('/help')} - Show agent tools & commands (${AGENT_TOOLS.length} tools)
471
+ ${chalk.cyan('/screenshare')} - Toggle real-time live screen share stream with model
472
+ ${chalk.cyan('/tree')} - View workspace directory tree
473
+ ${chalk.cyan('/stats')} - View real-time session & model telemetry
474
+ ${chalk.cyan('/test')} - Run automated project test suite
475
+ ${chalk.cyan('/diff')} - Show git diff of uncommitted changes
476
+ ${chalk.cyan('/export')} - Save session report to .ft/
477
+ ${chalk.cyan('/undo')} - Revert last file modifications
478
+ ${chalk.cyan('/clear')} - Reset conversation context
479
+ ${chalk.cyan('/scan')} - Run vulnerability scan
480
+ ${chalk.cyan('/plan')} - Generate implementation roadmap
481
+ ${chalk.cyan('/exit')} - Exit session
378
482
  `.trim();
379
483
  safeNote(welcomeBanner, 'Scout Agent Active Session');
380
484
  while (true) {
@@ -409,7 +513,8 @@ ${chalk.cyan('/exit')} - Exit session
409
513
  const KNOWN_SLASH_COMMANDS = [
410
514
  '/help', '/tree', '/status', '/stats', '/test', '/explain',
411
515
  '/export', '/save-session', '/diff', '/clear', '/compact',
412
- '/undo', '/history', '/scan', '/plan', '/scratchpad', '/voice', '/speak', '/exit', '/quit'
516
+ '/undo', '/history', '/scan', '/plan', '/scratchpad', '/voice', '/speak',
517
+ '/screenshare', '/livescreen', '/live', '/exit', '/quit'
413
518
  ];
414
519
  if (trimmedInput.startsWith('/')) {
415
520
  const cmdName = trimmedInput.split(/\s+/)[0].toLowerCase();
@@ -417,6 +522,7 @@ ${chalk.cyan('/exit')} - Exit session
417
522
  const filterStr = trimmedInput.toLowerCase();
418
523
  let availableOptions = [
419
524
  { value: '/help', label: '/help', hint: 'Show agent tools & commands guide' },
525
+ { value: '/screenshare', label: '/screenshare', hint: 'Toggle real-time live screen share stream with model' },
420
526
  { value: '/voice', label: '/voice', hint: 'Activate microphone STT speech input' },
421
527
  { value: '/speak', label: '/speak', hint: 'Speak text out loud via TTS' },
422
528
  { value: '/scratchpad', label: '/scratchpad', hint: 'View current agent working memory & plan (.ft/scratchpad.md)' },
@@ -511,6 +617,7 @@ ${chalk.cyan('/exit')} - Exit session
511
617
 
512
618
  ### 🚀 Interactive Slash Commands
513
619
  - \`/help\`: Show this tools & commands guide.
620
+ - \`/screenshare\` or \`/live\`: Toggle real-time live screen share stream with model.
514
621
  - \`/tree [dir]\`: Display workspace directory tree structure.
515
622
  - \`/status\` or \`/stats\`: Show session telemetry & statistics.
516
623
  - \`/test [cmd]\`: Run project verification test suite.
@@ -529,6 +636,19 @@ ${chalk.cyan('/exit')} - Exit session
529
636
  safeNote(renderMarkdown(helpText), ' Scout Slash Commands & Tools');
530
637
  continue;
531
638
  }
639
+ if (trimmedInput.toLowerCase() === '/screenshare' || trimmedInput.toLowerCase() === '/livescreen' || trimmedInput.toLowerCase() === '/live') {
640
+ if (isLiveScreenShareActive()) {
641
+ const stopRes = stopLiveScreenShare();
642
+ isScreenShareMode = false;
643
+ safeNote(chalk.yellow(stopRes.summary), ' Live Screen Share Stopped');
644
+ }
645
+ else {
646
+ const startRes = await startLiveScreenShare();
647
+ isScreenShareMode = true;
648
+ safeNote(chalk.green(startRes.message), ' Live Screen Share Active');
649
+ }
650
+ continue;
651
+ }
532
652
  if (trimmedInput.toLowerCase().startsWith('/speak')) {
533
653
  const phrase = trimmedInput.slice(6).trim();
534
654
  if (phrase) {