@mobileai/react-native 0.4.6 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/README.md +80 -4
  2. package/lib/module/components/AIAgent.js +179 -38
  3. package/lib/module/components/AIAgent.js.map +1 -1
  4. package/lib/module/components/AgentChatBar.js +53 -29
  5. package/lib/module/components/AgentChatBar.js.map +1 -1
  6. package/lib/module/components/Icons.js +337 -0
  7. package/lib/module/components/Icons.js.map +1 -0
  8. package/lib/module/core/AgentRuntime.js +74 -3
  9. package/lib/module/core/AgentRuntime.js.map +1 -1
  10. package/lib/module/core/systemPrompt.js +66 -39
  11. package/lib/module/core/systemPrompt.js.map +1 -1
  12. package/lib/module/index.js +3 -9
  13. package/lib/module/index.js.map +1 -1
  14. package/lib/module/services/AudioInputService.js +73 -2
  15. package/lib/module/services/AudioInputService.js.map +1 -1
  16. package/lib/module/services/AudioOutputService.js +58 -5
  17. package/lib/module/services/AudioOutputService.js.map +1 -1
  18. package/lib/module/services/VoiceService.js +281 -275
  19. package/lib/module/services/VoiceService.js.map +1 -1
  20. package/lib/typescript/src/components/AIAgent.d.ts.map +1 -1
  21. package/lib/typescript/src/components/AgentChatBar.d.ts.map +1 -1
  22. package/lib/typescript/src/components/Icons.d.ts +43 -0
  23. package/lib/typescript/src/components/Icons.d.ts.map +1 -0
  24. package/lib/typescript/src/core/AgentRuntime.d.ts +12 -0
  25. package/lib/typescript/src/core/AgentRuntime.d.ts.map +1 -1
  26. package/lib/typescript/src/core/systemPrompt.d.ts.map +1 -1
  27. package/lib/typescript/src/index.d.ts +4 -0
  28. package/lib/typescript/src/index.d.ts.map +1 -1
  29. package/lib/typescript/src/services/AudioInputService.d.ts +13 -0
  30. package/lib/typescript/src/services/AudioInputService.d.ts.map +1 -1
  31. package/lib/typescript/src/services/AudioOutputService.d.ts.map +1 -1
  32. package/lib/typescript/src/services/VoiceService.d.ts +38 -29
  33. package/lib/typescript/src/services/VoiceService.d.ts.map +1 -1
  34. package/package.json +1 -1
  35. package/src/components/AIAgent.tsx +192 -39
  36. package/src/components/AgentChatBar.tsx +44 -25
  37. package/src/components/Icons.tsx +253 -0
  38. package/src/core/AgentRuntime.ts +70 -3
  39. package/src/core/systemPrompt.ts +66 -39
  40. package/src/index.ts +8 -8
  41. package/src/services/AudioInputService.ts +77 -2
  42. package/src/services/AudioOutputService.ts +59 -5
  43. package/src/services/VoiceService.ts +278 -290
package/README.md CHANGED
@@ -16,15 +16,27 @@ Wrap your navigation with `<AIAgent>`. The AI automatically understands your ent
16
16
 
17
17
  ## ✨ Features
18
18
 
19
+ ### Text Mode
19
20
  - 🤖 **Zero-config UI understanding** — No annotations needed. The AI sees and understands your entire screen automatically.
20
21
  - 🎯 **Works with every component** — Buttons, switches, inputs, custom components — all work out of the box.
21
22
  - 🖼️ **Sees images & videos** — The AI knows what media is on screen and can describe it.
22
23
  - 🧭 **Auto-navigation** — Navigates between screens to complete multi-step tasks.
23
- - 🔒 **Production-grade security** — Element gating, content masking, lifecycle hooks, human-in-the-loop confirmation.
24
24
  - 🧩 **Custom actions** — Expose any business logic (checkout, API calls) as AI-callable tools with `useAction`.
25
25
  - 🌐 **MCP bridge** — Let external AI agents (OpenClaw, Claude Desktop) control your app remotely.
26
26
  - 🌍 **Bilingual** — English and Arabic support built-in.
27
27
 
28
+ ### 🎤 Voice Mode (Live Agent)
29
+ - 🗣️ **Real-time voice chat** — Bidirectional audio with Gemini Live API. Speak naturally, the agent responds with voice.
30
+ - 🔄 **Screen change detection** — The agent automatically detects when the screen changes (e.g., loading finishes) and updates its context — no polling tool needed.
31
+ - 🛡️ **Tool-first protocol** — Tool calls are emitted before speech to prevent server crashes (Gemini Live API limitation).
32
+ - 🔇 **Audio gating** — Mic is automatically paused during tool execution and resumed after, preventing race conditions.
33
+ - 🚫 **Auto-navigation guard** — Code-level gate rejects tool calls before the user speaks, preventing the model from acting on screen context alone.
34
+
35
+ ### Security & Production
36
+ - 🔒 **Production-grade security** — Element gating, content masking, lifecycle hooks, human-in-the-loop confirmation.
37
+
38
+ > **Provider support:** Currently supports **Google Gemini** only (`gemini-2.5-flash` for text, `gemini-2.5-flash-native-audio-preview` for voice). Additional providers may be added in future releases.
39
+
28
40
  ## 📦 Installation
29
41
 
30
42
  ```bash
@@ -37,13 +49,75 @@ No native modules required by default. Works with Expo managed workflow out of t
37
49
 
38
50
  ### Optional Native Dependencies
39
51
 
52
+ #### Screenshots
53
+
40
54
  If you want to use **Screenshots** (for image/video content), install this optional peer dependency:
41
55
 
42
56
  ```bash
43
- # For Screenshots (when asking AI "what do you see in this image?"):
44
57
  npx expo install react-native-view-shot
45
58
  ```
46
59
 
60
+ #### 🎤 Voice Mode (Real-time Voice Chat)
61
+
62
+ Voice mode enables real-time bidirectional audio with the Gemini Live API. It requires one native module:
63
+
64
+ ```bash
65
+ # Audio capture + playback (required for voice mode):
66
+ npm install react-native-audio-api
67
+ ```
68
+
69
+ **After installing, you need native configuration based on your setup:**
70
+
71
+ <details>
72
+ <summary><b>Expo Managed Workflow</b></summary>
73
+
74
+ Add permissions to your `app.json`:
75
+
76
+ ```json
77
+ {
78
+ "expo": {
79
+ "android": {
80
+ "permissions": [
81
+ "RECORD_AUDIO",
82
+ "MODIFY_AUDIO_SETTINGS"
83
+ ]
84
+ },
85
+ "ios": {
86
+ "infoPlist": {
87
+ "NSMicrophoneUsageDescription": "Required for voice chat with AI assistant"
88
+ }
89
+ }
90
+ }
91
+ }
92
+ ```
93
+
94
+ Then rebuild: `npx expo prebuild && npx expo run:android` (or `run:ios`)
95
+
96
+ </details>
97
+
98
+ <details>
99
+ <summary><b>Expo Bare / React Native CLI</b></summary>
100
+
101
+ **Android** — add to `android/app/src/main/AndroidManifest.xml`:
102
+
103
+ ```xml
104
+ <uses-permission android:name="android.permission.RECORD_AUDIO"/>
105
+ <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>
106
+ ```
107
+
108
+ **iOS** — add to `ios/YourApp/Info.plist`:
109
+
110
+ ```xml
111
+ <key>NSMicrophoneUsageDescription</key>
112
+ <string>Required for voice chat with AI assistant</string>
113
+ ```
114
+
115
+ Then rebuild: `npx react-native run-android` (or `run-ios`)
116
+
117
+ </details>
118
+
119
+ > **Note:** Hardware echo cancellation (AEC) is automatically enabled through `react-native-audio-api`'s AudioManager — no extra setup needed.
120
+
47
121
  ## 🚀 Quick Start
48
122
 
49
123
  ```tsx
@@ -75,10 +149,12 @@ The root provider. Wrap your app once at the top level.
75
149
  | Prop | Type | Default | Description |
76
150
  |------|------|---------|-------------|
77
151
  | `apiKey` | `string` | — | **Required.** Gemini API key. |
78
- | `model` | `string` | `'gemini-2.5-flash'` | Gemini model name. |
152
+ | `model` | `string` | `'gemini-2.5-flash'` | Gemini model name for text mode. |
79
153
  | `navRef` | `NavigationContainerRef` | — | Navigation ref for auto-navigation. |
80
- | `maxSteps` | `number` | `10` | Max steps per task. |
154
+ | `maxSteps` | `number` | `10` | Max steps per task (text mode). |
81
155
  | `showChatBar` | `boolean` | `true` | Show the floating chat bar. |
156
+ | `enableVoice` | `boolean` | `true` | Enable voice mode tab in the chat bar. |
157
+ | `language` | `'en' \| 'ar'` | `'en'` | Agent language (English/Arabic). |
82
158
  | `onResult` | `(result) => void` | — | Called when the agent finishes. |
83
159
 
84
160
  ### `useAction(name, description, params, handler)`
@@ -80,6 +80,10 @@ export function AIAgent({
80
80
  const voiceServiceRef = useRef(null);
81
81
  const audioInputRef = useRef(null);
82
82
  const audioOutputRef = useRef(null);
83
+ const toolLockRef = useRef(false);
84
+ const userHasSpokenRef = useRef(false);
85
+ const lastScreenContextRef = useRef('');
86
+ const screenPollIntervalRef = useRef(null);
83
87
 
84
88
  // Compute available modes from props
85
89
  const availableModes = useMemo(() => {
@@ -105,8 +109,10 @@ export function AIAgent({
105
109
  onAfterStep,
106
110
  onBeforeTask,
107
111
  onAfterTask,
108
- transformScreenContent,
109
- customTools,
112
+ customTools: mode === 'voice' ? {
113
+ ...customTools,
114
+ ask_user: null
115
+ } : customTools,
110
116
  instructions,
111
117
  stepDelay,
112
118
  mcpServerUrl,
@@ -115,7 +121,7 @@ export function AIAgent({
115
121
  onStatusUpdate: setStatusText,
116
122
  onTokenUsage,
117
123
  // Page-agent pattern: block the agent loop until user responds
118
- onAskUser: question => {
124
+ onAskUser: mode === 'voice' ? undefined : question => {
119
125
  return new Promise(resolve => {
120
126
  askUserResolverRef.current = resolve;
121
127
  // Show question in chat bar, allow user input
@@ -128,7 +134,7 @@ export function AIAgent({
128
134
  setStatusText('');
129
135
  });
130
136
  }
131
- }), [apiKey, model, language, maxSteps, interactiveBlacklist, interactiveWhitelist, onBeforeStep, onAfterStep, onBeforeTask, onAfterTask, transformScreenContent, customTools, instructions, stepDelay, mcpServerUrl, router, pathname, onTokenUsage]);
137
+ }), [mode, apiKey, model, language, maxSteps, interactiveBlacklist, interactiveWhitelist, onBeforeStep, onAfterStep, onBeforeTask, onAfterTask, transformScreenContent, customTools, instructions, stepDelay, mcpServerUrl, router, pathname, onTokenUsage]);
132
138
  const provider = useMemo(() => new GeminiProvider(apiKey, model), [apiKey, model]);
133
139
  const runtime = useMemo(() => new AgentRuntime(provider, config, rootViewRef.current, navRef),
134
140
  // eslint-disable-next-line react-hooks/exhaustive-deps
@@ -160,6 +166,9 @@ export function AIAgent({
160
166
  }
161
167
  logger.info('AIAgent', `Mode changed to "${mode}" — initializing voice services...`);
162
168
 
169
+ // Track async audio output init — mic MUST wait for this
170
+ let audioOutputInitPromise = Promise.resolve();
171
+
163
172
  // Create VoiceService with runtime's built-in tools (navigate, tap, type, done, etc.)
164
173
  if (!voiceServiceRef.current) {
165
174
  logger.info('AIAgent', 'Creating VoiceService...');
@@ -168,13 +177,14 @@ export function AIAgent({
168
177
  // Use voice-adapted system prompt — same core rules as text mode
169
178
  // but without agent-loop directives that trigger autonomous actions
170
179
  const voicePrompt = buildVoiceSystemPrompt(language, instructions?.system);
180
+ logger.info('AIAgent', `📝 Voice system prompt (${voicePrompt.length} chars):\n${voicePrompt}`);
171
181
  voiceServiceRef.current = new VoiceService({
172
182
  apiKey,
173
183
  systemPrompt: voicePrompt,
174
184
  tools: runtimeTools,
175
185
  language
176
186
  });
177
- logger.info('AIAgent', 'VoiceService created with full voice system prompt and tools');
187
+ logger.info('AIAgent', `VoiceService created with ${runtimeTools.length} tools: ${runtimeTools.map(t => t.name).join(', ')}`);
178
188
  }
179
189
 
180
190
  // Create AudioOutputService if not exists
@@ -183,7 +193,11 @@ export function AIAgent({
183
193
  audioOutputRef.current = new AudioOutputService({
184
194
  onError: err => logger.error('AIAgent', `AudioOutput error: ${err}`)
185
195
  });
186
- audioOutputRef.current.initialize().then(ok => {
196
+ // IMPORTANT: Must await initialize() BEFORE starting mic.
197
+ // initialize() calls setAudioSessionOptions which reconfigures the
198
+ // audio hardware. If the mic starts before this finishes, the native
199
+ // audio session change kills the recorder's device handle.
200
+ audioOutputInitPromise = audioOutputRef.current.initialize().then(ok => {
187
201
  logger.info('AIAgent', `AudioOutputService initialized: ${ok}`);
188
202
  });
189
203
  }
@@ -194,7 +208,7 @@ export function AIAgent({
194
208
  audioInputRef.current = new AudioInputService({
195
209
  // Default 16kHz — Gemini Live API input standard
196
210
  onAudioChunk: chunk => {
197
- logger.debug('AIAgent', `Mic chunk: ${chunk.length} chars`);
211
+ logger.info('AIAgent', `🎤 onAudioChunk: ${chunk.length} chars, voiceService=${!!voiceServiceRef.current}, connected=${voiceServiceRef.current?.isConnected}`);
198
212
  voiceServiceRef.current?.sendAudio(chunk);
199
213
  },
200
214
  onError: err => logger.error('AIAgent', `AudioInput error: ${err}`),
@@ -202,67 +216,194 @@ export function AIAgent({
202
216
  });
203
217
  }
204
218
 
205
- // Connect VoiceService
219
+ // Connect VoiceService (async — SDK's ai.live.connect returns a Promise)
206
220
  logger.info('AIAgent', 'Connecting VoiceService...');
207
- voiceServiceRef.current.connect({
221
+ void voiceServiceRef.current.connect({
208
222
  onAudioResponse: audio => {
209
- logger.info('AIAgent', `Received audio response (${audio.length} chars)`);
223
+ logger.info('AIAgent', `🔊 Audio response: ${audio.length} chars, audioOutputRef=${!!audioOutputRef.current}`);
210
224
  setIsAISpeaking(true);
211
- audioOutputRef.current?.enqueue(audio);
225
+ if (!audioOutputRef.current) {
226
+ logger.error('AIAgent', '❌ audioOutputRef.current is NULL — cannot play audio!');
227
+ return;
228
+ }
229
+ audioOutputRef.current.enqueue(audio);
212
230
  },
213
231
  onStatusChange: status => {
214
232
  logger.info('AIAgent', `Voice status: ${status}`);
215
233
  const connected = status === 'connected';
216
234
  setIsVoiceConnected(connected);
217
235
  if (connected) {
218
- logger.info('AIAgent', '✅ VoiceService connected — auto-starting mic...');
219
- // Auto-start mic streaming once WebSocket is ready
220
- audioInputRef.current?.start().then(ok => {
221
- if (ok) {
222
- setIsMicActive(true);
223
- logger.info('AIAgent', '🎙️ Mic auto-started after connection');
224
- }
236
+ logger.info('AIAgent', '✅ VoiceService connected — waiting for audio session init before starting mic...');
237
+ // Wait for audio session config to finish BEFORE starting mic.
238
+ // If mic starts while setAudioSessionOptions is in flight,
239
+ // the native audio device gets killed (AudioDeviceStop error).
240
+ audioOutputInitPromise.then(() => {
241
+ logger.info('AIAgent', ' Audio session ready — starting mic now...');
242
+ audioInputRef.current?.start().then(ok => {
243
+ if (ok) {
244
+ setIsMicActive(true);
245
+ logger.info('AIAgent', '🎙️ Mic auto-started after connection');
246
+ }
247
+ });
225
248
  });
226
- // Send initial screen context so the model knows what's on screen.
227
- // sendScreenContext uses turnComplete: false (passive context)
228
- // so the model should NOT act on it until the user speaks.
229
- const initialContext = runtime.getScreenContext();
230
- voiceServiceRef.current?.sendScreenContext(initialContext);
231
- logger.info('AIAgent', '📡 Initial screen context sent (passive, turnComplete=false)');
249
+ }
250
+
251
+ // Handle unexpected disconnection auto-reconnect ONLY if not intentional
252
+ if (status === 'disconnected' && mode === 'voice' && voiceServiceRef.current && !voiceServiceRef.current.intentionalDisconnect) {
253
+ // Stop mic & audio immediately so user isn't stuck
254
+ audioInputRef.current?.stop();
255
+ audioOutputRef.current?.stop();
256
+ setIsMicActive(false);
257
+ setIsAISpeaking(false);
258
+ logger.warn('AIAgent', '⚠️ Voice connection dropped unexpectedly — attempting reconnect in 2s...');
259
+
260
+ // Auto-reconnect after short delay
261
+ setTimeout(() => {
262
+ if (mode === 'voice' && voiceServiceRef.current && !voiceServiceRef.current.intentionalDisconnect && voiceServiceRef.current.lastCallbacks) {
263
+ logger.info('AIAgent', '🔄 Reconnecting VoiceService...');
264
+ void voiceServiceRef.current.connect(voiceServiceRef.current.lastCallbacks);
265
+ }
266
+ }, 2000);
232
267
  }
233
268
  },
234
269
  onTranscript: (text, isFinal, role) => {
235
270
  logger.info('AIAgent', `Transcript [${role}] (final=${isFinal}): "${text}"`);
271
+ if (role === 'user') {
272
+ userHasSpokenRef.current = true;
273
+ }
274
+ },
275
+ onSetupComplete: () => {
276
+ logger.info('AIAgent', '✅ SDK setup complete — sending initial screen context now');
277
+ userHasSpokenRef.current = false;
278
+ const initialContext = runtime.getScreenContext();
279
+ lastScreenContextRef.current = initialContext; // Seed for screen change detection
280
+ const passiveContext = `[SYSTEM CONTEXT — THIS IS NOT A USER REQUEST. DO NOT RESPOND. DO NOT NAVIGATE. DO NOT CALL ANY TOOLS. DO NOT SPEAK. Just silently read and memorize this screen layout. You will use it when the user speaks their first voice command.]\n\n${initialContext}`;
281
+ logger.info('AIAgent', `📡 Initial screen context (${passiveContext.length} chars):\n${passiveContext}`);
282
+ voiceServiceRef.current?.sendScreenContext(passiveContext);
283
+ logger.info('AIAgent', '📡 Initial screen context sent (turnComplete=true)');
236
284
  },
237
285
  onToolCall: async toolCall => {
238
- logger.info('AIAgent', `Voice tool call: ${toolCall.name}(${JSON.stringify(toolCall.args)})`);
239
- // Execute the tool via AgentRuntime and send result back to Gemini
240
- const result = await runtime.executeTool(toolCall.name, toolCall.args);
241
- logger.info('AIAgent', `Voice tool result: ${result}`);
242
- voiceServiceRef.current?.sendFunctionResponse(toolCall.name, toolCall.id, {
243
- result
244
- });
286
+ logger.info('AIAgent', `🔧 Voice tool call: ${toolCall.name}(${JSON.stringify(toolCall.args)}) [id=${toolCall.id}]`);
287
+
288
+ // Code-level gate: reject tool calls before the user has spoken.
289
+ // The model sometimes auto-navigates on receiving screen context.
290
+ if (!userHasSpokenRef.current) {
291
+ logger.warn('AIAgent', `🚫 Rejected tool call ${toolCall.name} — user hasn't spoken yet`);
292
+ voiceServiceRef.current?.sendFunctionResponse(toolCall.name, toolCall.id, {
293
+ result: 'Action rejected: wait for the user to speak before performing any actions.'
294
+ });
295
+ return;
296
+ }
297
+
298
+ // CRITICAL: Gate audio input during tool execution.
299
+ // The Gemini Live API crashes (code 1008) if sendRealtimeInput
300
+ // (audio) is called while a tool call is pending. Stop the mic
301
+ // before executing the tool and resume after the response is sent.
302
+ audioInputRef.current?.stop();
303
+ logger.info('AIAgent', `🔇 Mic paused for tool execution: ${toolCall.name}`);
304
+
305
+ // One-tool-at-a-time enforcement (mirrors text mode's line 752).
306
+ if (toolLockRef.current) {
307
+ logger.warn('AIAgent', `⏳ Tool locked — waiting for previous tool to finish before executing ${toolCall.name}`);
308
+ while (toolLockRef.current) {
309
+ await new Promise(resolve => setTimeout(resolve, 50));
310
+ }
311
+ }
312
+ toolLockRef.current = true;
313
+ try {
314
+ // Execute the tool via AgentRuntime and send result back to Gemini
315
+ const result = await runtime.executeTool(toolCall.name, toolCall.args);
316
+ logger.info('AIAgent', `🔧 Tool result for ${toolCall.name}: ${result}`);
245
317
 
246
- // After tool execution, push updated screen context
247
- // (the screen may have changed from tap/type/navigate)
248
- const updatedContext = runtime.getScreenContext();
249
- voiceServiceRef.current?.sendScreenContext(updatedContext);
250
- logger.info('AIAgent', '📡 Updated screen context sent after tool call');
318
+ // Step delay matches text mode's stepDelay (line 820 in AgentRuntime).
319
+ await new Promise(resolve => setTimeout(resolve, 300));
320
+
321
+ // Include updated screen context IN the tool response
322
+ const updatedContext = runtime.getScreenContext();
323
+ lastScreenContextRef.current = updatedContext; // Sync with poll tracker
324
+ logger.info('AIAgent', `📡 Updated screen context after ${toolCall.name} (${updatedContext.length} chars):\n${updatedContext}`);
325
+ const enrichedResult = `${result}\n\n<updated_screen>\n${updatedContext}\n</updated_screen>`;
326
+ logger.info('AIAgent', `📡 Enriched tool response (${enrichedResult.length} chars):\n${enrichedResult}`);
327
+ voiceServiceRef.current?.sendFunctionResponse(toolCall.name, toolCall.id, {
328
+ result: enrichedResult
329
+ });
330
+ logger.info('AIAgent', `📡 Tool response sent for ${toolCall.name} [id=${toolCall.id}]`);
331
+ } finally {
332
+ toolLockRef.current = false;
333
+ // Resume mic after tool response is sent
334
+ if (voiceServiceRef.current?.isConnected) {
335
+ audioInputRef.current?.start().then(ok => {
336
+ if (ok) {
337
+ setIsMicActive(true);
338
+ logger.info('AIAgent', `🔊 Mic resumed after tool execution: ${toolCall.name}`);
339
+ }
340
+ });
341
+ }
342
+ }
251
343
  },
252
344
  onError: err => {
253
345
  logger.error('AIAgent', `VoiceService error: ${err}`);
346
+ // Stop mic & audio on error to prevent stale state
347
+ audioInputRef.current?.stop();
348
+ audioOutputRef.current?.stop();
349
+ setIsMicActive(false);
350
+ setIsAISpeaking(false);
254
351
  },
255
352
  onTurnComplete: () => {
256
353
  logger.info('AIAgent', 'AI turn complete');
257
354
  setIsAISpeaking(false);
355
+ // No cool-down or echo gate needed — hardware AEC handles everything.
356
+ // Mic stays active and ready for the next voice command immediately.
258
357
  }
259
358
  });
260
359
 
360
+ // ─── Screen Change Detection ───────────────────────────────
361
+ // Poll the Fiber tree every 5s and resend context if the screen meaningfully changed.
362
+ // This gives voice mode the same screen-awareness as text mode's per-step re-read.
363
+ const SCREEN_POLL_INTERVAL = 5000;
364
+ const MIN_DIFF_RATIO = 0.05; // Ignore changes smaller than 5% of total length (animation flicker)
365
+
366
+ screenPollIntervalRef.current = setInterval(() => {
367
+ if (!voiceServiceRef.current?.isConnected) return;
368
+ // Skip during tool execution — the enriched tool response handles that
369
+ if (toolLockRef.current) {
370
+ logger.debug('AIAgent', '🔄 Screen poll skipped — tool lock active');
371
+ return;
372
+ }
373
+ try {
374
+ const currentContext = runtime.getScreenContext();
375
+ if (currentContext === lastScreenContextRef.current) return; // No change
376
+
377
+ // Check if the change is meaningful (not just animation/cursor flicker)
378
+ const lastLen = lastScreenContextRef.current.length;
379
+ const diff = Math.abs(currentContext.length - lastLen);
380
+ const diffRatio = lastLen > 0 ? diff / lastLen : 1;
381
+ if (diffRatio < MIN_DIFF_RATIO) {
382
+ logger.debug('AIAgent', `🔄 Screen poll: minor change ignored (${diff} chars, ${(diffRatio * 100).toFixed(1)}% < ${MIN_DIFF_RATIO * 100}% threshold)`);
383
+ return;
384
+ }
385
+ logger.info('AIAgent', `🔄 Screen change detected (${lastLen} → ${currentContext.length} chars, ${(diffRatio * 100).toFixed(1)}% diff)`);
386
+ lastScreenContextRef.current = currentContext;
387
+ const passiveUpdate = `[SCREEN UPDATE — The UI has changed. Here is the current screen layout. This is not a user request — do not act unless the user asks.]\n\n${currentContext}`;
388
+ voiceServiceRef.current?.sendScreenContext(passiveUpdate);
389
+ logger.info('AIAgent', '🔄 Updated screen context sent to voice model');
390
+ } catch (err) {
391
+ logger.warn('AIAgent', `🔄 Screen poll error: ${err}`);
392
+ }
393
+ }, SCREEN_POLL_INTERVAL);
394
+
261
395
  // Cleanup on mode change back to text
262
396
  return () => {
263
397
  logger.info('AIAgent', `Cleaning up voice services (leaving "${mode}" mode)`);
398
+ // Stop screen change polling
399
+ if (screenPollIntervalRef.current) {
400
+ clearInterval(screenPollIntervalRef.current);
401
+ screenPollIntervalRef.current = null;
402
+ logger.info('AIAgent', '🔄 Screen poll stopped');
403
+ }
404
+ lastScreenContextRef.current = '';
264
405
  voiceServiceRef.current?.disconnect();
265
- voiceServiceRef.current = null; // Ensure fresh instance on next connect
406
+ voiceServiceRef.current = null;
266
407
  audioInputRef.current?.stop();
267
408
  setIsMicActive(false);
268
409
  setIsAISpeaking(false);
@@ -286,7 +427,7 @@ export function AIAgent({
286
427
  setIsMicActive(false);
287
428
  setIsAISpeaking(false);
288
429
  setIsVoiceConnected(false);
289
- // 5. Switch back to text mode (triggers cleanup effect naturally)
430
+ // 6. Switch back to text mode (triggers cleanup effect naturally)
290
431
  setMode('text');
291
432
  logger.info('AIAgent', '🛑 Voice session fully stopped');
292
433
  }, [runtime]);
@@ -1 +1 @@
1
- {"version":3,"names":["React","useCallback","useEffect","useMemo","useRef","useState","View","StyleSheet","AgentRuntime","GeminiProvider","AgentContext","AgentChatBar","AgentOverlay","logger","buildVoiceSystemPrompt","MCPBridge","VoiceService","AudioInputService","AudioOutputService","jsx","_jsx","jsxs","_jsxs","console","log","AIAgent","apiKey","model","navRef","language","maxSteps","showChatBar","children","onResult","interactiveBlacklist","interactiveWhitelist","onBeforeStep","onAfterStep","onBeforeTask","onAfterTask","transformScreenContent","customTools","instructions","stepDelay","mcpServerUrl","router","pathname","enableVoice","onTokenUsage","debug","setEnabled","info","rootViewRef","isThinking","setIsThinking","statusText","setStatusText","lastResult","setLastResult","mode","setMode","isMicActive","setIsMicActive","isSpeakerMuted","setIsSpeakerMuted","isAISpeaking","setIsAISpeaking","isVoiceConnected","setIsVoiceConnected","voiceServiceRef","audioInputRef","audioOutputRef","availableModes","modes","push","join","askUserResolverRef","config","onStatusUpdate","onAskUser","question","Promise","resolve","current","success","message","steps","provider","runtime","updateRefs","bridge","destroy","runtimeTools","getTools","length","map","t","name","voicePrompt","system","systemPrompt","tools","onError","err","error","initialize","then","ok","onAudioChunk","chunk","sendAudio","onPermissionDenied","warn","connect","onAudioResponse","audio","enqueue","onStatusChange","status","connected","start","initialContext","getScreenContext","sendScreenContext","onTranscript","text","isFinal","role","onToolCall","toolCall","JSON","stringify","args","result","executeTool","sendFunctionResponse","id","updatedContext","onTurnComplete","disconnect","stop","stopVoiceSession","handleSend","trim","resolver","execute","Provider","value","ref","style","styles","root","collapsable","visible","onSend","onDismiss","onModeChange","newMode","onStopSession","onMicToggle","active","onSpeakerToggle","muted","mute","unmute","create","flex"],"sourceRoot":"../../../src","sources":["components/AIAgent.tsx"],"mappings":";;AAAA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;;AAEA,OAAOA,KAAK,IACVC,WAAW,EACXC,SAAS,EACTC,OAAO,EACPC,MAAM,EACNC,QAAQ,QACH,OAAO;AACd,SAASC,IAAI,EAAEC,UAAU,QAAQ,cAAc;AAC/C,SAASC,YAAY,QAAQ,yBAAsB;AACnD,SAASC,cAAc,QAAQ,gCAA6B;AAC5D,SAASC,YAAY,QAAQ,uBAAoB;AACjD,SAASC,YAAY,QAAQ,mBAAgB;AAC7C,SAASC,YAAY,QAAQ,mBAAgB;AAC7C,SAASC,MAAM,QAAQ,oBAAiB;AACxC,SAASC,sBAAsB,QAAQ,yBAAsB;AAC7D,SAASC,SAAS,QAAQ,sBAAmB;AAC7C,SAASC,YAAY,QAAQ,6BAA0B;AACvD,SAASC,iBAAiB,QAAQ,kCAA+B;AACjE,SAASC,kBAAkB,QAAQ,mCAAgC;AAAC,SAAAC,GAAA,IAAAC,IAAA,EAAAC,IAAA,IAAAC,KAAA;AAGpE;AACAC,OAAO,CAACC,GAAG,CAAC,8BAA8B,CAAC;;AAE3C;;AA+DA;;AAEA,OAAO,SAASC,OAAOA,CAAC;EACtBC,MAAM;EACNC,KAAK,GAAG,kBAAkB;EAC1BC,MAAM;EACNC,QAAQ,GAAG,IAAI;EACfC,QAAQ,GAAG,EAAE;EACbC,WAAW,GAAG,IAAI;EAClBC,QAAQ;EACRC,QAAQ;EACR;EACAC,oBAAoB;EACpBC,oBAAoB;EACpBC,YAAY;EACZC,WAAW;EACXC,YAAY;EACZC,WAAW;EACXC,sBAAsB;EACtBC,WAAW;EACXC,YAAY;EACZC,SAAS;EACTC,YAAY;EACZC,MAAM;EACNC,QAAQ;EACRC,WAAW,GAAG,KAAK;EACnBC,YAAY;EACZC,KAAK,GAAG;AACI,CAAC,EAAE;EACf;EACAjD,KAAK,CAACE,SAAS,CAAC,MAAM;IACpBqB,OAAO,CAACC,GAAG,CAAC,wBAAwB,EAAEyB,KAAK,EAAE,mBAAmB,CAAC;IACjEpC,MAAM,CAACqC,UAAU,CAACD,KAAK,CAAC;IACxB,IAAIA,KAAK,EAAE;MACTpC,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,0BAA0B,CAAC;IACpD;EACF,CAAC,EAAE,CAACF,KAAK,CAAC,CAAC;EAEX,MAAMG,WAAW,GAAGhD,MAAM,CAAM,IAAI,CAAC;EACrC,MAAM,CAACiD,UAAU,EAAEC,aAAa,CAAC,GAAGjD,QAAQ,CAAC,KAAK,CAAC;EACnD,MAAM,CAACkD,UAAU,EAAEC,aAAa,CAAC,GAAGnD,QAAQ,CAAC,EAAE,CAAC;EAChD,MAAM,CAACoD,UAAU,EAAEC,aAAa,CAAC,GAAGrD,QAAQ,CAAyB,IAAI,CAAC;;EAE1E;EACA,MAAM,CAACsD,IAAI,EAAEC,OAAO,CAAC,GAAGvD,QAAQ,CAAY,MAAM,CAAC;EACnD,MAAM,CAACwD,WAAW,EAAEC,cAAc,CAAC,GAAGzD,QAAQ,CAAC,KAAK,CAAC;EACrD,MAAM,CAAC0D,cAAc,EAAEC,iBAAiB,CAAC,GAAG3D,QAAQ,CAAC,KAAK,CAAC;EAC3D,MAAM,CAAC4D,YAAY,EAAEC,eAAe,CAAC,GAAG7D,QAAQ,CAAC,KAAK,CAAC;EACvD,MAAM,CAAC8D,gBAAgB,EAAEC,mBAAmB,CAAC,GAAG/D,QAAQ,CAAC,KAAK,CAAC;EAE/D,MAAMgE,eAAe,GAAGjE,MAAM,CAAsB,IAAI,CAAC;EACzD,MAAMkE,aAAa,GAAGlE,MAAM,CAA2B,IAAI,CAAC;EAC5D,MAAMmE,cAAc,GAAGnE,MAAM,CAA4B,IAAI,CAAC;;EAE9D;EACA,MAAMoE,cAA2B,GAAGrE,OAAO,CAAC,MAAM;IAChD,MAAMsE,KAAkB,GAAG,CAAC,MAAM,CAAC;IACnC,IAAI1B,WAAW,EAAE0B,KAAK,CAACC,IAAI,CAAC,OAAO,CAAC;IACpC7D,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,oBAAoBsB,KAAK,CAACE,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;IAC9D,OAAOF,KAAK;EACd,CAAC,EAAE,CAAC1B,WAAW,CAAC,CAAC;;EAEjB;EACA,MAAM6B,kBAAkB,GAAGxE,MAAM,CAAoC,IAAI,CAAC;;EAE1E;;EAEA,MAAMyE,MAAmB,GAAG1E,OAAO,CAAC,OAAO;IACzCuB,MAAM;IACNC,KAAK;IACLE,QAAQ;IACRC,QAAQ;IACRI,oBAAoB;IACpBC,oBAAoB;IACpBC,YAAY;IACZC,WAAW;IACXC,YAAY;IACZC,WAAW;IACXC,sBAAsB;IACtBC,WAAW;IACXC,YAAY;IACZC,SAAS;IACTC,YAAY;IACZC,MAAM;IACNC,QAAQ;IACRgC,cAAc,EAAEtB,aAAa;IAC7BR,YAAY;IACZ;IACA+B,SAAS,EAAGC,QAAgB,IAAK;MAC/B,OAAO,IAAIC,OAAO,CAAUC,OAAO,IAAK;QACtCN,kBAAkB,CAACO,OAAO,GAAGD,OAAO;QACpC;QACAxB,aAAa,CAAC;UAAE0B,OAAO,EAAE,IAAI;UAAEC,OAAO,EAAE,KAAKL,QAAQ,EAAE;UAAEM,KAAK,EAAE;QAAG,CAAC,CAAC;QACrEhC,aAAa,CAAC,KAAK,CAAC;QACpBE,aAAa,CAAC,EAAE,CAAC;MACnB,CAAC,CAAC;IACJ;EACF,CAAC,CAAC,EAAE,CACF9B,MAAM,EAAEC,KAAK,EAAEE,QAAQ,EAAEC,QAAQ,EACjCI,oBAAoB,EAAEC,oBAAoB,EAC1CC,YAAY,EAAEC,WAAW,EAAEC,YAAY,EAAEC,WAAW,EACpDC,sBAAsB,EAAEC,WAAW,EAAEC,YAAY,EAAEC,SAAS,EAC5DC,YAAY,EAAEC,MAAM,EAAEC,QAAQ,EAAEE,YAAY,CAC7C,CAAC;EAEF,MAAMuC,QAAQ,GAAGpF,OAAO,CAAC,MAAM,IAAIM,cAAc,CAACiB,MAAM,EAAEC,KAAK,CAAC,EAAE,CAACD,MAAM,EAAEC,KAAK,CAAC,CAAC;EAElF,MAAM6D,OAAO,GAAGrF,OAAO,CACrB,MAAM,IAAIK,YAAY,CAAC+E,QAAQ,EAAEV,MAAM,EAAEzB,WAAW,CAAC+B,OAAO,EAAEvD,MAAM,CAAC;EACrE;EACA,CAAC2D,QAAQ,EAAEV,MAAM,CACnB,CAAC;;EAED;EACA3E,SAAS,CAAC,MAAM;IACdsF,OAAO,CAACC,UAAU,CAACrC,WAAW,CAAC+B,OAAO,EAAEvD,MAAM,CAAC;EACjD,CAAC,EAAE,CAAC4D,OAAO,EAAE5D,MAAM,CAAC,CAAC;;EAErB;;EAEA1B,SAAS,CAAC,MAAM;IACd,IAAI,CAAC0C,YAAY,EAAE;IAEnB/B,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,4BAA4BP,YAAY,EAAE,CAAC;IAClE,MAAM8C,MAAM,GAAG,IAAI3E,SAAS,CAAC6B,YAAY,EAAE4C,OAAO,CAAC;IAEnD,OAAO,MAAM;MACXE,MAAM,CAACC,OAAO,CAAC,CAAC;IAClB,CAAC;EACH,CAAC,EAAE,CAAC/C,YAAY,EAAE4C,OAAO,CAAC,CAAC;;EAE3B;;EAEA;EACAtF,SAAS,CAAC,MAAM;IACd,IAAIyD,IAAI,KAAK,MAAM,EAAE;MACnB9C,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,yCAAyC,CAAC;MACjE;IACF;IAEAtC,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,oBAAoBQ,IAAI,oCAAoC,CAAC;;IAEpF;IACA,IAAI,CAACU,eAAe,CAACc,OAAO,EAAE;MAC5BtE,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,0BAA0B,CAAC;MAClD,MAAMyC,YAAY,GAAGJ,OAAO,CAACK,QAAQ,CAAC,CAAC;MACvChF,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,eAAeyC,YAAY,CAACE,MAAM,6BAA6BF,YAAY,CAACG,GAAG,CAACC,CAAC,IAAIA,CAAC,CAACC,IAAI,CAAC,CAACtB,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;MACjI;MACA;MACA,MAAMuB,WAAW,GAAGpF,sBAAsB,CAACe,QAAQ,EAAEa,YAAY,EAAEyD,MAAM,CAAC;MAC1E9B,eAAe,CAACc,OAAO,GAAG,IAAInE,YAAY,CAAC;QACzCU,MAAM;QACN0E,YAAY,EAAEF,WAAW;QACzBG,KAAK,EAAET,YAAY;QACnB/D;MACF,CAAC,CAAC;MACFhB,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,8DAA8D,CAAC;IACxF;;IAEA;IACA,IAAI,CAACoB,cAAc,CAACY,OAAO,EAAE;MAC3BtE,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,gCAAgC,CAAC;MACxDoB,cAAc,CAACY,OAAO,GAAG,IAAIjE,kBAAkB,CAAC;QAC9CoF,OAAO,EAAGC,GAAG,IAAK1F,MAAM,CAAC2F,KAAK,CAAC,SAAS,EAAE,sBAAsBD,GAAG,EAAE;MACvE,CAAC,CAAC;MACFhC,cAAc,CAACY,OAAO,CAACsB,UAAU,CAAC,CAAC,CAACC,IAAI,CAAEC,EAAE,IAAK;QAC/C9F,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,mCAAmCwD,EAAE,EAAE,CAAC;MACjE,CAAC,CAAC;IACJ;;IAEA;IACA,IAAI,CAACrC,aAAa,CAACa,OAAO,EAAE;MAC1BtE,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,+BAA+B,CAAC;MACvDmB,aAAa,CAACa,OAAO,GAAG,IAAIlE,iBAAiB,CAAC;QAC5C;QACA2F,YAAY,EAAGC,KAAK,IAAK;UACvBhG,MAAM,CAACoC,KAAK,CAAC,SAAS,EAAE,cAAc4D,KAAK,CAACf,MAAM,QAAQ,CAAC;UAC3DzB,eAAe,CAACc,OAAO,EAAE2B,SAAS,CAACD,KAAK,CAAC;QAC3C,CAAC;QACDP,OAAO,EAAGC,GAAG,IAAK1F,MAAM,CAAC2F,KAAK,CAAC,SAAS,EAAE,qBAAqBD,GAAG,EAAE,CAAC;QACrEQ,kBAAkB,EAAEA,CAAA,KAAMlG,MAAM,CAACmG,IAAI,CAAC,SAAS,EAAE,+BAA+B;MAClF,CAAC,CAAC;IACJ;;IAEA;IACAnG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,4BAA4B,CAAC;IACpDkB,eAAe,CAACc,OAAO,CAAC8B,OAAO,CAAC;MAC9BC,eAAe,EAAGC,KAAK,IAAK;QAC1BtG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,4BAA4BgE,KAAK,CAACrB,MAAM,SAAS,CAAC;QACzE5B,eAAe,CAAC,IAAI,CAAC;QACrBK,cAAc,CAACY,OAAO,EAAEiC,OAAO,CAACD,KAAK,CAAC;MACxC,CAAC;MACDE,cAAc,EAAGC,MAAM,IAAK;QAC1BzG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,iBAAiBmE,MAAM,EAAE,CAAC;QACjD,MAAMC,SAAS,GAAGD,MAAM,KAAK,WAAW;QACxClD,mBAAmB,CAACmD,SAAS,CAAC;QAC9B,IAAIA,SAAS,EAAE;UACb1G,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,iDAAiD,CAAC;UACzE;UACAmB,aAAa,CAACa,OAAO,EAAEqC,KAAK,CAAC,CAAC,CAACd,IAAI,CAAEC,EAAE,IAAK;YAC1C,IAAIA,EAAE,EAAE;cACN7C,cAAc,CAAC,IAAI,CAAC;cACpBjD,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,uCAAuC,CAAC;YACjE;UACF,CAAC,CAAC;UACF;UACA;UACA;UACA,MAAMsE,cAAc,GAAGjC,OAAO,CAACkC,gBAAgB,CAAC,CAAC;UACjDrD,eAAe,CAACc,OAAO,EAAEwC,iBAAiB,CAACF,cAAc,CAAC;UAC1D5G,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,8DAA8D,CAAC;QACxF;MACF,CAAC;MACDyE,YAAY,EAAEA,CAACC,IAAI,EAAEC,OAAO,EAAEC,IAAI,KAAK;QACrClH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,eAAe4E,IAAI,YAAYD,OAAO,OAAOD,IAAI,GAAG,CAAC;MAC9E,CAAC;MACDG,UAAU,EAAE,MAAOC,QAAQ,IAAK;QAC9BpH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,oBAAoB8E,QAAQ,CAAChC,IAAI,IAAIiC,IAAI,CAACC,SAAS,CAACF,QAAQ,CAACG,IAAI,CAAC,GAAG,CAAC;QAC7F;QACA,MAAMC,MAAM,GAAG,MAAM7C,OAAO,CAAC8C,WAAW,CAACL,QAAQ,CAAChC,IAAI,EAAEgC,QAAQ,CAACG,IAAI,CAAC;QACtEvH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,sBAAsBkF,MAAM,EAAE,CAAC;QAEtDhE,eAAe,CAACc,OAAO,EAAEoD,oBAAoB,CAACN,QAAQ,CAAChC,IAAI,EAAEgC,QAAQ,CAACO,EAAE,EAAE;UAAEH;QAAO,CAAC,CAAC;;QAErF;QACA;QACA,MAAMI,cAAc,GAAGjD,OAAO,CAACkC,gBAAgB,CAAC,CAAC;QACjDrD,eAAe,CAACc,OAAO,EAAEwC,iBAAiB,CAACc,cAAc,CAAC;QAC1D5H,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,gDAAgD,CAAC;MAC1E,CAAC;MACDmD,OAAO,EAAGC,GAAG,IAAK;QAChB1F,MAAM,CAAC2F,KAAK,CAAC,SAAS,EAAE,uBAAuBD,GAAG,EAAE,CAAC;MACvD,CAAC;MACDmC,cAAc,EAAEA,CAAA,KAAM;QACpB7H,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,kBAAkB,CAAC;QAC1Ce,eAAe,CAAC,KAAK,CAAC;MACxB;IACF,CAAC,CAAC;;IAEF;IACA,OAAO,MAAM;MACXrD,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wCAAwCQ,IAAI,SAAS,CAAC;MAC7EU,eAAe,CAACc,OAAO,EAAEwD,UAAU,CAAC,CAAC;MACrCtE,eAAe,CAACc,OAAO,GAAG,IAAI,CAAC,CAAC;MAChCb,aAAa,CAACa,OAAO,EAAEyD,IAAI,CAAC,CAAC;MAC7B9E,cAAc,CAAC,KAAK,CAAC;MACrBI,eAAe,CAAC,KAAK,CAAC;MACtBE,mBAAmB,CAAC,KAAK,CAAC;IAC5B,CAAC;IACD;EACF,CAAC,EAAE,CAACT,IAAI,EAAEjC,MAAM,EAAE8D,OAAO,EAAE3D,QAAQ,EAAEa,YAAY,CAAC,CAAC;;EAEnD;;EAEA,MAAMmG,gBAAgB,GAAG5I,WAAW,CAAC,MAAM;IACzCY,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,6CAA6C,CAAC;IACrE;IACAmB,aAAa,CAACa,OAAO,EAAEyD,IAAI,CAAC,CAAC;IAC7B;IACArE,cAAc,CAACY,OAAO,EAAEyD,IAAI,CAAC,CAAC;IAC9B;IACAvE,eAAe,CAACc,OAAO,EAAEwD,UAAU,CAAC,CAAC;IACrCtE,eAAe,CAACc,OAAO,GAAG,IAAI;IAC9B;IACArB,cAAc,CAAC,KAAK,CAAC;IACrBI,eAAe,CAAC,KAAK,CAAC;IACtBE,mBAAmB,CAAC,KAAK,CAAC;IAC1B;IACAR,OAAO,CAAC,MAAM,CAAC;IACf/C,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,gCAAgC,CAAC;EAC1D,CAAC,EAAE,CAACqC,OAAO,CAAC,CAAC;;EAEb;;EAEA,MAAMsD,UAAU,GAAG7I,WAAW,CAAC,MAAOoF,OAAe,IAAK;IACxD,IAAI,CAACA,OAAO,CAAC0D,IAAI,CAAC,CAAC,EAAE;IAErBlI,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,kBAAkBkC,OAAO,GAAG,CAAC;;IAEpD;IACA,IAAIT,kBAAkB,CAACO,OAAO,EAAE;MAC9B,MAAM6D,QAAQ,GAAGpE,kBAAkB,CAACO,OAAO;MAC3CP,kBAAkB,CAACO,OAAO,GAAG,IAAI;MACjC7B,aAAa,CAAC,IAAI,CAAC;MACnBE,aAAa,CAAC,2BAA2B,CAAC;MAC1CE,aAAa,CAAC,IAAI,CAAC;MACnBsF,QAAQ,CAAC3D,OAAO,CAAC;MACjB;IACF;;IAEA;IACA/B,aAAa,CAAC,IAAI,CAAC;IACnBE,aAAa,CAAC,aAAa,CAAC;IAC5BE,aAAa,CAAC,IAAI,CAAC;IAEnB,IAAI;MACF;MACA8B,OAAO,CAACC,UAAU,CAACrC,WAAW,CAAC+B,OAAO,EAAEvD,MAAM,CAAC;MAE/C,MAAMyG,MAAM,GAAG,MAAM7C,OAAO,CAACyD,OAAO,CAAC5D,OAAO,CAAC;MAE7C3B,aAAa,CAAC2E,MAAM,CAAC;MACrBpG,QAAQ,GAAGoG,MAAM,CAAC;MAElBxH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,WAAWkF,MAAM,CAACjD,OAAO,GAAG,GAAG,GAAG,GAAG,IAAIiD,MAAM,CAAChD,OAAO,EAAE,CAAC;IACnF,CAAC,CAAC,OAAOmB,KAAU,EAAE;MACnB3F,MAAM,CAAC2F,KAAK,CAAC,SAAS,EAAE,mBAAmB,EAAEA,KAAK,CAAC;MACnD9C,aAAa,CAAC;QACZ0B,OAAO,EAAE,KAAK;QACdC,OAAO,EAAE,UAAUmB,KAAK,CAACnB,OAAO,EAAE;QAClCC,KAAK,EAAE;MACT,CAAC,CAAC;IACJ,CAAC,SAAS;MACRhC,aAAa,CAAC,KAAK,CAAC;MACpBE,aAAa,CAAC,EAAE,CAAC;IACnB;EACF,CAAC,EAAE,CAACgC,OAAO,EAAE5D,MAAM,EAAEK,QAAQ,CAAC,CAAC;;EAE/B;;EAEA,oBACEX,KAAA,CAACZ,YAAY,CAACwI,QAAQ;IAACC,KAAK,EAAE3D,OAAQ;IAAAxD,QAAA,gBACpCZ,IAAA,CAACd,IAAI;MAAC8I,GAAG,EAAEhG,WAAY;MAACiG,KAAK,EAAEC,MAAM,CAACC,IAAK;MAACC,WAAW,EAAE,KAAM;MAAAxH,QAAA,EAC5DA;IAAQ,CACL,CAAC,eAGPZ,IAAA,CAACR,YAAY;MAAC6I,OAAO,EAAEpG,UAAW;MAACE,UAAU,EAAEA;IAAW,CAAE,CAAC,EAG5DxB,WAAW,iBACVX,IAAA,CAACT,YAAY;MACX+I,MAAM,EAAEZ,UAAW;MACnBzF,UAAU,EAAEA,UAAW;MACvBI,UAAU,EAAEA,UAAW;MACvB5B,QAAQ,EAAEA,QAAS;MACnB8H,SAAS,EAAEA,CAAA,KAAMjG,aAAa,CAAC,IAAI,CAAE;MACrCc,cAAc,EAAEA,cAAe;MAC/Bb,IAAI,EAAEA,IAAK;MACXiG,YAAY,EAAGC,OAAO,IAAK;QACzBhJ,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,gBAAgBQ,IAAI,MAAMkG,OAAO,EAAE,CAAC;QAC3DjG,OAAO,CAACiG,OAAO,CAAC;MAClB,CAAE;MACFhG,WAAW,EAAEA,WAAY;MACzBE,cAAc,EAAEA,cAAe;MAC/BE,YAAY,EAAEA,YAAa;MAC3B6F,aAAa,EAAEjB,gBAAiB;MAChC1E,gBAAgB,EAAEA,gBAAiB;MACnC4F,WAAW,EAAGC,MAAM,IAAK;QACvB,IAAIA,MAAM,IAAI,CAAC7F,gBAAgB,EAAE;UAC/BtD,MAAM,CAACmG,IAAI,CAAC,SAAS,EAAE,oDAAoD,CAAC;UAC5E;QACF;QACAnG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,eAAe6G,MAAM,GAAG,IAAI,GAAG,KAAK,EAAE,CAAC;QAC9DlG,cAAc,CAACkG,MAAM,CAAC;QACtB,IAAIA,MAAM,EAAE;UACVnJ,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wBAAwB,CAAC;UAChDmB,aAAa,CAACa,OAAO,EAAEqC,KAAK,CAAC,CAAC,CAACd,IAAI,CAAEC,EAAE,IAAK;YAC1C9F,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,4BAA4BwD,EAAE,EAAE,CAAC;UAC1D,CAAC,CAAC;QACJ,CAAC,MAAM;UACL9F,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wBAAwB,CAAC;UAChDmB,aAAa,CAACa,OAAO,EAAEyD,IAAI,CAAC,CAAC;QAC/B;MACF,CAAE;MACFqB,eAAe,EAAGC,KAAK,IAAK;QAC1BrJ,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,mBAAmB+G,KAAK,GAAG,OAAO,GAAG,SAAS,EAAE,CAAC;QACxElG,iBAAiB,CAACkG,KAAK,CAAC;QACxB,IAAIA,KAAK,EAAE;UACT3F,cAAc,CAACY,OAAO,EAAEgF,IAAI,CAAC,CAAC;QAChC,CAAC,MAAM;UACL5F,cAAc,CAACY,OAAO,EAAEiF,MAAM,CAAC,CAAC;QAClC;MACF;IAAE,CAEH,CACF;EAAA,CACoB,CAAC;AAE5B;AAGA,MAAMd,MAAM,GAAG/I,UAAU,CAAC8J,MAAM,CAAC;EAC/Bd,IAAI,EAAE;IACJe,IAAI,EAAE;EACR;AACF,CAAC,CAAC","ignoreList":[]}
1
+ {"version":3,"names":["React","useCallback","useEffect","useMemo","useRef","useState","View","StyleSheet","AgentRuntime","GeminiProvider","AgentContext","AgentChatBar","AgentOverlay","logger","buildVoiceSystemPrompt","MCPBridge","VoiceService","AudioInputService","AudioOutputService","jsx","_jsx","jsxs","_jsxs","console","log","AIAgent","apiKey","model","navRef","language","maxSteps","showChatBar","children","onResult","interactiveBlacklist","interactiveWhitelist","onBeforeStep","onAfterStep","onBeforeTask","onAfterTask","transformScreenContent","customTools","instructions","stepDelay","mcpServerUrl","router","pathname","enableVoice","onTokenUsage","debug","setEnabled","info","rootViewRef","isThinking","setIsThinking","statusText","setStatusText","lastResult","setLastResult","mode","setMode","isMicActive","setIsMicActive","isSpeakerMuted","setIsSpeakerMuted","isAISpeaking","setIsAISpeaking","isVoiceConnected","setIsVoiceConnected","voiceServiceRef","audioInputRef","audioOutputRef","toolLockRef","userHasSpokenRef","lastScreenContextRef","screenPollIntervalRef","availableModes","modes","push","join","askUserResolverRef","config","ask_user","onStatusUpdate","onAskUser","undefined","question","Promise","resolve","current","success","message","steps","provider","runtime","updateRefs","bridge","destroy","audioOutputInitPromise","runtimeTools","getTools","length","map","t","name","voicePrompt","system","systemPrompt","tools","onError","err","error","initialize","then","ok","onAudioChunk","chunk","isConnected","sendAudio","onPermissionDenied","warn","connect","onAudioResponse","audio","enqueue","onStatusChange","status","connected","start","intentionalDisconnect","stop","setTimeout","lastCallbacks","onTranscript","text","isFinal","role","onSetupComplete","initialContext","getScreenContext","passiveContext","sendScreenContext","onToolCall","toolCall","JSON","stringify","args","id","sendFunctionResponse","result","executeTool","updatedContext","enrichedResult","onTurnComplete","SCREEN_POLL_INTERVAL","MIN_DIFF_RATIO","setInterval","currentContext","lastLen","diff","Math","abs","diffRatio","toFixed","passiveUpdate","clearInterval","disconnect","stopVoiceSession","handleSend","trim","resolver","execute","Provider","value","ref","style","styles","root","collapsable","visible","onSend","onDismiss","onModeChange","newMode","onStopSession","onMicToggle","active","onSpeakerToggle","muted","mute","unmute","create","flex"],"sourceRoot":"../../../src","sources":["components/AIAgent.tsx"],"mappings":";;AAAA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;;AAEA,OAAOA,KAAK,IACVC,WAAW,EACXC,SAAS,EACTC,OAAO,EACPC,MAAM,EACNC,QAAQ,QACH,OAAO;AACd,SAASC,IAAI,EAAEC,UAAU,QAAQ,cAAc;AAC/C,SAASC,YAAY,QAAQ,yBAAsB;AACnD,SAASC,cAAc,QAAQ,gCAA6B;AAC5D,SAASC,YAAY,QAAQ,uBAAoB;AACjD,SAASC,YAAY,QAAQ,mBAAgB;AAC7C,SAASC,YAAY,QAAQ,mBAAgB;AAC7C,SAASC,MAAM,QAAQ,oBAAiB;AACxC,SAASC,sBAAsB,QAAQ,yBAAsB;AAC7D,SAASC,SAAS,QAAQ,sBAAmB;AAC7C,SAASC,YAAY,QAAQ,6BAA0B;AACvD,SAASC,iBAAiB,QAAQ,kCAA+B;AACjE,SAASC,kBAAkB,QAAQ,mCAAgC;AAAC,SAAAC,GAAA,IAAAC,IAAA,EAAAC,IAAA,IAAAC,KAAA;AAGpE;AACAC,OAAO,CAACC,GAAG,CAAC,8BAA8B,CAAC;;AAE3C;;AA+DA;;AAEA,OAAO,SAASC,OAAOA,CAAC;EACtBC,MAAM;EACNC,KAAK,GAAG,kBAAkB;EAC1BC,MAAM;EACNC,QAAQ,GAAG,IAAI;EACfC,QAAQ,GAAG,EAAE;EACbC,WAAW,GAAG,IAAI;EAClBC,QAAQ;EACRC,QAAQ;EACR;EACAC,oBAAoB;EACpBC,oBAAoB;EACpBC,YAAY;EACZC,WAAW;EACXC,YAAY;EACZC,WAAW;EACXC,sBAAsB;EACtBC,WAAW;EACXC,YAAY;EACZC,SAAS;EACTC,YAAY;EACZC,MAAM;EACNC,QAAQ;EACRC,WAAW,GAAG,KAAK;EACnBC,YAAY;EACZC,KAAK,GAAG;AACI,CAAC,EAAE;EACf;EACAjD,KAAK,CAACE,SAAS,CAAC,MAAM;IACpBqB,OAAO,CAACC,GAAG,CAAC,wBAAwB,EAAEyB,KAAK,EAAE,mBAAmB,CAAC;IACjEpC,MAAM,CAACqC,UAAU,CAACD,KAAK,CAAC;IACxB,IAAIA,KAAK,EAAE;MACTpC,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,0BAA0B,CAAC;IACpD;EACF,CAAC,EAAE,CAACF,KAAK,CAAC,CAAC;EAEX,MAAMG,WAAW,GAAGhD,MAAM,CAAM,IAAI,CAAC;EACrC,MAAM,CAACiD,UAAU,EAAEC,aAAa,CAAC,GAAGjD,QAAQ,CAAC,KAAK,CAAC;EACnD,MAAM,CAACkD,UAAU,EAAEC,aAAa,CAAC,GAAGnD,QAAQ,CAAC,EAAE,CAAC;EAChD,MAAM,CAACoD,UAAU,EAAEC,aAAa,CAAC,GAAGrD,QAAQ,CAAyB,IAAI,CAAC;;EAE1E;EACA,MAAM,CAACsD,IAAI,EAAEC,OAAO,CAAC,GAAGvD,QAAQ,CAAY,MAAM,CAAC;EACnD,MAAM,CAACwD,WAAW,EAAEC,cAAc,CAAC,GAAGzD,QAAQ,CAAC,KAAK,CAAC;EACrD,MAAM,CAAC0D,cAAc,EAAEC,iBAAiB,CAAC,GAAG3D,QAAQ,CAAC,KAAK,CAAC;EAC3D,MAAM,CAAC4D,YAAY,EAAEC,eAAe,CAAC,GAAG7D,QAAQ,CAAC,KAAK,CAAC;EACvD,MAAM,CAAC8D,gBAAgB,EAAEC,mBAAmB,CAAC,GAAG/D,QAAQ,CAAC,KAAK,CAAC;EAE/D,MAAMgE,eAAe,GAAGjE,MAAM,CAAsB,IAAI,CAAC;EACzD,MAAMkE,aAAa,GAAGlE,MAAM,CAA2B,IAAI,CAAC;EAC5D,MAAMmE,cAAc,GAAGnE,MAAM,CAA4B,IAAI,CAAC;EAC9D,MAAMoE,WAAW,GAAGpE,MAAM,CAAU,KAAK,CAAC;EAC1C,MAAMqE,gBAAgB,GAAGrE,MAAM,CAAU,KAAK,CAAC;EAC/C,MAAMsE,oBAAoB,GAAGtE,MAAM,CAAS,EAAE,CAAC;EAC/C,MAAMuE,qBAAqB,GAAGvE,MAAM,CAAwC,IAAI,CAAC;;EAEjF;EACA,MAAMwE,cAA2B,GAAGzE,OAAO,CAAC,MAAM;IAChD,MAAM0E,KAAkB,GAAG,CAAC,MAAM,CAAC;IACnC,IAAI9B,WAAW,EAAE8B,KAAK,CAACC,IAAI,CAAC,OAAO,CAAC;IACpCjE,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,oBAAoB0B,KAAK,CAACE,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;IAC9D,OAAOF,KAAK;EACd,CAAC,EAAE,CAAC9B,WAAW,CAAC,CAAC;;EAEjB;EACA,MAAMiC,kBAAkB,GAAG5E,MAAM,CAAoC,IAAI,CAAC;;EAE1E;;EAEA,MAAM6E,MAAmB,GAAG9E,OAAO,CAAC,OAAO;IACzCuB,MAAM;IACNC,KAAK;IACLE,QAAQ;IACRC,QAAQ;IACRI,oBAAoB;IACpBC,oBAAoB;IACpBC,YAAY;IACZC,WAAW;IACXC,YAAY;IACZC,WAAW;IACXE,WAAW,EAAEkB,IAAI,KAAK,OAAO,GAAG;MAAE,GAAGlB,WAAW;MAAEyC,QAAQ,EAAE;IAAK,CAAC,GAAGzC,WAAW;IAChFC,YAAY;IACZC,SAAS;IACTC,YAAY;IACZC,MAAM;IACNC,QAAQ;IACRqC,cAAc,EAAE3B,aAAa;IAC7BR,YAAY;IACZ;IACAoC,SAAS,EAAEzB,IAAI,KAAK,OAAO,GAAG0B,SAAS,GAAKC,QAAgB,IAAK;MAC/D,OAAO,IAAIC,OAAO,CAAUC,OAAO,IAAK;QACtCR,kBAAkB,CAACS,OAAO,GAAGD,OAAO;QACpC;QACA9B,aAAa,CAAC;UAAEgC,OAAO,EAAE,IAAI;UAAEC,OAAO,EAAE,KAAKL,QAAQ,EAAE;UAAEM,KAAK,EAAE;QAAG,CAAC,CAAC;QACrEtC,aAAa,CAAC,KAAK,CAAC;QACpBE,aAAa,CAAC,EAAE,CAAC;MACnB,CAAC,CAAC;IACJ;EACF,CAAC,CAAC,EAAE,CACFG,IAAI,EAAEjC,MAAM,EAAEC,KAAK,EAAEE,QAAQ,EAAEC,QAAQ,EACvCI,oBAAoB,EAAEC,oBAAoB,EAC1CC,YAAY,EAAEC,WAAW,EAAEC,YAAY,EAAEC,WAAW,EACpDC,sBAAsB,EAAEC,WAAW,EAAEC,YAAY,EAAEC,SAAS,EAC5DC,YAAY,EAAEC,MAAM,EAAEC,QAAQ,EAAEE,YAAY,CAC7C,CAAC;EAEF,MAAM6C,QAAQ,GAAG1F,OAAO,CAAC,MAAM,IAAIM,cAAc,CAACiB,MAAM,EAAEC,KAAK,CAAC,EAAE,CAACD,MAAM,EAAEC,KAAK,CAAC,CAAC;EAElF,MAAMmE,OAAO,GAAG3F,OAAO,CACrB,MAAM,IAAIK,YAAY,CAACqF,QAAQ,EAAEZ,MAAM,EAAE7B,WAAW,CAACqC,OAAO,EAAE7D,MAAM,CAAC;EACrE;EACA,CAACiE,QAAQ,EAAEZ,MAAM,CACnB,CAAC;;EAED;EACA/E,SAAS,CAAC,MAAM;IACd4F,OAAO,CAACC,UAAU,CAAC3C,WAAW,CAACqC,OAAO,EAAE7D,MAAM,CAAC;EACjD,CAAC,EAAE,CAACkE,OAAO,EAAElE,MAAM,CAAC,CAAC;;EAErB;;EAEA1B,SAAS,CAAC,MAAM;IACd,IAAI,CAAC0C,YAAY,EAAE;IAEnB/B,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,4BAA4BP,YAAY,EAAE,CAAC;IAClE,MAAMoD,MAAM,GAAG,IAAIjF,SAAS,CAAC6B,YAAY,EAAEkD,OAAO,CAAC;IAEnD,OAAO,MAAM;MACXE,MAAM,CAACC,OAAO,CAAC,CAAC;IAClB,CAAC;EACH,CAAC,EAAE,CAACrD,YAAY,EAAEkD,OAAO,CAAC,CAAC;;EAE3B;;EAEA;EACA5F,SAAS,CAAC,MAAM;IACd,IAAIyD,IAAI,KAAK,MAAM,EAAE;MACnB9C,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,yCAAyC,CAAC;MACjE;IACF;IAEAtC,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,oBAAoBQ,IAAI,oCAAoC,CAAC;;IAEpF;IACA,IAAIuC,sBAAqC,GAAGX,OAAO,CAACC,OAAO,CAAC,CAAC;;IAE7D;IACA,IAAI,CAACnB,eAAe,CAACoB,OAAO,EAAE;MAC5B5E,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,0BAA0B,CAAC;MAClD,MAAMgD,YAAY,GAAGL,OAAO,CAACM,QAAQ,CAAC,CAAC;MACvCvF,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,eAAegD,YAAY,CAACE,MAAM,6BAA6BF,YAAY,CAACG,GAAG,CAACC,CAAC,IAAIA,CAAC,CAACC,IAAI,CAAC,CAACzB,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;MACjI;MACA;MACA,MAAM0B,WAAW,GAAG3F,sBAAsB,CAACe,QAAQ,EAAEa,YAAY,EAAEgE,MAAM,CAAC;MAC1E7F,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,2BAA2BsD,WAAW,CAACJ,MAAM,aAAaI,WAAW,EAAE,CAAC;MAC/FpC,eAAe,CAACoB,OAAO,GAAG,IAAIzE,YAAY,CAAC;QACzCU,MAAM;QACNiF,YAAY,EAAEF,WAAW;QACzBG,KAAK,EAAET,YAAY;QACnBtE;MACF,CAAC,CAAC;MACFhB,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,6BAA6BgD,YAAY,CAACE,MAAM,WAAWF,YAAY,CAACG,GAAG,CAACC,CAAC,IAAIA,CAAC,CAACC,IAAI,CAAC,CAACzB,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;IAC/H;;IAEA;IACA,IAAI,CAACR,cAAc,CAACkB,OAAO,EAAE;MAC3B5E,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,gCAAgC,CAAC;MACxDoB,cAAc,CAACkB,OAAO,GAAG,IAAIvE,kBAAkB,CAAC;QAC9C2F,OAAO,EAAGC,GAAG,IAAKjG,MAAM,CAACkG,KAAK,CAAC,SAAS,EAAE,sBAAsBD,GAAG,EAAE;MACvE,CAAC,CAAC;MACF;MACA;MACA;MACA;MACAZ,sBAAsB,GAAG3B,cAAc,CAACkB,OAAO,CAACuB,UAAU,CAAC,CAAC,CAACC,IAAI,CAAEC,EAAE,IAAK;QACxErG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,mCAAmC+D,EAAE,EAAE,CAAC;MACjE,CAAC,CAAC;IACJ;;IAEA;IACA,IAAI,CAAC5C,aAAa,CAACmB,OAAO,EAAE;MAC1B5E,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,+BAA+B,CAAC;MACvDmB,aAAa,CAACmB,OAAO,GAAG,IAAIxE,iBAAiB,CAAC;QAC5C;QACAkG,YAAY,EAAGC,KAAK,IAAK;UACvBvG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,oBAAoBiE,KAAK,CAACf,MAAM,wBAAwB,CAAC,CAAChC,eAAe,CAACoB,OAAO,eAAepB,eAAe,CAACoB,OAAO,EAAE4B,WAAW,EAAE,CAAC;UAC9JhD,eAAe,CAACoB,OAAO,EAAE6B,SAAS,CAACF,KAAK,CAAC;QAC3C,CAAC;QACDP,OAAO,EAAGC,GAAG,IAAKjG,MAAM,CAACkG,KAAK,CAAC,SAAS,EAAE,qBAAqBD,GAAG,EAAE,CAAC;QACrES,kBAAkB,EAAEA,CAAA,KAAM1G,MAAM,CAAC2G,IAAI,CAAC,SAAS,EAAE,+BAA+B;MAClF,CAAC,CAAC;IACJ;;IAEA;IACA3G,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,4BAA4B,CAAC;IACpD,KAAKkB,eAAe,CAACoB,OAAO,CAACgC,OAAO,CAAC;MACnCC,eAAe,EAAGC,KAAK,IAAK;QAC1B9G,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,sBAAsBwE,KAAK,CAACtB,MAAM,0BAA0B,CAAC,CAAC9B,cAAc,CAACkB,OAAO,EAAE,CAAC;QAC9GvB,eAAe,CAAC,IAAI,CAAC;QACrB,IAAI,CAACK,cAAc,CAACkB,OAAO,EAAE;UAC3B5E,MAAM,CAACkG,KAAK,CAAC,SAAS,EAAE,uDAAuD,CAAC;UAChF;QACF;QACAxC,cAAc,CAACkB,OAAO,CAACmC,OAAO,CAACD,KAAK,CAAC;MACvC,CAAC;MACDE,cAAc,EAAGC,MAAM,IAAK;QAC1BjH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,iBAAiB2E,MAAM,EAAE,CAAC;QACjD,MAAMC,SAAS,GAAGD,MAAM,KAAK,WAAW;QACxC1D,mBAAmB,CAAC2D,SAAS,CAAC;QAC9B,IAAIA,SAAS,EAAE;UACblH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,kFAAkF,CAAC;UAC1G;UACA;UACA;UACA+C,sBAAsB,CAACe,IAAI,CAAC,MAAM;YAChCpG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,6CAA6C,CAAC;YACrEmB,aAAa,CAACmB,OAAO,EAAEuC,KAAK,CAAC,CAAC,CAACf,IAAI,CAAEC,EAAE,IAAK;cAC1C,IAAIA,EAAE,EAAE;gBACNpD,cAAc,CAAC,IAAI,CAAC;gBACpBjD,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,uCAAuC,CAAC;cACjE;YACF,CAAC,CAAC;UACJ,CAAC,CAAC;QACJ;;QAEA;QACA,IACE2E,MAAM,KAAK,cAAc,IACzBnE,IAAI,KAAK,OAAO,IAChBU,eAAe,CAACoB,OAAO,IACvB,CAACpB,eAAe,CAACoB,OAAO,CAACwC,qBAAqB,EAC9C;UACA;UACA3D,aAAa,CAACmB,OAAO,EAAEyC,IAAI,CAAC,CAAC;UAC7B3D,cAAc,CAACkB,OAAO,EAAEyC,IAAI,CAAC,CAAC;UAC9BpE,cAAc,CAAC,KAAK,CAAC;UACrBI,eAAe,CAAC,KAAK,CAAC;UAEtBrD,MAAM,CAAC2G,IAAI,CAAC,SAAS,EAAE,0EAA0E,CAAC;;UAElG;UACAW,UAAU,CAAC,MAAM;YACf,IACExE,IAAI,KAAK,OAAO,IAChBU,eAAe,CAACoB,OAAO,IACvB,CAACpB,eAAe,CAACoB,OAAO,CAACwC,qBAAqB,IAC9C5D,eAAe,CAACoB,OAAO,CAAC2C,aAAa,EACrC;cACAvH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,iCAAiC,CAAC;cACzD,KAAKkB,eAAe,CAACoB,OAAO,CAACgC,OAAO,CAACpD,eAAe,CAACoB,OAAO,CAAC2C,aAAa,CAAC;YAC7E;UACF,CAAC,EAAE,IAAI,CAAC;QACV;MACF,CAAC;MACDC,YAAY,EAAEA,CAACC,IAAI,EAAEC,OAAO,EAAEC,IAAI,KAAK;QACrC3H,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,eAAeqF,IAAI,YAAYD,OAAO,OAAOD,IAAI,GAAG,CAAC;QAC5E,IAAIE,IAAI,KAAK,MAAM,EAAE;UACnB/D,gBAAgB,CAACgB,OAAO,GAAG,IAAI;QACjC;MACF,CAAC;MACDgD,eAAe,EAAEA,CAAA,KAAM;QACrB5H,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,2DAA2D,CAAC;QACnFsB,gBAAgB,CAACgB,OAAO,GAAG,KAAK;QAChC,MAAMiD,cAAc,GAAG5C,OAAO,CAAC6C,gBAAgB,CAAC,CAAC;QACjDjE,oBAAoB,CAACe,OAAO,GAAGiD,cAAc,CAAC,CAAC;QAC/C,MAAME,cAAc,GAAG,+OAA+OF,cAAc,EAAE;QACtR7H,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,8BAA8ByF,cAAc,CAACvC,MAAM,aAAauC,cAAc,EAAE,CAAC;QACxGvE,eAAe,CAACoB,OAAO,EAAEoD,iBAAiB,CAACD,cAAc,CAAC;QAC1D/H,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,oDAAoD,CAAC;MAC9E,CAAC;MACD2F,UAAU,EAAE,MAAOC,QAAQ,IAAK;QAC9BlI,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,uBAAuB4F,QAAQ,CAACvC,IAAI,IAAIwC,IAAI,CAACC,SAAS,CAACF,QAAQ,CAACG,IAAI,CAAC,SAASH,QAAQ,CAACI,EAAE,GAAG,CAAC;;QAEpH;QACA;QACA,IAAI,CAAC1E,gBAAgB,CAACgB,OAAO,EAAE;UAC7B5E,MAAM,CAAC2G,IAAI,CAAC,SAAS,EAAE,yBAAyBuB,QAAQ,CAACvC,IAAI,2BAA2B,CAAC;UACzFnC,eAAe,CAACoB,OAAO,EAAE2D,oBAAoB,CAACL,QAAQ,CAACvC,IAAI,EAAEuC,QAAQ,CAACI,EAAE,EAAE;YACxEE,MAAM,EAAE;UACV,CAAC,CAAC;UACF;QACF;;QAEA;QACA;QACA;QACA;QACA/E,aAAa,CAACmB,OAAO,EAAEyC,IAAI,CAAC,CAAC;QAC7BrH,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,qCAAqC4F,QAAQ,CAACvC,IAAI,EAAE,CAAC;;QAE5E;QACA,IAAIhC,WAAW,CAACiB,OAAO,EAAE;UACvB5E,MAAM,CAAC2G,IAAI,CAAC,SAAS,EAAE,wEAAwEuB,QAAQ,CAACvC,IAAI,EAAE,CAAC;UAC/G,OAAOhC,WAAW,CAACiB,OAAO,EAAE;YAC1B,MAAM,IAAIF,OAAO,CAACC,OAAO,IAAI2C,UAAU,CAAC3C,OAAO,EAAE,EAAE,CAAC,CAAC;UACvD;QACF;QACAhB,WAAW,CAACiB,OAAO,GAAG,IAAI;QAE1B,IAAI;UACF;UACA,MAAM4D,MAAM,GAAG,MAAMvD,OAAO,CAACwD,WAAW,CAACP,QAAQ,CAACvC,IAAI,EAAEuC,QAAQ,CAACG,IAAI,CAAC;UACtErI,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,sBAAsB4F,QAAQ,CAACvC,IAAI,KAAK6C,MAAM,EAAE,CAAC;;UAExE;UACA,MAAM,IAAI9D,OAAO,CAACC,OAAO,IAAI2C,UAAU,CAAC3C,OAAO,EAAE,GAAG,CAAC,CAAC;;UAEtD;UACA,MAAM+D,cAAc,GAAGzD,OAAO,CAAC6C,gBAAgB,CAAC,CAAC;UACjDjE,oBAAoB,CAACe,OAAO,GAAG8D,cAAc,CAAC,CAAC;UAC/C1I,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,mCAAmC4F,QAAQ,CAACvC,IAAI,KAAK+C,cAAc,CAAClD,MAAM,aAAakD,cAAc,EAAE,CAAC;UAC/H,MAAMC,cAAc,GAAG,GAAGH,MAAM,yBAAyBE,cAAc,qBAAqB;UAC5F1I,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,8BAA8BqG,cAAc,CAACnD,MAAM,aAAamD,cAAc,EAAE,CAAC;UAExGnF,eAAe,CAACoB,OAAO,EAAE2D,oBAAoB,CAACL,QAAQ,CAACvC,IAAI,EAAEuC,QAAQ,CAACI,EAAE,EAAE;YAAEE,MAAM,EAAEG;UAAe,CAAC,CAAC;UACrG3I,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,6BAA6B4F,QAAQ,CAACvC,IAAI,QAAQuC,QAAQ,CAACI,EAAE,GAAG,CAAC;QAC1F,CAAC,SAAS;UACR3E,WAAW,CAACiB,OAAO,GAAG,KAAK;UAC3B;UACA,IAAIpB,eAAe,CAACoB,OAAO,EAAE4B,WAAW,EAAE;YACxC/C,aAAa,CAACmB,OAAO,EAAEuC,KAAK,CAAC,CAAC,CAACf,IAAI,CAAEC,EAAE,IAAK;cAC1C,IAAIA,EAAE,EAAE;gBACNpD,cAAc,CAAC,IAAI,CAAC;gBACpBjD,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wCAAwC4F,QAAQ,CAACvC,IAAI,EAAE,CAAC;cACjF;YACF,CAAC,CAAC;UACJ;QACF;MACF,CAAC;MACDK,OAAO,EAAGC,GAAG,IAAK;QAChBjG,MAAM,CAACkG,KAAK,CAAC,SAAS,EAAE,uBAAuBD,GAAG,EAAE,CAAC;QACrD;QACAxC,aAAa,CAACmB,OAAO,EAAEyC,IAAI,CAAC,CAAC;QAC7B3D,cAAc,CAACkB,OAAO,EAAEyC,IAAI,CAAC,CAAC;QAC9BpE,cAAc,CAAC,KAAK,CAAC;QACrBI,eAAe,CAAC,KAAK,CAAC;MACxB,CAAC;MACDuF,cAAc,EAAEA,CAAA,KAAM;QACpB5I,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,kBAAkB,CAAC;QAC1Ce,eAAe,CAAC,KAAK,CAAC;QACtB;QACA;MACF;IACF,CAAC,CAAC;;IAEF;IACA;IACA;IACA,MAAMwF,oBAAoB,GAAG,IAAI;IACjC,MAAMC,cAAc,GAAG,IAAI,CAAC,CAAC;;IAE7BhF,qBAAqB,CAACc,OAAO,GAAGmE,WAAW,CAAC,MAAM;MAChD,IAAI,CAACvF,eAAe,CAACoB,OAAO,EAAE4B,WAAW,EAAE;MAC3C;MACA,IAAI7C,WAAW,CAACiB,OAAO,EAAE;QACvB5E,MAAM,CAACoC,KAAK,CAAC,SAAS,EAAE,2CAA2C,CAAC;QACpE;MACF;MAEA,IAAI;QACF,MAAM4G,cAAc,GAAG/D,OAAO,CAAC6C,gBAAgB,CAAC,CAAC;QACjD,IAAIkB,cAAc,KAAKnF,oBAAoB,CAACe,OAAO,EAAE,OAAO,CAAC;;QAE7D;QACA,MAAMqE,OAAO,GAAGpF,oBAAoB,CAACe,OAAO,CAACY,MAAM;QACnD,MAAM0D,IAAI,GAAGC,IAAI,CAACC,GAAG,CAACJ,cAAc,CAACxD,MAAM,GAAGyD,OAAO,CAAC;QACtD,MAAMI,SAAS,GAAGJ,OAAO,GAAG,CAAC,GAAGC,IAAI,GAAGD,OAAO,GAAG,CAAC;QAElD,IAAII,SAAS,GAAGP,cAAc,EAAE;UAC9B9I,MAAM,CAACoC,KAAK,CAAC,SAAS,EAAE,yCAAyC8G,IAAI,WAAW,CAACG,SAAS,GAAG,GAAG,EAAEC,OAAO,CAAC,CAAC,CAAC,OAAOR,cAAc,GAAG,GAAG,cAAc,CAAC;UACtJ;QACF;QAEA9I,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,8BAA8B2G,OAAO,MAAMD,cAAc,CAACxD,MAAM,WAAW,CAAC6D,SAAS,GAAG,GAAG,EAAEC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC;QACxIzF,oBAAoB,CAACe,OAAO,GAAGoE,cAAc;QAC7C,MAAMO,aAAa,GAAG,6IAA6IP,cAAc,EAAE;QACnLxF,eAAe,CAACoB,OAAO,EAAEoD,iBAAiB,CAACuB,aAAa,CAAC;QACzDvJ,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,+CAA+C,CAAC;MACzE,CAAC,CAAC,OAAO2D,GAAG,EAAE;QACZjG,MAAM,CAAC2G,IAAI,CAAC,SAAS,EAAE,yBAAyBV,GAAG,EAAE,CAAC;MACxD;IACF,CAAC,EAAE4C,oBAAoB,CAAC;;IAExB;IACA,OAAO,MAAM;MACX7I,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wCAAwCQ,IAAI,SAAS,CAAC;MAC7E;MACA,IAAIgB,qBAAqB,CAACc,OAAO,EAAE;QACjC4E,aAAa,CAAC1F,qBAAqB,CAACc,OAAO,CAAC;QAC5Cd,qBAAqB,CAACc,OAAO,GAAG,IAAI;QACpC5E,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wBAAwB,CAAC;MAClD;MACAuB,oBAAoB,CAACe,OAAO,GAAG,EAAE;MACjCpB,eAAe,CAACoB,OAAO,EAAE6E,UAAU,CAAC,CAAC;MACrCjG,eAAe,CAACoB,OAAO,GAAG,IAAI;MAC9BnB,aAAa,CAACmB,OAAO,EAAEyC,IAAI,CAAC,CAAC;MAC7BpE,cAAc,CAAC,KAAK,CAAC;MACrBI,eAAe,CAAC,KAAK,CAAC;MACtBE,mBAAmB,CAAC,KAAK,CAAC;IAC5B,CAAC;IACD;EACF,CAAC,EAAE,CAACT,IAAI,EAAEjC,MAAM,EAAEoE,OAAO,EAAEjE,QAAQ,EAAEa,YAAY,CAAC,CAAC;;EAEnD;;EAEA,MAAM6H,gBAAgB,GAAGtK,WAAW,CAAC,MAAM;IACzCY,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,6CAA6C,CAAC;IACrE;IACAmB,aAAa,CAACmB,OAAO,EAAEyC,IAAI,CAAC,CAAC;IAC7B;IACA3D,cAAc,CAACkB,OAAO,EAAEyC,IAAI,CAAC,CAAC;IAC9B;IACA7D,eAAe,CAACoB,OAAO,EAAE6E,UAAU,CAAC,CAAC;IACrCjG,eAAe,CAACoB,OAAO,GAAG,IAAI;IAC9B;IACA3B,cAAc,CAAC,KAAK,CAAC;IACrBI,eAAe,CAAC,KAAK,CAAC;IACtBE,mBAAmB,CAAC,KAAK,CAAC;IAC1B;IACAR,OAAO,CAAC,MAAM,CAAC;IACf/C,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,gCAAgC,CAAC;EAC1D,CAAC,EAAE,CAAC2C,OAAO,CAAC,CAAC;;EAEb;;EAEA,MAAM0E,UAAU,GAAGvK,WAAW,CAAC,MAAO0F,OAAe,IAAK;IACxD,IAAI,CAACA,OAAO,CAAC8E,IAAI,CAAC,CAAC,EAAE;IAErB5J,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,kBAAkBwC,OAAO,GAAG,CAAC;;IAEpD;IACA,IAAIX,kBAAkB,CAACS,OAAO,EAAE;MAC9B,MAAMiF,QAAQ,GAAG1F,kBAAkB,CAACS,OAAO;MAC3CT,kBAAkB,CAACS,OAAO,GAAG,IAAI;MACjCnC,aAAa,CAAC,IAAI,CAAC;MACnBE,aAAa,CAAC,2BAA2B,CAAC;MAC1CE,aAAa,CAAC,IAAI,CAAC;MACnBgH,QAAQ,CAAC/E,OAAO,CAAC;MACjB;IACF;;IAEA;IACArC,aAAa,CAAC,IAAI,CAAC;IACnBE,aAAa,CAAC,aAAa,CAAC;IAC5BE,aAAa,CAAC,IAAI,CAAC;IAEnB,IAAI;MACF;MACAoC,OAAO,CAACC,UAAU,CAAC3C,WAAW,CAACqC,OAAO,EAAE7D,MAAM,CAAC;MAE/C,MAAMyH,MAAM,GAAG,MAAMvD,OAAO,CAAC6E,OAAO,CAAChF,OAAO,CAAC;MAE7CjC,aAAa,CAAC2F,MAAM,CAAC;MACrBpH,QAAQ,GAAGoH,MAAM,CAAC;MAElBxI,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,WAAWkG,MAAM,CAAC3D,OAAO,GAAG,GAAG,GAAG,GAAG,IAAI2D,MAAM,CAAC1D,OAAO,EAAE,CAAC;IACnF,CAAC,CAAC,OAAOoB,KAAU,EAAE;MACnBlG,MAAM,CAACkG,KAAK,CAAC,SAAS,EAAE,mBAAmB,EAAEA,KAAK,CAAC;MACnDrD,aAAa,CAAC;QACZgC,OAAO,EAAE,KAAK;QACdC,OAAO,EAAE,UAAUoB,KAAK,CAACpB,OAAO,EAAE;QAClCC,KAAK,EAAE;MACT,CAAC,CAAC;IACJ,CAAC,SAAS;MACRtC,aAAa,CAAC,KAAK,CAAC;MACpBE,aAAa,CAAC,EAAE,CAAC;IACnB;EACF,CAAC,EAAE,CAACsC,OAAO,EAAElE,MAAM,EAAEK,QAAQ,CAAC,CAAC;;EAE/B;;EAEA,oBACEX,KAAA,CAACZ,YAAY,CAACkK,QAAQ;IAACC,KAAK,EAAE/E,OAAQ;IAAA9D,QAAA,gBACpCZ,IAAA,CAACd,IAAI;MAACwK,GAAG,EAAE1H,WAAY;MAAC2H,KAAK,EAAEC,MAAM,CAACC,IAAK;MAACC,WAAW,EAAE,KAAM;MAAAlJ,QAAA,EAC5DA;IAAQ,CACL,CAAC,eAGPZ,IAAA,CAACR,YAAY;MAACuK,OAAO,EAAE9H,UAAW;MAACE,UAAU,EAAEA;IAAW,CAAE,CAAC,EAG5DxB,WAAW,iBACVX,IAAA,CAACT,YAAY;MACXyK,MAAM,EAAEZ,UAAW;MACnBnH,UAAU,EAAEA,UAAW;MACvBI,UAAU,EAAEA,UAAW;MACvB5B,QAAQ,EAAEA,QAAS;MACnBwJ,SAAS,EAAEA,CAAA,KAAM3H,aAAa,CAAC,IAAI,CAAE;MACrCkB,cAAc,EAAEA,cAAe;MAC/BjB,IAAI,EAAEA,IAAK;MACX2H,YAAY,EAAGC,OAAO,IAAK;QACzB1K,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,gBAAgBQ,IAAI,MAAM4H,OAAO,EAAE,CAAC;QAC3D3H,OAAO,CAAC2H,OAAO,CAAC;MAClB,CAAE;MACF1H,WAAW,EAAEA,WAAY;MACzBE,cAAc,EAAEA,cAAe;MAC/BE,YAAY,EAAEA,YAAa;MAC3BuH,aAAa,EAAEjB,gBAAiB;MAChCpG,gBAAgB,EAAEA,gBAAiB;MACnCsH,WAAW,EAAGC,MAAM,IAAK;QACvB,IAAIA,MAAM,IAAI,CAACvH,gBAAgB,EAAE;UAC/BtD,MAAM,CAAC2G,IAAI,CAAC,SAAS,EAAE,oDAAoD,CAAC;UAC5E;QACF;QACA3G,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,eAAeuI,MAAM,GAAG,IAAI,GAAG,KAAK,EAAE,CAAC;QAC9D5H,cAAc,CAAC4H,MAAM,CAAC;QACtB,IAAIA,MAAM,EAAE;UACV7K,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wBAAwB,CAAC;UAChDmB,aAAa,CAACmB,OAAO,EAAEuC,KAAK,CAAC,CAAC,CAACf,IAAI,CAAEC,EAAE,IAAK;YAC1CrG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,4BAA4B+D,EAAE,EAAE,CAAC;UAC1D,CAAC,CAAC;QACJ,CAAC,MAAM;UACLrG,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,wBAAwB,CAAC;UAChDmB,aAAa,CAACmB,OAAO,EAAEyC,IAAI,CAAC,CAAC;QAC/B;MACF,CAAE;MACFyD,eAAe,EAAGC,KAAK,IAAK;QAC1B/K,MAAM,CAACsC,IAAI,CAAC,SAAS,EAAE,mBAAmByI,KAAK,GAAG,OAAO,GAAG,SAAS,EAAE,CAAC;QACxE5H,iBAAiB,CAAC4H,KAAK,CAAC;QACxB,IAAIA,KAAK,EAAE;UACTrH,cAAc,CAACkB,OAAO,EAAEoG,IAAI,CAAC,CAAC;QAChC,CAAC,MAAM;UACLtH,cAAc,CAACkB,OAAO,EAAEqG,MAAM,CAAC,CAAC;QAClC;MACF;IAAE,CAEH,CACF;EAAA,CACoB,CAAC;AAE5B;AAGA,MAAMd,MAAM,GAAGzK,UAAU,CAACwL,MAAM,CAAC;EAC/Bd,IAAI,EAAE;IACJe,IAAI,EAAE;EACR;AACF,CAAC,CAAC","ignoreList":[]}