opencode-smart-voice-notify 1.1.2 → 1.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +41 -2
- package/example.config.jsonc +65 -0
- package/index.js +134 -79
- package/package.json +1 -1
- package/util/ai-messages.js +205 -0
- package/util/config.js +48 -0
package/README.md
CHANGED
|
@@ -35,6 +35,13 @@ The plugin automatically tries multiple TTS engines in order, falling back if on
|
|
|
35
35
|
- **Permission Batching**: Multiple simultaneous permission requests are batched into a single notification (e.g., "5 permission requests require your attention")
|
|
36
36
|
- **Question Tool Support** (SDK v1.1.7+): Notifies when the agent asks questions and needs user input
|
|
37
37
|
|
|
38
|
+
### AI-Generated Messages (Experimental)
|
|
39
|
+
- **Dynamic notifications**: Use a local AI to generate unique, contextual messages instead of preset static ones
|
|
40
|
+
- **OpenAI-compatible**: Works with Ollama, LM Studio, LocalAI, vLLM, llama.cpp, Jan.ai, or any OpenAI-compatible endpoint
|
|
41
|
+
- **User-hosted**: You provide your own AI endpoint - no cloud API keys required
|
|
42
|
+
- **Custom prompts**: Configure prompts per notification type for full control over AI personality
|
|
43
|
+
- **Smart fallback**: Automatically falls back to static messages if AI is unavailable
|
|
44
|
+
|
|
38
45
|
### System Integration
|
|
39
46
|
- **Native Edge TTS**: No external dependencies (Python/pip) required
|
|
40
47
|
- Wake monitor from sleep before notifying
|
|
@@ -383,8 +390,40 @@ If you prefer to create the config manually, add a `smart-voice-notify.jsonc` fi
|
|
|
383
390
|
}
|
|
384
391
|
```
|
|
385
392
|
|
|
386
|
-
See `example.config.jsonc` for more details.
|
|
387
|
-
|
|
393
|
+
See `example.config.jsonc` for more details.
|
|
394
|
+
|
|
395
|
+
### AI Message Generation (Optional)
|
|
396
|
+
|
|
397
|
+
If you want dynamic, AI-generated notification messages instead of preset ones, you can connect to a local AI server:
|
|
398
|
+
|
|
399
|
+
1. **Install a local AI server** (e.g., [Ollama](https://ollama.ai)):
|
|
400
|
+
```bash
|
|
401
|
+
# Install Ollama and pull a model
|
|
402
|
+
ollama pull llama3
|
|
403
|
+
```
|
|
404
|
+
|
|
405
|
+
2. **Enable AI messages in your config**:
|
|
406
|
+
```jsonc
|
|
407
|
+
{
|
|
408
|
+
"enableAIMessages": true,
|
|
409
|
+
"aiEndpoint": "http://localhost:11434/v1",
|
|
410
|
+
"aiModel": "llama3",
|
|
411
|
+
"aiApiKey": "",
|
|
412
|
+
"aiFallbackToStatic": true
|
|
413
|
+
}
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
3. **The AI will generate unique messages** for each notification, which are then spoken by your TTS engine.
|
|
417
|
+
|
|
418
|
+
**Supported AI Servers:**
|
|
419
|
+
| Server | Default Endpoint | API Key |
|
|
420
|
+
|--------|-----------------|---------|
|
|
421
|
+
| Ollama | `http://localhost:11434/v1` | Not needed |
|
|
422
|
+
| LM Studio | `http://localhost:1234/v1` | Not needed |
|
|
423
|
+
| LocalAI | `http://localhost:8080/v1` | Not needed |
|
|
424
|
+
| vLLM | `http://localhost:8000/v1` | Use "EMPTY" |
|
|
425
|
+
| Jan.ai | `http://localhost:1337/v1` | Required |
|
|
426
|
+
|
|
388
427
|
## Requirements
|
|
389
428
|
|
|
390
429
|
### For ElevenLabs TTS
|
package/example.config.jsonc
CHANGED
|
@@ -250,6 +250,71 @@
|
|
|
250
250
|
// Question batch window (ms) - how long to wait for more questions before notifying
|
|
251
251
|
"questionBatchWindowMs": 800,
|
|
252
252
|
|
|
253
|
+
// ============================================================
|
|
254
|
+
// AI MESSAGE GENERATION (OpenAI-Compatible Endpoints)
|
|
255
|
+
// ============================================================
|
|
256
|
+
// Use a local/self-hosted AI to generate dynamic notification messages
|
|
257
|
+
// instead of using preset static messages. The AI generates the text,
|
|
258
|
+
// which is then spoken by your configured TTS engine (ElevenLabs, Edge, etc.)
|
|
259
|
+
//
|
|
260
|
+
// Supports: Ollama, LM Studio, LocalAI, vLLM, llama.cpp, Jan.ai, and any
|
|
261
|
+
// OpenAI-compatible endpoint. You provide your own endpoint URL and API key.
|
|
262
|
+
//
|
|
263
|
+
// HOW IT WORKS:
|
|
264
|
+
// 1. When a notification is triggered (task complete, permission needed, etc.)
|
|
265
|
+
// 2. If AI is enabled, the plugin sends a prompt to your AI server
|
|
266
|
+
// 3. The AI generates a unique, contextual notification message
|
|
267
|
+
// 4. That message is spoken by your TTS engine (ElevenLabs, Edge, SAPI)
|
|
268
|
+
// 5. If AI fails, it falls back to the static messages defined above
|
|
269
|
+
|
|
270
|
+
// Enable AI-generated messages (experimental feature)
|
|
271
|
+
// Default: false (uses static messages defined above)
|
|
272
|
+
"enableAIMessages": false,
|
|
273
|
+
|
|
274
|
+
// Your AI server endpoint URL
|
|
275
|
+
// Common local AI servers and their default endpoints:
|
|
276
|
+
// Ollama: http://localhost:11434/v1
|
|
277
|
+
// LM Studio: http://localhost:1234/v1
|
|
278
|
+
// LocalAI: http://localhost:8080/v1
|
|
279
|
+
// vLLM: http://localhost:8000/v1
|
|
280
|
+
// llama.cpp: http://localhost:8080/v1
|
|
281
|
+
// Jan.ai: http://localhost:1337/v1
|
|
282
|
+
// text-gen-webui: http://localhost:5000/v1
|
|
283
|
+
"aiEndpoint": "http://localhost:11434/v1",
|
|
284
|
+
|
|
285
|
+
// Model name to use (must match a model loaded in your AI server)
|
|
286
|
+
// Examples for Ollama: "llama3", "llama3.2", "mistral", "phi3", "gemma2", "qwen2"
|
|
287
|
+
// For LM Studio: Use the model name shown in the UI
|
|
288
|
+
"aiModel": "llama3",
|
|
289
|
+
|
|
290
|
+
// API key for your AI server
|
|
291
|
+
// Most local servers (Ollama, LM Studio, LocalAI) don't require a key - leave empty
|
|
292
|
+
// Only set this if your server requires authentication
|
|
293
|
+
// For vLLM with auth disabled, use "EMPTY"
|
|
294
|
+
"aiApiKey": "",
|
|
295
|
+
|
|
296
|
+
// Request timeout in milliseconds
|
|
297
|
+
// Local AI can be slow on first request (model loading), so 15 seconds is recommended
|
|
298
|
+
// Increase if you have a slower machine or larger models
|
|
299
|
+
"aiTimeout": 15000,
|
|
300
|
+
|
|
301
|
+
// Fall back to static messages (defined above) if AI generation fails
|
|
302
|
+
// Recommended: true - ensures notifications always work even if AI is down
|
|
303
|
+
"aiFallbackToStatic": true,
|
|
304
|
+
|
|
305
|
+
// Custom prompts for each notification type
|
|
306
|
+
// You can customize these to change the AI's personality/style
|
|
307
|
+
// The AI will generate a short message based on these prompts
|
|
308
|
+
// TIP: Keep prompts concise - they're sent with each notification
|
|
309
|
+
"aiPrompts": {
|
|
310
|
+
"idle": "Generate a single brief, friendly notification sentence (max 15 words) saying a coding task is complete. Be encouraging and warm. Output only the message, no quotes.",
|
|
311
|
+
"permission": "Generate a single brief, urgent but friendly notification sentence (max 15 words) asking the user to approve a permission request. Output only the message, no quotes.",
|
|
312
|
+
"question": "Generate a single brief, polite notification sentence (max 15 words) saying the assistant has a question and needs user input. Output only the message, no quotes.",
|
|
313
|
+
"idleReminder": "Generate a single brief, gentle reminder sentence (max 15 words) that a completed task is waiting for review. Be slightly more insistent. Output only the message, no quotes.",
|
|
314
|
+
"permissionReminder": "Generate a single brief, urgent reminder sentence (max 15 words) that permission approval is still needed. Convey importance. Output only the message, no quotes.",
|
|
315
|
+
"questionReminder": "Generate a single brief, polite but persistent reminder sentence (max 15 words) that a question is still waiting for an answer. Output only the message, no quotes."
|
|
316
|
+
},
|
|
317
|
+
|
|
253
318
|
// ============================================================
|
|
254
319
|
// SOUND FILES (For immediate notifications)
|
|
255
320
|
// These are played first before TTS reminder kicks in
|
package/index.js
CHANGED
|
@@ -2,6 +2,7 @@ import fs from 'fs';
|
|
|
2
2
|
import os from 'os';
|
|
3
3
|
import path from 'path';
|
|
4
4
|
import { createTTS, getTTSConfig } from './util/tts.js';
|
|
5
|
+
import { getSmartMessage } from './util/ai-messages.js';
|
|
5
6
|
|
|
6
7
|
/**
|
|
7
8
|
* OpenCode Smart Voice Notify Plugin
|
|
@@ -251,11 +252,11 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
251
252
|
const storedCount = reminder?.itemCount || 1;
|
|
252
253
|
let reminderMessage;
|
|
253
254
|
if (type === 'permission') {
|
|
254
|
-
reminderMessage = getPermissionMessage(storedCount, true);
|
|
255
|
+
reminderMessage = await getPermissionMessage(storedCount, true);
|
|
255
256
|
} else if (type === 'question') {
|
|
256
|
-
reminderMessage = getQuestionMessage(storedCount, true);
|
|
257
|
+
reminderMessage = await getQuestionMessage(storedCount, true);
|
|
257
258
|
} else {
|
|
258
|
-
reminderMessage =
|
|
259
|
+
reminderMessage = await getSmartMessage('idle', true, config.idleReminderTTSMessages);
|
|
259
260
|
}
|
|
260
261
|
|
|
261
262
|
// Check for ElevenLabs API key configuration issues
|
|
@@ -305,11 +306,11 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
305
306
|
const followUpStoredCount = followUpReminder?.itemCount || 1;
|
|
306
307
|
let followUpMessage;
|
|
307
308
|
if (type === 'permission') {
|
|
308
|
-
followUpMessage = getPermissionMessage(followUpStoredCount, true);
|
|
309
|
+
followUpMessage = await getPermissionMessage(followUpStoredCount, true);
|
|
309
310
|
} else if (type === 'question') {
|
|
310
|
-
followUpMessage = getQuestionMessage(followUpStoredCount, true);
|
|
311
|
+
followUpMessage = await getQuestionMessage(followUpStoredCount, true);
|
|
311
312
|
} else {
|
|
312
|
-
followUpMessage =
|
|
313
|
+
followUpMessage = await getSmartMessage('idle', true, config.idleReminderTTSMessages);
|
|
313
314
|
}
|
|
314
315
|
|
|
315
316
|
await tts.wakeMonitor();
|
|
@@ -356,8 +357,8 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
356
357
|
soundLoops = 1,
|
|
357
358
|
ttsMessage,
|
|
358
359
|
fallbackSound,
|
|
359
|
-
permissionCount
|
|
360
|
-
questionCount
|
|
360
|
+
permissionCount, // Support permission count for batched notifications
|
|
361
|
+
questionCount // Support question count for batched notifications
|
|
361
362
|
} = options;
|
|
362
363
|
|
|
363
364
|
// Step 1: Play the immediate sound notification
|
|
@@ -391,11 +392,11 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
391
392
|
if (config.notificationMode === 'tts-first' || config.notificationMode === 'both') {
|
|
392
393
|
let immediateMessage;
|
|
393
394
|
if (type === 'permission') {
|
|
394
|
-
immediateMessage =
|
|
395
|
+
immediateMessage = await getSmartMessage('permission', false, config.permissionTTSMessages);
|
|
395
396
|
} else if (type === 'question') {
|
|
396
|
-
immediateMessage =
|
|
397
|
+
immediateMessage = await getSmartMessage('question', false, config.questionTTSMessages);
|
|
397
398
|
} else {
|
|
398
|
-
immediateMessage =
|
|
399
|
+
immediateMessage = await getSmartMessage('idle', false, config.idleTTSMessages);
|
|
399
400
|
}
|
|
400
401
|
|
|
401
402
|
await tts.speak(immediateMessage, {
|
|
@@ -407,69 +408,88 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
407
408
|
|
|
408
409
|
/**
|
|
409
410
|
* Get a count-aware TTS message for permission requests
|
|
411
|
+
* Uses AI generation when enabled, falls back to static messages
|
|
410
412
|
* @param {number} count - Number of permission requests
|
|
411
413
|
* @param {boolean} isReminder - Whether this is a reminder message
|
|
412
|
-
* @returns {string} The formatted message
|
|
414
|
+
* @returns {Promise<string>} The formatted message
|
|
413
415
|
*/
|
|
414
|
-
const getPermissionMessage = (count, isReminder = false) => {
|
|
416
|
+
const getPermissionMessage = async (count, isReminder = false) => {
|
|
415
417
|
const messages = isReminder
|
|
416
418
|
? config.permissionReminderTTSMessages
|
|
417
419
|
: config.permissionTTSMessages;
|
|
418
420
|
|
|
421
|
+
// If AI messages are enabled, ALWAYS try AI first (regardless of count)
|
|
422
|
+
if (config.enableAIMessages) {
|
|
423
|
+
const aiMessage = await getSmartMessage('permission', isReminder, messages, { count, type: 'permission' });
|
|
424
|
+
// getSmartMessage returns static message as fallback, so if AI was attempted
|
|
425
|
+
// and succeeded, we'll get the AI message. If it failed, we get static.
|
|
426
|
+
// Check if we got a valid message (not the generic fallback)
|
|
427
|
+
if (aiMessage && aiMessage !== 'Notification') {
|
|
428
|
+
return aiMessage;
|
|
429
|
+
}
|
|
430
|
+
}
|
|
431
|
+
|
|
432
|
+
// Fallback to static messages (AI disabled or failed with generic fallback)
|
|
419
433
|
if (count === 1) {
|
|
420
|
-
// Single permission - use regular message
|
|
421
434
|
return getRandomMessage(messages);
|
|
422
435
|
} else {
|
|
423
|
-
// Multiple permissions - use count-aware messages if available, or format dynamically
|
|
424
436
|
const countMessages = isReminder
|
|
425
437
|
? config.permissionReminderTTSMessagesMultiple
|
|
426
438
|
: config.permissionTTSMessagesMultiple;
|
|
427
439
|
|
|
428
440
|
if (countMessages && countMessages.length > 0) {
|
|
429
|
-
// Use configured multi-permission messages (replace {count} placeholder)
|
|
430
441
|
const template = getRandomMessage(countMessages);
|
|
431
442
|
return template.replace('{count}', count.toString());
|
|
432
|
-
} else {
|
|
433
|
-
// Fallback: generate a dynamic message
|
|
434
|
-
return `Attention! There are ${count} permission requests waiting for your approval.`;
|
|
435
443
|
}
|
|
444
|
+
return `Attention! There are ${count} permission requests waiting for your approval.`;
|
|
436
445
|
}
|
|
437
446
|
};
|
|
438
447
|
|
|
439
448
|
/**
|
|
440
449
|
* Get a count-aware TTS message for question requests (SDK v1.1.7+)
|
|
450
|
+
* Uses AI generation when enabled, falls back to static messages
|
|
441
451
|
* @param {number} count - Number of question requests
|
|
442
452
|
* @param {boolean} isReminder - Whether this is a reminder message
|
|
443
|
-
* @returns {string} The formatted message
|
|
453
|
+
* @returns {Promise<string>} The formatted message
|
|
444
454
|
*/
|
|
445
|
-
const getQuestionMessage = (count, isReminder = false) => {
|
|
455
|
+
const getQuestionMessage = async (count, isReminder = false) => {
|
|
446
456
|
const messages = isReminder
|
|
447
457
|
? config.questionReminderTTSMessages
|
|
448
458
|
: config.questionTTSMessages;
|
|
449
459
|
|
|
460
|
+
// If AI messages are enabled, ALWAYS try AI first (regardless of count)
|
|
461
|
+
if (config.enableAIMessages) {
|
|
462
|
+
const aiMessage = await getSmartMessage('question', isReminder, messages, { count, type: 'question' });
|
|
463
|
+
// getSmartMessage returns static message as fallback, so if AI was attempted
|
|
464
|
+
// and succeeded, we'll get the AI message. If it failed, we get static.
|
|
465
|
+
// Check if we got a valid message (not the generic fallback)
|
|
466
|
+
if (aiMessage && aiMessage !== 'Notification') {
|
|
467
|
+
return aiMessage;
|
|
468
|
+
}
|
|
469
|
+
}
|
|
470
|
+
|
|
471
|
+
// Fallback to static messages (AI disabled or failed with generic fallback)
|
|
450
472
|
if (count === 1) {
|
|
451
|
-
// Single question - use regular message
|
|
452
473
|
return getRandomMessage(messages);
|
|
453
474
|
} else {
|
|
454
|
-
// Multiple questions - use count-aware messages if available, or format dynamically
|
|
455
475
|
const countMessages = isReminder
|
|
456
476
|
? config.questionReminderTTSMessagesMultiple
|
|
457
477
|
: config.questionTTSMessagesMultiple;
|
|
458
478
|
|
|
459
479
|
if (countMessages && countMessages.length > 0) {
|
|
460
|
-
// Use configured multi-question messages (replace {count} placeholder)
|
|
461
480
|
const template = getRandomMessage(countMessages);
|
|
462
481
|
return template.replace('{count}', count.toString());
|
|
463
|
-
} else {
|
|
464
|
-
// Fallback: generate a dynamic message
|
|
465
|
-
return `Hey! I have ${count} questions for you. Please check your screen.`;
|
|
466
482
|
}
|
|
483
|
+
return `Hey! I have ${count} questions for you. Please check your screen.`;
|
|
467
484
|
}
|
|
468
485
|
};
|
|
469
486
|
|
|
470
487
|
/**
|
|
471
488
|
* Process the batched permission requests as a single notification
|
|
472
489
|
* Called after the batch window expires
|
|
490
|
+
*
|
|
491
|
+
* FIX: Play sound IMMEDIATELY before any AI generation to avoid delay.
|
|
492
|
+
* AI message generation can take 3-15+ seconds, which was delaying sound playback.
|
|
473
493
|
*/
|
|
474
494
|
const processPermissionBatch = async () => {
|
|
475
495
|
// Capture and clear the batch
|
|
@@ -489,40 +509,42 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
489
509
|
// We track all IDs in the batch for proper cleanup
|
|
490
510
|
activePermissionId = batch[0];
|
|
491
511
|
|
|
492
|
-
// Show toast
|
|
512
|
+
// Step 1: Show toast IMMEDIATELY (fire and forget - no await)
|
|
493
513
|
const toastMessage = batchCount === 1
|
|
494
514
|
? "⚠️ Permission request requires your attention"
|
|
495
515
|
: `⚠️ ${batchCount} permission requests require your attention`;
|
|
496
|
-
|
|
516
|
+
showToast(toastMessage, "warning", 8000); // No await - instant display
|
|
517
|
+
|
|
518
|
+
// Step 2: Play sound (after toast is triggered)
|
|
519
|
+
const soundLoops = batchCount === 1 ? 2 : Math.min(3, batchCount);
|
|
520
|
+
await playSound(config.permissionSound, soundLoops);
|
|
497
521
|
|
|
498
|
-
// CHECK: Did user already respond while
|
|
522
|
+
// CHECK: Did user already respond while sound was playing?
|
|
499
523
|
if (pendingPermissionBatch.length > 0) {
|
|
500
|
-
// New permissions arrived during
|
|
501
|
-
debugLog('processPermissionBatch: new permissions arrived during
|
|
524
|
+
// New permissions arrived during sound - they'll be handled in next batch
|
|
525
|
+
debugLog('processPermissionBatch: new permissions arrived during sound');
|
|
502
526
|
}
|
|
503
527
|
|
|
504
|
-
// Check
|
|
528
|
+
// Step 3: Check race condition - did user respond during sound?
|
|
505
529
|
if (activePermissionId === null) {
|
|
506
|
-
debugLog('processPermissionBatch:
|
|
530
|
+
debugLog('processPermissionBatch: user responded during sound - aborting');
|
|
507
531
|
return;
|
|
508
532
|
}
|
|
509
533
|
|
|
510
|
-
//
|
|
511
|
-
const
|
|
512
|
-
const reminderMessage = getPermissionMessage(batchCount, true);
|
|
513
|
-
|
|
514
|
-
// Smart notification: sound first, TTS reminder later
|
|
515
|
-
await smartNotify('permission', {
|
|
516
|
-
soundFile: config.permissionSound,
|
|
517
|
-
soundLoops: batchCount === 1 ? 2 : Math.min(3, batchCount), // More loops for more permissions
|
|
518
|
-
ttsMessage: reminderMessage,
|
|
519
|
-
fallbackSound: config.permissionSound,
|
|
520
|
-
// Pass count for potential use in notification
|
|
521
|
-
permissionCount: batchCount
|
|
522
|
-
});
|
|
534
|
+
// Step 4: Generate AI message for reminder AFTER sound played
|
|
535
|
+
const reminderMessage = await getPermissionMessage(batchCount, true);
|
|
523
536
|
|
|
524
|
-
//
|
|
537
|
+
// Step 5: Schedule TTS reminder if enabled
|
|
538
|
+
if (config.enableTTSReminder && reminderMessage) {
|
|
539
|
+
scheduleTTSReminder('permission', reminderMessage, {
|
|
540
|
+
fallbackSound: config.permissionSound,
|
|
541
|
+
permissionCount: batchCount
|
|
542
|
+
});
|
|
543
|
+
}
|
|
544
|
+
|
|
545
|
+
// Step 6: If TTS-first or both mode, generate and speak immediate message
|
|
525
546
|
if (config.notificationMode === 'tts-first' || config.notificationMode === 'both') {
|
|
547
|
+
const ttsMessage = await getPermissionMessage(batchCount, false);
|
|
526
548
|
await tts.wakeMonitor();
|
|
527
549
|
await tts.forceVolume();
|
|
528
550
|
await tts.speak(ttsMessage, {
|
|
@@ -541,6 +563,9 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
541
563
|
/**
|
|
542
564
|
* Process the batched question requests as a single notification (SDK v1.1.7+)
|
|
543
565
|
* Called after the batch window expires
|
|
566
|
+
*
|
|
567
|
+
* FIX: Play sound IMMEDIATELY before any AI generation to avoid delay.
|
|
568
|
+
* AI message generation can take 3-15+ seconds, which was delaying sound playback.
|
|
544
569
|
*/
|
|
545
570
|
const processQuestionBatch = async () => {
|
|
546
571
|
// Capture and clear the batch
|
|
@@ -563,41 +588,41 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
563
588
|
// We track all IDs in the batch for proper cleanup
|
|
564
589
|
activeQuestionId = batch[0]?.id;
|
|
565
590
|
|
|
566
|
-
// Show toast
|
|
591
|
+
// Step 1: Show toast IMMEDIATELY (fire and forget - no await)
|
|
567
592
|
const toastMessage = totalQuestionCount === 1
|
|
568
593
|
? "❓ The agent has a question for you"
|
|
569
594
|
: `❓ The agent has ${totalQuestionCount} questions for you`;
|
|
570
|
-
|
|
595
|
+
showToast(toastMessage, "info", 8000); // No await - instant display
|
|
596
|
+
|
|
597
|
+
// Step 2: Play sound (after toast is triggered)
|
|
598
|
+
await playSound(config.questionSound, 2);
|
|
571
599
|
|
|
572
|
-
// CHECK: Did user already respond while
|
|
600
|
+
// CHECK: Did user already respond while sound was playing?
|
|
573
601
|
if (pendingQuestionBatch.length > 0) {
|
|
574
|
-
// New questions arrived during
|
|
575
|
-
debugLog('processQuestionBatch: new questions arrived during
|
|
602
|
+
// New questions arrived during sound - they'll be handled in next batch
|
|
603
|
+
debugLog('processQuestionBatch: new questions arrived during sound');
|
|
576
604
|
}
|
|
577
605
|
|
|
578
|
-
// Check
|
|
606
|
+
// Step 3: Check race condition - did user respond during sound?
|
|
579
607
|
if (activeQuestionId === null) {
|
|
580
|
-
debugLog('processQuestionBatch:
|
|
608
|
+
debugLog('processQuestionBatch: user responded during sound - aborting');
|
|
581
609
|
return;
|
|
582
610
|
}
|
|
583
611
|
|
|
584
|
-
//
|
|
585
|
-
const
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
fallbackSound: config.questionSound,
|
|
595
|
-
// Pass count for use in reminders
|
|
596
|
-
questionCount: totalQuestionCount
|
|
597
|
-
});
|
|
612
|
+
// Step 4: Generate AI message for reminder AFTER sound played
|
|
613
|
+
const reminderMessage = await getQuestionMessage(totalQuestionCount, true);
|
|
614
|
+
|
|
615
|
+
// Step 5: Schedule TTS reminder if enabled
|
|
616
|
+
if (config.enableTTSReminder && reminderMessage) {
|
|
617
|
+
scheduleTTSReminder('question', reminderMessage, {
|
|
618
|
+
fallbackSound: config.questionSound,
|
|
619
|
+
questionCount: totalQuestionCount
|
|
620
|
+
});
|
|
621
|
+
}
|
|
598
622
|
|
|
599
|
-
//
|
|
623
|
+
// Step 6: If TTS-first or both mode, generate and speak immediate message
|
|
600
624
|
if (config.notificationMode === 'tts-first' || config.notificationMode === 'both') {
|
|
625
|
+
const ttsMessage = await getQuestionMessage(totalQuestionCount, false);
|
|
601
626
|
await tts.wakeMonitor();
|
|
602
627
|
await tts.forceVolume();
|
|
603
628
|
await tts.speak(ttsMessage, {
|
|
@@ -736,6 +761,9 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
736
761
|
|
|
737
762
|
// ========================================
|
|
738
763
|
// NOTIFICATION 1: Session Idle (Agent Finished)
|
|
764
|
+
//
|
|
765
|
+
// FIX: Play sound IMMEDIATELY before any AI generation to avoid delay.
|
|
766
|
+
// AI message generation can take 3-15+ seconds, which was delaying sound playback.
|
|
739
767
|
// ========================================
|
|
740
768
|
if (event.type === "session.idle") {
|
|
741
769
|
const sessionID = event.properties?.sessionID;
|
|
@@ -753,15 +781,42 @@ export default async function SmartVoiceNotifyPlugin({ project, client, $, direc
|
|
|
753
781
|
lastSessionIdleTime = Date.now();
|
|
754
782
|
|
|
755
783
|
debugLog(`session.idle: notifying for session ${sessionID} (idleTime=${lastSessionIdleTime})`);
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
763
|
-
|
|
764
|
-
}
|
|
784
|
+
|
|
785
|
+
// Step 1: Show toast IMMEDIATELY (fire and forget - no await)
|
|
786
|
+
showToast("✅ Agent has finished working", "success", 5000); // No await - instant display
|
|
787
|
+
|
|
788
|
+
// Step 2: Play sound (after toast is triggered)
|
|
789
|
+
// Only play sound in sound-first, sound-only, or both mode
|
|
790
|
+
if (config.notificationMode !== 'tts-first') {
|
|
791
|
+
await playSound(config.idleSound, 1);
|
|
792
|
+
}
|
|
793
|
+
|
|
794
|
+
// Step 3: Check race condition - did user respond during sound?
|
|
795
|
+
if (lastUserActivityTime > lastSessionIdleTime) {
|
|
796
|
+
debugLog(`session.idle: user active during sound - aborting`);
|
|
797
|
+
return;
|
|
798
|
+
}
|
|
799
|
+
|
|
800
|
+
// Step 4: Generate AI message for reminder AFTER sound played
|
|
801
|
+
const reminderMessage = await getSmartMessage('idle', true, config.idleReminderTTSMessages);
|
|
802
|
+
|
|
803
|
+
// Step 5: Schedule TTS reminder if enabled
|
|
804
|
+
if (config.enableTTSReminder && reminderMessage) {
|
|
805
|
+
scheduleTTSReminder('idle', reminderMessage, {
|
|
806
|
+
fallbackSound: config.idleSound
|
|
807
|
+
});
|
|
808
|
+
}
|
|
809
|
+
|
|
810
|
+
// Step 6: If TTS-first or both mode, generate and speak immediate message
|
|
811
|
+
if (config.notificationMode === 'tts-first' || config.notificationMode === 'both') {
|
|
812
|
+
const ttsMessage = await getSmartMessage('idle', false, config.idleTTSMessages);
|
|
813
|
+
await tts.wakeMonitor();
|
|
814
|
+
await tts.forceVolume();
|
|
815
|
+
await tts.speak(ttsMessage, {
|
|
816
|
+
enableTTS: true,
|
|
817
|
+
fallbackSound: config.idleSound
|
|
818
|
+
});
|
|
819
|
+
}
|
|
765
820
|
}
|
|
766
821
|
|
|
767
822
|
// ========================================
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "opencode-smart-voice-notify",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.2.2",
|
|
4
4
|
"description": "Smart voice notification plugin for OpenCode with multiple TTS engines (ElevenLabs, Edge TTS, Windows SAPI) and intelligent reminder system",
|
|
5
5
|
"main": "index.js",
|
|
6
6
|
"type": "module",
|
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* AI Message Generation Module
|
|
3
|
+
*
|
|
4
|
+
* Generates dynamic notification messages using OpenAI-compatible AI endpoints.
|
|
5
|
+
* Supports: Ollama, LM Studio, LocalAI, vLLM, llama.cpp, Jan.ai, etc.
|
|
6
|
+
*
|
|
7
|
+
* Uses native fetch() - no external dependencies required.
|
|
8
|
+
*/
|
|
9
|
+
|
|
10
|
+
import { getTTSConfig } from './tts.js';
|
|
11
|
+
|
|
12
|
+
/**
|
|
13
|
+
* Generate a message using an OpenAI-compatible AI endpoint
|
|
14
|
+
* @param {string} promptType - The type of prompt ('idle', 'permission', 'question', 'idleReminder', 'permissionReminder', 'questionReminder')
|
|
15
|
+
* @param {object} context - Optional context about the notification (for future use)
|
|
16
|
+
* @returns {Promise<string|null>} Generated message or null if failed
|
|
17
|
+
*/
|
|
18
|
+
export async function generateAIMessage(promptType, context = {}) {
|
|
19
|
+
const config = getTTSConfig();
|
|
20
|
+
|
|
21
|
+
// Check if AI messages are enabled
|
|
22
|
+
if (!config.enableAIMessages) {
|
|
23
|
+
return null;
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
// Get the prompt for this type
|
|
27
|
+
let prompt = config.aiPrompts?.[promptType];
|
|
28
|
+
if (!prompt) {
|
|
29
|
+
return null;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
// Inject count context if multiple items
|
|
33
|
+
if (context.count && context.count > 1) {
|
|
34
|
+
// Use type-specific terminology
|
|
35
|
+
let itemType = 'items';
|
|
36
|
+
if (context.type === 'question') {
|
|
37
|
+
itemType = 'questions';
|
|
38
|
+
} else if (context.type === 'permission') {
|
|
39
|
+
itemType = 'permission requests';
|
|
40
|
+
}
|
|
41
|
+
prompt = `${prompt} Important: There are ${context.count} ${itemType} (not just one) waiting for the user's attention. Mention the count in your message.`;
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
try {
|
|
45
|
+
// Build headers
|
|
46
|
+
const headers = { 'Content-Type': 'application/json' };
|
|
47
|
+
if (config.aiApiKey) {
|
|
48
|
+
headers['Authorization'] = `Bearer ${config.aiApiKey}`;
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
// Build endpoint URL (ensure it ends with /chat/completions)
|
|
52
|
+
let endpoint = config.aiEndpoint || 'http://localhost:11434/v1';
|
|
53
|
+
if (!endpoint.endsWith('/chat/completions')) {
|
|
54
|
+
endpoint = endpoint.replace(/\/$/, '') + '/chat/completions';
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
// Create abort controller for timeout
|
|
58
|
+
const controller = new AbortController();
|
|
59
|
+
const timeout = setTimeout(() => controller.abort(), config.aiTimeout || 15000);
|
|
60
|
+
|
|
61
|
+
// Make the request
|
|
62
|
+
const response = await fetch(endpoint, {
|
|
63
|
+
method: 'POST',
|
|
64
|
+
headers,
|
|
65
|
+
signal: controller.signal,
|
|
66
|
+
body: JSON.stringify({
|
|
67
|
+
model: config.aiModel || 'llama3',
|
|
68
|
+
messages: [
|
|
69
|
+
{
|
|
70
|
+
role: 'system',
|
|
71
|
+
content: 'You are a helpful assistant that generates short notification messages. Output only the message text, nothing else. No quotes, no explanations.'
|
|
72
|
+
},
|
|
73
|
+
{
|
|
74
|
+
role: 'user',
|
|
75
|
+
content: prompt
|
|
76
|
+
}
|
|
77
|
+
],
|
|
78
|
+
max_tokens: 1000, // High value to accommodate thinking models (e.g., Gemini 2.5) that use internal reasoning tokens
|
|
79
|
+
temperature: 0.7
|
|
80
|
+
})
|
|
81
|
+
});
|
|
82
|
+
|
|
83
|
+
clearTimeout(timeout);
|
|
84
|
+
|
|
85
|
+
if (!response.ok) {
|
|
86
|
+
return null;
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
const data = await response.json();
|
|
90
|
+
|
|
91
|
+
// Extract the message content
|
|
92
|
+
const message = data.choices?.[0]?.message?.content?.trim();
|
|
93
|
+
|
|
94
|
+
if (!message) {
|
|
95
|
+
return null;
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
// Clean up the message (remove quotes if AI added them)
|
|
99
|
+
let cleanMessage = message.replace(/^["']|["']$/g, '').trim();
|
|
100
|
+
|
|
101
|
+
// Validate message length (sanity check)
|
|
102
|
+
if (cleanMessage.length < 5 || cleanMessage.length > 200) {
|
|
103
|
+
return null;
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
return cleanMessage;
|
|
107
|
+
|
|
108
|
+
} catch (error) {
|
|
109
|
+
return null;
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
/**
|
|
114
|
+
* Get a smart message - tries AI first, falls back to static messages
|
|
115
|
+
* @param {string} eventType - 'idle', 'permission', 'question'
|
|
116
|
+
* @param {boolean} isReminder - Whether this is a reminder message
|
|
117
|
+
* @param {string[]} staticMessages - Array of static fallback messages
|
|
118
|
+
* @param {object} context - Optional context (e.g., { count: 3 } for batched notifications)
|
|
119
|
+
* @returns {Promise<string>} The message to speak
|
|
120
|
+
*/
|
|
121
|
+
export async function getSmartMessage(eventType, isReminder, staticMessages, context = {}) {
|
|
122
|
+
const config = getTTSConfig();
|
|
123
|
+
|
|
124
|
+
// Determine the prompt type
|
|
125
|
+
const promptType = isReminder ? `${eventType}Reminder` : eventType;
|
|
126
|
+
|
|
127
|
+
// Try AI generation if enabled
|
|
128
|
+
if (config.enableAIMessages) {
|
|
129
|
+
try {
|
|
130
|
+
const aiMessage = await generateAIMessage(promptType, context);
|
|
131
|
+
if (aiMessage) {
|
|
132
|
+
return aiMessage;
|
|
133
|
+
}
|
|
134
|
+
} catch (error) {
|
|
135
|
+
// Silently fall through to fallback
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
// Check if fallback is disabled
|
|
139
|
+
if (!config.aiFallbackToStatic) {
|
|
140
|
+
// Return a generic message if fallback disabled and AI failed
|
|
141
|
+
return 'Notification: Please check your screen.';
|
|
142
|
+
}
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
// Fallback to static messages
|
|
146
|
+
if (!Array.isArray(staticMessages) || staticMessages.length === 0) {
|
|
147
|
+
return 'Notification';
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
return staticMessages[Math.floor(Math.random() * staticMessages.length)];
|
|
151
|
+
}
|
|
152
|
+
|
|
153
|
+
/**
|
|
154
|
+
* Test connectivity to the AI endpoint
|
|
155
|
+
* @returns {Promise<{success: boolean, message: string, model?: string}>}
|
|
156
|
+
*/
|
|
157
|
+
export async function testAIConnection() {
|
|
158
|
+
const config = getTTSConfig();
|
|
159
|
+
|
|
160
|
+
if (!config.enableAIMessages) {
|
|
161
|
+
return { success: false, message: 'AI messages not enabled' };
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
try {
|
|
165
|
+
const headers = { 'Content-Type': 'application/json' };
|
|
166
|
+
if (config.aiApiKey) {
|
|
167
|
+
headers['Authorization'] = `Bearer ${config.aiApiKey}`;
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
// Try to list models (simpler endpoint to test connectivity)
|
|
171
|
+
let endpoint = config.aiEndpoint || 'http://localhost:11434/v1';
|
|
172
|
+
endpoint = endpoint.replace(/\/$/, '') + '/models';
|
|
173
|
+
|
|
174
|
+
const controller = new AbortController();
|
|
175
|
+
const timeout = setTimeout(() => controller.abort(), 5000);
|
|
176
|
+
|
|
177
|
+
const response = await fetch(endpoint, {
|
|
178
|
+
method: 'GET',
|
|
179
|
+
headers,
|
|
180
|
+
signal: controller.signal
|
|
181
|
+
});
|
|
182
|
+
|
|
183
|
+
clearTimeout(timeout);
|
|
184
|
+
|
|
185
|
+
if (response.ok) {
|
|
186
|
+
const data = await response.json();
|
|
187
|
+
const models = data.data?.map(m => m.id) || [];
|
|
188
|
+
return {
|
|
189
|
+
success: true,
|
|
190
|
+
message: `Connected! Available models: ${models.slice(0, 3).join(', ')}${models.length > 3 ? '...' : ''}`,
|
|
191
|
+
models
|
|
192
|
+
};
|
|
193
|
+
} else {
|
|
194
|
+
return { success: false, message: `HTTP ${response.status}: ${response.statusText}` };
|
|
195
|
+
}
|
|
196
|
+
|
|
197
|
+
} catch (error) {
|
|
198
|
+
if (error.name === 'AbortError') {
|
|
199
|
+
return { success: false, message: 'Connection timed out' };
|
|
200
|
+
}
|
|
201
|
+
return { success: false, message: error.message };
|
|
202
|
+
}
|
|
203
|
+
}
|
|
204
|
+
|
|
205
|
+
export default { generateAIMessage, getSmartMessage, testAIConnection };
|
package/util/config.js
CHANGED
|
@@ -297,6 +297,54 @@ const generateDefaultConfig = (overrides = {}, version = '1.0.0') => {
|
|
|
297
297
|
// Question batch window (ms) - how long to wait for more questions before notifying
|
|
298
298
|
"questionBatchWindowMs": ${overrides.questionBatchWindowMs !== undefined ? overrides.questionBatchWindowMs : 800},
|
|
299
299
|
|
|
300
|
+
// ============================================================
|
|
301
|
+
// AI MESSAGE GENERATION (OpenAI-Compatible Endpoints)
|
|
302
|
+
// ============================================================
|
|
303
|
+
// Use a local/self-hosted AI to generate dynamic notification messages
|
|
304
|
+
// instead of using preset static messages. The AI generates the text,
|
|
305
|
+
// which is then spoken by your configured TTS engine (ElevenLabs, Edge, etc.)
|
|
306
|
+
//
|
|
307
|
+
// Supports: Ollama, LM Studio, LocalAI, vLLM, llama.cpp, Jan.ai, and any
|
|
308
|
+
// OpenAI-compatible endpoint. You provide your own endpoint URL and API key.
|
|
309
|
+
|
|
310
|
+
// Enable AI-generated messages (experimental feature)
|
|
311
|
+
"enableAIMessages": ${overrides.enableAIMessages !== undefined ? overrides.enableAIMessages : false},
|
|
312
|
+
|
|
313
|
+
// Your AI server endpoint URL (e.g., Ollama: http://localhost:11434/v1)
|
|
314
|
+
// Common endpoints:
|
|
315
|
+
// Ollama: http://localhost:11434/v1
|
|
316
|
+
// LM Studio: http://localhost:1234/v1
|
|
317
|
+
// LocalAI: http://localhost:8080/v1
|
|
318
|
+
// vLLM: http://localhost:8000/v1
|
|
319
|
+
// Jan.ai: http://localhost:1337/v1
|
|
320
|
+
"aiEndpoint": "${overrides.aiEndpoint || 'http://localhost:11434/v1'}",
|
|
321
|
+
|
|
322
|
+
// Model name to use (depends on what's loaded in your AI server)
|
|
323
|
+
// Examples: "llama3", "mistral", "phi3", "gemma2", "qwen2"
|
|
324
|
+
"aiModel": "${overrides.aiModel || 'llama3'}",
|
|
325
|
+
|
|
326
|
+
// API key for your AI server (leave empty for Ollama/LM Studio/LocalAI)
|
|
327
|
+
// Only needed if your server requires authentication
|
|
328
|
+
"aiApiKey": "${overrides.aiApiKey || ''}",
|
|
329
|
+
|
|
330
|
+
// Request timeout in milliseconds (local AI can be slow on first request)
|
|
331
|
+
"aiTimeout": ${overrides.aiTimeout !== undefined ? overrides.aiTimeout : 15000},
|
|
332
|
+
|
|
333
|
+
// Fallback to static preset messages if AI generation fails
|
|
334
|
+
"aiFallbackToStatic": ${overrides.aiFallbackToStatic !== undefined ? overrides.aiFallbackToStatic : true},
|
|
335
|
+
|
|
336
|
+
// Custom prompts for each notification type
|
|
337
|
+
// The AI will generate a short message based on these prompts
|
|
338
|
+
// Keep prompts concise - they're sent with each notification
|
|
339
|
+
"aiPrompts": ${formatJSON(overrides.aiPrompts || {
|
|
340
|
+
"idle": "Generate a single brief, friendly notification sentence (max 15 words) saying a coding task is complete. Be encouraging and warm. Output only the message, no quotes.",
|
|
341
|
+
"permission": "Generate a single brief, urgent but friendly notification sentence (max 15 words) asking the user to approve a permission request. Output only the message, no quotes.",
|
|
342
|
+
"question": "Generate a single brief, polite notification sentence (max 15 words) saying the assistant has a question and needs user input. Output only the message, no quotes.",
|
|
343
|
+
"idleReminder": "Generate a single brief, gentle reminder sentence (max 15 words) that a completed task is waiting for review. Be slightly more insistent. Output only the message, no quotes.",
|
|
344
|
+
"permissionReminder": "Generate a single brief, urgent reminder sentence (max 15 words) that permission approval is still needed. Convey importance. Output only the message, no quotes.",
|
|
345
|
+
"questionReminder": "Generate a single brief, polite but persistent reminder sentence (max 15 words) that a question is still waiting for an answer. Output only the message, no quotes."
|
|
346
|
+
}, 4)},
|
|
347
|
+
|
|
300
348
|
// ============================================================
|
|
301
349
|
// SOUND FILES (For immediate notifications)
|
|
302
350
|
// These are played first before TTS reminder kicks in
|