vibes-plug 2.11.0 → 3.9.0

package/skills/voice-ai-realtime-agent/SKILL.md

@@ -1,202 +1,243 @@
----
-name: voice-ai-realtime-agent
-description: "Expert guide for Ultra-Low Latency Conversational Voice AI (<300ms), WebRTC bidirectional streaming, OpenAI Realtime API, Gemini Multimodal Live Audio, LiveKit Agents, and Semantic VAD / Panduan ahli AI suara percakapan real-time berlatensi ultra-rendah."
-author: "Roedy Rustam"
----
-
-# Voice AI Realtime Agent (2026 Edition)
-
-Expert guide for building ultra-low-latency (<300ms), bi-directional conversational voice AI applications. Covers WebRTC, full-duplex WebSocket audio streaming (PCM 24kHz), OpenAI Realtime API, Gemini Multimodal Live API, LiveKit Agents SDK, and smart interruption (barge-in) handling.
-
-*Panduan ahli untuk membangun aplikasi AI suara percakapan dua arah berlatensi ultra-rendah (<300ms) menggunakan WebRTC, WebSocket full-duplex, OpenAI Realtime API, Gemini Multimodal Live API, LiveKit Agents SDK, dan penanganan interupsi (barge-in).*
-
----
-
-## 1. Core Architecture: Full-Duplex Speech-to-Speech
-
-Traditional voice pipelines chain STT ➔ LLM ➔ TTS with cumulative latency exceeding 1,200ms–2,500ms. Modern 2026 voice agents use **native speech-to-speech** or **streamable full-duplex WebRTC pipelines** achieving natural, human-like reaction times (~250–350ms).
-
-```
-User Mic ──► [WebRTC / WebSocket] ──► [VAD: Silero / WebRTC VAD]
-                                                │
-                                                ▼
-User Speaks <── [Audio Output] ◄── [Native Audio Stream / Cartesia] ◄── [OpenAI Realtime / Gemini Live]
-     │
-     └── User Interrupts (Barge-in) ──► Instant Buffer Flush & Cancel Audio Frame Emission
-```
-
----
-
-## 2. Production Recipe: LiveKit Agents + OpenAI Realtime (Python)
-
-```python
-# agent.py - Production Voice Agent Worker with LiveKit & OpenAI Realtime
-import asyncio
-import os
-from livekit import rtc
-from livekit.agents import (
-    AutoSubscribe,
-    JobContext,
-    JobProcess,
-    WorkerOptions,
-    cli,
-    llm,
-)
-from livekit.agents.pipeline import VoicePipelineAgent
-from livekit.plugins import deepgram, openai, silero
-
-async def entrypoint(ctx: JobContext):
-    # Connect to room with audio only to minimize bandwidth & latency
-    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
-    
-    # Wait for the user participant to join
-    participant = await ctx.wait_for_participant()
-    
-    # Define agent instructions and tools
-    initial_ctx = llm.ChatContext().append(
-        role="system",
-        text=(
-            "You are a helpful, concise voice assistant. "
-            "Respond naturally in 1-2 short sentences. Never output markdown, bullet points, or emojis."
-        )
-    )
-
-    # Realtime Voice Pipeline: Deepgram (STT) + OpenAI (LLM) + Cartesia/OpenAI (TTS)
-    # Or use native OpenAI Realtime Model: gpt-4o-realtime-preview
-    agent = VoicePipelineAgent(
-        vad=silero.VAD.load(
-            min_speech_duration=0.1,
-            min_silence_duration=0.3, # Snappy turn-taking
-            prefix_padding_duration=0.2,
-        ),
-        stt=deepgram.STT(model="nova-2", language="id"), # Multi-language support
-        llm=openai.LLM(model="gpt-4o-mini"),
-        tts=openai.TTS(voice="alloy"),
-        chat_ctx=initial_ctx,
-        allow_interruptions=True, # Barge-in capability
-        interrupt_speech_duration=0.3, # Immediate cutoff when user talks
-    )
-
-    agent.start(ctx.room, participant)
-    
-    # Greet user immediately
-    await agent.say("Halo! Ada yang bisa saya bantu hari ini?", now=True)
-
-if __name__ == "__main__":
-    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
-```
-
----
-
-## 3. Production Recipe: Gemini Multimodal Live Audio (TypeScript / Node.js)
-
-```typescript
-// gemini-live-audio.ts - Bidirectional WebSocket PCM 24kHz
-import WebSocket from 'ws';
-
-interface GeminiAudioConfig {
-  apiKey: string;
-  model?: string;
-  systemInstruction?: string;
-}
-
-export class GeminiVoiceAgent {
-  private ws: WebSocket | null = null;
-  private isConnected = false;
-
-  constructor(private config: GeminiAudioConfig) {}
-
-  public async connect(): Promise<void> {
-    const url = `wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1alpha.GenerativeService.BidiGenerateContent?key=${this.config.apiKey}`;
-    
-    this.ws = new WebSocket(url);
-
-    this.ws.on('open', () => {
-      this.isConnected = true;
-      this.sendInitialHandshake();
-    });
-
-    this.ws.on('message', (data: WebSocket.Data) => {
-      this.handleIncomingAudio(data);
-    });
-  }
-
-  private sendInitialHandshake(): void {
-    const setupMessage = {
-      setup: {
-        model: `models/${this.config.model || 'gemini-2.0-flash-exp'}`,
-        generationConfig: {
-          responseModalities: ["AUDIO"],
-          speechConfig: {
-            voiceConfig: {
-              prebuiltVoiceConfig: { voiceName: "Puck" }
-            }
-          }
-        },
-        systemInstruction: {
-          parts: [{ text: this.config.systemInstruction || "You are a conversational voice agent. Keep answers brief." }]
-        }
-      }
-    };
-    this.ws?.send(JSON.stringify(setupMessage));
-  }
-
-  // Stream raw PCM 16-bit 24kHz mono audio from mic
-  public sendAudioChunk(pcm16Chunk: Buffer): void {
-    if (!this.isConnected || !this.ws) return;
-
-    const base64Audio = pcm16Chunk.toString('base64');
-    const msg = {
-      realtimeInput: {
-        mediaChunks: [
-          {
-            mimeType: "audio/pcm;rate=24000",
-            data: base64Audio
-          }
-        ]
-      }
-    };
-    this.ws.send(JSON.stringify(msg));
-  }
-
-  private handleIncomingAudio(data: WebSocket.Data): void {
-    try {
-      const response = JSON.parse(data.toString());
-      const parts = response.serverContent?.modelTurn?.parts;
-      if (parts) {
-        for (const part of parts) {
-          if (part.inlineData?.data) {
-            const pcmBuffer = Buffer.from(part.inlineData.data, 'base64');
-            this.playAudioSpeaker(pcmBuffer);
-          }
-        }
-      }
-    } catch {
-      // Binary PCM frame handler
-    }
-  }
-
-  private playAudioSpeaker(pcmChunk: Buffer): void {
-    // Send to WebRTC audio track or audio output device
-  }
-}
-```
-
----
-
-## 4. Key 2026 Performance Guardrails
-
-1. **Barge-in Latency Budget (<150ms)**: When the user speaks while the bot is talking, cancel outgoing audio immediately. Do not wait for the LLM to finish streaming its chunk.
-2. **Audio Sample Rates**:
-   - Mic Input: 16kHz or 24kHz 16-bit Linear PCM Mono.
-   - Bot Output: 24kHz PCM for crystal-clear natural prosody.
-3. **Turn-Taking Jitter Prevention**: Use minimum silence thresholds between `300ms` and `450ms`. Lower thresholds cause the bot to interrupt users when they pause to think; higher thresholds make the conversation feel robotic.
-
----
-
-## Orchestration & Integration
-
-- **`ai-llm-integration-expert`**: For base LLM prompt routing and function calling during conversation.
-- **`realtime-collaboration-expert`**: For syncing WebRTC tracks and room states with client applications.
-- **`gemini-agent-booster`**: Connects Gemini 3.x / 2.0 Flash thinking models to live voice agents.
-- **`mobile-expo-expert`**: Audio streaming implementation in React Native with `expo-av` and WebRTC shim.
+---
+name: voice-ai-realtime-agent
+description: "Expert guide for Ultra-Low Latency Conversational Voice AI (<300ms), WebRTC bidirectional streaming, OpenAI Realtime API, Gemini Multimodal Live Audio, LiveKit Agents, and Semantic VAD / Panduan ahli AI suara percakapan real-time berlatensi ultra-rendah."
+author: "Roedy Rustam"
+version: "3.0.0"
+---
+
+# Voice AI Realtime Agent (2026 Edition)
+
+[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
+
+---
+
+<a name="english"></a>
+## English
+
+### Description
+Expert guide for building ultra-low-latency (<300ms), bi-directional conversational voice AI applications. Covers WebRTC, full-duplex WebSocket audio streaming (PCM 24kHz), OpenAI Realtime API, Gemini Multimodal Live API, LiveKit Agents SDK, and smart interruption (barge-in) handling.
+
+### Trigger Conditions
+- Applications requiring sub-second, spoken conversation with an AI agent.
+- Voice customer service bots, verbal copilots, language tutors, and interactive voice assistants.
+- Implementation of WebRTC audio streaming, full-duplex WebSocket audio (PCM 24kHz), and Silero VAD.
+- Setting up OpenAI Realtime API (`gpt-4o-realtime-preview`) or Gemini Multimodal Live API.
+
+---
+
+## 1. Core Architecture: Full-Duplex Speech-to-Speech
+
+Traditional voice pipelines chain STT ➔ LLM ➔ TTS with cumulative latency exceeding 1,200ms–2,500ms. Modern 2026 voice agents use **native speech-to-speech** or **streamable full-duplex WebRTC pipelines** achieving natural, human-like reaction times (~250–350ms).
+
+```
+User Mic ──► [WebRTC / WebSocket] ──► [VAD: Silero / WebRTC VAD]
+                                                │
+                                                ▼
+User Speaks <── [Audio Output] ◄── [Native Audio Stream / Cartesia] ◄── [OpenAI Realtime / Gemini Live]
+     │
+     └── User Interrupts (Barge-in) ──► Instant Buffer Flush & Cancel Audio Frame Emission
+```
+
+---
+
+## 2. Production Recipe: LiveKit Agents + OpenAI Realtime (Python)
+
+```python
+# agent.py - Production Voice Agent Worker with LiveKit & OpenAI Realtime
+import asyncio
+import os
+from livekit import rtc
+from livekit.agents import (
+    AutoSubscribe,
+    JobContext,
+    JobProcess,
+    WorkerOptions,
+    cli,
+    llm,
+)
+from livekit.agents.pipeline import VoicePipelineAgent
+from livekit.plugins import deepgram, openai, silero
+
+async def entrypoint(ctx: JobContext):
+    # Connect to room with audio only to minimize bandwidth & latency
+    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
+    
+    # Wait for the user participant to join
+    participant = await ctx.wait_for_participant()
+    
+    # Define agent instructions and tools
+    initial_ctx = llm.ChatContext().append(
+        role="system",
+        text=(
+            "You are a helpful, concise voice assistant. "
+            "Respond naturally in 1-2 short sentences. Never output markdown, bullet points, or emojis."
+        )
+    )
+
+    # Realtime Voice Pipeline: Deepgram (STT) + OpenAI (LLM) + Cartesia/OpenAI (TTS)
+    # Or use native OpenAI Realtime Model: gpt-4o-realtime-preview
+    agent = VoicePipelineAgent(
+        vad=silero.VAD.load(
+            min_speech_duration=0.1,
+            min_silence_duration=0.3, # Snappy turn-taking
+            prefix_padding_duration=0.2,
+        ),
+        stt=deepgram.STT(model="nova-2", language="id"), # Multi-language support
+        llm=openai.LLM(model="gpt-4o-mini"),
+        tts=openai.TTS(voice="alloy"),
+        chat_ctx=initial_ctx,
+        allow_interruptions=True, # Barge-in capability
+        interrupt_speech_duration=0.3, # Immediate cutoff when user talks
+    )
+
+    agent.start(ctx.room, participant)
+    
+    # Greet user immediately
+    await agent.say("Halo! Ada yang bisa saya bantu hari ini?", now=True)
+
+if __name__ == "__main__":
+    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
+```
+
+---
+
+## 3. Production Recipe: Gemini Multimodal Live Audio (TypeScript / Node.js)
+
+```typescript
+// gemini-live-audio.ts - Bidirectional WebSocket PCM 24kHz
+import WebSocket from 'ws';
+
+interface GeminiAudioConfig {
+  apiKey: string;
+  model?: string;
+  systemInstruction?: string;
+}
+
+export class GeminiVoiceAgent {
+  private ws: WebSocket | null = null;
+  private isConnected = false;
+
+  constructor(private config: GeminiAudioConfig) {}
+
+  public async connect(): Promise<void> {
+    const url = `wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1alpha.GenerativeService.BidiGenerateContent?key=${this.config.apiKey}`;
+    
+    this.ws = new WebSocket(url);
+
+    this.ws.on('open', () => {
+      this.isConnected = true;
+      this.sendInitialHandshake();
+    });
+
+    this.ws.on('message', (data: WebSocket.Data) => {
+      this.handleIncomingAudio(data);
+    });
+  }
+
+  private sendInitialHandshake(): void {
+    const setupMessage = {
+      setup: {
+        model: `models/${this.config.model || 'gemini-2.0-flash-exp'}`,
+        generationConfig: {
+          responseModalities: ["AUDIO"],
+          speechConfig: {
+            voiceConfig: {
+              prebuiltVoiceConfig: { voiceName: "Puck" }
+            }
+          }
+        },
+        systemInstruction: {
+          parts: [{ text: this.config.systemInstruction || "You are a conversational voice agent. Keep answers brief." }]
+        }
+      }
+    };
+    this.ws?.send(JSON.stringify(setupMessage));
+  }
+
+  // Stream raw PCM 16-bit 24kHz mono audio from mic
+  public sendAudioChunk(pcm16Chunk: Buffer): void {
+    if (!this.isConnected || !this.ws) return;
+
+    const base64Audio = pcm16Chunk.toString('base64');
+    const msg = {
+      realtimeInput: {
+        mediaChunks: [
+          {
+            mimeType: "audio/pcm;rate=24000",
+            data: base64Audio
+          }
+        ]
+      }
+    };
+    this.ws.send(JSON.stringify(msg));
+  }
+
+  private handleIncomingAudio(data: WebSocket.Data): void {
+    try {
+      const response = JSON.parse(data.toString());
+      const parts = response.serverContent?.modelTurn?.parts;
+      if (parts) {
+        for (const part of parts) {
+          if (part.inlineData?.data) {
+            const pcmBuffer = Buffer.from(part.inlineData.data, 'base64');
+            this.playAudioSpeaker(pcmBuffer);
+          }
+        }
+      }
+    } catch {
+      // Binary PCM frame handler
+    }
+  }
+
+  private playAudioSpeaker(pcmChunk: Buffer): void {
+    // Send to WebRTC audio track or audio output device
+  }
+}
+```
+
+---
+
+## 4. Key 2026 Performance Guardrails
+
+1. **Barge-in Latency Budget (<150ms)**: When the user speaks while the bot is talking, cancel outgoing audio immediately. Do not wait for the LLM to finish streaming its chunk.
+2. **Audio Sample Rates**:
+   - Mic Input: 16kHz or 24kHz 16-bit Linear PCM Mono.
+   - Bot Output: 24kHz PCM for crystal-clear natural prosody.
+3. **Turn-Taking Jitter Prevention**: Use minimum silence thresholds between `300ms` and `450ms`. Lower thresholds cause the bot to interrupt users when they pause to think; higher thresholds make the conversation feel robotic.
+
+---
+
+## Orchestration & Integration
+
+- **`ai-llm-integration-expert`**: For base LLM prompt routing and function calling during conversation.
+- **`realtime-collaboration-expert`**: For syncing WebRTC tracks and room states with client applications.
+- **`gemini-agent-booster`**: Connects Gemini 3.x / 2.0 Flash thinking models to live voice agents.
+- **`mobile-expo-expert`**: Audio streaming implementation in React Native with `expo-av` and WebRTC shim.
+
+---
+
+<a name="bahasa-indonesia"></a>
+## Bahasa Indonesia
+
+### Deskripsi
+Panduan ahli untuk membangun aplikasi AI suara percakapan dua arah berlatensi ultra-rendah (<300ms). Mencakup integrasi WebRTC, streaming audio WebSocket full-duplex (PCM 24kHz), OpenAI Realtime API, Gemini Multimodal Live API, LiveKit Agents SDK, dan penanganan interupsi cerdas (*barge-in*).
+
+### Kondisi Pemicu
+- Kebutuhan interaksi percakapan verbal instan di bawah satu detik dengan agen AI.
+- Bot layanan pelanggan berbasis suara, asisten verbal, tutor bahasa interaktif.
+- Implementasi streaming audio WebRTC, WebSocket PCM 24kHz dua arah, dan Silero Voice Activity Detection (VAD).
+- Konfigurasi OpenAI Realtime API (`gpt-4o-realtime-preview`) atau Gemini Multimodal Live API.
+
+### Panduan Inti Arsitektur Suara Real-time
+1. **Full-Duplex Speech-to-Speech**: Mengganti pipeline sekuensial tradisional (STT ➔ LLM ➔ TTS) dengan pipeline streamable WebRTC atau model native speech-to-speech untuk memangkas latensi dari ~2000ms menjadi ~300ms.
+2. **Penanganan Interupsi (Barge-In)**: Saat VAD mendeteksi suara pengguna baru saat bot sedang berbicara, buffer audio keluar harus di-flush dalam waktu <150ms tanpa menunggu LLM menyelesaikan kalimatnya.
+3. **Standar Format Audio**: Input mikrofon PCM 16-bit 16kHz/24kHz Mono, dan output speaker 24kHz untuk intonasi yang alami dan jernih.
+
+---
+
+## Integrasi Orkestrasi
+
+- **`ai-llm-integration-expert`**: Routing instruksi sistem dasar dan pemanggilan tool fungsi selama percakapan suara.
+- **`realtime-collaboration-expert`**: Sinkronisasi track audio WebRTC dan status room pengguna.
+- **`gemini-agent-booster`**: Integrasi model multimodal Gemini 2.0/3.x Flash untuk live audio.
+- **`mobile-expo-expert`**: Implementasi audio streaming di React Native menggunakan `expo-av` dan WebRTC shim.

package/skills/vue-frontend-expert/SKILL.md

@@ -1,7 +1,8 @@
 ---
 name: vue-frontend-expert
 description: "Expert guide for Vue 3 (Composition API), Nuxt 3, and Pinia. Covers advanced reactive state management, `<script setup>` syntax, Vue Router, VueUse, and SPA/SSR architectural patterns in English and Indonesian."
-author: "Roedy Rustam"
+author: "Roedy Rustam"
+version: "3.0.0"
 ---
 
 # Vue Frontend Expert (Vue 3 / Nuxt 3)
@@ -14,7 +15,7 @@ author: "Roedy Rustam"
 ## English
 
 ### Orchestration & Integration
-Connects and orchestrates with relevant domain skills like `brainstorming`, `zero-to-prod-orchestrator`, and `project-context-mapper` to ensure cohesive execution.
+Connects and orchestrates with relevant domain skills like `brainstorming`, `zero-to-prod-orchestrator`, and `session-memory-manager` to ensure cohesive execution.
 
 ### Description
 Production-grade guidance for building highly reactive and scalable frontend applications using **Vue 3 (Composition API)** and **Nuxt 3**. Covers the latest ecosystem tools including **Pinia** for state management, **VueUse** for composables, and **Tailwind CSS v4** integration.
@@ -111,7 +112,7 @@ This skill should be referenced by the following orchestrators:
 ## Bahasa Indonesia
 
 ### Integrasi Orkestrasi
-Terhubung dan mengorkestrasi skill domain yang relevan seperti `brainstorming`, `zero-to-prod-orchestrator`, dan `project-context-mapper` untuk memastikan eksekusi yang kohesif.
+Terhubung dan mengorkestrasi skill domain yang relevan seperti `brainstorming`, `zero-to-prod-orchestrator`, dan `session-memory-manager` untuk memastikan eksekusi yang kohesif.
 
 ### Deskripsi
 Panduan tingkat produksi untuk membangun aplikasi frontend reaktif dan terukur menggunakan **Vue 3 (Composition API)** dan **Nuxt 3**. Mencakup pengelolaan state dengan **Pinia**, utilitas **VueUse**, dan integrasi **Tailwind CSS v4**.
@@ -129,4 +130,4 @@ Aktifkan skill ini ketika pengguna sedang:
 - **Pinia:** Gunakan Pinia dengan pola *Setup Store* (mirip Composition API) alih-alih pola Options (state, getters, actions).
 - **Nuxt 3 Fetching:** Gunakan `useFetch` atau `useAsyncData` di dalam komponen Nuxt untuk pengambilan data saat SSR, bukan `onMounted` dengan `fetch` biasa.
 - **Reaktivitas:** Gunakan `ref` untuk nilai primitif (string, number) dan `reactive` untuk objek bersarang yang kompleks. Utamakan `computed` daripada `watch` untuk state turunan.
-- **VueUse:** Jangan menulis fungsi utilitas dari nol jika sudah ada di *library* VueUse (contoh: `useIntersectionObserver`, `useLocalStorage`).
+- **VueUse:** Jangan menulis fungsi utilitas dari nol jika sudah ada di *library* VueUse (contoh: `useIntersectionObserver`, `useLocalStorage`).

package/skills/wasm-edge-computing-expert/SKILL.md

@@ -1,7 +1,8 @@
 ---
 name: wasm-edge-computing-expert
 description: "Expert guide for WebAssembly (WASM) and Edge Computing. Covers WASI preview 2, Spin/Fermyon, Cloudflare Workers WASM, and high-performance browser computing / Panduan ahli untuk WebAssembly (WASM) dan Edge Computing. Mencakup WASI preview 2, Spin/Fermyon, Cloudflare Workers WASM, dan komputasi performa tinggi di browser."
-author: "Roedy Rustam"
+author: "Roedy Rustam"
+version: "3.0.0"
 ---
 
 # WASM & Edge Computing Expert
@@ -94,4 +95,4 @@ Untuk performa multi-threading sejati di browser menggunakan WASM, gunakan `Shar
 ## Integrasi Orkestrasi
 - Memperkuat `api-gateway-proxy-expert` untuk logika kustom di edge.
 - Terintegrasi dengan `rust-programming-expert` untuk kompilasi modul.
-- Melengkapi `performance-web-vitals` dengan memindahkan komputasi berat dari *main thread* JavaScript.
+- Melengkapi `performance-web-vitals` dengan memindahkan komputasi berat dari *main thread* JavaScript.