@namch/agent-assistant 1.0.0

package/skills/podcast-generation/SKILL.md

@@ -0,0 +1,123 @@
+```skill
+---
+name: podcast-generation
+description: Generate AI-powered podcast-style audio narratives using Azure OpenAI's GPT Realtime Mini model via WebSocket. Use when building text-to-speech features, audio narrative generation, podcast creation from content, or integrating with Azure OpenAI Realtime API for real audio output. Covers full-stack implementation from React frontend to Python FastAPI backend with WebSocket streaming.
+---
+
+# Podcast Generation with GPT Realtime Mini
+
+Generate real audio narratives from text content using Azure OpenAI's Realtime API.
+
+## Quick Start
+
+1. Configure environment variables for Realtime API
+2. Connect via WebSocket to Azure OpenAI Realtime endpoint
+3. Send text prompt, collect PCM audio chunks + transcript
+4. Convert PCM to WAV format
+5. Return base64-encoded audio to frontend for playback
+
+## Environment Configuration
+
+```env
+AZURE_OPENAI_AUDIO_API_KEY=your_realtime_api_key
+AZURE_OPENAI_AUDIO_ENDPOINT=https://your-resource.cognitiveservices.azure.com
+AZURE_OPENAI_AUDIO_DEPLOYMENT=gpt-realtime-mini
+```
+
+**Note**: Endpoint should NOT include `/openai/v1/` - just the base URL.
+
+## Core Workflow
+
+### Backend Audio Generation
+
+```python
+from openai import AsyncOpenAI
+import base64
+
+# Convert HTTPS endpoint to WebSocket URL
+ws_url = endpoint.replace("https://", "wss://") + "/openai/v1"
+
+client = AsyncOpenAI(
+    websocket_base_url=ws_url,
+    api_key=api_key
+)
+
+audio_chunks = []
+transcript_parts = []
+
+async with client.realtime.connect(model="gpt-realtime-mini") as conn:
+    # Configure for audio-only output
+    await conn.session.update(session={
+        "output_modalities": ["audio"],
+        "instructions": "You are a narrator. Speak naturally."
+    })
+    
+    # Send text to narrate
+    await conn.conversation.item.create(item={
+        "type": "message",
+        "role": "user",
+        "content": [{"type": "input_text", "text": prompt}]
+    })
+    
+    await conn.response.create()
+    
+    # Collect streaming events
+    async for event in conn:
+        if event.type == "response.output_audio.delta":
+            audio_chunks.append(base64.b64decode(event.delta))
+        elif event.type == "response.output_audio_transcript.delta":
+            transcript_parts.append(event.delta)
+        elif event.type == "response.done":
+            break
+
+# Convert PCM to WAV (see scripts/pcm_to_wav.py)
+pcm_audio = b''.join(audio_chunks)
+wav_audio = pcm_to_wav(pcm_audio, sample_rate=24000)
+```
+
+### Frontend Audio Playback
+
+```javascript
+// Convert base64 WAV to playable blob
+const base64ToBlob = (base64, mimeType) => {
+  const bytes = atob(base64);
+  const arr = new Uint8Array(bytes.length);
+  for (let i = 0; i < bytes.length; i++) arr[i] = bytes.charCodeAt(i);
+  return new Blob([arr], { type: mimeType });
+};
+
+const audioBlob = base64ToBlob(response.audio_data, 'audio/wav');
+const audioUrl = URL.createObjectURL(audioBlob);
+new Audio(audioUrl).play();
+```
+
+## Voice Options
+
+| Voice | Character |
+|-------|-----------|
+| alloy | Neutral |
+| echo | Warm |
+| fable | Expressive |
+| onyx | Deep |
+| nova | Friendly |
+| shimmer | Clear |
+
+## Realtime API Events
+
+- `response.output_audio.delta` - Base64 audio chunk
+- `response.output_audio_transcript.delta` - Transcript text
+- `response.done` - Generation complete
+- `error` - Handle with `event.error.message`
+
+## Audio Format
+
+- **Input**: Text prompt
+- **Output**: PCM audio (24kHz, 16-bit, mono)
+- **Storage**: Base64-encoded WAV
+
+## References
+
+- **Full architecture**: See [references/architecture.md](references/architecture.md) for complete stack design
+- **Code examples**: See [references/code-examples.md](references/code-examples.md) for production patterns
+- **PCM conversion**: Use [scripts/pcm_to_wav.py](scripts/pcm_to_wav.py) for audio format conversion
+```

package/skills/podcast-generation/references/architecture.md

@@ -0,0 +1,87 @@
+# Architecture Reference
+
+## Full-Stack Flow
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                       FRONTEND (React)                          │
+│  AudioNarrative.jsx / PodcastOverview.jsx                       │
+│    ↓ POST /api/v1/ai/audio                                     │
+│  api.js → aiAPI.generateAudio(sourceType, sourceId, ...)       │
+└────────────────────────────┬────────────────────────────────────┘
+                             │
+┌────────────────────────────▼────────────────────────────────────┐
+│                       BACKEND (FastAPI)                         │
+│  API: ai.py                                                     │
+│    @router.post("/audio") → AudioNarrativeRequest               │
+│    @router.get("/audio/{id}/stream") → WAV file                │
+│                             │                                   │
+│  Service: ai_service.py                                         │
+│    generate_audio_narrative()                                   │
+│    - Fetch content from DB (tag/bookmark/custom)               │
+│    - Build styled prompt                                        │
+│    - WebSocket connect to Azure Realtime API                   │
+│    - Stream audio chunks + transcript                           │
+│    - PCM → WAV conversion                                       │
+│    - Save to database                                           │
+│                             │                                   │
+│  Model: database.py                                             │
+│    AudioNarrative table                                         │
+└────────────────────────────┬────────────────────────────────────┘
+                             │ wss://
+┌────────────────────────────▼────────────────────────────────────┐
+│              Azure OpenAI Realtime API                          │
+│  Model: gpt-realtime-mini                                       │
+│  Output: PCM audio (24kHz, 16-bit mono) + transcript           │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Database Schema
+
+```python
+class AudioNarrative(Base):
+    __tablename__ = "audio_narratives"
+    
+    id: int                          # Primary key
+    source_type: str                 # "tag", "bookmark", "custom"
+    source_id: Optional[int]         # Reference to source
+    source_name: Optional[str]       # Display name
+    title: str                       # Generated title
+    script: str                      # Transcript text
+    audio_url: Optional[str]         # Stream endpoint
+    audio_data: Optional[str]        # Base64 WAV
+    duration_seconds: Optional[int]  # Calculated from PCM length
+    voice_name: str                  # "alloy", "echo", etc.
+    created_at: datetime
+```
+
+## Pydantic Schemas
+
+```python
+class AudioNarrativeRequest(BaseModel):
+    source_type: str          # Required: "tag", "bookmark", "custom"
+    source_id: Optional[Union[int, str]] = None
+    custom_query: Optional[str] = None
+    voice_name: str = "alloy"
+    style: str = "podcast"    # "podcast", "summary", "lecture"
+
+class AudioNarrativeResponse(BaseModel):
+    id: int
+    title: str
+    script: str
+    audio_url: Optional[str]
+    audio_data: Optional[str]  # Base64 WAV
+    duration_seconds: Optional[int]
+    voice_name: str
+    created_at: datetime
+```
+
+## Style Instructions
+
+```python
+STYLE_INSTRUCTIONS = {
+    "podcast": "Speak in a conversational, engaging podcast style with natural transitions. Use phrases like 'Let's dive into...' and 'What's fascinating here is...'",
+    "summary": "Speak clearly and informatively, getting straight to the key points in a news anchor style.",
+    "lecture": "Speak in an educational, thorough style suitable for learning. Explain concepts clearly like a professor."
+}
+```

package/skills/podcast-generation/references/code-examples.md

@@ -0,0 +1,189 @@
+# Code Examples
+
+## Complete Backend Service Method
+
+```python
+async def generate_audio_narrative(
+    self,
+    source_type: str,
+    source_id: Optional[Union[int, str]] = None,
+    custom_query: Optional[str] = None,
+    voice_name: str = "alloy",
+    style: str = "podcast"
+) -> Dict[str, Any]:
+    """Generate podcast-style audio using gpt-realtime-mini"""
+    
+    # 1. Validate configuration
+    if not settings.azure_openai_audio_api_key:
+        raise ValueError("AZURE_OPENAI_AUDIO_API_KEY not configured")
+    
+    # 2. Gather content based on source
+    if source_type == "tag":
+        tag = await db.get_tag(source_id)
+        bookmarks = tag.bookmarks
+        title = f"Exploring {tag.name}"
+    elif source_type == "bookmark":
+        bookmark = await db.get_bookmark(source_id)
+        bookmarks = [bookmark]
+        title = f"Deep Dive: {bookmark.title}"
+    else:  # custom
+        bookmarks = await db.get_recent_bookmarks(limit=10)
+        title = f"Research Summary: {custom_query[:50]}"
+    
+    # 3. Build prompt with style
+    content = "\n".join([f"**{b.title}**\n{b.summary}" for b in bookmarks[:10]])
+    prompt = f"""Create a {style} narrative from these sources:
+{content}
+
+{STYLE_INSTRUCTIONS[style]}
+Make it 1-2 minutes (150-250 words). Speak naturally."""
+    
+    # 4. Connect to Realtime API
+    ws_url = settings.azure_openai_audio_endpoint.replace("https://", "wss://") + "/openai/v1"
+    client = AsyncOpenAI(websocket_base_url=ws_url, api_key=settings.azure_openai_audio_api_key)
+    
+    audio_chunks, transcript_parts = [], []
+    
+    async with client.realtime.connect(model=settings.azure_openai_audio_deployment) as conn:
+        await conn.session.update(session={
+            "output_modalities": ["audio"],
+            "instructions": f"Narrator creating {style}-style content. Speak naturally, don't ask questions."
+        })
+        
+        await conn.conversation.item.create(item={
+            "type": "message",
+            "role": "user", 
+            "content": [{"type": "input_text", "text": prompt}]
+        })
+        
+        await conn.response.create()
+        
+        async for event in conn:
+            if event.type == "response.output_audio.delta":
+                audio_chunks.append(base64.b64decode(event.delta))
+            elif event.type == "response.output_audio_transcript.delta":
+                transcript_parts.append(event.delta)
+            elif event.type == "response.done":
+                break
+            elif event.type == "error":
+                raise ValueError(f"Realtime API error: {event.error.message}")
+    
+    # 5. Process audio
+    pcm_audio = b''.join(audio_chunks)
+    wav_audio = pcm_to_wav(pcm_audio, sample_rate=24000)
+    audio_base64 = base64.b64encode(wav_audio).decode('utf-8')
+    duration = len(pcm_audio) // (24000 * 2)  # 24kHz, 16-bit
+    
+    # 6. Save and return
+    narrative = AudioNarrative(
+        source_type=source_type,
+        title=title,
+        script=''.join(transcript_parts),
+        audio_data=audio_base64,
+        duration_seconds=duration,
+        voice_name=voice_name
+    )
+    db.add(narrative)
+    await db.commit()
+    
+    return {
+        "id": narrative.id,
+        "title": title,
+        "script": narrative.script,
+        "audio_data": audio_base64,
+        "audio_url": f"/api/v1/ai/audio/{narrative.id}/stream",
+        "duration_seconds": duration,
+        "voice_name": voice_name
+    }
+```
+
+## FastAPI Endpoints
+
+```python
+@router.post("/audio", response_model=AudioNarrativeResponse)
+async def generate_audio(request: AudioNarrativeRequest, db: AsyncSession = Depends(get_db)):
+    ai_service = AIService(db)
+    result = await ai_service.generate_audio_narrative(
+        source_type=request.source_type,
+        source_id=request.source_id,
+        custom_query=request.custom_query,
+        voice_name=request.voice_name,
+        style=request.style
+    )
+    return AudioNarrativeResponse(**result)
+
+@router.get("/audio/{narrative_id}/stream")
+async def stream_audio(narrative_id: int, db: AsyncSession = Depends(get_db)):
+    ai_service = AIService(db)
+    narrative = await ai_service.get_audio_narrative_by_id(narrative_id)
+    if not narrative:
+        raise HTTPException(404, "Not found")
+    
+    audio_bytes = base64.b64decode(narrative["audio_data"])
+    return Response(
+        content=audio_bytes,
+        media_type="audio/wav",
+        headers={"Content-Disposition": f'filename="narrative-{narrative_id}.wav"'}
+    )
+```
+
+## React Component Pattern
+
+```jsx
+function AudioPlayer({ sourceType, sourceId }) {
+  const [narrative, setNarrative] = useState(null);
+  const [loading, setLoading] = useState(false);
+  const [isPlaying, setIsPlaying] = useState(false);
+  const audioRef = useRef(null);
+
+  const generate = async () => {
+    setLoading(true);
+    const response = await aiAPI.generateAudio(sourceType, sourceId, null, 'alloy', 'podcast');
+    setNarrative(response.data);
+    setLoading(false);
+  };
+
+  const play = () => {
+    if (!narrative?.audio_data) return;
+    
+    const blob = base64ToBlob(narrative.audio_data, 'audio/wav');
+    const url = URL.createObjectURL(blob);
+    
+    if (!audioRef.current) audioRef.current = new Audio();
+    audioRef.current.src = url;
+    audioRef.current.onended = () => setIsPlaying(false);
+    audioRef.current.play();
+    setIsPlaying(true);
+  };
+
+  return (
+    <div>
+      {!narrative && <button onClick={generate} disabled={loading}>Generate Podcast</button>}
+      {narrative && (
+        <>
+          <button onClick={play}>{isPlaying ? 'Pause' : 'Play'}</button>
+          <p>{narrative.script}</p>
+        </>
+      )}
+    </div>
+  );
+}
+```
+
+## Frontend API Service
+
+```javascript
+export const aiAPI = {
+  generateAudio: (sourceType, sourceId = null, customQuery = null, 
+                  voiceName = 'alloy', style = 'podcast') =>
+    api.post('/ai/audio', {
+      source_type: sourceType,
+      source_id: sourceId,
+      custom_query: customQuery,
+      voice_name: voiceName,
+      style
+    }),
+  
+  listAudioNarratives: (limit = 10) => api.get(`/ai/audio?limit=${limit}`),
+};
+```

package/skills/podcast-generation/scripts/pcm_to_wav.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""Convert raw PCM audio data to WAV format.
+
+Azure OpenAI Realtime API outputs PCM audio (24kHz, 16-bit, mono).
+This script converts it to standard WAV format for browser playback.
+
+Usage:
+    from pcm_to_wav import pcm_to_wav
+    wav_bytes = pcm_to_wav(pcm_data, sample_rate=24000)
+"""
+
+import struct
+import io
+
+
+def pcm_to_wav(
+    pcm_data: bytes,
+    sample_rate: int = 24000,
+    channels: int = 1,
+    sample_width: int = 2
+) -> bytes:
+    """Convert raw PCM audio data to WAV format.
+    
+    Args:
+        pcm_data: Raw PCM audio bytes
+        sample_rate: Samples per second (default 24000 for gpt-realtime-mini)
+        channels: Number of audio channels (default 1 for mono)
+        sample_width: Bytes per sample (default 2 for 16-bit)
+    
+    Returns:
+        WAV-formatted audio bytes
+    """
+    wav_buffer = io.BytesIO()
+    
+    # RIFF header
+    wav_buffer.write(b'RIFF')
+    wav_buffer.write(struct.pack('<I', 36 + len(pcm_data)))  # File size - 8
+    wav_buffer.write(b'WAVE')
+    
+    # fmt subchunk
+    wav_buffer.write(b'fmt ')
+    wav_buffer.write(struct.pack('<I', 16))                    # Subchunk size
+    wav_buffer.write(struct.pack('<H', 1))                     # Audio format (1 = PCM)
+    wav_buffer.write(struct.pack('<H', channels))              # Number of channels
+    wav_buffer.write(struct.pack('<I', sample_rate))           # Sample rate
+    wav_buffer.write(struct.pack('<I', sample_rate * channels * sample_width))  # Byte rate
+    wav_buffer.write(struct.pack('<H', channels * sample_width))  # Block align
+    wav_buffer.write(struct.pack('<H', sample_width * 8))      # Bits per sample
+    
+    # data subchunk
+    wav_buffer.write(b'data')
+    wav_buffer.write(struct.pack('<I', len(pcm_data)))
+    wav_buffer.write(pcm_data)
+    
+    return wav_buffer.getvalue()
+
+
+def calculate_duration(pcm_data: bytes, sample_rate: int = 24000, sample_width: int = 2) -> int:
+    """Calculate audio duration in seconds from PCM data length."""
+    return len(pcm_data) // (sample_rate * sample_width)
+
+
+if __name__ == "__main__":
+    # Example usage
+    import sys
+    if len(sys.argv) > 1:
+        with open(sys.argv[1], 'rb') as f:
+            pcm_data = f.read()
+        wav_data = pcm_to_wav(pcm_data)
+        output_path = sys.argv[1].replace('.pcm', '.wav')
+        with open(output_path, 'wb') as f:
+            f.write(wav_data)
+        print(f"Converted to {output_path} ({calculate_duration(pcm_data)}s)")

package/skills/postgres-pro/SKILL.md

@@ -0,0 +1,98 @@
+---
+name: postgres-pro
+description: Use when optimizing PostgreSQL queries, configuring replication, or implementing advanced database features. Invoke for EXPLAIN analysis, JSONB operations, extension usage, VACUUM tuning, performance monitoring.
+triggers:
+  - PostgreSQL
+  - Postgres
+  - EXPLAIN ANALYZE
+  - pg_stat
+  - JSONB
+  - streaming replication
+  - logical replication
+  - VACUUM
+  - PostGIS
+  - pgvector
+role: specialist
+scope: implementation
+output-format: code
+---
+
+# PostgreSQL Pro
+
+Senior PostgreSQL expert with deep expertise in database administration, performance optimization, and advanced PostgreSQL features.
+
+## Role Definition
+
+You are a senior PostgreSQL DBA with 10+ years of production experience. You specialize in query optimization, replication strategies, JSONB operations, extension usage, and database maintenance. You build reliable, high-performance PostgreSQL systems that scale.
+
+## When to Use This Skill
+
+- Analyzing and optimizing slow queries with EXPLAIN
+- Implementing JSONB storage and indexing strategies
+- Setting up streaming or logical replication
+- Configuring and using PostgreSQL extensions
+- Tuning VACUUM, ANALYZE, and autovacuum
+- Monitoring database health with pg_stat views
+- Designing indexes for optimal performance
+
+## Core Workflow
+
+1. **Analyze performance** - Use EXPLAIN ANALYZE, pg_stat_statements
+2. **Design indexes** - B-tree, GIN, GiST, BRIN based on workload
+3. **Optimize queries** - Rewrite inefficient queries, update statistics
+4. **Setup replication** - Streaming or logical based on requirements
+5. **Monitor and maintain** - VACUUM, ANALYZE, bloat tracking
+
+## Reference Guide
+
+Load detailed guidance based on context:
+
+| Topic | Reference | Load When |
+|-------|-----------|-----------|
+| Performance | `references/performance.md` | EXPLAIN ANALYZE, indexes, statistics, query tuning |
+| JSONB | `references/jsonb.md` | JSONB operators, indexing, GIN indexes, containment |
+| Extensions | `references/extensions.md` | PostGIS, pg_trgm, pgvector, uuid-ossp, pg_stat_statements |
+| Replication | `references/replication.md` | Streaming replication, logical replication, failover |
+| Maintenance | `references/maintenance.md` | VACUUM, ANALYZE, pg_stat views, monitoring, bloat |
+
+## Constraints
+
+### MUST DO
+- Use EXPLAIN ANALYZE for query optimization
+- Create appropriate indexes (B-tree, GIN, GiST, BRIN)
+- Update statistics with ANALYZE after bulk changes
+- Monitor autovacuum and tune if needed
+- Use connection pooling (pgBouncer, pgPool)
+- Setup replication for high availability
+- Monitor with pg_stat_statements, pg_stat_user_tables
+- Use prepared statements to prevent SQL injection
+
+### MUST NOT DO
+- Disable autovacuum globally
+- Create indexes without analyzing query patterns
+- Use SELECT * in production queries
+- Ignore replication lag monitoring
+- Skip VACUUM on high-churn tables
+- Use text for UUID storage (use uuid type)
+- Store large BLOBs in database (use object storage)
+- Ignore pg_stat_statements warnings
+
+## Output Templates
+
+When implementing PostgreSQL solutions, provide:
+1. Query with EXPLAIN ANALYZE output
+2. Index definitions with rationale
+3. Configuration changes with before/after values
+4. Monitoring queries for ongoing health checks
+5. Brief explanation of performance impact
+
+## Knowledge Reference
+
+PostgreSQL 12-16, EXPLAIN ANALYZE, B-tree/GIN/GiST/BRIN indexes, JSONB operators, streaming replication, logical replication, VACUUM/ANALYZE, pg_stat views, PostGIS, pgvector, pg_trgm, WAL archiving, PITR
+
+## Related Skills
+
+- **Database Optimizer** - General database optimization
+- **Backend Developer** - Application query patterns
+- **DevOps Engineer** - Deployment and automation
+- **SRE Engineer** - Reliability and monitoring