remdb 0.2.6__py3-none-any.whl

rem/services/audio/INTEGRATION.md

@@ -0,0 +1,308 @@
+# Audio Provider Integration
+
+The AudioProvider is now fully integrated into REM's ContentService with a **consistent interface** that matches all other content providers.
+
+## Architecture
+
+```
+┌────────────────────────────────────────────────────────────┐
+│                     ContentService                         │
+│                  (Pluggable Providers)                     │
+├────────────────────────────────────────────────────────────┤
+│                                                            │
+│  ┌──────────────┐  ┌────────────┐  ┌─────────────────┐   │
+│  │ TextProvider │→│DocProvider │→│ AudioProvider   │   │
+│  └──────────────┘  └────────────┘  └─────────────────┘   │
+│         │                  │                  │           │
+│         ▼                  ▼                  ▼           │
+│      extract()         extract()         extract()        │
+│         │                  │                  │           │
+│         ▼                  ▼                  ▼           │
+│     Markdown           Markdown           Markdown        │
+│       text               text               text          │
+│         │                  │                  │           │
+│         └──────────────────┼──────────────────┘           │
+│                            │                              │
+│                            ▼                              │
+│                    chunk_text() → embed()                 │
+│                            │                              │
+│                            ▼                              │
+│                   Save to Database                        │
+│                 (File + Resource entities)                │
+└────────────────────────────────────────────────────────────┘
+```
+
+## Consistent Interface
+
+All content providers implement the same `ContentProvider` base class:
+
+```python
+class ContentProvider(ABC):
+    @property
+    @abstractmethod
+    def name(self) -> str:
+        """Provider name for logging/debugging."""
+        pass
+
+    @abstractmethod
+    def extract(self, content: bytes, metadata: dict[str, Any]) -> dict[str, Any]:
+        """
+        Extract text content from file bytes.
+
+        Args:
+            content: Raw file bytes
+            metadata: File metadata (size, type, etc.)
+
+        Returns:
+            dict with:
+                - text: Extracted text content
+                - metadata: Additional metadata from extraction (optional)
+        """
+        pass
+```
+
+## Provider Implementations
+
+### 1. TextProvider
+```python
+def extract(self, content: bytes, metadata: dict) -> dict:
+    text = content.decode("utf-8")
+    return {
+        "text": text,
+        "metadata": {"line_count": len(text.split("\n"))}
+    }
+```
+
+### 2. DocProvider (Kreuzberg)
+```python
+def extract(self, content: bytes, metadata: dict) -> dict:
+    # Uses Kreuzberg for PDF extraction
+    result = extract_file_sync(tmp_path, config=config)
+    return {
+        "text": result.content,
+        "metadata": {"table_count": len(result.tables)}
+    }
+```
+
+### 3. AudioProvider (AudioChunker + Whisper)
+```python
+def extract(self, content: bytes, metadata: dict) -> dict:
+    # 1. Chunk audio by silence
+    chunks = chunker.chunk_audio(tmp_path)
+
+    # 2. Transcribe chunks
+    results = transcriber.transcribe_chunks(chunks)
+
+    # 3. Format as markdown with timestamps
+    markdown_parts = []
+    for result in results:
+        timestamp = f"{result.start_seconds:.1f}s - {result.end_seconds:.1f}s"
+        markdown_parts.append(f"## [{timestamp}]\n\n{result.text}\n")
+
+    return {
+        "text": "\n".join(markdown_parts),
+        "metadata": {
+            "chunk_count": len(chunks),
+            "duration_seconds": total_duration,
+            "estimated_cost": estimated_cost,
+        }
+    }
+```
+
+## Markdown Format
+
+All providers return markdown-formatted text. AudioProvider returns:
+
+```markdown
+## [0.0s - 60.0s]
+
+Transcription of first minute goes here...
+
+## [60.0s - 120.0s]
+
+Transcription of second minute goes here...
+
+## [120.0s - 180.0s]
+
+Transcription of third minute goes here...
+```
+
+This format:
+- ✅ Is valid markdown
+- ✅ Has clear section boundaries
+- ✅ Preserves temporal information
+- ✅ Can be chunked further if needed
+- ✅ Embeds naturally with other content
+
+## Processing Pipeline
+
+### Example: Audio File Processing
+
+```python
+from rem.services.content import ContentService
+
+service = ContentService()
+
+# Process audio file (same interface as PDF/markdown!)
+result = service.process_uri("s3://bucket/meeting.m4a")
+
+# Result structure (same for all providers):
+{
+    "uri": "s3://bucket/meeting.m4a",
+    "content": "## [0.0s - 60.0s]\n\nDiscussion about...\n\n## [60.0s - 120.0s]...",
+    "metadata": {
+        "chunk_count": 5,
+        "duration_seconds": 300.0,
+        "estimated_cost": 0.030,
+        "parser": "whisper_api"
+    },
+    "provider": "audio"
+}
+```
+
+### End-to-End Processing
+
+```python
+# Process and save to database
+await service.process_and_save(
+    uri="s3://bucket/meeting.m4a",
+    user_id="user-123"
+)
+
+# This automatically:
+# 1. Downloads from S3
+# 2. Chunks audio by silence
+# 3. Transcribes with Whisper
+# 4. Converts to markdown
+# 5. Chunks markdown text
+# 6. Saves File entity
+# 7. Saves Resource entities (one per chunk)
+# 8. Generates embeddings (ready for vector search)
+```
+
+## Registered Extensions
+
+The AudioProvider is automatically registered for:
+- `.wav` - Uncompressed audio
+- `.mp3` - Compressed audio
+- `.m4a` - Apple audio format
+- `.flac` - Lossless compression
+- `.ogg` - Ogg Vorbis
+
+## Graceful Degradation
+
+Without OpenAI API key:
+```python
+result = audio_provider.extract(content, metadata)
+
+# Returns:
+{
+    "text": "[Audio transcription requires OPENAI_API_KEY environment variable]",
+    "metadata": {"error": "missing_api_key"}
+}
+```
+
+Without pydub installed:
+```python
+# Returns:
+{
+    "text": "[Audio processing requires: pip install rem[audio]]",
+    "metadata": {"error": "missing_dependencies"}
+}
+```
+
+## Testing
+
+All providers tested for interface consistency:
+
+```bash
+# Run integration tests
+pytest tests/integration/services/test_content_providers.py -v
+
+# Results:
+# ✓ test_markdown_provider_interface PASSED
+# ✓ test_pdf_provider_interface PASSED
+# ✓ test_audio_provider_interface PASSED
+# ✓ test_content_service_has_all_providers PASSED
+# ✓ test_markdown_file_processing PASSED
+# ✓ test_audio_file_processing_without_api_key PASSED
+# ✓ test_all_providers_return_text_and_metadata PASSED
+# ✓ test_all_providers_handle_empty_content PASSED
+# ✓ test_markdown_to_audio_consistency PASSED
+# ✓ test_audio_returns_markdown_with_timestamps PASSED
+```
+
+## Consistency Guarantees
+
+All providers:
+
+1. **Accept same input**: `extract(content: bytes, metadata: dict)`
+2. **Return same structure**: `{"text": str, "metadata": dict}`
+3. **Return markdown format**: Text is markdown-compatible
+4. **Handle errors gracefully**: Return error messages, don't crash
+5. **Register with ContentService**: Via file extension mapping
+6. **Follow pipeline**: extract → markdown → chunk → embed → save
+
+## Usage Examples
+
+### Process Single File
+
+```python
+from rem.services.content import ContentService
+
+service = ContentService()
+
+# Process markdown
+md_result = service.process_uri("document.md")
+
+# Process PDF
+pdf_result = service.process_uri("report.pdf")
+
+# Process audio (same interface!)
+audio_result = service.process_uri("meeting.m4a")
+
+# All return same structure
+assert "content" in md_result
+assert "content" in pdf_result
+assert "content" in audio_result
+```
+
+### Process with S3
+
+```python
+# S3 URI - automatic download and processing
+result = service.process_uri("s3://recordings/standup.m4a")
+
+# Transcribed, chunked, and ready to save
+```
+
+### Custom Provider Registration
+
+```python
+# Register custom provider
+service.register_provider(
+    extensions=[".custom"],
+    provider=CustomProvider()
+)
+
+# Now .custom files use CustomProvider
+```
+
+## Future Enhancements
+
+1. **Streaming Transcription**: Process long audio files in streams
+2. **Speaker Diarization**: Identify different speakers
+3. **Language Detection**: Auto-detect language for transcription
+4. **Timestamp Refinement**: More accurate timestamps via VAD
+5. **Batch Processing**: Parallel transcription of multiple files
+
+## Key Takeaways
+
+✅ **Pluggable**: Easy to add new content types
+✅ **Consistent**: Same interface for all providers
+✅ **Testable**: All providers tested for consistency
+✅ **Graceful**: Handles missing dependencies/keys elegantly
+✅ **Integrated**: Works with ContentService out of the box
+✅ **Production-Ready**: Error handling, logging, cleanup
+
+The AudioProvider is a **first-class citizen** in REM's content processing pipeline!

rem/services/audio/README.md

@@ -0,0 +1,376 @@
+# REM Audio Processing
+
+Lightweight audio processing service with minimal dependencies for chunking and transcribing audio files.
+
+## Design Philosophy
+
+**Minimal Dependencies:**
+- `wave` (stdlib) for WAV file handling
+- `pydub` for audio format conversion (wraps ffmpeg)
+- `requests` for OpenAI Whisper API (already a REM dependency)
+- `loguru` for logging (REM standard)
+
+**No Heavy ML Libraries:**
+- No `torch`, `torchaudio`, or other heavyweight dependencies
+- No `librosa` for audio analysis
+- Keep the Docker image lean and fast
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                  REM Audio Service                      │
+├─────────────────────────────────────────────────────────┤
+│                                                         │
+│  ┌──────────────┐         ┌──────────────┐            │
+│  │ AudioChunker │────────▶│AudioTranscriber│           │
+│  └──────────────┘         └──────────────┘            │
+│         │                        │                     │
+│         │                        │                     │
+│    Split by silence         OpenAI Whisper API         │
+│    near minute              ($0.006/minute)            │
+│    boundaries                                          │
+│                                                         │
+└─────────────────────────────────────────────────────────┘
+```
+
+## Components
+
+### 1. AudioChunker
+
+Splits audio files by detecting silence near minute boundaries.
+
+**Strategy:**
+- Target chunks around 60 seconds (configurable)
+- Look for silence in window around target (±2 seconds)
+- Split at longest silence in window
+- If no silence, split at target boundary
+
+**Benefits:**
+- Keeps chunks under OpenAI's 25MB limit (~10 minutes)
+- Natural breaks at silence points
+- Maintains speech context within chunks
+
+**Example:**
+```python
+from rem.services.audio import AudioChunker
+
+chunker = AudioChunker(
+    target_chunk_seconds=60.0,  # 1 minute target
+    chunk_window_seconds=2.0,    # ±2 second search window
+    silence_threshold_db=-40.0,  # Silence detection threshold
+    min_silence_ms=500,          # Minimum 500ms silence
+)
+
+# Chunk audio file
+chunks = chunker.chunk_audio("recording.m4a")
+
+# Process chunks
+for chunk in chunks:
+    print(f"Chunk {chunk.chunk_index}: {chunk.start_seconds:.1f}s - {chunk.end_seconds:.1f}s")
+    print(f"Duration: {chunk.duration_seconds:.1f}s")
+    print(f"File: {chunk.file_path}")
+
+# Cleanup when done
+chunker.cleanup_chunks(chunks)
+```
+
+### 2. AudioTranscriber
+
+Transcribes audio using OpenAI Whisper API.
+
+**Features:**
+- Uses `requests` (no httpx dependency)
+- Handles file uploads efficiently
+- Automatic cost estimation
+- Detailed logging with loguru
+
+**Example:**
+```python
+from rem.services.audio import AudioTranscriber
+
+transcriber = AudioTranscriber(
+    api_key="sk-...",           # Or from OPENAI_API_KEY env
+    model="whisper-1",           # OpenAI Whisper model
+    language=None,               # Auto-detect language
+    temperature=0.0,             # Deterministic transcription
+)
+
+# Transcribe single file
+result = transcriber.transcribe_file("audio.wav")
+print(result.text)
+
+# Transcribe chunks
+results = transcriber.transcribe_chunks(chunks)
+for result in results:
+    print(f"[{result.start_seconds:.1f}s - {result.end_seconds:.1f}s]: {result.text}")
+```
+
+### 3. Complete Workflow
+
+```python
+from rem.services.audio import AudioChunker, AudioTranscriber
+
+# 1. Chunk audio by silence
+chunker = AudioChunker()
+chunks = chunker.chunk_audio("meeting_recording.m4a")
+
+print(f"Created {len(chunks)} chunks")
+
+# 2. Transcribe chunks
+transcriber = AudioTranscriber()
+results = transcriber.transcribe_chunks(chunks)
+
+print(f"Transcribed {len(results)} chunks")
+
+# 3. Combine results
+full_transcription = "\n\n".join([
+    f"[{r.start_seconds:.1f}s]: {r.text}"
+    for r in results
+])
+
+print(full_transcription)
+
+# 4. Cleanup
+chunker.cleanup_chunks(chunks)
+```
+
+## Configuration
+
+### Environment Variables
+
+```bash
+# OpenAI API Key (required for transcription)
+OPENAI_API_KEY=sk-...
+
+# Chunker Settings (optional)
+AUDIO_CHUNK_TARGET_SECONDS=60       # Target chunk duration
+AUDIO_CHUNK_WINDOW_SECONDS=2        # Silence search window
+AUDIO_SILENCE_THRESHOLD_DB=-40      # Silence detection threshold
+AUDIO_MIN_SILENCE_MS=500            # Minimum silence duration
+```
+
+### Transcription Costs
+
+OpenAI Whisper API pricing: **$0.006 per minute**
+
+Examples:
+- 10 minute recording: $0.06
+- 1 hour recording: $0.36
+- 10 hour recording: $3.60
+
+## Supported Formats
+
+### With pydub + ffmpeg:
+- WAV (uncompressed)
+- MP3 (compressed)
+- M4A (Apple audio)
+- FLAC (lossless)
+- OGG (Vorbis)
+- WMA (Windows)
+
+### Without pydub:
+- Only WAV files (requires pydub for format conversion)
+
+## Docker Setup
+
+The Dockerfile includes ffmpeg for audio processing:
+
+```dockerfile
+# Runtime dependencies
+RUN apt-get install -y \
+    ffmpeg  # Required by pydub for format conversion
+```
+
+Install pydub dependency:
+
+```bash
+# Install audio extras
+pip install rem[audio]
+
+# Or install all extras
+pip install rem[all]
+```
+
+## Dependencies
+
+### Core (always installed with rem[audio]):
+- `pydub>=0.25.0` - Audio manipulation
+
+### System (Docker):
+- `ffmpeg` - Audio codec support (installed in Dockerfile)
+
+### External APIs:
+- OpenAI Whisper API - Speech-to-text transcription
+
+## Error Handling
+
+### Missing API Key
+```python
+transcriber = AudioTranscriber()  # No API key
+
+# Raises: ValueError("OpenAI API key required for transcription")
+result = transcriber.transcribe_file("audio.wav")
+```
+
+### File Too Large
+```python
+# Whisper API limit: 25 MB
+transcriber.transcribe_file("huge_file.wav")
+
+# Raises: ValueError("Audio file too large: 30.5 MB (max 25 MB)")
+```
+
+### No pydub
+```python
+# Without pydub installed
+chunker = AudioChunker()
+chunker.chunk_audio("audio.m4a")
+
+# Raises: RuntimeError("pydub required for .m4a files")
+```
+
+## Best Practices
+
+1. **Chunk Before Transcribing**
+   - Don't send entire 2-hour recordings to Whisper
+   - Chunk into 1-minute segments for better quality
+   - Easier to debug and retry failed segments
+
+2. **Monitor Costs**
+   - Log transcription duration and cost
+   - Set budgets for long recordings
+   - Use `transcriber.transcribe_chunks()` for cost estimation
+
+3. **Handle Failures Gracefully**
+   - Chunks can fail independently
+   - Retry logic for transient errors
+   - Save partial results
+
+4. **Cleanup Temporary Files**
+   - Always call `chunker.cleanup_chunks()` when done
+   - Or use context manager (future enhancement)
+
+5. **Use Silence Detection**
+   - Default settings work well for most speech
+   - Adjust `silence_threshold_db` for noisy recordings
+   - Increase `min_silence_ms` for natural pauses
+
+## Integration with REM
+
+### File Processing
+
+```python
+# rem/workers/file_processor.py
+from rem.services.audio import AudioChunker, AudioTranscriber
+
+async def process_audio_file(file_path: Path, user_id: str):
+    """Process audio file and create REM resources."""
+
+    # 1. Chunk audio
+    chunker = AudioChunker()
+    chunks = chunker.chunk_audio(file_path)
+
+    # 2. Transcribe chunks
+    transcriber = AudioTranscriber()
+    results = transcriber.transcribe_chunks(chunks)
+
+    # 3. Create REM resources
+    for i, result in enumerate(results):
+        resource = Resource(
+            name=f"{file_path.stem} - Part {i+1}",
+            uri=f"{file_path.as_uri()}#t={result.start_seconds},{result.end_seconds}",
+            content=result.text,
+            timestamp=datetime.now(),
+            category="transcription",
+            user_id=user_id,
+        )
+        await repository.upsert(resource)
+
+    # 4. Cleanup
+    chunker.cleanup_chunks(chunks)
+```
+
+### Dreaming Worker
+
+```python
+# rem/workers/dreaming.py
+from rem.services.audio import AudioChunker, AudioTranscriber
+
+async def extract_moments_from_audio(audio_resource: Resource):
+    """Extract moments from audio transcription."""
+
+    # Audio already transcribed and stored as Resource
+    # Use transcription content to identify temporal moments
+
+    # Example: Split by speaker changes, topic shifts, etc.
+    moments = extract_temporal_segments(audio_resource.content)
+
+    for moment in moments:
+        await repository.upsert(moment)
+```
+
+## Logging
+
+All logs use loguru (REM standard):
+
+```python
+from loguru import logger
+
+# Chunker logs
+logger.info("Chunking audio: /path/to/file.m4a")
+logger.debug("Found silence at 58.3s (target: 60.0s)")
+logger.info("Created 5 chunks in /tmp/rem_audio_chunks_xyz")
+
+# Transcriber logs
+logger.info("Transcribing chunk 1/5 (58.0s - 118.0s)")
+logger.debug("Sending 2.3 MB to OpenAI Whisper API")
+logger.info("✓ Transcription complete: 245 characters")
+logger.info("Estimated cost: $0.180 (30.0 minutes)")
+```
+
+## Testing
+
+```bash
+# Run audio service tests
+pytest tests/unit/services/audio/
+
+# Test with real files (requires OpenAI API key)
+export OPENAI_API_KEY=sk-...
+pytest tests/integration/services/audio/
+```
+
+## Future Enhancements
+
+1. **Context Manager for Cleanup**
+   ```python
+   with AudioChunker() as chunker:
+       chunks = chunker.chunk_audio("file.m4a")
+       # Auto-cleanup on exit
+   ```
+
+2. **Batch Transcription**
+   - Parallel API requests
+   - Rate limiting
+   - Progress tracking
+
+3. **Speaker Diarization**
+   - Detect speaker changes
+   - Label speakers
+   - Split on speaker boundaries
+
+4. **Advanced Silence Detection**
+   - Machine learning-based VAD
+   - Energy-based fallback
+   - Adaptive thresholds
+
+5. **Format Detection**
+   - Auto-detect audio format
+   - Validate before processing
+   - Better error messages
+
+## References
+
+- [OpenAI Whisper API](https://platform.openai.com/docs/guides/speech-to-text)
+- [pydub Documentation](https://github.com/jiaaro/pydub)
+- [ffmpeg Documentation](https://ffmpeg.org/documentation.html)

rem/services/audio/__init__.py

@@ -0,0 +1,15 @@
+"""
+Audio processing service for REM.
+
+Lightweight audio processing with minimal dependencies:
+- wav module (stdlib) for WAV file handling
+- pydub (optional) for format conversion (M4A, MP3, etc.)
+- requests (already a dependency) for OpenAI Whisper API
+
+No torch, torchaudio, or other heavy ML dependencies.
+"""
+
+from .chunker import AudioChunker
+from .transcriber import AudioTranscriber
+
+__all__ = ["AudioChunker", "AudioTranscriber"]