content-core 1.6.0__tar.gz → 1.7.0__tar.gz

{content_core-1.6.0 → content_core-1.7.0}/CHANGELOG.md

@@ -11,6 +11,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Pure Python file type detection via the new `FileDetector` class
 - Comprehensive file signature detection for 25+ file formats
 - Smart detection for ZIP-based formats (DOCX, XLSX, PPTX, EPUB)
+- Custom audio model configuration - override speech-to-text provider and model at runtime
+  - Pass `audio_provider` and `audio_model` parameters through `extract_content()` API
+  - Supports any provider/model combination available through Esperanto library
+  - Maintains full backward compatibility - existing code works unchanged
+  - Includes validation with helpful warnings and error messages
 
 ### Changed
 - File type detection now uses pure Python implementation instead of libmagic

{content_core-1.6.0 → content_core-1.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: content-core
-Version: 1.6.0
+Version: 1.7.0
 Summary: Extract what matters from any media source. Available as Python Library, macOS Service, CLI and MCP Server
 Author-email: LUIS NOVO <lfnovo@gmail.com>
 License-File: LICENSE
@@ -263,6 +263,14 @@ cleaned_text = await cc.clean("...messy text with [brackets] and extra spaces...
 
 # Summarize content with optional context
 summary = await cc.summarize_content("long article text", context="explain to a child")
+
+# Extract audio with custom speech-to-text model
+from content_core.common import ProcessSourceInput
+result = await cc.extract(ProcessSourceInput(
+    file_path="interview.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
 ```
 
 ## Documentation

{content_core-1.6.0 → content_core-1.7.0}/README.md

@@ -228,6 +228,14 @@ cleaned_text = await cc.clean("...messy text with [brackets] and extra spaces...
 
 # Summarize content with optional context
 summary = await cc.summarize_content("long article text", context="explain to a child")
+
+# Extract audio with custom speech-to-text model
+from content_core.common import ProcessSourceInput
+result = await cc.extract(ProcessSourceInput(
+    file_path="interview.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
 ```
 
 ## Documentation

{content_core-1.6.0 → content_core-1.7.0}/docs/usage.md

@@ -362,6 +362,203 @@ result = await cc.extract({"file_path": "conference_talk.mp4"})
 print(result.content)  # Full transcript
 ```
 
+## Custom Audio Model Configuration
+
+Content Core allows you to override the default speech-to-text model at runtime, enabling you to choose different AI providers and models based on your specific needs (language support, cost, accuracy, etc.).
+
+### Overview
+
+By default, audio and video files are transcribed using the model configured in `models_config.yaml` (typically OpenAI Whisper-1). You can override this on a per-call basis by specifying both `audio_provider` and `audio_model` parameters.
+
+**Key Features:**
+- ✅ **Runtime flexibility**: Choose different models for different use cases
+- ✅ **Backward compatible**: Existing code works unchanged
+- ✅ **Multiple providers**: Support for any provider supported by Esperanto
+- ✅ **Automatic fallback**: Graceful handling of invalid configurations
+
+### Basic Usage
+
+```python
+from content_core.common import ProcessSourceInput
+import content_core as cc
+
+# Use custom audio model for transcription
+result = await cc.extract(ProcessSourceInput(
+    file_path="interview.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
+
+print(result.content)  # Transcribed text using specified model
+```
+
+### Supported Providers
+
+Content Core uses the Esperanto library for AI model abstraction, which supports multiple providers:
+
+- **OpenAI**: `provider="openai"`, models: `whisper-1`
+- **Google**: `provider="google"`, models: `chirp` (if available)
+- **Other providers**: Any provider supported by Esperanto
+
+Check the [Esperanto documentation](https://github.com/yourusername/esperanto) for the full list of supported providers and models.
+
+### Use Cases
+
+**Multilingual Transcription:**
+```python
+from content_core.common import ProcessSourceInput
+import content_core as cc
+
+# Use a model optimized for a specific language
+result = await cc.extract(ProcessSourceInput(
+    file_path="spanish_interview.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"  # Whisper supports 99 languages
+))
+```
+
+**Cost Optimization:**
+```python
+from content_core.common import ProcessSourceInput
+import content_core as cc
+
+# Use different models based on quality requirements
+# For high-value content, use premium model
+premium_result = await cc.extract(ProcessSourceInput(
+    file_path="important_meeting.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
+
+# For casual content, use default or cost-effective model
+casual_result = await cc.extract(ProcessSourceInput(
+    file_path="casual_recording.mp3"
+    # No custom params = uses default configured model
+))
+```
+
+**Video Transcription with Custom Model:**
+```python
+from content_core.common import ProcessSourceInput
+import content_core as cc
+
+# Custom model works for video files too (audio is extracted automatically)
+result = await cc.extract(ProcessSourceInput(
+    file_path="conference_presentation.mp4",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
+```
+
+### Parameter Requirements
+
+Both `audio_provider` and `audio_model` must be specified together:
+
+```python
+# ✅ CORRECT: Both parameters provided
+result = await cc.extract(ProcessSourceInput(
+    file_path="audio.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
+
+# ✅ CORRECT: Neither parameter (uses default)
+result = await cc.extract(ProcessSourceInput(
+    file_path="audio.mp3"
+))
+
+# ⚠️ WARNING: Only one parameter (logs warning, uses default)
+result = await cc.extract(ProcessSourceInput(
+    file_path="audio.mp3",
+    audio_provider="openai"  # Missing audio_model
+))
+# Logs: "audio_provider provided without audio_model. Both must be specified together. Falling back to default model."
+```
+
+### Error Handling
+
+Content Core gracefully handles invalid model configurations:
+
+**Invalid Provider:**
+```python
+result = await cc.extract(ProcessSourceInput(
+    file_path="audio.mp3",
+    audio_provider="invalid_provider",
+    audio_model="whisper-1"
+))
+# Logs error and falls back to default model
+# Transcription continues successfully
+```
+
+**Invalid Model Name:**
+```python
+result = await cc.extract(ProcessSourceInput(
+    file_path="audio.mp3",
+    audio_provider="openai",
+    audio_model="nonexistent-model"
+))
+# Logs error and falls back to default model
+# Transcription continues successfully
+```
+
+**Error Message Example:**
+```
+ERROR: Failed to create custom audio model 'invalid_provider/whisper-1': Unsupported provider.
+Check that the provider and model are supported by Esperanto. Falling back to default model.
+```
+
+### Concurrency Control
+
+Custom audio models respect the same concurrency limits as the default model (configured via `CCORE_AUDIO_CONCURRENCY` or `set_audio_concurrency()`). This ensures consistent API rate limit handling regardless of which model you use.
+
+```python
+from content_core.config import set_audio_concurrency
+from content_core.common import ProcessSourceInput
+import content_core as cc
+
+# Set concurrency for all transcriptions (default and custom models)
+set_audio_concurrency(5)
+
+# Both use the same concurrency limit
+default_result = await cc.extract(ProcessSourceInput(file_path="audio1.mp3"))
+custom_result = await cc.extract(ProcessSourceInput(
+    file_path="audio2.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
+```
+
+### Backward Compatibility
+
+All existing code continues to work without any changes:
+
+```python
+import content_core as cc
+
+# Old code (no custom params) - still works perfectly
+result = await cc.extract("audio.mp3")
+result = await cc.extract({"file_path": "audio.mp3"})
+
+# New capability (optional custom params)
+from content_core.common import ProcessSourceInput
+result = await cc.extract(ProcessSourceInput(
+    file_path="audio.mp3",
+    audio_provider="openai",
+    audio_model="whisper-1"
+))
+```
+
+### Troubleshooting
+
+**Issue**: "Both audio_provider and audio_model must be specified together"
+- **Solution**: Provide both parameters or neither. Don't specify just one.
+
+**Issue**: "Failed to create custom audio model"
+- **Solution**: Verify the provider and model are supported by Esperanto. Check your API keys are configured correctly.
+
+**Issue**: Custom model seems to be ignored
+- **Solution**: Ensure you're using `ProcessSourceInput` class (not plain dict) when passing custom parameters.
+
 ## File Type Detection
 
 Content Core uses a pure Python implementation for file type detection, eliminating the need for system dependencies like libmagic. This ensures consistent behavior across all platforms (Windows, macOS, Linux).

{content_core-1.6.0 → content_core-1.7.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "content-core"
-version = "1.6.0"
+version = "1.7.0"
 description = "Extract what matters from any media source. Available as Python Library, macOS Service, CLI and MCP Server"
 readme = "README.md"
 homepage = "https://github.com/lfnovo/content-core"

{content_core-1.6.0 → content_core-1.7.0}/src/content_core/common/state.py

@@ -27,6 +27,14 @@ class ProcessSourceState(BaseModel):
         default=None,
         description="Override Docling output format: 'markdown', 'html', or 'json'",
     )
+    audio_provider: Optional[str] = Field(
+        default=None,
+        description="Override speech-to-text provider (e.g., 'openai', 'google')",
+    )
+    audio_model: Optional[str] = Field(
+        default=None,
+        description="Override speech-to-text model name (e.g., 'whisper-1', 'chirp')",
+    )
 
 
 class ProcessSourceInput(BaseModel):
@@ -36,6 +44,8 @@ class ProcessSourceInput(BaseModel):
     document_engine: Optional[str] = None
     url_engine: Optional[str] = None
     output_format: Optional[str] = None
+    audio_provider: Optional[str] = None
+    audio_model: Optional[str] = None
 
 
 class ProcessSourceOutput(BaseModel):

{content_core-1.6.0 → content_core-1.7.0}/src/content_core/processors/audio.py

@@ -190,8 +190,39 @@ async def extract_audio_data(data: ProcessSourceState):
 
             # Transcribe audio files in parallel with concurrency limit
             from content_core.models import ModelFactory
+            from esperanto import AIFactory
+
+            # Determine which model to use based on state parameters
+            if data.audio_provider and data.audio_model:
+                # Custom model provided - create new instance
+                try:
+                    logger.info(
+                        f"Using custom audio model: {data.audio_provider}/{data.audio_model}"
+                    )
+                    speech_to_text_model = AIFactory.create_speech_to_text(
+                        data.audio_provider, data.audio_model
+                    )
+                except Exception as e:
+                    logger.error(
+                        f"Failed to create custom audio model '{data.audio_provider}/{data.audio_model}': {e}. "
+                        f"Check that the provider and model are supported by Esperanto. "
+                        f"Falling back to default model."
+                    )
+                    speech_to_text_model = ModelFactory.get_model("speech_to_text")
+            elif data.audio_provider or data.audio_model:
+                # Only one parameter provided - log warning and use default
+                missing = "audio_model" if data.audio_provider else "audio_provider"
+                provided = "audio_provider" if data.audio_provider else "audio_model"
+                logger.warning(
+                    f"{provided} provided without {missing}. "
+                    f"Both audio_provider and audio_model must be specified together. "
+                    f"Falling back to default model."
+                )
+                speech_to_text_model = ModelFactory.get_model("speech_to_text")
+            else:
+                # No custom parameters - use default (backward compatible)
+                speech_to_text_model = ModelFactory.get_model("speech_to_text")
 
-            speech_to_text_model = ModelFactory.get_model("speech_to_text")
             concurrency = get_audio_concurrency()
             semaphore = asyncio.Semaphore(concurrency)
 

{content_core-1.6.0 → content_core-1.7.0}/src/content_core/templated_message.py

@@ -2,7 +2,6 @@ from typing import Dict, Optional, Union
 
 from ai_prompter import Prompter
 from esperanto import LanguageModel
-from esperanto.common_types import Message
 from pydantic import BaseModel, Field
 
 from content_core.models import ModelFactory

{content_core-1.6.0 → content_core-1.7.0}/tests/integration/conftest.py

@@ -1,7 +1,6 @@
 """Pytest configuration for integration tests."""
 import asyncio
 import gc
-import warnings
 
 import pytest