media-toolkit 0.2.2__tar.gz → 0.2.4__tar.gz

{media_toolkit-0.2.2 → media_toolkit-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: media-toolkit
-Version: 0.2.2
+Version: 0.2.4
 Summary: Web-ready standardized file processing and serialization. Read, load and convert to standard file types with a common interface.
 Author: SocAIty
 License: MIT License
@@ -49,11 +49,11 @@ Requires-Dist: vidgear[core]; extra == "videofile"
 Dynamic: license-file
 
 
-  <h1 align="center" style="margin-top:-25px">MediaToolkit</h1>
+<h1 align="center" style="margin-top:-25px">MediaToolkit</h1>
 <p align="center">
   <img align="center" src="docs/media-file-icon.png" height="200" />
 </p>
-  <h3 align="center" style="margin-top:-10px">Web-ready standardized file processing and serialization</h3>
+<h3 align="center" style="margin-top:-10px">Web-ready standardized file processing and serialization</h3>
 
 
 # Features
@@ -64,7 +64,7 @@ Especially useful for code that works with multiple file types like images, audi
 Load and convert from and to common data types:
 - numpy arrays 
 - file paths 
-- bytes,
+- bytes
 - base64
 - json
 - urls
@@ -92,6 +92,7 @@ pip install media-toolkit[VideoFile]  # or [AudioFile, VideoFile, ...]
 # install from github for newest release
 pip install git+git://github.com/SocAIty/media-toolkit
 ```
+
 The package checks if you have missing dependencies for certain file types while using. 
 Use the ```--no-deps``` flag for a minimal tiny pure python installation.
 The package with dependencies is quite small < 39kb itself.
@@ -144,7 +145,58 @@ as_base64 = my_file.to_base64()
 as_json = my_file.to_json()
 ```
 
-### Working with VideoFiles.
+## Working with Collections of Files
+
+### MediaList
+A flexible list that can handle multiple media files with type safety:
+
+```python
+from media_toolkit import MediaList, AudioFile
+
+# Create a list that only accepts AudioFiles
+audio_list = MediaList[AudioFile]()
+
+# Add files to the list
+audio_list.append("path/to/audio.mp3")
+audio_list.extend(["url1", "url2"])
+
+# Process all files
+for audio in audio_list:
+    print(audio.file_size())
+
+# Convert all files to base64
+base64_files = audio_list.to_base64()
+```
+
+### MediaDict
+A dictionary for organizing media files with keys:
+
+```python
+from media_toolkit import MediaDict, ImageFile
+
+# Create a dictionary that only accepts ImageFiles
+image_dict = MediaDict[ImageFile]()
+
+# Add files with keys
+image_dict["profile"] = "path/to/profile.jpg"
+image_dict["banner"] = "https://example.com/banner.png"
+
+# Process files
+for key, image in image_dict.items():
+    print(f"{key}: {image.file_size()}")
+
+# Convert to JSON
+json_data = image_dict.to_json()
+```
+
+Both `MediaList` and `MediaDict` support:
+- Type safety with generic types (e.g., `MediaList[AudioFile]`)
+- Lazy loading of files
+- Batch processing
+- Common operations (to_base64, to_bytes, etc.)
+- Nested structures (MediaDict inside MediaList and vice versa)
+
+### Working with VideoFiles
 
 The VideoFiles wrap the famous [vidgear](https://abhitronix.github.io/vidgear/latest/) package as well as [pydub](https://github.com/jiaaro/pydub).
 VideoFiles support extra methods like audio extraction, combining video and audio.
@@ -193,7 +245,7 @@ You can use the files in fastapi and transform the starlette upload file to a Me
 ```python
 @app.post("/upload")
 async def upload_file(file: UploadFile = File(...)):
-    mf = ImageFile().from_starlette_upload_file(file)
+    mf = ImageFile().from_any(file)
     return {"filename": file.filename}
 ```
 
@@ -211,16 +263,12 @@ my_files = {
 response = httpx.Client().post(url, files=my_files)
 ```
 
-
 # How it works
 
 If media-file is instantiated with ```from_*``` it converts it to an intermediate representation.
 The ```to_*``` methods then convert it to the desired format.
 
-Currently the intermediate representation is supported in memory with (BytesIO).
-
+Currently the intermediate representation is supported in memory with (BytesIO) or on disk with temporary files.
 
 # ToDo:
-
-- [x] additionally support tempfile backend instead of working bytesio memory mode only.
 - [x] decreasing redundancies for _file_info() method

{media_toolkit-0.2.2 → media_toolkit-0.2.4}/README.md

@@ -1,9 +1,9 @@
 
-  <h1 align="center" style="margin-top:-25px">MediaToolkit</h1>
+<h1 align="center" style="margin-top:-25px">MediaToolkit</h1>
 <p align="center">
   <img align="center" src="docs/media-file-icon.png" height="200" />
 </p>
-  <h3 align="center" style="margin-top:-10px">Web-ready standardized file processing and serialization</h3>
+<h3 align="center" style="margin-top:-10px">Web-ready standardized file processing and serialization</h3>
 
 
 # Features
@@ -14,7 +14,7 @@ Especially useful for code that works with multiple file types like images, audi
 Load and convert from and to common data types:
 - numpy arrays 
 - file paths 
-- bytes,
+- bytes
 - base64
 - json
 - urls
@@ -42,6 +42,7 @@ pip install media-toolkit[VideoFile]  # or [AudioFile, VideoFile, ...]
 # install from github for newest release
 pip install git+git://github.com/SocAIty/media-toolkit
 ```
+
 The package checks if you have missing dependencies for certain file types while using. 
 Use the ```--no-deps``` flag for a minimal tiny pure python installation.
 The package with dependencies is quite small < 39kb itself.
@@ -94,7 +95,58 @@ as_base64 = my_file.to_base64()
 as_json = my_file.to_json()
 ```
 
-### Working with VideoFiles.
+## Working with Collections of Files
+
+### MediaList
+A flexible list that can handle multiple media files with type safety:
+
+```python
+from media_toolkit import MediaList, AudioFile
+
+# Create a list that only accepts AudioFiles
+audio_list = MediaList[AudioFile]()
+
+# Add files to the list
+audio_list.append("path/to/audio.mp3")
+audio_list.extend(["url1", "url2"])
+
+# Process all files
+for audio in audio_list:
+    print(audio.file_size())
+
+# Convert all files to base64
+base64_files = audio_list.to_base64()
+```
+
+### MediaDict
+A dictionary for organizing media files with keys:
+
+```python
+from media_toolkit import MediaDict, ImageFile
+
+# Create a dictionary that only accepts ImageFiles
+image_dict = MediaDict[ImageFile]()
+
+# Add files with keys
+image_dict["profile"] = "path/to/profile.jpg"
+image_dict["banner"] = "https://example.com/banner.png"
+
+# Process files
+for key, image in image_dict.items():
+    print(f"{key}: {image.file_size()}")
+
+# Convert to JSON
+json_data = image_dict.to_json()
+```
+
+Both `MediaList` and `MediaDict` support:
+- Type safety with generic types (e.g., `MediaList[AudioFile]`)
+- Lazy loading of files
+- Batch processing
+- Common operations (to_base64, to_bytes, etc.)
+- Nested structures (MediaDict inside MediaList and vice versa)
+
+### Working with VideoFiles
 
 The VideoFiles wrap the famous [vidgear](https://abhitronix.github.io/vidgear/latest/) package as well as [pydub](https://github.com/jiaaro/pydub).
 VideoFiles support extra methods like audio extraction, combining video and audio.
@@ -143,7 +195,7 @@ You can use the files in fastapi and transform the starlette upload file to a Me
 ```python
 @app.post("/upload")
 async def upload_file(file: UploadFile = File(...)):
-    mf = ImageFile().from_starlette_upload_file(file)
+    mf = ImageFile().from_any(file)
     return {"filename": file.filename}
 ```
 
@@ -161,16 +213,12 @@ my_files = {
 response = httpx.Client().post(url, files=my_files)
 ```
 
-
 # How it works
 
 If media-file is instantiated with ```from_*``` it converts it to an intermediate representation.
 The ```to_*``` methods then convert it to the desired format.
 
-Currently the intermediate representation is supported in memory with (BytesIO).
-
+Currently the intermediate representation is supported in memory with (BytesIO) or on disk with temporary files.
 
 # ToDo:
-
-- [x] additionally support tempfile backend instead of working bytesio memory mode only.
-- [x] decreasing redundancies for _file_info() method
+- [x] decreasing redundancies for _file_info() method

media_toolkit-0.2.4/media_toolkit/__init__.py

@@ -0,0 +1,4 @@
+from media_toolkit.core import MediaFile, ImageFile, VideoFile, AudioFile, MediaList, MediaDict
+from media_toolkit.utils.file_conversion import media_from_file, media_from_any
+
+__all__ = ["MediaFile", "ImageFile", "VideoFile", "AudioFile", "MediaList", "MediaDict", "media_from_file", "media_from_any"]

media_toolkit-0.2.4/media_toolkit/core/IMediaFile.py

@@ -0,0 +1,86 @@
+from abc import ABC, abstractmethod
+import io
+from typing import Dict, Any, Optional
+
+
+class IMediaFile(ABC):
+    """
+    Abstract base interface defining the core contract for media file handling
+    in the Media Toolkit ecosystem.
+    """
+    @abstractmethod
+    def from_any(self, data: Any, allow_reads_from_disk: bool = True) -> 'IMediaFile':
+        """
+        Load file content from various input sources.
+
+        Args:
+            data: Input source (bytes, file path, URL, base64, etc.)
+            allow_reads_from_disk: Flag to control disk file reading
+
+        Returns:
+            Self, for method chaining
+        """
+        pass
+
+    @abstractmethod
+    def to_bytes(self) -> bytes:
+        """Convert file content to raw bytes."""
+        pass
+
+    @abstractmethod
+    def to_base64(self) -> str:
+        """Encode file content to base64."""
+        pass
+
+    @abstractmethod
+    def to_bytes_io(self) -> io.BytesIO:
+        """Convert file content to BytesIO object."""
+        pass
+
+    @abstractmethod
+    def to_httpx_send_able_tuple(self) -> tuple:
+        """
+        Prepare file for HTTP transmission.
+
+        Returns:
+            Tuple of (filename, content, content_type)
+        """
+        pass
+
+    @abstractmethod
+    def save(self, path: Optional[str] = None):
+        """
+        Save file to specified path.
+
+        Args:
+            path: Destination path. Uses current directory if None.
+        """
+        pass
+
+    @abstractmethod
+    def file_size(self, unit: str = "bytes") -> float:
+        """
+        Get file size in specified units.
+
+        Args:
+            unit: Size unit (bytes, kb, mb, gb)
+
+        Returns:
+            File size in specified unit
+        """
+        pass
+
+    @abstractmethod
+    def to_json(self) -> Dict[str, Any]:
+        """
+        Serialize file to JSON-compatible dictionary.
+
+        Returns:
+            Dictionary representation of the file
+        """
+        pass
+
+    @abstractmethod
+    def __sizeof__(self):
+        """Get the size of the file in bytes."""
+        pass

media_toolkit-0.2.4/media_toolkit/core/MediaDict.py

@@ -0,0 +1,353 @@
+import io
+import uuid
+from typing import List, Union, Optional, Any, Dict, TypeVar, Generic
+from media_toolkit.core.IMediaFile import IMediaFile
+from media_toolkit.core.media_file import MediaFile
+from media_toolkit.core.MediaList import MediaList
+
+T = TypeVar('T', bound=IMediaFile)
+
+
+class MediaDict(IMediaFile, Generic[T]):
+    """
+    A flexible media file dictionary that handles multiple file types
+    and sources with configurable loading behaviors.
+
+    Supports:
+    - Multiple MediaFile types as dictionary values
+    - Batch media processing
+    - Generic type restrictions (e.g. MediaDict[AudioFile])
+    """
+    def __init__(
+            self,
+            files: Optional[Dict[str, Union[str, T, MediaList[T], 'MediaDict[T]']]] = None,
+            download_files: bool = True,
+            read_system_files: bool = True,
+            file_name: str = "MediaDict",
+            use_temp_file: bool = False,
+            temp_dir: str = None
+    ):
+        """
+        Initialize MediaDict with optional files and configuration.
+
+        Args:
+            files: Dictionary of files with keys as identifiers
+            download_files: Flag if files provided as URLs are downloaded and converted
+            read_system_files: Flag if files provided as paths are read and converted
+            file_name: Name of the media dictionary
+            use_temp_file: Flag to use temp file for file processing
+            temp_dir: Temp directory path for file processing
+        """
+        self.file_name = file_name
+        self.use_temp_file = use_temp_file
+        self.temp_dir = temp_dir
+        self.download_files = download_files
+        self.read_system_files = read_system_files
+        self._media_files: Dict[str, Union[str, T, MediaList[T]]] = {}
+
+        if files:
+            self.update(files)
+
+    @staticmethod
+    def _is_empty_file(file: Any) -> bool:
+        """ Check if file has any content. """
+        if isinstance(file, list) and all(MediaDict._is_empty_file(item) for item in file):
+            return True
+        return file is None or (hasattr(file, '__len__') and len(file) == 0)
+            
+    def _process_file(
+            self,
+            file: Union[str, T, MediaList[T], 'MediaDict[T]']
+    ) -> Union[str, T, MediaList[T], 'MediaDict[T]']:
+        """
+        Process a single file based on configuration.
+
+        Args:
+            file: File to process (URL, path, MediaFile, MediaList)
+        Returns:
+            Processed file (MediaFile, MediaList, or original str)
+        """
+        if isinstance(file, (IMediaFile, MediaList, MediaDict)):
+            return file
+
+        # check if is empty
+        if MediaDict._is_empty_file(file):
+            return file
+
+        # perform conversion
+        if isinstance(file, str):
+            if MediaFile._is_url(file):
+                if not self.download_files:
+                    return file
+                return MediaFile(use_temp_file=self.use_temp_file, temp_dir=self.temp_dir).from_url(file)
+
+            if MediaFile._is_valid_file_path(file):
+                if not self.read_system_files:
+                    return file
+                return MediaFile(use_temp_file=self.use_temp_file, temp_dir=self.temp_dir).from_file(file)
+
+        if MediaFile._is_file_model(file):
+            from media_toolkit.utils import media_from_FileModel
+            return media_from_FileModel(file, allow_reads_from_disk=self.read_system_files)
+
+        if isinstance(file, list):
+            return MediaList[T](
+                files=file,
+                download_files=self.download_files,
+                read_system_files=self.read_system_files,
+                use_temp_file=self.use_temp_file,
+                temp_dir=self.temp_dir
+            )
+
+        if isinstance(file, dict):
+            return MediaDict[T](
+                files=file,
+                download_files=self.download_files,
+                read_system_files=self.read_system_files,
+                use_temp_file=self.use_temp_file,
+                temp_dir=self.temp_dir
+            )
+
+        return MediaFile(use_temp_file=self.use_temp_file, temp_dir=self.temp_dir).from_any(file)
+
+    def from_any(
+            self,
+            data: Union[Dict[str, Union[str, T, MediaList[T]]], Any]
+    ) -> 'MediaDict[T]':
+        """
+        Load files from a dictionary of files.
+
+        Args:
+            data: Dictionary of files to load
+        Returns:
+            Self, for method chaining
+        """
+        self.update(data)
+        return self
+
+    def get_processable_files(
+            self,
+            ignore_all_potential_errors: bool = False,
+            raise_exception: bool = True,
+            silent: bool = False
+    ) -> 'MediaDict[T]':
+        """
+        Validate that all files can be processed for batch operations.
+
+        Args:
+            ignore_all_potential_errors: Ignore processing errors
+            raise_exception: Raise exceptions for unprocessable files
+            silent: Suppress error messages
+        Returns:
+            Dictionary of processable files
+        """
+        if ignore_all_potential_errors:
+            return self
+
+        processable_files = {
+            key: file for key, file in self._media_files.items()
+            if isinstance(file, (IMediaFile, MediaList))
+        }
+
+        if len(processable_files) != len(self._media_files):
+            not_processable_file_names = [
+                str(key) for key, file in self._media_files.items()
+                if file not in processable_files.values()
+            ]
+            message = (
+                f"Files not processed: {not_processable_file_names}. "
+                f"Check configuration (download_files={self.download_files}, "
+                f"read_system_files={self.read_system_files})"
+            )
+
+            if raise_exception:
+                raise ValueError(message)
+            if not silent:
+                print(message)
+
+            return self._shallow_copy_with_settings(processable_files)
+
+        return self
+
+    def _shallow_copy_with_settings(self, data: dict | None = None) -> 'MediaDict[T]':
+        """
+        Creates a new MediaDict with the same settings but shallow copies the media files dictionary.
+        This avoids re-reading all files when creating a copy.
+        """
+        md = MediaDict[T](
+            file_name=self.file_name, download_files=self.download_files,
+            read_system_files=self.read_system_files, use_temp_file=self.use_temp_file, temp_dir=self.temp_dir
+        )
+        md._media_files = data
+        return md
+
+    def get_url_files(self) -> Union['MediaDict[T]', dict]:
+        """
+        Get all non-processed files that are URLs.
+
+        Returns:
+            Dictionary of URL files
+        """
+        if self.download_files:
+            return {}
+
+        return self._shallow_copy_with_settings({
+            key: file for key, file in self._media_files.items()
+            if isinstance(file, str) and MediaFile._is_url(file)
+        })
+
+    def get_file_path_files(self) -> Union['MediaDict[T]', dict]:
+        """
+        Get all non-processed files that are file paths.
+
+        Returns:
+            Dictionary of file path files
+        """
+        if self.read_system_files:
+            return {}
+        return self._shallow_copy_with_settings({
+            key: file for key, file in self._media_files.items()
+            if isinstance(file, str) and MediaFile._is_valid_file_path(file)
+        })
+
+    def to_base64(self) -> Dict[str, str]:
+        """Convert all processable files to base64."""
+        return {
+            key: file.to_base64()
+            for key, file in self.get_processable_files(raise_exception=False).items()
+        }
+
+    def to_bytes_io(self) -> Dict[str, io.BytesIO]:
+        """Convert all processable files to BytesIO."""
+        return {
+            key: file.to_bytes_io()
+            for key, file in self.get_processable_files(raise_exception=False).items()
+        }
+
+    def file_size(self, unit: str = "bytes") -> float:
+        """Calculate total file size."""
+        return sum(
+            file.file_size(unit)
+            for file in self.get_processable_files(raise_exception=False).values()
+        )
+
+    def to_json(self) -> Dict[str, Any]:
+        """Convert files to JSON representation."""
+        files = self.get_processable_files(ignore_all_potential_errors=True)
+        return {
+            key: (file.to_json() if isinstance(file, (IMediaFile, MediaList)) else file)
+            for key, file in files.items()
+        }
+
+    def to_bytes(self) -> Dict[str, bytes]:
+        """Convert all processable files to bytes."""
+        return {
+            key: file.to_bytes()
+            for key, file in self.get_processable_files(raise_exception=False).items()
+        }
+
+    def to_httpx_send_able_tuple(self) -> List[tuple] | dict:
+        """
+        Convert files to httpx-send-able format.
+
+        Args:
+            param_name: Optional parameter name for API endpoint
+        Returns:
+            List of tuples  for httpx file transmission
+        """
+        files = self.get_processable_files(raise_exception=False, silent=True)
+
+        ret = []
+        for k, file in files.items():
+            if isinstance(file, MediaList):
+                ret.extend(file.to_httpx_sendable_tuple(k))
+            elif isinstance(file, MediaDict):
+                fls = file.to_httpx_sendable_tuple()
+                if isinstance(fls, dict):
+                    ret.append((k, fls))
+                else:
+                    ret.extend(fls)
+            else:
+                ret.append((k, file.to_httpx_send_able_tuple()))
+
+        if len(ret) == 1:
+            return {ret[0][0]: ret[0][1]}
+        return ret
+
+    def save(self, directory: Optional[str] = None):
+        """
+        Save all processable files to a specified directory.
+
+        Args:
+            directory: Target directory (uses current directory if None)
+        """
+        import os
+        directory = directory or os.path.curdir
+        os.makedirs(directory, exist_ok=True)
+
+        for key, file in self.get_processable_files(raise_exception=False).items():
+            file.save(directory)
+
+    def __getitem__(self, key: str):
+        """Allow dictionary-style access."""
+        return self._media_files[key]
+
+    def __setitem__(self, key: str, value: Union[str, T, MediaList[T]]):
+        """Allow dictionary-style assignment with processing."""
+        self._media_files[key] = self._process_file(value)
+
+    def __delitem__(self, key: str):
+        """Allow dictionary-style deletion."""
+        del self._media_files[key]
+
+    def __iter__(self):
+        """Make the class iterable."""
+        return iter(self._media_files)
+
+    def __len__(self):
+        """Return the number of files in the dictionary."""
+        return len(self._media_files)
+
+    def __contains__(self, key: str):
+        """Check if a key exists in the dictionary."""
+        return key in self._media_files
+
+    def keys(self):
+        """Return dictionary keys."""
+        return self._media_files.keys()
+
+    def values(self):
+        """Return dictionary values."""
+        return self._media_files.values()
+
+    def items(self):
+        """Return dictionary items."""
+        return self._media_files.items()
+
+    def update(self, files: Union['MediaDict[T]', Dict[str, Union[str, T, MediaList[T]]]]):
+        """
+        Update the dictionary with new files.
+
+        Args:
+            files: Dictionary of files to add or update
+        """
+        if files is None:
+            return
+
+        if not isinstance(files, dict) and not isinstance(files, MediaDict):
+            files = {str(uuid.uuid4()): files}
+
+        for key, file in files.items():
+            self[key] = self._process_file(file)
+
+    def __sizeof__(self):
+        """Returns the memory size of the instance + actual file/buffer size."""
+        size = super().__sizeof__() + self.file_size("bytes")
+        return size
+
+    def to_dict(self) -> Dict[str, Union[str, T, MediaList[T]]]:
+        """Convert MediaDict to a standard dictionary."""
+        return {
+            key: (file.to_dict() if isinstance(file, MediaDict) else file)
+            for key, file in self._media_files.items()
+        }