deffcode 0.2.7__tar.gz → 0.2.9__tar.gz

{deffcode-0.2.7 → deffcode-0.2.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deffcode
-Version: 0.2.7
+Version: 0.2.9
 Summary: A cross-platform High-performance & Flexible Real-time Video Frames Decoder in Python.
 Author-email: Abhishek Thakur <abhi.una12@gmail.com>
 License: Apache License 2.0
@@ -22,11 +22,11 @@ Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Science/Research
 Classifier: Intended Audience :: Education
 Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
-Requires-Python: >=3.10
+Classifier: Programming Language :: Python :: 3.14
+Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: cython
@@ -108,12 +108,13 @@ Here are some key features that stand out:
 - Hands down the easiest [**Index based Camera Device Capturing**][decoding-camera-devices-using-indexes], similar to OpenCV.
 - Easy to code **Real-time [Simple][transcoding-live-simple-filtergraphs] & [Complex][transcoding-live-complex-filtergraphs] Filtergraphs**. _(Yes, You read it correctly "Real-time"!)_
 - Native **[Multi-Input Source Configurations][multi-input-source-configurations]** support for decoding complex topologies.
+- Native **[Multi-Outputs Configurations][multi-outputs-configurations]** support for recording/streaming while decoding, in a single FFmpeg pipeline.
 - Lightning fast dedicated **GPU-Accelerated Video [Decoding][hardware-accelerated-video-decoding] & [Transcoding][hardware-accelerated-video-transcoding]**.
 - Enables precise FFmpeg [**Key-frame Seeking**][extracting-key-frames-as-png-image] with pinpoint accuracy.
 - Effortless [**Metadata Extraction**][extracting-video-metadata] from all streams available in the source.
 - Maintains the standard easy to learn [**OpenCV-Python**](https://docs.opencv.org/4.x/d6/d00/tutorial_py_root.html) coding syntax.
 - Out-of-the-box support for all prominent Computer Vision libraries.
-- Cross-platform, runs on Python 3.10+, and easy to install.
+- Cross-platform, runs on Python 3.11+, and easy to install.
 
 <!--
 - [x] Lossless Transcoding support with [WriteGear](https://abhitronix.github.io/deffcode/latest/gears/writegear/introduction/). #TODO
@@ -235,6 +236,11 @@ Once you have DeFFcode installed, checkout our Well-Documented **[Recipes 🍱][
 - [CUDA-NVENC-accelerated Video Transcoding with WriteGear API][cuda-nvenc-accelerated-video-transcoding-with-writegear-api]
 - [CUDA-NVENC-accelerated End-to-end Lossless Video Transcoding with WriteGear API][cuda-nvenc-accelerated-end-to-end-lossless-video-transcoding-with-writegear-api]
 - [Multi-Input Source Configurations][multi-input-source-configurations]
+  - [Decoding multiple inputs as 2x2 video grid _(mosaic)_][decoding-multiple-inputs-as-2x2-video-grid-mosaic]
+  - [Decoding crossfade transition between multiple inputs][decoding-crossfade-transition-between-multiple-inputs]
+- [Multi-Outputs Configurations][multi-outputs-configurations]
+  - [Recording while decoding][recording-while-decoding]
+  - [Encode once, duplicate to file and stream][encode-once-duplicate-to-file-and-stream]
 
 </details>
 
@@ -271,9 +277,8 @@ decoder = FFdecoder("https://abhitronix.github.io/html/Big_Buck_Bunny_1080_10s_1
 
 # grab RGB24(default) 3D frames from decoder
 for frame in decoder.generateFrame():
-
     # lets print its shape
-    print(frame.shape) # (1080, 1920, 3)
+    print(frame.shape)  # (1080, 1920, 3)
 
 # terminate the decoder
 decoder.terminate()
@@ -288,7 +293,9 @@ The primary function of [**Sourcer API**][sourcer-api] is to **gather metadata i
 from deffcode import Sourcer
 
 # initialize and formulate the decoder using suitable source
-sourcer = Sourcer("https://abhitronix.github.io/html/Big_Buck_Bunny_1080_10s_1MB.mp4").probe_stream()
+sourcer = Sourcer(
+    "https://abhitronix.github.io/html/Big_Buck_Bunny_1080_10s_1MB.mp4"
+).probe_stream()
 
 # print metadata as `json.dump`
 print(sourcer.retrieve_metadata(pretty_json=True))
@@ -354,18 +361,18 @@ It is something I am doing with my own free time. But so much more needs to be d
 
 Here is a Bibtex entry you can use to cite this project in a publication:
 
-[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.12689394.svg)](https://doi.org/10.5281/zenodo.12689394)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23446604.svg)](https://doi.org/10.5281/zenodo.23446604)
 
 ```BibTeX
 @software{deffcode,
   author       = {Abhishek Thakur},
-  title        = {abhiTronix/deffcode: v0.2.6},
-  month        = jul,
-  year         = 2024,
+  title        = {abhiTronix/deffcode: v0.2.8},
+  month        = oct,
+  year         = 2026,
   publisher    = {Zenodo},
-  version      = {v0.2.6},
-  doi          = {10.5281/zenodo.12689394},
-  url          = {https://doi.org/10.5281/zenodo.12689394},
+  version      = {v0.2.8},
+  doi          = {10.5281/zenodo.23446604},
+  url          = {https://doi.org/10.5281/zenodo.23446604},
 }
 ```
 
@@ -478,6 +485,11 @@ Advanced Recipes
 [smart-keyframe-only-decoding-for-heavy-ai-inference]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/extract-frame-metadata/#smart-keyframe-only-decoding-for-heavy-ai-inference
 [variable-frame-rate-vfr-synchronization-via-pts_time]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/extract-frame-metadata/#variable-frame-rate-vfr-synchronization-via-pts_time
 [multi-input-source-configurations]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_input/#multi-input-source-configurations
+[decoding-multiple-inputs-as-2x2-video-grid-mosaic]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_input/#decoding-multiple-inputs-as-2x2-video-grid-mosaic
+[decoding-crossfade-transition-between-multiple-inputs]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_input/#decoding-crossfade-transition-between-multiple-inputs
+[multi-outputs-configurations]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_output/#multi-outputs-configurations
+[recording-while-decoding]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_output/#recording-while-decoding
+[encode-once-duplicate-to-file-and-stream]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_output/#encode-once-duplicate-to-file-and-stream
 
 <!--
 CI Apps URLs

{deffcode-0.2.7 → deffcode-0.2.9}/README.md

@@ -68,12 +68,13 @@ Here are some key features that stand out:
 - Hands down the easiest [**Index based Camera Device Capturing**][decoding-camera-devices-using-indexes], similar to OpenCV.
 - Easy to code **Real-time [Simple][transcoding-live-simple-filtergraphs] & [Complex][transcoding-live-complex-filtergraphs] Filtergraphs**. _(Yes, You read it correctly "Real-time"!)_
 - Native **[Multi-Input Source Configurations][multi-input-source-configurations]** support for decoding complex topologies.
+- Native **[Multi-Outputs Configurations][multi-outputs-configurations]** support for recording/streaming while decoding, in a single FFmpeg pipeline.
 - Lightning fast dedicated **GPU-Accelerated Video [Decoding][hardware-accelerated-video-decoding] & [Transcoding][hardware-accelerated-video-transcoding]**.
 - Enables precise FFmpeg [**Key-frame Seeking**][extracting-key-frames-as-png-image] with pinpoint accuracy.
 - Effortless [**Metadata Extraction**][extracting-video-metadata] from all streams available in the source.
 - Maintains the standard easy to learn [**OpenCV-Python**](https://docs.opencv.org/4.x/d6/d00/tutorial_py_root.html) coding syntax.
 - Out-of-the-box support for all prominent Computer Vision libraries.
-- Cross-platform, runs on Python 3.10+, and easy to install.
+- Cross-platform, runs on Python 3.11+, and easy to install.
 
 <!--
 - [x] Lossless Transcoding support with [WriteGear](https://abhitronix.github.io/deffcode/latest/gears/writegear/introduction/). #TODO
@@ -195,6 +196,11 @@ Once you have DeFFcode installed, checkout our Well-Documented **[Recipes 🍱][
 - [CUDA-NVENC-accelerated Video Transcoding with WriteGear API][cuda-nvenc-accelerated-video-transcoding-with-writegear-api]
 - [CUDA-NVENC-accelerated End-to-end Lossless Video Transcoding with WriteGear API][cuda-nvenc-accelerated-end-to-end-lossless-video-transcoding-with-writegear-api]
 - [Multi-Input Source Configurations][multi-input-source-configurations]
+  - [Decoding multiple inputs as 2x2 video grid _(mosaic)_][decoding-multiple-inputs-as-2x2-video-grid-mosaic]
+  - [Decoding crossfade transition between multiple inputs][decoding-crossfade-transition-between-multiple-inputs]
+- [Multi-Outputs Configurations][multi-outputs-configurations]
+  - [Recording while decoding][recording-while-decoding]
+  - [Encode once, duplicate to file and stream][encode-once-duplicate-to-file-and-stream]
 
 </details>
 
@@ -231,9 +237,8 @@ decoder = FFdecoder("https://abhitronix.github.io/html/Big_Buck_Bunny_1080_10s_1
 
 # grab RGB24(default) 3D frames from decoder
 for frame in decoder.generateFrame():
-
     # lets print its shape
-    print(frame.shape) # (1080, 1920, 3)
+    print(frame.shape)  # (1080, 1920, 3)
 
 # terminate the decoder
 decoder.terminate()
@@ -248,7 +253,9 @@ The primary function of [**Sourcer API**][sourcer-api] is to **gather metadata i
 from deffcode import Sourcer
 
 # initialize and formulate the decoder using suitable source
-sourcer = Sourcer("https://abhitronix.github.io/html/Big_Buck_Bunny_1080_10s_1MB.mp4").probe_stream()
+sourcer = Sourcer(
+    "https://abhitronix.github.io/html/Big_Buck_Bunny_1080_10s_1MB.mp4"
+).probe_stream()
 
 # print metadata as `json.dump`
 print(sourcer.retrieve_metadata(pretty_json=True))
@@ -314,18 +321,18 @@ It is something I am doing with my own free time. But so much more needs to be d
 
 Here is a Bibtex entry you can use to cite this project in a publication:
 
-[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.12689394.svg)](https://doi.org/10.5281/zenodo.12689394)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23446604.svg)](https://doi.org/10.5281/zenodo.23446604)
 
 ```BibTeX
 @software{deffcode,
   author       = {Abhishek Thakur},
-  title        = {abhiTronix/deffcode: v0.2.6},
-  month        = jul,
-  year         = 2024,
+  title        = {abhiTronix/deffcode: v0.2.8},
+  month        = oct,
+  year         = 2026,
   publisher    = {Zenodo},
-  version      = {v0.2.6},
-  doi          = {10.5281/zenodo.12689394},
-  url          = {https://doi.org/10.5281/zenodo.12689394},
+  version      = {v0.2.8},
+  doi          = {10.5281/zenodo.23446604},
+  url          = {https://doi.org/10.5281/zenodo.23446604},
 }
 ```
 
@@ -438,6 +445,11 @@ Advanced Recipes
 [smart-keyframe-only-decoding-for-heavy-ai-inference]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/extract-frame-metadata/#smart-keyframe-only-decoding-for-heavy-ai-inference
 [variable-frame-rate-vfr-synchronization-via-pts_time]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/extract-frame-metadata/#variable-frame-rate-vfr-synchronization-via-pts_time
 [multi-input-source-configurations]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_input/#multi-input-source-configurations
+[decoding-multiple-inputs-as-2x2-video-grid-mosaic]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_input/#decoding-multiple-inputs-as-2x2-video-grid-mosaic
+[decoding-crossfade-transition-between-multiple-inputs]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_input/#decoding-crossfade-transition-between-multiple-inputs
+[multi-outputs-configurations]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_output/#multi-outputs-configurations
+[recording-while-decoding]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_output/#recording-while-decoding
+[encode-once-duplicate-to-file-and-stream]: https://abhitronix.github.io/deffcode/latest/recipes/advanced/multi_output/#encode-once-duplicate-to-file-and-stream
 
 <!--
 CI Apps URLs

{deffcode-0.2.7 → deffcode-0.2.9}/deffcode/ffdecoder.py

@@ -26,16 +26,19 @@ import platform
 import queue
 import re
 import subprocess as sp
+import sys
 import threading
-from collections import OrderedDict
+from collections import OrderedDict, deque
 from collections.abc import Generator
+from itertools import pairwise
 from types import TracebackType
-from typing import Any
+from typing import Any, Self
 
 import numpy as np
 
 # import utils packages
 from .ffhelper import (
+    TERMINATE_TIMEOUT_SUBPROCESS,
     get_supported_pixfmts,
     get_supported_vdecoders,
 )
@@ -207,6 +210,15 @@ class FFdecoder:
                 self.__ffmpeg_prefixes = []
             sourcer_params["-ffprefixes"] = self.__ffmpeg_prefixes
 
+        # handles user-defined additional FFmpeg outputs, appended after the rawvideo pipe output (must be a flat list)
+        self.__extra_outputs = self.__extra_params.pop("-extra_outputs", [])
+        if not isinstance(self.__extra_outputs, list) or any(
+            isinstance(x, (list, tuple, dict)) for x in self.__extra_outputs
+        ):
+            logger.warning("Discarding invalid `-extra_outputs` value! It must be a flat list.")
+            self.__extra_outputs = []
+        self.__extra_outputs = [str(x) for x in self.__extra_outputs]
+
         # pass parameter(if specified) to Sourcer API, specifying where to save the downloaded FFmpeg Static
         # assets on Windows(if specified)
         sourcer_params["-ffmpeg_download_path"] = self.__extra_params.pop(
@@ -226,6 +238,20 @@ class FFdecoder:
         if {"-vf", "-filter_complex"}.intersection(self.__extra_params.keys()):
             key = "-vf" if "-vf" in self.__extra_params else "-filter_complex"
             sourcer_params[key] = self.__extra_params[key]
+        # multi-input output streams are selected via `-map`, so probe with it too
+        if self.__is_multi and "-map" in self.__extra_params:
+            sourcer_params["-map"] = self.__extra_params["-map"]
+        # filtergraph labels consumed only by `-extra_outputs` must also be consumed while
+        # probing, otherwise FFmpeg aborts with "unconnected output"; map all into the probe sink.
+        if self.__extra_outputs and "-filter_complex" in sourcer_params:
+            if "-map" in self.__extra_params:
+                sourcer_params["-map"] = self.__extra_params["-map"]
+            sourcer_params["-clones"] = [
+                arg
+                for key, val in pairwise(self.__extra_outputs)
+                if key == "-map" and val.startswith("[")
+                for arg in (key, val)
+            ]
 
         # define dict to store user-defined parameters
         self.__user_metadata = {}
@@ -277,6 +303,13 @@ class FFdecoder:
                 "Enabling asynchronous `showinfo` per-frame metadata extraction."
             )
 
+        # handle capturing FFmpeg stderr tail for diagnosing pipeline failures
+        self.__capture_stderr = self.__extra_params.pop("-capture_stderr", False)
+        if not isinstance(self.__capture_stderr, bool):
+            self.__capture_stderr = False
+        self.__stderr_tail: deque[str] = deque(maxlen=50)
+        self.__returncode: int | None = None
+
         # handle disabling window for ffmpeg subprocess on Windows OS
         # this patch prevents ffmpeg creation window from opening when
         # building exe files
@@ -364,7 +397,7 @@ class FFdecoder:
             # reset improper values
             self.__custom_resolution = None
 
-    def formulate(self) -> FFdecoder:
+    def formulate(self) -> Self:
         """
         This method formulates all necessary FFmpeg pipeline arguments and executes it inside the FFmpeg `subprocess` pipe.
 
@@ -662,7 +695,9 @@ class FFdecoder:
             if "-frames:v" in input_params:
                 self.__raw_frame_num = input_params["-frames:v"]
             elif (
-                self.__sourcer_metadata["approx_video_nframes"] is not None
+                # composite output length is unrelated to the primary source's length
+                not self.__is_multi
+                and self.__sourcer_metadata["approx_video_nframes"] is not None
                 and self.__sourcer_metadata["approx_video_nframes"] > 0
             ):
                 self.__raw_frame_num = self.__sourcer_metadata["approx_video_nframes"]
@@ -778,16 +813,39 @@ class FFdecoder:
             while not self.__terminate_stream:  # infinite raw frames
                 frame = self.__fetchNextFrame()
                 if frame is None:
-                    self.__terminate_stream = True
+                    self.__handle_stream_end()
                     break
                 yield self.__attach_metadata(frame)
         else:
             for _ in range(self.__raw_frame_num):  # finite raw frames
                 frame = self.__fetchNextFrame()
                 if frame is None:
-                    self.__terminate_stream = True
+                    self.__handle_stream_end()
                     break
                 yield self.__attach_metadata(frame)
+        # generator exhausted, FFmpeg may still be flushing/exiting cleanly
+        self.__terminate_stream = True
+
+    def __handle_stream_end(self) -> None:
+        """
+        Internal: marks stream as ended on a short read and, when `-capture_stderr`
+        is enabled and FFmpeg exited with a non-zero code, logs the `stderr_tail`.
+        """
+        self.__terminate_stream = True
+        if not self.__capture_stderr or self.__process is None:
+            return
+        try:
+            retcode = self.__process.wait(timeout=TERMINATE_TIMEOUT_SUBPROCESS)
+        except sp.TimeoutExpired:
+            return
+        if retcode:
+            # let the reader thread drain remaining stderr lines first
+            self.__stderr_thread and self.__stderr_thread.join(timeout=TERMINATE_TIMEOUT_SUBPROCESS)
+            logger.error(
+                "FFmpeg exited with code {}. Last stderr lines:\n{}".format(
+                    retcode, "\n".join(self.__stderr_tail)
+                )
+            )
 
     def __attach_metadata(self, frame: np.ndarray):
         """
@@ -825,6 +883,25 @@ class FFdecoder:
         """
         self.terminate()
 
+    @property
+    def returncode(self) -> int | None:
+        """
+        A property object that holds FFmpeg process exit code, set by `terminate()`.
+
+        **Returns:** FFmpeg exit code, or `None` if pipeline has not been terminated yet.
+        """
+        return self.__returncode
+
+    @property
+    def stderr_tail(self) -> list[str]:
+        """
+        A property object that holds the last 50 lines of FFmpeg's stderr output.
+        Only populated when `-capture_stderr` (or `-extract_metadata`) is enabled.
+
+        **Returns:** List of stderr lines (oldest first).
+        """
+        return list(self.__stderr_tail)
+
     @property
     def metadata(self) -> str:
         """
@@ -903,8 +980,10 @@ class FFdecoder:
                 else:
                     # otherwise discard and log it
                     logger.warning(
-                        "Manually assigned `{}` metadata property value is of invalid type. Discarding!"
-                    ).format(key)
+                        "Manually assigned `{}` metadata property value is of invalid type. Discarding!".format(
+                            key
+                        )
+                    )
                 # delete invalid key
                 del value[key]
             # There is no concept of a tuple in the JSON format.
@@ -956,7 +1035,7 @@ class FFdecoder:
                 )
                 if _src_meta.get("source_demuxer"):
                     cmd += ["-f", _src_meta["source_demuxer"]]
-                cmd += ["-i", _src]
+                cmd += ["-i", _src_meta.get("source", _src)]
         else:
             cmd += self.__ffmpeg_prefixes
             cmd += input_parameters
@@ -969,11 +1048,12 @@ class FFdecoder:
 
         cmd += output_parameters
         cmd += ["-f", "rawvideo", "-"]
-        # When metadata extraction is enabled we must capture stderr regardless
-        # of verbose mode so the background reader thread can parse showinfo
-        # lines. Without PIPE the reader would have nothing to read (verbose
+        cmd += self.__extra_outputs
+        # When metadata extraction or stderr capture is enabled we must pipe stderr
+        # regardless of verbose mode so the background reader thread can consume
+        # it. Without PIPE the reader would have nothing to read (verbose
         # inherits parent stderr; silent discards it).
-        if self.__extract_metadata:
+        if self.__extract_metadata or self.__capture_stderr:
             stderr_target = sp.PIPE
         elif self.__verbose_logs:
             stderr_target = None
@@ -996,18 +1076,20 @@ class FFdecoder:
                 ),
             )
 
-        # spin up the stderr reader thread that parses `showinfo` lines and
-        # feeds per-frame metadata dicts into the queue consumed by generateFrame()
-        if self.__extract_metadata:
-            self.__metadata_queue = queue.Queue()
+        # spin up the stderr reader thread that records stderr tail, and parses
+        # `showinfo` lines into the metadata queue consumed by generateFrame()
+        if self.__extract_metadata or self.__capture_stderr:
+            if self.__extract_metadata:
+                self.__metadata_queue = queue.Queue()
             self.__stderr_stop.clear()
             self.__stderr_thread = threading.Thread(target=self.__read_stderr, daemon=True)
             self.__stderr_thread.start()
 
     def __read_stderr(self) -> None:
         """
-        Internal: background daemon that parses FFmpeg `showinfo` lines off
-        stderr and pushes per-frame metadata dicts onto `__metadata_queue`.
+        Internal: background daemon that appends FFmpeg stderr lines to `__stderr_tail`
+        (echoing them when verbose), and parses `showinfo` lines into per-frame metadata
+        dicts on `__metadata_queue` when `-extract_metadata` is enabled.
         Exits when FFmpeg closes stderr or when `__stderr_stop` is signalled.
         """
         assert self.__process is not None and self.__process.stderr is not None
@@ -1017,6 +1099,11 @@ class FFdecoder:
                 if self.__stderr_stop.is_set():
                     break
                 decoded = line.decode("utf-8", errors="ignore")
+                self.__stderr_tail.append(decoded.rstrip())
+                # echo so verbose console output stays unchanged
+                self.__verbose_logs and sys.stderr.write(decoded)
+                if not self.__extract_metadata:
+                    continue
                 match = _SHOWINFO_REGEX.search(decoded)
                 if not match:
                     continue
@@ -1032,35 +1119,51 @@ class FFdecoder:
             pass
         finally:
             # sentinel so consumers unblock on EOF
-            self.__metadata_queue.put(None)
+            self.__metadata_queue is not None and self.__metadata_queue.put(None)
 
-    def terminate(self) -> None:
+    def terminate(self) -> int | None:
         """
         Safely terminates all processes.
+
+        **Returns:** FFmpeg process exit code (also available via `returncode`), or `None` if no process was launched.
         """
 
         # signal we are closing
         self.__verbose_logs and logger.debug("Terminating FFdecoder Pipeline...")
+        stream_exhausted = self.__terminate_stream
         self.__terminate_stream = True
         self.__stderr_stop.set()
         # check if no process was initiated at first place
-        if self.__process is None or self.__process.poll() is not None:
+        if self.__process is None:
             logger.info("Pipeline already terminated.")
-            return
+            return self.__returncode
         # Attempt to close pipeline.
         # close `stdin` output
         self.__process.stdin and self.__process.stdin.close()
         # close `stdout` output
         self.__process.stdout and self.__process.stdout.close()
-        # close `stderr` so the background reader thread's blocking readline() unblocks
-        self.__process.stderr and self.__process.stderr.close()
-        # terminate/kill process if still processing
-        self.__process.poll() is None and self.__process.terminate()
-        # wait if not exiting
-        self.__process.wait()
-        # join the stderr reader thread so it does not outlive the pipeline
+        # let FFmpeg finalize `-extra_outputs` (trailers/moov) on EPIPE, or exit cleanly
+        # after stream end, before forcing termination
+        if self.__extra_outputs or stream_exhausted:
+            try:
+                self.__process.wait(TERMINATE_TIMEOUT_SUBPROCESS)
+            except sp.TimeoutExpired:
+                pass
+        # terminate process if still processing, escalate to kill if unresponsive
+        if self.__process.poll() is None:
+            self.__process.terminate()
+            try:
+                self.__process.wait(TERMINATE_TIMEOUT_SUBPROCESS)
+            except sp.TimeoutExpired:
+                self.__process.kill()
+                self.__process.wait()
+        # join the stderr reader thread (it hits EOF once FFmpeg exits) so it does not outlive
+        # the pipeline, then close `stderr`; closing earlier blocks on the reader's buffer lock
         if self.__stderr_thread is not None and self.__stderr_thread.is_alive():
             self.__stderr_thread.join(timeout=2.0)
         self.__stderr_thread = None
+        self.__process.stderr and self.__process.stderr.close()
+        self.__returncode = self.__process.returncode
         self.__process = None
         logger.info("Pipeline terminated successfully.")
+        return self.__returncode

{deffcode-0.2.7 → deffcode-0.2.9}/deffcode/ffhelper.py

@@ -42,8 +42,14 @@ logger.propagate = False
 logger.addHandler(logger_handler())
 logger.setLevel(logging.DEBUG)
 
+# set default timeout for subprocesses
+MAX_TIMEOUT_SUBPROCESS: float = float(os.getenv("MAX_TIMEOUT_SUBPROCESS", 10.0))
+# set grace period (seconds) between SIGTERM and SIGKILL when terminating
+# a timed-out subprocess. Allows FFmpeg to flush buffers and release hardware
+# resources before being force-killed.
+TERMINATE_TIMEOUT_SUBPROCESS: float = float(os.getenv("TERMINATE_TIMEOUT_SUBPROCESS", 2.0))
 # set default timer for download requests
-DEFAULT_TIMEOUT = 3
+DEFAULT_TIMEOUT_REQUESTS: float = float(os.getenv("DEFAULT_TIMEOUT_REQUESTS", 3.0))
 
 
 class TimeoutHTTPAdapter(HTTPAdapter):
@@ -52,7 +58,7 @@ class TimeoutHTTPAdapter(HTTPAdapter):
     """
 
     def __init__(self, *args: Any, **kwargs: Any) -> None:
-        self.timeout: float = DEFAULT_TIMEOUT
+        self.timeout: float = DEFAULT_TIMEOUT_REQUESTS
         if "timeout" in kwargs:
             self.timeout = kwargs["timeout"]
             del kwargs["timeout"]
@@ -210,7 +216,9 @@ def download_ffmpeg_binaries(path: str, os_windows: bool = False, os_bit: str =
                         status_forcelist=[429, 500, 502, 503, 504],
                     )
                     # Mount it for https usage
-                    adapter = TimeoutHTTPAdapter(timeout=2.0, max_retries=retries)
+                    adapter = TimeoutHTTPAdapter(
+                        timeout=MAX_TIMEOUT_SUBPROCESS, max_retries=retries
+                    )
                     http.mount("https://", adapter)
                     response = http.get(file_url, stream=True)
                     response.raise_for_status()
@@ -410,6 +418,7 @@ def extract_device_n_demuxer(
         metadata = check_sp_output(
             [path, *default_ffcommand.split(" ")],
             force_retrieve_stderr=True,
+            timeout=MAX_TIMEOUT_SUBPROCESS,
         )
         # clean and split metadata
         splitted = [x.decode("utf-8").strip() for x in metadata.split(b"\n")]
@@ -469,6 +478,7 @@ def extract_device_n_demuxer(
                         # search in path properties
                         metadata_path = check_sp_output(
                             ["v4l2-ctl", "--device={}".format(path), "--all"],
+                            timeout=MAX_TIMEOUT_SUBPROCESS,
                         )
                         # decode path metadata
                         decoded_path = metadata_path.decode("utf-8").strip()
@@ -488,6 +498,7 @@ def extract_device_n_demuxer(
         metadata = check_sp_output(
             [path, *default_ffcommand.split(" ")],
             force_retrieve_stderr=True,
+            timeout=MAX_TIMEOUT_SUBPROCESS,
         )
         # clean and split metadata
         splitted = [x.decode("utf-8").strip() for x in metadata.split(b"\n")]
@@ -641,44 +652,85 @@ def check_sp_output(*args: Any, **kwargs: Any) -> bytes:
     """
     ## check_sp_output
 
-    Returns FFmpeg `stdout` output from subprocess module.
+    Executes a subprocess command and returns its `stdout` (or `stderr` when
+    requested). On timeout, performs a two-step graceful shutdown — sends
+    `SIGTERM` first to allow FFmpeg to flush buffers and release hardware
+    resources (decoders, capture devices), then escalates to `SIGKILL` if the
+    process fails to exit within a short grace period.
 
     Parameters:
         args (based on input): Non Keyword Arguments
         kwargs (based on input): Keyword Arguments
+            force_retrieve_stderr (bool): If True, returns stderr. Also
+                suppresses `CalledProcessError` on non-zero exit, since some
+                FFmpeg diagnostic commands (e.g. `-list_devices`) emit useful
+                output on stderr while exiting non-zero by design.
+            timeout (float): Seconds to wait before terminating the process.
 
-    **Returns:** A string value.
+    **Returns:** A bytes value.
     """
     # workaround for python bug: https://bugs.python.org/issue37380
     if platform.system() == "Windows":
         # see comment https://bugs.python.org/msg370334
         sp._cleanup = lambda: None
+
     # handle additional params
     retrieve_stderr = kwargs.pop("force_retrieve_stderr", False)
+    timeout = kwargs.pop("timeout", None)
+    # never inherit parent's stdin, avoids blocking on unexpected input reads
+    kwargs.setdefault("stdin", sp.DEVNULL)
+
     # execute command in subprocess
     process = sp.Popen(
         *args,
         stdout=sp.PIPE,
-        stderr=sp.DEVNULL if not (retrieve_stderr) else sp.PIPE,
+        stderr=sp.PIPE if retrieve_stderr else sp.DEVNULL,
         **kwargs,
     )
-    # communicate and poll process
-    output, stderr = process.communicate()
+
+    # communicate and poll process with two-step timeout handling
+    timeout_occurred = False
+    try:
+        output, stderr = process.communicate(timeout=timeout)
+    except sp.TimeoutExpired:
+        timeout_occurred = True
+        logger.warning(
+            f"[Pipeline-Warning] :: Process exceeded timeout of {timeout}s. "
+            "Attempting graceful termination..."
+        )
+        # Step 1: polite SIGTERM, give process a chance to clean up
+        process.terminate()
+        try:
+            output, stderr = process.communicate(timeout=TERMINATE_TIMEOUT_SUBPROCESS)
+        except sp.TimeoutExpired:
+            # Step 2: process ignored SIGTERM, force kill
+            logger.error("[Pipeline-Error] :: Process unresponsive to SIGTERM. Hard killing...")
+            process.kill()
+            output, stderr = process.communicate()
+
     retcode = process.poll()
+
     # handle return code
-    if retcode and not (retrieve_stderr):
-        logger.error("[Pipeline-Error] :: {}".format(output.decode("utf-8")))
+    # Bypass CalledProcessError if caller wants stderr (some FFmpeg commands
+    # exit non-zero by design) or if we killed the process via our timeout.
+    if retcode and not retrieve_stderr and not timeout_occurred:
+        logger.error(
+            "[Pipeline-Error] :: {}".format(output.decode("utf-8") if output else "No output")
+        )
         cmd = kwargs.get("args")
         if cmd is None:
             cmd = args[0]
         error = sp.CalledProcessError(retcode, cmd)
         error.output = output
         raise error
-    # raise error if no output
-    bool(output) or bool(stderr) or logger.error(
-        "[Pipeline-Error] :: Pipeline failed to exact any data from command: {}!".format(
-            args[0] if args else []
+
+    # warn if process emitted nothing on either stream
+    if not (bool(output) or bool(stderr)):
+        logger.error(
+            "[Pipeline-Error] :: Pipeline failed to extract any data from command: {}!".format(
+                args[0] if args else []
+            )
         )
-    )
-    # return output otherwise
+
+    # return stderr when explicitly requested (and present), else stdout
     return stderr if retrieve_stderr and stderr else output