gst-python-ml 0.1.0__tar.gz → 0.3.0__tar.gz

{gst_python_ml-0.1.0/plugins/python/gst_python_ml.egg-info → gst_python_ml-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: gst-python-ml
-Version: 0.1.0
+Version: 0.3.0
 Summary: An ML package for GStreamer
 Home-page: https://github.com/collabora/gst-python-ml
 Author: Aaron Boxer
@@ -22,6 +22,7 @@ Requires-Dist: huggingface-hub
 Requires-Dist: lap
 Requires-Dist: ultralytics
 Requires-Dist: pycairo
+Requires-Dist: pytest>=7.0
 Provides-Extra: kafka
 Requires-Dist: confluent-kafka; extra == "kafka"
 Provides-Extra: captioning
@@ -47,6 +48,7 @@ Dynamic: description
 Dynamic: description-content-type
 Dynamic: home-page
 Dynamic: license
+Dynamic: license-file
 Dynamic: provides-extra
 Dynamic: requires-dist
 Dynamic: requires-python
@@ -209,9 +211,24 @@ Run `gst-inspect-1.0 python` to see all of the pyml elements listed.
 
 # Building PyPI Package
 
-1. `pip install setuptools wheel twine`
-2. `python setup.py sdist bdist_wheel`
-3. ls dist/
+## Setup
+1. Generate token on PyPI and add to `.pypirc` :
+
+```
+[pypi]
+  username = __token__
+  password = FOOBAR
+```
+
+2. `pip install setuptools wheel twine`
+
+## Build
+
+`python -m build`
+
+## Upload
+
+`twine upload dist/*`
 
 
 ## Using GStreamer Python ML Elements
@@ -293,58 +310,105 @@ Note: make sure to set the following in `.bashrc` file :
 
 `GST_DEBUG=4 gst-launch-1.0 filesrc location=data/soccer_single_camera.mp4 ! decodebin ! videorate ! video/x-raw,framerate=30/1 ! videoconvert ! pyml_birdseye ! videoconvert ! openh264enc ! h264parse ! matroskamux ! filesink location=output.mkv`
 
+### Classification
+
+```
+GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_classifier model-name=resnet18 device=cuda !  videoconvert !  autovideosink
+```
+
+
 ### Object Detection
 
-Possible model names:
-`fasterrcnn_resnet50_fpn`
-`retinanet_resnet50_fpn`
+#### TorchVision
 
-#### fasterrcnn/kafka
+`pyml_objectdetector` supports all TorchVision  object detection models.
+Simply choose a suitable model name and set it on the `model-name` property.
+A few possible model names:
 
-`GST_DEBUG=4 gst-launch-1.0 multifilesrc location=data/000015.jpg ! jpegdec ! videoconvert ! videoscale ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! pyml_kafkasink schema-file=data/pyml_object_detector.json broker=kafka:9092 topic=test-kafkasink-topic  2>&1 | grep pyml_kafkasink`
+```
+fasterrcnn_resnet50_fpn
+ssdlite320_mobilenet_v3_large
+```
 
-#### maskrcnn
+##### fasterrcnn
+
+`GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! videoconvert ! pyml_overlay ! videoconvert ! autovideosink`
+
+##### fasterrcnn/kafka
+
+a) run pipeline from host
+
+```
+GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! pyml_kafkasink schema-file=data/pyml_object_detector.json broker=localhost:29092 topic=test-kafkasink-topic
+```
+
+b) run pipeline from docker
+
+```
+GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! pyml_kafkasink schema-file=data/pyml_object_detector.json broker=kafka:9092 topic=test-kafkasink-topic
+```
 
-`GST_DEBUG=4 gst-launch-1.0   filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! pyml_maskrcnn device=cuda batch-size=4 model-name=maskrcnn_resnet50_fpn ! videoconvert ! objectdetectionoverlay labels-color=0xFFFF0000 object-detection-outline-color=0xFFFF0000  ! autovideosink`
 
+#### maskrcnn
+
+```
+GST_DEBUG=4 gst-launch-1.0   filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! pyml_maskrcnn device=cuda batch-size=4 model-name=maskrcnn_resnet50_fpn ! videoconvert ! pyml_overlay ! videoconvert ! autovideosink
+```
 
 #### yolo with tracking
 
-`gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True ! videoconvert  !  pyml_overlay labels-color=0xFFFF0000 object-detection-outline-color=0xFFFF0000 ! autovideosink`
+```
+GST_DEBUG=4 gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin !  videoconvertscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True ! pyml_overlay  ! videoconvert ! autovideosink
+```
 
-#### yolo with overlay
+```
+GST_DEBUG=4 gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvertscale ! video/x-raw,width=640,height=480,format=RGB ! pyml_streammux name=mux   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvertscale ! video/x-raw,width=640,height=480,format=RGB ! mux.   mux. ! pyml_yolo model-name=yolo11m device=cuda:0 track=True ! pyml_streamdemux name=demux   demux. ! queue ! videoconvert ! pyml_overlay ! videoconvert ! autovideosink sync=false   demux. ! queue ! videoconvert ! pyml_overlay ! videoconvert !  autovideosink sync=false
 
- `gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True !  pyml_overlay ! videoconvert !  autovideosink`
+```
 
+#### yolo with overlay
 
-### streammux pipeline
+```
+ GST_DEBUG=4 gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True !  pyml_overlay ! videoconvert !  autovideosink
+```
 
-`GST_DEBUG=4 gst-launch-1.0 pyml_streammux name=mux  ! videoconvert ! fakesink videotestsrc ! mux. videotestsrc pattern=ball ! mux. videotestsrc pattern=snow ! mux.`
+### streammux/streamdemux pipeline
 
+```
+ GST_DEBUG=4 gst-launch-1.0   videotestsrc pattern=ball ! video/x-raw, width=320, height=240 ! queue ! pyml_streammux name=mux   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_1   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_2   mux.src ! queue ! pyml_streamdemux name=demux   demux.src_0 ! queue ! glimagesink  demux.src_1 ! queue ! glimagesink   demux.src_2 ! queue  ! glimagesink
+```
 
 ### Transcription
 
 #### transcription with initial prompt set
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko initial_prompt = "Air Traffic Control은, radar systems를,  weather conditions에, flight paths를, communication은, unexpected weather conditions가, continuous training을, dedication과, professionalism" ! fakesink`
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko initial_prompt = "Air Traffic Control은, radar systems를,  weather conditions에, flight paths를, communication은, unexpected weather conditions가, continuous training을, dedication과, professionalism" ! fakesink
+```
 
 #### translation to English
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! fakesink`
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! fakesink
+```
 
 #### coquitts
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_coquitts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav`
-
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_coquitts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav
+```
 
 #### whisperspeechtts
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_whisperspeechtts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav`
-
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_whisperspeechtts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav
+```
 
 #### mariantranslate
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_mariantranslate device=cuda src=en target=fr ! fakesink`
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_mariantranslate device=cuda src=en target=fr ! fakesink
+```
 
 Supported src/target languages:
 

{gst_python_ml-0.1.0 → gst_python_ml-0.3.0}/README.md

@@ -155,9 +155,24 @@ Run `gst-inspect-1.0 python` to see all of the pyml elements listed.
 
 # Building PyPI Package
 
-1. `pip install setuptools wheel twine`
-2. `python setup.py sdist bdist_wheel`
-3. ls dist/
+## Setup
+1. Generate token on PyPI and add to `.pypirc` :
+
+```
+[pypi]
+  username = __token__
+  password = FOOBAR
+```
+
+2. `pip install setuptools wheel twine`
+
+## Build
+
+`python -m build`
+
+## Upload
+
+`twine upload dist/*`
 
 
 ## Using GStreamer Python ML Elements
@@ -239,58 +254,105 @@ Note: make sure to set the following in `.bashrc` file :
 
 `GST_DEBUG=4 gst-launch-1.0 filesrc location=data/soccer_single_camera.mp4 ! decodebin ! videorate ! video/x-raw,framerate=30/1 ! videoconvert ! pyml_birdseye ! videoconvert ! openh264enc ! h264parse ! matroskamux ! filesink location=output.mkv`
 
+### Classification
+
+```
+GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_classifier model-name=resnet18 device=cuda !  videoconvert !  autovideosink
+```
+
+
 ### Object Detection
 
-Possible model names:
-`fasterrcnn_resnet50_fpn`
-`retinanet_resnet50_fpn`
+#### TorchVision
 
-#### fasterrcnn/kafka
+`pyml_objectdetector` supports all TorchVision  object detection models.
+Simply choose a suitable model name and set it on the `model-name` property.
+A few possible model names:
 
-`GST_DEBUG=4 gst-launch-1.0 multifilesrc location=data/000015.jpg ! jpegdec ! videoconvert ! videoscale ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! pyml_kafkasink schema-file=data/pyml_object_detector.json broker=kafka:9092 topic=test-kafkasink-topic  2>&1 | grep pyml_kafkasink`
+```
+fasterrcnn_resnet50_fpn
+ssdlite320_mobilenet_v3_large
+```
 
-#### maskrcnn
+##### fasterrcnn
+
+`GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! videoconvert ! pyml_overlay ! videoconvert ! autovideosink`
+
+##### fasterrcnn/kafka
+
+a) run pipeline from host
+
+```
+GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! pyml_kafkasink schema-file=data/pyml_object_detector.json broker=localhost:29092 topic=test-kafkasink-topic
+```
+
+b) run pipeline from docker
+
+```
+GST_DEBUG=4 gst-launch-1.0  filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_objectdetector model-name=fasterrcnn_resnet50_fpn device=cuda batch-size=4 ! pyml_kafkasink schema-file=data/pyml_object_detector.json broker=kafka:9092 topic=test-kafkasink-topic
+```
 
-`GST_DEBUG=4 gst-launch-1.0   filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! pyml_maskrcnn device=cuda batch-size=4 model-name=maskrcnn_resnet50_fpn ! videoconvert ! objectdetectionoverlay labels-color=0xFFFF0000 object-detection-outline-color=0xFFFF0000  ! autovideosink`
 
+#### maskrcnn
+
+```
+GST_DEBUG=4 gst-launch-1.0   filesrc location=data/people.mp4 ! decodebin ! videoconvert ! videoscale ! pyml_maskrcnn device=cuda batch-size=4 model-name=maskrcnn_resnet50_fpn ! videoconvert ! pyml_overlay ! videoconvert ! autovideosink
+```
 
 #### yolo with tracking
 
-`gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True ! videoconvert  !  pyml_overlay labels-color=0xFFFF0000 object-detection-outline-color=0xFFFF0000 ! autovideosink`
+```
+GST_DEBUG=4 gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin !  videoconvertscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True ! pyml_overlay  ! videoconvert ! autovideosink
+```
 
-#### yolo with overlay
+```
+GST_DEBUG=4 gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvertscale ! video/x-raw,width=640,height=480,format=RGB ! pyml_streammux name=mux   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvertscale ! video/x-raw,width=640,height=480,format=RGB ! mux.   mux. ! pyml_yolo model-name=yolo11m device=cuda:0 track=True ! pyml_streamdemux name=demux   demux. ! queue ! videoconvert ! pyml_overlay ! videoconvert ! autovideosink sync=false   demux. ! queue ! videoconvert ! pyml_overlay ! videoconvert !  autovideosink sync=false
 
- `gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True !  pyml_overlay ! videoconvert !  autovideosink`
+```
 
+#### yolo with overlay
 
-### streammux pipeline
+```
+ GST_DEBUG=4 gst-launch-1.0   filesrc location=data/soccer_tracking.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw,width=640,height=480 ! pyml_yolo model-name=yolo11m device=cuda:0 track=True !  pyml_overlay ! videoconvert !  autovideosink
+```
 
-`GST_DEBUG=4 gst-launch-1.0 pyml_streammux name=mux  ! videoconvert ! fakesink videotestsrc ! mux. videotestsrc pattern=ball ! mux. videotestsrc pattern=snow ! mux.`
+### streammux/streamdemux pipeline
 
+```
+ GST_DEBUG=4 gst-launch-1.0   videotestsrc pattern=ball ! video/x-raw, width=320, height=240 ! queue ! pyml_streammux name=mux   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_1   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_2   mux.src ! queue ! pyml_streamdemux name=demux   demux.src_0 ! queue ! glimagesink  demux.src_1 ! queue ! glimagesink   demux.src_2 ! queue  ! glimagesink
+```
 
 ### Transcription
 
 #### transcription with initial prompt set
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko initial_prompt = "Air Traffic Control은, radar systems를,  weather conditions에, flight paths를, communication은, unexpected weather conditions가, continuous training을, dedication과, professionalism" ! fakesink`
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko initial_prompt = "Air Traffic Control은, radar systems를,  weather conditions에, flight paths를, communication은, unexpected weather conditions가, continuous training을, dedication과, professionalism" ! fakesink
+```
 
 #### translation to English
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! fakesink`
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! fakesink
+```
 
 #### coquitts
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_coquitts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav`
-
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_coquitts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav
+```
 
 #### whisperspeechtts
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_whisperspeechtts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav`
-
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_whisperspeechtts device=cuda ! audioconvert ! wavenc ! filesink location=output_audio.wav
+```
 
 #### mariantranslate
 
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_mariantranslate device=cuda src=en target=fr ! fakesink`
+```
+GST_DEBUG=4 gst-launch-1.0 filesrc location=data/air_traffic_korean_with_english.wav ! decodebin ! audioconvert ! pyml_whispertranscribe device=cuda language=ko translate=yes ! pyml_mariantranslate device=cuda src=en target=fr ! fakesink
+```
 
 Supported src/target languages:
 

gst_python_ml-0.1.0/plugins/python/gst_aggregator.py → gst_python_ml-0.3.0/plugins/python/aggregator_base.py

@@ -1,4 +1,4 @@
-# GstAggregator
+# AggregatorBase
 # Copyright (C) 2024-2025 Collabora Ltd.
 #
 # This library is free software; you can redistribute it and/or
@@ -18,15 +18,18 @@
 
 from abc import abstractmethod
 import gi
-from engine.gst_engine_factory import GstEngineFactory
 
 gi.require_version("Gst", "1.0")
 gi.require_version("GstBase", "1.0")
 gi.require_version("GLib", "2.0")
 from gi.repository import Gst, GObject, GstBase  # noqa: E402
 
+from engine.engine_factory import EngineFactory
+from log.logger_factory import LoggerFactory
+from model_engine_helper import ModelEngineHelper
 
-class GstAggregator(GstBase.Aggregator):
+
+class AggregatorBase(GstBase.Aggregator):
     """
     Base class for GStreamer aggregator elements that perform inference
     with a machine learning model. This class manages shared properties
@@ -34,7 +37,7 @@ class GstAggregator(GstBase.Aggregator):
     """
 
     __gstmetadata__ = (
-        "GstAggregator",
+        "AggregatorBase",
         "Aggregator",
         "Generic machine learning model aggregator element",
         "Aaron Boxer <aaron.boxer@collabora.com>",
@@ -74,7 +77,7 @@ class GstAggregator(GstBase.Aggregator):
         blurb="Name of the pre-trained model or local model path",
         flags=GObject.ParamFlags.READWRITE,
     )
-    ml_engine = GObject.Property(
+    engine_name = GObject.Property(
         type=str,
         default=None,
         nick="ML Engine",
@@ -84,9 +87,9 @@ class GstAggregator(GstBase.Aggregator):
 
     device_queue_id = GObject.Property(
         type=int,
-        default=0,  # Default to queue ID 0
+        default=0,
         minimum=0,
-        maximum=32,  # You can adjust the maximum depending on the size of your pool
+        maximum=32,
         nick="Device Queue ID",
         blurb="ID of the DeviceQueue from the pool to use",
         flags=GObject.ParamFlags.READWRITE,
@@ -94,8 +97,9 @@ class GstAggregator(GstBase.Aggregator):
 
     def __init__(self):
         super().__init__()
-        self.ml_engine = GstEngineFactory.PYTORCH_ENGINE
-        self.engine = None
+        self.logger = LoggerFactory.get(LoggerFactory.LOGGER_TYPE_GST)
+        self.engine_helper = ModelEngineHelper(self.logger)
+        self.engine_name = self.engine_helper.engine_name
         self.kwargs = {}
         self.segment_pushed = False
 
@@ -107,12 +111,9 @@ class GstAggregator(GstBase.Aggregator):
         elif prop.name == "model-name":
             return self.model_name
         elif prop.name == "device":
-            if self.engine:
-                return self.engine.get_device()
-            else:
-                return None
-        elif prop.name == "ml-engine":
-            return self.ml_engine
+            return self.device  # Return from AggregatorBase, not from helper
+        elif prop.name == "engine-name":
+            return self.engine_name
         elif prop.name == "device-queue-id":
             return self.device_queue_id
         else:
@@ -121,86 +122,72 @@ class GstAggregator(GstBase.Aggregator):
     def do_set_property(self, prop: GObject.ParamSpec, value):
         if prop.name == "batch-size":
             self.batch_size = value
-            if self.engine:
-                self.engine.batch_size = value
+            if self.engine_helper.engine:
+                self.engine_helper.engine.batch_size = value
         elif prop.name == "frame-stride":
             self.frame_stride = value
-            if self.engine:
-                self.engine.frame_stride = value
+            if self.engine_helper.engine:
+                self.engine_helper.engine.frame_stride = value
         elif prop.name == "model-name":
             self.model_name = value
-            self.do_load_model()
+            self.engine_helper.load_model(value)
         elif prop.name == "device":
             self.device = value
-            # Only set the device if the engine is initialized
-            if self.engine:
-                self.engine.set_device(value)
-                self.do_load_model()
-        elif prop.name == "ml-engine":
-            if self.device:
-                self.ml_engine = GstEngineFactory.create_engine(value, self.device)
-                self.initialize_engine()
-                self.do_load_model()
+            self.engine_helper.set_device(value)  # Update device in helper
+            self.engine_helper.initialize_engine(self.engine_name)
+            self.engine_helper.load_model(self.model_name)
+        elif prop.name == "engine-name":
+            self.engine_name = value
+            self.engine_helper.initialize_engine(value)
+            self.engine_helper.load_model(self.model_name)
         elif prop.name == "device-queue-id":
             self.device_queue_id = value
-            if self.engine:
-                self.engine.device_queue_id = value
+            if self.engine_helper.engine:
+                self.engine_helper.engine.device_queue_id = value
         else:
             raise AttributeError(f"Unknown property {prop.name}")
 
     def _initialize_engine_if_needed(self):
-        """Initialize the engine if it hasn't been initialized yet."""
-        if not self.engine and self.ml_engine:
-            self.initialize_engine()
+        if not self.engine_helper.engine and self.engine_name:
+            self.engine_helper.initialize_engine(self.engine_name)
 
     def initialize_engine(self):
-        """Initialize the machine learning engine based on the ml_engine property."""
-        if self.ml_engine is not None:
-            self.engine = GstEngineFactory.create_engine(self.ml_engine, self.device)
-            self.engine.batch_size = self.batch_size
-            self.engine.frame_stride = self.frame_stride
+        if self.engine_name is not None:
+            self.engine_helper.initialize_engine(self.engine_name)
+            self.engine_helper.engine.batch_size = self.batch_size
+            self.engine_helper.engine.frame_stride = self.frame_stride
             if self.device_queue_id:
-                self.engine.device_queue_id = self.device_queue_id
+                self.engine_helper.engine.device_queue_id = self.device_queue_id
         else:
-            Gst.error(f"Unsupported ML engine: {self.ml_engine}")
-            return
+            self.logger.error(f"Unsupported ML engine: {self.engine_name}")
 
     def do_load_model(self):
-        """Loads the model using the current engine."""
-        if self.engine and self.model_name:
-            self.engine.load_model(self.model_name, **self.kwargs)
+        if self.engine_helper.engine and self.model_name:
+            self.engine_helper.load_model(self.model_name)
         else:
-            Gst.warning("Engine is not present, unable to load the model.")
+            self.logger.warning("Engine is not present, unable to load the model.")
 
     def get_model(self):
-        """Gets the model from the engine."""
         self._initialize_engine_if_needed()
-        """Gets the model from the engine."""
-        if self.engine:
-            return self.engine.get_model()
+        if self.engine_helper.engine:
+            return self.engine_helper.get_model()
         else:
-            Gst.warning("Engine is not present, unable to get the model.")
+            self.logger.warning("Engine is not present, unable to get the model.")
             return None
 
     def set_model(self, model):
-        """Gets the model from the engine."""
         self._initialize_engine_if_needed()
-        """Sets the model in the engine."""
-        if self.engine:
-            self.engine.set_model(model)
+        if self.engine_helper.engine:
+            self.engine_helper.set_model(model)
         else:
-            Gst.warning("Engine is not present, unable to set the model.")
+            self.logger.warning("Engine is not present, unable to set the model.")
 
     def get_tokenizer(self):
-        """Gets the model from the engine."""
         self._initialize_engine_if_needed()
-        if self.get_model() is None:
-            self.do_load_model()
-        """Gets the model from the engine."""
-        if self.engine:
-            return self.engine.tokenizer
+        if self.engine_helper.engine:
+            return self.engine_helper.get_tokenizer()
         else:
-            Gst.warning("Engine is not present, unable to get the tokenizer.")
+            self.logger.warning("Engine is not present, unable to get the tokenizer.")
             return None
 
     def push_segment_if_needed(self):
@@ -215,10 +202,6 @@ class GstAggregator(GstBase.Aggregator):
             self.segment_pushed = True
 
     def do_aggregate(self, timeout):
-        """
-        Aggregates the buffers from the sink pads,
-        processes with the model, and pushes the result downstream.
-        """
         self.push_segment_if_needed()
         self.process_all_sink_pads()
         return Gst.FlowReturn.OK
@@ -232,5 +215,4 @@ class GstAggregator(GstBase.Aggregator):
 
     @abstractmethod
     def do_process(self, buf):
-        """Process a buffer using the loaded model."""
         pass

{gst_python_ml-0.1.0 → gst_python_ml-0.3.0}/plugins/python/analytics_utils.py

@@ -25,27 +25,36 @@ try:
     gi.require_version("GLib", "2.0")
     gi.require_version("GstAnalytics", "1.0")
     from gi.repository import Gst, GstAnalytics, GLib  # noqa: E402
+
+    from log.logger_factory import LoggerFactory
 except ImportError:
     ANALYTICS_UTILS_AVAILABLE = False
 
 
 class AnalyticsUtils:
+    def __init__(self):
+        super().__init__()
+        self.logger = LoggerFactory.get(LoggerFactory.LOGGER_TYPE_GST)
+
     def extract_analytics_metadata(self, buffer):
         metadata = []
         meta = GstAnalytics.buffer_get_analytics_relation_meta(buffer)
         if not meta:
+            self.logger.info("No analytics relation metadata found on buffer")
             return metadata
 
         try:
             count = GstAnalytics.relation_get_length(meta)
+            self.logger.info(f"Found {count} analytics relations in metadata")
             for index in range(count):
                 ret, od_mtd = meta.get_od_mtd(index)
                 if not ret or od_mtd is None:
+                    # self.logger.warning(f"Failed to get od_mtd at index {index}")
                     continue
-
                 label_quark = od_mtd.get_obj_type()
-                label = GLib.quark_to_string(label_quark)
-                track_id = self.extract_id_from_label(label)
+                full_label = GLib.quark_to_string(label_quark)
+                self.logger.debug(f"Index {index}: quark={full_label}")
+                track_id, label = self.extract_id_from_label(full_label)
                 location = od_mtd.get_location()
                 presence, x, y, w, h, loc_conf_lvl = location
                 if presence:
@@ -57,16 +66,27 @@ class AnalyticsUtils:
                             "box": {"x1": x, "y1": y, "x2": x + w, "y2": y + h},
                         }
                     )
+                    self.logger.debug(f"Added metadata entry: {metadata[-1]}")
         except Exception as e:
-            Gst.error(f"Error while extracting analytics metadata: {e}")
+            self.logger.error(f"Error while extracting analytics metadata: {e}")
         return metadata
 
-    def extract_id_from_label(self, label):
-        """Extracts the numeric ID from a label formatted as 'id_<number>'."""
-        match = re.match(r"id_(\d+)", label)
+    def extract_id_from_label(self, full_label):
+        match = re.match(r"stream_\d+_id_(\d+)", full_label)
         if match:
             track_id = int(match.group(1))
-            return track_id
-        else:
-            print("No ID found in label")  # Optional debug message for unmatched format
-            return None  # Return None if the ID format is not found
+            label = f"id_{track_id}"
+            self.logger.debug(
+                f"Extracted track_id {track_id} and label '{label}' from '{full_label}'"
+            )
+            return track_id, label
+        match = re.match(
+            r"stream_\d+_(.+)", full_label
+        )  # Match class name after stream_<idx>_
+        if match:
+            class_name = match.group(1)
+            label = class_name  # Use class name directly
+            self.logger.debug(f"Extracted class label '{label}' from '{full_label}'")
+            return None, label
+        self.logger.info(f"No recognizable format in label '{full_label}', using as-is")
+        return None, full_label