gst-python-ml 1.0.0__tar.gz → 1.0.2__tar.gz

{gst_python_ml-1.0.0 → gst_python_ml-1.0.2}/MANIFEST.in

@@ -1,4 +1,3 @@
-include requirements.txt
 include README.md
 include COPYING
 recursive-include plugins/python *

{gst_python_ml-1.0.0/plugins/python/gst_python_ml.egg-info → gst_python_ml-1.0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gst-python-ml
-Version: 1.0.0
+Version: 1.0.2
 Summary: An ML package for GStreamer
 Author-email: Aaron Boxer <aaron.boxer@collabora.com>
 Project-URL: Homepage, https://github.com/collabora/gst-python-ml
@@ -18,12 +18,14 @@ Requires-Dist: transformers==4.56.2
 Requires-Dist: qwen-vl-utils[decord]==0.0.8
 Requires-Dist: autoawq==0.2.9
 Requires-Dist: accelerate==1.12.0
+Requires-Dist: bitsandbytes==0.46.1
+Requires-Dist: opencv-python==4.10.0.84
+Requires-Dist: opencv-contrib-python==4.10.0.84
 Requires-Dist: numpy
 Requires-Dist: huggingface-hub
 Requires-Dist: lap
 Requires-Dist: pycairo
 Requires-Dist: ultralytics
-Requires-Dist: bitsandbytes==0.46.1
 Requires-Dist: confluent_kafka
 Requires-Dist: diffusers
 Requires-Dist: sentencepiece
@@ -56,24 +58,21 @@ Supported functionality includes:
 1. LLMs
 1. serializing model metadata to Kafka server
 
-ML toolkits are supported via the `MLEngine` abstraction - we have nominal support for
+Different ML toolkits are supported via the `MLEngine` abstraction - we have nominal support for
 TensorFlow, LiteRT and OpenVINO, but all testing thus far has been done with PyTorch.
 
-These elements will work with your distribution's GStreamer packages. They have been tested on Ubuntu 24 with GStreamer 1.24.
-
-## Python Version
-
-All elements have been tested with Python 3.12, the installed version of Python on Ubuntu 24
+These elements will work with your distribution's GStreamer packages as long as the GStreamer version
+is >= 1.24.
 
 ## Install
 
-There are two installation options described below: installing on your host machine,
-or installing with a Docker container:
+There are two installation options described below: on host machine or on Docker container:
 
-### Host Install (Ubuntu 24)
+### Host Install
 
-#### Install packages
+#### Install distribution packages
 
+##### Ubuntu
 ```
 sudo apt update && sudo apt -y upgrade
 sudo apt install -y python3-pip  python3-venv \
@@ -83,7 +82,28 @@ sudo apt install -y python3-pip  python3-venv \
     libcairo2 libcairo2-dev git
 ```
 
-#### uv
+##### Fedora
+
+(adjust Fedora version from 42 to match your version number)
+
+```
+sudo dnf install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-42.noarch.rpm https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-42.noarch.rpm
+sudo dnf update -y
+sudo dnf install akmod-nvidia xorg-x11-drv-nvidia-cuda -y
+```
+
+```
+sudo dnf upgrade -y
+sudo dnf install -y python3-pip \
+    python3-devel cairo cairo-devel cairo-gobject-devel pkgconfig git \
+    gstreamer1-plugins-base gstreamer1-plugins-base-tools \
+    gstreamer1-plugins-good gstreamer1-plugins-bad-free \
+    gstreamer1-plugins-bad-free-devel python3-gstreamer1
+```
+
+
+
+#### Manage Python packages with uv
 
 ##### install
 curl -LsSf https://astral.sh/uv/install.sh | sh
@@ -93,38 +113,31 @@ curl -LsSf https://astral.sh/uv/install.sh | sh
 ```
 uv venv --system-site-packages
 source .venv/bin/activate
+uv pip install --upgrade pip
 uv sync
 ```
 
 Now manually install flash-attn wheel (must match your version of python, torch and cuda)
-eg:  
+For example:
+
 `uv pip install ./flash_attn-2.8.3+cu128torch2.9-cp313-cp313-linux_x86_64.whl`
 
-A pre-built wheel can be found here:
+Pe-built wheels can be found here:
 https://github.com/mjun0812/flash-attention-prebuild-wheels/releases
 
 
-#### Clone repo (host)
-
-`git clone https://github.com/collabora/gst-python-ml.git`
-
-#### Update .bashrc
+#### Clone repo
 
 ```
-export VIRTUAL_ENV=$HOME/venv
-export PATH=$VIRTUAL_ENV/bin:$PATH
-export GST_PLUGIN_PATH=$HOME/src/gst-python-ml/plugins
+cd $HOME/src
+git clone https://github.com/collabora/gst-python-ml.git
 ```
 
-and then
-
-`source ~/.bashrc`
-
-#### Activate venv and install basic pip packages
+#### Update .bashrc
 
 ```
-source $VIRTUAL_ENV/bin/activate
-pip install --upgrade pip
+echo 'export GST_PLUGIN_PATH=$HOME/src/gst-python-ml/demos:$HOME/src/gst-python-ml/plugins:$GST_PLUGIN_PATH' >> ~/.bashrc
+source ~/.bashrc
 ```
 
 ### Docker Install
@@ -134,7 +147,7 @@ pip install --upgrade pip
 Important Note:
 
 This Dockerfile maps a local `gst-python-ml` repository to the container,
-and expects this repository to be located in `~/src` i.e.  `~/src/gst-python-ml`.
+and expects this repository to be located in `$HOME/src` i.e.  `$HOME/src/gst-python-ml`.
 
 
 #### Enable Docker GPU Support on Host
@@ -143,19 +156,12 @@ To use the host GPU in a docker container, you will need to install the nvidia c
 
 
 ##### Ubuntu
-
-Add nvidia repository
-
 ```
 curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
   && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
     sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
     sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
-```
 
-Then
-
-```
 sudo apt update
 sudo apt install -y nvidia-container-toolkit
 sudo systemctl restart docker
@@ -164,26 +170,58 @@ sudo systemctl restart docker
 ##### Fedora
 
 ```
-curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/
-nvidia-container-toolkit.repo
-
-sudo dnf install nvidia-container-toolkit
-sudo nvidia-ctk runtime configure --runtime=docker
-sudo nvidia-ctk runtime configure --runtime=containerd
-sudo systemctl restart containerd
-sudo systemctl restart docker
+sudo dnf install docker
+sudo usermod -aG docker $USER
+# Then either log out/in completely, or:
+newgrp docker
 ```
 
-To test:
 
 ```
-docker pull nvidia/cuda:12.0.0-base-ubuntu22.04
-docker run --rm --runtime=nvidia --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
+# 1. Add NVIDIA Container Toolkit repository
+curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
+  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
+
+# 2. Remove Fedora's conflicting partial package (if present)
+sudo dnf remove -y golang-github-nvidia-container-toolkit 2>/dev/null || true
+
+# 3. Install the full NVIDIA Container Toolkit
+sudo dnf install -y nvidia-container-toolkit
+
+# 4. Configure Docker to use the NVIDIA runtime as default
+sudo mkdir -p /etc/docker
+sudo tee /etc/docker/daemon.json > /dev/null <<EOF
+{
+  "runtimes": {
+    "nvidia": {
+      "path": "/usr/bin/nvidia-container-runtime",
+      "runtimeArgs": []
+    }
+  },
+  "default-runtime": "nvidia"
+}
+EOF
 
+# 5. Fix Fedora's broken dockerd ExecStart (required!)
+sudo mkdir -p /etc/systemd/system/docker.service.d
+sudo tee /etc/systemd/system/docker.service.d/override.conf >/dev/null <<EOF
+[Service]
+ExecStart=
+ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
+EOF
+
+# 6. Reload and restart Docker
+sudo systemctl daemon-reload
+sudo systemctl restart docker
+
+# 7. Verify it works
+docker info --format '{{.DefaultRuntime}}'   # → should print: nvidia
+docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
 ```
 
 
 #### Build Ubuntu 24.04 Container
+
 `docker build -f ./Dockerfile -t ubuntu24:latest .`
 
 #### Run Docker Container
@@ -194,16 +232,11 @@ b) This command assumes you have set up a Kafka network as described below
 
 `docker run -v ~/src/gst-python-ml/:/root/gst-python-ml -it --rm --gpus all --name ubuntu24 ubuntu24:latest /bin/bash`
 
-In the container shell, run
-
-`pip install -r requirements.txt`
-
-to install base requirements, and then
+In the container shell, install `uv` following steps above
 
-`cd gst-python-ml` to run the pipelines below. After installing requirements,
-it is recommended to open another terminal on host and run
+`cd gst-python-ml` to run the pipelines below.
 
-`docker ps` to get the container id, and then run
+To persist the container, run `docker ps` to get the container id, and then run
 
 `docker commit $CONTAINER_ID` to commit the changes, where `$CONTAINER_ID`
 is the id for your docker instance.
@@ -219,15 +252,6 @@ docker image prune -a -f
 
 ## IMPORTANT NOTES
 
-### Language Elements
-
-1. To use the language elements included in this project, additional pip requirements must be installed from `requirements/language.txt`
-
-2. A specfic version of Cuda is required for these elements: LD_LIBRARY_PATH in `~/.bashrc` must be updated with the following line (!!! adjust for your python version) :
-
-`export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$VIRTUAL_ENV/lib/python3.12/site-packages/nvidia/cublas/lib:$VIRTUAL_ENV/lib/python3.12/site-packages/nvidia/cudnn/lib`
-
-
 ### Birdseye
 
 To use `pyml_birdseye`, additional pip requirements must be installed from the `plugins/python/birdseye` folder.
@@ -235,7 +259,7 @@ To use `pyml_birdseye`, additional pip requirements must be installed from the `
 
 ## Post Install
 
-Run `gst-inspect-1.0 python` to see all of the pyml elements listed.
+Run `gst-inspect-1.0 python` to list pyml elements.
 
 # Building PyPI Package
 
@@ -248,8 +272,12 @@ Run `gst-inspect-1.0 python` to see all of the pyml elements listed.
   password = FOOBAR
 ```
 
-2. `pip install setuptools wheel twine`
+2. 
 
+```
+pip install setuptools wheel twine
+pip install --upgrade build
+```
 ## Build
 
 `python -m build`
@@ -265,86 +293,6 @@ Run `gst-inspect-1.0 python` to see all of the pyml elements listed.
 
 Below are some sample pipelines for the various elements in this project.
 
-### kafkasink
-
-#### Setting up kafka network
-
-`docker network create kafka-network`
-
-and list networks
-
-`docker network ls`
-
-#### docker launch
-
-To launch a docker instance with the kafka network, add ` --network kafka-network  `
-to the docker launch command above.
-
-#### Set up kafka and zookeeper
-
-Note: setup below assumes you are running your pipeline in a docker container. 
-If running pipeline from host, then the port changes from `9092` to `29092`,
-and the broker changes from `kafka` to `localhost`.
-
-```
-docker stop kafka zookeeper
-docker rm kafka zookeeper
-docker run -d --name zookeeper --network kafka-network -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:latest
-docker run -d --name kafka --network kafka-network \
-  -e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \
-  -e KAFKA_ADVERTISED_LISTENERS=INSIDE://kafka:9092,OUTSIDE://localhost:29092 \
-  -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT \
-  -e KAFKA_LISTENERS=INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:29092 \
-  -e KAFKA_INTER_BROKER_LISTENER_NAME=INSIDE \
-  -e KAFKA_BROKER_ID=1 \
-  -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
-  -p 9092:9092 \
-  -p 29092:29092 \
-  confluentinc/cp-kafka:latest
-```
-
-#### Create test topic
-```
-docker exec kafka kafka-topics --create --topic test-kafkasink-topic --bootstrap-server kafka:9092 --partitions 1 --replication-factor 1
-```
-
-#### list topics
-
-`docker exec -it kafka kafka-topics --list --bootstrap-server kafka:9092`
-
-
-#### delete topic
-
-`docker exec -it kafka kafka-topics --delete --topic test-topic --bootstrap-server kafka:9092`
-
-
-#### consume topic
-
-`docker exec -it kafka kafka-console-consumer --bootstrap-server kafka:9092 --topic test-kafkasink-topic --from-beginning`
-
-
-### non ML
-
-`GST_DEBUG=4 gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720 ! pyml_overlay meta-path=data/sample_metadata.json tracking=true ! videoconvert ! autovideosink`
-
-Note: make sure to set the following in `.bashrc` file :
-
-`export GST_PLUGIN_PATH=/home/$USER/src/gst-python-ml/plugins:$GST_PLUGIN_PATH`
-
-
-### streammux/streamdemux pipeline
-
-```
- GST_DEBUG=4 gst-launch-1.0   videotestsrc pattern=ball ! video/x-raw, width=320, height=240 ! queue ! pyml_streammux name=mux   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_1   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_2   mux.src ! queue ! pyml_streamdemux name=demux   demux.src_0 ! queue ! glimagesink  demux.src_1 ! queue ! glimagesink   demux.src_2 ! queue  ! glimagesink
-```
-
-
-### Bird's Eye View
-
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/soccer_single_camera.mp4 ! decodebin ! videoconvert ! pyml_birdseye ! videoconvert ! autovideosink`
-
-`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/soccer_single_camera.mp4 ! decodebin ! videorate ! video/x-raw,framerate=30/1 ! videoconvert ! pyml_birdseye ! videoconvert ! openh264enc ! h264parse ! matroskamux ! filesink location=output.mkv`
-
 ### Classification
 
 ```
@@ -479,3 +427,80 @@ https://huggingface.co/models?sort=trending&search=Helsinki
 ```
 GST_DEBUG=3 gst-launch-1.0 filesrc location=data/soccer_single_camera.mp4 ! decodebin ! videoconvertscale ! video/x-raw,width=640,height=480 ! tee name=t t. ! queue ! textoverlay name=overlay wait-text=false ! videoconvert ! autovideosink t. ! queue leaky=2 max-size-buffers=1 ! videoconvertscale ! video/x-raw,width=240,height=180 ! pyml_caption_qwen device=cuda:0 prompt="In one sentence, describe what you see?" model-name="Qwen/Qwen2.5-VL-3B-Instruct-AWQ" name=cap cap.src ! fakesink async=0 sync=0 cap.text_src ! queue ! coalescehistory history-length=10 ! pyml_llm model-name="Qwen/Qwen3-0.6B" device=cuda system-prompt="You receive the history of what happened in recent times, summarize it nicely with excitement but NEVER mention the specific times. Focus on the most recent events." ! queue ! overlay.text_sink
 ```
+
+
+### Bird's Eye View
+
+`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/soccer_single_camera.mp4 ! decodebin ! videoconvert ! pyml_birdseye ! videoconvert ! autovideosink`
+
+`GST_DEBUG=4 gst-launch-1.0 filesrc location=data/soccer_single_camera.mp4 ! decodebin ! videorate ! video/x-raw,framerate=30/1 ! videoconvert ! pyml_birdseye ! videoconvert ! openh264enc ! h264parse ! matroskamux ! filesink location=output.mkv`
+
+
+### kafkasink
+
+#### Setting up kafka network
+
+`docker network create kafka-network`
+
+and list networks
+
+`docker network ls`
+
+#### docker launch
+
+To launch a docker instance with the kafka network, add ` --network kafka-network  `
+to the docker launch command above.
+
+#### Set up kafka and zookeeper
+
+Note: setup below assumes you are running your pipeline in a docker container. 
+If running pipeline from host, then the port changes from `9092` to `29092`,
+and the broker changes from `kafka` to `localhost`.
+
+```
+docker stop kafka zookeeper
+docker rm kafka zookeeper
+docker run -d --name zookeeper --network kafka-network -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:latest
+docker run -d --name kafka --network kafka-network \
+  -e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \
+  -e KAFKA_ADVERTISED_LISTENERS=INSIDE://kafka:9092,OUTSIDE://localhost:29092 \
+  -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT \
+  -e KAFKA_LISTENERS=INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:29092 \
+  -e KAFKA_INTER_BROKER_LISTENER_NAME=INSIDE \
+  -e KAFKA_BROKER_ID=1 \
+  -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
+  -p 9092:9092 \
+  -p 29092:29092 \
+  confluentinc/cp-kafka:latest
+```
+
+#### Create test topic
+```
+docker exec kafka kafka-topics --create --topic test-kafkasink-topic --bootstrap-server kafka:9092 --partitions 1 --replication-factor 1
+```
+
+#### list topics
+
+`docker exec -it kafka kafka-topics --list --bootstrap-server kafka:9092`
+
+
+#### delete topic
+
+`docker exec -it kafka kafka-topics --delete --topic test-topic --bootstrap-server kafka:9092`
+
+
+#### consume topic
+
+`docker exec -it kafka kafka-console-consumer --bootstrap-server kafka:9092 --topic test-kafkasink-topic --from-beginning`
+
+
+### non ML
+
+`GST_DEBUG=4 gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720 ! pyml_overlay meta-path=data/sample_metadata.json tracking=true ! videoconvert ! autovideosink`
+
+
+### streammux/streamdemux pipeline
+
+```
+ GST_DEBUG=4 gst-launch-1.0   videotestsrc pattern=ball ! video/x-raw, width=320, height=240 ! queue ! pyml_streammux name=mux   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_1   videotestsrc pattern=smpte ! video/x-raw, width=320, height=240 ! queue ! mux.sink_2   mux.src ! queue ! pyml_streamdemux name=demux   demux.src_0 ! queue ! glimagesink  demux.src_1 ! queue ! glimagesink   demux.src_2 ! queue  ! glimagesink
+```