rocket-welder-sdk 1.1.36.dev14__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- rocket_welder_sdk/__init__.py +95 -0
- rocket_welder_sdk/bytes_size.py +234 -0
- rocket_welder_sdk/connection_string.py +291 -0
- rocket_welder_sdk/controllers.py +831 -0
- rocket_welder_sdk/external_controls/__init__.py +30 -0
- rocket_welder_sdk/external_controls/contracts.py +100 -0
- rocket_welder_sdk/external_controls/contracts_old.py +105 -0
- rocket_welder_sdk/frame_metadata.py +138 -0
- rocket_welder_sdk/gst_metadata.py +411 -0
- rocket_welder_sdk/high_level/__init__.py +54 -0
- rocket_welder_sdk/high_level/client.py +235 -0
- rocket_welder_sdk/high_level/connection_strings.py +331 -0
- rocket_welder_sdk/high_level/data_context.py +169 -0
- rocket_welder_sdk/high_level/frame_sink_factory.py +118 -0
- rocket_welder_sdk/high_level/schema.py +195 -0
- rocket_welder_sdk/high_level/transport_protocol.py +238 -0
- rocket_welder_sdk/keypoints_protocol.py +642 -0
- rocket_welder_sdk/opencv_controller.py +278 -0
- rocket_welder_sdk/periodic_timer.py +303 -0
- rocket_welder_sdk/py.typed +2 -0
- rocket_welder_sdk/rocket_welder_client.py +497 -0
- rocket_welder_sdk/segmentation_result.py +420 -0
- rocket_welder_sdk/session_id.py +238 -0
- rocket_welder_sdk/transport/__init__.py +31 -0
- rocket_welder_sdk/transport/frame_sink.py +122 -0
- rocket_welder_sdk/transport/frame_source.py +74 -0
- rocket_welder_sdk/transport/nng_transport.py +197 -0
- rocket_welder_sdk/transport/stream_transport.py +193 -0
- rocket_welder_sdk/transport/tcp_transport.py +154 -0
- rocket_welder_sdk/transport/unix_socket_transport.py +339 -0
- rocket_welder_sdk/ui/__init__.py +48 -0
- rocket_welder_sdk/ui/controls.py +362 -0
- rocket_welder_sdk/ui/icons.py +21628 -0
- rocket_welder_sdk/ui/ui_events_projection.py +226 -0
- rocket_welder_sdk/ui/ui_service.py +358 -0
- rocket_welder_sdk/ui/value_types.py +72 -0
- rocket_welder_sdk-1.1.36.dev14.dist-info/METADATA +845 -0
- rocket_welder_sdk-1.1.36.dev14.dist-info/RECORD +40 -0
- rocket_welder_sdk-1.1.36.dev14.dist-info/WHEEL +5 -0
- rocket_welder_sdk-1.1.36.dev14.dist-info/top_level.txt +1 -0
|
@@ -0,0 +1,845 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: rocket-welder-sdk
|
|
3
|
+
Version: 1.1.36.dev14
|
|
4
|
+
Summary: High-performance video streaming SDK for RocketWelder services using ZeroBuffer IPC
|
|
5
|
+
Home-page: https://github.com/modelingevolution/rocket-welder-sdk
|
|
6
|
+
Author: ModelingEvolution
|
|
7
|
+
Author-email: ModelingEvolution <info@modelingevolution.com>
|
|
8
|
+
Maintainer-email: ModelingEvolution <info@modelingevolution.com>
|
|
9
|
+
License: MIT
|
|
10
|
+
Project-URL: Homepage, https://github.com/modelingevolution/rocket-welder-sdk
|
|
11
|
+
Project-URL: Repository, https://github.com/modelingevolution/rocket-welder-sdk.git
|
|
12
|
+
Project-URL: Issues, https://github.com/modelingevolution/rocket-welder-sdk/issues
|
|
13
|
+
Keywords: video,streaming,gstreamer,ipc,shared-memory,zerobuffer,computer-vision
|
|
14
|
+
Classifier: Development Status :: 4 - Beta
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Topic :: Multimedia :: Video
|
|
17
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
18
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
19
|
+
Classifier: Programming Language :: Python :: 3
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
24
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
25
|
+
Classifier: Operating System :: OS Independent
|
|
26
|
+
Classifier: Typing :: Typed
|
|
27
|
+
Requires-Python: >=3.8
|
|
28
|
+
Description-Content-Type: text/markdown
|
|
29
|
+
Requires-Dist: numpy>=1.20.0
|
|
30
|
+
Requires-Dist: opencv-python>=4.5.0
|
|
31
|
+
Requires-Dist: zerobuffer-ipc>=1.1.17
|
|
32
|
+
Requires-Dist: pydantic>=2.5.0
|
|
33
|
+
Requires-Dist: py-micro-plumberd>=0.1.8
|
|
34
|
+
Requires-Dist: typing-extensions>=4.0.0
|
|
35
|
+
Provides-Extra: nng
|
|
36
|
+
Requires-Dist: pynng>=0.7.2; extra == "nng"
|
|
37
|
+
Provides-Extra: dev
|
|
38
|
+
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
39
|
+
Requires-Dist: pytest-cov>=4.0; extra == "dev"
|
|
40
|
+
Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
|
|
41
|
+
Requires-Dist: black>=22.0; extra == "dev"
|
|
42
|
+
Requires-Dist: mypy>=1.0; extra == "dev"
|
|
43
|
+
Requires-Dist: ruff>=0.1.0; extra == "dev"
|
|
44
|
+
Requires-Dist: types-setuptools; extra == "dev"
|
|
45
|
+
Requires-Dist: pynng>=0.7.2; extra == "dev"
|
|
46
|
+
Dynamic: author
|
|
47
|
+
Dynamic: home-page
|
|
48
|
+
Dynamic: requires-python
|
|
49
|
+
|
|
50
|
+
# Rocket Welder SDK
|
|
51
|
+
|
|
52
|
+
[](https://www.nuget.org/packages/RocketWelder.SDK/)
|
|
53
|
+
[](https://pypi.org/project/rocket-welder-sdk/)
|
|
54
|
+
[](https://github.com/modelingevolution/rocket-welder-sdk-vcpkg-registry)
|
|
55
|
+
[](https://opensource.org/licenses/MIT)
|
|
56
|
+
|
|
57
|
+
**Client libraries for building custom AI/ML video processing containers that integrate with RocketWelder (Neuron) devices.**
|
|
58
|
+
|
|
59
|
+
## Overview
|
|
60
|
+
|
|
61
|
+
The Rocket Welder SDK enables AI/ML developers to build custom video processing containers for Neuron industrial vision devices. It provides high-performance, **zero-copy** frame access via shared memory, supporting real-time computer vision, object detection, and AI inference workloads.
|
|
62
|
+
|
|
63
|
+
**Target Audience**: AI/ML developers building containerized applications for:
|
|
64
|
+
- Real-time object detection (YOLO, custom models)
|
|
65
|
+
- Computer vision processing
|
|
66
|
+
- AI inference on video streams
|
|
67
|
+
- Industrial vision applications
|
|
68
|
+
|
|
69
|
+
## Table of Contents
|
|
70
|
+
|
|
71
|
+
- [Quick Start](#quick-start)
|
|
72
|
+
- [Your First AI Processing Container](#your-first-ai-processing-container)
|
|
73
|
+
- [Development Workflow](#development-workflow)
|
|
74
|
+
- [Deploying to Neuron Device](#deploying-to-neuron-device)
|
|
75
|
+
- [RocketWelder Integration](#rocketwelder-integration)
|
|
76
|
+
- [API Reference](#api-reference)
|
|
77
|
+
- [Production Best Practices](#production-best-practices)
|
|
78
|
+
|
|
79
|
+
## Quick Start
|
|
80
|
+
|
|
81
|
+
### Installation
|
|
82
|
+
|
|
83
|
+
| Language | Package Manager | Package Name |
|
|
84
|
+
|----------|----------------|--------------|
|
|
85
|
+
| C++ | vcpkg | rocket-welder-sdk |
|
|
86
|
+
| C# | NuGet | RocketWelder.SDK |
|
|
87
|
+
| Python | pip | rocket-welder-sdk |
|
|
88
|
+
|
|
89
|
+
#### Python
|
|
90
|
+
```bash
|
|
91
|
+
pip install rocket-welder-sdk
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
#### C#
|
|
95
|
+
```bash
|
|
96
|
+
dotnet add package RocketWelder.SDK
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
#### C++
|
|
100
|
+
```bash
|
|
101
|
+
vcpkg install rocket-welder-sdk
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## Your First AI Processing Container
|
|
105
|
+
|
|
106
|
+
### Starting with Examples
|
|
107
|
+
|
|
108
|
+
The SDK includes ready-to-use examples in the `/examples` directory:
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
examples/
|
|
112
|
+
├── python/
|
|
113
|
+
│ ├── simple_client.py # Timestamp overlay example
|
|
114
|
+
│ ├── integration_client.py # Testing with --exit-after
|
|
115
|
+
│ └── Dockerfile # Ready-to-build container
|
|
116
|
+
├── csharp/
|
|
117
|
+
│ └── SimpleClient/
|
|
118
|
+
│ ├── Program.cs # Full example with UI controls
|
|
119
|
+
│ └── Dockerfile # Ready-to-build container
|
|
120
|
+
└── cpp/
|
|
121
|
+
├── simple_client.cpp
|
|
122
|
+
└── CMakeLists.txt
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
### Python Example - Simple Timestamp Overlay
|
|
126
|
+
|
|
127
|
+
```python
|
|
128
|
+
#!/usr/bin/env python3
|
|
129
|
+
import sys
|
|
130
|
+
import cv2
|
|
131
|
+
import numpy as np
|
|
132
|
+
from datetime import datetime
|
|
133
|
+
import rocket_welder_sdk as rw
|
|
134
|
+
|
|
135
|
+
# Create client - reads CONNECTION_STRING from environment or args
|
|
136
|
+
client = rw.Client.from_(sys.argv)
|
|
137
|
+
|
|
138
|
+
def process_frame(frame: np.ndarray) -> None:
|
|
139
|
+
"""Add timestamp overlay to frame - zero copy!"""
|
|
140
|
+
timestamp = datetime.now().strftime("%H:%M:%S")
|
|
141
|
+
cv2.putText(frame, timestamp, (10, 30),
|
|
142
|
+
cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)
|
|
143
|
+
|
|
144
|
+
# Start processing
|
|
145
|
+
client.start(process_frame)
|
|
146
|
+
|
|
147
|
+
# Keep running
|
|
148
|
+
while client.is_running:
|
|
149
|
+
time.sleep(0.1)
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
### Building Your Container
|
|
153
|
+
|
|
154
|
+
```bash
|
|
155
|
+
# Navigate to examples directory
|
|
156
|
+
cd python/examples
|
|
157
|
+
|
|
158
|
+
# Build Docker image
|
|
159
|
+
docker build -t my-ai-app:v1 -f Dockerfile ..
|
|
160
|
+
|
|
161
|
+
# Test locally with file
|
|
162
|
+
docker run --rm \
|
|
163
|
+
-e CONNECTION_STRING="file:///data/test.mp4?loop=true" \
|
|
164
|
+
-v /path/to/video.mp4:/data/test.mp4:ro \
|
|
165
|
+
my-ai-app:v1
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
## Development Workflow
|
|
169
|
+
|
|
170
|
+
### Step 1: Test Locally with Video File
|
|
171
|
+
|
|
172
|
+
Start by testing your container locally before deploying to Neuron:
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
# Build your container
|
|
176
|
+
docker build -t my-ai-app:v1 -f python/examples/Dockerfile .
|
|
177
|
+
|
|
178
|
+
# Test with a video file
|
|
179
|
+
docker run --rm \
|
|
180
|
+
-e CONNECTION_STRING="file:///data/test.mp4?loop=true&preview=false" \
|
|
181
|
+
-v $(pwd)/examples/test_stream.mp4:/data/test.mp4:ro \
|
|
182
|
+
my-ai-app:v1
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
You can also see preview in your terminal.
|
|
186
|
+
|
|
187
|
+
```bash
|
|
188
|
+
# Install x11-apps
|
|
189
|
+
sudo apt install x11-apps
|
|
190
|
+
|
|
191
|
+
# Test with a video file
|
|
192
|
+
docker run --rm \
|
|
193
|
+
-e CONNECTION_STRING="file:///data/test.mp4?loop=true&preview=true" \
|
|
194
|
+
-e DISPLAY=$DISPLAY \
|
|
195
|
+
-v /path/to/your/file.mp4:/data/test.mp4:ro -v /tmp/.X11-unix:/tmp/.X11-unix my-ai-app:v1
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
### Step 2: Test with Live Stream from Neuron
|
|
199
|
+
|
|
200
|
+
Once your container works locally, test it with a live stream from your Neuron device:
|
|
201
|
+
|
|
202
|
+
#### Configure RocketWelder Pipeline for Streaming
|
|
203
|
+
|
|
204
|
+
1. Access RocketWelder UI on your Neuron device (usually `http://neuron-ip:8080`)
|
|
205
|
+
2. Open **Pipeline Designer**
|
|
206
|
+
3. Click **"Add Element"**
|
|
207
|
+
4. Choose your video source (e.g., `pylonsrc` for Basler cameras)
|
|
208
|
+
5. Add **caps filter** to specify format: `video/x-raw,width=1920,height=1080,format=GRAY8`
|
|
209
|
+
6. Add **jpegenc** element
|
|
210
|
+
7. Add **tcpserversink** element with properties:
|
|
211
|
+
- `host`: `0.0.0.0`
|
|
212
|
+
- `port`: `5000`
|
|
213
|
+
8. Start the pipeline
|
|
214
|
+
|
|
215
|
+
Example pipeline:
|
|
216
|
+
```
|
|
217
|
+
pylonsrc → video/x-raw,width=1920,height=1080,format=GRAY8 → queue max-buffers-size=1, Leaky=Upstream → jpegenc → tcpserversink host=0.0.0.0 port=5000 sync=false
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
#### Connect from Your Dev Laptop
|
|
221
|
+
|
|
222
|
+
```bash
|
|
223
|
+
# On your laptop - connect to Neuron's TCP stream
|
|
224
|
+
docker run --rm \
|
|
225
|
+
-e CONNECTION_STRING="mjpeg+tcp://neuron-ip:5000" \
|
|
226
|
+
--network host \
|
|
227
|
+
my-ai-app:v1
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
You can also see preview in your terminal.
|
|
231
|
+
```bash
|
|
232
|
+
docker run --rm \
|
|
233
|
+
-e CONNECTION_STRING="mjpeg+tcp://<neuron-ip>:<tcp-server-sink-port>?preview=true" \
|
|
234
|
+
-e DISPLAY=$DISPLAY \
|
|
235
|
+
-v /tmp/.X11-unix:/tmp/.X11-unix \
|
|
236
|
+
--network host my-ai-app:v1
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
This allows you to:
|
|
240
|
+
- Test your AI processing with real camera feeds
|
|
241
|
+
- Debug frame processing logic
|
|
242
|
+
- Measure performance with actual hardware
|
|
243
|
+
|
|
244
|
+
## Deploying to Neuron Device
|
|
245
|
+
|
|
246
|
+
### Option 1: Local Docker Registry (Recommended for Development)
|
|
247
|
+
|
|
248
|
+
This is the fastest workflow for iterative development:
|
|
249
|
+
|
|
250
|
+
#### Setup Registry on Your Laptop (One-time)
|
|
251
|
+
|
|
252
|
+
```bash
|
|
253
|
+
# Start a local Docker registry
|
|
254
|
+
docker run -d \
|
|
255
|
+
-p 5000:5000 \
|
|
256
|
+
--restart=always \
|
|
257
|
+
--name registry \
|
|
258
|
+
registry:2
|
|
259
|
+
|
|
260
|
+
# Verify it's running
|
|
261
|
+
curl http://localhost:5000/v2/_catalog
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
#### Configure Neuron to Use Your Laptop Registry (One-time)
|
|
265
|
+
|
|
266
|
+
```bash
|
|
267
|
+
# SSH to Neuron device
|
|
268
|
+
ssh user@neuron-ip
|
|
269
|
+
|
|
270
|
+
# Edit Docker daemon config
|
|
271
|
+
sudo nano /etc/docker/daemon.json
|
|
272
|
+
|
|
273
|
+
# Add your laptop's IP to insecure registries:
|
|
274
|
+
{
|
|
275
|
+
"insecure-registries": ["laptop-ip:5000"]
|
|
276
|
+
}
|
|
277
|
+
|
|
278
|
+
# Restart Docker
|
|
279
|
+
sudo systemctl restart docker
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
**Note**: Replace `laptop-ip` with your laptop's actual IP address (e.g., `192.168.1.100`).
|
|
283
|
+
To find it: `ip addr show` or `ifconfig`
|
|
284
|
+
|
|
285
|
+
#### Push Image to Your Registry
|
|
286
|
+
|
|
287
|
+
```bash
|
|
288
|
+
# On your laptop - tag for local registry
|
|
289
|
+
docker tag my-ai-app:v1 localhost:5000/my-ai-app:v1
|
|
290
|
+
|
|
291
|
+
# Push to registry
|
|
292
|
+
docker push localhost:5000/my-ai-app:v1
|
|
293
|
+
|
|
294
|
+
# Verify push
|
|
295
|
+
curl http://localhost:5000/v2/my-ai-app/tags/list
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
#### Pull on Neuron Device
|
|
299
|
+
|
|
300
|
+
```bash
|
|
301
|
+
# SSH to Neuron
|
|
302
|
+
ssh user@neuron-ip
|
|
303
|
+
|
|
304
|
+
# Pull from laptop registry
|
|
305
|
+
docker pull laptop-ip:5000/my-ai-app:v1
|
|
306
|
+
|
|
307
|
+
# Verify image
|
|
308
|
+
docker images | grep my-ai-app
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
#### Workflow Summary
|
|
312
|
+
|
|
313
|
+
```bash
|
|
314
|
+
# Iterative development loop:
|
|
315
|
+
1. Edit code on laptop
|
|
316
|
+
2. docker build -t localhost:5000/my-ai-app:v1 .
|
|
317
|
+
3. docker push localhost:5000/my-ai-app:v1
|
|
318
|
+
4. Configure in RocketWelder UI (once)
|
|
319
|
+
5. RocketWelder pulls and runs your container
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
### Option 2: Export/Import (For One-off Transfers)
|
|
323
|
+
|
|
324
|
+
Useful when you don't want to set up a registry:
|
|
325
|
+
|
|
326
|
+
```bash
|
|
327
|
+
# On your laptop - save image to tar
|
|
328
|
+
docker save my-ai-app:v1 | gzip > my-ai-app-v1.tar.gz
|
|
329
|
+
|
|
330
|
+
# Transfer to Neuron
|
|
331
|
+
scp my-ai-app-v1.tar.gz user@neuron-ip:/tmp/
|
|
332
|
+
|
|
333
|
+
# SSH to Neuron and load
|
|
334
|
+
ssh user@neuron-ip
|
|
335
|
+
docker load < /tmp/my-ai-app-v1.tar.gz
|
|
336
|
+
|
|
337
|
+
# Verify
|
|
338
|
+
docker images | grep my-ai-app
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
### Option 3: Azure Container Registry (Production)
|
|
342
|
+
|
|
343
|
+
For production deployments:
|
|
344
|
+
|
|
345
|
+
```bash
|
|
346
|
+
# Login to ACR (Azure Container Registry)
|
|
347
|
+
az acr login --name your-registry
|
|
348
|
+
|
|
349
|
+
# Tag and push
|
|
350
|
+
docker tag my-ai-app:v1 your-registry.azurecr.io/my-ai-app:v1
|
|
351
|
+
docker push your-registry.azurecr.io/my-ai-app:v1
|
|
352
|
+
|
|
353
|
+
# Configure Neuron to use ACR (credentials required)
|
|
354
|
+
```
|
|
355
|
+
|
|
356
|
+
## RocketWelder Integration
|
|
357
|
+
|
|
358
|
+
### Understanding zerosink vs zerofilter
|
|
359
|
+
|
|
360
|
+
RocketWelder provides two GStreamer elements for container integration:
|
|
361
|
+
|
|
362
|
+
| Element | Mode | Use Case |
|
|
363
|
+
|---------|------|----------|
|
|
364
|
+
| **zerosink** | One-way | RocketWelder → Your Container<br/>Read frames, process, log results |
|
|
365
|
+
| **zerofilter** | Duplex | RocketWelder ↔ Your Container<br/>Read frames, modify them, return modified frames |
|
|
366
|
+
|
|
367
|
+
**Most AI use cases use `zerosink`** (one-way mode):
|
|
368
|
+
- Object detection (draw bounding boxes)
|
|
369
|
+
- Classification (overlay labels)
|
|
370
|
+
- Analytics (count objects, log events)
|
|
371
|
+
|
|
372
|
+
**Use `zerofilter`** (duplex mode) when:
|
|
373
|
+
- You need to modify frames and return them to the pipeline
|
|
374
|
+
- Real-time visual effects/filters
|
|
375
|
+
- Frame enhancement before encoding
|
|
376
|
+
|
|
377
|
+
### Configuring Your Container in RocketWelder
|
|
378
|
+
|
|
379
|
+
#### Step-by-Step UI Configuration
|
|
380
|
+
|
|
381
|
+
1. **Access RocketWelder UI**
|
|
382
|
+
- Navigate to `http://neuron-ip:8080`
|
|
383
|
+
- Log in to your Neuron device
|
|
384
|
+
|
|
385
|
+
2. **Open Pipeline Designer**
|
|
386
|
+
- Go to **Pipelines** section
|
|
387
|
+
- Create new pipeline or edit existing
|
|
388
|
+
|
|
389
|
+
3. **Add Video Source**
|
|
390
|
+
- Click **"Add Element"**
|
|
391
|
+
- Choose your camera source (e.g., `pylonsrc`, `aravissrc`)
|
|
392
|
+
- Configure camera properties
|
|
393
|
+
|
|
394
|
+
4. **Add Format**
|
|
395
|
+
- Add caps filter: `video/x-raw,format=RGB`
|
|
396
|
+
|
|
397
|
+
5. **Add queueue**
|
|
398
|
+
- max-num-buffers: 1
|
|
399
|
+
- leaky: upstream
|
|
400
|
+
|
|
401
|
+
5. **Add ZeroBuffer Element**
|
|
402
|
+
- Click **"Add Element"**
|
|
403
|
+
- Select **"zerosink"** (or **"zerofilter"** for duplex mode)
|
|
404
|
+
- Scroll down in properties panel on the right
|
|
405
|
+
|
|
406
|
+
6. **Configure Consumer**
|
|
407
|
+
- Toggle **"Enable ZeroBuffer Consumer"** ✓
|
|
408
|
+
- Select **"Consumer Mode"** dropdown
|
|
409
|
+
- Choose **"Docker Container"** (not Process)
|
|
410
|
+
|
|
411
|
+
7. **Configure Docker Settings**
|
|
412
|
+
- **Image**: Enter your image name
|
|
413
|
+
- Local registry: `laptop-ip:5000/my-ai-app`
|
|
414
|
+
- ACR: `your-registry.azurecr.io/my-ai-app`
|
|
415
|
+
- Loaded image: `my-ai-app`
|
|
416
|
+
- **Tag**: `v1` (or your version tag)
|
|
417
|
+
- **Environment Variables**: (optional) Add custom env vars if needed
|
|
418
|
+
- **Auto-remove**: ✓ (recommended - cleans up container on stop)
|
|
419
|
+
|
|
420
|
+
8. **Save Pipeline Configuration**
|
|
421
|
+
|
|
422
|
+
9. **Start Pipeline**
|
|
423
|
+
- Click **"Start"** button
|
|
424
|
+
- RocketWelder will automatically:
|
|
425
|
+
- Pull your Docker image (if not present)
|
|
426
|
+
- Create shared memory buffer
|
|
427
|
+
- Launch your container with `CONNECTION_STRING` env var
|
|
428
|
+
- Start streaming frames
|
|
429
|
+
|
|
430
|
+
### Automatic Environment Variables
|
|
431
|
+
|
|
432
|
+
When RocketWelder launches your container, it automatically sets:
|
|
433
|
+
|
|
434
|
+
```bash
|
|
435
|
+
CONNECTION_STRING=shm://zerobuffer-abc123-456?size=20MB&metadata=4KB&mode=oneway
|
|
436
|
+
SessionId=def789-012 # For UI controls (if enabled)
|
|
437
|
+
EventStore=esdb://host.docker.internal:2113?tls=false # For external controls
|
|
438
|
+
```
|
|
439
|
+
|
|
440
|
+
Your SDK code simply reads `CONNECTION_STRING`:
|
|
441
|
+
|
|
442
|
+
```python
|
|
443
|
+
# Python - automatically reads CONNECTION_STRING from environment
|
|
444
|
+
client = rw.Client.from_(sys.argv)
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
```csharp
|
|
448
|
+
// C# - automatically reads CONNECTION_STRING
|
|
449
|
+
var client = RocketWelderClient.From(args);
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
### Example Pipeline Configurations
|
|
453
|
+
|
|
454
|
+
#### AI Object Detection Pipeline
|
|
455
|
+
|
|
456
|
+
```
|
|
457
|
+
pylonsrc
|
|
458
|
+
→ video/x-raw,width=1920,height=1080,format=Gray8
|
|
459
|
+
→ videoconvert
|
|
460
|
+
→ zerosink
|
|
461
|
+
└─ Docker: laptop-ip:5000/yolo-detector:v1
|
|
462
|
+
```
|
|
463
|
+
|
|
464
|
+
Your YOLO container receives frames, detects objects, draws bounding boxes.
|
|
465
|
+
|
|
466
|
+
#### Dual Output: AI Processing
|
|
467
|
+
|
|
468
|
+
```
|
|
469
|
+
pylonsrc
|
|
470
|
+
→ video/x-raw,width=1920,height=1080,format=Gray8
|
|
471
|
+
→ tee name=t
|
|
472
|
+
t. → queue → jpegenc → tcpserversink
|
|
473
|
+
t. → queue → zerofilter → queue → jpegenc → tcpserversink
|
|
474
|
+
└─ Docker: laptop-ip:5000/my-ai-app:v1
|
|
475
|
+
```
|
|
476
|
+
|
|
477
|
+
#### Real-time Frame Enhancement with Live Preview (Duplex Mode)
|
|
478
|
+
|
|
479
|
+
```
|
|
480
|
+
→ pylonsrc hdr-sequence="5000,5500" hdr-sequence2="19,150" hdr-profile=0
|
|
481
|
+
→ video/x-raw,width=1920,height=1080,format=Gray8
|
|
482
|
+
→ queue max-num-buffers=1 leaky=upstream
|
|
483
|
+
→ hdr mode=burst num-frames=2
|
|
484
|
+
→ sortingbuffer
|
|
485
|
+
→ queue max-num-buffers=1 leaky=upstream
|
|
486
|
+
→ zerofilter
|
|
487
|
+
└─ Docker: laptop-ip:5000/frame-enhancer:v1
|
|
488
|
+
→ queue max-num-buffers=1 leaky=upstream
|
|
489
|
+
→ jpegenc
|
|
490
|
+
→ multipartmux enable-html=true
|
|
491
|
+
→ tcpserversink host=0.0.0.0 port=5000 sync=false
|
|
492
|
+
```
|
|
493
|
+
|
|
494
|
+
In duplex mode with `zerofilter`, your container:
|
|
495
|
+
1. Receives input frames via shared memory (automatically configured by RocketWelder)
|
|
496
|
+
2. Processes them in real-time (e.g., AI enhancement, object detection, overlays)
|
|
497
|
+
3. Writes modified frames back to shared memory
|
|
498
|
+
4. Modified frames flow back into RocketWelder pipeline for streaming/display
|
|
499
|
+
|
|
500
|
+
**Pipeline elements explained:**
|
|
501
|
+
- `pylonsrc hdr-sequence="5000,5500"`: Configures HDR Profile 0 with 5000μs and 5500μs exposures (cycles automatically via camera sequencer)
|
|
502
|
+
- `hdr-sequence2="19,150"`: Configures HDR Profile 1 with 2 exposures for runtime switching
|
|
503
|
+
- `hdr-profile=0`: Starts with Profile 0 (can be changed at runtime to switch between lighting conditions), requires a branch with histogram, dre and pylontarget.
|
|
504
|
+
- `hdr processing-mode=burst num-frames=2`: HDR blending element - combines multiple exposures into single HDR frame
|
|
505
|
+
- `sortingbuffer skip-behaviour=hdr`: Reorders out-of-order frames from Pylon camera using HDR metadata (MasterSequence, ExposureSequenceIndex) - automatically detects frame order using `image_number` from Pylon metadata
|
|
506
|
+
- `zerofilter`: Bidirectional shared memory connection to your Docker container
|
|
507
|
+
- `jpegenc`: JPEG compression for network streaming
|
|
508
|
+
- `multipartmux enable-html=true`: Creates MJPEG stream with CORS headers for browser viewing
|
|
509
|
+
- `tcpserversink`: Streams to RocketWelder UI at `http://neuron-ip:5000`
|
|
510
|
+
|
|
511
|
+
**View live preview:**
|
|
512
|
+
Open in browser: `http://neuron-ip:5000` to see the processed video stream with your AI enhancements in real-time!
|
|
513
|
+
|
|
514
|
+
**HDR Profile Switching:**
|
|
515
|
+
The dual-profile system allows runtime switching between lighting conditions:
|
|
516
|
+
- Profile 0 (2 exposures): Fast cycling for normal conditions
|
|
517
|
+
- Profile 1 (2 exposures): More exposures for challenging lighting
|
|
518
|
+
- Switch dynamically via `hdr-profile` property without stopping the pipeline (requires another branch, histogram, dre, pylon-target)
|
|
519
|
+
|
|
520
|
+
**Use case examples:**
|
|
521
|
+
- **AI object detection**: Draw bounding boxes that appear in RocketWelder preview
|
|
522
|
+
- **Real-time enhancement**: AI super-resolution, denoising, stabilization
|
|
523
|
+
- **Visual feedback**: Add crosshairs, tracking overlays, status indicators
|
|
524
|
+
- **Quality control**: Highlight defects or areas of interest in industrial inspection
|
|
525
|
+
|
|
526
|
+
## Connection String Format
|
|
527
|
+
|
|
528
|
+
The SDK uses URI-style connection strings:
|
|
529
|
+
|
|
530
|
+
```
|
|
531
|
+
protocol://[host[:port]]/[path][?param1=value1¶m2=value2]
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
### Supported Protocols
|
|
535
|
+
|
|
536
|
+
#### Shared Memory (Production - Automatic)
|
|
537
|
+
```
|
|
538
|
+
shm://buffer-name?size=20MB&metadata=4KB&mode=oneway
|
|
539
|
+
```
|
|
540
|
+
|
|
541
|
+
When deployed with RocketWelder, this is set automatically via `CONNECTION_STRING` environment variable.
|
|
542
|
+
|
|
543
|
+
**Parameters:**
|
|
544
|
+
- `size`: Buffer size (default: 20MB, supports: B, KB, MB, GB)
|
|
545
|
+
- `metadata`: Metadata size (default: 4KB)
|
|
546
|
+
- `mode`: `oneway` (zerosink) or `duplex` (zerofilter)
|
|
547
|
+
|
|
548
|
+
#### File Protocol (Local Testing)
|
|
549
|
+
```
|
|
550
|
+
file:///path/to/video.mp4?loop=true&preview=false
|
|
551
|
+
```
|
|
552
|
+
|
|
553
|
+
**Parameters:**
|
|
554
|
+
- `loop`: Loop playback (`true`/`false`, default: `false`)
|
|
555
|
+
- `preview`: Show preview window (`true`/`false`, default: `false`)
|
|
556
|
+
|
|
557
|
+
#### MJPEG over TCP (Development/Testing)
|
|
558
|
+
```
|
|
559
|
+
mjpeg+tcp://neuron-ip:5000
|
|
560
|
+
```
|
|
561
|
+
|
|
562
|
+
Connect to RocketWelder's `tcpserversink` for development testing.
|
|
563
|
+
|
|
564
|
+
#### MJPEG over HTTP
|
|
565
|
+
```
|
|
566
|
+
mjpeg+http://camera-ip:8080
|
|
567
|
+
```
|
|
568
|
+
|
|
569
|
+
For network cameras or HTTP streamers.
|
|
570
|
+
|
|
571
|
+
## API Reference
|
|
572
|
+
|
|
573
|
+
### Python API
|
|
574
|
+
|
|
575
|
+
```python
|
|
576
|
+
import rocket_welder_sdk as rw
|
|
577
|
+
|
|
578
|
+
# Create client (reads CONNECTION_STRING from env or args)
|
|
579
|
+
client = rw.Client.from_(sys.argv)
|
|
580
|
+
|
|
581
|
+
# Or specify connection string directly
|
|
582
|
+
client = rw.Client.from_connection_string("shm://buffer-name?size=20MB")
|
|
583
|
+
|
|
584
|
+
# Process frames - one-way mode
|
|
585
|
+
@client.on_frame
|
|
586
|
+
def process_frame(frame: np.ndarray) -> None:
|
|
587
|
+
# frame is a numpy array (height, width, channels)
|
|
588
|
+
# Modify in-place for zero-copy performance
|
|
589
|
+
cv2.putText(frame, "AI Processing", (10, 30),
|
|
590
|
+
cv2.FONT_HERSHEY_SIMPLEX, 1.0, (0, 255, 0), 2)
|
|
591
|
+
|
|
592
|
+
# Process frames - duplex mode
|
|
593
|
+
def process_frame_duplex(input_frame: np.ndarray, output_frame: np.ndarray) -> None:
|
|
594
|
+
# Copy input to output and modify
|
|
595
|
+
np.copyto(output_frame, input_frame)
|
|
596
|
+
# Add AI overlay to output_frame
|
|
597
|
+
cv2.putText(output_frame, "Processed", (10, 30),
|
|
598
|
+
cv2.FONT_HERSHEY_SIMPLEX, 1.0, (0, 255, 0), 2)
|
|
599
|
+
|
|
600
|
+
# Start processing
|
|
601
|
+
client.start(process_frame) # or process_frame_duplex for duplex mode
|
|
602
|
+
|
|
603
|
+
# Keep running
|
|
604
|
+
while client.is_running:
|
|
605
|
+
time.sleep(0.1)
|
|
606
|
+
|
|
607
|
+
# Stop
|
|
608
|
+
client.stop()
|
|
609
|
+
```
|
|
610
|
+
|
|
611
|
+
### C# API
|
|
612
|
+
|
|
613
|
+
```csharp
|
|
614
|
+
using RocketWelder.SDK;
|
|
615
|
+
using Emgu.CV;
|
|
616
|
+
|
|
617
|
+
// Create client (reads CONNECTION_STRING from env or config)
|
|
618
|
+
var client = RocketWelderClient.From(args);
|
|
619
|
+
|
|
620
|
+
// Or specify connection string directly
|
|
621
|
+
var client = RocketWelderClient.FromConnectionString("shm://buffer-name?size=20MB");
|
|
622
|
+
|
|
623
|
+
// Process frames - one-way mode
|
|
624
|
+
client.Start((Mat frame) =>
|
|
625
|
+
{
|
|
626
|
+
// frame is an Emgu.CV.Mat (zero-copy)
|
|
627
|
+
CvInvoke.PutText(frame, "AI Processing", new Point(10, 30),
|
|
628
|
+
FontFace.HersheySimplex, 1.0, new MCvScalar(0, 255, 0), 2);
|
|
629
|
+
});
|
|
630
|
+
|
|
631
|
+
// Process frames - duplex mode
|
|
632
|
+
client.Start((Mat input, Mat output) =>
|
|
633
|
+
{
|
|
634
|
+
input.CopyTo(output);
|
|
635
|
+
CvInvoke.PutText(output, "Processed", new Point(10, 30),
|
|
636
|
+
FontFace.HersheySimplex, 1.0, new MCvScalar(0, 255, 0), 2);
|
|
637
|
+
});
|
|
638
|
+
```
|
|
639
|
+
|
|
640
|
+
### C++ API
|
|
641
|
+
|
|
642
|
+
```cpp
|
|
643
|
+
#include <rocket_welder/client.hpp>
|
|
644
|
+
#include <opencv2/opencv.hpp>
|
|
645
|
+
|
|
646
|
+
// Create client (reads CONNECTION_STRING from env or args)
|
|
647
|
+
auto client = rocket_welder::Client::from(argc, argv);
|
|
648
|
+
|
|
649
|
+
// Or specify connection string directly
|
|
650
|
+
auto client = rocket_welder::Client::from_connection_string("shm://buffer-name?size=20MB");
|
|
651
|
+
|
|
652
|
+
// Process frames - one-way mode
|
|
653
|
+
client.on_frame([](cv::Mat& frame) {
|
|
654
|
+
// frame is a cv::Mat reference (zero-copy)
|
|
655
|
+
cv::putText(frame, "AI Processing", cv::Point(10, 30),
|
|
656
|
+
cv::FONT_HERSHEY_SIMPLEX, 1.0, cv::Scalar(0, 255, 0), 2);
|
|
657
|
+
});
|
|
658
|
+
|
|
659
|
+
// Process frames - duplex mode
|
|
660
|
+
client.on_frame([](const cv::Mat& input, cv::Mat& output) {
|
|
661
|
+
input.copyTo(output);
|
|
662
|
+
cv::putText(output, "Processed", cv::Point(10, 30),
|
|
663
|
+
cv::FONT_HERSHEY_SIMPLEX, 1.0, cv::Scalar(0, 255, 0), 2);
|
|
664
|
+
});
|
|
665
|
+
|
|
666
|
+
// Start processing
|
|
667
|
+
client.start();
|
|
668
|
+
```
|
|
669
|
+
|
|
670
|
+
## Production Best Practices
|
|
671
|
+
|
|
672
|
+
### Performance Optimization
|
|
673
|
+
|
|
674
|
+
1. **Zero-Copy Processing**
|
|
675
|
+
- Modify frames in-place when possible
|
|
676
|
+
- Avoid unnecessary memory allocations in the frame processing loop
|
|
677
|
+
- Use OpenCV operations that work directly on the frame buffer
|
|
678
|
+
|
|
679
|
+
2. **Frame Rate Management**
|
|
680
|
+
```python
|
|
681
|
+
# Process every Nth frame for expensive AI operations
|
|
682
|
+
frame_count = 0
|
|
683
|
+
|
|
684
|
+
def process_frame(frame):
|
|
685
|
+
global frame_count
|
|
686
|
+
frame_count += 1
|
|
687
|
+
if frame_count % 5 == 0: # Process every 5th frame
|
|
688
|
+
run_expensive_ai_model(frame)
|
|
689
|
+
```
|
|
690
|
+
|
|
691
|
+
3. **Logging**
|
|
692
|
+
- Use structured logging with appropriate levels
|
|
693
|
+
- Avoid logging in the frame processing loop for production
|
|
694
|
+
- Log only important events (errors, detections, etc.)
|
|
695
|
+
|
|
696
|
+
### Error Handling
|
|
697
|
+
|
|
698
|
+
```python
|
|
699
|
+
import logging
|
|
700
|
+
import rocket_welder_sdk as rw
|
|
701
|
+
|
|
702
|
+
logger = logging.getLogger(__name__)
|
|
703
|
+
|
|
704
|
+
client = rw.Client.from_(sys.argv)
|
|
705
|
+
|
|
706
|
+
def on_error(sender, error):
|
|
707
|
+
logger.error(f"Client error: {error.Exception}")
|
|
708
|
+
# Implement recovery logic or graceful shutdown
|
|
709
|
+
|
|
710
|
+
client.OnError += on_error
|
|
711
|
+
```
|
|
712
|
+
|
|
713
|
+
### Monitoring
|
|
714
|
+
|
|
715
|
+
```python
|
|
716
|
+
import time
|
|
717
|
+
from datetime import datetime
|
|
718
|
+
|
|
719
|
+
class FrameStats:
|
|
720
|
+
def __init__(self):
|
|
721
|
+
self.frame_count = 0
|
|
722
|
+
self.start_time = time.time()
|
|
723
|
+
|
|
724
|
+
def update(self):
|
|
725
|
+
self.frame_count += 1
|
|
726
|
+
if self.frame_count % 100 == 0:
|
|
727
|
+
elapsed = time.time() - self.start_time
|
|
728
|
+
fps = self.frame_count / elapsed
|
|
729
|
+
logger.info(f"Processed {self.frame_count} frames, {fps:.1f} FPS")
|
|
730
|
+
|
|
731
|
+
stats = FrameStats()
|
|
732
|
+
|
|
733
|
+
def process_frame(frame):
|
|
734
|
+
stats.update()
|
|
735
|
+
# Your processing logic
|
|
736
|
+
```
|
|
737
|
+
|
|
738
|
+
### Docker Best Practices
|
|
739
|
+
|
|
740
|
+
1. **Use Multi-stage Builds**
|
|
741
|
+
```dockerfile
|
|
742
|
+
FROM python:3.12-slim as builder
|
|
743
|
+
# Build dependencies
|
|
744
|
+
|
|
745
|
+
FROM python:3.12-slim
|
|
746
|
+
# Copy only runtime artifacts
|
|
747
|
+
```
|
|
748
|
+
|
|
749
|
+
2. **Minimize Image Size**
|
|
750
|
+
- Use slim base images
|
|
751
|
+
- Remove build tools in final stage
|
|
752
|
+
- Clean apt cache: `rm -rf /var/lib/apt/lists/*`
|
|
753
|
+
|
|
754
|
+
3. **Health Checks**
|
|
755
|
+
```dockerfile
|
|
756
|
+
HEALTHCHECK --interval=30s --timeout=3s \
|
|
757
|
+
CMD pgrep -f my_app.py || exit 1
|
|
758
|
+
```
|
|
759
|
+
|
|
760
|
+
4. **Resource Limits** (in RocketWelder docker-compose or deployment)
|
|
761
|
+
```yaml
|
|
762
|
+
deploy:
|
|
763
|
+
resources:
|
|
764
|
+
limits:
|
|
765
|
+
cpus: '2.0'
|
|
766
|
+
memory: 2G
|
|
767
|
+
```
|
|
768
|
+
|
|
769
|
+
## Examples
|
|
770
|
+
|
|
771
|
+
The `examples/` directory contains complete working examples:
|
|
772
|
+
|
|
773
|
+
- **python/simple_client.py** - Minimal timestamp overlay
|
|
774
|
+
- **python/integration_client.py** - Testing with --exit-after flag
|
|
775
|
+
- **python/advanced_client.py** - Full-featured with UI controls
|
|
776
|
+
- **csharp/SimpleClient/** - Complete C# example with crosshair controls
|
|
777
|
+
- **cpp/simple_client.cpp** - C++ example
|
|
778
|
+
|
|
779
|
+
## Troubleshooting
|
|
780
|
+
|
|
781
|
+
### Container Doesn't Start
|
|
782
|
+
|
|
783
|
+
**Check Docker logs:**
|
|
784
|
+
```bash
|
|
785
|
+
docker ps -a | grep my-ai-app
|
|
786
|
+
docker logs <container-id>
|
|
787
|
+
```
|
|
788
|
+
|
|
789
|
+
**Common issues:**
|
|
790
|
+
- Image not found (check `docker images`)
|
|
791
|
+
- Insecure registry not configured on Neuron
|
|
792
|
+
|
|
793
|
+
### Cannot Pull from Laptop Registry
|
|
794
|
+
|
|
795
|
+
```bash
|
|
796
|
+
# On Neuron - test connectivity
|
|
797
|
+
ping laptop-ip
|
|
798
|
+
|
|
799
|
+
# Test registry access
|
|
800
|
+
curl http://laptop-ip:5000/v2/_catalog
|
|
801
|
+
|
|
802
|
+
# Check Docker daemon config
|
|
803
|
+
cat /etc/docker/daemon.json
|
|
804
|
+
|
|
805
|
+
# Restart Docker after config change
|
|
806
|
+
sudo systemctl restart docker
|
|
807
|
+
```
|
|
808
|
+
|
|
809
|
+
### SDK Connection Timeout
|
|
810
|
+
|
|
811
|
+
**Check shared memory buffer exists:**
|
|
812
|
+
```bash
|
|
813
|
+
# On Neuron device
|
|
814
|
+
ls -lh /dev/shm/
|
|
815
|
+
|
|
816
|
+
# Should see zerobuffer-* files
|
|
817
|
+
```
|
|
818
|
+
|
|
819
|
+
**Check RocketWelder pipeline status:**
|
|
820
|
+
- Is pipeline running?
|
|
821
|
+
- Is zerosink element configured correctly?
|
|
822
|
+
- Check RocketWelder logs for errors
|
|
823
|
+
|
|
824
|
+
### Low Frame Rate / Performance
|
|
825
|
+
|
|
826
|
+
1. **Check CPU usage:** `htop` or `docker stats`
|
|
827
|
+
2. **Reduce AI model complexity** or process every Nth frame
|
|
828
|
+
3. **Profile your code** to find bottlenecks
|
|
829
|
+
4. **Use GPU acceleration** if available (NVIDIA runtime)
|
|
830
|
+
|
|
831
|
+
## Support
|
|
832
|
+
|
|
833
|
+
- **Issues**: [GitHub Issues](https://github.com/modelingevolution/rocket-welder-sdk/issues)
|
|
834
|
+
- **Discussions**: [GitHub Discussions](https://github.com/modelingevolution/rocket-welder-sdk/discussions)
|
|
835
|
+
- **Documentation**: [https://docs.rocket-welder.io](https://docs.rocket-welder.io)
|
|
836
|
+
|
|
837
|
+
## License
|
|
838
|
+
|
|
839
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
840
|
+
|
|
841
|
+
## Acknowledgments
|
|
842
|
+
|
|
843
|
+
- GStreamer Project for the multimedia framework
|
|
844
|
+
- ZeroBuffer contributors for the zero-copy buffer implementation
|
|
845
|
+
- OpenCV community for computer vision tools
|