img-phy-sim 0.6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of img-phy-sim might be problematic. Click here for more details.
- img_phy_sim-0.6/PKG-INFO +668 -0
- img_phy_sim-0.6/README.md +626 -0
- img_phy_sim-0.6/img_phy_sim/__init__.py +102 -0
- img_phy_sim-0.6/img_phy_sim/copy_22_01ray_tracing.py +2399 -0
- img_phy_sim-0.6/img_phy_sim/data.py +417 -0
- img_phy_sim-0.6/img_phy_sim/eval.py +146 -0
- img_phy_sim-0.6/img_phy_sim/img.py +652 -0
- img_phy_sim-0.6/img_phy_sim/ism.py +948 -0
- img_phy_sim-0.6/img_phy_sim/ism_slow.py +498 -0
- img_phy_sim-0.6/img_phy_sim/math.py +185 -0
- img_phy_sim-0.6/img_phy_sim/ray_tracing.py +1640 -0
- img_phy_sim-0.6/img_phy_sim.egg-info/PKG-INFO +668 -0
- img_phy_sim-0.6/img_phy_sim.egg-info/SOURCES.txt +17 -0
- img_phy_sim-0.6/img_phy_sim.egg-info/dependency_links.txt +1 -0
- img_phy_sim-0.6/img_phy_sim.egg-info/requires.txt +12 -0
- img_phy_sim-0.6/img_phy_sim.egg-info/top_level.txt +1 -0
- img_phy_sim-0.6/pyproject.toml +4 -0
- img_phy_sim-0.6/setup.cfg +4 -0
- img_phy_sim-0.6/setup.py +82 -0
img_phy_sim-0.6/PKG-INFO
ADDED
|
@@ -0,0 +1,668 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: img-phy-sim
|
|
3
|
+
Version: 0.6
|
|
4
|
+
Summary: Physical Simulations on Images.
|
|
5
|
+
Home-page: https://github.com/M-106/Image-Physics-Simulation
|
|
6
|
+
Download-URL: https://github.com/M-106/Image-Physics-Simulation/archive/v_03.tar.gz
|
|
7
|
+
Author: Tobia Ippolito
|
|
8
|
+
Project-URL: Documentation, https://M-106.github.io/Image-Physics-Simulation/img_phy_sim
|
|
9
|
+
Project-URL: Source, https://github.com/M-106/Image-Physics-Simulation
|
|
10
|
+
Keywords: Simulation,Computer-Vision,Physgen
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Classifier: Operating System :: OS Independent
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
Requires-Dist: numpy
|
|
21
|
+
Requires-Dist: opencv-python
|
|
22
|
+
Requires-Dist: matplotlib
|
|
23
|
+
Requires-Dist: scikit-image
|
|
24
|
+
Requires-Dist: joblib
|
|
25
|
+
Requires-Dist: shapely
|
|
26
|
+
Provides-Extra: full
|
|
27
|
+
Requires-Dist: torch; extra == "full"
|
|
28
|
+
Requires-Dist: torchvision; extra == "full"
|
|
29
|
+
Requires-Dist: datasets==3.6.0; extra == "full"
|
|
30
|
+
Requires-Dist: prime_printer; extra == "full"
|
|
31
|
+
Dynamic: author
|
|
32
|
+
Dynamic: classifier
|
|
33
|
+
Dynamic: description
|
|
34
|
+
Dynamic: description-content-type
|
|
35
|
+
Dynamic: download-url
|
|
36
|
+
Dynamic: home-page
|
|
37
|
+
Dynamic: keywords
|
|
38
|
+
Dynamic: project-url
|
|
39
|
+
Dynamic: provides-extra
|
|
40
|
+
Dynamic: requires-dist
|
|
41
|
+
Dynamic: summary
|
|
42
|
+
|
|
43
|
+
# **I**mage-**P**hysics-**S**imulation
|
|
44
|
+
|
|
45
|
+
This is a library for 2D Ray-Tracing on an image. For example the Physgen Dataset, [see here](https://huggingface.co/datasets/mspitzna/physicsgen).
|
|
46
|
+
|
|
47
|
+
Contents:
|
|
48
|
+
- [Installation](#installation)
|
|
49
|
+
- [Download Example Data](#download-example-data)
|
|
50
|
+
- [Pre-Compute Rays](#pre-compute-rays)
|
|
51
|
+
- [Library Overview](#library-overview)
|
|
52
|
+
- [Raytracing Computation](#raytracing-computation)
|
|
53
|
+
- [Raytracing Tutorial](#raytracing-tutorial)
|
|
54
|
+
- [Performance Test](#performance-test)
|
|
55
|
+
- [Ray-Tracing Formats](#ray-tracing-formats)
|
|
56
|
+
|
|
57
|
+
[> Documentation <](https://M-106.github.io/Image-Physics-Simulation/img_phy_sim.html)
|
|
58
|
+
|
|
59
|
+
<img src="https://github.com/M-106/Image-Physics-Simulation/raw/main/img_phy_sim/raytracing_example.png" width="46%"></img>
|
|
60
|
+
<img src="./img_phy_sim/ism_example.png" width="46%"></img>
|
|
61
|
+
|
|
62
|
+
> Ray-Beams and ISM
|
|
63
|
+
|
|
64
|
+
<br><br>
|
|
65
|
+
|
|
66
|
+
### Installation
|
|
67
|
+
|
|
68
|
+
This repo only need some basic libraries:
|
|
69
|
+
- `numpy`
|
|
70
|
+
- `matplotlib`
|
|
71
|
+
- `opencv-python`
|
|
72
|
+
- `scikit-image`
|
|
73
|
+
- `joblib`
|
|
74
|
+
|
|
75
|
+
If you want to use the `data` module then this package needs also:
|
|
76
|
+
- `torch`
|
|
77
|
+
- `torchvision`
|
|
78
|
+
- `datasets`
|
|
79
|
+
- `prime_printer`
|
|
80
|
+
|
|
81
|
+
You can download / clone this repo and run the example notebook via following Python/Anaconda setup:
|
|
82
|
+
```bash
|
|
83
|
+
conda create -n img-phy-sim python=3.13 pip -y
|
|
84
|
+
conda activate img-phy-sim
|
|
85
|
+
pip install numpy matplotlib opencv-python ipython jupyter shapely prime_printer datasets==3.6.0 scikit-image joblib shapely
|
|
86
|
+
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
You can also use this repo via [Python Package Index (PyPI)](https://pypi.org/) as a package:
|
|
90
|
+
```bash
|
|
91
|
+
pip install img-phy-sim
|
|
92
|
+
# or for using `data` module:
|
|
93
|
+
pip install img-phy-sim[full]
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Here the instructions to use the package version of `ips` and an anconda setup:
|
|
97
|
+
```bash
|
|
98
|
+
conda create -n img-phy-sim python=3.13 pip -y
|
|
99
|
+
conda activate img-phy-sim
|
|
100
|
+
pip install img-phy-sim
|
|
101
|
+
# or for using `data` module:
|
|
102
|
+
pip install img-phy-sim[full]
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
To run the example code you also need (this is included in `img-phy-sim[full]`):
|
|
106
|
+
```bash
|
|
107
|
+
pip install prime_printer datasets==3.6.0
|
|
108
|
+
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
<br><br>
|
|
112
|
+
|
|
113
|
+
### Download Example Data
|
|
114
|
+
|
|
115
|
+
You can download Physgen data if wanted via the `data.py` using following commands:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
conda activate img-phy-sim
|
|
119
|
+
cd "D:\Informatik\Projekte\Image-Physics-Simulation\img_phy_sim" && D:
|
|
120
|
+
python data.py --output_real_path ./datasets/physgen_train_raw/real --output_osm_path ./datasets/physgen_train_raw/osm --variation sound_reflection --input_type osm --output_type standard --data_mode train
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
```bash
|
|
124
|
+
python data.py --output_real_path ./datasets/physgen_test_raw/real --output_osm_path ./datasets/physgen_test_raw/osm --variation sound_reflection --input_type osm --output_type standard --data_mode test
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
python data.py --output_real_path ./datasets/physgen_val_raw/real --output_osm_path ./datasets/physgen_val_raw/osm --variation sound_reflection --input_type osm --output_type standard --data_mode validation
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
<br><br>
|
|
132
|
+
|
|
133
|
+
### Pre-Compute Rays
|
|
134
|
+
|
|
135
|
+
We can use the saving/loading functionality of the `ips`-framework to reduce the computational cost during training a deep learning model.
|
|
136
|
+
|
|
137
|
+
The [ray_tracing_saver.py](./example/ray_tracing_saver.py) is an example of how you can pre-compute all your rays.
|
|
138
|
+
|
|
139
|
+
During training you can load the rays like this (which will try to load pre-computed but still can compute during runtime.):
|
|
140
|
+
```python
|
|
141
|
+
class PhysGenDataset(Dataset):
|
|
142
|
+
|
|
143
|
+
# def __init__(self, ...):
|
|
144
|
+
# ...
|
|
145
|
+
|
|
146
|
+
# def __len__(self):
|
|
147
|
+
# ..
|
|
148
|
+
|
|
149
|
+
def __getitem__(self, idx):
|
|
150
|
+
sample = self.dataset[idx]
|
|
151
|
+
|
|
152
|
+
if self.input_type == "base_simulation":
|
|
153
|
+
input_img = self.basesimulation_dataset[idx]["soundmap"]
|
|
154
|
+
else:
|
|
155
|
+
input_img = sample["osm"] # PIL Image
|
|
156
|
+
target_img = sample["soundmap"] # PIL Image
|
|
157
|
+
|
|
158
|
+
input_img = self.transform(input_img)
|
|
159
|
+
target_img = self.transform(target_img)
|
|
160
|
+
|
|
161
|
+
# Fix real image size 512x512 > 256x256
|
|
162
|
+
input_img = F.interpolate(input_img.unsqueeze(0), size=(256, 256), mode='bilinear', align_corners=False)
|
|
163
|
+
input_img = input_img.squeeze(0)
|
|
164
|
+
|
|
165
|
+
|
|
166
|
+
# reflexions
|
|
167
|
+
if self.reflexion_channels:
|
|
168
|
+
height, width = np.squeeze(input_img.cpu().numpy(), axis=0).shape
|
|
169
|
+
ray_path = os.path.join("./rays", self.mode, str(self.reflexion_steps), f"rays_[{str(idx)}].txt")
|
|
170
|
+
if os.path.exists(ray_path):
|
|
171
|
+
rays = ips.ray_tracing.open(path=ray_path)
|
|
172
|
+
else:
|
|
173
|
+
rays = ips.ray_tracing.trace_beams(rel_position=(0.5, 0.5),
|
|
174
|
+
img_src=np.squeeze(input_img.cpu().numpy(), axis=0),
|
|
175
|
+
directions_in_degree=ips.math.get_linear_degree_range(step_size=(self.reflexion_steps/360)*100),
|
|
176
|
+
wall_values=[0],
|
|
177
|
+
wall_thickness=0,
|
|
178
|
+
img_border_also_collide=False,
|
|
179
|
+
reflexion_order=3,
|
|
180
|
+
should_scale_rays=True,
|
|
181
|
+
should_scale_img=False)
|
|
182
|
+
ray_img = ips.ray_tracing.draw_rays(rays,
|
|
183
|
+
detail_draw=False,
|
|
184
|
+
output_format='channels' if self.reflexions_as_channels else 'single_image',
|
|
185
|
+
img_background=None,
|
|
186
|
+
ray_value=[50, 100, 255],
|
|
187
|
+
ray_thickness=1,
|
|
188
|
+
img_shape=(height, width),
|
|
189
|
+
should_scale_rays_to_image=True,
|
|
190
|
+
show_only_reflections=True)
|
|
191
|
+
# (256, 256)
|
|
192
|
+
ray_img = self.transform(ray_img)
|
|
193
|
+
ray_img = ray_img.float()
|
|
194
|
+
if ray_img.ndim == 2:
|
|
195
|
+
ray_img = ray_img.unsqueeze(0) # (1, H, W)
|
|
196
|
+
|
|
197
|
+
# Merging with input image
|
|
198
|
+
if ray_img.shape[1:] == input_img.shape[1:]:
|
|
199
|
+
input_img = torch.cat((input_img, ray_img), dim=0)
|
|
200
|
+
else:
|
|
201
|
+
raise ValueError(f"Ray image shape {ray_img.shape} does not match input image shape {input_img.shape}.")
|
|
202
|
+
|
|
203
|
+
return input_img, target_img, idx
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
|
|
207
|
+
<br><br>
|
|
208
|
+
|
|
209
|
+
### Library Overview
|
|
210
|
+
|
|
211
|
+
- Img-Phy-Sim (ips)
|
|
212
|
+
- `ray_tracing`
|
|
213
|
+
- `get_wall_map`: extracting a wall-map from an image
|
|
214
|
+
- `trace-beam`: calculates one ray
|
|
215
|
+
- `trace_beams`: load an image, extract the wall-map and trace multiple beams
|
|
216
|
+
- `draw_rays`: draw/export the rays as/inside an image
|
|
217
|
+
- `print_rays_info`: get some interesting informations about your rays
|
|
218
|
+
- `save`: save your rays as txt file
|
|
219
|
+
- `open`: load your saved rays txt file
|
|
220
|
+
- `get_linear_degree_range`: get a range for your beam-directions -> example beams between 0-360 with stepsize 10
|
|
221
|
+
- `merge_rays`: merge 2 rays to one 'object'
|
|
222
|
+
- `ism`
|
|
223
|
+
- `reflect_point_across_infinite_line`: Reflects a point across the infinite line defined by two endpoints.
|
|
224
|
+
- `paths_to_rays`: Converts polyline paths into your ray/segment representation, optionally normalizing points to ([0,1]) image space.
|
|
225
|
+
- `reflection_map_to_img`: Normalizes a float reflection/energy map to a uint8 visualization image in ([0,255]).
|
|
226
|
+
- `Segment`: Immutable dataclass representing a 2D line segment with convenience access to its endpoints.
|
|
227
|
+
- `_seg_seg_intersection`: Computes the unique intersection point of two finite 2D segments, returning None for parallel/colinear/no-hit cases.
|
|
228
|
+
- `_bresenham_points`: Generates all integer grid points along a line between two pixels using Bresenham’s algorithm.
|
|
229
|
+
- `is_visible_raster`: Tests line-of-sight between two points by checking whether Bresenham-sampled pixels hit an occlusion raster.
|
|
230
|
+
- `build_wall_mask`: Builds a binary 0/255 wall mask from an input image using explicit wall labels or mask-like heuristics.
|
|
231
|
+
- `get_wall_segments_from_mask`: Extracts wall boundary contours from a binary mask and converts them into geometric Segment primitives.
|
|
232
|
+
- `build_occlusion_from_wallmask`: Produces a binary occlusion raster (optionally dilated) used for fast visibility checks.
|
|
233
|
+
- `enumerate_wall_sequences_indices`: Enumerates all reflection sequences (as wall-index tuples) up to a maximum reflection order.
|
|
234
|
+
- `precompute_image_sources`: Computes image-source positions for each reflection sequence by iteratively mirroring the source across the corresponding walls.
|
|
235
|
+
- `build_path_for_sequence`: Reconstructs the specular reflection polyline for a given wall sequence by backtracking virtual receivers and segment intersections.
|
|
236
|
+
- `path_energy`: Computes a simple physically-inspired path contribution based on total path length and per-reflection losses.
|
|
237
|
+
- `check_path_visibility_raster`: Verifies that every segment of a candidate path is unobstructed using raster line-of-sight tests.
|
|
238
|
+
- `compute_reflection_map`: Evaluates all valid ISM paths from one source to a receiver grid and accumulates either path counts or energy (optionally dB).
|
|
239
|
+
- `img`
|
|
240
|
+
- `open`: load an image via Open-CV
|
|
241
|
+
- `save`: save an image
|
|
242
|
+
- `imshow`: show an single image (without much features)
|
|
243
|
+
- `advanced_imshow`: show multiple images with many options
|
|
244
|
+
- `show_image_with_line_and_profile`: show an image with a red line + the values of the image on this line
|
|
245
|
+
- `plot_image_with_values`: plot an image with it's value plotted and averaged to see your image in values
|
|
246
|
+
- `math`
|
|
247
|
+
- `get_linear_degree_range`: generate evenly spaced degrees within a range
|
|
248
|
+
- `degree_to_vector`: convert a degree angle to a 2D unit vector
|
|
249
|
+
- `vector_to_degree`: convert a 2D vector into its corresponding degree
|
|
250
|
+
- `normalize_point`: Normalize a 2D point to [0, 1] range
|
|
251
|
+
- `denormalize_point`: Denormalize a 2D point to pixel coordinates
|
|
252
|
+
- `numpy_info`: Get statistics about an numpy array
|
|
253
|
+
- `eval`
|
|
254
|
+
- `calc_metrices`: calculate F1, Recall and Precision between rays (or optinal an image) and an image
|
|
255
|
+
- `data`
|
|
256
|
+
- `PhysGenDataset()`: PyTorch dataset wrapper for PhysGen with flexible input/output configuration
|
|
257
|
+
- `resize_tensor_to_divisible_by_14`: resize tensors so height and width are divisible by 14
|
|
258
|
+
- `get_dataloader`: create a PyTorch DataLoader for the PhysGen dataset
|
|
259
|
+
- `get_image`: retrieve a single dataset sample (optionally as NumPy arrays)
|
|
260
|
+
- `save_dataset`: export PhysGen inputs and targets as PNG images to disk
|
|
261
|
+
|
|
262
|
+
|
|
263
|
+
That are not all functions but the ones which should be most useful. Check out the documentation for all functions.
|
|
264
|
+
|
|
265
|
+
<br><br>
|
|
266
|
+
|
|
267
|
+
### Raytracing Computation
|
|
268
|
+
|
|
269
|
+
1. A map is created which contains the vector of all "walls" (collision objects) in the image as a single degree value. This is done using the `ips.ray_tracing.get_wall_map`-function, which internally uses the canny-edge detection algorithm from OpenCV. The whole process is masked to get the edges for the given wall-values. In order to put the degree/direction values into the wall-map, we following the `Bresenham's line algorithm` to go from one point pixel-wise to the end-point of the edge and on the way we put the value of thes epixels and the pixels around, depending of the used *thickness*.
|
|
270
|
+
2. The `ips.ray_tracing.trace_beam`-function goes from pixel to pixel and checks if the pixel-value is a "wall" (collision-object). If yes, a new beam is added using the wall-map to calculate the right direction-vector. To know, which pixel is the next pixel, we use the average vector between the direction vector (the target direction which is available since the beginning of a beam calculation) and a vector pointing to the closest point of the optimal way/vector. Why so complicated? Because we move per pixel and pixel does only know 8 directions, but we want to move more smooth in more directions. The direction vector from the beginning is helpful but will lead every update to the same pixel, because it does not use the information of the current position. THis is bad because so we will move very wrong if going only after the goal vector with our limited step-directions. And so we add the described "to the perfect line"-vector into the calculation weighted a bit less then the real direction (0.5).<br>The "perfect line" is calculated very easy. At the beginning of a beam we move in small steps (like 0.01) forward towards the target direction, ignoring pixel-wise approach, until we hit with x or y a boundary of the image.
|
|
271
|
+
|
|
272
|
+
> You may have noticed that we use a own and Bresenham's line algorithm for moving in the pixel-space, which have no further reason and we may only use one of them in future.
|
|
273
|
+
|
|
274
|
+
|
|
275
|
+
|
|
276
|
+
<br><br>
|
|
277
|
+
|
|
278
|
+
### Raytracing Tutorial
|
|
279
|
+
|
|
280
|
+
[See also the example notebook 👀](./example/physgen.ipynb)
|
|
281
|
+
|
|
282
|
+
In general you need to do:
|
|
283
|
+
1. **Load your Image** -> using `ips.img.open`
|
|
284
|
+
2. **Calculate the Wall-Map** -> using `ips.ray_tracing.get_wall_map`
|
|
285
|
+
3. **Calculate the Beams** -> using `ips.img.open`
|
|
286
|
+
4. **Draw (Export) the Beams** -> using `ips.img.open`
|
|
287
|
+
|
|
288
|
+
Using this lib, this is reduced to:
|
|
289
|
+
1. **Calculate the Beams** (including Wall-Map and loading your Image) -> using `ips.img.open`
|
|
290
|
+
2. **Draw (Export) the Beams** -> using `ips.img.open`
|
|
291
|
+
|
|
292
|
+
See this example:
|
|
293
|
+
|
|
294
|
+
```python
|
|
295
|
+
rays = ips.ray_tracing.trace_beams(rel_position=[0.5, 0.5],
|
|
296
|
+
img_src=img_src,
|
|
297
|
+
directions_in_degree=ips.ray_tracing.get_linear_degree_range(step_size=10),
|
|
298
|
+
wall_values=None,
|
|
299
|
+
wall_thickness=1,
|
|
300
|
+
img_border_also_collide=False,
|
|
301
|
+
reflexion_order=1,
|
|
302
|
+
should_scale_rays=True,
|
|
303
|
+
should_scale_img=True)
|
|
304
|
+
ips.ray_tracing.print_rays_info(rays)
|
|
305
|
+
|
|
306
|
+
ray_img = ips.ray_tracing.draw_rays(rays, detail_draw=False,
|
|
307
|
+
output_format="single_image",
|
|
308
|
+
img_background=img, ray_value=2, ray_thickness=1,
|
|
309
|
+
img_shape=(256, 256), dtype=float, standard_value=0,
|
|
310
|
+
should_scale_rays_to_image=True, original_max_width=None, original_max_height=None)
|
|
311
|
+
ips.img.imshow(ray_img, size=5)
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
**Rays structure:**<br>
|
|
315
|
+
The Raytracing result is a list of rays, where every ray can consist of multiple beams which comes from reflections. One beam is a list of multiple points, where the first point is the start point and the last element is the end point.
|
|
316
|
+
|
|
317
|
+
Example:
|
|
318
|
+
```text
|
|
319
|
+
[
|
|
320
|
+
[[start-point, ..., end-point], [start-point, ..., end-point]], # one reflection
|
|
321
|
+
[[start-point, ..., end-point]], # no reflection
|
|
322
|
+
# ...
|
|
323
|
+
]
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
<br><br>
|
|
327
|
+
|
|
328
|
+
Now let's go step by step how to apply ray-tracing to your image.
|
|
329
|
+
|
|
330
|
+
<br><br>
|
|
331
|
+
|
|
332
|
+
**1. Analyzing your image**<br>
|
|
333
|
+
First it is important you know the values of your image and which values should consider an object for collision. For that use the tools given with this package:
|
|
334
|
+
```python
|
|
335
|
+
img = ips.img.open(src=img_src, should_scale=False, should_print=True)
|
|
336
|
+
ips.img.imshow(img, size=4, axis_off=False)
|
|
337
|
+
ips.img.show_image_with_line_and_profile(imgs=[img], axis='row', index=None, titles=None, figsize=(10, 8));
|
|
338
|
+
```
|
|
339
|
+
If you see super small values then you might tried to scale your already scaled image.
|
|
340
|
+
|
|
341
|
+
<br><br>
|
|
342
|
+
|
|
343
|
+
**2. Get the Collision Ready**<br>
|
|
344
|
+
Next it is helpful to check if your collision is ready by running the wall-map by yourself (later you will use the wrapper, but here you can find the right params).
|
|
345
|
+
|
|
346
|
+
```python
|
|
347
|
+
wall_map = ips.ray_tracing.get_wall_map(img, wall_values=None, thickness=0)
|
|
348
|
+
ips.img.imshow(wall_map, size=4, axis_off=False)
|
|
349
|
+
ips.img.show_image_with_line_and_profile(imgs=[wall_map], axis='row', index=None, titles=None, figsize=(10, 8));
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
You can give `wall_values` a list of values, which you want to collide with.
|
|
353
|
+
|
|
354
|
+
<br><br>
|
|
355
|
+
|
|
356
|
+
**3. Let's start tracing the rays!**<br>
|
|
357
|
+
Now everything should be ready for tracing the rays. The following code include loading your image and creating the wall-map.
|
|
358
|
+
|
|
359
|
+
```python
|
|
360
|
+
rays = ips.ray_tracing.trace_beams(rel_position=[0.5, 0.5],
|
|
361
|
+
img_src="./my_image.png",
|
|
362
|
+
directions_in_degree=ips.ray_tracing.get_linear_degree_range(start=0, stop=360, step_size=10),
|
|
363
|
+
wall_values=[0.0],
|
|
364
|
+
wall_thickness=0,
|
|
365
|
+
img_border_also_collide=False,
|
|
366
|
+
reflexion_order=1,
|
|
367
|
+
should_scale_rays=True,
|
|
368
|
+
should_scale_img=True)
|
|
369
|
+
ips.ray_tracing.print_rays_info(rays)
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
Following features are included:
|
|
373
|
+
- Setting custom startposition for raytracing
|
|
374
|
+
- Adding custom beam shooting positions in degree (where 0° is the east/right of the image and 90° is south/bottom and so on)
|
|
375
|
+
- Setting reflexion order (how many maxium reflexions should be calculated)
|
|
376
|
+
- Setting if the border of the image should be reflective or not
|
|
377
|
+
- And setting if the input image should be scaled and if the rays itself should be scaled
|
|
378
|
+
- You can also set the "wall" object values, which should get detected as objects with collision. If set to None, the programm will find all clear edges.
|
|
379
|
+
- Setting if the rays should be in 0.0-1.0 range or the real image range
|
|
380
|
+
- Whether to scale the image or not
|
|
381
|
+
|
|
382
|
+
<br><br>
|
|
383
|
+
|
|
384
|
+
**4. Export your rays**<br>
|
|
385
|
+
At the end you might want to use your rays in an image. We provide you with a draw/export function with many flexibility.
|
|
386
|
+
|
|
387
|
+
Features are:
|
|
388
|
+
- Custom value of ray-traces
|
|
389
|
+
- Thickness of ray-traces
|
|
390
|
+
- Drawing on empty image or an existing image
|
|
391
|
+
- Given image-shape, dtype and fill-value (standard-value)
|
|
392
|
+
- Scaling rays to the given image
|
|
393
|
+
- Different Format Types
|
|
394
|
+
- One Image -> `single_image`
|
|
395
|
+
- Multiple Images (each ray on one image) -> `multiple_images`
|
|
396
|
+
- One Image and each channel is one ray -> `channels`
|
|
397
|
+
- Showing only the reflexions
|
|
398
|
+
- Give different values for different reflexion orders
|
|
399
|
+
```
|
|
400
|
+
ray_img = ips.ray_tracing.draw_rays(rays, detail_draw=False,
|
|
401
|
+
output_format="single_image",
|
|
402
|
+
img_background=None, ray_value=2, ray_thickness=1,
|
|
403
|
+
img_shape=(256, 256), dtype=float, standard_value=0,
|
|
404
|
+
should_scale_rays_to_image=False, original_max_width=None, original_max_height=None,
|
|
405
|
+
show_only_reflections=False)
|
|
406
|
+
ips.img.imshow(ray_img, size=4)
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
<br><br>
|
|
410
|
+
|
|
411
|
+
I hope this little tutorial could be helpful. Good luck with your project <3
|
|
412
|
+
|
|
413
|
+
<br><br>
|
|
414
|
+
|
|
415
|
+
### Performance Test
|
|
416
|
+
|
|
417
|
+
<br>
|
|
418
|
+
|
|
419
|
+
[> See the notebook/code <](./example/physgen_performance.ipynb) [(or parallel notebook)](./example/physgen_parallel_performance.ipynb)
|
|
420
|
+
|
|
421
|
+
<br><br>
|
|
422
|
+
|
|
423
|
+
Comparison no parallel vs parallel computing:
|
|
424
|
+
- Parallel Mean Experiment time: 3.48 seconds (mean first experiment: 8.85 seconds)
|
|
425
|
+
- Standard Mean Experiment time: 4.53 seconds (mean first experiment: 16.00 seconds)
|
|
426
|
+
|
|
427
|
+
<br><br>
|
|
428
|
+
|
|
429
|
+
Parameter Experiments:
|
|
430
|
+
|
|
431
|
+
Executed with 50 random images.
|
|
432
|
+
|
|
433
|
+
Standard Settings were:
|
|
434
|
+
- test_amount=50
|
|
435
|
+
- step_size=10
|
|
436
|
+
- reflexion_order=3
|
|
437
|
+
- ray_scaling=True
|
|
438
|
+
- detail_draw=False
|
|
439
|
+
- output_format="channels"
|
|
440
|
+
|
|
441
|
+
<br>
|
|
442
|
+
|
|
443
|
+
Investigated Factors:
|
|
444
|
+
- `Ray Amount` (step_size)
|
|
445
|
+
- `Scaling of Rays` (ray_scaling)
|
|
446
|
+
- `Reflexion Order` (reflexion_order)
|
|
447
|
+
- `Detail Drawing of Rays` (detail_draw)
|
|
448
|
+
|
|
449
|
+
|
|
450
|
+
<br><br>
|
|
451
|
+
|
|
452
|
+
Experiment 1: Ray Amount
|
|
453
|
+
|
|
454
|
+
```text
|
|
455
|
+
Number of experiments: 4
|
|
456
|
+
|
|
457
|
+
avg_time : mean=16.0064, std=23.6257, min=0.5744, max=56.7918, rel_change=351.22%
|
|
458
|
+
median_time : mean=15.4329, std=22.7908, min=0.5546, max=54.7799, rel_change=351.36%
|
|
459
|
+
var_time : mean=17.1176, std=29.2774, min=0.0076, max=67.8259, rel_change=396.19%
|
|
460
|
+
avg_compute_time : mean=15.1070, std=22.2322, min=0.5529, max=53.4815, rel_change=350.36%
|
|
461
|
+
median_compute_time : mean=14.5613, std=21.4404, min=0.5346, max=51.5723, rel_change=350.50%
|
|
462
|
+
var_compute_time : mean=14.7739, std=25.2362, min=0.0074, max=58.4825, rel_change=395.80%
|
|
463
|
+
avg_draw_time : mean=0.8994, std=1.3940, min=0.0215, max=3.3103, rel_change=365.67%
|
|
464
|
+
median_draw_time : mean=0.8373, std=1.2914, min=0.0207, max=3.0705, rel_change=364.22%
|
|
465
|
+
var_draw_time : mean=0.1525, std=0.2639, min=0.0000, max=0.6096, rel_change=399.67%
|
|
466
|
+
|
|
467
|
+
Overall trend in avg_time: increasing (1.7306e+01 change per experiment)
|
|
468
|
+
Conclusion: Performance changes significantly across experiments.
|
|
469
|
+
```
|
|
470
|
+
|
|
471
|
+
|
|
472
|
+
<br><br>
|
|
473
|
+
|
|
474
|
+
Experiment 2: Ray Scaling
|
|
475
|
+
|
|
476
|
+
```text
|
|
477
|
+
Number of experiments: 2
|
|
478
|
+
|
|
479
|
+
avg_time : mean=0.5744, std=0.0078, min=0.5666, max=0.5823, rel_change=2.73%
|
|
480
|
+
median_time : mean=0.5558, std=0.0080, min=0.5478, max=0.5637, rel_change=2.86%
|
|
481
|
+
var_time : mean=0.0078, std=0.0000, min=0.0078, max=0.0078, rel_change=0.16%
|
|
482
|
+
avg_compute_time : mean=0.5581, std=0.0024, min=0.5557, max=0.5605, rel_change=0.86%
|
|
483
|
+
median_compute_time : mean=0.5398, std=0.0026, min=0.5372, max=0.5423, rel_change=0.95%
|
|
484
|
+
var_compute_time : mean=0.0075, std=0.0000, min=0.0075, max=0.0075, rel_change=0.13%
|
|
485
|
+
avg_draw_time : mean=0.0163, std=0.0055, min=0.0108, max=0.0218, rel_change=66.90%
|
|
486
|
+
median_draw_time : mean=0.0160, std=0.0053, min=0.0106, max=0.0213, rel_change=66.86%
|
|
487
|
+
var_draw_time : mean=0.0000, std=0.0000, min=0.0000, max=0.0000, rel_change=67.83%
|
|
488
|
+
|
|
489
|
+
Overall trend in avg_time: decreasing (-1.5690e-02 change per experiment)
|
|
490
|
+
Conclusion: Performance changes slightly across experiments.
|
|
491
|
+
```
|
|
492
|
+
|
|
493
|
+
<br><br>
|
|
494
|
+
|
|
495
|
+
Experiment 3: Reflexion Order
|
|
496
|
+
|
|
497
|
+
```text
|
|
498
|
+
Number of experiments: 6
|
|
499
|
+
|
|
500
|
+
avg_time : mean=0.9300, std=0.7354, min=0.3231, max=2.4409, rel_change=227.71%
|
|
501
|
+
median_time : mean=0.8502, std=0.6128, min=0.3253, max=2.0917, rel_change=207.76%
|
|
502
|
+
var_time : mean=0.4635, std=0.9329, min=0.0002, max=2.5421, rel_change=548.45%
|
|
503
|
+
avg_compute_time : mean=0.9014, std=0.7194, min=0.3088, max=2.3798, rel_change=229.74%
|
|
504
|
+
median_compute_time : mean=0.8241, std=0.6002, min=0.3108, max=2.0401, rel_change=209.83%
|
|
505
|
+
var_compute_time : mean=0.4449, std=0.8953, min=0.0002, max=2.4396, rel_change=548.32%
|
|
506
|
+
avg_draw_time : mean=0.0286, std=0.0160, min=0.0143, max=0.0611, rel_change=163.74%
|
|
507
|
+
median_draw_time : mean=0.0266, std=0.0129, min=0.0144, max=0.0521, rel_change=142.21%
|
|
508
|
+
var_draw_time : mean=0.0002, std=0.0004, min=0.0000, max=0.0011, rel_change=550.06%
|
|
509
|
+
|
|
510
|
+
Overall trend in avg_time: increasing (3.7365e-01 change per experiment)
|
|
511
|
+
Conclusion: Performance changes significantly across experiments.
|
|
512
|
+
```
|
|
513
|
+
|
|
514
|
+
<br><br>
|
|
515
|
+
|
|
516
|
+
Experiment 4: Detail Draw
|
|
517
|
+
|
|
518
|
+
```text
|
|
519
|
+
Number of experiments: 2
|
|
520
|
+
|
|
521
|
+
avg_time : mean=0.6282, std=0.0515, min=0.5767, max=0.6796, rel_change=16.39%
|
|
522
|
+
median_time : mean=0.6101, std=0.0507, min=0.5593, max=0.6608, rel_change=16.63%
|
|
523
|
+
var_time : mean=0.0115, std=0.0040, min=0.0074, max=0.0155, rel_change=70.54%
|
|
524
|
+
avg_compute_time : mean=0.5572, std=0.0020, min=0.5552, max=0.5592, rel_change=0.70%
|
|
525
|
+
median_compute_time : mean=0.5430, std=0.0047, min=0.5383, max=0.5478, rel_change=1.75%
|
|
526
|
+
var_compute_time : mean=0.0074, std=0.0002, min=0.0072, max=0.0076, rel_change=5.32%
|
|
527
|
+
avg_draw_time : mean=0.0709, std=0.0495, min=0.0214, max=0.1204, rel_change=139.55%
|
|
528
|
+
median_draw_time : mean=0.0663, std=0.0454, min=0.0209, max=0.1117, rel_change=137.05%
|
|
529
|
+
var_draw_time : mean=0.0011, std=0.0011, min=0.0000, max=0.0022, rel_change=199.31%
|
|
530
|
+
|
|
531
|
+
Overall trend in avg_time: increasing (1.0292e-01 change per experiment)
|
|
532
|
+
Conclusion: Performance changes slightly across experiments.
|
|
533
|
+
```
|
|
534
|
+
|
|
535
|
+
<br><br>
|
|
536
|
+
|
|
537
|
+
Summary:
|
|
538
|
+
|
|
539
|
+
The Stepsize/amount of rays have the biggest impact on the performance. The other parameters have rather a small impact.
|
|
540
|
+
|
|
541
|
+
|
|
542
|
+
| **Experiment** | **Number of Experiments** | **avg_time (mean ± std)** | **avg_compute_time (mean ± std)** | **avg_draw_time (mean ± std)** | **rel_change (avg_time)** | **Trend** | **Conclusion** |
|
|
543
|
+
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
544
|
+
| **1. Ray Amount** | 4 | 16.01 ± 23.63 s | 15.11 ± 22.23 s | 0.90 ± 1.39 s | **351.22 %** | Increasing (+17.31 s/exp) | Performance changes **significantly** |
|
|
545
|
+
| **2. Ray Scaling** | 2 | 0.57 ± 0.01 s | 0.56 ± 0.00 s | 0.016 ± 0.006 s | **2.73 %** | Decreasing (−0.016 s/exp) | Performance changes **slightly** |
|
|
546
|
+
| **3. Reflection Order** | 6 | 0.93 ± 0.74 s | 0.90 ± 0.72 s | 0.029 ± 0.016 s | **227.71 %** | Increasing (+0.37 s/exp) | Performance changes **significantly** |
|
|
547
|
+
| **4. Detail Draw** | 2 | 0.63 ± 0.05 s | 0.56 ± 0.00 s | 0.071 ± 0.050 s | **16.39 %** | Increasing (+0.10 s/exp) | Performance changes **slightly** |
|
|
548
|
+
|
|
549
|
+
<!--
|
|
550
|
+
<br><br>
|
|
551
|
+
|
|
552
|
+
### Optimization
|
|
553
|
+
|
|
554
|
+
<br>
|
|
555
|
+
|
|
556
|
+
Speed comparison between `standard`, `with joblib` and `joblib + CPython`.
|
|
557
|
+
|
|
558
|
+
FIXME -> table
|
|
559
|
+
|
|
560
|
+
|
|
561
|
+
<br>
|
|
562
|
+
|
|
563
|
+
Cython is Pyhon code which is closer to C. Instead of compiling to Python-Bytecode (`.pyc`), your code will be compiled as C-Extension (`.so`/`.pyd`).
|
|
564
|
+
|
|
565
|
+
There are 3 layers of Cython optimization:
|
|
566
|
+
1. Writing in `.pyx` files not `.py` files -> you can just rename your file<br>
|
|
567
|
+
- +5% to +30% speedup
|
|
568
|
+
2. Set `cdef` for local variables + helper-functions + `cpdef` for API-functions -> add types<br>
|
|
569
|
+
Example:
|
|
570
|
+
```python
|
|
571
|
+
cdef double x0, y0, dx, dy
|
|
572
|
+
cdef int cell_x, cell_y, steps
|
|
573
|
+
```
|
|
574
|
+
- +5x to +50x speedup
|
|
575
|
+
3. Add types everywhere, especially in numpy arrays. <br>
|
|
576
|
+
Example:
|
|
577
|
+
```python
|
|
578
|
+
cimport numpy as cnp
|
|
579
|
+
|
|
580
|
+
def trace(..., cnp.ndarray[double, ndim=2] img):
|
|
581
|
+
cdef double value = img[y, x]
|
|
582
|
+
```
|
|
583
|
+
- +20× to +500×
|
|
584
|
+
|
|
585
|
+
|
|
586
|
+
|
|
587
|
+
| **Optimization Layer** | **Effort** | **Speedup** | **What It Does** |
|
|
588
|
+
|---|---|---|---|
|
|
589
|
+
| **1. `.py` → `.pyx`** | minimal | **+5–30%** | Reduces Python interpreter overhead and applies basic Cython optimizations |
|
|
590
|
+
| **2. `cdef` variables & `cpdef` functions** | medium | **+5×–50×** | Moves loops and math into pure C, eliminating Python object overhead |
|
|
591
|
+
| **3. `cimport numpy` + typed NumPy arrays** | high | **+20×–500×** | Enables direct C‑level memory access with zero Python indexing overhead |
|
|
592
|
+
|
|
593
|
+
|
|
594
|
+
|
|
595
|
+
> Use `pip install cypthon` to install it.
|
|
596
|
+
|
|
597
|
+
In `setup.py` you need following changes:
|
|
598
|
+
```python
|
|
599
|
+
from setuptools import setup, find_packages, Extension
|
|
600
|
+
from Cython.Build import cythonize
|
|
601
|
+
|
|
602
|
+
...
|
|
603
|
+
|
|
604
|
+
ext_1 = Extension(
|
|
605
|
+
name="img_phy_sim.ray_tracing",
|
|
606
|
+
sources=["img_phy_sim/ray_tracing.pyx"],
|
|
607
|
+
include_dirs=[],
|
|
608
|
+
extra_compile_args=["-O3"],
|
|
609
|
+
)
|
|
610
|
+
|
|
611
|
+
ext_2 = Extension(
|
|
612
|
+
name="img_phy_sim.math",
|
|
613
|
+
sources=["img_phy_sim/math.pyx"],
|
|
614
|
+
include_dirs=[],
|
|
615
|
+
extra_compile_args=["-O3"],
|
|
616
|
+
)
|
|
617
|
+
|
|
618
|
+
setup(
|
|
619
|
+
ext_modules=cythonize(
|
|
620
|
+
[ext_1, ext_2],
|
|
621
|
+
compiler_directives={
|
|
622
|
+
"language_level": "3",
|
|
623
|
+
"boundscheck": False,
|
|
624
|
+
"wraparound": False,
|
|
625
|
+
"initializedcheck": False,
|
|
626
|
+
"nonecheck": False,
|
|
627
|
+
"cdivision": True,
|
|
628
|
+
},
|
|
629
|
+
annotate=True,
|
|
630
|
+
),
|
|
631
|
+
...
|
|
632
|
+
)
|
|
633
|
+
```
|
|
634
|
+
|
|
635
|
+
-->
|
|
636
|
+
|
|
637
|
+
<br><br>
|
|
638
|
+
|
|
639
|
+
### Ray-Tracing Formats
|
|
640
|
+
|
|
641
|
+
<br>
|
|
642
|
+
|
|
643
|
+
|
|
644
|
+
**Your current approach (DDA / Pixel Ray Marching)**
|
|
645
|
+
* **Forward integration**: Ray is propagated step by step through a **discrete grid** (pixel/grid).
|
|
646
|
+
* **Collision model**: A "hit" occurs when the ray enters a **wall cell** (quantization).
|
|
647
|
+
* **Reflection**: Occurs **locally at the collision pixel** with a (often quantized) normal/orientation.
|
|
648
|
+
* Result: good for "many rays" / field of view, but **not deterministic with regard to reflection paths** (you need directions/sampling).
|
|
649
|
+
|
|
650
|
+
**Noise modeling style (image source method / geometric acoustics)**
|
|
651
|
+
* **Path construction**: Reflection paths are constructed **deterministically** via **mirror sources** (virtual sources).
|
|
652
|
+
* **Continuous geometry**: works in $\mathbb{R}^2 / \mathbb{R}^3$ with lines/segments/polygons ("infinity-based" in the sense of *continuous space*, not raster).
|
|
653
|
+
* **Validation**: Path is then accepted/rejected via **visibility/occlusion checks**.
|
|
654
|
+
* Result: Delivers **all specular paths up to order N** without angle sampling.
|
|
655
|
+
|
|
656
|
+
Short form:
|
|
657
|
+
|
|
658
|
+
* **Pixel-based + stochastic/directed** (original approach here) vs. **continuous + deterministically constructed** (noise modelling).
|
|
659
|
+
|
|
660
|
+
|
|
661
|
+
<br>
|
|
662
|
+
|
|
663
|
+
How to use which of them in `img-phy-sim`:
|
|
664
|
+
|
|
665
|
+
FIXME
|
|
666
|
+
|
|
667
|
+
|
|
668
|
+
|