DeepGPR 0.0.18__tar.gz → 0.0.20__tar.gz

deepgpr-0.0.20/MANIFEST.in

@@ -0,0 +1,19 @@
+include README.md
+include license
+include pyproject.toml
+
+recursive-include src/DeepGPR *.py *.txt *.cu *.h *.c *.dll *.so *.dylib
+
+prune .agents
+prune .codex
+prune .github
+prune Fig
+prune docs
+prune examples
+prune scripts
+prune tests
+prune website
+
+global-exclude .DS_Store
+global-exclude *.py[cod]
+global-exclude __pycache__

{deepgpr-0.0.18 → deepgpr-0.0.20}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: DeepGPR
-Version: 0.0.18
+Version: 0.0.20
 Summary: PyTorch and CUDA for GPR FWI
 Author-email: Lei Liu <liulei990222@gmail.com>
 License-Expression: MIT
@@ -23,6 +23,8 @@ Dynamic: license-file
 
 # DeepGPR
 
+**Official Website / Documentation:** [https://songc0a.github.io/DeepGPR/](https://songc0a.github.io/DeepGPR/)
+
 DeepGPR provides a wave propagation module for PyTorch, designed for applications such as Ground Penetrating Radar (GPR) imaging and inversion. Its core concepts are derived from Deepwave. You can use it to perform both forward modeling and backpropagation—thereby enabling the simulation of wave propagation to generate synthetic data—as well as for Full Waveform Inversion (FWI). Furthermore, you can integrate this wave propagation functionality into a larger operational pipeline—incorporating various wavelets, loss functions, and other components—to achieve end-to-end forward and reverse propagation, powered by automatic differentiation and our high-performance operators.
 
 
@@ -241,7 +243,13 @@ This section defines the geometric observation system (coordinates) and the exci
 | :--- | :--- | :--- | :--- |
 | **`pmlthick`** | `int` / `list` / `Tensor`| Scalar or list of 6 | Thickness (in grid layers) of the PML (Perfectly Matched Layer) absorbing boundaries.<br>- Integer `p`: All six boundaries have thickness `p` (Z-boundaries are ignored in 2D).<br>- List `[x0, xm, y0, ym, z0, zm]`: Specific thicknesses for the 6 boundaries. |
 | **`model_gradient_sampling_interval`**| `int` | Scalar | Wavefield sampling interval during forward propagation (Default: 1).<br>A larger integer reduces VRAM use for `E_saved` and `R_saved`, but uses an explicitly approximate model gradient. The last incomplete sampling block is weighted by its actual length. |
+| **`save_wavefield_history`** | `bool` | Scalar | Independently controls allocation and native writes of the E/R histories used by adjoint model-gradient backward (Default: `True`). `False` still executes the complete FDTD, CPML, source-injection, and receiver-recording path, returns an empty `E_saved`, and raises a clear error if history-dependent backward is attempted. |
 | **`wavefield_storage_dtype`** | `torch.dtype` / `str` | `float32`, `float16`, or `bfloat16` | Storage format for saved `E_saved` and `R_saved` model-gradient wavefields. FDTD propagation remains float32. `float16` and `bfloat16` halve saved-wavefield memory at the cost of gradient accuracy; `bfloat16` has the safer dynamic range. String aliases such as `"fp16"` and `"bf16"` are accepted. |
+| **`wavefield_conversion_backend`** | `str` | `"auto"`, `"legacy"`, `"native_scalar"`, or `"native_vec2"` | CUDA FP16/BF16 history conversion. The audited default `"auto"` uses NVIDIA scalar intrinsics for CUDA FP16 and the legacy path otherwise. Explicit values retain the correctness/performance A/B paths; vec2 is not the default because it was slower on RTX 4090. |
+| **`wavefield_compression`** | `str` | `"none"`, `"int8"`, or `"zfp"` | `"int8"` enables CUDA-native per-block symmetric INT8 histories with FP32 scales and inline decode in the fused material-gradient kernel. `"none"` is the unchanged default. `"zfp"` is reserved for an optional fused CUDA decoder and is rejected by the current dependency-free build instead of silently materializing a decoded global-memory history. |
+| **`wavefield_compression_block_size`** | sequence / `None` | 2D or 3D spatial block | INT8 block shape; defaults to `(8, 8)` in 2D and `(4, 4, 4)` in 3D. The block volume must be a power of two no larger than 256. Partial boundary blocks are supported. |
+| **`int8_reduction_backend`** | `str` | `"auto"`, `"current"`, `"cub_block"`, or `"warp_shuffle"` | Tile maximum reduction. The audited default `"auto"` selects NVIDIA CUB `BlockReduce` for 64-voxel tiles and preserves the prior shared-memory tree for other valid tile sizes. Explicit values expose the retained A/B implementations. |
+| **`wavefield_compression_rate`** | scalar / `None` | Optional ZFP setting | Reserved for an optional ZFP backend. It is rejected unless that backend is selected and available. |
 | **`use_async_offload`** | `bool` | Scalar | CUDA-only VRAM optimization flag (Default: `False`).<br>If `True`, `E_saved` and `R_saved` are asynchronously offloaded to page-locked host memory (`pin_memory` CPU RAM). This reduces GPU VRAM consumption at the cost of PCIe transfers. On CPU this option is ignored. |
 
 ### 4.1 FWI Gradient Mode
@@ -259,8 +267,55 @@ The backward solver applies the exact reverse-mode transpose of each executed op
 
 CPML is treated as a fixed numerical boundary. Its boundary material averages are explicitly detached, and CPML cells are excluded from the returned material gradients. This separation must be retained when optimizing a model.
 
+The material-gradient formulation in DeepGPR was informed in part by the differentiable FDTD implementation in [TIDE](https://github.com/Vcholerae1/tide-GPR), particularly its treatment of the discrete Maxwell electric-field update in gradient computation. We gratefully acknowledge the TIDE project and its authors for this work.
+
 Use `model_gradient_sampling_interval=1` and `wavefield_storage_dtype=torch.float32` for a directional derivative check in the physical model region. Temporal subsampling and lower-precision storage deliberately approximate the gradient. Run [the gradient-check notebook](examples/4.GradientCheck.ipynb) after rebuilding the native ABI 6 libraries.
 
+### 4.3 GPU-native block INT8 history
+
+Only the sampled forward state used by the material adjoint is compressed. The
+live Ex/Ey/Ez/Hx/Hy/Hz and CPML states remain float32. The executed discrete
+update requires `E^n` and `R^n`, where `E^(n+1) = ca E^n + cb R^n`; consequently
+`mode=2` stores compressed Ez/Rz and `mode=3` stores compressed Ex/Ey/Ez and all
+three corresponding RHS components. Magnetic histories are not stored.
+
+The packed tensor contains a contiguous signed-INT8 payload followed by a
+four-byte-aligned contiguous FP32 scale array. Backward maps one CUDA block to
+one compression tile, loads each E/R scale once into shared memory, decodes each
+value in a register, and immediately accumulates the epsilon/conductivity
+gradient. It does not allocate or write a reconstructed global-memory history.
+`use_async_offload=True`, CPU execution, and a non-float32
+`wavefield_storage_dtype` are explicitly incompatible with `"int8"`.
+
+The current shared-memory maximum reduction remains available as
+`int8_reduction_backend="current"`. The RTX 4090 audit selected CUB
+`BlockReduce` for the default 64-voxel tiles; `"auto"` falls back to the current
+tree for other supported power-of-two tile volumes.
+
+`E_saved` is an opaque one-dimensional `torch.int8` packed tensor in this mode.
+For diagnostics only, reconstruct it with
+`DeepGPR.decompress_wavefield_history(E_saved, original_shape, block_size)`.
+This helper materializes FP32 and is never called by autograd backward.
+
+## CUDA Backend Build
+
+Build the CUDA shared library from the repository root on Linux. `-lineinfo`
+preserves source correlation for Nsight; `-Xptxas=-v` prints registers and spill
+stores/loads for each kernel.
+
+```bash
+nvcc -std=c++14 -O3 -lineinfo -Xptxas=-v -arch=sm_89 --shared -Xcompiler -fPIC \
+  -o src/DeepGPR/lib/deepgpr.so src/DeepGPR/lib/deepgpr.cu
+```
+
+The command above is the audited RTX 4090 build path. Select the matching
+architecture when building for a different GPU; do not add `--use_fast_math`
+unless a separate numerical and gradient validation explicitly permits it.
+
+The new INT8 path deliberately keeps native ABI 6 because the forward/backward
+C signatures are unchanged. A capability symbol prevents an older ABI-6 CUDA
+library from accepting the packed storage code and corrupting memory.
+
 ## CPU Backend Build
 
 The CPU backend is a plain C shared library and is built with OpenMP by default. Build it into `src/DeepGPR/lib` before running with `device='cpu'`. You can control CPU thread count with `OMP_NUM_THREADS`.
@@ -309,11 +364,12 @@ return E_saved, (Ex, Ey, Ez), (Hx, Hy, Hz), (x0EPhi1...zmHPhi2), receiver_amplit
 ```
 
 1.  **`E_saved`**: The pre-update electric field history `E^n` saved for gradient calculation and diagnostics. An internal `R_saved` tensor stores the corresponding discrete right-hand side `R^n`.
+    *   With `save_wavefield_history=False`, `E_saved` is a zero-length tensor and neither E nor R history storage/compression kernels are launched.
     *   **Shape when `mode=2`**: `(nt_saved, nstep, nx, ny, nz)`, storing Ez only.
     *   **Shape when `mode=3`**: `(3, nt_saved, nstep, nx, ny, nz)`, storing components in `[Ex, Ey, Ez]` order.
     *   `nt_saved` depends on `nt` and `model_gradient_sampling_interval`.
-    *   Dtype is selected by `wavefield_storage_dtype`.
-    *   Set `save_forward_wavefield_path="/path/to/output"` to save this tensor as a CPU-loadable `.pt` file. Load it with `torch.load(path, map_location="cpu")`.
+    *   Dtype is selected by `wavefield_storage_dtype` when compression is disabled. With `wavefield_compression="int8"`, this is an opaque packed one-dimensional `torch.int8` tensor containing values and FP32 scales.
+    *   Set `save_forward_wavefield_path="/path/to/output"` to save a CPU-loadable `.pt` file. Uncompressed modes save the tensor directly. INT8 mode saves a dictionary containing `wavefield`, `compression`, `block_size`, and `uncompressed_shape` so diagnostics can reconstruct it safely.
 2.  **`(Ex, Ey, Ez)`**: The 3D electric field state at the final time step. 
 3.  **`(Hx, Hy, Hz)`**: The 3D magnetic field state at the final time step.
 4.  **`(PML_Tuple)`**: A tuple of 24 Tensors recording the final time step state of the PML auxiliary $\Phi$ variables.

{deepgpr-0.0.18 → deepgpr-0.0.20}/README.md

@@ -1,5 +1,7 @@
 # DeepGPR
 
+**Official Website / Documentation:** [https://songc0a.github.io/DeepGPR/](https://songc0a.github.io/DeepGPR/)
+
 DeepGPR provides a wave propagation module for PyTorch, designed for applications such as Ground Penetrating Radar (GPR) imaging and inversion. Its core concepts are derived from Deepwave. You can use it to perform both forward modeling and backpropagation—thereby enabling the simulation of wave propagation to generate synthetic data—as well as for Full Waveform Inversion (FWI). Furthermore, you can integrate this wave propagation functionality into a larger operational pipeline—incorporating various wavelets, loss functions, and other components—to achieve end-to-end forward and reverse propagation, powered by automatic differentiation and our high-performance operators.
 
 
@@ -218,7 +220,13 @@ This section defines the geometric observation system (coordinates) and the exci
 | :--- | :--- | :--- | :--- |
 | **`pmlthick`** | `int` / `list` / `Tensor`| Scalar or list of 6 | Thickness (in grid layers) of the PML (Perfectly Matched Layer) absorbing boundaries.<br>- Integer `p`: All six boundaries have thickness `p` (Z-boundaries are ignored in 2D).<br>- List `[x0, xm, y0, ym, z0, zm]`: Specific thicknesses for the 6 boundaries. |
 | **`model_gradient_sampling_interval`**| `int` | Scalar | Wavefield sampling interval during forward propagation (Default: 1).<br>A larger integer reduces VRAM use for `E_saved` and `R_saved`, but uses an explicitly approximate model gradient. The last incomplete sampling block is weighted by its actual length. |
+| **`save_wavefield_history`** | `bool` | Scalar | Independently controls allocation and native writes of the E/R histories used by adjoint model-gradient backward (Default: `True`). `False` still executes the complete FDTD, CPML, source-injection, and receiver-recording path, returns an empty `E_saved`, and raises a clear error if history-dependent backward is attempted. |
 | **`wavefield_storage_dtype`** | `torch.dtype` / `str` | `float32`, `float16`, or `bfloat16` | Storage format for saved `E_saved` and `R_saved` model-gradient wavefields. FDTD propagation remains float32. `float16` and `bfloat16` halve saved-wavefield memory at the cost of gradient accuracy; `bfloat16` has the safer dynamic range. String aliases such as `"fp16"` and `"bf16"` are accepted. |
+| **`wavefield_conversion_backend`** | `str` | `"auto"`, `"legacy"`, `"native_scalar"`, or `"native_vec2"` | CUDA FP16/BF16 history conversion. The audited default `"auto"` uses NVIDIA scalar intrinsics for CUDA FP16 and the legacy path otherwise. Explicit values retain the correctness/performance A/B paths; vec2 is not the default because it was slower on RTX 4090. |
+| **`wavefield_compression`** | `str` | `"none"`, `"int8"`, or `"zfp"` | `"int8"` enables CUDA-native per-block symmetric INT8 histories with FP32 scales and inline decode in the fused material-gradient kernel. `"none"` is the unchanged default. `"zfp"` is reserved for an optional fused CUDA decoder and is rejected by the current dependency-free build instead of silently materializing a decoded global-memory history. |
+| **`wavefield_compression_block_size`** | sequence / `None` | 2D or 3D spatial block | INT8 block shape; defaults to `(8, 8)` in 2D and `(4, 4, 4)` in 3D. The block volume must be a power of two no larger than 256. Partial boundary blocks are supported. |
+| **`int8_reduction_backend`** | `str` | `"auto"`, `"current"`, `"cub_block"`, or `"warp_shuffle"` | Tile maximum reduction. The audited default `"auto"` selects NVIDIA CUB `BlockReduce` for 64-voxel tiles and preserves the prior shared-memory tree for other valid tile sizes. Explicit values expose the retained A/B implementations. |
+| **`wavefield_compression_rate`** | scalar / `None` | Optional ZFP setting | Reserved for an optional ZFP backend. It is rejected unless that backend is selected and available. |
 | **`use_async_offload`** | `bool` | Scalar | CUDA-only VRAM optimization flag (Default: `False`).<br>If `True`, `E_saved` and `R_saved` are asynchronously offloaded to page-locked host memory (`pin_memory` CPU RAM). This reduces GPU VRAM consumption at the cost of PCIe transfers. On CPU this option is ignored. |
 
 ### 4.1 FWI Gradient Mode
@@ -236,8 +244,55 @@ The backward solver applies the exact reverse-mode transpose of each executed op
 
 CPML is treated as a fixed numerical boundary. Its boundary material averages are explicitly detached, and CPML cells are excluded from the returned material gradients. This separation must be retained when optimizing a model.
 
+The material-gradient formulation in DeepGPR was informed in part by the differentiable FDTD implementation in [TIDE](https://github.com/Vcholerae1/tide-GPR), particularly its treatment of the discrete Maxwell electric-field update in gradient computation. We gratefully acknowledge the TIDE project and its authors for this work.
+
 Use `model_gradient_sampling_interval=1` and `wavefield_storage_dtype=torch.float32` for a directional derivative check in the physical model region. Temporal subsampling and lower-precision storage deliberately approximate the gradient. Run [the gradient-check notebook](examples/4.GradientCheck.ipynb) after rebuilding the native ABI 6 libraries.
 
+### 4.3 GPU-native block INT8 history
+
+Only the sampled forward state used by the material adjoint is compressed. The
+live Ex/Ey/Ez/Hx/Hy/Hz and CPML states remain float32. The executed discrete
+update requires `E^n` and `R^n`, where `E^(n+1) = ca E^n + cb R^n`; consequently
+`mode=2` stores compressed Ez/Rz and `mode=3` stores compressed Ex/Ey/Ez and all
+three corresponding RHS components. Magnetic histories are not stored.
+
+The packed tensor contains a contiguous signed-INT8 payload followed by a
+four-byte-aligned contiguous FP32 scale array. Backward maps one CUDA block to
+one compression tile, loads each E/R scale once into shared memory, decodes each
+value in a register, and immediately accumulates the epsilon/conductivity
+gradient. It does not allocate or write a reconstructed global-memory history.
+`use_async_offload=True`, CPU execution, and a non-float32
+`wavefield_storage_dtype` are explicitly incompatible with `"int8"`.
+
+The current shared-memory maximum reduction remains available as
+`int8_reduction_backend="current"`. The RTX 4090 audit selected CUB
+`BlockReduce` for the default 64-voxel tiles; `"auto"` falls back to the current
+tree for other supported power-of-two tile volumes.
+
+`E_saved` is an opaque one-dimensional `torch.int8` packed tensor in this mode.
+For diagnostics only, reconstruct it with
+`DeepGPR.decompress_wavefield_history(E_saved, original_shape, block_size)`.
+This helper materializes FP32 and is never called by autograd backward.
+
+## CUDA Backend Build
+
+Build the CUDA shared library from the repository root on Linux. `-lineinfo`
+preserves source correlation for Nsight; `-Xptxas=-v` prints registers and spill
+stores/loads for each kernel.
+
+```bash
+nvcc -std=c++14 -O3 -lineinfo -Xptxas=-v -arch=sm_89 --shared -Xcompiler -fPIC \
+  -o src/DeepGPR/lib/deepgpr.so src/DeepGPR/lib/deepgpr.cu
+```
+
+The command above is the audited RTX 4090 build path. Select the matching
+architecture when building for a different GPU; do not add `--use_fast_math`
+unless a separate numerical and gradient validation explicitly permits it.
+
+The new INT8 path deliberately keeps native ABI 6 because the forward/backward
+C signatures are unchanged. A capability symbol prevents an older ABI-6 CUDA
+library from accepting the packed storage code and corrupting memory.
+
 ## CPU Backend Build
 
 The CPU backend is a plain C shared library and is built with OpenMP by default. Build it into `src/DeepGPR/lib` before running with `device='cpu'`. You can control CPU thread count with `OMP_NUM_THREADS`.
@@ -286,11 +341,12 @@ return E_saved, (Ex, Ey, Ez), (Hx, Hy, Hz), (x0EPhi1...zmHPhi2), receiver_amplit
 ```
 
 1.  **`E_saved`**: The pre-update electric field history `E^n` saved for gradient calculation and diagnostics. An internal `R_saved` tensor stores the corresponding discrete right-hand side `R^n`.
+    *   With `save_wavefield_history=False`, `E_saved` is a zero-length tensor and neither E nor R history storage/compression kernels are launched.
     *   **Shape when `mode=2`**: `(nt_saved, nstep, nx, ny, nz)`, storing Ez only.
     *   **Shape when `mode=3`**: `(3, nt_saved, nstep, nx, ny, nz)`, storing components in `[Ex, Ey, Ez]` order.
     *   `nt_saved` depends on `nt` and `model_gradient_sampling_interval`.
-    *   Dtype is selected by `wavefield_storage_dtype`.
-    *   Set `save_forward_wavefield_path="/path/to/output"` to save this tensor as a CPU-loadable `.pt` file. Load it with `torch.load(path, map_location="cpu")`.
+    *   Dtype is selected by `wavefield_storage_dtype` when compression is disabled. With `wavefield_compression="int8"`, this is an opaque packed one-dimensional `torch.int8` tensor containing values and FP32 scales.
+    *   Set `save_forward_wavefield_path="/path/to/output"` to save a CPU-loadable `.pt` file. Uncompressed modes save the tensor directly. INT8 mode saves a dictionary containing `wavefield`, `compression`, `block_size`, and `uncompressed_shape` so diagnostics can reconstruct it safely.
 2.  **`(Ex, Ey, Ez)`**: The 3D electric field state at the final time step. 
 3.  **`(Hx, Hy, Hz)`**: The 3D magnetic field state at the final time step.
 4.  **`(PML_Tuple)`**: A tuple of 24 Tensors recording the final time step state of the PML auxiliary $\Phi$ variables.

{deepgpr-0.0.18 → deepgpr-0.0.20}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "DeepGPR"
-version = "0.0.18"
+version = "0.0.20"
 authors = [
     { name = "Lei Liu", email = "liulei990222@gmail.com" }
 ]

{deepgpr-0.0.18 → deepgpr-0.0.20}/src/DeepGPR/__init__.py

@@ -232,6 +232,29 @@ def _configure_deepgpr_library(lib: ctypes.CDLL) -> None:
         lib.set_fdtd_order.argtypes = [ctypes.c_int]
         lib.set_fdtd_order.restype = None
 
+    if hasattr(lib, "deepgpr_supports_int8_wavefield"):
+        lib.deepgpr_supports_int8_wavefield.argtypes = []
+        lib.deepgpr_supports_int8_wavefield.restype = ctypes.c_int
+
+    if hasattr(lib, "deepgpr_supports_conversion_backends"):
+        lib.deepgpr_supports_conversion_backends.argtypes = []
+        lib.deepgpr_supports_conversion_backends.restype = ctypes.c_int
+
+    if hasattr(lib, "deepgpr_supports_int8_reduction_backends"):
+        lib.deepgpr_supports_int8_reduction_backends.argtypes = []
+        lib.deepgpr_supports_int8_reduction_backends.restype = ctypes.c_int
+
+    if hasattr(lib, "deepgpr_test_wavefield_conversion"):
+        lib.deepgpr_test_wavefield_conversion.argtypes = [
+            _FLOAT_P,
+            _VOID_P,
+            _FLOAT_P,
+            ctypes.c_longlong,
+            ctypes.c_int,
+            ctypes.c_int,
+        ]
+        lib.deepgpr_test_wavefield_conversion.restype = None
+
     lib._deepgpr_argtypes_configured = True