diff-fvr 1.0.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- diff_fvr-1.0.0.dist-info/METADATA +137 -0
- diff_fvr-1.0.0.dist-info/RECORD +18 -0
- diff_fvr-1.0.0.dist-info/WHEEL +5 -0
- diff_fvr-1.0.0.dist-info/entry_points.txt +3 -0
- diff_fvr-1.0.0.dist-info/top_level.txt +1 -0
- fvr/__init__.py +35 -0
- fvr/articulated_spine.py +163 -0
- fvr/cli.py +160 -0
- fvr/core_engine.py +189 -0
- fvr/dicom_loader.py +165 -0
- fvr/diff_fvr.py +194 -0
- fvr/fvr_engine.py +147 -0
- fvr/gradient_optimizer.py +90 -0
- fvr/gradients.py +135 -0
- fvr/raycasting_drr.py +80 -0
- fvr/registration_3d2d.py +161 -0
- fvr/robust_registration.py +134 -0
- fvr/rotation_2d.py +69 -0
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: diff-fvr
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Dual-Domain Differentiable Fourier Volume Rendering for Ultra-Fast Biplanar 3D-2D Spine Registration
|
|
5
|
+
Author-email: Pareel Amre <pareel.amre@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Keywords: digitally-reconstructed-radiographs,fourier-volume-rendering,3d-2d-registration,spine-surgery,medical-imaging,image-guided-interventions
|
|
8
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
9
|
+
Classifier: Intended Audience :: Science/Research
|
|
10
|
+
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
13
|
+
Requires-Python: >=3.10
|
|
14
|
+
Description-Content-Type: text/markdown
|
|
15
|
+
Requires-Dist: numpy>=1.24.0
|
|
16
|
+
Requires-Dist: scipy>=1.10.0
|
|
17
|
+
Requires-Dist: pandas>=2.0.0
|
|
18
|
+
Requires-Dist: matplotlib>=3.7.0
|
|
19
|
+
Requires-Dist: pillow>=9.5.0
|
|
20
|
+
Requires-Dist: pydicom>=2.4.0
|
|
21
|
+
Requires-Dist: simpleitk>=2.2.0
|
|
22
|
+
Requires-Dist: scikit-image>=0.20.0
|
|
23
|
+
|
|
24
|
+
# Dual-Domain Differentiable Fourier Volume Rendering (Diff-FVR) for Real-Time Biplanar 3D–2D Spine Registration
|
|
25
|
+
|
|
26
|
+
[](https://github.com/pareelamre/diff-fvr/actions/workflows/ci.yml)
|
|
27
|
+
[](https://pareelamre.github.io/diff-fvr/)
|
|
28
|
+
[](https://pareelamre.github.io/diff-fvr/)
|
|
29
|
+
[](LICENSE)
|
|
30
|
+
[](https://www.python.org/)
|
|
31
|
+
|
|
32
|
+
**Author:** Pareel Amre
|
|
33
|
+
**Affiliation:** Faculty of Computer Science, Technische Universität Dresden (TU Dresden), 01062 Dresden, Germany
|
|
34
|
+
**Email:** `pareel.amre@gmail.com`
|
|
35
|
+
**Target Venue:** *IEEE Transactions on Medical Imaging* / *Medical Image Analysis*
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## 🔬 Scientific Overview
|
|
40
|
+
|
|
41
|
+
Intensity-based 3D–2D registration between pre-operative volumetric Computed Tomography (pCT) and intra-operative 2D fluoroscopy is essential for computer-assisted spine surgery (e.g. pedicle screw navigation). However, conventional ray-casting DRR synthesis scales cubically ($\mathcal{O}(N^3)$), requiring $> 1.0\text{ s}$ per projection on CPU.
|
|
42
|
+
|
|
43
|
+
**Diff-FVR** formulates Digitally Reconstructed Radiographs (DRRs) via the **continuous Fourier Projection-Slice Theorem**, reducing rendering complexity to $\mathcal{O}(N^2)$ frequency slice extraction:
|
|
44
|
+
|
|
45
|
+
$$
|
|
46
|
+
\mathcal{F}_{2D}\{\mathcal{P}[f(x,y,z)]\} = \mathcal{S}_{2D}\{\mathcal{F}_{3D}[f(x,y,z)]\}
|
|
47
|
+
$$
|
|
48
|
+
|
|
49
|
+
1. **Mathematical Anti-Aliasing:** Enforces $2\times$ spatial zero-padding bound ($Z_p = 2.0 \ge \sqrt{3}$) and $\text{sinc}^2$ spatial pre-multiplication to eliminate boundary wrap-around and peripheral roll-off.
|
|
50
|
+
2. **Exact 1:1 Scale Preservation:** Implements full-bandwidth frequency slicing to preserve physical anatomical scale.
|
|
51
|
+
3. **Normalized Spectral Cross-Correlation (Spectral-NCC):** Computes similarity directly in 2D Fourier space via Parseval's identity ($< 1.0\text{ ms}$ per pose) with **zero inverse Fourier transforms** inside the optimization loop.
|
|
52
|
+
4. **Articulated Multi-Vertebra Kinematics:** Models inter-vertebral flexion (L3–L4–L5) on surgical tables using Fourier linearity:
|
|
53
|
+
$$
|
|
54
|
+
\mathcal{F}\left\{\sum_i I_i\right\} = \sum_i S_i(\mathbf{k}, \mathbf{T}_i)
|
|
55
|
+
$$
|
|
56
|
+
5. **Robustness to Physical Failure Limits:** Automatic radio-opaque metal hardware saturation masking + Masked Spatial Loss (achieving $0.000\text{ mm}$ error under $30\%$ occlusion) and isocenter geometric calibration ($1/\text{mag} = \text{SAD}/\text{SDD}$).
|
|
57
|
+
|
|
58
|
+
---
|
|
59
|
+
|
|
60
|
+
## 📊 Comprehensive Experimental Suite (23 Benchmarks)
|
|
61
|
+
|
|
62
|
+
| Experiment | Title | Key Quantitative Finding |
|
|
63
|
+
| :--- | :--- | :--- |
|
|
64
|
+
| **Exp 1** | O(N²) vs O(N³) Asymptotic Complexity | **11.5× to 72.5× speedup** over spatial raycasting (1.38 ms DRR). |
|
|
65
|
+
| **Exp 2** | Rotational Invariance (72 Angles) | PSNR = 23.96 dB, SSIM = 0.9329, NCC = 0.9943 ± 0.0007. |
|
|
66
|
+
| **Exp 3** | Zero-Padding & Sinc² Premultiplication | 2× padding eliminates high-frequency wrap-around; roll-off removed. |
|
|
67
|
+
| **Exp 4** | Spectral-NCC 50-Trial Benchmark | 0.84 mm mTRE (98.0% success < 2.0 mm) in 1.82 s (212× per-pose acceleration). |
|
|
68
|
+
| **Exp 5** | Energy Landscape & Capture Range | Smooth, strictly convex basin over ±25° / ±25 mm displacement. |
|
|
69
|
+
| **Exp 6** | Biplanar vs. Monoplanar 6-DoF | Depth error Δtx reduced from 3.59 mm down to **0.03 ± 0.04 mm**. |
|
|
70
|
+
| **Exp 7** | Real Hospital Patient DICOM CT Series | **mTRE = 0.020 mm** (Clean), **0.031 mm** (Low-Dose); 100% success. |
|
|
71
|
+
| **Exp 8** | Capture Basin Sensitivity Bounds | 100% success at ±5° / ±5 mm; 90% at ±16° / ±16 mm. |
|
|
72
|
+
| **Exp 9** | Cone-Beam Perspective Divergence | Identified 1.4× magnification distortion; isocenter calibration cuts error by 89%. |
|
|
73
|
+
| **Exp 10** | Surgical Metal Hardware Occlusion | Automatic saturation threshold + Masked Spatial Loss drops error from 49.15 mm to 0.000 mm. |
|
|
74
|
+
| **Exp 11** | Radiation Exposure Starvation Sweep | Sub-0.05 mm mTRE maintained across 6 orders of magnitude (10⁶ down to 10³ ph/px). |
|
|
75
|
+
| **Exp 12** | Multi-Center Kaggle Lumbar CT Cohort | **mTRE = 1.341 mm** (Clean), **1.351 mm** (Low-Dose); 85.0% success. |
|
|
76
|
+
| **Exp 13** | Multi-Center AMOS Cohort (NCT Dresden) | **mTRE = 1.118 mm** (Clean), **1.121 mm** (Low-Dose); 90.0% success. |
|
|
77
|
+
| **Exp 14** | Resolved Failure Limits Verification | Unit tests confirm 0.000 mm precision under 30% metal occlusion. |
|
|
78
|
+
| **Exp 15** | Articulated Multi-Vertebra Kinematics | Hierarchical L3-L4-L5 model recovers base pose (0.767 mm) and in-vivo flexion (Δloss = −54.7%). |
|
|
79
|
+
| **Exp 16** | Unified Multi-Center CED Benchmark | Standardized linear and logarithmic CED curves across all 3 independent cohorts. |
|
|
80
|
+
| **Exp 17** | Detector FOV Truncation & Clipping | 2D Tukey cosine tapering preserves 0.75 mm mTRE up to 30% clipping. |
|
|
81
|
+
| **Exp 18** | Non-Orthogonal C-Arm Angulations | Sub-millimeter (0.021–0.055 mm) down to Δϕ = 30° safe operating angle. |
|
|
82
|
+
| **Exp 19** | Polychromatic Beam Hardening & Soft Tissue | High-Pass Spectral-NCC achieves 100% sub-mm (0.028–0.358 mm) across all tiers. |
|
|
83
|
+
| **Exp 20** | Intraoperative Respiratory Motion Blur | Phase invariance preserves sub-millimeter tracking across 0–5 mm breathing smear. |
|
|
84
|
+
| **Exp 21** | Preoperative CT Anisotropic Slice Thickness | Biplanar decoupling preserves sub-millimeter precision (0.0008 mm) on 5.0 mm trauma slices. |
|
|
85
|
+
| **Exp 22** | Dynamic Streaming Fluoroscopy Tracking | Real-time video tracking across 60 frames with warm-start analytical gradients. |
|
|
86
|
+
| **Exp 23** | Joint Articulation + Metal Implants | Base mTRE 0.776 mm, cuts residual loss by 48.2%, recovers in-vivo flexion angles. |
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## 🎮 Interactive Surgical Registration Cockpit
|
|
91
|
+
|
|
92
|
+
An interactive in-browser simulator demonstrating real-time 6-DoF biplanar registration, pedicle screw metal saturation, and physical breakdown regimes is live:
|
|
93
|
+
|
|
94
|
+
👉 **[Launch Interactive Cockpit](https://pareelamre.github.io/diff-fvr/)**
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## 🚀 Installation & CLI Quick Start
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
# Install package in editable mode
|
|
102
|
+
pip install -e .
|
|
103
|
+
|
|
104
|
+
# 1. Synthesize a fast 2D DRR from 3D CT:
|
|
105
|
+
fvr-render --ct data/dicom --yaw 15.0 --pitch -5.0 --view ap --res 96 --out drr_ap.png
|
|
106
|
+
|
|
107
|
+
# 2. Run biplanar 6-DoF spine registration:
|
|
108
|
+
fvr-register --ct data/dicom --ap data/ap_xray.png --lat data/lat_xray.png --handle-metal --out pose_result.json
|
|
109
|
+
|
|
110
|
+
# 3. Run production integration test suite:
|
|
111
|
+
python tests/test_core_engine.py
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## 📁 Repository Structure
|
|
117
|
+
|
|
118
|
+
```
|
|
119
|
+
fft-drr-project/
|
|
120
|
+
├── data/
|
|
121
|
+
│ ├── dicom/ # Real clinical hospital patient CT DICOM series
|
|
122
|
+
│ ├── kaggle_lumbar/ # Multi-center Kaggle lumbar spine CT
|
|
123
|
+
│ └── amos/ # AMOS multi-organ abdominal CT cohort (NCT Dresden)
|
|
124
|
+
├── docs/ # GitHub Pages interactive surgical cockpit simulator
|
|
125
|
+
├── src/fvr/
|
|
126
|
+
│ ├── core_engine.py # Production engine (calibration, metal masking, bounded search)
|
|
127
|
+
│ ├── diff_fvr.py # Core Differentiable Fourier Volume Renderer
|
|
128
|
+
│ ├── articulated_spine.py # Multi-body kinematic chain (L3-L4-L5)
|
|
129
|
+
│ ├── gradients.py # Closed-form analytical Fourier phase & spectral gradients
|
|
130
|
+
│ ├── robust_registration.py # Masked spatial loss engine
|
|
131
|
+
│ └── dicom_loader.py # DICOM series loader & Hounsfield-to-attenuation converter
|
|
132
|
+
├── experiments/ # Complete 23 benchmark experiment scripts
|
|
133
|
+
├── outputs/paper_figures/ # High-resolution 300 DPI publication figures & CSV tables
|
|
134
|
+
├── tests/ # Verification and regression test suite
|
|
135
|
+
├── main.tex # IEEE Transactions on Medical Imaging manuscript
|
|
136
|
+
└── paper_manuscript.md # Comprehensive audited research report
|
|
137
|
+
```
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
fvr/__init__.py,sha256=6dAIlgWZYlrKWEu7vSUAnJk4HUvn102zGZ4s6KzXgY4,1100
|
|
2
|
+
fvr/articulated_spine.py,sha256=-FzPOPK9CUXJVSLrhGcQZlwqiF0yQGdXpWH4ZUKCgo0,6607
|
|
3
|
+
fvr/cli.py,sha256=OHlIYLgOyn_zPwiM4Ve1rH2FplXoxOTU_WmQdtpHW1w,7103
|
|
4
|
+
fvr/core_engine.py,sha256=pvTaTnyb5_xxLDggvM9mpZfy9h5NT3ElhT2Pqqe_hu8,7351
|
|
5
|
+
fvr/dicom_loader.py,sha256=nvC6_FntSWRkf_v7aQ-100jxbmT6_-0qvg8jTMasZoo,6399
|
|
6
|
+
fvr/diff_fvr.py,sha256=YWD4w0h-m6q9GN3AjS4JVp2ww3s_wnzWOI00S-wqQnk,8116
|
|
7
|
+
fvr/fvr_engine.py,sha256=vYBp66kFqU9lVhQKSRWODUNGndUF7KQwwS9bvnYD8-4,6125
|
|
8
|
+
fvr/gradient_optimizer.py,sha256=BEnsCzqu2NHN4XQMtGXlnBtoN6lPI0-8nLXsa2xr-Rg,2872
|
|
9
|
+
fvr/gradients.py,sha256=KLjxceU8sltsZvbnBGWHX5UV3dMCnjwoinBopqSgkDY,5631
|
|
10
|
+
fvr/raycasting_drr.py,sha256=ciS8jxIUNUBgujvFfrRiOD7Zi-iTsHx2aHPnPsmHbVU,2753
|
|
11
|
+
fvr/registration_3d2d.py,sha256=KkjO-yGn0A5cGPGBWiPdMMLOLSgl6BNhJlM-UaEfxp8,5924
|
|
12
|
+
fvr/robust_registration.py,sha256=doZ0rq41R5IKsYSbhva5ciXavjMXSd7eDmfHvT7X_i0,5705
|
|
13
|
+
fvr/rotation_2d.py,sha256=WKVSykE1I3O0YV5TmZiLhWB5hYlaABxfXOb8E-iaMJQ,2574
|
|
14
|
+
diff_fvr-1.0.0.dist-info/METADATA,sha256=nm15xez1MTWjlbw4A5xKhOO7aJLFkb-33lXUG9gwt9c,8966
|
|
15
|
+
diff_fvr-1.0.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
|
|
16
|
+
diff_fvr-1.0.0.dist-info/entry_points.txt,sha256=zTLs9n81t3KR7p4HJbf1AXgoABNSnHB3VulCPDqxork,88
|
|
17
|
+
diff_fvr-1.0.0.dist-info/top_level.txt,sha256=v00arNuDKSbJkDGsfQJPu4RRdOr6gqGqBaTbDnHRkUQ,4
|
|
18
|
+
diff_fvr-1.0.0.dist-info/RECORD,,
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
fvr
|
fvr/__init__.py
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
"""
|
|
2
|
+
Diff-FVR: Differentiable Fourier Volume Rendering for 3D-2D Registration.
|
|
3
|
+
Author: Pareel Amre, TU Dresden
|
|
4
|
+
"""
|
|
5
|
+
|
|
6
|
+
from .fvr_engine import FourierVolumeRenderer
|
|
7
|
+
from .diff_fvr import DifferentiableFourierVolumeRenderer
|
|
8
|
+
from .core_engine import RobustDiffFVREngine, detect_radio_opaque_metal
|
|
9
|
+
from .gradients import SpectralGradientEngine
|
|
10
|
+
from .articulated_spine import ArticulatedSpineModel, VertebralSegment
|
|
11
|
+
from .raycasting_drr import raycast_drr
|
|
12
|
+
from .rotation_2d import rotate_2d_fft
|
|
13
|
+
from .dicom_loader import load_dicom_series, create_spine_phantom, hu_to_attenuation
|
|
14
|
+
from .registration_3d2d import compute_mtre_6dof
|
|
15
|
+
from .cli import main_render, main_register
|
|
16
|
+
|
|
17
|
+
__version__ = "1.0.0"
|
|
18
|
+
|
|
19
|
+
__all__ = [
|
|
20
|
+
"FourierVolumeRenderer",
|
|
21
|
+
"DifferentiableFourierVolumeRenderer",
|
|
22
|
+
"RobustDiffFVREngine",
|
|
23
|
+
"detect_radio_opaque_metal",
|
|
24
|
+
"SpectralGradientEngine",
|
|
25
|
+
"ArticulatedSpineModel",
|
|
26
|
+
"VertebralSegment",
|
|
27
|
+
"raycast_drr",
|
|
28
|
+
"rotate_2d_fft",
|
|
29
|
+
"load_dicom_series",
|
|
30
|
+
"create_spine_phantom",
|
|
31
|
+
"hu_to_attenuation",
|
|
32
|
+
"compute_mtre_6dof",
|
|
33
|
+
"main_render",
|
|
34
|
+
"main_register"
|
|
35
|
+
]
|
fvr/articulated_spine.py
ADDED
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
"""
|
|
2
|
+
Articulated Multi-Vertebra Kinematic Registration Framework.
|
|
3
|
+
|
|
4
|
+
Models the lumbar spine as a kinematically coupled multi-body system (e.g., L3-L4-L5)
|
|
5
|
+
where each vertebral segment undergoes independent relative rotation (flexion/extension,
|
|
6
|
+
lateral bending, axial twist) and translation within physiological disc limits.
|
|
7
|
+
|
|
8
|
+
Leverages Fourier Linearity:
|
|
9
|
+
F_2D{ sum_i I_i } = sum_i F_2D{ I_i } = sum_i S_i(k, T_i)
|
|
10
|
+
allowing ultra-fast simultaneous articulated DRR synthesis and hierarchical registration.
|
|
11
|
+
"""
|
|
12
|
+
|
|
13
|
+
import numpy as np
|
|
14
|
+
import time
|
|
15
|
+
from typing import List, Dict, Any, Tuple, Optional
|
|
16
|
+
from scipy.optimize import minimize
|
|
17
|
+
from .diff_fvr import DifferentiableFourierVolumeRenderer
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
class VertebralSegment:
|
|
21
|
+
"""
|
|
22
|
+
Represents an individual segmented vertebra in 3D Fourier space.
|
|
23
|
+
"""
|
|
24
|
+
def __init__(self, name: str, sub_volume: np.ndarray, nominal_offset_z: float, pad_factor: float = 2.0):
|
|
25
|
+
self.name = name
|
|
26
|
+
self.nominal_offset_z = nominal_offset_z
|
|
27
|
+
self.renderer = DifferentiableFourierVolumeRenderer(sub_volume, pad_factor=pad_factor)
|
|
28
|
+
self.shape = sub_volume.shape
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
class ArticulatedSpineModel:
|
|
32
|
+
"""
|
|
33
|
+
Hierarchical Articulated Spine Model with Physiological Joint Bounds.
|
|
34
|
+
"""
|
|
35
|
+
def __init__(self, segments: List[VertebralSegment]):
|
|
36
|
+
self.segments = segments
|
|
37
|
+
self.num_segments = len(segments)
|
|
38
|
+
|
|
39
|
+
def render_articulated_spectrum(
|
|
40
|
+
self,
|
|
41
|
+
base_pose: np.ndarray, # [yaw, pitch, roll, tx, ty, tz]
|
|
42
|
+
relative_articulations: List[np.ndarray], # list of [d_yaw, d_pitch, d_roll, d_tx, d_ty, d_tz]
|
|
43
|
+
view: str = 'lateral'
|
|
44
|
+
) -> np.ndarray:
|
|
45
|
+
"""
|
|
46
|
+
Synthesize combined 2D Fourier spectrum of articulated spine via Fourier linearity.
|
|
47
|
+
"""
|
|
48
|
+
ref_seg = self.segments[0]
|
|
49
|
+
slice_h = ref_seg.renderer.pz
|
|
50
|
+
slice_w = ref_seg.renderer.py if view == 'lateral' else ref_seg.renderer.px
|
|
51
|
+
|
|
52
|
+
total_spectrum = np.zeros((slice_h, slice_w), dtype=np.complex128)
|
|
53
|
+
|
|
54
|
+
curr_pose = np.copy(base_pose)
|
|
55
|
+
for i, seg in enumerate(self.segments):
|
|
56
|
+
if i > 0 and len(relative_articulations) >= i:
|
|
57
|
+
# Accumulate relative inter-vertebral articulation
|
|
58
|
+
curr_pose = curr_pose + relative_articulations[i - 1]
|
|
59
|
+
|
|
60
|
+
# In-plane translation for the specific view
|
|
61
|
+
yaw, pitch, roll, tx, ty, tz = curr_pose
|
|
62
|
+
if view == 'lateral':
|
|
63
|
+
f_slice = seg.renderer.extract_frequency_slice(yaw, pitch, roll, tx=ty, ty=tz, view='lateral')
|
|
64
|
+
elif view == 'ap':
|
|
65
|
+
f_slice = seg.renderer.extract_frequency_slice(yaw, pitch, roll, tx=tx, ty=tz, view='ap')
|
|
66
|
+
else:
|
|
67
|
+
f_slice = seg.renderer.extract_frequency_slice(yaw, pitch, roll, tx=tx, ty=ty, view='axial')
|
|
68
|
+
|
|
69
|
+
total_spectrum += f_slice
|
|
70
|
+
|
|
71
|
+
return total_spectrum
|
|
72
|
+
|
|
73
|
+
def render_drr(
|
|
74
|
+
self,
|
|
75
|
+
base_pose: np.ndarray,
|
|
76
|
+
relative_articulations: List[np.ndarray],
|
|
77
|
+
view: str = 'lateral'
|
|
78
|
+
) -> np.ndarray:
|
|
79
|
+
"""
|
|
80
|
+
Render spatial projection of the articulated spine.
|
|
81
|
+
"""
|
|
82
|
+
f_spectrum = self.render_articulated_spectrum(base_pose, relative_articulations, view=view)
|
|
83
|
+
from scipy import fft
|
|
84
|
+
spatial = np.real(fft.fftshift(fft.ifft2(fft.ifftshift(f_spectrum))))
|
|
85
|
+
return np.maximum(spatial, 0.0)
|
|
86
|
+
|
|
87
|
+
def register_articulated_biplanar(
|
|
88
|
+
self,
|
|
89
|
+
f_target_ap: np.ndarray,
|
|
90
|
+
f_target_lat: np.ndarray,
|
|
91
|
+
init_base_pose: np.ndarray,
|
|
92
|
+
max_iter_global: int = 40,
|
|
93
|
+
max_iter_local: int = 30
|
|
94
|
+
) -> Dict[str, Any]:
|
|
95
|
+
"""
|
|
96
|
+
Two-Stage Hierarchical Articulated Registration:
|
|
97
|
+
Stage 1: Global Spine Alignment (all vertebrae rigidly locked).
|
|
98
|
+
Stage 2: Local Inter-Vertebral Joint Refinement within physiological bounds.
|
|
99
|
+
"""
|
|
100
|
+
t0 = time.time()
|
|
101
|
+
num_joints = self.num_segments - 1
|
|
102
|
+
|
|
103
|
+
# --- Stage 1: Global Rigid Base Registration ---
|
|
104
|
+
def stage1_obj(pose):
|
|
105
|
+
zero_art = [np.zeros(6) for _ in range(num_joints)]
|
|
106
|
+
f_ap = self.render_articulated_spectrum(pose, zero_art, view='ap')
|
|
107
|
+
f_lat = self.render_articulated_spectrum(pose, zero_art, view='lateral')
|
|
108
|
+
|
|
109
|
+
ncc_ap = self.segments[0].renderer.compute_spectral_ncc(f_ap, f_target_ap)
|
|
110
|
+
ncc_lat = self.segments[0].renderer.compute_spectral_ncc(f_lat, f_target_lat)
|
|
111
|
+
return float(0.5 * (1.0 - ncc_ap) + 0.5 * (1.0 - ncc_lat))
|
|
112
|
+
|
|
113
|
+
res_stage1 = minimize(
|
|
114
|
+
stage1_obj,
|
|
115
|
+
x0=init_base_pose,
|
|
116
|
+
method='Powell',
|
|
117
|
+
options={'maxiter': max_iter_global, 'ftol': 1e-4}
|
|
118
|
+
)
|
|
119
|
+
base_converged = res_stage1.x
|
|
120
|
+
|
|
121
|
+
# --- Stage 2: Articulated Joint Refinement ---
|
|
122
|
+
# State vector: [d_yaw_1..K-1, d_pitch_1..K-1, d_roll_1..K-1, d_tx_1..K-1, ...]
|
|
123
|
+
if num_joints > 0:
|
|
124
|
+
init_art = np.zeros(num_joints * 6)
|
|
125
|
+
|
|
126
|
+
# Physiological limits: +/- 8 deg rotation, +/- 2.5 mm translation
|
|
127
|
+
joint_bounds = []
|
|
128
|
+
for _ in range(num_joints):
|
|
129
|
+
joint_bounds.extend([(-8.0, 8.0), (-8.0, 8.0), (-8.0, 8.0), (-2.5, 2.5), (-2.5, 2.5), (-2.5, 2.5)])
|
|
130
|
+
|
|
131
|
+
def stage2_obj(art_flat):
|
|
132
|
+
arts = [art_flat[j*6:(j+1)*6] for j in range(num_joints)]
|
|
133
|
+
f_ap = self.render_articulated_spectrum(base_converged, arts, view='ap')
|
|
134
|
+
f_lat = self.render_articulated_spectrum(base_converged, arts, view='lateral')
|
|
135
|
+
|
|
136
|
+
ncc_ap = self.segments[0].renderer.compute_spectral_ncc(f_ap, f_target_ap)
|
|
137
|
+
ncc_lat = self.segments[0].renderer.compute_spectral_ncc(f_lat, f_target_lat)
|
|
138
|
+
|
|
139
|
+
# Prior: small regularization on inter-vertebral strain
|
|
140
|
+
strain_penalty = 1e-4 * np.sum(art_flat ** 2)
|
|
141
|
+
return float(0.5 * (1.0 - ncc_ap) + 0.5 * (1.0 - ncc_lat) + strain_penalty)
|
|
142
|
+
|
|
143
|
+
res_stage2 = minimize(
|
|
144
|
+
stage2_obj,
|
|
145
|
+
x0=init_art,
|
|
146
|
+
method='Powell',
|
|
147
|
+
options={'maxiter': max_iter_local, 'ftol': 1e-4}
|
|
148
|
+
)
|
|
149
|
+
arts_converged = [res_stage2.x[j*6:(j+1)*6] for j in range(num_joints)]
|
|
150
|
+
final_loss = float(res_stage2.fun)
|
|
151
|
+
else:
|
|
152
|
+
arts_converged = []
|
|
153
|
+
final_loss = float(res_stage1.fun)
|
|
154
|
+
|
|
155
|
+
elapsed = time.time() - t0
|
|
156
|
+
|
|
157
|
+
return {
|
|
158
|
+
'base_pose': base_converged,
|
|
159
|
+
'articulations': arts_converged,
|
|
160
|
+
'final_loss': final_loss,
|
|
161
|
+
'time_sec': elapsed,
|
|
162
|
+
'stage1_success': res_stage1.success
|
|
163
|
+
}
|
fvr/cli.py
ADDED
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
"""
|
|
2
|
+
Command-Line Interface (CLI) for Diff-FVR.
|
|
3
|
+
Provides:
|
|
4
|
+
- fvr-render: Ultra-fast DRR projection synthesis from 3D CT.
|
|
5
|
+
- fvr-register: Biplanar 3D-2D 6-DoF registration against intraoperative fluoroscopy.
|
|
6
|
+
"""
|
|
7
|
+
|
|
8
|
+
import argparse
|
|
9
|
+
import json
|
|
10
|
+
from pathlib import Path
|
|
11
|
+
import time
|
|
12
|
+
import numpy as np
|
|
13
|
+
from PIL import Image
|
|
14
|
+
from scipy.ndimage import zoom
|
|
15
|
+
|
|
16
|
+
from .dicom_loader import load_dicom_series, hu_to_attenuation
|
|
17
|
+
from .diff_fvr import DifferentiableFourierVolumeRenderer
|
|
18
|
+
from .core_engine import RobustDiffFVREngine
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
def load_volume(ct_path: Path, target_res: int = 96):
|
|
22
|
+
"""
|
|
23
|
+
Load CT from DICOM directory or numpy array, with isotropic cubic resampling.
|
|
24
|
+
"""
|
|
25
|
+
if ct_path.is_dir():
|
|
26
|
+
vol_hu, spacing = load_dicom_series(ct_path)
|
|
27
|
+
elif ct_path.suffix in ['.npy', '.npz']:
|
|
28
|
+
vol_hu = np.load(ct_path)
|
|
29
|
+
spacing = (1.0, 1.0, 1.0)
|
|
30
|
+
else:
|
|
31
|
+
raise ValueError(f"Unsupported CT path: {ct_path}. Must be a DICOM directory or .npy file.")
|
|
32
|
+
|
|
33
|
+
if target_res is not None and vol_hu.shape[1] != target_res:
|
|
34
|
+
target_xy = target_res
|
|
35
|
+
phys_z = vol_hu.shape[0] * spacing[0]
|
|
36
|
+
phys_x = vol_hu.shape[2] * spacing[2]
|
|
37
|
+
vox_size = phys_x / target_xy
|
|
38
|
+
target_z = int(round(phys_z / vox_size))
|
|
39
|
+
|
|
40
|
+
vol_iso = zoom(vol_hu, (target_z / vol_hu.shape[0], target_xy / vol_hu.shape[1], target_xy / vol_hu.shape[2]), order=1)
|
|
41
|
+
pad_z_before = (target_xy - target_z) // 2
|
|
42
|
+
pad_z_after = target_xy - target_z - pad_z_before
|
|
43
|
+
vol_cubic = np.pad(vol_iso, ((pad_z_before, pad_z_after), (0, 0), (0, 0)), mode='constant', constant_values=-1000.0)
|
|
44
|
+
else:
|
|
45
|
+
vol_cubic = vol_hu
|
|
46
|
+
|
|
47
|
+
return hu_to_attenuation(vol_cubic)
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
def main_render():
|
|
51
|
+
parser = argparse.ArgumentParser(description="Diff-FVR: Ultra-fast Digitally Reconstructed Radiograph (DRR) Renderer.")
|
|
52
|
+
parser.add_argument("--ct", type=str, required=True, help="Path to pre-op CT DICOM directory or .npy file.")
|
|
53
|
+
parser.add_argument("--yaw", type=float, default=0.0, help="Rotation yaw in degrees (default: 0.0).")
|
|
54
|
+
parser.add_argument("--pitch", type=float, default=0.0, help="Rotation pitch in degrees (default: 0.0).")
|
|
55
|
+
parser.add_argument("--roll", type=float, default=0.0, help="Rotation roll in degrees (default: 0.0).")
|
|
56
|
+
parser.add_argument("--tx", type=float, default=0.0, help="Translation Tx in mm (default: 0.0).")
|
|
57
|
+
parser.add_argument("--ty", type=float, default=0.0, help="Translation Ty in mm (default: 0.0).")
|
|
58
|
+
parser.add_argument("--tz", type=float, default=0.0, help="Translation Tz in mm (default: 0.0).")
|
|
59
|
+
parser.add_argument("--view", type=str, choices=['ap', 'lateral'], default='ap', help="View angle ('ap' or 'lateral').")
|
|
60
|
+
parser.add_argument("--res", type=int, default=128, help="Isotropic cubic grid dimension (default: 128).")
|
|
61
|
+
parser.add_argument("--out", type=str, default="drr_output.png", help="Output PNG path (default: drr_output.png).")
|
|
62
|
+
|
|
63
|
+
args = parser.parse_args()
|
|
64
|
+
|
|
65
|
+
ct_path = Path(args.ct)
|
|
66
|
+
if not ct_path.exists():
|
|
67
|
+
print(f"Error: CT path not found: {ct_path}")
|
|
68
|
+
return
|
|
69
|
+
|
|
70
|
+
print(f"Loading CT from: {ct_path} (resampling to {args.res}^3)...")
|
|
71
|
+
t0 = time.time()
|
|
72
|
+
vol_mu = load_volume(ct_path, target_res=args.res)
|
|
73
|
+
renderer = DifferentiableFourierVolumeRenderer(vol_mu, pad_factor=2.0)
|
|
74
|
+
print(f"Initialized 3D FFT volume in {time.time() - t0:.2f} s.")
|
|
75
|
+
|
|
76
|
+
t_render = time.perf_counter()
|
|
77
|
+
drr = renderer.render_drr(args.yaw, args.pitch, args.roll, tx=args.tx, ty=args.ty, view=args.view)
|
|
78
|
+
render_ms = (time.perf_counter() - t_render) * 1000.0
|
|
79
|
+
|
|
80
|
+
print(f"Rendered {args.view.upper()} DRR in {render_ms:.2f} ms ({1000.0 / render_ms:.1f} FPS)!")
|
|
81
|
+
|
|
82
|
+
# Normalize and save image
|
|
83
|
+
drr_norm = (drr - drr.min()) / (drr.max() - drr.min() + 1e-8)
|
|
84
|
+
img_uint8 = (drr_norm * 255.0).astype(np.uint8)
|
|
85
|
+
out_path = Path(args.out)
|
|
86
|
+
out_path.parent.mkdir(parents=True, exist_ok=True)
|
|
87
|
+
Image.fromarray(img_uint8).save(out_path)
|
|
88
|
+
print(f"Saved rendered DRR to: {out_path.resolve()}")
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
def main_register():
|
|
92
|
+
parser = argparse.ArgumentParser(description="Diff-FVR: Biplanar 3D-2D 6-DoF Rigid Spine Registration.")
|
|
93
|
+
parser.add_argument("--ct", type=str, required=True, help="Path to pre-op CT DICOM directory or .npy file.")
|
|
94
|
+
parser.add_argument("--ap", type=str, required=True, help="Path to target intraoperative AP X-ray (PNG/TIFF/NPY).")
|
|
95
|
+
parser.add_argument("--lat", type=str, required=True, help="Path to target intraoperative Lateral X-ray.")
|
|
96
|
+
parser.add_argument("--res", type=int, default=96, help="Isotropic CT grid dimension (default: 96).")
|
|
97
|
+
parser.add_argument("--max-iter", type=int, default=30, help="Maximum optimizer iterations (default: 30).")
|
|
98
|
+
parser.add_argument("--handle-metal", action="store_true", help="Enable automatic radio-opaque surgical metal saturation masking.")
|
|
99
|
+
parser.add_argument("--out", type=str, default="registration_result.json", help="Output JSON results path.")
|
|
100
|
+
|
|
101
|
+
args = parser.parse_args()
|
|
102
|
+
|
|
103
|
+
ct_path = Path(args.ct)
|
|
104
|
+
ap_path = Path(args.ap)
|
|
105
|
+
lat_path = Path(args.lat)
|
|
106
|
+
|
|
107
|
+
if not ct_path.exists() or not ap_path.exists() or not lat_path.exists():
|
|
108
|
+
print(f"Error: One or more input paths do not exist:\n CT: {ct_path}\n AP: {ap_path}\n LAT: {lat_path}")
|
|
109
|
+
return
|
|
110
|
+
|
|
111
|
+
# Load targets
|
|
112
|
+
def load_img(p: Path):
|
|
113
|
+
if p.suffix == '.npy':
|
|
114
|
+
return np.load(p)
|
|
115
|
+
im = Image.open(p).convert('L')
|
|
116
|
+
return np.array(im, dtype=np.float64)
|
|
117
|
+
|
|
118
|
+
targ_ap = load_img(ap_path)
|
|
119
|
+
targ_lat = load_img(lat_path)
|
|
120
|
+
|
|
121
|
+
print(f"Loading and initializing CT from: {ct_path}...")
|
|
122
|
+
vol_mu = load_volume(ct_path, target_res=args.res)
|
|
123
|
+
engine = RobustDiffFVREngine(vol_mu, pad_factor=2.0)
|
|
124
|
+
|
|
125
|
+
print(f"Running Biplanar 6-DoF Registration (handle_metal={args.handle_metal}, max_iter={args.max_iter})...")
|
|
126
|
+
res = engine.register_biplanar_robust(
|
|
127
|
+
target_ap=targ_ap,
|
|
128
|
+
target_lat=targ_lat,
|
|
129
|
+
handle_metal=args.handle_metal,
|
|
130
|
+
max_iter=args.max_iter
|
|
131
|
+
)
|
|
132
|
+
|
|
133
|
+
pose = res['final_pose']
|
|
134
|
+
print(f"\nREGISTRATION CONVERGED in {res['time_sec']:.2f} s ({res['evaluations']} evaluations):")
|
|
135
|
+
print(f" Rotation: Yaw={pose[0]:.3f}°, Pitch={pose[1]:.3f}°, Roll={pose[2]:.3f}°")
|
|
136
|
+
print(f" Translation: Tx={pose[3]:.3f} mm, Ty={pose[4]:.3f} mm, Tz={pose[5]:.3f} mm")
|
|
137
|
+
print(f" Final Loss: {res['final_loss']:.6f}")
|
|
138
|
+
|
|
139
|
+
out_data = {
|
|
140
|
+
'yaw_deg': float(pose[0]),
|
|
141
|
+
'pitch_deg': float(pose[1]),
|
|
142
|
+
'roll_deg': float(pose[2]),
|
|
143
|
+
'tx_mm': float(pose[3]),
|
|
144
|
+
'ty_mm': float(pose[4]),
|
|
145
|
+
'tz_mm': float(pose[5]),
|
|
146
|
+
'final_loss': float(res['final_loss']),
|
|
147
|
+
'time_sec': float(res['time_sec']),
|
|
148
|
+
'evaluations': int(res['evaluations']),
|
|
149
|
+
'metal_detected': bool(res['metal_detected'])
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
out_path = Path(args.out)
|
|
153
|
+
out_path.parent.mkdir(parents=True, exist_ok=True)
|
|
154
|
+
with open(out_path, 'w', encoding='utf-8') as f:
|
|
155
|
+
json.dump(out_data, f, indent=2)
|
|
156
|
+
print(f"Saved results to: {out_path.resolve()}")
|
|
157
|
+
|
|
158
|
+
|
|
159
|
+
if __name__ == '__main__':
|
|
160
|
+
main_render()
|