cdi-stools 0.2.2__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cdi_stools-0.3.0/CHANGELOG.md +91 -0
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/PKG-INFO +2 -2
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/README.md +279 -279
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/pyproject.toml +1 -1
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/__init__.py +1 -1
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/__main__.py +4 -5
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/bcdi_core.py +706 -668
- cdi_stools-0.3.0/src/cdi_st/bcdi_gui.py +503 -0
- cdi_stools-0.3.0/src/cdi_st/facets.py +82 -0
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_autophase_infer.py +4 -133
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_autophase_model.py +265 -405
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_autophase_train.py +369 -409
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_data_generator.py +349 -440
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_dataset.py +0 -88
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_demo.py +294 -334
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_experimental_loader.py +0 -572
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_gui_tabs.py +0 -27
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_phase_model.py +3 -129
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_phase_retrieval.py +505 -720
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_train.py +303 -354
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/nn_visualize.py +329 -378
- cdi_stools-0.3.0/src/cdi_st/ortho.py +104 -0
- cdi_stools-0.3.0/src/cdi_st/recon.py +198 -0
- cdi_stools-0.3.0/src/cdi_st/trainlog.py +146 -0
- cdi_stools-0.3.0/src/cdi_st/viz3d.py +102 -0
- cdi_stools-0.2.2/.gitignore +0 -45
- cdi_stools-0.2.2/CHANGELOG.md +0 -122
- cdi_stools-0.2.2/src/cdi_st/bcdi_gui.py +0 -937
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/LICENSE +0 -0
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/CDI_ST_logo.png +0 -0
- {cdi_stools-0.2.2 → cdi_stools-0.3.0}/src/cdi_st/requirements.txt +0 -0
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## [0.3.0] - 2026-11-08
|
|
4
|
+
|
|
5
|
+
Faceted particles, photon-budget noise, detector/sample space, standalone phase
|
|
6
|
+
retrieval and training diagnostics.
|
|
7
|
+
|
|
8
|
+
With thanks to **Dr. Gerards Hinsley**, whose review of v0.2.2 prompted most of
|
|
9
|
+
the changes in this release: the facet options, the photon-count control, the
|
|
10
|
+
detector-space toggle, the standalone reconstruction path, the training log, and
|
|
11
|
+
the corrections to the noise model and colour-scale controls.
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
**Simulation**
|
|
16
|
+
- Crystallographic facet shapes built from a Wulff half-space construction
|
|
17
|
+
(`facets.py`): octahedron, cuboctahedron, truncated octahedron, truncated cube,
|
|
18
|
+
rhombic dodecahedron, tunable Wulff, and hexagonal prism / bipyramid /
|
|
19
|
+
truncated bipyramid. Facet normals come from the reciprocal lattice, so the
|
|
20
|
+
construction is correct for every crystal system.
|
|
21
|
+
- Facet multiplicities control: a ratio of facet distances, off by default,
|
|
22
|
+
enabled by a tick box, with a plain-language tooltip giving the shape each
|
|
23
|
+
value produces.
|
|
24
|
+
- Particle size entered directly in nm; supercell multiples follow from the
|
|
25
|
+
lattice vectors. Cell limit raised from 2500 to 100000 per axis, with a size
|
|
26
|
+
notice above 2M atoms and a confirmation above 10M.
|
|
27
|
+
- Photon budget: the pattern is scaled to a chosen number of photons before
|
|
28
|
+
Poisson sampling, so the counting statistics mean something. The field accepts
|
|
29
|
+
scientific notation (1e9, 2.5e12).
|
|
30
|
+
- Detector / sample space toggle. `ortho.det_matrix` builds the real
|
|
31
|
+
(dq/dpx, dq/dpy, dq/drocking) transform; the Results tab switches between the
|
|
32
|
+
orthogonal lab q grid and the raw skewed detector grid and reports the skew.
|
|
33
|
+
- Custom 2theta with a live readout: the offset from the nearest reflection, the
|
|
34
|
+
resulting dq, and the peak shift in both simulation voxels and real detector
|
|
35
|
+
pixels, with a warning when the peak leaves the frame.
|
|
36
|
+
- Prismatic dislocation loop alongside the existing line dislocations.
|
|
37
|
+
- Separate simulation grid control, independent of the detector panel size.
|
|
38
|
+
- Visualization controls (bonds, strain, dislocation) grouped together.
|
|
39
|
+
|
|
40
|
+
**Data processing**
|
|
41
|
+
- `recon.py` — standalone ER / HIO / RAAR phase retrieval with shrinkwrap,
|
|
42
|
+
bad-pixel masking, chi-squared history, centring, phase-ramp removal and PRTF.
|
|
43
|
+
No network needed.
|
|
44
|
+
- `recon.load_recon` — reads reconstructions from `.npz`, `.npy`, `.h5`, `.cxi`,
|
|
45
|
+
`.nxs` and `.mat`, including PyNX `/entry_1/image_1/data` and amplitude/phase
|
|
46
|
+
pairs, so work reconstructed outside CDI-St can be loaded.
|
|
47
|
+
- `trainlog.py` — per-step training log (`.log` + `.jsonl`) that flags NaN/Inf
|
|
48
|
+
losses, non-finite gradients and weights, and loss spikes, then dumps input
|
|
49
|
+
statistics and the likely cause. `scan_dataset()` pre-flights a training set.
|
|
50
|
+
- `viz3d.py` — 14 colormaps including Turbo; clip plane with a filled cut face
|
|
51
|
+
showing the data on the cut layer; explicit colour-scale scope (whole volume /
|
|
52
|
+
rendered shell / current slice) with a label stating which was used.
|
|
53
|
+
|
|
54
|
+
### Changed
|
|
55
|
+
|
|
56
|
+
- Read noise is applied only to integrating detectors. Hybrid photon-counting
|
|
57
|
+
detectors (EIGER, PILATUS, MAXIPIX, MERLIN, TIMEPIX) have Poisson statistics
|
|
58
|
+
only, and the Readout option disables itself when one is selected. A
|
|
59
|
+
`CCD (integrating)` preset was added for the case where read noise applies.
|
|
60
|
+
- Noise controls are named per effect (Poisson / Readout / Air / Dead px /
|
|
61
|
+
Hot px) instead of a single "Add noise", and all are off by default, so the
|
|
62
|
+
ideal pattern is what you see until you ask for noise.
|
|
63
|
+
- Default simulation grid raised from 128 to 192, giving 2.25x the pixels in the
|
|
64
|
+
2D slices for about 2 s per simulation.
|
|
65
|
+
- The simulation panel is resizable; the fixed 520 px left column is a splitter.
|
|
66
|
+
- Frame transform uses cubic interpolation (0.7% round-trip error, against 3.6%
|
|
67
|
+
for linear).
|
|
68
|
+
|
|
69
|
+
### Fixed
|
|
70
|
+
|
|
71
|
+
- Poisson noise was on by default and applied the photon budget unconditionally,
|
|
72
|
+
which quantised 83% of the volume to zero counts and erased the weak fringes
|
|
73
|
+
in the 3D peak and the 2D slices. Noise is now opt-in.
|
|
74
|
+
- The simulation grid was capped by the detector panel size, so a 128 x 128
|
|
75
|
+
panel forced 128^3 slices regardless of the grid setting. Panel geometry
|
|
76
|
+
(exported as metadata) and simulation grid are now independent.
|
|
77
|
+
- Reflections are labelled by their canonical Friedel mate, so selecting (1 1 1)
|
|
78
|
+
no longer displays (-1 -1 -1). Both label the same physical peak; the hkl
|
|
79
|
+
search simply reached the negative representative first. The raw index is kept
|
|
80
|
+
as `hkl_raw`.
|
|
81
|
+
- `select_reflection` returns full-precision d, 2theta, q, |F|^2 and LP. It
|
|
82
|
+
previously used the four-decimal values rounded for the table.
|
|
83
|
+
- Half-voxel registration error in the detector/lab transform. An orthogonal
|
|
84
|
+
transform is now an exact no-op for any q-grid convention.
|
|
85
|
+
- Shrinkwrap could inflate the support until the algorithm fit the noise (89% of
|
|
86
|
+
the array at 1e5 counts). An absolute cap keeps reconstructions stable from
|
|
87
|
+
1e3 to 1e8 photons.
|
|
88
|
+
- Selecting a large detector preset set the transform size to the full panel:
|
|
89
|
+
Eiger2 4M attempted a 2068^3 complex FFT, about 140 GB.
|
|
90
|
+
- The facet ratio control was not read when building, and the large-supercell
|
|
91
|
+
notice was missing.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
Metadata-Version: 2.
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
2
|
Name: cdi-stools
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.0
|
|
4
4
|
Summary: Coherent Diffraction Imaging Simulation Tools — BCDI simulation and NN-based phase retrieval
|
|
5
5
|
Project-URL: Homepage, https://github.com/SAIDIsoufiane/CDI-St
|
|
6
6
|
Project-URL: Documentation, https://github.com/SAIDIsoufiane/CDI-St#readme
|
|
@@ -1,279 +1,279 @@
|
|
|
1
|
-
<div align="center">
|
|
2
|
-
|
|
3
|
-
<img src="docs/screenshots/CDI_ST_logo.png" alt="CDI-ST" width="600"/>
|
|
4
|
-
|
|
5
|
-
# CDI-ST
|
|
6
|
-
|
|
7
|
-
**Coherent Diffraction Imaging — Simulation Tools**
|
|
8
|
-
|
|
9
|
-
An end-to-end Bragg Coherent Diffraction Imaging (BCDI) Python application using low computational resources with an easy to learn GUI:
|
|
10
|
-
crystal lattice construction, X-ray scattering simulation, neural-network phase retrieval,
|
|
11
|
-
and 3D visualization, in a single GUI.
|
|
12
|
-
|
|
13
|
-
[](https://pypi.org/project/cdi-stools/)
|
|
14
|
-
[](https://pypi.org/project/cdi-stools/)
|
|
15
|
-
[](https://opensource.org/licenses/MIT)
|
|
16
|
-
[]()
|
|
17
|
-
|
|
18
|
-
</div>
|
|
19
|
-
|
|
20
|
-
## Screenshots
|
|
21
|
-
|
|
22
|
-
<table>
|
|
23
|
-
<tr>
|
|
24
|
-
<td><img src="docs/screenshots/launcher.png" width="280"/></td>
|
|
25
|
-
<td><img src="docs/screenshots/material_tab.png" width="280"/></td>
|
|
26
|
-
<td><img src="docs/screenshots/results.png" width="280"/></td>
|
|
27
|
-
</tr>
|
|
28
|
-
<tr>
|
|
29
|
-
<td align="center"><sub>Launcher</sub></td>
|
|
30
|
-
<td align="center"><sub>Material — crystal builder</sub></td>
|
|
31
|
-
<td align="center"><sub>Results — simulated Bragg peak</sub></td>
|
|
32
|
-
</tr>
|
|
33
|
-
<tr>
|
|
34
|
-
<td><img src="docs/screenshots/3d_viewer.png" width="280"/></td>
|
|
35
|
-
<td><img src="docs/screenshots/strain.png" width="280"/></td>
|
|
36
|
-
<td><img src="docs/screenshots/reconstruction.png" width="280"/></td>
|
|
37
|
-
</tr>
|
|
38
|
-
<tr>
|
|
39
|
-
<td align="center"><sub>3D Viewer — diffraction simulation</sub></td>
|
|
40
|
-
<td align="center"><sub>2D reconstruction</sub></td>
|
|
41
|
-
<td align="center"><sub>3D Viewer — reconstruction</sub></td>
|
|
42
|
-
</tr>
|
|
43
|
-
</table>
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
---
|
|
47
|
-
|
|
48
|
-
## Installation
|
|
49
|
-
|
|
50
|
-
```bash
|
|
51
|
-
pip install cdi-stools
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
After installation, launch the GUI from any terminal:
|
|
55
|
-
|
|
56
|
-
```bash
|
|
57
|
-
cdi-st or cdi-stools
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
**Support for other beamlines files is coming in future updates...**
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
> **Note**: `cdi-st` ships a Qt-based GUI. On Linux, you may need system Qt
|
|
64
|
-
> dependencies (`sudo apt install libxcb-cursor0 libxkbcommon-x11-0` on Ubuntu/Debian).
|
|
65
|
-
> On Windows and macOS the wheel includes everything.
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## What is BCDI?
|
|
70
|
-
|
|
71
|
-
Bragg Coherent Diffraction Imaging is a synchrotron technique that reconstructs the
|
|
72
|
-
3D shape and atomic-displacement field of nanocrystals by inverting the diffraction
|
|
73
|
-
intensity around a Bragg peak. The intensity gives only |F(q)|², so reconstruction
|
|
74
|
-
must recover the missing phase by iterative algorithms (HIO, RAAR, ER),
|
|
75
|
-
more recently by neural networks.
|
|
76
|
-
|
|
77
|
-
**CDI-ST integrates the entire workflow into one tool:**
|
|
78
|
-
|
|
79
|
-
| Step | What you do | Where in the GUI |
|
|
80
|
-
|---|---|---|
|
|
81
|
-
| 1. Build a crystal | Define material, shape, orientation, optional defects | **BCDI Simulation → Material** |
|
|
82
|
-
| 2. Set the X-ray geometry | Beam energy, detector geometry, oversampling | **BCDI Simulation → Beam** |
|
|
83
|
-
| 3. Simulate diffraction | Compute the 3D Bragg peak | **BCDI Simulation → Results** |
|
|
84
|
-
| 4. Generate training data | Build a labeled dataset of (object, diffraction) pairs | **BCDI Data Analysis → Generate Data** |
|
|
85
|
-
| 5. Train an unsupervised NN | AutoPhase_NN: physics-driven, no ground truth | **BCDI Data Analysis → AutoPhase_NN** |
|
|
86
|
-
| 6. Train a supervised NN | CDI_NN: PhaseUNet3D learning the phase mapping | **BCDI Data Analysis → CDI_NN** |
|
|
87
|
-
| 7. Reconstruct | NN → optionally refine with HIO/RAAR/ER | **BCDI Data Analysis → Reconstruction** |
|
|
88
|
-
| 8. Visualize and export | Interactive 3D viewer, ParaView VTI export | **BCDI Data Analysis → 3D Viewer** |
|
|
89
|
-
|
|
90
|
-
---
|
|
91
|
-
|
|
92
|
-
## The simulation pipeline
|
|
93
|
-
|
|
94
|
-
### 1. Material — build the crystal
|
|
95
|
-
|
|
96
|
-
Choose from preset materials (Si, Ge, Pt, Au, SiC, GaAs, ...) or import a custom
|
|
97
|
-
CIF file. The crystal is built atom-by-atom from the unit cell, replicated to the
|
|
98
|
-
requested supercell size.
|
|
99
|
-
|
|
100
|
-
You can:
|
|
101
|
-
- Select the **particle shape**: cube, sphere, cylinder, hexagonal prism, octahedron, dodecahedron
|
|
102
|
-
- Set the **target oversampling ratio** (controls how much of the diffraction
|
|
103
|
-
volume the object occupies)
|
|
104
|
-
- Add **strain fields**: radial gradient, edge dislocation, random
|
|
105
|
-
- Add **dislocations**: edge / screw / mixed, at a configurable position with
|
|
106
|
-
an arbitrary Burgers vector
|
|
107
|
-
|
|
108
|
-
The 3D lattice view (powered by Plotly) shows every atom in the supercell.
|
|
109
|
-
|
|
110
|
-
### 2. Beam — define the X-ray geometry
|
|
111
|
-
|
|
112
|
-
Select a reflection (h k l) from the auto-computed list of allowed peaks.
|
|
113
|
-
The Beam tab also lets you set:
|
|
114
|
-
- **Beam energy** (and the resulting wavelength)
|
|
115
|
-
- **Detector geometry**: pixel pitch, sample-to-detector distance, detector size
|
|
116
|
-
- **Beam size** in microns
|
|
117
|
-
- **Detector preset**: Maxipix (516×516), Eiger 2M...etc, custom rectangular
|
|
118
|
-
|
|
119
|
-
Detector preset auto-fills the geometry parameters.
|
|
120
|
-
|
|
121
|
-
### 3. Results — run the simulation
|
|
122
|
-
|
|
123
|
-
CDI-ST computes the diffraction in **two paths**:
|
|
124
|
-
|
|
125
|
-
- **Analytical path** (cube/rectangular shapes, no strain): closed-form Fourier transform of the support
|
|
126
|
-
- **FFT path** (any shape, any strain field): full 3D FFT of the complex object
|
|
127
|
-
|
|
128
|
-
Output: 3D diffraction volume `|F(q)|²`, displayed as 2D slices (qx-qy, qx-qz)
|
|
129
|
-
and in an interactive 3D Plotly viewer with iso-surface controls.
|
|
130
|
-
|
|
131
|
-
You can export the simulation as a `.npz` file ready for the data-analysis side,
|
|
132
|
-
or for use as ground truth when training NNs.
|
|
133
|
-
|
|
134
|
-
---
|
|
135
|
-
|
|
136
|
-
## The data-analysis pipeline
|
|
137
|
-
|
|
138
|
-
### 4. Generate Data — create training samples
|
|
139
|
-
|
|
140
|
-
Generates a dataset of synthetic BCDI samples for NN training. Each sample
|
|
141
|
-
contains the ground-truth phase, support, and the simulated diffraction
|
|
142
|
-
intensity. You control:
|
|
143
|
-
|
|
144
|
-
- **Number of samples** (typically 2,000–10,000)
|
|
145
|
-
- **Grid size** (64³ recommended for training, 128³ for production inference)
|
|
146
|
-
- **Materials** (random or fixed)
|
|
147
|
-
- **Particle size variation** (random supercell multiplier)
|
|
148
|
-
- **Random dislocations** adds occasional edge/screw/mixed dislocations
|
|
149
|
-
- **Random strain** adds occasional radial/edge/random strain fields
|
|
150
|
-
|
|
151
|
-
Samples save as `sample_XXXXX.npz` and a `manifest.json`. Generation is
|
|
152
|
-
parallelizable and resumable.
|
|
153
|
-
|
|
154
|
-
### 5. AutoPhase_NN — unsupervised physics-driven NN
|
|
155
|
-
|
|
156
|
-
AutoPhase_NN is a 3D dual-decoder convolutional network that learns to invert the
|
|
157
|
-
Fourier modulus constraint **without ground truth**. It predicts the object's
|
|
158
|
-
amplitude and phase such that `|FFT(amp · exp(iφ))|` matches the measured
|
|
159
|
-
magnitude. The loss is a normalized-magnitude error plus optional regularizers
|
|
160
|
-
on phase smoothness and support compactness. This is the network from (Yao, Yudong,
|
|
161
|
-
et al. npj Computational Materials 8.1 (2022) ) reimplemented for this codebase.
|
|
162
|
-
|
|
163
|
-
Training is GPU-accelerated (CUDA preferred) and converges in a few dozen
|
|
164
|
-
epochs on 2,000+ samples. The best checkpoint is saved automatically.
|
|
165
|
-
|
|
166
|
-
### 6. CDI_NN — supervised PhaseUNet3D
|
|
167
|
-
|
|
168
|
-
A 3D U-Net that predicts the phase field from the diffraction magnitude, trained
|
|
169
|
-
against the ground-truth phase from the data generator.
|
|
170
|
-
|
|
171
|
-
Use AutoPhase_NN when you don't trust your simulator's ground truth (or have
|
|
172
|
-
none). Use CDI_NN when you have a high-fidelity simulator and want a direct
|
|
173
|
-
mapping. Use both via the **ensemble** mode in the Reconstruction tab.
|
|
174
|
-
|
|
175
|
-
### 7. Reconstruction — invert experimental or simulated data
|
|
176
|
-
|
|
177
|
-
Load any `.npz` (simulated) or `.h5` (experimental) BCDI file, or convert a
|
|
178
|
-
SPEC + EDF scan from ID01 directly via the dialog. Choose:
|
|
179
|
-
|
|
180
|
-
- **NN-only (Autophase_NN)**: forward pass, twin-image suppression, COM centering
|
|
181
|
-
- **Refined**: NN seed → HIO + RAAR + ER iterative phase retrieval
|
|
182
|
-
- **Ensemble**: AutoPhase_NN + CDI_NN combined, optionally refined
|
|
183
|
-
- **Compare**: runs both modes side-by-side
|
|
184
|
-
|
|
185
|
-
The output is a complex object (amplitude + phase), a binary support, the
|
|
186
|
-
R-factor convergence curve, and 2D slices at the COM of the reconstruction.
|
|
187
|
-
|
|
188
|
-
For experimental data, the loader handles:
|
|
189
|
-
- Detector chip-gap masking (Maxipix, Eiger)
|
|
190
|
-
- Hot-pixel removal
|
|
191
|
-
- Beamstop / direct-beam streak removal
|
|
192
|
-
- Bragg-peak-centered cropping
|
|
193
|
-
- Optional q-space orthogonalization via xrayutilities (with `[id01]` extra)
|
|
194
|
-
|
|
195
|
-
### 8. 3D Viewer — interactive visualization
|
|
196
|
-
|
|
197
|
-
The viewer renders the reconstruction in three modes:
|
|
198
|
-
|
|
199
|
-
- **Point cloud**: every voxel above threshold, colored by amplitude / phase / strain,
|
|
200
|
-
with per-voxel opacity weighted by intensity
|
|
201
|
-
- **Isosurface**: smoothed iso-shell with internal data points showing through
|
|
202
|
-
- **Surface only**: bare iso-mesh with edges visible
|
|
203
|
-
|
|
204
|
-
Controls:
|
|
205
|
-
- Threshold (% of peak amplitude), voxel size, opacity, colormap
|
|
206
|
-
- Color scale auto / manual (`vmin` / `vmax`)
|
|
207
|
-
- Clip plane on any axis
|
|
208
|
-
- Mouse: left-drag rotate, right-drag pan, wheel zoom, double-click reset
|
|
209
|
-
- Export PNG snapshot or VTI (ParaView)
|
|
210
|
-
|
|
211
|
-
---
|
|
212
|
-
|
|
213
|
-
## Project structure
|
|
214
|
-
|
|
215
|
-
```
|
|
216
|
-
cdi-st/
|
|
217
|
-
├── src/cdi_st/
|
|
218
|
-
│ ├── __init__.py
|
|
219
|
-
│ ├── __main__.py # python -m cdi_st
|
|
220
|
-
│ ├── bcdi_gui.py # main entry point + launcher + splash
|
|
221
|
-
│ ├── bcdi_core.py # crystal builder, simulator, reflection calculator
|
|
222
|
-
│ ├── nn_gui_tabs.py # all NN-related GUI tabs
|
|
223
|
-
│ ├── nn_data_generator.py # training-data generator
|
|
224
|
-
│ ├── nn_dataset.py # PyTorch Dataset wrapper
|
|
225
|
-
│ ├── nn_phase_model.py # PhaseUNet3D (supervised)
|
|
226
|
-
│ ├── nn_autophase_model.py # AutoPhaseNet3D (unsupervised)
|
|
227
|
-
│ ├── nn_train.py # supervised training CLI
|
|
228
|
-
│ ├── nn_autophase_train.py # unsupervised training CLI
|
|
229
|
-
│ ├── nn_phase_retrieval.py # HIO / RAAR / ER iterative algorithms
|
|
230
|
-
│ ├── nn_autophase_infer.py # inference: nn_only / refined / ensemble
|
|
231
|
-
│ ├── nn_experimental_loader.py # h5/spec/edf loading + preprocessing
|
|
232
|
-
│ ├── nn_demo.py
|
|
233
|
-
│ ├── nn_visualize.py
|
|
234
|
-
│ └── CDI_ST_logo.png
|
|
235
|
-
├── tests/
|
|
236
|
-
├── docs/
|
|
237
|
-
├── pyproject.toml
|
|
238
|
-
├── README.md
|
|
239
|
-
├── LICENSE
|
|
240
|
-
└── CHANGELOG.md
|
|
241
|
-
```
|
|
242
|
-
|
|
243
|
-
---
|
|
244
|
-
|
|
245
|
-
## Citing
|
|
246
|
-
|
|
247
|
-
If you use CDI-ST in published research, please cite:
|
|
248
|
-
|
|
249
|
-
```bibtex
|
|
250
|
-
@software{saidi_cdi_st_2026,
|
|
251
|
-
author = {Saidi, Soufiane},
|
|
252
|
-
title = {CDI-ST: Coherent Diffraction Imaging — Simulation Tools},
|
|
253
|
-
year = {2026},
|
|
254
|
-
publisher = {GitHub},
|
|
255
|
-
url = {https://github.com/Refze/CDI-St}
|
|
256
|
-
}
|
|
257
|
-
```
|
|
258
|
-
|
|
259
|
-
---
|
|
260
|
-
|
|
261
|
-
## License
|
|
262
|
-
|
|
263
|
-
MIT — see [LICENSE](LICENSE).
|
|
264
|
-
|
|
265
|
-
## Author
|
|
266
|
-
|
|
267
|
-
**Soufiane SAIDI** — saidisoufiane@hotmail.com
|
|
268
|
-
|
|
269
|
-
We heavily encourage you to use the in-app **Reports & Suggestions** button to send any type of feedback.
|
|
270
|
-
|
|
271
|
-
---
|
|
272
|
-
|
|
273
|
-
## Acknowledgments
|
|
274
|
-
|
|
275
|
-
Built on the shoulders of the BCDI and python community:
|
|
276
|
-
[PyNX](https://pynx.esrf.fr/), [BCDI-Utilities](https://github.com/carnisj/bcdi),
|
|
277
|
-
[cdiutils](https://github.com/clatlan/cdiutils), [Atomsk](https://atomsk.univ-lille.fr/) , [xrayutilities](https://xrayutilities.sourceforge.io/),
|
|
278
|
-
[scikit-image](https://scikit-image.org/), [PyQt6](https://www.riverbankcomputing.com/software/pyqt/),
|
|
279
|
-
and the AutoPhaseNN paper by Yao et al. (2022).
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
<img src="docs/screenshots/CDI_ST_logo.png" alt="CDI-ST" width="600"/>
|
|
4
|
+
|
|
5
|
+
# CDI-ST
|
|
6
|
+
|
|
7
|
+
**Coherent Diffraction Imaging — Simulation Tools**
|
|
8
|
+
|
|
9
|
+
An end-to-end Bragg Coherent Diffraction Imaging (BCDI) Python application using low computational resources with an easy to learn GUI:
|
|
10
|
+
crystal lattice construction, X-ray scattering simulation, neural-network phase retrieval,
|
|
11
|
+
and 3D visualization, in a single GUI.
|
|
12
|
+
|
|
13
|
+
[](https://pypi.org/project/cdi-stools/)
|
|
14
|
+
[](https://pypi.org/project/cdi-stools/)
|
|
15
|
+
[](https://opensource.org/licenses/MIT)
|
|
16
|
+
[]()
|
|
17
|
+
|
|
18
|
+
</div>
|
|
19
|
+
|
|
20
|
+
## Screenshots
|
|
21
|
+
|
|
22
|
+
<table>
|
|
23
|
+
<tr>
|
|
24
|
+
<td><img src="docs/screenshots/launcher.png" width="280"/></td>
|
|
25
|
+
<td><img src="docs/screenshots/material_tab.png" width="280"/></td>
|
|
26
|
+
<td><img src="docs/screenshots/results.png" width="280"/></td>
|
|
27
|
+
</tr>
|
|
28
|
+
<tr>
|
|
29
|
+
<td align="center"><sub>Launcher</sub></td>
|
|
30
|
+
<td align="center"><sub>Material — crystal builder</sub></td>
|
|
31
|
+
<td align="center"><sub>Results — simulated Bragg peak</sub></td>
|
|
32
|
+
</tr>
|
|
33
|
+
<tr>
|
|
34
|
+
<td><img src="docs/screenshots/3d_viewer.png" width="280"/></td>
|
|
35
|
+
<td><img src="docs/screenshots/strain.png" width="280"/></td>
|
|
36
|
+
<td><img src="docs/screenshots/reconstruction.png" width="280"/></td>
|
|
37
|
+
</tr>
|
|
38
|
+
<tr>
|
|
39
|
+
<td align="center"><sub>3D Viewer — diffraction simulation</sub></td>
|
|
40
|
+
<td align="center"><sub>2D reconstruction</sub></td>
|
|
41
|
+
<td align="center"><sub>3D Viewer — reconstruction</sub></td>
|
|
42
|
+
</tr>
|
|
43
|
+
</table>
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Installation
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
pip install cdi-stools
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
After installation, launch the GUI from any terminal:
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
cdi-st or cdi-stools
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
**Support for other beamlines files is coming in future updates...**
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
> **Note**: `cdi-st` ships a Qt-based GUI. On Linux, you may need system Qt
|
|
64
|
+
> dependencies (`sudo apt install libxcb-cursor0 libxkbcommon-x11-0` on Ubuntu/Debian).
|
|
65
|
+
> On Windows and macOS the wheel includes everything.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## What is BCDI?
|
|
70
|
+
|
|
71
|
+
Bragg Coherent Diffraction Imaging is a synchrotron technique that reconstructs the
|
|
72
|
+
3D shape and atomic-displacement field of nanocrystals by inverting the diffraction
|
|
73
|
+
intensity around a Bragg peak. The intensity gives only |F(q)|², so reconstruction
|
|
74
|
+
must recover the missing phase by iterative algorithms (HIO, RAAR, ER),
|
|
75
|
+
more recently by neural networks.
|
|
76
|
+
|
|
77
|
+
**CDI-ST integrates the entire workflow into one tool:**
|
|
78
|
+
|
|
79
|
+
| Step | What you do | Where in the GUI |
|
|
80
|
+
|---|---|---|
|
|
81
|
+
| 1. Build a crystal | Define material, shape, orientation, optional defects | **BCDI Simulation → Material** |
|
|
82
|
+
| 2. Set the X-ray geometry | Beam energy, detector geometry, oversampling | **BCDI Simulation → Beam** |
|
|
83
|
+
| 3. Simulate diffraction | Compute the 3D Bragg peak | **BCDI Simulation → Results** |
|
|
84
|
+
| 4. Generate training data | Build a labeled dataset of (object, diffraction) pairs | **BCDI Data Analysis → Generate Data** |
|
|
85
|
+
| 5. Train an unsupervised NN | AutoPhase_NN: physics-driven, no ground truth | **BCDI Data Analysis → AutoPhase_NN** |
|
|
86
|
+
| 6. Train a supervised NN | CDI_NN: PhaseUNet3D learning the phase mapping | **BCDI Data Analysis → CDI_NN** |
|
|
87
|
+
| 7. Reconstruct | NN → optionally refine with HIO/RAAR/ER | **BCDI Data Analysis → Reconstruction** |
|
|
88
|
+
| 8. Visualize and export | Interactive 3D viewer, ParaView VTI export | **BCDI Data Analysis → 3D Viewer** |
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## The simulation pipeline
|
|
93
|
+
|
|
94
|
+
### 1. Material — build the crystal
|
|
95
|
+
|
|
96
|
+
Choose from preset materials (Si, Ge, Pt, Au, SiC, GaAs, ...) or import a custom
|
|
97
|
+
CIF file. The crystal is built atom-by-atom from the unit cell, replicated to the
|
|
98
|
+
requested supercell size.
|
|
99
|
+
|
|
100
|
+
You can:
|
|
101
|
+
- Select the **particle shape**: cube, sphere, cylinder, hexagonal prism, octahedron, dodecahedron
|
|
102
|
+
- Set the **target oversampling ratio** (controls how much of the diffraction
|
|
103
|
+
volume the object occupies)
|
|
104
|
+
- Add **strain fields**: radial gradient, edge dislocation, random
|
|
105
|
+
- Add **dislocations**: edge / screw / mixed, at a configurable position with
|
|
106
|
+
an arbitrary Burgers vector
|
|
107
|
+
|
|
108
|
+
The 3D lattice view (powered by Plotly) shows every atom in the supercell.
|
|
109
|
+
|
|
110
|
+
### 2. Beam — define the X-ray geometry
|
|
111
|
+
|
|
112
|
+
Select a reflection (h k l) from the auto-computed list of allowed peaks.
|
|
113
|
+
The Beam tab also lets you set:
|
|
114
|
+
- **Beam energy** (and the resulting wavelength)
|
|
115
|
+
- **Detector geometry**: pixel pitch, sample-to-detector distance, detector size
|
|
116
|
+
- **Beam size** in microns
|
|
117
|
+
- **Detector preset**: Maxipix (516×516), Eiger 2M...etc, custom rectangular
|
|
118
|
+
|
|
119
|
+
Detector preset auto-fills the geometry parameters.
|
|
120
|
+
|
|
121
|
+
### 3. Results — run the simulation
|
|
122
|
+
|
|
123
|
+
CDI-ST computes the diffraction in **two paths**:
|
|
124
|
+
|
|
125
|
+
- **Analytical path** (cube/rectangular shapes, no strain): closed-form Fourier transform of the support
|
|
126
|
+
- **FFT path** (any shape, any strain field): full 3D FFT of the complex object
|
|
127
|
+
|
|
128
|
+
Output: 3D diffraction volume `|F(q)|²`, displayed as 2D slices (qx-qy, qx-qz)
|
|
129
|
+
and in an interactive 3D Plotly viewer with iso-surface controls.
|
|
130
|
+
|
|
131
|
+
You can export the simulation as a `.npz` file ready for the data-analysis side,
|
|
132
|
+
or for use as ground truth when training NNs.
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## The data-analysis pipeline
|
|
137
|
+
|
|
138
|
+
### 4. Generate Data — create training samples
|
|
139
|
+
|
|
140
|
+
Generates a dataset of synthetic BCDI samples for NN training. Each sample
|
|
141
|
+
contains the ground-truth phase, support, and the simulated diffraction
|
|
142
|
+
intensity. You control:
|
|
143
|
+
|
|
144
|
+
- **Number of samples** (typically 2,000–10,000)
|
|
145
|
+
- **Grid size** (64³ recommended for training, 128³ for production inference)
|
|
146
|
+
- **Materials** (random or fixed)
|
|
147
|
+
- **Particle size variation** (random supercell multiplier)
|
|
148
|
+
- **Random dislocations** adds occasional edge/screw/mixed dislocations
|
|
149
|
+
- **Random strain** adds occasional radial/edge/random strain fields
|
|
150
|
+
|
|
151
|
+
Samples save as `sample_XXXXX.npz` and a `manifest.json`. Generation is
|
|
152
|
+
parallelizable and resumable.
|
|
153
|
+
|
|
154
|
+
### 5. AutoPhase_NN — unsupervised physics-driven NN
|
|
155
|
+
|
|
156
|
+
AutoPhase_NN is a 3D dual-decoder convolutional network that learns to invert the
|
|
157
|
+
Fourier modulus constraint **without ground truth**. It predicts the object's
|
|
158
|
+
amplitude and phase such that `|FFT(amp · exp(iφ))|` matches the measured
|
|
159
|
+
magnitude. The loss is a normalized-magnitude error plus optional regularizers
|
|
160
|
+
on phase smoothness and support compactness. This is the network from (Yao, Yudong,
|
|
161
|
+
et al. npj Computational Materials 8.1 (2022) ) reimplemented for this codebase.
|
|
162
|
+
|
|
163
|
+
Training is GPU-accelerated (CUDA preferred) and converges in a few dozen
|
|
164
|
+
epochs on 2,000+ samples. The best checkpoint is saved automatically.
|
|
165
|
+
|
|
166
|
+
### 6. CDI_NN — supervised PhaseUNet3D
|
|
167
|
+
|
|
168
|
+
A 3D U-Net that predicts the phase field from the diffraction magnitude, trained
|
|
169
|
+
against the ground-truth phase from the data generator.
|
|
170
|
+
|
|
171
|
+
Use AutoPhase_NN when you don't trust your simulator's ground truth (or have
|
|
172
|
+
none). Use CDI_NN when you have a high-fidelity simulator and want a direct
|
|
173
|
+
mapping. Use both via the **ensemble** mode in the Reconstruction tab.
|
|
174
|
+
|
|
175
|
+
### 7. Reconstruction — invert experimental or simulated data
|
|
176
|
+
|
|
177
|
+
Load any `.npz` (simulated) or `.h5` (experimental) BCDI file, or convert a
|
|
178
|
+
SPEC + EDF scan from ID01 directly via the dialog. Choose:
|
|
179
|
+
|
|
180
|
+
- **NN-only (Autophase_NN)**: forward pass, twin-image suppression, COM centering
|
|
181
|
+
- **Refined**: NN seed → HIO + RAAR + ER iterative phase retrieval
|
|
182
|
+
- **Ensemble**: AutoPhase_NN + CDI_NN combined, optionally refined
|
|
183
|
+
- **Compare**: runs both modes side-by-side
|
|
184
|
+
|
|
185
|
+
The output is a complex object (amplitude + phase), a binary support, the
|
|
186
|
+
R-factor convergence curve, and 2D slices at the COM of the reconstruction.
|
|
187
|
+
|
|
188
|
+
For experimental data, the loader handles:
|
|
189
|
+
- Detector chip-gap masking (Maxipix, Eiger)
|
|
190
|
+
- Hot-pixel removal
|
|
191
|
+
- Beamstop / direct-beam streak removal
|
|
192
|
+
- Bragg-peak-centered cropping
|
|
193
|
+
- Optional q-space orthogonalization via xrayutilities (with `[id01]` extra)
|
|
194
|
+
|
|
195
|
+
### 8. 3D Viewer — interactive visualization
|
|
196
|
+
|
|
197
|
+
The viewer renders the reconstruction in three modes:
|
|
198
|
+
|
|
199
|
+
- **Point cloud**: every voxel above threshold, colored by amplitude / phase / strain,
|
|
200
|
+
with per-voxel opacity weighted by intensity
|
|
201
|
+
- **Isosurface**: smoothed iso-shell with internal data points showing through
|
|
202
|
+
- **Surface only**: bare iso-mesh with edges visible
|
|
203
|
+
|
|
204
|
+
Controls:
|
|
205
|
+
- Threshold (% of peak amplitude), voxel size, opacity, colormap
|
|
206
|
+
- Color scale auto / manual (`vmin` / `vmax`)
|
|
207
|
+
- Clip plane on any axis
|
|
208
|
+
- Mouse: left-drag rotate, right-drag pan, wheel zoom, double-click reset
|
|
209
|
+
- Export PNG snapshot or VTI (ParaView)
|
|
210
|
+
|
|
211
|
+
---
|
|
212
|
+
|
|
213
|
+
## Project structure
|
|
214
|
+
|
|
215
|
+
```
|
|
216
|
+
cdi-st/
|
|
217
|
+
├── src/cdi_st/
|
|
218
|
+
│ ├── __init__.py
|
|
219
|
+
│ ├── __main__.py # python -m cdi_st
|
|
220
|
+
│ ├── bcdi_gui.py # main entry point + launcher + splash
|
|
221
|
+
│ ├── bcdi_core.py # crystal builder, simulator, reflection calculator
|
|
222
|
+
│ ├── nn_gui_tabs.py # all NN-related GUI tabs
|
|
223
|
+
│ ├── nn_data_generator.py # training-data generator
|
|
224
|
+
│ ├── nn_dataset.py # PyTorch Dataset wrapper
|
|
225
|
+
│ ├── nn_phase_model.py # PhaseUNet3D (supervised)
|
|
226
|
+
│ ├── nn_autophase_model.py # AutoPhaseNet3D (unsupervised)
|
|
227
|
+
│ ├── nn_train.py # supervised training CLI
|
|
228
|
+
│ ├── nn_autophase_train.py # unsupervised training CLI
|
|
229
|
+
│ ├── nn_phase_retrieval.py # HIO / RAAR / ER iterative algorithms
|
|
230
|
+
│ ├── nn_autophase_infer.py # inference: nn_only / refined / ensemble
|
|
231
|
+
│ ├── nn_experimental_loader.py # h5/spec/edf loading + preprocessing
|
|
232
|
+
│ ├── nn_demo.py
|
|
233
|
+
│ ├── nn_visualize.py
|
|
234
|
+
│ └── CDI_ST_logo.png
|
|
235
|
+
├── tests/
|
|
236
|
+
├── docs/
|
|
237
|
+
├── pyproject.toml
|
|
238
|
+
├── README.md
|
|
239
|
+
├── LICENSE
|
|
240
|
+
└── CHANGELOG.md
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
## Citing
|
|
246
|
+
|
|
247
|
+
If you use CDI-ST in published research, please cite:
|
|
248
|
+
|
|
249
|
+
```bibtex
|
|
250
|
+
@software{saidi_cdi_st_2026,
|
|
251
|
+
author = {Saidi, Soufiane},
|
|
252
|
+
title = {CDI-ST: Coherent Diffraction Imaging — Simulation Tools},
|
|
253
|
+
year = {2026},
|
|
254
|
+
publisher = {GitHub},
|
|
255
|
+
url = {https://github.com/Refze/CDI-St}
|
|
256
|
+
}
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
---
|
|
260
|
+
|
|
261
|
+
## License
|
|
262
|
+
|
|
263
|
+
MIT — see [LICENSE](LICENSE).
|
|
264
|
+
|
|
265
|
+
## Author
|
|
266
|
+
|
|
267
|
+
**Soufiane SAIDI** — saidisoufiane@hotmail.com
|
|
268
|
+
|
|
269
|
+
We heavily encourage you to use the in-app **Reports & Suggestions** button to send any type of feedback.
|
|
270
|
+
|
|
271
|
+
---
|
|
272
|
+
|
|
273
|
+
## Acknowledgments
|
|
274
|
+
|
|
275
|
+
Built on the shoulders of the BCDI and python community:
|
|
276
|
+
[PyNX](https://pynx.esrf.fr/), [BCDI-Utilities](https://github.com/carnisj/bcdi),
|
|
277
|
+
[cdiutils](https://github.com/clatlan/cdiutils), [Atomsk](https://atomsk.univ-lille.fr/) , [xrayutilities](https://xrayutilities.sourceforge.io/),
|
|
278
|
+
[scikit-image](https://scikit-image.org/), [PyQt6](https://www.riverbankcomputing.com/software/pyqt/),
|
|
279
|
+
and the AutoPhaseNN paper by Yao et al. (2022).
|