pyREyes 3.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pyreyes-3.4.0/.gitignore +6 -0
- pyreyes-3.4.0/LICENSE +15 -0
- pyreyes-3.4.0/PKG-INFO +117 -0
- pyreyes-3.4.0/README.md +91 -0
- pyreyes-3.4.0/docs/scripts/SerialEMsettings-scripts_CETA-D.txt +1104 -0
- pyreyes-3.4.0/docs/scripts/SerialEMsettings-scripts_DE_Apollo.txt +1099 -0
- pyreyes-3.4.0/docs/tutorials/Configurable_SerialEM_params.md +274 -0
- pyreyes-3.4.0/docs/tutorials/Installation_guide.md +92 -0
- pyreyes-3.4.0/docs/tutorials/User_guide.pdf +0 -0
- pyreyes-3.4.0/pyproject.toml +51 -0
- pyreyes-3.4.0/src/pyREyes/REyes.py +958 -0
- pyreyes-3.4.0/src/pyREyes/append_targets_nav.py +343 -0
- pyreyes-3.4.0/src/pyREyes/config/filtering_options.json +19 -0
- pyreyes-3.4.0/src/pyREyes/config/microscope_configs.json +92 -0
- pyreyes-3.4.0/src/pyREyes/create_final_targets_nav.py +476 -0
- pyreyes-3.4.0/src/pyREyes/dif_map.py +1219 -0
- pyreyes-3.4.0/src/pyREyes/docs/README_FULL.md +522 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/diffraction-map.md +90 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/eucentricity.md +64 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/final-targets.md +92 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/grid-squares.md +130 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/manual-squares.md +61 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/map-plotter.md +95 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/monitor.md +113 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/navigation.md +88 -0
- pyreyes-3.4.0/src/pyREyes/docs/modules/targets.md +96 -0
- pyreyes-3.4.0/src/pyREyes/eucentricity.py +392 -0
- pyreyes-3.4.0/src/pyREyes/grid_squares.py +530 -0
- pyreyes-3.4.0/src/pyREyes/lib/REyes_errors.py +18 -0
- pyreyes-3.4.0/src/pyREyes/lib/REyes_logging.py +44 -0
- pyreyes-3.4.0/src/pyREyes/lib/REyes_microscope_configurations.py +36 -0
- pyreyes-3.4.0/src/pyREyes/lib/REyes_utils.py +35 -0
- pyreyes-3.4.0/src/pyREyes/lib/__init__.py +0 -0
- pyreyes-3.4.0/src/pyREyes/lib/diffraction/DiffractionDataProcessor.py +210 -0
- pyreyes-3.4.0/src/pyREyes/lib/grid_processing/CentroidProcessor.py +692 -0
- pyreyes-3.4.0/src/pyREyes/lib/grid_processing/GridProcessor.py +50 -0
- pyreyes-3.4.0/src/pyREyes/lib/grid_processing/filtering_options.py +30 -0
- pyreyes-3.4.0/src/pyREyes/lib/image_processing/CartImage.py +660 -0
- pyreyes-3.4.0/src/pyREyes/lib/image_processing/CartImage_utils.py +51 -0
- pyreyes-3.4.0/src/pyREyes/lib/image_processing/PlottingManager.py +210 -0
- pyreyes-3.4.0/src/pyREyes/lib/image_processing/__init__.py +0 -0
- pyreyes-3.4.0/src/pyREyes/lib/image_processing/image_processor.py +531 -0
- pyreyes-3.4.0/src/pyREyes/lib/ui/REyes_ui.py +16 -0
- pyreyes-3.4.0/src/pyREyes/lib/ui/__init__.py +0 -0
- pyreyes-3.4.0/src/pyREyes/manual_squares.py +148 -0
- pyreyes-3.4.0/src/pyREyes/mnt_maps_targets.py +650 -0
- pyreyes-3.4.0/src/pyREyes/write_sorted_targets_nav.py +158 -0
- pyreyes-3.4.0/tools/diffraction_quality/diffraction_quality.ipynb +1034 -0
- pyreyes-3.4.0/tools/diffraction_quality/example_1/20240807_19495_integrated_movie.mrc +0 -0
- pyreyes-3.4.0/tools/diffraction_quality/example_1/20240807_19889_integrated_movie.mrc +0 -0
- pyreyes-3.4.0/tools/diffraction_quality/example_1/20240807_21132_integrated_movie.mrc +0 -0
- pyreyes-3.4.0/tools/diffraction_quality/example_1/20240821_34369_integrated_movie.mrc +0 -0
- pyreyes-3.4.0/tools/diffraction_quality/example_1/20240821_34422_integrated_movie.mrc +0 -0
- pyreyes-3.4.0/tools/diffraction_quality/example_1/20240821_34445_integrated_movie.mrc +0 -0
- pyreyes-3.4.0/tools/diffraction_quality/example_1/20240830_41002_integrated_movie.mrc +0 -0
pyreyes-3.4.0/.gitignore
ADDED
pyreyes-3.4.0/LICENSE
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
Copyright © 2025, California Institute of Technology (Caltech). All rights reserved.
|
|
2
|
+
|
|
3
|
+
Redistribution and use in source and binary forms for academic and other non-commercial purposes only, with or without modification, are permitted provided that the following conditions are met:
|
|
4
|
+
|
|
5
|
+
1. Redistributions of source code, including modified source code, must retain the above copyright notice, this list of conditions and the following disclaimer.
|
|
6
|
+
|
|
7
|
+
2. Redistributions in binary form, or a modified version of the binary form, must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
|
8
|
+
|
|
9
|
+
3. Neither the name of California Institute of Technology (Caltech), any of its trademarks, the names of its employees, nor contributors to the source code may be used to endorse or promote products derived from this software without specific prior written permission.
|
|
10
|
+
|
|
11
|
+
4. Where a modified version of the source code is redistributed publicly in source or binary forms, the modified source code must be published in a freely accessible manner, or otherwise redistributed at no charge to anyone requesting a copy of the modified source code, subject to the same terms as this agreement.
|
|
12
|
+
|
|
13
|
+
5. This software is part of a patented invention by Dmitry Eremin, Jose Rodriguez, and Hosea Nelson. Use is permitted for academic and non-commercial research purposes only. Commercial use requires a separate license agreement. Please contact the California Institute of Technology's Office of Technology Transfer for commercial licensing inquiries.
|
|
14
|
+
|
|
15
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
pyreyes-3.4.0/PKG-INFO
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: pyREyes
|
|
3
|
+
Version: 3.4.0
|
|
4
|
+
Summary: Automated MicroED diffraction data collection and analysis package
|
|
5
|
+
Project-URL: Homepage, https://github.com/theNelsonLab/REyes
|
|
6
|
+
Project-URL: Issues, https://github.com/theNelsonLab/REyes/issues
|
|
7
|
+
Project-URL: Documentation, https://github.com/theNelsonLab/REyes/tree/main/src/pyREyes/docs
|
|
8
|
+
Author-email: Dmitry Eremin <eremin@caltech.edu>, Hongyu Zhang <hzhang7@caltech.edu>
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
13
|
+
Requires-Python: >=3.10
|
|
14
|
+
Requires-Dist: hyperspy==2.3.0
|
|
15
|
+
Requires-Dist: matplotlib==3.10.3
|
|
16
|
+
Requires-Dist: numba==0.61.2
|
|
17
|
+
Requires-Dist: numpy==2.2.6
|
|
18
|
+
Requires-Dist: pandas==2.2.3
|
|
19
|
+
Requires-Dist: psutil==7.0.0
|
|
20
|
+
Requires-Dist: pyautoprocess>=0.1.1
|
|
21
|
+
Requires-Dist: scikit-image==0.25.2
|
|
22
|
+
Requires-Dist: scikit-learn==1.6.1
|
|
23
|
+
Requires-Dist: scipy==1.15.3
|
|
24
|
+
Requires-Dist: shapely==2.1.1
|
|
25
|
+
Description-Content-Type: text/markdown
|
|
26
|
+
|
|
27
|
+
# Diffraction Map Processing Pipeline (REyes v3.4)
|
|
28
|
+
[](LICENSE)
|
|
29
|
+
[](https://doi.org/10.5281/zenodo.16971798)
|
|
30
|
+
|
|
31
|
+
## Quick Start
|
|
32
|
+
```bash
|
|
33
|
+
# Install the package
|
|
34
|
+
pip install .
|
|
35
|
+
|
|
36
|
+
# Run automated pipeline
|
|
37
|
+
reyes-monitor --microscope Arctica-CETA --autoprocess
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Overview
|
|
41
|
+
pyREyes is a Python package to support REyes (Reciprocal Eyes), an end-to-end autonomous electron diffraction data collection and processing pipeline. REyes generates diffraction heatmaps, identifies key targets, and manages navigator files for SerialEM.
|
|
42
|
+
|
|
43
|
+
📚 [Full Documentation](src/pyREyes/docs/README_FULL.md)
|
|
44
|
+
|
|
45
|
+
## Key Features
|
|
46
|
+
- Grid squares detection and atlas management
|
|
47
|
+
- Manual or automated grid square selection
|
|
48
|
+
- Eucentricity correction
|
|
49
|
+
- Diffraction pattern quality classification using DQI (Diffraction Quality Index)
|
|
50
|
+
- Navigation file generation with spatial awareness
|
|
51
|
+
- Comprehensive visualization suite
|
|
52
|
+
- Automated data processing with state machine monitoring
|
|
53
|
+
- Optional movie processing and structure solution integration
|
|
54
|
+
|
|
55
|
+
## Diffraction Quality Analysis
|
|
56
|
+
|
|
57
|
+
REyes uses advanced quality metrics to assess diffraction patterns:
|
|
58
|
+
|
|
59
|
+
**LQP (Lattice Quality Peaks)**: A number of peaks in an FFT of the thresholded and binarized diffraction snapshot that correspond to lattice quality.
|
|
60
|
+
|
|
61
|
+
**DQI (Diffraction Quality Index)**: A quality metric calculated as the ratio of LQP to total diffraction peaks (DQI = LQP / Diffraction Peaks). DQI values above 3.0 indicate higher crystalline quality with better-ordered lattice structures. This index is used to automatically classify and prioritize crystal targets for data collection.
|
|
62
|
+
|
|
63
|
+
## Pipeline Components
|
|
64
|
+
|
|
65
|
+
### Core Modules
|
|
66
|
+
1. [Grid Squares Searcher](src/pyREyes/docs/modules/grid-squares.md)
|
|
67
|
+
- 1.1. [Manual Grid Squares Adder](src/pyREyes/docs/modules/manual-squares.md)
|
|
68
|
+
2. [Navigator Eucentricity Corrector](src/pyREyes/docs/modules/eucentricity.md)
|
|
69
|
+
3. [Diffraction Map Processor](src/pyREyes/docs/modules/diffraction-map.md)
|
|
70
|
+
4. [Navigation File Generator](src/pyREyes/docs/modules/navigation.md)
|
|
71
|
+
- 4.1. [Selected Targets Adder](src/pyREyes/docs/modules/targets.md)
|
|
72
|
+
5. [Final Targets Processor](src/pyREyes/docs/modules/final-targets.md)
|
|
73
|
+
6. [Collection Executor](src/pyREyes/docs/modules/map-plotter.md)
|
|
74
|
+
|
|
75
|
+
**Pipeline Orchestration:**
|
|
76
|
+
- [Processing Monitor](src/pyREyes/docs/modules/monitor.md) - Automatically runs all steps above and pauses for user input on optional manual steps
|
|
77
|
+
|
|
78
|
+
## Directory Structure
|
|
79
|
+
```
|
|
80
|
+
working_directory/
|
|
81
|
+
├── dif_maps/ # Diffraction maps and visualizations
|
|
82
|
+
│ └── diff_blocks_maps/ # Block-specific maps
|
|
83
|
+
├── grid_squares/ # Grid square & eucentricity outputs
|
|
84
|
+
├── movies/ # Continuous rotation diffraction movies
|
|
85
|
+
├── REyes_logs/ # Processing logs
|
|
86
|
+
└── targets/ # Navigation files and diffraction snapshots
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## Reference
|
|
90
|
+
The REyes methodology and its applications to small molecules, materials, and protein crystallography are described in detail in:
|
|
91
|
+
|
|
92
|
+
Eremin, D.B.; Jha, K.K.; Delgadillo, D.A.; Zhang, H.; Foxman, S.H.; Johnson, S.N.; Vlahakis, N.W.; Cascio, D.; Lavallo, V.; Rodríguez, J.A.; Nelson, H.M. **"Spatially-Aware Diffraction Mapping Enables Fully Autonomous MicroED"** *ChemRxiv* **2025**, DOI: [10.26434/chemrxiv-2025-4p4c3](https://doi.org/10.26434/chemrxiv-2025-4p4c3)
|
|
93
|
+
|
|
94
|
+
## Data Availability
|
|
95
|
+
Electron diffraction data are publicly available through the Caltech.
|
|
96
|
+
|
|
97
|
+
**Repository:** https://miledd.caltech.edu/shared/
|
|
98
|
+
|
|
99
|
+
**ZENODO Dataset:** Complete Raw and Processed REyes Dataset: Autonomous MicroED Collection for (S,S)-Salen Ligand - https://doi.org/10.5281/zenodo.16971798
|
|
100
|
+
|
|
101
|
+
## Support
|
|
102
|
+
For support and questions, please contact:
|
|
103
|
+
- 📫 Dmitry Eremin - eremin@caltech.edu
|
|
104
|
+
|
|
105
|
+
## Developers
|
|
106
|
+
- [Dmitry Eremin (mit-eremin)](https://github.com/mit-eremin)
|
|
107
|
+
- [Hongyu Zhang (Eta2Zeta)](https://github.com/Eta2Zeta)
|
|
108
|
+
|
|
109
|
+
## Patent Disclosure
|
|
110
|
+
Dmitry B. Eremin, Hongyu Zhang, Jose A. Rodríguez, and Hosea M. Nelson have filed a provisional patent application related to the methods implemented in this software:
|
|
111
|
+
|
|
112
|
+
**Patent Application:** CIT-9295-P2, filed May 8, 2025
|
|
113
|
+
|
|
114
|
+
This disclosure is provided in accordance with academic transparency standards. The software remains available for academic and research use under the terms specified in the license.
|
|
115
|
+
|
|
116
|
+
Copyright © 2025, California Institute of Technology (Caltech). All rights reserved.
|
|
117
|
+
Use of this software is permitted for academic and non‑commercial research only. Commercial use is prohibited without a license, please contact Caltech Office of Technology Transfer & Corporate Partnerships.
|
pyreyes-3.4.0/README.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# Diffraction Map Processing Pipeline (REyes v3.4)
|
|
2
|
+
[](LICENSE)
|
|
3
|
+
[](https://doi.org/10.5281/zenodo.16971798)
|
|
4
|
+
|
|
5
|
+
## Quick Start
|
|
6
|
+
```bash
|
|
7
|
+
# Install the package
|
|
8
|
+
pip install .
|
|
9
|
+
|
|
10
|
+
# Run automated pipeline
|
|
11
|
+
reyes-monitor --microscope Arctica-CETA --autoprocess
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
## Overview
|
|
15
|
+
pyREyes is a Python package to support REyes (Reciprocal Eyes), an end-to-end autonomous electron diffraction data collection and processing pipeline. REyes generates diffraction heatmaps, identifies key targets, and manages navigator files for SerialEM.
|
|
16
|
+
|
|
17
|
+
📚 [Full Documentation](src/pyREyes/docs/README_FULL.md)
|
|
18
|
+
|
|
19
|
+
## Key Features
|
|
20
|
+
- Grid squares detection and atlas management
|
|
21
|
+
- Manual or automated grid square selection
|
|
22
|
+
- Eucentricity correction
|
|
23
|
+
- Diffraction pattern quality classification using DQI (Diffraction Quality Index)
|
|
24
|
+
- Navigation file generation with spatial awareness
|
|
25
|
+
- Comprehensive visualization suite
|
|
26
|
+
- Automated data processing with state machine monitoring
|
|
27
|
+
- Optional movie processing and structure solution integration
|
|
28
|
+
|
|
29
|
+
## Diffraction Quality Analysis
|
|
30
|
+
|
|
31
|
+
REyes uses advanced quality metrics to assess diffraction patterns:
|
|
32
|
+
|
|
33
|
+
**LQP (Lattice Quality Peaks)**: A number of peaks in an FFT of the thresholded and binarized diffraction snapshot that correspond to lattice quality.
|
|
34
|
+
|
|
35
|
+
**DQI (Diffraction Quality Index)**: A quality metric calculated as the ratio of LQP to total diffraction peaks (DQI = LQP / Diffraction Peaks). DQI values above 3.0 indicate higher crystalline quality with better-ordered lattice structures. This index is used to automatically classify and prioritize crystal targets for data collection.
|
|
36
|
+
|
|
37
|
+
## Pipeline Components
|
|
38
|
+
|
|
39
|
+
### Core Modules
|
|
40
|
+
1. [Grid Squares Searcher](src/pyREyes/docs/modules/grid-squares.md)
|
|
41
|
+
- 1.1. [Manual Grid Squares Adder](src/pyREyes/docs/modules/manual-squares.md)
|
|
42
|
+
2. [Navigator Eucentricity Corrector](src/pyREyes/docs/modules/eucentricity.md)
|
|
43
|
+
3. [Diffraction Map Processor](src/pyREyes/docs/modules/diffraction-map.md)
|
|
44
|
+
4. [Navigation File Generator](src/pyREyes/docs/modules/navigation.md)
|
|
45
|
+
- 4.1. [Selected Targets Adder](src/pyREyes/docs/modules/targets.md)
|
|
46
|
+
5. [Final Targets Processor](src/pyREyes/docs/modules/final-targets.md)
|
|
47
|
+
6. [Collection Executor](src/pyREyes/docs/modules/map-plotter.md)
|
|
48
|
+
|
|
49
|
+
**Pipeline Orchestration:**
|
|
50
|
+
- [Processing Monitor](src/pyREyes/docs/modules/monitor.md) - Automatically runs all steps above and pauses for user input on optional manual steps
|
|
51
|
+
|
|
52
|
+
## Directory Structure
|
|
53
|
+
```
|
|
54
|
+
working_directory/
|
|
55
|
+
├── dif_maps/ # Diffraction maps and visualizations
|
|
56
|
+
│ └── diff_blocks_maps/ # Block-specific maps
|
|
57
|
+
├── grid_squares/ # Grid square & eucentricity outputs
|
|
58
|
+
├── movies/ # Continuous rotation diffraction movies
|
|
59
|
+
├── REyes_logs/ # Processing logs
|
|
60
|
+
└── targets/ # Navigation files and diffraction snapshots
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
## Reference
|
|
64
|
+
The REyes methodology and its applications to small molecules, materials, and protein crystallography are described in detail in:
|
|
65
|
+
|
|
66
|
+
Eremin, D.B.; Jha, K.K.; Delgadillo, D.A.; Zhang, H.; Foxman, S.H.; Johnson, S.N.; Vlahakis, N.W.; Cascio, D.; Lavallo, V.; Rodríguez, J.A.; Nelson, H.M. **"Spatially-Aware Diffraction Mapping Enables Fully Autonomous MicroED"** *ChemRxiv* **2025**, DOI: [10.26434/chemrxiv-2025-4p4c3](https://doi.org/10.26434/chemrxiv-2025-4p4c3)
|
|
67
|
+
|
|
68
|
+
## Data Availability
|
|
69
|
+
Electron diffraction data are publicly available through the Caltech.
|
|
70
|
+
|
|
71
|
+
**Repository:** https://miledd.caltech.edu/shared/
|
|
72
|
+
|
|
73
|
+
**ZENODO Dataset:** Complete Raw and Processed REyes Dataset: Autonomous MicroED Collection for (S,S)-Salen Ligand - https://doi.org/10.5281/zenodo.16971798
|
|
74
|
+
|
|
75
|
+
## Support
|
|
76
|
+
For support and questions, please contact:
|
|
77
|
+
- 📫 Dmitry Eremin - eremin@caltech.edu
|
|
78
|
+
|
|
79
|
+
## Developers
|
|
80
|
+
- [Dmitry Eremin (mit-eremin)](https://github.com/mit-eremin)
|
|
81
|
+
- [Hongyu Zhang (Eta2Zeta)](https://github.com/Eta2Zeta)
|
|
82
|
+
|
|
83
|
+
## Patent Disclosure
|
|
84
|
+
Dmitry B. Eremin, Hongyu Zhang, Jose A. Rodríguez, and Hosea M. Nelson have filed a provisional patent application related to the methods implemented in this software:
|
|
85
|
+
|
|
86
|
+
**Patent Application:** CIT-9295-P2, filed May 8, 2025
|
|
87
|
+
|
|
88
|
+
This disclosure is provided in accordance with academic transparency standards. The software remains available for academic and research use under the terms specified in the license.
|
|
89
|
+
|
|
90
|
+
Copyright © 2025, California Institute of Technology (Caltech). All rights reserved.
|
|
91
|
+
Use of this software is permitted for academic and non‑commercial research only. Commercial use is prohibited without a license, please contact Caltech Office of Technology Transfer & Corporate Partnerships.
|