scipion-em-xmipp 23.3.0.3__tar.gz → 23.11.0.0__tar.gz

scipion-em-xmipp-23.11.0.0/CHANGES.md

@@ -0,0 +1,211 @@
+## Release 3.23.11 - Nereus
+   - New protocols
+      - Volume local adjustment
+   - Protocols updated
+      - convert_pdb: Allowed conversion natively from CIFs
+      - particle_pick_automatic: The model can now be given by a directory
+      - volume_local_adjust: Save occupancy volume
+      - extract_particles: Added two different cases for downsampling, by dimensions and by sampling rate
+   - Protocols fixed
+      - movie_resize: Fixed movie resize output size
+      - movie_gain: Fix update output step by using a generic one from scipion
+      - tilt_analysis: Fixes in the generated tilt images and in updating correctly the output sets
+      - ctf_consensus: Fix the dependencies of the step
+      - preprocess_micrographs: Fixed output size in preprocess micrographs
+      - deep_center_assignment: Fixed deep center calls
+      - extract_particles_movies: Get coords correctly
+      - particle_pick_consensus: Fix MicsPointer
+      - trigger_data: fix updateOutput and close correclty the output set
+   - Protocols deprecated (For more details visit [this](https://github.com/I2PC/xmipp/wiki/Deprecating-programs-and-protocols))
+      - classification_gpuCorr
+      - classification_gpuCorr_full
+      - classification_gpuCorr_semi
+   - More scipio-em-xmipp
+      - Updated Nvidia driver required version
+
+
+## Release 3.23.07 - Morpheus 
+   - New protocols
+      - Movie Dose analysis
+      - deep_center
+      - deep_global_assignment
+      - deep_center_predict
+      - deep_global_assignment_predict
+   - Protocols updated
+      - consensus_classes (Efficient p-value calculation, updated intersection merging process, generalized protocol for other set of classes)
+      - Movie Gain: changed _stepsCheckSecs and fixed inputMovies calling, np.asscalar discontinued in numpy 1.16
+      - convert_pdb: dont allow set size if template volume, to convert a set of pdbs to volumes, generates an mrc file
+      - CTF_consensus: add 4 threads by default
+      - process: Better instantiation of Scipion subclasses
+      - create_mask3d: Addding a validate in 3dmask, add :mrc to input filename
+      - consensus_local_ctf: save defocus in proper fields,  compute consensus for local defocus U and V separately, add consensus angle
+      - align_volume: Included the label in the volumes
+      - crop_resize: Add mask as input. Mask resize is now possible
+      - subtract_projection: change pad validation error for warning, parallelized
+   - Protocols fixed
+      - Tilt analysis: Close correctly the output sets once finished
+      - Deep micrograph cleaner: fix two bugs that occured during streaming implementation bug 
+      - volume_adjust_sub: fix with :mrc
+      - Picking consensus: define correctly the possibleOutputs bug 
+      - Center particles: streaming bug when definining the outputs bug
+      - volume_subtraction: bug fixed in filename
+      - compare_reprojections: fix update subtract projection output
+      - deep_micrograph_screen: Bug fix that prevents using small GPUs
+      - consensus_classes:Fixed manual output generation
+   - Protocols deprecated (For more details visit [this](https://github.com/I2PC/xmipp/wiki/Deprecating-programs-and-protocols))
+      - apply_deformation_zernike3d
+      - classify_kmeans2d
+      - kmeans_clustering
+      - particle_boxSize
+      - rotational_spectra
+      - split_volume_hierarchical_cluster
+   - Viewers
+      - viewer_resolution_fs: fixing 0.1 threshold not found
+      - viewer_projmatch, viewer_metaprotocol_golden_highres: Fixing viewers, change removed ChimeraClientView to ChimeraView
+      - monores_viewer: fix histogram
+      - viewer_structure_map: Change the label for each volume
+
+
+## Release 3.23.03 - Kratos
+  - New protocol status: beta, new, production and updated. Will appear in the left pannel of Scipion 
+  - Protocol subtract_projection: user experience improvements, no final mask by default, apply ciruclar mask in adjustment image to avoid edge artifacts, validate same sampling rate with tolerance in third decimal
+  - Protocol convert_pdb: Allowed to save centered PDB used for conversion. 
+  - Protocol align_volume_and_particles: add alingment validation
+  - Protocol FlexAlign: updating protocol to reflect changes in the executable, fixed test, removing unused protocol (Movie average)
+  - Protocol align_volume_and_particles:Align volume and particles adapted to tomography and works in the absence of tomo plugin.
+  - Protocol volume_consensus: validate same sampling rate with tolerance in third decimal
+  - Protocols deprecated (for more details visit the [wiki](https://github.com/I2PC/xmipp/wiki/Deprecating-programs)): protocol_deep _align, reconstruct_heterogeneous, protocol_metaprotocol_create_output, protocol_metaprotocol_discrete_heterogeneity_scheduler
+
+  
+## Hot fix 3.22.11.2
+- Align volume and particles works in the absence of tomo plugin.
+
+## Hot fix 3.22.11.1
+- Align volume and particles adapted to tomography. Defines possible outputs. Optimized. Test more exhaustive for matrices
+
+## Release 3.22.11 - Iris
+  - Protocol_cl2d_align: The input can now be a set of averages or a set of 2D classes 
+  - Protocol_local_ctf: Default value are now changed for maxDefocusChange
+  - Protocol_apply_zernike3d: Now accepts either a Volume or SetOfVolumes and applies the coefficients in a loop in the deform step
+  - Protocol_postProcessing_deepPostProcessing: Managed GPU memory to avoid errors
+  - Protocol_resolution_deepres: Mandatory mask
+  - Protocol center particles and Gl2d (all options): Fix streaming
+  - Protocol_create_3d_mask: Allows volume Null=True
+  - Protocol_reconstruct_fourier: Set pixel size
+  - GL2D static: Bug fixing
+  - Protocol_trigger_data: Bug fixing
+  - Protocol_crop_resize: Set sampling rate of mrc files when cropping resizing volumes or particles
+  - subtract_projection: New protocol for boosting particles. Add protocol to wizard XmippParticleMaskRadiusWizard as now the protocol uses it
+
+  - **New tests:** deep_hand, pick_noise, screen_deep_learning, resolution_B_factor
+  - Fixed TestHighres test
+
+## Release 3.22.07 - Helios 
+- rotate_volume: New protocol
+- subtract_projection: New implementation based on adjustment by regression instead of POCS and improved performance
+- local_ctf: Add new sameDefocus option + formatting
+- compare_reprojections & protocol_align_volume: Fast Fourier by default
+- crop_resize: Allows input pointers
+- resolution_deepres: Resize output to original size
+- denoise_particles: Added setOfAverages as input option
+- process: Change output from stk (spider) to mrcs (mrc)
+- trigger_data: Bug fixed
+- screen_deeplearning:  Added descriptive help
+- center_particles: Added summary info
+- align_volume_and_particles: Summary error fixed
+- cl2d: Summary errors solved 
+- New tests: test_protocol_reconstruct_fourier, test_protocols_local_defocus, test_protocols_local_defocus, TestXmippAlignVolumeAndParticles,  TestXmippRotateVolume
+- Improved tests: test_protocols_deepVolPostprocessing, test_protocols_xmipp_3d, Test ProjSubtracion
+- Excluded tests: test_protocols_zernike3d, test_protocols_metaprotocol_heterogeneity
+
+## Release 3.22.04 - Gaia
+- protocol_core_analysis: New protocol
+- protocol_compare_angles: Bug fix in compare angles under some conditions
+- protocol_center_particles: protocol simplified (removed setofCoordinates as output)
+- protocol_CTF_consensus: concurrency error fixed
+- protocol_convert_pdb: remove size if deactivated
+- protocol_resolution_deepres: binary masked not stored in Extra folder and avoiding memory problems on GPUs
+- protocol_add_noise: fixes
+- protocol_compare_reprojections: improve computation of residuals + tests + fix + formatting
+- protocol_screen_deepConsensus: multiple fixes in batch processing, trainging and streaming mode
+- protocol_shift_particles: apply transform is now optional
+
+
+## Release 3.22.01 - Eris
+- Visit changeLog.md of xmipp
+
+## Release 3.21.06 - Caerus
+
+- CUDA-11 support
+- New protocol: Deep align
+- ChimeraX support
+- Improvements of streaming process
+- Several performance optimizations
+- Build time optimization
+- Multiple bug fixes
+- Improved documentation
+
+
+## Release 3.20.07 - Boreas
+
+- New Protocol: MicrographCleaner is a new algorithm that removes coordinates picked from carbon edges, aggregations, ice crystals and other contaminations
+- New functionality: The protocol compare reprojections can now compute the residuals after alignment
+- New protocol: Split frames divide input movies into odd and even movies so that they can be processed independently
+- New protocol: Continuous heterogeneity analysis using spherical harmonics (not ready to be used)
+- Bug fixing when some micrograph has no coordinates in the consensus-picking.
+- New functionalities: Different architectures and training modes
+- Normal Mode Analysis protocols have been moved to the plugin ContinuousFlex
+- Fixing MPI version of the Fourier Reconstruction
+- New protocol: local CTF integration and consensus protocol for local ctf (also the viewers)
+- Local CTF analysis tools: Not yet ready for general public
+- New functionallity: Introducing the posibility of automatic estimation of the gain orientation.
+- Bugs fixings regarding stability on streaming processing
+- Support of heterogeneous movie sets
+- New protocol: Clustering of subtomogram coordinates into connected components that can be processed independently
+- New Protocol: Removing duplicated coordinates
+- New protocol: Subtomograms can be projected in several ways to 2D images so that 2D clustering tools can be used
+- New protocol: Regions of Interest can be defined in tomograms (e.g., membranes)
+- Bug fixing in mask3d protocol
+- Bug fix: in helical search symmetry protocol
+- Enhanced precision of the FlexAlign program
+- Now, deepLearningToolkit is under its own conda environment
+- Multiple protocols accelerated using GPU
+- New functionality: Xmipp CTF estimation can now take a previous defocus and do not change it
+- New functionallity: CTF-consensus is able to take the primary main values or an average of the two.
+- New functionallity: CTF-consensus is able to append metadata from the secondary input
+- New functionality: Xmipp Highres can now work with non-phase flipped images
+- New functionality: Xmipp Preprocess particles can now phase flip the images
+- New protocol: Tool to evaluate the quality of a map-model fitting
+- Allowing multi-GPU processing using FlexAlign
+- Improvement in monores and localdeblur
+- Randomize phases also available for images
+- Change the plugin to the new Scipion structure
+- Migrating the code to python3
+
+## Release 3.19.04 -
+
+- Highres can now take a global alignment performed by any other method
+- New protocol: 3D bionotes
+- New protocol: Align volume and particles
+- New protocol: Center particles
+- New protocols: GL2D, GL2D streaming and GL2D static
+- New protocol: 2D kmeans clustering
+- New protocol: compare angles
+- New protocol: consensus 3D classes
+- New protocol: CTF consensus
+- New protocol: deep denoising
+- New protocols: Eliminate empty particles and eliminate empty classes
+- New protocol: Extract unit cell
+- New protocol: Generate reprojections
+- New protocol: metaprotocol heterogenety output, metaprotocol heterogeneity subset and metaprotocol heterogeneity
+- New protocol: Movie Max Shift
+- New protocol: particle boxsize
+- New protocol: pick noise
+- New protocol: significant heterogeneity
+- New protocol: swarm consensus intial volumes
+- New protocol: directional ResDir
+- New protocol: local monoTomo
+- New protocol: deep consensus picking
+- New protocol: screen deep learning
+- New protocol: split volume hierarchical
+- New protocol: trigger data

{scipion-em-xmipp-23.3.0.3/scipion_em_xmipp.egg-info → scipion-em-xmipp-23.11.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.0
 Name: scipion-em-xmipp
-Version: 23.3.0.3
+Version: 23.11.0.0
 Summary: Plugin to use Xmipp programs within the Scipion framework (see https://scipion-em.github.io/docs/).
 Home-page: https://github.com/i2pc/scipion-em-xmipp
 Author: I2PC

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/requirements.txt

@@ -1,7 +1,7 @@
 scipion-pyworkflow>=3.0.31
+scikit-learn==0.22
 scipion-em
-scikit-learn==0.22.1
-scipy<=1.8.1
+scipy<=1.10.0
 joblib
 peppercorn
 plotter

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0/scipion_em_xmipp.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.0
 Name: scipion-em-xmipp
-Version: 23.3.0.3
+Version: 23.11.0.0
 Summary: Plugin to use Xmipp programs within the Scipion framework (see https://scipion-em.github.io/docs/).
 Home-page: https://github.com/i2pc/scipion-em-xmipp
 Author: I2PC

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/scipion_em_xmipp.egg-info/SOURCES.txt

@@ -1,4 +1,4 @@
-CHANGES.txt
+CHANGES.md
 LICENSE
 MANIFEST.in
 README.md
@@ -28,8 +28,8 @@ xmipp3/protocols/protocol_align_volume.py
 xmipp3/protocols/protocol_align_volume_and_particles.py
 xmipp3/protocols/protocol_analyze_local_ctf.py
 xmipp3/protocols/protocol_angular_graph_consistency.py
+xmipp3/protocols/protocol_angular_resolution_alignment.py
 xmipp3/protocols/protocol_apply_alignment.py
-xmipp3/protocols/protocol_apply_deformation_zernike3d.py
 xmipp3/protocols/protocol_apply_transformation_matrix.py
 xmipp3/protocols/protocol_apply_zernike3d.py
 xmipp3/protocols/protocol_assignment_tilt_pair.py
@@ -38,13 +38,9 @@ xmipp3/protocols/protocol_center_particles.py
 xmipp3/protocols/protocol_cl2d.py
 xmipp3/protocols/protocol_cl2d_align.py
 xmipp3/protocols/protocol_classes_2d_mapping.py
-xmipp3/protocols/protocol_classification_gpuCorr.py
-xmipp3/protocols/protocol_classification_gpuCorr_full.py
-xmipp3/protocols/protocol_classification_gpuCorr_semi.py
-xmipp3/protocols/protocol_classify_kmeans2d.py
 xmipp3/protocols/protocol_compare_angles.py
 xmipp3/protocols/protocol_compare_reprojections.py
-xmipp3/protocols/protocol_consensus_classes3D.py
+xmipp3/protocols/protocol_consensus_classes.py
 xmipp3/protocols/protocol_consensus_local_ctf.py
 xmipp3/protocols/protocol_convert_pdb.py
 xmipp3/protocols/protocol_core_analysis.py
@@ -53,6 +49,7 @@ xmipp3/protocols/protocol_ctf_consensus.py
 xmipp3/protocols/protocol_ctf_correct_wiener2d.py
 xmipp3/protocols/protocol_ctf_defocus_group.py
 xmipp3/protocols/protocol_ctf_micrographs.py
+xmipp3/protocols/protocol_deep_center_assignment.py
 xmipp3/protocols/protocol_deep_denoising.py
 xmipp3/protocols/protocol_deep_hand.py
 xmipp3/protocols/protocol_deep_micrograph_screen.py
@@ -67,11 +64,11 @@ xmipp3/protocols/protocol_flexalign.py
 xmipp3/protocols/protocol_generate_reprojections.py
 xmipp3/protocols/protocol_helical_parameters.py
 xmipp3/protocols/protocol_kerdensom.py
-xmipp3/protocols/protocol_kmeans_clustering.py
 xmipp3/protocols/protocol_local_ctf.py
 xmipp3/protocols/protocol_metaprotocol_create_subset.py
 xmipp3/protocols/protocol_metaprotocol_golden_highres.py
 xmipp3/protocols/protocol_ml2d.py
+xmipp3/protocols/protocol_movie_dose_analysis.py
 xmipp3/protocols/protocol_movie_gain.py
 xmipp3/protocols/protocol_movie_max_shift.py
 xmipp3/protocols/protocol_movie_opticalflow.py
@@ -79,7 +76,6 @@ xmipp3/protocols/protocol_movie_split_frames.py
 xmipp3/protocols/protocol_multiple_fscs.py
 xmipp3/protocols/protocol_multireference_alignability.py
 xmipp3/protocols/protocol_normalize_strain.py
-xmipp3/protocols/protocol_particle_boxsize.py
 xmipp3/protocols/protocol_particle_pick.py
 xmipp3/protocols/protocol_particle_pick_automatic.py
 xmipp3/protocols/protocol_particle_pick_consensus.py
@@ -102,7 +98,6 @@ xmipp3/protocols/protocol_resolution_directional.py
 xmipp3/protocols/protocol_resolution_fso.py
 xmipp3/protocols/protocol_resolution_monogenic_signal.py
 xmipp3/protocols/protocol_rotate_volume.py
-xmipp3/protocols/protocol_rotational_spectra.py
 xmipp3/protocols/protocol_rotational_symmetry.py
 xmipp3/protocols/protocol_screen_deepConsensus.py
 xmipp3/protocols/protocol_screen_deeplearning.py
@@ -112,7 +107,6 @@ xmipp3/protocols/protocol_shift_volume.py
 xmipp3/protocols/protocol_simulate_ctf.py
 xmipp3/protocols/protocol_solid_angles.py
 xmipp3/protocols/protocol_split_volume.py
-xmipp3/protocols/protocol_split_volume_hierarchical_cluster.py
 xmipp3/protocols/protocol_structure_map.py
 xmipp3/protocols/protocol_structure_map_zernike3d.py
 xmipp3/protocols/protocol_subtract_projection.py
@@ -124,6 +118,7 @@ xmipp3/protocols/protocol_validate_overfitting.py
 xmipp3/protocols/protocol_volume_adjust_sub.py
 xmipp3/protocols/protocol_volume_consensus.py
 xmipp3/protocols/protocol_volume_deform_zernike3d.py
+xmipp3/protocols/protocol_volume_local_adjust.py
 xmipp3/protocols/protocol_volume_local_sharpening.py
 xmipp3/protocols/protocol_volume_strain.py
 xmipp3/protocols/protocol_write_testC.py
@@ -152,6 +147,7 @@ xmipp3/tests/test_protocol_apply_transformation_matrix.py
 xmipp3/tests/test_protocol_compare_angles.py
 xmipp3/tests/test_protocol_consensus_classes3D.py
 xmipp3/tests/test_protocol_ctf_consensus.py
+xmipp3/tests/test_protocol_deep_center_assignment.py
 xmipp3/tests/test_protocol_deep_denoising.py
 xmipp3/tests/test_protocol_extract_asymmetric_unit.py
 xmipp3/tests/test_protocol_monodir.py
@@ -163,13 +159,11 @@ xmipp3/tests/test_protocol_simulate_ctf.py
 xmipp3/tests/test_protocol_validate_fscq.py
 xmipp3/tests/test_protocol_validate_overfitting.py
 xmipp3/tests/test_protocols_add_noise.py
+xmipp3/tests/test_protocols_angular_resolution_alignment.py
 xmipp3/tests/test_protocols_continuousflex.py
 xmipp3/tests/test_protocols_deepVolPostprocessing.py
 xmipp3/tests/test_protocols_deepres.py
 xmipp3/tests/test_protocols_fso.py
-xmipp3/tests/test_protocols_gpuCorr_classifier.py
-xmipp3/tests/test_protocols_gpuCorr_fullStreaming.py
-xmipp3/tests/test_protocols_gpuCorr_semiStreaming.py
 xmipp3/tests/test_protocols_highres.py
 xmipp3/tests/test_protocols_local_defocus.py
 xmipp3/tests/test_protocols_localdeblur.py
@@ -188,13 +182,14 @@ xmipp3/viewers/__init__.py
 xmipp3/viewers/plotter.py
 xmipp3/viewers/viewer.py
 xmipp3/viewers/viewer_analyze_local_ctf.py
+xmipp3/viewers/viewer_angular_resolution_alignment.py
 xmipp3/viewers/viewer_cl2d.py
-xmipp3/viewers/viewer_classes3D.py
-xmipp3/viewers/viewer_consensus_classes3D.py
+xmipp3/viewers/viewer_consensus_classes.py
 xmipp3/viewers/viewer_ctf_consensus.py
 xmipp3/viewers/viewer_deepEMHancer.py
 xmipp3/viewers/viewer_deep_consensus.py
 xmipp3/viewers/viewer_deep_micrograph_cleaner.py
+xmipp3/viewers/viewer_dose_analysis.py
 xmipp3/viewers/viewer_eliminate_empty_images.py
 xmipp3/viewers/viewer_extract_asymmetric_unit.py
 xmipp3/viewers/viewer_local_sharpening.py

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/scipion_em_xmipp.egg-info/requires.txt

@@ -1,7 +1,7 @@
 scipion-pyworkflow>=3.0.31
+scikit-learn==0.22
 scipion-em
-scikit-learn==0.22.1
-scipy<=1.8.1
+scipy<=1.10.0
 joblib
 peppercorn
 plotter

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/xmipp3/__init__.py

@@ -37,12 +37,15 @@ import pyworkflow.utils as pwutils
 from .base import *
 from .constants import XMIPP_HOME, XMIPP_URL, XMIPP_DLTK_NAME
 
-_logo = "xmipp_logo.png"
-_references = ['delaRosaTrevin2013', 'Sorzano2013']
-_currentBinVersion = '3.23.03.0'
-# This will allow scipion to detect the plugin version and print it in the log.
-__version__ = _currentBinVersion[2:] + ".3"  # Set this to ".0" on each xmipp binary release, otherwise increase it --> ".1", ".2", ...
+type_of_version = 'release' #'devel'
+_logo = "xmipp_logo" + ("" if type_of_version == 'release' else '_devel') + '.png'
 
+_references = ['delaRosaTrevin2013', 'Sorzano2013', 'Strelak2021']
+_currentBinVersion = '3.23.11.0'
+__version__ = _currentBinVersion[2:] + ".0"  # Set this to ".0" on each xmipp binary release, otherwise increase it --> ".1", ".2", ...
+
+# Requirement version variables
+NVIDIA_DRIVERS_MINIMUM_VERSION = 450
 
 class Plugin(pwem.Plugin):
     _homeVar = XMIPP_HOME
@@ -209,38 +212,37 @@ def installDeepLearningToolkit(plugin, env):
                                                env=plugin.getEnviron(),
                                                stdout=subprocess.PIPE
                                                ).stdout.read().decode('utf-8').split(".")[0]
-            try:
-                int(nvidiaDriverVer)
-            except ValueError:
-                nvidiaDriverVer = None
-            if int(nvidiaDriverVer) < 390:
+            if int(nvidiaDriverVer) < NVIDIA_DRIVERS_MINIMUM_VERSION:
                 preMsgs.append("Incompatible driver %s" % nvidiaDriverVer)
-                cudaMsgs.append("Your NVIDIA drivers are too old (<390). "
+                cudaMsgs.append(f"Your NVIDIA drivers are too old (<{NVIDIA_DRIVERS_MINIMUM_VERSION}). "
                                 "Tensorflow was installed without GPU support. "
-                                "Just CPU computations enabled (slow computations).")
+                                "Just CPU computations enabled (slow computations)."
+                                f"To enable CUDA (drivers>{NVIDIA_DRIVERS_MINIMUM_VERSION} needed), "
+                                "set CUDA=True in 'scipion.conf' file")
                 nvidiaDriverVer = None
-        except Exception as e:
-            preMsgs.append(str(e))
+        except (ValueError, TypeError):
+            nvidiaDriverVer = None
+            preMsgs.append("Not nvidia driver found. Type: "
+                           " nvidia-smi --query-gpu=driver_version --format=csv,noheader")
+            preMsgs.append(
+                "CUDA will NOT be USED. (not found or incompatible)")
+            msg = ("Tensorflow installed without GPU. Just CPU computations "
+                   "enabled (slow computations).")
+            cudaMsgs.append(msg)
+            useGpu = False
 
     if nvidiaDriverVer is not None:
-        preMsgs.append("CUDA support find. Driver version: %s" % nvidiaDriverVer)
-        msg = "Tensorflow installed with CUDA SUPPORT."
+        preMsgs.append("CUDA support found. Driver version: %s" % nvidiaDriverVer)
+        msg = "Tensorflow will be installed with CUDA SUPPORT."
         cudaMsgs.append(msg)
         useGpu = True
-    else:
-        preMsgs.append("CUDA will NOT be USED. (not found or incompatible)")
-        msg = ("Tensorflow installed without GPU. Just CPU computations "
-               "enabled (slow computations). To enable CUDA (drivers>390 needed), "
-               "set CUDA=True in 'scipion.conf' file")
-        cudaMsgs.append(msg)
-        useGpu = False
+
 
     # commands  = [(command, target), (cmd, tgt), ...]
-    cmdsInstall = [(cmd, envName + ".yml") for cmd, envName in
-                   CondaEnvManager.yieldInstallAllCmds(useGpu=useGpu)]
+    cmdsInstall = list(CondaEnvManager.yieldInstallAllCmds(useGpu=useGpu))
 
     now = datetime.now()
-    installDLvars = {'modelsUrl': "http://scipion.cnb.csic.es/downloads/scipion/software/em",
+    installDLvars = {'modelsUrl': "https://scipion.cnb.csic.es/downloads/scipion/software/em",
                      'syncBin': plugin.getHome('bin/xmipp_sync_data'),
                      'modelsDir': plugin.getHome('models'),
                      'modelsPrefix': "models_UPDATED_on",
@@ -267,6 +269,6 @@ def installDeepLearningToolkit(plugin, env):
                          " fi" % installDLvars,           # End of command
                          installDLvars['xmippLibToken'])  # Target
 
-    env.addPackage(XMIPP_DLTK_NAME, version='0.2', urlSuffix='external',
+    env.addPackage(XMIPP_DLTK_NAME, version='1.0', urlSuffix='external',
                    commands=[xmippInstallCheck]+cmdsInstall+[modelsDownloadCmd],
                    deps=[], tar=XMIPP_DLTK_NAME+'.tgz')

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/xmipp3/bibtex.py

@@ -604,5 +604,59 @@ url = {https://www.sciencedirect.com/science/article/pii/S104784772100085X},
 author = {E. Fernández-Giménez and M. Martínez and R. Sánchez-García and R. Marabini and E. Ramírez-Aportela and
 P. Conesa and J.M. Carazo and C.O.S. Sorzano},
 }
+
+@Article{Strelak2021,
+AUTHOR = {Strelak, David and Jiménez-Moreno, Amaya and Vilas, José L. and Ramírez-Aportela, Erney and Sánchez-García, Ruben and Maluenda, David and Vargas, Javier and Herreros, David and Fernández-Giménez, Estrella and de Isidro-Gómez, Federico P. and Horacek, Jan and Myska, David and Horacek, Martin and Conesa, Pablo and Fonseca-Reyna, Yunior C. and Jiménez, Jorge and Martínez, Marta and Harastani, Mohamad and Jonić, Slavica and Filipovic, Jiri and Marabini, Roberto and Carazo, José M. and Sorzano, Carlos O. S.},
+TITLE = {Advances in Xmipp for Cryo–Electron Microscopy: From Xmipp to Scipion},
+JOURNAL = {Molecules},
+VOLUME = {26},
+YEAR = {2021},
+NUMBER = {20},
+ARTICLE-NUMBER = {6224},
+URL = {https://www.mdpi.com/1420-3049/26/20/6224},
+PubMedID = {34684805},
+ISSN = {1420-3049},
+DOI = {https://doi.org/10.3390/molecules26206224}
+}
+
+
+@article{Fernandez-Gimenez2023a,
+title = {A new algorithm for particle weighted subtraction to eliminate signals from unwanted components in Single Particle Analysis},
+journal = {Journal of Structural Biology},
+volume = {215},
+number = {4},
+year = {2023},
+issn = {1047-8477},
+doi = {https://doi.org/10.1016/j.jsb.2023.108024},
+url = {https://www.sciencedirect.com/science/article/pii/S1047847723000874},
+author = {E. Fernández-Giménez, M. Martínez, R. Marabini, D. Strelak, R. Sánchez-García, J.M. Carazo and C.O.S. Sorzano},
+}
+
+@article{Fernandez-Gimenez2023b,
+title = {Local defocus estimation in Single Particle Analysis in Cryo-Electron Microscopy},
+journal = {Journal of Structural Biology},
+volume = {215},
+number = {4},
+year = {2023},
+issn = {1047-8477},
+doi = {https://doi.org/10.1016/j.jsb.2023.108030},
+url = {https://www.sciencedirect.com/science/article/pii/S104784772300093X},
+author = {E. Fernández-Giménez, J.M. Carazo and C.O.S. Sorzano},
+
+
+@Article{Strelak2023performance,
+AUTHOR = {Střelák, David and Marchán, Daniel and Carazo, José María and S. Sorzano, Carlos O.},
+TITLE = {Performance and Quality Comparison of Movie Alignment Software for Cryogenic Electron Microscopy},
+JOURNAL = {Micromachines},
+VOLUME = {14},
+YEAR = {2023},
+NUMBER = {10},
+ARTICLE-NUMBER = {1835},
+URL = {https://www.mdpi.com/2072-666X/14/10/1835},
+ISSN = {2072-666X},
+DOI = {https://doi.org/10.3390/mi14101835}
+
+}
+
 """
 

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/xmipp3/convert/io_coordinates.py

@@ -17,9 +17,10 @@ def readSetOfCoordsFromPosFnames(posDir, setOfInputCoords, sqliteOutName,
     """
 
     inputMics = setOfInputCoords.getMicrographs()
+    inputMicsPointer = setOfInputCoords.getMicrographs(asPointer=True)
     cleanPath(sqliteOutName)
     setOfOutputCoordinates= SetOfCoordinates(filename= sqliteOutName)
-    setOfOutputCoordinates.setMicrographs(inputMics)
+    setOfOutputCoordinates.setMicrographs(inputMicsPointer)
     setOfOutputCoordinates.setBoxSize(setOfInputCoords.getBoxSize())
     readSetOfCoordinates(posDir, micSet=inputMics,
                            coordSet=setOfOutputCoordinates,

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/xmipp3/protocols/__init__.py

@@ -31,6 +31,7 @@ from .protocol_preprocess import *
 from .protocol_assignment_tilt_pair import XmippProtAssignmentTiltPair
 from .protocol_align_volume import XmippProtAlignVolume, XmippProtAlignVolumeForWeb
 from .protocol_angular_graph_consistency import XmippProtAngularGraphConsistency
+from .protocol_angular_resolution_alignment import XmippProtResolutionAlignment
 from .protocol_preprocess.protocol_add_noise import (XmippProtAddNoiseVolumes,
                                                      XmippProtAddNoiseParticles)
 from .protocol_apply_alignment import XmippProtApplyAlignment
@@ -38,10 +39,7 @@ from .protocol_apply_transformation_matrix import XmippProtApplyTransformationMa
 from .protocol_break_symmetry import XmippProtAngBreakSymmetry
 from .protocol_cl2d_align import XmippProtCL2DAlign
 from .protocol_cl2d import XmippProtCL2D
-from .protocol_classify_kmeans2d import XmippProtKmeansClassif2D
-from .protocol_classification_gpuCorr import XmippProtGpuCrrCL2D
-from .protocol_classification_gpuCorr_semi import XmippProtStrGpuCrrSimple
-from .protocol_classification_gpuCorr_full import XmippProtStrGpuCrrCL2D
+#from .protocol_classify_kmeans2d import XmippProtKmeansClassif2D
 from .protocol_ctf_defocus_group import XmippProtCTFDefocusGroup
 from .protocol_compare_reprojections import XmippProtCompareReprojections
 from .protocol_compare_angles import XmippProtCompareAngles
@@ -51,7 +49,7 @@ from .protocol_create_gallery import XmippProtCreateGallery
 from .protocol_ctf_consensus import XmippProtCTFConsensus
 from .protocol_ctf_micrographs import XmippProtCTFMicrographs
 from .protocol_ctf_correct_wiener2d import XmippProtCTFCorrectWiener2D
-from .protocol_consensus_classes3D import XmippProtConsensusClasses3D
+from .protocol_consensus_classes import XmippProtConsensusClasses
 from .protocol_denoise_particles import XmippProtDenoiseParticles
 from .protocol_deep_micrograph_screen import  XmippProtDeepMicrographScreen
 from .protocol_eliminate_empty_images import (XmippProtEliminateEmptyParticles,
@@ -68,6 +66,7 @@ from .protocol_movie_gain import XmippProtMovieGain
 from .protocol_flexalign import XmippProtFlexAlign
 from .protocol_movie_opticalflow import XmippProtOFAlignment, ProtMovieAlignment
 from .protocol_movie_max_shift import XmippProtMovieMaxShift
+from .protocol_movie_dose_analysis import XmippProtMovieDoseAnalysis
 from .protocol_movie_split_frames import XmippProtSplitFrames
 from .protocol_multiple_fscs import XmippProtMultipleFSCs
 from .protocol_multireference_alignability import XmippProtMultiRefAlignability
@@ -75,7 +74,7 @@ from .protocol_normalize_strain import XmippProtNormalizeStrain
 from .protocol_particle_pick_automatic import XmippParticlePickingAutomatic
 from .protocol_particle_pick_consensus import XmippProtConsensusPicking
 from .protocol_pick_noise import XmippProtPickNoise
-from .protocol_particle_boxsize import XmippProtParticleBoxsize
+#from .protocol_particle_boxsize import XmippProtParticleBoxsize
 from .protocol_particle_pick import XmippProtParticlePicking
 from .protocol_particle_pick_pairs import XmippProtParticlePickingPairs
 from .protocol_phantom_create import XmippProtPhantom
@@ -96,7 +95,7 @@ from .protocol_resolution_monogenic_signal import XmippProtMonoRes
 from .protocol_resolution_deepres import XmippProtDeepRes
 from .protocol_postProcessing_deepPostProcessing import XmippProtDeepVolPostProc
 from .protocol_rotate_volume import XmippProtRotateVolume
-from .protocol_rotational_spectra import XmippProtRotSpectra
+#from .protocol_rotational_spectra import XmippProtRotSpectra
 from .protocol_rotational_symmetry import XmippProtRotationalSymmetry
 from .protocol_screen_particles import XmippProtScreenParticles
 from .protocol_screen_deepConsensus import XmippProtScreenDeepConsensus, XmippProtDeepConsSubSet
@@ -119,7 +118,6 @@ from .protocol_write_testC import XmippProtWriteTestC
 from .protocol_write_testP import XmippProtWriteTestP
 from .protocol_generate_reprojections import XmippProtGenerateReprojections
 from .protocol_deep_denoising import XmippProtDeepDenoising  # deepDenoising has to be after XmippProtGenerateReprojections and XmippProtCompareReprojections
-from .protocol_split_volume_hierarchical_cluster import XmippProtSplitVolumeHierarchical
 from .protocol_metaprotocol_create_subset import XmippMetaProtCreateSubset
 from .protocol_volume_deform_zernike3d import XmippProtVolumeDeformZernike3D
 from .protocol_structure_map_zernike3d import XmippProtStructureMapZernike3D
@@ -135,5 +133,7 @@ from .protocol_structure_map import XmippProtStructureMap
 from .protocol_apply_zernike3d import XmippApplyZernike3D
 from .protocol_volume_adjust_sub import XmippProtVolAdjust, XmippProtVolSubtraction
 from .protocol_volume_consensus import XmippProtVolConsensus
+from .protocol_volume_local_adjust import XmippProtLocalVolAdj
 from .protocol_classes_2d_mapping import XmippProtCL2DMap
 from .protocol_deep_hand import XmippProtDeepHand
+from .protocol_deep_center_assignment import XmippProtDeepCenter, XmippProtDeepGlobalAssignment

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/xmipp3/protocols/protocol_align_volume.py

@@ -32,6 +32,7 @@ from pwem.protocols import ProtAlignVolume
 from pwem.objects import Volume, Transform, SetOfVolumes
 
 from xmipp3.convert import getImageLocation
+from pyworkflow import BETA, UPDATED, NEW, PROD
 
 
 ALIGN_MASK_CIRCULAR = 0
@@ -51,6 +52,8 @@ class XmippProtAlignVolume(ProtAlignVolume):
      """
     _label = 'align volume'
     nVols = 0
+    _devStatus = PROD
+
     
     def __init__(self, **args):
         ProtAlignVolume.__init__(self, **args)
@@ -200,7 +203,9 @@ class XmippProtAlignVolume(ProtAlignVolume):
             fnOutVol = self._getExtraPath("vol%02d.mrc"%idx)
             outVol.setLocation(fnOutVol)
             outVol.setObjComment(vol.getObjComment())
-            #set transformation matrix             
+            outVol.setObjLabel(vol.getObjLabel())
+
+            #set transformation matrix
             fhInputTranMat = self._getExtraPath('transformation-matrix_vol%06d.txt'%idx)
             transMatFromFile = np.loadtxt(fhInputTranMat)
             transformationMat = np.reshape(transMatFromFile,(4,4))
@@ -329,7 +334,7 @@ class XmippProtAlignVolume(ProtAlignVolume):
         return maskArgs
     
     def _getAlignArgs(self):
-        alignArgs = ''
+        alignArgs = ' --dontWrap'
         
         if self.alignmentAlgorithm == ALIGN_ALGORITHM_FAST_FOURIER:
             alignArgs += " --frm"

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/xmipp3/protocols/protocol_align_volume_and_particles.py

@@ -256,7 +256,7 @@ class XmippProtAlignVolumeParticles(ProtAlignVolume):
             if self.maskType == ALIGN_MASK_CIRCULAR:
                 maskArgs+=" --mask circular -%d" % self.maskRadius
             else:
-                maskArgs+=" --mask binary_file %s" % self.volMask
+                maskArgs+=" --mask binary_file %s" % self.maskFile.get().getFileName()
         return maskArgs
 
           

{scipion-em-xmipp-23.3.0.3 → scipion-em-xmipp-23.11.0.0}/xmipp3/protocols/protocol_analyze_local_ctf.py

@@ -34,6 +34,8 @@ from pwem.protocols import ProtAnalysis3D
 import pwem.emlib.metadata as md
 from xmipp3.convert import readSetOfMicrographs, writeSetOfMicrographs
 
+CITE ='Fernandez-Gimenez2023b'
+
 
 class XmippProtAnalyzeLocalCTF(ProtAnalysis3D):
     """Assigns to each micrograph a coefficient (R2) which evaluates the result of the