scipion-pyworkflow 3.10.5__tar.gz → 3.11.0__tar.gz

{scipion_pyworkflow-3.10.5 → scipion_pyworkflow-3.11.0}/CHANGES.txt

@@ -1,3 +1,52 @@
+V3.11.0
+users:
+  - New behaviour: Saving a protocol will unload the form and log the confirmation message. There will not be any pop up message.
+        -New variable: SCIPION_UNLOAD_FORM_ON_SAVE (True). define it in the config file to False to go back to old behaviour.
+  - GPU ID are now always based on the PCI slow. Should match what nvidia-smi output as IDs: CUDA_DEVICE_ORDER='PCI_BUS_ID'
+  - Improving subset creation performance
+  - Usage report:
+      Sent upon project closing and not upon opening.
+      Sent once day(default), even if it hasn't changed.
+  - Suggestion mode added to the "add protocol" dialog:
+      Button added. An empty search "" activates the suggestions. Otherwise search on the installed protocols
+      Suggestions works with empty selection (suggestion as first protocol).
+      Adding a new protocol may be linked automatically with a compatible output of the "Selected protocol"
+  - Host.conf parsing feedback. When there is a parsing error in the host.conf it will report more precisely the cause
+  - Parallel section refactored: Scipion threads and or "Package compute" lines.
+  - Allow for having the three compute fields: Scipion threads, threads and mpis.
+  - Fix: jobs, processes killed are detected by scipion.
+  - Project log has info about protocol execution modes: queue, or subprocess
+  - Killed protocol process (timeout in queue or plain killed) are detected
+  - SCIPION_PROJECT label can be used in host.conf: richer job names
+  - PLUGIN_MODULE variable can be used in host.conf to customize mpirun command: mpirun.PLUGIN_MODULE(mpirun.relion)
+  - Live change of loglevel during GUI
+  - Bug fix: viewers shown when"searching for input" are now correct.
+  - Report better when running a protocol in node and protocol does not load (installation mismatch).
+  - Bug fix: stop launched protocol with PID and  the PID died.
+  - Menu > Project > "Remove temporary files" remove folders too.
+  - File browser: 2 new actions on right click: "delete" and "copy"
+ developers:
+  - Protocol.getParam(name): returns the parameter definition by name.
+  - stepsExecutionMode has to be declared at class level not instance level.
+  - updateProtocol refactored to cover QUEUE timeout cases
+  - Updating the protocol process id (_pid) to 0 when the finish it.
+  - Viewers can evaluate the object that is going to be used in can_handle_this method
+  - using toml
+  - Config.getEnvName(): return Scipion's env name
+V3.10.6:
+  - Improving subset creation
+  - PLUGIN_MODULE variable can be used in host.conf to customize mpirun command: mpirun.PLUGIN_MODULE --> mpirun.relion)
+       This approach implies having this pattern for all mpi plugins that use mpi.
+  - Alternatively, in the scipion.conf file a variable called PARALLEL_COMMAND_XXX could be defined for a more
+       versatile way to have a mpi command only for some plugins. Example:
+       PARALLEL_COMMAND_RELION=mpirun.relion -np %%(JOB_NODES)d --map-by node %%(COMMAND)s
+       NOTE: in this config file, %_ (valid and needed in host.conf) should be replaced by %%
+  - GPU Ids are automatically anonymized (0-len(GPUs)) when using queues
+  - Bug-fix: creation time is not loaded when loadig run.db (updating a protocol)
+  developers:
+  - Adding a Streaming section in ProtStreamingBase
+  - runJob call without %(GPU)s will not reserve GPU slots anymore.
+   
 V3.10.5: cancel usage of SCIPION_PROJECT in the host template
 V3.10.4: hotfix: Parallelization with GPUs is fixed and step queue executor fixed with the new implementation.
 V3.10.3: Failed hotfix
@@ -5,11 +54,12 @@ V3.10.2: hotfix: Executor frees the GPU/s used via step id instead of node.
 V3.10.1: hotfix: Avoid double Gpu assignment in scipion parallelize protocols with concurrent GPU steps
 V3.10.0
 developers:
-  -  stepsExecutionMode = STEPS_SERIAL at class level. Should a class attribute for future versions. Now both approaches are valid.
+  - stepsExecutionMode = STEPS_SERIAL at class level. Should be a class attribute for future versions. Now both approaches are valid.
 
 V3.9.2 - hotfix: rescue commented methods: Protocol._insertRunJobStep and Protocol._insertCopyFileStep
 V3.9.1 - hotfix: loading related objects (CTF - Mics in particle extraction)
 V3.9.0
+users:
   - Project can have a comment (tooltip over the project name and click will edit it). Will be populated with templates description.
   - Project Color modes:
        size color mode removed: Problematic in slow setups or huge projects.

{scipion_pyworkflow-3.10.5/scipion_pyworkflow.egg-info → scipion_pyworkflow-3.11.0}/PKG-INFO

@@ -1,22 +1,15 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: scipion-pyworkflow
-Version: 3.10.5
-Summary: Simple workflow platform used in scientific applications, initially developed within the Scipion framework for image processing in Electron Microscopy.
-Home-page: https://github.com/scipion-em/scipion-pyworkflow
-Author: J.M. De la Rosa Trevin, Roberto Marabini, Grigory Sharov, Josue Gomez Blanco, Pablo Conesa, Yunior Fonseca Reyna
-Author-email: delarosatrevin@scilifelab.se, roberto@cnb.csic.es, gsharov@mrc-lmb.cam.ac.uk, josue.gomez-blanco@mcgill.ca, pconesa@cnb.csic.es, fonsecareyna@cnb.csic.es
-Project-URL: Bug Reports, https://github.com/scipion-em/scipion-pyworkflow/issues
-Project-URL: Source, https://github.com/scipion-em/scipion-pyworkflow/
-Keywords: workflows science electron-microscopy cryo-em structural-biology image-processing scipion
-Classifier: Development Status :: 5 - Production/Stable
-Classifier: Intended Audience :: Science/Research
-Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Scientific/Engineering
+Version: 3.11.0
+Summary: Workflow platform used in scientific applications, initially developed
+Author-email: Pablo Conesa <pconesa@cnb.csic.es>, Yunior Fonseca <cfonseca@cnb.csic.es>, "J.M. De la Rosa Trevin" <josemiguel.delarosatrevin@stjude.org>, Roberto Marabini <roberto@cnb.csic.es>, Grigory Sharov <sharov.grigory@gmail.com>, Josue Gomez Blanco <josue.gomez-blanco@mcgill.ca>
+License: GNU General Public License v3 (GPLv3)
+Project-URL: Homepage, https://scipion.i2pc.es
+Project-URL: Repository, https://github.com/scipion-em/scipion-pyworkflow
+Project-URL: Issues, https://github.com/scipion-em/scipion-pyworkflow/issues
+Keywords: scipion,electron-microscopy,cryo-em,structural-biology,generic-workflow-engine,image-processing,scipion-3.0
+Requires-Python: >=3.8
+Description-Content-Type: text/x-rst
 License-File: LICENSE.txt
 Requires-Dist: bibtexparser<=1.4.1
 Requires-Dist: psutil<=5.9.6
@@ -27,9 +20,12 @@ Requires-Dist: pillow==10.1.0; python_version >= "3.8"
 Requires-Dist: numpy==1.24.4; python_version == "3.8"
 Requires-Dist: numpy==1.26.1; python_version >= "3.9"
 Requires-Dist: requests==2.31.0; python_version >= "3.8"
-Requires-Dist: tkcolorpicker
+Requires-Dist: tkcolorpicker2
 Requires-Dist: distro<=1.8
 Requires-Dist: importlib-metadata<=6.8.0
+Requires-Dist: setuptools>=62.6
+Requires-Dist: pyperclip==1.9.0
+Dynamic: license-file
 
 .. image:: https://img.shields.io/pypi/v/scipion-pyworkflow.svg
         :target: https://pypi.python.org/pypi/scipion-pyworkflow

scipion_pyworkflow-3.11.0/pyproject.toml

@@ -0,0 +1,43 @@
+[build-system]
+requires = ["setuptools>=42", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "scipion-pyworkflow"
+dynamic = ["version", "dependencies"]
+
+authors = [
+    {name = "Pablo Conesa", email = "pconesa@cnb.csic.es"},
+    {name = "Yunior Fonseca", email = "cfonseca@cnb.csic.es"},
+    {name = "J.M. De la Rosa Trevin", email = "josemiguel.delarosatrevin@stjude.org"},
+    {name = "Roberto Marabini", email = "roberto@cnb.csic.es"},
+    {name = "Grigory Sharov", email = "sharov.grigory@gmail.com"},
+    {name = "Josue Gomez Blanco", email = "josue.gomez-blanco@mcgill.ca"},
+]
+description = """Workflow platform used in scientific applications, initially developed
+            within the Scipion framework for image processing in Electron Microscopy but
+            generic by design to be applied to any domain."""
+
+readme = "README.rst"
+requires-python = ">=3.8"
+keywords = ["scipion", "electron-microscopy", "cryo-em", "structural-biology", "generic-workflow-engine", "image-processing", "scipion-3.0"]
+license = {text = "GNU General Public License v3 (GPLv3)"}
+
+[project.urls]
+Homepage = "https://scipion.i2pc.es"
+Repository = "https://github.com/scipion-em/scipion-pyworkflow"
+Issues = "https://github.com/scipion-em/scipion-pyworkflow/issues"
+
+[tool.setuptools]
+packages = ["pyworkflow", "pyworkflowtests"]
+
+[tool.setuptools.package-data]
+"pyworkflow" = ["resources/*", "project/scripts/*"]
+
+[project.entry-points."pyworkflow.plugin"]
+pyworkflowtests = "pyworkflowtests"
+
+
+[tool.setuptools.dynamic]
+version = {attr = "pyworkflow.__version__"}
+dependencies = {file = ["requirements.txt"]}

{scipion_pyworkflow-3.10.5 → scipion_pyworkflow-3.11.0}/pyworkflow/config.py

@@ -97,12 +97,15 @@ class Variable:
         self.value = new_value
         self.isDefault= self._isValueDefault()
     def _isValueDefault(self):
-        return self.value==self.default
+        return self.value == self.default
+
+
 class VariablesRegistry:
-    _variables={}
+    _variables = {}
 
     def __init__(self):
         raise RuntimeError("Variables class doesn't need to be instantiated.")
+
     @classmethod
     def register(cls, variable: Variable):
         cls._variables[variable.name] = variable
@@ -146,7 +149,7 @@ class Config:
 
         if key in os.environ:
             value = os.environ.get(key)
-            isDefault = (value==default)
+            isDefault = (value == default)
         else:
             isDefault = True
             value = default
@@ -154,10 +157,10 @@ class Config:
         # If the caster is passed do the casting, if fails go back to default
         if caster:
             try:
-                value=caster(value)
+                value = caster(value)
             except:
                 logger.warning("Variable %s has this value %s that can't be casted to the right type (%s). Using %s (default value)" %
-                               (key,value, caster, default))
+                               (key, value, caster, default))
                 value = default
         # If empty use default value
         if value == "" != default:
@@ -168,10 +171,23 @@ class Config:
         if isinstance(value, str):
             value = os.path.expandvars(os.path.expanduser(value))
 
-        # Register the variable
+        # Register the variable. Boolean variables are converted to booleans after this call. May not be accurate.
+        # 1 turns into False in the Config
         VariablesRegistry.register(Variable(key,description, source, value, default, var_type=var_type, isDefault=isDefault))
         return value
 
+    @staticmethod
+    def __notFalse(value):
+        return value != FALSE_STR
+
+    @staticmethod
+    def __notTrue(value):
+        return value != TRUE_STR
+
+    @staticmethod
+    def __bool(value):
+        return value.lower() in TRUE_YES_ON_
+
     class Root:
         """ Simple helper to return path from a root. """
 
@@ -185,13 +201,17 @@ class Config:
             # join will not join if expanded is absolute
             return os.path.join(self._root, expanded)
 
-    # Home for scipion
     _get = __get.__func__
-    SCIPION_HOME = os.path.abspath(_get(SCIPION_HOME_VAR, '',
-    "Path where Scipion is installed. Other paths are based on this like SCIPION_SOFTWARE, SCIPION_TESTS,... unless specified"))
+    _notFalse = __notFalse.__func__
+    __bool = __bool.__func__
+
+    # Home for scipion
+    _home_var = _get(SCIPION_HOME_VAR, '',
+                     "Path where Scipion is installed. Other paths are based on this like SCIPION_SOFTWARE, SCIPION_TESTS,... unless specified")
+    SCIPION_HOME = os.path.abspath(_home_var)
 
     # False if SCIPION_HOME is not found in the environment. To distinguish API documentation generation execution.
-    SCIPION_HOME_DEFINED = SCIPION_HOME != ''
+    SCIPION_HOME_DEFINED = _home_var != ''
 
     _root = Root(str(SCIPION_HOME))
     _join = _root.join
@@ -201,68 +221,75 @@ class Config:
     __defaultSpritesFile = _join(getResourcesPath(),'sprites.png')
 
     CONDA_ACTIVATION_CMD = _get(CONDA_ACTIVATION_CMD_VAR,'',
-    "str: Command to activate/initialize conda itself. Do not confuse it with 'conda activate'. It should be defined at installation time. It looks like this: eval \"$(/extra/miniconda3/bin/conda shell.bash hook)\"")
+                                "str: Command to activate/initialize conda itself. Do not confuse it with 'conda activate'. It should be defined at installation time. It looks like this: eval \"$(/extra/miniconda3/bin/conda shell.bash hook)\"")
 
     # SCIPION PATHS
     SCIPION_SOFTWARE = _get('SCIPION_SOFTWARE', _join('software'),
-    "Path where Scipion will install the software. Defaults to SCIPION_HOME/software.", var_type=VarTypes.FOLDER)
+                            "Path where Scipion will install the software. Defaults to SCIPION_HOME/software.", var_type=VarTypes.FOLDER)
 
     SCIPION_TESTS = _get('SCIPION_TESTS', _join('data', 'tests'),
-    "Path where to find/download test data. Defaults to SCIPION_HOME/data/tests.", var_type=VarTypes.FOLDER)
+                         "Path where to find/download test data. Defaults to SCIPION_HOME/data/tests.", var_type=VarTypes.FOLDER)
 
     # User dependent paths
     SCIPION_USER_DATA = _get('SCIPION_USER_DATA', '~/ScipionUserData',
-    "Path where Scipion projects are or will be created. Defaults to ~/ScipionUserData", var_type=VarTypes.FOLDER)
+                             "Path where Scipion projects are or will be created. Defaults to ~/ScipionUserData", var_type=VarTypes.FOLDER)
 
     # LOGGING variables
     SCIPION_LOGS = _get('SCIPION_LOGS', _join(SCIPION_USER_DATA, 'logs'),
-    "Folder for Scipion logs used by the GUI. Defaults to SCIPION_USER_DATA/logs.", var_type=VarTypes.FOLDER)
+                        "Folder for Scipion logs used by the GUI. Defaults to SCIPION_USER_DATA/logs.", var_type=VarTypes.FOLDER)
 
     SCIPION_LOG_CONFIG = _get('SCIPION_LOG_CONFIG', None,
-    "Optional. Path to a python logging configuration file to fine tune the logging.", var_type=VarTypes.PATH)
+                              "Optional. Path to a python logging configuration file to fine tune the logging.", var_type=VarTypes.PATH)
 
     SCIPION_LOG = _get('SCIPION_LOG', _join(SCIPION_LOGS, 'scipion.log'),
-    "Path to the file where scipion will write GUI logging messages. Defaults to SCIPION_LOGS/scipion.log", var_type=VarTypes.PATH)
+                       "Path to the file where scipion will write GUI logging messages. Defaults to SCIPION_LOGS/scipion.log", var_type=VarTypes.PATH)
 
     SCIPION_LOG_FORMAT = _get('SCIPION_LOG_FORMAT', "%(message)s",
-    "str: Format for all the log lines, defaults to %(message)s. To compose the format see https://docs.python.org/3/library/logging.html#logrecord-attributes")
+                              "str: Format for all the log lines, defaults to %(message)s. To compose the format see https://docs.python.org/3/library/logging.html#logrecord-attributes")
 
     SCIPION_LOG_LEVEL = _get(SCIPION_LOG_LEVEL, 'INFO',
-    "Default logging level. String among CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET. Default value is INFO.")
+                             "Default logging level. String among CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET. Default value is INFO.")
 
     NO_COLOR = _get('NO_COLOR', '',
-    "str: Comply with https://no-color.org/ initiative. Set it to something different than '' to deactivate colors in the output.")
+                    "str: Comply with https://no-color.org/ initiative. Set it to something different than '' to deactivate colors in the output.")
 
     SCIPION_SCRATCH = _get(SCIPION_SCRATCH, None,
-    "Optional. Path to a location mounted in a scratch drive (SSD,...)")
+                           "Optional. Path to a location mounted in a scratch drive (SSD,...)")
 
     SCIPION_TESTS_OUTPUT = _get('SCIPION_TESTS_OUTPUT', _join(SCIPION_USER_DATA, 'Tests'),
-    "Path to a folder where the output of the tests will be written. Defaults to SCIPION_USER_DATA/Tests.", var_type=VarTypes.FOLDER)
+                                "Path to a folder where the output of the tests will be written. Defaults to SCIPION_USER_DATA/Tests.", var_type=VarTypes.FOLDER)
 
     SCIPION_TEST_NOSYNC = _get('SCIPION_TEST_NOSYNC', FALSE_STR,
-    "Set it to 1, True, Yes or y to cancel test dataset synchronization. Needed when updating files in a dataset.") != FALSE_STR
+                               "Set it to any value except False to cancel test dataset synchronization."
+                               " Needed when updating files in a dataset.", caster=_notFalse)
 
     SCIPION_SUPPORT_EMAIL = 'scipion@cnb.csic.es'
 
     # Config variables
     SCIPION_CONFIG = _get('SCIPION_CONFIG', _join('config','scipion.conf'),
-    "Path to the scipion configuration file where all this variables could be defined.", var_type=VarTypes.PATH)
+                          "Path to the scipion configuration file where all this variables could be defined.", var_type=VarTypes.PATH)
 
     SCIPION_LOCAL_CONFIG = _get('SCIPION_LOCAL_CONFIG', SCIPION_CONFIG,
-    "Path to an optional/extra/user configuration file meant to overwrite default variables.", var_type=VarTypes.PATH)
+                                "Path to an optional/extra/user configuration file meant to overwrite default variables.", var_type=VarTypes.PATH)
 
     SCIPION_HOSTS = _get('SCIPION_HOSTS', _join('config','hosts.conf'),
-    "Path to the host.cof file to allow scipion to use queue engines and run in HPC environments.")
+                         "Path to the host.cof file to allow scipion to use queue engines and run in HPC environments.")
 
     SCIPION_PROTOCOLS = _get('SCIPION_PROTOCOLS', _join('config','protocols.conf'),
-    "Custom conf file to extend the protocols tree view panel (panel on the left)")
+                             "Custom conf file to extend the protocols tree view panel (panel on the left)")
 
     SCIPION_PLUGIN_JSON = _get('SCIPION_PLUGIN_JSON', None,
-    "Optional. Path to get the json file with all the plugins available for Scipion.")
+                               "Optional. Path to get the json file with all the plugins available for Scipion.")
+
+    SCIPION_SITE = _get('SCIPION_SITE', 'https://scipion.i2pc.es',
+                        "Scipion site URL.")
+    SCIPION_SITE_API = SCIPION_SITE + '/report_protocols/api/v2'
+    SCIPION_STATS_WORKFLOW_APP = SCIPION_SITE_API + '/workflow/'
+    SCIPION_STATS_SUGGESTION = SCIPION_SITE_API + '/nextprotocol/suggestion/%s'
 
     SCIPION_PLUGIN_REPO_URL = _get('SCIPION_PLUGIN_REPO_URL',
-                                   'https://scipion.i2pc.es/getplugins/',
-    "Url from where to get the list of plugins.")
+                                   SCIPION_SITE + '/getplugins/',
+                                   "Url from where to get the list of plugins.")
 
     # REMOTE Section
     SCIPION_URL = 'https://scipion.cnb.csic.es/downloads/scipion'
@@ -271,65 +298,69 @@ class Config:
 
     # Scipion Notes
     SCIPION_NOTES_FILE = _get(SCIPION_NOTES_FILE, 'notes.txt',
-    "Name of the file where to write per project notes.")
+                              "Name of the file where to write per project notes.")
 
     SCIPION_NOTES_PROGRAM = _get(SCIPION_NOTES_PROGRAM, None,
-    "Command or program to use to open the notes file. Otherwise system will extension association will take place.")
+                                 "Command or program to use to open the notes file. Otherwise system will extension association will take place.")
 
     SCIPION_NOTES_ARGS = _get(SCIPION_NOTES_ARGS, None)
 
     # External text editor:
     SCIPION_TEXT_EDITOR = _get(SCIPION_TEXT_EDITOR, '',
-    "Preferred text editor executable.", caster=str)
+                               "Preferred text editor executable.", caster=str)
 
     # Aspect
     SCIPION_FONT_NAME = _get('SCIPION_FONT_NAME', "Helvetica",
-    "Name of the font to use in Scipion GUI. Defaults to Helvetica.")
+                             "Name of the font to use in Scipion GUI. Defaults to Helvetica.")
 
-    SCIPION_FONT_SIZE = _get('SCIPION_FONT_SIZE', SCIPION_DEFAULT_FONT_SIZE,
-    "Size of the 'normal' font to be used in Scipion GUI. Defaults to 10.", caster=int)
+    SCIPION_FONT_SIZE = _get('SCIPION_FONT_SIZE', str(SCIPION_DEFAULT_FONT_SIZE),
+                             "Size of the 'normal' font to be used in Scipion GUI. "
+                             "Defaults to 10.", caster=int)
 
     SCIPION_MAIN_COLOR = _get('SCIPION_MAIN_COLOR', Color.MAIN_COLOR,
-    "str: Main color of the GUI. Background will be white, so for better contrast choose a dark color. Probably any name here will work: https://matplotlib.org/stable/gallery/color/named_colors.html",
+                              "str: Main color of the GUI. Background will be white, so for better contrast choose a dark color. Probably any name here will work: https://matplotlib.org/stable/gallery/color/named_colors.html",
                               caster=validColor)
 
     SCIPION_BG_COLOR = _get('SCIPION_BG_COLOR', Color.BG_COLOR,
-    "str: Main background color of the GUI. Default is white, chose a light one. Probably any name here will work: https://matplotlib.org/stable/gallery/color/named_colors.html",
+                            "str: Main background color of the GUI. Default is white, chose a light one. Probably any name here will work: https://matplotlib.org/stable/gallery/color/named_colors.html",
                             validColor)
 
-    SCIPION_CONTRAST_COLOR = _get('SCIPION_CONTRAST_COLOR', 'cyan',
-    "Color used to highlight features over grayscaled images.", caster=validColor)
+    SCIPION_CONTRAST_COLOR = _get('SCIPION_CONTRAST_COLOR', 'cyan', "Color used to highlight features over "
+                                                                    "gray-scaled images.", caster=validColor)
 
     SCIPION_SPRITES_FILE = _get('SCIPION_SPRITES_FILE', __defaultSpritesFile,
-    "File (png) with the icons in a collage. Default is found at pyworkflow/resources/sprites.png. And a GIMP file could be found at the same folder in the github repo.")
+                                "File (png) with the icons in a collage. Default is found at "
+                                "pyworkflow/resources/sprites.png. And a GIMP file could be found at the same "
+                                "folder in the github repo.")
 
-    SCIPION_SHOW_TEXT_IN_TOOLBAR = _get('SCIPION_SHOW_TEXT_IN_TOOLBAR', TRUE_STR,
-    "Define it to anything else except False to show the label of the icons. It will take more space.") == TRUE_STR
+    SCIPION_SHOW_TEXT_IN_TOOLBAR = _get('SCIPION_SHOW_TEXT_IN_TOOLBAR', TRUE_STR, "Define it to anything else except "
+                                                                                  "%s to hide labels. It will take less space." % TRUE_YES_ON_, caster=__bool)
 
-    SCIPION_ICON_ZOOM = _get('SCIPION_ICON_ZOOM', 50,
-    "Define it to anything else except False to show the label of the icons. It will take more space.", var_type=VarTypes.INTEGER, caster=int)
+    SCIPION_ICON_ZOOM = _get('SCIPION_ICON_ZOOM', "50",
+                             "Define it to any integer value(percentage) to increase/decrease the size of the icons.",
+                             var_type=VarTypes.INTEGER, caster=int)
 
     # Notification
-    SCIPION_NOTIFY = _get('SCIPION_NOTIFY', TRUE_STR,
-    "If set to False, Scipion developers will know almost nothing about Scipion usage and will have less information to improve it.") == TRUE_STR
+    SCIPION_NOTIFY = _get('SCIPION_NOTIFY', TRUE_STR, "If set to anything except %s Scipion developers will know "
+                                                      "nothing abut Scipion usage and will have less information to improve it." % TRUE_YES_ON_,
+                          caster=__bool)
 
     # *** Execution variables ***
-    SCIPION_CWD = _get('SCIPION_CWD', os.path.abspath(os.getcwd()),
-    "Directory when scipion was launched")
+    SCIPION_CWD = _get('SCIPION_CWD', os.path.abspath(os.getcwd()), "Directory when scipion was launched")
 
     SCIPION_GUI_REFRESH_IN_THREAD = _get('SCIPION_GUI_REFRESH_IN_THREAD', FALSE_STR,
-    "True to refresh the runs graph with a thread. Unstable.") != FALSE_STR
+                                         "True to refresh the runs graph with a thread. Unstable.") != FALSE_STR
 
     SCIPION_GUI_REFRESH_INITIAL_WAIT = _get("SCIPION_GUI_REFRESH_INITIAL_WAIT", 5,
-    "Seconds to wait after a manual refresh", caster=int)
+                                            "Seconds to wait after a manual refresh", caster=int)
 
     SCIPION_GUI_CANCEL_AUTO_REFRESH = _get("SCIPION_GUI_CANCEL_AUTO_REFRESH",FALSE_STR,
-    "Set it to True to cancel automatic refresh of the runs.") != FALSE_STR
+                                           "Set it to True to cancel automatic refresh of the runs.") != FALSE_STR
 
     # Cancel shutil fast copy. In GPFS, shutil.copy does fail when trying a fastcopy and does not
     # fall back on the slow copy. For legacy reasons None is also False.
     SCIPION_CANCEL_FASTCOPY = _get('SCIPION_CANCEL_FASTCOPY', FALSE_STR,
-    "Cancel fast copy done by shutil (copying files) when it fails. Has happened in GPFS environments. Defaults to False. None is also False otherwise fastcopy is cancelled."
+                                   "Cancel fast copy done by shutil (copying files) when it fails. Has happened in GPFS environments. Defaults to False. None is also False otherwise fastcopy is cancelled."
                                    ) not in [NONE_STR, FALSE_STR]
 
     # Priority package list: This variable is used in the view protocols in
@@ -339,26 +370,40 @@ class Config:
     SCIPION_PRIORITY_PACKAGE_LIST = _get('SCIPION_PRIORITY_PACKAGE_LIST', EMPTY_STR)
 
     SCIPION_STEPS_CHECK_SEC = _get('SCIPION_STEPS_CHECK_SEC', 5,
-    "Number of seconds to wait before checking if new input is available in streamified protocols.", caster=int)
+                                   "Number of seconds to wait before checking if new input is available in "
+                                   "streamified protocols.", caster=int)
 
     SCIPION_UPDATE_SET_ATTEMPTS = _get('SCIPION_UPDATE_SET_ATTEMPTS', 3,
-    "Number of attempts to modify the protocol output before failing. The default value is 3", caster=int)
+                                       "Number of attempts to modify the protocol output before failing. "
+                                       "The default value is 3", caster=int)
 
     SCIPION_UPDATE_SET_ATTEMPT_WAIT = _get('SCIPION_UPDATE_SET_ATTEMPT_WAIT', 2,
-    "Time in seconds to wait until the next attempt when checking new outputs. The default value is 2 seconds", caster=int)
+                                           "Time in seconds to wait until the next attempt when checking new outputs. "
+                                           "The default value is 2 seconds", caster=int)
+
+    SCIPION_UNLOAD_FORM_ON_SAVE = _get('SCIPION_UNLOAD_FORM_ON_SAVE', TRUE_STR,
+                                       "When a protocol is saved it is unloaded if all goes fine. Set it to false "
+                                       "otherwise to rescue old behaviour.", caster=__bool)
 
     SCIPION_USE_QUEUE = _get("SCIPION_USE_QUEUE", FALSE_STR,
-    "Default value for using the queue. By default is False. ANY value will be True except and empty value. \"False\" or \"0\" will be True too.")!= FALSE_STR
+                             "Default value for using the queue. By default is False. "
+                             "ANY value will be True except and empty value. \"False\" or \"0\" "
+                             "will be True too.") != FALSE_STR
 
     SCIPION_DEFAULT_EXECUTION_ACTION = _get('SCIPION_DEFAULT_EXECUTION_ACTION', DEFAULT_EXECUTION_ACTION_ASK,
-    """Ask if you want to launch a single protocol or a sub-workflow. The default value is 1
-       1: Scipion always ask
-       2: Run a single protocol
-       3: Run a sub-workflow """, caster=int)
+                                            """Ask if you want to launch a single protocol or a sub-workflow. The default value is 1
+                                               1: Scipion always ask
+                                               2: Run a single protocol
+                                               3: Run a sub-workflow """, caster=int)
 
     SCIPION_MAPPER_USE_TEMPLATE = _get('SCIPION_MAPPER_USE_TEMPLATE', TRUE_STR,
-    "Set it to False to force instantiation for each item during sets iterations. Experimental. This penalize the iteration but avoids"
-    "the use of .clone() ot the items.") == TRUE_STR
+                                       "Set it to False to force instantiation for each item during sets iterations. Experimental. This penalize the iteration but avoids"
+                                       "the use of .clone() ot the items.") == TRUE_STR
+
+    CUDA_DEVICE_ORDER = _get('CUDA_DEVICE_ORDER', 'PCI_BUS_ID',
+                             "To make GPU ID match what you see in nvidia-smi, that is the PCI ID."
+                             " Use FASTEST_FIRST for default behaviour but this may not match what is returned by nvidia-smi."
+                             " See https://docs.nvidia.com/cuda/cuda-c-programming-guide/#:~:text=in%20device%20memory.-,CUDA_DEVICE_ORDER,-FASTEST_FIRST%2C%20PCI_BUS_ID%2C%20(default ")
 
     try:
         VIEWERS = ast.literal_eval(_get('VIEWERS', "{}", "Json string to define which viewer are the default ones per output type."))
@@ -377,7 +422,8 @@ class Config:
         :return: Folder where libraries must be placed in case a binding needs them
         """
         lib = cls._join(cls.SCIPION_SOFTWARE, 'lib')
-        os.makedirs(lib, exist_ok=True)
+        if cls.SCIPION_HOME_DEFINED:
+            os.makedirs(lib, exist_ok=True)
         return lib
 
     @classmethod
@@ -388,7 +434,8 @@ class Config:
         :return:   The bindings folder
         """
         bindings = cls._join(cls.SCIPION_SOFTWARE, 'bindings')
-        os.makedirs(bindings, exist_ok=True)
+        if cls.SCIPION_HOME_DEFINED:
+            os.makedirs(bindings, exist_ok=True)
         return bindings
 
     @classmethod
@@ -397,7 +444,8 @@ class Config:
         Folder where scipion logs must be placed. The folder is created         
         """
         logsFolder = cls.SCIPION_LOGS
-        os.makedirs(logsFolder, exist_ok=True)
+        if cls.SCIPION_HOME_DEFINED:
+            os.makedirs(logsFolder, exist_ok=True)
         return logsFolder
 
     @classmethod
@@ -455,8 +503,13 @@ class Config:
         debugOn = not Config.debugOn()
         os.environ[SCIPION_DEBUG] = str(debugOn)
         os.environ[SCIPION_DEBUG_NOCLEAN] = str(debugOn)
-        os.environ[SCIPION_LOG_LEVEL] = "INFO" if not debugOn else "DEBUG"
 
+        newLevel = "DEBUG" if debugOn else "INFO"
+        os.environ[SCIPION_LOG_LEVEL] = newLevel
+
+        from pyworkflow.utils import changeLogLevel
+        changeLogLevel(newLevel)
+        logger.info("Log level set to %s" % newLevel)
     @staticmethod
     def debugSQLOn():
         from .utils import envVarOn
@@ -545,6 +598,17 @@ class Config:
             condaExe = os.path.join(envFolder, "bin", "python")
             return condaExe == getPython()
 
+    @classmethod
+    def getEnvName(cls):
+        """ Returns Scipion's environment name(conda or venv)"""
+        if cls.isCondaInstallation():
+            envPath = os.environ['CONDA_PREFIX']
+        else:  # Virtualenv
+            envPath = os.environ['VIRTUAL_ENV']
+
+        return os.path.basename(envPath)
+
+
     @classmethod
     def getSpritesFile(cls):
         if not os.path.exists(Config.SCIPION_SPRITES_FILE):

{scipion_pyworkflow-3.10.5 → scipion_pyworkflow-3.11.0}/pyworkflow/constants.py

@@ -43,7 +43,7 @@ VERSION_1 = '1.0.0'
 VERSION_1_1 = '1.1.0'
 VERSION_1_2 = '1.2.0'
 VERSION_2_0 = '2.0.0'
-VERSION_3_0 = '3.10.5'
+VERSION_3_0 = '3.11.0'
 
 # For a new release, define a new constant and assign it to LAST_VERSION
 # The existing one has to be added to OLD_VERSIONS list.
@@ -73,6 +73,16 @@ SCIPION_HOME_VAR = 'SCIPION_HOME'
 SCIPION_TESTS = 'SCIPION_TESTS'
 SCIPION_SCRATCH = 'SCIPION_SCRATCH'
 
+# VARIABLE names in host.conf
+PARALLEL_COMMAND_VAR = 'PARALLEL_COMMAND'
+PLUGIN_MODULE_VAR = 'PLUGIN_MODULE'
+QUEUE_FOR_JOBS = 'QUEUE_FOR_JOBS'
+
+
+# Launching constants
+RUN_JOB_GPU_PARAM = 'GPU' # Param name to "place" GPU ids used to build a run command.
+RUN_JOB_GPU_PARAM_SEARCH = "%("+ RUN_JOB_GPU_PARAM +")s"
+
 # FONT
 SCIPION_DEFAULT_FONT_SIZE = 10
 
@@ -131,6 +141,7 @@ PROJECT_SETTINGS = 'settings.sqlite'
 FALSE_STR = str(False)
 TRUE_STR = str(True)
 NONE_STR = str(None)
+TRUE_YES_ON_ = ['true', 'yes', 'on', '1']
 EMPTY_STR = ""
 
 # GUI colors
@@ -215,4 +226,3 @@ DEFAULT_EXECUTION_ACTION_ALL = 3
 # Id field/attribute constants
 ID_COLUMN='id'
 ID_ATTRIBUTE='_objId'
-

{scipion_pyworkflow-3.10.5 → scipion_pyworkflow-3.11.0}/pyworkflow/object.py

@@ -1233,12 +1233,13 @@ class Set(Object):
         return self._getMapper().exists(itemId)
 
     def iterItems(self, orderBy='id', direction='ASC', where=None,
-                  limit=None, iterate=True):
+                  limit=None, iterate=True, rowFilter=None):
         return self._getMapper().selectAll(orderBy=orderBy,
                                            direction=direction,
                                            where=where,
                                            limit=limit,
-                                           iterate=iterate)  # has flat mapper, iterate is true
+                                           iterate=iterate,
+                                           rowFilter=rowFilter)  # has flat mapper, iterate is true
 
     def getFirstItem(self):
         """ Return the first item in the Set. """