apsbits 1.0.3__tar.gz → 1.0.5__tar.gz

{apsbits-1.0.3 → apsbits-1.0.5}/HISTORY.rst

@@ -30,10 +30,59 @@ describe future plans.
 .. Coming release content can be gathered here.
     Some people object to publishing unreleased changes.
 
-    1.0.2
+    1.1.0
     #####
 
-    release expected 2025-Q2
+    release expected ?
+
+    New Features
+    ---------------
+
+    * Hoist support to setup baseline stream using labels kwarg from USAXS.
+
+    Maintenance
+    ---------------
+
+    * Bump iconfig version to 2.0.1 for the baseline addition.
+    * Remove run_engine section from QS config.yml file and pin QS to 0.0.22+.
+
+1.0.4
+#####
+
+released 2025-05-14
+
+1.0.3
+#####
+
+released 2025-05-01
+
+Enhancements
+---------------
+
+* arguments for run engine
+
+Fixes
+-----
+
+* 'make_devices()' from yaml file
+
+Maintenance
+---------------
+
+* Clean backend
+
+1.0.2
+#####
+
+released 2025-04-18
+
+Maintenance
+---------------
+
+* Add a release history file
+* Documentation overhaul1
+* adding install docs given new workflow
+* Feature/API_functionalities and Makedevices
 
 1.0.1
 #####

{apsbits-1.0.3/apsbits.egg-info → apsbits-1.0.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apsbits
-Version: 1.0.3
+Version: 1.0.5
 Summary: Model of a Bluesky Data Acquisition Instrument in console, notebook, & queueserver.
 Author-email: Eric Codrea <ecodrea@anl.gov>, Pete Jemian <prjemian+instrument@gmail.com>, Rafael Vescovi <rvescovi@anl.gov>
 Maintainer-email: Eric Codrea <ecodrea@anl.gov>, Pete Jemian <prjemian+instrument@gmail.com>, Rafael Vescovi <rvescovi@anl.gov>
@@ -19,7 +19,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: apstools>=1.7.2
 Requires-Dist: bluesky-queueserver-api
-Requires-Dist: bluesky-queueserver
+Requires-Dist: bluesky-queueserver>=0.0.22
 Requires-Dist: bluesky-widgets
 Requires-Dist: bluesky
 Requires-Dist: caproto

{apsbits-1.0.3 → apsbits-1.0.5/apsbits.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apsbits
-Version: 1.0.3
+Version: 1.0.5
 Summary: Model of a Bluesky Data Acquisition Instrument in console, notebook, & queueserver.
 Author-email: Eric Codrea <ecodrea@anl.gov>, Pete Jemian <prjemian+instrument@gmail.com>, Rafael Vescovi <rvescovi@anl.gov>
 Maintainer-email: Eric Codrea <ecodrea@anl.gov>, Pete Jemian <prjemian+instrument@gmail.com>, Rafael Vescovi <rvescovi@anl.gov>
@@ -19,7 +19,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: apstools>=1.7.2
 Requires-Dist: bluesky-queueserver-api
-Requires-Dist: bluesky-queueserver
+Requires-Dist: bluesky-queueserver>=0.0.22
 Requires-Dist: bluesky-widgets
 Requires-Dist: bluesky
 Requires-Dist: caproto

{apsbits-1.0.3 → apsbits-1.0.5}/apsbits.egg-info/SOURCES.txt

@@ -61,18 +61,21 @@ docs/source/api/generated/apsbits.utils.logging_setup.rst
 docs/source/api/generated/apsbits.utils.metadata.rst
 docs/source/api/generated/apsbits.utils.stored_dict.rst
 docs/source/deprecated/console.rst
+docs/source/deprecated/dm.md
 docs/source/deprecated/logging_config.rst
 docs/source/deprecated/notebook.rst
 docs/source/deprecated/script.rst
 docs/source/guides/creating_devices.rst
 docs/source/guides/creating_instrument.rst
 docs/source/guides/developing_bits.rst
-docs/source/guides/dm.md
+docs/source/guides/dm.rst
 docs/source/guides/index.rst
+docs/source/guides/logging.rst
 docs/source/guides/qserver.rst
 docs/source/guides/qserver_service.rst
 docs/source/guides/sessions.rst
 docs/source/guides/setting_iconfig.rst
+docs/source/guides/startup.rst
 docs/source/guides/template_creation.rst
 src/apsbits/__init__.py
 src/apsbits/_version.py
@@ -95,6 +98,7 @@ src/apsbits/demo_instrument/callbacks/spec_data_file_writer.py
 src/apsbits/demo_instrument/configs/__init__.py
 src/apsbits/demo_instrument/configs/devices.yml
 src/apsbits/demo_instrument/configs/devices_aps_only.yml
+src/apsbits/demo_instrument/configs/extra_logging.yml
 src/apsbits/demo_instrument/configs/iconfig.yml
 src/apsbits/demo_instrument/devices/__init__.py
 src/apsbits/demo_instrument/plans/__init__.py
@@ -116,6 +120,7 @@ src/apsbits/tests/test_run_instrument.py
 src/apsbits/tests/test_stored_dict.py
 src/apsbits/utils/__init__.py
 src/apsbits/utils/aps_functions.py
+src/apsbits/utils/baseline_setup.py
 src/apsbits/utils/config_loaders.py
 src/apsbits/utils/controls_setup.py
 src/apsbits/utils/helper_functions.py

{apsbits-1.0.3 → apsbits-1.0.5}/apsbits.egg-info/requires.txt

@@ -1,6 +1,6 @@
 apstools>=1.7.2
 bluesky-queueserver-api
-bluesky-queueserver
+bluesky-queueserver>=0.0.22
 bluesky-widgets
 bluesky
 caproto

{apsbits-1.0.3 → apsbits-1.0.5}/docs/source/guides/creating_devices.rst

@@ -46,32 +46,24 @@ You can also add a device from an external package to the ``devices.yml`` file.
        prefix: ioc:Slit1
        labels: ["slits"]
 
-2. Make sure all your device files are loaded on the ``iconfig.yml`` file.
-
-.. code-block:: yaml
-
-      DEVICES_FILES:
-        - devices.yml
-      APS_DEVICES_FILES:
-        - devices_aps_only.yml
-
-Note that the ``devices.yml`` file is loaded by default and the ``devices_aps_only.yml`` file is only loaded if the device is on the APS network.
 
 .. tip::
-    All kwargs of your device can be specified in the yaml file making it easy to resuse classes.
+    `APSTOOLS <https://github.com/BCDA-APS/apstools/tree/main/apstools>`_ has a lot of devices commonly used at the APS. Consider first checking the package and overwriting the device class to fit your needs before creating a new device.
 
 .. tip::
-   You can add any number of device files inside the iconfig file used by your instrument ``startup.py`` file. Just add another entry to the list.
+    You can add the `label: baseline` to any device in your configuration to automatically track its state during scans. This is particularly useful for monitoring environmental conditions or instrument parameters. For example:
 
-   .. code-block:: yaml
+    .. code-block:: yaml
 
-      ### Local OPHYD Device Control Yaml
-      DEVICES_FILES:
-        - devices.yml ## standard device file
-        - devices_2.yml ## another device file
-      APS_DEVICES_FILES:
-        - devices_aps_only.yml ## APS only device file
-        - devices_aps_only_2.yml ## another APS only device file
+        apstools.devices.ApsMachineParametersDevice:
+        - name: aps
+          labels:
+          - baseline
 
-.. tip::
-    `APSTOOLS <https://github.com/BCDA-APS/apstools/tree/main/apstools>`_ has a lot of devices commonly used at the APS. Consider first checking the package and overwriting the device class to fit your needs before creating a new device.
+        ophyd.EpicsSignalRO:
+        - name: temperature_monitor
+          prefix: "IOC:TEMP"
+          labels:
+          - baseline
+
+    Baseline devices will be automatically included in the metadata of each scan, making it easy to track changes in environmental conditions or instrument parameters over time.

apsbits-1.0.5/docs/source/guides/dm.rst

@@ -0,0 +1,88 @@
+Guide: APS Data Management Plans
+==============================
+
+Provides a few examples of the plans that interact with APS Data Management (DM)
+tools.
+
+Required
+--------
+
+The DM tools rely on the existence of a set of environment variables that define various aspects of the DM system.
+
+Show any DM jobs still processing
+--------------------------------
+
+Use the ``dm_list_processing_jobs()`` plan stub to show DM any workflow jobs that
+are still running or pending.  These are installed by calling
+``aps_dm_setup(DM_SETUP_SCRIPT)`` in each session before you call any other DM
+code.
+
+Here, ``DM_SETUP_SCRIPT`` is the full path to the bash setup shell script provided
+by DM for this account.  The exact path can be different for some installations.
+If unsure, contact the APS DM team for advice.
+
+Note: ``aps_dm_setup`` is not a bluesky plan stub.  Call it as a standard Python
+function.
+
+Here's an example:
+
+.. code-block:: python
+
+    from apsbits.utils.aps_functions import aps_dm_setup
+
+    aps_dm_setup("/home/dm/etc/dm.setup.sh")
+
+Start a new workflow job
+-----------------------
+
+The ``dm_kickoff_workflow()`` plan can be used to start a DM workflow job.  See
+the source code for additional options (such as how often to report progress and
+how to wait for the workflow to finish before the bluesky plan proceeds).
+
+.. code-block:: python
+
+    from new_instrument.plans.dm_plans import dm_kickoff_workflow
+
+    # Use the run with `uid` from the catalog `cat`.
+    run = cat[uid]
+
+    # Create the dictionary of arguments for the chosen workflow.
+    argsDict = {
+        "filePath": "path/to/data/file.mda",  # example
+        "experimentName": "testing-2024-11",  # example
+        "workflowName": "processing",  # existing workflow name
+        # ... any other items required by the workflow
+    }
+
+    # Start the workflow job from the command line:
+    RE(dm_kickoff_workflow(run, argsDict))
+
+In a plan, replace the call to ``RE(...)`` with ``yield from ...``, such as:
+
+.. code-block:: python
+
+    def a_plan():
+        # earlier steps
+        yield from dm_kickoff_workflow(run, argsDict)
+        # later steps
+
+Start a new workflow job (Low-level)
+----------------------------------
+
+If the ``dm_kickoff_workflow()`` plan stub does more than you want, you might consider the ``dm_submit_workflow_job()``
+plan stub.  The ``dm_submit_workflow_job()`` plan stub is
+a thin wrapper around DM's ``startProcessingJob()`` function.
+The plan stub converts this DM function into a bluesky plan, and reports the DM workflow job ``id`` once the job has been submitted.
+
+As above, you'll need the ``workflowName`` and the ``argsDict``.
+
+From the command line: ``RE(dm_submit_workflow_job(workflowName, argsDict))``
+
+In a plan: ``yield from dm_submit_workflow_job(workflowName, argsDict)``
+
+References
+----------
+
+The `apstools` package has more support to integrate various capabilities of the DM tools.
+
+For more information about DM, see its `API Reference <https://git.aps.anl.gov/DM/dm-docs/-/wikis/DM/Beamline-Services/API-Reference>`_.

{apsbits-1.0.3 → apsbits-1.0.5}/docs/source/guides/index.rst

@@ -13,6 +13,7 @@ User Documentation
 
    template_creation
    creating_instrument
+   startup
    setting_iconfig
    creating_devices
    sessions

apsbits-1.0.5/docs/source/guides/logging.rst

@@ -0,0 +1,283 @@
+Logging Configuration Guide
+=========================
+
+This guide explains how to configure and customize logging in the BITS package.
+
+Overview
+--------
+
+The logging system provides:
+
+* Console output for immediate feedback
+* File logging for persistent records
+* IPython session logging
+* Module-specific log levels
+* Log rotation to manage disk space
+
+Default Configuration
+-------------------
+
+The default logging configuration is defined in ``src/apsbits/configs/logging.yml``:
+
+.. code-block:: yaml
+
+    console_logs:
+      date_format: "%a-%H:%M:%S"
+      log_format: "%(levelname)-.1s %(asctime)s.%(msecs)03d: %(message)s"
+      level: info
+      root_level: bsdev
+
+    file_logs:
+      date_format: "%Y-%m-%d %H:%M:%S"
+      log_directory: .logs
+      log_filename_base: logging.log
+      log_format: "|\
+        %(asctime)s.%(msecs)03d|\
+        %(levelname)s|\
+        %(process)d|\
+        %(name)s|\
+        %(module)s|\
+        %(lineno)d|\
+        %(threadName)s| - \
+        %(message)s"
+      maxBytes: 1_000_000
+      backupCount: 9
+      level: info
+      rotate_on_startup: true
+
+    ipython_logs:
+      log_directory: .logs
+      log_filename_base: ipython_log.py
+      log_mode: rotate
+      options: -o -t
+
+    modules:
+      apstools: warning
+      bluesky-queueserver: warning
+      bluesky: warning
+      bluesky.RE: warning
+      caproto: warning
+      databroker: warning
+      ophyd: warning
+
+Log Levels
+---------
+
+The package defines seven log levels in order of increasing detail:
+
+=========   =========   ==================================================
+name        severity    comments
+=========   =========   ==================================================
+CRITICAL    50          Examine immediately. **Quietest** level.
+ERROR       40          Something has failed.
+WARNING     30          Something needs attention.
+INFO        20          A report that may be of interest.
+BSDEV       15          A report of interest to developers.
+DEBUG       10          Diagnostic. **Noisiest** level.
+NOTSET      0           Initial setting, defaults to WARNING.
+=========   =========   ==================================================
+
+Custom Configuration
+------------------
+
+To customize logging for your instrument, create an ``extra_logging.yml`` file in your instrument's config directory. The settings in this file will override the default configuration:
+
+.. code-block:: yaml
+
+    # Custom logging configuration
+    # These settings will override the default configuration
+    console_logs:
+      level: debug  # Override console log level
+      root_level: debug  # Override root log level
+
+    file_logs:
+      log_directory: /custom/path/to/logs  # Override log directory
+      maxBytes: 2_000_000  # Override max file size
+      backupCount: 5  # Override backup count
+
+    modules:
+      your_module: debug  # Add or override module logging level
+
+The configuration system will:
+1. Load the default configuration from ``src/apsbits/configs/logging.yml``
+2. If an extra configuration is provided, merge it with the default configuration:
+   * For dictionary settings (like ``console_logs``, ``file_logs``), only the specified keys are overridden
+   * For non-dictionary settings, the entire setting is replaced
+   * New settings not present in the default configuration are added
+
+Then, in your ``startup.py``, configure logging with your custom configuration:
+
+.. code-block:: python
+
+    from pathlib import Path
+    from apsbits.utils.logging_setup import configure_logging
+
+    # Get the path to your instrument's config directory
+    instrument_path = Path(__file__).parent
+    extra_logging_configs_path = instrument_path / "configs" / "extra_logging.yml"
+
+    # Configure logging with your custom settings
+    # The extra configuration will override the default settings
+    configure_logging(extra_logging_configs_path=extra_logging_configs_path)
+
+Example of Configuration Override
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Default configuration:
+.. code-block:: yaml
+
+    console_logs:
+      level: info
+      root_level: bsdev
+
+    file_logs:
+      maxBytes: 1_000_000
+      backupCount: 9
+
+Extra configuration:
+.. code-block:: yaml
+
+    console_logs:
+      level: debug  # Only overrides the level, root_level remains bsdev
+
+    file_logs:
+      maxBytes: 2_000_000  # Only overrides maxBytes, backupCount remains 9
+
+Resulting configuration:
+.. code-block:: yaml
+
+    console_logs:
+      level: debug      # From extra config
+      root_level: bsdev # From default config
+
+    file_logs:
+      maxBytes: 2_000_000  # From extra config
+      backupCount: 9       # From default config
+
+Error Handling
+-------------
+
+The logging configuration system includes error handling for common issues:
+
+1. Invalid or Empty Configuration Files:
+   * The main logging configuration file must be valid and non-empty
+   * If the main configuration is invalid, a ``ValueError`` is raised
+   * If the extra configuration is invalid, a warning is logged and the default configuration is used
+
+2. Missing Configuration Files:
+   * The main logging configuration file must exist
+   * Extra configuration files are optional
+   * If an extra configuration file is missing, the default configuration is used
+
+Example error handling in your code:
+
+.. code-block:: python
+
+    try:
+        configure_logging(extra_logging_configs_path=extra_logging_configs_path)
+    except ValueError as e:
+        logger.error("Failed to configure logging: %s", e)
+        # Handle the error appropriately
+    except Exception as e:
+        logger.exception("Unexpected error configuring logging")
+        # Handle unexpected errors
+
+Log File Locations
+----------------
+
+By default, logs are stored in a ``.logs`` directory at the root of your package:
+
+.. code-block:: text
+
+    your_package/
+    ├── .logs/
+    │   ├── logging.log        # Main log file
+    │   ├── logging.log.1      # Rotated logs
+    │   ├── logging.log.2
+    │   └── ipython_log.py     # IPython session logs
+    ├── src/
+    ├── tests/
+    └── ...
+
+You can override the log directory location by specifying ``log_directory`` in your configuration:
+
+.. code-block:: yaml
+
+    file_logs:
+      log_directory: /custom/path/to/logs
+      log_filename_base: my_instrument.log
+
+Log Rotation
+-----------
+
+Log files are automatically rotated to manage disk space:
+
+* Files are rotated when they reach the specified size (default: 1MB)
+* A maximum number of backup files is kept (default: 9)
+* Logs can be rotated on startup (default: true)
+
+Configure these settings in your logging configuration:
+
+.. code-block:: yaml
+
+    file_logs:
+      maxBytes: 2_000_000  # 2MB
+      backupCount: 5       # Keep 5 backup files
+      rotate_on_startup: true
+
+Using Logging in Your Code
+------------------------
+
+To use logging in your code:
+
+.. code-block:: python
+
+    import logging
+
+    # Get a logger for your module
+    logger = logging.getLogger(__name__)
+
+    # Log messages at different levels
+    logger.debug("Detailed information for debugging")
+    logger.info("General information about program execution")
+    logger.warning("Warning messages for potentially problematic situations")
+    logger.error("Error messages for serious problems")
+    logger.critical("Critical messages for fatal errors")
+    logger.bsdev("Developer-specific information")
+
+Best Practices
+-------------
+
+1. Use appropriate log levels:
+   * DEBUG: Detailed information for debugging
+   * INFO: General information about program execution
+   * WARNING: Potentially problematic situations
+   * ERROR: Serious problems
+   * CRITICAL: Fatal errors
+   * BSDEV: Developer-specific information
+
+2. Include relevant context in log messages:
+   * Use string formatting for variables
+   * Include error details when catching exceptions
+   * Add module and function names for clarity
+
+3. Configure module-specific logging levels to reduce noise:
+   * Set verbose logging for your modules
+   * Set higher levels for third-party modules
+
+4. Use log rotation to manage disk space:
+   * Set appropriate file sizes
+   * Configure backup count based on available space
+   * Consider rotating logs on startup for clean sessions
+
+5. Handle logging configuration errors:
+   * Always wrap logging configuration in try-except blocks
+   * Provide fallback configurations when needed
+   * Log configuration errors appropriately
+
+References
+----------
+
+* `Python logging documentation <https://docs.python.org/3/library/logging.html>`_
+* `Bluesky debugging guide <https://blueskyproject.io/bluesky/main/debugging.html>`_
+* `Ophyd logging reference <https://blueskyproject.io/ophyd/user_v1/reference/logging.html>`_

{apsbits-1.0.3 → apsbits-1.0.5}/docs/source/guides/sessions.rst

@@ -35,4 +35,26 @@ When ready to load the bluesky data acquisition for use, type this command. For
 
 .. code-block:: bash
 
+    from demo_instrument.startup import *
+
+.. note::
+Change the demo_instrument to your instrument installed package name.
+
+Adding BITS to ipython profile
+----------------------------------
+
+To add BITS to your ipython profile, first create a new profile:
+
+.. code-block:: bash
+
+    ipython profile create bits-xx
+
+Then, add the following to your startup file:
+
+.. code-block:: bash
+
+    cat > ~/.ipython/profile_bits-xx/startup/00-start-bits.py  << EOF
     from new_instrument.startup import *
+    EOF
+
+For more detailed guidance on creating and configuring an ipython profile, see the `bluesky training documentation <https://github.com/BCDA-APS/bluesky_training/blob/304b8d02503044932afa5657cb43afd1f6be2f40/docs/source/instrument/_create_bluesky_ipython_profile.rst#L2>`_.