mx-bluesky 1.5.12__tar.gz → 1.5.15__tar.gz

{mx_bluesky-1.5.12 → mx_bluesky-1.5.15}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mx-bluesky
-Version: 1.5.12
+Version: 1.5.15
 Summary: Bluesky tools for MX Beamlines at DLS
 Author-email: Dominic Oram <dominic.oram@diamond.ac.uk>
 License:                                  Apache License
@@ -237,11 +237,11 @@ Requires-Dist: deepdiff
 Requires-Dist: matplotlib
 Requires-Dist: cachetools
 Requires-Dist: daq-config-server>=v1.0.0-rc.2
-Requires-Dist: blueapi>=1.6.3
+Requires-Dist: blueapi>=1.8.0
 Requires-Dist: ophyd>=1.10.5
 Requires-Dist: ophyd-async>=0.14.0
 Requires-Dist: bluesky>=1.14.6
-Requires-Dist: dls-dodal==1.66.0
+Requires-Dist: dls-dodal==1.68.0
 Provides-Extra: dev
 Requires-Dist: black; extra == "dev"
 Requires-Dist: build; extra == "dev"

{mx_bluesky-1.5.12 → mx_bluesky-1.5.15}/docs/developer/general/how-to/create-a-release.rst

@@ -3,7 +3,7 @@ Create a New Release
 ===========
 Pre-release
 ===========
-Make sure that dodal has an up-to-date release. If not, create one (see `dodal release instructions <https://diamondlightsource.github.io/dodal/main/developer/how-to/make-release.html>`_).
+Make sure that dodal has an up-to-date release. If not, create one (see `dodal release instructions <https://diamondlightsource.github.io/dodal/main/how-to/make-release.html>`_).
 
 =======
 Release
@@ -19,7 +19,7 @@ Release
 8. Publish the release
 
 NOTE FOR USING THE MX-BLUESKY-BLUEAPI IMAGE: If using this image with BlueAPI's helmchart for deployment, the version of dodal which is installed will be the version which is pinned in the BlueAPI, rather than what's in mx-bluesky's pyproject.toml. Before releasing, you should pin BlueAPI to a version which uses a dodal version which is compatible with itself AND mx-bluesky. The deployment will fail if BlueAPI has no version which works with the desired dodal version
-Follow these :ref:`instructions <deploying-to-i04-blueapi>` on how to deploy to I04 BlueAPI.
+Follow `these instructions <setup-blueapi-for-mx.html>`_ on using the mx-bluesky-blueapi image.
 
 ------------------------
 Deciding release numbers

mx_bluesky-1.5.15/docs/developer/general/how-to/deploy-a-release.rst

@@ -0,0 +1,37 @@
+Deploy a New Release
+====================
+
+.. warning::
+
+   This guide is for pre-containerised deployments. To see how deployments work using BlueAPI with kubernetes, please see `the updated guide <setup-blueapi-for-mx.html>`_.
+
+**Remember to discuss any new deployments with the appropriate beamline scientist.**
+
+The ``utility_scripts/deploy/deploy_mx_bluesky.py`` script will deploy the latest mx-bluesky version to a specified beamline. Deployments live in ``/dls_sw/ixx/software/bluesky/mx-bluesky_X.X.X``. To do a new deployment you should run the deploy script from your mx-bluesky dev environment with e.g.
+If you have just created a new release, you may need to run git fetch --tags to get the newest release.
+
+.. code:: console
+
+    python ./utility_scripts/deploy/deploy_mx_bluesky.py i24
+
+
+If you want to test the script for a specific beamline you can run:
+
+.. code:: console
+
+    python ./deploy/deploy_mx_bluesky.py i03 --dev
+
+
+which will create the beamline deployment of the new release in ``/scratch/30day_tmp/mx-bluesky_release_test``.
+
+
+.. note::
+
+    When deploying on I24, the edm screens for serial crystallography will be deployed automatically along with the mx-bluesky release.
+
+
+The script has a few additional optional arguments, which can be viewed with:
+
+.. code:: console
+
+    python ./deploy/deploy_mx_bluesky.py -h

mx_bluesky-1.5.15/docs/developer/general/how-to/setup-blueapi-for-mx.md

@@ -0,0 +1,40 @@
+# Setting up BlueAPI for an MX beamline
+
+## Background
+
+The Athena DAQ platform is intended to run in microservices using kubernetes. A beamline ixx has a gitlab repository `ixx-services`, which contains that beamline's available services and their helm configurations, as well as `ixx-deployments` which says which available services should be deployed. Services managed in this way will be deployed into the `ixx-beamline` namespace. ArgoCD provides a useful dashboard for the deployed services and their kubernetes objects. See [here](https://dev-guide.diamond.ac.uk/athena/how-tos/migration/) for more information. This guide will explain how to setup BlueAPI using mx-bluesky within this framework.
+
+## Prerequisites
+
+1. The beamline has a kubernetes cluster with gitlab deployment and services repos. See [here](https://dev-guide.diamond.ac.uk/epics-containers/how-tos/intro/) for a guide on this. DAQ members should be codeowners of these repos.
+2. Confirm the beamline cluster has a Blueapi ingress and DNS entry for `ixx-blueapi.diamond.ac.uk` - create a scicomp ticket if it doesn't.
+
+## Guide to run BlueAPI with mx-bluesky through kubernetes
+
+The best way to add BlueAPI to the services and deployments repo with the correct configuration is to copy from an existing setup and change the relevant values. i04 will be used as an example for this guide - [see their blueapi configuration](https://gitlab.diamond.ac.uk/controls/containers/beamline/i04-services/-/tree/main/services/i04-blueapi?ref_type=heads). The `chart.yaml` tells us which helmcharts are being used. The BlueAPI helmchart gives us the bulk of the configuration. See [here](https://github.com/DiamondLightSource/blueapi/tree/main/helm/blueapi) for a table of its settings. In MX we use an additional `mx-bluesky-blueapi` helmchart - right now this is purely for configuring Zocalo, but anything else which is generic to MX should be added.
+
+`values.yaml` lets us configure the helmcharts that we had listed in dependencies. This file should look the same for each MX beamline, other than the actual beamline name. Unfortunately, there's no way to get a templated file here which automatically fills in the beamline name. Some notes:
+
+- The BlueAPI image should be set to `ghcr.io/diamondlightsource/mx-bluesky-blueapi` and you should specify a version tag. See all releases of this image [here](https://github.com/DiamondLightSource/mx-bluesky/pkgs/container/mx-bluesky-blueapi). This image will install any MX-specific dependencies on top of the BlueAPI image.
+- We are currently mounting `/dls/ixx` and `/dls_sw/ixx` to BlueAPI, but the long-term goal is to not mount the file system. Nexgen currently requires `/dls/ixx` to be mounted. Once Nexgen is migrated into a separate service, we can unmount `/dls/ixx` from Blueapi and mount it in Nexgen only. Unmounting `/dls_sw/ixx`requires us to be using the config server for all settings - this is currently being worked on.
+- Until the above is done, BlueAPI needs to run as the detector user so that it has permission to write files. Type `id ixxdetector` into a terminal to find the user id. This then needs to be added in the `runAsUser: [number]` part of the values.
+
+MX Bluesky plans require some secrets to be added to the beamline cluster. The configuration from the above requires these secrets, so will not work without this. At the time of writing this guide, there are 3 that we need: ispyb (expeye) credentials, rabbitMQ credentials for zocalo, and config for rabbitMQ api reader for zocalo. These should be added as sealed secrets and committed to the services repo. Here's how this works:
+
+1. Open a terminal and enter your beamline cluster with `module load k8s-ixx`
+2. Set the default namespace to ixx-beamline with `kubectl config set-context --current --namespace=ixx-beamline`
+3. Create a regular secret and save to a yaml file with `kubectl create secret generic [secret-name] --from-file=/path/to/sensitive/file --save-config --dry-run=client -o yaml > [secret-file-name.yaml]`
+4. Seal the secret with `kubeseal --controller-namespace kube-system --controller-name sealed-secrets-controller --format yaml < [secret-file-name.yaml] > [sealed-secret-file-name.yaml]`. This will encrypt your secret - the beamline cluster comes installed with the private key to decrypt information sealed with this command.
+5. Commit the sealed secret into `ixx-services/services/ixx-blueapi/templates`. Importantly, do not commit the unsealed secret!
+6. Once this is pushed to the main branch, you can check the k8s dashboard to see if the secret has successfully been added.
+
+<div style="border: 2px solid #f39c12; background-color: #fef2e4; padding: 10px; margin: 15px 0; border-radius: 5px;">
+  <strong>Warning:</strong> The mx-bluesky-blueapi image is responsible for your blueapi version and your python environment. It is independent from the mx-bluesky and dodal repos in the scratch folder. Currently, the versions of these repos must be manually managed. This will be addressed <a href="https://github.com/DiamondLightSource/blueapi/issues/933" target="_blank" style="color: #e67e22; text-decoration: underline;">here</a>. 
+</div>
+
+Paths to required credentials:
+
+- Zocalo api reader: `/dls_sw/apps/zocalo/secrets/rabbitmq-api-reader.yml`
+- Zocalo RMQ credentials: `/dls_sw/apps/zocalo/secrets/rabbitmq-credentials.yml`
+- ispyb (expeye) credentials: `/dls_sw/dasc/mariadb/credentials/ispyb-mx-bluesky-ixx.cfg`
+  See [here](https://gitlab.diamond.ac.uk/controls/containers/beamline/i24-services/-/tree/main/services/i24-blueapi/templates?ref_type=heads) For what the templates folder should look like once the above steps have been done.

{mx_bluesky-1.5.12 → mx_bluesky-1.5.15}/docs/developer/general/index.rst

@@ -1,3 +1,5 @@
+:: _dev-guide:
+
 General MX-Bluesky Developer Guide
 ==================================
 
@@ -25,6 +27,7 @@ Documentation is split into four categories, and each is also accessible from li
             :maxdepth: 1
 
             how-to/get-started
+            how-to/setup-blueapi-for-mx
             how-to/contribute
             how-to/create-a-release
             how-to/deploy-a-release

{mx_bluesky-1.5.12 → mx_bluesky-1.5.15}/docs/developer/hyperion/index.rst

@@ -16,7 +16,6 @@ Documentation specific for the Hyperion module within MX-Bluesky
             reference/baton
             reference/param-hierarchy
             reference/coordinate-systems
-            deploying-hyperion
             system-tests
 
         +++

mx_bluesky-1.5.15/docs/user/hyperion/advanced/concurrent_operations.puml

@@ -0,0 +1,131 @@
+@startuml
+!pragma teoz true
+participant "BlueSky RunEngine" as RE
+box "Opyhd Async Devices"
+participant "Aperture\nScatterguard" as ap_sg
+participant "Smargon" as smargon
+participant "Robot" as robot
+participant "Backlight" as backlight
+participant "OAV" as oav
+participant "Panda" as panda
+participant "Zebra" as zebra
+participant "Zocalo\nResults" as zocalo
+participant "Eiger" as eiger
+end box
+box "Callbacks"
+participant "Robot Load\nISPyB" as RobotLoadISPyB
+participant "Gridscan\nISPyB" as GridscanISPyB
+participant "Grid\nDetection" as GridDetection
+participant "GridScan\nNexus" as GridScanNexus
+participant "Zocalo\nTrigger" as ZocaloCallback
+participant "XRC\nResults" as XRayCentreEventHandler
+participant "Snapshot\nBeam Centre\nAnnotation" as BeamDrawingCallback 
+end box
+
+RE -> RE:robot_load_then_xray_centre
+activate RE #ffeeee
+    opt pin not already loaded
+        RE -> RE: robot_load_and_change_energy_plan
+        activate RE  #ffdddd
+            RE -> RE: prepare_for_robot_load
+            activate RE  #ffcccc
+                RE -> ap_sg: Move Aperture Scatterguard out of beam
+                activate ap_sg
+                    RE -> smargon: Home smargon
+                    activate smargon
+                deactivate ap_sg
+            deactivate RE /' prepare_for_robot_load '/
+            RE -> RE: robot_load_and_snapshots
+            activate RE #ffcccc
+                activate RobotLoadISPyB
+                RE -> RobotLoadISPyB: send parameters to robot load callback
+                RE -> backlight: move backlight in
+                RE -> RE: do_robot_load
+                activate RE  #ffbbbb
+                    RE -> robot: trigger robot load
+                    activate robot
+                    RE -> RE: set_energy_plan
+                    note right of RE: (see below)
+                    deactivate robot
+                deactivate RE /' do_robot_load '/
+                RE -> smargon: Restore motor positions
+                deactivate smargon
+                RE -> oav: trigger snapshots
+                RE -> RobotLoadISPyB: read hardware
+                RE -> RobotLoadISPyB: record snapshots
+            deactivate RE /'robot_load_and_snapshots'/
+        return /'robot_load_and_change_energy_plan'/
+        deactivate RobotLoadISPyB
+    end /'pin not already loaded'/
+    alt pin is loaded
+        RE -> RE: set_energy_plan
+    end /'pin is loaded'/
+    RE -> RE++ : start_preparing_data_collection_then_do_plan
+        RE -> eiger ++ : arm the eiger
+        RE -> GridscanISPyB ++ : send parameters to grdscan callback
+            RE -> RE ++ #ffdddd : pin_centre_then_flyscan_plan
+                RE -> RE: setup_beamline_for_oav
+                RE -> RE: pin_tip_centre_plan
+                ...do pin tip centring...
+                RE -> RE ++ #ffcccc: detect_grid_and_do_gridscan
+                    RE -> RE: setup_beamline_for_oav
+                    RE -> ap_sg ++: prepare aperture scatterguard\nfor gridscan in background
+                    activate GridDetection
+                        RE -> RE ++ #ffbbbb: grid_detection_plan
+                            loop xy and xz grids
+                                RE -> smargon: rotate omega
+                                RE -> oav: trigger OAV snapshot
+                                RE -> GridscanISPyB: snapshot event with oav + smargon state
+                                & RE -> BeamDrawingCallback ++
+                                BeamDrawingCallback -> BeamDrawingCallback : save snapshot details\n\
+to generate rotation\n\
+snapshots later
+                            end
+                        deactivate RE /'grid_detection_plan'/
+                    deactivate GridDetection
+                    RE -> RE ++
+                    ...grid detection...
+                    return
+                    RE -> backlight: move backlight out 
+                    ap_sg -> RE --: aperture scatterguard completes
+                    RE -> RE ++ #ffaaaa : common_flycan_xray_centre
+                        RE -> eiger : setup detector parameters
+                        RE -> GridScanNexus++: send gridscan parameters
+                        RE -> panda : configure panda
+                        RE -> zebra : configure zebra for panda
+                        RE -> panda ++ : arm panda
+                        RE -> zocalo ++ : stage zocalo device
+                        RE -> RE ++ #ff9999 : run_gridscan 
+                            RE -> GridscanISPyB : read hardware before
+                            & RE -> GridScanNexus
+                            RE <-> eiger : stage eiger (ensures arming complete if not already)
+                            RE -> ZocaloCallback ++ : send params to Zocalo callback
+                            zocalo -> RE : zocalo stage completes
+                            RE -> ZocaloCallback : trigger zocalo with detector filename
+                            RE -> panda ++ : kickoff gridscan
+                            RE -> GridscanISPyB : read hardware during
+                            & RE -> GridScanNexus
+                            deactivate GridScanNexus
+                            ...perform gridscan...
+                            panda -> RE --
+                            eiger -> RE -- : unstage eiger
+                            RE -> ZocaloCallback : notify Zocalo of completion
+                            deactivate ZocaloCallback
+                        deactivate RE /' run_gridscan '/
+                        RE -> zocalo ++: fetch results from zocalo
+                        ...wait for Zocalo processing...
+                        zocalo -> RE -- : detected xtal centres
+                        RE -> XRayCentreEventHandler ++ : post detected centres to callback
+                        zocalo -> RE -- : unstage zocalo
+                        panda -> RE -- : unstage panda
+                        RE -> zebra : tidy zebra
+                    deactivate RE /'common_flyscan_xray_centre'/
+                deactivate RE /' detect_grid_and_do_gridscan'/
+            return /' pin_centre_then_flyscan_plan'/
+        deactivate GridscanISPyB
+    deactivate RE /' start_preparing_data_collection_then_do_plan '/
+deactivate RE /'robot_load_then_xray_centre'/
+note over XRayCentreEventHandler
+Results collected later by rotation
+end note
+@enduml

mx_bluesky-1.5.15/docs/user/hyperion/advanced/concurrent_operations_rotation.puml

@@ -0,0 +1,77 @@
+@startuml
+!pragma teoz true
+participant "BlueSky RunEngine" as RE
+box "Opyhd Async Devices"
+participant "Aperture\nScatterguard" as ap_sg
+participant "Smargon" as smargon
+participant "Robot" as robot
+participant "Backlight" as backlight
+participant "OAV" as oav
+participant "Zebra" as zebra
+participant "Eiger" as eiger
+participant "Synchrotron" as synchrotron
+end box
+box "Callbacks"
+participant "Rotation\nISPyB" as RotationISPyB
+participant "Rotation\nNexus" as RotationScanNexus
+participant "XRC\nResults" as XRayCentreEventHandler
+participant "Snapshot\nBeam Centre\nAnnotation" as BeamDrawingCallback
+end box
+
+activate XRayCentreEventHandler
+activate BeamDrawingCallback
+XRayCentreEventHandler -> RE : results from gridscan
+deactivate XRayCentreEventHandler
+
+RE -> eiger : configure eiger parameters
+
+RE -> RE ++ : start_preparing_data_collection_then_do_plan
+    RE -> eiger ++ : start arming the eiger
+    RE -> RotationScanNexus ++ : send rotation parameters
+    loop for each rotation
+        RE -> RotationISPyB ++: send rotation parameters (ROTATION_OUTER)
+        & RotationISPyB -> RotationScanNexus
+        RE -> smargon : move to xtal centre
+        RE -> RE : setup_beamline_for_oav
+        note right of RE : (if snapshots not derived from gridscan snapshots)
+        RE -> ap_sg ++ : prepare aperture
+        RE -> RE ++ #ffdddd : oav_snapshot_plan
+            loop for each snapshot
+                RE -> smargon : rotate
+                RE -> oav : configure + trigger snapshot (if taking real snapshots)
+                oav -> oav : save snapshots
+                RE -> RotationISPyB : generate snapshot event
+                & RE -> BeamDrawingCallback
+                BeamDrawingCallback -> BeamDrawingCallback : annotate snapshots
+            end
+            deactivate BeamDrawingCallback
+        deactivate RE /' oav_snapshot_plan '/
+        RE -> RE ++ #ffdddd : rotation_scan_plan
+            RE -> RotationISPyB ++ : send scan points (ROTATION_SCAN_MAIN)
+            RE -> zebra ++ : configure zebra for rotation
+            RE -> RE ++ #ffcccc : setup_sample_environment
+                RE -> backlight : move backlight out
+                ap_sg -> RE -- : select aperture completed
+            deactivate RE /' setup_sample_environment '/
+            RE -> smargon : set velocity
+            RE -> zebra : arm zebra PC
+            opt wait for topup
+                RE -> synchrotron ++ : check synchrotron status vs desired exposure
+                ...wait until topup is complete if necessary...
+                return 
+            end
+            RE -> smargon : trigger scan
+            RE -> RotationISPyB : read hardware during scan
+            & -> RotationScanNexus
+        deactivate RE /' rotation_scan_plan '/
+        RE -> RotationISPyB : record subplan exit status
+        deactivate RotationISPyB
+    end /' loop for each rotation '/
+    deactivate RotationISPyB
+    eiger -> RE -- : unstage eiger
+    RE -> zebra : tidy zebra
+    deactivate zebra
+    deactivate RotationScanNexus
+deactivate RE /' start_preparing_data_collection_then_do_plan '/
+
+@enduml

mx_bluesky-1.5.15/docs/user/hyperion/advanced/concurrent_operations_set_energy.puml

@@ -0,0 +1,38 @@
+@startuml
+!pragma teoz true
+participant "BlueSky RunEngine" as RE
+box "Opyhd Async Devices"
+participant "Undulator" as undulator
+participant "DCM" as dcm
+participant "Transmission\nand Feedback" as feedback
+participant "Mirrors" as mirrors
+participant "Mirror\nVoltages" as voltages
+end box
+
+group set_energy_plan
+    RE -> RE: set_energy_plan
+    activate RE
+    RE -> feedback: feedback off
+    activate feedback
+    RE -> undulator: set undulator gap for requested energy
+    activate undulator
+    RE -> dcm: Adjust DCM pitch and roll for energy
+    activate dcm
+    opt mirror stripe needs changing
+        RE -> mirrors: Adjust mirror stripe
+        activate mirrors
+        RE -> mirrors: Adjust yaw, lat
+        {start_voltages} RE -> voltages: Adjust mirror voltages
+        activate voltages
+        ...Wait for all voltages to settle...
+        {end_voltages} voltages -> RE
+        deactivate voltages
+        deactivate mirrors
+    end
+    undulator -> RE
+    deactivate undulator
+    deactivate dcm
+    deactivate feedback
+    deactivate RE /' set energy plan '/
+end
+@enduml

mx_bluesky-1.5.15/docs/user/hyperion/advanced/index.rst

@@ -0,0 +1,9 @@
+Advanced Topics
+===============
+
+.. toctree::
+    :maxdepth: 1
+    :glob:
+    
+
+    *

mx_bluesky-1.5.15/docs/user/hyperion/advanced/install.rst

@@ -0,0 +1,57 @@
+Server Install
+--------------
+
+Install Location
+~~~~~~~~~~~~~~~~
+
+Hyperion is generally installed in the beamline software directory:
+
+``/dls_sw/<beamline>/software/bluesky``
+
+This directory contains versioned installation folders ``mx-bluesky_vx.y.z`` for each installed version. Within the 
+directory there is a symlink ``hyperion`` to the currently active version. This is generally a symlink to either 
+``hyperion_stable`` or ``hyperion_latest`` symlinks which then point to the latest stable and development releases 
+respectively.
+
+Switching Versions
+~~~~~~~~~~~~~~~~~~
+
+For example to switch from the development to the stable release simply:
+
+::
+
+    rm hyperion
+    ln -s hyperion_latest hyperion
+
+After this you will need to run ``hyperion_restart()`` in the GDA jython console to restart hyperion
+
+Verifying that Hyperion is running
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To verify that the correct version of Hyperion is running, you should be able to view the process on the control 
+server, ssh into the control server, the output of ``ps ax`` should be something like the following, showing which 
+deployment is currently running. You should see two processes, the main process and also the external callback process. 
+
+::
+
+    $ ps ax | grep hyperion
+    1181420 ?        Sl     2:36 /dls_sw/i03/software/bluesky/mx-bluesky_v1.5.0/mx-bluesky/.venv/bin/python /dls_sw/i03/software/bluesky/mx-bluesky_v1.5.0/mx-bluesky/.venv/bin/hyperion
+    1181422 ?        Sl     1:54 /dls_sw/i03/software/bluesky/mx-bluesky_v1.5.0/mx-bluesky/.venv/bin/python /dls_sw/i03/software/bluesky/mx-bluesky_v1.5.0/mx-bluesky/.venv/bin/hyperion-callbacks
+
+Kubernetes Install
+------------------
+
+If Hyperion is deployed on Kubernetes (currently experimental) then it can be managed from the beamline kubernetes 
+dashboard, e.g. 
+https://k8s-i03-dashboard.diamond.ac.uk
+
+In the beamline namespace there will be a deployment ``hyperion-deployment``, a ``hyperion-svc`` service and associated 
+pods, ingress etc. through which the current state may be observed / managed.
+
+Logging
+~~~~~~~
+
+On kubernetes deployments, the initial startup is sent to standard IO and is captured as part of the standard 
+kubernetes logging facility.
+
+The configured logging locations are defined in the ``values.yaml`` for the specific deployment Helm chart. 

mx_bluesky-1.5.15/docs/user/hyperion/advanced/operations.rst

@@ -0,0 +1,46 @@
+Overview of the Data Collection Plan
+====================================
+
+This page shows a high level overview of the overall data collection plan, called ``load_centre_collect``.
+
+The intention of this is to show the sequencing of operations, including those operations that happen concurrently.
+
+This has been broken down into three main diagrams 
+
+- robot load and centring
+- rotations
+- changing energy
+
+At a high level ``load_centre_collect`` performs all three operations, however depending on the requested collection 
+parameters and whether the correct pin is already loaded, it may skip some of these steps.
+
+Note that not all of the operations are shown, in order to reduce the complexity of the diagrams, and some of the 
+steps have been slightly simplified.
+
+In conjunction with the `Code Map`_ this should provide a good overview of operations that may assist in interpreting
+logging output should any issues occur.
+
+
+.. _`Code Map`: ../../../developer/code-map/index.rst
+
+Robot Load and X-ray Centring
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This shows
+
+- Robot Load
+- Pin Tip Detection
+- Grid Detection
+- X-ray Centring
+
+.. uml:: concurrent_operations.puml
+
+Rotations
+~~~~~~~~~
+
+.. uml:: concurrent_operations_rotation.puml
+
+Set energy
+~~~~~~~~~~
+
+.. uml:: concurrent_operations_set_energy.puml

{mx_bluesky-1.5.12 → mx_bluesky-1.5.15}/docs/user/hyperion/configuration.rst

@@ -13,6 +13,7 @@ Configuration of several properties that control Hyperion execution are availabl
 
 Note that making a change to these properties generally requires a restart of the GDA server.
 
+Also note that some of these configuration properties will be removed in a future release of Hyperion. 
 
 .. csv-table:: Configuration properties
     :widths: auto

{mx_bluesky-1.5.12 → mx_bluesky-1.5.15}/docs/user/hyperion/index.rst

@@ -1,8 +1,7 @@
 Hyperion User Guide
 ===================
 
-The Hyperion User Guide describes how to run, configure and troubleshoot Hyperion. For installation instructions, see
-the `Developer Guide`_.
+The Hyperion User Guide describes how to run, configure and troubleshoot Hyperion. See the `getting started <../../developer/general/how-to/get-started.html>`_ guide for installation instructions.
 
 What is Hyperion?
 -----------------
@@ -17,7 +16,7 @@ The software supports two modes of operation:
   instructions). GDA mode will be removed in a future release.
 
 The mode of operation is determined by configuration using the ``gda.mx.udc.hyperion.enable`` parameter inside 
-GDA properties. See `configuration`_ for other properties that control Hyperions behavior. 
+GDA properties. See :doc:`configuration<./configuration>` for other properties that control Hyperions behavior. 
 
 Once Hyperion has received a request, either from GDA or directly from Agamemnon, it will do the following tasks:
 
@@ -43,15 +42,14 @@ to a user is:
 - If an xray centre cannot be done on a pin (because it is too long/short or bent), or if no diffraction is found in the 
   xray centring data, the sample will be immediately unloaded without a rotation scan being done and the next sample loaded.
 
-For a more detailed breakdown of the operation, it may be helpful to browse the  `Code Map`_
-
-.. _Code Map: ../../developer/code-map 
-.. _Developer Guide: ../../developer/hyperion/deploying-hyperion
-.. _configuration: ./configuration
+For a more detailed overview of operations, it may be helpful to view the
+:doc:`Data Collection Plan Diagrams<./advanced/operations>`
 
 .. toctree::
     :caption: Topics
     :maxdepth: 1
     :glob:
+    
 
     *
+    advanced/index

mx_bluesky-1.5.15/docs/user/hyperion/logging.rst

@@ -0,0 +1,43 @@
+Logging
+=======
+
+Hyperion logs to a number of different locations. The sections below describe the locations that are configured via 
+environment variables for a standard server install.
+
+Graylog
+~~~~~~~
+
+Graylog is the recommended way to view logs. It is used for more centralised logging which is also more easily 
+searched and archived. Log messages are sent to the `Hyperion graylog stream <https://graylog.diamond.ac.uk/streams/66264f5519ccca6d1c9e4e03/search>`_.
+
+To find different severity of message you can search for ``level: X``. Where X is:
+
+* ``3``: Errors messages - usually causing Hyperion to have stopped. Note that no diffraction is currently logged as an error but this doesn't stop Hyperion, only skip the sample. This will be improved by https://github.com/DiamondLightSource/mx-bluesky/issues/427.
+* ``4``: Warnings - usually not a major problem
+* ``6``: Info - Just telling you what Hyperion is doing
+
+
+Startup Log
+~~~~~~~~~~~
+
+When ``hyperion_restart()`` is called by GDA, it will log the initial console output to a log file. This log file 
+location is 
+controlled by the ``gda.logs.dir`` property and is typically ``/dls_sw/<beamline>/logs/bluesky``.
+
+Log files
+~~~~~~~~~
+
+By default, Hyperion logs to the filesystem.
+
+The log file location is controlled by the ``LOG_DIR`` environment value. Typically this can be found in 
+``/dls_sw/<beamline>/logs/bluesky``
+
+Debug Log
+~~~~~~~~~
+
+The standard logs files do not record all messages, only those at INFO level or higher, in order to keep storage 
+requirements to a minimum. 
+In the event of an error occurring, then trace-level logging for the most recent events (by default 5000) is flushed 
+to a separate set of log files. Due to the size of these files, they are stored separately from the main log files
+. These logs are located in ``/dls/tmp/<beamline>/logs/bluesky`` by default, or 
+otherwise as specified by the ``DEBUG_LOG_DIR`` environment variable. 

mx_bluesky-1.5.15/docs/user/hyperion/troubleshooting.rst

@@ -0,0 +1,53 @@
+Troubleshooting
+===============
+
+.. contents::
+
+Known Issues
+------------
+
+Odin Errors when there are filesystem issues
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+https://github.com/DiamondLightSource/mx-bluesky/issues/1199
+
+On occasions where there are issues with the filesystem you may see errors similar to
+
+::
+
+    ophyd.utils.errors.UnknownStatusFailure: The status (Status(obj=EpicsSignalWithRBV
+    (read_pv='BL03I-EA-EIGER-01:OD:Capture_RBV', name='eiger_odin_file_writer_capture', parent='eiger_odin_file_writer',
+    value=0, timestamp=1754488753.208739, auto_monitor=False, string=False, write_pv='BL03I-EA-EIGER-01:OD:Capture',
+    limits=False, put_complete=False), done=True, success=False) & SubscriptionStatus(device=eiger_odin_meta_ready,
+    done=False, success=False)) has failed. To obtain more specific, helpful errors in the future, update the Device
+    to use set_exception(...) instead of _finished(success=False).
+
+hyperion_restart() sometimes times out
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Sometimes hyperion_restart() will time out waiting for Hyperion to start, in the Jython console you may see the 
+following
+
+::
+
+    InteractiveConsole exception: hyperion_utils.exceptions.HyperionFailedException: Hyperion failed to start, see /dls_sw/i03/logs/bluesky/start_log.log for log
+    org.python.core.PyException: hyperion_utils.exceptions.HyperionFailedException: Hyperion failed to start, see /dls_sw/i03/logs/bluesky/start_log.log for log
+	at org.python.core.PyException.doRaise(PyException.java:239)
+	at org.python.core.Py.makeException(Py.java:1654)
+	at org.python.core.Py.makeException(Py.java:1658)
+	at org.python.core.Py.makeException(Py.java:1662)
+
+However on inspection the start log will not show any errors. Hyperion running can be verified as above `Verifying 
+that Hyperion is running`_
+
+.. _`Verifying that Hyperion is running`: advanced/install.rst
+
+Smargon Motion
+~~~~~~~~~~~~~~
+
+There are potential race conditions surrounding smargon moves. This may manifest as a CancelledError or TimeoutError 
+on one of the smargon axes. The workaround if this occurs is to run a collection in GDA to confirm correct operation 
+and then return control to Hyperion. 
+
+We believe this is fixed with https://github.com/DiamondLightSource/mx-bluesky/issues/1049 however it is possible 
+that this may recur.