mx-bluesky 1.4.6__tar.gz → 1.4.8__tar.gz

mx_bluesky-1.4.8/.github/CODEOWNERS

@@ -0,0 +1 @@
+* @DiamondLightSource/developers-mx-daq

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/.github/CONTRIBUTING.md

@@ -17,6 +17,23 @@ Contributions and issues are most welcome! All issues and pull requests are hand
 
 **Work should not be done without an associated ticket describing what the work is**
 
+## Developing mx-bluesky and dodal
+
+Some issues will span the mx-bluesky repo and the dodal repo. When working on both at the same time, follow these instructions:
+1. An issue is created for the work, inside mx-bluesky. This issue should describe in as much detail as possible the work that needs to be done. Anyone is free to make a ticket and/or comment on one.
+2. If a developer is going to do the work they assign themselves to the issue.
+3. The developer makes a new branch in mx-bluesky with the format `issue_short_description` e.g. `122_create_a_contributing_file`. The developer also creates a new branch in dodal, with the format `mx_bluesky_issue_short_description` - the same as the mx-bluesky branch name but with `mx_bluesky_` in front. (External developers are also welcome to make forks)
+4. The developer does the work on these branches, adding their work in small commits. Commit messages should be informative and prefixed with the issue number e.g. `(#122) Added contributing file`.
+5. Once the developer has done the final commit in both repos, they find the latest commit SHA in the dodal repo using `git log`. In `mx-bluesky/pyproject.toml`, the developer changes `dls-dodal @ git+https://github.com/DiamondLightSource/dodal.git@{dodal_commmit_SHA}` using the SHA they just found.
+6. The developer commits and pushes this final change on mx-bluesky.
+7. The developer submits a dodal PR for the work they have done in the dodal repo. The pull request should start with `Fixes #issue_num` e.g. `Fixes #122`, and should link to the issue created in mx-bluesky. The developer should also add some background on how the reviewer might test the change.
+8. The developer submits a mx-bluesky PR for the work they have done in the mx-bluesky repo. The pull request should start with `Fixes #issue_num` e.g. `Fixes #122`. This PR should link to the dodal PR just created with something like "Also needs #456". Again, the developer should add some background on how the reviewer might test the change.
+9. If the developer has a particular person in mind to review the work they should assign that person to the PRs as a reviewer.
+10. The reviewer and developer go back and forth on the code until the reviewer approves it. (See "Reviewing Work" below)
+11. Once the work is approved on both PRs, the original developer merges them. The PRs should be merged at the same time.
+
+**Work should not be done without an associated ticket describing what the work is**
+
 ## Reviewing Work
 
 **Work must be reviewed by another developer before it can be merged**. Remember that work is reviewed for a number of reasons:

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/.github/workflows/_test.yml

@@ -51,7 +51,9 @@ jobs:
           pip-install: ".[dev]"
 
       - name: Run tests
-        run: tox -e tests -- -m "not dlstbx"
+        # Longer timeout for CI due to likelihood of cold caches,
+        # cloud infra hw contention etc.
+        run: tox -e tests -- -m "not dlstbx" --timeout=2
 
       - name: Upload coverage to Codecov
         uses: codecov/codecov-action@v4

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/.gitignore

@@ -11,7 +11,8 @@ __pycache__/
 env/
 # Output
 *.png
-!docs/*.png
+!docs/**/*.png
+!tests/test_data/**/*.png
 
 # Distribution / packaging
 .Python

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: mx-bluesky
-Version: 1.4.6
+Version: 1.4.8
 Summary: Bluesky tools for MX Beamlines at DLS
 Author-email: Dominic Oram <dominic.oram@diamond.ac.uk>
 License:                                  Apache License
@@ -215,7 +215,6 @@ Description-Content-Type: text/x-rst
 License-File: LICENSE
 Requires-Dist: annotated_types
 Requires-Dist: caproto
-Requires-Dist: confluent-kafka==2.8.0
 Requires-Dist: fastapi[all]
 Requires-Dist: flask-restful
 Requires-Dist: ispyb
@@ -237,10 +236,10 @@ Requires-Dist: semver
 Requires-Dist: matplotlib
 Requires-Dist: blueapi>=0.5.0
 Requires-Dist: daq-config-server>=0.1.1
-Requires-Dist: ophyd==1.9.0
-Requires-Dist: ophyd-async>=0.9.0a2
-Requires-Dist: bluesky>=1.13
-Requires-Dist: dls-dodal==1.41.0
+Requires-Dist: ophyd>=1.10.5
+Requires-Dist: ophyd-async>=0.10.0a2
+Requires-Dist: bluesky>=1.13.1
+Requires-Dist: dls-dodal==1.47.0
 Provides-Extra: dev
 Requires-Dist: black; extra == "dev"
 Requires-Dist: build; extra == "dev"
@@ -254,10 +253,11 @@ Requires-Dist: pipdeptree; extra == "dev"
 Requires-Dist: plantweb; extra == "dev"
 Requires-Dist: pre-commit; extra == "dev"
 Requires-Dist: pydata-sphinx-theme>=0.12; extra == "dev"
-Requires-Dist: pyright==1.1.394; extra == "dev"
+Requires-Dist: pyright; extra == "dev"
 Requires-Dist: pytest-asyncio; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
 Requires-Dist: pytest-random-order; extra == "dev"
+Requires-Dist: pytest-timeout; extra == "dev"
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: ruff; extra == "dev"
 Requires-Dist: sphinx-autobuild; extra == "dev"
@@ -267,6 +267,7 @@ Requires-Dist: tox-direct; extra == "dev"
 Requires-Dist: tox; extra == "dev"
 Requires-Dist: types-mock; extra == "dev"
 Requires-Dist: types-requests; extra == "dev"
+Dynamic: license-file
 
 mx-bluesky
 ===========================
@@ -285,7 +286,7 @@ Releases       https://github.com/DiamondLightSource/mx-bluesky/releases
 Getting Started
 ===============
 
-To get started with developing this repo at DLS run ``dls_dev_setup.sh``.
+To get started with developing this repo at DLS run ``./utility_scripts/dls_dev_env.sh``.
 
 If you want to develop interactively at the beamline we recommend using jupyter notebooks. You can get started with this by running::
 

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/README.rst

@@ -15,7 +15,7 @@ Releases       https://github.com/DiamondLightSource/mx-bluesky/releases
 Getting Started
 ===============
 
-To get started with developing this repo at DLS run ``dls_dev_setup.sh``.
+To get started with developing this repo at DLS run ``./utility_scripts/dls_dev_env.sh``.
 
 If you want to develop interactively at the beamline we recommend using jupyter notebooks. You can get started with this by running::
 

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/conftest.py

@@ -1,21 +1,11 @@
 from collections.abc import Iterator
-from os import environ, getenv
+from os import environ
 from unittest.mock import patch
 
 import pytest
 
 environ["HYPERION_TEST_MODE"] = "true"
 
-print("Adjusting S03 EPICS environment ...")
-s03_epics_server_port = getenv("S03_EPICS_CA_SERVER_PORT")
-s03_epics_repeater_port = getenv("S03_EPICS_CA_REPEATER_PORT")
-if s03_epics_server_port:
-    environ["EPICS_CA_SERVER_PORT"] = s03_epics_server_port
-    print(f"[EPICS_CA_SERVER_PORT] = {s03_epics_server_port}")
-if s03_epics_repeater_port:
-    environ["EPICS_CA_REPEATER_PORT"] = s03_epics_repeater_port
-    print(f"[EPICS_CA_REPEATER_PORT] = {s03_epics_repeater_port}")
-
 
 def pytest_addoption(parser):
     parser.addoption(

mx_bluesky-1.4.8/docs/developer/code-map/grid_detect_xrc.rst

@@ -0,0 +1,60 @@
+Grid Detect Then Xray Centre
+============================
+
+The aim of this page is to provide a general overview of the Hyperion ``grid_detect_then_xray_centre_plan``.
+
+Most of the code for the ``grid_detect_then_xray_centre_plan`` plan is in ``mx-bluesky/hyperion``. Some code is located in ``mx-bluesky/common`` and `dodal <https://github.com/DiamondLightSource/dodal>`_, and plans are increasingly being generalised and moved into ``mx-bluesky/common`` where possible. Therefore, the structure of the codebase is subject to change and this document should be updated accordingly.
+
+There are then a number of plans that make up the ``grid_detect_then_xray_centre_plan`` plan. Some important ones:
+
+* :ref:`grid_detection_plan<grid-detect>` - Use the OAV to optically calculate a grid for a scan that would cover the whole sample.
+* :ref:`flyscan_xray_centre_plan<flyscan>` - Triggers a hardware-based grid scan and moves to the X-ray centre as returned from ``zocalo``.
+* :ref:`grid_detect_then_xray_centre_plan<grid-detect-xrc>` - This top-level plan performs an :ref:`OAV grid detection <grid-detect>` then a :ref:`flyscan x-ray centre <flyscan>`.
+
+The diagram below shows all the plans that make up the ``grid_detect_then_xray_centre_plan``. The colors indicate where these plans can be found.
+
+.. image:: grid_detect_then_xray_centre.drawio.png
+
+.. _grid-detect-xrc:
+
+Grid Detect Then Xray Centre Plan
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+`This plan <https://github.com/DiamondLightSource/mx-bluesky/blob/main/src/mx_bluesky/hyperion/experiment_plans/grid_detect_then_xray_centre_plan.py>`__ does the following, in roughly this order:
+
+1. If called standalone, start preparing for data collection.
+2. Perform an :ref:`OAV grid detection <grid-detect>`.
+3. Convert the parameters calculated in step 2 into something we can send to the flyscan X-ray centre.
+4. Move the backlight out, set the aperture to small, and wait for the detector to finish moving.
+5. Perform a :ref:`flyscan X-ray centre <flyscan>`.
+6. Move the sample based on the results of step 5.
+
+.. _grid-detect:
+
+OAV Grid Detection
+~~~~~~~~~~~~~~~~~~
+
+`This plan <https://github.com/DiamondLightSource/mx-bluesky/blob/main/src/mx_bluesky/hyperion/experiment_plans/oav_grid_detection_plan.py>`__ does the following, in roughly this order:
+
+1. Move to omega 0.
+2. Calculate the 2D grid size using the edge arrays from the OAV.
+3. Trigger the OAV device to take snapshots, both with and without the grid.
+4. Read the snapshot paths (which will be gathered for ispyb in the background).
+5. Repeat steps 2–4 for omega 90.
+6. Return the grid positions.
+
+.. _flyscan:
+
+Flyscan Xray Centre No Move
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+`This plan <https://github.com/DiamondLightSource/mx-bluesky/blob/main/src/mx_bluesky/hyperion/experiment_plans/flyscan_xray_centre_plan.py>`__ does the following, in roughly this order:
+
+1. Move to the desired transmission (and turn off xbpm feedback).
+2. Move to omega 0.
+3. Read hardware values for ispyb (a grid scan entry will be added in the background).
+4. Set up zebra and motion devices for a grid scan to be done in the motion controller.
+5. Wait for the Eiger to finish arming or arm the Eiger if it hasn't already been done.
+6. Run the motion control grid scan.
+7. Wait for the grid scan to end.
+8. Retrieve the X-ray centering results from ``Zocalo`` (which will be gathered in the background).

mx_bluesky-1.4.8/docs/developer/code-map/index.rst

@@ -0,0 +1,17 @@
+Code Map
+==================================
+
+Explanation of the structure of the codebase
+
+.. grid:: 2
+    :gutter: 2
+
+    .. grid-item-card:: :material-regular:`code;3em`
+
+        .. toctree::
+            :caption: Grid Detect Then Xray Centre
+            :maxdepth: 1
+
+            grid_detect_xrc
+
+        +++

mx_bluesky-1.4.8/docs/developer/general/explanations/callback_and_run_logic.rst

@@ -0,0 +1,27 @@
+Callbacks and Run Logic
+========================
+
+Callbacks are used to trigger external services:
+
+- Ispyb deposition
+- Nexus writing
+- Zocalo triggering
+
+These are linked in that to trigger zocalo you need to have made an ispyb deposition, written a nexus file and have finished writing raw data to disk. Nexus files and ispyb depositions can be made at anytime, we do not need to have necessarily finished writing raw data. 
+
+Currently, the requirement of needing to have written to ispyb is explicit as the ispyb callback will emit to the zocalo callback. The nexus file is written when the hardware is read during a collection and so its ordering is implied. When instantiated the zocalo callback is told on which plan to trigger and it is up to the plan developer to make sure this plan finishes after data is written to the detector.
+
+In general, the ordering flow of when callbacks are triggered is controlled by emitting documents with the expected plan name and data.
+
+Rotation Scans
+---------------------
+
+There are currently two ways of doing rotation scans. A single scan creates one hdf file, one ispyb deposition and then triggers zocalo once. Multi rotation scans create one hdf file for all rotations but then N nexus files, N ispyb depositions and triggers zocalo N times.
+
+Single scans will be removed in https://github.com/DiamondLightSource/mx-bluesky/issues/847
+
+For multi rotations this is does by starting 1+2*N different runs:
+
+1. ``CONST.PLAN.ROTATION_MULTI``: This is emitted once for the whole multiple rotation. It is used by the nexus callback to get the full number of images and meta_data_run_number so that it knows which hdf file to use. When this is finished zocalo end is triggered.
+2. ``CONST.PLAN.ROTATION_OUTER``: Emitted N times, inside a ``CONST.PLAN.ROTATION_MULTI`` run. This is used to create the initial ispyb deposition and create the nexus writer (but not actually write the file)
+3. ``CONST.PLAN.ROTATION_MAIN``: Emitted N times, inside ``CONST.PLAN.ROTATION_OUTER`` run. Used to finish writing to ispyb (i.e. write success/failure) and to send collection information to zocalo.

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/general/how-to/get-started.rst

@@ -4,8 +4,8 @@ Get Started with mx-bluesky
 Development Environment
 =======================
 
-- Clone this repo: ``git@github.com:DiamondLightSource/mx-bluesky.git``
-- To install a ``dev`` environment run ``./dls_dev_setup.sh``. Note that this will also clone and install a local version of ``dodal``, as the ``mx-bluesky`` package makes use of the devices instantiated there. 
+- Clone this repo using SSH: ``git clone git@github.com:DiamondLightSource/mx-bluesky.git``
+- To install a ``dev`` environment, from the top level ``mx-bluesky`` directory, run ``./utility_scripts/dls_dev_env.sh``. Note that this will also clone and install a local version of ``dodal``, as the ``mx-bluesky`` package makes use of the devices instantiated there. 
 
 The recommended IDE is vscode, and a workspace which includes dodal has been set up in the repo. This can be used on a DLS machine as follows:
 

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/general/index.rst

@@ -41,6 +41,7 @@ Documentation is split into four categories, and each is also accessible from li
             :maxdepth: 1
 
             explanations/decisions
+            explanations/callback_and_run_logic
 
         +++
 

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/general/tutorials/profile-tests.rst

@@ -27,7 +27,7 @@ e.g.
 
 .. code-block:: bash
 
-    pytest -m "not (s03 or dlstbx)" --durations=10
+    pytest -m "not dlstbx" --durations=10
 
 in order to find the top 10 slowest tests
 

mx_bluesky-1.4.8/docs/developer/hyperion/how-to/update-panda-ioc.rst

@@ -0,0 +1,44 @@
+Update the PandA IOC
+=====================
+
+The IOC for the PandA can sometimes get out of sync with the production version of Ophyd-Async. Here is how to update it
+
+Check if the IOC needs updating
+""""""""""""""""""""""""""""""""
+
+- Check the minimum panda IOC version required in the version of ophyd-async used in Hyperion's release version. This can be found in the ``MINIMUM_PANDA_IOC`` constant in `_hdf_panda.py <https://github.com/bluesky/ophyd-async/blob/main/src/ophyd_async/fastcs/panda/_hdf_panda.py>`_
+
+- In a terminal:
+
+.. code-block:: bash
+
+    cd /dls_sw/work/R3.14.12.7/ioc/BL03I/BL03I-PY-IOC-02
+    source venv/bin/activate
+    pip freeze | grep pandablocks-ioc
+
+Note that the IOC location will soon be moving to ``prod`` instead of ``work``.
+
+- Compare the version from step one
+
+Updating the IOC
+""""""""""""""""""""""""""
+
+- Update the Python IOC repo in the venv:
+
+.. code-block:: bash
+
+    cd /dls_sw/work/R3.14.12.7/ioc/BL03I/BL03I-PY-IOC-02
+    source venv/bin/activate
+    pip install --upgrade pandablocks-ioc=={desired version}
+
+
+- After making sure the beamline isn't in use, restart the IOC:
+
+.. code-block:: bash
+
+    module load controls-tools
+
+    console BL03I-PY-IOC-02
+
+- Press ``ctrl+x``
+- Disconnect by pressing ``ctrl+e``, then press ``c``, then press ``.``

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/hyperion/index.rst

@@ -20,3 +20,15 @@ Documentation specific for the Hyperion module within MX-Bluesky
         +++
 
         Technical reference material on standards in use.
+
+    .. grid-item-card:: :material-regular:`task;3em`
+
+        .. toctree::
+            :caption: How-to Guides
+            :maxdepth: 1
+
+            how-to/update-panda-ioc
+
+        +++
+
+        Practical step-by-step guides for day-to-day dev tasks.

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/hyperion/reference/gridscan.puml

@@ -77,7 +77,7 @@ GridParamUpdate --> HyperionThreeDGridScan : combines with GridScanWithEdgeDetec
 
 class experiment_plans {
     grid_detect_then_xray_centre()
-    flyscan_xray_centre_no_move()
+    common_flyscan_xray_centre()
     create_parameters_for_flyscan_xray_centre(GridScanWithEdgeDetect, GridParamUpdate) -> HyperionThreeDGridScan
 }
 

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/hyperion/reference/readme.md

@@ -1,7 +1,5 @@
 # hyperion
 
-![Tests](https://github.com/DiamondLightSource/hyperion/actions/workflows/code.yml/badge.svg) [![codecov](https://codecov.io/gh/DiamondLightSource/hyperion/branch/main/graph/badge.svg?token=00Ww81MHe8)](https://codecov.io/gh/DiamondLightSource/hyperion)
-
 Repository for the Hyperion project to implement Unattended Data Collections on the Diamond MX beamlines using the [BlueSky](https://nsls-ii.github.io/bluesky/) / Ophyd framework from BNL.
 
 Currently the software is able to:
@@ -27,9 +25,14 @@ Note that because Hyperion makes heavy use of [Dodal](https://github.com/Diamond
 
 ## Starting the bluesky runner
 
-You can start the bluesky runner by running `run_hyperion.sh`. Note that this will fail on a developer machine unless you have a simulated beamline running, instead you should do `run_hyperion.sh --dev --skip-startup-connection`, which will give you a running instance (note that without hardware trying to run a plan on this will fail). The `--dev` flag ensures that logging will not be sent to the production Graylog.
+The bluesky runner is (re)started in production by GDA when it invokes `run_hyperion.sh`. 
+
+If you wish to attempt to run from a developer machine, then (if you are able to connect all devices) 
+you may `run_hyperion.sh --dev --beamline=<beamline>`, which will give you a running instance albeit with
+read-only devices. The `--dev` flag ensures that logging will not be sent to the production Graylog/output folders.
 
-This script will determine whether you are on a beamline or a production machine based on the `BEAMLINE` environment variable. If on a beamline Hyperion will run with `INFO` level logging, sending its logs to both production graylog and to the beamline/log/bluesky/hyperion.log on the shared file system.
+This script will determine which beamline you are on based on the `BEAMLINE` environment variable. If on a 
+beamline Hyperion will run with `INFO` level logging, sending its logs to both production graylog and to the beamline/log/bluesky/hyperion.log on the shared file system.
 
 If in a dev environment Hyperion will log to a local graylog instance instead and into a file at `./tmp/dev/hyperion.log`. A local instance of graylog will need to be running for this to work correctly. To set this up and run up the containers on your local machine run the `setup_graylog.sh` script.
 
@@ -43,17 +46,9 @@ The hyperion python module can also be run directly without the startup script.
 python -m hyperion --dev --verbose-event-logging
 ```
 
-Lastly, you can choose to skip running the hardware connection scripts on startup with the flag
-
-```
-python -m hyperion --skip-startup-connection
-```
-
 ## Testing
 
-Unit tests can be run with `python -m pytest -m "not s03" --random-order`. To see log output from tests you can turn on logging with the `--logging` command line option and then use the `-s` command line option to print logs into the console. So to run the unit tests such that all logs are at printed to the terminal, you can use `python -m pytest -m "not s03" --random-order --logging -s`. Note that this will likely overrun your terminal buffer, so you can narrow the selection of tests with the `-k "<test name pattern>"` option.
-
-To be able to run the system tests, or a complete fake scan, we need the simulated S03 beamline. This can be found at: https://gitlab.diamond.ac.uk/controls/python3/s03_utils
+Unit tests can be run with `pytest --random-order`. To see log output from tests you can turn on logging with the `--logging` command line option and then use the `-s` command line option to print logs into the console. So to run the unit tests such that all logs are at printed to the terminal, you can use `python -m pytest --random-order --logging -s`. Note that this will likely overrun your terminal buffer, so you can narrow the selection of tests with the `-k "<test name pattern>"` option.
 
 To fake interaction and processing with Zocalo, you can run `fake_zocalo/dls_start_fake_zocalo.sh`, and make sure to run `module load dials/latest` before starting hyperion (in the same terminal).
 

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/index.rst

@@ -53,3 +53,15 @@ Documentation is split general and topic-specific sections. Each section is spli
         +++
 
         Documentation relating to integrating the murko image recognition system
+
+    .. grid-item-card:: :material-regular:`code;3em`
+
+        .. toctree::
+            :caption: Code Map
+            :maxdepth: 1
+
+            code-map/index
+
+        +++
+
+        Code map explaining the structure of the codebase

mx_bluesky-1.4.6/docs/developer/serial-crystallography-on-i24/how-to/stage-pmac-moves.rst → mx_bluesky-1.4.8/docs/developer/serial-crystallography-on-i24/how-to/pmac-docs.rst

@@ -1,3 +1,20 @@
+=================================
+PMAC-run operations on I24 serial
+=================================
+
+
+Several operations in a fixed target (FT) collection are run vie the PMAC, by sending a command to the ``PMAC_STRING`` PV.
+
+Among these:
+
+- Stage moves
+- Kickoff data collection
+- Laser control
+
+
+For reference, see also the PMAC device implementation in dodal: `PMAC docs <https://diamondlightsource.github.io/dodal/main/reference/generated/dodal.devices.i24.pmac.html>`_
+
+
 Stage motor moves using the PMAC device
 ---------------------------------------
 
@@ -18,6 +35,8 @@ to motors. The axes allocations defined for the chip stages set up are:
 
 When an X-axis move is executed, the #1 motor will make the move.
 
+It should be noted that the PMAC coordinate system used by I24 serial is number 2, which can be specified in a PMAC string by starting the command with ``&2``.
+
 When running chip collections, the stage motors are moved via the `PMAC
 device <https://github.com/DiamondLightSource/dodal/blob/main/src/dodal/devices/i24/pmac.py>`__
 in a couple of different ways.
@@ -28,19 +47,31 @@ in a couple of different ways.
    -  Using a JOG command ``J:{const}``, to jog the motor a specified
       distance from the current position. For example, this will move
       motor Y by 10 motor steps:
-      ``python      yield from bps.abs_set(pmac.pmac_string, "#2J:10")``
+
+      .. code:: python
+
+         yield from bps.abs_set(pmac.pmac_string, "#2J:10")
+
 
    -  The ``hmz`` strings are homing commands which will reset the
       encoders counts to 0 for the axis. All three motors are homed by
       sending the string: ``#1hmz#2hmz#3hmz``. In the plans this is done
       by triggering the home move:
-      ``python       yield from bps.trigger(pmac.home)``
+
+      .. code:: python
+
+         yield from bps.trigger(pmac.home)
+
 
    -  Another pmac_string that can start a move has the format
       ``!x..y..``. This is a command designed to blend any ongoing move
       into a new position. A common one through the serial collection
       code is ``!x0y0z0``, which will start a move to 0 for all motors.
-      ``python      yield from bps.trigger(pmac.to_xyz_zero)``
+
+      .. code:: python
+
+         yield from bps.trigger(pmac.to_xyz_zero)
+
 
 2. The stage motors can also be moved directly through the existing PVs
    ``ME14E-MO-CHIP-01:{X,Y,Z}``, for example:
@@ -49,6 +80,7 @@ in a couple of different ways.
 
       yield from bps.mv(pmac.x, 0, pmac.y, 1)
 
+
 Notes on the coordinate system for a fixed-target collection
 ============================================================
 
@@ -81,7 +113,7 @@ Theory for this computation
 
 The plan needs information stored in a few files:
 
-* The motor directions are stored in ``src/mx_bluesky/i24/serial/parameters/fixed_target/cs/motor_directions.txt.`` The motor number multiplied by the motor direction should give the positive chip direction. 
+* The motor directions are stored in ``src/mx_bluesky/i24/serial/parameters/fixed_target/cs/motor_direction.txt.`` The motor number multiplied by the motor direction should give the positive chip direction.
 * The scale values for x,y,z, the skew value and the sign of Sx, Sy, Sz are all stored in ``src/mx_bluesky/i24/serial/parameters/fixed_target/cs/cs_maker.json``
 * The fiducials 1 and 2 positions are written to file when selecting the fiducials (Setting fiducial 0 instead sends a homing command directly to the pmac_string PV)
 
@@ -106,3 +138,78 @@ the following pmac_strings:
    "#1->10000X+0Y+0Z"
    "#2->+0X-10000Y+0Z"
    "#3->0X+0Y-10000Z"
+
+
+
+
+Data collection via the PMAC
+----------------------------
+
+The data collection for a FT experiment is kicked off by sending a PMAC_STRING with the program number of the motion program the PMAC should run.
+
+Two P-variables - general purpose variables that can be used to store information on the PMAC - have been set aside to monitor the collection run:
+
+::
+
+   P2401 is the "scan_status" variable. It goes to 1 once the motion program starts and will go back to 0 at the very end of the collection
+   P2402 is the "counter" variable. It keeps count of how many images have been acquired so far in the collection.
+
+
+The program number is chosen depending on the input collection parameters:
+
+::
+
+   11 -> Custom, Mini and PSI type chip collections, as well as Oxford chips with mapping set to "None" (full chip collections)
+   12 -> Oxford Chips with Lite mapping (only some blocks collected)
+   13 -> In the past was used for "Full Mapping". **CURRENTLY DISABLED**
+   14 -> Any Pump Probe collection, with any chip type. **WARNING** Assumes Lite mapping for Oxford chips.
+
+To do this, the PMAC device in dodal implements a Flyable device (``ProgramRunner``) and a soft signal (``program_number``).
+The ``kickoff_and_complete_collection_plan`` first sets up the PMAC by setting the program_number signal and calculating the expected duration of the collection, and then triggers the collection by:
+
+.. code:: python
+
+   yield from bps.kickoff(pmac.run_program, wait=True)
+   yield from bps.complete(pmac.run_program, wait=True)
+
+
+The ``kickoff`` method works out the pmac_string to send from the program number in the following way:
+
+::
+
+   "&2b{prog_num}r" where
+      - &2 is the coordinate system in use
+      - b sets the motion program to run
+      - r runs it
+
+
+and then waits for the scan status P-variable to go to 1.
+The ``complete`` method instead monitors the scan status variable and waits for it to go back to 0.
+
+
+In the event of an aborted data collection, an additional Triggerable signal has been added to the PMAC device to be able to reset the PMAC.
+The abort plan for FT will call:
+
+.. code:: python
+
+   yield from bps.trigger(pmac.abort_program)
+
+
+which first sends a ``A`` command to the PMAC to tell it to abort the motion program being currently run and then resets the ``P2041`` variable to 0.
+There is no need to reset the ``P2402`` variable as it's automatically reset once the new motion program starts.
+
+
+
+Laser control
+-------------
+
+The ``laser_control`` plan switches a laser on and off by sending PMAC_STRINGS that set a pair of M-variables.
+M-variables point to a location in memory and are usually used for user access or I/O operations - in this case they have to do with position compare settings.
+
+The M-variables used here are M712/M711 for laser1 and M812/M811 for laser2.
+M711 and M811 are set to 1, while and the value set to M712/M812 indicates when the triggering happens, eg:
+
+::
+
+   M712 = 0 if triggering on the falling edge -> laser off
+   M712 = 1 if triggering on the rising edge -> laser on

{mx_bluesky-1.4.6 → mx_bluesky-1.4.8}/docs/developer/serial-crystallography-on-i24/index.rst

@@ -1,11 +1,11 @@
-I24 Serial Crystallography 
+I24 Serial Crystallography
 ==========================
 
 Documentation is split into four categories, and each is also accessible from links in the side-bar.
 
 .. grid:: 2
     :gutter: 2
-        
+
     .. grid-item-card:: :material-regular:`map;3em`
 
         .. toctree::
@@ -26,7 +26,7 @@ Documentation is split into four categories, and each is also accessible from li
 
             how-to/environment-setup
             how-to/run-a-collection
-            how-to/stage-pmac-moves
+            how-to/pmac-docs
 
         +++
 
@@ -62,4 +62,4 @@ Documentation is split into four categories, and each is also accessible from li
 
         +++
 
-        Tutorials for getting up and running as a developer.    
+        Tutorials for getting up and running as a developer.