mx-bluesky 0.3.1__tar.gz → 1.1.0__tar.gz

mx_bluesky-1.1.0/.coveragerc

@@ -0,0 +1,18 @@
+[run]
+omit = 
+    */test_*
+    **/conftest.py
+data_file = /tmp/hyperion.coverage
+
+[report]
+exclude_also = 
+    if TYPE_CHECKING:
+    def __repr__
+    raise NotImplementedError
+    @(abc\.)?abstractmethod
+
+[paths]
+# Tests are run from installed location, map back to the src directory
+source =
+    src
+    **/site-packages/

{mx_bluesky-0.3.1 → mx_bluesky-1.1.0}/.devcontainer/Dockerfile

@@ -3,7 +3,7 @@
 # The devcontainer should use the build target and run as root with podman
 # or docker with user namespaces.
 #
-FROM python:3.11 as build
+FROM python:3.11 AS build
 
 ARG PIP_OPTIONS
 
@@ -24,7 +24,7 @@ WORKDIR /context
 # install python package into /venv
 RUN pip install ${PIP_OPTIONS}
 
-FROM python:3.11-slim as runtime
+FROM python:3.11-slim AS runtime
 
 # Add apt-get system dependecies for runtime here if needed
 

mx_bluesky-1.1.0/.dockerignore

@@ -0,0 +1,12 @@
+# List of folders and files to be excluded from the docker image
+.devcontainer
+.github
+.pytest_cache
+.ruff_cache
+**/__pycache__/
+
+# virtualenv stuff - this gets built by the docker script
+.venv
+activate
+
+tmp

mx_bluesky-1.1.0/.github/CONTRIBUTING.md

@@ -0,0 +1,57 @@
+# How to contribute to mx-bluesky
+
+We should prioritise working, agile code over strict standards, particularly for contributors that are not full time developers. The standards below are thus guidelines that developers should try to follow as much as possible (apart from when something is specified in bold, which is required).
+
+Contributions and issues are most welcome! All issues and pull requests are handled through [GitHub](https://github.com/DiamondLightSource/mx-bluesky/issues). Also, please check for any existing issues before filing a new one. If you have a great idea but it involves big changes, please file a ticket before making a pull request! We want to make sure you don't spend your time coding something that might not fit the scope of the project.
+
+## General Workflow
+
+1. An issue is created for the work. This issue should describe in as much detail as possible the work that needs to be done. Anyone is free to make a ticket and/or comment on one.
+2. If a developer is going to do the work they assign themselves to the issue.
+3. The developer makes a new branch with the format `issue_short_description` e.g. `122_create_a_contributing_file`. (External developers are also welcome to make forks)
+4. The developer does the work on this branch, adding their work in small commits. Commit messages should be informative and prefixed with the issue number e.g. `(#122) Added contributing file`.
+5. The developer submits a PR for the work. In the pull request should start with `Fixes #issue_num` e.g. `Fixes #122`, this will ensure the issue is automatically closed when the PR is merged. The developer should also add some background on how the reviewer might test the change.
+6. If the developer has a particular person in mind to review the work they should assign that person to the PR as a reviewer.
+7. The reviewer and developer go back and forth on the code until the reviewer approves it. (See "Reviewing Work" below)
+8. Once the work is approved the original developer merges it.
+
+**Work should not be done without an associated ticket describing what the work is**
+
+## Reviewing Work
+
+**Work must be reviewed by another developer before it can be merged**. Remember that work is reviewed for a number of reasons:
+
+- In order to maintain quality and avoid errors
+- Help people learn
+
+It is not a judgement on someone's technical abilities so be kind.
+
+It is suggested that the reviewer prefixes comments with something like the following:
+
+- **must**: A change that must be made before merging e.g. it will break something if not made
+- **should/sugg**: A change that should be made e.g. definitely improves code quality but does not neccessarily break anything
+- **nit**: A minor suggestion that the reviewer would like to see but is happy to leave as is e.g. rename a variable to something
+
+Developers are welcome to ignore **nit** comments if they wish and can choose not to do **should** comments but the must give a reason as to why they disagree with the change.
+
+For minor changes to code reviewers are welcome to make the changes themselves but in this case the original developer should then recheck what the reviewer has done.
+
+## Issue or Discussion?
+
+Github also offers [discussions](https://github.com/DiamondLightSource/mx-bluesky/discussions) as a place to ask questions and share ideas. If
+your issue is open ended and it is not obvious when it can be "closed", please
+raise it as a discussion instead.
+
+## Code Coverage
+
+While 100% code coverage does not make a library bug-free, it significantly
+reduces the number of easily caught bugs! Please make sure coverage remains the
+same or is improved by a pull request!
+
+## Developer Information
+
+It is recommended that developers use a [vscode devcontainer](https://code.visualstudio.com/docs/devcontainers/containers). This repository contains configuration to set up a containerized development environment that suits its own needs.
+
+This project was created using the [Diamond Light Source Copier Template](https://github.com/DiamondLightSource/python-copier-template) for Python projects.
+
+For more information on common tasks like setting up a developer environment, running the tests, and setting a pre-commit hook, see the template's [How-to guides](https://diamondlightsource.github.io/python-copier-template/2.2.0/how-to.html).

{mx_bluesky-0.3.1 → mx_bluesky-1.1.0}/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md

@@ -1,8 +1,13 @@
 Fixes #ISSUE
 
+Link to dodal PR (if required): #XXX
+(remember to update `setup.cfg` with the dodal commit tag if you need it for tests to pass!)
+
 ### Instructions to reviewer on how to test:
+
 1. Do thing x
 2. Confirm thing y happens
 
 ### Checks for reviewer
+
 - [ ] Would the PR title make sense to a user on a set of release notes

mx_bluesky-1.1.0/.github/actions/verify-nexus/Dockerfile

@@ -0,0 +1,3 @@
+FROM ghcr.io/githubgphl/imginfo:main
+COPY entrypoint.sh /entrypoint.sh
+ENTRYPOINT ["/entrypoint.sh"]

mx_bluesky-1.1.0/.github/actions/verify-nexus/action.yml

@@ -0,0 +1,18 @@
+name: "Verify nexus"
+description: "Verify nexus files against imginfo"
+inputs:
+  filename:
+    description: "nexus file to verify"
+    required: true
+outputs:
+  imginfo_stdout:
+    description: "imginfo output"
+  imginfo_stderr:
+    description: "imginfo error output"
+  imginfo_exit_code:
+    description: "imginfo exit code"
+runs:
+  using: "docker"
+  image: "Dockerfile"
+  args:
+    - ${{ inputs.filename }}

mx_bluesky-1.1.0/.github/actions/verify-nexus/entrypoint.sh

@@ -0,0 +1,25 @@
+#!/bin/sh -l
+echo "Contents of /github/workspace:"
+ls /github/workspace
+echo "current dir: $(pwd)"
+echo "Running imginfo on $1"
+/imginfo/imginfo  $1 >> imginfo_out_file 2>> imginfo_err_file
+{ echo "imginfo_output<<EOF"
+  cat imginfo_out_file
+  echo EOF
+} >> "$GITHUB_OUTPUT"
+{ echo "imginfo_output<<EOF"
+  cat imginfo_err_file
+  echo EOF
+} >> "$GITHUB_OUTPUT"
+echo "imginfo_exit_code=$?" >> "$GITHUB_OUTPUT"
+echo "------------- IMGINFO STDOUT -------------"
+cat imginfo_out_file
+echo "------------- IMGINFO STDERR -------------"
+cat imginfo_err_file
+echo "------------------------------------------"
+if [ -s imginfo_err_file ]; then
+  echo "ERRORS IN IMGINFO PROCESSING!"
+  exit 1
+fi
+echo "VALIDATED SUCCESSFULLY!"

{mx_bluesky-0.3.1 → mx_bluesky-1.1.0}/.github/workflows/_test.yml

@@ -51,7 +51,7 @@ jobs:
           pip-install: ".[dev]"
 
       - name: Run tests
-        run: tox -e tests
+        run: tox -e tests -- -m "not dlstbx"
 
       - name: Upload coverage to Codecov
         uses: codecov/codecov-action@v4

{mx_bluesky-0.3.1 → mx_bluesky-1.1.0}/.github/workflows/ci.yml

@@ -21,11 +21,7 @@ jobs:
     strategy:
       matrix:
         runs-on: ["ubuntu-latest"] # can add windows-latest, macos-latest
-        python-version: ["3.10", "3.11"] # , "3.12"] # add 3.12 when p4p #145 is fixed
-        include:
-          # Include one that runs in the dev environment
-          - runs-on: "ubuntu-latest"
-            python-version: "dev"
+        python-version: ["3.11"] # , "3.12"] # add 3.12 when p4p #145 is fixed
       fail-fast: false
     uses: ./.github/workflows/_test.yml
     with:

mx_bluesky-1.1.0/.github/workflows/publish_docker_image.yml

@@ -0,0 +1,50 @@
+name: Publish Docker Image
+on:
+  release:
+    types: [published]
+  # Allow the workflow to be triggered manually
+  workflow_dispatch:
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+jobs:
+  build_and_push_image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - name: Checkout
+        # v4.1.7
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332
+      - name: Log in to GHCR
+        # v3.3.0
+        uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        # v5.5.1
+        uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+      - name: Build and push Docker image
+        # v6.5.0
+        uses: docker/build-push-action@5176d81f87c23d6fc96624dfdbcd9f3830bbe445
+        with:
+          context: .
+          push: true
+          tags: |
+            ${{ steps.meta.outputs.tags }}
+            ${{ github.sha }}
+          labels: ${{ steps.meta.outputs.labels }}
+          file: Dockerfile.release
+      - name: Generate artifact attestation
+        # v1.4.0
+        uses: actions/attest-build-provenance@210c1913531870065f03ce1f9440dd87bc0938cd
+        with:
+          subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
+          subject-digest: ${{ steps.push.outputs.digest }}
+          push-to-registry: true

mx_bluesky-1.1.0/.gitignore

@@ -0,0 +1,173 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+# Output
+*.png
+!docs/*.png
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+**/_version.py
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+cov.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cov.xml
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+.venv*/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# gedit backup files
+*~
+
+# Log files
+**/tmp
+
+# further build artifacts
+lockfiles/
+
+# ruff cache
+.ruff_cache/
+
+# Temporary files
+tmp/
+
+# EDM screens
+edm/
+edm_serial/
+
+# Edm backup files
+*.edl~
+
+# Parameter files that could be modified by hand by scientists
+# or created during collection
+**/parameters/*.json
+**/parameters/*.txt
+**/parameters/fixed_target/*.txt
+**/parameters/fixed_target/cs/fiducial_*.txt
+**/parameters/fixed_target/*.json
+**/parameters/fixed_target/*/*.map
+# idea project files
+.idea/

{mx_bluesky-0.3.1 → mx_bluesky-1.1.0}/.pre-commit-config.yaml

@@ -3,7 +3,10 @@ repos:
     rev: v4.5.0
     hooks:
       - id: check-added-large-files
+        args: ["--maxkb=500"]
       - id: check-yaml
+        args: ["--allow-multiple-documents"]
+        exclude: ^helmchart/
       - id: check-merge-conflict
       - id: end-of-file-fixer
 
@@ -29,5 +32,5 @@ repos:
     hooks:
       - id: mypy
         files: 'src/.*\.py$'
-        additional_dependencies: [types-requests, types-redis]
+        additional_dependencies: [types-requests, types-redis, pydantic]
         args: ["--ignore-missing-imports", "--no-strict-optional"]

mx_bluesky-1.1.0/.vscode/extensions.json

@@ -0,0 +1,10 @@
+{
+    "recommendations": [
+        "ms-vscode-remote.remote-containers",
+        "ms-python.python",
+        "tamasfe.even-better-toml",
+        "redhat.vscode-yaml",
+        "ryanluker.vscode-coverage-gutters",
+        "charliermarsh.Ruff"
+    ]
+}

{mx_bluesky-0.3.1 → mx_bluesky-1.1.0}/Dockerfile

@@ -1,7 +1,7 @@
 # The devcontainer should use the developer target and run as root with podman
 # or docker with user namespaces.
 ARG PYTHON_VERSION=3.11
-FROM python:${PYTHON_VERSION} as developer
+FROM python:${PYTHON_VERSION} AS developer
 
 # Add any system dependencies for the developer/build environment here
 RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -13,13 +13,13 @@ RUN python -m venv /venv
 ENV PATH=/venv/bin:$PATH
 
 # The build stage installs the context into the venv
-FROM developer as build
+FROM developer AS build
 COPY . /context
 WORKDIR /context
 RUN touch dev-requirements.txt && pip install -c dev-requirements.txt .
 
 # The runtime stage copies the built venv into a slim runtime container
-FROM python:${PYTHON_VERSION}-slim as runtime
+FROM python:${PYTHON_VERSION}-slim AS runtime
 # Add apt-get system dependecies for runtime here if needed
 COPY --from=build /venv/ /venv/
 ENV PATH=/venv/bin:$PATH

mx_bluesky-1.1.0/Dockerfile.release

@@ -0,0 +1,38 @@
+FROM python:3.11 AS build
+
+RUN pip install setuptools_scm
+
+# Copy the pyproject.toml and install dependencies for better caching when developing
+# & rerunning deployment scripts
+COPY pyproject.toml /app/hyperion/
+WORKDIR "/app/hyperion"
+RUN mkdir -p src/mx_bluesky
+
+# This enables us to cache the pip install without needing _version.py
+# see https://setuptools-scm.readthedocs.io/en/latest/usage/
+RUN SETUPTOOLS_SCM_PRETEND_VERSION_FOR_MX_BLUESKY=1.0.0 pip install \
+    --no-cache-dir --no-compile -e .
+
+# Check out and install dodal locally with no dependencies as this may be a different version to what
+# is referred to in the setup.cfg, but we don't care as it will be overridden by bind mounts in the
+# running container
+RUN mkdir ../dodal && \
+git clone https://github.com/DiamondLightSource/dodal.git ../dodal && \
+pip install --no-cache-dir --no-compile --no-deps -e ../dodal
+
+#
+# Everything above this line should be in the image cache unless pyproject.toml changes
+#
+ADD .git /app/hyperion/.git
+# Restore the repository at the current commit instead of copying, to exclude uncommitted changes
+# This is so that if you build a developer image from this dockerfile then _version.py will not
+# append the dirty workdir hash, which causes complications during deployments that mount from a clean folder.
+RUN git restore .
+
+# Regenerate _version.py with the correct version - this should run quickly since we already have our dependencies
+RUN rm src/mx_bluesky/_version.py
+RUN pip install --no-cache-dir --no-compile -e .
+
+ENTRYPOINT /app/hyperion/utility_scripts/docker/entrypoint.sh
+
+EXPOSE 5005

{mx_bluesky-0.3.1 → mx_bluesky-1.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mx-bluesky
-Version: 0.3.1
+Version: 1.1.0
 Summary: Bluesky tools for MX Beamlines at DLS
 Author-email: Dominic Oram <dominic.oram@diamond.ac.uk>
 License:                                  Apache License
@@ -208,29 +208,43 @@ License:                                  Apache License
 Project-URL: GitHub, https://github.com/DiamondLightSource/mx-bluesky
 Classifier: Development Status :: 3 - Alpha
 Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.10
+Requires-Python: >=3.11
 Description-Content-Type: text/x-rst
 License-File: LICENSE
-Requires-Dist: bluesky
-Requires-Dist: ophyd
-Requires-Dist: pyepics
+Requires-Dist: annotated_types
 Requires-Dist: caproto
+Requires-Dist: fastapi[all]
+Requires-Dist: flask-restful
+Requires-Dist: ispyb
 Requires-Dist: jupyterlab
 Requires-Dist: matplotlib
-Requires-Dist: requests
+Requires-Dist: nexgen
+Requires-Dist: numpy
 Requires-Dist: opencv-python
+Requires-Dist: opentelemetry-distro
+Requires-Dist: opentelemetry-exporter-otlp
 Requires-Dist: pydantic
+Requires-Dist: pyepics
+Requires-Dist: pyzmq
+Requires-Dist: requests
+Requires-Dist: scanspec
+Requires-Dist: scipy
 Requires-Dist: semver
-Requires-Dist: fastapi[all]
-Requires-Dist: dls-dodal==1.30.0
-Requires-Dist: blueapi==0.4.4
+Requires-Dist: matplotlib
+Requires-Dist: blueapi>=0.5.0
+Requires-Dist: daq-config-server>=0.1.1
+Requires-Dist: ophyd==1.9.0
+Requires-Dist: ophyd-async>=0.3a5
+Requires-Dist: bluesky>=1.13.0a4
+Requires-Dist: dls-dodal==1.32.0
 Provides-Extra: dev
 Requires-Dist: black; extra == "dev"
-Requires-Dist: copier; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Requires-Dist: diff-cover; extra == "dev"
 Requires-Dist: GitPython; extra == "dev"
+Requires-Dist: ipython; extra == "dev"
 Requires-Dist: mypy; extra == "dev"
 Requires-Dist: myst-parser; extra == "dev"
 Requires-Dist: pipdeptree; extra == "dev"
@@ -244,8 +258,10 @@ Requires-Dist: pytest; extra == "dev"
 Requires-Dist: ruff; extra == "dev"
 Requires-Dist: sphinx-autobuild; extra == "dev"
 Requires-Dist: sphinx-copybutton; extra == "dev"
+Requires-Dist: sphinxcontrib-plantuml; extra == "dev"
 Requires-Dist: sphinx-design; extra == "dev"
 Requires-Dist: tox-direct; extra == "dev"
+Requires-Dist: tox; extra == "dev"
 Requires-Dist: types-mock; extra == "dev"
 Requires-Dist: types-requests; extra == "dev"
 

mx_bluesky-1.1.0/codecov.yml

@@ -0,0 +1,34 @@
+comment:
+  layout: "header, diff, flags, components" # show component info in the PR comment
+
+component_management:
+  default_rules: # default rules that will be inherited by all components
+    statuses:
+      - type: project
+        target: 85%
+        threshold: 1%
+  individual_components:
+    - component_id: module_i24_ssx
+      name: i24 SSX
+      paths:
+        - src/mx_bluesky/beamlines/i24/serial/**
+        - tests/unit_tests/beamlines/i24/serial/**
+      statuses:
+        - type: project
+          target: 55%
+          threshold: 1%
+    - component_id: module_hyperion
+      name: hyperion
+      paths:
+        - src/mx_bluesky/hyperion/**
+        - tests/unit_tests//hyperion/**
+    - component_id: module_other
+      name: other
+      paths:
+        - "!src/mx_bluesky/hyperion/"
+        - "!tests/unit_tests//hyperion/**"
+        - "!src/mx_bluesky/beamlines/i24/serial/**"
+        - "!tests/unit_tests/beamlines/i24/serial/**"
+
+ignore:
+  - "**/conftest.py"

mx_bluesky-1.1.0/conftest.py

@@ -0,0 +1,33 @@
+from collections.abc import Iterator
+from os import environ, getenv
+from unittest.mock import patch
+
+import pytest
+
+environ["HYPERION_TEST_MODE"] = "true"
+
+print("Adjusting S03 EPICS environment ...")
+s03_epics_server_port = getenv("S03_EPICS_CA_SERVER_PORT")
+s03_epics_repeater_port = getenv("S03_EPICS_CA_REPEATER_PORT")
+if s03_epics_server_port:
+    environ["EPICS_CA_SERVER_PORT"] = s03_epics_server_port
+    print(f"[EPICS_CA_SERVER_PORT] = {s03_epics_server_port}")
+if s03_epics_repeater_port:
+    environ["EPICS_CA_REPEATER_PORT"] = s03_epics_repeater_port
+    print(f"[EPICS_CA_REPEATER_PORT] = {s03_epics_repeater_port}")
+
+
+def pytest_addoption(parser):
+    parser.addoption(
+        "--logging",
+        action="store_true",
+        default=False,
+        help="Log during all tests (not just those that are testing logging logic)",
+    )
+
+
+@pytest.fixture(scope="session", autouse=True)
+def default_session_fixture() -> Iterator[None]:
+    print("Patching bluesky 0MQ Publisher in __main__ for the whole session")
+    with patch("mx_bluesky.hyperion.__main__.Publisher"):
+        yield