holowizard 1.1.13__tar.gz

holowizard-1.1.13/.github/workflows/python-publish-to-production.yml

@@ -0,0 +1,64 @@
+# This workflow will upload a Python Package to PyPI when a release is created
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries
+
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+name: Upload Python Package
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+jobs:
+  release-build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Build release distributions
+        run: |
+          # NOTE: put your own distribution build steps here.
+          python -m pip install build
+          python -m build
+
+      - name: Upload distributions
+        uses: actions/upload-artifact@v4
+        with:
+          name: release-dists
+          path: dist/
+
+  pypi-publish:
+    #if: startsWith(github.ref, 'refs/tags/')
+    runs-on: ubuntu-latest
+    needs:
+      - release-build
+    permissions:
+      # IMPORTANT: this permission is mandatory for trusted publishing
+      id-token: write
+
+    # Dedicated environments with protections for publishing are strongly recommended.
+    # For more information, see: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#deployment-protection-rules
+    environment:
+      name: pypi
+      url: https://pypi.org/p/holowizard
+
+    steps:
+      - name: Retrieve release distributions
+        uses: actions/download-artifact@v4
+        with:
+          name: release-dists
+          path: dist/
+
+      - name: Publish release distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

holowizard-1.1.13/.github/workflows/python-publish-to-test.yml

@@ -0,0 +1,68 @@
+# This workflow will upload a Python Package to PyPI when a release is created
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries
+
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+name: Upload Python Package
+
+on:
+  push:
+    # Pattern matched against refs/tags
+    tags:
+      - '**'           # Push events to every tag including hierarchical tags like v1.0/beta
+
+permissions:
+  contents: read
+
+jobs:
+  release-build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Build release distributions
+        run: |
+          # NOTE: put your own distribution build steps here.
+          python -m pip install build
+          python -m build
+
+      - name: Upload distributions
+        uses: actions/upload-artifact@v4
+        with:
+          name: release-dists
+          path: dist/
+
+  pypi-publish:
+    #if: startsWith(github.ref, 'refs/tags/')
+    runs-on: ubuntu-latest
+    needs:
+      - release-build
+    permissions:
+      # IMPORTANT: this permission is mandatory for trusted publishing
+      id-token: write
+
+    # Dedicated environments with protections for publishing are strongly recommended.
+    # For more information, see: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#deployment-protection-rules
+    environment:
+      name: testpypi
+      url: https://test.pypi.org/p/holowizard
+
+    steps:
+      - name: Retrieve release distributions
+        uses: actions/download-artifact@v4
+        with:
+          name: release-dists
+          path: dist/
+
+      - name: Publish release distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          repository-url: https://test.pypi.org/legacy/

holowizard-1.1.13/.gitignore

@@ -0,0 +1,182 @@
+# Created by https://www.toptal.com/developers/gitignore/api/python
+# Edit at https://www.toptal.com/developers/gitignore?templates=python
+
+### Python ###
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+.idea/
+
+### Python Patch ###
+# Poetry local configuration file - https://python-poetry.org/docs/configuration/#local-configuration
+poetry.toml
+
+# ruff
+.ruff_cache/
+
+# LSP config files
+pyrightconfig.json
+
+# End of https://www.toptal.com/developers/gitignore/api/python
+
+*.tiff
+holowizard/core/scripts/Examples/logs/*
+
+data_generation_test
+logs

holowizard-1.1.13/.gitlab-ci.yml

@@ -0,0 +1,79 @@
+default:
+  image: python:3.11
+  cache:
+    paths:
+      - .pip-cache/
+  before_script:
+    - python --version
+    - pip install --upgrade pip
+    - pip install build twine
+
+stages:
+  - build
+  - test
+  - deploy
+
+build:
+  stage: build
+  script:
+    - python -m build
+  artifacts:
+    paths:
+      - dist/
+
+run-gpu-tests:
+  stage: test
+  image: ubuntu:22.04
+  tags:
+    - DESY-intern
+  rules:
+    # runs when you push a commit with "ci!" in the message or when a merge request is created
+    - if: '$CI_COMMIT_MESSAGE =~ /ci!/ || $CI_MERGE_REQUEST_ID'
+    # runs when you push a commit to the versions branch or main branch
+    - if: '$CI_COMMIT_BRANCH =~ /versions\/[0-9]+\.[0-9]+\.[0-9]+$/'
+    - if: $CI_COMMIT_BRANCH == "main"
+  script:
+    - apt-get update && apt-get install -y openssh-client rsync
+    - mkdir -p ~/.ssh
+    - echo "-----BEGIN OPENSSH PRIVATE KEY-----" > ~/.ssh/gitlab-ci
+    - FORMATTED_BODY=$(echo "$SSH_PRIVATE_KEY" | fold -w 70)
+    - echo "${FORMATTED_BODY[@]}" >> ~/.ssh/gitlab-ci
+    - echo "-----END OPENSSH PRIVATE KEY-----" >> ~/.ssh/gitlab-ci
+    - chmod 600 ~/.ssh/gitlab-ci
+    - export SSH_OPTS="-o StrictHostKeyChecking=no -i ~/.ssh/gitlab-ci"
+    # Generate a unique remote folder
+    - ssh $SSH_OPTS gruenjoh@max-display.desy.de "
+        mkdir ~/jgruen/ci-package/
+      "
+    - export REMOTE_CI_DIR="/home/gruenjoh/jgruen/ci-package/ci_${CI_JOB_ID}"
+    - echo "Using remote folder" + "$REMOTE_CI_DIR"
+    - rsync -az -e "ssh $SSH_OPTS" . gruenjoh@max-display.desy.de:$REMOTE_CI_DIR/
+    # Setup environment, run tests and clean up
+    - ssh $SSH_OPTS gruenjoh@max-display.desy.de "
+        export CI_JOB_ID=${CI_JOB_ID} &&
+        ml maxwell mamba &&
+        . mamba-init &&
+        mamba create -y -p /home/gruenjoh/jgruen/ci-package/ci_${CI_JOB_ID}_env python=3.11 &&
+        mamba run -p /home/gruenjoh/jgruen/ci-package/ci_${CI_JOB_ID}_env bash -c '
+          pip install -r /home/gruenjoh/jgruen/ci-package/ci_${CI_JOB_ID}/requirements.txt --no-cache-dir &&
+          pip install /home/gruenjoh/jgruen/ci-package/ci_${CI_JOB_ID} --no-cache-dir
+        ' &&
+        bash /home/gruenjoh/jgruen/ci-package/ci_${CI_JOB_ID}/tests/test_run.sh
+      "
+  after_script:
+    - apt-get update && apt-get install -y openssh-client rsync
+    - mkdir -p ~/.ssh
+    - echo "-----BEGIN OPENSSH PRIVATE KEY-----" > ~/.ssh/gitlab-ci
+    - FORMATTED_BODY=$(echo "$SSH_PRIVATE_KEY" | fold -w 70)
+    - echo "${FORMATTED_BODY[@]}" >> ~/.ssh/gitlab-ci
+    - echo "-----END OPENSSH PRIVATE KEY-----" >> ~/.ssh/gitlab-ci
+    - chmod 600 ~/.ssh/gitlab-ci
+    - export SSH_OPTS="-o StrictHostKeyChecking=no -i ~/.ssh/gitlab-ci"
+    # Cleanup
+    - ssh $SSH_OPTS gruenjoh@max-display.desy.de "
+        ml maxwell mamba &&
+        . mamba-init &&
+        mamba deactivate &&
+        mamba remove -y -n ci-env --all &&
+        rm -rf ~/jgruen/ci-package/*
+      "

holowizard-1.1.13/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Johannes Dora
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

holowizard-1.1.13/PKG-INFO

@@ -0,0 +1,261 @@
+Metadata-Version: 2.4
+Name: holowizard
+Version: 1.1.13
+Author-email: Johannes Dora <johannes.dora@desy.de>, Johannes Hagemann <johannes.hagemann@desy.de>, André Lopes <andre.lopes@hereon.de>, Johannes Hagemann <johannes.hagemann@desy.de>, Silja Flenner <silja.flenner@hereon.de>, Johannes Grün <johannes.gruen@desy.de>, Sebastian Eberle <sebastian.eberle@desy.de>, Cem Cinar <cem.cinar@tuhh.de>
+Maintainer-email: Johannes Dora <johannes.dora@desy.de>, André Lopes <andre.lopes@hereon.de>, Johannes Grün <johannes.gruen@desy.de>, Sebastian Eberle <sebastian.eberle@desy.de>, Johannes Hagemann <johannes.hagemann@desy.de>
+License-Expression: MIT
+Project-URL: Repository, https://github.com/DESY-FS-PETRA/holowizard
+Project-URL: Paper_ASRM, https://doi.org/10.1364/OE.514641
+Project-URL: Paper_Autofocus, https://doi.org/10.1364/OE.544573
+Project-URL: Zenodo, https://doi.org/10.5281/zenodo.8349364
+Keywords: near-field holography,phase retrieval,tomography
+Classifier: Programming Language :: Python
+Requires-Python: <3.12,>=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: holowizard/forge/LICENSE
+License-File: holowizard/core/LICENSE
+License-File: holowizard/pipe/LICENSE
+Requires-Dist: tqdm
+Requires-Dist: numpy==1.26.4
+Requires-Dist: torch
+Requires-Dist: torchvision
+Requires-Dist: matplotlib
+Requires-Dist: pandas
+Requires-Dist: h5py
+Requires-Dist: hdf5plugin
+Requires-Dist: selenium
+Requires-Dist: bs4
+Requires-Dist: requests
+Requires-Dist: xraylib
+Requires-Dist: cupy-cuda12x
+Requires-Dist: pyzmq
+Requires-Dist: pyqt5-tools
+Requires-Dist: scikit-learn
+Requires-Dist: scikit-image
+Requires-Dist: GitPython
+Requires-Dist: dask
+Requires-Dist: dask_jobqueue
+Requires-Dist: pytest
+Requires-Dist: bokeh
+Requires-Dist: scipy
+Requires-Dist: astra-toolbox
+Requires-Dist: pyqtgraph
+Requires-Dist: uvicorn
+Requires-Dist: hydra-core
+Requires-Dist: fastapi[standard]
+Requires-Dist: markdown
+Requires-Dist: plotly
+Dynamic: license-file
+
+ <img src="https://connect.helmholtz-imaging.de/media/solution/Holowizard_logo.png" width="200"/> 
+
+
+# Table of Contents
+1. [General Information](#general-information)
+2. [Installation](#installation)
+3. [HoloWizard Core](#holowizard-core)
+4. [HoloWizard Pipe](#holowizard-pipe)
+5. [HoloWizard Forge](#holowizard-forge)
+6. [Please Cite](#citations)
+
+## General Information
+- Repository: https://github.com/DESY-FS-PETRA/holowizard
+- Zenodo: https://doi.org/10.5281/zenodo.8349364
+
+### Further links
+https://helmholtz.software/software/holowizard \
+https://connect.helmholtz-imaging.de/solution/71 \
+https://connect.helmholtz-imaging.de/blog_gallery/blogpost/10 
+
+## Installation
+### Python Environment
+Create a new environment with python 3.11., i.e. with mamba
+```bash
+$ mamba create -p <path_to_env> python=3.11 
+```
+
+Activate enviroment
+```bash
+$ mamba activate <path_to_env>
+```
+
+### Install package
+```bash
+$ pip install holowizard
+```
+
+# HoloWizard Core
+
+To create examples, open a terminal and run
+
+```{bash}
+$ holowizard_core_create_examples <directory>
+```
+
+# HoloWizard Pipe
+
+## Setting Up an Instance
+
+We provide a CLI command to initialize everything:
+
+```bash
+holopipe beamtime.name=YOUR_BEAMTIME_NAME beamtime.year=YEAR_OF_BEAMTIME
+```
+
+This command sets up the pipeline. You can override any other configuration value using Hydra’s override syntax:  
+👉 [Hydra Override Syntax Documentation](https://hydra.cc/docs/advanced/override_grammar/basic/)
+
+If the startup is successful, you’ll see output like:
+
+```
+INFO:     Uvicorn running on http://MY_IP_ADDRESS:MY_PORT (Press CTRL+C to quit)
+```
+
+Click the address to open a browser window showing that `holopipe` is running.  
+Visit: `http://MY_IP_ADDRESS:MY_PORT/dashboard` for useful runtime information.
+
+
+---
+
+## Usage
+
+### Add a scan with default parameters. 
+
+You can submit scans using a simple `curl` POST request:
+
+```bash
+curl -X POST \
+     -H "Content-Type: application/json" \
+     -d '{ "a0": 1.0,
+           "scan-name": "nano145014_s7_1_tum",
+           "holder": 220,
+           "base_dir": "holopipe",
+           "reconstruction": "wire",
+           "find_focus": "wire",
+           "energy": 17.0
+         }' \
+     http://MY_IP_ADDRESS:MY_PORT/api/submit_scan
+```
+
+If you are on the same machine as the server is running you can use the python script:
+```bash
+recontruct-scan --help # will tell you all important parameters
+```
+
+#### Required Parameters
+
+- `name`: Folder name of the current scan  
+- `holder`: Height of the holder  
+- `energy`: Scan energy in keV  
+
+#### Optional Parameters
+
+- `a0`: Optional numeric parameter; if not provided, it will be computed automatically  
+- `reconstruction`: Instruction set for reconstruction — `wire` (default) or `spider`  
+- `find_focus`: Instruction set to find focus — `wire` (default) or `spider`  
+- `base_dir`: Root directory for output files (default: `holopipe`)
+
+### Parameter Optimization
+
+To performe parameter optimization go to `http://MY_IP_ADDRESS:MY_PORT/parameter`. Here you can set all parameter for all stages. To test them click `Reconstruct`. If the parameters work well, you can save them to the current beamtime folder using the `Save as` at the lower left. 
+If you want to reconstruct the whole scan you can click `Submit All` after chosing the Options. If you select `Custom` it will take the parameters from the left. 
+
+### Other changes during beamtime
+
+If you change the detector or anything else like removing tasks adapt parameters the full config files are located in the `beamtime/processed/holowizard_config/`folder. Changes here will reflect onto future curl requests!
+
+# HoloWizard Forge
+
+This framework can be used to generate large datasets of simulated holograms of randomly sampled objects. 
+
+## Create New Dataset
+
+Open a terminal and create a new config file with 
+
+```{bash}
+$ holowziard_forge_create_testconfig <args>
+```
+
+| Argument        | Description                                               | Position |
+|-----------------|-----------------------------------------------------------|----------|
+| `name`          | Name of the new config file (without file name extension) | 1        |
+| `--output_dir`  | Output directory                                          | optional |
+| `--override`    | Overrides existing configuration file                     | optional |
+
+
+The config file can then be customized and used to create a new dataset with
+
+```{bash}
+$ holowizard_forge_generate_data <args>
+```
+
+| Argument      | Description                                          | Position |
+|---------------|------------------------------------------------------|----------|
+| `config`      | Path to the custom configuration file.               | 1        |
+| `output_dir`  | Output directory where the generated data is stored. | 2        |
+| `num_samples` | Number of data samples that should be generated.     | 3        |
+| `--override`  | Override the output folder if it already exists.     | optional |
+
+## Output Structure
+```
+output/
+└── train.hdf5
+└── train.json
+```
+
+The file train.hdf5 contains the training data
+The file `train.json` contains the config parameters which have been used for the training data creation.
+
+## Developer Info
+
+### Add new Parameters
+To add a new parameter, add it to the default configuration `holowizard/forge/configs/default.json`.
+
+# Citations
+### Artifact-suppressing reconstruction method:
+- URL: https://opg.optica.org/oe/fulltext.cfm?uri=oe-32-7-10801&id=547807###
+- DOI: 10.1364/OE.514641
+
+```{bibtex}
+@article{Dora:24,
+author = {Johannes Dora and Martin M\"{o}ddel and Silja Flenner and Christian G. Schroer and Tobias Knopp and Johannes Hagemann},
+journal = {Opt. Express},
+keywords = {Free electron lasers; Holographic microscopy; Imaging techniques; Phase shift; X-ray imaging; Zone plates},
+number = {7},
+pages = {10801--10828},
+publisher = {Optica Publishing Group},
+title = {{Artifact-suppressing reconstruction of strongly interacting objects in X-ray near-field holography without a spatial support constraint}},
+volume = {32},
+month = {Mar},
+year = {2024},
+url = {https://opg.optica.org/oe/abstract.cfm?URI=oe-32-7-10801},
+doi = {10.1364/OE.514641},
+abstract = {The phase problem is a well known ill-posed reconstruction problem of coherent lens-less microscopic imaging, where only the squared magnitude of a complex wavefront is measured by a detector while the phase information of the wave field is lost. To retrieve the lost information, common algorithms rely either on multiple data acquisitions under varying measurement conditions or on the application of strong constraints such as a spatial support. In X-ray near-field holography, however, these methods are rendered impractical in the setting of time sensitive in situ and operando measurements. In this paper, we will forego the spatial support constraint and propose a projected gradient descent (PGD) based reconstruction scheme in combination with proper preprocessing and regularization that significantly reduces artifacts for refractive reconstructions from only a single acquired hologram without a spatial support constraint. We demonstrate the feasibility and robustness of our approach on different data sets obtained at the nano imaging endstation of P05 at PETRA III (DESY, Hamburg) operated by Helmholtz-Zentrum Hereon.},
+}
+```
+
+### Model-based autofocus:
+- URL: https://opg.optica.org/oe/abstract.cfm?doi=10.1364/OE.544573
+- DOI: 10.1364/OE.544573
+
+```{bibtex}
+@article{Dora:25,
+author = {Johannes Dora and Martin M\"{o}ddel and Silja Flenner and Jan Reimers and Berit Zeller-Plumhoff and Christian G. Schroer and Tobias Knopp and Johannes Hagemann},
+journal = {Opt. Express},
+keywords = {Image analysis; Image metrics; Imaging systems; Phase retrieval; X-ray imaging; Zone plates},
+number = {4},
+pages = {6641--6657},
+publisher = {Optica Publishing Group},
+title = {Model-based autofocus for near-field phase retrieval},
+volume = {33},
+month = {Feb},
+year = {2025},
+url = {https://opg.optica.org/oe/abstract.cfm?URI=oe-33-4-6641},
+doi = {10.1364/OE.544573},
+}
+```
+
+### Python Repository on Zenodo
+- URL: https://zenodo.org/records/14024980
+- DOI: 10.5281/zenodo.8349364