maybempi 0.1.0__tar.gz

maybempi-0.1.0/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Max Lindqvist
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

maybempi-0.1.0/PKG-INFO

@@ -0,0 +1,162 @@
+Metadata-Version: 2.4
+Name: maybempi
+Version: 0.1.0
+Summary: Use MPI only when launched under MPI, and a serial stand-in for mpi4py otherwise.
+Author: Max
+License: MIT License
+        
+        Copyright (c) 2026 Max Lindqvist
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Changelog, https://github.com/max-models/maybempi/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://max-models.github.io/maybempi/
+Project-URL: Source, https://github.com/max-models/maybempi
+Keywords: hpc,mpi,mpi4py,parallel,serial,slurm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: System :: Distributed Computing
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+Provides-Extra: dev
+Requires-Dist: jupyterlab; extra == "dev"
+Requires-Dist: maybempi[docs,test]; extra == "dev"
+Requires-Dist: pre-commit; extra == "dev"
+Requires-Dist: pyproject-fmt; extra == "dev"
+Requires-Dist: pyright; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: griffe; extra == "docs"
+Requires-Dist: ipykernel; extra == "docs"
+Requires-Dist: nbclient>=0.10; extra == "docs"
+Requires-Dist: nbformat>=5.10; extra == "docs"
+Requires-Dist: numpy; extra == "docs"
+Provides-Extra: mpi
+Requires-Dist: mpi4py; extra == "mpi"
+Provides-Extra: test
+Requires-Dist: numpy; extra == "test"
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-cov; extra == "test"
+Dynamic: license-file
+
+# maybempi
+
+
+<!-- README.md is generated from README.qmd: edit the .qmd and run `make readme`. -->
+
+[![Tests](https://github.com/max-models/maybempi/actions/workflows/test_pytest.yml/badge.svg)](https://github.com/max-models/maybempi/actions/workflows/test_pytest.yml)
+[![Static
+analysis](https://github.com/max-models/maybempi/actions/workflows/static_analysis.yml/badge.svg)](https://github.com/max-models/maybempi/actions/workflows/static_analysis.yml)
+[![Docs](https://github.com/max-models/maybempi/actions/workflows/docs.yml/badge.svg)](https://max-models.github.io/maybempi/)
+[![codecov](https://codecov.io/gh/max-models/maybempi/branch/main/graph/badge.svg)](https://codecov.io/gh/max-models/maybempi)
+[![PyPI](https://img.shields.io/pypi/v/maybempi.png)](https://pypi.org/project/maybempi/)
+[![Python](https://img.shields.io/pypi/pyversions/maybempi.png)](https://pypi.org/project/maybempi/)
+
+Use MPI only when the process was launched under MPI, and a serial
+stand-in for `mpi4py` otherwise. The same code runs under
+`mpirun`/`srun` and as a plain `python script.py`, and the serial run
+never starts MPI.
+
+Documentation: <https://max-models.github.io/maybempi/>
+
+``` python
+from maybempi import MPI  # instead of: from mpi4py import MPI
+
+# mpi4py.MPI under mpirun/mpiexec/srun, a serial stand-in otherwise
+comm = MPI.COMM_WORLD
+total = comm.allreduce(local_total, op=MPI.SUM)  # local_total in a serial run
+```
+
+## Why
+
+Importing `mpi4py.MPI` calls `MPI_Init`, which costs close to a second
+and makes every collective cost something, even on one process. maybempi
+decides from the environment variables that MPI launchers export (Open
+MPI, MPICH, Intel MPI, PMIx/`srun`, MVAPICH2, Cray), without importing
+mpi4py. In a serial run it returns a stand-in with the module constants
+and communicator methods of mpi4py, which give the results MPI gives on
+one process: `allreduce(x)` is `x`, `gather(x)` is `[x]`,
+`Allreduce(send, recv)` copies. Methods it does not implement raise
+`AttributeError` instead of silently doing nothing.
+
+- `MAYBEMPI=0`/`1` overrides the decision.
+- `maybempi.is_serial(MPI)` tells the stand-in from mpi4py.
+- `maybempi.local_rank()` gives the node-local rank before `MPI_Init`,
+  e.g. to pick a GPU.
+- The `maybempi` command prints what it decides, and why:
+  `mpirun -n 2 maybempi --init`.
+
+maybempi is pure Python and has no dependencies.
+
+## Install
+
+``` bash
+pip install maybempi            # serial runs only
+pip install "maybempi[mpi]"     # with mpi4py, for MPI runs
+```
+
+## Development
+
+With [uv](https://docs.astral.sh/uv/):
+
+``` bash
+make install    # uv sync --extra dev, plus the pre-commit hooks
+make lint       # ruff check, ruff format --check, pyright
+make test       # pytest with coverage
+mpiexec -n 2 .venv/bin/python -m pytest src/maybempi/tests/mpi   # tests under a real launcher
+```
+
+Commit messages follow [Conventional
+Commits](https://www.conventionalcommits.org/); see
+[CONTRIBUTING.md](CONTRIBUTING.md).
+
+The documentation in `docs/` is an [Astro](https://astro.build/) +
+[Starlight](https://starlight.astro.build/) site with the executed
+notebooks from `tutorials/` and an API reference generated from the
+docstrings. It needs Node 22 or newer:
+
+``` bash
+make docs-install     # npm packages and the Python docs extra
+make docs-notebooks   # execute tutorials/*.ipynb and convert them to pages
+make docs-dev         # live preview at http://localhost:4321/maybempi/
+```
+
+`README.md` is rendered from `README.qmd` with
+[Quarto](https://quarto.org/): `make readme`.
+
+## Releases
+
+Before merging a release to `main`, update the version in
+`pyproject.toml`, `src/maybempi/__init__.py` and `CITATION.cff`
+(including its release date), and add the release notes to
+`CHANGELOG.md`. The push to `main` creates a GitHub release with a
+`vX.Y.Z` tag and publishes the package to PyPI with trusted publishing
+(OIDC). See the [publishing
+guide](https://max-models.github.io/maybempi/development/publishing/).

maybempi-0.1.0/README.md

@@ -0,0 +1,94 @@
+# maybempi
+
+
+<!-- README.md is generated from README.qmd: edit the .qmd and run `make readme`. -->
+
+[![Tests](https://github.com/max-models/maybempi/actions/workflows/test_pytest.yml/badge.svg)](https://github.com/max-models/maybempi/actions/workflows/test_pytest.yml)
+[![Static
+analysis](https://github.com/max-models/maybempi/actions/workflows/static_analysis.yml/badge.svg)](https://github.com/max-models/maybempi/actions/workflows/static_analysis.yml)
+[![Docs](https://github.com/max-models/maybempi/actions/workflows/docs.yml/badge.svg)](https://max-models.github.io/maybempi/)
+[![codecov](https://codecov.io/gh/max-models/maybempi/branch/main/graph/badge.svg)](https://codecov.io/gh/max-models/maybempi)
+[![PyPI](https://img.shields.io/pypi/v/maybempi.png)](https://pypi.org/project/maybempi/)
+[![Python](https://img.shields.io/pypi/pyversions/maybempi.png)](https://pypi.org/project/maybempi/)
+
+Use MPI only when the process was launched under MPI, and a serial
+stand-in for `mpi4py` otherwise. The same code runs under
+`mpirun`/`srun` and as a plain `python script.py`, and the serial run
+never starts MPI.
+
+Documentation: <https://max-models.github.io/maybempi/>
+
+``` python
+from maybempi import MPI  # instead of: from mpi4py import MPI
+
+# mpi4py.MPI under mpirun/mpiexec/srun, a serial stand-in otherwise
+comm = MPI.COMM_WORLD
+total = comm.allreduce(local_total, op=MPI.SUM)  # local_total in a serial run
+```
+
+## Why
+
+Importing `mpi4py.MPI` calls `MPI_Init`, which costs close to a second
+and makes every collective cost something, even on one process. maybempi
+decides from the environment variables that MPI launchers export (Open
+MPI, MPICH, Intel MPI, PMIx/`srun`, MVAPICH2, Cray), without importing
+mpi4py. In a serial run it returns a stand-in with the module constants
+and communicator methods of mpi4py, which give the results MPI gives on
+one process: `allreduce(x)` is `x`, `gather(x)` is `[x]`,
+`Allreduce(send, recv)` copies. Methods it does not implement raise
+`AttributeError` instead of silently doing nothing.
+
+- `MAYBEMPI=0`/`1` overrides the decision.
+- `maybempi.is_serial(MPI)` tells the stand-in from mpi4py.
+- `maybempi.local_rank()` gives the node-local rank before `MPI_Init`,
+  e.g. to pick a GPU.
+- The `maybempi` command prints what it decides, and why:
+  `mpirun -n 2 maybempi --init`.
+
+maybempi is pure Python and has no dependencies.
+
+## Install
+
+``` bash
+pip install maybempi            # serial runs only
+pip install "maybempi[mpi]"     # with mpi4py, for MPI runs
+```
+
+## Development
+
+With [uv](https://docs.astral.sh/uv/):
+
+``` bash
+make install    # uv sync --extra dev, plus the pre-commit hooks
+make lint       # ruff check, ruff format --check, pyright
+make test       # pytest with coverage
+mpiexec -n 2 .venv/bin/python -m pytest src/maybempi/tests/mpi   # tests under a real launcher
+```
+
+Commit messages follow [Conventional
+Commits](https://www.conventionalcommits.org/); see
+[CONTRIBUTING.md](CONTRIBUTING.md).
+
+The documentation in `docs/` is an [Astro](https://astro.build/) +
+[Starlight](https://starlight.astro.build/) site with the executed
+notebooks from `tutorials/` and an API reference generated from the
+docstrings. It needs Node 22 or newer:
+
+``` bash
+make docs-install     # npm packages and the Python docs extra
+make docs-notebooks   # execute tutorials/*.ipynb and convert them to pages
+make docs-dev         # live preview at http://localhost:4321/maybempi/
+```
+
+`README.md` is rendered from `README.qmd` with
+[Quarto](https://quarto.org/): `make readme`.
+
+## Releases
+
+Before merging a release to `main`, update the version in
+`pyproject.toml`, `src/maybempi/__init__.py` and `CITATION.cff`
+(including its release date), and add the release notes to
+`CHANGELOG.md`. The push to `main` creates a GitHub release with a
+`vX.Y.Z` tag and publishes the package to PyPI with trusted publishing
+(OIDC). See the [publishing
+guide](https://max-models.github.io/maybempi/development/publishing/).

maybempi-0.1.0/pyproject.toml

@@ -0,0 +1,82 @@
+[build-system]
+build-backend = "setuptools.build_meta"
+requires = [ "setuptools", "wheel" ]
+
+[project]
+name = "maybempi"
+version = "0.1.0"
+description = "Use MPI only when launched under MPI, and a serial stand-in for mpi4py otherwise."
+readme = "README.md"
+keywords = [ "hpc", "mpi", "mpi4py", "parallel", "serial", "slurm" ]
+license = { file = "LICENSE.txt" }
+authors = [ { name = "Max" } ]
+requires-python = ">=3.10"
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Science/Research",
+  "License :: OSI Approved :: MIT License",
+  "Operating System :: OS Independent",
+  "Programming Language :: Python :: 3 :: Only",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Programming Language :: Python :: 3.14",
+  "Topic :: Scientific/Engineering",
+  "Topic :: System :: Distributed Computing",
+  "Typing :: Typed",
+]
+dependencies = []
+optional-dependencies.dev = [
+  "jupyterlab",
+  "maybempi[docs,test]",
+  "pre-commit",
+  "pyproject-fmt",
+  "pyright",
+  "ruff",
+]
+optional-dependencies.docs = [
+  "griffe",
+  "ipykernel",
+  "nbclient>=0.10",
+  "nbformat>=5.10",
+  "numpy",
+]
+optional-dependencies.mpi = [ "mpi4py" ]
+optional-dependencies.test = [ "numpy", "pytest", "pytest-cov" ]
+urls.Changelog = "https://github.com/max-models/maybempi/blob/main/CHANGELOG.md"
+urls.Documentation = "https://max-models.github.io/maybempi/"
+urls.Source = "https://github.com/max-models/maybempi"
+scripts.maybempi = "maybempi.cli:main"
+
+[tool.setuptools]
+package-data."*" = [ "py.typed" ]
+packages.find.where = [ "src" ]
+
+[tool.ruff]
+target-version = "py310"
+line-length = 88
+extend-include = [ "*.ipynb" ]
+# pycodestyle, pyflakes, isort, pyupgrade, bugbear and pydocstyle
+lint.select = [ "B", "D", "E", "F", "I", "UP", "W" ]
+lint.ignore = [ "E501" ]  # line length is handled by the formatter
+lint.per-file-ignores."**/tests/**" = [ "D" ]
+lint.per-file-ignores."*.ipynb" = [ "D" ]
+lint.per-file-ignores."docs/tools/*" = [ "D" ]
+lint.pydocstyle.convention = "google"
+
+[tool.pyright]
+include = [ "src" ]
+exclude = [ "**/tests" ]
+pythonVersion = "3.10"
+typeCheckingMode = "standard"
+
+[tool.pytest]
+ini_options.pythonpath = [ "src" ]
+ini_options.testpaths = [ "src" ]
+ini_options.addopts = "-ra"
+
+[tool.coverage]
+run.omit = [ "*/tests/*" ]
+run.source = [ "maybempi" ]
+report.show_missing = true

maybempi-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

maybempi-0.1.0/src/maybempi/__init__.py

@@ -0,0 +1,66 @@
+"""Use MPI only when the process was launched under MPI, and a serial stand-in otherwise.
+
+Importing ``mpi4py.MPI`` starts MPI, which costs close to a second and makes every
+collective cost something, even on one process. maybempi decides from the
+environment the launcher sets up, without importing mpi4py, and returns either
+``mpi4py.MPI`` or a serial stand-in with the same interface::
+
+    import maybempi
+
+    MPI = maybempi.get_mpi()  # mpi4py.MPI under mpirun/srun, else the stand-in
+    comm = MPI.COMM_WORLD
+    total = comm.allreduce(local_total, op=MPI.SUM)
+
+``from maybempi import MPI`` is the drop-in replacement for ``from mpi4py import
+MPI``: the name resolves to ``get_mpi()`` when it is first imported.
+
+:mod:`maybempi.launch` holds the detection, :mod:`maybempi.serial` the stand-in.
+"""
+
+from typing import Any
+
+from maybempi.launch import (
+    LAUNCHER_VARIABLES,
+    LOCAL_RANK_VARIABLES,
+    OVERRIDE_VARIABLE,
+    get_mpi,
+    is_serial,
+    launched_under_mpi,
+    launcher_variable,
+    local_rank,
+)
+from maybempi.serial import (
+    SerialComm,
+    SerialMPI,
+    SerialPrequest,
+    SerialRequest,
+    SerialStatus,
+    set_copy_hook,
+)
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "LAUNCHER_VARIABLES",
+    "LOCAL_RANK_VARIABLES",
+    "OVERRIDE_VARIABLE",
+    "SerialComm",
+    "SerialMPI",
+    "SerialPrequest",
+    "SerialRequest",
+    "SerialStatus",
+    "__version__",
+    "get_mpi",
+    "is_serial",
+    "launched_under_mpi",
+    "launcher_variable",
+    "local_rank",
+    "set_copy_hook",
+]
+
+
+def __getattr__(name: str) -> Any:
+    """Resolve ``maybempi.MPI`` (and ``from maybempi import MPI``) to :func:`get_mpi`."""
+    if name == "MPI":
+        return get_mpi()
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

maybempi-0.1.0/src/maybempi/__main__.py

@@ -0,0 +1,5 @@
+"""``python -m maybempi``: the same as the ``maybempi`` command."""
+
+from maybempi.cli import main
+
+main()

maybempi-0.1.0/src/maybempi/cli.py

@@ -0,0 +1,142 @@
+"""The ``maybempi`` command: show what maybempi decides in this environment.
+
+Run it the way the application is run, to see whether it would use MPI::
+
+    maybempi                      # serial: one table
+    mpirun -n 2 maybempi          # one table per rank (MPI is not started)
+    mpirun -n 2 maybempi --init   # start MPI: rank 0 prints one table for all ranks
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import os
+import socket
+from collections.abc import Sequence
+from typing import Any
+
+from maybempi import __version__
+from maybempi.launch import (
+    OVERRIDE_VARIABLE,
+    get_mpi,
+    is_serial,
+    launched_under_mpi,
+    launcher_variable,
+    local_rank,
+)
+
+# columns of the per-rank table, in this order; other items become columns only
+# if they differ between ranks
+_RANK_COLUMNS = ("rank", "host", "local rank", "launcher variable")
+
+
+def info(init: bool = False) -> dict[str, Any]:
+    """Collect what maybempi decides in this process, and why.
+
+    Args:
+        init: Also call :func:`~maybempi.launch.get_mpi` (which starts MPI under a
+            launcher) and add the rank and size of ``COMM_WORLD``.
+
+    Returns:
+        The items of the report, by name.
+    """
+    items: dict[str, Any] = {
+        "maybempi": __version__,
+        "host": socket.gethostname(),
+        "launched under MPI": launched_under_mpi(),
+        "launcher variable": launcher_variable() or "-",
+        f"{OVERRIDE_VARIABLE} override": os.environ.get(OVERRIDE_VARIABLE, "-"),
+        "local rank": local_rank(),
+        "mpi4py installed": importlib.util.find_spec("mpi4py") is not None,
+    }
+    if init:
+        mpi = get_mpi()
+        comm = mpi.COMM_WORLD
+        items["MPI"] = "serial stand-in" if is_serial(mpi) else "mpi4py"
+        items["rank"] = comm.Get_rank()
+        items["size"] = comm.Get_size()
+    return items
+
+
+def _table(header: Sequence[str], rows: Sequence[Sequence[Any]]) -> str:
+    """Format `rows` as a plain-text table with aligned columns."""
+    cells = [[str(value) for value in row] for row in rows]
+    widths = [
+        max(len(str(h)), *(len(row[i]) for row in cells)) for i, h in enumerate(header)
+    ]
+
+    def line(values: Sequence[str]) -> str:
+        return "  ".join(
+            v.ljust(w) for v, w in zip(values, widths, strict=True)
+        ).rstrip()
+
+    return "\n".join(
+        [line(header), line(["-" * w for w in widths]), *(line(row) for row in cells)]
+    )
+
+
+def format_report(items: dict[str, Any]) -> str:
+    """Format the report of one process as a two-column table.
+
+    Args:
+        items: What :func:`info` returned.
+
+    Returns:
+        The table, one item per line.
+    """
+    items = dict(items)
+    if "rank" in items:
+        items["rank"] = f"{items['rank']} of {items.pop('size')}"
+    return _table(("item", "value"), list(items.items()))
+
+
+def format_reports(reports: Sequence[dict[str, Any]]) -> str:
+    """Format the reports of all ranks: the common items, then one row per rank.
+
+    Args:
+        reports: What :func:`info` returned on each rank, in rank order.
+
+    Returns:
+        A table of the items that are the same on every rank, and a table with one
+        row per rank for the others (rank, host, local rank, launcher variable and
+        whatever else differs).
+    """
+    names = list(reports[0])
+    differs = {n for n in names if any(r.get(n) != reports[0].get(n) for r in reports)}
+    columns = [n for n in _RANK_COLUMNS if n in names]
+    columns += [n for n in names if n in differs and n not in columns]
+    common = [(n, reports[0][n]) for n in names if n not in columns]
+    rows = [[report.get(n, "-") for n in columns] for report in reports]
+    return f"{_table(('item', 'value'), common)}\n\n{_table(columns, rows)}"
+
+
+def report(init: bool = False) -> str:
+    """Return the report of this process as a table (see :func:`info`)."""
+    return format_report(info(init))
+
+
+def main(argv: list[str] | None = None) -> None:
+    """Print the report; the entry point of the ``maybempi`` command."""
+    parser = argparse.ArgumentParser(
+        prog="maybempi",
+        description="Show whether this process would use MPI, and why.",
+    )
+    parser.add_argument(
+        "--init",
+        action="store_true",
+        help="also start MPI (under a launcher); rank 0 then prints one table for all ranks",
+    )
+    parser.add_argument("--version", action="version", version=__version__)
+    args = parser.parse_args(argv)
+    items = info(init=args.init)
+    if args.init and items["size"] > 1:
+        reports = get_mpi().COMM_WORLD.gather(items, root=0)
+        if items["rank"] == 0:
+            print(format_reports(reports), flush=True)
+        return
+    print(format_report(items), flush=True)
+
+
+if __name__ == "__main__":
+    main()