numba-mpi 0.40__py3-none-any.whl → 0.41__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- numba_mpi-0.41.dist-info/METADATA +172 -0
- {numba_mpi-0.40.dist-info → numba_mpi-0.41.dist-info}/RECORD +5 -5
- {numba_mpi-0.40.dist-info → numba_mpi-0.41.dist-info}/WHEEL +1 -1
- numba_mpi-0.40.dist-info/METADATA +0 -66
- {numba_mpi-0.40.dist-info → numba_mpi-0.41.dist-info}/LICENSE +0 -0
- {numba_mpi-0.40.dist-info → numba_mpi-0.41.dist-info}/top_level.txt +0 -0
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: numba-mpi
|
|
3
|
+
Version: 0.41
|
|
4
|
+
Summary: Numba @njittable MPI wrappers tested on Linux, macOS and Windows
|
|
5
|
+
Home-page: https://github.com/numba-mpi/numba-mpi
|
|
6
|
+
Author: https://github.com/numba-mpi/numba-mpi/graphs/contributors
|
|
7
|
+
License: GPL v3
|
|
8
|
+
Project-URL: Tracker, https://github.com/numba-mpi/numba-mpi/issues
|
|
9
|
+
Project-URL: Documentation, https://numba-mpi.github.io/numba-mpi
|
|
10
|
+
Project-URL: Source, https://github.com/numba-mpi/numba-mpi
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Requires-Dist: numba
|
|
14
|
+
Requires-Dist: numpy
|
|
15
|
+
Requires-Dist: mpi4py
|
|
16
|
+
Requires-Dist: psutil
|
|
17
|
+
Provides-Extra: tests
|
|
18
|
+
Requires-Dist: pytest <8.0.0 ; extra == 'tests'
|
|
19
|
+
|
|
20
|
+
# <img src="https://raw.githubusercontent.com/numba-mpi/numba-mpi/main/.github/numba_mpi_logo.svg" style="height:50pt" alt="numba-mpi logo"> numba-mpi
|
|
21
|
+
|
|
22
|
+
[](https://www.python.org/)
|
|
23
|
+
[](https://numba.pydata.org)
|
|
24
|
+
[](https://en.wikipedia.org/wiki/Linux)
|
|
25
|
+
[](https://en.wikipedia.org/wiki/macOS)
|
|
26
|
+
[](https://en.wikipedia.org/wiki/Windows)
|
|
27
|
+
[](https://github.com/numba-mpi/numba-mpi/actions/workflows/tests+pypi.yml)
|
|
28
|
+
[](https://GitHub.com/numba-mpi/numba-mpi/graphs/commit-activity)
|
|
29
|
+
[](https://www.gnu.org/licenses/gpl-3.0.html)
|
|
30
|
+
[](https://pypi.org/project/numba-mpi)
|
|
31
|
+
[](https://anaconda.org/conda-forge/numba-mpi)
|
|
32
|
+
[](https://aur.archlinux.org/packages/python-numba-mpi)
|
|
33
|
+
[](https://zenodo.org/badge/latestdoi/316911228)
|
|
34
|
+
|
|
35
|
+
### Overview
|
|
36
|
+
numba-mpi provides Python wrappers to the C MPI API callable from within [Numba JIT-compiled code](https://numba.readthedocs.io/en/stable/user/jit.html) (@njit mode).
|
|
37
|
+
|
|
38
|
+
Support is provided for a subset of MPI routines covering: `size`/`rank`, `send`/`recv`, `allreduce`, `bcast`, `scatter`/`gather` & `allgather`, `barrier`, `wtime`
|
|
39
|
+
and basic asynchronous communication with `isend`/`irecv` (only for contiguous arrays); for request handling including `wait`/`waitall`/`waitany` and `test`/`testall`/`testany`.
|
|
40
|
+
|
|
41
|
+
The API uses NumPy and supports both numeric and character datatypes (e.g., `broadcast`).
|
|
42
|
+
Auto-generated docstring-based API docs are published on the web: https://numba-mpi.github.io/numba-mpi
|
|
43
|
+
|
|
44
|
+
Packages can be obtained from
|
|
45
|
+
[PyPI](https://pypi.org/project/numba-mpi),
|
|
46
|
+
[Conda Forge](https://anaconda.org/conda-forge/numba-mpi),
|
|
47
|
+
[Arch Linux](https://aur.archlinux.org/packages/python-numba-mpi)
|
|
48
|
+
or by invoking `pip install git+https://github.com/numba-mpi/numba-mpi.git`.
|
|
49
|
+
|
|
50
|
+
numba-mpi is a pure-Python package.
|
|
51
|
+
The codebase includes a test suite used through the GitHub Actions workflows ([thanks to mpi4py's setup-mpi](https://github.com/mpi4py/setup-mpi)!)
|
|
52
|
+
for automated testing on: Linux ([MPICH](https://www.mpich.org/), [OpenMPI](https://www.open-mpi.org/doc/)
|
|
53
|
+
& [Intel MPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html)),
|
|
54
|
+
macOS ([MPICH](https://www.mpich.org/) & [OpenMPI](https://www.open-mpi.org/doc/)) and
|
|
55
|
+
Windows ([MS MPI](https://docs.microsoft.com/en-us/message-passing-interface/microsoft-mpi)).
|
|
56
|
+
|
|
57
|
+
Features that are not implemented yet include (help welcome!):
|
|
58
|
+
- support for non-default communicators
|
|
59
|
+
- support for `MPI_IN_PLACE` in `[all]gather`/`scatter` and `allreduce`
|
|
60
|
+
- support for `MPI_Type_create_struct` (Numpy structured arrays)
|
|
61
|
+
- ...
|
|
62
|
+
|
|
63
|
+
### Hello world send/recv example:
|
|
64
|
+
```python
|
|
65
|
+
import numba, numba_mpi, numpy
|
|
66
|
+
|
|
67
|
+
@numba.njit()
|
|
68
|
+
def hello():
|
|
69
|
+
src = numpy.array([1., 2., 3., 4., 5.])
|
|
70
|
+
dst_tst = numpy.empty_like(src)
|
|
71
|
+
|
|
72
|
+
if numba_mpi.rank() == 0:
|
|
73
|
+
numba_mpi.send(src, dest=1, tag=11)
|
|
74
|
+
elif numba_mpi.rank() == 1:
|
|
75
|
+
numba_mpi.recv(dst_tst, source=0, tag=11)
|
|
76
|
+
|
|
77
|
+
hello()
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### Example comparing numba-mpi vs. mpi4py performance:
|
|
81
|
+
|
|
82
|
+
The example below compares Numba + mpi4py vs. Numba + numba-mpi performance.
|
|
83
|
+
The sample code estimates $\pi$ by integration of $4/(1+x^2)$ between 0 and 1
|
|
84
|
+
dividing the workload into `n_intervals` handled by separate MPI processes
|
|
85
|
+
and then obtaining a sum using `allreduce`.
|
|
86
|
+
The computation is carried out in a JIT-compiled function and is repeated
|
|
87
|
+
`N_TIMES`, the repetitions and the MPI-handled reduction are done outside or
|
|
88
|
+
inside of the JIT-compiled block for mpi4py and numba-mpi, respectively.
|
|
89
|
+
Timing is repeated `N_REPEAT` times and the minimum time is reported.
|
|
90
|
+
The generated plot shown below depicts the speedup obtained by replacing mpi4py
|
|
91
|
+
with numba_mpi as a function of `n_intervals` - the more often communication
|
|
92
|
+
is needed (smaller `n_intervals`), the larger the expected speedup.
|
|
93
|
+
|
|
94
|
+
```python
|
|
95
|
+
import timeit, mpi4py, numba, numpy as np, numba_mpi
|
|
96
|
+
|
|
97
|
+
N_TIMES = 10000
|
|
98
|
+
N_REPEAT = 10
|
|
99
|
+
RTOL = 1e-3
|
|
100
|
+
|
|
101
|
+
@numba.njit
|
|
102
|
+
def get_pi_part(out, n_intervals, rank, size):
|
|
103
|
+
h = 1 / n_intervals
|
|
104
|
+
partial_sum = 0.0
|
|
105
|
+
for i in range(rank + 1, n_intervals, size):
|
|
106
|
+
x = h * (i - 0.5)
|
|
107
|
+
partial_sum += 4 / (1 + x**2)
|
|
108
|
+
out[0] = h * partial_sum
|
|
109
|
+
|
|
110
|
+
@numba.njit
|
|
111
|
+
def pi_numba_mpi(n_intervals):
|
|
112
|
+
pi = np.array([0.])
|
|
113
|
+
part = np.empty_like(pi)
|
|
114
|
+
for _ in range(N_TIMES):
|
|
115
|
+
get_pi_part(part, n_intervals, numba_mpi.rank(), numba_mpi.size())
|
|
116
|
+
numba_mpi.allreduce(part, pi, numba_mpi.Operator.SUM)
|
|
117
|
+
assert abs(pi[0] - np.pi) / np.pi < RTOL
|
|
118
|
+
|
|
119
|
+
def pi_mpi4py(n_intervals):
|
|
120
|
+
pi = np.array([0.])
|
|
121
|
+
part = np.empty_like(pi)
|
|
122
|
+
for _ in range(N_TIMES):
|
|
123
|
+
get_pi_part(part, n_intervals, mpi4py.MPI.COMM_WORLD.rank, mpi4py.MPI.COMM_WORLD.size)
|
|
124
|
+
mpi4py.MPI.COMM_WORLD.Allreduce(part, (pi, mpi4py.MPI.DOUBLE), op=mpi4py.MPI.SUM)
|
|
125
|
+
assert abs(pi[0] - np.pi) / np.pi < RTOL
|
|
126
|
+
|
|
127
|
+
plot_x = [1000 * k for k in range(1, 11)]
|
|
128
|
+
plot_y = {'numba_mpi': [], 'mpi4py': []}
|
|
129
|
+
for n_intervals in plot_x:
|
|
130
|
+
for impl in plot_y:
|
|
131
|
+
plot_y[impl].append(min(timeit.repeat(
|
|
132
|
+
f"pi_{impl}({n_intervals})",
|
|
133
|
+
globals=locals(),
|
|
134
|
+
number=1,
|
|
135
|
+
repeat=N_REPEAT
|
|
136
|
+
)))
|
|
137
|
+
|
|
138
|
+
if numba_mpi.rank() == 0:
|
|
139
|
+
from matplotlib import pyplot
|
|
140
|
+
pyplot.figure(figsize=(8.3, 3.5), tight_layout=True)
|
|
141
|
+
pyplot.plot(plot_x, np.array(plot_y['mpi4py'])/np.array(plot_y['numba_mpi']), marker='o')
|
|
142
|
+
pyplot.xlabel('n_intervals (workload in between communication)')
|
|
143
|
+
pyplot.ylabel('wall time ratio (mpi4py / numba_mpi)')
|
|
144
|
+
pyplot.title(f'mpiexec -np {numba_mpi.size()}')
|
|
145
|
+
pyplot.grid()
|
|
146
|
+
pyplot.savefig('readme_plot.png')
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
|
|
150
|
+
|
|
151
|
+
|
|
152
|
+
### MPI resources on the web:
|
|
153
|
+
|
|
154
|
+
- MPI standard and general information:
|
|
155
|
+
- https://www.mpi-forum.org/docs
|
|
156
|
+
- https://en.wikipedia.org/wiki/Message_Passing_Interface
|
|
157
|
+
- MPI implementations:
|
|
158
|
+
- OpenMPI: https://www.open-mpi.org
|
|
159
|
+
- MPICH: https://www.mpich.org
|
|
160
|
+
- MS MPI: https://learn.microsoft.com/en-us/message-passing-interface
|
|
161
|
+
- Intel MPI: https://intel.com/content/www/us/en/developer/tools/oneapi/mpi-library-documentation.html
|
|
162
|
+
- MPI bindings:
|
|
163
|
+
- Python: https://mpi4py.readthedocs.io
|
|
164
|
+
- Julia: https://juliaparallel.org/MPI.jl
|
|
165
|
+
- Rust: https://docs.rs/mpi
|
|
166
|
+
- C++: https://boost.org/doc/html/mpi.html
|
|
167
|
+
- R: https://cran.r-project.org/web/packages/Rmpi
|
|
168
|
+
|
|
169
|
+
### Acknowledgements:
|
|
170
|
+
|
|
171
|
+
Development of numba-mpi has been supported by the [Polish National Science Centre](https://ncn.gov.pl/en) (grant no. 2020/39/D/ST10/01220).
|
|
172
|
+
|
|
@@ -16,8 +16,8 @@ numba_mpi/api/scatter_gather.py,sha256=goZn4BxMKakWQHjfXIOdjzK3DJ-lTeaiQQwgnyQeZ
|
|
|
16
16
|
numba_mpi/api/send.py,sha256=jn1hPw0YHBHOaeJop_ZbjaBChaqgfw3nM1xGhW9sabI,909
|
|
17
17
|
numba_mpi/api/size.py,sha256=fYLeUrygvz_XcxIDsLiZlMtS-aiWfp58Zi7aIOAgaj8,549
|
|
18
18
|
numba_mpi/api/wtime.py,sha256=qrTqlefW7K7hqnAQKkGYm8kgdiRGuSAGiHmPcTrhLzE,279
|
|
19
|
-
numba_mpi-0.
|
|
20
|
-
numba_mpi-0.
|
|
21
|
-
numba_mpi-0.
|
|
22
|
-
numba_mpi-0.
|
|
23
|
-
numba_mpi-0.
|
|
19
|
+
numba_mpi-0.41.dist-info/LICENSE,sha256=OXLcl0T2SZ8Pmy2_dmlvKuetivmyPd5m1q-Gyd-zaYY,35149
|
|
20
|
+
numba_mpi-0.41.dist-info/METADATA,sha256=4POxiWzQaU7S3NeU8b_GKcog4H6OBjvvV62h-nPHj5I,8147
|
|
21
|
+
numba_mpi-0.41.dist-info/WHEEL,sha256=GJ7t_kWBFywbagK5eo9IoUwLW6oyOeTKmQ-9iHFVNxQ,92
|
|
22
|
+
numba_mpi-0.41.dist-info/top_level.txt,sha256=yb_ktLmrfuhOZS0rjS81FFNC-gK_4c19WbLG2ViP73g,10
|
|
23
|
+
numba_mpi-0.41.dist-info/RECORD,,
|
|
@@ -1,66 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: numba-mpi
|
|
3
|
-
Version: 0.40
|
|
4
|
-
Summary: Numba @njittable MPI wrappers tested on Linux, macOS and Windows
|
|
5
|
-
Home-page: https://github.com/numba-mpi/numba-mpi
|
|
6
|
-
Author: https://github.com/numba-mpi/numba-mpi/graphs/contributors
|
|
7
|
-
License: GPL v3
|
|
8
|
-
Project-URL: Tracker, https://github.com/numba-mpi/numba-mpi/issues
|
|
9
|
-
Project-URL: Documentation, https://numba-mpi.github.io/numba-mpi
|
|
10
|
-
Project-URL: Source, https://github.com/numba-mpi/numba-mpi
|
|
11
|
-
Description-Content-Type: text/markdown
|
|
12
|
-
License-File: LICENSE
|
|
13
|
-
Requires-Dist: numba
|
|
14
|
-
Requires-Dist: numpy
|
|
15
|
-
Requires-Dist: mpi4py
|
|
16
|
-
Requires-Dist: psutil
|
|
17
|
-
Provides-Extra: tests
|
|
18
|
-
Requires-Dist: pytest <8.0.0 ; extra == 'tests'
|
|
19
|
-
|
|
20
|
-
# <img src="https://raw.githubusercontent.com/numba-mpi/numba-mpi/main/.github/numba_mpi_logo.svg" style="height:50pt" alt="numba-mpi logo"> numba-mpi
|
|
21
|
-
|
|
22
|
-
[](https://www.python.org/)
|
|
23
|
-
[](https://numba.pydata.org)
|
|
24
|
-
[](https://en.wikipedia.org/wiki/Linux)
|
|
25
|
-
[](https://en.wikipedia.org/wiki/macOS)
|
|
26
|
-
[](https://en.wikipedia.org/wiki/Windows)
|
|
27
|
-
[](https://github.com/numba-mpi/numba-mpi/actions/workflows/tests+pypi.yml)
|
|
28
|
-
[](https://GitHub.com/numba-mpi/numba-mpi/graphs/commit-activity)
|
|
29
|
-
[](https://www.gnu.org/licenses/gpl-3.0.html)
|
|
30
|
-
[](https://pypi.org/project/numba-mpi)
|
|
31
|
-
[](https://anaconda.org/conda-forge/numba-mpi)
|
|
32
|
-
[](https://aur.archlinux.org/packages/python-numba-mpi)
|
|
33
|
-
[](https://zenodo.org/badge/latestdoi/316911228)
|
|
34
|
-
|
|
35
|
-
#### Numba @njittable MPI wrappers
|
|
36
|
-
- covering: `size`/`rank`, `send`/`recv`, `allreduce`, `bcast`, `scatter`/`gather` & `allgather`, `barrier` and `wtime`
|
|
37
|
-
- basic asynchronous communication with `isend`/`irecv` (only for contiguous arrays); for request handling including `wait`/`waitall`/`waitany` and `test`/`testall`/`testany`
|
|
38
|
-
- not yet implemented: support for non-default communicators, ...
|
|
39
|
-
- API based on NumPy and supporting numeric and character datatypes
|
|
40
|
-
- auto-generated docstring-based API docs on the web: https://numba-mpi.github.io/numba-mpi
|
|
41
|
-
- pure-Python implementation with packages available at [PyPI](https://pypi.org/project/numba-mpi), [Conda Forge](https://anaconda.org/conda-forge/numba-mpi) and for [Arch Linux](https://aur.archlinux.org/packages/python-numba-mpi)
|
|
42
|
-
- CI-tested on: Linux ([MPICH](https://www.mpich.org/), [OpenMPI](https://www.open-mpi.org/doc/) & [Intel MPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html)), macOS ([MPICH](https://www.mpich.org/) & [OpenMPI](https://www.open-mpi.org/doc/)) and Windows ([MS MPI](https://docs.microsoft.com/en-us/message-passing-interface/microsoft-mpi))
|
|
43
|
-
|
|
44
|
-
Hello world example:
|
|
45
|
-
```python
|
|
46
|
-
import numba, numba_mpi, numpy
|
|
47
|
-
|
|
48
|
-
@numba.njit()
|
|
49
|
-
def hello():
|
|
50
|
-
print(numba_mpi.rank())
|
|
51
|
-
print(numba_mpi.size())
|
|
52
|
-
|
|
53
|
-
src = numpy.array([1., 2., 3., 4., 5.])
|
|
54
|
-
dst_tst = numpy.empty_like(src)
|
|
55
|
-
|
|
56
|
-
if numba_mpi.rank() == 0:
|
|
57
|
-
numba_mpi.send(src, dest=1, tag=11)
|
|
58
|
-
elif numba_mpi.rank() == 1:
|
|
59
|
-
numba_mpi.recv(dst_tst, source=0, tag=11)
|
|
60
|
-
|
|
61
|
-
hello()
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
For information on MPI, see:
|
|
65
|
-
- https://www.mpi-forum.org/docs/
|
|
66
|
-
- https://mpi4py.readthedocs.io/en/stable/
|
|
File without changes
|
|
File without changes
|