numexpr 2.10.2__tar.gz → 2.11.0__tar.gz

{numexpr-2.10.2 → numexpr-2.11.0}/ANNOUNCE.rst

@@ -1,28 +1,33 @@
 =========================
-Announcing NumExpr 2.10.2
+Announcing NumExpr 2.11.0
 =========================
 
 Hi everyone,
 
-NumExpr 2.10.2 provides wheels for Python 2.13 for first time.
-Also, there is better support for CPUs that do not have a power
-of 2 number of cores.  Finally, numexpr is allowed to run with
-the multithreading package in Python.
+NumExpr 2.11.0 Initial support for free-threaded Python 3.13t has been added.
+This is still experimental, so please report any issues you find.
+Finally, Python 3.10 is now the minimum supported version.
 
 Project documentation is available at:
 
 http://numexpr.readthedocs.io/
 
-Changes from 2.10.1 to 2.10.2
+Changes from 2.10.2 to 2.11.0
 -----------------------------
 
-* Better support for CPUs that do not have a power of 2 number of
-  cores.  See #479 and #490.  Thanks to @avalentino.
+* Initial support for free-threaded Python 3.13t has been added.
+  This is still experimental, so please report any issues you find.
+  For more info, see discussions PRs #504, #505 and #508.
+  Thanks to @andfoy, @rgommers and @FrancescAlted for the work.
 
-* Allow numexpr to run with the multithreading package in Python.
-  See PR #496.  Thanks to @emmaai
+* Fix imaginary evaluation in the form of `1.1e1j`.  This was
+  previously not supported and would raise an error.  Thanks to @27rabbitlt
+  for the fix.
 
-* Wheels for Python 3.13 are now provided.
+* The test suite has been modernized to use `pytest` instead of `unittest`.
+  This should make it easier to run the tests and contribute to the project.
+
+* Python 3.10 is now the minimum supported version.
 
 What's Numexpr?
 ---------------

{numexpr-2.10.2 → numexpr-2.11.0}/AUTHORS.txt

@@ -23,7 +23,7 @@ Google Inc. contributed bug fixes.
 
 David Cox improved readability of the Readme.
 
-Robert A. McLeod contributed bug fixes and ported the documentation to 
+Robert A. McLeod contributed bug fixes and ported the documentation to
 numexpr.readthedocs.io. He has served as the maintainer of the package
 since 2016 to 2023.
 

{numexpr-2.10.2 → numexpr-2.11.0}/PKG-INFO

@@ -1,9 +1,10 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: numexpr
-Version: 2.10.2
+Version: 2.11.0
 Summary: Fast numerical expression evaluator for NumPy
 Author-email: "David M. Cooke, Francesc Alted, and others" <blosc@blosc.org>
 Maintainer-email: Blosc Development Team <blosc@blosc.org>
+License-Expression: MIT
 Project-URL: homepage, https://github.com/pydata/numexpr
 Project-URL: documentation, https://numexpr.readthedocs.io
 Project-URL: repository, https://github.com/pydata/numexpr
@@ -11,21 +12,21 @@ Classifier: Development Status :: 6 - Mature
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Information Technology
 Classifier: Intended Audience :: Science/Research
-Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Operating System :: Microsoft :: Windows
 Classifier: Operating System :: Unix
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.9
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.10
 Description-Content-Type: text/x-rst
 License-File: LICENSE.txt
 License-File: AUTHORS.txt
 Requires-Dist: numpy>=1.23.0
+Dynamic: license-file
 
 ======================================================
 NumExpr: Fast numerical expression evaluator for NumPy
@@ -188,6 +189,24 @@ Usage
   array([ True, False, False], dtype=bool)
 
 
+Free-threading support
+----------------------
+Starting on CPython 3.13 onwards there is a new distribution that disables the
+Global Interpreter Lock (GIL) altogether, thus increasing the performance yields
+under multi-threaded conditions on a single interpreter, as opposed to having to use
+multiprocessing.
+
+Whilst numexpr has been demonstrated to work under free-threaded
+CPython, considerations need to be taken when using numexpr native parallel
+implementation vs using Python threads directly in order to prevent oversubscription,
+we recommend either using the main CPython interpreter thread to spawn multiple C threads
+using the parallel numexpr API, or spawning multiple CPython threads that do not use
+the parallel API.
+
+For more information about free-threaded CPython, we recommend visiting the following
+`community Wiki <https://py-free-threading.github.io/>`
+
+
 Documentation
 -------------
 

{numexpr-2.10.2 → numexpr-2.11.0}/README.rst

@@ -159,6 +159,24 @@ Usage
   array([ True, False, False], dtype=bool)
 
 
+Free-threading support
+----------------------
+Starting on CPython 3.13 onwards there is a new distribution that disables the
+Global Interpreter Lock (GIL) altogether, thus increasing the performance yields
+under multi-threaded conditions on a single interpreter, as opposed to having to use
+multiprocessing.
+
+Whilst numexpr has been demonstrated to work under free-threaded
+CPython, considerations need to be taken when using numexpr native parallel
+implementation vs using Python threads directly in order to prevent oversubscription,
+we recommend either using the main CPython interpreter thread to spawn multiple C threads
+using the parallel numexpr API, or spawning multiple CPython threads that do not use
+the parallel API.
+
+For more information about free-threaded CPython, we recommend visiting the following
+`community Wiki <https://py-free-threading.github.io/>`
+
+
 Documentation
 -------------
 

{numexpr-2.10.2 → numexpr-2.11.0}/RELEASE_NOTES.rst

@@ -1,7 +1,24 @@
 =====================================
-Release notes for NumExpr 2.10 series
+Release notes for NumExpr 2.11 series
 =====================================
 
+Changes from 2.10.2 to 2.11.0
+-----------------------------
+
+* Initial support for free-threaded Python 3.13t has been added.
+  This is still experimental, so please report any issues you find.
+  For more info, see discussions PRs #504, #505 and #508.
+  Thanks to @andfoy, @rgommers and @FrancescAlted for the work.
+
+* Fix imaginary evaluation in the form of `1.1e1j`.  This was
+  previously not supported and would raise an error.  Thanks to @27rabbitlt
+  for the fix.
+
+* The test suite has been modernized to use `pytest` instead of `unittest`.
+  This should make it easier to run the tests and contribute to the project.
+
+* Python 3.10 is now the minimum supported version.
+
 Changes from 2.10.1 to 2.10.2
 -----------------------------
 

numexpr-2.11.0/VERSION

@@ -0,0 +1 @@
+2.11.0

{numexpr-2.10.2 → numexpr-2.11.0}/bench/boolean_timing.py

@@ -9,8 +9,10 @@
 ####################################################################
 
 from __future__ import print_function
+
 import sys
 import timeit
+
 import numpy
 
 array_size = 5_000_000

numexpr-2.11.0/bench/free_threading.py

@@ -0,0 +1,171 @@
+#################################################################################
+# To compare the performance of numexpr when free-threading CPython is used.
+#
+# This example makes use of Python threads, as opposed to C native ones
+# in order to highlight the improvement introduced by free-threading CPython,
+# which now disables the GIL altogether.
+#################################################################################
+"""
+Results with GIL-enabled CPython:
+
+Benchmarking Expression 1:
+NumPy time (threaded over 32 chunks with 16 threads): 1.173090 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 0.951071 seconds
+numexpr speedup: 1.23x
+----------------------------------------
+Benchmarking Expression 2:
+NumPy time (threaded over 32 chunks with 16 threads): 10.410874 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 8.248753 seconds
+numexpr speedup: 1.26x
+----------------------------------------
+Benchmarking Expression 3:
+NumPy time (threaded over 32 chunks with 16 threads): 9.605909 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 11.087108 seconds
+numexpr speedup: 0.87x
+----------------------------------------
+Benchmarking Expression 4:
+NumPy time (threaded over 32 chunks with 16 threads): 3.836962 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 18.054531 seconds
+numexpr speedup: 0.21x
+----------------------------------------
+
+Results with free-threading CPython:
+
+Benchmarking Expression 1:
+NumPy time (threaded over 32 chunks with 16 threads): 3.415349 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 2.618876 seconds
+numexpr speedup: 1.30x
+----------------------------------------
+Benchmarking Expression 2:
+NumPy time (threaded over 32 chunks with 16 threads): 19.005238 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 12.611407 seconds
+numexpr speedup: 1.51x
+----------------------------------------
+Benchmarking Expression 3:
+NumPy time (threaded over 32 chunks with 16 threads): 20.555149 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 17.690749 seconds
+numexpr speedup: 1.16x
+----------------------------------------
+Benchmarking Expression 4:
+NumPy time (threaded over 32 chunks with 16 threads): 38.338372 seconds
+numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 35.074684 seconds
+numexpr speedup: 1.09x
+----------------------------------------
+"""
+
+import os
+
+os.environ["NUMEXPR_NUM_THREADS"] = "2"
+import threading
+import timeit
+
+import numpy as np
+
+import numexpr as ne
+
+array_size = 10**8
+num_runs = 10
+num_chunks = 32  # Number of chunks
+num_threads = 16  # Number of threads constrained by how many chunks memory can hold
+
+a = np.random.rand(array_size).reshape(10**4, -1)
+b = np.random.rand(array_size).reshape(10**4, -1)
+c = np.random.rand(array_size).reshape(10**4, -1)
+
+chunk_size = array_size // num_chunks
+
+expressions_numpy = [
+    lambda a, b, c: a + b * c,
+    lambda a, b, c: a**2 + b**2 - 2 * a * b * np.cos(c),
+    lambda a, b, c: np.sin(a) + np.log(b) * np.sqrt(c),
+    lambda a, b, c: np.exp(a) + np.tan(b) - np.sinh(c),
+]
+
+expressions_numexpr = [
+    "a + b * c",
+    "a**2 + b**2 - 2 * a * b * cos(c)",
+    "sin(a) + log(b) * sqrt(c)",
+    "exp(a) + tan(b) - sinh(c)",
+]
+
+
+def benchmark_numpy_chunk(func, a, b, c, results, indices):
+    for index in indices:
+        start = index * chunk_size
+        end = (index + 1) * chunk_size
+        time_taken = timeit.timeit(
+            lambda: func(a[start:end], b[start:end], c[start:end]), number=num_runs
+        )
+        results.append(time_taken)
+
+
+def benchmark_numexpr_re_evaluate(expr, a, b, c, results, indices):
+    for index in indices:
+        start = index * chunk_size
+        end = (index + 1) * chunk_size
+        # if index == 0:
+        # Evaluate the first chunk with evaluate
+        time_taken = timeit.timeit(
+            lambda: ne.evaluate(
+                expr,
+                local_dict={
+                    "a": a[start:end],
+                    "b": b[start:end],
+                    "c": c[start:end],
+                },
+            ),
+            number=num_runs,
+        )
+        results.append(time_taken)
+
+
+def run_benchmark_threaded():
+    chunk_indices = list(range(num_chunks))
+
+    for i in range(len(expressions_numpy)):
+        print(f"Benchmarking Expression {i+1}:")
+
+        results_numpy = []
+        results_numexpr = []
+
+        threads_numpy = []
+        for j in range(num_threads):
+            indices = chunk_indices[j::num_threads]  # Distribute chunks across threads
+            thread = threading.Thread(
+                target=benchmark_numpy_chunk,
+                args=(expressions_numpy[i], a, b, c, results_numpy, indices),
+            )
+            threads_numpy.append(thread)
+            thread.start()
+
+        for thread in threads_numpy:
+            thread.join()
+
+        numpy_time = sum(results_numpy)
+        print(
+            f"NumPy time (threaded over {num_chunks} chunks with {num_threads} threads): {numpy_time:.6f} seconds"
+        )
+
+        threads_numexpr = []
+        for j in range(num_threads):
+            indices = chunk_indices[j::num_threads]  # Distribute chunks across threads
+            thread = threading.Thread(
+                target=benchmark_numexpr_re_evaluate,
+                args=(expressions_numexpr[i], a, b, c, results_numexpr, indices),
+            )
+            threads_numexpr.append(thread)
+            thread.start()
+
+        for thread in threads_numexpr:
+            thread.join()
+
+        numexpr_time = sum(results_numexpr)
+        print(
+            f"numexpr time (threaded with re_evaluate over {num_chunks} chunks with {num_threads} threads): {numexpr_time:.6f} seconds"
+        )
+        print(f"numexpr speedup: {numpy_time / numexpr_time:.2f}x")
+        print("-" * 40)
+
+
+if __name__ == "__main__":
+    run_benchmark_threaded()

{numexpr-2.10.2 → numexpr-2.11.0}/bench/issue-36.py

@@ -2,10 +2,14 @@
 # performs better than the serial code.  See issue #36 for details.
 
 from __future__ import print_function
+
+from time import time
+
 import numpy as np
-import numexpr as ne
 from numpy.testing import assert_array_equal
-from time import time
+
+import numexpr as ne
+
 
 def bench(N):
     print("*** array length:", N)
@@ -31,4 +35,3 @@ if __name__ == "__main__":
     ne.set_num_threads(2)
     for N in range(10, 20):
         bench(2**N)
-

{numexpr-2.10.2 → numexpr-2.11.0}/bench/issue-47.py

@@ -1,4 +1,5 @@
 import numpy
+
 import numexpr
 
 numexpr.set_num_threads(8)

{numexpr-2.10.2 → numexpr-2.11.0}/bench/large_array_vs_numpy.py

@@ -31,10 +31,12 @@ numexpr speedup: 2.38x
 import os
 
 os.environ["NUMEXPR_NUM_THREADS"] = "16"
+import threading
+import timeit
+
 import numpy as np
+
 import numexpr as ne
-import timeit
-import threading
 
 array_size = 10**8
 num_runs = 10

{numexpr-2.10.2 → numexpr-2.11.0}/bench/multidim.py

@@ -12,9 +12,12 @@
 # Based on a script provided by Andrew Collette.
 
 from __future__ import print_function
+
+import time
+
 import numpy as np
+
 import numexpr as nx
-import time
 
 test_shapes = [
     (100*100*100),
@@ -90,5 +93,3 @@ for test_shape in test_shapes:
     print("Simple: ", (stop1-start1)/nruns)
     print("Numexpr: ", (stop2-start2)/nruns)
     print("Chunked: ", (stop3-start3)/nruns)
-
-

{numexpr-2.10.2 → numexpr-2.11.0}/bench/poly.py

@@ -17,11 +17,13 @@
 #######################################################################
 
 from __future__ import print_function
+
 import sys
 from time import time
+
 import numpy as np
-import numexpr as ne
 
+import numexpr as ne
 
 #expr = ".25*x**3 + .75*x**2 - 1.5*x - 2"  # the polynomial to compute
 expr = "((.25*x + .75)*x - 1.5)*x - 2"  # a computer-friendly polynomial

{numexpr-2.10.2 → numexpr-2.11.0}/bench/timing.py

@@ -9,7 +9,10 @@
 ####################################################################
 
 from __future__ import print_function
-import timeit, numpy
+
+import timeit
+
+import numpy
 
 array_size = 5e6
 iterations = 2

{numexpr-2.10.2 → numexpr-2.11.0}/bench/unaligned-simple.py

@@ -13,8 +13,11 @@ aligned vs unaligned arrays.
 """
 
 from __future__ import print_function
+
 from timeit import Timer
+
 import numpy as np
+
 import numexpr as ne
 
 niter = 10

{numexpr-2.10.2 → numexpr-2.11.0}/bench/varying-expr.py

@@ -13,9 +13,12 @@
 # the latency of numexpr when working with small arrays.
 
 from __future__ import print_function
+
 import sys
 from time import time
+
 import numpy as np
+
 import numexpr as ne
 
 N = 100

{numexpr-2.10.2 → numexpr-2.11.0}/bench/vml_timing.py

@@ -9,9 +9,12 @@
 ####################################################################
 
 from __future__ import print_function
+
 import sys
 import timeit
+
 import numpy
+
 import numexpr
 
 array_size = 5_000_000

{numexpr-2.10.2 → numexpr-2.11.0}/bench/vml_timing2.py

@@ -4,11 +4,14 @@
 # https://github.com/pydata/numexpr/wiki/NumexprMKL
 
 from __future__ import print_function
+
 import datetime
 import sys
+from time import time
+
 import numpy as np
+
 import numexpr as ne
-from time import time
 
 N = int(2**26)
 

{numexpr-2.10.2 → numexpr-2.11.0}/bench/vml_timing3.py

@@ -1,7 +1,9 @@
 # -*- coding: utf-8 -*-
+from timeit import default_timer as timer
+
 import numpy as np
+
 import numexpr as ne
-from timeit import default_timer as timer
 
 x = np.ones(100000)
 scaler = -1J

{numexpr-2.10.2 → numexpr-2.11.0}/numexpr/__init__.py

@@ -21,21 +21,20 @@ for more info about it.
 
 """
 
-from numexpr.interpreter import MAX_THREADS, use_vml, __BLOCK_SIZE1__
+from numexpr.interpreter import __BLOCK_SIZE1__, MAX_THREADS, use_vml
 
 is_cpu_amd_intel = False # DEPRECATION WARNING: WILL BE REMOVED IN FUTURE RELEASE
 
-# cpuinfo imports were moved into the test submodule function that calls them 
+# cpuinfo imports were moved into the test submodule function that calls them
 # to improve import times.
 
 from numexpr.expressions import E
-from numexpr.necompiler import (NumExpr, disassemble, evaluate, re_evaluate, 
-    validate)
-
-from numexpr.utils import (_init_num_threads,
-    get_vml_version, set_vml_accuracy_mode, set_vml_num_threads,
-    set_num_threads, get_num_threads,
-    detect_number_of_cores, detect_number_of_threads)
+from numexpr.necompiler import (NumExpr, disassemble, evaluate, re_evaluate,
+                                validate)
+from numexpr.utils import (_init_num_threads, detect_number_of_cores,
+                           detect_number_of_threads, get_num_threads,
+                           get_vml_version, set_num_threads,
+                           set_vml_accuracy_mode, set_vml_num_threads)
 
 # Detect the number of cores
 ncores = detect_number_of_cores()
@@ -45,6 +44,7 @@ nthreads = _init_num_threads()
 # set_vml_num_threads(1)
 
 from . import version
+
 __version__ = version.version
 
 def print_versions():
@@ -63,4 +63,4 @@ def test(verbosity=1):
         return numexpr.tests.test(verbosity=verbosity)
     except ImportError:
         # To maintain Python 2.6 compatibility we have simple error handling
-        raise ImportError('`numexpr.tests` could not be imported, likely it was excluded from the distribution.')
+        raise ImportError('`numexpr.tests` could not be imported, likely it was excluded from the distribution.')

{numexpr-2.10.2 → numexpr-2.11.0}/numexpr/cpuinfo.py

@@ -23,12 +23,14 @@ Pearu Peterson
 
 __all__ = ['cpu']
 
-import sys, re, types
+import inspect
 import os
+import platform
+import re
 import subprocess
+import sys
+import types
 import warnings
-import platform
-import inspect
 
 is_cpu_amd_intel = False # DEPRECATION WARNING: WILL BE REMOVED IN FUTURE RELEASE
 

{numexpr-2.10.2 → numexpr-2.11.0}/numexpr/expressions.py

@@ -35,6 +35,7 @@ scalar_constant_types = tuple(scalar_constant_types)
 
 from numexpr import interpreter
 
+
 class Expression():
 
     def __getattr__(self, name):
@@ -269,10 +270,10 @@ def rtruediv_op(a, b):
 
 @ophelper
 def pow_op(a, b):
-    
+
     if isinstance(b, ConstantNode):
         x = b.value
-        if (    a.astKind in ('int', 'long') and 
+        if (    a.astKind in ('int', 'long') and
                 b.astKind in ('int', 'long') and x < 0) :
             raise ValueError(
                 'Integers to negative integer powers are not allowed.')

{numexpr-2.10.2 → numexpr-2.11.0}/numexpr/interp_body.cpp

@@ -7,13 +7,13 @@
   See LICENSE.txt for details about copyright and rights to use.
 **********************************************************************/
 
-// WARNING: This file is included multiple times in `interpreter.cpp`. It is 
-// essentially a very macro-heavy jump table. Interpretation is best done by 
+// WARNING: This file is included multiple times in `interpreter.cpp`. It is
+// essentially a very macro-heavy jump table. Interpretation is best done by
 // the developer by expanding all macros (e.g. adding `'-E'` to the `extra_cflags`
 // argument in `setup.py` and looking at the resulting `interpreter.cpp`.
 //
-// Changes made to this file will not be recognized by the compile, so the developer 
-// must make a trivial change is made to `interpreter.cpp` or delete the `build/` 
+// Changes made to this file will not be recognized by the compile, so the developer
+// must make a trivial change is made to `interpreter.cpp` or delete the `build/`
 // directory in-between each build.
 {
 #define VEC_LOOP(expr) for(j = 0; j < BLOCK_SIZE; j++) {       \