kernel-tuner 0.1.8__tar.gz → 0.2.0__tar.gz

{kernel_tuner-0.1.8 → kernel_tuner-0.2.0}/PKG-INFO

@@ -1,15 +1,15 @@
-Metadata-Version: 1.1
+Metadata-Version: 2.1
 Name: kernel_tuner
-Version: 0.1.8
+Version: 0.2.0
 Summary: A simple CUDA/OpenCL kernel tuner in Python
 Home-page: http://benvanwerkhoven.github.io/kernel_tuner/
 Author: Ben van Werkhoven
 Author-email: b.vanwerkhoven@esciencecenter.nl
 License: Apache 2.0
-Description: Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python
-        =========================================================
+Description: Kernel Tuner
+        ============
         
-        |Build Status| |Codacy Badge| |Codacy Badge2|
+        |Build Status| |Codacy Badge| |Codacy Badge2| |PyPi Badge|
         
         Kernel Tuner simplifies the software development of optimized and auto-tuned GPU programs, by enabling Python-based unit testing of GPU code and making it easy to develop scripts for auto-tuning GPU kernels. This also means no extensive changes and no new dependencies are required in the kernel code. The kernels can still be compiled and used as normal from any host programming language.
         
@@ -99,6 +99,23 @@ Description: Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python
         and in the `Kernel Tuner
         documentation pages <http://benvanwerkhoven.github.io/kernel_tuner/index.html>`__.
         
+        Search strategies for tuning
+        ----------------------------
+        
+        Kernel Tuner supports many optimization algorithms to accelerate the auto-tuning process. Currently 
+        implemented search algorithms are: Brute Force (default), Nelder-Mead, Powell, CG, BFGS, L-BFGS-B, TNC, 
+        COBYLA, SLSQP, Random Search, Basinhopping, Differential Evolution, a Genetic Algorithm, Particle Swarm 
+        Optimization, the Firefly Algorithm, and Simulated Annealing.
+        
+        .. image:: doc/gemm-amd-summary.png
+            :width: 100%
+            :align: center
+        
+        Using a search strategy is easy, you only need to specify to ``tune_kernel`` which strategy and method 
+        you would like to use, for example ``strategy="genetic_algorithm"`` or ``strategy="basinhopping", 
+        method="Powell"``. For a full overview of the supported search strategies and methods please see the `user 
+        api documentation <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
+        
         Tuning host and kernel code
         ---------------------------
         
@@ -113,12 +130,6 @@ Description: Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python
         `documentation <http://benvanwerkhoven.github.io/kernel_tuner/hostcode.html>`__
         for a detailed explanation of the kernel tuner Python script.
         
-        Search strategies for tuning
-        ----------------------------
-        
-        Kernel Tuner supports several strategies: brute_force (default), random_sample, minimize, basinhopping, and diff_evo (differential
-        evolution). Using different strategies is easy, you only need to specify to ``tune_kernel`` which strategy you would like to use, for example: ``strategy="basinhopping", method="TNC"``. For a full overview of the supported search strategies and methods please see the
-        `user api <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
         
         Correctness verification
         ------------------------
@@ -140,23 +151,26 @@ Description: Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python
         
         Citation
         --------
-        A scientific paper about the Kernel Tuner is in preparation, in the meantime please cite the Kernel Tuner as follows:
+        A scientific paper about Kernel Tuner has been accepted for publication, please cite
+        Kernel Tuner as follows:
         
         .. code:: latex
         
-            @misc{
-              author = {Ben van Werkhoven},
-              title = {Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python},
-              year = {2018}
+            @article{kerneltuner,
+              author  = {Ben van Werkhoven},
+              title   = {Kernel Tuner: A search-optimizing GPU code auto-tuner},
+              journal = {Future Generation Computer Systems},
+              year = {2019},
+              volume  = {90},
+              pages = {347-358},
+              doi = {https://doi.org/10.1016/j.future.2018.08.004},
             }
         
         Related work
         ------------
         
         You may also like `CLTune <https://github.com/CNugteren/CLTune>`__ by
-        Cedric Nugteren. CLTune is a C++ library for kernel tuning and supports
-        various advanced features like machine learning to optimize the time
-        spent on tuning kernels.
+        Cedric Nugteren. CLTune is a C++ library for kernel tuning.
         
         .. |Build Status| image:: https://api.travis-ci.org/benvanwerkhoven/kernel_tuner.svg?branch=master
            :target: https://travis-ci.org/benvanwerkhoven/kernel_tuner
@@ -164,6 +178,8 @@ Description: Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python
            :target: https://www.codacy.com/app/b-vanwerkhoven/kernel_tuner
         .. |Codacy Badge2| image:: https://api.codacy.com/project/badge/coverage/016dc85044ab4d57b777449d93275608
            :target: https://www.codacy.com/app/b-vanwerkhoven/kernel_tuner
+        .. |PyPi Badge| image:: https://img.shields.io/pypi/v/kernel_tuner.svg?colorB=blue 
+           :target: https://pypi.python.org/pypi/kernel_tuner/
         
 Keywords: auto-tuning gpu computing pycuda cuda pyopencl opencl
 Platform: UNKNOWN
@@ -180,3 +196,9 @@ Classifier: Topic :: Scientific/Engineering
 Classifier: Topic :: Software Development
 Classifier: Topic :: System :: Distributed Computing
 Classifier: Development Status :: 4 - Beta
+Provides-Extra: cuda_opencl
+Provides-Extra: cuda
+Provides-Extra: dev
+Provides-Extra: tutorial
+Provides-Extra: opencl
+Provides-Extra: doc

{kernel_tuner-0.1.8 → kernel_tuner-0.2.0}/README.rst

@@ -1,7 +1,7 @@
-Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python
-=========================================================
+Kernel Tuner
+============
 
-|Build Status| |Codacy Badge| |Codacy Badge2|
+|Build Status| |Codacy Badge| |Codacy Badge2| |PyPi Badge|
 
 Kernel Tuner simplifies the software development of optimized and auto-tuned GPU programs, by enabling Python-based unit testing of GPU code and making it easy to develop scripts for auto-tuning GPU kernels. This also means no extensive changes and no new dependencies are required in the kernel code. The kernels can still be compiled and used as normal from any host programming language.
 
@@ -91,6 +91,23 @@ You can find many - more extensive - example codes, in the
 and in the `Kernel Tuner
 documentation pages <http://benvanwerkhoven.github.io/kernel_tuner/index.html>`__.
 
+Search strategies for tuning
+----------------------------
+
+Kernel Tuner supports many optimization algorithms to accelerate the auto-tuning process. Currently 
+implemented search algorithms are: Brute Force (default), Nelder-Mead, Powell, CG, BFGS, L-BFGS-B, TNC, 
+COBYLA, SLSQP, Random Search, Basinhopping, Differential Evolution, a Genetic Algorithm, Particle Swarm 
+Optimization, the Firefly Algorithm, and Simulated Annealing.
+
+.. image:: doc/gemm-amd-summary.png
+    :width: 100%
+    :align: center
+
+Using a search strategy is easy, you only need to specify to ``tune_kernel`` which strategy and method 
+you would like to use, for example ``strategy="genetic_algorithm"`` or ``strategy="basinhopping", 
+method="Powell"``. For a full overview of the supported search strategies and methods please see the `user 
+api documentation <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
+
 Tuning host and kernel code
 ---------------------------
 
@@ -105,12 +122,6 @@ and the
 `documentation <http://benvanwerkhoven.github.io/kernel_tuner/hostcode.html>`__
 for a detailed explanation of the kernel tuner Python script.
 
-Search strategies for tuning
-----------------------------
-
-Kernel Tuner supports several strategies: brute_force (default), random_sample, minimize, basinhopping, and diff_evo (differential
-evolution). Using different strategies is easy, you only need to specify to ``tune_kernel`` which strategy you would like to use, for example: ``strategy="basinhopping", method="TNC"``. For a full overview of the supported search strategies and methods please see the
-`user api <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
 
 Correctness verification
 ------------------------
@@ -132,23 +143,26 @@ Please see the `Contributions Guide <http://benvanwerkhoven.github.io/kernel_tun
 
 Citation
 --------
-A scientific paper about the Kernel Tuner is in preparation, in the meantime please cite the Kernel Tuner as follows:
+A scientific paper about Kernel Tuner has been accepted for publication, please cite
+Kernel Tuner as follows:
 
 .. code:: latex
 
-    @misc{
-      author = {Ben van Werkhoven},
-      title = {Kernel Tuner: A simple CUDA/OpenCL Auto-Tuner in Python},
-      year = {2018}
+    @article{kerneltuner,
+      author  = {Ben van Werkhoven},
+      title   = {Kernel Tuner: A search-optimizing GPU code auto-tuner},
+      journal = {Future Generation Computer Systems},
+      year = {2019},
+      volume  = {90},
+      pages = {347-358},
+      doi = {https://doi.org/10.1016/j.future.2018.08.004},
     }
 
 Related work
 ------------
 
 You may also like `CLTune <https://github.com/CNugteren/CLTune>`__ by
-Cedric Nugteren. CLTune is a C++ library for kernel tuning and supports
-various advanced features like machine learning to optimize the time
-spent on tuning kernels.
+Cedric Nugteren. CLTune is a C++ library for kernel tuning.
 
 .. |Build Status| image:: https://api.travis-ci.org/benvanwerkhoven/kernel_tuner.svg?branch=master
    :target: https://travis-ci.org/benvanwerkhoven/kernel_tuner
@@ -156,3 +170,5 @@ spent on tuning kernels.
    :target: https://www.codacy.com/app/b-vanwerkhoven/kernel_tuner
 .. |Codacy Badge2| image:: https://api.codacy.com/project/badge/coverage/016dc85044ab4d57b777449d93275608
    :target: https://www.codacy.com/app/b-vanwerkhoven/kernel_tuner
+.. |PyPi Badge| image:: https://img.shields.io/pypi/v/kernel_tuner.svg?colorB=blue 
+   :target: https://pypi.python.org/pypi/kernel_tuner/

{kernel_tuner-0.1.8 → kernel_tuner-0.2.0}/kernel_tuner/c.py

@@ -1,8 +1,10 @@
 """ This module contains the functionality for running and compiling C functions """
 
+from collections import namedtuple
 import subprocess
 import platform
 import errno
+import re
 import logging
 import ctypes as C
 import _ctypes
@@ -12,6 +14,15 @@ import numpy.ctypeslib
 
 from kernel_tuner.util import get_temp_filename, delete_temp_file, write_file
 
+dtype_map = {"int8": C.c_int8,
+             "int16": C.c_int16,
+             "int32": C.c_int32,
+             "int64": C.c_int64,
+             "float32": C.c_float,
+             "float64": C.c_double}
+
+Argument = namedtuple("Argument", ["type", "shape"])
+
 
 class CFunctions(object):
     """Class that groups the code for running and compiling C functions"""
@@ -67,31 +78,23 @@ class CFunctions(object):
         :returns: A list of arguments that can be passed to the C function.
         :rtype: list()
         """
-        ctype_args = []
-
-        dtype_map = {"int32": C.c_int32,
-                     "int64": C.c_int64,
-                     "float32": C.c_float,
-                     "float64": C.c_double}
-        np_to_c_type_map = {numpy.int32: C.c_int32,
-                            numpy.int64: C.c_int64,
-                            numpy.float32: C.c_float,
-                            numpy.float64: C.c_double}
-
-        for arg in arguments:
+        ctype_args = [None for _ in arguments]
+        self.arg_mapping = dict()
+
+        for i, arg in enumerate(arguments):
+            if not isinstance(arg, (numpy.ndarray, numpy.generic)):
+                raise TypeError("Argument is not numpy ndarray or numpy scalar %s" % type(arg))
+            dtype_str = str(arg.dtype)
+            arg_info = Argument(dtype_str, arg.shape)
             if isinstance(arg, numpy.ndarray):
-                dtype_str = str(arg.dtype)
                 if dtype_str in dtype_map.keys():
-                    ctype_args.append(arg.ctypes.data_as(C.POINTER(dtype_map[dtype_str])))
+                    ctype_args[i] = arg.ctypes.data_as(C.POINTER(dtype_map[dtype_str]))
                 else:
                     raise TypeError("unknown dtype for ndarray")
-                self.arg_mapping[str(ctype_args[-1])] = arg.shape
-            elif isinstance(arg, tuple(np_to_c_type_map.keys())):
-                ctype_args.append(np_to_c_type_map[type(arg)](arg))
-                self.arg_mapping[str(ctype_args[-1])] = ()
-            else:
-                raise TypeError("Argument is not numpy ndarray or numpy scalar %s" % type(arg))
-
+                self.arg_mapping[str(ctype_args[i])] = arg_info
+            elif isinstance(arg, numpy.generic):
+                ctype_args[i] = dtype_map[dtype_str](arg)
+                self.arg_mapping[str(i)] = arg_info
         return ctype_args
 
 
@@ -113,25 +116,35 @@ class CFunctions(object):
         if self.lib != None:
             self.cleanup_lib()
 
-        suffix = ".cc"
-
-        if not "extern \"C\"" in kernel_string:
-            kernel_string = "extern \"C\" {\n" + kernel_string + "\n}"
-
         compiler_options = ["-fPIC"]
-        if "#include <omp.h>" in kernel_string:
+
+        #detect openmp
+        if "#include <omp.h>" in kernel_string or "use omp_lib" in kernel_string:
             logging.debug('set using_openmp to true')
             self.using_openmp = True
-            compiler_options.append("-fopenmp")
+            if self.compiler == "pgfortran":
+                compiler_options.append("-mp")
+            else:
+                compiler_options.append("-fopenmp")
 
+        #detect whether to use nvcc as default instead of g++, may overrule an explicitly passed g++
         if ("#include <cuda" in kernel_string) or ("__global__" in kernel_string):
-            if self.nvcc_available:
+            if self.compiler == "g++" and self.nvcc_available:
                 self.compiler = "nvcc"
 
+        #select right suffix based on compiler
+        suffix = ".cc"
+        if self.compiler in ["gfortran", "pgfortran", "ftn", "ifort"]:
+            suffix = ".F90"
         if self.compiler == "nvcc":
             suffix = suffix[:-1] + "u"
             compiler_options = ["-Xcompiler=" + c for c in compiler_options]
 
+        if ".c" in suffix:
+            if not "extern \"C\"" in kernel_string:
+                kernel_string = "extern \"C\" {\n" + kernel_string + "\n}"
+
+        #copy user specified compiler options to current list
         if self.compiler_options:
             compiler_options += self.compiler_options
 
@@ -146,6 +159,17 @@ class CFunctions(object):
         source_file = get_temp_filename(suffix=suffix)
         filename = ".".join(source_file.split(".")[:-1])
 
+        #detect Fortran modules
+        match = re.search(r"\s*module\s+([a-zA-Z_]*)", kernel_string)
+        if match:
+            if self.compiler == "gfortran":
+                kernel_name = "__" + match.group(1) + "_MOD_" + kernel_name
+            elif self.compiler in ["ftn", "ifort"]:
+                kernel_name = match.group(1) + "_mp_" + kernel_name + "_"
+            elif self.compiler == "pgfortran":
+                kernel_name = match.group(1) + "_" + kernel_name + "_"
+
+
         try:
             write_file(source_file, kernel_string)
 
@@ -156,8 +180,8 @@ class CFunctions(object):
             subprocess.check_call([self.compiler, "-c", source_file] + compiler_options + ["-o", filename + ".o"])
             subprocess.check_call([self.compiler, filename + ".o"] + compiler_options + ["-shared", "-o", filename + lib_extension] + lib_args)
 
-            self.lib = numpy.ctypeslib.load_library(filename, '.')
 
+            self.lib = numpy.ctypeslib.load_library(filename, '.')
             func = getattr(self.lib, kernel_name)
             func.restype = C.c_float
 
@@ -167,9 +191,9 @@ class CFunctions(object):
             delete_temp_file(filename+".so")
             delete_temp_file(filename+".dylib")
 
-
         return func
 
+
     def benchmark(self, func, c_args, threads, grid, times):
         """runs the kernel repeatedly, returns averaged returned value
 
@@ -287,7 +311,8 @@ class CFunctions(object):
         :param src: A ctypes pointer to some memory allocation
         :type src: ctypes.pointer
         """
-        dest[:] = numpy.ctypeslib.as_array(src, shape=self.arg_mapping[str(src)])
+        arginfo = self.arg_mapping[str(src)]
+        dest[:] = numpy.ctypeslib.as_array(src, shape=arginfo.shape)
 
 
     def cleanup_lib(self):

{kernel_tuner-0.1.8 → kernel_tuner-0.2.0}/kernel_tuner/cuda.py

@@ -47,11 +47,22 @@ class CudaFunctions(object):
         #inspect device properties
         devprops = {str(k): v for (k, v) in self.context.get_device().get_attributes().items()}
         self.max_threads = devprops['MAX_THREADS_PER_BLOCK']
-        self.cc = str(devprops.get('COMPUTE_CAPABILITY_MAJOR','0')) + str(devprops.get('COMPUTE_CAPABILITY_MINOR','0'))
+        cc = str(devprops.get('COMPUTE_CAPABILITY_MAJOR', '0')) + str(devprops.get('COMPUTE_CAPABILITY_MINOR', '0'))
+        if cc == "00":
+            cc = self.context.get_device().compute_capability()
+        self.cc = str(cc[0])+str(cc[1])
         self.iterations = iterations
         self.current_module = None
         self.compiler_options = compiler_options or []
 
+        #select PyCUDA source module
+        if int(self.cc) >= 35:
+            self.source_mod = DynamicSourceModule
+        else:
+            self.source_mod = SourceModule
+        if not self.source_mod:
+            raise ImportError("Error: pycuda not correctly installed, please ensure pycuda is installed on the same CUDA installation as you're using right now")
+
         #collect environment information
         env = dict()
         env["device_name"] = self.context.get_device().name()
@@ -114,16 +125,10 @@ class CudaFunctions(object):
             if self.compiler_options:
                 compiler_options += self.compiler_options
 
-            if int(self.cc) >= 35:
-                source_mod = DynamicSourceModule
-            else:
-                source_mod = SourceModule
-
-            self.current_module = source_mod(kernel_string, options=compiler_options + ["-e", kernel_name],
-                                               arch=('compute_' + self.cc) if self.cc != "00" else None,
-                                               code=('sm_' + self.cc) if self.cc != "00" else None,
-                                               cache_dir=False, no_extern_c=no_extern_c)
-
+            self.current_module = self.source_mod(kernel_string, options=compiler_options + ["-e", kernel_name],
+                                             arch=('compute_' + self.cc) if self.cc != "00" else None,
+                                             code=('sm_' + self.cc) if self.cc != "00" else None,
+                                             cache_dir=False, no_extern_c=no_extern_c)
 
             func = self.current_module.get_function(kernel_name)
             return func

{kernel_tuner-0.1.8 → kernel_tuner-0.2.0}/kernel_tuner/interface.py

@@ -35,7 +35,7 @@ import numpy
 import kernel_tuner.util as util
 import kernel_tuner.core as core
 
-from kernel_tuner.strategies import brute_force, random_sample, diff_evo, minimize, basinhopping
+from kernel_tuner.strategies import brute_force, random_sample, diff_evo, minimize, basinhopping, genetic_algorithm, pso, simulated_annealing, firefly_algorithm
 
 class Options(OrderedDict):
     """read-only class for passing options around"""
@@ -194,6 +194,10 @@ _tuning_options = Options([
             * "random_sample", specify: *sample_fraction*,
             * "minimize" or "basinhopping", specify: *method*,
             * "diff_evo", specify: *method*.
+            * "genetic_algorithm"
+            * "pso"
+            * "firefly_algorithm"
+            * "simulated_annealing"
 
         "brute_force" is the default and iterates over the entire search
         space.
@@ -213,6 +217,17 @@ _tuning_options = Options([
         "rand2exp", "randtobest1bin", "best2bin", "rand2bin", "rand1bin".
         The default is "best1bin".
 
+        "genetic_algorithm" implements a simple Genetic Algorithm, default
+        setting uses a population size of 20 for 100 generations.
+
+        "pso" implements Particle Swarm Optimization, using the default
+        setting of 20 particles for 100 iterations.
+
+        "firefly_algorithm" implements the Firefly Algorithm, using 20
+        fireflies for 100 iterations.
+
+        "simulated_annealing" uses Simulated Annealing.
+
         """, "")),
     ("method", ("""Specify a method for the strategy that searches through
         the parameter space during tuning.
@@ -324,6 +339,11 @@ def tune_kernel(kernel_name, kernel_string, problem_size, arguments,
         raise ValueError("It's not possible to use both sample_fraction in combination with other strategies. " \
                          'Please set strategy=None or strategy="random_sample", when using sample_fraction')
 
+    strategy_map = {"genetic_algorithm": genetic_algorithm,
+                    "pso": pso,
+                    "simulated_annealing": simulated_annealing,
+                    "firefly_algorithm": firefly_algorithm}
+
     if strategy in [None, 'sample_fraction', 'brute_force']:
         if sample_fraction:
             use_strategy = random_sample
@@ -346,6 +366,8 @@ def tune_kernel(kernel_name, kernel_string, problem_size, arguments,
             if not method in ["best1bin", "best1exp", "rand1exp", "randtobest1exp", "best2exp",
                               "rand2exp", "randtobest1bin", "best2bin", "rand2bin", "rand1bin"]:
                 raise ValueError("method option not recognized")
+    elif strategy in strategy_map.keys():
+        use_strategy = strategy_map[strategy]
     else:
         raise ValueError("strategy option not recognized")
     strategy = use_strategy
@@ -474,6 +496,9 @@ def run_kernel(kernel_name, kernel_string, problem_size, arguments,
             results.append(numpy.zeros_like(arg))
             dev.memcpy_dtoh(results[-1], gpu_args[i])
 
+    #trying to make run_kernel work nicely with the Nvidia Visual Profiler
+    del dev
+
     return results
 
 

{kernel_tuner-0.1.8 → kernel_tuner-0.2.0}/kernel_tuner/strategies/basinhopping.py

@@ -55,4 +55,3 @@ def tune(runner, kernel_options, device_options, tuning_options):
         print(opt_result.message)
 
     return results, runner.dev.get_environment()
-

{kernel_tuner-0.1.8 → kernel_tuner-0.2.0}/kernel_tuner/strategies/diff_evo.py

@@ -4,7 +4,7 @@ from __future__ import print_function
 from scipy.optimize import differential_evolution
 from kernel_tuner import util
 
-import kernel_tuner.strategies.minimize as minimize
+from kernel_tuner.strategies.minimize import get_bounds, _cost_func
 
 def tune(runner, kernel_options, device_options, tuning_options):
     """ Find the best performing kernel configuration in the parameter space
@@ -31,40 +31,21 @@ def tune(runner, kernel_options, device_options, tuning_options):
     """
 
     results = []
+    cache = {}
 
+    tuning_options["scaling"] = False
     #build a bounds array as needed for the optimizer
-    bounds = minimize.get_bounds(tuning_options.tune_params)
+    bounds = get_bounds(tuning_options.tune_params)
+
+    args = (kernel_options, tuning_options, runner, results, cache)
 
     #call the differential evolution optimizer
-    opt_result = differential_evolution(_cost_func, bounds, [kernel_options, tuning_options, runner, results],
-                                        maxiter=1, polish=False, disp=tuning_options.verbose)
+    opt_result = differential_evolution(_cost_func, bounds, args, maxiter=1,
+                                        polish=False, disp=tuning_options.verbose)
 
     if tuning_options.verbose:
         print(opt_result.message)
-        print('best config:', minimize.snap_to_nearest_config(opt_result.x, tuning_options.tune_params))
 
     return results, runner.dev.get_environment()
 
 
-
-def _cost_func(x, kernel_options, tuning_options, runner, results):
-    """ Cost function used by the differential evolution optimizer """
-
-    #snap values in x to nearest actual value for each parameter
-    params = minimize.snap_to_nearest_config(x, tuning_options.tune_params)
-
-    #check if this is a legal (non-restricted) parameter instance
-    if tuning_options.restrictions:
-        legal = util.check_restrictions(tuning_options.restrictions, params, tuning_options.tune_params.keys(), tuning_options.verbose)
-        if not legal:
-            return 1e20
-
-    #compile and benchmark this instance
-    res, _ = runner.run([params], kernel_options, tuning_options)
-
-    #append to tuning results
-    if len(res) > 0:
-        results.append(res[0])
-        return res[0]['time']
-
-    return 1e20

kernel_tuner-0.2.0/kernel_tuner/strategies/firefly_algorithm.py

@@ -0,0 +1,116 @@
+""" The strategy that uses the firefly algorithm for optimization"""
+from __future__ import print_function
+import numpy as np
+
+from kernel_tuner.strategies.minimize import _cost_func, get_bounds_x0_eps
+from kernel_tuner.strategies.pso import Particle
+
+def tune(runner, kernel_options, device_options, tuning_options):
+    """ Find the best performing kernel configuration in the parameter space
+
+    :params runner: A runner from kernel_tuner.runners
+    :type runner: kernel_tuner.runner
+
+    :param kernel_options: A dictionary with all options for the kernel.
+    :type kernel_options: dict
+
+    :param device_options: A dictionary with all options for the device
+        on which the kernel should be tuned.
+    :type device_options: dict
+
+    :param tuning_options: A dictionary with all options regarding the tuning
+        process.
+    :type tuning_options: dict
+
+    :returns: A list of dictionaries for executed kernel configurations and their
+        execution times. And a dictionary that contains a information
+        about the hardware/software environment on which the tuning took place.
+    :rtype: list(dict()), dict()
+
+    """
+
+    results = []
+    cache = {}
+
+    #scale variables in x because PSO works with velocities to visit different configurations
+    tuning_options["scaling"] = True
+
+    #using this instead of get_bounds because scaling is used
+    bounds, _, _ = get_bounds_x0_eps(tuning_options)
+
+    args = (kernel_options, tuning_options, runner, results, cache)
+
+    num_particles = 20
+    maxiter = 100
+
+    #parameters needed by the Firefly Algorithm
+    B0 = 1.0
+    gamma = 1.0
+    alpha = 0.20
+
+    best_time_global = 1e20
+    best_position_global = []
+
+    # init particle swarm
+    swarm = []
+    for i in range(0, num_particles):
+        swarm.append(Firefly(bounds, args))
+
+    # compute initial intensities
+    for j in range(num_particles):
+        swarm[j].compute_intensity(_cost_func)
+
+    for c in range(maxiter):
+        if tuning_options.verbose:
+            print("start iteration ", c, "best time global", best_time_global)
+
+        # compare all to all and compute attractiveness
+        for i in range(num_particles):
+            for j in range(num_particles):
+
+                if swarm[i].intensity < swarm[j].intensity:
+                    dist = swarm[i].distance_to(swarm[j])
+                    beta = B0 * np.exp(-gamma * dist * dist)
+
+                    swarm[i].move_towards(swarm[j], beta, alpha)
+                    swarm[i].compute_intensity(_cost_func)
+
+                    # update global best if needed, actually only used for printing
+                    if swarm[i].time <= best_time_global:
+                        best_position_global = swarm[i].position
+                        best_time_global = swarm[i].time
+
+        swarm.sort(key=lambda x: x.time)
+
+    if tuning_options.verbose:
+        print('Final result:')
+        print(best_position_global)
+        print(best_time_global)
+
+    return results, runner.dev.get_environment()
+
+
+class Firefly(Particle):
+    """Firefly object for use in the Firefly Algorithm"""
+
+    def __init__(self, bounds, args):
+        """Create Firefly at random position within bounds"""
+        super().__init__(bounds, args)
+        self.bounds = bounds
+        self.intensity = 1 / self.time
+
+    def distance_to(self, other):
+        """Return Euclidian distance between self and other Firefly"""
+        return np.linalg.norm(self.position-other.position)
+
+    def compute_intensity(self, _cost_func):
+        """Evaluate cost function and compute intensity at this position"""
+        self.evaluate(_cost_func)
+        self.intensity = 1 / self.time
+
+    def move_towards(self, other, beta, alpha):
+        """Move firefly towards another given beta and alpha values"""
+        self.position += beta * (other.position - self.position)
+        self.position += alpha * (np.random.uniform(-0.5, 0.5, len(self.position)))
+        self.position = np.minimum(self.position, [b[1] for b in self.bounds])
+        self.position = np.maximum(self.position, [b[0] for b in self.bounds])