kernel-tuner 0.1.9__tar.gz → 0.2.0__tar.gz

{kernel_tuner-0.1.9 → kernel_tuner-0.2.0}/PKG-INFO

@@ -1,13 +1,13 @@
-Metadata-Version: 1.1
+Metadata-Version: 2.1
 Name: kernel_tuner
-Version: 0.1.9
+Version: 0.2.0
 Summary: A simple CUDA/OpenCL kernel tuner in Python
 Home-page: http://benvanwerkhoven.github.io/kernel_tuner/
 Author: Ben van Werkhoven
 Author-email: b.vanwerkhoven@esciencecenter.nl
 License: Apache 2.0
-Description: Kernel Tuner: A Search-Optimizing GPU Code Auto-Tuner
-        =========================================================
+Description: Kernel Tuner
+        ============
         
         |Build Status| |Codacy Badge| |Codacy Badge2| |PyPi Badge|
         
@@ -99,6 +99,23 @@ Description: Kernel Tuner: A Search-Optimizing GPU Code Auto-Tuner
         and in the `Kernel Tuner
         documentation pages <http://benvanwerkhoven.github.io/kernel_tuner/index.html>`__.
         
+        Search strategies for tuning
+        ----------------------------
+        
+        Kernel Tuner supports many optimization algorithms to accelerate the auto-tuning process. Currently 
+        implemented search algorithms are: Brute Force (default), Nelder-Mead, Powell, CG, BFGS, L-BFGS-B, TNC, 
+        COBYLA, SLSQP, Random Search, Basinhopping, Differential Evolution, a Genetic Algorithm, Particle Swarm 
+        Optimization, the Firefly Algorithm, and Simulated Annealing.
+        
+        .. image:: doc/gemm-amd-summary.png
+            :width: 100%
+            :align: center
+        
+        Using a search strategy is easy, you only need to specify to ``tune_kernel`` which strategy and method 
+        you would like to use, for example ``strategy="genetic_algorithm"`` or ``strategy="basinhopping", 
+        method="Powell"``. For a full overview of the supported search strategies and methods please see the `user 
+        api documentation <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
+        
         Tuning host and kernel code
         ---------------------------
         
@@ -113,12 +130,6 @@ Description: Kernel Tuner: A Search-Optimizing GPU Code Auto-Tuner
         `documentation <http://benvanwerkhoven.github.io/kernel_tuner/hostcode.html>`__
         for a detailed explanation of the kernel tuner Python script.
         
-        Search strategies for tuning
-        ----------------------------
-        
-        Kernel Tuner supports several strategies: brute_force (default), random_sample, minimize, basinhopping, and diff_evo (differential
-        evolution). Using different strategies is easy, you only need to specify to ``tune_kernel`` which strategy you would like to use, for example: ``strategy="basinhopping", method="TNC"``. For a full overview of the supported search strategies and methods please see the
-        `user api <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
         
         Correctness verification
         ------------------------
@@ -140,23 +151,26 @@ Description: Kernel Tuner: A Search-Optimizing GPU Code Auto-Tuner
         
         Citation
         --------
-        A scientific paper about the Kernel Tuner is in preparation, in the meantime please cite the Kernel Tuner as follows:
+        A scientific paper about Kernel Tuner has been accepted for publication, please cite
+        Kernel Tuner as follows:
         
         .. code:: latex
         
-            @misc{vanwerkhoven2018kerneltuner,
-              author = {Ben van Werkhoven},
-              title = {Kernel Tuner: A Search-Optimizing GPU Code Auto-Tuner},
-              year = {2018}
+            @article{kerneltuner,
+              author  = {Ben van Werkhoven},
+              title   = {Kernel Tuner: A search-optimizing GPU code auto-tuner},
+              journal = {Future Generation Computer Systems},
+              year = {2019},
+              volume  = {90},
+              pages = {347-358},
+              doi = {https://doi.org/10.1016/j.future.2018.08.004},
             }
         
         Related work
         ------------
         
         You may also like `CLTune <https://github.com/CNugteren/CLTune>`__ by
-        Cedric Nugteren. CLTune is a C++ library for kernel tuning and supports
-        various advanced features like machine learning to optimize the time
-        spent on tuning kernels.
+        Cedric Nugteren. CLTune is a C++ library for kernel tuning.
         
         .. |Build Status| image:: https://api.travis-ci.org/benvanwerkhoven/kernel_tuner.svg?branch=master
            :target: https://travis-ci.org/benvanwerkhoven/kernel_tuner
@@ -182,3 +196,9 @@ Classifier: Topic :: Scientific/Engineering
 Classifier: Topic :: Software Development
 Classifier: Topic :: System :: Distributed Computing
 Classifier: Development Status :: 4 - Beta
+Provides-Extra: cuda_opencl
+Provides-Extra: cuda
+Provides-Extra: dev
+Provides-Extra: tutorial
+Provides-Extra: opencl
+Provides-Extra: doc

{kernel_tuner-0.1.9 → kernel_tuner-0.2.0}/README.rst

@@ -1,5 +1,5 @@
-Kernel Tuner: A Search-Optimizing GPU Code Auto-Tuner
-=========================================================
+Kernel Tuner
+============
 
 |Build Status| |Codacy Badge| |Codacy Badge2| |PyPi Badge|
 
@@ -91,6 +91,23 @@ You can find many - more extensive - example codes, in the
 and in the `Kernel Tuner
 documentation pages <http://benvanwerkhoven.github.io/kernel_tuner/index.html>`__.
 
+Search strategies for tuning
+----------------------------
+
+Kernel Tuner supports many optimization algorithms to accelerate the auto-tuning process. Currently 
+implemented search algorithms are: Brute Force (default), Nelder-Mead, Powell, CG, BFGS, L-BFGS-B, TNC, 
+COBYLA, SLSQP, Random Search, Basinhopping, Differential Evolution, a Genetic Algorithm, Particle Swarm 
+Optimization, the Firefly Algorithm, and Simulated Annealing.
+
+.. image:: doc/gemm-amd-summary.png
+    :width: 100%
+    :align: center
+
+Using a search strategy is easy, you only need to specify to ``tune_kernel`` which strategy and method 
+you would like to use, for example ``strategy="genetic_algorithm"`` or ``strategy="basinhopping", 
+method="Powell"``. For a full overview of the supported search strategies and methods please see the `user 
+api documentation <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
+
 Tuning host and kernel code
 ---------------------------
 
@@ -105,12 +122,6 @@ and the
 `documentation <http://benvanwerkhoven.github.io/kernel_tuner/hostcode.html>`__
 for a detailed explanation of the kernel tuner Python script.
 
-Search strategies for tuning
-----------------------------
-
-Kernel Tuner supports several strategies: brute_force (default), random_sample, minimize, basinhopping, and diff_evo (differential
-evolution). Using different strategies is easy, you only need to specify to ``tune_kernel`` which strategy you would like to use, for example: ``strategy="basinhopping", method="TNC"``. For a full overview of the supported search strategies and methods please see the
-`user api <http://benvanwerkhoven.github.io/kernel_tuner/user-api.html>`__.
 
 Correctness verification
 ------------------------
@@ -132,23 +143,26 @@ Please see the `Contributions Guide <http://benvanwerkhoven.github.io/kernel_tun
 
 Citation
 --------
-A scientific paper about the Kernel Tuner is in preparation, in the meantime please cite the Kernel Tuner as follows:
+A scientific paper about Kernel Tuner has been accepted for publication, please cite
+Kernel Tuner as follows:
 
 .. code:: latex
 
-    @misc{vanwerkhoven2018kerneltuner,
-      author = {Ben van Werkhoven},
-      title = {Kernel Tuner: A Search-Optimizing GPU Code Auto-Tuner},
-      year = {2018}
+    @article{kerneltuner,
+      author  = {Ben van Werkhoven},
+      title   = {Kernel Tuner: A search-optimizing GPU code auto-tuner},
+      journal = {Future Generation Computer Systems},
+      year = {2019},
+      volume  = {90},
+      pages = {347-358},
+      doi = {https://doi.org/10.1016/j.future.2018.08.004},
     }
 
 Related work
 ------------
 
 You may also like `CLTune <https://github.com/CNugteren/CLTune>`__ by
-Cedric Nugteren. CLTune is a C++ library for kernel tuning and supports
-various advanced features like machine learning to optimize the time
-spent on tuning kernels.
+Cedric Nugteren. CLTune is a C++ library for kernel tuning.
 
 .. |Build Status| image:: https://api.travis-ci.org/benvanwerkhoven/kernel_tuner.svg?branch=master
    :target: https://travis-ci.org/benvanwerkhoven/kernel_tuner

{kernel_tuner-0.1.9 → kernel_tuner-0.2.0}/kernel_tuner/c.py

@@ -4,6 +4,7 @@ from collections import namedtuple
 import subprocess
 import platform
 import errno
+import re
 import logging
 import ctypes as C
 import _ctypes
@@ -115,25 +116,35 @@ class CFunctions(object):
         if self.lib != None:
             self.cleanup_lib()
 
-        suffix = ".cc"
-
-        if not "extern \"C\"" in kernel_string:
-            kernel_string = "extern \"C\" {\n" + kernel_string + "\n}"
-
         compiler_options = ["-fPIC"]
-        if "#include <omp.h>" in kernel_string:
+
+        #detect openmp
+        if "#include <omp.h>" in kernel_string or "use omp_lib" in kernel_string:
             logging.debug('set using_openmp to true')
             self.using_openmp = True
-            compiler_options.append("-fopenmp")
+            if self.compiler == "pgfortran":
+                compiler_options.append("-mp")
+            else:
+                compiler_options.append("-fopenmp")
 
+        #detect whether to use nvcc as default instead of g++, may overrule an explicitly passed g++
         if ("#include <cuda" in kernel_string) or ("__global__" in kernel_string):
-            if self.nvcc_available:
+            if self.compiler == "g++" and self.nvcc_available:
                 self.compiler = "nvcc"
 
+        #select right suffix based on compiler
+        suffix = ".cc"
+        if self.compiler in ["gfortran", "pgfortran", "ftn", "ifort"]:
+            suffix = ".F90"
         if self.compiler == "nvcc":
             suffix = suffix[:-1] + "u"
             compiler_options = ["-Xcompiler=" + c for c in compiler_options]
 
+        if ".c" in suffix:
+            if not "extern \"C\"" in kernel_string:
+                kernel_string = "extern \"C\" {\n" + kernel_string + "\n}"
+
+        #copy user specified compiler options to current list
         if self.compiler_options:
             compiler_options += self.compiler_options
 
@@ -148,6 +159,17 @@ class CFunctions(object):
         source_file = get_temp_filename(suffix=suffix)
         filename = ".".join(source_file.split(".")[:-1])
 
+        #detect Fortran modules
+        match = re.search(r"\s*module\s+([a-zA-Z_]*)", kernel_string)
+        if match:
+            if self.compiler == "gfortran":
+                kernel_name = "__" + match.group(1) + "_MOD_" + kernel_name
+            elif self.compiler in ["ftn", "ifort"]:
+                kernel_name = match.group(1) + "_mp_" + kernel_name + "_"
+            elif self.compiler == "pgfortran":
+                kernel_name = match.group(1) + "_" + kernel_name + "_"
+
+
         try:
             write_file(source_file, kernel_string)
 
@@ -158,8 +180,8 @@ class CFunctions(object):
             subprocess.check_call([self.compiler, "-c", source_file] + compiler_options + ["-o", filename + ".o"])
             subprocess.check_call([self.compiler, filename + ".o"] + compiler_options + ["-shared", "-o", filename + lib_extension] + lib_args)
 
-            self.lib = numpy.ctypeslib.load_library(filename, '.')
 
+            self.lib = numpy.ctypeslib.load_library(filename, '.')
             func = getattr(self.lib, kernel_name)
             func.restype = C.c_float
 

{kernel_tuner-0.1.9 → kernel_tuner-0.2.0}/kernel_tuner/cuda.py

@@ -47,11 +47,22 @@ class CudaFunctions(object):
         #inspect device properties
         devprops = {str(k): v for (k, v) in self.context.get_device().get_attributes().items()}
         self.max_threads = devprops['MAX_THREADS_PER_BLOCK']
-        self.cc = str(devprops.get('COMPUTE_CAPABILITY_MAJOR','0')) + str(devprops.get('COMPUTE_CAPABILITY_MINOR','0'))
+        cc = str(devprops.get('COMPUTE_CAPABILITY_MAJOR', '0')) + str(devprops.get('COMPUTE_CAPABILITY_MINOR', '0'))
+        if cc == "00":
+            cc = self.context.get_device().compute_capability()
+        self.cc = str(cc[0])+str(cc[1])
         self.iterations = iterations
         self.current_module = None
         self.compiler_options = compiler_options or []
 
+        #select PyCUDA source module
+        if int(self.cc) >= 35:
+            self.source_mod = DynamicSourceModule
+        else:
+            self.source_mod = SourceModule
+        if not self.source_mod:
+            raise ImportError("Error: pycuda not correctly installed, please ensure pycuda is installed on the same CUDA installation as you're using right now")
+
         #collect environment information
         env = dict()
         env["device_name"] = self.context.get_device().name()
@@ -114,16 +125,10 @@ class CudaFunctions(object):
             if self.compiler_options:
                 compiler_options += self.compiler_options
 
-            if int(self.cc) >= 35:
-                source_mod = DynamicSourceModule
-            else:
-                source_mod = SourceModule
-
-            self.current_module = source_mod(kernel_string, options=compiler_options + ["-e", kernel_name],
-                                               arch=('compute_' + self.cc) if self.cc != "00" else None,
-                                               code=('sm_' + self.cc) if self.cc != "00" else None,
-                                               cache_dir=False, no_extern_c=no_extern_c)
-
+            self.current_module = self.source_mod(kernel_string, options=compiler_options + ["-e", kernel_name],
+                                             arch=('compute_' + self.cc) if self.cc != "00" else None,
+                                             code=('sm_' + self.cc) if self.cc != "00" else None,
+                                             cache_dir=False, no_extern_c=no_extern_c)
 
             func = self.current_module.get_function(kernel_name)
             return func

{kernel_tuner-0.1.9 → kernel_tuner-0.2.0}/kernel_tuner/interface.py

@@ -35,7 +35,7 @@ import numpy
 import kernel_tuner.util as util
 import kernel_tuner.core as core
 
-from kernel_tuner.strategies import brute_force, random_sample, diff_evo, minimize, basinhopping
+from kernel_tuner.strategies import brute_force, random_sample, diff_evo, minimize, basinhopping, genetic_algorithm, pso, simulated_annealing, firefly_algorithm
 
 class Options(OrderedDict):
     """read-only class for passing options around"""
@@ -194,6 +194,10 @@ _tuning_options = Options([
             * "random_sample", specify: *sample_fraction*,
             * "minimize" or "basinhopping", specify: *method*,
             * "diff_evo", specify: *method*.
+            * "genetic_algorithm"
+            * "pso"
+            * "firefly_algorithm"
+            * "simulated_annealing"
 
         "brute_force" is the default and iterates over the entire search
         space.
@@ -213,6 +217,17 @@ _tuning_options = Options([
         "rand2exp", "randtobest1bin", "best2bin", "rand2bin", "rand1bin".
         The default is "best1bin".
 
+        "genetic_algorithm" implements a simple Genetic Algorithm, default
+        setting uses a population size of 20 for 100 generations.
+
+        "pso" implements Particle Swarm Optimization, using the default
+        setting of 20 particles for 100 iterations.
+
+        "firefly_algorithm" implements the Firefly Algorithm, using 20
+        fireflies for 100 iterations.
+
+        "simulated_annealing" uses Simulated Annealing.
+
         """, "")),
     ("method", ("""Specify a method for the strategy that searches through
         the parameter space during tuning.
@@ -324,6 +339,11 @@ def tune_kernel(kernel_name, kernel_string, problem_size, arguments,
         raise ValueError("It's not possible to use both sample_fraction in combination with other strategies. " \
                          'Please set strategy=None or strategy="random_sample", when using sample_fraction')
 
+    strategy_map = {"genetic_algorithm": genetic_algorithm,
+                    "pso": pso,
+                    "simulated_annealing": simulated_annealing,
+                    "firefly_algorithm": firefly_algorithm}
+
     if strategy in [None, 'sample_fraction', 'brute_force']:
         if sample_fraction:
             use_strategy = random_sample
@@ -346,6 +366,8 @@ def tune_kernel(kernel_name, kernel_string, problem_size, arguments,
             if not method in ["best1bin", "best1exp", "rand1exp", "randtobest1exp", "best2exp",
                               "rand2exp", "randtobest1bin", "best2bin", "rand2bin", "rand1bin"]:
                 raise ValueError("method option not recognized")
+    elif strategy in strategy_map.keys():
+        use_strategy = strategy_map[strategy]
     else:
         raise ValueError("strategy option not recognized")
     strategy = use_strategy
@@ -474,6 +496,9 @@ def run_kernel(kernel_name, kernel_string, problem_size, arguments,
             results.append(numpy.zeros_like(arg))
             dev.memcpy_dtoh(results[-1], gpu_args[i])
 
+    #trying to make run_kernel work nicely with the Nvidia Visual Profiler
+    del dev
+
     return results
 
 

{kernel_tuner-0.1.9 → kernel_tuner-0.2.0}/kernel_tuner/strategies/basinhopping.py

@@ -55,4 +55,3 @@ def tune(runner, kernel_options, device_options, tuning_options):
         print(opt_result.message)
 
     return results, runner.dev.get_environment()
-

{kernel_tuner-0.1.9 → kernel_tuner-0.2.0}/kernel_tuner/strategies/diff_evo.py

@@ -4,7 +4,7 @@ from __future__ import print_function
 from scipy.optimize import differential_evolution
 from kernel_tuner import util
 
-import kernel_tuner.strategies.minimize as minimize
+from kernel_tuner.strategies.minimize import get_bounds, _cost_func
 
 def tune(runner, kernel_options, device_options, tuning_options):
     """ Find the best performing kernel configuration in the parameter space
@@ -31,40 +31,21 @@ def tune(runner, kernel_options, device_options, tuning_options):
     """
 
     results = []
+    cache = {}
 
+    tuning_options["scaling"] = False
     #build a bounds array as needed for the optimizer
-    bounds = minimize.get_bounds(tuning_options.tune_params)
+    bounds = get_bounds(tuning_options.tune_params)
+
+    args = (kernel_options, tuning_options, runner, results, cache)
 
     #call the differential evolution optimizer
-    opt_result = differential_evolution(_cost_func, bounds, [kernel_options, tuning_options, runner, results],
-                                        maxiter=1, polish=False, disp=tuning_options.verbose)
+    opt_result = differential_evolution(_cost_func, bounds, args, maxiter=1,
+                                        polish=False, disp=tuning_options.verbose)
 
     if tuning_options.verbose:
         print(opt_result.message)
-        print('best config:', minimize.snap_to_nearest_config(opt_result.x, tuning_options.tune_params))
 
     return results, runner.dev.get_environment()
 
 
-
-def _cost_func(x, kernel_options, tuning_options, runner, results):
-    """ Cost function used by the differential evolution optimizer """
-
-    #snap values in x to nearest actual value for each parameter
-    params = minimize.snap_to_nearest_config(x, tuning_options.tune_params)
-
-    #check if this is a legal (non-restricted) parameter instance
-    if tuning_options.restrictions:
-        legal = util.check_restrictions(tuning_options.restrictions, params, tuning_options.tune_params.keys(), tuning_options.verbose)
-        if not legal:
-            return 1e20
-
-    #compile and benchmark this instance
-    res, _ = runner.run([params], kernel_options, tuning_options)
-
-    #append to tuning results
-    if len(res) > 0:
-        results.append(res[0])
-        return res[0]['time']
-
-    return 1e20

kernel_tuner-0.2.0/kernel_tuner/strategies/firefly_algorithm.py

@@ -0,0 +1,116 @@
+""" The strategy that uses the firefly algorithm for optimization"""
+from __future__ import print_function
+import numpy as np
+
+from kernel_tuner.strategies.minimize import _cost_func, get_bounds_x0_eps
+from kernel_tuner.strategies.pso import Particle
+
+def tune(runner, kernel_options, device_options, tuning_options):
+    """ Find the best performing kernel configuration in the parameter space
+
+    :params runner: A runner from kernel_tuner.runners
+    :type runner: kernel_tuner.runner
+
+    :param kernel_options: A dictionary with all options for the kernel.
+    :type kernel_options: dict
+
+    :param device_options: A dictionary with all options for the device
+        on which the kernel should be tuned.
+    :type device_options: dict
+
+    :param tuning_options: A dictionary with all options regarding the tuning
+        process.
+    :type tuning_options: dict
+
+    :returns: A list of dictionaries for executed kernel configurations and their
+        execution times. And a dictionary that contains a information
+        about the hardware/software environment on which the tuning took place.
+    :rtype: list(dict()), dict()
+
+    """
+
+    results = []
+    cache = {}
+
+    #scale variables in x because PSO works with velocities to visit different configurations
+    tuning_options["scaling"] = True
+
+    #using this instead of get_bounds because scaling is used
+    bounds, _, _ = get_bounds_x0_eps(tuning_options)
+
+    args = (kernel_options, tuning_options, runner, results, cache)
+
+    num_particles = 20
+    maxiter = 100
+
+    #parameters needed by the Firefly Algorithm
+    B0 = 1.0
+    gamma = 1.0
+    alpha = 0.20
+
+    best_time_global = 1e20
+    best_position_global = []
+
+    # init particle swarm
+    swarm = []
+    for i in range(0, num_particles):
+        swarm.append(Firefly(bounds, args))
+
+    # compute initial intensities
+    for j in range(num_particles):
+        swarm[j].compute_intensity(_cost_func)
+
+    for c in range(maxiter):
+        if tuning_options.verbose:
+            print("start iteration ", c, "best time global", best_time_global)
+
+        # compare all to all and compute attractiveness
+        for i in range(num_particles):
+            for j in range(num_particles):
+
+                if swarm[i].intensity < swarm[j].intensity:
+                    dist = swarm[i].distance_to(swarm[j])
+                    beta = B0 * np.exp(-gamma * dist * dist)
+
+                    swarm[i].move_towards(swarm[j], beta, alpha)
+                    swarm[i].compute_intensity(_cost_func)
+
+                    # update global best if needed, actually only used for printing
+                    if swarm[i].time <= best_time_global:
+                        best_position_global = swarm[i].position
+                        best_time_global = swarm[i].time
+
+        swarm.sort(key=lambda x: x.time)
+
+    if tuning_options.verbose:
+        print('Final result:')
+        print(best_position_global)
+        print(best_time_global)
+
+    return results, runner.dev.get_environment()
+
+
+class Firefly(Particle):
+    """Firefly object for use in the Firefly Algorithm"""
+
+    def __init__(self, bounds, args):
+        """Create Firefly at random position within bounds"""
+        super().__init__(bounds, args)
+        self.bounds = bounds
+        self.intensity = 1 / self.time
+
+    def distance_to(self, other):
+        """Return Euclidian distance between self and other Firefly"""
+        return np.linalg.norm(self.position-other.position)
+
+    def compute_intensity(self, _cost_func):
+        """Evaluate cost function and compute intensity at this position"""
+        self.evaluate(_cost_func)
+        self.intensity = 1 / self.time
+
+    def move_towards(self, other, beta, alpha):
+        """Move firefly towards another given beta and alpha values"""
+        self.position += beta * (other.position - self.position)
+        self.position += alpha * (np.random.uniform(-0.5, 0.5, len(self.position)))
+        self.position = np.minimum(self.position, [b[1] for b in self.bounds])
+        self.position = np.maximum(self.position, [b[0] for b in self.bounds])

kernel_tuner-0.2.0/kernel_tuner/strategies/genetic_algorithm.py

@@ -0,0 +1,117 @@
+""" A simple genetic algorithm for parameter search """
+from __future__ import print_function
+
+import random
+
+from kernel_tuner.strategies.minimize import _cost_func
+
+def tune(runner, kernel_options, device_options, tuning_options):
+    """ Find the best performing kernel configuration in the parameter space
+
+    :params runner: A runner from kernel_tuner.runners
+    :type runner: kernel_tuner.runner
+
+    :param kernel_options: A dictionary with all options for the kernel.
+    :type kernel_options: kernel_tuner.interface.Options
+
+    :param device_options: A dictionary with all options for the device
+        on which the kernel should be tuned.
+    :type device_options: kernel_tuner.interface.Options
+
+    :param tuning_options: A dictionary with all options regarding the tuning
+        process.
+    :type tuning_options: kernel_tuner.interface.Options
+
+    :returns: A list of dictionaries for executed kernel configurations and their
+        execution times. And a dictionary that contains a information
+        about the hardware/software environment on which the tuning took place.
+    :rtype: list(dict()), dict()
+
+    """
+
+    dna_size = len(tuning_options.tune_params.keys())
+    pop_size = 20
+    generations = 100
+    tuning_options["scaling"] = False
+
+    tune_params = tuning_options.tune_params
+
+    population = random_population(dna_size, pop_size, tune_params)
+
+    best_time = 1e20
+    all_results = []
+    cache = {}
+
+    for generation in range(generations):
+        if tuning_options.verbose:
+            print("Generation %d, best_time %f" % (generation, best_time))
+
+        #determine fitness of population members
+        weighted_population = []
+        for dna in population:
+            time = _cost_func(dna, kernel_options, tuning_options, runner, all_results, cache)
+            weighted_population.append((dna, time))
+        population = []
+
+        #'best_time' is used only for printing
+        if tuning_options.verbose and all_results:
+            best_time = min(all_results, key=lambda x: x["time"])["time"]
+
+        #population is sorted such that better configs have higher chance of reproducing
+        weighted_population.sort(key=lambda x: x[1])
+
+        #crossover and mutate
+        for _ in range(pop_size//2):
+            ind1 = weighted_choice(weighted_population)
+            ind2 = weighted_choice(weighted_population)
+
+            ind1, ind2 = crossover(ind1, ind2)
+
+            population.append(mutate(ind1, dna_size, tune_params))
+            population.append(mutate(ind2, dna_size, tune_params))
+
+    return all_results, runner.dev.get_environment()
+
+
+
+def weighted_choice(population):
+    """Randomly select, fitness determines probability of being selected"""
+    random_number = random.betavariate(1, 2.5) #increased probability of selecting members early in the list
+    #random_number = random.random()
+    ind = int(random_number*len(population))
+    ind = min(max(ind, 0), len(population)-1)
+    return population[ind][0]
+
+def random_population(dna_size, pop_size, tune_params):
+    """create a random population"""
+    population = []
+    for _ in range(pop_size):
+        dna = []
+        for i in range(dna_size):
+            dna.append(random_val(i, tune_params))
+        population.append(dna)
+    return population
+
+def random_val(index, tune_params):
+    """return a random value for a parameter"""
+    key = list(tune_params.keys())[index]
+    return random.choice(tune_params[key])
+
+def mutate(dna, dna_size, tune_params):
+    """Mutate DNA with 1/mutation_chance chance"""
+    dna_out = []
+    mutation_chance = 10
+    for i in range(dna_size):
+        if int(random.random()*mutation_chance) == 1:
+            dna_out.append(random_val(i, tune_params))
+        else:
+            dna_out.append(dna[i])
+    return dna_out
+
+def crossover(dna1, dna2):
+    """crossover dna1 and dna2 at a random index"""
+    pos = int(random.random()*len(dna1))
+    if random.random() < 0.5:
+        return (dna1[:pos]+dna2[pos:], dna2[:pos]+dna1[pos:])
+    else:
+        return (dna2[:pos]+dna1[pos:], dna1[:pos]+dna2[pos:])