cross_entropy 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 04a027fd6b1ff464e5845bd8a85657fda3cef628
+  data.tar.gz: 1e542c3df5f6a36d2f13c564c3dfcb46d5e2d69f
+SHA512:
+  metadata.gz: 040644e34aed5dbe019789af16ba95226ac4e9c09e8be119d76aff5cf5d30946b286cfe0cdd7c700356217389c4b90060fcb564856e3f618a20fff6830e7fca6
+  data.tar.gz: a57a8845f9e8e3e56bc1c91ccda84bb6961d861fb69ef7f1c8c1b4518e3561b7fcdeb8ff410a19905b4971eba4f9a84a1fcc01d66f8d62896b7e55cca2a11e16

data/README.md

@@ -0,0 +1,99 @@
+# cross_entropy
+
+[![Build Status](https://travis-ci.org/jdleesmiller/cross_entropy.svg?branch=master)](https://travis-ci.org/jdleesmiller/cross_entropy)
+
+https://github.com/jdleesmiller/cross_entropy 
+
+## SYNOPSIS
+
+Implementations of the [Cross Entropy Method](https://en.wikipedia.org/wiki/Cross-entropy_method) for several types of problems. Uses [NArray](http://masa16.github.io/narray/) for the numerics, to achieve reasonable performance.
+
+### What is the Cross Entropy method?
+
+It's basically like a [genetic algorithm](https://en.wikipedia.org/wiki/Genetic_algorithm) without the biological stuff. Instead, it works on nice, pure probability distributions. You start by specifying a probability distribution for the optimal values, based on your initial guess. The CEM then
+- generates samples based on that distribution,
+- scores them according to the objective function, and
+- uses the highest-scoring samples to update the parameters of the probability distribution, so it converges on an optimal value.
+
+It has relatively few tunable parameters, and it automatically balances diversification and intensification. It is robust to noise in the objective function, so it is very useful for parameter tuning and simulation work.
+
+### Supported problem types
+
+- MatrixProblem: For discrete optimisation problems. Each variable can take one of a fixed number of states. The sampling distribution is a defined by a probability mass function for each variable. The term "matrix problem" is based on the idea that we can write the PMFs for each variable into the rows (NArray dimension 1) of a matrix. For example:
+    ```
+               value 1 | value 2
+    variable 1     0.3 | 0.7
+    variable 2     0.9 | 0.1
+    ```
+
+- ContinuousProblem: For continuous unbounded problems. The sampling
+  distribution is a univariate Gaussian.
+
+- BetaProblem: For continous bounded problems. The sampling distribution is a
+  Beta distribution.
+
+### Usage
+
+For example, here is the [Rosenbrock banana function](http://en.wikipedia.org/wiki/Rosenbrock_function) and a custom smooth updater. The function has a global minimum at `(a, a^2)`, but it's hard to find.
+```{ruby}
+# Parameters for the "banana" objective function.
+a = 1.0
+b = 100.0
+
+# Our initial guess at the optimal solution.
+# This is just a guess, so we give it a large standard deviation.
+mean = NArray[0.0, 0.0]
+stddev = NArray[10.0, 10.0]
+
+# Set up the problem. These are the CEM parameters.
+problem = CrossEntropy::ContinuousProblem.new(mean, stddev)
+problem.num_samples = 1000
+problem.num_elite   = 10
+problem.max_iters   = 300
+smooth = 0.1
+
+# Objective function.
+problem.to_score_sample {|x| (a - x[0])**2 + b*(x[1] - x[0]**2)**2 }
+
+# Do some smoothing when updating the parameters based on new samples.
+# This isn't strictly required, but I find it often helps convergence.
+problem.to_update {|new_mean, new_stddev|
+  smooth_mean = smooth*new_mean + (1 - smooth)*problem.param_mean
+  smooth_stddev = smooth*new_stddev + (1 - smooth)*problem.param_stddev
+  [smooth_mean, smooth_stddev]
+}
+
+# It's all calculation from now on...
+problem.solve
+# problems.param_mean => NArray[1.0, 1.0]
+```
+
+## INSTALLATION
+
+    gem install cross_entropy
+
+## LICENSE
+
+(The MIT License)
+
+Copyright (c) 2015 John Lees-Miller
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+

data/lib/cross_entropy.rb

@@ -0,0 +1,10 @@
+require 'cross_entropy/version'
+
+require 'narray'
+
+require 'cross_entropy/narray_extensions'
+require 'cross_entropy/abstract_problem'
+require 'cross_entropy/matrix_problem'
+require 'cross_entropy/continuous_problem'
+require 'cross_entropy/beta_problem'
+

data/lib/cross_entropy/abstract_problem.rb

@@ -0,0 +1,102 @@
+module CrossEntropy
+  #
+  # Base class for specific problem types.
+  #
+  class AbstractProblem
+    #
+    # @param [Array] params
+    #
+    def initialize params
+      @params = params
+
+      @max_iters = nil
+      @track_overall_min = false
+      @overall_min_score = 1.0 / 0.0
+      @overall_min_score_sample = nil
+
+      @generate_samples = proc { raise "no generating function provided" }
+      @score_sample     = proc {|sample| raise "no score function provided" }
+      @estimate         = proc {|elite| raise "no estimate function provided" }
+      @update           = proc {|estimated_params| estimated_params }
+      @stop_decision    = proc {
+        raise "no max_iters provided" unless self.max_iters
+        self.num_iters >= self.max_iters
+      }
+
+      yield(self) if block_given?
+    end
+
+    attr_accessor :params
+
+    attr_accessor :num_samples
+    attr_accessor :num_elite
+    attr_accessor :max_iters
+
+    def to_generate_samples █ @generate_samples = block end
+
+    def to_score_sample █ @score_sample = block end
+
+    def to_estimate █ @estimate = block end
+
+    def to_update █ @update = block end
+
+    def for_stop_decision █ @stop_decision = block end
+
+    attr_reader :num_iters
+    attr_reader :min_score
+    attr_reader :elite_score
+
+    # Keep track of the best sample we've ever seen; if the scoring function is
+    # deterministic, then this is a quantity of major interest.
+    attr_reader :overall_min_score
+    attr_reader :overall_min_score_sample
+    attr_accessor :track_overall_min
+
+    #
+    # Generic cross entropy routine.
+    #
+    def solve
+      @num_iters = 0
+
+      begin
+        @min_score   = nil
+        @elite_score = nil
+
+        samples = @generate_samples.call
+
+        # Score each sample.
+        scores = NArray.float(self.num_samples)
+        for i in 0...self.num_samples
+          sample_i = samples[i,true]
+          score_i  = @score_sample.call(sample_i)
+
+          # Keep track of best ever if requested.
+          if track_overall_min && score_i < overall_min_score
+            @overall_min_score        = score_i
+            @overall_min_score_sample = sample_i
+          end
+
+          scores[i] = score_i
+        end
+
+        # Find elite quantile (gamma).
+        scores_sorted = scores.sort
+        @min_score   = scores_sorted[0]
+        @elite_score = scores_sorted[self.num_elite-1]
+
+        # Take all samples with scores below (or equal to) gamma; note that
+        # there may be more than num_elite, due to ties.
+        elite = samples[(scores <= elite_score).where, true]
+
+        # Compute new parameter estimates.
+        estimated_params = @estimate.call(elite)
+
+        # Update main parameter estimates.
+        self.params = @update.call(estimated_params)
+
+        @num_iters += 1
+      end until @stop_decision.call
+    end
+  end
+end
+

data/lib/cross_entropy/beta_problem.rb

@@ -0,0 +1,75 @@
+module CrossEntropy
+  #
+  # Solve a continuous optimisation problem in which the variables are bounded
+  # to the unit interval, [0, 1]. The sampling distribution of each parameter
+  # is assumed to be a Beta distribution with parameters alpha and
+  # beta.
+  #
+  class BetaProblem < AbstractProblem
+    include NMath
+
+    def initialize alpha, beta
+      super [alpha, beta]
+
+      @generate_samples = proc { self.generate_beta_samples }
+      @estimate         = proc {|elite| self.estimate_mom(elite) }
+
+      yield(self) if block_given?
+    end
+
+    def param_alpha; params[0] end
+    def param_beta; params[1] end
+
+    #
+    # Generate samples.
+    #
+    def generate_beta_samples
+      NArray[*param_alpha.to_a.zip(param_beta.to_a).map {|alpha, beta|
+        generate_beta_sample(alpha, beta)
+      }]
+    end
+
+    #
+    # Method of moments estimate using only the given 'elite' solutions.
+    #
+    # Maximum likelihood estimates for the parameters of the beta distribution
+    # are difficult to compute, so we use the method of moments instead; see
+    # http://www.itl.nist.gov/div898/handbook/eda/section3/eda366h.htm
+    # for more information.
+    #
+    # @param [NArray] elite elite samples; dimension 0 is the sample index; the
+    #        remaining dimensions contain the samples
+    #
+    # @return [Array] the estimated parameter arrays
+    #
+    def estimate_mom elite
+      mean = elite.mean(0)
+      variance = elite.stddev(0)**2
+
+      q = mean * (1.0 - mean)
+      valid = 0 < variance && variance < q
+      r = q[valid] / variance[valid] - 1
+
+      alpha = NArray[*param_alpha.map(&:to_f)]
+      alpha[valid] = mean[valid] * r
+
+      beta = NArray[*param_beta.map(&:to_f)]
+      beta[valid] = (1.0 - mean[valid]) * r
+
+      [alpha, beta]
+    end
+
+    private
+
+    def generate_erlang_samples k
+      -log(NArray.float(k, num_samples).random).sum(0)
+    end
+
+    def generate_beta_sample alpha, beta
+      a = generate_erlang_samples(alpha)
+      b = generate_erlang_samples(beta)
+      a / (a + b)
+    end
+  end
+end
+

data/lib/cross_entropy/continuous_problem.rb

@@ -0,0 +1,44 @@
+module CrossEntropy
+  #
+  # Solve a continuous optimisation problem. The sampling distribution of each
+  # parameter is assumed to be a 1D Gaussian with given mean and variance.
+  #
+  class ContinuousProblem < AbstractProblem
+    def initialize mean, stddev
+      super [mean, stddev]
+
+      @generate_samples = proc { self.generate_gaussian_samples }
+      @estimate         = proc {|elite| self.estimate_ml(elite) }
+
+      yield(self) if block_given?
+    end
+
+    def param_mean; params[0] end
+    def param_stddev; params[1] end
+
+    def sample_shape; param_mean.shape end
+
+    #
+    # Generate samples.
+    #
+    def generate_gaussian_samples
+      r = NArray.float(num_samples, *sample_shape).randomn
+      mean = param_mean.reshape(1, *sample_shape)
+      stddev = param_stddev.reshape(1, *sample_shape)
+      mean + stddev * r
+    end
+
+    #
+    # Maximum likelihood estimate using only the given 'elite' solutions.
+    #
+    # @param [NArray] elite elite samples; dimension 0 is the sample index; the
+    #        remaining dimensions contain the samples
+    #
+    # @return [Array] the estimated parameter arrays
+    #
+    def estimate_ml elite
+      [elite.mean(0), elite.stddev(0)]
+    end
+  end
+end
+

data/lib/cross_entropy/matrix_problem.rb

@@ -0,0 +1,79 @@
+module CrossEntropy
+  #
+  # Assuming that the data are probabilities in an NArray (say dim 1 or dim 2
+  # for now). Rows (NArray dimension 1) must sum to one. Columns (NArray
+  # dimension 0) represent the quantities to be optimized.
+  #
+  # Caller should set seed with NArray.srand before calling.
+  #
+  class MatrixProblem < AbstractProblem
+    using NArrayExtensions
+
+    def initialize(params = nil)
+      super(params)
+
+      # Configurable procs.
+      @generate_samples = proc { self.generate_samples_directly }
+      @estimate         = proc {|elite|  self.estimate_ml(elite) }
+      @update           = proc {|pr_est| pr_est }
+    end
+
+    def num_variables; @params.shape[1] end
+    def num_values;    @params.shape[0] end
+
+    #
+    # Generate samples directly from the probabilities matrix {#pr}.
+    #
+    # If your problem is tightly constrained, you may want to provide a custom
+    # sample generation routine that avoids infeasible solutions; see
+    # {#to_generate_samples}.
+    #
+    def generate_samples_directly
+      self.params.tile(1,1,self.num_samples).sample_pmf_dim.transpose(1,0)
+    end
+
+    #
+    # Maximum likelihood estimate using only the given 'elite' solutions.
+    #
+    # This is often (but not always) the optimal estimate for the probabilities
+    # from the elite samples for problems of this form.
+    #
+    # @param [NArray] elite {#num_variables} rows; the number of columns depends
+    # on the {#num_elite} parameter, but is typically less than {#num_samples};
+    # elements are integer in [0, {#num_values})
+    #
+    # @return [NArray] {#num_variables} rows; {#num_values} columns; entries are
+    # non-negative floats in [0,1] and sum to 1
+    #
+    def estimate_ml elite
+      pr_est = NArray.float(self.num_values, self.num_variables)
+      for i in 0...num_variables
+        elite_i = elite[true,i]
+        for j in 0...num_values
+          pr_est[j,i] = elite_i.eq(j).count_true
+        end
+      end
+      pr_est /= elite.shape[0]
+      pr_est
+    end
+
+    #
+    # Find most likely solution so far based on given probabilities.
+    #
+    # @param [NArray] pr probability matrix with {#num_variables} rows and
+    # {#num_values} columns; if not specified, the current {#pr} matrix is used
+    #
+    # @return [Narray] column vector with {#num_variables} integer entries in
+    # [0, {#num_values})
+    #
+    def most_likely_solution pr=self.params
+      pr_eq = pr.eq(pr.max(0).tile(1,pr.shape[0]).transpose(1,0))
+      pr_ml = NArray.int(pr_eq.shape[1])
+      for i in 0...pr_eq.shape[1]
+        pr_ml[i] = pr_eq[true,i].where[0]
+      end
+      pr_ml
+    end
+  end
+end
+

data/lib/cross_entropy/narray_extensions.rb

@@ -0,0 +1,230 @@
+module CrossEntropy
+  #
+  # Some extensions to NArray.
+  #
+  # Note that I've opened a pull request for general cumsum and tile, but it's
+  # still open without comment after three years, so maybe they don't like them.
+  # https://github.com/masa16/narray/pull/7
+  #
+  module NArrayExtensions
+    refine NArray do
+      #
+      # Cumulative sum along dimension +dim+; modifies this array in place.
+      #
+      # @param [Number] dim non-negative
+      #
+      # @return [NArray] self
+      #
+      def cumsum_general! dim=0
+        if self.dim > dim
+          if self.dim == 1
+            # use the built-in version for dimension 1
+            self.cumsum_1!
+          else
+            # for example, if this is a matrix and dim = 0, mask_0 selects the
+            # first column of the matrix and mask_1 selects the second column;
+            # then we just shuffle them along and accumulate.
+            mask_0 = (0...self.dim).map{|d| d == dim ? 0 : true}
+            mask_1 = (0...self.dim).map{|d| d == dim ? 1 : true}
+            while mask_1[dim] < self.shape[dim]
+              self[*mask_1] += self[*mask_0]
+              mask_0[dim] += 1
+              mask_1[dim] += 1
+            end
+          end
+        end
+        self
+      end
+
+      #
+      # Cumulative sum along dimension +dim+.
+      #
+      # @param [Number] dim non-negative
+      #
+      # @return [NArray]
+      #
+      def cumsum_general dim=0
+        self.dup.cumsum_general!(dim)
+      end
+
+      # The built-in cumsum only does vectors (dim 1).
+      alias cumsum_1 cumsum
+      alias cumsum cumsum_general
+      alias cumsum_1! cumsum!
+      alias cumsum! cumsum_general!
+
+      #
+      # Replicate this array to make a tiled array; this is the matlab function
+      # repmat.
+      #
+      # @param [Array<Number>] reps number of times to repeat in each dimension;
+      # note that reps.size is allowed to be different from self.dim, and
+      # dimensions of size 1 will be added to compensate
+      #
+      # @return [NArray] with same typecode as self
+      #
+      def tile *reps
+        if self.dim == 0 || reps.member?(0)
+          # Degenerate case: 0 dimensions or dimension 0
+          res = NArray.new(self.typecode, 0)
+        else
+          if reps.size <= self.dim
+            # Repeat any extra dims once.
+            reps = reps + [1]*(self.dim - reps.size)
+            tile = self
+          else
+            # Have to add some more dimensions (with implicit shape[dim] = 1).
+            tile_shape = self.shape + [1]*(reps.size - self.dim)
+            tile = self.reshape(*tile_shape)
+          end
+
+          # Allocate tiled matrix.
+          res_shape = (0...tile.dim).map{|i| tile.shape[i] * reps[i]}
+          res = NArray.new(self.typecode, *res_shape)
+
+          # Copy tiles.
+          # This probably isn't the most efficient way of doing this; just doing
+          # res[] = tile doesn't seem to work in general
+          nested_for_zero_to(reps) do |tile_pos|
+            tile_slice = (0...tile.dim).map{|i|
+              (tile.shape[i] * tile_pos[i])...(tile.shape[i] * (tile_pos[i]+1))}
+            res[*tile_slice] = tile
+          end
+        end
+        res
+      end
+
+      #
+      # Convert a linear (1D) index into subscripts for an array with the given
+      # shape; this is the matlab function ind2sub.
+      #
+      # (TODO: There must be a function in NArray to do this, but I can't find
+      # it.)
+      #
+      # @param [Integer] index non-negative
+      #
+      # @return [Array<Integer>] subscript corresponding to the given linear
+      #         index; this is the same size as +shape+
+      #
+      def index_to_subscript index
+        raise IndexError.new("out of bounds: index=#{index} for shape=#{
+          self.shape.inspect}") if index >= self.size
+
+        self.shape.map {|s| index, r = index.divmod(s); r }
+      end
+
+      #
+      # Sample from an array that represents an empirical probability mass
+      # function (pmf). It is assumed that this is an array of probabilities,
+      # and that the sum over the whole array is one (up to rounding error). An
+      # index into the array is chosen in proportion to its probability.
+      #
+      # @example select a subscript uniform-randomly
+      #   NArray.float(3,3,3).fill!(1).div!(3*3*3).sample_pmf #=> [2, 2, 0]
+      #
+      # @param [NArray] r if you have already generated the random sample, you
+      #        can pass it in here; if nil, a random sample will be generated;
+      #        this is used for testing; must be have shape <tt>[1]</tt> if
+      #        specified
+      #
+      # @return [Array<Integer>] subscripts of a randomly selected into the
+      #         array; this is the same size as +shape+
+      #
+      def sample_pmf r=nil
+        self.index_to_subscript(self.flatten.sample_pmf_dim(0, r))
+      end
+
+      #
+      # Sample from an array in which the given dimension, +dim+, represents an
+      # empirical probability mass function (pmf). It is assumed that the
+      # entries along +dim+ are probabilities that sum to one (up to rounding
+      # error).
+      #
+      # @example a matrix in which dim 0 sums to 1
+      #   NArray[[0.1,0.2,0.7],
+      #          [0.3,0.5,0.2],
+      #          [0.0,0.2,0.8],
+      #          [0.7,0.1,0.2]].sample_pmf(1)
+      #   #=> NArray.int(2) [ 1, 1, 2, 0 ] # random indices into dimension 1
+      #
+      # @param [Integer] dim dimension to sample along
+      #
+      # @param [NArray] r if you have already generated the random sample, you
+      #        can pass it in here; if nil, a random sample will be generated;
+      #        this is used for testing; see also sample_cdf_dim
+      #
+      # @return [NArray] integer subscripts
+      #
+      def sample_pmf_dim dim=0, r=nil
+        self.cumsum(dim).sample_cdf_dim(dim, r)
+      end
+
+      #
+      # Sample from an array in which the given dimension, +dim+, represents an
+      # empirical cumulative distribution function (cdf). It is assumed that the
+      # entries along +dim+ are sums of probabilities, and that the last entry
+      # along dim should be 1 (up to rounding error)
+      #
+      # @param [Integer] dim dimension to sample along
+      #
+      # @param [NArray] r if you have already generated the random sample, you
+      #        can pass it in here; if nil, a random sample will be generated;
+      #        this is used for testing; see also sample_cdf_dim
+      #
+      # @return [NArray] integer subscripts
+      #
+      def sample_cdf_dim dim=0, r=nil
+        raise 'self.dim must be > dim' unless self.dim > dim
+
+        # generate random sample, unless one was given for testing
+        r_shape = (0...self.dim).map {|i| i == dim ? 1 : self.shape[i]}
+        r = NArray.new(self.typecode, *r_shape).random! unless r
+
+        # allocate space for results -- same size as the random sample
+        res = NArray.int(*r_shape)
+
+        # for every other dimension, look for the first element that is over the
+        # threshold
+        nested_for_zero_to(r_shape) do |slice|
+          r_thresh    = r[*slice]
+          res[*slice] = self.shape[dim] - 1 # default to last
+          self_slice = slice.dup
+          for self_slice[dim] in 0...self.shape[dim]
+            if r_thresh < self[*self_slice]
+              res[*slice] = self_slice[dim]
+              break
+            end
+          end
+        end
+
+        res[*(0...self.dim).map {|i| i == dim ? 0 : true}]
+      end
+
+      private
+
+      #
+      # This is effectively <tt>suprema.size</tt> nested 'for' loops, in which
+      # the outermost loop runs over <tt>0...suprema.first</tt>, and the
+      # innermost loop runs over <tt>0...suprema.last</tt>.
+      #
+      # For example, when +suprema+ is [3], it yields [0], [1] and [2], and when
+      # +suprema+ is [3,2] it yields [0,0], [0,1], [1,0], [1,1], [2,0] and
+      # [2,1].
+      #
+      # @param [Array<Integer>] suprema non-negative entries; does not yield if
+      #        empty
+      #
+      # @return [nil]
+      #
+      def nested_for_zero_to suprema
+        unless suprema.empty?
+          nums = suprema.map{|n| (0...n).to_a}
+          nums.first.product(*nums.drop(1)).each do |num|
+            yield num
+          end
+        end
+        nil
+      end
+    end
+  end
+end

data/lib/cross_entropy/version.rb

@@ -0,0 +1,6 @@
+module CrossEntropy
+  VERSION_MAJOR = 1
+  VERSION_MINOR = 0
+  VERSION_PATCH = 0
+  VERSION = [VERSION_MAJOR, VERSION_MINOR, VERSION_PATCH].join('.')
+end

data/test/cross_entropy/beta_problem_test.rb

@@ -0,0 +1,47 @@
+require 'cross_entropy'
+require 'minitest/autorun'
+
+class TestBetaProblem < MiniTest::Test
+  # tolerance for numerical comparisons
+  DELTA = 1e-3
+
+  def assert_narray_close exp, obs
+    assert exp.shape == obs.shape && ((exp - obs).abs < DELTA).all?,
+      "#{exp.inspect} expected; got\n#{obs.inspect}"
+  end
+
+  #
+  # See http://en.wikipedia.org/wiki/Rosenbrock_function
+  #
+  # The function has a global minimum at $(a, a^2)$, but it's hard to find.
+  #
+  def test_rosenbrock_banana
+    NArray.srand(567) # must use NArray's generator, not Ruby's
+
+    a = 0.5
+    b = 100.0
+    smooth = 0.1
+
+    alpha = NArray[1.0, 1.0]
+    beta = NArray[1.0, 1.0]
+
+    problem = CrossEntropy::BetaProblem.new(alpha, beta)
+    problem.num_samples = 1000
+    problem.num_elite   = 10
+    problem.max_iters   = 10
+
+    problem.to_score_sample {|x| (a - x[0])**2 + b*(x[1] - x[0]**2)**2 }
+
+    problem.to_update {|new_alpha, new_beta|
+      smooth_alpha = smooth*new_alpha + (1 - smooth)*problem.param_alpha
+      smooth_beta = smooth*new_beta + (1 - smooth)*problem.param_beta
+      [smooth_alpha, smooth_beta]
+    }
+
+    problem.solve
+
+    estimates = problem.param_alpha / (problem.param_alpha + problem.param_beta)
+    assert_narray_close NArray[0.5, 0.25], estimates
+    assert problem.num_iters <= problem.max_iters
+  end
+end

data/test/cross_entropy/continuous_problem_test.rb

@@ -0,0 +1,78 @@
+require 'cross_entropy'
+require 'minitest/autorun'
+
+class TestContinuousProblem < MiniTest::Test
+  # tolerance for numerical comparisons
+  DELTA = 1e-6
+
+  include NMath
+
+  def assert_narray_close exp, obs
+    assert exp.shape == obs.shape && ((exp - obs).abs < DELTA).all?,
+      "#{exp.inspect} expected; got\n#{obs.inspect}"
+  end
+
+  #
+  # Example 3.1 from Kroese et al. 2006.
+  #
+  # Maximise $e^{-(x-2)^2} + 0.8 e^{−(x+2)^2}$ for real $x$. The function has a
+  # global maximum at x = 2 and a local maximum at x = -2, which we should
+  # avoid.
+  #
+  # (This is also the example on Wikipedia.)
+  #
+  def test_Kroese_3_1
+    NArray.srand(567) # must use NArray's generator, not Ruby's
+
+    mean = NArray[0.0]
+    stddev = NArray[10.0]
+
+    problem = CrossEntropy::ContinuousProblem.new(mean, stddev)
+    problem.num_samples = 100
+    problem.num_elite   = 10
+    problem.max_iters   = 100
+
+    # NB: maximising
+    problem.to_score_sample {|x| -(exp(-(x-2)**2) + 0.8 * exp(-(x+2)**2)) }
+
+    problem.solve
+
+    assert_narray_close NArray[2.0], problem.param_mean
+    assert problem.num_iters <= problem.max_iters
+  end
+
+  #
+  # See http://en.wikipedia.org/wiki/Rosenbrock_function
+  #
+  # The function has a global minimum at $(a, a^2)$, but it's hard to find.
+  #
+  def test_rosenbrock_banana
+    NArray.srand(567) # must use NArray's generator, not Ruby's
+
+    a = 1.0
+    b = 100.0
+    smooth = 0.1
+
+    mean = NArray[0.0, 0.0]
+    stddev = NArray[10.0, 10.0]
+
+    problem = CrossEntropy::ContinuousProblem.new(mean, stddev)
+    problem.num_samples = 1000
+    problem.num_elite   = 10
+    problem.max_iters   = 300
+
+    problem.to_score_sample {|x| (a - x[0])**2 + b*(x[1] - x[0]**2)**2 }
+
+    problem.to_update {|new_mean, new_stddev|
+      smooth_mean = smooth*new_mean + (1 - smooth)*problem.param_mean
+      smooth_stddev = smooth*new_stddev + (1 - smooth)*problem.param_stddev
+      [smooth_mean, smooth_stddev]
+    }
+
+    problem.solve
+
+    assert_narray_close NArray[1.0, 1.0], problem.param_mean
+    assert problem.num_iters <= problem.max_iters
+  end
+end
+

data/test/cross_entropy/cross_entropy_test.rb

@@ -0,0 +1,149 @@
+require 'cross_entropy'
+require 'minitest/autorun'
+
+class TestCrossEntropy < MiniTest::Test
+  # tolerance for numerical comparisons
+  DELTA = 1e-6
+
+  def assert_narray_close exp, obs
+    assert exp.shape == obs.shape && ((exp - obs).abs < DELTA).all?,
+      "#{exp.inspect} expected; got\n#{obs.inspect}"
+  end
+
+  def test_ce_estimate_ml
+    mp = CrossEntropy::MatrixProblem.new
+    mp.params        = NArray.float(2, 4).fill!(0.5)
+    mp.num_samples   = 50
+    mp.num_elite     = 3
+
+    # Note that the number of columns in elite can be > num_elite due to ties.
+    elite = NArray[[1,0,0,0],
+                   [0,1,0,0],
+                   [0,0,1,0],
+                   [0,0,0,1]]
+    pr_est = mp.estimate_ml(elite)
+    assert_equal [2, 4], pr_est.shape
+    assert_narray_close NArray[[0.75, 0.25],
+                               [0.75, 0.25],
+                               [0.75, 0.25],
+                               [0.75, 0.25]], pr_est
+
+    # All samples the same.
+    elite = NArray[[0,0,0,0],
+                   [1,1,1,1],
+                   [0,0,0,0],
+                   [0,0,0,0]]
+    pr_est = mp.estimate_ml(elite)
+    assert_equal [2, 4], pr_est.shape
+    assert_narray_close NArray[[1.0, 0.0],
+                               [0.0, 1.0],
+                               [1.0, 0.0],
+                               [1.0, 0.0]], pr_est
+  end
+
+  def test_ce_most_likely_solution
+    mp = CrossEntropy::MatrixProblem.new
+    mp.params        = NArray.float(4, 3).fill!(0.25)
+    mp.num_samples   = 50
+    mp.num_elite     = 3
+
+    # When there is a tie, the lowest value is taken.
+    assert_equal NArray[0,0,0], mp.most_likely_solution
+
+    mp.params = NArray[[0.0,0.0,0.0,1.0],
+                       [1.0,0.0,0.0,0.0],
+                       [0.2,0.2,0.2,0.4]]
+    assert_equal NArray[3,0,3], mp.most_likely_solution
+
+    mp.params = NArray[[0.0,0.0,1.0,0.0],
+                       [0.0,1.0,0.0,0.0],
+                       [0.1,0.3,0.4,0.2]]
+    assert_equal NArray[2,1,2], mp.most_likely_solution
+  end
+
+  #
+  # Example 1.2 from de Boer et al. 2005.
+  # The aim is to search for the given Boolean vector y_true.
+  # The MatrixProblem's default estimation rule is equivalent to equation (8).
+  #
+  def test_ce_deBoer_1
+    NArray.srand(567) # must use NArray's generator, not Ruby's
+
+    n = 10
+    y_true = NArray[1,1,1,1,1,0,0,0,0,0]
+
+    mp = CrossEntropy::MatrixProblem.new
+    mp.params        = NArray.float(2, n).fill!(0.5)
+    mp.num_samples   = 50
+    mp.num_elite     = 5
+    mp.max_iters     = 10
+
+    mp.to_score_sample do |sample|
+      y_true.eq(sample).count_false # to be minimized
+    end
+
+    mp.solve
+
+    if y_true != mp.most_likely_solution
+      warn "expected #{y_true}; found #{mp.most_likely_solution}"
+    end
+    assert mp.num_iters <= mp.max_iters
+  end
+
+  #
+  # Example 3.1 from de Boer et al. 2005.
+  # This is a max-cut problem.
+  # We also do some smoothing.
+  #
+  def test_ce_deBoer_2
+    NArray.srand(567) # must use NArray's generator, not Ruby's
+
+    # Cost matrix
+    n = 5
+    c = NArray[[0,1,3,5,6],
+               [1,0,3,6,5],
+               [3,3,0,2,2],
+               [5,6,2,0,2],
+               [6,5,2,2,0]]
+
+    mp = CrossEntropy::MatrixProblem.new
+    mp.params         = NArray.float(2, n).fill!(0.5)
+    mp.params[true,0] = NArray[0.0,1.0] # put vertex 0 in subset 1
+    mp.num_samples    = 50
+    mp.num_elite      = 5
+    mp.max_iters      = 10
+    smooth            = 0.4
+
+    max_cut_score = proc do |sample|
+      weight = 0
+      for i in 0...n
+        for j in 0...n
+          weight += c[j,i] if sample[i] < sample[j]
+        end
+      end
+      -weight # to be minimized
+    end
+    best_cut = NArray[1,1,0,0,0]
+    assert_equal(-15, max_cut_score.call(NArray[1,0,0,0,0]))
+    assert_equal(-28, max_cut_score.call(best_cut))
+
+    mp.to_score_sample(&max_cut_score)
+
+    mp.to_update do |pr_iter|
+      smooth*pr_iter + (1 - smooth)*mp.params
+    end
+
+    mp.for_stop_decision do
+      #p mp.params
+      mp.num_iters >= mp.max_iters
+    end
+
+    mp.solve
+
+    if best_cut != mp.most_likely_solution
+      warn "expected #{best_cut}; found #{mp.most_likely_solution}"
+    end
+    assert mp.num_iters <= mp.max_iters
+  end
+end
+

metadata

@@ -0,0 +1,92 @@
+--- !ruby/object:Gem::Specification
+name: cross_entropy
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- John Lees-Miller
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2016-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: narray
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.6'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.6'
+- !ruby/object:Gem::Dependency
+  name: gemma
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+description: Includes solvers for continuous and discrete multivariate optimisation
+  problems.
+email:
+- jdleesmiller@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files:
+- README.md
+files:
+- README.md
+- lib/cross_entropy.rb
+- lib/cross_entropy/abstract_problem.rb
+- lib/cross_entropy/beta_problem.rb
+- lib/cross_entropy/continuous_problem.rb
+- lib/cross_entropy/matrix_problem.rb
+- lib/cross_entropy/narray_extensions.rb
+- lib/cross_entropy/version.rb
+- test/cross_entropy/beta_problem_test.rb
+- test/cross_entropy/continuous_problem_test.rb
+- test/cross_entropy/cross_entropy_test.rb
+homepage: https://github.com/jdleesmiller/cross_entropy
+licenses: []
+metadata: {}
+post_install_message: 
+rdoc_options:
+- "--main"
+- README.md
+- "--title"
+- cross_entropy-1.0.0 Documentation
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.4.6
+signing_key: 
+specification_version: 4
+summary: Solve optimisation problems with the Cross Entropy Method.
+test_files:
+- test/cross_entropy/beta_problem_test.rb
+- test/cross_entropy/continuous_problem_test.rb
+- test/cross_entropy/cross_entropy_test.rb
+has_rdoc: