rumale 0.13.2 → 0.13.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 948ea0c8e1c7d41704f0259ecd75dc2c4dd3e10f
-  data.tar.gz: 66adfaeb23d85aafc8cdea65bdda628ed1b481a8
+  metadata.gz: '088ba275c0027e5f4a816a681bac8f0ff08d9d9c'
+  data.tar.gz: 61f2d2e2e8a2557eb18a045cfb76cbc36d1876dd
 SHA512:
-  metadata.gz: c424d21b6c49e55606e26d946ea6df05fd5f860914ba016a8c32da82a63865e74be24ce1f82e14cb787352979960aedaae953093854756df81c6ef57079f7ed5
-  data.tar.gz: 04ba83211d4a296fda4f109f92439b76e494ee2ac9321a9d46cf1c5cd4d07a5736e86c9217d180438ea360b0f48564ef03d4dfad5caa8e2c9586e096d703adbc
+  metadata.gz: e13dfbee846fd28b10f8f5fa04b166efad17269a14537c9dcee8ff50f56353ac740549f2e616d418907bcc83cf37ac80834c1d3c2a26da33ce0acaa632416790
+  data.tar.gz: 46c895ce3b5dee436d83887c2d1a028048cc934054d91482c7b3148f6f128d59c9de947ba19cd3a69aef2fc42273579a413257203a7e4ee46b66534acc71866b

data/CHANGELOG.md

@@ -1,3 +1,7 @@
+# 0.13.3
+- Add transformer class for [Kernel PCA](https://yoshoku.github.io/rumale/doc/Rumale/KernelMachine/KernelPCA.html).
+- Add regressor class for [Kernel Ridge](https://yoshoku.github.io/rumale/doc/Rumale/KernelMachine/KernelRidge.html).
+
 # 0.13.2
 - Add preprocessing class for label binarization.
 - Fix to use LabelBinarizer instead of OneHotEncoder.

data/README.md

@@ -11,10 +11,10 @@
 Rumale (**Ru**by **ma**chine **le**arning) is a machine learning library in Ruby.
 Rumale provides machine learning algorithms with interfaces similar to Scikit-Learn in Python.
 Rumale supports Linear / Kernel Support Vector Machine,
-Logistic Regression, Linear Regression, Ridge, Lasso, Factorization Machine,
+Logistic Regression, Linear Regression, Ridge, Lasso, Kernel Ridge, Factorization Machine,
 Naive Bayes, Decision Tree, AdaBoost, Gradient Tree Boosting, Random Forest, Extra-Trees, K-nearest neighbor classifier,
 K-Means, K-Medoids, Gaussian Mixture Model, DBSCAN, SNN, Power Iteration Clustering,
-Mutidimensional Scaling, t-SNE, Principal Component Analysis, and Non-negative Matrix Factorization.
+Mutidimensional Scaling, t-SNE, Principal Component Analysis, Kernel PCA and Non-negative Matrix Factorization.
 
 This project was formerly known as "SVMKit".
 If you are using SVMKit, please install Rumale and replace `SVMKit` constants with `Rumale`.

data/lib/rumale/kernel_machine/kernel_pca.rb

@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+
+require 'rumale/base/base_estimator'
+require 'rumale/base/transformer'
+
+module Rumale
+  module KernelMachine
+    # KernelPCA is a class that implements Kernel Principal Component Analysis.
+    #
+    # @example
+    #   kernel_mat_train = Rumale::PairwiseMetric::rbf_kernel(training_samples)
+    #   kpca = Rumale::KernelMachine::KernelPCA(n_components: 2)
+    #   mapped_traininig_samples = kpca.fit_transform(kernel_mat_train)
+    #
+    #   kernel_mat_test = Rumale::PairwiseMetric::rbf_kernel(test_samples, training_samples)
+    #   mapped_test_samples = kpca.transform(kernel_mat_test)
+    #
+    # *Reference*
+    # - B. Scholkopf, A. Smola, and K-R. Muller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, Vol. 10 (5), pp. 1299--1319, 1998.
+    class KernelPCA
+      include Base::BaseEstimator
+      include Base::Transformer
+
+      # Returns the eigenvalues of the centered kernel matrix.
+      # @return [Numo::DFloat] (shape: [n_components])
+      attr_reader :lambdas
+
+      # Returns the eigenvectros of the centered kernel matrix.
+      # @return [Numo::DFloat] (shape: [n_training_sampes, n_components])
+      attr_reader :alphas
+
+      # Create a new transformer with Kernel PCA.
+      #
+      # @param n_components [Integer] The number of components.
+      def initialize(n_components: 2)
+        check_params_integer(n_components: n_components)
+        @params = {}
+        @params[:n_components] = n_components
+        @alphas = nil
+        @lambdas = nil
+        @row_mean = nil
+        @all_mean = nil
+      end
+
+      # Fit the model with given training data.
+      # To execute this method, Numo::Linalg must be loaded.
+      #
+      # @overload fit(x) -> KernelPCA
+      #   @param x [Numo::DFloat] (shape: [n_training_samples, n_training_samples])
+      #     The kernel matrix of the training data to be used for fitting the model.
+      # @return [KernelPCA] The learned transformer itself.
+      def fit(x, _y = nil)
+        check_sample_array(x)
+        raise ArgumentError, 'Expect the kernel matrix of training data to be square.' unless x.shape[0] == x.shape[1]
+        raise 'KernelPCA#fit requires Numo::Linalg but that is not loaded.' unless enable_linalg?
+
+        n_samples = x.shape[0]
+        @row_mean = x.mean(0)
+        @all_mean = @row_mean.sum.fdiv(n_samples)
+        centered_kernel_mat = x - x.mean(1).expand_dims(1) - @row_mean + @all_mean
+        eig_vals, eig_vecs = Numo::Linalg.eigh(centered_kernel_mat, vals_range: (n_samples - @params[:n_components])...n_samples)
+        @alphas = eig_vecs.reverse(1).dup
+        @lambdas = eig_vals.reverse.dup
+        self
+      end
+
+      # Fit the model with training data, and then transform them with the learned model.
+      # To execute this method, Numo::Linalg must be loaded.
+      #
+      # @overload fit_transform(x) -> Numo::DFloat
+      #   @param x [Numo::DFloat] (shape: [n_samples, n_samples])
+      #     The kernel matrix of the training data to be used for fitting the model and transformed.
+      # @return [Numo::DFloat] (shape: [n_samples, n_components]) The transformed data
+      def fit_transform(x, _y = nil)
+        check_sample_array(x)
+        fit(x).transform(x)
+      end
+
+      # Transform the given data with the learned model.
+      #
+      # @param x [Numo::DFloat] (shape: [n_testing_samples, n_training_samples])
+      #   The kernel matrix between testing samples and training samples to be transformed.
+      # @return [Numo::DFloat] (shape: [n_testing_samples, n_components]) The transformed data.
+      def transform(x)
+        check_sample_array(x)
+        col_mean = x.sum(1) / @row_mean.shape[0]
+        centered_kernel_mat = x - col_mean.expand_dims(1) - @row_mean + @all_mean
+        transform_mat = @alphas.dot((1.0 / Numo::NMath.sqrt(@lambdas)).diag)
+        transformed = centered_kernel_mat.dot(transform_mat)
+        @params[:n_components] == 1 ? transformed[true, 0].dup : transformed
+      end
+
+      # Dump marshal data.
+      # @return [Hash] The marshal data.
+      def marshal_dump
+        { params: @params,
+          row_mean: @row_mean,
+          all_mean: @all_mean,
+          alphas: @alphas,
+          lambdas: @lambdas }
+      end
+
+      # Load marshal data.
+      # @return [nil]
+      def marshal_load(obj)
+        @params = obj[:params]
+        @row_mean = obj[:row_mean]
+        @all_mean = obj[:all_mean]
+        @alphas = obj[:alphas]
+        @lambdas = obj[:lambdas]
+        nil
+      end
+    end
+  end
+end

data/lib/rumale/kernel_machine/kernel_ridge.rb

@@ -0,0 +1,93 @@
+# frozen_string_literal: true
+
+require 'rumale/base/base_estimator'
+require 'rumale/base/regressor'
+
+module Rumale
+  module KernelMachine
+    # KernelRidge is a class that implements kernel ridge regression.
+    #
+    # @example
+    #   kernel_mat_train = Rumale::PairwiseMetric::rbf_kernel(training_samples)
+    #   kridge = Rumale::KernelMachine::KernelRidge.new(reg_param: 1.0)
+    #   kridge.fit(kernel_mat_train, traininig_values)
+    #
+    #   kernel_mat_test = Rumale::PairwiseMetric::rbf_kernel(test_samples, training_samples)
+    #   results = kridge.predict(kernel_mat_test)
+    class KernelRidge
+      include Base::BaseEstimator
+      include Base::Regressor
+
+      # Return the weight vector.
+      # @return [Numo::DFloat] (shape: [n_training_sample, n_outputs])
+      attr_reader :weight_vec
+
+      # Create a new regressor with kernel ridge regression.
+      #
+      # @param reg_param [Float/Numo::DFloat] The regularization parameter.
+      def initialize(reg_param: 1.0)
+        raise TypeError, 'Expect class of reg_param to be Float or Numo::DFloat' unless reg_param.is_a?(Float) || reg_param.is_a?(Numo::DFloat)
+        raise ArgumentError, 'Expect reg_param array to be 1-D arrray' if reg_param.is_a?(Numo::DFloat) && reg_param.shape.size != 1
+        @params = {}
+        @params[:reg_param] = reg_param
+        @weight_vec = nil
+      end
+
+      # Fit the model with given training data.
+      #
+      # @param x [Numo::DFloat] (shape: [n_training_samples, n_training_samples])
+      #   The kernel matrix of the training data to be used for fitting the model.
+      # @param y [Numo::DFloat] (shape: [n_samples, n_outputs]) The taget values to be used for fitting the model.
+      # @return [KernelRidge] The learned regressor itself.
+      def fit(x, y)
+        check_sample_array(x)
+        check_tvalue_array(y)
+        check_sample_tvalue_size(x, y)
+        raise ArgumentError, 'Expect the kernel matrix of training data to be square.' unless x.shape[0] == x.shape[1]
+        raise 'KernelRidge#fit requires Numo::Linalg but that is not loaded.' unless enable_linalg?
+
+        n_samples = x.shape[0]
+
+        if @params[:reg_param].is_a?(Float)
+          reg_kernel_mat = x + Numo::DFloat.eye(n_samples) * @params[:reg_param]
+          @weight_vec = Numo::Linalg.solve(reg_kernel_mat, y, driver: 'sym')
+        else
+          raise ArgumentError, 'Expect y and reg_param to have the same number of elements.' unless y.shape[1] == @params[:reg_param].shape[0]
+          n_outputs = y.shape[1]
+          @weight_vec = Numo::DFloat.zeros(n_samples, n_outputs)
+          n_outputs.times do |n|
+            reg_kernel_mat = x + Numo::DFloat.eye(n_samples) * @params[:reg_param][n]
+            @weight_vec[true, n] = Numo::Linalg.solve(reg_kernel_mat, y[true, n], driver: 'sym')
+          end
+        end
+
+        self
+      end
+
+      # Predict values for samples.
+      #
+      # @param x [Numo::DFloat] (shape: [n_testing_samples, n_training_samples])
+      #     The kernel matrix between testing samples and training samples to predict values.
+      # @return [Numo::DFloat] (shape: [n_samples, n_outputs]) Predicted values per sample.
+      def predict(x)
+        check_sample_array(x)
+        x.dot(@weight_vec)
+      end
+
+      # Dump marshal data.
+      # @return [Hash] The marshal data.
+      def marshal_dump
+        { params: @params,
+          weight_vec: @weight_vec }
+      end
+
+      # Load marshal data.
+      # @return [nil]
+      def marshal_load(obj)
+        @params = obj[:params]
+        @weight_vec = obj[:weight_vec]
+        nil
+      end
+    end
+  end
+end

data/lib/rumale/version.rb

@@ -3,5 +3,5 @@
 # Rumale is a machine learning library in Ruby.
 module Rumale
   # The version of Rumale you are using.
-  VERSION = '0.13.2'
+  VERSION = '0.13.3'
 end

data/lib/rumale.rb

@@ -34,6 +34,8 @@ require 'rumale/linear_model/linear_regression'
 require 'rumale/linear_model/ridge'
 require 'rumale/linear_model/lasso'
 require 'rumale/kernel_machine/kernel_svc'
+require 'rumale/kernel_machine/kernel_pca'
+require 'rumale/kernel_machine/kernel_ridge'
 require 'rumale/polynomial_model/base_factorization_machine'
 require 'rumale/polynomial_model/factorization_machine_classifier'
 require 'rumale/polynomial_model/factorization_machine_regressor'

data/rumale.gemspec

@@ -17,10 +17,10 @@ Gem::Specification.new do |spec|
     Rumale is a machine learning library in Ruby.
     Rumale provides machine learning algorithms with interfaces similar to Scikit-Learn in Python.
     Rumale currently supports Linear / Kernel Support Vector Machine,
-    Logistic Regression, Linear Regression, Ridge, Lasso, Factorization Machine,
+    Logistic Regression, Linear Regression, Ridge, Lasso, Kernel Ridge, Factorization Machine,
     Naive Bayes, Decision Tree, AdaBoost, Gradient Tree Boosting, Random Forest, Extra-Trees, K-nearest neighbor algorithm,
     K-Means, K-Medoids, Gaussian Mixture Model, DBSCAN, SNN, Power Iteration Clustering,
-    Multidimensional Scaling, t-SNE, Principal Component Analysis, and Non-negative Matrix Factorization.
+    Multidimensional Scaling, t-SNE, Principal Component Analysis, Kernel PCA, and Non-negative Matrix Factorization.
   MSG
   spec.homepage      = 'https://github.com/yoshoku/rumale'
   spec.license       = 'BSD-2-Clause'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rumale
 version: !ruby/object:Gem::Version
-  version: 0.13.2
+  version: 0.13.3
 platform: ruby
 authors:
 - yoshoku
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2019-09-06 00:00:00.000000000 Z
+date: 2019-09-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: numo-narray
@@ -126,10 +126,10 @@ description: |
   Rumale is a machine learning library in Ruby.
   Rumale provides machine learning algorithms with interfaces similar to Scikit-Learn in Python.
   Rumale currently supports Linear / Kernel Support Vector Machine,
-  Logistic Regression, Linear Regression, Ridge, Lasso, Factorization Machine,
+  Logistic Regression, Linear Regression, Ridge, Lasso, Kernel Ridge, Factorization Machine,
   Naive Bayes, Decision Tree, AdaBoost, Gradient Tree Boosting, Random Forest, Extra-Trees, K-nearest neighbor algorithm,
   K-Means, K-Medoids, Gaussian Mixture Model, DBSCAN, SNN, Power Iteration Clustering,
-  Multidimensional Scaling, t-SNE, Principal Component Analysis, and Non-negative Matrix Factorization.
+  Multidimensional Scaling, t-SNE, Principal Component Analysis, Kernel PCA, and Non-negative Matrix Factorization.
 email:
 - yoshoku@outlook.com
 executables: []
@@ -196,6 +196,8 @@ files:
 - lib/rumale/evaluation_measure/recall.rb
 - lib/rumale/evaluation_measure/roc_auc.rb
 - lib/rumale/kernel_approximation/rbf.rb
+- lib/rumale/kernel_machine/kernel_pca.rb
+- lib/rumale/kernel_machine/kernel_ridge.rb
 - lib/rumale/kernel_machine/kernel_svc.rb
 - lib/rumale/linear_model/base_linear_model.rb
 - lib/rumale/linear_model/lasso.rb