svmkit 0.7.3 → 0.8.1

data/.rubocop_todo.yml

@@ -1,58 +0,0 @@
-# This configuration was generated by
-# `rubocop --auto-gen-config`
-# on 2018-06-10 12:21:53 +0900 using RuboCop version 0.57.1.
-# The point is for the user to remove these configuration records
-# one by one as the offenses are removed from the code base.
-# Note that changes in the inspected code, or installation of new
-# versions of RuboCop, may require this file to be generated again.
-
-# Offense count: 2
-# Cop supports --auto-correct.
-Layout/ClosingHeredocIndentation:
-  Exclude:
-    - 'svmkit.gemspec'
-
-# Offense count: 2
-# Cop supports --auto-correct.
-# Configuration parameters: EnforcedStyle.
-# SupportedStyles: auto_detection, squiggly, active_support, powerpack, unindent
-Layout/IndentHeredoc:
-  Exclude:
-    - 'svmkit.gemspec'
-
-# Offense count: 1
-# Cop supports --auto-correct.
-Layout/LeadingBlankLines:
-  Exclude:
-    - 'svmkit.gemspec'
-
-# Offense count: 1
-# Configuration parameters: CountComments, ExcludedMethods.
-Metrics/BlockLength:
-  Max: 29
-
-# Offense count: 3
-Metrics/CyclomaticComplexity:
-  Max: 12
-
-# Offense count: 3
-Metrics/PerceivedComplexity:
-  Max: 13
-
-# Offense count: 1
-# Cop supports --auto-correct.
-# Configuration parameters: EnforcedStyle, UseHashRocketsWithSymbolValues, PreferHashRocketsForNonAlnumEndingSymbols.
-# SupportedStyles: ruby19, hash_rockets, no_mixed_keys, ruby19_no_mixed_keys
-Style/HashSyntax:
-  Exclude:
-    - 'Rakefile'
-
-# Offense count: 6
-# Cop supports --auto-correct.
-# Configuration parameters: EnforcedStyle, ConsistentQuotesInMultiline.
-# SupportedStyles: single_quotes, double_quotes
-Style/StringLiterals:
-  Exclude:
-    - 'Gemfile'
-    - 'Rakefile'
-    - 'bin/console'

data/HISTORY.md

@@ -1,168 +0,0 @@
-# 0.7.3
-- Add class for grid search performing hyperparameter optimization.
-- Add argument validations to Pipeline.
-
-# 0.7.2
-- Add class for Pipeline that constructs chain of transformers and estimators.
-- Fix some typos on document ([#1](https://github.com/yoshoku/SVMKit/pull/1)).
-
-# 0.7.1
-- Fix to use CSV class in parsing libsvm format file.
-- Refactor ensemble estimators.
-
-# 0.7.0
-- Add class for AdaBoost classifier.
-- Add class for AdaBoost regressor.
-
-# 0.6.3
-- Fix bug on setting random seed and max_features parameter of Random Forest estimators.
-
-# 0.6.2
-- Refactor decision tree classes for improving performance.
-
-# 0.6.1
-- Add abstract class for linear estimator with stochastic gradient descent.
-- Refactor linear estimators to use linear esitmator abstract class.
-- Refactor decision tree classes to avoid unneeded type conversion.
-
-# 0.6.0
-- Add class for Principal Component Analysis.
-- Add class for Non-negative Matrix Factorization.
-
-# 0.5.2
-- Add class for DBSCAN clustering.
-
-# 0.5.1
-- Fix bug on class probability calculation of DecisionTreeClassifier.
-
-# 0.5.0
-- Add class for K-Means clustering.
-- Add class for evaluating purity.
-- Add class for evaluating normalized mutual information.
-
-# 0.4.1
-- Add class for linear regressor.
-- Add class for SGD optimizer.
-- Add class for RMSProp optimizer.
-- Add class for YellowFin optimizer.
-- Fix to be able to select optimizer on estimators of LineaModel and PolynomialModel.
-
-# 0.4.0
-## Breaking changes
-
-SVMKit introduces optimizer algorithm that calculates learning rates adaptively
-on each iteration of stochastic gradient descent (SGD).
-While Pegasos SGD runs fast, it sometimes fails to optimize complicated models
-like Factorization Machine.
-To solve this problem, in version 0.3.3, SVMKit introduced optimization with RMSProp on
-FactorizationMachineRegressor, Ridge and Lasso.
-This attempt realized stable optimization of those estimators.
-Following the success of the attempt, author decided to use modern optimizer algorithms
-with all SGD optimizations in SVMKit.
-Through some preliminary experiments, author implemented Nadam as the default optimizer.
-SVMKit plans to add other optimizer algorithms sequentially, so that users can select them.
-
-- Fix to use Nadam for optimization on SVC, SVR, LogisticRegression, Ridge, Lasso, and Factorization Machine estimators.
-  - Combine reg_param_weight and reg_param_bias parameters on Factorization Machine estimators into the unified parameter named reg_param_linear.
-  - Remove init_std paramter on Factorization Machine estimators.
-  - Remove learning_rate, decay, and momentum parameters on Ridge, Lasso, and FactorizationMachineRegressor.
-  - Remove normalize parameter on SVC, SVR, and LogisticRegression.
-
-# 0.3.3
-- Add class for Ridge regressor.
-- Add class for Lasso regressor.
-- Fix bug on gradient calculation of FactorizationMachineRegressor.
-- Fix some documents.
-
-# 0.3.2
-- Add class for Factorization Machine regressor.
-- Add class for Decision Tree regressor.
-- Add class for Random Forest regressor.
-- Fix to support loading and dumping libsvm file with multi-target variables.
-- Fix to require DecisionTreeClassifier on RandomForestClassifier.
-- Fix some mistakes on document.
-
-# 0.3.1
-- Fix bug on decision function calculation of FactorizationMachineClassifier.
-- Fix bug on weight updating process of KernelSVC.
-
-# 0.3.0
-- Add class for Support Vector Regression.
-- Add class for K-Nearest Neighbor Regression.
-- Add class for evaluating coefficient of determination.
-- Add class for evaluating mean squared error.
-- Add class for evaluating mean absolute error.
-- Fix to use min method instead of sort and first methods.
-- Fix cross validation class to be able to use for regression problem.
-- Fix some typos on document.
-- Rename spec filename for Factorization Machine classifier.
-
-# 0.2.9
-- Add predict_proba method to SVC and KernelSVC.
-- Add class for evaluating logarithmic loss.
-- Add classes for Label- and One-Hot- encoding.
-- Add some validator.
-- Fix bug on training data score calculation of cross validation.
-- Fix fit method of SVC for performance.
-- Fix criterion calculation on Decision Tree for performance.
-- Fix data structure of Decision Tree for performance.
-
-# 0.2.8
-- Fix bug on gradient calculation of Logistic Regression.
-- Fix to change accessor of params of estimators to read only.
-- Add parameter validation.
-
-# 0.2.7
-- Fix to support multiclass classifiction into LinearSVC, LogisticRegression, KernelSVC, and FactorizationMachineClassifier
-
-# 0.2.6
-- Add class for Decision Tree classifier.
-- Add class for Random Forest classifier.
-- Fix to use frozen string literal.
-- Refactor marshal dump method on some classes.
-- Introduce Coveralls to confirm test coverage.
-
-# 0.2.5
-- Add classes for Naive Bayes classifier.
-- Fix decision function method on Logistic Regression class.
-- Fix method visibility on RBF kernel approximation class.
-
-# 0.2.4
-- Add class for Factorization Machine classifier.
-- Add classes for evaluation measures.
-- Fix the method for prediction of class probability in Logistic Regression.
-
-# 0.2.3
-- Add class for cross validation.
-- Add specs for base modules.
-- Fix validation of the number of splits when a negative label is given.
-
-# 0.2.2
-- Add data splitter classes for K-fold cross validation.
-
-# 0.2.1
-- Add class for K-nearest neighbors classifier.
-
-# 0.2.0
-- Migrated the linear algebra library to Numo::NArray.
-- Add module for loading and saving libsvm format file.
-
-# 0.1.3
-- Add class for Kernel Support Vector Machine with Pegasos algorithm.
-- Add module for calculating pairwise kernel fuctions and euclidean distances.
-
-# 0.1.2
-- Add the function learning a model with bias term to the PegasosSVC and LogisticRegression classes.
-- Rewrite the document with yard notation.
-
-# 0.1.1
-- Add class for Logistic Regression with SGD optimization.
-- Fix some mistakes on the document.
-
-# 0.1.0
-- Add basic classes.
-- Add an utility module.
-- Add class for RBF kernel approximation.
-- Add class for Support Vector Machine with Pegasos alogrithm.
-- Add class that performs mutlclass classification with one-vs.-rest strategy.
-- Add classes for preprocessing such as min-max scaling, standardization, and L2 normalization.

data/lib/svmkit/base/base_estimator.rb

@@ -1,13 +0,0 @@
-# frozen_string_literal: true
-
-module SVMKit
-  # This module consists of basic mix-in classes.
-  module Base
-    # Base module for all estimators in SVMKit.
-    module BaseEstimator
-      # Return parameters about an estimator.
-      # @return [Hash]
-      attr_reader :params
-    end
-  end
-end

data/lib/svmkit/base/classifier.rb

@@ -1,34 +0,0 @@
-# frozen_string_literal: true
-
-require 'svmkit/validation'
-require 'svmkit/evaluation_measure/accuracy'
-
-module SVMKit
-  module Base
-    # Module for all classifiers in SVMKit.
-    module Classifier
-      # An abstract method for fitting a model.
-      def fit
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-
-      # An abstract method for predicting labels.
-      def predict
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-
-      # Calculate the mean accuracy of the given testing data.
-      #
-      # @param x [Numo::DFloat] (shape: [n_samples, n_features]) Testing data.
-      # @param y [Numo::Int32] (shape: [n_samples]) True labels for testing data.
-      # @return [Float] Mean accuracy
-      def score(x, y)
-        SVMKit::Validation.check_sample_array(x)
-        SVMKit::Validation.check_label_array(y)
-        SVMKit::Validation.check_sample_label_size(x, y)
-        evaluator = SVMKit::EvaluationMeasure::Accuracy.new
-        evaluator.score(y, predict(x))
-      end
-    end
-  end
-end

data/lib/svmkit/base/cluster_analyzer.rb

@@ -1,29 +0,0 @@
-# frozen_string_literal: true
-
-require 'svmkit/validation'
-require 'svmkit/evaluation_measure/purity'
-
-module SVMKit
-  module Base
-    # Module for all clustering algorithms in SVMKit.
-    module ClusterAnalyzer
-      # An abstract method for analyzing clusters and predicting cluster indices.
-      def fit_predict
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-
-      # Calculate purity of clustering result.
-      #
-      # @param x [Numo::DFloat] (shape: [n_samples, n_features]) Testing data.
-      # @param y [Numo::Int32] (shape: [n_samples]) True labels for testing data.
-      # @return [Float] Purity
-      def score(x, y)
-        SVMKit::Validation.check_sample_array(x)
-        SVMKit::Validation.check_label_array(y)
-        SVMKit::Validation.check_sample_label_size(x, y)
-        evaluator = SVMKit::EvaluationMeasure::Purity.new
-        evaluator.score(y, fit_predict(x))
-      end
-    end
-  end
-end

data/lib/svmkit/base/evaluator.rb

@@ -1,13 +0,0 @@
-# frozen_string_literal: true
-
-module SVMKit
-  module Base
-    # Module for all evaluation measures in SVMKit.
-    module Evaluator
-      # An abstract method for evaluation of model.
-      def score
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-    end
-  end
-end

data/lib/svmkit/base/regressor.rb

@@ -1,34 +0,0 @@
-# frozen_string_literal: true
-
-require 'svmkit/validation'
-require 'svmkit/evaluation_measure/r2_score'
-
-module SVMKit
-  module Base
-    # Module for all regressors in SVMKit.
-    module Regressor
-      # An abstract method for fitting a model.
-      def fit
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-
-      # An abstract method for predicting labels.
-      def predict
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-
-      # Calculate the coefficient of determination for the given testing data.
-      #
-      # @param x [Numo::DFloat] (shape: [n_samples, n_features]) Testing data.
-      # @param y [Numo::DFloat] (shape: [n_samples, n_outputs]) Target values for testing data.
-      # @return [Float] Coefficient of determination
-      def score(x, y)
-        SVMKit::Validation.check_sample_array(x)
-        SVMKit::Validation.check_tvalue_array(y)
-        SVMKit::Validation.check_sample_tvalue_size(x, y)
-        evaluator = SVMKit::EvaluationMeasure::R2Score.new
-        evaluator.score(y, predict(x))
-      end
-    end
-  end
-end

data/lib/svmkit/base/splitter.rb

@@ -1,17 +0,0 @@
-# frozen_string_literal: true
-
-module SVMKit
-  module Base
-    # Module for all validation methods in SVMKit.
-    module Splitter
-      # Return the number of splits.
-      # @return [Integer]
-      attr_reader :n_splits
-
-      # An abstract method for splitting dataset.
-      def split
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-    end
-  end
-end

data/lib/svmkit/base/transformer.rb

@@ -1,18 +0,0 @@
-# frozen_string_literal: true
-
-module SVMKit
-  module Base
-    # Module for all transfomers in SVMKit.
-    module Transformer
-      # An abstract method for fitting a model.
-      def fit
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-
-      # An abstract method for fitting a model and transforming given data.
-      def fit_transform
-        raise NotImplementedError, "#{__method__} has to be implemented in #{self.class}."
-      end
-    end
-  end
-end

data/lib/svmkit/clustering/dbscan.rb

@@ -1,127 +0,0 @@
-# frozen_string_literal: true
-
-require 'svmkit/validation'
-require 'svmkit/base/base_estimator'
-require 'svmkit/base/cluster_analyzer'
-require 'svmkit/pairwise_metric'
-
-module SVMKit
-  module Clustering
-    # DBSCAN is a class that implements DBSCAN cluster analysis.
-    # The current implementation uses the Euclidean distance for analyzing the clusters.
-    #
-    # @example
-    #   analyzer = SVMKit::Clustering::DBSCAN.new(eps: 0.5, min_samples: 5)
-    #   cluster_labels = analyzer.fit_predict(samples)
-    #
-    # *Reference*
-    # - M. Ester, H-P. Kriegel, J. Sander, and X. Xu, "A density-based algorithm for discovering clusters in large spatial databases with noise," Proc. KDD' 96, pp. 266--231, 1996.
-    class DBSCAN
-      include Base::BaseEstimator
-      include Base::ClusterAnalyzer
-      include Validation
-
-      # Return the core sample indices.
-      # @return [Numo::Int32] (shape: [n_core_samples])
-      attr_reader :core_sample_ids
-
-      # Return the cluster labels. The negative cluster label indicates that the point is noise.
-      # @return [Numo::Int32] (shape: [n_samples])
-      attr_reader :labels
-
-      # Create a new cluster analyzer with DBSCAN method.
-      #
-      # @param eps [Float] The radius of neighborhood.
-      # @param min_samples [Integer] The number of neighbor samples to be used for the criterion whether a point is a core point.
-      def initialize(eps: 0.5, min_samples: 5)
-        check_params_float(eps: eps)
-        check_params_integer(min_samples: min_samples)
-        @params = {}
-        @params[:eps] = eps
-        @params[:min_samples] = min_samples
-        @core_sample_ids = nil
-        @labels = nil
-      end
-
-      # Analysis clusters with given training data.
-      #
-      # @overload fit(x) -> DBSCAN
-      #
-      # @param x [Numo::DFloat] (shape: [n_samples, n_features]) The training data to be used for cluster analysis.
-      # @return [DBSCAN] The learned cluster analyzer itself.
-      def fit(x, _y = nil)
-        check_sample_array(x)
-        partial_fit(x)
-        self
-      end
-
-      # Analysis clusters and assign samples to clusters.
-      #
-      # @param x [Numo::DFloat] (shape: [n_samples, n_features]) The training data to be used for cluster analysis.
-      # @return [Numo::Int32] (shape: [n_samples]) Predicted cluster label per sample.
-      def fit_predict(x)
-        check_sample_array(x)
-        partial_fit(x)
-        labels
-      end
-
-      # Dump marshal data.
-      # @return [Hash] The marshal data.
-      def marshal_dump
-        { params: @params,
-          core_sample_ids: @core_sample_ids,
-          labels: @labels }
-      end
-
-      # Load marshal data.
-      # @return [nil]
-      def marshal_load(obj)
-        @params = obj[:params]
-        @core_sample_ids = obj[:core_sample_ids]
-        @labels = obj[:labels]
-        nil
-      end
-
-      private
-
-      def partial_fit(x)
-        cluster_id = 0
-        n_samples  = x.shape[0]
-        @core_sample_ids = []
-        @labels = Numo::Int32.zeros(n_samples) - 2
-        n_samples.times do |q|
-          next if @labels[q] >= -1
-          cluster_id += 1 if expand_cluster(x, q, cluster_id)
-        end
-        @core_sample_ids = Numo::Int32[*@core_sample_ids.flatten]
-        nil
-      end
-
-      def expand_cluster(x, query_id, cluster_id)
-        target_ids = region_query(x[query_id, true], x)
-        if target_ids.size < @params[:min_samples]
-          @labels[query_id] = -1
-          false
-        else
-          @labels[target_ids] = cluster_id
-          @core_sample_ids.push(target_ids.dup)
-          target_ids.delete(query_id)
-          while (m = target_ids.shift)
-            neighbor_ids = region_query(x[m, true], x)
-            next if neighbor_ids.size < @params[:min_samples]
-            neighbor_ids.each do |n|
-              target_ids.push(n) if @labels[n] < -1
-              @labels[n] = cluster_id if @labels[n] <= -1
-            end
-          end
-          true
-        end
-      end
-
-      def region_query(query, targets)
-        distance_arr = PairwiseMetric.euclidean_distance(query.expand_dims(0), targets)[0, true]
-        distance_arr.lt(@params[:eps]).where.to_a
-      end
-    end
-  end
-end