lbfgsb 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 10f6e681a7b2d60d8915837bc7001a79d9ea796237d9907651481eeb96618213
+  data.tar.gz: d95ab1a0f2ac0e5d3f040de8309e4c3a9e3b3babe057c02fe553a3702dd3e75c
+SHA512:
+  metadata.gz: 7eae04f38bb591a1a2b13b18c2be747cfccd28551c5ff3cdc5095442f00044d6489407abc1b6db33425cedd5368cff0efc88202b76c5ca27ba3a9a98094ed60d
+  data.tar.gz: e75187c9e10cd4e9670c5f3c3a795509d35307515dbe98c951eb8059fde47eec4e7dc4190fdf80dbbf45104b2d0a4306caee050a3e0b485b30c8451bfc50ad3f

data/.github/workflows/build.yml

@@ -0,0 +1,21 @@
+name: build
+
+on: [push]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        ruby: [ '2.5', '2.6', '2.7' ]
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Ruby ${{ matrix.ruby }}
+        uses: actions/setup-ruby@v1
+        with:
+          ruby-version: ${{ matrix.ruby }}
+      - name: Build and test with Rake
+        run: |
+          gem install bundler
+          bundle install --jobs 4 --retry 3
+          bundle exec rake

data/.gitignore

@@ -0,0 +1,17 @@
+/.bundle/
+/.yardoc
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+*.bundle
+*.so
+*.o
+*.a
+mkmf.log
+iterate.dat
+
+# rspec failure tracking
+.rspec_status

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.yardopts

@@ -0,0 +1 @@
+--exclude ext/lbfgsb/src/

data/Gemfile

@@ -0,0 +1,8 @@
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in lbfgsb.gemspec
+gemspec
+
+gem "rake", "~> 12.0"
+gem "rake-compiler"
+gem "rspec", "~> 3.0"

data/LICENSE.txt

@@ -0,0 +1,27 @@
+Copyright (c) 2020 Atsushi Tatsuma
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the copyright holder nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,110 @@
+# Lbfgsb.rb
+
+[![Build Status](https://github.com/yoshoku/lbfgsb.rb/workflows/build/badge.svg)](https://github.com/yoshoku/lbfgsb.rb/actions?query=workflow%3Abuild)
+[![Gem Version](https://badge.fury.io/rb/lbfgsb.svg)](https://badge.fury.io/rb/lbfgsb)
+[![BSD 3-Clause License](https://img.shields.io/badge/License-BSD%203--Clause-orange.svg)](https://github.com/yoshoku/suika/blob/master/LICENSE.txt)
+
+Lbfgsb.rb is a Ruby binding for [L-BFGS-B](http://users.iems.northwestern.edu/~nocedal/lbfgsb.html)
+that is a limited-memory algorithm for solving large nonlinear optimization problems
+subject to simple bounds on the variables.
+L-BFGS-B is written in FORTRAN. Author converted the codes into C-lang
+and call that with Ruby native exntesion.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'lbfgsb'
+```
+
+And then execute:
+
+    $ bundle install
+
+Or install it yourself as:
+
+    $ gem install lbfgsb
+
+## Usage
+Example 1. Logistic Regression
+
+```ruby
+require 'numo/narray'
+require 'lbfgsb'
+require 'rumale'
+
+
+## Training:
+# Define objective function and its derivative.
+obj_fnc = proc do |w, x, y|
+  Numo::NMath.log(1 + Numo::NMath.exp(-y * x.dot(w))).sum
+end
+
+d_obj_fnc = proc do |w, x, y|
+  (y / (1 + Numo::NMath.exp(-y * x.dot(w))) - y).dot(x)
+end
+
+# Load dataset.
+x, y = Rumale::Dataset.load_libsvm_file('svmguide3')
+
+# Extend variable for intercept.
+xb = Numo::DFloat.hstack([x, Numo::DFloat.ones(x.shape[0]).expand_dims(1)])
+
+# Initialize weight vector.
+w0 = xb.mean(0)
+
+# Optimize weight vector.
+res = Lbfgsb.minimize(fnc: obj_fnc, jcb: d_obj_fnc, x_init: w0, args: [xb, y])
+pp res
+w = res[:x]
+
+
+## Testing:
+# Load dataset.
+x_test, y_test = Rumale::Dataset.load_libsvm_file('svmguide3.t')
+xb_test = Numo::DFloat.hstack([x_test, Numo::DFloat.ones(x_test.shape[0]).expand_dims(1)])
+
+# Predict class label.
+probs = 1.0 / (Numo::NMath.exp(-xb_test.dot(w)) + 1.0)
+predicted = Numo::Int32.zeros(x_test.shape[0])
+predicted[probs >= 0.5] = 1
+predicted[probs <  0.5] =-1
+
+# Evaluate result.
+evaluator = Rumale::EvaluationMeasure::Accuracy.new
+puts("Accuracy: %.1f%%" % (100.0 * evaluator.score(predicted, y_test)))
+```
+
+```sh
+$ gem install rumale
+$ wget https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/svmguide3
+$ wget https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/svmguide3.t
+$ ruby example1.rb
+{:task=>"CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH",
+ :x=>
+  Numo::DFloat#shape=[23]
+[6.36634, -86.6413, -18.9489, -11.4564, -16.6738, -23.9111, -6.19714, ...],
+ :fnc=>512.3210359536802,
+ :jcb=>
+  Numo::DFloat#shape=[23]
+[8.01782e-05, -0.000226254, -0.00144915, 4.26451e-05, 0.000677138, ...],
+ :n_iter=>623,
+ :n_fev=>716,
+ :n_jev=>716,
+ :success=>true}
+Accuracy: 61.0%
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/yoshoku/lbfgsb.
+
+## License
+
+The gem is available as open source under the terms of the [BSD-3-Clause License](https://opensource.org/licenses/BSD-3-Clause).
+In addition, L-BFGS-B carries the following condition for use:
+
+This software is freely available, but we expect that all publications describing  work using this software ,
+or all commercial products using it, quote at least one of the references given below.
+This software is released under the "New BSD License" (aka "Modified BSD License" or "3-clause license").

data/Rakefile

@@ -0,0 +1,15 @@
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+require 'rake/extensiontask'
+
+task :build => :compile
+
+Rake::ExtensionTask.new('lbfgsbext') do |ext|
+  ext.ext_dir = 'ext/lbfgsb'
+  ext.lib_dir = 'lib/lbfgsb'
+end
+
+task :default => [:clobber, :compile, :spec]

data/ext/lbfgsb/extconf.rb

@@ -0,0 +1,35 @@
+require 'mkmf'
+require 'numo/narray'
+
+$LOAD_PATH.each do |lp|
+  if File.exist?(File.join(lp, 'numo/numo/narray.h'))
+    $INCFLAGS = "-I#{lp}/numo #{$INCFLAGS}"
+    break
+  end
+end
+
+unless have_header('numo/narray.h')
+  puts 'numo/narray.h not found.'
+  exit(1)
+end
+
+if RUBY_PLATFORM =~ /mswin|cygwin|mingw/
+  $LOAD_PATH.each do |lp|
+    if File.exist?(File.join(lp, 'numo/libnarray.a'))
+      $LDFLAGS = "-L#{lp}/numo #{$LDFLAGS}"
+      break
+    end
+  end
+  unless have_library('narray', 'nary_new')
+    puts 'libnarray.a not found.'
+    exit(1)
+  end
+end
+
+$srcs = Dir.glob("#{$srcdir}/*.c").map { |path| File.basename(path) }
+$srcs.concat(%w[blas.c linpack.c lbfgsb.c])
+
+$INCFLAGS << " -I$(srcdir)/src"
+$VPATH << "$(srcdir)/src"
+
+create_makefile('lbfgsb/lbfgsbext')

data/ext/lbfgsb/lbfgsbext.c

@@ -0,0 +1,164 @@
+#include "lbfgsbext.h"
+
+VALUE rb_mLbfgsb;
+
+static
+VALUE lbfgsb_min_l_bfgs_b(VALUE self,
+  VALUE fnc, VALUE x_val, VALUE jcb, VALUE args,
+  VALUE l_val, VALUE u_val, VALUE nbd_val,
+  VALUE maxcor, VALUE ftol, VALUE gtol, VALUE maxiter, VALUE disp)
+{
+  long i;
+  long n_iter;
+  long n_fev;
+  long n_jev;
+  long max_iter = NUM2LONG(maxiter);
+  narray_t* x_nary;
+  narray_t* l_nary;
+  narray_t* u_nary;
+  narray_t* nbd_nary;
+  long n;
+  long m = NUM2LONG(maxcor);
+  double *x_ptr;
+  double *l_ptr;
+  double *u_ptr;
+  long *nbd_ptr;
+  double f;
+  double *g;
+  double factr = NUM2DBL(ftol);
+  double pgtol = NUM2DBL(gtol);
+  double* wa;
+  long* iwa;
+  char task[60];
+  long iprint = NIL_P(disp) ? -1 : NUM2LONG(disp);
+  char csave[60];
+  long lsave[4];
+  long isave[44];
+  double dsave[29];
+  double* g_ptr;
+  VALUE g_val;
+  VALUE ret;
+
+  GetNArray(x_val, x_nary);
+  if (NA_NDIM(x_nary) != 1) {
+    rb_raise(rb_eArgError, "x must be a 1-D array.");
+    return Qnil;
+  }
+  n = (long)NA_SIZE(x_nary);
+  if (CLASS_OF(x_val) != numo_cDFloat) {
+    x_val = rb_funcall(numo_cDFloat, rb_intern("cast"), 1, x_val);
+  }
+  if (!RTEST(nary_check_contiguous(x_val))) {
+    x_val = nary_dup(x_val);
+  }
+
+  GetNArray(l_val, l_nary);
+  if (NA_NDIM(l_nary) != 1) {
+    rb_raise(rb_eArgError, "l must be a 1-D array.");
+    return Qnil;
+  }
+  if ((long)NA_SIZE(l_nary) != n) {
+    rb_raise(rb_eArgError, "The size of l must be equal to that of x.");
+    return Qnil;
+  }
+  if (CLASS_OF(l_val) != numo_cDFloat) {
+    l_val = rb_funcall(numo_cDFloat, rb_intern("cast"), 1, l_val);
+  }
+  if (!RTEST(nary_check_contiguous(l_val))) {
+    l_val = nary_dup(l_val);
+  }
+
+  GetNArray(u_val, u_nary);
+  if (NA_NDIM(u_nary) != 1) {
+    rb_raise(rb_eArgError, "u must be a 1-D array.");
+    return Qnil;
+  }
+  if ((long)NA_SIZE(u_nary) != n) {
+    rb_raise(rb_eArgError, "The size of u must be equal to that of x.");
+    return Qnil;
+  }
+  if (CLASS_OF(u_val) != numo_cDFloat) {
+    u_val = rb_funcall(numo_cDFloat, rb_intern("cast"), 1, u_val);
+  }
+  if (!RTEST(nary_check_contiguous(u_val))) {
+    u_val = nary_dup(u_val);
+  }
+
+  GetNArray(nbd_val, nbd_nary);
+  if (NA_NDIM(nbd_nary) != 1) {
+    rb_raise(rb_eArgError, "nbd must be a 1-D array.");
+    return Qnil;
+  }
+  if ((long)NA_SIZE(nbd_nary) != n) {
+    rb_raise(rb_eArgError, "The size of nbd must be equal to that of x.");
+    return Qnil;
+  }
+  if (CLASS_OF(nbd_val) != numo_cInt64) {
+    nbd_val = rb_funcall(numo_cInt64, rb_intern("cast"), 1, nbd_val);
+  }
+  if (!RTEST(nary_check_contiguous(nbd_val))) {
+    nbd_val = nary_dup(nbd_val);
+  }
+
+  x_ptr = (double*)na_get_pointer_for_read_write(x_val);
+  l_ptr = (double*)na_get_pointer_for_read(l_val);
+  u_ptr = (double*)na_get_pointer_for_read(u_val);
+  nbd_ptr = (long*)na_get_pointer_for_read(nbd_val);
+  g = ALLOC_N(double, n);
+  wa = ALLOC_N(double, (2 * m + 5) * n + 12 * m * m + 12 * m);
+  iwa = ALLOC_N(long, 3 * n);
+
+  f = 0.0;
+  for (i = 0; i < n; g[i++] = 0.0);
+  strcpy(task, "START");
+  n_fev = 0;
+  n_jev = 0;
+
+  for (n_iter = 0; n_iter < max_iter;) {
+    setulb_(
+      &n, &m, x_ptr, l_ptr, u_ptr, nbd_ptr, &f, g, &factr, &pgtol, wa, iwa,
+      task, &iprint, csave, lsave, isave, dsave
+    );
+    if (strncmp(task, "FG", 2) == 0) {
+      f = NUM2DBL(rb_funcall(self, rb_intern("fnc"), 3, fnc, x_val, args));
+      g_val = rb_funcall(self, rb_intern("jcb"), 3, jcb, x_val, args);
+      n_fev += 1;
+      n_jev += 1;
+      if (CLASS_OF(g_val) != numo_cDFloat) g_val = rb_funcall(numo_cDFloat, rb_intern("cast"), 1, g_val);
+      if (!RTEST(nary_check_contiguous(g_val))) g_val = nary_dup(g_val);
+      g_ptr = (double*)na_get_pointer_for_read(g_val);
+      for (i = 0; i < n; i++) g[i] = g_ptr[i];
+    } else if (strncmp(task, "NEW_X", 5) == 0) {
+      n_iter += 1;
+      continue;
+    } else {
+      break;
+    }
+  }
+
+  xfree(g);
+  xfree(wa);
+  xfree(iwa);
+
+  ret = rb_hash_new();
+  rb_hash_aset(ret, ID2SYM(rb_intern("task")), rb_str_new_cstr(task));
+  rb_hash_aset(ret, ID2SYM(rb_intern("x")), x_val);
+  rb_hash_aset(ret, ID2SYM(rb_intern("fnc")), DBL2NUM(f));
+  rb_hash_aset(ret, ID2SYM(rb_intern("jcb")), g_val);
+  rb_hash_aset(ret, ID2SYM(rb_intern("n_iter")), LONG2NUM(n_iter));
+  rb_hash_aset(ret, ID2SYM(rb_intern("n_fev")), LONG2NUM(n_fev));
+  rb_hash_aset(ret, ID2SYM(rb_intern("n_jev")), LONG2NUM(n_jev));
+  rb_hash_aset(ret, ID2SYM(rb_intern("success")), strncmp(task, "CONV", 4) == 0 ? Qtrue : Qfalse);
+
+  return ret;
+}
+
+void
+Init_lbfgsbext(void)
+{
+  rb_mLbfgsb = rb_define_module("Lbfgsb");
+  /* The value of double epsilon used in the native extension. */
+  rb_define_const(rb_mLbfgsb, "DBL_EPSILON", DBL2NUM(DBL_EPSILON));
+  /* @!visibility private */
+  rb_define_module_function(rb_mLbfgsb, "min_l_bfgs_b", lbfgsb_min_l_bfgs_b, 12);
+}

data/ext/lbfgsb/lbfgsbext.h

@@ -0,0 +1,12 @@
+#ifndef LBFGSB_RB_H
+#define LBFGSB_RB_H 1
+
+#include <float.h>
+
+#include <ruby.h>
+#include <numo/narray.h>
+#include <numo/template.h>
+
+#include <lbfgsb.h>
+
+#endif /* LBFGSB_RB_H */

data/ext/lbfgsb/src/License.txt

@@ -0,0 +1,71 @@
+3-clause license ("New BSD License" or "Modified BSD License")
+New BSD License
+Author	Regents of the University of California
+Publisher	Public Domain
+Published	July 22, 1999[8]
+DFSG compatible	Yes[7]
+FSF approved	Yes[1]
+OSI approved	Yes[3]
+GPL compatible	Yes[1]
+Copyleft	No[1]
+Copyfree	Yes
+Linking from code with a different license	Yes
+
+The advertising clause was removed from the license text in the official BSD on July 22, 1999 by William Hoskins, Director of 
+the Office of Technology Licensing for UC Berkeley.[8] Other BSD distributions removed the clause, but many similar clauses 
+remain in BSD-derived code from other sources, and unrelated code using a derived license.
+
+While the original license is sometimes referred to as "BSD-old", the resulting 3-clause version is sometimes referred to by 
+"BSD-new." Other names include "New BSD", "revised BSD", "BSD-3", or "3-clause BSD". This version has been vetted as 
+an Open source license by the OSI as the "The BSD License".[3] The Free Software Foundation, which refers to the license 
+as the "Modified BSD License", states that it is compatible with the GNU GPL. The FSF encourages users to be specific 
+when referring to the license by name (i.e. not simply referring to it as "a BSD license" or "BSD-style") to avoid confusion with 
+the original BSD license.[1]
+
+This version allows unlimited redistribution for any purpose as long as its copyright notices and the license's disclaimers of 
+warranty are maintained. The license also contains a clause restricting use of the names of contributors for endorsement of a 
+derived work without specific permission.
+
+Copyright (c) <year>, <copyright holder>
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+    * Redistributions of source code must retain the above copyright
+      notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+      notice, this list of conditions and the following disclaimer in the
+      documentation and/or other materials provided with the distribution.
+    * Neither the name of the <organization> nor the
+      names of its contributors may be used to endorse or promote products
+      derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+References
+
+1. ^ a b c d e "Various Licenses and Comments about Them - GNU Project - Free Software Foundation (FSF): Modified BSD license". 
+     Free Software Foundation. Retrieved 02 October 2010.
+2. ^ a b c d e "Various Licenses and Comments about Them - GNU Project - Free Software Foundation (FSF): FreeBSD license". 
+     Free Software Foundation. Retrieved 02 October 2010.
+3. ^ a b c d e f "Open Source Initiative OSI - The BSD License:Licensing". Open Source Initiative. Retrieved 06 December 2009.
+4. ^ a b c d "Various Licenses and Comments about Them - GNU Project - Free Software Foundation (FSF): Original BSD license". 
+     Free Software Foundation. Retrieved 02 October 2010.
+5. ^ The year given is the year 4.3BSD-Tahoe was released. Whether this is the first use of the license is not known.
+6. ^ The year given is the year 4.3BSD-Reno was released. Whether this is the first use of the license is not known.
+7. ^ a b "Debian -- License information". Debian. Retrieved 18 February 2010.
+8. ^ a b c "To All Licensees, Distributors of Any Version of BSD". University of California, Berkeley. 1999-07-22. Retrieved 
+     2006-11-15.
+9. ^ Richard Stallman. "The BSD License Problem". Free Software Foundation. Retrieved 2006-11-15.
+10. ^ "The FreeBSD Copyright". The FreeBSD Project. Retrieved 6 December 2009.
+11. ^ "NetBSD Licensing and Redistribution". The NetBSD Foundation. Retrieved 06 December 2009.