ruby_brain 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 243add4b6f64c99d4bc2de70f69ff74f8fe439fd
+  data.tar.gz: 979c86696fd11c0892031495e57e7f3746b8df2f
+SHA512:
+  metadata.gz: a4d8118899133f660ebfb9a64dd1cd068bb7501dff0cf371b6a8937a51fb9d9c0949b45f76176e71404dc745aa766fd9d3853ef34b6015f1d9bc33025df00b48
+  data.tar.gz: e0aec12b42540e424555f50e3bd7784d136fd7d831ee01486967198461d2940d63f174cdcb4a6b7583bb0b5ffdd0aae8968879a45d137d03a07c02b725d13e49

data/.gitignore

@@ -0,0 +1,10 @@
+*~
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.1
+before_install: gem install bundler -v 1.12.5

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in ruby_brain.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 TODO: Write your name
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,48 @@
+# RubyBrain
+
+RubyBrain is a library of neural net, deep learning for Ruby.
+You can install/use this library easily because the core is created by using only Ruby standard library.
+
+The code of RubyBrain is the neuron oriented style.
+This means that a class which represents a neuraon exists and each neurons are instances of the class.
+So, you can treat neurons flexibly in a network.
+Instead, the speed is very slow and it might not be reasonable for applications to use this library in the core.
+However this library may help you get more deep knowledge around neuralnet/deep learning.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'ruby_brain'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install ruby_brain
+
+## Usage
+
+Please refer to
+[github.com/elgoog/ruby_brain/README.org](https://github.com/elgoog/ruby_brain/blob/master/README.org)
+for detail.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/elgoog/ruby_brain.
+
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
+

data/README.org

@@ -0,0 +1,237 @@
+* Introduction
+
+  RubyBrain is a library of neural net, deep learning for Ruby.
+  You can install/use this library easily because the core is created by using only Ruby standard library.
+
+  The code of RubyBrain is the neuron oriented style.
+  This means that a class which represents a neuraon exists and each neurons are instances of the class.
+  So, you can treat neurons flexibly in a network.
+  Instead, the speed is very slow and it might not be reasonable for applications to use this library in the core.
+  However this library may help you get more deep knowledge around neuralnet/deep learning.
+  
+* Installation
+
+  Add this line to your application's Gemfile:
+
+  #+BEGIN_SRC ruby
+    gem 'ruby_brain'
+  #+END_SRC
+
+  And then execute:
+
+  #+BEGIN_SRC shell
+    $ bundle
+  #+END_SRC
+
+  Or install it yourself as:
+
+  #+BEGIN_SRC shell
+    $ gem install ruby_brain
+  #+END_SRC
+  
+* Usage
+  
+** dataset
+
+   All dataset used for training/test/predicate must be 2 dimension array structure.
+   1st dimension indicates samples and 2nd dimension is used for features.
+
+*** example "AND"
+    Now we assume that we train a network to operate as "AND operator"
+    The true table of "AND operator" is as below.
+    In short, when both "in 1" and "in 2" are 1, the "out" should be 1
+    and 0 should be output for other input combinations.
+    
+    | in 1 | in 2 | out |
+    |------+------+-----|
+    | 0    | 0    | 0   |
+    | 0    | 1    | 0   |
+    | 1    | 0    | 0   |
+    | 1    | 1    | 1   |
+
+    In this situation, you can prepare the dataset lake following Ruby array.
+    #+BEGIN_SRC ruby
+      training_input_set = [
+        [0, 0],
+        [0, 1],
+        [1, 0],
+        [1, 1],
+      ]
+
+      training_supervisor_set = [
+        [0],
+        [0],
+        [0],
+        [1],
+      ]
+    #+END_SRC
+
+** constructing a network
+
+   RubyBrain::Network class represents a network. The constructor accepts an array which specifies the network structure.
+   If we use 1 hidden layer which has 3 neurons for above "AND operator" example, following array indicates the structure.
+   #+BEGIN_SRC ruby
+     # 2 inputs
+     # 3 units in a hidden layer
+     # 1 output
+     [2, 3, 1]
+   #+END_SRC
+   
+   You can use 2 hidden layers with following code.
+   #+BEGIN_SRC ruby
+     # 2 inputs
+     # 4 units in 1st hidden layer
+     # 3 units in 2nd hidden layer
+     # 1 output
+     [2, 4, 2, 1]
+   #+END_SRC
+   
+   So, a netowrk is created by
+   #+BEGIN_SRC ruby
+     a_network = RubyBrain::Network.new([2, 3, 1])
+
+     # learning_rate can be set
+     a_network.learning_rate = 0.5
+
+     # the networks must be initialized before it is used
+     a_network.init_network
+   #+END_SRC
+
+   There are other options for the constructor.
+   Please refer to the code. Sorry for missing document.
+
+** training
+
+   An instance method =learn= is used for training the network.
+   You can specify not only dataset but also other options for training.
+   #+BEGIN_SRC ruby
+     # max_training_cout : max epoch
+     # tolerance : stop training if RMS error become smaller than this value.
+     a_network.learn(training_input_set, training_supervisor_set, max_training_count=100, tolerance=0.0004, monitoring_channels=[:best_params_training])
+   #+END_SRC
+
+** predicate
+
+   Use =get_forward_outputs= with input data for predicating something.
+   Input data should be 1 sample.
+   #+BEGIN_SRC ruby
+     a_network.get_forward_outputs([1, 0])
+   #+END_SRC
+
+** save weights to a file
+
+   You can save optimized weights into a file.
+   Weights are saved as YAML format.
+
+   #+BEGIN_SRC ruby
+     a_network.dump_weights_to_yaml('/path/to/saved/weights/file.yml')
+   #+END_SRC
+   
+** restore weights from a file
+
+   Optimized weights can be saved into a YAML file and you can use it for initializing weights when you create a new network.
+   #+BEGIN_SRC ruby
+     a_network = RubyBrain::Network.new([2, 3, 1])
+     a_network.init_network
+     a_network.load_weights_from_yaml_file('/path/to/saved/weights/file.yml')
+   #+END_SRC
+   
+* Examples
+
+** MNIST
+   Following code is included in [[https://github.com/elgoog/ruby_brain/blob/master/examples/mnist.rb][examples/mnist.rb]]
+
+   This module dependos on [[https://rubygems.org/gems/mnist][mnist]] gem to load mnist data into ruby array.
+
+   #+BEGIN_SRC ruby
+     require 'ruby_brain'
+     require 'ruby_brain/dataset/mnist/data'
+   #+END_SRC
+
+   Get MNIST dataset from [[http://yann.lecun.com/exdb/mnist/][THE MNIST DATABASE of handwritten digits]] if the dataset files don't exist in the working directory.
+   And load them into Ruby array =dataset=.
+
+   #+BEGIN_SRC ruby
+      dataset = RubyBrain::DataSet::Mnist::data
+   #+END_SRC
+
+   Divide =dataset= into training and test data.
+   NUM_TRAIN_DATA means how many first images are used as training data.
+   We use first 5000 images for training here.
+
+   #+BEGIN_SRC ruby
+     NUM_TRAIN_DATA = 5000
+     training_input = dataset[:input][0..(NUM_TRAIN_DATA-1)]
+     training_supervisor = dataset[:output][0..(NUM_TRAIN_DATA-1)]
+   #+END_SRC
+
+   Then construct the network and initialize.
+   In this case, an image has 784(28x28) pixcels and 10 classes(0..9).
+   So, the network structure should be [784, 50, 10] with 1 hidden layer which has 50 units.
+   You can construct the structure with following code.
+   
+   #+BEGIN_SRC ruby
+     # network structure [784, 50, 10]
+     network = RubyBrain::Network.new([dataset[:input].first.size, 50, dataset[:output].first.size])
+     # learning rate is 0.7
+     network.learning_rate = 0.7
+     # initialize network
+     network.init_network
+   #+END_SRC
+
+   Run training.
+   #+BEGIN_SRC ruby
+     network.learn(training_input, training_supervisor, max_training_count=100, tolerance=0.0004, monitoring_channels=[:best_params_training])
+   #+END_SRC
+
+   Now, An optimized network was completed.
+   You can check it.
+
+   First, add =argmax= function into Array class.
+   This method finds the index of the array position the max value exists.
+   We use this method for finding the class(label 0~9) whose probability is the highest.
+
+   #+BEGIN_SRC ruby
+     class Array
+       def argmax
+         max_i, max_val = 0, self.first
+         self.each_with_index do |v, i|
+           max_val, max_i = v, i if v > max_val
+         end
+         max_i
+       end
+     end
+   #+END_SRC
+
+   Then, you can review each classes(labels) predicated by the model with following code.
+   
+   #+BEGIN_SRC ruby
+     results = []
+     test_input.each_with_index do |input, i|
+       input.each_with_index do |e, j|
+         print(e > 0.3 ? 'x' : ' ')
+         puts if (j % 28) == 0
+       end
+       puts
+       supervisor_label = test_supervisor[i].argmax
+       predicated_label = network.get_forward_outputs(test_input[i]).argmax
+       puts "test_supervisor: #{supervisor_label}"
+       puts "predicate: #{predicated_label}"
+       results << (supervisor_label == predicated_label)
+       puts "------------------------------------------------------------"
+     end
+
+     puts "accuracy: #{results.count(true).to_f/results.size}"
+   #+END_SRC
+
+   I tried to train wioth above conditions.
+   The accuracy of trained model was 92.3%.
+   The weights file is [[https://github.com/elgoog/weights_ruby_brain/blob/master/weights_782_50_10_1.yml][here]].
+   
+* Contributing
+
+  1. Fork it ( https://github.com/elgoog/ruby_brain/fork )
+  2. Create your feature branch (`git checkout -b my-new-feature`)
+  3. Commit your changes (`git commit -am 'Add some feature'`)
+  4. Push to the branch (`git push origin my-new-feature`)
+  5. Create a new Pull Request

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "ruby_brain"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/examples/mnist.rb

@@ -0,0 +1,79 @@
+require 'ruby_brain'
+require 'ruby_brain/dataset/mnist/data'
+
+# NUM_TEST_DATA = 50000
+
+dataset = RubyBrain::DataSet::Mnist::data
+
+NUM_TRAIN_DATA = 5000
+
+training_input = dataset[:input][0..(NUM_TRAIN_DATA-1)]
+training_supervisor = dataset[:output][0..(NUM_TRAIN_DATA-1)]
+
+# test_input = dataset[:input][NUM_TRAIN_DATA..(NUM_TRAIN_DATA+NUM_TEST_DATA-1)]
+# test_supervisor = dataset[:output][NUM_TRAIN_DATA..(NUM_TRAIN_DATA+NUM_TEST_DATA-1)]
+test_input = dataset[:input][NUM_TRAIN_DATA..-1]
+test_supervisor = dataset[:output][NUM_TRAIN_DATA..-1]
+
+network = RubyBrain::Network.new([dataset[:input].first.size, 50, dataset[:output].first.size])
+network.learning_rate = 0.7
+network.init_network
+### You can load weights from file in this timing if you want.
+# network.load_weights_from_yaml_file(File.dirname(__FILE__) + '/../best_weights_1469044985.yml')
+
+### You can initializes weights by loading weights from file if you want.
+# network.load_weights_from_yaml_file('path/to/weights.yml.file')
+
+network.learn(training_input, training_supervisor, max_training_count=100, tolerance=0.0004, monitoring_channels=[:best_params_training])
+
+### You can save weights into a yml file if you want.
+# network.dump_weights_to_yaml('path/to/weights.yml.file')
+
+
+class Array
+  def argmax
+    max_i, max_val = 0, self.first
+    self.each_with_index do |v, i|
+      max_val, max_i = v, i if v > max_val
+    end
+    max_i
+  end
+end
+
+results = []
+test_input.each_with_index do |input, i|
+  ### You can see test input, label and predicated lable in standard out if you uncomment in this block 
+  # input.each_with_index do |e, j|
+  #   print(e > 0.3 ? 'x' : ' ')
+  #   puts if (j % 28) == 0
+  # end
+  # puts
+  supervisor_label = test_supervisor[i].argmax
+  predicated_label = network.get_forward_outputs(test_input[i]).argmax
+  # puts "test_supervisor: #{supervisor_label}"
+  # puts "predicate: #{predicated_label}"
+  results << (supervisor_label == predicated_label)
+  # puts "------------------------------------------------------------"
+end
+
+puts "accuracy: #{results.count(true).to_f/results.size}"
+
+
+
+### you can do above procedure simply by using Trainer
+
+# training_option = {
+#   learning_rate: 0.5,
+#   max_training_count: 50,
+#   tolerance: 0.0004,
+#   # initial_weights_file: 'weights_3_30_10_1429166740.yml',
+#   # initial_weights_file: 'best_weights_1429544001.yml',
+#   monitoring_channels: [:best_params_training]
+# }
+
+# RubyBrain::Trainer.normal_learning([dataset[:input].first.size, 50, dataset[:output].first.size],
+#                                    training_input, training_supervisor,
+#                                    training_option)
+
+
+