rb-brain 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: c16b832f4b21365c93b698c01c14fbea54eb2e74
+  data.tar.gz: e65a2a8c1c76c8b83c00972184942d02fbbeefa3
+SHA512:
+  metadata.gz: cabeba60da560694cb1343f0c0f384533e563ea4f829838783a5cea8ac30c81cf487e0cc2faede4d832ae6e1ea8da3156eca6b4dd7da66c9b29ae2c5be7d6a4f
+  data.tar.gz: 80aab9ccc24424d6b1763c985ff693f81e09b6c41aa1d613d4dc46f431b438d8ce83a4468d78da67df16a5b75858c14747b67e0132b4ccc34df8de3438fab9f7

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in carrierwave-webdav.gemspec
+gemspec

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2014 Qinix
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,105 @@
+# rb-brain
+
+`rb-brain` is an easy-to-use [neural network](http://en.wikipedia.org/wiki/Artificial_neural_network) library written in Ruby. Here's an example of using it to approximate the XOR function:
+
+```ruby
+require 'brain'
+
+net = Brain::NeuralNetwork.new
+
+net.train([{input: [0, 0], output: [0]},
+           {input: [0, 1], output: [1]},
+           {input: [1, 0], output: [1]},
+           {input: [1, 1], output: [0]}])
+
+output = net.run([1, 0])  # [0.948]
+```
+
+## Installation
+
+Install the latest release:
+
+    gem install rb-brain
+
+Require it in your code:
+
+    require 'brain'
+
+Or, in Rails you can add it to your Gemfile:
+
+    gem 'rb-brain', :require => 'brain'
+
+## Training
+Use `train()` to train the network with an array of training data. The network has to be trained with all the data in bulk in one call to `train()`. The more training patterns, the longer it will probably take to train, but the better the network will be at classifiying new patterns.
+
+#### Data format
+Each training pattern should have an `input` and an `output`, both of which can be either an array of numbers from `0` to `1` or a hash of numbers from `0` to `1`. For hash it looks something like this:
+
+```ruby
+require 'brain'
+
+net = Brain::NeuralNetwork.new
+
+net.train([{input: {x: 0,y: 0}, output: {result: 0}},
+           {input: {x: 0,y: 1}, output: {result: 1}},
+           {input: {x: 1,y: 0}, output: {result: 1}},
+           {input: {x: 1,y: 1}, output: {result: 0}}])
+
+output = net.run({x: 1,y: 0})  # {result: 0.948}
+```
+
+#### Options
+`train()` takes a hash of options as its second argument:
+
+```ruby
+net.train(data,
+  error_thresh: 0.005,  # error threshold to reach
+  iterations: 20000,   # maximum training iterations
+  log: true,           # print progress periodically
+  log_period: 10,       # number of iterations between logging
+  learning_rate: 0.3    # learning rate
+)
+```
+
+The network will train until the training error has gone below the threshold (default `0.003`) or the max number of iterations (default `20000`) has been reached, whichever comes first.
+
+By default training won't let you know how its doing until the end, but set `log` to `true` to get periodic updates on the current training error of the network. The training error should decrease every time.
+
+The learning rate is a parameter that influences how quickly the network trains. It's a number from `0` to `1`. If the learning rate is close to `0` it will take longer to train. If the learning rate is closer to `1` it will train faster but it's in danger of training to a local minimum and performing badly on new data. The default learning rate is `0.3`.
+
+#### Output
+The output of `train()` is a hash of information about how the training went:
+
+```ruby
+{
+  error: 0.0039139985510105032,  # training error
+  iterations: 406                # training iterations
+}
+```
+
+#### Failing
+If the network failed to train, the error will be above the error threshold. This could happen because the training data is too noisy (most likely), the network doesn't have enough hidden layers or nodes to handle the complexity of the data, or it hasn't trained for enough iterations.
+
+If the training error is still something huge like `0.4` after 20000 iterations, it's a good sign that the network can't make sense of the data you're giving it.
+
+## Options
+`NeuralNetwork` takes a hash of options:
+
+```ruby
+net = Brain::NeuralNetwork.new(
+  hidden_layers: [4],
+  learning_rate: 0.6 # global learning rate, useful when training using streams
+)
+```
+
+#### hidden_ayers
+Specify the number of hidden layers in the network and the size of each layer. For example, if you want two hidden layers - the first with 3 nodes and the second with 4 nodes, you'd give:
+
+```
+hidden_layers: [3, 4]
+```
+
+By default `brain` uses one hidden layer with size proportionate to the size of the input array.
+
+# Acknowledgement
+I learned a lot from [harthur/brain](https://github.com/harthur/brain), Most of the code is rewritten from this repo. I would like to thank the author of the repo.

data/lib/brain.rb

@@ -0,0 +1 @@
+require 'brain/neuralnetwork'

data/lib/brain/lookup.rb

@@ -0,0 +1,51 @@
+module Brain
+  class Lookup
+    class << self
+      def build_lookup(hashes)
+        # [{a: 1}, {b: 6, c: 7}] -> {a: 0, b: 1, c: 2}
+        h = hashes.reduce do |memo, hash|
+          memo.merge hash
+        end
+        lookup_from_hash h
+      end
+
+      def lookup_from_hash(hash)
+        # {a: 6, b: 7} -> {a: 0, b: 1}
+        lookup = {}
+        index = 0
+        hash.each do |k, v|
+          lookup[k] = index
+          index += 1
+        end
+        lookup
+      end
+
+      def to_array(lookup, hash)
+        # {a: 0, b: 1}, {a: 6} -> [6, 0]
+        array = []
+        lookup.each do |k, v|
+          array[lookup[k]] = hash[k] || 0
+        end
+        array
+      end
+
+      def to_hash(lookup, array)
+        # {a: 0, b: 1}, [6, 7] -> {a: 6, b: 7}
+        hash = {}
+        lookup.each do |k, v|
+          hash[k] = array[lookup[k]]
+        end
+        hash
+      end
+
+      def lookup_from_array(array)
+        lookup = {}
+        z = 0
+        array.reverse.each do |i|
+          lookup[i] = z
+          z += 1
+        end
+      end
+    end
+  end
+end

data/lib/brain/neuralnetwork.rb

@@ -0,0 +1,209 @@
+require 'brain/lookup'
+
+module Brain
+  class NeuralNetwork
+    def initialize(options = {})
+      @learning_rate = options[:learning_rate] || 0.3
+      @momentum = options[:momentum] || 0.1
+      @hidden_sizes = options[:hidden_layers]
+      @binary_thresh = options[:binary_thresh] || 0.5
+    end
+
+    def init(sizes)
+      @sizes = sizes
+      @output_layer = @sizes.length - 1
+
+      @biases = [] # weights for bias nodes
+      @weights = []
+      @outputs = []
+
+      # state for training
+      @deltas = []
+      @changes = [] # for momentum
+      @errors = []
+
+      (0..@output_layer).each do |layer|
+        size = @sizes[layer]
+        @deltas[layer] = Array.new size, 0
+        @errors[layer] = Array.new size, 0
+        @outputs[layer] = Array.new size, 0
+
+        if layer > 0
+          @biases[layer] = randos size
+          @weights[layer] = Array.new size
+          @changes[layer] = Array.new size
+
+          (0...size).each do |node|
+            prev_size = @sizes[layer - 1]
+            @weights[layer][node] = randos prev_size
+            @changes[layer][node] = Array.new prev_size, 0
+          end
+        end
+      end
+    end
+
+    def run(input)
+      input = Lookup.to_array(@input_lookup, input) if @input_lookup
+
+      output = run_input input
+      output = Lookup.to_hash(@output_lookup, output) if @output_lookup
+
+      output
+    end
+
+    def run_input(input)
+      @outputs[0] = input
+      output = 0
+
+      (1..@output_layer).each do |layer|
+        (0...@sizes[layer]).each do |node|
+          weights = @weights[layer][node]
+
+          sum = @biases[layer][node]
+          (0...weights.length).each do |k|
+            sum += weights[k] * input[k]
+          end
+          @outputs[layer][node] = 1 / (1 + Math.exp(-sum))
+        end
+        output = input = @outputs[layer]
+      end
+
+      output
+    end
+
+    def train(data, options = {})
+      data = format_data data
+
+      iterations = options[:iterations] || 20000
+      error_thresh = options[:error_thresh] || 0.003
+      log = options[:log] || false
+      log_period = options[:log_period] || 10
+      learning_rate = options[:learning_rate] || @learning_rate || 0.3
+
+      input_size = data[0][:input].length
+      output_size = data[0][:output].length
+
+      hidden_sizes = @hidden_sizes
+      hidden_sizes = [[3, (input_size / 2.0).floor].max] unless hidden_sizes
+      sizes = [input_size, hidden_sizes, output_size].flatten
+      init sizes
+
+      error = 1
+      done_iterations = iterations
+      (0...iterations).each do |i|
+        unless error > error_thresh
+          done_iterations = i
+          break
+        end
+        sum = 0
+        data.each do |d|
+          err = train_pattern d[:input], d[:output], learning_rate
+          sum += err
+        end
+        error = sum / data.length
+
+        puts "iterations: #{i}, training error: #{error}" if log and (i % log_period == 0)
+      end
+
+      {
+        error: error,
+        iterations: done_iterations
+      }
+    end
+
+    def train_pattern(input, target, learning_rate)
+      learning_rate ||= @learning_rate
+
+      # forward propogate
+      run_input input
+      calculate_deltas target
+      adjust_weights learning_rate
+
+      mse @errors[@output_layer]
+    end
+
+    def calculate_deltas(target)
+      (0..@output_layer).to_a.reverse.each do |layer|
+        (0...@sizes[layer]).each do |node|
+          output = @outputs[layer][node]
+
+          error = 0
+          if layer == @output_layer
+            error = target[node] - output
+          else
+            deltas = @deltas[layer + 1]
+            (0...deltas.length).each do |k|
+              error += deltas[k] * @weights[layer + 1][k][node]
+            end
+          end
+          @errors[layer][node] = error
+          @deltas[layer][node] = error * output * (1 - output)
+        end
+      end
+    end
+
+    def adjust_weights(learning_rate)
+      (1..@output_layer).each do |layer|
+        incoming = @outputs[layer - 1]
+
+        (0...@sizes[layer]).each do |node|
+          delta = @deltas[layer][node]
+
+          (0...incoming.length).each do |k|
+            change = @changes[layer][node][k]
+
+            change = (learning_rate * delta * incoming[k]) + (@momentum * change)
+
+            @changes[layer][node][k] = change
+            @weights[layer][node][k] += change
+          end
+          @biases[layer][node] += learning_rate * delta
+        end
+      end
+    end
+
+    def format_data(data)
+      unless data.is_a? Array
+        data = [data]
+      end
+
+      #turn sparse hash input into arrays with 0s as filler
+      unless data[0][:input].is_a? Array
+        @input_lookup = Lookup.build_lookup data.map {|d| d[:input]} unless @input_lookup
+        data.map! do |datum|
+          array = Lookup.to_array @input_lookup, datum[:input]
+          datum.merge({ input: array })
+        end
+      end
+
+      unless data[0][:output].is_a? Array
+        @output_lookup = Lookup.build_lookup data.map {|d| d[:output]} unless @output_lookup
+        data.map! do |datum|
+          array = Lookup.to_array @output_lookup, datum[:output]
+          datum.merge({ output: array })
+        end
+      end
+
+      data
+    end
+
+  private
+    def random_weight
+      Random.rand * 0.4 - 0.2
+    end
+
+    def randos(size)
+      array = Array.new size
+      array.map! {|item| item = random_weight}
+    end
+
+    def mse(errors)
+      # mean squared error
+      sum = 0
+      errors.each do |e|
+        sum += e ** 2
+      end
+      sum / errors.length
+    end
+  end
+end

data/rb-brain.gemspec

@@ -0,0 +1,20 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+Gem::Specification.new do |gem|
+  gem.name          = "rb-brain"
+  gem.version       = "0.0.1"
+  gem.authors       = ["Eric Zhang"]
+  gem.email         = ["i@qinix.com"]
+  gem.description   = %q{rb-brain is an easy-to-use neural network written in ruby}
+  gem.summary       = %q{rb-brain is an easy-to-use neural network written in ruby}
+  gem.homepage      = "https://github.com/qinix/rb-brain"
+  gem.license       = "MIT"
+
+  gem.files         = `git ls-files`.split($/)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
+
+end

metadata

@@ -0,0 +1,51 @@
+--- !ruby/object:Gem::Specification
+name: rb-brain
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Eric Zhang
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2014-12-22 00:00:00.000000000 Z
+dependencies: []
+description: rb-brain is an easy-to-use neural network written in ruby
+email:
+- i@qinix.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- Gemfile
+- LICENSE
+- README.md
+- lib/brain.rb
+- lib/brain/lookup.rb
+- lib/brain/neuralnetwork.rb
+- rb-brain.gemspec
+homepage: https://github.com/qinix/rb-brain
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.4.3
+signing_key: 
+specification_version: 4
+summary: rb-brain is an easy-to-use neural network written in ruby
+test_files: []