cerebrum 0.1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 0f30759c9f08ac49ff7c52a6a0fd97dcf288d255
+  data.tar.gz: 94a095b0fe5853e8b7e5b8e6f88c4cfe8ea1fd7a
+SHA512:
+  metadata.gz: 7c6cc26527c029730a51e2c1f5a36eed725d1b4e3c473732b67bc02673da9b28557adeb9a88c6ea2b1916f432a5d673ffc17cd52d8236decba912bf54c2b85a7
+  data.tar.gz: 6b755cd92c8fbb84c69dde3b4751247c670278272112128ca6412c798b15a57ad26770367afed48315b5c7dce905ef1d6e0a4949e165927985f8890fd79782d8

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.travis.yml

@@ -0,0 +1,5 @@
+language: ruby
+rvm:
+  - 2.1.8
+before_install: gem install bundler -v 1.11.2
+script: bundle exec rake

data/Gemfile

@@ -0,0 +1,7 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in cerebrum.gemspec
+gemspec
+gem 'pry-nav', group: [:development, :test]
+gem 'bundler', group: [:development, :test]
+gem 'minitest', group: [:development, :test]

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Irfan Sharif
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,136 @@
+# GEM: cerebrum ![](https://travis-ci.org/irfansharif/cerebrum.svg?branch=master)
+
+`cerebrum` is an implementation of
+[ANNs](https://en.wikipedia.org/wiki/Artificial_neural_networ://en.wikipedia.org/wiki/Artificial_neural_network)
+in Ruby.  There's no reason to train a neural network in Ruby, I'm using it to
+experiment and play around with the bare fundamentals of ANNs, original idea
+for this project [here](https://github.com/harthur/brain) is currently
+unmaintained. Extensions on top of that are personal experimentation.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'cerebrum'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install cerebrum
+
+## Usage
+
+```ruby
+require 'cerebrum'
+
+network = Cerebrum.new
+
+network.train([
+  {input: [0, 0], output: [0]},
+  {input: [0, 1], output: [1]},
+  {input: [1, 0], output: [1]},
+  {input: [1, 1], output: [0]}
+])
+
+result = network.run([1, 0])
+# => [0.9333206724219677]
+
+```
+
+### Training
+
+Use `Cerebrum#train` to train the network with an array of training data.
+
+#### Data format
+
+Each training pattern should have an `input:` and an `output:`, both of which
+can either be an array of numbers from `0` to `1` or a hash of numbers from `0`
+to `1`. An example of the latter is demonstrated below:
+
+```ruby
+network = Cerebrum.new
+
+network.train([
+    {input: { r: 0.03, g: 0.7, b: 0.5 }, output: { black: 1 }},
+    {input: { r: 0.16, g: 0.09, b: 0.2 }, output: { white: 1 }},
+    {input: { r: 0.5, g: 0.5, b: 1.0 }, output: { white: 1 }}
+]);
+
+result = network.run({ r: 1, g: 0.4, b: 0 })
+# => { :black=>0.011967728530458011, :white=>0.9871010273923573 }
+```
+
+#### Cerebrum Options
+
+`Cerebrum#new` takes a hash of options that would set defaults if not specified in the `Cerebrum#train` procedure call:
+
+```ruby
+network = Cerebrum.new({
+  learning_rate:  0.3,
+  momentum:       0.1,
+  binary_thresh:  0.5,
+  hidden_layers:  [3, 4]
+})
+```
+
+#### Training Options
+
+`Cerebrum#train` optionally takes in a configuration hash as the second argument:
+
+```ruby
+network.train(data, {
+  error_threshold: 0.005,
+  iterations:      20000,
+  log:             true,
+  log_period:      100,
+  learning_rate:   0.3
+})
+```
+
+The network will train until the training error has gone below the threshold or
+the max number of iterations has been reached, whichever comes first.
+
+By default training won't let you know how its doing until the end, but set `log`
+to `true` to get periodic updates on the current training error of the network.
+The training error should decrease every time. The updates will be printed to
+console. If you set `log` to a function, this function will be called with the
+updates instead of printing to the console.
+
+The `learning_rate` is a parameter that influences how quickly the network
+trains, a number from `0` to `1`. If the learning rate is close to `0` it will
+take longer to train. If the learning rate is closer to `1` it will train faster
+but it's in danger of training to a local minimum and performing badly on new
+data.
+
+#### Output
+
+The output of `Cerebrum#train` is a hash of information about how the training went:
+
+```ruby
+network.train(data, options)
+# => { error: 0.005324233132423, iterations: 9001 }
+```
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run
+`rake test` to run the tests. You can also run `bin/console` for an interactive
+prompt that will allow you to experiment.  To install this gem onto your local
+machine, run `bundle exec rake install`. To release a new version, update the
+version number in `version.rb`, and then run `bundle exec rake release`, which
+will create a git tag for the version, push git commits and tags, and push the
+`.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at [irfansharif](https://github.com/irfansharif/cerebrum).
+
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
+

data/Rakefile

@@ -0,0 +1,10 @@
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList['test/**/*_test.rb']
+end
+
+task default: :test

data/bin/console

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "cerebrum"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+require "pry"
+Pry.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/cerebrum.gemspec

@@ -0,0 +1,32 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'cerebrum/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "cerebrum"
+  spec.version       = Cerebrum::VERSION
+  spec.authors       = ["Irfan Sharif", "Arham Ahmed"]
+  spec.email         = ["irfanmahmoudsharif@gmail.com", "mohammad.a.ahmed@uwaterloo.ca"]
+
+  spec.summary       = %q{Artificial Neural Networks in Ruby}
+  spec.homepage      = "https://github.com/irfansharif/cerebrum"
+  spec.license       = "MIT"
+
+  # Prevent pushing this gem to RubyGems.org by setting 'allowed_push_host', or
+  # delete this section to allow pushing this gem to any host.
+  if spec.respond_to?(:metadata)
+    spec.metadata['allowed_push_host'] = "https://rubygems.org/"
+  else
+    raise "RubyGems 2.0 or newer is required to protect against public gem pushes."
+  end
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.11"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "minitest", "~> 5.0"
+end

data/lib/cerebrum.rb

@@ -0,0 +1,2 @@
+require "cerebrum/version"
+require "cerebrum/cerebrum"

data/lib/cerebrum/cerebrum.rb

@@ -0,0 +1,166 @@
+require_relative "data_scrubber"
+require_relative "cerebrum_helper"
+
+class Cerebrum
+  include CerebrumHelper
+  include DataScrubber
+
+  attr_accessor :learning_rate, :momentum, :binary_thresh, :hidden_layers,
+                :input_lookup_table, :output_lookup_table
+
+  def initialize(learning_rate: 0.3, momentum: 0.1, binary_thresh: 0.5, hidden_layers: nil)
+    @learning_rate  = learning_rate
+    @momentum       = momentum
+    @binary_thresh  = binary_thresh
+    @hidden_layers  = hidden_layers
+  end
+
+  def train_pattern(input, target, learning_rate)
+    learning_rate = learning_rate || @learning_rate
+
+    run_input(input)
+    calculate_deltas(target)
+    adjust_weights(learning_rate)
+    mean_squared_error(@errors[@layers])
+  end
+
+  def train(training_set, options = Hash.new)
+    @input_lookup_table ||= get_input_lookup_table(training_set)
+    @output_lookup_table ||= get_output_lookup_table(training_set)
+    training_set = scrub_dataset(training_set)
+
+    iterations        = options[:iterations] || 20000
+    error_threshold   = options[:error_threshold] || 0.005
+    log               = options[:log] || false
+    log_period        = options[:log_period] || 10
+    learning_rate     = options[:learning_rate] || 0.3
+    error             = Float::INFINITY
+    current_iteration = 0
+
+    input_size = training_set[0][:input].length
+    output_size = training_set[0][:output].length
+
+    hidden_layer_sizes = [ [3, (input_size/2).floor].max ] unless @hidden_layers
+    layer_sizes = [input_size, hidden_layer_sizes, output_size].flatten
+    construct_network(layer_sizes)
+
+    iterations.times do |i|
+      current_iteration = i
+      training_set_errors = training_set.map { |ex| train_pattern(ex[:input], ex[:output], learning_rate) }
+      error = training_set_errors.inject(:+) / training_set.length
+      puts "(#{i}) training error: #{error}" if (log && (i % log_period) == 0)
+
+      break if error < error_threshold
+    end
+
+    { error: error, iterations: current_iteration }
+  end
+
+  def run(input)
+    input = to_vector_given_features(input, @input_lookup_table) if @input_lookup_table
+    output = run_input(input)
+    @output_lookup_table ? to_features_given_vector(output, @output_lookup_table) : output
+  end
+
+  private
+
+  def construct_network(layer_sizes)
+    @layer_sizes = layer_sizes
+    @layers = layer_sizes.length - 1 # Excluding output layer
+
+    @biases, @weights, @outputs = [], [], []
+    @deltas, @changes, @errors = [], [], []
+
+    (@layers + 1).times do |layer| # Including output layer
+      layer_size = @layer_sizes[layer]
+      @deltas[layer] = zeros(layer_size)
+      @errors[layer] = zeros(layer_size)
+      @outputs[layer] = zeros(layer_size)
+
+      next if layer == 0
+
+      @biases[layer] = randos(layer_size)
+      @weights[layer] = Array.new(layer_size)
+      @changes[layer] = Array.new(layer_size)
+      previous_layer_size = @layer_sizes[layer - 1]
+
+      layer_size.times do |node|
+        @weights[layer][node] = randos(previous_layer_size)
+        @changes[layer][node] = zeros(previous_layer_size)
+      end
+    end
+  end
+
+  def mean_squared_error(errors)
+    sum_of_squares = errors.map{ |error| error ** 2 }.reduce(:+)
+    Float(sum_of_squares) / errors.length
+  end
+
+  def run_input(input)
+    @outputs[0] = input
+
+    (@layers + 1).times do |layer|  # Include output layer
+      next if layer == 0
+
+      layer_size = @layer_sizes[layer]
+      previous_layer_size = @layer_sizes[layer - 1]
+
+      layer_size.times do |node|
+        weights = @weights[layer][node]
+        sum = @biases[layer][node]
+        previous_layer_size.times do |prev_node|
+          sum += @outputs[layer - 1][prev_node] * weights[prev_node]
+        end
+        @outputs[layer][node] = activation_function(sum)
+      end
+    end
+
+    @outputs.last
+  end
+
+  def calculate_deltas(target)
+    @layers.downto(0) do |layer|
+      layer_size = @layer_sizes[layer]
+
+      layer_size.times do |node|
+        output = @outputs[layer][node]
+        error = 0
+
+        if layer == @layers # Output layer
+          error = target[node] - output
+        else # Hidden layer
+          deltas = @deltas[layer + 1]
+          deltas.each_with_index do |delta, next_node|
+            error += delta * @weights[layer + 1][next_node][node]
+          end
+        end
+        @errors[layer][node] = error
+        @deltas[layer][node] = error * output * (1 - output)
+      end
+    end
+  end
+
+  def adjust_weights(rate)
+    1.upto(@layers) do |layer|
+      prev_layer_output = @outputs[layer - 1]
+      layer_size = @layer_sizes[layer]
+
+      layer_size.times do |node|
+        delta = @deltas[layer][node]
+        prev_layer_output.length.times do |prev_node|
+          change = @changes[layer][node][prev_node]
+          change = rate * delta * prev_layer_output[prev_node] + (@momentum * change)
+
+          @changes[layer][node][prev_node] = change
+          @weights[layer][node][prev_node] += change
+        end
+
+        @biases[layer][node] += rate * delta
+      end
+    end
+  end
+
+  def activation_function(sum)
+    1 / (1 + Math.exp( -sum ))
+  end
+end

data/lib/cerebrum/cerebrum_helper.rb

@@ -0,0 +1,37 @@
+module CerebrumHelper
+  private
+
+  def zeros(size)
+    Array.new(size, 0)
+  end
+
+  def randos(size)
+    Array.new(size) { rand }
+  end
+
+  # [{a: 1}, {b: 6, c: 7}] -> {a: 0, b: 1, c: 2}
+  def features_to_vector_index_lookup_table(features)
+    flattened_feature_keys = features.inject(:merge)
+    reindex_hash_values(flattened_feature_keys)
+  end
+
+  # changes hash {a: 6, b: 7} to {a: 0, b: 1}
+  def reindex_hash_values(hash)
+    hash.each_with_index{ |pair, index| hash[pair[0]] = index }
+  end
+
+  # formats {a: 0, b: 1}, {a: 6} to [6, 0]
+  def to_vector_given_features(features, lookup_table)
+    lookup_table.map { |k,v| features[k] || 0 }
+  end
+
+  # {a: 0, b: 1}, [6, 7] to {a: 6, b: 7}
+  def to_features_given_vector(vector, lookup_table)
+    lookup_table.keys.zip(vector).to_h
+  end
+
+  # [5, 3] to {5: 0, 3: 1}
+  def lookup_table_from_array(arr)
+    Hash[arr.each_with_index.map { |val, i| [val, i] }]
+  end
+end

data/lib/cerebrum/data_scrubber.rb

@@ -0,0 +1,33 @@
+module DataScrubber
+  private
+
+  def scrub_dataset(dataset)
+    dataset = scrub_input(dataset) unless dataset[0][:input].is_a? Array
+    dataset = scrub_output(dataset) unless dataset[0][:output].is_a? Array
+    dataset
+  end
+
+  def get_input_lookup_table(dataset)
+    input_features = dataset.map { |ex| ex[:input] }
+    (input_features.first.is_a? Array) ? nil : features_to_vector_index_lookup_table(input_features)
+  end
+
+  def get_output_lookup_table(dataset)
+    output_features = dataset.map { |ex| ex[:output] }
+    (output_features.first.is_a? Array) ? nil : features_to_vector_index_lookup_table(output_features)
+  end
+
+  def scrub_input(dataset)
+    input_lookup_table = get_input_lookup_table(dataset)
+    dataset.each do |ex|
+      ex[:input] = to_vector_given_features(ex[:input], input_lookup_table)
+    end
+  end
+
+  def scrub_output(dataset)
+    output_lookup_table = get_output_lookup_table(dataset)
+    dataset.each do |ex|
+      ex[:output] = to_vector_given_features(ex[:output], output_lookup_table)
+    end
+  end
+end

data/lib/cerebrum/version.rb

@@ -0,0 +1,3 @@
+class Cerebrum
+  VERSION = "0.1.1"
+end

metadata

@@ -0,0 +1,103 @@
+--- !ruby/object:Gem::Specification
+name: cerebrum
+version: !ruby/object:Gem::Version
+  version: 0.1.1
+platform: ruby
+authors:
+- Irfan Sharif
+- Arham Ahmed
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2016-04-21 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.11'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.11'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+description: 
+email:
+- irfanmahmoudsharif@gmail.com
+- mohammad.a.ahmed@uwaterloo.ca
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".travis.yml"
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- cerebrum.gemspec
+- lib/cerebrum.rb
+- lib/cerebrum/cerebrum.rb
+- lib/cerebrum/cerebrum_helper.rb
+- lib/cerebrum/data_scrubber.rb
+- lib/cerebrum/version.rb
+homepage: https://github.com/irfansharif/cerebrum
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org/
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.3
+signing_key: 
+specification_version: 4
+summary: Artificial Neural Networks in Ruby
+test_files: []