hopfield 1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 Bart Olsthoorn, website: bartolsthoorn.nl
2
+
3
+ Permission is hereby granted, free of charge, to any person
4
+ obtaining a copy of this software and associated documentation
5
+ files (the "Software"), to deal in the Software without
6
+ restriction, including without limitation the rights to use,
7
+ copy, modify, merge, publish, distribute, sublicense, and/or sell
8
+ copies of the Software, and to permit persons to whom the
9
+ Software is furnished to do so, subject to the following
10
+ conditions:
11
+
12
+ The above copyright notice and this permission notice shall be
13
+ included in all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
17
+ OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
19
+ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
20
+ WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
21
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22
+ OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,30 @@
1
+ # Hopfield Network in Ruby
2
+
3
+ ## What is it?
4
+ [Hopfield Networks](http://en.wikipedia.org/wiki/Hopfield_network) model the way humans recall memories, or more specific, how neurons recall the pattern. This means you first train the network with a set of known patterns and then pass an unknown or perturbed version of the pattern. The neurons will restore the missing information to create an exact match.
5
+
6
+ The patterns can be passed using multi dimensional array of either 0 and 1 or -1 and 1. An artifical neural network will learn the patterns. Now let's move on to an example.
7
+
8
+ ## How do I use it?
9
+ ```ruby
10
+ training = Hopfield::Training.new([pattern1, pattern2])
11
+ network = Hopfield::Network.new(training, perturbed_pattern)
12
+ network.pattern # the matched pattern
13
+ network.runs # how many propagations it took
14
+ ```
15
+
16
+ ## Example with images
17
+ See examples/image.rb for a memory association of Charlie Sheen, with a cat hiding him.
18
+ ```
19
+ $ cd examples
20
+ $ ruby image.rb
21
+ Image 1 is now in an array of [20x20]
22
+ Image 2 is now in an array of [20x20]
23
+ Hopfield neurons are trained!
24
+ Neurons propagated: 1776
25
+ Errors: [0]
26
+ ```
27
+ The script also creates black and white pattern images for you.
28
+
29
+ ## Credits
30
+ I was introduced to Hopfield networks through the book [Clever Algorithms](www.cleveralgorithms.com), and I've borrowed bits of the implementation shown in the book. Also used the `.associated?` syntax found here: [Brain](https://github.com/brainopia/brain).
data/lib/hopfield.rb ADDED
@@ -0,0 +1,10 @@
1
+ require_relative 'hopfield/neuron'
2
+ require_relative 'hopfield/training'
3
+ require_relative 'hopfield/network'
4
+
5
+ module Hopfield
6
+ # Hopfield consists of three parts:
7
+ # => Neuron (hopfield/neuron.rb)
8
+ # => Training (hopfield/training.rb)
9
+ # => Network (hopfield/network.rb)
10
+ end
@@ -0,0 +1,92 @@
1
+ module Hopfield
2
+ class Network
3
+ attr_accessor :neurons, :patterns, :state, :pattern_width, :vector, :last_error, :runs
4
+
5
+ def initialize(training, perturbed_pattern)
6
+ unless training.class.to_s == 'Hopfield::Training'
7
+ raise TypeError, 'Training has to be an instance of Hopfield::Training'
8
+ end
9
+
10
+ unless training.patterns.first.size == perturbed_pattern.size
11
+ raise SyntaxError, 'Given pattern does not match size of the training patterns'
12
+ end
13
+
14
+ # Turn 0 into -1
15
+ perturbed_pattern.map {|value| (value == 0 ? -1 : value) }
16
+
17
+ self.neurons = training.neurons
18
+ self.patterns = training.patterns
19
+ self.pattern_width = training.pattern_width
20
+ self.vector = perturbed_pattern.flatten
21
+ self.neurons.each_with_index { |neuron,i| neuron.output = self.vector[i] }
22
+
23
+ self.last_error = [1]
24
+ self.runs = 0
25
+ end
26
+
27
+ def associated?
28
+ return self.last_error.include? 0
29
+ end
30
+
31
+ def pattern
32
+ return self.state
33
+ end
34
+
35
+ def propagate
36
+ # Select random neuron
37
+ i = rand(self.neurons.size)
38
+ activation = 0
39
+ self.neurons.each_with_index do |other, j|
40
+ activation += other.weights[i]*other.output if i!=j
41
+ end
42
+ output = transfer(activation)
43
+ change = output != self.neurons[i].output
44
+ self.neurons[i].output = output
45
+
46
+ # Compile state of outputs
47
+ state = Array.new(self.neurons.size){|i| self.neurons[i].output}
48
+
49
+ # Calculate the current error
50
+ self.last_error = calculate_error(state)
51
+
52
+ # Convert state to binary and back to a multi dimensional array
53
+ state = to_binary(state)
54
+ state = state.each_slice(self.pattern_width).to_a
55
+ self.state = state
56
+
57
+ self.runs += 1
58
+
59
+ return {
60
+ :did_change => change,
61
+ :state => self.state,
62
+ :error => self.last_error
63
+ }
64
+ end
65
+
66
+ def calculate_error(current_pattern)
67
+ errors = Array.new(0)
68
+
69
+ self.patterns.each do |pattern|
70
+ sum = 0
71
+
72
+ expected = pattern.flatten
73
+ actual = current_pattern
74
+
75
+ expected.each_with_index do |v, i|
76
+ sum += 1 if expected[i]!=actual[i]
77
+ end
78
+ errors << sum
79
+ end
80
+ return errors
81
+ end
82
+
83
+ def transfer(activation)
84
+ (activation >= 0 ? 1 : -1)
85
+ end
86
+
87
+ def to_binary(vector)
88
+ return Array.new(vector.size){|i| ((vector[i]==-1) ? 0 : 1)}
89
+ end
90
+
91
+ end
92
+ end
@@ -0,0 +1,18 @@
1
+ module Hopfield
2
+ class Neuron
3
+ attr_accessor :weights, :output
4
+
5
+ def initialize(pattern_size)
6
+ minmax = Array.new(pattern_size) { [-0.5, 0.5] }
7
+
8
+ self.weights = random_vector(minmax)
9
+ end
10
+
11
+ def random_vector(minmax)
12
+ return Array.new(minmax.size) do |i|
13
+ minmax[i][0] + ((minmax[i][1] - minmax[i][0]) * rand())
14
+ end
15
+ end
16
+
17
+ end
18
+ end
@@ -0,0 +1,47 @@
1
+ module Hopfield
2
+ class Training
3
+ attr_accessor :patterns, :neurons, :pattern_width
4
+
5
+ def initialize(patterns)
6
+ # Check if patterns are the same size
7
+ unless patterns.map(&:size).uniq.count == 1
8
+ raise SyntaxError, 'Inconsistent pattern size'
9
+ end
10
+
11
+ # Turn 0 into -1
12
+ patterns.map { |pattern| pattern.map {|value| (value == 0 ? -1 : value) }}
13
+
14
+ # Set the patterns for this training
15
+ self.patterns = patterns
16
+
17
+ # Calculate the amount of required neurons
18
+ # This number is based on the number of inputs of a pattern
19
+ connections = patterns.first.map(&:size).inject{|sum,x| sum + x }
20
+ self.pattern_width = patterns.first.first.size
21
+
22
+ # Create neurons
23
+ self.neurons = Array.new(connections) { Neuron.new connections }
24
+
25
+ # Train the neurons
26
+ train(connections)
27
+ end
28
+
29
+ def train(connections)
30
+ self.neurons.each_with_index do |neuron, i|
31
+ for j in ((i+1)...connections) do
32
+ wij = 0.0
33
+ self.patterns.each do |pattern|
34
+ vector = pattern.flatten
35
+ #puts "Pattern: " + pattern.size.to_s
36
+ #puts "Pattern Y: " + pattern.first.size.to_s
37
+ #puts "Vector: " + vector.size.to_s
38
+ wij += vector[i]*vector[j]
39
+ end
40
+ self.neurons[i].weights[j] = wij
41
+ self.neurons[j].weights[i] = wij
42
+ end
43
+ end
44
+ end
45
+
46
+ end
47
+ end
metadata ADDED
@@ -0,0 +1,85 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: hopfield
3
+ version: !ruby/object:Gem::Version
4
+ version: '1.0'
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Bart Olsthoorn
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2013-02-17 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rspec
16
+ requirement: !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ! '>='
20
+ - !ruby/object:Gem::Version
21
+ version: '0'
22
+ type: :development
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ! '>='
28
+ - !ruby/object:Gem::Version
29
+ version: '0'
30
+ - !ruby/object:Gem::Dependency
31
+ name: chunky_png
32
+ requirement: !ruby/object:Gem::Requirement
33
+ none: false
34
+ requirements:
35
+ - - ! '>='
36
+ - !ruby/object:Gem::Version
37
+ version: '0'
38
+ type: :development
39
+ prerelease: false
40
+ version_requirements: !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ! '>='
44
+ - !ruby/object:Gem::Version
45
+ version: '0'
46
+ description: Hopfield networks can be used for smart pattern recollections. It's recalling
47
+ patterns by modelling associative memory of a neural network
48
+ email:
49
+ - bartolsthoorn@gmail.com
50
+ executables: []
51
+ extensions: []
52
+ extra_rdoc_files: []
53
+ files:
54
+ - lib/hopfield/network.rb
55
+ - lib/hopfield/neuron.rb
56
+ - lib/hopfield/training.rb
57
+ - lib/hopfield.rb
58
+ - LICENSE
59
+ - README.md
60
+ homepage: http://github.com/bartolsthoorn/ruby-hopfield
61
+ licenses: []
62
+ post_install_message:
63
+ rdoc_options: []
64
+ require_paths:
65
+ - lib
66
+ required_ruby_version: !ruby/object:Gem::Requirement
67
+ none: false
68
+ requirements:
69
+ - - ! '>='
70
+ - !ruby/object:Gem::Version
71
+ version: '0'
72
+ required_rubygems_version: !ruby/object:Gem::Requirement
73
+ none: false
74
+ requirements:
75
+ - - ! '>='
76
+ - !ruby/object:Gem::Version
77
+ version: '0'
78
+ requirements: []
79
+ rubyforge_project:
80
+ rubygems_version: 1.8.24
81
+ signing_key:
82
+ specification_version: 3
83
+ summary: Ruby implementation of a Hopfield Network
84
+ test_files: []
85
+ has_rdoc: