neuronet 6.0.1 → 6.1.0

data/README.md

@@ -315,7 +315,7 @@ For the example above, we can check their lengths.
 	puts neuronet.yang.length #=> 6
 	puts neuronet.out.length #=> 3
 	
-## Tao, Yin, and Yang
+## Tao, Yin, Yang, and Brahma
 
 Tao
 :	The absolute principle underlying the universe,
@@ -346,6 +346,8 @@ Training begins the process that sets the weights to associate the two.
 But you can also manually set the initial weights.
 One useful way to initially set the weigths is to have one layer mirror another.
 The [Yin](http://rubydoc.info/gems/neuronet/Neuronet/Yin) bless makes yin mirror the input.
+The length of yin must be at least that of in.
+The pairing starts with in.first and yin.first on up.
 
 	Yin.bless(neuronet)
 
@@ -353,11 +355,24 @@ Yang
 :	The active male principle of the universe, characterized as male and
 	creative and associated with heaven, heat, and light.
 
-One the other hand, the [Yang](http://rubydoc.info/gems/neuronet/Neuronet/Yang)
+On the other hand, the [Yang](http://rubydoc.info/gems/neuronet/Neuronet/Yang)
 bless makes the output mirror yang.
+The length of yang must be a least that of out.
+The pairing starts from yang.last and out.last on down.
 
 	Yang.bless(neuronet)
 
+Brahma
+:	The creator god in later Hinduism, who forms a triad with Vishnu the preserver and Shiva the destroyer.
+
+[Brahma](http://rubydoc.info/gems/neuronet/Neuronet/Brahma)
+pairs each input node with two yin neurons sending them respectively the positive and negative value of its activation.
+I'd say then that yin both mirrors and shadows input.
+The length of yin must be at least twice that of in.
+The pairing starts with in.first and yin.first on up.
+
+	Brahma.bless(neuronet)
+
 Bless
 :	Pronounce words in a religious rite, to confer or invoke divine favor upon.
 
@@ -493,26 +508,10 @@ would be constructed this way:
 Here yinyang's hidden layer (which is both yin and yang)
 initially would have the first n neurons mirror the input and
 the last m neurons be mirrored by the output.
-Another interesting YinYang would be:
+Another interesting YinYang would be an input to output mirror:
 
 	yinyang = YinYang.bless FeedForward.new( [n, n, n] )
 
-The following code demonstrates what is meant by "mirroring":
-
-	yinyang = YinYang.bless FeedForward.new( [3, 3, 3] )
-	yinyang.set( [-1,0,1] )
-	puts yinyang.in.map{|x| x.activation}.join(', ')
-	puts yinyang.yin.map{|x| x.activation}.join(', ')
-	puts yinyang.out.map{|x| x.activation}.join(', ')
-	puts yinyang.output.join(', ')
-
-Here's the output:
-
-	0.268941421369995, 0.5, 0.731058578630005
-	0.442490985892539, 0.5, 0.557509014107461
-	0.485626707638021, 0.5, 0.514373292361979
-	-0.0575090141074614, 0.0, 0.057509014107461
-
 # Theory
 
 ## The Biological Description of a Neuron
@@ -709,6 +708,79 @@ So I provide a way to set the learning constant based on the size of the data wi
 
 The value of #num(n) is #muk(1.0)/Math.sqrt(n)).
 
+## Mirroring
+
+Because the squash function is not linear, mirroring is going to be warped.
+Nonetheless, I'd like to map zeroes to zeroes and ones to ones.
+That gives us the following two equations:
+
+	weight*sigmoid(1.0) + bias = 1.0
+	weight*sigmoid(0.0) + bias = 0.0
+
+We can solve that!  Consider the zeroes to zeroes map:
+
+	weight*sigmoid(0.0) + bias = 0.0
+	weight*sigmoid(0.0) = -bias
+	weight*0.5 = -bias
+	weight = -2*bias
+
+Now the ones to ones:
+
+	weight*sigmoid(1.0) + bias = 1.0
+	-2.0*bias*sigmoid(1.0) + bias = 1.0
+	bias*(-2.0*sigmoid(1.0) + 1.0) = 1.0
+	bias = 1.0 / (1.0 - 2.0*sigmoid(1.0))
+
+We get the numerical values:
+
+	bias = -2.163953413738653  # BZERO
+	weight = 4.327906827477306 # WONE
+
+In the code I call this bias and weight BZERO and WONE respectively.
+What about "shadowing"?
+
+	weight*sigmoid(1.0) + bias = -1.0
+	weight*sigmoid(0.0) + bias = 0.0
+
+	weight = -2.0*bias # <== same a before
+
+	weight*sigmoid(1.0) + bias = -1.0
+	-2.0*bias*sigmoid(1.0) + bias = -1.0
+	bias*(-2.0*sigmoid(1.0) + 1.0) = -1.0
+	bias = -1.0 / (-2.0*sigmoid(1.0) + 1.0)
+	bias = 1.0 / (2.0*sigmoid(1.0) - 1.0)
+	# ^== this is just negative what we got before.
+
+Shadowing is just the negative of mirroring.
+There's a test, [tests/mirror.rb](https://github.com/carlosjhr64/neuronet/blob/master/tests/mirror.rb),
+which demostrates mirroring.  Here's the output:
+
+	### YinYang ###
+	Input:
+	-1.0,	0.0,	1.0
+	In:
+	0.2689414213699951,	0.5,	0.7310585786300049
+	Yin/Yang:
+	0.2689414213699951,	0.5,	0.7310585786300049
+	0.2689414213699951,	0.5,	0.7310585786300049
+	Out:
+	0.2689414213699951,	0.5,	0.7310585786300049
+	Output:
+	-1.0000000000000002,	0.0,	1.0
+
+	### BrahmaYang ###
+	Input:
+	-1.0,	0.0,	1.0
+	In:
+	0.2689414213699951,	0.5,	0.7310585786300049
+	Yin/Yang:
+	0.2689414213699951,	0.7310585786300049,	0.5,	0.5,	0.7310585786300049,	0.2689414213699951
+	0.2689414213699951,	0.7310585786300049,	0.5,	0.5,	0.7310585786300049,	0.2689414213699951
+	Out:
+	0.2689414213699951,	0.7310585786300049,	0.5,	0.5,	0.7310585786300049,	0.2689414213699951
+	Output:
+	-1.0000000000000002,	1.0,	0.0,	0.0,	1.0,	-1.0000000000000002
+
 # Questions?
 
 Email me!

data/lib/neuronet.rb

@@ -1,6 +1,6 @@
 # Neuronet module
 module Neuronet
-  VERSION = '6.0.1'
+  VERSION = '6.1.0'
 
   # An artificial neural network uses a squash function
   # to determine the activation value of a neuron.
@@ -26,6 +26,9 @@ module Neuronet
     Math.log(squashed / (1.0 - squashed))
   end
 
+  BZERO = 1.0/(1.0-2.0*squash(1.0))
+  WONE  = -2.0*BZERO
+
   # Although the implementation is free to set all parameters for each neuron,
   # Neuronet by default creates zeroed neurons.
   # Association between inputs and outputs are trained, and
@@ -478,10 +481,10 @@ module Neuronet
     end
   end
 
-  # Yin is a network wich has its @yin layer initially mirroring @in.
+  # Yin is a network which has its @yin layer initially mirroring @in.
   module Yin
-    # Yin.bless sets the bias of each @yin[i] to 0.5, and
-    # the weight of pairing (@yin[i], @in[i]) connections to one.
+    # Yin.bless increments the bias of each @yin[i] by BZERO, and
+    # the weight of pairing (@yin[i], @in[i]) connections by WONE.
     # This makes @yin initially mirror @in.
     # The pairing is done starting with (@yin[0], @in[0]).
     # That is, starting with (@yin.first, @in.first).
@@ -490,11 +493,11 @@ module Neuronet
       if yin.length < (in_length = myself.in.length)
         raise "First hidden layer, yin, needs to have at least the same length as input"
       end
-      # connections from yin[i] to in[i] are 1... mirroring to start.
+      # connections from yin[i] to in[i] are WONE... mirroring to start.
       0.upto(in_length-1) do |index|
         node = yin[index]
-        node.connections[index].weight = 1.0
-        node.bias = -0.5
+        node.connections[index].weight += WONE
+        node.bias += BZERO
       end
       return myself
     end
@@ -502,8 +505,8 @@ module Neuronet
 
   # Yang is a network wich has its @out layer initially mirroring @yang.
   module Yang
-    # Yang.bless sets the bias of each @yang[i] to 0.5, and
-    # the weight of pairing (@out[i], @yang[i]) connections to one.
+    # Yang.bless increments the bias of each @yang[i] by BZERO, and
+    # the weight of pairing (@out[i], @yang[i]) connections by WONE.
     # This makes @out initially mirror @yang.
     # The pairing is done starting with (@out[-1], @yang[-1]).
     # That is, starting with (@out.last, @yang.last).
@@ -514,8 +517,8 @@ module Neuronet
       # the net effect to is pair @out.last with @yang.last, and so on down.
       0.upto(out_length-1) do |index|
         node = out[index]
-        node.connections[offset+index].weight = 1.0
-        node.bias = -0.5
+        node.connections[offset+index].weight += WONE
+        node.bias += BZERO
       end
       return myself
     end
@@ -524,8 +527,8 @@ module Neuronet
   # A Yin Yang composite provided for convenience.
   module YinYang
     def self.bless(myself)
-      Yin.bless(myself)
       Yang.bless(myself)
+      Yin.bless(myself)
       return myself
     end
   end
@@ -533,9 +536,9 @@ module Neuronet
   # A Tao Yin Yang composite provided for convenience.
   module TaoYinYang
     def self.bless(myself)
-      Tao.bless(myself)
-      Yin.bless(myself)
       Yang.bless(myself)
+      Yin.bless(myself)
+      Tao.bless(myself)
       return myself
     end
   end
@@ -543,8 +546,8 @@ module Neuronet
   # A Tao Yin composite provided for convenience.
   module TaoYin
     def self.bless(myself)
-      Tao.bless(myself)
       Yin.bless(myself)
+      Tao.bless(myself)
       return myself
     end
   end
@@ -552,10 +555,67 @@ module Neuronet
   # A Tao Yang composite provided for convenience.
   module TaoYang
     def self.bless(myself)
+      Yang.bless(myself)
       Tao.bless(myself)
+      return myself
+    end
+  end
+
+  # Brahma is a network which has its @yin layer initially mirror and "shadow" @in.
+  # I'm calling it shadow until I can think of a better name.
+  # Note that a Brahma, Yin bless combination overwrite eachother and is probably useless.
+  module Brahma
+    # Brahma.bless increments the weights of pairing even yin (@yin[2*i], @in[i]) connections by WONE.
+    # and pairing odd yin (@yin[2*i+1], @in[i]) connections by negative WONE.
+    # Likewise the bias with BZERO.
+    # This makes @yin initially mirror and shadow @in.
+    # The pairing is done starting with (@yin[0], @in[0]).
+    # That is, starting with (@yin.first, @in.first).
+    def self.bless(myself)
+      yin = myself.yin
+      if yin.length < 2*(in_length = myself.in.length)
+        raise "First hidden layer, yin, needs to be at least twice the length as input"
+      end
+      # connections from yin[2*i] to in[i] are WONE... mirroring to start.
+      # connections from yin[2*i+1] to in[i] are -WONE... shadowing to start.
+      0.upto(in_length-1) do |index|
+        even = yin[2*index]
+        odd = yin[(2*index)+1]
+        even.connections[index].weight += WONE
+        even.bias += BZERO
+        odd.connections[index].weight  -= WONE
+        odd.bias -= BZERO
+      end
+      return myself
+    end
+  end
+
+  # A Brahma Yang composite provided for convenience.
+  module BrahmaYang
+    def self.bless(myself)
+      Brahma.bless(myself)
       Yang.bless(myself)
       return myself
     end
   end
 
+  # A Brahma Yang composite provided for convenience.
+  module TaoBrahma
+    def self.bless(myself)
+      Brahma.bless(myself)
+      Tao.bless(myself)
+      return myself
+    end
+  end
+
+  # A Tao Brahma Yang composite provided for convenience.
+  module TaoBrahmaYang
+    def self.bless(myself)
+      Yang.bless(myself)
+      Brahma.bless(myself)
+      Tao.bless(myself)
+      return myself
+    end
+  end
+
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: neuronet
 version: !ruby/object:Gem::Version
-  version: 6.0.1
+  version: 6.1.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-06-18 00:00:00.000000000 Z
+date: 2013-06-20 00:00:00.000000000 Z
 dependencies: []
 description: Build custom neural networks. 100% 1.9 Ruby.
 email: carlosjhr64@gmail.com