ai4r 1.1 → 1.2
Sign up to get free protection for your applications and to get access to all the features.
- data/README.rdoc +21 -20
- data/examples/decision_trees/id3_example.rb +3 -2
- data/examples/genetic_algorithm/genetic_algorithm_example.rb +6 -6
- data/examples/neural_network/backpropagation_example.rb +2 -2
- data/lib/ai4r/classifiers/classifier_helper.rb +54 -0
- data/lib/ai4r/classifiers/id3.rb +356 -0
- data/lib/ai4r/classifiers/one_r.rb +148 -0
- data/lib/ai4r/classifiers/prism.rb +231 -0
- data/lib/ai4r/classifiers/zero_r.rb +104 -0
- data/lib/ai4r/genetic_algorithm/genetic_algorithm.rb +272 -0
- data/lib/ai4r/neural_network/backpropagation.rb +271 -0
- data/site/build/tmp/locationmap.xml +14 -14
- data/site/build/tmp/output.xmap +23 -23
- data/site/build/tmp/pluginlist2fetchbuild.xml +144 -144
- data/site/build/tmp/plugins-1.xml +0 -11
- data/site/build/tmp/plugins-2.xml +54 -0
- data/site/build/tmp/projfilters.properties +41 -41
- data/site/build/webapp/WEB-INF/logs/core.log +681 -788
- data/site/build/webapp/WEB-INF/logs/error.log +281 -248
- data/site/build/webapp/WEB-INF/logs/sitemap.log +1015 -0
- data/site/src/documentation/content/xdocs/forum.html +9 -0
- data/site/src/documentation/content/xdocs/geneticAlgorithms.xml +82 -68
- data/site/src/documentation/content/xdocs/index.xml +47 -18
- data/site/src/documentation/content/xdocs/machineLearning.xml +10 -9
- data/site/src/documentation/content/xdocs/neuralNetworks.xml +60 -36
- data/site/src/documentation/content/xdocs/site.xml +8 -5
- data/site/src/documentation/content/xdocs/svn.xml +11 -1
- data/site/src/documentation/resources/images/Thumbs.db +0 -0
- data/site/src/documentation/resources/images/ai4r-logo.png +0 -0
- data/site/src/documentation/resources/images/genetic_algorithms_example.png +0 -0
- data/site/src/documentation/resources/images/jadeferret.png +0 -0
- data/site/src/documentation/resources/images/neural_network_example.png +0 -0
- data/site/src/documentation/resources/images/sub-dir/Thumbs.db +0 -0
- data/site/src/documentation/skinconf.xml +18 -18
- data/test/classifiers/id3_test.rb +206 -0
- data/test/classifiers/one_r_test.rb +62 -0
- data/test/classifiers/prism_test.rb +83 -0
- data/test/classifiers/zero_r_test.rb +48 -0
- data/test/genetic_algorithm/chromosome_test.rb +41 -38
- data/test/genetic_algorithm/genetic_algorithm_test.rb +64 -61
- data/test/neural_network/backpropagation_test.rb +20 -18
- metadata +109 -199
- data/lib/decision_tree/id3.rb +0 -354
- data/lib/genetic_algorithm/genetic_algorithm.rb +0 -268
- data/lib/neural_network/backpropagation.rb +0 -264
- data/site/build/site/en/broken-links.xml +0 -2
- data/site/build/site/en/downloads.html +0 -187
- data/site/build/site/en/downloads.pdf +0 -151
- data/site/build/site/en/geneticAlgorithms.html +0 -564
- data/site/build/site/en/geneticAlgorithms.pdf +0 -911
- data/site/build/site/en/images/ai4r-logo.png +0 -0
- data/site/build/site/en/images/built-with-forrest-button.png +0 -0
- data/site/build/site/en/images/c.png +0 -0
- data/site/build/site/en/images/c_wbn.png +0 -0
- data/site/build/site/en/images/c_wn.png +0 -0
- data/site/build/site/en/images/ero.gif +0 -0
- data/site/build/site/en/images/europe2.png +0 -0
- data/site/build/site/en/images/europe3.png +0 -0
- data/site/build/site/en/images/fitness.png +0 -0
- data/site/build/site/en/images/instruction_arrow.png +0 -0
- data/site/build/site/en/images/my_email.png +0 -0
- data/site/build/site/en/images/rubyforge.png +0 -0
- data/site/build/site/en/images/s.png +0 -0
- data/site/build/site/en/images/s_wbn.png +0 -0
- data/site/build/site/en/images/s_wn.png +0 -0
- data/site/build/site/en/images/sigmoid.png +0 -0
- data/site/build/site/en/images/t.png +0 -0
- data/site/build/site/en/images/t_wbn.png +0 -0
- data/site/build/site/en/images/t_wn.png +0 -0
- data/site/build/site/en/index.html +0 -258
- data/site/build/site/en/index.pdf +0 -306
- data/site/build/site/en/linkmap.html +0 -231
- data/site/build/site/en/linkmap.pdf +0 -94
- data/site/build/site/en/locationmap.xml +0 -72
- data/site/build/site/en/machineLearning.html +0 -325
- data/site/build/site/en/machineLearning.pdf +0 -337
- data/site/build/site/en/neuralNetworks.html +0 -446
- data/site/build/site/en/neuralNetworks.pdf +0 -604
- data/site/build/site/en/skin/CommonMessages_de.xml +0 -23
- data/site/build/site/en/skin/CommonMessages_en_US.xml +0 -23
- data/site/build/site/en/skin/CommonMessages_es.xml +0 -23
- data/site/build/site/en/skin/CommonMessages_fr.xml +0 -23
- data/site/build/site/en/skin/basic.css +0 -166
- data/site/build/site/en/skin/breadcrumbs-optimized.js +0 -90
- data/site/build/site/en/skin/breadcrumbs.js +0 -237
- data/site/build/site/en/skin/fontsize.js +0 -166
- data/site/build/site/en/skin/getBlank.js +0 -40
- data/site/build/site/en/skin/getMenu.js +0 -45
- data/site/build/site/en/skin/images/README.txt +0 -1
- data/site/build/site/en/skin/images/add.jpg +0 -0
- data/site/build/site/en/skin/images/built-with-forrest-button.png +0 -0
- data/site/build/site/en/skin/images/chapter.gif +0 -0
- data/site/build/site/en/skin/images/chapter_open.gif +0 -0
- data/site/build/site/en/skin/images/current.gif +0 -0
- data/site/build/site/en/skin/images/error.png +0 -0
- data/site/build/site/en/skin/images/external-link.gif +0 -0
- data/site/build/site/en/skin/images/fix.jpg +0 -0
- data/site/build/site/en/skin/images/forrest-credit-logo.png +0 -0
- data/site/build/site/en/skin/images/hack.jpg +0 -0
- data/site/build/site/en/skin/images/header_white_line.gif +0 -0
- data/site/build/site/en/skin/images/info.png +0 -0
- data/site/build/site/en/skin/images/instruction_arrow.png +0 -0
- data/site/build/site/en/skin/images/label.gif +0 -0
- data/site/build/site/en/skin/images/page.gif +0 -0
- data/site/build/site/en/skin/images/pdfdoc.gif +0 -0
- data/site/build/site/en/skin/images/poddoc.png +0 -0
- data/site/build/site/en/skin/images/printer.gif +0 -0
- data/site/build/site/en/skin/images/rc-b-l-15-1body-2menu-3menu.png +0 -0
- data/site/build/site/en/skin/images/rc-b-r-15-1body-2menu-3menu.png +0 -0
- data/site/build/site/en/skin/images/rc-b-r-5-1header-2tab-selected-3tab-selected.png +0 -0
- data/site/build/site/en/skin/images/rc-t-l-5-1header-2searchbox-3searchbox.png +0 -0
- data/site/build/site/en/skin/images/rc-t-l-5-1header-2tab-selected-3tab-selected.png +0 -0
- data/site/build/site/en/skin/images/rc-t-l-5-1header-2tab-unselected-3tab-unselected.png +0 -0
- data/site/build/site/en/skin/images/rc-t-r-15-1body-2menu-3menu.png +0 -0
- data/site/build/site/en/skin/images/rc-t-r-5-1header-2searchbox-3searchbox.png +0 -0
- data/site/build/site/en/skin/images/rc-t-r-5-1header-2tab-selected-3tab-selected.png +0 -0
- data/site/build/site/en/skin/images/rc-t-r-5-1header-2tab-unselected-3tab-unselected.png +0 -0
- data/site/build/site/en/skin/images/remove.jpg +0 -0
- data/site/build/site/en/skin/images/rss.png +0 -0
- data/site/build/site/en/skin/images/spacer.gif +0 -0
- data/site/build/site/en/skin/images/success.png +0 -0
- data/site/build/site/en/skin/images/txtdoc.png +0 -0
- data/site/build/site/en/skin/images/update.jpg +0 -0
- data/site/build/site/en/skin/images/valid-html401.png +0 -0
- data/site/build/site/en/skin/images/vcss.png +0 -0
- data/site/build/site/en/skin/images/warning.png +0 -0
- data/site/build/site/en/skin/images/xmldoc.gif +0 -0
- data/site/build/site/en/skin/menu.js +0 -48
- data/site/build/site/en/skin/note.txt +0 -50
- data/site/build/site/en/skin/print.css +0 -54
- data/site/build/site/en/skin/profile.css +0 -163
- data/site/build/site/en/skin/prototype.js +0 -1257
- data/site/build/site/en/skin/screen.css +0 -587
- data/site/build/site/en/svn.html +0 -223
- data/site/build/site/en/svn.pdf +0 -239
- data/site/build/site/en/wholesite.pdf +0 -1686
- data/site/build/tmp/brokenlinks.xml +0 -2
- data/site/build/tmp/cocoon-work/cache-dir/cocoon-ehcache-1.data +0 -0
- data/site/build/tmp/cocoon-work/cache-dir/cocoon-ehcache-1.index +0 -0
- data/test/decision_tree/id3_test.rb +0 -209
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
<html>
|
|
2
|
+
<head>
|
|
3
|
+
<title>Artificial intelligence for ruby :: ai4r :: Forum page at RubyForge</title>
|
|
4
|
+
</head>
|
|
5
|
+
<body>
|
|
6
|
+
<h1><a name="intro" />Artificial intelligence for ruby - ai4r - Forum page at RubyForge</h1>
|
|
7
|
+
<iframe width="100%" height="800px" frameborder="0" src="http://rubyforge.org/forum/forum.php?forum_id=19341" />
|
|
8
|
+
</body>
|
|
9
|
+
</html>
|
|
@@ -5,46 +5,6 @@
|
|
|
5
5
|
<title>Genetics Algorithms in Ruby :: ai4r</title>
|
|
6
6
|
</header>
|
|
7
7
|
<body>
|
|
8
|
-
<section id="Introduction">
|
|
9
|
-
<title>Introduction to Genetics Algorithms in Ruby</title>
|
|
10
|
-
<p>The GeneticAlgorithm module implements the GeneticSearch and Chromosome classes. The GeneticSearch is a generic class, and can be used to solved any kind of problems. The GeneticSearch class performs a stochastic search
|
|
11
|
-
of the solution of a given problem. It uses the following pseudocode:
|
|
12
|
-
</p>
|
|
13
|
-
<ol>
|
|
14
|
-
<li>Choose initial population</li>
|
|
15
|
-
<li>Evaluate the fitness of each individual in the population</li>
|
|
16
|
-
<li>Repeat as many times as generations we allow
|
|
17
|
-
<ol>
|
|
18
|
-
<li>Select randomly best-ranking individuals to reproduce</li>
|
|
19
|
-
<li>Breed new generation through crossover and mutation (genetic operations) and give birth to offspring</li>
|
|
20
|
-
<li>Evaluate the individual fitnesses of the offspring</li>
|
|
21
|
-
<li>Replace worst ranked part of population with offspring</li>
|
|
22
|
-
</ol>
|
|
23
|
-
</li>
|
|
24
|
-
</ol>
|
|
25
|
-
<p>The Chromosome is "problem specific". Ai4r built-in Chromosomeclass was designed to model the <a href="http://en.wikipedia.org/wiki/Traveling_salesman_problem" title="Link to Wikipedia">Travelling salesman problem</a>. You have to provide a matrix with the cost of traveling from one point to another (array of arrays of float values). If you want to solve other type of problem, you will have to modify the Chromosome class, by overwriting its fitness, reproduce, and mutate functions, to model you specific problem.</p>
|
|
26
|
-
</section>
|
|
27
|
-
|
|
28
|
-
<section id="how-to-run">
|
|
29
|
-
<title>How to use it</title>
|
|
30
|
-
<source>
|
|
31
|
-
<![CDATA[
|
|
32
|
-
#Cost of traveling from one point to another. E.g. Travelling from Node 0 to Node 2 costs 5.
|
|
33
|
-
data_set = [ [ 0, 10, 5],
|
|
34
|
-
[ 6, 0, 4],
|
|
35
|
-
[25, 4, 0]
|
|
36
|
-
]
|
|
37
|
-
|
|
38
|
-
GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
39
|
-
|
|
40
|
-
search = GeneticAlgorithm::GeneticSearch.new(10, 20)
|
|
41
|
-
result = search.run
|
|
42
|
-
puts "Result cost: #{result.fitness}"
|
|
43
|
-
puts "Result nodes: #{result.data.inspect}"
|
|
44
|
-
]]>
|
|
45
|
-
</source>
|
|
46
|
-
</section>
|
|
47
|
-
|
|
48
8
|
<section id="Example">
|
|
49
9
|
<title>The European Rock Tour Problem (Also known as the Travelling salesman problem)</title>
|
|
50
10
|
<p>An ageing rock band was planning its (hopefully) last european tour. They were planning to visite 15 european cities: Barcelona, Berlin, Brussels, Dublin, Hamburg, Kiev, London, Madrid, Milan, Moscow, Munich, Paris, Rome, Vienna, and Warsaw.</p>
|
|
@@ -72,29 +32,45 @@ puts "Result nodes: #{result.data.inspect}"
|
|
|
72
32
|
</p>
|
|
73
33
|
</section>
|
|
74
34
|
|
|
75
|
-
|
|
76
|
-
<
|
|
77
|
-
<
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
35
|
+
<section id="Introduction">
|
|
36
|
+
<title>Introduction to Genetics Algorithms in Ruby</title>
|
|
37
|
+
<p>A Genetic Algorithm is a particular class of evolutionary algorithm and stochastic search.
|
|
38
|
+
It aims to find the best possible solution in a solution domain, by selecting a simple
|
|
39
|
+
set of solutions (chosed randomly or with a simple heuristic), and making it "evolve".
|
|
40
|
+
</p>
|
|
41
|
+
<p>It works based on the following pseudocode:</p>
|
|
42
|
+
<ol>
|
|
43
|
+
<li>Choose initial population</li>
|
|
44
|
+
<li>Evaluate the fitness of each individual in the population</li>
|
|
45
|
+
<li>Repeat as many times as generations we allow
|
|
46
|
+
<ol>
|
|
47
|
+
<li>Select randomly best-ranking individuals to reproduce</li>
|
|
48
|
+
<li>Breed new generation through crossover and mutation (genetic operations) and give birth to offspring</li>
|
|
49
|
+
<li>Evaluate the individual fitnesses of the offspring</li>
|
|
50
|
+
<li>Replace worst ranked part of population with offspring</li>
|
|
51
|
+
</ol>
|
|
52
|
+
</li>
|
|
53
|
+
</ol>
|
|
54
|
+
</section>
|
|
94
55
|
|
|
95
56
|
<section id="chromosome-impl">
|
|
96
57
|
<title>Implementation of Chromosome class for the Travelling salesman problem</title>
|
|
97
|
-
|
|
58
|
+
<p>In AI4R, the GeneticAlgorithm module implements the GeneticSearch and Chromosome classes.
|
|
59
|
+
GeneticSearch is a generic class, and can be used to solved any kind of problems.
|
|
60
|
+
The GeneticSearch class performs a stochastic search following the algorithm mentioned in the
|
|
61
|
+
previous section.
|
|
62
|
+
</p>
|
|
63
|
+
<p>
|
|
64
|
+
However, the Chromosome class implementation is problem specific.
|
|
65
|
+
Ai4r built-in Chromosomeclass was designed to model the
|
|
66
|
+
<a href="http://en.wikipedia.org/wiki/Traveling_salesman_problem" title="Link to Wikipedia">
|
|
67
|
+
Travelling salesman problem.
|
|
68
|
+
</a>
|
|
69
|
+
You have to provide a matrix with the cost of traveling from one point to another
|
|
70
|
+
(array of arrays of float values).
|
|
71
|
+
If you want to solve other type of problem, you will have to modify the Chromosome class,
|
|
72
|
+
by overwriting its fitness, reproduce, and mutate functions, to model you specific problem.
|
|
73
|
+
</p>
|
|
98
74
|
<section id="chromosome-impl-data">
|
|
99
75
|
<title>Data representation</title>
|
|
100
76
|
<p>Each chromosome must represent a posible solution for the problem. This class conatins an array
|
|
@@ -105,7 +81,7 @@ data_set = [ [ 0, 10, 5],
|
|
|
105
81
|
[ 6, 0, 4],
|
|
106
82
|
[25, 4, 0]
|
|
107
83
|
]
|
|
108
|
-
GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
84
|
+
Ai4r::GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
109
85
|
]]>
|
|
110
86
|
</source>
|
|
111
87
|
</section>
|
|
@@ -122,8 +98,8 @@ data_set = [ [ 0, 10, 5],
|
|
|
122
98
|
[ 6, 0, 4],
|
|
123
99
|
[25, 4, 0]
|
|
124
100
|
]
|
|
125
|
-
GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
126
|
-
chromosome = GeneticAlgorithm::Chromosome.new([0, 2, 1])
|
|
101
|
+
Ai4r::GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
102
|
+
chromosome = Ai4r::GeneticAlgorithm::Chromosome.new([0, 2, 1])
|
|
127
103
|
chromosome.fitness
|
|
128
104
|
# => -9
|
|
129
105
|
]]>
|
|
@@ -185,7 +161,7 @@ want to model another type of problem, you will have to modify the Chromosome cl
|
|
|
185
161
|
<p>You have to provide two parameters during instantiation: The initial population size, and the how many generations produce. Large numbers will usually converge to better results, while small numbers will have better performance.</p>
|
|
186
162
|
<source>
|
|
187
163
|
<![CDATA[
|
|
188
|
-
search = GeneticAlgorithm::GeneticSearch.new(10, 20)
|
|
164
|
+
search = Ai4r::GeneticAlgorithm::GeneticSearch.new(10, 20)
|
|
189
165
|
result = search.run
|
|
190
166
|
]]>
|
|
191
167
|
</source>
|
|
@@ -237,7 +213,25 @@ rand < ((1 - chromosome.normalized_fitness) * 0.4)
|
|
|
237
213
|
|
|
238
214
|
|
|
239
215
|
</section>
|
|
216
|
+
<section id="how-to-run">
|
|
217
|
+
<title>How to use AI4R Genetic Search implementation</title>
|
|
218
|
+
<source>
|
|
219
|
+
<![CDATA[
|
|
220
|
+
#Cost of traveling from one point to another. E.g. Travelling from Node 0 to Node 2 costs 5.
|
|
221
|
+
data_set = [ [ 0, 10, 5],
|
|
222
|
+
[ 6, 0, 4],
|
|
223
|
+
[25, 4, 0]
|
|
224
|
+
]
|
|
240
225
|
|
|
226
|
+
Ai4r::GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
227
|
+
|
|
228
|
+
search = Ai4r::GeneticAlgorithm::GeneticSearch.new(10, 20)
|
|
229
|
+
result = search.run
|
|
230
|
+
puts "Result cost: #{result.fitness}"
|
|
231
|
+
puts "Result nodes: #{result.data.inspect}"
|
|
232
|
+
]]>
|
|
233
|
+
</source>
|
|
234
|
+
</section>
|
|
241
235
|
|
|
242
236
|
<section id="example-run">
|
|
243
237
|
<title>How to run the example</title>
|
|
@@ -245,8 +239,9 @@ rand < ((1 - chromosome.normalized_fitness) * 0.4)
|
|
|
245
239
|
contains:</p>
|
|
246
240
|
<source>
|
|
247
241
|
<![CDATA[
|
|
248
|
-
require
|
|
249
|
-
require
|
|
242
|
+
require "rubygems"
|
|
243
|
+
require "ai4r/genetic_algorithm/genetic_algorithm"
|
|
244
|
+
require "csv"
|
|
250
245
|
|
|
251
246
|
# Load data from data_set.csv
|
|
252
247
|
data_set = []
|
|
@@ -258,10 +253,10 @@ data_set.collect! do |column|
|
|
|
258
253
|
column.collect { |element| element.to_f}
|
|
259
254
|
end
|
|
260
255
|
|
|
261
|
-
GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
256
|
+
Ai4r::GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
|
|
262
257
|
|
|
263
258
|
puts "Beginning genetic search, please wait... "
|
|
264
|
-
search = GeneticAlgorithm::GeneticSearch.new(800, 100)
|
|
259
|
+
search = Ai4r::GeneticAlgorithm::GeneticSearch.new(800, 100)
|
|
265
260
|
result = search.run
|
|
266
261
|
puts "Result cost: #{result.fitness}"
|
|
267
262
|
puts "Result tour: "
|
|
@@ -271,6 +266,25 @@ result.data.each { |c| print " #{data_labels[c]}"}
|
|
|
271
266
|
|
|
272
267
|
</section>
|
|
273
268
|
|
|
269
|
+
<section id="Results">
|
|
270
|
+
<title>Results of using Genetic Algorithms to the The European Rock Tour Problem (or Travelling salesman problem)</title>
|
|
271
|
+
<p>The cost of 3 randomly selected tours:</p>
|
|
272
|
+
<ul>
|
|
273
|
+
<li>$17486.01 : Madrid Vienna Moscow Berlin Brussels Munich Milan Barcelona London Hamburg Warsaw Dublin Kiev Paris Rome</li>
|
|
274
|
+
<li>$20198.92 : London Rome Brussels Kiev Hamburg Warsaw Barcelona Paris Munich Dublin Vienna Moscow Madrid Milan Berlin</li>
|
|
275
|
+
<li>$17799.34 : Madrid Milan Kiev Vienna Warsaw London Barcelona Hamburg Paris Munich Dublin Berlin Moscow Rome Brussels</li>
|
|
276
|
+
</ul>
|
|
277
|
+
<p>3 tours obtained with an initial population of 800, and after 100 generations:</p>
|
|
278
|
+
<ul>
|
|
279
|
+
<li>$7611.99 : Moscow Kiev Warsaw Hamburg Berlin Munich Vienna Milan Rome Barcelona Madrid Paris Brussels London Dublin</li>
|
|
280
|
+
<li>$7596.74 : Moscow Kiev Warsaw Berlin Hamburg Munich Vienna Milan Rome Barcelona Madrid Paris Brussels London Dublin (See Image)</li>
|
|
281
|
+
<li>$7641.61 : Madrid Barcelona Rome Milan Paris Dublin London Brussels Hamburg Berlin Vienna Munich Warsaw Kiev Moscow</li>
|
|
282
|
+
</ul>
|
|
283
|
+
<p><img src="images/europe3.png" alt="Best tour result using Genetic Algorithms in ruby" /></p>
|
|
284
|
+
<p>The GeneticSearch class is an generic class to try to solve any kind of problem using genetic algorithms. If you
|
|
285
|
+
want to model another type of problem, you will have to modify the Chromosome class, defining its fitness, mutate, and reproduce functions.</p>
|
|
286
|
+
</section>
|
|
287
|
+
|
|
274
288
|
<section id="more-genetic-run">
|
|
275
289
|
<title>More about Genetic Algorithms and the Travelling salesman problem</title>
|
|
276
290
|
<p><a href="http://en.wikipedia.org/wiki/Traveling_salesman_problem" title="Link to Wikipedia">Travelling salesman problem at Wikipedia</a></p>
|
|
@@ -19,35 +19,66 @@
|
|
|
19
19
|
<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" "http://forrest.apache.org/dtd/document-v20.dtd">
|
|
20
20
|
<document>
|
|
21
21
|
<header>
|
|
22
|
-
<title>Artificial Intelligence for Ruby
|
|
22
|
+
<title>AI4R :: Artificial Intelligence for Ruby</title>
|
|
23
23
|
</header>
|
|
24
24
|
<body>
|
|
25
25
|
<section id="Introduction">
|
|
26
26
|
<title>Introduction</title>
|
|
27
27
|
<p>
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
intelligence fields, including:
|
|
28
|
+
AI4R is a collection of ruby algorithms implementations, covering several Artificial
|
|
29
|
+
intelligence fields, and simple practical examples using them. It implements:
|
|
31
30
|
</p>
|
|
32
31
|
<ul>
|
|
33
|
-
<li><a href="
|
|
34
|
-
<li><a href="
|
|
35
|
-
<li><a href="
|
|
32
|
+
<li><a href="http://en.wikipedia.org/wiki/Genetic_algorithms">Genetic algorithms</a></li>
|
|
33
|
+
<li><a href="http://en.wikipedia.org/wiki/Neural_network">Neural networks (Backpropagation networks)</a></li>
|
|
34
|
+
<li><a href="machineLearning.html">Automatic classifiers (Machine Learning)</a>
|
|
35
|
+
<ul>
|
|
36
|
+
<li><a href="http://en.wikipedia.org/wiki/ID3_algorithm">ID3 (Decision Trees)</a></li>
|
|
37
|
+
<li>PRISM (J. Cendrowska, 1987)</li>
|
|
38
|
+
<li><a href="http://en.wikipedia.org/wiki/Association_rule_learning#One-attribute-rule">OneR (AKA One Attribute Rule, 1R)</a></li>
|
|
39
|
+
<li><a href="http://en.wikipedia.org/wiki/Association_rule_learning#Zero-attribute-rule">ZeroR</a></li>
|
|
40
|
+
</ul>
|
|
41
|
+
</li>
|
|
36
42
|
</ul>
|
|
37
43
|
</section>
|
|
38
44
|
|
|
45
|
+
<section id="Examples_offered">
|
|
46
|
+
<title>Practical examples</title>
|
|
47
|
+
<p>
|
|
48
|
+
<strong>Genetic Algorithms</strong><br />
|
|
49
|
+
<a href="geneticAlgorithms.html">Optimization of the Travelling salesman problem (NP-hard problem)</a><br />
|
|
50
|
+
<a href="geneticAlgorithms.html">
|
|
51
|
+
<img src="images/genetic_algorithms_example.png" alt="Genetic Algorithms Example" />
|
|
52
|
+
</a>
|
|
53
|
+
</p>
|
|
54
|
+
<p>
|
|
55
|
+
<strong>Neural networks</strong><br />
|
|
56
|
+
<a href="neuralNetworks.html">Simple OCR (recognition of visual patterns)</a><br />
|
|
57
|
+
<a href="neuralNetworks.html">
|
|
58
|
+
<img src="images/neural_network_example.png" alt="Neural Network Example" />
|
|
59
|
+
</a>
|
|
60
|
+
</p>
|
|
61
|
+
<p>
|
|
62
|
+
<strong>Automatic classifiers</strong><br />
|
|
63
|
+
<a href="machineLearning.html">Automatic identification of relevant marketing targets</a>
|
|
64
|
+
</p>
|
|
65
|
+
|
|
66
|
+
</section>
|
|
67
|
+
|
|
39
68
|
<section id="howtoinstall">
|
|
40
69
|
<title>How to install</title>
|
|
41
70
|
<p>1. Install the gem:</p>
|
|
42
|
-
<source>gem install
|
|
43
|
-
<p>Include require statements in your code:</p>
|
|
71
|
+
<source>gem install ai4r</source>
|
|
72
|
+
<p>2. Include require statements in your code:</p>
|
|
44
73
|
<source>
|
|
45
74
|
require "rubygems"
|
|
46
|
-
require "
|
|
47
|
-
require "
|
|
48
|
-
require "
|
|
75
|
+
require "ai4r/classifiers/id3"en
|
|
76
|
+
require "ai4r/classifiers/prism"
|
|
77
|
+
require "ai4r/classifiers/one_r"
|
|
78
|
+
require "ai4r/classifiers/zero_r"
|
|
79
|
+
require "ai4r/neural_network/backpropagation"
|
|
80
|
+
require "ai4r/genetic_algorithm/genetic_algorithm"
|
|
49
81
|
</source>
|
|
50
|
-
|
|
51
82
|
</section>
|
|
52
83
|
|
|
53
84
|
<section id="ContactMe">
|
|
@@ -62,11 +93,9 @@ require "genetic_algorithm/genetic_algorithm"
|
|
|
62
93
|
<section id="warranty">
|
|
63
94
|
<title>Disclaimer Note</title>
|
|
64
95
|
<p>
|
|
65
|
-
This software is
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
the implied warranties of merchantibility and fitness for a
|
|
69
|
-
particular purpose.
|
|
96
|
+
This software is provided "as is" and without any express or implied
|
|
97
|
+
warranties, including, without limitation, the implied warranties of
|
|
98
|
+
merchantibility and fitness for a particular purpose.
|
|
70
99
|
</p>
|
|
71
100
|
</section>
|
|
72
101
|
</body>
|
|
@@ -8,14 +8,14 @@
|
|
|
8
8
|
<section id="mach-intro">
|
|
9
9
|
<title>Introduction to ID3 algorithm</title>
|
|
10
10
|
<p>
|
|
11
|
-
|
|
11
|
+
AI4R implements the ID3 algorithm (Quinlan) as one of its automatic classifiers.
|
|
12
12
|
Given a set of preclassified examples, it builds a top-down
|
|
13
13
|
induction of decision tree, biased by the information gain and
|
|
14
14
|
entropy measure.
|
|
15
15
|
</p>
|
|
16
16
|
<p>
|
|
17
|
-
The good thing about this learning method is that humans learns as well.
|
|
18
|
-
Unlike other
|
|
17
|
+
The good thing about this automatic learning method is that humans learns as well.
|
|
18
|
+
Unlike other AI techniques like neural networks, classifiers can
|
|
19
19
|
generate ruby code with if / else sentences. You
|
|
20
20
|
can use this to evaluate parameters on realtime, copy paste them in a
|
|
21
21
|
code, or just read them to learn about your problem domain.
|
|
@@ -24,7 +24,8 @@
|
|
|
24
24
|
|
|
25
25
|
<section id="mach-HowTo">
|
|
26
26
|
<title>Marketing target strategy example using ID3 Decision Trees in Ruby</title>
|
|
27
|
-
<p>Let's suppose that you are
|
|
27
|
+
<p>Let's suppose that you are writting an application that must identify people as relevant marketing targets or not.
|
|
28
|
+
The only information that you have is a collection of examples, provided by a marketing survey:</p>
|
|
28
29
|
<source>
|
|
29
30
|
<![CDATA[
|
|
30
31
|
DATA_LABELS = [ 'city', 'age_range', 'gender', 'marketing_target' ]
|
|
@@ -51,7 +52,7 @@
|
|
|
51
52
|
<p>You can create an ID3 Decision tree to do the dirty job for you:</p>
|
|
52
53
|
<source>
|
|
53
54
|
<![CDATA[
|
|
54
|
-
id3 =
|
|
55
|
+
id3 = ID3.new(DATA_SET, DATA_LABELS)
|
|
55
56
|
]]>
|
|
56
57
|
</source>
|
|
57
58
|
<p>The Decision tree will automatically create the "rules" to parse new data,
|
|
@@ -75,7 +76,7 @@
|
|
|
75
76
|
<section id="mach-dataload">
|
|
76
77
|
<title>Better data loading</title>
|
|
77
78
|
<p>
|
|
78
|
-
In
|
|
79
|
+
In real life you will use many more data training examples,
|
|
79
80
|
with more attributes.
|
|
80
81
|
Consider moving your data to an external CSV (comma separate values) file.
|
|
81
82
|
</p>
|
|
@@ -87,7 +88,7 @@
|
|
|
87
88
|
end
|
|
88
89
|
data_labels = data_set.shift
|
|
89
90
|
|
|
90
|
-
id3 =
|
|
91
|
+
id3 = ID3.new(data_set, data_labels)
|
|
91
92
|
]]></source>
|
|
92
93
|
|
|
93
94
|
</section>
|
|
@@ -107,7 +108,7 @@ But instead of going through the tree every time, you can take advantage of the
|
|
|
107
108
|
</p>
|
|
108
109
|
<source>
|
|
109
110
|
<![CDATA[
|
|
110
|
-
id3 =
|
|
111
|
+
id3 = ID3.new(DATA_SET, DATA_LABELS)
|
|
111
112
|
age_range = '<30'
|
|
112
113
|
city = 'New York'
|
|
113
114
|
gender = 'M'
|
|
@@ -126,4 +127,4 @@ But instead of going through the tree every time, you can take advantage of the
|
|
|
126
127
|
</p>
|
|
127
128
|
</section>
|
|
128
129
|
</body>
|
|
129
|
-
</document>
|
|
130
|
+
</document>
|
|
@@ -25,10 +25,22 @@
|
|
|
25
25
|
<section id="nn-Introduction">
|
|
26
26
|
<title>Introduction to Neural Networks in Ruby</title>
|
|
27
27
|
<p>
|
|
28
|
-
The utility of artificial neural network models lies in the fact
|
|
28
|
+
The utility of artificial neural network models lies in the fact
|
|
29
|
+
that they can be used to infer a function from observations. This is
|
|
30
|
+
particularly useful in applications where the complexity of the data
|
|
31
|
+
or task makes the design of such a function by hand impractical.
|
|
32
|
+
Neural Networks are being used in many businesses and applications.
|
|
33
|
+
Their ability to learn by example makes them attractive in environments
|
|
34
|
+
where the business rules are either not well defined or are hard to
|
|
35
|
+
enumerate and define. Many people believe that Neural Networks can
|
|
36
|
+
only solve toy problems. Give them a try, and let you decide if they
|
|
37
|
+
are good enough to solve your needs.
|
|
29
38
|
</p>
|
|
30
39
|
<p>
|
|
31
|
-
In this module you will find an implementation of neural networks
|
|
40
|
+
In this module you will find an implementation of neural networks
|
|
41
|
+
using the Backpropagation is a supervised learning technique
|
|
42
|
+
(described by Paul Werbos in 1974, and further developed by David E.
|
|
43
|
+
Rumelhart, Geoffrey E. Hinton and Ronald J. Williams in 1986)
|
|
32
44
|
</p>
|
|
33
45
|
</section>
|
|
34
46
|
<section id="nn-HowTo">
|
|
@@ -36,46 +48,57 @@ In this module you will find an implementation of neural networks using the Back
|
|
|
36
48
|
<source>
|
|
37
49
|
<![CDATA[
|
|
38
50
|
# Create the network
|
|
39
|
-
net = Backpropagation.new([4, 3, 2])
|
|
40
|
-
|
|
41
|
-
|
|
51
|
+
net = Ai4r::NeuralNetwork::Backpropagation.new([4, 3, 2])
|
|
52
|
+
# 4 inputs
|
|
53
|
+
# 1 hidden layer with 3 neurons
|
|
54
|
+
# 2 outputs
|
|
42
55
|
# Train the network
|
|
43
56
|
1..upto(100) do |i|
|
|
44
57
|
net.train(example[i], result[i])
|
|
45
58
|
end
|
|
46
59
|
|
|
47
60
|
# Use it: Evaluate data with the trained network
|
|
48
|
-
net.eval([12, 48, 12, 25]) # => [
|
|
61
|
+
net.eval([12, 48, 12, 25]) # => [0.89, 0.04]
|
|
49
62
|
]]>
|
|
50
63
|
</source>
|
|
51
64
|
</section>
|
|
52
65
|
|
|
53
|
-
<section id="nn-example">
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
Let's imagine that we have to implement a program to identify simple patterns
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
</
|
|
77
|
-
<
|
|
78
|
-
|
|
66
|
+
<section id="nn-example">
|
|
67
|
+
<title>OCR example using Backpropagation networks in ruby</title>
|
|
68
|
+
<p>
|
|
69
|
+
Let's imagine that we have to implement a program to identify simple patterns
|
|
70
|
+
(triangles, squares, crosses, etc). The main problem is that this program must
|
|
71
|
+
be resistant to random noise in the image (pixels with wrong values) and
|
|
72
|
+
line noise (similar to the unwanted direct current that we usually have in a
|
|
73
|
+
signal).
|
|
74
|
+
</p>
|
|
75
|
+
<p>
|
|
76
|
+
We can take an example of each pattern to be recognized, and train a
|
|
77
|
+
neural network to identify them.
|
|
78
|
+
</p>
|
|
79
|
+
<p>
|
|
80
|
+
We create a network with the following architecture: 256 input neurons,
|
|
81
|
+
128 neurons in a hidden layer, 3 output neurons. We feed this network
|
|
82
|
+
with 16x16 matrices (in fact will convert them to vectors of length 256).
|
|
83
|
+
Each pixel is represented with a number from 0 (white pixel) to 10
|
|
84
|
+
(black pixel). The output of this network if a 3 vector of dimension 3,
|
|
85
|
+
where ideally:
|
|
86
|
+
</p>
|
|
87
|
+
<ul>
|
|
88
|
+
<li>(1, 0, 0) for triangles</li>
|
|
89
|
+
<li>(0, 1, 0) for squares</li>
|
|
90
|
+
<li>(0, 0, 1) for crosses</li>
|
|
91
|
+
</ul>
|
|
92
|
+
<p>We train our backpropagation neural network using the following examples:</p>
|
|
93
|
+
<table>
|
|
94
|
+
<caption>Training patterns</caption>
|
|
95
|
+
<tr>
|
|
96
|
+
<td><img alt="Triangule training example" src="/images/t.png" /></td>
|
|
97
|
+
<td><img alt="Square training example" src="/images/s.png" /></td>
|
|
98
|
+
<td><img alt="Cross training example" src="/images/c.png" /></td>
|
|
99
|
+
</tr>
|
|
100
|
+
</table>
|
|
101
|
+
<p>And we repeat the training 20 times.</p>
|
|
79
102
|
<p>
|
|
80
103
|
The results we got when we evaluate patterns with our trained network are:
|
|
81
104
|
</p>
|
|
@@ -110,12 +133,13 @@ applications is OCR (opticar character recognition).</p>
|
|
|
110
133
|
<p>This is the source code used to elaborate this example (You can find it inside the zip file):</p>
|
|
111
134
|
<source>
|
|
112
135
|
<![CDATA[
|
|
136
|
+
require "rubygems"
|
|
137
|
+
require "ai4r/neural_network/backpropagation"
|
|
113
138
|
require File.dirname(__FILE__) + '/training_patterns'
|
|
114
139
|
require File.dirname(__FILE__) + '/patterns_with_noise'
|
|
115
140
|
require File.dirname(__FILE__) + '/patterns_with_base_noise'
|
|
116
|
-
require File.dirname(__FILE__) + '/../../lib/neural_network/backpropagation'
|
|
117
141
|
|
|
118
|
-
net = NeuralNetwork::Backpropagation.new([256, 128, 3])
|
|
142
|
+
net = Ai4r::NeuralNetwork::Backpropagation.new([256, 128, 3])
|
|
119
143
|
|
|
120
144
|
tr_input = TRIANGLE.flatten.collect { |input| input.to_f / 10}
|
|
121
145
|
sq_input = SQUARE.flatten.collect { |input| input.to_f / 10}
|
|
@@ -177,11 +201,11 @@ puts "#{net.eval(cr_with_base_noise).inspect} => #{result_label(net.eval(cr_with
|
|
|
177
201
|
value between 0 and 1. </p>
|
|
178
202
|
<p><img src="images/sigmoid.png" alt="Sigmoid function" /></p>
|
|
179
203
|
<p>
|
|
180
|
-
Sometimes you will have better results with f(x) = x. You can change the transference function from the default sigmoidal function to the linear one, overriding NeuralNetwork::Neuron.f and NeuralNetwork::Neuron.f_prime (derived function
|
|
204
|
+
Sometimes you will have better results with f(x) = x. You can change the transference function from the default sigmoidal function to the linear one, overriding Ai4r::NeuralNetwork::Neuron.f and Ai4r::NeuralNetwork::Neuron.f_prime (derived function
|
|
181
205
|
of f):</p>
|
|
182
206
|
<source>
|
|
183
207
|
<![CDATA[
|
|
184
|
-
class NeuralNetwork::Neuron
|
|
208
|
+
class Ai4r::NeuralNetwork::Neuron
|
|
185
209
|
def self.f(x)
|
|
186
210
|
x
|
|
187
211
|
end
|