biopsy 0.1.6.alpha → 0.1.7
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/README.md +4 -6
- data/lib/biopsy.rb +0 -1
- data/lib/biopsy/experiment.rb +21 -4
- data/lib/biopsy/objective_handler.rb +3 -4
- data/lib/biopsy/optimisers/spea2.rb +297 -0
- data/lib/biopsy/optimisers/tabu_search.rb +28 -2
- data/lib/biopsy/version.rb +2 -3
- data/test/helper.rb +2 -2
- data/test/test_experiment.rb +1 -1
- metadata +37 -55
- data/lib/biopsy/opt_algorithm.rb +0 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 64634183ae77fe731606799110dad261a5e65ac3
|
4
|
+
data.tar.gz: c0f9775091fde41e4856c2109a910ef7d4b31f7d
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 487ea9c1efa2d5447eec05293e4f6673459828a7f9e08918112893898dfe9fe65daa3675c36f07269fdb4376281c24fc2e62bf3350e58cd81496f89d1fac70c6
|
7
|
+
data.tar.gz: e04fc594a7b60fc4fbd2098421cdd7f99f83fe2c58542a37a7fe0bed4dc7fb2369fa96d8221a0c4a64cfaf6a146fd0e700d71e94d0f944691813cf28aad021fc
|
data/README.md
CHANGED
@@ -5,7 +5,7 @@ An automatic optimisation framework for programs and pipelines.
|
|
5
5
|
|
6
6
|
Biopsy is a framework for optimising any program or pipeline which produces a measurable output. By reducing the settings of one or more programs to a parameter space, and by carefully choosing objective functions with which to measure the output of the program(s), biopsy can use a range of optimisation strategies to rapidly find the settings that perform the best. Combined with a strategy for subsampling the input data, this can lead to vast time and performance improvements.
|
7
7
|
|
8
|
-
A simple example of the power of this approach is *de-novo* transcriptome assembly. Typically, the assembly process takes many GB of data as input, uses many GB of RAM and takes many hours to complete. This prevents researchers from performing full parameter sweeps, and they are therefore forced to use word-of-mouth and very basic optimisation to choose assembler settings. Assemblotron, which uses the Biopsy framework, can fully optimise any *de-novo* assembler to produce the optimal assembly possible given a particular input. This typically takes little more time than running a single assembly.
|
8
|
+
A simple example of the power of this approach is *de-novo* transcriptome assembly. Typically, the assembly process takes many GB of data as input, uses many GB of RAM and takes many hours to complete. This prevents researchers from performing full parameter sweeps, and they are therefore forced to use word-of-mouth and very basic optimisation to choose assembler settings. [Assemblotron](https://github.com/Blahah/assemblotron), which uses the Biopsy framework, can fully optimise any *de-novo* assembler to produce the optimal assembly possible given a particular input. This typically takes little more time than running a single assembly.
|
9
9
|
|
10
10
|
## Installation
|
11
11
|
|
@@ -15,8 +15,6 @@ Make sure you have Ruby installed, then:
|
|
15
15
|
|
16
16
|
## Usage
|
17
17
|
|
18
|
-
[]
|
19
|
-
|
20
18
|
Detailed usage instructions are on the wiki. Here's a quick overview:
|
21
19
|
|
22
20
|
1. Define your optimisation target. This is a program or pipeline you want to optimise, and you define it by filling in a template [YAML file](http://en.wikipedia.org/wiki/YAML). Easy!
|
@@ -59,8 +57,6 @@ Please don't report issues or request documentation until we are ready for beta
|
|
59
57
|
| ObjectiveHandler | DONE | DONE | DONE |
|
60
58
|
|
61
59
|
* ~ 20/24 tasks completed, ~83% done overall
|
62
|
-
* alpha released: 6th September 2013
|
63
|
-
* planned beta release date: 17th September 2013
|
64
60
|
|
65
61
|
### Documentation
|
66
62
|
|
@@ -69,4 +65,6 @@ Documentation is in development and will be released with the beta.
|
|
69
65
|
### Citation
|
70
66
|
|
71
67
|
This is *pre-release*, *pre-publication* academic software. In lieu of a paper to cite, please cite this Github repo and/or the [Figshare DOI (http://dx.doi.org/10.6084/m9.figshare.790660
|
72
|
-
)](http://dx.doi.org/10.6084/m9.figshare.790660) if your use of the software leads to a publication.
|
68
|
+
)](http://dx.doi.org/10.6084/m9.figshare.790660) if your use of the software leads to a publication.
|
69
|
+
|
70
|
+
[![Analytics](https://ga-beacon.appspot.com/UA-46900280-1/Blahah/biopsy)](https://github.com/Blahah/biopsy)
|
data/lib/biopsy.rb
CHANGED
@@ -5,7 +5,6 @@ require "biopsy/experiment"
|
|
5
5
|
require "biopsy/target"
|
6
6
|
require "biopsy/objective_handler"
|
7
7
|
require "biopsy/objective_function"
|
8
|
-
require "biopsy/opt_algorithm"
|
9
8
|
require "biopsy/optimisers/genetic_algorithm"
|
10
9
|
require "biopsy/optimisers/tabu_search"
|
11
10
|
require "biopsy/optimisers/parameter_sweeper"
|
data/lib/biopsy/experiment.rb
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
# Optimisation Framework: Experiment
|
1
|
+
1# Optimisation Framework: Experiment
|
2
2
|
#
|
3
3
|
# == Description
|
4
4
|
#
|
@@ -19,7 +19,8 @@ module Biopsy
|
|
19
19
|
attr_reader :inputs, :outputs, :retain_intermediates, :target, :start, :algorithm
|
20
20
|
|
21
21
|
# Returns a new Experiment
|
22
|
-
def initialize(target, options={}, start=nil, algorithm=nil)
|
22
|
+
def initialize(target, options={}, threads=4, start=nil, algorithm=nil)
|
23
|
+
@threads = threads
|
23
24
|
@start = start
|
24
25
|
@algorithm = algorithm
|
25
26
|
if target.is_a? Target
|
@@ -77,10 +78,16 @@ module Biopsy
|
|
77
78
|
while in_progress do
|
78
79
|
run_iteration
|
79
80
|
# update the best result
|
81
|
+
best = @best
|
80
82
|
@best = @algorithm.best
|
83
|
+
ptext = @best[:parameters].each_pair.map{ |k, v| "#{k}:#{v}" }.join(", ")
|
84
|
+
if (@best && @best.has_key?(:score) && best && best.has_key?(:score) && @best[:score] > best[:score])
|
85
|
+
puts "found a new best score: #{@best[:score]} for parameters #{ptext}"
|
86
|
+
end
|
81
87
|
# have we finished?
|
82
88
|
in_progress = !@algorithm.finished?
|
83
89
|
end
|
90
|
+
@algorithm.write_data if @algorithm.respond_to? :write_data
|
84
91
|
puts "found optimum score: #{@best[:score]} for parameters #{@best[:parameters]} in #{@iteration_count} iterations."
|
85
92
|
return @best
|
86
93
|
end
|
@@ -99,8 +106,9 @@ module Biopsy
|
|
99
106
|
if @scores.has_key? param_key
|
100
107
|
result = @scores[param_key]
|
101
108
|
else
|
102
|
-
result = @objective.run_for_output
|
109
|
+
result = @objective.run_for_output(raw_output, @threads)
|
103
110
|
@iteration_count += 1
|
111
|
+
self.print_progress(@iteration_count, @current_params, result, @best)
|
104
112
|
end
|
105
113
|
@scores[@current_params.to_s] = result
|
106
114
|
# get next steps from optimiser
|
@@ -109,6 +117,13 @@ module Biopsy
|
|
109
117
|
self.cleanup
|
110
118
|
end
|
111
119
|
|
120
|
+
def print_progress(iteration, params, score, best)
|
121
|
+
ptext = params.each_pair.map{ |k, v| "#{k}:#{v}" }.join(", ")
|
122
|
+
msg = "run #{iteration}. parameters: #{ptext} | score: #{score}"
|
123
|
+
msg += " | best #{best[:score]}" if (best && best.has_key?(:score))
|
124
|
+
puts msg
|
125
|
+
end
|
126
|
+
|
112
127
|
def cleanup
|
113
128
|
# TODO: make this work
|
114
129
|
# remove all but essential files
|
@@ -116,6 +131,8 @@ module Biopsy
|
|
116
131
|
@objectives.values.each{ |objective| essential_files += objective.essential_files }
|
117
132
|
end
|
118
133
|
Dir["*"].each do |file|
|
134
|
+
next
|
135
|
+
# TODO: implement this
|
119
136
|
next if File.directory? file
|
120
137
|
if essential_files && essential_files.include?(file)
|
121
138
|
`gzip #{file}` if Settings.instance.gzip_intermediates
|
@@ -141,4 +158,4 @@ module Biopsy
|
|
141
158
|
|
142
159
|
end # end of class RunHandler
|
143
160
|
|
144
|
-
end # end of module Biopsy
|
161
|
+
end # end of module Biopsy
|
@@ -113,8 +113,7 @@ module Biopsy
|
|
113
113
|
files = Dir[glob]
|
114
114
|
zerosize = files.reduce(false) { |empty, f| File.size(f) == 0 }
|
115
115
|
if files.empty? || zerosize
|
116
|
-
puts
|
117
|
-
raise ObjectiveHandlerError.new "output files for #{key} matching #{glob} do not exist or are empty"
|
116
|
+
puts ObjectiveHandlerError.new "output files for #{key} matching #{glob} do not exist or are empty"
|
118
117
|
return nil
|
119
118
|
end
|
120
119
|
output_files[key] = files.map { |f| File.expand_path(f) }
|
@@ -130,8 +129,8 @@ module Biopsy
|
|
130
129
|
return {:results => results,
|
131
130
|
:reduced => self.dimension_reduce(results)}
|
132
131
|
else
|
133
|
-
results.
|
134
|
-
return value[:result]
|
132
|
+
results.each_pair do |key, value|
|
133
|
+
return value.kind_of?(Hash) ? value[:result] : value
|
135
134
|
end
|
136
135
|
end
|
137
136
|
end
|
@@ -0,0 +1,297 @@
|
|
1
|
+
require 'csv'
|
2
|
+
require 'pp'
|
3
|
+
|
4
|
+
module Biopsy
|
5
|
+
class SPEA2
|
6
|
+
attr_reader :current, :probability_of_cross
|
7
|
+
attr_writer :probability_of_cross
|
8
|
+
attr_accessor :population_size, :archive_size, :probability_of_cross
|
9
|
+
def initialize(parameter_ranges, threads=8)
|
10
|
+
@parameter_ranges = parameter_ranges
|
11
|
+
@currently_mating = false
|
12
|
+
@probability_of_cross = 0.94
|
13
|
+
@population_size = 50
|
14
|
+
@archive_size = 15
|
15
|
+
@mutation_rate = 0.20
|
16
|
+
@generation_handler = Generation.new(@population_size, @archive_size)
|
17
|
+
@mating_handler = MatingHandler.new(@population_size, @probability_of_cross, @parameter_ranges, @mutation_rate)
|
18
|
+
end
|
19
|
+
def setup start_point
|
20
|
+
@current = {:parameters => start_point, :score => nil}
|
21
|
+
@best = @current
|
22
|
+
end
|
23
|
+
# accepts a parameter set and a score (which are added to population)
|
24
|
+
# returns another parameter set to be scored which is then accepted next iteration
|
25
|
+
def run_one_iteration(parameters, score)
|
26
|
+
self.run_one_mating_iteration(parameters, score) if @currently_mating == true
|
27
|
+
@current = {:parameters => parameters, :score => score}
|
28
|
+
# update best score?
|
29
|
+
self.update_best?
|
30
|
+
# push new parameter set into generation
|
31
|
+
@currently_mating = @generation_handler.add_new_individual(@current)
|
32
|
+
if @currently_mating == true
|
33
|
+
# begin mating process
|
34
|
+
# empty current population
|
35
|
+
@generation_handler.population_array = []
|
36
|
+
selected_to_mate = @mating_handler.select_mating_population(@generation_handler.archive_array)
|
37
|
+
self.get_children_to_score(selected_to_mate)
|
38
|
+
@children_to_score.pop[:parameters]
|
39
|
+
else
|
40
|
+
# get next parameter set to score
|
41
|
+
self.random_parameter_set
|
42
|
+
end
|
43
|
+
end
|
44
|
+
def get_children_to_score selected_to_mate
|
45
|
+
@children_to_score = @mating_handler.run(selected_to_mate)
|
46
|
+
end
|
47
|
+
def run_one_mating_iteration(parameters, score)
|
48
|
+
@generation_handler.add_new_individual({:parameters => parameters, :score => score})
|
49
|
+
if @children_to_score.length == 1
|
50
|
+
@currently_mating = false
|
51
|
+
end
|
52
|
+
@children_to_score.pop
|
53
|
+
end
|
54
|
+
# return true if algorithm knows its own starting point
|
55
|
+
def knows_starting_point?
|
56
|
+
return true
|
57
|
+
end
|
58
|
+
def update_best?
|
59
|
+
if @best[:score].nil? || @current[:score] > @best[:score]
|
60
|
+
@best = @current.clone
|
61
|
+
@iterations_since_best = 0
|
62
|
+
else
|
63
|
+
@iterations_since_best += 1
|
64
|
+
end
|
65
|
+
end
|
66
|
+
# returns a parameter set to be used as the starting point
|
67
|
+
def select_starting_point
|
68
|
+
self.random_parameter_set
|
69
|
+
end
|
70
|
+
# generates a random parameter set
|
71
|
+
def random_parameter_set
|
72
|
+
Hash[@parameter_ranges.map { |k, v| [k, v.sample] }]
|
73
|
+
end
|
74
|
+
# check if termination conditions are met
|
75
|
+
def finished?
|
76
|
+
true if rand < 0.00025
|
77
|
+
end
|
78
|
+
def best
|
79
|
+
@best
|
80
|
+
end
|
81
|
+
end # Algorithm
|
82
|
+
|
83
|
+
class Generation
|
84
|
+
attr_accessor :population_array, :archive_array
|
85
|
+
def initialize(population_size, archive_size)
|
86
|
+
@archive_generation = ArchiveGeneration.new
|
87
|
+
@population_size = population_size
|
88
|
+
@archive_size = archive_size
|
89
|
+
@population_array = []
|
90
|
+
@archive_array = []
|
91
|
+
end
|
92
|
+
def add_new_individual individual
|
93
|
+
@population_array << Individual.new(individual)
|
94
|
+
# run generation
|
95
|
+
if @population_array.length == @population_size
|
96
|
+
self.run_generation
|
97
|
+
true
|
98
|
+
else
|
99
|
+
false
|
100
|
+
end
|
101
|
+
end
|
102
|
+
def run_generation
|
103
|
+
@pop_and_archive = @population_array + @archive_array
|
104
|
+
FitnessAssignment.run(@pop_and_archive)
|
105
|
+
@archive_array = @archive_generation.run(@pop_and_archive, 10)
|
106
|
+
end
|
107
|
+
end # Generation
|
108
|
+
|
109
|
+
class Individual
|
110
|
+
attr_reader :parameters, :score, :fitness, :raw_fitness, :density, :distance_to_kth_point, :distance_to_origin
|
111
|
+
attr_writer :parameters, :fitness, :raw_fitness, :density, :distance_to_kth_point
|
112
|
+
def initialize(individual)
|
113
|
+
@parameters = individual[:parameters]
|
114
|
+
@score = individual[:score]
|
115
|
+
@distance_to_origin = FitnessAssignment.distance_to_origin(@score)
|
116
|
+
end
|
117
|
+
end # Individual
|
118
|
+
|
119
|
+
class FitnessAssignment
|
120
|
+
def self.run generation
|
121
|
+
self.score_raw_fitness(generation)
|
122
|
+
self.score_density(generation)
|
123
|
+
generation.each do |individual|
|
124
|
+
individual.fitness = individual.density + individual.raw_fitness
|
125
|
+
end
|
126
|
+
end
|
127
|
+
def self.distance_to_origin coordinates
|
128
|
+
# coordinates is a hash of coordinates
|
129
|
+
return rand(coordinates)
|
130
|
+
end
|
131
|
+
def self.distance_between_points(individual_one, individual_two)
|
132
|
+
if (individual_one.score - individual_two.score) < 0
|
133
|
+
return (individual_one.score - individual_two.score)*-1
|
134
|
+
else
|
135
|
+
return (individual_one.score - individual_two.score)
|
136
|
+
end
|
137
|
+
end
|
138
|
+
def self.score_density generation
|
139
|
+
generation_hash = map_points_distance(generation)
|
140
|
+
|
141
|
+
find_distance_to_kth_point(generation_hash)
|
142
|
+
|
143
|
+
generation_hash.each do |key,value|
|
144
|
+
value[0].density = (1.to_f/(value[0].distance_to_kth_point+2))
|
145
|
+
end
|
146
|
+
end
|
147
|
+
def self.map_points_distance generation
|
148
|
+
generation_clone = generation.clone
|
149
|
+
generation_hash = {}
|
150
|
+
generation_length = generation.length
|
151
|
+
(1..generation_length).each do |num|
|
152
|
+
generation_hash[num.to_s] = [generation_clone.pop, {}]
|
153
|
+
end
|
154
|
+
generation_hash.each do |key, value|
|
155
|
+
(1..generation_length).each do |num|
|
156
|
+
next if key.to_i == num or generation_hash[key][1].has_key?(num.to_s)
|
157
|
+
generation_hash[key][1][num.to_s] = distance_between_points(generation_hash[key][0], generation_hash[num.to_s][0])
|
158
|
+
generation_hash[num.to_s][1][key] = distance_between_points(generation_hash[key][0], generation_hash[num.to_s][0])
|
159
|
+
end
|
160
|
+
end
|
161
|
+
return generation_hash
|
162
|
+
end
|
163
|
+
def self.score_raw_fitness generation
|
164
|
+
generation.sort! { |a, b| a.distance_to_origin <=> b.distance_to_origin }.reverse!
|
165
|
+
counter = 0
|
166
|
+
previous_distance = +1.0/0.0
|
167
|
+
generation.each do |individual|
|
168
|
+
if previous_distance > individual.distance_to_origin
|
169
|
+
individual.raw_fitness = counter
|
170
|
+
else
|
171
|
+
individual.raw_fitness = counter - 1
|
172
|
+
end
|
173
|
+
previous_distance = individual.distance_to_origin
|
174
|
+
counter += 1
|
175
|
+
end
|
176
|
+
end
|
177
|
+
def self.find_distance_to_kth_point generation_hash
|
178
|
+
kth_point = Math.sqrt(generation_hash.length).round(0)
|
179
|
+
generation_hash.each do |key, value|
|
180
|
+
sorted_distances_array = value[1].sort_by {|k,v| v}
|
181
|
+
(1..sorted_distances_array.length).each do |num|
|
182
|
+
if num == (kth_point-1)
|
183
|
+
value[0].distance_to_kth_point = sorted_distances_array[num][1]
|
184
|
+
end
|
185
|
+
end
|
186
|
+
end
|
187
|
+
end
|
188
|
+
end # FitnessAssignment
|
189
|
+
|
190
|
+
class ArchiveGeneration
|
191
|
+
def run(generation, archive_size)
|
192
|
+
@generation = generation
|
193
|
+
archive = fill_archive(@generation)
|
194
|
+
|
195
|
+
if archive.length < archive_size
|
196
|
+
archive = self.further_archive_selection(archive, archive_size)
|
197
|
+
elsif archive.length > archive_size
|
198
|
+
archive = self.archive_truncation(archive)
|
199
|
+
end
|
200
|
+
return archive
|
201
|
+
end
|
202
|
+
def fill_archive generation
|
203
|
+
archive = []
|
204
|
+
@generation.clone.each do |individual|
|
205
|
+
if individual.fitness < 1
|
206
|
+
archive << individual
|
207
|
+
@generation.delete(individual)
|
208
|
+
end
|
209
|
+
end
|
210
|
+
return archive
|
211
|
+
end
|
212
|
+
def archive_truncation archive
|
213
|
+
return archive
|
214
|
+
end
|
215
|
+
def further_archive_selection archive, archive_size
|
216
|
+
@generation.sort!{|a, b| a.fitness <=> b.fitness}
|
217
|
+
@generation.clone.each do |individual|
|
218
|
+
if archive.length < archive_size
|
219
|
+
archive << individual
|
220
|
+
@generation.delete(individual)
|
221
|
+
else
|
222
|
+
break
|
223
|
+
end
|
224
|
+
end
|
225
|
+
return archive
|
226
|
+
end
|
227
|
+
end # ArchiveGeneration
|
228
|
+
|
229
|
+
class MatingHandler
|
230
|
+
def initialize(population_size, probability_of_cross, parameter_ranges, mutation_rate)
|
231
|
+
@probability_of_cross = probability_of_cross
|
232
|
+
@parameter_ranges = parameter_ranges
|
233
|
+
@population_size = population_size
|
234
|
+
@mutation_rate = mutation_rate
|
235
|
+
end
|
236
|
+
def run selected_to_mate
|
237
|
+
new_population = reproduce(selected_to_mate)
|
238
|
+
return new_population
|
239
|
+
end
|
240
|
+
def select_mating_population archive
|
241
|
+
Array.new(@population_size) {self.binary_tournament(archive)}
|
242
|
+
end
|
243
|
+
def binary_tournament(archive)
|
244
|
+
individual_one = rand(archive.size)
|
245
|
+
individual_two = rand(archive.size)
|
246
|
+
individual_two = rand(archive.size) while individual_one == individual_two
|
247
|
+
if archive[individual_one].fitness > archive[individual_two].fitness
|
248
|
+
return archive[individual_one]
|
249
|
+
else
|
250
|
+
return archive[individual_two]
|
251
|
+
end
|
252
|
+
end
|
253
|
+
def reproduce selected_to_mate
|
254
|
+
# loop through selected
|
255
|
+
# mate first with last
|
256
|
+
# mate index 1 with index 2
|
257
|
+
# mate index 2 with index 1
|
258
|
+
# mate index 3 with index 4
|
259
|
+
children = []
|
260
|
+
selected_to_mate.each_with_index do |parent_one, index|
|
261
|
+
if index == selected_to_mate.size+1
|
262
|
+
parent_two = selected_to_mate[0]
|
263
|
+
elsif index.modulo(2) == 1
|
264
|
+
parent_two = selected_to_mate[index-1]
|
265
|
+
else
|
266
|
+
parent_two = selected_to_mate[index+1]
|
267
|
+
end
|
268
|
+
children << self.mate(parent_one, parent_two)
|
269
|
+
end
|
270
|
+
return children
|
271
|
+
end
|
272
|
+
def mate(parent_one, parent_two)
|
273
|
+
child = self.crossover(parent_one, parent_two)
|
274
|
+
child = self.mutation(child)
|
275
|
+
return child
|
276
|
+
end
|
277
|
+
def crossover(parent_one, parent_two)
|
278
|
+
return {:parameters => parent_one.parameters} if rand >= @probability_of_cross
|
279
|
+
# create the hash if parameter_ranges that can be passed to the +:Individual:+ class
|
280
|
+
child_parameters = {}
|
281
|
+
parent_one.parameters.each do |parent_one_key, parent_one_value|
|
282
|
+
if rand < 0.5
|
283
|
+
child_parameters[parent_one_key.to_sym] = parent_one_value
|
284
|
+
else
|
285
|
+
child_parameters[parent_one_key.to_sym] = parent_two.parameters[parent_one_key.to_sym]
|
286
|
+
end
|
287
|
+
end
|
288
|
+
return {:parameters => child_parameters}
|
289
|
+
end
|
290
|
+
def mutation child
|
291
|
+
child[:parameters].each do |key, value|
|
292
|
+
child[:parameters][key.to_sym] = @parameter_ranges[key.to_sym].sample(1)[0] if rand < @mutation_rate
|
293
|
+
end
|
294
|
+
return child
|
295
|
+
end
|
296
|
+
end # MatingHandler
|
297
|
+
end # Biopsy
|
@@ -165,7 +165,8 @@ module Biopsy
|
|
165
165
|
Thread = Struct.new(:best, :tabu, :distributions,
|
166
166
|
:standard_deviations, :recent_scores,
|
167
167
|
:iterations_since_best, :backtracks,
|
168
|
-
:current, :current_hood, :loaded
|
168
|
+
:current, :current_hood, :loaded,
|
169
|
+
:score_history, :best_history)
|
169
170
|
|
170
171
|
def initialize(parameter_ranges, threads=8, limit=nil)
|
171
172
|
|
@@ -191,6 +192,8 @@ module Biopsy
|
|
191
192
|
@jump_cutoff = 10
|
192
193
|
|
193
194
|
# logging
|
195
|
+
@score_history = []
|
196
|
+
@best_history = []
|
194
197
|
@log_data = false
|
195
198
|
@logfiles = {}
|
196
199
|
self.log_setup
|
@@ -241,6 +244,8 @@ module Biopsy
|
|
241
244
|
@best = @current
|
242
245
|
@standard_deviations = {}
|
243
246
|
@recent_scores = []
|
247
|
+
@score_history = []
|
248
|
+
@best_history = []
|
244
249
|
@tabu = Set.new
|
245
250
|
self.define_neighbourhood_structure
|
246
251
|
@current_hood = Biopsy::Hood.new(@distributions, @max_hood_size, @tabu)
|
@@ -434,6 +439,10 @@ module Biopsy
|
|
434
439
|
end
|
435
440
|
|
436
441
|
def log
|
442
|
+
if @current[:score]
|
443
|
+
@score_history << @current[:score]
|
444
|
+
@best_history << @best[:score]
|
445
|
+
end
|
437
446
|
if @log_data
|
438
447
|
@logfiles[:standard_deviations] << @distributions.map { |k, d| d.sd }
|
439
448
|
@logfiles[:best] << [@best[:score]]
|
@@ -460,6 +469,23 @@ module Biopsy
|
|
460
469
|
Hash[@ranges.map { |p, r| [p, r.sample] }]
|
461
470
|
end
|
462
471
|
|
472
|
+
def write_data
|
473
|
+
require 'csv'
|
474
|
+
now = Time.now.to_i
|
475
|
+
CSV.open("../#{now}_scores.csv", "w") do |c|
|
476
|
+
c << %w(iteration thread score best)
|
477
|
+
@threads.each_with_index do |t, t_idx|
|
478
|
+
sh = t.score_history
|
479
|
+
bh = t.best_history
|
480
|
+
sh.zip(bh).each_with_index do |pair, i|
|
481
|
+
c << [i, t_idx] + pair
|
482
|
+
end
|
483
|
+
end
|
484
|
+
end
|
485
|
+
path = File.expand_path("../#{now}_scores.csv")
|
486
|
+
puts "wrote TabuSearch run data to #{path}"
|
487
|
+
end
|
488
|
+
|
463
489
|
end # TabuSearch
|
464
490
|
|
465
|
-
end # Biopsy
|
491
|
+
end # Biopsy
|
data/lib/biopsy/version.rb
CHANGED
data/test/helper.rb
CHANGED
@@ -7,7 +7,7 @@ SimpleCov.formatter = SimpleCov::Formatter::MultiFormatter[
|
|
7
7
|
]
|
8
8
|
SimpleCov.start
|
9
9
|
|
10
|
-
require '
|
10
|
+
require 'minitest/autorun'
|
11
11
|
begin; require 'turn/autorun'; rescue LoadError; end
|
12
12
|
require 'shoulda-context'
|
13
13
|
require 'biopsy'
|
@@ -161,4 +161,4 @@ end
|
|
161
161
|
end
|
162
162
|
end
|
163
163
|
|
164
|
-
end
|
164
|
+
end
|
data/test/test_experiment.rb
CHANGED
@@ -47,7 +47,7 @@ class TestExperiment < Test::Unit::TestCase
|
|
47
47
|
|
48
48
|
should "respect user's choice of starting point" do
|
49
49
|
s = {:a => 2, :b => 4}
|
50
|
-
e = Biopsy::Experiment.new('target_test', s)
|
50
|
+
e = Biopsy::Experiment.new('target_test', start=s)
|
51
51
|
assert_equal s, e.start
|
52
52
|
end
|
53
53
|
|
metadata
CHANGED
@@ -1,142 +1,125 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: biopsy
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
5
|
-
prerelease: 6
|
4
|
+
version: 0.1.7
|
6
5
|
platform: ruby
|
7
6
|
authors:
|
8
7
|
- Richard Smith
|
9
8
|
autorequire:
|
10
9
|
bindir: bin
|
11
10
|
cert_chain: []
|
12
|
-
date:
|
11
|
+
date: 2014-04-22 00:00:00.000000000 Z
|
13
12
|
dependencies:
|
14
13
|
- !ruby/object:Gem::Dependency
|
15
14
|
name: rake
|
16
15
|
requirement: !ruby/object:Gem::Requirement
|
17
|
-
none: false
|
18
16
|
requirements:
|
19
|
-
- - ~>
|
17
|
+
- - "~>"
|
20
18
|
- !ruby/object:Gem::Version
|
21
19
|
version: 10.1.0
|
22
20
|
type: :runtime
|
23
21
|
prerelease: false
|
24
22
|
version_requirements: !ruby/object:Gem::Requirement
|
25
|
-
none: false
|
26
23
|
requirements:
|
27
|
-
- - ~>
|
24
|
+
- - "~>"
|
28
25
|
- !ruby/object:Gem::Version
|
29
26
|
version: 10.1.0
|
30
27
|
- !ruby/object:Gem::Dependency
|
31
28
|
name: threach
|
32
29
|
requirement: !ruby/object:Gem::Requirement
|
33
|
-
none: false
|
34
30
|
requirements:
|
35
|
-
- -
|
31
|
+
- - ">="
|
36
32
|
- !ruby/object:Gem::Version
|
37
33
|
version: '0'
|
38
34
|
type: :runtime
|
39
35
|
prerelease: false
|
40
36
|
version_requirements: !ruby/object:Gem::Requirement
|
41
|
-
none: false
|
42
37
|
requirements:
|
43
|
-
- -
|
38
|
+
- - ">="
|
44
39
|
- !ruby/object:Gem::Version
|
45
40
|
version: '0'
|
46
41
|
- !ruby/object:Gem::Dependency
|
47
42
|
name: rubystats
|
48
43
|
requirement: !ruby/object:Gem::Requirement
|
49
|
-
none: false
|
50
44
|
requirements:
|
51
|
-
- -
|
45
|
+
- - ">="
|
52
46
|
- !ruby/object:Gem::Version
|
53
47
|
version: '0'
|
54
48
|
type: :runtime
|
55
49
|
prerelease: false
|
56
50
|
version_requirements: !ruby/object:Gem::Requirement
|
57
|
-
none: false
|
58
51
|
requirements:
|
59
|
-
- -
|
52
|
+
- - ">="
|
60
53
|
- !ruby/object:Gem::Version
|
61
54
|
version: '0'
|
62
55
|
- !ruby/object:Gem::Dependency
|
63
56
|
name: statsample
|
64
57
|
requirement: !ruby/object:Gem::Requirement
|
65
|
-
none: false
|
66
58
|
requirements:
|
67
|
-
- -
|
59
|
+
- - ">="
|
68
60
|
- !ruby/object:Gem::Version
|
69
61
|
version: '0'
|
70
62
|
type: :runtime
|
71
63
|
prerelease: false
|
72
64
|
version_requirements: !ruby/object:Gem::Requirement
|
73
|
-
none: false
|
74
65
|
requirements:
|
75
|
-
- -
|
66
|
+
- - ">="
|
76
67
|
- !ruby/object:Gem::Version
|
77
68
|
version: '0'
|
78
69
|
- !ruby/object:Gem::Dependency
|
79
70
|
name: turn
|
80
71
|
requirement: !ruby/object:Gem::Requirement
|
81
|
-
none: false
|
82
72
|
requirements:
|
83
|
-
- -
|
73
|
+
- - ">="
|
84
74
|
- !ruby/object:Gem::Version
|
85
75
|
version: '0'
|
86
76
|
type: :development
|
87
77
|
prerelease: false
|
88
78
|
version_requirements: !ruby/object:Gem::Requirement
|
89
|
-
none: false
|
90
79
|
requirements:
|
91
|
-
- -
|
80
|
+
- - ">="
|
92
81
|
- !ruby/object:Gem::Version
|
93
82
|
version: '0'
|
94
83
|
- !ruby/object:Gem::Dependency
|
95
84
|
name: simplecov
|
96
85
|
requirement: !ruby/object:Gem::Requirement
|
97
|
-
none: false
|
98
86
|
requirements:
|
99
|
-
- -
|
87
|
+
- - ">="
|
100
88
|
- !ruby/object:Gem::Version
|
101
89
|
version: '0'
|
102
90
|
type: :development
|
103
91
|
prerelease: false
|
104
92
|
version_requirements: !ruby/object:Gem::Requirement
|
105
|
-
none: false
|
106
93
|
requirements:
|
107
|
-
- -
|
94
|
+
- - ">="
|
108
95
|
- !ruby/object:Gem::Version
|
109
96
|
version: '0'
|
110
97
|
- !ruby/object:Gem::Dependency
|
111
98
|
name: shoulda-context
|
112
99
|
requirement: !ruby/object:Gem::Requirement
|
113
|
-
none: false
|
114
100
|
requirements:
|
115
|
-
- -
|
101
|
+
- - ">="
|
116
102
|
- !ruby/object:Gem::Version
|
117
103
|
version: '0'
|
118
104
|
type: :development
|
119
105
|
prerelease: false
|
120
106
|
version_requirements: !ruby/object:Gem::Requirement
|
121
|
-
none: false
|
122
107
|
requirements:
|
123
|
-
- -
|
108
|
+
- - ">="
|
124
109
|
- !ruby/object:Gem::Version
|
125
110
|
version: '0'
|
126
111
|
- !ruby/object:Gem::Dependency
|
127
112
|
name: coveralls
|
128
113
|
requirement: !ruby/object:Gem::Requirement
|
129
|
-
none: false
|
130
114
|
requirements:
|
131
|
-
- - ~>
|
115
|
+
- - "~>"
|
132
116
|
- !ruby/object:Gem::Version
|
133
117
|
version: 0.6.7
|
134
118
|
type: :development
|
135
119
|
prerelease: false
|
136
120
|
version_requirements: !ruby/object:Gem::Requirement
|
137
|
-
none: false
|
138
121
|
requirements:
|
139
|
-
- - ~>
|
122
|
+
- - "~>"
|
140
123
|
- !ruby/object:Gem::Version
|
141
124
|
version: 0.6.7
|
142
125
|
description:
|
@@ -146,51 +129,50 @@ extensions: []
|
|
146
129
|
extra_rdoc_files: []
|
147
130
|
files:
|
148
131
|
- Rakefile
|
149
|
-
- lib/biopsy
|
150
|
-
- lib/biopsy/
|
151
|
-
- lib/biopsy/objective_function.rb
|
132
|
+
- lib/biopsy.rb
|
133
|
+
- lib/biopsy/version.rb
|
152
134
|
- lib/biopsy/objective_handler.rb
|
153
135
|
- lib/biopsy/objectives/fastest_optimum.rb
|
154
|
-
- lib/biopsy/
|
136
|
+
- lib/biopsy/settings.rb
|
137
|
+
- lib/biopsy/optimisers/tabu_search.rb
|
155
138
|
- lib/biopsy/optimisers/genetic_algorithm.rb
|
139
|
+
- lib/biopsy/optimisers/spea2.rb
|
156
140
|
- lib/biopsy/optimisers/parameter_sweeper.rb
|
157
|
-
- lib/biopsy/
|
158
|
-
- lib/biopsy/settings.rb
|
141
|
+
- lib/biopsy/objective_function.rb
|
159
142
|
- lib/biopsy/target.rb
|
160
|
-
- lib/biopsy/
|
161
|
-
- lib/biopsy.rb
|
162
|
-
- test/
|
163
|
-
- test/test_experiment.rb
|
164
|
-
- test/test_file.rb
|
143
|
+
- lib/biopsy/base_extensions.rb
|
144
|
+
- lib/biopsy/experiment.rb
|
145
|
+
- test/test_target.rb
|
165
146
|
- test/test_hash.rb
|
147
|
+
- test/test_string.rb
|
148
|
+
- test/helper.rb
|
166
149
|
- test/test_objective_handler.rb
|
150
|
+
- test/test_file.rb
|
167
151
|
- test/test_settings.rb
|
168
|
-
- test/
|
169
|
-
- test/test_target.rb
|
152
|
+
- test/test_experiment.rb
|
170
153
|
- README.md
|
171
154
|
- LICENSE.txt
|
172
155
|
homepage: https://github.com/Blahah/biopsy
|
173
156
|
licenses: []
|
157
|
+
metadata: {}
|
174
158
|
post_install_message:
|
175
159
|
rdoc_options: []
|
176
160
|
require_paths:
|
177
161
|
- lib
|
178
162
|
required_ruby_version: !ruby/object:Gem::Requirement
|
179
|
-
none: false
|
180
163
|
requirements:
|
181
|
-
- -
|
164
|
+
- - ">="
|
182
165
|
- !ruby/object:Gem::Version
|
183
166
|
version: '0'
|
184
167
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
185
|
-
none: false
|
186
168
|
requirements:
|
187
|
-
- -
|
169
|
+
- - ">="
|
188
170
|
- !ruby/object:Gem::Version
|
189
|
-
version:
|
171
|
+
version: '0'
|
190
172
|
requirements: []
|
191
173
|
rubyforge_project:
|
192
|
-
rubygems_version: 1.
|
174
|
+
rubygems_version: 2.1.4
|
193
175
|
signing_key:
|
194
|
-
specification_version:
|
176
|
+
specification_version: 4
|
195
177
|
summary: framework for optimising any computational pipeline or program
|
196
178
|
test_files: []
|
data/lib/biopsy/opt_algorithm.rb
DELETED
File without changes
|