sisfc 0.1.0 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.projections.json +12 -0
- data/.travis.yml +6 -0
- data/README.md +37 -19
- data/Rakefile +4 -2
- data/TODO +5 -0
- data/bin/sisfc +29 -6
- data/examples/generator.R +3 -1
- data/examples/simulator.conf +80 -29
- data/lib/sisfc.rb +4 -4
- data/lib/sisfc/configuration.rb +73 -6
- data/lib/sisfc/data_center.rb +42 -29
- data/lib/sisfc/evaluation.rb +23 -15
- data/lib/sisfc/event.rb +9 -6
- data/lib/sisfc/generator.rb +14 -21
- data/lib/sisfc/latency_manager.rb +65 -0
- data/lib/sisfc/logger.rb +28 -0
- data/lib/sisfc/request.rb +42 -85
- data/lib/sisfc/service_type.rb +2 -0
- data/lib/sisfc/simulation.rb +234 -47
- data/lib/sisfc/sorted_array.rb +2 -0
- data/lib/sisfc/statistics.rb +37 -3
- data/lib/sisfc/support/dsl_helper.rb +2 -0
- data/lib/sisfc/version.rb +3 -1
- data/lib/sisfc/vm.rb +46 -27
- data/sisfc.gemspec +9 -5
- data/spec/minitest_helper.rb +9 -0
- data/{test/sisfc/configuration_test.rb → spec/sisfc/configuration_spec.rb} +4 -2
- data/spec/sisfc/data_center_spec.rb +19 -0
- data/{test/sisfc/evaluation_test.rb → spec/sisfc/evaluation_spec.rb} +5 -3
- data/{test/sisfc/generator_test.rb → spec/sisfc/generator_spec.rb} +21 -18
- data/spec/sisfc/latency_manager_spec.rb +13 -0
- data/spec/sisfc/reference_configuration.rb +534 -0
- data/spec/sisfc/request_spec.rb +19 -0
- metadata +115 -49
- data/test/sisfc/reference_configuration.rb +0 -191
- data/test/sisfc/request_test.rb +0 -13
- data/test/test_helper.rb +0 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: fac842a4649d5c5a521c3281fa86c202dac7ca7e96a558e919e0fa325acebe3d
|
4
|
+
data.tar.gz: 0ad6254580131f7c5249ba558b793d9d5a493bfd852f1c39226fdfc50be1ca3a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: aecd32a8ea709fe01e17c38af9cebb5bb0bc018c4f7265e5bb0ecf76cc76164ec455706eadf3368afefc7ce0a4444f54c78d281e370363fd99b7e6dfdfb84361
|
7
|
+
data.tar.gz: '06923f6dc02f509e7e22ff52cf35e6ed905ea7731791300dbd45db03df265f756a927e2a4f1472a2d122a1e7960f6e5040dadd8be892e3fa66af8677101194af'
|
data/.projections.json
ADDED
data/.travis.yml
ADDED
data/README.md
CHANGED
@@ -1,28 +1,13 @@
|
|
1
1
|
# A Simulator for IT Service in Federated Clouds (SISFC)
|
2
2
|
|
3
|
+
[![Gem Version](https://badge.fury.io/rb/sisfc.svg)](https://badge.fury.io/rb/sisfc)
|
4
|
+
[![Build Status](https://travis-ci.org/mtortonesi/sisfc.png?branch=master)](https://travis-ci.org/mtortonesi/sisfc)
|
5
|
+
[![Code Climate](https://codeclimate.com/github/mtortonesi/sisfc.png)](https://codeclimate.com/github/mtortonesi/sisfc)
|
6
|
+
|
3
7
|
SISFC is a simulator designed to reenact the behaviour of IT services in
|
4
8
|
federated Cloud environments.
|
5
9
|
|
6
10
|
|
7
|
-
## References
|
8
|
-
|
9
|
-
This simulator (more precisely an earlier version of it) was used in the
|
10
|
-
following research papers:
|
11
|
-
|
12
|
-
1. L. Foschini, M. Tortonesi, "Adaptive and Business-driven Service Placement
|
13
|
-
in Federated Cloud Computing Environments", in Proceedings of the 8th
|
14
|
-
IFIP/IEEE International Workshop on Business-driven IT Management (BDIM 2013),
|
15
|
-
27 May 2013, Ghent, Belgium.
|
16
|
-
|
17
|
-
2. G. Grabarnik, L. Shwartz, M. Tortonesi, "Business-Driven Optimization of
|
18
|
-
Component Placement for Complex Services in Federated Clouds", to appear in
|
19
|
-
Proceedings of the 14th IEEE/IFIP Network Operations and Management Symposium
|
20
|
-
(NOMS 2014) - Mini-conference track, 5-9 May 2014, Krakow, Poland.
|
21
|
-
|
22
|
-
Please, consider citing some of these papers if you find this simulator useful
|
23
|
-
for your research.
|
24
|
-
|
25
|
-
|
26
11
|
## Installation
|
27
12
|
|
28
13
|
You can install the SISFC simulator through RubyGems:
|
@@ -53,3 +38,36 @@ Note that the SISFC was not designed to be run directly by users, but instead
|
|
53
38
|
to be integrated within automated tools that implement a continuous
|
54
39
|
optimization framework (for instance, built on top of our [mhl metaheuristics
|
55
40
|
library](https://github.com/mtortonesi/ruby-mhl)).
|
41
|
+
|
42
|
+
|
43
|
+
## References
|
44
|
+
|
45
|
+
The SISFC simulator was used in the following research papers:
|
46
|
+
|
47
|
+
1. L. Foschini, M. Tortonesi, "Adaptive and Business-driven Service Placement
|
48
|
+
in Federated Cloud Computing Environments", in Proceedings of the 8th
|
49
|
+
IFIP/IEEE International Workshop on Business-driven IT Management (BDIM 2013),
|
50
|
+
27 May 2013, Ghent, Belgium.
|
51
|
+
|
52
|
+
2. G. Grabarnik, L. Shwartz, M. Tortonesi, "Business-Driven Optimization of
|
53
|
+
Component Placement for Complex Services in Federated Clouds", in
|
54
|
+
Proceedings of the 14th IEEE/IFIP Network Operations and Management Symposium
|
55
|
+
(NOMS 2014) - Mini-conference track, 5-9 May 2014, Krakow, Poland.
|
56
|
+
|
57
|
+
3. M. Tortonesi, L. Foschini, "Business-driven Service Placement for Highly
|
58
|
+
Dynamic and Distributed Cloud Systems", IEEE Transactions on Cloud
|
59
|
+
Computing, 2016 (in print).
|
60
|
+
|
61
|
+
Please, consider citing some of these papers if you find this simulator useful
|
62
|
+
for your research.
|
63
|
+
|
64
|
+
|
65
|
+
## Acknowledgements
|
66
|
+
|
67
|
+
The research work that led to the development of SISFC was supported in part by
|
68
|
+
the [DICET - INMOTO - ORganization of Cultural HEritage for Smart
|
69
|
+
Tourism and Real-time Accessibility (OR.C.HE.S.T.R.A.)](http://www.ponrec.it/open-data/progetti/scheda-progetto?ProgettoID=5835)
|
70
|
+
project, funded by the Italian Ministry of University and Research on Axis II
|
71
|
+
of the National operative programme (PON) for Research and Competitiveness
|
72
|
+
2007-13 within the call 'Smart Cities and Communities and Social Innovation'
|
73
|
+
(D.D. n.84/Ric., 2 March 2012).
|
data/Rakefile
CHANGED
@@ -3,7 +3,9 @@ require 'bundler/gem_tasks'
|
|
3
3
|
require 'rake/testtask'
|
4
4
|
|
5
5
|
Rake::TestTask.new do |t|
|
6
|
-
t.libs <<
|
7
|
-
t.test_files =
|
6
|
+
t.libs << "spec"
|
7
|
+
t.test_files = FileList['spec/**/*_spec.rb']
|
8
8
|
t.verbose = true
|
9
9
|
end
|
10
|
+
|
11
|
+
# task(default: :test)
|
data/TODO
ADDED
data/bin/sisfc
CHANGED
@@ -3,18 +3,41 @@
|
|
3
3
|
begin
|
4
4
|
require 'sisfc'
|
5
5
|
require 'sisfc/evaluation'
|
6
|
-
require 'awesome_print'
|
7
6
|
rescue LoadError
|
8
7
|
require 'rubygems'
|
9
8
|
require 'sisfc'
|
10
9
|
require 'sisfc/evaluation'
|
11
|
-
require 'awesome_print'
|
12
10
|
end
|
13
11
|
|
14
12
|
|
15
|
-
|
16
|
-
|
17
|
-
|
13
|
+
def do_abort(message)
|
14
|
+
abort <<-EOS.gsub(/^\s+\|/, '')
|
15
|
+
|#{message}
|
16
|
+
|
|
17
|
+
|Usage:
|
18
|
+
| #{File.basename(__FILE__)} simulator_config_file vm_allocation_config_file
|
19
|
+
|
|
20
|
+
EOS
|
21
|
+
end
|
22
|
+
|
23
|
+
|
24
|
+
if File.expand_path(__FILE__) == File.expand_path($0)
|
25
|
+
# make sure both required arguments were given
|
26
|
+
case ARGV.size
|
27
|
+
when 1 then
|
28
|
+
do_abort("Missing VM allocation configuration file!")
|
29
|
+
when 0 then
|
30
|
+
do_abort("Missing simulator and VM allocation configuration files!")
|
31
|
+
end
|
32
|
+
|
33
|
+
# make sure simulator config file exists
|
34
|
+
unless File.exists? ARGV[0]
|
35
|
+
do_abort("Invalid simulator configuration file!")
|
36
|
+
end
|
37
|
+
|
38
|
+
# make sure vm allocation config file exists
|
39
|
+
unless File.exists? ARGV[1]
|
40
|
+
do_abort("Invalid VM allocation configuration file!")
|
18
41
|
end
|
19
42
|
|
20
43
|
# load simulation configuration
|
@@ -30,5 +53,5 @@ if __FILE__ == $0
|
|
30
53
|
|
31
54
|
# Print results
|
32
55
|
puts 'Result:'
|
33
|
-
|
56
|
+
puts res
|
34
57
|
end
|
data/examples/generator.R
CHANGED
@@ -36,10 +36,12 @@ data.center.ids <- sample.int(num.data.centers, length(generation.times), replac
|
|
36
36
|
latencies <- rtruncnorm(length(generation.times), a=min.latency, b=Inf, mean=mu, sd=sigma)
|
37
37
|
arrival.times <- generation.times + latencies
|
38
38
|
workflow.type.ids <- rep(1, length(generation.times))
|
39
|
+
customer.ids <- rep(0, length(generation.times))
|
39
40
|
|
40
41
|
# prepare data frame and output it on the console
|
41
42
|
df <- data.frame(Generation.Time = generation.times,
|
42
43
|
Data.Center.ID = data.center.ids,
|
43
44
|
Arrival.Time = arrival.times,
|
44
|
-
Workflow.Type.ID = workflow.type.ids
|
45
|
+
Workflow.Type.ID = workflow.type.ids,
|
46
|
+
Customer.ID = customer.ids)
|
45
47
|
write.csv(df[order(df$Arrival.Time),], row.names=F)
|
data/examples/simulator.conf
CHANGED
@@ -1,3 +1,15 @@
|
|
1
|
+
####################################################################################
|
2
|
+
# #
|
3
|
+
# ▄▄ ▄▄▄▄ #
|
4
|
+
# ██ ▀▀██ ██ #
|
5
|
+
# ▄████▄ ██▄███▄ ▄▄█████▄ ▄████▄ ██ ▄████▄ ███████ ▄████▄ #
|
6
|
+
# ██▀ ▀██ ██▀ ▀██ ██▄▄▄▄ ▀ ██▀ ▀██ ██ ██▄▄▄▄██ ██ ██▄▄▄▄██ #
|
7
|
+
# ██ ██ ██ ██ ▀▀▀▀██▄ ██ ██ ██ ██▀▀▀▀▀▀ ██ ██▀▀▀▀▀▀ #
|
8
|
+
# ▀██▄▄██▀ ███▄▄██▀ █▄▄▄▄▄██ ▀██▄▄██▀ ██▄▄▄ ▀██▄▄▄▄█ ██▄▄▄ ▀██▄▄▄▄█ #
|
9
|
+
# ▀▀▀▀ ▀▀ ▀▀▀ ▀▀▀▀▀▀ ▀▀▀▀ ▀▀▀▀ ▀▀▀▀▀ ▀▀▀▀ ▀▀▀▀▀ #
|
10
|
+
# #
|
11
|
+
####################################################################################
|
12
|
+
|
1
13
|
start_time DateTime.civil(2013,1,18,0,0,0)
|
2
14
|
warmup_duration 10.seconds
|
3
15
|
duration 1.minute
|
@@ -5,66 +17,105 @@ duration 1.minute
|
|
5
17
|
|
6
18
|
data_centers \
|
7
19
|
1 => {
|
8
|
-
:
|
9
|
-
:
|
10
|
-
:
|
20
|
+
maximum_vm_capacity: {
|
21
|
+
medium: 50_000,
|
22
|
+
large: 50_000,
|
11
23
|
},
|
24
|
+
location_id: 0,
|
12
25
|
},
|
13
26
|
2 => {
|
14
|
-
:
|
15
|
-
:
|
16
|
-
:
|
27
|
+
maximum_vm_capacity: {
|
28
|
+
medium: 50_000,
|
29
|
+
large: 50_000,
|
17
30
|
},
|
31
|
+
location_id: 1,
|
18
32
|
}
|
19
33
|
|
34
|
+
latency_models \
|
35
|
+
[
|
36
|
+
# location 0
|
37
|
+
[
|
38
|
+
{ distribution: :gaussian,
|
39
|
+
mean: 0.009,
|
40
|
+
sd: 0.001 },
|
41
|
+
{ distribution: :gaussian,
|
42
|
+
mean: 0.009,
|
43
|
+
sd: 0.001 },
|
44
|
+
],
|
45
|
+
# location 1
|
46
|
+
[
|
47
|
+
{ distribution: :gaussian,
|
48
|
+
mean: 0.009,
|
49
|
+
sd: 0.001 },
|
50
|
+
],
|
51
|
+
]
|
52
|
+
|
53
|
+
|
54
|
+
customers \
|
55
|
+
[
|
56
|
+
# first and only customer (id: 0) is in location with id=2
|
57
|
+
{ location_id: 2 },
|
58
|
+
]
|
59
|
+
|
20
60
|
|
21
61
|
service_component_types \
|
22
62
|
'Web Server' => {
|
23
|
-
:
|
24
|
-
:
|
25
|
-
:
|
26
|
-
|
27
|
-
|
63
|
+
allowed_vm_types: [ :medium ],
|
64
|
+
service_time_distribution: {
|
65
|
+
medium: { distribution: :gaussian,
|
66
|
+
mean: 0.009, # 1 request processed every 9ms
|
67
|
+
sd: 0.001 },
|
28
68
|
},
|
29
|
-
:
|
69
|
+
estimated_workload: 50,
|
30
70
|
},
|
31
71
|
'App Server' => {
|
32
|
-
:
|
33
|
-
:
|
34
|
-
:
|
35
|
-
|
36
|
-
|
72
|
+
allowed_vm_types: [ :large ],
|
73
|
+
service_time_distribution: {
|
74
|
+
large: { distribution: :gaussian,
|
75
|
+
mean: 0.012, # 1 request processed every 12ms
|
76
|
+
sd: 0.002 },
|
37
77
|
},
|
38
|
-
:
|
78
|
+
estimated_workload: 70,
|
39
79
|
}
|
40
80
|
|
41
81
|
|
42
82
|
# workflow (or job) types descriptions
|
43
83
|
workflow_types \
|
44
84
|
1 => {
|
45
|
-
:
|
46
|
-
{ :
|
47
|
-
{ :
|
85
|
+
component_sequence: [
|
86
|
+
{ name: 'Web Server' }, # no need for type: dedicated / shared
|
87
|
+
{ name: 'App Server' },
|
48
88
|
],
|
49
|
-
:
|
89
|
+
next_component_selection: :random,
|
50
90
|
}
|
51
91
|
|
52
92
|
|
93
|
+
constraints \
|
94
|
+
'Web Server' => [
|
95
|
+
{ data_center: 1, min: 0, max: 300 },
|
96
|
+
{ data_center: 2, min: 0, max: 300 },
|
97
|
+
],
|
98
|
+
'App Server' => [
|
99
|
+
{ data_center: 1, min: 0, max: 300 },
|
100
|
+
{ data_center: 2, min: 0, max: 300 },
|
101
|
+
]
|
102
|
+
|
103
|
+
|
53
104
|
# input request source (the generator.R script)
|
54
105
|
request_generation \
|
55
|
-
:
|
106
|
+
command: '<pwd>/generator.R'
|
56
107
|
|
57
108
|
|
58
109
|
# evaluation model
|
59
110
|
evaluation \
|
60
|
-
:
|
61
|
-
{ :
|
62
|
-
{ :
|
63
|
-
{ :
|
64
|
-
{ :
|
111
|
+
vm_hourly_cost: [
|
112
|
+
{ data_center: 1, vm_type: :medium, cost: 0.160 },
|
113
|
+
{ data_center: 1, vm_type: :large, cost: 0.320 },
|
114
|
+
{ data_center: 2, vm_type: :medium, cost: 0.184 },
|
115
|
+
{ data_center: 2, vm_type: :large, cost: 0.368 }
|
65
116
|
],
|
66
117
|
# 500$ penalties if MTTR takes more than 50 msecs
|
67
|
-
:
|
118
|
+
penalties: lambda {|kpis,dc_kpis| 500.0 if kpis[:mttr] > 0.050 }
|
68
119
|
|
69
120
|
|
70
121
|
# vim: filetype=ruby
|
data/lib/sisfc.rb
CHANGED
@@ -1,5 +1,5 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
|
3
|
+
require_relative './sisfc/configuration'
|
4
|
+
require_relative './sisfc/simulation'
|
5
|
+
require_relative './sisfc/evaluation'
|
data/lib/sisfc/configuration.rb
CHANGED
@@ -1,20 +1,69 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './support/dsl_helper'
|
4
|
+
|
5
|
+
require_relative './logger'
|
6
|
+
|
7
|
+
require 'as-duration'
|
8
|
+
require 'ice_nine'
|
9
|
+
|
10
|
+
module ERV
|
11
|
+
module GaussianMixtureHelper
|
12
|
+
def self.RawParametersToMixtureArgs(*args)
|
13
|
+
raise ArgumentError, "Arguments must be a multiple of 3!" if (args.count % 3) != 0
|
14
|
+
args.each_slice(3).map do |(a,b,c)|
|
15
|
+
{ distribution: :gaussian, weight: a * c, args: { mean: b, sd: c } }
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
if defined? JRUBY_VERSION
|
22
|
+
# JRuby 9.2 still has a buggy support for refinements, so we need to revert
|
23
|
+
# to the brutal monkeypatching of the Integer class
|
24
|
+
class Integer
|
25
|
+
# def minute; self * 60; end
|
26
|
+
# def minutes; self * 60; end
|
27
|
+
# def second; self; end
|
28
|
+
# def seconds; self; end
|
29
|
+
def msec; self * 1E-3; end
|
30
|
+
def msecs; self * 1E-3; end
|
31
|
+
end
|
32
|
+
else
|
33
|
+
module TimeExtensions
|
34
|
+
refine Integer do
|
35
|
+
# def minute; self * 60; end
|
36
|
+
# def minutes; self * 60; end
|
37
|
+
# def second; self; end
|
38
|
+
# def seconds; self; end
|
39
|
+
def msec; self * 1E-3; end
|
40
|
+
def msecs; self * 1E-3; end
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
2
44
|
|
3
45
|
module SISFC
|
4
46
|
|
5
47
|
module Configurable
|
6
|
-
dsl_accessor :
|
48
|
+
dsl_accessor :constraints,
|
49
|
+
:customers,
|
50
|
+
:custom_stats,
|
51
|
+
:data_centers,
|
7
52
|
:duration,
|
8
|
-
:
|
53
|
+
:evaluation,
|
54
|
+
:kpi_customization,
|
55
|
+
:latency_models,
|
9
56
|
:request_generation,
|
10
|
-
:data_centers,
|
11
57
|
:service_component_types,
|
12
|
-
:
|
58
|
+
:start_time,
|
59
|
+
:warmup_duration,
|
13
60
|
:workflow_types
|
14
61
|
end
|
15
62
|
|
16
63
|
class Configuration
|
17
64
|
include Configurable
|
65
|
+
include Logging
|
66
|
+
using TimeExtensions unless defined? JRUBY_VERSION
|
18
67
|
|
19
68
|
attr_accessor :filename
|
20
69
|
|
@@ -32,11 +81,29 @@ module SISFC
|
|
32
81
|
@duration = @duration.to_f
|
33
82
|
@warmup_duration = @warmup_duration.to_f
|
34
83
|
|
84
|
+
# initialize kpi_customization to empty hash if needed
|
85
|
+
@kpi_customization ||= {}
|
86
|
+
|
35
87
|
# TODO: might want to restrict this substitution only to the :filename
|
36
88
|
# and :command keys
|
37
89
|
@request_generation.each do |k,v|
|
38
|
-
v.gsub
|
90
|
+
v = v.gsub('<pwd>', File.expand_path(File.dirname(@filename)))
|
39
91
|
end
|
92
|
+
|
93
|
+
# freeze everything!
|
94
|
+
IceNine.deep_freeze(@constraints)
|
95
|
+
IceNine.deep_freeze(@customers)
|
96
|
+
IceNine.deep_freeze(@custom_stats)
|
97
|
+
IceNine.deep_freeze(@data_centers)
|
98
|
+
IceNine.deep_freeze(@duration)
|
99
|
+
IceNine.deep_freeze(@evaluation)
|
100
|
+
IceNine.deep_freeze(@kpi_customization)
|
101
|
+
IceNine.deep_freeze(@latency_models)
|
102
|
+
IceNine.deep_freeze(@request_generation)
|
103
|
+
IceNine.deep_freeze(@service_component_types)
|
104
|
+
IceNine.deep_freeze(@start_time)
|
105
|
+
IceNine.deep_freeze(@warmup_duration)
|
106
|
+
IceNine.deep_freeze(@workflow_types)
|
40
107
|
end
|
41
108
|
|
42
109
|
def self.load_from_file(filename)
|