scaltainer 0.1.5 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: ad9d8453d71a28ce6344fa3b5e8a3185cf21e613
4
- data.tar.gz: 3033587d946ce00606b9420ff2645d36c617bd43
3
+ metadata.gz: 642f49828031f354f271d0ad4e28268ab40e1f1a
4
+ data.tar.gz: 2be74e2694d99bc5edde992e8e277867db0de44b
5
5
  SHA512:
6
- metadata.gz: 503333de51f96a8d4df01410378cfd26b8b9db46d6023ed50949199396032ad4ef3c02e0da3baeb12e3f1bdb92721e69ef0ad3264d6b43cc4aa18ae3d9bff440
7
- data.tar.gz: be2a993a5589276e95fc46145953b65c3a20ee53d3512ec7a85c8d00f06566dff436f4bf1e2723eda24ee52420a4f2da0de7236a0d45e85f824590ea3aadcac6
6
+ metadata.gz: 34a7ddda22e20d4e909a29c7b2183344712a7fc72dd8d970055c578834e6d1707496bfeebfd09cc4224d62a806615756cb964bddb07653b60798ded0ffe0cbe8
7
+ data.tar.gz: 07b3b919496e8cc1fbdbc25199e53509e5bb44b3be47a0257430ce6be22102dc08eb43e7d6a2e9bc2eb6ef2ad8d0d33b84ebec9a5c4be8ee83b5094df7e25bc5
data/.dockerignore ADDED
@@ -0,0 +1,3 @@
1
+ pkg
2
+ spec
3
+ .env
data/.gitignore CHANGED
@@ -6,4 +6,8 @@
6
6
  /pkg/
7
7
  /spec/reports/
8
8
  /tmp/
9
- Gemfile.lock
9
+ Gemfile.lock
10
+ .env
11
+ scaltainer.yml
12
+ scaltainer.yml.state
13
+ kube-manifest.yaml
data/Dockerfile CHANGED
@@ -1,8 +1,10 @@
1
1
  FROM ruby:2.3
2
2
 
3
- MAINTAINER Hossam Hammady <github@hammady.net>
3
+ label maintainer="Hossam Hammady <github@hammady.net>"
4
4
 
5
- RUN gem install scaltainer
5
+ WORKDIR /home
6
+ COPY / /home/
7
+ RUN bundle install && bundle exec rake install
6
8
 
7
9
  ENTRYPOINT ["scaltainer"]
8
10
 
data/README.md CHANGED
@@ -4,32 +4,38 @@
4
4
 
5
5
  # Scaltainer
6
6
 
7
- A Ruby gem to monitor docker swarm mode services and auto-scale them based on user configuration.
7
+ A Ruby gem to monitor Docker Swarm mode services and Kubernetes resources
8
+ and auto-scale them based on user configuration.
8
9
  It can be used to monitor web services and worker services. The web services type has metrics like response time using [New Relic](https://newrelic.com/). The worker services type metrics are basically the queue size for each.
9
10
  This gem is inspired by [HireFire](https://manager.hirefire.io/) and was indeed motivated by the migration
10
- from [Heroku](https://www.heroku.com/) to Docker Swarm mode.
11
+ from [Heroku](https://www.heroku.com/) to Docker.
11
12
 
12
13
  ## Installation
13
14
 
14
15
  Add this line to your application's Gemfile:
15
16
 
16
- ```ruby
17
- gem 'scaltainer'
18
- ```
17
+ Install using rubygems:
19
18
 
20
- And then execute:
19
+ $ gem install scaltainer
21
20
 
22
- $ bundle
21
+ ## Usage
23
22
 
24
- Or install it yourself as:
23
+ For Docker swarm:
25
24
 
26
- $ gem install scaltainer
25
+ scaltainer -o swarm
27
26
 
28
- ## Usage
27
+ Or simply:
29
28
 
30
29
  scaltainer
31
30
 
32
- This will do a one-time check on the running service replicas and sends scaling out/in commands to the swarm cluster as appropriate.
31
+ For Kubernetes:
32
+
33
+ scaltainer -o kubernetes
34
+
35
+
36
+ This will do a one-time check on the running docker service replicas
37
+ or Kubernetes replication controllers, replica sets, or deployments.
38
+ Then it sends scaling out/in commands to the cluster as appropriate.
33
39
  Configuration is read from `scaltainer.yml` by default. If you want to read from another file add `-f yourconfig.yml`:
34
40
 
35
41
  scaltainer -f yourconfig.yml
@@ -47,15 +53,44 @@ specify the wait time between repetitions using the `-w` parameter in seconds:
47
53
 
48
54
  scaltainer -w 60
49
55
 
50
- This will repeatedly call scaltainer every 60 seconds, sleeping in between.
56
+ This will repeatedly call scaltainer every 60 seconds, sleeping in-between.
51
57
 
52
58
  ## Configuration
53
59
 
54
60
  ### Environment variables
55
61
 
62
+ #### Docker swarm options
63
+
56
64
  - `DOCKER_URL`: Should point to the docker engine URL.
57
65
  If not set, it defaults to local unix socket.
58
66
 
67
+ #### Kubernetes options
68
+
69
+ - `KUBECONFIG`: set to Kubernetes config
70
+ (default: `$HOME/.kube/config`) if you want to connect
71
+ to the current configured cluster.
72
+
73
+ - `KUBERNETES_API_SERVER`: overrides option in `KUBECONFIG`
74
+ and defaults to `https://kubernetes.default:443`.
75
+
76
+ - `KUBERNETES_SKIP_SSL_VERIFY`: `KUBECONFIG` option overrides
77
+ this, set to any value to skip SSL verification.
78
+
79
+ - `KUBERNETES_API_ENDPOINT`: defaults to `/api`.
80
+
81
+ - `KUBERNETES_API_VERSION`: overrides option in `KUBECONFIG`
82
+ and defaults to `v1`.
83
+
84
+ - `KUBERNETES_CONTROLLER_KIND`: controller kind to scale,
85
+ allowed values: `deployment` (default),
86
+ `replication_controller`, or `replica_set`.
87
+
88
+ Make sure the `KUBERNETES_CONTROLLER_KIND` you specify is
89
+ part of the api specified using `KUBERNETES_API_ENDPOINT`
90
+ and `KUBERNETES_API_VERSION`.
91
+
92
+ #### General options
93
+
59
94
  - `HIREFIRE_TOKEN`: If your application is configured the
60
95
  [hirefire](https://help.hirefire.io/guides/hirefire/job-queue-any-programming-language) way, you need to
61
96
  set `HIREFIRE_TOKEN` environment variable before invoking
@@ -86,8 +121,8 @@ The configuration file (determined by `-f FILE` command line parameter) should b
86
121
 
87
122
  # to get worker metrics
88
123
  endpoint: https://your-app.com/hirefire/$HIREFIRE_TOKEN/info
89
- # optional docker swarm stack name
90
- stack_name: mystack
124
+ # optional docker swarm stack name or kubernetes namespace
125
+ namespace: mynamespace
91
126
  # list of web services to monitor
92
127
  web_services:
93
128
  # each service name should match docker service name
@@ -126,25 +161,15 @@ The configuration file (determined by `-f FILE` command line parameter) should b
126
161
 
127
162
  More details about configuration parameters can be found in [HireFire docs](https://help.hirefire.io/guides).
128
163
 
129
- ## Docker installation and usage
164
+ ## Docker Swarm usage
130
165
 
131
- Scaltainer is availabe on Docker Hub, so you can `docker run` it:
132
-
133
- docker run -it --rm rayyanqcri/scaltainer
134
-
135
- Which will print the usage. To add arguments, just append them:
136
-
137
- docker run -it --rm rayyanqcri/scaltainer -f scaltainer.yml
138
-
139
- Scaltainer should typically be run as a minutely cron service.
140
- If you are using [rayyanqcri/swarm-scheduler](https://github.com/rayyanqcri/swarm-scheduler),
141
- a service definition for scaltainer is typically something like this:
166
+ A service definition for scaltainer is typically something like this:
142
167
 
143
168
  version: '3.3'
144
169
  services:
145
170
  scaltainer:
146
171
  image: rayyanqcri/scaltainer:latest
147
- command: -f /scaltainer.yml --state-file /tmp/scaltainer-state.yml
172
+ command: -f /scaltainer.yml --state-file /tmp/scaltainer-state.yml -w 60
148
173
  volumes:
149
174
  - /var/run/docker.sock:/var/run/docker.sock
150
175
  environment:
@@ -157,9 +182,7 @@ a service definition for scaltainer is typically something like this:
157
182
  secrets:
158
183
  - scaltainer
159
184
  deploy:
160
- replicas: 0
161
- restart_policy:
162
- condition: none
185
+ replicas: 1
163
186
  placement:
164
187
  constraints:
165
188
  - node.role == manager
@@ -177,6 +200,75 @@ Where `scaltainer.env` is a file containing HireFire and NewRelic secrets:
177
200
 
178
201
  And `scaltainer.yml` is the scaltainer configuration file.
179
202
 
203
+ ## Kubernetes usage
204
+
205
+ ### Create a ConfigMap
206
+
207
+ kubectl create configmap scaltainer --from-file=scaltainer.yaml=/path/to/your/scaltainer.yml
208
+
209
+ Where `/path/to/your/scaltainer.yml` is the scaltainer configuration file.
210
+
211
+ ### Create a Secret
212
+
213
+ kubectl create secret generic scaltainer --from-env-file=/path/to/scaltainer.env
214
+
215
+ Where `/path/to/scaltainer.env` is a file containing HireFire and NewRelic secrets:
216
+
217
+ HIREFIRE_TOKEN=
218
+ NEW_RELIC_API_KEY=
219
+
220
+ ### Create a Deployment:
221
+
222
+ kubectl apply -f scaltainer-kube.yaml
223
+
224
+ Where scaltainer-kube.yaml has the following content:
225
+
226
+ apiVersion: extensions/v1beta1
227
+ kind: Deployment
228
+ metadata:
229
+ labels:
230
+ app: scaltainer
231
+ name: scaltainer
232
+ spec:
233
+ replicas: 1
234
+ template:
235
+ metadata:
236
+ labels:
237
+ app: scaltainer
238
+ spec:
239
+ containers:
240
+ - image: rayyanqcri/scaltainer:latest
241
+ name: scaltainer
242
+ args:
243
+ - -o
244
+ - kubernetes
245
+ - -f
246
+ - /etc/config/scaltainer.yaml
247
+ - --state-file
248
+ - /tmp/scaltainer-state.yaml
249
+ - -w
250
+ - "60"
251
+ env:
252
+ - name: KUBERNETES_SKIP_SSL_VERIFY
253
+ value: "yes"
254
+ - name: KUBERNETES_API_ENDPOINT
255
+ value: /apis/extensions
256
+ - name: KUBERNETES_API_VERSION
257
+ value: v1beta1
258
+ - name: KUBERNETES_CONTROLLER_KIND
259
+ value: deployment
260
+ envFrom:
261
+ - secretRef:
262
+ name: scaltainer
263
+ volumeMounts:
264
+ - name: scaltainer-config
265
+ mountPath: "/etc/config"
266
+ volumes:
267
+ - name: scaltainer-config
268
+ configMap:
269
+ name: scaltainer
270
+
271
+
180
272
  ## Development
181
273
 
182
274
  After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
@@ -187,9 +279,9 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
187
279
 
188
280
  Bug reports and pull requests are welcome on GitHub at https://github.com/hammady/scaltainer.
189
281
 
190
- ## TODOs
282
+ ## Testing
191
283
 
192
- - Rspec
284
+ rake
193
285
 
194
286
  ## License
195
287
 
data/exe/scaltainer CHANGED
@@ -3,8 +3,8 @@
3
3
  require 'scaltainer'
4
4
 
5
5
  begin
6
- configfile, statefile, logger, wait = Scaltainer::Command.parse ARGV
7
- Scaltainer::Runner.new configfile, statefile, logger, wait
6
+ configfile, statefile, logger, wait, orchestrator = Scaltainer::Command.parse ARGV
7
+ Scaltainer::Runner.new configfile, statefile, logger, wait, orchestrator
8
8
  rescue => e
9
9
  $stderr.puts e.message
10
10
  $stderr.puts e.backtrace
data/lib/scaltainer.rb CHANGED
@@ -3,6 +3,6 @@ require "scaltainer/exceptions"
3
3
  require "scaltainer/service_types"
4
4
  require "scaltainer/runner"
5
5
  require "scaltainer/command"
6
- require "scaltainer/docker/service"
6
+ require "scaltainer/orchestrators"
7
7
  require "scaltainer/newrelic/metrics"
8
8
  require "json"
@@ -4,7 +4,7 @@ require "optparse"
4
4
  module Scaltainer
5
5
  class Command
6
6
  def self.parse(args)
7
- configfile, statefile, wait = 'scaltainer.yml', nil, 0
7
+ configfile, statefile, wait, orchestrator = 'scaltainer.yml', nil, 0, :swarm
8
8
  OptionParser.new do |opts|
9
9
  opts.banner = "Usage: scaltainer [options]"
10
10
  opts.on("-f", "--conf-file FILE", "Specify configuration file (default: scaltainer.yml)") do |file|
@@ -16,10 +16,27 @@ module Scaltainer
16
16
  opts.on("-w", "--wait SECONDS", "Specify wait time between repeated calls, 0 for no repetition (default: 0)") do |w|
17
17
  wait = w.to_i
18
18
  end
19
+ opts.on("-o", "--orchestrator swarm:kubernetes", [:swarm, :kubernetes], "Specify orchestrator type (default: swarm)") do |o|
20
+ orchestrator = o
21
+ end
22
+ opts.on("-v", "--version", "Show version and exit") do
23
+ puts Scaltainer::VERSION
24
+ exit 0
25
+ end
19
26
  opts.on_tail("-h", "--help", "Show this message") do
20
27
  puts opts
21
28
  puts "\nEnvironment variables: \n"
29
+ puts "Docker Swarm options:"
22
30
  puts "- DOCKER_URL: defaults to local socket"
31
+ puts "Kubernetes options:"
32
+ puts "- KUBECONFIG: set to Kubernetes config (default: $HOME/.kube/config) if you want to connect to the current configured cluster"
33
+ puts "- KUBERNETES_API_SERVER: overrides option in KUBECONFIG and defaults to https://kubernetes.default:443"
34
+ puts "- KUBERNETES_SKIP_SSL_VERIFY: KUBECONFIG option overrides this, set to any value to skip SSL verification"
35
+ puts "- KUBERNETES_API_ENDPOINT: defaults to /api"
36
+ puts "- KUBERNETES_API_VERSION: overrides option in KUBECONFIG and defaults to v1"
37
+ puts "- KUBERNETES_CONTROLLER_KIND: controller kind to scale, allowed values: deployment (default), replication_controller, or replica_set"
38
+ puts " Make sure the KUBERNETES_CONTROLLER_KIND you specify is part of the api specified using KUBERNETES_API_ENDPOINT and KUBERNETES_API_VERSION"
39
+ puts "General options:"
23
40
  puts "- HIREFIRE_TOKEN"
24
41
  puts "- NEW_RELIC_API_KEY"
25
42
  puts "- RESPONSE_TIME_WINDOW: defaults to 5"
@@ -38,7 +55,7 @@ module Scaltainer
38
55
  logger = Logger.new(STDOUT)
39
56
  logger.level = %w(debug info warn error fatal unknown).find_index((ENV['LOG_LEVEL'] || '').downcase) || 1
40
57
 
41
- return configfile, statefile, logger, wait
58
+ return configfile, statefile, logger, wait, orchestrator
42
59
  end
43
60
 
44
61
  private
@@ -0,0 +1,3 @@
1
+ require "scaltainer/orchestrators/base"
2
+ require "scaltainer/orchestrators/swarm"
3
+ require "scaltainer/orchestrators/kubernetes"
@@ -0,0 +1,17 @@
1
+ module Scaltainer
2
+ class ReplicaSetBase
3
+ attr_accessor :id, :name, :type, :namespace
4
+
5
+ def initialize(name, type, namespace)
6
+ @name, @type, @namespace = name, type, namespace
7
+ end
8
+
9
+ def get_replicas
10
+ raise 'Abstract method, please override'
11
+ end
12
+
13
+ def set_replicas(replicas)
14
+ raise 'Abstract method, please override'
15
+ end
16
+ end
17
+ end
@@ -0,0 +1,84 @@
1
+ require 'kubeclient'
2
+
3
+ module Scaltainer
4
+ class KubeResource < ReplicaSetBase
5
+ def initialize(name, namespace)
6
+ @@client ||= self.class.get_client
7
+ type = ENV['KUBERNETES_CONTROLLER_KIND'] || 'deployment'
8
+ # if namespace not specified, use the one found in configuration
9
+ namespace ||= @@namespace || 'default'
10
+ super(name, type, namespace)
11
+ @resource = @@client.send("get_#{@type}", normalize_name(@name), @namespace)
12
+ @id = @resource.metadata.uid
13
+ end
14
+
15
+ def get_replicas
16
+ @resource.spec.replicas
17
+ end
18
+
19
+ def set_replicas(replicas)
20
+ @@client.send("patch_#{@type}", normalize_name(@name), {spec: {replicas: replicas}}, @namespace)
21
+ end
22
+
23
+ private
24
+
25
+ def self.get_client
26
+ if ENV['KUBECONFIG']
27
+ get_client_from_kubeconfig ENV['KUBECONFIG']
28
+ else
29
+ get_client_from_serviceaccount '/var/run/secrets/kubernetes.io/serviceaccount'
30
+ end
31
+ end
32
+
33
+ def self.get_client_from_kubeconfig(kubeconfig)
34
+ config = Kubeclient::Config.read(kubeconfig)
35
+ url = get_api_url(config.context.api_endpoint)
36
+ version = get_api_version(config.context.api_version)
37
+ # @@namespace = config.context.namespace # wait till PR#308 merged into kubeclient
38
+ Kubeclient::Client.new(
39
+ url, version,
40
+ ssl_options: config.context.ssl_options,
41
+ auth_options: config.context.auth_options
42
+ )
43
+ end
44
+
45
+ def self.get_client_from_serviceaccount(serviceaccount)
46
+ ssl_verify = if ENV['KUBERNETES_SKIP_SSL_VERIFY']
47
+ OpenSSL::SSL::VERIFY_NONE
48
+ else
49
+ OpenSSL::SSL::VERIFY_PEER
50
+ end
51
+ ssl_options = {
52
+ client_cert: OpenSSL::X509::Certificate.new(read_secret(serviceaccount, 'ca.crt')),
53
+ verify_ssl: ssl_verify
54
+ }
55
+ auth_options = {bearer_token: read_secret(serviceaccount, 'token')}
56
+ @@namespace = read_secret(serviceaccount, 'namespace')
57
+ url = get_api_url('https://kubernetes.default:443')
58
+ version = get_api_version('v1')
59
+ Kubeclient::Client.new(
60
+ url, version,
61
+ ssl_options: ssl_options,
62
+ auth_options: auth_options
63
+ )
64
+ end
65
+
66
+ def self.get_api_url(default_server)
67
+ server = ENV['KUBERNETES_API_SERVER'] || default_server
68
+ endpoint = ENV['KUBERNETES_API_ENDPOINT'] || '/api'
69
+ "#{server}#{endpoint}"
70
+ end
71
+
72
+ def self.get_api_version(default_version)
73
+ ENV['KUBERNETES_API_VERSION'] || default_version
74
+ end
75
+
76
+ def self.read_secret(serviceaccount, secret)
77
+ File.read("#{serviceaccount}/#{secret}")
78
+ end
79
+
80
+ def normalize_name(name)
81
+ name.gsub(/_/, '-')
82
+ end
83
+ end
84
+ end
@@ -1,3 +1,26 @@
1
+ module Scaltainer
2
+ class DockerService < ReplicaSetBase
3
+ def initialize(service_name, namespace)
4
+ # set logger?
5
+ full_name = namespace ? "#{namespace}_#{service_name}" : service_name
6
+ @service = Docker::Service.all(filters: {name: [full_name]}.to_json)[0]
7
+ raise "Docker Service not found: #{full_name}" unless @service
8
+ @id = @service.id
9
+ super(service_name, 'service', namespace)
10
+ end
11
+
12
+ def get_replicas
13
+ replicated = @service.info["Spec"]["Mode"]["Replicated"]
14
+ raise ConfigurationError.new "Cannot replicate a global service: #{@name}" unless replicated
15
+ replicated["Replicas"]
16
+ end
17
+
18
+ def set_replicas(replicas)
19
+ @service.scale replicas
20
+ end
21
+ end
22
+ end
23
+
1
24
  # source: https://github.com/Stazer/docker-api/blob/feature/swarm_support/lib/docker/service.rb
2
25
 
3
26
  # This class represents a Docker Service. It's important to note that nothing
@@ -2,7 +2,8 @@ require "yaml"
2
2
 
3
3
  module Scaltainer
4
4
  class Runner
5
- def initialize(configfile, statefile, logger, wait)
5
+ def initialize(configfile, statefile, logger, wait, orchestrator)
6
+ @orchestrator = orchestrator
6
7
  @logger = logger
7
8
  @default_service_config = {
8
9
  "min" => 0,
@@ -13,7 +14,7 @@ module Scaltainer
13
14
  }
14
15
  @logger.debug "Scaltainer initialized with configuration file: #{configfile}, and state file: #{statefile}"
15
16
  config = YAML.load_file configfile
16
- Docker.logger = @logger
17
+ Docker.logger = @logger if orchestrator == :swarm
17
18
  state = get_state(statefile) || {}
18
19
  endpoint = config["endpoint"]
19
20
  service_type_web = ServiceTypeWeb.new(endpoint)
@@ -29,9 +30,9 @@ module Scaltainer
29
30
  private
30
31
 
31
32
  def run(config, state, service_type_web, service_type_worker)
32
- service_prefix = config["stack_name"]
33
- iterate_services config["web_services"], service_prefix, service_type_web, state
34
- iterate_services config["worker_services"], service_prefix, service_type_worker, state
33
+ namespace = config["namespace"] || config["stack_name"]
34
+ iterate_services config["web_services"], namespace, service_type_web, state
35
+ iterate_services config["worker_services"], namespace, service_type_worker, state
35
36
  end
36
37
 
37
38
  def get_state(statefile)
@@ -42,35 +43,18 @@ module Scaltainer
42
43
  File.write(statefile, state.to_yaml)
43
44
  end
44
45
 
45
- def get_service(service_name)
46
- begin
47
- service = Docker::Service.all(filters: {name: [service_name]}.to_json)[0]
48
- rescue => e
49
- raise NetworkError.new "Could not get service with name #{service_name} from docker engine at #{Docker.url}.\n#{e.message}"
50
- end
51
- raise ConfigurationError.new "Unknown service to docker: #{service_name}" unless service
52
- service
53
- end
54
-
55
- def get_service_replicas(service)
56
- # ask docker about replicas for service
57
- replicated = service.info["Spec"]["Mode"]["Replicated"]
58
- raise ConfigurationError.new "Cannot replicate a global service: #{service.info['Spec']['Name']}" unless replicated
59
- replicated["Replicas"]
60
- end
61
-
62
- def iterate_services(services, service_prefix, type, state)
46
+ def iterate_services(services, namespace, type, state)
63
47
  begin
64
48
  metrics = type.get_metrics services
65
- @logger.debug "Retrieved metrics for #{type} services: #{metrics}"
49
+ @logger.debug "Retrieved metrics for #{type} resources: #{metrics}"
66
50
  services.each do |service_name, service_config|
67
51
  begin
68
52
  state[service_name] ||= {}
69
53
  service_state = state[service_name]
70
- @logger.debug "Service #{service_name} currently has state: #{service_state}"
54
+ @logger.debug "Resource #{service_name} in namespace #{namespace} currently has state: #{service_state}"
71
55
  service_config = @default_service_config.merge service_config
72
- @logger.debug "Service #{service_name} configuration: #{service_config}"
73
- process_service service_name, service_config, service_state, service_prefix, type, metrics
56
+ @logger.debug "Resource #{service_name} in namespace #{namespace} configuration: #{service_config}"
57
+ process_service service_name, service_config, service_state, namespace, type, metrics
74
58
  rescue RuntimeError => e
75
59
  # skipping service
76
60
  log_exception e
@@ -86,34 +70,46 @@ module Scaltainer
86
70
  @logger.log (e.class == Scaltainer::Warning ? Logger::WARN : Logger::ERROR), e.message
87
71
  end
88
72
 
89
- def process_service(service_name, config, state, prefix, type, metrics)
90
- full_service_name = prefix ? "#{prefix}_#{service_name}" : service_name
91
- service = get_service full_service_name
92
- @logger.debug "Found service at docker with name '#{service_name}' and id '#{service.id}'"
93
- current_replicas = get_service_replicas service
94
- @logger.debug "Service #{service_name} is currently configured for #{current_replicas} replica(s)"
95
- metric = metrics[service_name]
96
- raise Scaltainer::Warning.new("Configured service '#{service_name}' not found in metrics endpoint") unless metric
73
+ def process_service(service_name, config, state, namespace, type, metrics)
74
+ service = get_service service_name, namespace
75
+ @logger.debug "Found #{service.type} at orchestrator with name '#{service.name}' and id '#{service.id}'"
76
+ current_replicas = service.get_replicas
77
+ @logger.debug "#{service.type.capitalize} #{service.name} is currently configured for #{current_replicas} replica(s)"
78
+ metric = metrics[service.name]
79
+ raise Scaltainer::Warning.new("Configured #{service.type} '#{service.name}' not found in metrics endpoint") unless metric
97
80
  desired_replicas = type.determine_desired_replicas metric, config, current_replicas
98
- @logger.debug "Desired number of replicas for service #{service_name} is #{desired_replicas}"
81
+ @logger.debug "Desired number of replicas for #{service.type} #{service.name} is #{desired_replicas}"
99
82
  adjusted_replicas = type.adjust_desired_replicas(desired_replicas, config)
100
- @logger.debug "Desired number of replicas for service #{service_name} is adjusted to #{adjusted_replicas}"
83
+ @logger.debug "Desired number of replicas for #{service.type} #{service.name} is adjusted to #{adjusted_replicas}"
101
84
  replica_diff = adjusted_replicas - current_replicas
102
85
  type.yield_to_scale(replica_diff, config, state, metric,
103
- service_name, @logger) do
86
+ service.name, @logger) do
104
87
  scale_out service, current_replicas, adjusted_replicas
105
88
  end
106
89
  end
107
90
 
91
+ def get_service(service_name, namespace)
92
+ begin
93
+ service = if @orchestrator == :swarm
94
+ DockerService.new service_name, namespace
95
+ elsif @orchestrator == :kubernetes
96
+ KubeResource.new service_name, namespace
97
+ end
98
+ rescue => e
99
+ raise NetworkError.new "Could not find resource with name #{service_name} in namespace #{namespace}: #{e.message}"
100
+ end
101
+ raise ConfigurationError.new "Unknown resource: #{service_name} in namespace #{namespace}" unless service
102
+ service
103
+ end
104
+
108
105
  def scale_out(service, current_replicas, desired_replicas)
109
106
  return if current_replicas == desired_replicas
110
- # send scale command to docker
111
- service_name = service.info['Spec']['Name']
112
- @logger.info "Scaling #{service_name} from #{current_replicas} to #{desired_replicas}"
107
+ # send scale command to orchestrator
108
+ @logger.info "Scaling #{service.type} #{service.name} from #{current_replicas} to #{desired_replicas}"
113
109
  begin
114
- service.scale desired_replicas
110
+ service.set_replicas desired_replicas
115
111
  rescue => e
116
- raise NetworkError.new "Could not scale service #{service_name} due to docker engine error at #{Docker.url}.\n#{e.message}"
112
+ raise NetworkError.new "Could not scale #{service.type} #{service.name} due to error: #{e.message}"
117
113
  end
118
114
  end
119
115
 
@@ -6,12 +6,12 @@ module Scaltainer
6
6
 
7
7
  def get_metrics(services)
8
8
  services_count = services.keys.length rescue 0
9
- raise Scaltainer::Warning.new "No services found for #{self.class.name}" if services_count == 0
9
+ raise Scaltainer::Warning.new "No resources found for #{self.class.name}" if services_count == 0
10
10
  end
11
11
 
12
12
  def determine_desired_replicas(metric, service_config, current_replicas)
13
- raise ConfigurationError.new 'No metric found for requested service' unless metric
14
- raise ConfigurationError.new 'No configuration found for requested service' unless service_config
13
+ raise ConfigurationError.new 'No metric found for requested resource' unless metric
14
+ raise ConfigurationError.new 'No configuration found for requested resource' unless service_config
15
15
  end
16
16
 
17
17
  def adjust_desired_replicas(desired_replicas, config)
@@ -32,7 +32,7 @@ module Scaltainer
32
32
  yield
33
33
  state["upscale_sensitivity"] = 0
34
34
  else
35
- logger.debug "Scaling up of service #{service_name} blocked by upscale_sensitivity at level " +
35
+ logger.debug "Scaling up of resource #{service_name} blocked by upscale_sensitivity at level " +
36
36
  "#{state["upscale_sensitivity"]} while level #{config["upscale_sensitivity"]} is required"
37
37
  end
38
38
  elsif replica_diff < 0 # TODO force down when above max?
@@ -45,17 +45,17 @@ module Scaltainer
45
45
  yield
46
46
  state["downscale_sensitivity"] = 0
47
47
  else
48
- logger.debug "Scaling down of service #{service_name} blocked by downscale_sensitivity at level " +
48
+ logger.debug "Scaling down of resource #{service_name} blocked by downscale_sensitivity at level " +
49
49
  "#{state["downscale_sensitivity"]} while level #{config["downscale_sensitivity"]} is required"
50
50
  end
51
51
  else
52
- logger.debug "Scaling down of service #{service_name} to #{metric} replicas blocked by a non-decrementable config"
52
+ logger.debug "Scaling down of resource #{service_name} to #{metric} replicas blocked by a non-decrementable config"
53
53
  end
54
54
  else
55
55
  # no breach, change state
56
56
  state["upscale_sensitivity"] = 0
57
57
  state["downscale_sensitivity"] = 0
58
- logger.info "No need to scale service #{service_name}"
58
+ logger.info "No need to scale resource #{service_name}"
59
59
  end
60
60
  end
61
61
 
@@ -14,7 +14,7 @@ module Scaltainer
14
14
 
15
15
  services.reduce({}) do |hash, (service_name, service_config)|
16
16
  app_id = service_config["newrelic_app_id"]
17
- raise ConfigurationError.new "Service #{service_name} does not have a corresponding newrelic_app_id" unless app_id
17
+ raise ConfigurationError.new "Resource #{service_name} does not have a corresponding newrelic_app_id" unless app_id
18
18
 
19
19
  begin
20
20
  metric = nr.get_avg_response_time app_id, from, to
@@ -28,8 +28,8 @@ module Scaltainer
28
28
 
29
29
  def determine_desired_replicas(metric, service_config, current_replicas)
30
30
  super
31
- raise ConfigurationError.new "Missing max_response_time in web service configuration" unless service_config["max_response_time"]
32
- raise ConfigurationError.new "Missing min_response_time in web service configuration" unless service_config["min_response_time"]
31
+ raise ConfigurationError.new "Missing max_response_time in web resource configuration" unless service_config["max_response_time"]
32
+ raise ConfigurationError.new "Missing min_response_time in web resource configuration" unless service_config["min_response_time"]
33
33
  unless service_config["min_response_time"] <= service_config["max_response_time"]
34
34
  raise ConfigurationError.new "min_response_time and max_response_time are not in order"
35
35
  end
@@ -21,7 +21,7 @@ module Scaltainer
21
21
 
22
22
  def determine_desired_replicas(metric, service_config, current_replicas)
23
23
  super
24
- raise ConfigurationError.new "Missing ratio in worker service configuration" unless service_config["ratio"]
24
+ raise ConfigurationError.new "Missing ratio in worker resource configuration" unless service_config["ratio"]
25
25
  if !metric.is_a?(Integer) || metric < 0
26
26
  raise ConfigurationError.new "#{metric} is an invalid metric value, must be a non-negative number"
27
27
  end
@@ -1,3 +1,3 @@
1
1
  module Scaltainer
2
- VERSION = "0.1.5"
2
+ VERSION = "0.2.0"
3
3
  end
data/scaltainer.gemspec CHANGED
@@ -30,5 +30,6 @@ Gem::Specification.new do |spec|
30
30
 
31
31
  spec.add_runtime_dependency 'excon', '>= 0.47.0'
32
32
  spec.add_runtime_dependency "docker-api"
33
+ spec.add_runtime_dependency "kubeclient"
33
34
  spec.add_runtime_dependency "dotenv"
34
35
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: scaltainer
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.5
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Hossam Hammady
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2017-11-30 00:00:00.000000000 Z
11
+ date: 2018-03-01 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -108,6 +108,20 @@ dependencies:
108
108
  - - ">="
109
109
  - !ruby/object:Gem::Version
110
110
  version: '0'
111
+ - !ruby/object:Gem::Dependency
112
+ name: kubeclient
113
+ requirement: !ruby/object:Gem::Requirement
114
+ requirements:
115
+ - - ">="
116
+ - !ruby/object:Gem::Version
117
+ version: '0'
118
+ type: :runtime
119
+ prerelease: false
120
+ version_requirements: !ruby/object:Gem::Requirement
121
+ requirements:
122
+ - - ">="
123
+ - !ruby/object:Gem::Version
124
+ version: '0'
111
125
  - !ruby/object:Gem::Dependency
112
126
  name: dotenv
113
127
  requirement: !ruby/object:Gem::Requirement
@@ -132,6 +146,7 @@ executables:
132
146
  extensions: []
133
147
  extra_rdoc_files: []
134
148
  files:
149
+ - ".dockerignore"
135
150
  - ".gitignore"
136
151
  - ".rspec"
137
152
  - ".travis.yml"
@@ -145,9 +160,12 @@ files:
145
160
  - exe/scaltainer
146
161
  - lib/scaltainer.rb
147
162
  - lib/scaltainer/command.rb
148
- - lib/scaltainer/docker/service.rb
149
163
  - lib/scaltainer/exceptions.rb
150
164
  - lib/scaltainer/newrelic/metrics.rb
165
+ - lib/scaltainer/orchestrators.rb
166
+ - lib/scaltainer/orchestrators/base.rb
167
+ - lib/scaltainer/orchestrators/kubernetes.rb
168
+ - lib/scaltainer/orchestrators/swarm.rb
151
169
  - lib/scaltainer/runner.rb
152
170
  - lib/scaltainer/service_types.rb
153
171
  - lib/scaltainer/service_types/base.rb