floatyhelper 1.4 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b9107871b6337de1b4508247eebc2f59e1250e5ed7b61a4ee358f3e7d920e5e5
4
- data.tar.gz: d346fccc76b850c599d9ab7bef16e2a5eebb51897482158e273d45c7082bb9fa
3
+ metadata.gz: ab5a427189944d8960bab3ac42f1def0053e19fbed5a76d23fe866bef85ea28d
4
+ data.tar.gz: acddc4079e86b93b58a3ac1f4df3163fa440595e5f499f130ad5babeaa42f0cd
5
5
  SHA512:
6
- metadata.gz: 23e2932ddc4a3f11edbd8cff698420d83f3e41c9e0e13b92b8391e6eb474cfe5078958542e4b572a9f99728ed2a7fe6832fba17d9f3958d744da153cfb537832
7
- data.tar.gz: 1600273e214fb4abf841f64150ccaf393e14088e312e3147af8adb9c2634e4ce4c3642c0974470e03bf96a37147657d7750a5e7ae44940a38fe8bb0b0445747e
6
+ metadata.gz: 97a90ee999e59877c3125cd96611b3de3a5eead211800eb0f233e9e2a9951680583309b053b173a3cb402c2b8ed4f8e7f698fb44e5484c86520e1875f5820604
7
+ data.tar.gz: cbb27ccfe2310492034eb123a0bff8c7e39dc681091fc6bec444084a390c581c8e18d8d1b9ffd6a2729e1b591c0c1d52c93d7bb34bdf37e6c460f7d0e428bf6f
data/README.md CHANGED
@@ -1,28 +1,14 @@
1
1
  # Floatyhelper
2
2
 
3
- A CLI app for manipulating groups of VMs provisioned with vmpooler. Works on top of https://github.com/briancain/vmfloaty
3
+ A CLI app for manipulating groups of VMs provisioned with vmpooler. Works on top of https://github.com/puppetlabs/vmfloaty
4
4
 
5
- This was written to aid the installer and management team with testing various configurations of Puppet Enterprise. Groups of hosts can be added (either directly from a Beaker run from inside the pe_acceptance_tests folder it was run with, or manually with the --hosts flag) and given a "tag" to refer to the group of hosts by. These hosts can be snapshotted together, with these snapshots given a "snaptag" that can then be reverted to if needed.
5
+ This was written to aid Puppet development teams with testing various configurations of Puppet Enterprise or other Puppet products. Groups of hosts can be added (either directly from a Beaker run from inside the pe_acceptance_tests folder it was run with, or manually with the --hosts flag) and given a "tag" to refer to the group of hosts by. These hosts can be snapshotted together, with these snapshots given a "snaptag" that can then be reverted to if needed.
6
6
 
7
- NOTE: This requires that you have already set up your .vmfloaty file with the appropriate token. The app currently does not check that this is the case.
8
-
9
- This tool could use some specs and linting, probably.
7
+ The app will install vmfloaty for you and ensure your .vmfloaty.yml config file is correct. If no tokens are present in the file, it will request new tokens for you.
10
8
 
11
9
  ## Installation
12
10
 
13
- Add this line to your application's Gemfile:
14
-
15
- ```ruby
16
- gem 'floatyhelper'
17
- ```
18
-
19
- And then execute:
20
-
21
- $ bundle install
22
-
23
- Or install it yourself as:
24
-
25
- $ gem install floatyhelper
11
+ This app lives on RubyGems, so to install, simply run `gem install floatyhelper`
26
12
 
27
13
  ## Usage
28
14
 
@@ -31,21 +17,27 @@ floatyhelper <command> <options>
31
17
  ```
32
18
  The following commands are currently available:
33
19
 
34
- `floatyhelper tags` - List all currently tracked tags that specify groups of hosts
20
+ `floatyhelper config` - Lists, gets, or sets floatyhelper configuration settings.
21
+
22
+ `floatyhelper tags` - List all currently tracked tags that specify groups of hosts.
35
23
 
36
- `floatyhelper list` - List all VMs and their associated tags, and checks the remaining lifetime for each VM. If --nocheck is passed, lifetime checking will not occur.
24
+ `floatyhelper list` - List all VMs and their associated tags, and optionally checks the type and remaining lifetime for each VM.
37
25
 
38
- `floatyhelper snaplist <tag>` - List all of the snaptags associated with the given VM group tag
26
+ `floatyhelper check` - Same as `floatyhelper list --check`.
27
+
28
+ `floatyhelper snaplist <tag>` - List all of the snaptags associated with the given VM group tag.
39
29
 
40
30
  `floatyhelper showconfig` - Dumps the .floatyhelper.yaml file. Primarily used for debug purposes.
41
31
 
42
- `floatyhelper addhosts <tag> [--hosts HOST1,HOST2,...] [--file sutfile]` - Takes all hosts defined in a beaker sut.log file and adds them to the list of hosts floatyhelper is tracking. These hosts will be grouped under the given tag, so that this tag can be used to operate on the whole set of hosts at once. If `--file` is not specified, this assumes you are inside a pe_acceptance_tests folder and grabs the lsit of hosts from `log/latest/sut.log`. If --hosts is specified, it will add the given comma-separated list of hosts.
32
+ `floatyhelper addhosts <tag> [--hosts HOST1,HOST2,...] [--file sutfile]` - Takes all hosts defined in a beaker sut.log file and adds them to the list of hosts floatyhelper is tracking. These hosts will be grouped under the given tag, so that this tag can be used to operate on the whole set of hosts at once. If `--file` is not specified, this assumes you are inside a pe_acceptance_tests folder and grabs the list of hosts from `log/latest/sut.log`. If --hosts is specified, it will add the given comma-separated list of hosts.
33
+
34
+ `floatyhelper movehost <host> <tag>` - Moves a host between two different tags.
43
35
 
44
- `floatyhelper appendhosts <tag> [--hosts HOST1,HOST2,...] [--file sutfile]` - Similar to the addhosts command, but adds the given hosts to the list of hosts tracked under a given tag.
36
+ `floatyhelper destroy <tag> [--all] [--hosts HOSTS]` - Calls `floaty delete` on the vmpooler hosts with the given tag, or the given hosts, or all hosts managed by floatyhelper.
45
37
 
46
- `floatyhelper destroy <tag> [--all] [--hosts HOSTS]` - Calls `floaty delete` on the vmpooler hosts with the given tag.
38
+ `floatyhelper destroy_from_token` - This destroys all hosts tied to your vmpooler token. Note that this does not currently remove the hosts from floatyhelper management. This is mostly a helper for when you have been using `floaty` directly to request VMs and you have some that are not managed by floatyhelper.
47
39
 
48
- `floatyhelper increaselife <tag> [options]` - Increases the lifetime of the given host or tagged hosts. Defaults to 100 hours.
40
+ `floatyhelper increaselife <tag> [options]` - Increases the lifetime of the given host or tagged hosts. Defaults to the increaselife config setting value.
49
41
 
50
42
  `floatyhelper snapshot <host|tag> <snaptag>` - Snapshots the given VM or group of tagged hosts, and defines a string to use as a snaptag to refer to this group of snapshots.
51
43
 
@@ -53,13 +45,13 @@ The following commands are currently available:
53
45
 
54
46
  `floatyhelper refresh` - Removes tags that contain only expired VMs. You may pass the -y argument to run non-interactively.
55
47
 
56
- `floatyhelper getvm [platform] [tag]` - Request a VM from floaty, and add the host to the given tag. Defaults to centos-7-x86_64 if no platform is given, and defaults to "Unassigned" tag.
48
+ `floatyhelper getvm [platform] [tag]` - Request a VM from floaty, and add the host to the given tag. Defaults to centos-7-x86_64 if no platform is given, and defaults to "Unassigned" tag. If requesting a platform that is currently pooled, it will request it from vmpooler. Otherwise, it will request it from ABS (or if the --abs flag is used).
57
49
 
58
- `floatyhelper movehost <host> <tag>` - Moves the given host from its existing tag to the given tag
50
+ `floatyhelper checktokens` - Checks the tokens in .vmfloaty.yml and requests new ones if needed.
59
51
 
60
52
  ## Contributing
61
53
 
62
- Bug reports and pull requests are welcome on GitHub at https://github.com/nmburgan/floatyhelper.
54
+ Bug reports and pull requests are welcome on GitHub at https://github.com/puppetlabs/floatyhelper.
63
55
 
64
56
  To build locally,
65
57
  ```
@@ -68,19 +60,10 @@ bundle exec rake build
68
60
  ```
69
61
  This will create the gem in the `pkg` directory. You can then install it locally with `gem install`.
70
62
 
71
- ## License
63
+ Run specs with `bundle exec rake spec` and Rubocop with `bundle exec rake rubocop`. These will be run automatically by Travis on all PRs.
72
64
 
73
- The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
65
+ To release a new version, update lib/version.rb, then run `bundle exec build release`.
74
66
 
75
- ## Changelog
76
-
77
- ### 0.9
78
- * First release
67
+ ## License
79
68
 
80
- ### 1.0
81
- * Added refresh command
82
- * Added getvm command
83
- * Added movehost command
84
- * Added functionality to check for remaining VM lifetime with the list command
85
- * Added --nocheck option for list command to prevent checking VM lifetime
86
- * Tag automatically added if appendhosts is used with a tag that doesn't exist
69
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
@@ -0,0 +1,75 @@
1
+ # This is for directly manipulating the yaml file, as well as
2
+ # storing configuration for floatyhelper itself.
3
+ require 'yaml'
4
+ require 'etc'
5
+
6
+ class Config
7
+ DEFAULT_CONF_FILE = <<~EOT
8
+ ---
9
+ config: {}
10
+ vms: {}
11
+ snapshots: {}
12
+ EOT
13
+ .freeze
14
+
15
+ VALID_SETTINGS = {
16
+ 'increaselife' => {
17
+ 'description' => 'Number of hours beyond the service default to increase the VM lifetime when adding a VM to floatyhelper.',
18
+ 'default' => 0,
19
+ },
20
+ 'vertical_snapshot_status' => {
21
+ 'description' => 'When true, clear the screen and show the snapshot in progress status vertically. Otherwise, show status on a single line.',
22
+ 'default' => true,
23
+ },
24
+ }.freeze
25
+
26
+ attr_reader :VALID_SETTINGS
27
+
28
+ def self.fhfile
29
+ "#{Etc.getpwuid.dir}/.floatyhelper.yaml"
30
+ end
31
+
32
+ def self.load_data
33
+ File.write(fhfile, DEFAULT_CONF_FILE) unless File.exist?(fhfile)
34
+ data = YAML.load_file(fhfile)
35
+ # If someone is using an older version, ensure the conf file has
36
+ # all the correct top-level keys. Only write it back if we modify
37
+ # the data to add a top-level key.
38
+ writeback = false
39
+ ['config', 'vms', 'snapshots'].each do |key|
40
+ if !data.keys.include?(key)
41
+ data[key] = {}
42
+ writeback = true
43
+ end
44
+ end
45
+
46
+ # Config setting defaults
47
+ VALID_SETTINGS.each do |setting, info|
48
+ writeback = true if data['config'][setting].nil?
49
+ data['config'][setting] ||= info['default']
50
+ end
51
+
52
+ write_data(data) if writeback
53
+ data
54
+ end
55
+
56
+ # It is up to the calling function to ensure the data object
57
+ # being passed in is a properly formatted hash. It should only be
58
+ # used to modify after reading the current state with load_data.
59
+ def self.write_data(data)
60
+ File.write(fhfile, data.to_yaml)
61
+ end
62
+
63
+ # NOTE: Anything currently set with 'floatyhelper config set' will return a string,
64
+ # regardless of its intended type. Need to add typing one of these days.
65
+ def self.get_config_setting(setting)
66
+ data = load_data
67
+ data['config'][setting]
68
+ end
69
+
70
+ def self.set_config_setting(setting, value)
71
+ data = load_data
72
+ data['config'][setting] = value
73
+ write_data(data)
74
+ end
75
+ end
@@ -0,0 +1,154 @@
1
+ # This is for talking to floaty and getting data back from it.
2
+ require 'json'
3
+ require 'open3'
4
+ require 'colorize'
5
+ require 'pty'
6
+
7
+ class Floaty
8
+ FLOATY_SERVICES = {
9
+ 'abs' => {
10
+ 'type' => 'abs',
11
+ 'url' => 'https://abs-prod.k8s.infracore.puppet.net/api/v2',
12
+ },
13
+ 'vmpooler' => {
14
+ 'type' => 'vm',
15
+ 'url' => 'http://vmpooler.delivery.puppetlabs.net',
16
+ },
17
+ 'nspooler' => {
18
+ 'type' => 'nonstandard',
19
+ 'url' => 'https://nspooler-prod.k8s.infracore.puppet.net',
20
+ },
21
+ }.freeze
22
+
23
+ def self.vmfloatyfile
24
+ "#{Etc.getpwuid.dir}/.vmfloaty.yml"
25
+ end
26
+
27
+ def self.load_vmfloaty_config
28
+ if File.exist?(vmfloatyfile)
29
+ YAML.safe_load(File.read(vmfloatyfile))
30
+ else
31
+ {}
32
+ end
33
+ end
34
+
35
+ # This adds some stdout printing here which I was trying to avoid, but
36
+ # I think it makes sense to put it here rather than the main floatyhelper
37
+ # somewhere.
38
+ def self.check_floaty(data = nil)
39
+ data ||= load_vmfloaty_config
40
+
41
+ user = data['user']
42
+ unless user
43
+ puts 'No username found in .vmfloaty.yml.'.yellow
44
+ user = ask('Username: ').chomp
45
+ data['user'] = user
46
+ end
47
+ data['services'] ||= {}
48
+ puts 'No services defined in .vmfloaty.yml.' if data['services'].empty?
49
+
50
+ need_token = FLOATY_SERVICES.keys.any? { |k| !data['services'].keys.include?(k) } || data['services'].any? { |_svc, val| val['token'].nil? }
51
+ return unless need_token
52
+
53
+ puts 'It appears we need to fetch one or more tokens for the ~/.vmfloaty.yml file. Please enter your Puppet password.'
54
+ password = ask('Password: ') { |q| q.echo = '*' }
55
+ FLOATY_SERVICES.each do |service, info|
56
+ next if data['services'][service] && data['services'][service]['token']
57
+
58
+ data['services'][service] ||= {}
59
+ # Kind of silly to call out to curl here. Replace this with a Net::HTTP call
60
+ output, status = Open3.capture2e("curl -s -X POST -u #{user}:#{password} --url #{info['url']}/token")
61
+ unless status.exitstatus.zero?
62
+ puts "Bad return status from curl to #{info['url']}: #{status.exitstatus}".red
63
+ exit status.exitstatus
64
+ end
65
+ result = JSON.parse(output)
66
+ if result['ok']
67
+ data['services'][service]['type'] = info['type']
68
+ data['services'][service]['url'] = info['url']
69
+ data['services'][service]['token'] = result['token']
70
+ puts "Successfully fetched token for #{service}".green
71
+ else
72
+ puts "Could not get a token from #{service}. Please ensure your username and password are correct.".red
73
+ puts "Result: #{result}".red
74
+ exit 1
75
+ end
76
+ end
77
+ File.write(vmfloatyfile, data.to_yaml)
78
+ end
79
+
80
+ # Make sure all tokens are valid
81
+ def self.check_tokens
82
+ data = load_vmfloaty_config
83
+ issues = false
84
+ FLOATY_SERVICES.each do |service, _info|
85
+ if data['services'].nil? || data['services'][service].nil? || data['services'][service]['token'].nil?
86
+ puts "#{service} service token not found in .vmfloaty.yml".yellow
87
+ issues = true
88
+ next
89
+ end
90
+ # TODO: See what exitcode is when token is bad vs. some other fatal error
91
+ output, _status = Open3.capture2e("/usr/bin/env floaty token status --service #{service}")
92
+ result = parse_floaty_json_output(output)
93
+ next if result['ok']
94
+
95
+ puts "Problem checking #{service} token: #{result['reason']}".red
96
+ data['services']['token'] = nil
97
+ issues = true
98
+ end
99
+ if issues
100
+ check_floaty(data)
101
+ else
102
+ puts 'All tokens valid'.green
103
+ end
104
+ end
105
+
106
+ def self.parse_floaty_json_output(output)
107
+ # Sometimes there will be extra stuff before the hash output,
108
+ # so ignore it until we reach the hash.
109
+ lines = output.split("\n")
110
+ index = lines.index { |l| l[0] == '{' }
111
+ output = lines[index..-1].join
112
+ JSON.parse(output.gsub('=>',':'))
113
+ end
114
+
115
+ def self.floaty_cmd(command, ignore_error: false, use_pty: false)
116
+ check_floaty
117
+ # While we could potentially make a Vmfloaty object and send a command
118
+ # that way, Commander doesn't make it easy. Parsing floaty's stdout
119
+ # isn't great, but it works for now.
120
+ if use_pty
121
+ output = ''
122
+ status = nil
123
+ PTY.spawn("/usr/bin/env floaty #{command}") do |read, write, pid|
124
+ write.close
125
+ while status.nil?
126
+ begin
127
+ line = read.gets
128
+ puts line
129
+ output += line
130
+ rescue EOFError, Errno::EIO
131
+ # GNU/Linux raises EIO on read operation when pty is closed - see pty.rb docs
132
+ # Ensure child process finishes and then pass through to ensure below to get status
133
+ nil
134
+ rescue IO::WaitReadable, IO::WaitWritable
135
+ retry
136
+ ensure
137
+ status ||= PTY.check(pid)
138
+ end
139
+ end
140
+
141
+ Process.waitall
142
+ # Double check we have the status
143
+ status ||= PTY.check(pid)
144
+ end
145
+ else
146
+ output, status = Open3.capture2e("/usr/bin/env floaty #{command}")
147
+ end
148
+ if !status.exitstatus.zero? && !ignore_error
149
+ puts "Error running 'floaty #{command}': #{output}".red
150
+ exit status.exitstatus
151
+ end
152
+ output
153
+ end
154
+ end
@@ -1,65 +1,76 @@
1
- require 'floatyhelper/conf'
1
+ # This is for the management of hosts and snapshots in the yaml config file.
2
+ # Anything for adding, removing, or modifying hosts or snapshots in the
3
+ # floatyhelper.yaml file belongs here. Anything for modifying the VMs themselves
4
+ # does not.
5
+ require 'floatyhelper/config'
2
6
 
3
7
  class Groups
4
- def self.is_tag?(id)
5
- data = Conf.load_data
8
+ def self.get_tags
9
+ data = Config.load_data
10
+ data['vms'].keys
11
+ end
12
+
13
+ def self.tag?(id)
14
+ data = Config.load_data
6
15
  data['vms'].keys.include?(id)
7
16
  end
8
17
 
9
- def self.delete_tag(tag)
10
- data = Conf.load_data
18
+ def self.delete_tag(tag, data: nil)
19
+ data ||= Config.load_data
11
20
  data['vms'].delete(tag)
12
21
  data['snapshots'].delete(tag) if data['snapshots'].keys.include?(tag)
13
- Conf.write_data(data)
22
+ Config.write_data(data)
14
23
  end
15
24
 
16
25
  def self.delete_all
17
- data = Conf.load_data
26
+ data = Config.load_data
18
27
  data['vms'] = {}
19
28
  data['snapshots'] = {}
20
- Conf.write_data(data)
29
+ Config.write_data(data)
21
30
  end
22
31
 
23
- def self.addhosts(hosts, tag)
24
- data = Conf.load_data
25
- data['vms'] ||= {}
26
- tag = 'Blank Tag' unless tag #Removed the ability to do this
27
- hosts.each do |host|
28
- data['vms'][tag] ||= []
29
- data['vms'][tag] << host unless data['vms'][tag].include?(host)
30
- end
31
- Conf.write_data(data)
32
+ def self.get_hosts(tag)
33
+ data = Config.load_data
34
+ data['vms'][tag]
32
35
  end
33
36
 
34
- def self.appendhosts(hosts, tag)
35
- data = Conf.load_data
37
+ def self.addhosts(hosts, tag)
38
+ data = Config.load_data
39
+ data['vms'] ||= {}
40
+ tag ||= 'Blank Tag' # Removed the ability to do this
36
41
  hosts.each do |host|
37
42
  data['vms'][tag] ||= []
38
43
  data['vms'][tag] << host unless data['vms'][tag].include?(host)
39
44
  end
40
- Conf.write_data(data)
45
+ Config.write_data(data)
41
46
  end
42
47
 
43
48
  def self.removehosts(hosts, tag)
44
- data = Conf.load_data
49
+ data = Config.load_data
50
+ return if data['vms'][tag].nil?
51
+
45
52
  hosts.each do |host|
46
53
  data['vms'][tag] ||= []
47
54
  data['vms'][tag].delete(host)
48
- data['vms'].delete(tag) if data['vms'][tag].empty?
55
+ if data['snapshots'][tag]
56
+ data['snapshots'][tag].each do |snaptag, _vals|
57
+ data['snapshots'][tag][snaptag].delete(host)
58
+ end
59
+ end
60
+ delete_tag(tag, data: data) if data['vms'][tag].empty?
49
61
  end
50
- Conf.write_data(data)
62
+ Config.write_data(data)
51
63
  end
52
64
 
53
- def self.is_managed_host?(host)
54
- data = Conf.load_data
55
- data['vms'].any? { |tag, hosts| hosts.include?(host) }
65
+ def self.managed_host?(host)
66
+ data = Config.load_data
67
+ data['vms'].any? { |_tag, hosts| hosts.include?(host) }
56
68
  end
57
69
 
58
70
  def self.host_tag(host)
59
- data = Conf.load_data
60
- raise 'Host not found' unless is_managed_host?(host)
61
- tags = data['vms'].select { |tag, hosts| hosts.include?(host) }.keys
71
+ data = Config.load_data
72
+ raise 'Host not found' unless managed_host?(host)
73
+ tags = data['vms'].select { |_tag, hosts| hosts.include?(host) }.keys
62
74
  tags[0]
63
75
  end
64
-
65
- end
76
+ end
@@ -1,14 +1,15 @@
1
- require 'floatyhelper/conf'
1
+ # This is for finding hosts in various places (sut.log, --hosts flag)
2
+ # and returning an array of hostnames.
3
+ require 'floatyhelper/config'
2
4
  require 'floatyhelper/groups'
3
5
 
4
6
  class Hosts
5
-
6
7
  def self.get_hosts_from_sut_log(file)
7
8
  hosts = []
8
9
  File.open(file).each do |line|
9
10
  items = line.split("\t")
10
- hostname, tag = items[4].split(" ")
11
- short_host = hostname.split(".")[0]
11
+ hostname, tag = items[4].split # rubocop:disable Lint/UselessAssignment
12
+ short_host = hostname.split('.')[0]
12
13
  hosts << short_host
13
14
  end
14
15
  hosts
@@ -20,20 +21,19 @@ class Hosts
20
21
  end
21
22
 
22
23
  def self.get_hosts_from_id(id)
23
- data = Conf.load_data
24
+ data = Config.load_data
24
25
  if id == 'all'
25
26
  hosts = []
26
- data['vms'].each do |tag, hostlist|
27
+ data['vms'].each do |_tag, hostlist|
27
28
  hostlist.each do |host|
28
29
  hosts << host unless hosts.include?(host)
29
30
  end
30
31
  end
31
- elsif Groups.is_tag?(id)
32
+ elsif Groups.tag?(id)
32
33
  hosts = data['vms'][id]
33
34
  else
34
35
  hosts = [id].flatten
35
36
  end
36
37
  hosts
37
38
  end
38
-
39
- end
39
+ end
@@ -1,3 +1,3 @@
1
- module Floatyhelper
2
- VERSION = "1.4"
1
+ module FloatyhelperVersion
2
+ VERSION = '2.0.1'.freeze
3
3
  end