vmfloaty 0.11.1 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +19 -75
- data/extras/completions/floaty.bash +2 -2
- data/extras/completions/floaty.zsh +37 -0
- data/lib/vmfloaty.rb +92 -7
- data/lib/vmfloaty/abs.rb +118 -49
- data/lib/vmfloaty/conf.rb +1 -1
- data/lib/vmfloaty/service.rb +20 -0
- data/lib/vmfloaty/utils.rb +79 -11
- data/lib/vmfloaty/version.rb +2 -1
- data/spec/vmfloaty/abs_spec.rb +52 -5
- data/spec/vmfloaty/utils_spec.rb +436 -73
- metadata +11 -10
    
        checksums.yaml
    CHANGED
    
    | @@ -1,7 +1,7 @@ | |
| 1 1 | 
             
            ---
         | 
| 2 2 | 
             
            SHA256:
         | 
| 3 | 
            -
              metadata.gz:  | 
| 4 | 
            -
              data.tar.gz:  | 
| 3 | 
            +
              metadata.gz: 4cf673c0a3e005fc4ef5102d232206f3105215de4f19cd76725de335b62937fd
         | 
| 4 | 
            +
              data.tar.gz: e4791593e75d31d42a8f05cfb28b3655e42218ac1a7191ae1c8391ccdad5133b
         | 
| 5 5 | 
             
            SHA512:
         | 
| 6 | 
            -
              metadata.gz:  | 
| 7 | 
            -
              data.tar.gz:  | 
| 6 | 
            +
              metadata.gz: b22ad1332f3a663407a6ce964525606938134925e0b91ccd151a8f9a4abc265cab53cddd9509ce92555abb357dc213f2beef50a02ea72eb501db31c9e3936983
         | 
| 7 | 
            +
              data.tar.gz: 8436c38cc6a99b76c4e65da0dfa653edbbdbe9b7fcefb9607942983cd24014c40ab14f556b06e7d6a449dc57f84740e09df2668e15129cdb5dccbb3b6e746eff
         | 
    
        data/README.md
    CHANGED
    
    | @@ -14,9 +14,8 @@ A CLI helper tool for [Puppet's vmpooler](https://github.com/puppetlabs/vmpooler | |
| 14 14 | 
             
              - [Example workflow](#example-workflow)
         | 
| 15 15 | 
             
              - [vmfloaty dotfile](#vmfloaty-dotfile)
         | 
| 16 16 | 
             
                - [Basic configuration](#basic-configuration)
         | 
| 17 | 
            -
                - [ | 
| 18 | 
            -
                - [ | 
| 19 | 
            -
                - [Using a Nonstandard Pooler service](#using-a-nonstandard-pooler-service)
         | 
| 17 | 
            +
                - [Using multiple services](#using-multiple-services)
         | 
| 18 | 
            +
                - [Using backends besides VMPooler](#using-backends-besides-vmpooler)
         | 
| 20 19 | 
             
                - [Valid config keys](#valid-config-keys)
         | 
| 21 20 | 
             
              - [Tab Completion](#tab-completion)
         | 
| 22 21 | 
             
            - [vmpooler API](#vmpooler-api)
         | 
| @@ -54,6 +53,7 @@ $ floaty --help | |
| 54 53 | 
             
                modify     Modify a VM's tags, time to live, disk space, or reservation reason
         | 
| 55 54 | 
             
                query      Get information about a given vm
         | 
| 56 55 | 
             
                revert     Reverts a vm to a specified snapshot
         | 
| 56 | 
            +
                service    Display information about floaty services and their configuration
         | 
| 57 57 | 
             
                snapshot   Takes a snapshot of a given vm
         | 
| 58 58 | 
             
                ssh        Grabs a single vm and sshs into it
         | 
| 59 59 | 
             
                status     Prints the status of pools in the pooler service
         | 
| @@ -90,95 +90,32 @@ floaty get centos-7-x86_64=2 debian-7-x86_64 windows-10=3 --token mytokenstring | |
| 90 90 |  | 
| 91 91 | 
             
            ### vmfloaty dotfile
         | 
| 92 92 |  | 
| 93 | 
            -
            If you do not wish to continually specify various config options with the cli, you can  | 
| 93 | 
            +
            If you do not wish to continually specify various config options with the cli, you can `~/.vmfloaty.yml` for some defaults. You can get a list of valid service types and example configuration files via `floaty service types` and `floaty service examples`, respectively.
         | 
| 94 94 |  | 
| 95 95 | 
             
            #### Basic configuration
         | 
| 96 96 |  | 
| 97 | 
            -
             | 
| 98 | 
            -
            # file at ~/.vmfloaty.yml
         | 
| 99 | 
            -
            url: 'https://vmpooler.example.net/api/v1'
         | 
| 100 | 
            -
            user: 'brian'
         | 
| 101 | 
            -
            token: 'tokenstring'
         | 
| 102 | 
            -
            ```
         | 
| 103 | 
            -
             | 
| 104 | 
            -
            Now vmfloaty will use those config files if no flag was specified.
         | 
| 105 | 
            -
             | 
| 106 | 
            -
            #### Default to Puppet's ABS instead of vmpooler
         | 
| 97 | 
            +
            This is the simplest type of configuration where you only need a single service:
         | 
| 107 98 |  | 
| 108 99 | 
             
            ```yaml
         | 
| 109 100 | 
             
            # file at ~/.vmfloaty.yml
         | 
| 110 | 
            -
            url: 'https:// | 
| 101 | 
            +
            url: 'https://vmpooler.example.net/api/v1'
         | 
| 111 102 | 
             
            user: 'brian'
         | 
| 112 103 | 
             
            token: 'tokenstring'
         | 
| 113 | 
            -
            type: 'abs'
         | 
| 114 104 | 
             
            ```
         | 
| 115 105 |  | 
| 116 | 
            -
             | 
| 106 | 
            +
            Run `floaty service examples` to see additional configuration options
         | 
| 117 107 |  | 
| 118 | 
            -
             | 
| 108 | 
            +
            #### Using multiple services
         | 
| 119 109 |  | 
| 120 | 
            -
             | 
| 121 | 
            -
             | 
| 122 | 
            -
            ```yaml
         | 
| 123 | 
            -
            # file at /Users/me/.vmfloaty.yml
         | 
| 124 | 
            -
            user: 'brian'
         | 
| 125 | 
            -
            services:
         | 
| 126 | 
            -
              main:
         | 
| 127 | 
            -
                url: 'https://vmpooler.example.net/api/v1'
         | 
| 128 | 
            -
                token: 'tokenstring'
         | 
| 129 | 
            -
              alternate:
         | 
| 130 | 
            -
                url: 'https://vmpooler.example.com/api/v1'
         | 
| 131 | 
            -
                token: 'alternate-tokenstring'
         | 
| 132 | 
            -
            ```
         | 
| 110 | 
            +
            Most commands allow you to specify a `--service <servicename>` option to allow the use of multiple pooler instances. This can be useful when you'd rather not specify a `--url` or `--token` by hand for alternate services.
         | 
| 133 111 |  | 
| 134 112 | 
             
            - If you run `floaty` without a `--service <name>` option, vmfloaty will use the first configured service by default.
         | 
| 135 | 
            -
              With the config file above, the default would be to use the 'main' vmpooler instance.
         | 
| 136 113 | 
             
            - If keys are missing for a configured service, vmfloaty will attempt to fall back to the top-level values.
         | 
| 137 | 
            -
               | 
| 138 | 
            -
             | 
| 139 | 
            -
            Examples using the above configuration:
         | 
| 140 | 
            -
             | 
| 141 | 
            -
            List available vm types from our main vmpooler instance:
         | 
| 142 | 
            -
             | 
| 143 | 
            -
            ```bash
         | 
| 144 | 
            -
            floaty list --service main
         | 
| 145 | 
            -
            # or, since the first configured service is used by default:
         | 
| 146 | 
            -
            floaty list
         | 
| 147 | 
            -
            ```
         | 
| 148 | 
            -
             | 
| 149 | 
            -
            List available vm types from our alternate vmpooler instance:
         | 
| 150 | 
            -
             | 
| 151 | 
            -
            ```bash
         | 
| 152 | 
            -
            floaty list --service alternate
         | 
| 153 | 
            -
            ```
         | 
| 114 | 
            +
              This makes it so you can specify things like `user` once at the top of your `~/.vmfloaty.yml`.
         | 
| 154 115 |  | 
| 155 | 
            -
            #### Using  | 
| 116 | 
            +
            #### Using backends besides VMPooler
         | 
| 156 117 |  | 
| 157 | 
            -
            vmfloaty  | 
| 158 | 
            -
             | 
| 159 | 
            -
            ```yaml
         | 
| 160 | 
            -
            # file at /Users/me/.vmfloaty.yml
         | 
| 161 | 
            -
            user: 'brian'
         | 
| 162 | 
            -
            services:
         | 
| 163 | 
            -
              vm:
         | 
| 164 | 
            -
                url: 'https://vmpooler.example.net/api/v1'
         | 
| 165 | 
            -
                token: 'tokenstring'
         | 
| 166 | 
            -
              ns:
         | 
| 167 | 
            -
                url: 'https://nspooler.example.net/api/v1'
         | 
| 168 | 
            -
                token: 'nspooler-tokenstring'
         | 
| 169 | 
            -
                type: 'nonstandard'  # <-- 'type' is necessary for any non-vmpooler service
         | 
| 170 | 
            -
              abs:
         | 
| 171 | 
            -
                url: 'https://abs.example.net/'
         | 
| 172 | 
            -
                token: 'abs-tokenstring'
         | 
| 173 | 
            -
                type: 'abs'  # <-- 'type' is necessary for any non-vmpooler service
         | 
| 174 | 
            -
             | 
| 175 | 
            -
            ```
         | 
| 176 | 
            -
             | 
| 177 | 
            -
            With this configuration, you could list available OS types from nspooler like this:
         | 
| 178 | 
            -
             | 
| 179 | 
            -
            ```bash
         | 
| 180 | 
            -
            floaty list --service ns
         | 
| 181 | 
            -
            ```
         | 
| 118 | 
            +
            vmfloaty supports additional backends besides VMPooler. To see a complete list, run `floaty service types`. The output of `floaty service examples` will show you how to configure each of the supported backends.
         | 
| 182 119 |  | 
| 183 120 | 
             
            #### Valid config keys
         | 
| 184 121 |  | 
| @@ -190,6 +127,7 @@ Here are the keys that vmfloaty currently supports: | |
| 190 127 | 
             
            - url (String)
         | 
| 191 128 | 
             
            - services (String)
         | 
| 192 129 | 
             
            - type (String)
         | 
| 130 | 
            +
            - vmpooler_fallback (String)
         | 
| 193 131 |  | 
| 194 132 | 
             
            ### Tab Completion
         | 
| 195 133 |  | 
| @@ -207,6 +145,12 @@ If you are running on macOS and use Homebrew's `bash-completion` formula, you ca | |
| 207 145 | 
             
            ln -s $(floaty completion --shell bash) /usr/local/etc/bash_completion.d/floaty
         | 
| 208 146 | 
             
            ```
         | 
| 209 147 |  | 
| 148 | 
            +
            There is also tab completion for zsh:
         | 
| 149 | 
            +
             | 
| 150 | 
            +
            ```zsh
         | 
| 151 | 
            +
            source $(floaty completion --shell zsh)
         | 
| 152 | 
            +
            ```
         | 
| 153 | 
            +
             | 
| 210 154 | 
             
            ## vmpooler API
         | 
| 211 155 |  | 
| 212 156 | 
             
            This cli tool uses the [vmpooler API](https://github.com/puppetlabs/vmpooler/blob/master/API.md).
         | 
| @@ -7,7 +7,7 @@ _vmfloaty() | |
| 7 7 | 
             
              cur="${COMP_WORDS[COMP_CWORD]}"
         | 
| 8 8 | 
             
              prev="${COMP_WORDS[COMP_CWORD-1]}"
         | 
| 9 9 |  | 
| 10 | 
            -
              subcommands="delete get help list modify query revert snapshot ssh status summary token"
         | 
| 10 | 
            +
              subcommands="delete get help list modify query revert service snapshot ssh status summary token"
         | 
| 11 11 | 
             
              template_subcommands="get ssh"
         | 
| 12 12 | 
             
              hostname_subcommands="delete modify query revert snapshot"
         | 
| 13 13 |  | 
| @@ -21,7 +21,7 @@ _vmfloaty() | |
| 21 21 |  | 
| 22 22 | 
             
                COMPREPLY=( $(compgen -W "${_vmfloaty_avail_templates}" -- "${cur}") )
         | 
| 23 23 | 
             
              elif [[ $hostname_subcommands =~ (^| )$prev($| ) ]] ; then
         | 
| 24 | 
            -
                _vmfloaty_active_hostnames=$(floaty list --active 2>/dev/null | 
| 24 | 
            +
                _vmfloaty_active_hostnames=$(floaty list --active --hostnameonly 2>/dev/null)
         | 
| 25 25 | 
             
                COMPREPLY=( $(compgen -W "${_vmfloaty_active_hostnames}" -- "${cur}") )
         | 
| 26 26 | 
             
              else
         | 
| 27 27 | 
             
                COMPREPLY=( $(compgen -W "${subcommands}" -- "${cur}") )
         | 
| @@ -0,0 +1,37 @@ | |
| 1 | 
            +
            _floaty()
         | 
| 2 | 
            +
            {
         | 
| 3 | 
            +
              local line subcommands template_subcommands hostname_subcommands
         | 
| 4 | 
            +
             | 
| 5 | 
            +
              subcommands="delete get help list modify query revert snapshot ssh status summary token"
         | 
| 6 | 
            +
             | 
| 7 | 
            +
              template_subcommands=("get" "ssh")
         | 
| 8 | 
            +
              hostname_subcommands=("delete" "modify" "query" "revert" "snapshot")
         | 
| 9 | 
            +
             | 
| 10 | 
            +
              _arguments -C \
         | 
| 11 | 
            +
                "1: :(${subcommands})" \
         | 
| 12 | 
            +
                "*::arg:->args"
         | 
| 13 | 
            +
             | 
| 14 | 
            +
              if ((template_subcommands[(Ie)$line[1]])); then
         | 
| 15 | 
            +
                _floaty_template_sub
         | 
| 16 | 
            +
              elif ((hostname_subcommands[(Ie)$line[1]])); then
         | 
| 17 | 
            +
                _floaty_hostname_sub
         | 
| 18 | 
            +
              fi
         | 
| 19 | 
            +
            }
         | 
| 20 | 
            +
             | 
| 21 | 
            +
            _floaty_template_sub()
         | 
| 22 | 
            +
            {
         | 
| 23 | 
            +
              if [[ -z "$_vmfloaty_avail_templates" ]] ; then
         | 
| 24 | 
            +
                _vmfloaty_avail_templates=$(floaty list 2>/dev/null)
         | 
| 25 | 
            +
              fi
         | 
| 26 | 
            +
             | 
| 27 | 
            +
              _arguments "1: :(${_vmfloaty_avail_templates})"
         | 
| 28 | 
            +
            }
         | 
| 29 | 
            +
             | 
| 30 | 
            +
            _floaty_hostname_sub()
         | 
| 31 | 
            +
            {
         | 
| 32 | 
            +
              _vmfloaty_active_hostnames=$(floaty list --active --hostnameonly 2>/dev/null)
         | 
| 33 | 
            +
             | 
| 34 | 
            +
              _arguments "1: :(${_vmfloaty_active_hostnames})"
         | 
| 35 | 
            +
            }
         | 
| 36 | 
            +
             | 
| 37 | 
            +
            compdef _floaty floaty
         | 
    
        data/lib/vmfloaty.rb
    CHANGED
    
    | @@ -20,7 +20,7 @@ class Vmfloaty | |
| 20 20 |  | 
| 21 21 | 
             
              def run # rubocop:disable Metrics/AbcSize
         | 
| 22 22 | 
             
                program :version, Vmfloaty::VERSION
         | 
| 23 | 
            -
                program :description, "A CLI helper tool for Puppet's vmpooler to help you stay afloat"
         | 
| 23 | 
            +
                program :description, "A CLI helper tool for Puppet's vmpooler to help you stay afloat.\n\nConfiguration may be placed in a ~/.vmfloaty.yml file."
         | 
| 24 24 |  | 
| 25 25 | 
             
                config = Conf.read_config
         | 
| 26 26 |  | 
| @@ -88,8 +88,10 @@ class Vmfloaty | |
| 88 88 | 
             
                  c.option '--service STRING', String, 'Configured pooler service name'
         | 
| 89 89 | 
             
                  c.option '--active', 'Prints information about active vms for a given token'
         | 
| 90 90 | 
             
                  c.option '--json', 'Prints information as JSON'
         | 
| 91 | 
            +
                  c.option '--hostnameonly', 'When listing active vms, prints only hostnames, one per line'
         | 
| 91 92 | 
             
                  c.option '--token STRING', String, 'Token for pooler service'
         | 
| 92 93 | 
             
                  c.option '--url STRING', String, 'URL of pooler service'
         | 
| 94 | 
            +
                  c.option '--user STRING', String, 'User to authenticate with'
         | 
| 93 95 | 
             
                  c.action do |args, options|
         | 
| 94 96 | 
             
                    verbose = options.verbose || config['verbose']
         | 
| 95 97 |  | 
| @@ -98,7 +100,12 @@ class Vmfloaty | |
| 98 100 |  | 
| 99 101 | 
             
                    if options.active
         | 
| 100 102 | 
             
                      # list active vms
         | 
| 101 | 
            -
                       | 
| 103 | 
            +
                      if service.type == "ABS"
         | 
| 104 | 
            +
                        # this is actually job_ids
         | 
| 105 | 
            +
                        running_vms = service.list_active_job_ids(verbose, service.url, service.user)
         | 
| 106 | 
            +
                      else
         | 
| 107 | 
            +
                        running_vms = service.list_active(verbose)
         | 
| 108 | 
            +
                      end
         | 
| 102 109 | 
             
                      host = URI.parse(service.url).host
         | 
| 103 110 | 
             
                      if running_vms.empty?
         | 
| 104 111 | 
             
                        if options.json
         | 
| @@ -109,6 +116,10 @@ class Vmfloaty | |
| 109 116 | 
             
                      else
         | 
| 110 117 | 
             
                        if options.json
         | 
| 111 118 | 
             
                          puts Utils.get_host_data(verbose, service, running_vms).to_json
         | 
| 119 | 
            +
                        elsif options.hostnameonly
         | 
| 120 | 
            +
                          Utils.get_host_data(verbose, service, running_vms).each do |hostname, host_data|
         | 
| 121 | 
            +
                            Utils.print_fqdn_for_host(service, hostname, host_data)
         | 
| 122 | 
            +
                          end
         | 
| 112 123 | 
             
                        else
         | 
| 113 124 | 
             
                          puts "Your VMs on #{host}:"
         | 
| 114 125 | 
             
                          Utils.pretty_print_hosts(verbose, service, running_vms)
         | 
| @@ -125,7 +136,7 @@ class Vmfloaty | |
| 125 136 | 
             
                command :query do |c|
         | 
| 126 137 | 
             
                  c.syntax = 'floaty query hostname [options]'
         | 
| 127 138 | 
             
                  c.summary = 'Get information about a given vm'
         | 
| 128 | 
            -
                  c.description = 'Given a hostname from the pooler service, vmfloaty with query the service to get various details about the vm.'
         | 
| 139 | 
            +
                  c.description = 'Given a hostname from the pooler service, vmfloaty with query the service to get various details about the vm. If using ABS, you can query a job_id'
         | 
| 129 140 | 
             
                  c.example 'Get information about a sample host', 'floaty query hostname --url http://vmpooler.example.com'
         | 
| 130 141 | 
             
                  c.option '--verbose', 'Enables verbose output'
         | 
| 131 142 | 
             
                  c.option '--service STRING', String, 'Configured pooler service name'
         | 
| @@ -164,7 +175,12 @@ class Vmfloaty | |
| 164 175 | 
             
                      FloatyLogger.error 'ERROR: Provide a hostname or specify --all.'
         | 
| 165 176 | 
             
                      exit 1
         | 
| 166 177 | 
             
                    end
         | 
| 167 | 
            -
                    running_vms = | 
| 178 | 
            +
                    running_vms =
         | 
| 179 | 
            +
                        if modify_all
         | 
| 180 | 
            +
                          service.list_active(verbose)
         | 
| 181 | 
            +
                        else
         | 
| 182 | 
            +
                          hostname.split(',')
         | 
| 183 | 
            +
                        end
         | 
| 168 184 |  | 
| 169 185 | 
             
                    tags = options.tags ? JSON.parse(options.tags) : nil
         | 
| 170 186 | 
             
                    modify_hash = {
         | 
| @@ -188,7 +204,7 @@ class Vmfloaty | |
| 188 204 | 
             
                      end
         | 
| 189 205 | 
             
                      if ok
         | 
| 190 206 | 
             
                        if modify_all
         | 
| 191 | 
            -
                          puts  | 
| 207 | 
            +
                          puts "Successfully modified all #{running_vms.count} VMs."
         | 
| 192 208 | 
             
                        else
         | 
| 193 209 | 
             
                          puts "Successfully modified VM #{hostname}."
         | 
| 194 210 | 
             
                        end
         | 
| @@ -212,6 +228,7 @@ class Vmfloaty | |
| 212 228 | 
             
                  c.option '--json', 'Outputs hosts scheduled for deletion as JSON'
         | 
| 213 229 | 
             
                  c.option '--token STRING', String, 'Token for pooler service'
         | 
| 214 230 | 
             
                  c.option '--url STRING', String, 'URL of pooler service'
         | 
| 231 | 
            +
                  c.option '--user STRING', String, 'User to authenticate with'
         | 
| 215 232 | 
             
                  c.action do |args, options|
         | 
| 216 233 | 
             
                    verbose = options.verbose || config['verbose']
         | 
| 217 234 | 
             
                    service = Service.new(options, config)
         | 
| @@ -223,7 +240,12 @@ class Vmfloaty | |
| 223 240 | 
             
                    successes = []
         | 
| 224 241 |  | 
| 225 242 | 
             
                    if delete_all
         | 
| 226 | 
            -
                       | 
| 243 | 
            +
                      if service.type == "ABS"
         | 
| 244 | 
            +
                        # this is actually job_ids
         | 
| 245 | 
            +
                        running_vms = service.list_active_job_ids(verbose, service.url, service.user)
         | 
| 246 | 
            +
                      else
         | 
| 247 | 
            +
                        running_vms = service.list_active(verbose)
         | 
| 248 | 
            +
                      end
         | 
| 227 249 | 
             
                      if running_vms.empty?
         | 
| 228 250 | 
             
                        if options.json
         | 
| 229 251 | 
             
                          puts {}.to_json
         | 
| @@ -272,7 +294,6 @@ class Vmfloaty | |
| 272 294 | 
             
                    end
         | 
| 273 295 |  | 
| 274 296 | 
             
                    unless successes.empty?
         | 
| 275 | 
            -
                      FloatyLogger.info unless failures.empty?
         | 
| 276 297 | 
             
                      if options.json
         | 
| 277 298 | 
             
                        puts successes.to_json
         | 
| 278 299 | 
             
                      else
         | 
| @@ -481,6 +502,70 @@ class Vmfloaty | |
| 481 502 | 
             
                  end
         | 
| 482 503 | 
             
                end
         | 
| 483 504 |  | 
| 505 | 
            +
                command :service do |c|
         | 
| 506 | 
            +
                  c.syntax = 'floaty service <types examples>'
         | 
| 507 | 
            +
                  c.summary = 'Display information about floaty services and their configuration'
         | 
| 508 | 
            +
                  c.description = 'Display information about floaty services to aid in setting up a configuration file.'
         | 
| 509 | 
            +
                  c.example 'Print a list of the valid service types', 'floaty service types'
         | 
| 510 | 
            +
                  c.example 'Print a sample config file with multiple services', 'floaty service examples'
         | 
| 511 | 
            +
                  c.example 'list vms from the service named "nspooler-prod"', 'floaty list --service nspooler-prod'
         | 
| 512 | 
            +
                  c.action do |args, options|
         | 
| 513 | 
            +
                    action = args.first
         | 
| 514 | 
            +
             | 
| 515 | 
            +
                    example_config = Utils.strip_heredoc(<<-CONFIG)
         | 
| 516 | 
            +
                      # Sample ~/.vmfloaty.yml with just vmpooler
         | 
| 517 | 
            +
                      user: 'jdoe'
         | 
| 518 | 
            +
                      url: 'https://vmpooler.example.net'
         | 
| 519 | 
            +
                      token: '456def789'
         | 
| 520 | 
            +
             | 
| 521 | 
            +
                      # Sample ~/.vmfloaty.yml with multiple services
         | 
| 522 | 
            +
                      # Note: when the --service is not specified on the command line,
         | 
| 523 | 
            +
                      # the first service listed here is selected automatically
         | 
| 524 | 
            +
                      user: 'jdoe'
         | 
| 525 | 
            +
                      services:
         | 
| 526 | 
            +
                        abs-prod:
         | 
| 527 | 
            +
                          type: 'abs'
         | 
| 528 | 
            +
                          url: 'https://abs.example.net/api/v2'
         | 
| 529 | 
            +
                          token: '123abc456'
         | 
| 530 | 
            +
                          vmpooler_fallback: 'vmpooler-prod'
         | 
| 531 | 
            +
                        nspooler-prod:
         | 
| 532 | 
            +
                          type: 'nspooler'
         | 
| 533 | 
            +
                          url: 'https://nspooler.example.net'
         | 
| 534 | 
            +
                          token:  '789ghi012'
         | 
| 535 | 
            +
                        vmpooler-dev:
         | 
| 536 | 
            +
                          type: 'vmpooler'
         | 
| 537 | 
            +
                          url: 'https://vmpooler-dev.example.net'
         | 
| 538 | 
            +
                          token: '987dsa654'
         | 
| 539 | 
            +
                        vmpooler-prod:
         | 
| 540 | 
            +
                          type: 'vmpooler'
         | 
| 541 | 
            +
                          url: 'https://vmpooler.example.net'
         | 
| 542 | 
            +
                          token: '456def789'
         | 
| 543 | 
            +
             | 
| 544 | 
            +
                    CONFIG
         | 
| 545 | 
            +
             | 
| 546 | 
            +
                    types_output = Utils.strip_heredoc(<<-TYPES)
         | 
| 547 | 
            +
                      The values on the left below can be used in ~/.vmfloaty.yml as the value of type:
         | 
| 548 | 
            +
             | 
| 549 | 
            +
                      abs:       Puppet's Always Be Scheduling
         | 
| 550 | 
            +
                      nspooler:  Puppet's Non-standard Pooler, aka NSPooler
         | 
| 551 | 
            +
                      vmpooler:  Puppet's VMPooler
         | 
| 552 | 
            +
                    TYPES
         | 
| 553 | 
            +
             | 
| 554 | 
            +
                    case action
         | 
| 555 | 
            +
                    when 'examples'
         | 
| 556 | 
            +
                      FloatyLogger.info example_config
         | 
| 557 | 
            +
                    when 'types'
         | 
| 558 | 
            +
                      FloatyLogger.info types_output
         | 
| 559 | 
            +
                    when nil
         | 
| 560 | 
            +
                      FloatyLogger.error 'No action provided'
         | 
| 561 | 
            +
                      exit 1
         | 
| 562 | 
            +
                    else
         | 
| 563 | 
            +
                      FloatyLogger.error "Unknown action: #{action}"
         | 
| 564 | 
            +
                      exit 1
         | 
| 565 | 
            +
                    end
         | 
| 566 | 
            +
                  end
         | 
| 567 | 
            +
                end
         | 
| 568 | 
            +
             | 
| 484 569 | 
             
                run!
         | 
| 485 570 | 
             
              end
         | 
| 486 571 | 
             
            end
         | 
    
        data/lib/vmfloaty/abs.rb
    CHANGED
    
    | @@ -2,6 +2,7 @@ | |
| 2 2 |  | 
| 3 3 | 
             
            require 'vmfloaty/errors'
         | 
| 4 4 | 
             
            require 'vmfloaty/http'
         | 
| 5 | 
            +
            require 'vmfloaty/utils'
         | 
| 5 6 | 
             
            require 'faraday'
         | 
| 6 7 | 
             
            require 'json'
         | 
| 7 8 |  | 
| @@ -36,39 +37,61 @@ class ABS | |
| 36 37 | 
             
              #       }
         | 
| 37 38 | 
             
              #     }
         | 
| 38 39 | 
             
              #
         | 
| 39 | 
            -
             | 
| 40 40 | 
             
              @active_hostnames = {}
         | 
| 41 41 |  | 
| 42 | 
            -
              def self. | 
| 43 | 
            -
                 | 
| 42 | 
            +
              def self.list_active_job_ids(verbose, url, user)
         | 
| 43 | 
            +
                all_job_ids = []
         | 
| 44 44 | 
             
                @active_hostnames = {}
         | 
| 45 | 
            +
                get_active_requests(verbose, url, user).each do |req_hash|
         | 
| 46 | 
            +
                  @active_hostnames[req_hash['request']['job']['id']] = req_hash # full hash saved for later retrieval
         | 
| 47 | 
            +
                  all_job_ids.push(req_hash['request']['job']['id'])
         | 
| 48 | 
            +
                end
         | 
| 49 | 
            +
             | 
| 50 | 
            +
                all_job_ids
         | 
| 51 | 
            +
              end
         | 
| 45 52 |  | 
| 53 | 
            +
              def self.list_active(verbose, url, _token, user)
         | 
| 54 | 
            +
                hosts = []
         | 
| 46 55 | 
             
                get_active_requests(verbose, url, user).each do |req_hash|
         | 
| 47 | 
            -
                   | 
| 48 | 
            -
             | 
| 56 | 
            +
                  if req_hash.key?('allocated_resources')
         | 
| 57 | 
            +
                    req_hash['allocated_resources'].each do |onehost|
         | 
| 58 | 
            +
                      hosts.push(onehost['hostname'])
         | 
| 59 | 
            +
                    end
         | 
| 60 | 
            +
                  end
         | 
| 49 61 | 
             
                end
         | 
| 50 62 |  | 
| 51 | 
            -
                 | 
| 63 | 
            +
                hosts
         | 
| 52 64 | 
             
              end
         | 
| 53 65 |  | 
| 54 66 | 
             
              def self.get_active_requests(verbose, url, user)
         | 
| 55 67 | 
             
                conn = Http.get_conn(verbose, url)
         | 
| 56 68 | 
             
                res = conn.get 'status/queue'
         | 
| 57 | 
            -
                 | 
| 69 | 
            +
                if valid_json?(res.body)
         | 
| 70 | 
            +
                  requests = JSON.parse(res.body)
         | 
| 71 | 
            +
                else
         | 
| 72 | 
            +
                  FloatyLogger.warn "Warning: couldn't parse body returned from abs/status/queue"
         | 
| 73 | 
            +
                end
         | 
| 58 74 |  | 
| 59 75 | 
             
                ret_val = []
         | 
| 60 76 |  | 
| 61 77 | 
             
                requests.each do |req|
         | 
| 62 78 | 
             
                  next if req == 'null'
         | 
| 63 79 |  | 
| 64 | 
            -
                   | 
| 80 | 
            +
                  if valid_json?(req) # legacy ABS had another JSON string always-be-scheduling/pull/306
         | 
| 81 | 
            +
                    req_hash = JSON.parse(req)
         | 
| 82 | 
            +
                  elsif req.is_a?(Hash)
         | 
| 83 | 
            +
                    req_hash = req
         | 
| 84 | 
            +
                  else
         | 
| 85 | 
            +
                    FloatyLogger.warn "Warning: couldn't parse request returned from abs/status/queue"
         | 
| 86 | 
            +
                    next
         | 
| 87 | 
            +
                  end
         | 
| 65 88 |  | 
| 66 89 | 
             
                  begin
         | 
| 67 90 | 
             
                    next unless user == req_hash['request']['job']['user']
         | 
| 68 91 |  | 
| 69 92 | 
             
                    ret_val.push(req_hash)
         | 
| 70 93 | 
             
                  rescue NoMethodError
         | 
| 71 | 
            -
                    FloatyLogger.warn "Warning: couldn't parse  | 
| 94 | 
            +
                    FloatyLogger.warn "Warning: couldn't parse user returned from abs/status/queue: "
         | 
| 72 95 | 
             
                  end
         | 
| 73 96 | 
             
                end
         | 
| 74 97 |  | 
| @@ -145,30 +168,59 @@ class ABS | |
| 145 168 | 
             
                os_list = []
         | 
| 146 169 |  | 
| 147 170 | 
             
                res = conn.get 'status/platforms/vmpooler'
         | 
| 148 | 
            -
             | 
| 149 | 
            -
             | 
| 150 | 
            -
             | 
| 151 | 
            -
             | 
| 171 | 
            +
                if valid_json?(res.body)
         | 
| 172 | 
            +
                  res_body = JSON.parse(res.body)
         | 
| 173 | 
            +
                  if res_body.key?('vmpooler_platforms')
         | 
| 174 | 
            +
                    os_list << '*** VMPOOLER Pools ***'
         | 
| 175 | 
            +
                    if res_body['vmpooler_platforms'].is_a?(String)
         | 
| 176 | 
            +
                      os_list += JSON.parse(res_body['vmpooler_platforms']) # legacy ABS had another JSON string always-be-scheduling/pull/306
         | 
| 177 | 
            +
                    else
         | 
| 178 | 
            +
                      os_list += res_body['vmpooler_platforms']
         | 
| 179 | 
            +
                    end
         | 
| 180 | 
            +
                  end
         | 
| 181 | 
            +
                end
         | 
| 152 182 |  | 
| 153 183 | 
             
                res = conn.get 'status/platforms/ondemand_vmpooler'
         | 
| 154 | 
            -
                 | 
| 155 | 
            -
             | 
| 156 | 
            -
                   | 
| 157 | 
            -
             | 
| 158 | 
            -
             | 
| 184 | 
            +
                if valid_json?(res.body)
         | 
| 185 | 
            +
                  res_body = JSON.parse(res.body)
         | 
| 186 | 
            +
                  if res_body.key?('ondemand_vmpooler_platforms') && res_body['ondemand_vmpooler_platforms'] != '[]'
         | 
| 187 | 
            +
                    os_list << ''
         | 
| 188 | 
            +
                    os_list << '*** VMPOOLER ONDEMAND Pools ***'
         | 
| 189 | 
            +
                    if res_body['ondemand_vmpooler_platforms'].is_a?(String)
         | 
| 190 | 
            +
                      os_list += JSON.parse(res_body['ondemand_vmpooler_platforms']) # legacy ABS had another JSON string always-be-scheduling/pull/306
         | 
| 191 | 
            +
                    else
         | 
| 192 | 
            +
                      os_list += res_body['ondemand_vmpooler_platforms']
         | 
| 193 | 
            +
                    end
         | 
| 194 | 
            +
                  end
         | 
| 159 195 | 
             
                end
         | 
| 160 196 |  | 
| 161 197 | 
             
                res = conn.get 'status/platforms/nspooler'
         | 
| 162 | 
            -
                 | 
| 163 | 
            -
             | 
| 164 | 
            -
             | 
| 165 | 
            -
             | 
| 198 | 
            +
                if valid_json?(res.body)
         | 
| 199 | 
            +
                  res_body = JSON.parse(res.body)
         | 
| 200 | 
            +
                  if res_body.key?('nspooler_platforms')
         | 
| 201 | 
            +
                    os_list << ''
         | 
| 202 | 
            +
                    os_list << '*** NSPOOLER Pools ***'
         | 
| 203 | 
            +
                    if res_body['nspooler_platforms'].is_a?(String)
         | 
| 204 | 
            +
                      os_list += JSON.parse(res_body['nspooler_platforms']) # legacy ABS had another JSON string always-be-scheduling/pull/306
         | 
| 205 | 
            +
                    else
         | 
| 206 | 
            +
                      os_list += res_body['nspooler_platforms']
         | 
| 207 | 
            +
                    end
         | 
| 208 | 
            +
                  end
         | 
| 209 | 
            +
                end
         | 
| 166 210 |  | 
| 167 211 | 
             
                res = conn.get 'status/platforms/aws'
         | 
| 168 | 
            -
                 | 
| 169 | 
            -
             | 
| 170 | 
            -
             | 
| 171 | 
            -
             | 
| 212 | 
            +
                if valid_json?(res.body)
         | 
| 213 | 
            +
                  res_body = JSON.parse(res.body)
         | 
| 214 | 
            +
                  if res_body.key?('aws_platforms')
         | 
| 215 | 
            +
                    os_list << ''
         | 
| 216 | 
            +
                    os_list << '*** AWS Pools ***'
         | 
| 217 | 
            +
                    if res_body['aws_platforms'].is_a?(String)
         | 
| 218 | 
            +
                      os_list += JSON.parse(res_body['aws_platforms']) # legacy ABS had another JSON string always-be-scheduling/pull/306
         | 
| 219 | 
            +
                    else
         | 
| 220 | 
            +
                      os_list += res_body['aws_platforms']
         | 
| 221 | 
            +
                    end
         | 
| 222 | 
            +
                  end
         | 
| 223 | 
            +
                end
         | 
| 172 224 |  | 
| 173 225 | 
             
                os_list.delete 'ok'
         | 
| 174 226 |  | 
| @@ -176,7 +228,7 @@ class ABS | |
| 176 228 | 
             
              end
         | 
| 177 229 |  | 
| 178 230 | 
             
              # Retrieve an OS from ABS.
         | 
| 179 | 
            -
              def self.retrieve(verbose, os_types, token, url, user,  | 
| 231 | 
            +
              def self.retrieve(verbose, os_types, token, url, user, config, _ondemand = nil)
         | 
| 180 232 | 
             
                #
         | 
| 181 233 | 
             
                # Contents of post must be like:
         | 
| 182 234 | 
             
                #
         | 
| @@ -197,7 +249,7 @@ class ABS | |
| 197 249 | 
             
                conn.headers['X-AUTH-TOKEN'] = token if token
         | 
| 198 250 |  | 
| 199 251 | 
             
                saved_job_id = DateTime.now.strftime('%Q')
         | 
| 200 | 
            -
             | 
| 252 | 
            +
                vmpooler_config = Utils.get_vmpooler_service_config(config['vmpooler_fallback'])
         | 
| 201 253 | 
             
                req_obj = {
         | 
| 202 254 | 
             
                  :resources => os_types,
         | 
| 203 255 | 
             
                  :job       => {
         | 
| @@ -206,17 +258,18 @@ class ABS | |
| 206 258 | 
             
                      :user => user,
         | 
| 207 259 | 
             
                    },
         | 
| 208 260 | 
             
                  },
         | 
| 261 | 
            +
                  :vm_token => vmpooler_config['token'] # request with this token, on behalf of this user
         | 
| 209 262 | 
             
                }
         | 
| 210 263 |  | 
| 211 | 
            -
                if  | 
| 212 | 
            -
                  req_obj[:priority] = if  | 
| 264 | 
            +
                if config['priority']
         | 
| 265 | 
            +
                  req_obj[:priority] = if config['priority'] == 'high'
         | 
| 213 266 | 
             
                                         1
         | 
| 214 | 
            -
                                       elsif  | 
| 267 | 
            +
                                       elsif config['priority'] == 'medium'
         | 
| 215 268 | 
             
                                         2
         | 
| 216 | 
            -
                                       elsif  | 
| 269 | 
            +
                                       elsif config['priority'] == 'low'
         | 
| 217 270 | 
             
                                         3
         | 
| 218 271 | 
             
                                       else
         | 
| 219 | 
            -
                                          | 
| 272 | 
            +
                                         config['priority'].to_i
         | 
| 220 273 | 
             
                                       end
         | 
| 221 274 | 
             
                end
         | 
| 222 275 |  | 
| @@ -233,7 +286,7 @@ class ABS | |
| 233 286 |  | 
| 234 287 | 
             
                (1..retries).each do |i|
         | 
| 235 288 | 
             
                  queue_place, res_body = check_queue(conn, saved_job_id, req_obj, verbose)
         | 
| 236 | 
            -
                  return translated(res_body) if res_body
         | 
| 289 | 
            +
                  return translated(res_body, saved_job_id) if res_body
         | 
| 237 290 |  | 
| 238 291 | 
             
                  sleep_seconds = 10 if i >= 10
         | 
| 239 292 | 
             
                  sleep_seconds = i if i < 10
         | 
| @@ -247,8 +300,8 @@ class ABS | |
| 247 300 | 
             
              #
         | 
| 248 301 | 
             
              # We should fix the ABS API to be more like the vmpooler or nspooler api, but for now
         | 
| 249 302 | 
             
              #
         | 
| 250 | 
            -
              def self.translated(res_body)
         | 
| 251 | 
            -
                vmpooler_formatted_body = {}
         | 
| 303 | 
            +
              def self.translated(res_body, job_id)
         | 
| 304 | 
            +
                vmpooler_formatted_body = {'job_id' => job_id}
         | 
| 252 305 |  | 
| 253 306 | 
             
                res_body.each do |host|
         | 
| 254 307 | 
             
                  if vmpooler_formatted_body[host['type']] && vmpooler_formatted_body[host['type']]['hostname'].class == Array
         | 
| @@ -264,12 +317,17 @@ class ABS | |
| 264 317 |  | 
| 265 318 | 
             
              def self.check_queue(conn, job_id, req_obj, verbose)
         | 
| 266 319 | 
             
                queue_info_res = conn.get "status/queue/info/#{job_id}"
         | 
| 267 | 
            -
                 | 
| 320 | 
            +
                if valid_json?(queue_info_res.body)
         | 
| 321 | 
            +
                  queue_info = JSON.parse(queue_info_res.body)
         | 
| 322 | 
            +
                else
         | 
| 323 | 
            +
                  FloatyLogger.warn "Could not parse the status/queue/info/#{job_id}"
         | 
| 324 | 
            +
                  return [nil, nil]
         | 
| 325 | 
            +
                end
         | 
| 268 326 |  | 
| 269 327 | 
             
                res = conn.post 'request', req_obj.to_json
         | 
| 270 328 | 
             
                validate_queue_status_response(res.status, res.body, "Check queue request", verbose)
         | 
| 271 329 |  | 
| 272 | 
            -
                unless res.body.empty?
         | 
| 330 | 
            +
                unless res.body.empty? || !valid_json?(res.body)
         | 
| 273 331 | 
             
                  res_body = JSON.parse(res.body)
         | 
| 274 332 | 
             
                  return queue_info['queue_place'], res_body
         | 
| 275 333 | 
             
                end
         | 
| @@ -277,7 +335,7 @@ class ABS | |
| 277 335 | 
             
              end
         | 
| 278 336 |  | 
| 279 337 | 
             
              def self.snapshot(_verbose, _url, _hostname, _token)
         | 
| 280 | 
            -
                 | 
| 338 | 
            +
                raise NoMethodError, "Can't snapshot with ABS, use '--service vmpooler' (even for vms checked out with ABS)"
         | 
| 281 339 | 
             
              end
         | 
| 282 340 |  | 
| 283 341 | 
             
              def self.status(verbose, url)
         | 
| @@ -289,20 +347,24 @@ class ABS | |
| 289 347 | 
             
              end
         | 
| 290 348 |  | 
| 291 349 | 
             
              def self.summary(verbose, url)
         | 
| 292 | 
            -
                 | 
| 293 | 
            -
             | 
| 294 | 
            -
                res = conn.get 'summary'
         | 
| 295 | 
            -
                JSON.parse(res.body)
         | 
| 350 | 
            +
                raise NoMethodError, 'summary is not defined for ABS'
         | 
| 296 351 | 
             
              end
         | 
| 297 352 |  | 
| 298 | 
            -
              def self.query(verbose, url,  | 
| 299 | 
            -
                return  | 
| 353 | 
            +
              def self.query(verbose, url, job_id)
         | 
| 354 | 
            +
                # return saved hostnames from the last time list_active was run
         | 
| 355 | 
            +
                # preventing having to query the API again.
         | 
| 356 | 
            +
                # This works as long as query is called after list_active
         | 
| 357 | 
            +
                return @active_hostnames if @active_hostnames && !@active_hostnames.empty?
         | 
| 300 358 |  | 
| 301 | 
            -
                 | 
| 359 | 
            +
                # If using the cli query job_id
         | 
| 302 360 | 
             
                conn = Http.get_conn(verbose, url)
         | 
| 303 | 
            -
             | 
| 304 | 
            -
                 | 
| 305 | 
            -
             | 
| 361 | 
            +
                queue_info_res = conn.get "status/queue/info/#{job_id}"
         | 
| 362 | 
            +
                if valid_json?(queue_info_res.body)
         | 
| 363 | 
            +
                  queue_info = JSON.parse(queue_info_res.body)
         | 
| 364 | 
            +
                else
         | 
| 365 | 
            +
                  FloatyLogger.warn "Could not parse the status/queue/info/#{job_id}"
         | 
| 366 | 
            +
                end
         | 
| 367 | 
            +
                queue_info
         | 
| 306 368 | 
             
              end
         | 
| 307 369 |  | 
| 308 370 | 
             
              def self.modify(_verbose, _url, _hostname, _token, _modify_hash)
         | 
| @@ -333,4 +395,11 @@ class ABS | |
| 333 395 | 
             
                  raise "HTTP #{status_code}: #{request_name} request to ABS failed!\n#{body}"
         | 
| 334 396 | 
             
                end
         | 
| 335 397 | 
             
              end
         | 
| 398 | 
            +
             | 
| 399 | 
            +
              def self.valid_json?(json)
         | 
| 400 | 
            +
                JSON.parse(json)
         | 
| 401 | 
            +
                return true
         | 
| 402 | 
            +
              rescue TypeError, JSON::ParserError => e
         | 
| 403 | 
            +
                return false
         | 
| 404 | 
            +
              end
         | 
| 336 405 | 
             
            end
         |