hetzner-k3s 0.5.7 → 0.5.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1d75621e1a64fc2a2cc874811aed89a81086943d4099d1f651ee7193f01d1add
-  data.tar.gz: f6b6845cc0701d771701a3ca4551722da0d8fd4e71eb63001974026a266a6a5d
+  metadata.gz: c0d855f62ab9e222986d220edcdde203f7eb363ab725c8aa4e1f1389f2b251e2
+  data.tar.gz: 19d6a1ff6769cbec2207539d615d6a873eaede07aec9b15a43e6ef9d79101731
 SHA512:
-  metadata.gz: 32925df0b7b2d4af9705d8cc563ed34d8dc181ec37c5c7063504a24f56cc383fdb6871d27daa8fb456979db1f6e2310567a122455a5a53894260ba4bb65b1c92
-  data.tar.gz: d8374b7a45ec03331877c8307297be4fade4237ae2c5fc07545631f112106ed74e068a4605bd8414e3a6cc903838c39cad903df4278ab65f49442705b0460e2a
+  metadata.gz: d8fdc127e71f790e530d3abf97ca30d22b28d75eff687e436d1351336595ae7ba912c0c947c8b1738ab9d50447e577c4be753db3c20a2bfbcca195fcc6a0d193
+  data.tar.gz: 5cb4203c6270f0e82b66049fefd7244aaeb42e1dd89e257a0e1ba4b126db04ed8adf27b715b10874248f6661b6fed3126e562913ec2342f086bbff4c717c329b

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    hetzner-k3s (0.5.6)
+    hetzner-k3s (0.5.7)
       bcrypt_pbkdf
       ed25519
       http
@@ -30,7 +30,7 @@ GEM
       http-cookie (~> 1.0)
       http-form_data (~> 2.2)
       http-parser (~> 1.2.0)
-    http-cookie (1.0.4)
+    http-cookie (1.0.5)
       domain_name (~> 0.5)
     http-form_data (2.3.0)
     http-parser (1.2.3)
@@ -39,7 +39,7 @@ GEM
     parallel (1.21.0)
     parser (3.1.0.0)
       ast (~> 2.4.1)
-    public_suffix (4.0.6)
+    public_suffix (4.0.7)
     rainbow (3.1.1)
     rake (12.3.3)
     regexp_parser (2.2.0)
@@ -74,7 +74,7 @@ GEM
     thor (1.2.1)
     unf (0.1.4)
       unf_ext
-    unf_ext (0.0.8)
+    unf_ext (0.0.8.2)
     unicode-display_width (2.1.0)
 
 PLATFORMS

data/README.md

@@ -16,6 +16,8 @@ Using this tool, creating a highly available k3s cluster with 3 masters for the
 
 See roadmap [here](https://github.com/vitobotta/hetzner-k3s/projects/1) for the features planned or in progress.
 
+Also see this [wiki page](https://github.com/vitobotta/hetzner-k3s/wiki/Tutorial:---Setting-up-a-cluster) for a tutorial on how to set up a cluster with the most common setup to get you started.
+
 ## Requirements
 
 All that is needed to use this tool is
@@ -39,7 +41,12 @@ This will install the `hetzner-k3s` executable in your PATH.
 Alternatively, if you don't want to set up a Ruby runtime but have Docker installed, you can use a container. Run the following from inside the directory where you have the config file for the cluster (described in the next section):
 
 ```bash
-docker run --rm -it -v ${PWD}:/cluster -v ${HOME}/.ssh:/tmp/.ssh vitobotta/hetzner-k3s:v0.5.7 create-cluster --config-file /cluster/test.yaml
+docker run --rm -it \
+  -v ${PWD}:/cluster \
+  -v ${HOME}/.ssh:/tmp/.ssh \
+  vitobotta/hetzner-k3s:v0.5.8 \
+  create-cluster \
+  --config-file /cluster/test.yaml
 ```
 
 Replace `test.yaml` with the name of your config file.
@@ -164,6 +171,8 @@ The `create-cluster` command can be run any number of times with the same config
 
 To add one or more nodes to a node pool, just change the instance count in the configuration file for that node pool and re-run the create command.
 
+**Important**: if you are increasing the size of a node pool created prior to v0.5.7, please see [this thread](https://github.com/vitobotta/hetzner-k3s/issues/80).
+
 ### Scaling down a node pool
 
 To make a node pool smaller:
@@ -199,16 +208,6 @@ Note that the API server will briefly be unavailable during the upgrade of the c
 
 To check the upgrade progress, run `watch kubectl get nodes -owide`. You will see the masters being upgraded one per time, followed by the worker nodes.
 
-## Upgrade the OS on nodes
-
-The easiest way to upgrade the OS on existing nodes is actually to replace them, as it happens with managed Kubernetes service. To do this:
-
-- drain one node
-- delete the node from Kubernetes
-- delete the node from the Hetzner console
-- re-run the script to recreate the deleted node with an updated OS
-- proceed with the next node
-
 ### What to do if the upgrade doesn't go smoothly
 
 If the upgrade gets stuck for some reason, or it doesn't upgrade all the nodes:
@@ -234,7 +233,8 @@ I have noticed that sometimes I need to re-run the upgrade command a couple of t
 You can also check the logs of the system upgrade controller's pod:
 
 ```bash
-kubectl -n system-upgrade logs -f $(kubectl -n system-upgrade get pod -l pod-template-hash -o jsonpath="{.items[0].metadata.name}")
+kubectl -n system-upgrade \
+  logs -f $(kubectl -n system-upgrade get pod -l pod-template-hash -o jsonpath="{.items[0].metadata.name}")
 ```
 
 A final note about upgrades is that if for some reason the upgrade gets stuck after upgrading the masters and before upgrading the worker nodes, just cleaning up the resources as described above might not be enough. In that case also try running the following to tell the upgrade job for the workers that the masters have already been upgraded, so the upgrade can continue for the workers:
@@ -243,6 +243,15 @@ A final note about upgrades is that if for some reason the upgrade gets stuck af
 kubectl label node <master1> <master2> <master2> plan.upgrade.cattle.io/k3s-server=upgraded
 ```
 
+## Upgrading the OS on nodes
+
+- consider adding a temporary node during the process if you don't have enough spare capacity in the cluster
+- drain one node
+- update etc
+- reboot
+- uncordon
+- proceed with the next node
+
 ## Deleting a cluster
 
 To delete a cluster, running
@@ -277,7 +286,7 @@ I set `load-balancer.hetzner.cloud/hostname` to a valid hostname that I configur
 
 The annotation `load-balancer.hetzner.cloud/use-private-ip: "true"` ensures that the communication between the load balancer and the nodes happens through the private network, so we don't have to open any ports on the nodes (other than the port 6443 for the Kubernetes API server).
 
-The other annotations should be self explanatory. You can find a list of the available annotations here.
+The other annotations should be self explanatory. You can find a list of the available annotations [here](https://pkg.go.dev/github.com/hetznercloud/hcloud-cloud-controller-manager/internal/annotation).
 
 ## Persistent volumes
 
@@ -293,6 +302,10 @@ I recommend that you create a separate Hetzner project for each cluster, because
 
 Please create a PR if you want to propose any changes, or open an issue if you are having trouble with the tool - I will do my best to help if I can.
 
+Contributors:
+
+- [TitanFighter](https://github.com/TitanFighter) for [this awesome tutorial](https://github.com/vitobotta/hetzner-k3s/wiki/Tutorial:---Setting-up-a-cluster)
+
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/bin/build.sh

@@ -4,9 +4,9 @@ set -e
 
 IMAGE="vitobotta/hetzner-k3s"
 
-docker build -t ${IMAGE}:v0.5.7 \
+docker build -t ${IMAGE}:v0.5.8 \
   --platform=linux/amd64 \
-  --cache-from ${IMAGE}:v0.5.6 \
+  --cache-from ${IMAGE}:v0.5.7 \
   --build-arg BUILDKIT_INLINE_CACHE=1 .
 
-docker push vitobotta/hetzner-k3s:v0.5.7
+docker push vitobotta/hetzner-k3s:v0.5.8

data/lib/hetzner/infra/server.rb

@@ -8,13 +8,19 @@ module Hetzner
     end
 
     def create(location:, instance_type:, instance_id:, firewall_id:, network_id:, ssh_key_id:, placement_group_id:, image:, additional_packages: [], additional_post_create_commands: [])
+      @location = location
+      @instance_type = instance_type
+      @instance_id = instance_id
+      @firewall_id = firewall_id
+      @network_id = network_id
+      @ssh_key_id = ssh_key_id
+      @placement_group_id = placement_group_id
+      @image = image
       @additional_packages = additional_packages
       @additional_post_create_commands = additional_post_create_commands
 
       puts
 
-      server_name = "#{cluster_name}-#{instance_type}-#{instance_id}"
-
       if (server = find_server(server_name))
         puts "Server #{server_name} already exists, skipping."
         puts
@@ -23,44 +29,16 @@ module Hetzner
 
       puts "Creating server #{server_name}..."
 
-      server_config = {
-        name: server_name,
-        location:,
-        image:,
-        firewalls: [
-          { firewall: firewall_id }
-        ],
-        networks: [
-          network_id
-        ],
-        server_type: instance_type,
-        ssh_keys: [
-          ssh_key_id
-        ],
-        user_data:,
-        labels: {
-          cluster: cluster_name,
-          role: (server_name =~ /master/ ? 'master' : 'worker')
-        },
-        placement_group: placement_group_id
-      }
-
-      response = hetzner_client.post('/servers', server_config)
-      response_body = response.body
-
-      server = JSON.parse(response_body)['server']
+      if (server = make_request)
+        puts "...server #{server_name} created."
+        puts
 
-      unless server
+        server
+      else
         puts "Error creating server #{server_name}. Response details below:"
         puts
         p response
-        return
       end
-
-      puts "...server #{server_name} created."
-      puts
-
-      server
     end
 
     def delete(server_name:)
@@ -75,7 +53,7 @@ module Hetzner
 
     private
 
-    attr_reader :hetzner_client, :cluster_name, :additional_packages, :additional_post_create_commands
+    attr_reader :hetzner_client, :cluster_name, :location, :instance_type, :instance_id, :firewall_id, :network_id, :ssh_key_id, :placement_group_id, :image, :additional_packages, :additional_post_create_commands
 
     def find_server(server_name)
       hetzner_client.get('/servers?sort=created:desc')['servers'].detect { |network| network['name'] == server_name }
@@ -113,5 +91,40 @@ module Hetzner
         #{post_create_commands}
       YAML
     end
+
+    def server_name
+      @server_name ||= "#{cluster_name}-#{instance_type}-#{instance_id}"
+    end
+
+    def server_config
+      @server_config ||= {
+        name: server_name,
+        location:,
+        image:,
+        firewalls: [
+          { firewall: firewall_id }
+        ],
+        networks: [
+          network_id
+        ],
+        server_type: instance_type,
+        ssh_keys: [
+          ssh_key_id
+        ],
+        user_data:,
+        labels: {
+          cluster: cluster_name,
+          role: (server_name =~ /master/ ? 'master' : 'worker')
+        },
+        placement_group: placement_group_id
+      }
+    end
+
+    def make_request
+      response = hetzner_client.post('/servers', server_config)
+      response_body = response.body
+
+      JSON.parse(response_body)['server']
+    end
   end
 end

data/lib/hetzner/k3s/cli.rb

@@ -8,6 +8,7 @@ require 'open-uri'
 require 'yaml'
 
 require_relative 'cluster'
+require_relative 'configuration'
 require_relative 'version'
 
 module Hetzner
@@ -17,13 +18,6 @@ module Hetzner
         true
       end
 
-      def initialize(*args)
-        @errors = []
-        @used_server_types = []
-
-        super
-      end
-
       desc 'version', 'Print the version'
       def version
         puts Hetzner::K3s::VERSION
@@ -32,15 +26,15 @@ module Hetzner
       desc 'create-cluster', 'Create a k3s cluster in Hetzner Cloud'
       option :config_file, required: true
       def create_cluster
-        validate_configuration :create
-        Cluster.new(hetzner_client:, hetzner_token:).create configuration:
+        configuration.validate action: :create
+        Cluster.new(configuration:).create
       end
 
       desc 'delete-cluster', 'Delete an existing k3s cluster in Hetzner Cloud'
       option :config_file, required: true
       def delete_cluster
-        validate_configuration :delete
-        Cluster.new(hetzner_client:, hetzner_token:).delete configuration:
+        configuration.validate action: :delete
+        Cluster.new(configuration:).delete
       end
 
       desc 'upgrade-cluster', 'Upgrade an existing k3s cluster in Hetzner Cloud to a new version'
@@ -48,399 +42,27 @@ module Hetzner
       option :new_k3s_version, required: true
       option :force, default: 'false'
       def upgrade_cluster
-        validate_configuration :upgrade
-
-        Cluster.new(hetzner_client:, hetzner_token:)
-               .upgrade(configuration:, new_k3s_version: options[:new_k3s_version], config_file: options[:config_file])
+        configuration.validate action: :upgrade
+        Cluster.new(configuration:).upgrade(new_k3s_version: options[:new_k3s_version], config_file: options[:config_file])
       end
 
       desc 'releases', 'List available k3s releases'
       def releases
-        available_releases.each do |release|
+        Hetzner::Configuration.available_releases.each do |release|
           puts release
         end
       end
 
       private
 
-      attr_reader :configuration, :hetzner_client, :k3s_version
-      attr_accessor :errors, :used_server_types
-
-      def validate_configuration(action)
-        validate_configuration_file
-        validate_token
-        validate_cluster_name
-        validate_kubeconfig_path
-
-        case action
-        when :create
-          validate_create
-        when :delete
-          validate_kubeconfig_path_must_exist
-        when :upgrade
-          validate_upgrade
-        end
-
-        errors.flatten!
-
-        return if errors.empty?
-
-        puts 'Some information in the configuration file requires your attention:'
-
-        errors.each do |error|
-          puts " - #{error}"
-        end
-
-        exit 1
-      end
-
-      def valid_token?
-        return @valid unless @valid.nil?
-
-        begin
-          token = hetzner_token
-          @hetzner_client = Hetzner::Client.new(token:)
-          response = hetzner_client.get('/locations')
-          error_code = response.dig('error', 'code')
-          @valid = error_code&.size != 0
-        rescue StandardError
-          @valid = false
-        end
-      end
-
-      def validate_token
-        errors << 'Invalid Hetzner Cloud token' unless valid_token?
-      end
-
-      def validate_cluster_name
-        errors << 'Cluster name is an invalid format (only lowercase letters, digits and dashes are allowed)' unless configuration['cluster_name'] =~ /\A[a-z\d-]+\z/
-
-        return if configuration['cluster_name'] =~ /\A[a-z]+.*\z/
-
-        errors << 'Ensure that the cluster name starts with a normal letter'
-      end
-
-      def validate_kubeconfig_path
-        path = File.expand_path(configuration['kubeconfig_path'])
-        errors << 'kubeconfig path cannot be a directory' and return if File.directory? path
-
-        directory = File.dirname(path)
-        errors << "Directory #{directory} doesn't exist" unless File.exist? directory
-      rescue StandardError
-        errors << 'Invalid path for the kubeconfig'
-      end
-
-      def validate_public_ssh_key
-        path = File.expand_path(configuration['public_ssh_key_path'])
-        errors << 'Invalid Public SSH key path' and return unless File.exist? path
-
-        key = File.read(path)
-        errors << 'Public SSH key is invalid' unless ::SSHKey.valid_ssh_public_key?(key)
-      rescue StandardError
-        errors << 'Invalid Public SSH key path'
-      end
-
-      def validate_private_ssh_key
-        private_ssh_key_path = configuration['private_ssh_key_path']
-
-        return unless private_ssh_key_path
-
-        path = File.expand_path(private_ssh_key_path)
-        errors << 'Invalid Private SSH key path' and return unless File.exist?(path)
-      rescue StandardError
-        errors << 'Invalid Private SSH key path'
-      end
-
-      def validate_kubeconfig_path_must_exist
-        path = File.expand_path configuration['kubeconfig_path']
-        errors << 'kubeconfig path is invalid' and return unless File.exist? path
-
-        errors << 'kubeconfig path cannot be a directory' if File.directory? path
-      rescue StandardError
-        errors << 'Invalid kubeconfig path'
-      end
-
-      def server_types
-        return [] unless valid_token?
-
-        @server_types ||= hetzner_client.get('/server_types')['server_types'].map { |server_type| server_type['name'] }
-      rescue StandardError
-        @errors << 'Cannot fetch server types with Hetzner API, please try again later'
-        false
-      end
-
-      def locations
-        return [] unless valid_token?
-
-        @locations ||= hetzner_client.get('/locations')['locations'].map { |location| location['name'] }
-      rescue StandardError
-        @errors << 'Cannot fetch locations with Hetzner API, please try again later'
-        []
-      end
-
-      def valid_location?(location)
-        return if locations.empty? && !valid_token?
-
-        locations.include? location
-      end
-
-      def validate_masters_location
-        return if valid_location?(configuration['location'])
-
-        errors << 'Invalid location for master nodes - valid locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland) or ash (Ashburn, Virginia, USA)'
-      end
-
-      def available_releases
-        @available_releases ||= begin
-          response = HTTP.get('https://api.github.com/repos/k3s-io/k3s/tags?per_page=999').body
-          JSON.parse(response).map { |hash| hash['name'] }
-        end
-      rescue StandardError
-        errors << 'Cannot fetch the releases with Hetzner API, please try again later'
-      end
-
-      def validate_k3s_version
-        k3s_version = configuration['k3s_version']
-        errors << 'Invalid k3s version' unless available_releases.include? k3s_version
-      end
-
-      def validate_new_k3s_version
-        new_k3s_version = options[:new_k3s_version]
-        errors << 'The new k3s version is invalid' unless available_releases.include? new_k3s_version
-      end
+      attr_reader :hetzner_token, :hetzner_client
 
-      def validate_masters
-        masters_pool = nil
-
-        begin
-          masters_pool = configuration['masters']
-        rescue StandardError
-          errors << 'Invalid masters configuration'
-          return
-        end
-
-        if masters_pool.nil?
-          errors << 'Invalid masters configuration'
-          return
+      def configuration
+        @configuration ||= begin
+          config = ::Hetzner::Configuration.new(options:)
+          @hetzner_token = config.hetzner_token
+          config
         end
-
-        validate_instance_group masters_pool, workers: false
-      end
-
-      def validate_worker_node_pools
-        worker_node_pools = configuration['worker_node_pools'] || []
-
-        unless worker_node_pools.size.positive? || schedule_workloads_on_masters?
-          errors << 'Invalid node pools configuration'
-          return
-        end
-
-        return if worker_node_pools.size.zero? && schedule_workloads_on_masters?
-
-        if !worker_node_pools.is_a? Array
-          errors << 'Invalid node pools configuration'
-        elsif worker_node_pools.size.zero?
-          errors << 'At least one node pool is required in order to schedule workloads' unless schedule_workloads_on_masters?
-        elsif worker_node_pools.map { |worker_node_pool| worker_node_pool['name'] }.uniq.size != worker_node_pools.size
-          errors << 'Each node pool must have an unique name'
-        elsif server_types
-          worker_node_pools.each do |worker_node_pool|
-            validate_instance_group worker_node_pool
-          end
-        end
-      end
-
-      def schedule_workloads_on_masters?
-        schedule_workloads_on_masters = configuration['schedule_workloads_on_masters']
-        schedule_workloads_on_masters ? !!schedule_workloads_on_masters : false
-      end
-
-      def validate_instance_group(instance_group, workers: true)
-        instance_group_errors = []
-
-        instance_group_type = workers ? "Worker mode pool '#{instance_group['name']}'" : 'Masters pool'
-
-        instance_group_errors << "#{instance_group_type} has an invalid name" unless !workers || instance_group['name'] =~ /\A([A-Za-z0-9\-_]+)\Z/
-
-        instance_group_errors << "#{instance_group_type} is in an invalid format" unless instance_group.is_a? Hash
-
-        instance_group_errors << "#{instance_group_type} has an invalid instance type" unless !valid_token? || server_types.include?(instance_group['instance_type'])
-
-        if workers
-          location = instance_group.fetch('location', configuration['location'])
-          instance_group_errors << "#{instance_group_type} has an invalid location - valid locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland) or ash (Ashburn, Virginia, USA)" unless valid_location?(location)
-
-          in_network_zone = configuration['location'] == 'ash' ? location == 'ash' : location != 'ash'
-          instance_group_errors << "#{instance_group_type} must be in the same network zone as the masters. If the masters are located in Ashburn, all the node pools must be located in Ashburn too, otherwise none of the node pools should be located in Ashburn." unless in_network_zone
-        end
-
-        if instance_group['instance_count'].is_a? Integer
-          if instance_group['instance_count'] < 1
-            instance_group_errors << "#{instance_group_type} must have at least one node"
-          elsif instance_group['instance_count'] > 10
-            instance_group_errors << "#{instance_group_type} cannot have more than 10 nodes due to a limitation with the Hetzner placement groups. You can add more node pools if you need more nodes."
-          elsif !workers
-            instance_group_errors << 'Masters count must equal to 1 for non-HA clusters or an odd number (recommended 3) for an HA cluster' unless instance_group['instance_count'].odd?
-          end
-        else
-          instance_group_errors << "#{instance_group_type} has an invalid instance count"
-        end
-
-        used_server_types << instance_group['instance_type']
-
-        errors << instance_group_errors
-      end
-
-      def validate_verify_host_key
-        return unless [true, false].include?(configuration.fetch('public_ssh_key_path', false))
-
-        errors << 'Please set the verify_host_key option to either true or false'
-      end
-
-      def hetzner_token
-        @token = ENV.fetch('HCLOUD_TOKEN', nil)
-        return @token unless @token.nil?
-
-        @token = configuration['hetzner_token']
-      end
-
-      def validate_ssh_allowed_networks
-        networks ||= configuration['ssh_allowed_networks']
-
-        if networks.nil? || networks.empty?
-          errors << 'At least one network/IP range must be specified for SSH access'
-          return
-        end
-
-        invalid_networks = networks.reject do |network|
-          IPAddr.new(network)
-        rescue StandardError
-          false
-        end
-
-        unless invalid_networks.empty?
-          invalid_networks.each do |network|
-            errors << "The network #{network} is an invalid range"
-          end
-        end
-
-        invalid_ranges = networks.reject do |network|
-          network.include? '/'
-        end
-
-        unless invalid_ranges.empty?
-          invalid_ranges.each do |_network|
-            errors << 'Please use the CIDR notation for the networks to avoid ambiguity'
-          end
-        end
-
-        return unless invalid_networks.empty?
-
-        current_ip = URI.open('http://whatismyip.akamai.com').read
-
-        current_ip_networks = networks.detect do |network|
-          IPAddr.new(network).include?(current_ip)
-        rescue StandardError
-          false
-        end
-
-        errors << "Your current IP #{current_ip} is not included into any of the networks you've specified, so we won't be able to SSH into the nodes" unless current_ip_networks
-      end
-
-      def validate_additional_packages
-        additional_packages = configuration['additional_packages']
-        errors << 'Invalid additional packages configuration - it should be an array' if additional_packages && !additional_packages.is_a?(Array)
-      end
-
-      def validate_post_create_commands
-        post_create_commands = configuration['post_create_commands']
-        errors << 'Invalid post create commands configuration - it should be an array' if post_create_commands && !post_create_commands.is_a?(Array)
-      end
-
-      def validate_create
-        validate_public_ssh_key
-        validate_private_ssh_key
-        validate_ssh_allowed_networks
-        validate_masters_location
-        validate_k3s_version
-        validate_masters
-        validate_worker_node_pools
-        validate_verify_host_key
-        validate_additional_packages
-        validate_post_create_commands
-        validate_kube_api_server_args
-        validate_kube_scheduler_args
-        validate_kube_controller_manager_args
-        validate_kube_cloud_controller_manager_args
-        validate_kubelet_args
-        validate_kube_proxy_args
-      end
-
-      def validate_upgrade
-        validate_kubeconfig_path_must_exist
-        validate_new_k3s_version
-      end
-
-      def validate_configuration_file
-        config_file_path = options[:config_file]
-
-        if File.exist?(config_file_path)
-          begin
-            @configuration = YAML.load_file(options[:config_file])
-            unless configuration.is_a? Hash
-              puts 'Configuration is invalid'
-              exit 1
-            end
-          rescue StandardError
-            puts 'Please ensure that the config file is a correct YAML manifest.'
-            exit 1
-          end
-        else
-          puts 'Please specify a correct path for the config file.'
-          exit 1
-        end
-      end
-
-      def validate_kube_api_server_args
-        kube_api_server_args = configuration['kube_api_server_args']
-        return unless kube_api_server_args
-
-        errors << 'kube_api_server_args must be an array of arguments' unless kube_api_server_args.is_a? Array
-      end
-
-      def validate_kube_scheduler_args
-        kube_scheduler_args = configuration['kube_scheduler_args']
-        return unless kube_scheduler_args
-
-        errors << 'kube_scheduler_args must be an array of arguments' unless kube_scheduler_args.is_a? Array
-      end
-
-      def validate_kube_controller_manager_args
-        kube_controller_manager_args = configuration['kube_controller_manager_args']
-        return unless kube_controller_manager_args
-
-        errors << 'kube_controller_manager_args must be an array of arguments' unless kube_controller_manager_args.is_a? Array
-      end
-
-      def validate_kube_cloud_controller_manager_args
-        kube_cloud_controller_manager_args = configuration['kube_cloud_controller_manager_args']
-        return unless kube_cloud_controller_manager_args
-
-        errors << 'kube_cloud_controller_manager_args must be an array of arguments' unless kube_cloud_controller_manager_args.is_a? Array
-      end
-
-      def validate_kubelet_args
-        kubelet_args = configuration['kubelet_args']
-        return unless kubelet_args
-
-        errors << 'kubelet_args must be an array of arguments' unless kubelet_args.is_a? Array
-      end
-
-      def validate_kube_proxy_args
-        kube_proxy_args = configuration['kube_proxy_args']
-        return unless kube_proxy_args
-
-        errors << 'kube_proxy_args must be an array of arguments' unless kube_proxy_args.is_a? Array
       end
     end
   end

data/lib/hetzner/k3s/cluster.rb

@@ -19,13 +19,11 @@ require_relative '../utils'
 class Cluster
   include Utils
 
-  def initialize(hetzner_client:, hetzner_token:)
-    @hetzner_client = hetzner_client
-    @hetzner_token = hetzner_token
+  def initialize(configuration:)
+    @configuration = configuration
   end
 
-  def create(configuration:)
-    @configuration = configuration
+  def create
     @cluster_name = configuration['cluster_name']
     @kubeconfig_path = File.expand_path(configuration['kubeconfig_path'])
     @public_ssh_key_path = File.expand_path(configuration['public_ssh_key_path'])
@@ -57,8 +55,7 @@ class Cluster
     deploy_system_upgrade_controller
   end
 
-  def delete(configuration:)
-    @configuration = configuration
+  def delete
     @cluster_name = configuration['cluster_name']
     @kubeconfig_path = File.expand_path(configuration['kubeconfig_path'])
     @public_ssh_key_path = File.expand_path(configuration['public_ssh_key_path'])
@@ -68,8 +65,7 @@ class Cluster
     delete_resources
   end
 
-  def upgrade(configuration:, new_k3s_version:, config_file:)
-    @configuration = configuration
+  def upgrade(new_k3s_version:, config_file:)
     @cluster_name = configuration['cluster_name']
     @kubeconfig_path = File.expand_path(configuration['kubeconfig_path'])
     @new_k3s_version = new_k3s_version
@@ -82,10 +78,10 @@ class Cluster
 
   attr_accessor :servers
 
-  attr_reader :hetzner_client, :cluster_name, :kubeconfig_path, :k3s_version,
+  attr_reader :configuration, :cluster_name, :kubeconfig_path, :k3s_version,
               :masters_config, :worker_node_pools,
               :masters_location, :public_ssh_key_path,
-              :hetzner_token, :new_k3s_version, :configuration,
+              :hetzner_token, :new_k3s_version,
               :config_file, :verify_host_key, :networks, :private_ssh_key_path,
               :enable_encryption, :kube_api_server_args, :kube_scheduler_args,
               :kube_controller_manager_args, :kube_cloud_controller_manager_args,
@@ -190,9 +186,10 @@ class Cluster
     puts 'Upgrade will now start. Run `watch kubectl get nodes` to see the nodes being upgraded. This should take a few minutes for a small cluster.'
     puts 'The API server may be briefly unavailable during the upgrade of the controlplane.'
 
-    configuration['k3s_version'] = new_k3s_version
+    updated_configuration = configuration.raw
+    updated_configuration['k3s_version'] = new_k3s_version
 
-    File.write(config_file, configuration.to_yaml)
+    File.write(config_file, updated_configuration.to_yaml)
   end
 
   def master_script(master)
@@ -214,10 +211,8 @@ class Cluster
         --cluster-cidr=10.244.0.0/16 \
         --etcd-expose-metrics=true \
         #{flannel_wireguard} \
-        --kube-controller-manager-arg="address=0.0.0.0" \
         --kube-controller-manager-arg="bind-address=0.0.0.0" \
         --kube-proxy-arg="metrics-bind-address=0.0.0.0" \
-        --kube-scheduler-arg="address=0.0.0.0" \
         --kube-scheduler-arg="bind-address=0.0.0.0" \
         #{taint} #{extra_args} \
         --kubelet-arg="cloud-provider=external" \
@@ -299,7 +294,7 @@ class Cluster
           name: 'hcloud'
         stringData:
           network: "#{cluster_name}"
-          token: "#{hetzner_token}"
+          token: "#{configuration.hetzner_token}"
       EOF
     BASH
 
@@ -339,7 +334,7 @@ class Cluster
           namespace: 'kube-system'
           name: 'hcloud-csi'
         stringData:
-          token: "#{hetzner_token}"
+          token: "#{configuration.hetzner_token}"
       EOF
     BASH
 
@@ -643,4 +638,8 @@ class Cluster
       " --kube-proxy-arg=\"#{arg}\" "
     end.join
   end
+
+  def hetzner_client
+    configuration.hetzner_client
+  end
 end

data/lib/hetzner/k3s/configuration.rb

@@ -0,0 +1,454 @@
+# frozen_string_literal: true
+
+module Hetzner
+  class Configuration
+    GITHUB_DELIM_LINKS = ','.freeze
+    GITHUB_LINK_REGEX = /<([^>]+)>; rel=\"([^\"]+)\"/
+
+    attr_reader :hetzner_client
+
+    def initialize(options:)
+      @options = options
+      @errors = []
+
+      validate_configuration_file
+    end
+
+    def validate(action:)
+      validate_token
+
+      if valid_token?
+        validate_cluster_name
+        validate_kubeconfig_path
+
+        case action
+        when :create
+          validate_create
+        when :delete
+          validate_kubeconfig_path_must_exist
+        when :upgrade
+          validate_upgrade
+        end
+      end
+
+      errors.flatten!
+
+      return if errors.empty?
+
+      puts 'Some information in the configuration file requires your attention:'
+
+      errors.each do |error|
+        puts " - #{error}"
+      end
+
+      exit 1
+    end
+
+    def self.available_releases
+      @available_releases ||= begin
+        releases = []
+
+        response, page_releases = fetch_releases('https://api.github.com/repos/k3s-io/k3s/tags?per_page=100')
+        releases = page_releases
+        link_header = response.headers['link']
+
+        while !link_header.nil?
+          next_page_url = extract_next_github_page_url(link_header)
+
+          break if next_page_url.nil?
+
+          response, page_releases = fetch_releases(next_page_url)
+
+          releases += page_releases
+
+          link_header = response.headers['link']
+        end
+
+        releases.sort
+      end
+    rescue StandardError
+      if defined?errors
+        errors << 'Cannot fetch the releases with Hetzner API, please try again later'
+      else
+        puts 'Cannot fetch the releases with Hetzner API, please try again later'
+      end
+    end
+
+    def hetzner_token
+      return @token unless @token.nil?
+
+      @token = ENV.fetch('HCLOUD_TOKEN', configuration['hetzner_token'])
+    end
+
+    def [](key)
+      configuration[key]
+    end
+
+    def fetch(key, default)
+      configuration.fetch(key, default)
+    end
+
+    def raw
+      configuration
+    end
+
+    private
+
+    attr_reader :configuration, :errors, :options
+
+    def self.fetch_releases(url)
+      response = HTTP.get(url)
+      [response, JSON.parse(response.body).map { |hash| hash['name'] }]
+    end
+
+    def self.extract_next_github_page_url(link_header)
+      link_header.split(GITHUB_DELIM_LINKS).each do |link|
+        GITHUB_LINK_REGEX.match(link.strip) do |match|
+          url_part, meta_part = match[1], match[2]
+          next if !url_part || !meta_part
+          return url_part if meta_part == "next"
+        end
+      end
+
+      nil
+    end
+
+    def self.assign_url_part(meta_part, url_part)
+      case meta_part
+      when "next"
+        url_part
+      end
+    end
+
+    def validate_create
+      validate_public_ssh_key
+      validate_private_ssh_key
+      validate_ssh_allowed_networks
+      validate_masters_location
+      validate_k3s_version
+      validate_masters
+      validate_worker_node_pools
+      validate_verify_host_key
+      validate_additional_packages
+      validate_post_create_commands
+      validate_kube_api_server_args
+      validate_kube_scheduler_args
+      validate_kube_controller_manager_args
+      validate_kube_cloud_controller_manager_args
+      validate_kubelet_args
+      validate_kube_proxy_args
+    end
+
+    def validate_upgrade
+      validate_kubeconfig_path_must_exist
+      validate_new_k3s_version
+    end
+
+    def validate_public_ssh_key
+      path = File.expand_path(configuration['public_ssh_key_path'])
+      errors << 'Invalid Public SSH key path' and return unless File.exist? path
+
+      key = File.read(path)
+      errors << 'Public SSH key is invalid' unless ::SSHKey.valid_ssh_public_key?(key)
+    rescue StandardError
+      errors << 'Invalid Public SSH key path'
+    end
+
+    def validate_private_ssh_key
+      private_ssh_key_path = configuration['private_ssh_key_path']
+
+      return unless private_ssh_key_path
+
+      path = File.expand_path(private_ssh_key_path)
+      errors << 'Invalid Private SSH key path' and return unless File.exist?(path)
+    rescue StandardError
+      errors << 'Invalid Private SSH key path'
+    end
+
+    def validate_ssh_allowed_networks
+      networks ||= configuration['ssh_allowed_networks']
+
+      if networks.nil? || networks.empty?
+        errors << 'At least one network/IP range must be specified for SSH access'
+        return
+      end
+
+      invalid_networks = networks.reject do |network|
+        IPAddr.new(network)
+      rescue StandardError
+        false
+      end
+
+      unless invalid_networks.empty?
+        invalid_networks.each do |network|
+          errors << "The network #{network} is an invalid range"
+        end
+      end
+
+      invalid_ranges = networks.reject do |network|
+        network.include? '/'
+      end
+
+      unless invalid_ranges.empty?
+        invalid_ranges.each do |_network|
+          errors << 'Please use the CIDR notation for the networks to avoid ambiguity'
+        end
+      end
+
+      return unless invalid_networks.empty?
+
+      current_ip = URI.open('http://whatismyip.akamai.com').read
+
+      current_ip_networks = networks.detect do |network|
+        IPAddr.new(network).include?(current_ip)
+      rescue StandardError
+        false
+      end
+
+      errors << "Your current IP #{current_ip} is not included into any of the networks you've specified, so we won't be able to SSH into the nodes" unless current_ip_networks
+    end
+
+    def validate_masters_location
+      return if valid_location?(configuration['location'])
+
+      errors << 'Invalid location for master nodes - valid locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland) or ash (Ashburn, Virginia, USA)'
+    end
+
+    def validate_k3s_version
+      k3s_version = configuration['k3s_version']
+      errors << 'Invalid k3s version' unless Hetzner::Configuration.available_releases.include? k3s_version
+    end
+
+    def validate_masters
+      masters_pool = nil
+
+      begin
+        masters_pool = configuration['masters']
+      rescue StandardError
+        errors << 'Invalid masters configuration'
+        return
+      end
+
+      if masters_pool.nil?
+        errors << 'Invalid masters configuration'
+        return
+      end
+
+      validate_instance_group masters_pool, workers: false
+    end
+
+    def validate_worker_node_pools
+      worker_node_pools = configuration['worker_node_pools'] || []
+
+      unless worker_node_pools.size.positive? || schedule_workloads_on_masters?
+        errors << 'Invalid node pools configuration'
+        return
+      end
+
+      return if worker_node_pools.size.zero? && schedule_workloads_on_masters?
+
+      if !worker_node_pools.is_a? Array
+        errors << 'Invalid node pools configuration'
+      elsif worker_node_pools.size.zero?
+        errors << 'At least one node pool is required in order to schedule workloads' unless schedule_workloads_on_masters?
+      elsif worker_node_pools.map { |worker_node_pool| worker_node_pool['name'] }.uniq.size != worker_node_pools.size
+        errors << 'Each node pool must have an unique name'
+      elsif server_types
+        worker_node_pools.each do |worker_node_pool|
+          validate_instance_group worker_node_pool
+        end
+      end
+    end
+
+    def validate_verify_host_key
+      return unless [true, false].include?(configuration.fetch('public_ssh_key_path', false))
+
+      errors << 'Please set the verify_host_key option to either true or false'
+    end
+
+    def validate_additional_packages
+      additional_packages = configuration['additional_packages']
+      errors << 'Invalid additional packages configuration - it should be an array' if additional_packages && !additional_packages.is_a?(Array)
+    end
+
+    def validate_post_create_commands
+      post_create_commands = configuration['post_create_commands']
+      errors << 'Invalid post create commands configuration - it should be an array' if post_create_commands && !post_create_commands.is_a?(Array)
+    end
+
+    def validate_kube_api_server_args
+      kube_api_server_args = configuration['kube_api_server_args']
+      return unless kube_api_server_args
+
+      errors << 'kube_api_server_args must be an array of arguments' unless kube_api_server_args.is_a? Array
+    end
+
+    def validate_kube_scheduler_args
+      kube_scheduler_args = configuration['kube_scheduler_args']
+      return unless kube_scheduler_args
+
+      errors << 'kube_scheduler_args must be an array of arguments' unless kube_scheduler_args.is_a? Array
+    end
+
+    def validate_kube_controller_manager_args
+      kube_controller_manager_args = configuration['kube_controller_manager_args']
+      return unless kube_controller_manager_args
+
+      errors << 'kube_controller_manager_args must be an array of arguments' unless kube_controller_manager_args.is_a? Array
+    end
+
+    def validate_kube_cloud_controller_manager_args
+      kube_cloud_controller_manager_args = configuration['kube_cloud_controller_manager_args']
+      return unless kube_cloud_controller_manager_args
+
+      errors << 'kube_cloud_controller_manager_args must be an array of arguments' unless kube_cloud_controller_manager_args.is_a? Array
+    end
+
+    def validate_kubelet_args
+      kubelet_args = configuration['kubelet_args']
+      return unless kubelet_args
+
+      errors << 'kubelet_args must be an array of arguments' unless kubelet_args.is_a? Array
+    end
+
+    def validate_kube_proxy_args
+      kube_proxy_args = configuration['kube_proxy_args']
+      return unless kube_proxy_args
+
+      errors << 'kube_proxy_args must be an array of arguments' unless kube_proxy_args.is_a? Array
+    end
+
+    def validate_configuration_file
+      config_file_path = options[:config_file]
+
+      if File.exist?(config_file_path)
+        begin
+          @configuration = YAML.load_file(options[:config_file])
+          unless configuration.is_a? Hash
+            puts 'Configuration is invalid'
+            exit 1
+          end
+        rescue StandardError
+          puts 'Please ensure that the config file is a correct YAML manifest.'
+          exit 1
+        end
+      else
+        puts 'Please specify a correct path for the config file.'
+        exit 1
+      end
+    end
+
+    def validate_token
+      errors << 'Invalid Hetzner Cloud token' unless valid_token?
+    end
+
+    def validate_kubeconfig_path
+      path = File.expand_path(configuration['kubeconfig_path'])
+      errors << 'kubeconfig path cannot be a directory' and return if File.directory? path
+
+      directory = File.dirname(path)
+      errors << "Directory #{directory} doesn't exist" unless File.exist? directory
+    rescue StandardError
+      errors << 'Invalid path for the kubeconfig'
+    end
+
+    def validate_kubeconfig_path_must_exist
+      path = File.expand_path configuration['kubeconfig_path']
+      errors << 'kubeconfig path is invalid' and return unless File.exist? path
+
+      errors << 'kubeconfig path cannot be a directory' if File.directory? path
+    rescue StandardError
+      errors << 'Invalid kubeconfig path'
+    end
+
+    def validate_cluster_name
+      errors << 'Cluster name is an invalid format (only lowercase letters, digits and dashes are allowed)' unless configuration['cluster_name'] =~ /\A[a-z\d-]+\z/
+
+      return if configuration['cluster_name'] =~ /\A[a-z]+.*([a-z]|\d)+\z/
+
+      errors << 'Ensure that the cluster name starts and ends with a normal letter'
+    end
+
+    def validate_new_k3s_version
+      new_k3s_version = options[:new_k3s_version]
+      errors << 'The new k3s version is invalid' unless Hetzner::Configuration.available_releases.include? new_k3s_version
+    end
+
+    def valid_token?
+      return @valid unless @valid.nil?
+
+      begin
+        token = hetzner_token
+        @hetzner_client = Hetzner::Client.new(token:)
+        response = hetzner_client.get('/locations')
+        error_code = response.dig('error', 'code')
+        @valid = error_code != 'unauthorized'
+      rescue StandardError
+        @valid = false
+      end
+    end
+
+    def validate_instance_group(instance_group, workers: true)
+      instance_group_errors = []
+
+      instance_group_type = workers ? "Worker mode pool '#{instance_group['name']}'" : 'Masters pool'
+
+      instance_group_errors << "#{instance_group_type} has an invalid name" unless !workers || instance_group['name'] =~ /\A([A-Za-z0-9\-_]+)\Z/
+
+      instance_group_errors << "#{instance_group_type} is in an invalid format" unless instance_group.is_a? Hash
+
+      instance_group_errors << "#{instance_group_type} has an invalid instance type" unless !valid_token? || server_types.include?(instance_group['instance_type'])
+
+      if workers
+        location = instance_group.fetch('location', configuration['location'])
+        instance_group_errors << "#{instance_group_type} has an invalid location - valid locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland) or ash (Ashburn, Virginia, USA)" unless valid_location?(location)
+
+        in_network_zone = configuration['location'] == 'ash' ? location == 'ash' : location != 'ash'
+        instance_group_errors << "#{instance_group_type} must be in the same network zone as the masters. If the masters are located in Ashburn, all the node pools must be located in Ashburn too, otherwise none of the node pools should be located in Ashburn." unless in_network_zone
+      end
+
+      if instance_group['instance_count'].is_a? Integer
+        if instance_group['instance_count'] < 1
+          instance_group_errors << "#{instance_group_type} must have at least one node"
+        elsif instance_group['instance_count'] > 10
+          instance_group_errors << "#{instance_group_type} cannot have more than 10 nodes due to a limitation with the Hetzner placement groups. You can add more node pools if you need more nodes."
+        elsif !workers
+          instance_group_errors << 'Masters count must equal to 1 for non-HA clusters or an odd number (recommended 3) for an HA cluster' unless instance_group['instance_count'].odd?
+        end
+      else
+        instance_group_errors << "#{instance_group_type} has an invalid instance count"
+      end
+
+      errors << instance_group_errors
+    end
+
+    def valid_location?(location)
+      return if locations.empty? && !valid_token?
+
+      locations.include? location
+    end
+
+    def locations
+      return [] unless valid_token?
+
+      @locations ||= hetzner_client.get('/locations')['locations'].map { |location| location['name'] }
+    rescue StandardError
+      @errors << 'Cannot fetch locations with Hetzner API, please try again later'
+      []
+    end
+
+    def schedule_workloads_on_masters?
+      schedule_workloads_on_masters = configuration['schedule_workloads_on_masters']
+      schedule_workloads_on_masters ? !!schedule_workloads_on_masters : false
+    end
+
+    def server_types
+      return [] unless valid_token?
+
+      @server_types ||= hetzner_client.get('/server_types')['server_types'].map { |server_type| server_type['name'] }
+    rescue StandardError
+      @errors << 'Cannot fetch server types with Hetzner API, please try again later'
+      false
+    end
+  end
+end

data/lib/hetzner/k3s/version.rb

@@ -2,6 +2,6 @@
 
 module Hetzner
   module K3s
-    VERSION = '0.5.7'
+    VERSION = '0.5.8'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: hetzner-k3s
 version: !ruby/object:Gem::Version
-  version: 0.5.7
+  version: 0.5.8
 platform: ruby
 authors:
 - Vito Botta
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2022-05-28 00:00:00.000000000 Z
+date: 2022-08-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bcrypt_pbkdf
@@ -161,6 +161,7 @@ files:
 - lib/hetzner/infra/ssh_key.rb
 - lib/hetzner/k3s/cli.rb
 - lib/hetzner/k3s/cluster.rb
+- lib/hetzner/k3s/configuration.rb
 - lib/hetzner/k3s/version.rb
 - lib/hetzner/utils.rb
 homepage: https://github.com/vitobotta/hetzner-k3s