hetzner-k3s 0.4.2 → 0.4.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 703675cd5cbfeb28c857486046b88a1202835316b4522ac3b964c42479ca91e9
-  data.tar.gz: d496229db7199e5d72bf19e5534ce83c7235ffe91eb93aec2151435d4c3a9c2c
+  metadata.gz: 60167b5a135a2932bff53b3d030b22940e3a7a943584ab9645d420fdfa2f59f4
+  data.tar.gz: 6b36f71696ffb4f94d1dfde94b5215c46b9f92fc482c0da148ea84c0cfa5a56f
 SHA512:
-  metadata.gz: 75f9af0a29e2e34e7e4fd703ee1a98f2a2e2b9cd8143783583f74ce26427b1bbfbf53930f258b1a5fc9faf9fe1e1a84e7f15f5c92122ecfd1b4ae6ae7913031e
-  data.tar.gz: b89c51f47038eca49083cf8936d7cc4927a8edbe9fe41582d834dd546a2be8d5b80ef03bf3bf0bce7605637e35445e153c1c72bf206ed58df9632c4a245d9f3b
+  metadata.gz: 73242160d8701cfb92332bf18aefcf7e2c15275493b2e4dedfc16d1e577367e91e338b2c57fc3d5731a5f68317751e1644b7d2be4d505371ee1543f04365685f
+  data.tar.gz: 2da1c1ef1bf32b3c403868d47bc94fbeee1ad8479aa1272db9141c749b695e70489c85c19f8e891fd332a91de6deda000d273e959f1d1d6a1e88edf9efe4310f

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    hetzner-k3s (0.4.0)
+    hetzner-k3s (0.4.5)
       bcrypt_pbkdf
       ed25519
       http
@@ -20,12 +20,12 @@ GEM
     diff-lcs (1.4.4)
     domain_name (0.5.20190701)
       unf (>= 0.0.5, < 1.0.0)
-    dry-configurable (0.12.1)
+    dry-configurable (0.13.0)
       concurrent-ruby (~> 1.0)
-      dry-core (~> 0.5, >= 0.5.0)
-    dry-container (0.8.0)
+      dry-core (~> 0.6)
+    dry-container (0.9.0)
       concurrent-ruby (~> 1.0)
-      dry-configurable (~> 0.1, >= 0.1.3)
+      dry-configurable (~> 0.13, >= 0.13.0)
     dry-core (0.7.1)
       concurrent-ruby (~> 1.0)
     dry-equalizer (0.3.0)
@@ -99,7 +99,7 @@ GEM
     to_regexp (0.2.1)
     unf (0.1.4)
       unf_ext
-    unf_ext (0.0.7.7)
+    unf_ext (0.0.8)
     yajl-ruby (1.4.1)
     yaml-safe_load_stream (0.1.1)
 

data/README.md

@@ -2,7 +2,7 @@
 
 This is a CLI tool - based on a Ruby gem - to quickly create and manage Kubernetes clusters in [Hetzner Cloud](https://www.hetzner.com/cloud) using the lightweight Kubernetes distribution [k3s](https://k3s.io/) from [Rancher](https://rancher.com/).
 
-Hetzner Cloud is an awesome cloud provider which offers a truly great service with the best performance/cost ratio in the market. I highly recommend them if European locations (Germany and Finland) are OK for your projects (the Nuremberg data center has decent latency for US users as well). With Hetzner's Cloud Controller Manager and CSI driver you can provision load balancers and persistent volumes very easily.
+Hetzner Cloud is an awesome cloud provider which offers a truly great service with the best performance/cost ratio in the market. With Hetzner's Cloud Controller Manager and CSI driver you can provision load balancers and persistent volumes very easily.
 
 k3s is my favorite Kubernetes distribution now because it uses much less memory and CPU, leaving more resources to workloads. It is also super quick to deploy because it's a single binary.
 
@@ -38,7 +38,7 @@ This will install the `hetzner-k3s` executable in your PATH.
 Alternatively, if you don't want to set up a Ruby runtime but have Docker installed, you can use a container. Run the following from inside the directory where you have the config file for the cluster (described in the next section):
 
 ```bash
-docker run --rm -it -v ${PWD}:/cluster -v ${HOME}/.ssh:/tmp/.ssh vitobotta/hetzner-k3s:v0.4.1 create-cluster --config-file /cluster/test.yaml
+docker run --rm -it -v ${PWD}:/cluster -v ${HOME}/.ssh:/tmp/.ssh vitobotta/hetzner-k3s:v0.4.6 create-cluster --config-file /cluster/test.yaml
 ```
 
 Replace `test.yaml` with the name of your config file.
@@ -78,7 +78,7 @@ If you are using Docker, then set `kubeconfig_path` to `/cluster/kubeconfig` so
 
 If you don't want to specify the Hetzner token in the config file (for example if you want to use the tool with CI), then you can use the `HCLOUD_TOKEN` environment variable instead, which has predecence.
 
-**Important**: The tool assignes the label `cluster` to each server it creates, with the clsuter name you specify in the config file, as the value. So please ensure you don't create unrelated servers in the same project having
+**Important**: The tool assignes the label `cluster` to each server it creates, with the cluster name you specify in the config file, as the value. So please ensure you don't create unrelated servers in the same project having
 the label `cluster=<cluster name>`, because otherwise they will be deleted if you delete the cluster. I recommend you create a separate Hetzner project for each cluster, see note at the end of this README for more details.
 
 
@@ -86,7 +86,8 @@ If you set `masters.instance_count` to 1 then the tool will create a non highly
 
 You can specify any number of worker node pools for example to have mixed nodes with different specs for different workloads.
 
-At the moment Hetzner Cloud has three locations: two in Germany (`nbg1`, Nuremberg and `fsn1`, Falkensteing) and one in Finland (`hel1`, Helsinki).
+At the moment Hetzner Cloud has four locations: two in Germany (`nbg1`, Nuremberg and `fsn1`, Falkensteing), one in Finland (`hel1`, Helsinki) and one in the USA (`ash`, Ashburn, Virginia). Please note that the Ashburn, Virginia location has just
+been announced and it's limited to AMD instances for now.
 
 For the available instance types and their specs, either check from inside a project when adding a server manually or run the following with your Hetzner token:
 
@@ -241,6 +242,19 @@ I recommend that you create a separate Hetzner project for each cluster, because
 
 ## changelog
 
+- 0.4.5
+  - Added a check to abort gracefully when for some reason one or more servers are not created, for example due to temporary problems with the Hetzner API.
+
+- 0.4.5
+  - Fix network creation (bug introduced in the previous version)
+
+- 0.4.4
+  - Add support for the new Ashburn, Virginia (USA) location
+  - Automatically use a placement group so that the instances are all created on different physical hosts for high availability
+
+- 0.4.3
+  - Fix an issue with SSH key creation
+
 - 0.4.2
   - Update Hetzner CSI driver to v1.6.0
   - Update System Upgrade Controller to v0.8.0

data/bin/build.sh

@@ -6,9 +6,9 @@ set -e
 
 IMAGE="vitobotta/hetzner-k3s"
 
-docker build -t ${IMAGE}:v0.4.2 \
+docker build -t ${IMAGE}:v0.4.6 \
   --platform=linux/amd64 \
-  --cache-from ${IMAGE}:v0.4.1 \
+  --cache-from ${IMAGE}:v0.4.5 \
   --build-arg BUILDKIT_INLINE_CACHE=1 .
 
-docker push vitobotta/hetzner-k3s:v0.4.2
+docker push vitobotta/hetzner-k3s:v0.4.6

data/lib/hetzner/infra/network.rb

@@ -5,7 +5,8 @@ module Hetzner
       @cluster_name = cluster_name
     end
 
-    def create
+    def create(location:)
+      @location = location
       puts
 
       if network = find_network
@@ -38,7 +39,7 @@ module Hetzner
 
     private
 
-      attr_reader :hetzner_client, :cluster_name
+      attr_reader :hetzner_client, :cluster_name, :location
 
       def network_config
         {
@@ -47,7 +48,7 @@ module Hetzner
           subnets: [
             {
               ip_range: "10.0.0.0/16",
-              network_zone: "eu-central",
+              network_zone: (location == "ash" ? "us-east" : "eu-central"),
               type: "cloud"
             }
           ]

data/lib/hetzner/infra/placement_group.rb

@@ -0,0 +1,55 @@
+module Hetzner
+  class PlacementGroup
+    def initialize(hetzner_client:, cluster_name:)
+      @hetzner_client = hetzner_client
+      @cluster_name = cluster_name
+    end
+
+    def create
+      puts
+
+      if (placement_group = find_placement_group)
+        puts "Placement group already exists, skipping."
+        puts
+        return placement_group["id"]
+      end
+
+      puts "Creating placement group..."
+
+      response = hetzner_client.post("/placement_groups", placement_group_config).body
+
+      puts "...placement group created."
+      puts
+
+      JSON.parse(response)["placement_group"]["id"]
+    end
+
+    def delete
+      if (placement_group = find_placement_group)
+        puts "Deleting placement group..."
+        hetzner_client.delete("/placement_groups", placement_group["id"])
+        puts "...placement group deleted."
+      else
+        puts "Placement group no longer exists, skipping."
+      end
+
+      puts
+    end
+
+    private
+
+      attr_reader :hetzner_client, :cluster_name
+
+      def placement_group_config
+        {
+          name: cluster_name,
+          type: "spread"
+        }
+      end
+
+      def find_placement_group
+        hetzner_client.get("/placement_groups")["placement_groups"].detect{ |placement_group| placement_group["name"] == cluster_name }
+      end
+
+  end
+end

data/lib/hetzner/infra/server.rb

@@ -5,7 +5,7 @@ module Hetzner
       @cluster_name = cluster_name
     end
 
-    def create(location:, instance_type:, instance_id:, firewall_id:, network_id:, ssh_key_id:)
+    def create(location:, instance_type:, instance_id:, firewall_id:, network_id:, ssh_key_id:, placement_group_id:)
       puts
 
       server_name = "#{cluster_name}-#{instance_type}-#{instance_id}"
@@ -36,15 +36,26 @@ module Hetzner
         labels: {
           cluster: cluster_name,
           role: (server_name =~ /master/ ? "master" : "worker")
-        }
+        },
+        placement_group: placement_group_id
       }
 
-      response = hetzner_client.post("/servers", server_config).body
+      response = hetzner_client.post("/servers", server_config)
+      response_body = response.body
+
+      server = JSON.parse(response_body)["server"]
+
+      unless server
+        puts "Error creating server #{server_name}. Response details below:"
+        puts
+        p response
+        return
+      end
 
       puts "...server #{server_name} created."
       puts
 
-      JSON.parse(response)["server"]
+      server
     end
 
     def delete(server_name:)

data/lib/hetzner/infra/ssh_key.rb

@@ -55,7 +55,7 @@ module Hetzner
       def ssh_key_config
         {
           name: cluster_name,
-          public_ssh_key: public_ssh_key
+          public_key: public_ssh_key
         }
       end
 

data/lib/hetzner/k3s/cli.rb

@@ -193,7 +193,7 @@ module Hetzner
 
         def validate_location
           return if locations.empty? && !valid_token?
-          errors << "Invalid location - available locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland)" unless locations.include? configuration.dig("location")
+          errors << "Invalid location - available locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland) or ash (Ashburn, Virginia, USA)" unless locations.include? configuration.dig("location")
         end
 
         def find_available_releases

data/lib/hetzner/k3s/cluster.rb

@@ -11,6 +11,7 @@ require_relative "../infra/network"
 require_relative "../infra/ssh_key"
 require_relative "../infra/server"
 require_relative "../infra/load_balancer"
+require_relative "../infra/placement_group"
 
 require_relative "../k3s/client_patch"
 
@@ -89,6 +90,11 @@ class Cluster
       master_instance_type = masters_config["instance_type"]
       masters_count = masters_config["instance_count"]
 
+      placement_group_id = Hetzner::PlacementGroup.new(
+        hetzner_client: hetzner_client,
+        cluster_name: cluster_name
+      ).create
+
       firewall_id = Hetzner::Firewall.new(
         hetzner_client: hetzner_client,
         cluster_name: cluster_name
@@ -97,7 +103,7 @@ class Cluster
       network_id = Hetzner::Network.new(
         hetzner_client: hetzner_client,
         cluster_name: cluster_name
-      ).create
+      ).create(location: location)
 
       ssh_key_id = Hetzner::SSHKey.new(
         hetzner_client: hetzner_client,
@@ -113,7 +119,8 @@ class Cluster
           instance_id: "master#{i+1}",
           firewall_id: firewall_id,
           network_id: network_id,
-          ssh_key_id: ssh_key_id
+          ssh_key_id: ssh_key_id,
+          placement_group_id: placement_group_id
         }
       end
 
@@ -136,7 +143,8 @@ class Cluster
             instance_id: "pool-#{worker_node_pool_name}-worker#{i+1}",
             firewall_id: firewall_id,
             network_id: network_id,
-            ssh_key_id: ssh_key_id
+            ssh_key_id: ssh_key_id,
+            placement_group_id: placement_group_id
           }
         end
       end
@@ -149,6 +157,11 @@ class Cluster
 
       threads.each(&:join) unless threads.empty?
 
+      if server_configs.size != servers.size
+        puts "Something went wrong while creating some servers, please try again."
+        exit 1
+      end
+
       puts
       threads = servers.map do |server|
         Thread.new { wait_for_ssh server }
@@ -158,6 +171,11 @@ class Cluster
     end
 
     def delete_resources
+      Hetzner::PlacementGroup.new(
+        hetzner_client: hetzner_client,
+        cluster_name: cluster_name
+      ).delete
+
       Hetzner::LoadBalancer.new(
         hetzner_client: hetzner_client,
         cluster_name: cluster_name
@@ -217,28 +235,28 @@ class Cluster
       taint = schedule_workloads_on_masters? ? " " : " --node-taint CriticalAddonsOnly=true:NoExecute "
 
       <<~EOF
-      curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="#{k3s_version}" K3S_TOKEN="#{k3s_token}" INSTALL_K3S_EXEC="server \
-        --disable-cloud-controller \
-        --disable servicelb \
-        --disable traefik \
-        --disable local-storage \
-        --disable metrics-server \
-        --write-kubeconfig-mode=644 \
-        --node-name="$(hostname -f)" \
-        --cluster-cidr=10.244.0.0/16 \
-        --etcd-expose-metrics=true \
-        --kube-controller-manager-arg="address=0.0.0.0" \
-        --kube-controller-manager-arg="bind-address=0.0.0.0" \
-        --kube-proxy-arg="metrics-bind-address=0.0.0.0" \
-        --kube-scheduler-arg="address=0.0.0.0" \
-        --kube-scheduler-arg="bind-address=0.0.0.0" \
-        #{taint} \
-        --kubelet-arg="cloud-provider=external" \
-        --advertise-address=$(hostname -I | awk '{print $2}') \
-        --node-ip=$(hostname -I | awk '{print $2}') \
-        --node-external-ip=$(hostname -I | awk '{print $1}') \
-        --flannel-iface=#{flannel_interface} \
-        #{server} #{tls_sans}" sh -
+        curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="#{k3s_version}" K3S_TOKEN="#{k3s_token}" INSTALL_K3S_EXEC="server \
+          --disable-cloud-controller \
+          --disable servicelb \
+          --disable traefik \
+          --disable local-storage \
+          --disable metrics-server \
+          --write-kubeconfig-mode=644 \
+          --node-name="$(hostname -f)" \
+          --cluster-cidr=10.244.0.0/16 \
+          --etcd-expose-metrics=true \
+          --kube-controller-manager-arg="address=0.0.0.0" \
+          --kube-controller-manager-arg="bind-address=0.0.0.0" \
+          --kube-proxy-arg="metrics-bind-address=0.0.0.0" \
+          --kube-scheduler-arg="address=0.0.0.0" \
+          --kube-scheduler-arg="bind-address=0.0.0.0" \
+          #{taint} \
+          --kubelet-arg="cloud-provider=external" \
+          --advertise-address=$(hostname -I | awk '{print $2}') \
+          --node-ip=$(hostname -I | awk '{print $2}') \
+          --node-external-ip=$(hostname -I | awk '{print $1}') \
+          --flannel-iface=#{flannel_interface} \
+          #{server} #{tls_sans}" sh -
       EOF
     end
 
@@ -246,12 +264,12 @@ class Cluster
       flannel_interface = find_flannel_interface(worker)
 
       <<~EOF
-      curl -sfL https://get.k3s.io | K3S_TOKEN="#{k3s_token}" INSTALL_K3S_VERSION="#{k3s_version}" K3S_URL=https://#{first_master_private_ip}:6443 INSTALL_K3S_EXEC="agent \
-        --node-name="$(hostname -f)" \
-        --kubelet-arg="cloud-provider=external" \
-        --node-ip=$(hostname -I | awk '{print $2}') \
-        --node-external-ip=$(hostname -I | awk '{print $1}') \
-        --flannel-iface=#{flannel_interface}" sh -
+        curl -sfL https://get.k3s.io | K3S_TOKEN="#{k3s_token}" INSTALL_K3S_VERSION="#{k3s_version}" K3S_URL=https://#{first_master_private_ip}:6443 INSTALL_K3S_EXEC="agent \
+          --node-name="$(hostname -f)" \
+          --kubelet-arg="cloud-provider=external" \
+          --node-ip=$(hostname -I | awk '{print $2}') \
+          --node-external-ip=$(hostname -I | awk '{print $1}') \
+          --flannel-iface=#{flannel_interface}" sh -
       EOF
     end
 

data/lib/hetzner/k3s/version.rb

@@ -1,5 +1,5 @@
 module Hetzner
   module K3s
-    VERSION = "0.4.2"
+    VERSION = "0.4.6"
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: hetzner-k3s
 version: !ruby/object:Gem::Version
-  version: 0.4.2
+  version: 0.4.6
 platform: ruby
 authors:
 - Vito Botta
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-10-03 00:00:00.000000000 Z
+date: 2021-11-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
@@ -140,6 +140,7 @@ files:
 - lib/hetzner/infra/firewall.rb
 - lib/hetzner/infra/load_balancer.rb
 - lib/hetzner/infra/network.rb
+- lib/hetzner/infra/placement_group.rb
 - lib/hetzner/infra/server.rb
 - lib/hetzner/infra/ssh_key.rb
 - lib/hetzner/k3s/cli.rb