hetzner-k3s 0.4.0 → 0.4.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/README.md +28 -6
- data/bin/build.sh +14 -0
- data/lib/hetzner/infra/network.rb +4 -3
- data/lib/hetzner/infra/placement_group.rb +55 -0
- data/lib/hetzner/infra/server.rb +3 -2
- data/lib/hetzner/infra/ssh_key.rb +15 -15
- data/lib/hetzner/k3s/cli.rb +32 -8
- data/lib/hetzner/k3s/cluster.rb +61 -15
- data/lib/hetzner/k3s/version.rb +1 -1
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 153385c9fce84159b90d6b77bed3e3afebd3cb0739fbd6de1e4cc91e5f1e130f
|
|
4
|
+
data.tar.gz: '08a51842b854c2a438012c6fde115520e32fb782524a3ddd4048a772db8aeaa3'
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 9d6ac1e71d783a6b01d77863e30fae972c0983cda9e85dfcaac5fda8da0ea7a33565404aaa8efdd0594185f31c85ebc91cf92cdfe06fd9b92422aaf8c158feee
|
|
7
|
+
data.tar.gz: 4f93a9eb6635d2c757dfbc1205023e563bfbb564ca0b0f9b839ea4b722f9fc2d7db7978ce4ab7da70d2ce23fd3559a301342f3b222522ac4df0554a9f66317bd
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
This is a CLI tool - based on a Ruby gem - to quickly create and manage Kubernetes clusters in [Hetzner Cloud](https://www.hetzner.com/cloud) using the lightweight Kubernetes distribution [k3s](https://k3s.io/) from [Rancher](https://rancher.com/).
|
|
4
4
|
|
|
5
|
-
Hetzner Cloud is an awesome cloud provider which offers a truly great service with the best performance/cost ratio in the market.
|
|
5
|
+
Hetzner Cloud is an awesome cloud provider which offers a truly great service with the best performance/cost ratio in the market. With Hetzner's Cloud Controller Manager and CSI driver you can provision load balancers and persistent volumes very easily.
|
|
6
6
|
|
|
7
7
|
k3s is my favorite Kubernetes distribution now because it uses much less memory and CPU, leaving more resources to workloads. It is also super quick to deploy because it's a single binary.
|
|
8
8
|
|
|
@@ -25,7 +25,7 @@ All that is needed to use this tool is
|
|
|
25
25
|
|
|
26
26
|
## Installation
|
|
27
27
|
|
|
28
|
-
Once you have the Ruby runtime up and running, you just need to install the gem:
|
|
28
|
+
Once you have the Ruby runtime up and running (2.7.2 or newer in the 2.7 series is recommended at this stage), you just need to install the gem:
|
|
29
29
|
|
|
30
30
|
```bash
|
|
31
31
|
gem install hetzner-k3s
|
|
@@ -38,7 +38,7 @@ This will install the `hetzner-k3s` executable in your PATH.
|
|
|
38
38
|
Alternatively, if you don't want to set up a Ruby runtime but have Docker installed, you can use a container. Run the following from inside the directory where you have the config file for the cluster (described in the next section):
|
|
39
39
|
|
|
40
40
|
```bash
|
|
41
|
-
docker run --rm -it -v ${PWD}:/cluster -v ${HOME}/.ssh:/tmp/.ssh vitobotta/hetzner-k3s:v0.
|
|
41
|
+
docker run --rm -it -v ${PWD}:/cluster -v ${HOME}/.ssh:/tmp/.ssh vitobotta/hetzner-k3s:v0.4.4 create-cluster --config-file /cluster/test.yaml
|
|
42
42
|
```
|
|
43
43
|
|
|
44
44
|
Replace `test.yaml` with the name of your config file.
|
|
@@ -53,11 +53,13 @@ hetzner_token: <your token>
|
|
|
53
53
|
cluster_name: test
|
|
54
54
|
kubeconfig_path: "./kubeconfig"
|
|
55
55
|
k3s_version: v1.21.3+k3s1
|
|
56
|
-
|
|
56
|
+
public_ssh_key_path: "~/.ssh/id_rsa.pub"
|
|
57
|
+
private_ssh_key_path: "~/.ssh/id_rsa"
|
|
57
58
|
ssh_allowed_networks:
|
|
58
59
|
- 0.0.0.0/0
|
|
59
60
|
verify_host_key: false
|
|
60
61
|
location: nbg1
|
|
62
|
+
schedule_workloads_on_masters: false
|
|
61
63
|
masters:
|
|
62
64
|
instance_type: cpx21
|
|
63
65
|
instance_count: 3
|
|
@@ -76,7 +78,7 @@ If you are using Docker, then set `kubeconfig_path` to `/cluster/kubeconfig` so
|
|
|
76
78
|
|
|
77
79
|
If you don't want to specify the Hetzner token in the config file (for example if you want to use the tool with CI), then you can use the `HCLOUD_TOKEN` environment variable instead, which has predecence.
|
|
78
80
|
|
|
79
|
-
**Important**: The tool assignes the label `cluster` to each server it creates, with the
|
|
81
|
+
**Important**: The tool assignes the label `cluster` to each server it creates, with the cluster name you specify in the config file, as the value. So please ensure you don't create unrelated servers in the same project having
|
|
80
82
|
the label `cluster=<cluster name>`, because otherwise they will be deleted if you delete the cluster. I recommend you create a separate Hetzner project for each cluster, see note at the end of this README for more details.
|
|
81
83
|
|
|
82
84
|
|
|
@@ -84,7 +86,8 @@ If you set `masters.instance_count` to 1 then the tool will create a non highly
|
|
|
84
86
|
|
|
85
87
|
You can specify any number of worker node pools for example to have mixed nodes with different specs for different workloads.
|
|
86
88
|
|
|
87
|
-
At the moment Hetzner Cloud has
|
|
89
|
+
At the moment Hetzner Cloud has four locations: two in Germany (`nbg1`, Nuremberg and `fsn1`, Falkensteing), one in Finland (`hel1`, Helsinki) and one in the USA (`ash`, Ashburn, Virginia). Please note that the Ashburn, Virginia location has just
|
|
90
|
+
been announced and it's limited to AMD instances for now.
|
|
88
91
|
|
|
89
92
|
For the available instance types and their specs, either check from inside a project when adding a server manually or run the following with your Hetzner token:
|
|
90
93
|
|
|
@@ -239,11 +242,30 @@ I recommend that you create a separate Hetzner project for each cluster, because
|
|
|
239
242
|
|
|
240
243
|
## changelog
|
|
241
244
|
|
|
245
|
+
- 0.4.4
|
|
246
|
+
- Add support for the new Ashburn, Virginia (USA) location
|
|
247
|
+
- Automatically use a placement group so that the instances are all created on different physical hosts for high availability
|
|
248
|
+
|
|
249
|
+
- 0.4.3
|
|
250
|
+
- Fix an issue with SSH key creation
|
|
251
|
+
|
|
252
|
+
- 0.4.2
|
|
253
|
+
- Update Hetzner CSI driver to v1.6.0
|
|
254
|
+
- Update System Upgrade Controller to v0.8.0
|
|
255
|
+
|
|
256
|
+
- 0.4.1
|
|
257
|
+
- Allow to optionally specify the path of the private SSH key
|
|
258
|
+
- Set correct permissions for the kubeconfig file
|
|
259
|
+
- Retry fetching manifests a few times to allow for temporary network issues
|
|
260
|
+
- Allow to optionally schedule workloads on masters
|
|
261
|
+
- Allow clusters with no worker node pools if scheduling is enabled for the masters
|
|
262
|
+
|
|
242
263
|
- 0.4.0
|
|
243
264
|
- Ensure the masters are removed from the API load balancer before deleting the load balancer
|
|
244
265
|
- Ensure the servers are removed from the firewall before deleting it
|
|
245
266
|
- Allow using an environment variable to specify the Hetzner token
|
|
246
267
|
- Allow restricting SSH access to the nodes to specific networks
|
|
268
|
+
- Do not open the port 6443 on the nodes if a load balancer is created for an HA cluster
|
|
247
269
|
|
|
248
270
|
- 0.3.9
|
|
249
271
|
- Add command "version" to print the version of the tool in use
|
data/bin/build.sh
ADDED
|
@@ -5,7 +5,8 @@ module Hetzner
|
|
|
5
5
|
@cluster_name = cluster_name
|
|
6
6
|
end
|
|
7
7
|
|
|
8
|
-
def create
|
|
8
|
+
def create(location:)
|
|
9
|
+
@location = location
|
|
9
10
|
puts
|
|
10
11
|
|
|
11
12
|
if network = find_network
|
|
@@ -38,7 +39,7 @@ module Hetzner
|
|
|
38
39
|
|
|
39
40
|
private
|
|
40
41
|
|
|
41
|
-
attr_reader :hetzner_client, :cluster_name
|
|
42
|
+
attr_reader :hetzner_client, :cluster_name, :location
|
|
42
43
|
|
|
43
44
|
def network_config
|
|
44
45
|
{
|
|
@@ -47,7 +48,7 @@ module Hetzner
|
|
|
47
48
|
subnets: [
|
|
48
49
|
{
|
|
49
50
|
ip_range: "10.0.0.0/16",
|
|
50
|
-
network_zone: "eu-central",
|
|
51
|
+
network_zone: (location ? "us-east" : "eu-central"),
|
|
51
52
|
type: "cloud"
|
|
52
53
|
}
|
|
53
54
|
]
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
module Hetzner
|
|
2
|
+
class PlacementGroup
|
|
3
|
+
def initialize(hetzner_client:, cluster_name:)
|
|
4
|
+
@hetzner_client = hetzner_client
|
|
5
|
+
@cluster_name = cluster_name
|
|
6
|
+
end
|
|
7
|
+
|
|
8
|
+
def create
|
|
9
|
+
puts
|
|
10
|
+
|
|
11
|
+
if (placement_group = find_placement_group)
|
|
12
|
+
puts "Placement group already exists, skipping."
|
|
13
|
+
puts
|
|
14
|
+
return placement_group["id"]
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
puts "Creating placement group..."
|
|
18
|
+
|
|
19
|
+
response = hetzner_client.post("/placement_groups", placement_group_config).body
|
|
20
|
+
|
|
21
|
+
puts "...placement group created."
|
|
22
|
+
puts
|
|
23
|
+
|
|
24
|
+
JSON.parse(response)["placement_group"]["id"]
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def delete
|
|
28
|
+
if (placement_group = find_placement_group)
|
|
29
|
+
puts "Deleting placement group..."
|
|
30
|
+
hetzner_client.delete("/placement_groups", placement_group["id"])
|
|
31
|
+
puts "...placement group deleted."
|
|
32
|
+
else
|
|
33
|
+
puts "Placement group no longer exists, skipping."
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
puts
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
private
|
|
40
|
+
|
|
41
|
+
attr_reader :hetzner_client, :cluster_name
|
|
42
|
+
|
|
43
|
+
def placement_group_config
|
|
44
|
+
{
|
|
45
|
+
name: cluster_name,
|
|
46
|
+
type: "spread"
|
|
47
|
+
}
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
def find_placement_group
|
|
51
|
+
hetzner_client.get("/placement_groups")["placement_groups"].detect{ |placement_group| placement_group["name"] == cluster_name }
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
end
|
|
55
|
+
end
|
data/lib/hetzner/infra/server.rb
CHANGED
|
@@ -5,7 +5,7 @@ module Hetzner
|
|
|
5
5
|
@cluster_name = cluster_name
|
|
6
6
|
end
|
|
7
7
|
|
|
8
|
-
def create(location:, instance_type:, instance_id:, firewall_id:, network_id:, ssh_key_id:)
|
|
8
|
+
def create(location:, instance_type:, instance_id:, firewall_id:, network_id:, ssh_key_id:, placement_group_id:)
|
|
9
9
|
puts
|
|
10
10
|
|
|
11
11
|
server_name = "#{cluster_name}-#{instance_type}-#{instance_id}"
|
|
@@ -36,7 +36,8 @@ module Hetzner
|
|
|
36
36
|
labels: {
|
|
37
37
|
cluster: cluster_name,
|
|
38
38
|
role: (server_name =~ /master/ ? "master" : "worker")
|
|
39
|
-
}
|
|
39
|
+
},
|
|
40
|
+
placement_group: placement_group_id
|
|
40
41
|
}
|
|
41
42
|
|
|
42
43
|
response = hetzner_client.post("/servers", server_config).body
|
|
@@ -5,15 +5,15 @@ module Hetzner
|
|
|
5
5
|
@cluster_name = cluster_name
|
|
6
6
|
end
|
|
7
7
|
|
|
8
|
-
def create(
|
|
9
|
-
@
|
|
8
|
+
def create(public_ssh_key_path:)
|
|
9
|
+
@public_ssh_key_path = public_ssh_key_path
|
|
10
10
|
|
|
11
11
|
puts
|
|
12
12
|
|
|
13
|
-
if
|
|
13
|
+
if (public_ssh_key = find_public_ssh_key)
|
|
14
14
|
puts "SSH key already exists, skipping."
|
|
15
15
|
puts
|
|
16
|
-
return
|
|
16
|
+
return public_ssh_key["id"]
|
|
17
17
|
end
|
|
18
18
|
|
|
19
19
|
puts "Creating SSH key..."
|
|
@@ -26,13 +26,13 @@ module Hetzner
|
|
|
26
26
|
JSON.parse(response)["ssh_key"]["id"]
|
|
27
27
|
end
|
|
28
28
|
|
|
29
|
-
def delete(
|
|
30
|
-
@
|
|
29
|
+
def delete(public_ssh_key_path:)
|
|
30
|
+
@public_ssh_key_path = public_ssh_key_path
|
|
31
31
|
|
|
32
|
-
if
|
|
33
|
-
if
|
|
32
|
+
if (public_ssh_key = find_public_ssh_key)
|
|
33
|
+
if public_ssh_key["name"] == cluster_name
|
|
34
34
|
puts "Deleting ssh_key..."
|
|
35
|
-
hetzner_client.delete("/ssh_keys",
|
|
35
|
+
hetzner_client.delete("/ssh_keys", public_ssh_key["id"])
|
|
36
36
|
puts "...ssh_key deleted."
|
|
37
37
|
else
|
|
38
38
|
puts "The SSH key existed before creating the cluster, so I won't delete it."
|
|
@@ -46,24 +46,24 @@ module Hetzner
|
|
|
46
46
|
|
|
47
47
|
private
|
|
48
48
|
|
|
49
|
-
attr_reader :hetzner_client, :cluster_name, :
|
|
49
|
+
attr_reader :hetzner_client, :cluster_name, :public_ssh_key_path
|
|
50
50
|
|
|
51
|
-
def
|
|
52
|
-
@
|
|
51
|
+
def public_ssh_key
|
|
52
|
+
@public_ssh_key ||= File.read(public_ssh_key_path).chop
|
|
53
53
|
end
|
|
54
54
|
|
|
55
55
|
def ssh_key_config
|
|
56
56
|
{
|
|
57
57
|
name: cluster_name,
|
|
58
|
-
public_key:
|
|
58
|
+
public_key: public_ssh_key
|
|
59
59
|
}
|
|
60
60
|
end
|
|
61
61
|
|
|
62
62
|
def fingerprint
|
|
63
|
-
@fingerprint ||= ::SSHKey.fingerprint(
|
|
63
|
+
@fingerprint ||= ::SSHKey.fingerprint(public_ssh_key)
|
|
64
64
|
end
|
|
65
65
|
|
|
66
|
-
def
|
|
66
|
+
def find_public_ssh_key
|
|
67
67
|
key = hetzner_client.get("/ssh_keys")["ssh_keys"].detect do |ssh_key|
|
|
68
68
|
ssh_key["fingerprint"] == fingerprint
|
|
69
69
|
end
|
data/lib/hetzner/k3s/cli.rb
CHANGED
|
@@ -83,7 +83,8 @@ module Hetzner
|
|
|
83
83
|
|
|
84
84
|
case action
|
|
85
85
|
when :create
|
|
86
|
-
|
|
86
|
+
validate_public_ssh_key
|
|
87
|
+
validate_private_ssh_key
|
|
87
88
|
validate_ssh_allowed_networks
|
|
88
89
|
validate_location
|
|
89
90
|
validate_k3s_version
|
|
@@ -147,16 +148,25 @@ module Hetzner
|
|
|
147
148
|
errors << "Invalid path for the kubeconfig"
|
|
148
149
|
end
|
|
149
150
|
|
|
150
|
-
def
|
|
151
|
-
path = File.expand_path(configuration.dig("
|
|
151
|
+
def validate_public_ssh_key
|
|
152
|
+
path = File.expand_path(configuration.dig("public_ssh_key_path"))
|
|
152
153
|
errors << "Invalid Public SSH key path" and return unless File.exists? path
|
|
153
154
|
|
|
154
155
|
key = File.read(path)
|
|
155
|
-
errors << "Public SSH key is invalid" unless ::SSHKey.valid_ssh_public_key?
|
|
156
|
+
errors << "Public SSH key is invalid" unless ::SSHKey.valid_ssh_public_key?(key)
|
|
156
157
|
rescue
|
|
157
158
|
errors << "Invalid Public SSH key path"
|
|
158
159
|
end
|
|
159
160
|
|
|
161
|
+
def validate_private_ssh_key
|
|
162
|
+
return unless (private_ssh_key_path = configuration.dig("private_ssh_key_path"))
|
|
163
|
+
|
|
164
|
+
path = File.expand_path(private_ssh_key_path)
|
|
165
|
+
errors << "Invalid Private SSH key path" and return unless File.exists?(path)
|
|
166
|
+
rescue
|
|
167
|
+
errors << "Invalid Private SSH key path"
|
|
168
|
+
end
|
|
169
|
+
|
|
160
170
|
def validate_kubeconfig_path_must_exist
|
|
161
171
|
path = File.expand_path configuration.dig("kubeconfig_path")
|
|
162
172
|
errors << "kubeconfig path is invalid" and return unless File.exists? path
|
|
@@ -183,7 +193,7 @@ module Hetzner
|
|
|
183
193
|
|
|
184
194
|
def validate_location
|
|
185
195
|
return if locations.empty? && !valid_token?
|
|
186
|
-
errors << "Invalid location - available locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland)" unless locations.include? configuration.dig("location")
|
|
196
|
+
errors << "Invalid location - available locations: nbg1 (Nuremberg, Germany), fsn1 (Falkenstein, Germany), hel1 (Helsinki, Finland) or ash (Ashburn, Virginia, USA)" unless locations.include? configuration.dig("location")
|
|
187
197
|
end
|
|
188
198
|
|
|
189
199
|
def find_available_releases
|
|
@@ -231,14 +241,22 @@ module Hetzner
|
|
|
231
241
|
begin
|
|
232
242
|
worker_node_pools = configuration.dig("worker_node_pools")
|
|
233
243
|
rescue
|
|
234
|
-
|
|
244
|
+
unless schedule_workloads_on_masters?
|
|
245
|
+
errors << "Invalid node pools configuration"
|
|
246
|
+
return
|
|
247
|
+
end
|
|
248
|
+
end
|
|
249
|
+
|
|
250
|
+
if worker_node_pools.nil? && schedule_workloads_on_masters?
|
|
235
251
|
return
|
|
236
252
|
end
|
|
237
253
|
|
|
238
254
|
if !worker_node_pools.is_a? Array
|
|
239
255
|
errors << "Invalid node pools configuration"
|
|
240
256
|
elsif worker_node_pools.size == 0
|
|
241
|
-
|
|
257
|
+
unless schedule_workloads_on_masters?
|
|
258
|
+
errors << "At least one node pool is required in order to schedule workloads"
|
|
259
|
+
end
|
|
242
260
|
elsif worker_node_pools.map{ |worker_node_pool| worker_node_pool["name"]}.uniq.size != worker_node_pools.size
|
|
243
261
|
errors << "Each node pool must have an unique name"
|
|
244
262
|
elsif server_types
|
|
@@ -248,6 +266,11 @@ module Hetzner
|
|
|
248
266
|
end
|
|
249
267
|
end
|
|
250
268
|
|
|
269
|
+
def schedule_workloads_on_masters?
|
|
270
|
+
schedule_workloads_on_masters = configuration.dig("schedule_workloads_on_masters")
|
|
271
|
+
schedule_workloads_on_masters ? !!schedule_workloads_on_masters : false
|
|
272
|
+
end
|
|
273
|
+
|
|
251
274
|
def validate_new_k3s_version_must_be_more_recent
|
|
252
275
|
return if options[:force] == "true"
|
|
253
276
|
return unless kubernetes_client
|
|
@@ -316,12 +339,13 @@ module Hetzner
|
|
|
316
339
|
config_hash = YAML.load_file(File.expand_path(configuration["kubeconfig_path"]))
|
|
317
340
|
config_hash['current-context'] = configuration["cluster_name"]
|
|
318
341
|
@kubernetes_client = K8s::Client.config(K8s::Config.new(config_hash))
|
|
342
|
+
rescue
|
|
319
343
|
errors << "Cannot connect to the Kubernetes cluster"
|
|
320
344
|
false
|
|
321
345
|
end
|
|
322
346
|
|
|
323
347
|
def validate_verify_host_key
|
|
324
|
-
return unless [true, false].include?(configuration.fetch("
|
|
348
|
+
return unless [true, false].include?(configuration.fetch("public_ssh_key_path", false))
|
|
325
349
|
errors << "Please set the verify_host_key option to either true or false"
|
|
326
350
|
end
|
|
327
351
|
|
data/lib/hetzner/k3s/cluster.rb
CHANGED
|
@@ -11,6 +11,7 @@ require_relative "../infra/network"
|
|
|
11
11
|
require_relative "../infra/ssh_key"
|
|
12
12
|
require_relative "../infra/server"
|
|
13
13
|
require_relative "../infra/load_balancer"
|
|
14
|
+
require_relative "../infra/placement_group"
|
|
14
15
|
|
|
15
16
|
require_relative "../k3s/client_patch"
|
|
16
17
|
|
|
@@ -22,12 +23,15 @@ class Cluster
|
|
|
22
23
|
end
|
|
23
24
|
|
|
24
25
|
def create(configuration:)
|
|
26
|
+
@configuration = configuration
|
|
25
27
|
@cluster_name = configuration.dig("cluster_name")
|
|
26
28
|
@kubeconfig_path = File.expand_path(configuration.dig("kubeconfig_path"))
|
|
27
|
-
@
|
|
29
|
+
@public_ssh_key_path = File.expand_path(configuration.dig("public_ssh_key_path"))
|
|
30
|
+
private_ssh_key_path = configuration.dig("private_ssh_key_path")
|
|
31
|
+
@private_ssh_key_path = File.expand_path(private_ssh_key_path) if private_ssh_key_path
|
|
28
32
|
@k3s_version = configuration.dig("k3s_version")
|
|
29
33
|
@masters_config = configuration.dig("masters")
|
|
30
|
-
@worker_node_pools = configuration
|
|
34
|
+
@worker_node_pools = find_worker_node_pools(configuration)
|
|
31
35
|
@location = configuration.dig("location")
|
|
32
36
|
@verify_host_key = configuration.fetch("verify_host_key", false)
|
|
33
37
|
@servers = []
|
|
@@ -47,7 +51,7 @@ class Cluster
|
|
|
47
51
|
def delete(configuration:)
|
|
48
52
|
@cluster_name = configuration.dig("cluster_name")
|
|
49
53
|
@kubeconfig_path = File.expand_path(configuration.dig("kubeconfig_path"))
|
|
50
|
-
@
|
|
54
|
+
@public_ssh_key_path = File.expand_path(configuration.dig("public_ssh_key_path"))
|
|
51
55
|
|
|
52
56
|
delete_resources
|
|
53
57
|
end
|
|
@@ -64,13 +68,17 @@ class Cluster
|
|
|
64
68
|
|
|
65
69
|
private
|
|
66
70
|
|
|
71
|
+
def find_worker_node_pools(configuration)
|
|
72
|
+
configuration.fetch("worker_node_pools", [])
|
|
73
|
+
end
|
|
74
|
+
|
|
67
75
|
attr_accessor :servers
|
|
68
76
|
|
|
69
77
|
attr_reader :hetzner_client, :cluster_name, :kubeconfig_path, :k3s_version,
|
|
70
78
|
:masters_config, :worker_node_pools,
|
|
71
|
-
:location, :
|
|
79
|
+
:location, :public_ssh_key_path, :kubernetes_client,
|
|
72
80
|
:hetzner_token, :tls_sans, :new_k3s_version, :configuration,
|
|
73
|
-
:config_file, :verify_host_key, :networks
|
|
81
|
+
:config_file, :verify_host_key, :networks, :private_ssh_key_path, :configuration
|
|
74
82
|
|
|
75
83
|
|
|
76
84
|
def latest_k3s_version
|
|
@@ -82,6 +90,11 @@ class Cluster
|
|
|
82
90
|
master_instance_type = masters_config["instance_type"]
|
|
83
91
|
masters_count = masters_config["instance_count"]
|
|
84
92
|
|
|
93
|
+
placement_group_id = Hetzner::PlacementGroup.new(
|
|
94
|
+
hetzner_client: hetzner_client,
|
|
95
|
+
cluster_name: cluster_name
|
|
96
|
+
).create
|
|
97
|
+
|
|
85
98
|
firewall_id = Hetzner::Firewall.new(
|
|
86
99
|
hetzner_client: hetzner_client,
|
|
87
100
|
cluster_name: cluster_name
|
|
@@ -90,12 +103,12 @@ class Cluster
|
|
|
90
103
|
network_id = Hetzner::Network.new(
|
|
91
104
|
hetzner_client: hetzner_client,
|
|
92
105
|
cluster_name: cluster_name
|
|
93
|
-
).create
|
|
106
|
+
).create(location: location)
|
|
94
107
|
|
|
95
108
|
ssh_key_id = Hetzner::SSHKey.new(
|
|
96
109
|
hetzner_client: hetzner_client,
|
|
97
110
|
cluster_name: cluster_name
|
|
98
|
-
).create(
|
|
111
|
+
).create(public_ssh_key_path: public_ssh_key_path)
|
|
99
112
|
|
|
100
113
|
server_configs = []
|
|
101
114
|
|
|
@@ -106,7 +119,8 @@ class Cluster
|
|
|
106
119
|
instance_id: "master#{i+1}",
|
|
107
120
|
firewall_id: firewall_id,
|
|
108
121
|
network_id: network_id,
|
|
109
|
-
ssh_key_id: ssh_key_id
|
|
122
|
+
ssh_key_id: ssh_key_id,
|
|
123
|
+
placement_group_id: placement_group_id
|
|
110
124
|
}
|
|
111
125
|
end
|
|
112
126
|
|
|
@@ -129,7 +143,8 @@ class Cluster
|
|
|
129
143
|
instance_id: "pool-#{worker_node_pool_name}-worker#{i+1}",
|
|
130
144
|
firewall_id: firewall_id,
|
|
131
145
|
network_id: network_id,
|
|
132
|
-
ssh_key_id: ssh_key_id
|
|
146
|
+
ssh_key_id: ssh_key_id,
|
|
147
|
+
placement_group_id: placement_group_id
|
|
133
148
|
}
|
|
134
149
|
end
|
|
135
150
|
end
|
|
@@ -151,6 +166,11 @@ class Cluster
|
|
|
151
166
|
end
|
|
152
167
|
|
|
153
168
|
def delete_resources
|
|
169
|
+
Hetzner::PlacementGroup.new(
|
|
170
|
+
hetzner_client: hetzner_client,
|
|
171
|
+
cluster_name: cluster_name
|
|
172
|
+
).delete
|
|
173
|
+
|
|
154
174
|
Hetzner::LoadBalancer.new(
|
|
155
175
|
hetzner_client: hetzner_client,
|
|
156
176
|
cluster_name: cluster_name
|
|
@@ -169,7 +189,7 @@ class Cluster
|
|
|
169
189
|
Hetzner::SSHKey.new(
|
|
170
190
|
hetzner_client: hetzner_client,
|
|
171
191
|
cluster_name: cluster_name
|
|
172
|
-
).delete(
|
|
192
|
+
).delete(public_ssh_key_path: public_ssh_key_path)
|
|
173
193
|
|
|
174
194
|
threads = all_servers.map do |server|
|
|
175
195
|
Thread.new do
|
|
@@ -207,6 +227,8 @@ class Cluster
|
|
|
207
227
|
server = master == first_master ? " --cluster-init " : " --server https://#{first_master_private_ip}:6443 "
|
|
208
228
|
flannel_interface = find_flannel_interface(master)
|
|
209
229
|
|
|
230
|
+
taint = schedule_workloads_on_masters? ? " " : " --node-taint CriticalAddonsOnly=true:NoExecute "
|
|
231
|
+
|
|
210
232
|
<<~EOF
|
|
211
233
|
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="#{k3s_version}" K3S_TOKEN="#{k3s_token}" INSTALL_K3S_EXEC="server \
|
|
212
234
|
--disable-cloud-controller \
|
|
@@ -223,7 +245,7 @@ class Cluster
|
|
|
223
245
|
--kube-proxy-arg="metrics-bind-address=0.0.0.0" \
|
|
224
246
|
--kube-scheduler-arg="address=0.0.0.0" \
|
|
225
247
|
--kube-scheduler-arg="bind-address=0.0.0.0" \
|
|
226
|
-
|
|
248
|
+
#{taint} \
|
|
227
249
|
--kubelet-arg="cloud-provider=external" \
|
|
228
250
|
--advertise-address=$(hostname -I | awk '{print $2}') \
|
|
229
251
|
--node-ip=$(hostname -I | awk '{print $2}') \
|
|
@@ -313,7 +335,7 @@ class Cluster
|
|
|
313
335
|
end
|
|
314
336
|
|
|
315
337
|
|
|
316
|
-
manifest =
|
|
338
|
+
manifest = fetch_manifest("https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm-networks.yaml")
|
|
317
339
|
|
|
318
340
|
File.write("/tmp/cloud-controller-manager.yaml", manifest)
|
|
319
341
|
|
|
@@ -338,11 +360,18 @@ class Cluster
|
|
|
338
360
|
retry
|
|
339
361
|
end
|
|
340
362
|
|
|
363
|
+
def fetch_manifest(url)
|
|
364
|
+
retries ||= 1
|
|
365
|
+
HTTP.follow.get(url).body
|
|
366
|
+
rescue
|
|
367
|
+
retry if (retries += 1) <= 10
|
|
368
|
+
end
|
|
369
|
+
|
|
341
370
|
def deploy_system_upgrade_controller
|
|
342
371
|
puts
|
|
343
372
|
puts "Deploying k3s System Upgrade Controller..."
|
|
344
373
|
|
|
345
|
-
manifest = HTTP.follow.get("https://github.com/rancher/system-upgrade-controller/releases/download/v0.
|
|
374
|
+
manifest = HTTP.follow.get("https://github.com/rancher/system-upgrade-controller/releases/download/v0.8.0/system-upgrade-controller.yaml").body
|
|
346
375
|
|
|
347
376
|
File.write("/tmp/system-upgrade-controller.yaml", manifest)
|
|
348
377
|
|
|
@@ -391,7 +420,7 @@ class Cluster
|
|
|
391
420
|
end
|
|
392
421
|
|
|
393
422
|
|
|
394
|
-
manifest = HTTP.follow.get("https://raw.githubusercontent.com/hetznercloud/csi-driver/v1.
|
|
423
|
+
manifest = HTTP.follow.get("https://raw.githubusercontent.com/hetznercloud/csi-driver/v1.6.0/deploy/kubernetes/hcloud-csi.yml").body
|
|
395
424
|
|
|
396
425
|
File.write("/tmp/csi-driver.yaml", manifest)
|
|
397
426
|
|
|
@@ -442,7 +471,13 @@ class Cluster
|
|
|
442
471
|
public_ip = server.dig("public_net", "ipv4", "ip")
|
|
443
472
|
output = ""
|
|
444
473
|
|
|
445
|
-
|
|
474
|
+
params = { verify_host_key: (verify_host_key ? :always : :never) }
|
|
475
|
+
|
|
476
|
+
if private_ssh_key_path
|
|
477
|
+
params[:keys] = [private_ssh_key_path]
|
|
478
|
+
end
|
|
479
|
+
|
|
480
|
+
Net::SSH.start(public_ip, "root", params) do |session|
|
|
446
481
|
session.exec!(command) do |channel, stream, data|
|
|
447
482
|
output << data
|
|
448
483
|
puts data if print_output
|
|
@@ -453,6 +488,10 @@ class Cluster
|
|
|
453
488
|
retry unless e.message =~ /Too many authentication failures/
|
|
454
489
|
rescue Net::SSH::ConnectionTimeout, Errno::ECONNREFUSED, Errno::ENETUNREACH, Errno::EHOSTUNREACH
|
|
455
490
|
retry
|
|
491
|
+
rescue Net::SSH::AuthenticationFailed
|
|
492
|
+
puts
|
|
493
|
+
puts "Cannot continue: SSH authentication failed. Please ensure that the private SSH key is correct."
|
|
494
|
+
exit 1
|
|
456
495
|
rescue Net::SSH::HostKeyMismatch
|
|
457
496
|
puts
|
|
458
497
|
puts "Cannot continue: Unable to SSH into server with IP #{public_ip} because the existing fingerprint in the known_hosts file does not match that of the actual host key."
|
|
@@ -542,6 +581,8 @@ class Cluster
|
|
|
542
581
|
gsub("default", cluster_name)
|
|
543
582
|
|
|
544
583
|
File.write(kubeconfig_path, kubeconfig)
|
|
584
|
+
|
|
585
|
+
FileUtils.chmod "go-r", kubeconfig_path
|
|
545
586
|
end
|
|
546
587
|
|
|
547
588
|
def ugrade_plan_manifest_path
|
|
@@ -605,4 +646,9 @@ class Cluster
|
|
|
605
646
|
server.dig("labels", "cluster") == cluster_name
|
|
606
647
|
end
|
|
607
648
|
|
|
649
|
+
def schedule_workloads_on_masters?
|
|
650
|
+
schedule_workloads_on_masters = configuration.dig("schedule_workloads_on_masters")
|
|
651
|
+
schedule_workloads_on_masters ? !!schedule_workloads_on_masters : false
|
|
652
|
+
end
|
|
653
|
+
|
|
608
654
|
end
|
data/lib/hetzner/k3s/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: hetzner-k3s
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.4.
|
|
4
|
+
version: 0.4.4
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Vito Botta
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2021-
|
|
11
|
+
date: 2021-11-03 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: thor
|
|
@@ -127,6 +127,7 @@ files:
|
|
|
127
127
|
- LICENSE.txt
|
|
128
128
|
- README.md
|
|
129
129
|
- Rakefile
|
|
130
|
+
- bin/build.sh
|
|
130
131
|
- bin/console
|
|
131
132
|
- bin/setup
|
|
132
133
|
- cluster_config.yaml.example
|
|
@@ -139,6 +140,7 @@ files:
|
|
|
139
140
|
- lib/hetzner/infra/firewall.rb
|
|
140
141
|
- lib/hetzner/infra/load_balancer.rb
|
|
141
142
|
- lib/hetzner/infra/network.rb
|
|
143
|
+
- lib/hetzner/infra/placement_group.rb
|
|
142
144
|
- lib/hetzner/infra/server.rb
|
|
143
145
|
- lib/hetzner/infra/ssh_key.rb
|
|
144
146
|
- lib/hetzner/k3s/cli.rb
|