kubes 0.4.2 → 0.4.7
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +16 -0
- data/docs/_docs/helpers.md +8 -1
- data/docs/_docs/helpers/aws.md +15 -0
- data/docs/_docs/helpers/aws/iam-role.md +91 -0
- data/docs/_docs/helpers/aws/secrets.md +129 -0
- data/docs/_docs/helpers/aws/ssm.md +76 -0
- data/docs/_docs/helpers/custom.md +40 -0
- data/docs/_docs/helpers/google.md +17 -0
- data/docs/_docs/helpers/google/secrets.md +76 -0
- data/docs/_docs/helpers/google/service-account.md +52 -0
- data/docs/_docs/patterns/secrets.md +0 -2
- data/docs/_includes/helpers/base64.md +1 -0
- data/docs/_includes/layering/layers.md +1 -1
- data/docs/_includes/sidebar.html +21 -1
- data/kubes.gemspec +1 -0
- data/lib/kubes.rb +2 -1
- data/lib/kubes/autoloader.rb +9 -0
- data/lib/kubes/compiler/shared/custom_helpers.rb +17 -0
- data/lib/kubes/compiler/shared/helpers.rb +1 -1
- data/lib/kubes/compiler/strategy/base.rb +3 -3
- data/lib/kubes/compiler/strategy/dsl.rb +3 -2
- data/lib/kubes/compiler/strategy/erb.rb +10 -1
- data/lib/kubes/compiler/strategy/erb/yaml_error.rb +60 -0
- data/lib/kubes/compiler/util/normalize.rb +1 -1
- data/lib/kubes/compiler/util/save_file.rb +8 -0
- data/lib/kubes/hooks/builder.rb +1 -1
- data/lib/kubes/kubectl.rb +5 -23
- data/lib/kubes/kubectl/batch.rb +33 -6
- data/lib/kubes/version.rb +1 -1
- metadata +28 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 1b2e59aafa5ea2023bdf23dc43ba38c5279dcd719c3db793eca0abf24c8e1da2
|
4
|
+
data.tar.gz: ed004f619cba70d0d6257ca9559b3fd1b1fa7cb6c48943c64c3abcb8cddf3d38
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 7be4d6f0bd67e15b9ba3e3594f1b98d36c3fb5a4cc68c67fd603a75a973e4b90f8800a0a8fddf68951d0f3f09291c154bae5066830ec792aa167eb8675f5acca
|
7
|
+
data.tar.gz: 6187e6b6bc29e1a038f8c5a20530483a120cc6107854f718f8570648334f27cb7b4afb1a39d4d45ede4bb3cda97e8cc66d63c1cbbdde27a059997f65bd547704
|
data/CHANGELOG.md
CHANGED
@@ -3,6 +3,22 @@
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
4
4
|
This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
|
5
5
|
|
6
|
+
## [0.4.7]
|
7
|
+
- #33 improve switch context: earlier and only when needed
|
8
|
+
|
9
|
+
## [0.4.6]
|
10
|
+
- #32 custom helpers support
|
11
|
+
|
12
|
+
## [0.4.5]
|
13
|
+
- #31 kubes AWS helpers
|
14
|
+
|
15
|
+
## [0.4.4]
|
16
|
+
- #30 friendly message for rendered erb yaml and dsl errors
|
17
|
+
- fix backtrace_reject pattern
|
18
|
+
|
19
|
+
## [0.4.3]
|
20
|
+
- #29 fix edge case when user provides hook on option for non-kubectl hooks
|
21
|
+
|
6
22
|
## [0.4.2]
|
7
23
|
- #28 base64 helper
|
8
24
|
|
data/docs/_docs/helpers.md
CHANGED
@@ -17,4 +17,11 @@ Here's also the source code with most of the helpers: [helpers.rb](https://githu
|
|
17
17
|
|
18
18
|
## DSL Specific Methods
|
19
19
|
|
20
|
-
Each DSL resource has it's own specific methods. Refer to the [DSL Docs]({% link _docs/dsl.md %}) for their methods.
|
20
|
+
Each DSL resource has it's own specific methods. Refer to the [DSL Docs]({% link _docs/dsl.md %}) for their methods.
|
21
|
+
|
22
|
+
## Provider Helpers
|
23
|
+
|
24
|
+
There are also provider-specific helpers:
|
25
|
+
|
26
|
+
* [AWS Helpers]({% link _docs/helpers/aws.md %})
|
27
|
+
* [Google Helpers]({% link _docs/helpers/google.md %})
|
@@ -0,0 +1,15 @@
|
|
1
|
+
---
|
2
|
+
title: AWS Helpers
|
3
|
+
---
|
4
|
+
|
5
|
+
List of AWS helpers:
|
6
|
+
|
7
|
+
{% assign docs = site.docs | where: "categories","helpers-aws" %}
|
8
|
+
{% for doc in docs -%}
|
9
|
+
* [{{ doc.nav_text }}]({{ doc.url }})
|
10
|
+
{% endfor %}
|
11
|
+
|
12
|
+
## Notes
|
13
|
+
|
14
|
+
* By default, `KubeGoogle.logger = Kubes.logger`. This means, you can set `logger.level = "debug"` in `.kubes/config.rb` to see more details.
|
15
|
+
* The AWS helpers are provided by the [boltops-tools/kubes_aws](https://github.com/boltops-tools/kubes_aws) library.
|
@@ -0,0 +1,91 @@
|
|
1
|
+
---
|
2
|
+
title: AWS IAM Role
|
3
|
+
nav_text: IAM Role
|
4
|
+
categories: helpers-aws
|
5
|
+
---
|
6
|
+
|
7
|
+
You can automatically create the IAM Role associated with the Kubernetes Service Account, covered in [Introducing fine-grained IAM roles for service accounts](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/).
|
8
|
+
|
9
|
+
Here's a Kubes hook that creates an IAM Role:
|
10
|
+
|
11
|
+
.kubes/config/hooks/kubes.rb
|
12
|
+
|
13
|
+
```ruby
|
14
|
+
iam_role = KubesAws::IamRole.new(
|
15
|
+
app: "demo",
|
16
|
+
namespace: "demo-#{Kubes.env}", # defaults to APP-ENV when not set. IE: demo-dev
|
17
|
+
managed_policies: ["AmazonS3ReadOnlyAccess", "AmazonSSMReadOnlyAccess"], # defaults to empty when not set
|
18
|
+
inline_policies: [:secrets_read_only], # See Secrets Read Only Inline Policy at the bottom
|
19
|
+
)
|
20
|
+
before("apply",
|
21
|
+
label: "create iam role",
|
22
|
+
execute: iam_role,
|
23
|
+
)
|
24
|
+
KubesAws::IamRole.role_arn = iam_role.arn # used in .kubes/resources/shared/service_account.yaml
|
25
|
+
```
|
26
|
+
|
27
|
+
The corresponding Kubernetes Service account looks like this:
|
28
|
+
|
29
|
+
.kubes/resources/shared/service_account.yaml
|
30
|
+
|
31
|
+
```yaml
|
32
|
+
apiVersion: v1
|
33
|
+
kind: ServiceAccount
|
34
|
+
metadata:
|
35
|
+
annotations:
|
36
|
+
eks.amazonaws.com/role-arn: <%= KubesAws::IamRole.role_arn %>
|
37
|
+
name: demo
|
38
|
+
labels:
|
39
|
+
app: demo
|
40
|
+
```
|
41
|
+
|
42
|
+
The role policy permissions are currently always added to the existing permissions. So removing roles that were previously added does not remove them.
|
43
|
+
|
44
|
+
IamRole#initialize options:
|
45
|
+
|
46
|
+
Variable | Description | Default
|
47
|
+
---|---|---
|
48
|
+
app | The app name. It's used to set other variables conventionally. This is required. | nil
|
49
|
+
ksa | The Kubernetes Service Account name. The conventional name is APP. IE: demo | APP
|
50
|
+
namespace | The Kubernetes namespace. Defaults to the APP-ENV. IE: demo-dev. | APP-ENV
|
51
|
+
policies | IAM policies to add. This adds permissions to the IAM Role. | []
|
52
|
+
role_name | The IAM Role name. The conventional name is APP-ENV. IE: demo-dev. | APP-ENV
|
53
|
+
|
54
|
+
## OpenID Connect Provider
|
55
|
+
|
56
|
+
The `KubesAws::IamRole` class also automatically creates the OpenID Connect Provider if it doesn't already exist.
|
57
|
+
|
58
|
+
## Secrets Read-Only Inline Policy
|
59
|
+
|
60
|
+
Note the the `:secrets_read_only` is a way to generate an Inline Policy that represents read-only access for Secrets. Kubes does this since there's no managed policy for this yet. For example:
|
61
|
+
|
62
|
+
```ruby
|
63
|
+
inline_policies: [:secrets_read_only]
|
64
|
+
```
|
65
|
+
|
66
|
+
Is the same as:
|
67
|
+
|
68
|
+
```ruby
|
69
|
+
inline_secrets_read_only = {
|
70
|
+
policy_document: {
|
71
|
+
Version: "2012-10-17",
|
72
|
+
Statement: {
|
73
|
+
Effect: "Allow",
|
74
|
+
Action: [
|
75
|
+
"secretsmanager:Describe*",
|
76
|
+
"secretsmanager:Get*",
|
77
|
+
"secretsmanager:List*"
|
78
|
+
],
|
79
|
+
Resource: "*"
|
80
|
+
}
|
81
|
+
},
|
82
|
+
policy_name: "SecretsReadOnly",
|
83
|
+
}
|
84
|
+
iam_role = KubesAws::IamRole.new(
|
85
|
+
app: "rails",
|
86
|
+
cluster: "dev-cluster",
|
87
|
+
namespace: "rails-#{Kubes.env}", # defaults to APP-ENV when not set. IE: rails-dev
|
88
|
+
managed_policies: ["AmazonS3ReadOnlyAccess", "AmazonSSMReadOnlyAccess"], # defaults to empty when not set
|
89
|
+
inline_policies: [inline_secrets_read_only],
|
90
|
+
)
|
91
|
+
```
|
@@ -0,0 +1,129 @@
|
|
1
|
+
---
|
2
|
+
title: AWS Secrets
|
3
|
+
nav_text: Secrets
|
4
|
+
categories: helpers-aws
|
5
|
+
---
|
6
|
+
|
7
|
+
## Simple Values
|
8
|
+
|
9
|
+
For example if you have these secret values:
|
10
|
+
|
11
|
+
$ aws secretsmanager get-secret-value --secret-id demo/dev/db_user | jq '.SecretString'
|
12
|
+
user
|
13
|
+
$ aws secretsmanager get-secret-value --secret-id demo/dev/db_pass | jq '.SecretString'
|
14
|
+
pass
|
15
|
+
|
16
|
+
Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
|
17
|
+
|
18
|
+
.kubes/config/hooks/kubes.rb
|
19
|
+
|
20
|
+
```ruby
|
21
|
+
secrets = KubesAws::Secrets.new(upcase: true, prefix: "demo/dev/")
|
22
|
+
before("compile",
|
23
|
+
label: "Get secrets from AWS Secrets Manager",
|
24
|
+
execute: secrets,
|
25
|
+
)
|
26
|
+
```
|
27
|
+
|
28
|
+
Then set the secrets in the YAML:
|
29
|
+
|
30
|
+
.kubes/resources/shared/secret.yaml
|
31
|
+
|
32
|
+
```yaml
|
33
|
+
apiVersion: v1
|
34
|
+
kind: Secret
|
35
|
+
metadata:
|
36
|
+
name: demo
|
37
|
+
labels:
|
38
|
+
app: demo
|
39
|
+
data:
|
40
|
+
<% KubesAws::Secrets.data.each do |k,v| -%>
|
41
|
+
<%= k %>: <%= base64(v) %>
|
42
|
+
<% end -%>
|
43
|
+
```
|
44
|
+
|
45
|
+
This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data. The values are automatically base64 encoded. Produces:
|
46
|
+
|
47
|
+
.kubes/output/shared/secret.yaml
|
48
|
+
|
49
|
+
```yaml
|
50
|
+
metadata:
|
51
|
+
namespace: demo
|
52
|
+
name: demo-2a78a13682
|
53
|
+
labels:
|
54
|
+
app: demo
|
55
|
+
apiVersion: v1
|
56
|
+
kind: Secret
|
57
|
+
data:
|
58
|
+
db_pass: dGVzdDEK
|
59
|
+
db_user: dGVzdDIK
|
60
|
+
```
|
61
|
+
|
62
|
+
## JSON Values
|
63
|
+
|
64
|
+
For example if you have these secret values:
|
65
|
+
|
66
|
+
$ aws secretsmanager get-secret-value --secret-id demo/dev/k2 | jq '.SecretString'
|
67
|
+
{\"a\":1,\"b\":2}"
|
68
|
+
|
69
|
+
Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
|
70
|
+
|
71
|
+
.kubes/config/hooks/kubes.rb
|
72
|
+
|
73
|
+
```ruby
|
74
|
+
secrets = KubesAws::Secrets.new(prefix: "rails/dev/")
|
75
|
+
before("compile",
|
76
|
+
label: "Get secrets from AWS Secrets Manager",
|
77
|
+
execute: secrets,
|
78
|
+
)
|
79
|
+
```
|
80
|
+
|
81
|
+
Then set the secrets in the YAML:
|
82
|
+
|
83
|
+
.kubes/resources/shared/secret.yaml
|
84
|
+
|
85
|
+
```yaml
|
86
|
+
apiVersion: v1
|
87
|
+
kind: Secret
|
88
|
+
metadata:
|
89
|
+
name: demo
|
90
|
+
labels:
|
91
|
+
app: demo
|
92
|
+
data:
|
93
|
+
<% k2 = JSON.load(KubesAws::Secrets.data["k2"]) %>
|
94
|
+
a: <%= base64(k2["a"]) %>
|
95
|
+
b: <%= base64(k2["b"]) %>
|
96
|
+
```
|
97
|
+
|
98
|
+
Produces:
|
99
|
+
|
100
|
+
```yaml
|
101
|
+
metadata:
|
102
|
+
namespace: demo-dev
|
103
|
+
name: demo-a4cd604a95
|
104
|
+
labels:
|
105
|
+
app: demo
|
106
|
+
apiVersion: v1
|
107
|
+
kind: Secret
|
108
|
+
data:
|
109
|
+
a: MQ==
|
110
|
+
b: Mg==
|
111
|
+
```
|
112
|
+
|
113
|
+
## Variables
|
114
|
+
|
115
|
+
These environment variables can be set:
|
116
|
+
|
117
|
+
Name | Description
|
118
|
+
---|---
|
119
|
+
AWS_SECRET_PREFIX | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`.
|
120
|
+
|
121
|
+
Secrets#initialize options:
|
122
|
+
|
123
|
+
Variable | Description | Default
|
124
|
+
---|---|---
|
125
|
+
base64 | Automatically base64 encode the values. | false
|
126
|
+
upcase | Automatically upcase the Kubernetes secret data keys. | false
|
127
|
+
prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
|
128
|
+
|
129
|
+
{% include helpers/base64.md %}
|
@@ -0,0 +1,76 @@
|
|
1
|
+
---
|
2
|
+
title: AWS SSM Parameters
|
3
|
+
nav_text: SSM
|
4
|
+
categories: helpers-aws
|
5
|
+
---
|
6
|
+
|
7
|
+
For example if you have these secret values:
|
8
|
+
|
9
|
+
$ aws ssm get-parameter --name /demo/development/db_user --with-decryption | jq '.Parameter.Value'
|
10
|
+
user
|
11
|
+
$ aws ssm get-parameter --name /demo/development/db_pass --with-decryption | jq '.Parameter.Value'
|
12
|
+
pass
|
13
|
+
|
14
|
+
Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
|
15
|
+
|
16
|
+
.kubes/config/hooks/kubes.rb
|
17
|
+
|
18
|
+
```ruby
|
19
|
+
ssm = KubesAws::SSM.new(upcase: true, prefix: "/demo/development/")
|
20
|
+
before("compile",
|
21
|
+
label: "Get secrets from AWS SSM Manager",
|
22
|
+
execute: ssm,
|
23
|
+
)
|
24
|
+
```
|
25
|
+
|
26
|
+
Then set the secrets in the YAML:
|
27
|
+
|
28
|
+
.kubes/resources/shared/secret.yaml
|
29
|
+
|
30
|
+
```yaml
|
31
|
+
apiVersion: v1
|
32
|
+
kind: Secret
|
33
|
+
metadata:
|
34
|
+
name: demo
|
35
|
+
labels:
|
36
|
+
app: demo
|
37
|
+
data:
|
38
|
+
<% KubesAws::SSM.data.each do |k,v| -%>
|
39
|
+
<%= k %>: <%= base64(v) %>
|
40
|
+
<% end -%>
|
41
|
+
```
|
42
|
+
|
43
|
+
This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data. The values are automatically base64 encoded. Produces:
|
44
|
+
|
45
|
+
.kubes/output/shared/secret.yaml
|
46
|
+
|
47
|
+
```yaml
|
48
|
+
metadata:
|
49
|
+
namespace: demo
|
50
|
+
name: demo-2a78a13682
|
51
|
+
labels:
|
52
|
+
app: demo
|
53
|
+
apiVersion: v1
|
54
|
+
kind: Secret
|
55
|
+
data:
|
56
|
+
db_pass: dGVzdDEK
|
57
|
+
db_user: dGVzdDIK
|
58
|
+
```
|
59
|
+
|
60
|
+
## Variables
|
61
|
+
|
62
|
+
These environment variables can be set:
|
63
|
+
|
64
|
+
Name | Description
|
65
|
+
---|---
|
66
|
+
AWS_SSM_PREFIX | Prefixed used to list and filter AWS SSM Parameters. IE: `demo/dev/`.
|
67
|
+
|
68
|
+
Secrets#initialize options:
|
69
|
+
|
70
|
+
Variable | Description | Default
|
71
|
+
---|---|---
|
72
|
+
base64 | Automatically base64 encode the values. | false
|
73
|
+
upcase | Automatically upcase the Kubernetes secret data keys. | false
|
74
|
+
prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
|
75
|
+
|
76
|
+
{% include helpers/base64.md %}
|
@@ -0,0 +1,40 @@
|
|
1
|
+
---
|
2
|
+
title: Custom Helpers
|
3
|
+
---
|
4
|
+
|
5
|
+
Kubes ships with several built-in helpers. On top of this, you can define your own custom helpers. This allows you to define new methods and customize Kubes further.
|
6
|
+
|
7
|
+
## Example
|
8
|
+
|
9
|
+
You define custom helpers in the `.kubes/helpers` folder.
|
10
|
+
|
11
|
+
.kubes/helpers/my_helpers.rb
|
12
|
+
|
13
|
+
```ruby
|
14
|
+
module MyHelpers
|
15
|
+
def database_endpoint
|
16
|
+
case Kubes.env
|
17
|
+
when "dev"
|
18
|
+
"dev-db.cbuqdmc3nqvb.us-west-2.rds.amazonaws.com"
|
19
|
+
when "prod"
|
20
|
+
"prod-db.cbuqdmc3nqvb.us-west-2.rds.amazonaws.com"
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
24
|
+
```
|
25
|
+
|
26
|
+
The `database_endpoint` will be available to use in the `.kubes/resources` YAML files. IE:
|
27
|
+
|
28
|
+
.kubes/helpers/resources/shared/config_map.yaml
|
29
|
+
|
30
|
+
```yaml
|
31
|
+
apiVersion: v1
|
32
|
+
kind: ConfigMap
|
33
|
+
metadata:
|
34
|
+
name: demo
|
35
|
+
labels:
|
36
|
+
app: demo
|
37
|
+
data:
|
38
|
+
DATABASE_ENDPOINT: <%= database_endpoint %>
|
39
|
+
```
|
40
|
+
|
@@ -0,0 +1,17 @@
|
|
1
|
+
---
|
2
|
+
title: Google Helpers
|
3
|
+
---
|
4
|
+
|
5
|
+
List of Google helpers:
|
6
|
+
|
7
|
+
{% assign docs = site.docs | where: "categories","helpers-aws" %}
|
8
|
+
{% for doc in docs -%}
|
9
|
+
* [{{ doc.nav_text }}]({{ doc.url }})
|
10
|
+
{% endfor %}
|
11
|
+
|
12
|
+
## Notes
|
13
|
+
|
14
|
+
* By default, `KubeGoogle.logger = Kubes.logger`. This means, you can set `logger.level = "debug"` in `.kubes/config.rb` to see more details.
|
15
|
+
* The `gcloud` cli is used to create IAM roles. So `gcloud` is required.
|
16
|
+
* Note: Would like to use the google sdk, but it wasn't obvious how to do so. PRs are welcomed.
|
17
|
+
* The Google helpers are provided by the [boltops-tools/kubes_google](https://github.com/boltops-tools/kubes_google) library.
|
@@ -0,0 +1,76 @@
|
|
1
|
+
---
|
2
|
+
title: Google Secrets
|
3
|
+
nav_text: Secrets
|
4
|
+
categories: helpers-google
|
5
|
+
---
|
6
|
+
|
7
|
+
Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
|
8
|
+
|
9
|
+
.kubes/config/hooks/kubes.rb
|
10
|
+
|
11
|
+
```ruby
|
12
|
+
before("compile",
|
13
|
+
execute: KubesGoogle::Secrets.new(upcase: true, prefix: 'projects/686010496118/secrets/demo-dev-')
|
14
|
+
)
|
15
|
+
```
|
16
|
+
|
17
|
+
Then set the secrets in the YAML:
|
18
|
+
|
19
|
+
.kubes/resources/shared/secret.yaml
|
20
|
+
|
21
|
+
```yaml
|
22
|
+
apiVersion: v1
|
23
|
+
kind: Secret
|
24
|
+
metadata:
|
25
|
+
name: demo
|
26
|
+
labels:
|
27
|
+
app: demo
|
28
|
+
data:
|
29
|
+
<% KubesGoogle::Secrets.data.each do |k,v| -%>
|
30
|
+
<%= k %>: <%= base64(v) %>
|
31
|
+
<% end -%>
|
32
|
+
```
|
33
|
+
|
34
|
+
This results in Google secrets with the prefix the `demo-dev-` being added to the Kubernetes secret data. The values are automatically base64 encoded.
|
35
|
+
|
36
|
+
For example if you have these secret values:
|
37
|
+
|
38
|
+
$ gcloud secrets versions access latest --secret demo-dev-db_user
|
39
|
+
test1
|
40
|
+
$ gcloud secrets versions access latest --secret demo-dev-db_pass
|
41
|
+
test2
|
42
|
+
$
|
43
|
+
|
44
|
+
.kubes/output/shared/secret.yaml
|
45
|
+
|
46
|
+
```yaml
|
47
|
+
metadata:
|
48
|
+
namespace: demo
|
49
|
+
name: demo-2a78a13682
|
50
|
+
labels:
|
51
|
+
app: demo
|
52
|
+
apiVersion: v1
|
53
|
+
kind: Secret
|
54
|
+
data:
|
55
|
+
db_pass: dGVzdDEK
|
56
|
+
db_user: dGVzdDIK
|
57
|
+
```
|
58
|
+
|
59
|
+
## Variables
|
60
|
+
|
61
|
+
These environment variables can be set:
|
62
|
+
|
63
|
+
Name | Description
|
64
|
+
---|---
|
65
|
+
GCP_SECRET_PREFIX | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`.
|
66
|
+
GOOGLE_PROJECT | Google project id.
|
67
|
+
|
68
|
+
Secrets#initialize options:
|
69
|
+
|
70
|
+
Variable | Description | Default
|
71
|
+
---|---|---
|
72
|
+
base64 | Automatically base64 encode the values. | false
|
73
|
+
upcase | Automatically upcase the Kubernetes secret data keys. | false
|
74
|
+
prefix | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`. Can also be set with the `GCP_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
|
75
|
+
|
76
|
+
{% include helpers/base64.md %}
|
@@ -0,0 +1,52 @@
|
|
1
|
+
---
|
2
|
+
title: Google Service Account
|
3
|
+
nav_text: Service Account
|
4
|
+
categories: helpers-google
|
5
|
+
---
|
6
|
+
|
7
|
+
## Service Accounts
|
8
|
+
|
9
|
+
You can automatically create the Google Service Account associated with the [GKE Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity).
|
10
|
+
|
11
|
+
Here's a Kubes hook that creates a service account:
|
12
|
+
|
13
|
+
.kubes/config/hooks/kubes.rb
|
14
|
+
|
15
|
+
```ruby
|
16
|
+
service_account = KubesGoogle::ServiceAccount.new(
|
17
|
+
app: "demo",
|
18
|
+
namespace: "demo-#{Kubes.env}", # defaults to APP-ENV when not set. IE: demo-dev
|
19
|
+
roles: ["cloudsql.client", "secretmanager.viewer"], # defaults to empty when not set
|
20
|
+
)
|
21
|
+
before("apply",
|
22
|
+
label: "create service account",
|
23
|
+
execute: service_account,
|
24
|
+
)
|
25
|
+
```
|
26
|
+
|
27
|
+
The corresponding Kubernetes Service account looks like this:
|
28
|
+
|
29
|
+
.kubes/resources/shared/service_account.yaml
|
30
|
+
|
31
|
+
```yaml
|
32
|
+
apiVersion: v1
|
33
|
+
kind: ServiceAccount
|
34
|
+
metadata:
|
35
|
+
annotations:
|
36
|
+
iam.gke.io/gcp-service-account: demo-<%= Kubes.env %>@<%= ENV['GOOGLE_PROJECT'] %>.iam.gserviceaccount.com
|
37
|
+
name: demo
|
38
|
+
labels:
|
39
|
+
app: demo
|
40
|
+
```
|
41
|
+
|
42
|
+
The role permissions are currently always added to the existing permissions. So removing roles that were previously added does not remove them.
|
43
|
+
|
44
|
+
ServiceAccount#initialize options:
|
45
|
+
|
46
|
+
Variable | Description | Default
|
47
|
+
---|---|---
|
48
|
+
app | The app name. It's used to set other variables conventionally. This is required. | nil
|
49
|
+
gsa | The Google Service Account name. The conventional name is APP-ENV. IE: demo-dev. | APP-ENV
|
50
|
+
ksa | The Kubernetes Service Account name. The conventional name is APP. IE: demo | APP
|
51
|
+
namespace | The Kubernetes namespace. Defaults to the APP-ENV. IE: demo-dev. | APP-ENV
|
52
|
+
roles | Google IAM roles to add. This adds permissions to the Google service account. | []
|
@@ -78,5 +78,3 @@ upcase | Automatically upcase the Kubernetes secret data keys. | false
|
|
78
78
|
prefix | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`. Can also be set with the `GCP_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
|
79
79
|
|
80
80
|
Note, Kubernetes secrets are only base64 encoded. So users who have access to read Kubernetes secrets will be able to decode and get the value trivially. Depending on your security posture requirements, this may or may not suffice.
|
81
|
-
|
82
|
-
The Google helpers are provided by the [boltops-tools/kubes_google](https://github.com/boltops-tools/kubes_google) library. For more details, check out its README.
|
@@ -0,0 +1 @@
|
|
1
|
+
Note, Kubernetes secrets are only base64 encoded. So users who have access to read Kubernetes secrets will be able to decode and get the value trivially. Depending on your security posture requirements, this may or may not suffice.
|
@@ -27,7 +27,7 @@ To explain the layering, here's the general processing order that Kubes takes.
|
|
27
27
|
|
28
28
|
Note, both YAML and DSL forms support layering.
|
29
29
|
|
30
|
-
Layering only combines resources definitions with the same form. For example, `base/all.rb` will not be combined with `web/deployment.yaml`.
|
30
|
+
Layering only combines resources definitions with the same form. For example, the DSL form `base/all.rb` will not be combined with YAML form `web/deployment.yaml`.
|
31
31
|
|
32
32
|
## Full Layering
|
33
33
|
|
data/docs/_includes/sidebar.html
CHANGED
@@ -97,7 +97,27 @@
|
|
97
97
|
<li><a href="{% link _docs/dsl/multiple-resources.md %}">Multiple Resources</a>
|
98
98
|
</ul>
|
99
99
|
</li>
|
100
|
-
<li><a href="{% link _docs/helpers.md %}">Helpers</a
|
100
|
+
<li><a href="{% link _docs/helpers.md %}">Helpers</a>
|
101
|
+
<ul>
|
102
|
+
<li><a href="{% link _docs/helpers/custom.md %}">Custom</a></li>
|
103
|
+
<li><a href="{% link _docs/helpers/aws.md %}">AWS</a>
|
104
|
+
<ul>
|
105
|
+
{% assign docs = site.docs | where: "categories","helpers-aws" %}
|
106
|
+
{% for doc in docs -%}
|
107
|
+
<li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
|
108
|
+
{% endfor %}
|
109
|
+
</ul>
|
110
|
+
</li>
|
111
|
+
<li><a href="{% link _docs/helpers/google.md %}">Google</a>
|
112
|
+
<ul>
|
113
|
+
{% assign docs = site.docs | where: "categories","helpers-google" %}
|
114
|
+
{% for doc in docs -%}
|
115
|
+
<li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
|
116
|
+
{% endfor %}
|
117
|
+
</ul>
|
118
|
+
</li>
|
119
|
+
</ul>
|
120
|
+
</li>
|
101
121
|
<li><a href="{% link _docs/patterns.md %}">Patterns</a>
|
102
122
|
<ul>
|
103
123
|
{% assign docs = site.docs | where: "categories","patterns" %}
|
data/kubes.gemspec
CHANGED
data/lib/kubes.rb
CHANGED
@@ -15,9 +15,10 @@ require "rainbow/ext/string"
|
|
15
15
|
require "yaml"
|
16
16
|
|
17
17
|
# core helper libraries
|
18
|
+
require "kubes_aws"
|
18
19
|
require "kubes_google"
|
19
20
|
|
20
|
-
DslEvaluator.backtrace_reject = "
|
21
|
+
DslEvaluator.backtrace_reject = "lib/kubes"
|
21
22
|
|
22
23
|
require "kubes/autoloader"
|
23
24
|
Kubes::Autoloader.setup
|
data/lib/kubes/autoloader.rb
CHANGED
@@ -14,8 +14,17 @@ module Kubes
|
|
14
14
|
loader = Zeitwerk::Loader.new
|
15
15
|
loader.inflector = Inflector.new
|
16
16
|
loader.push_dir(File.dirname(__dir__)) # lib
|
17
|
+
|
18
|
+
helpers = "#{kubes_root}/.kubes/helpers"
|
19
|
+
loader.push_dir(helpers) if File.exist?(helpers) # project helpers
|
20
|
+
|
17
21
|
loader.setup
|
18
22
|
end
|
23
|
+
|
24
|
+
# Autoloader runs so early that Kubes.root is not available, so we must declare it here
|
25
|
+
def kubes_root
|
26
|
+
ENV['KUBES_ROOT'] || '.'
|
27
|
+
end
|
19
28
|
end
|
20
29
|
end
|
21
30
|
end
|
@@ -0,0 +1,17 @@
|
|
1
|
+
module Kubes::Compiler::Shared
|
2
|
+
module CustomHelpers
|
3
|
+
# Load custom helper methods from project
|
4
|
+
@@custom_helpers_loaded = false
|
5
|
+
def load_custom_helpers
|
6
|
+
return if @@custom_helpers_loaded
|
7
|
+
paths = Dir.glob("#{Kubes.root}/.kubes/helpers/**/*.rb")
|
8
|
+
paths.sort_by! { |p| p.size } # so namespaces are loaded first
|
9
|
+
paths.each do |path|
|
10
|
+
filename = path.sub(%r{.*.kubes/helpers/},'').sub('.rb','')
|
11
|
+
module_name = filename.camelize
|
12
|
+
self.class.send :include, module_name.constantize
|
13
|
+
end
|
14
|
+
@@custom_helpers_loaded = true
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
@@ -1,13 +1,13 @@
|
|
1
1
|
class Kubes::Compiler::Strategy
|
2
2
|
class Base
|
3
3
|
include Kubes::Logging
|
4
|
+
include Kubes::Compiler::Util::SaveFile
|
5
|
+
include Kubes::Compiler::Shared::CustomHelpers
|
4
6
|
|
5
7
|
def initialize(options={})
|
6
8
|
@options = options
|
7
9
|
@path = options[:path]
|
8
|
-
|
9
|
-
@filename = @path.sub(%r{.*\.kubes/resources/},'') # IE: web/deployment.rb or web/deployment.yaml
|
10
|
-
@save_file = @filename.sub('.yml','.yaml').sub('.rb','.yaml')
|
10
|
+
@save_file = save_file(@path)
|
11
11
|
end
|
12
12
|
end
|
13
13
|
end
|
@@ -3,6 +3,7 @@ class Kubes::Compiler::Strategy
|
|
3
3
|
include Kubes::Compiler::Util::Normalize
|
4
4
|
|
5
5
|
def run
|
6
|
+
load_custom_helpers
|
6
7
|
dsl = dsl_class.new(@options) # Deployment, Service, etc
|
7
8
|
data = dsl.run
|
8
9
|
Result.new(@save_file, data)
|
@@ -17,7 +18,7 @@ class Kubes::Compiler::Strategy
|
|
17
18
|
end
|
18
19
|
|
19
20
|
def syntax_class
|
20
|
-
klass_name = normalize_kind(@
|
21
|
+
klass_name = normalize_kind(@save_file)
|
21
22
|
"Kubes::Compiler::Dsl::Syntax::#{klass_name}".constantize
|
22
23
|
rescue NameError
|
23
24
|
logger.debug "Using default resource for: #{klass_name}"
|
@@ -25,7 +26,7 @@ class Kubes::Compiler::Strategy
|
|
25
26
|
end
|
26
27
|
|
27
28
|
def block_form?
|
28
|
-
type = extract_type(@
|
29
|
+
type = extract_type(@save_file)
|
29
30
|
type.pluralize == type
|
30
31
|
end
|
31
32
|
end
|
@@ -4,10 +4,12 @@ class Kubes::Compiler::Strategy
|
|
4
4
|
class Erb < Base
|
5
5
|
extend Kubes::Compiler::Dsl::Core::Fields
|
6
6
|
include Kubes::Compiler::Dsl::Core::Helpers
|
7
|
+
include Kubes::Compiler::Shared::CustomHelpers
|
7
8
|
include Kubes::Compiler::Shared::Helpers
|
8
9
|
include Kubes::Compiler::Layering
|
9
10
|
|
10
11
|
def run
|
12
|
+
load_custom_helpers
|
11
13
|
@data = {}
|
12
14
|
|
13
15
|
render_files(pre_layers)
|
@@ -32,11 +34,18 @@ class Kubes::Compiler::Strategy
|
|
32
34
|
def render_result(path)
|
33
35
|
if File.exist?(path)
|
34
36
|
yaml = RenderMePretty.result(path, context: self)
|
35
|
-
result =
|
37
|
+
result = yaml_load(path, yaml)
|
36
38
|
result.is_a?(Hash) ? result : {} # in case of blank yaml doc a Boolean false is returned
|
37
39
|
else
|
38
40
|
{}
|
39
41
|
end
|
40
42
|
end
|
43
|
+
|
44
|
+
def yaml_load(path, yaml)
|
45
|
+
YAML.load(yaml)
|
46
|
+
rescue Psych::SyntaxError
|
47
|
+
YamlError.new(path, yaml).show
|
48
|
+
exit 1
|
49
|
+
end
|
41
50
|
end
|
42
51
|
end
|
@@ -0,0 +1,60 @@
|
|
1
|
+
class Kubes::Compiler::Strategy::Erb
|
2
|
+
class YamlError
|
3
|
+
include Kubes::Compiler::Util::SaveFile
|
4
|
+
|
5
|
+
def initialize(path, rendered_yaml)
|
6
|
+
@path, @rendered_yaml = path, rendered_yaml
|
7
|
+
end
|
8
|
+
|
9
|
+
def show
|
10
|
+
FileUtils.mkdir_p(File.dirname(dest))
|
11
|
+
IO.write(dest, @rendered_yaml)
|
12
|
+
show_error(dest)
|
13
|
+
end
|
14
|
+
|
15
|
+
def dest
|
16
|
+
save_file = save_file(@path)
|
17
|
+
"#{Kubes.root}/.kubes/output/#{save_file}"
|
18
|
+
end
|
19
|
+
|
20
|
+
def show_error(path)
|
21
|
+
text = IO.read(path)
|
22
|
+
begin
|
23
|
+
YAML.load(text)
|
24
|
+
rescue Psych::SyntaxError => e
|
25
|
+
handle_yaml_syntax_error(e, path)
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
def handle_yaml_syntax_error(e, path)
|
30
|
+
io = StringIO.new
|
31
|
+
io.puts "Invalid yaml. Output written for debugging: #{path}".color(:red)
|
32
|
+
io.puts "ERROR: #{e.message}".color(:red)
|
33
|
+
|
34
|
+
# Grab line info. Example error:
|
35
|
+
# ERROR: (<unknown>): could not find expected ':' while scanning a simple key at line 2 column 1
|
36
|
+
md = e.message.match(/at line (\d+) column (\d+)/)
|
37
|
+
line = md[1].to_i
|
38
|
+
|
39
|
+
lines = IO.read(path).split("\n")
|
40
|
+
context = 5 # lines of context
|
41
|
+
top, bottom = [line-context-1, 0].max, line+context-1
|
42
|
+
spacing = lines.size.to_s.size
|
43
|
+
lines[top..bottom].each_with_index do |line_content, index|
|
44
|
+
line_number = top+index+1
|
45
|
+
if line_number == line
|
46
|
+
io.printf("%#{spacing}d %s\n".color(:red), line_number, line_content)
|
47
|
+
else
|
48
|
+
io.printf("%#{spacing}d %s\n", line_number, line_content)
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
if ENV['KUBES_TEST']
|
53
|
+
io.string
|
54
|
+
else
|
55
|
+
puts io.string
|
56
|
+
exit 1
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
data/lib/kubes/hooks/builder.rb
CHANGED
data/lib/kubes/kubectl.rb
CHANGED
@@ -17,13 +17,11 @@ module Kubes
|
|
17
17
|
params = args.flatten.join(' ')
|
18
18
|
args = "#{@name} #{params}" # @name: apply or delete
|
19
19
|
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
self.class.execute(args, options)
|
26
|
-
end
|
20
|
+
run_hooks("kubectl.rb", name: @name, file: @options[:file]) do
|
21
|
+
if options[:capture]
|
22
|
+
self.class.capture(args, options) # already includes kubectl
|
23
|
+
else
|
24
|
+
self.class.execute(args, options)
|
27
25
|
end
|
28
26
|
end
|
29
27
|
end
|
@@ -44,22 +42,6 @@ module Kubes
|
|
44
42
|
Kubes.config.kubectl.exit_on_fail[@name]
|
45
43
|
end
|
46
44
|
|
47
|
-
def switch_context(&block)
|
48
|
-
kubectl = Kubes.config.kubectl
|
49
|
-
context = kubectl.context
|
50
|
-
if context
|
51
|
-
previous_context = capture("kubectl config current-context")
|
52
|
-
sh("kubectl config use-context #{context}", mute: true)
|
53
|
-
result = block.call
|
54
|
-
if !previous_context.blank? && !kubectl.context_keep
|
55
|
-
sh("kubectl config use-context #{previous_context}", mute: true)
|
56
|
-
end
|
57
|
-
result
|
58
|
-
else
|
59
|
-
block.call
|
60
|
-
end
|
61
|
-
end
|
62
|
-
|
63
45
|
def args
|
64
46
|
# base at end in case of redirection. IE: command > /path
|
65
47
|
custom.args + default.args
|
data/lib/kubes/kubectl/batch.rb
CHANGED
@@ -3,6 +3,7 @@ class Kubes::Kubectl
|
|
3
3
|
include Kubes::Hooks::Concern
|
4
4
|
include Kubes::Logging
|
5
5
|
include Kubes::Util::Consider
|
6
|
+
include Kubes::Util::Sh
|
6
7
|
include Ordering
|
7
8
|
|
8
9
|
def initialize(name, options={})
|
@@ -12,17 +13,43 @@ class Kubes::Kubectl
|
|
12
13
|
def run
|
13
14
|
# @options[:preview] is really only used for kubectl delete
|
14
15
|
logger.info "Will run:" if @options[:preview]
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
16
|
+
switch_context do
|
17
|
+
run_hooks("kubes.rb", name: @name) do
|
18
|
+
sorted_files.each do |file|
|
19
|
+
if @options[:preview]
|
20
|
+
logger.info " kubectl #{@name} -f #{file}"
|
21
|
+
else
|
22
|
+
Kubes::Kubectl.run(@name, @options.merge(file: file))
|
23
|
+
end
|
21
24
|
end
|
22
25
|
end
|
23
26
|
end
|
24
27
|
end
|
25
28
|
|
29
|
+
def switch_context(&block)
|
30
|
+
kubectl = Kubes.config.kubectl
|
31
|
+
context = kubectl.context
|
32
|
+
|
33
|
+
unless context
|
34
|
+
block.call
|
35
|
+
return
|
36
|
+
end
|
37
|
+
|
38
|
+
previous_context = sh_capture("kubectl config current-context")
|
39
|
+
if previous_context == context
|
40
|
+
block.call
|
41
|
+
return
|
42
|
+
end
|
43
|
+
|
44
|
+
logger.debug "Switching kubectl context to: #{context}"
|
45
|
+
sh("kubectl config use-context #{context}", mute: true)
|
46
|
+
result = block.call
|
47
|
+
if !previous_context.blank? && !kubectl.context_keep
|
48
|
+
sh("kubectl config use-context #{previous_context}", mute: true)
|
49
|
+
end
|
50
|
+
result
|
51
|
+
end
|
52
|
+
|
26
53
|
# kubes apply # {role: nil, resource: nil}
|
27
54
|
# kubes apply clock # {role: "clock", resource: nil}
|
28
55
|
# kubes apply clock deployment # {role: "clock", resource: "deployment"}
|
data/lib/kubes/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: kubes
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.4.
|
4
|
+
version: 0.4.7
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tung Nguyen
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-10-
|
11
|
+
date: 2020-10-28 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -136,6 +136,20 @@ dependencies:
|
|
136
136
|
- - ">="
|
137
137
|
- !ruby/object:Gem::Version
|
138
138
|
version: '0'
|
139
|
+
- !ruby/object:Gem::Dependency
|
140
|
+
name: kubes_aws
|
141
|
+
requirement: !ruby/object:Gem::Requirement
|
142
|
+
requirements:
|
143
|
+
- - ">="
|
144
|
+
- !ruby/object:Gem::Version
|
145
|
+
version: '0'
|
146
|
+
type: :runtime
|
147
|
+
prerelease: false
|
148
|
+
version_requirements: !ruby/object:Gem::Requirement
|
149
|
+
requirements:
|
150
|
+
- - ">="
|
151
|
+
- !ruby/object:Gem::Version
|
152
|
+
version: '0'
|
139
153
|
- !ruby/object:Gem::Dependency
|
140
154
|
name: kubes_google
|
141
155
|
requirement: !ruby/object:Gem::Requirement
|
@@ -285,6 +299,14 @@ files:
|
|
285
299
|
- docs/_docs/extra-env/dsl.md
|
286
300
|
- docs/_docs/extra-env/yaml.md
|
287
301
|
- docs/_docs/helpers.md
|
302
|
+
- docs/_docs/helpers/aws.md
|
303
|
+
- docs/_docs/helpers/aws/iam-role.md
|
304
|
+
- docs/_docs/helpers/aws/secrets.md
|
305
|
+
- docs/_docs/helpers/aws/ssm.md
|
306
|
+
- docs/_docs/helpers/custom.md
|
307
|
+
- docs/_docs/helpers/google.md
|
308
|
+
- docs/_docs/helpers/google/secrets.md
|
309
|
+
- docs/_docs/helpers/google/service-account.md
|
288
310
|
- docs/_docs/intro.md
|
289
311
|
- docs/_docs/intro/concepts.md
|
290
312
|
- docs/_docs/intro/how-kubes-works.md
|
@@ -339,6 +361,7 @@ files:
|
|
339
361
|
- docs/_includes/footer.html
|
340
362
|
- docs/_includes/google_analytics.html
|
341
363
|
- docs/_includes/header.html
|
364
|
+
- docs/_includes/helpers/base64.md
|
342
365
|
- docs/_includes/intro/install.md
|
343
366
|
- docs/_includes/js.html
|
344
367
|
- docs/_includes/kubes-steps.md
|
@@ -535,14 +558,17 @@ files:
|
|
535
558
|
- lib/kubes/compiler/dsl/syntax/service.rb
|
536
559
|
- lib/kubes/compiler/dsl/syntax/service_account.rb
|
537
560
|
- lib/kubes/compiler/layering.rb
|
561
|
+
- lib/kubes/compiler/shared/custom_helpers.rb
|
538
562
|
- lib/kubes/compiler/shared/helpers.rb
|
539
563
|
- lib/kubes/compiler/strategy.rb
|
540
564
|
- lib/kubes/compiler/strategy/base.rb
|
541
565
|
- lib/kubes/compiler/strategy/dsl.rb
|
542
566
|
- lib/kubes/compiler/strategy/erb.rb
|
567
|
+
- lib/kubes/compiler/strategy/erb/yaml_error.rb
|
543
568
|
- lib/kubes/compiler/strategy/pass.rb
|
544
569
|
- lib/kubes/compiler/strategy/result.rb
|
545
570
|
- lib/kubes/compiler/util/normalize.rb
|
571
|
+
- lib/kubes/compiler/util/save_file.rb
|
546
572
|
- lib/kubes/compiler/util/yaml_dump.rb
|
547
573
|
- lib/kubes/completer.rb
|
548
574
|
- lib/kubes/completer/script.rb
|