consul-templaterb 1.0.5 → 1.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +36 -0
- data/README.md +60 -29
- data/bin/consul-templaterb +4 -0
- data/consul-templaterb.gemspec +7 -3
- data/lib/consul/async/version.rb +1 -1
- data/samples/services.html.erb +0 -17
- metadata +10 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c585b885aba40b8fcb72ac48c82f4dfd90ea4cbb9949976aa33eecf8a6a109a3
|
4
|
+
data.tar.gz: 1a35639786fb7c3dad6f23bd890816763997f025299a99608291212cc8c07695
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 50afc5d1cdfb2aa269673be86561dc4c8fde3334f9de4b2a9361714f1e35b47a0235d828a17c93d6e50345873bbb396075bd0762f6ad7062fe97ba51eda1c1c3
|
7
|
+
data.tar.gz: 9286bd23cf9d41e01b0585521eb0e7c41bead41a82b571baa687e7f3734fdfd238cce67b7b45bcadbe2565111dc118b49d659061ec20a9deaa2d0dd6fa0bf016
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,36 @@
|
|
1
|
+
## (UNRELEASED)
|
2
|
+
|
3
|
+
## 1.0.6 (March 16, 2018)
|
4
|
+
|
5
|
+
IMPROVEMENTS:
|
6
|
+
|
7
|
+
* Adds http:// to Consul URL if missing since `$CONSUL_HTTP_ADDR` environment
|
8
|
+
variable might not have it
|
9
|
+
* Updated gem description with more accurate information
|
10
|
+
* samples: removed KV/nodes from services.html.erb
|
11
|
+
* Fixed typos in README.md
|
12
|
+
* Added CHANGELOG.md
|
13
|
+
|
14
|
+
## 1.0.5 (March 12, 2018)
|
15
|
+
|
16
|
+
BUG FIXES:
|
17
|
+
|
18
|
+
* fixed default value for parameter -c to http://localhost:8500
|
19
|
+
|
20
|
+
## 1.0.4 (March 12, 2018)
|
21
|
+
|
22
|
+
BUG FIXES:
|
23
|
+
|
24
|
+
* First auto-built by Travis CI and fully deployed version
|
25
|
+
* Fixed String interpolation bug in help messages
|
26
|
+
* Bump rubocop to version 0.49.0 due to CVE (Travis CI)
|
27
|
+
|
28
|
+
IMPROVEMENTS:
|
29
|
+
|
30
|
+
* Various README.md improvements
|
31
|
+
|
32
|
+
## 1.0.0 -> 1.0.3 (March 12, 2018) First public release
|
33
|
+
|
34
|
+
BUG FIXES:
|
35
|
+
|
36
|
+
* Allow Travis to build properly using tags and publish to rubygems.org
|
data/README.md
CHANGED
@@ -2,13 +2,13 @@
|
|
2
2
|
|
3
3
|
This GEM is both a library and an executable that allows to generate files
|
4
4
|
using data from Consul (Discovery and Key/Value Store) easily using ruby's
|
5
|
-
erb templates. It also support launching programs and
|
5
|
+
erb templates. It also support launching programs and baby-sitting processes
|
6
6
|
when rendering the files, thus notifying programs when data do change.
|
7
7
|
|
8
8
|
It is intended for user accustomed to expressiveness or Ruby templating (ERB),
|
9
|
-
allowing for more flexibility and than Go
|
9
|
+
allowing for more flexibility and features than Go templates.
|
10
10
|
|
11
|
-
It also
|
11
|
+
It also allows to use all of ruby language, especially useful for generating
|
12
12
|
files in several formats (JSON, XML) where text substitutions are hard to get
|
13
13
|
right.
|
14
14
|
|
@@ -16,7 +16,7 @@ It also focuses on good performance and lightweight usage of bandwidth,
|
|
16
16
|
especially for very large clusters and watching lots of services.
|
17
17
|
|
18
18
|
For complicated rendering of templates and large Consul Clusters, it usually
|
19
|
-
|
19
|
+
renders faster with a more predictable way the template than the original
|
20
20
|
consul-template.
|
21
21
|
|
22
22
|
## Differences with HashiCorp's consul-template
|
@@ -31,12 +31,12 @@ has support for Hashicorp's Vault), but consul-templaterb focuses on getting
|
|
31
31
|
more power with the generation of templates and more performance.
|
32
32
|
|
33
33
|
Consul Template uses Go templates which is very limited in its set of
|
34
|
-
features
|
34
|
+
features: it is complicated to sort, apply real transformations
|
35
35
|
using code and even interact with the OS (ex: get the current date, format
|
36
36
|
timestamps...).
|
37
37
|
|
38
|
-
The sort feature for instances allow you to create
|
39
|
-
that the order of nodes is
|
38
|
+
The sort feature for instances allow you to create predictable output (i.e: meaning
|
39
|
+
that the order of nodes is predictable), thus it might offer better performance
|
40
40
|
since the reload of processes if happening ONLY when the files are binary
|
41
41
|
different. Thus, if using consul-templaterb, you will reload less your haproxy or
|
42
42
|
load-balancer than you would do with consul-template.
|
@@ -44,19 +44,19 @@ load-balancer than you would do with consul-template.
|
|
44
44
|
Compared to consul-template, consul-templaterb offers the following features:
|
45
45
|
|
46
46
|
* Hot-Reload of template files
|
47
|
-
*
|
47
|
+
* Bandwidth limitation per endpoint (will soon support dynamic bandwidth limiter)
|
48
48
|
* Supports baby sitting of multiple processes
|
49
49
|
* Supports all Ruby features (ex: base64, real JSON/XML generation...)
|
50
50
|
* Information about bandwidth
|
51
51
|
|
52
52
|
The executable supports close semantics to Consul template, it also supports
|
53
|
-
commands when files are modified and
|
54
|
-
ability to send signals to those processes
|
53
|
+
commands when files are modified and supervision of multiple processes with
|
54
|
+
ability to send signals to those processes when the files do change.
|
55
55
|
|
56
56
|
## Installation
|
57
57
|
|
58
|
-
You might either use the executable
|
59
|
-
adding this line to your application's Gemfile
|
58
|
+
You might either use the executable directly OR use this GEM as a library by
|
59
|
+
adding this line to your application's `Gemfile`:
|
60
60
|
|
61
61
|
```ruby
|
62
62
|
gem 'consul-templaterb'
|
@@ -65,7 +65,7 @@ gem 'consul-templaterb'
|
|
65
65
|
And then execute:
|
66
66
|
|
67
67
|
```shell
|
68
|
-
|
68
|
+
$ bundle
|
69
69
|
[...]
|
70
70
|
```
|
71
71
|
|
@@ -79,13 +79,34 @@ $ gem install consul-templaterb
|
|
79
79
|
If you simply want to use the executable on your favorite linux distribution, you
|
80
80
|
have to install first: ruby and ruby dev.
|
81
81
|
|
82
|
-
### Quick install on Ubuntu
|
82
|
+
### Quick install on Ubuntu-Linux
|
83
83
|
|
84
84
|
```shell
|
85
85
|
sudo apt-get install ruby ruby-dev && sudo gem install consul-templaterb
|
86
86
|
```
|
87
87
|
|
88
|
-
You can now use it directly using the binary `consul-templaterb` in your path
|
88
|
+
You can now use it directly using the binary `consul-templaterb` in your path.
|
89
|
+
|
90
|
+
### Playing with the samples templates
|
91
|
+
|
92
|
+
Samples are installed with the GEM, you can either
|
93
|
+
[download](https://github.com/criteo/consul-templaterb/tree/master/samples) them or
|
94
|
+
simply use the ones installed with the gem. To figure out where the templates are
|
95
|
+
installed:
|
96
|
+
|
97
|
+
```shell
|
98
|
+
$ gem contents consul-templaterb|grep samples
|
99
|
+
```
|
100
|
+
|
101
|
+
Will output the path where the samples are being installed, you can copy the directory
|
102
|
+
somewhere and then issue the command:
|
103
|
+
|
104
|
+
```shell
|
105
|
+
$ consul-templaterb samples/*.html.erb
|
106
|
+
```
|
107
|
+
|
108
|
+
It will render a full web site you may browse to look in real time the status of your
|
109
|
+
Consul Cluster.
|
89
110
|
|
90
111
|
## Usage of consul-templaterb
|
91
112
|
|
@@ -110,6 +131,14 @@ USAGE: bin/consul-templaterb [[options]]
|
|
110
131
|
--once Do not run the process as a daemon
|
111
132
|
```
|
112
133
|
|
134
|
+
When launched with file arguments ending with .erb, the executable will assume
|
135
|
+
the file is a template and will render the corresponding file without the
|
136
|
+
`.erb` extension.
|
137
|
+
|
138
|
+
It means that you can call consul-templaterb with *.erb arguments, the shell
|
139
|
+
will then substitute all files and render it by removing the .erb extension as
|
140
|
+
if the `--template my_file.ext.erb:myfile.ext` was used.
|
141
|
+
|
113
142
|
### Generate multiple templates
|
114
143
|
|
115
144
|
In the same way as consul-template, consul-templaterb supports multiple templates and executing
|
@@ -132,7 +161,7 @@ $ consul-templaterb \\
|
|
132
161
|
|
133
162
|
### Process management and signalisation of configuration files
|
134
163
|
|
135
|
-
With the
|
164
|
+
With the `--exec` argument (can be specified multiple times), consul-templaterb will launch
|
136
165
|
the process specified when all templates have been generated and will send a reload signal
|
137
166
|
if the content of any of the files do change (the signal will be sent atomically however,
|
138
167
|
meaning that if 2 results of templates are modified at the same time, the signal will be
|
@@ -154,7 +183,7 @@ By design, the GEM supports limiting the number of requests per endpoints (see c
|
|
154
183
|
`bin/consul-templaterb` file). It avoids using too much network to fetch data from Consul
|
155
184
|
in large Consul Clusters (especially when watching lots of files).
|
156
185
|
|
157
|
-
The limitation is
|
186
|
+
The limitation is static for now, but fair dynamic bandwidth allocation will allow to limit
|
158
187
|
the bandwidth used to get information for all services by capping the global bandwidth used
|
159
188
|
by consul-templaterb.
|
160
189
|
|
@@ -176,12 +205,12 @@ use (website updated automagically when values to change).
|
|
176
205
|
|
177
206
|
Here are the various functions you might use in your templates.
|
178
207
|
|
179
|
-
For each function, mandatory arguments are specified at the
|
208
|
+
For each function, mandatory arguments are specified at the beginning while optional ones are marked with `[]`.
|
180
209
|
Most of them support the optional dc attribute to access data from another datacenter. If the `dc`
|
181
210
|
attribute is not specified, the function will output data from the current datacenter.
|
182
211
|
|
183
212
|
To ease template development, `consul-templaterb` supports HOT reload of templates, thus it is possible to
|
184
|
-
develop the templates
|
213
|
+
develop the templates interactively. While developing, it is possible to use the switch `--hot-reload=keep`,
|
185
214
|
thus the application will display a warning if the template is invalid and won't stop
|
186
215
|
(`--hot-reload=die` is the default, thus if the hot-reloaded template has issue, the application will die).
|
187
216
|
|
@@ -228,15 +257,16 @@ mechanism.
|
|
228
257
|
|
229
258
|
### agent_self()
|
230
259
|
|
231
|
-
[Get the configuration of Consul Agent](https://www.consul.io/api/agent.html#read-configuration).
|
232
|
-
endpoint does not support blocking queries, data will be refreshed every few seconds,
|
233
|
-
queries mechanism.
|
260
|
+
[Get the configuration of Consul Agent](https://www.consul.io/api/agent.html#read-configuration).
|
261
|
+
Since this endpoint does not support blocking queries, data will be refreshed every few seconds,
|
262
|
+
but will not use blocking queries mechanism.
|
234
263
|
|
235
264
|
### render_file RELATIVE_PATH_TO_ERB_FILE
|
236
265
|
|
237
266
|
This allow to include a template file into another one. Beware, it does not check for infinite recursion!
|
238
267
|
The template can be either a static file either another template. The file has to be a valid template, but
|
239
|
-
can also be raw text (if it is a valid template
|
268
|
+
can also be raw text (if it is a valid template) and is resolved with a relative path regarding the file
|
269
|
+
including it.
|
240
270
|
|
241
271
|
Example:
|
242
272
|
|
@@ -250,16 +280,17 @@ We recommend using bundle using `bundle install`, you can now run `bundle exec b
|
|
250
280
|
|
251
281
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
|
252
282
|
version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
|
253
|
-
push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
283
|
+
push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org/gems/consul-templaterb).
|
254
284
|
|
255
285
|
## Known bugs
|
256
286
|
|
257
287
|
Here are the known bugs of the application:
|
258
288
|
|
259
289
|
* [ ] On Mac OS X, with many services, consul-templaterb sometimes crashes when watching lots of changes (more than 2k
|
260
|
-
watches) at the same time. This bug is
|
261
|
-
|
262
|
-
|
290
|
+
watches) at the same time. This bug is
|
291
|
+
[probably a race condition in `em-http-request`](https://github.com/igrigorik/em-http-request/issues/315). Only visible
|
292
|
+
on very large clusters or when watching thousands of individual KV keys.
|
293
|
+
* [ ] render_file might create an infinite recursion if a template includes itself indirectly.
|
263
294
|
|
264
295
|
## TODO
|
265
296
|
|
@@ -268,7 +299,7 @@ Here are the known bugs of the application:
|
|
268
299
|
* [ ] More samples: apache, nginx, full website displaying consul information...
|
269
300
|
* [ ] Optimize rendering speed at startup: an iteration is done very second by default, but it would be possible to speed
|
270
301
|
up rendering by iterating with higher frequency until the first write of result has been performed.
|
271
|
-
* [ ] Allow to tune
|
302
|
+
* [ ] Allow to tune bandwidth using a simple config file (while it should not be necessary for 90% of use-cases)
|
272
303
|
|
273
304
|
## Contributing
|
274
305
|
|
@@ -278,4 +309,4 @@ This project is intended to be a safe, welcoming space for collaboration, and co
|
|
278
309
|
|
279
310
|
## License
|
280
311
|
|
281
|
-
The gem is available as open source under the terms of the Apache v2 license. See LICENSE.txt file.
|
312
|
+
The gem is available as open source under the terms of the Apache v2 license. See [LICENSE.txt](LICENSE.txt) file.
|
data/bin/consul-templaterb
CHANGED
@@ -212,6 +212,10 @@ def find_max_descriptors(max_descripors)
|
|
212
212
|
max_size
|
213
213
|
end
|
214
214
|
|
215
|
+
unless options[:consul][:base_url].start_with? 'http'
|
216
|
+
options[:consul][:base_url] = "http://#{options[:consul][:base_url]}"
|
217
|
+
end
|
218
|
+
|
215
219
|
# Since we might be using a lots of descriptors, document this
|
216
220
|
new_size = find_max_descriptors(65_536)
|
217
221
|
STDERR.puts "Max number of descriptors set to #{new_size}" if options[:consul][:debug][:network]
|
data/consul-templaterb.gemspec
CHANGED
@@ -12,9 +12,13 @@ Gem::Specification.new do |spec|
|
|
12
12
|
spec.email = ['sre-core-services@criteo.com']
|
13
13
|
|
14
14
|
spec.summary = 'Implementation of Consul template using Ruby and .erb templating language'
|
15
|
-
spec.homepage = 'https://
|
16
|
-
spec.description = 'A ruby implementation of Consul Template with support of erb templating'
|
17
|
-
|
15
|
+
spec.homepage = 'https://rubygems.org/gems/consul-templaterb'
|
16
|
+
spec.description = 'A ruby implementation of Consul Template with support of erb templating ' \
|
17
|
+
'with hi-performance on large clusters and advanced process management features.'
|
18
|
+
spec.metadata = { 'bug_tracker_uri' => 'https://github.com/criteo/consul-templaterb/issues',
|
19
|
+
'changelog_uri' => 'https://github.com/criteo/consul-templaterb/blob/master/CHANGELOG.md',
|
20
|
+
'homepage_uri' => 'https://github.com/criteo/consul-templaterb',
|
21
|
+
'source_code_uri' => 'https://github.com/criteo/consul-templaterb' }
|
18
22
|
spec.license = 'Apache v2'
|
19
23
|
|
20
24
|
spec.files = `git ls-files -z`.split("\x0").reject do |f|
|
data/lib/consul/async/version.rb
CHANGED
data/samples/services.html.erb
CHANGED
@@ -25,8 +25,6 @@
|
|
25
25
|
<li><a href="#datacenters">DataCenters</a></li>
|
26
26
|
<li><a href="#list_services">List of services</a></li>
|
27
27
|
<li><a href="#services">Services with instances</a></li>
|
28
|
-
<li><a href="#nodes">All Nodes</a></li>
|
29
|
-
<li><a href="#kv">Key/Value Store</a></li>
|
30
28
|
</ul>
|
31
29
|
</nav>
|
32
30
|
<h1 id="datacenters">List of all datacenters</h1>
|
@@ -71,19 +69,4 @@
|
|
71
69
|
<% end%>
|
72
70
|
</ul>
|
73
71
|
<% end%>
|
74
|
-
|
75
|
-
|
76
|
-
<h1 id="nodes">List all nodes for DC, sorted by name</h1>
|
77
|
-
<ul>
|
78
|
-
<% nodes.sort {|a,b| a['Node'] <=> b['Node'] }.each do |snode|
|
79
|
-
%> <li id="node_#{snode['ID']"><%= snode['Address'].ljust(16) %> <%= snode['Node'] %></li>
|
80
|
-
<% end %>
|
81
|
-
</ul>
|
82
|
-
|
83
|
-
<h1 id="kv">KV of Current DC</h1>
|
84
|
-
<ul>
|
85
|
-
<% kv(keys:true).each do |key|
|
86
|
-
%><li id="kv_#{key}"><%= key %></li>
|
87
|
-
<% end %>
|
88
|
-
</ul>
|
89
72
|
<%= render_file 'common/footer.html.erb' %>
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: consul-templaterb
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0.
|
4
|
+
version: 1.0.6
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- SRE Core Services
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2018-03-
|
11
|
+
date: 2018-03-16 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: em-http-request
|
@@ -123,6 +123,7 @@ dependencies:
|
|
123
123
|
- !ruby/object:Gem::Version
|
124
124
|
version: '0'
|
125
125
|
description: A ruby implementation of Consul Template with support of erb templating
|
126
|
+
with hi-performance on large clusters and advanced process management features.
|
126
127
|
email:
|
127
128
|
- sre-core-services@criteo.com
|
128
129
|
executables:
|
@@ -136,6 +137,7 @@ files:
|
|
136
137
|
- ".rubocop.yml"
|
137
138
|
- ".ruby_app"
|
138
139
|
- ".travis.yml"
|
140
|
+
- CHANGELOG.md
|
139
141
|
- Gemfile
|
140
142
|
- LICENSE.txt
|
141
143
|
- README.md
|
@@ -162,10 +164,14 @@ files:
|
|
162
164
|
- samples/keys.html.erb
|
163
165
|
- samples/nodes.html.erb
|
164
166
|
- samples/services.html.erb
|
165
|
-
homepage: https://
|
167
|
+
homepage: https://rubygems.org/gems/consul-templaterb
|
166
168
|
licenses:
|
167
169
|
- Apache v2
|
168
|
-
metadata:
|
170
|
+
metadata:
|
171
|
+
bug_tracker_uri: https://github.com/criteo/consul-templaterb/issues
|
172
|
+
changelog_uri: https://github.com/criteo/consul-templaterb/blob/master/CHANGELOG.md
|
173
|
+
homepage_uri: https://github.com/criteo/consul-templaterb
|
174
|
+
source_code_uri: https://github.com/criteo/consul-templaterb
|
169
175
|
post_install_message:
|
170
176
|
rdoc_options: []
|
171
177
|
require_paths:
|