consul-templaterb 1.0.5 → 1.0.6
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +36 -0
- data/README.md +60 -29
- data/bin/consul-templaterb +4 -0
- data/consul-templaterb.gemspec +7 -3
- data/lib/consul/async/version.rb +1 -1
- data/samples/services.html.erb +0 -17
- metadata +10 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c585b885aba40b8fcb72ac48c82f4dfd90ea4cbb9949976aa33eecf8a6a109a3
|
4
|
+
data.tar.gz: 1a35639786fb7c3dad6f23bd890816763997f025299a99608291212cc8c07695
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 50afc5d1cdfb2aa269673be86561dc4c8fde3334f9de4b2a9361714f1e35b47a0235d828a17c93d6e50345873bbb396075bd0762f6ad7062fe97ba51eda1c1c3
|
7
|
+
data.tar.gz: 9286bd23cf9d41e01b0585521eb0e7c41bead41a82b571baa687e7f3734fdfd238cce67b7b45bcadbe2565111dc118b49d659061ec20a9deaa2d0dd6fa0bf016
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,36 @@
|
|
1
|
+
## (UNRELEASED)
|
2
|
+
|
3
|
+
## 1.0.6 (March 16, 2018)
|
4
|
+
|
5
|
+
IMPROVEMENTS:
|
6
|
+
|
7
|
+
* Adds http:// to Consul URL if missing since `$CONSUL_HTTP_ADDR` environment
|
8
|
+
variable might not have it
|
9
|
+
* Updated gem description with more accurate information
|
10
|
+
* samples: removed KV/nodes from services.html.erb
|
11
|
+
* Fixed typos in README.md
|
12
|
+
* Added CHANGELOG.md
|
13
|
+
|
14
|
+
## 1.0.5 (March 12, 2018)
|
15
|
+
|
16
|
+
BUG FIXES:
|
17
|
+
|
18
|
+
* fixed default value for parameter -c to http://localhost:8500
|
19
|
+
|
20
|
+
## 1.0.4 (March 12, 2018)
|
21
|
+
|
22
|
+
BUG FIXES:
|
23
|
+
|
24
|
+
* First auto-built by Travis CI and fully deployed version
|
25
|
+
* Fixed String interpolation bug in help messages
|
26
|
+
* Bump rubocop to version 0.49.0 due to CVE (Travis CI)
|
27
|
+
|
28
|
+
IMPROVEMENTS:
|
29
|
+
|
30
|
+
* Various README.md improvements
|
31
|
+
|
32
|
+
## 1.0.0 -> 1.0.3 (March 12, 2018) First public release
|
33
|
+
|
34
|
+
BUG FIXES:
|
35
|
+
|
36
|
+
* Allow Travis to build properly using tags and publish to rubygems.org
|
data/README.md
CHANGED
@@ -2,13 +2,13 @@
|
|
2
2
|
|
3
3
|
This GEM is both a library and an executable that allows to generate files
|
4
4
|
using data from Consul (Discovery and Key/Value Store) easily using ruby's
|
5
|
-
erb templates. It also support launching programs and
|
5
|
+
erb templates. It also support launching programs and baby-sitting processes
|
6
6
|
when rendering the files, thus notifying programs when data do change.
|
7
7
|
|
8
8
|
It is intended for user accustomed to expressiveness or Ruby templating (ERB),
|
9
|
-
allowing for more flexibility and than Go
|
9
|
+
allowing for more flexibility and features than Go templates.
|
10
10
|
|
11
|
-
It also
|
11
|
+
It also allows to use all of ruby language, especially useful for generating
|
12
12
|
files in several formats (JSON, XML) where text substitutions are hard to get
|
13
13
|
right.
|
14
14
|
|
@@ -16,7 +16,7 @@ It also focuses on good performance and lightweight usage of bandwidth,
|
|
16
16
|
especially for very large clusters and watching lots of services.
|
17
17
|
|
18
18
|
For complicated rendering of templates and large Consul Clusters, it usually
|
19
|
-
|
19
|
+
renders faster with a more predictable way the template than the original
|
20
20
|
consul-template.
|
21
21
|
|
22
22
|
## Differences with HashiCorp's consul-template
|
@@ -31,12 +31,12 @@ has support for Hashicorp's Vault), but consul-templaterb focuses on getting
|
|
31
31
|
more power with the generation of templates and more performance.
|
32
32
|
|
33
33
|
Consul Template uses Go templates which is very limited in its set of
|
34
|
-
features
|
34
|
+
features: it is complicated to sort, apply real transformations
|
35
35
|
using code and even interact with the OS (ex: get the current date, format
|
36
36
|
timestamps...).
|
37
37
|
|
38
|
-
The sort feature for instances allow you to create
|
39
|
-
that the order of nodes is
|
38
|
+
The sort feature for instances allow you to create predictable output (i.e: meaning
|
39
|
+
that the order of nodes is predictable), thus it might offer better performance
|
40
40
|
since the reload of processes if happening ONLY when the files are binary
|
41
41
|
different. Thus, if using consul-templaterb, you will reload less your haproxy or
|
42
42
|
load-balancer than you would do with consul-template.
|
@@ -44,19 +44,19 @@ load-balancer than you would do with consul-template.
|
|
44
44
|
Compared to consul-template, consul-templaterb offers the following features:
|
45
45
|
|
46
46
|
* Hot-Reload of template files
|
47
|
-
*
|
47
|
+
* Bandwidth limitation per endpoint (will soon support dynamic bandwidth limiter)
|
48
48
|
* Supports baby sitting of multiple processes
|
49
49
|
* Supports all Ruby features (ex: base64, real JSON/XML generation...)
|
50
50
|
* Information about bandwidth
|
51
51
|
|
52
52
|
The executable supports close semantics to Consul template, it also supports
|
53
|
-
commands when files are modified and
|
54
|
-
ability to send signals to those processes
|
53
|
+
commands when files are modified and supervision of multiple processes with
|
54
|
+
ability to send signals to those processes when the files do change.
|
55
55
|
|
56
56
|
## Installation
|
57
57
|
|
58
|
-
You might either use the executable
|
59
|
-
adding this line to your application's Gemfile
|
58
|
+
You might either use the executable directly OR use this GEM as a library by
|
59
|
+
adding this line to your application's `Gemfile`:
|
60
60
|
|
61
61
|
```ruby
|
62
62
|
gem 'consul-templaterb'
|
@@ -65,7 +65,7 @@ gem 'consul-templaterb'
|
|
65
65
|
And then execute:
|
66
66
|
|
67
67
|
```shell
|
68
|
-
|
68
|
+
$ bundle
|
69
69
|
[...]
|
70
70
|
```
|
71
71
|
|
@@ -79,13 +79,34 @@ $ gem install consul-templaterb
|
|
79
79
|
If you simply want to use the executable on your favorite linux distribution, you
|
80
80
|
have to install first: ruby and ruby dev.
|
81
81
|
|
82
|
-
### Quick install on Ubuntu
|
82
|
+
### Quick install on Ubuntu-Linux
|
83
83
|
|
84
84
|
```shell
|
85
85
|
sudo apt-get install ruby ruby-dev && sudo gem install consul-templaterb
|
86
86
|
```
|
87
87
|
|
88
|
-
You can now use it directly using the binary `consul-templaterb` in your path
|
88
|
+
You can now use it directly using the binary `consul-templaterb` in your path.
|
89
|
+
|
90
|
+
### Playing with the samples templates
|
91
|
+
|
92
|
+
Samples are installed with the GEM, you can either
|
93
|
+
[download](https://github.com/criteo/consul-templaterb/tree/master/samples) them or
|
94
|
+
simply use the ones installed with the gem. To figure out where the templates are
|
95
|
+
installed:
|
96
|
+
|
97
|
+
```shell
|
98
|
+
$ gem contents consul-templaterb|grep samples
|
99
|
+
```
|
100
|
+
|
101
|
+
Will output the path where the samples are being installed, you can copy the directory
|
102
|
+
somewhere and then issue the command:
|
103
|
+
|
104
|
+
```shell
|
105
|
+
$ consul-templaterb samples/*.html.erb
|
106
|
+
```
|
107
|
+
|
108
|
+
It will render a full web site you may browse to look in real time the status of your
|
109
|
+
Consul Cluster.
|
89
110
|
|
90
111
|
## Usage of consul-templaterb
|
91
112
|
|
@@ -110,6 +131,14 @@ USAGE: bin/consul-templaterb [[options]]
|
|
110
131
|
--once Do not run the process as a daemon
|
111
132
|
```
|
112
133
|
|
134
|
+
When launched with file arguments ending with .erb, the executable will assume
|
135
|
+
the file is a template and will render the corresponding file without the
|
136
|
+
`.erb` extension.
|
137
|
+
|
138
|
+
It means that you can call consul-templaterb with *.erb arguments, the shell
|
139
|
+
will then substitute all files and render it by removing the .erb extension as
|
140
|
+
if the `--template my_file.ext.erb:myfile.ext` was used.
|
141
|
+
|
113
142
|
### Generate multiple templates
|
114
143
|
|
115
144
|
In the same way as consul-template, consul-templaterb supports multiple templates and executing
|
@@ -132,7 +161,7 @@ $ consul-templaterb \\
|
|
132
161
|
|
133
162
|
### Process management and signalisation of configuration files
|
134
163
|
|
135
|
-
With the
|
164
|
+
With the `--exec` argument (can be specified multiple times), consul-templaterb will launch
|
136
165
|
the process specified when all templates have been generated and will send a reload signal
|
137
166
|
if the content of any of the files do change (the signal will be sent atomically however,
|
138
167
|
meaning that if 2 results of templates are modified at the same time, the signal will be
|
@@ -154,7 +183,7 @@ By design, the GEM supports limiting the number of requests per endpoints (see c
|
|
154
183
|
`bin/consul-templaterb` file). It avoids using too much network to fetch data from Consul
|
155
184
|
in large Consul Clusters (especially when watching lots of files).
|
156
185
|
|
157
|
-
The limitation is
|
186
|
+
The limitation is static for now, but fair dynamic bandwidth allocation will allow to limit
|
158
187
|
the bandwidth used to get information for all services by capping the global bandwidth used
|
159
188
|
by consul-templaterb.
|
160
189
|
|
@@ -176,12 +205,12 @@ use (website updated automagically when values to change).
|
|
176
205
|
|
177
206
|
Here are the various functions you might use in your templates.
|
178
207
|
|
179
|
-
For each function, mandatory arguments are specified at the
|
208
|
+
For each function, mandatory arguments are specified at the beginning while optional ones are marked with `[]`.
|
180
209
|
Most of them support the optional dc attribute to access data from another datacenter. If the `dc`
|
181
210
|
attribute is not specified, the function will output data from the current datacenter.
|
182
211
|
|
183
212
|
To ease template development, `consul-templaterb` supports HOT reload of templates, thus it is possible to
|
184
|
-
develop the templates
|
213
|
+
develop the templates interactively. While developing, it is possible to use the switch `--hot-reload=keep`,
|
185
214
|
thus the application will display a warning if the template is invalid and won't stop
|
186
215
|
(`--hot-reload=die` is the default, thus if the hot-reloaded template has issue, the application will die).
|
187
216
|
|
@@ -228,15 +257,16 @@ mechanism.
|
|
228
257
|
|
229
258
|
### agent_self()
|
230
259
|
|
231
|
-
[Get the configuration of Consul Agent](https://www.consul.io/api/agent.html#read-configuration).
|
232
|
-
endpoint does not support blocking queries, data will be refreshed every few seconds,
|
233
|
-
queries mechanism.
|
260
|
+
[Get the configuration of Consul Agent](https://www.consul.io/api/agent.html#read-configuration).
|
261
|
+
Since this endpoint does not support blocking queries, data will be refreshed every few seconds,
|
262
|
+
but will not use blocking queries mechanism.
|
234
263
|
|
235
264
|
### render_file RELATIVE_PATH_TO_ERB_FILE
|
236
265
|
|
237
266
|
This allow to include a template file into another one. Beware, it does not check for infinite recursion!
|
238
267
|
The template can be either a static file either another template. The file has to be a valid template, but
|
239
|
-
can also be raw text (if it is a valid template
|
268
|
+
can also be raw text (if it is a valid template) and is resolved with a relative path regarding the file
|
269
|
+
including it.
|
240
270
|
|
241
271
|
Example:
|
242
272
|
|
@@ -250,16 +280,17 @@ We recommend using bundle using `bundle install`, you can now run `bundle exec b
|
|
250
280
|
|
251
281
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
|
252
282
|
version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
|
253
|
-
push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
283
|
+
push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org/gems/consul-templaterb).
|
254
284
|
|
255
285
|
## Known bugs
|
256
286
|
|
257
287
|
Here are the known bugs of the application:
|
258
288
|
|
259
289
|
* [ ] On Mac OS X, with many services, consul-templaterb sometimes crashes when watching lots of changes (more than 2k
|
260
|
-
watches) at the same time. This bug is
|
261
|
-
|
262
|
-
|
290
|
+
watches) at the same time. This bug is
|
291
|
+
[probably a race condition in `em-http-request`](https://github.com/igrigorik/em-http-request/issues/315). Only visible
|
292
|
+
on very large clusters or when watching thousands of individual KV keys.
|
293
|
+
* [ ] render_file might create an infinite recursion if a template includes itself indirectly.
|
263
294
|
|
264
295
|
## TODO
|
265
296
|
|
@@ -268,7 +299,7 @@ Here are the known bugs of the application:
|
|
268
299
|
* [ ] More samples: apache, nginx, full website displaying consul information...
|
269
300
|
* [ ] Optimize rendering speed at startup: an iteration is done very second by default, but it would be possible to speed
|
270
301
|
up rendering by iterating with higher frequency until the first write of result has been performed.
|
271
|
-
* [ ] Allow to tune
|
302
|
+
* [ ] Allow to tune bandwidth using a simple config file (while it should not be necessary for 90% of use-cases)
|
272
303
|
|
273
304
|
## Contributing
|
274
305
|
|
@@ -278,4 +309,4 @@ This project is intended to be a safe, welcoming space for collaboration, and co
|
|
278
309
|
|
279
310
|
## License
|
280
311
|
|
281
|
-
The gem is available as open source under the terms of the Apache v2 license. See LICENSE.txt file.
|
312
|
+
The gem is available as open source under the terms of the Apache v2 license. See [LICENSE.txt](LICENSE.txt) file.
|
data/bin/consul-templaterb
CHANGED
@@ -212,6 +212,10 @@ def find_max_descriptors(max_descripors)
|
|
212
212
|
max_size
|
213
213
|
end
|
214
214
|
|
215
|
+
unless options[:consul][:base_url].start_with? 'http'
|
216
|
+
options[:consul][:base_url] = "http://#{options[:consul][:base_url]}"
|
217
|
+
end
|
218
|
+
|
215
219
|
# Since we might be using a lots of descriptors, document this
|
216
220
|
new_size = find_max_descriptors(65_536)
|
217
221
|
STDERR.puts "Max number of descriptors set to #{new_size}" if options[:consul][:debug][:network]
|
data/consul-templaterb.gemspec
CHANGED
@@ -12,9 +12,13 @@ Gem::Specification.new do |spec|
|
|
12
12
|
spec.email = ['sre-core-services@criteo.com']
|
13
13
|
|
14
14
|
spec.summary = 'Implementation of Consul template using Ruby and .erb templating language'
|
15
|
-
spec.homepage = 'https://
|
16
|
-
spec.description = 'A ruby implementation of Consul Template with support of erb templating'
|
17
|
-
|
15
|
+
spec.homepage = 'https://rubygems.org/gems/consul-templaterb'
|
16
|
+
spec.description = 'A ruby implementation of Consul Template with support of erb templating ' \
|
17
|
+
'with hi-performance on large clusters and advanced process management features.'
|
18
|
+
spec.metadata = { 'bug_tracker_uri' => 'https://github.com/criteo/consul-templaterb/issues',
|
19
|
+
'changelog_uri' => 'https://github.com/criteo/consul-templaterb/blob/master/CHANGELOG.md',
|
20
|
+
'homepage_uri' => 'https://github.com/criteo/consul-templaterb',
|
21
|
+
'source_code_uri' => 'https://github.com/criteo/consul-templaterb' }
|
18
22
|
spec.license = 'Apache v2'
|
19
23
|
|
20
24
|
spec.files = `git ls-files -z`.split("\x0").reject do |f|
|
data/lib/consul/async/version.rb
CHANGED
data/samples/services.html.erb
CHANGED
@@ -25,8 +25,6 @@
|
|
25
25
|
<li><a href="#datacenters">DataCenters</a></li>
|
26
26
|
<li><a href="#list_services">List of services</a></li>
|
27
27
|
<li><a href="#services">Services with instances</a></li>
|
28
|
-
<li><a href="#nodes">All Nodes</a></li>
|
29
|
-
<li><a href="#kv">Key/Value Store</a></li>
|
30
28
|
</ul>
|
31
29
|
</nav>
|
32
30
|
<h1 id="datacenters">List of all datacenters</h1>
|
@@ -71,19 +69,4 @@
|
|
71
69
|
<% end%>
|
72
70
|
</ul>
|
73
71
|
<% end%>
|
74
|
-
|
75
|
-
|
76
|
-
<h1 id="nodes">List all nodes for DC, sorted by name</h1>
|
77
|
-
<ul>
|
78
|
-
<% nodes.sort {|a,b| a['Node'] <=> b['Node'] }.each do |snode|
|
79
|
-
%> <li id="node_#{snode['ID']"><%= snode['Address'].ljust(16) %> <%= snode['Node'] %></li>
|
80
|
-
<% end %>
|
81
|
-
</ul>
|
82
|
-
|
83
|
-
<h1 id="kv">KV of Current DC</h1>
|
84
|
-
<ul>
|
85
|
-
<% kv(keys:true).each do |key|
|
86
|
-
%><li id="kv_#{key}"><%= key %></li>
|
87
|
-
<% end %>
|
88
|
-
</ul>
|
89
72
|
<%= render_file 'common/footer.html.erb' %>
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: consul-templaterb
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0.
|
4
|
+
version: 1.0.6
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- SRE Core Services
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2018-03-
|
11
|
+
date: 2018-03-16 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: em-http-request
|
@@ -123,6 +123,7 @@ dependencies:
|
|
123
123
|
- !ruby/object:Gem::Version
|
124
124
|
version: '0'
|
125
125
|
description: A ruby implementation of Consul Template with support of erb templating
|
126
|
+
with hi-performance on large clusters and advanced process management features.
|
126
127
|
email:
|
127
128
|
- sre-core-services@criteo.com
|
128
129
|
executables:
|
@@ -136,6 +137,7 @@ files:
|
|
136
137
|
- ".rubocop.yml"
|
137
138
|
- ".ruby_app"
|
138
139
|
- ".travis.yml"
|
140
|
+
- CHANGELOG.md
|
139
141
|
- Gemfile
|
140
142
|
- LICENSE.txt
|
141
143
|
- README.md
|
@@ -162,10 +164,14 @@ files:
|
|
162
164
|
- samples/keys.html.erb
|
163
165
|
- samples/nodes.html.erb
|
164
166
|
- samples/services.html.erb
|
165
|
-
homepage: https://
|
167
|
+
homepage: https://rubygems.org/gems/consul-templaterb
|
166
168
|
licenses:
|
167
169
|
- Apache v2
|
168
|
-
metadata:
|
170
|
+
metadata:
|
171
|
+
bug_tracker_uri: https://github.com/criteo/consul-templaterb/issues
|
172
|
+
changelog_uri: https://github.com/criteo/consul-templaterb/blob/master/CHANGELOG.md
|
173
|
+
homepage_uri: https://github.com/criteo/consul-templaterb
|
174
|
+
source_code_uri: https://github.com/criteo/consul-templaterb
|
169
175
|
post_install_message:
|
170
176
|
rdoc_options: []
|
171
177
|
require_paths:
|