consul-templaterb 1.25.2 → 1.27.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 939468d29eba6433fe6f2c93c1ca6eb8a2522d6827a7c6fa99e49415a1b2b589
-  data.tar.gz: 052ccf9fce6c083e6925439d0e2505eecbeb2b80371f418dc8b9c7ad6f42c65f
+  metadata.gz: ea2191ec559b6c2ccc871a6cac0083fd894200bd268421eb25b12361a8fc4373
+  data.tar.gz: db35acece2ea661ef35cefad9c1a94dd452e533458cff6cfa97f3114d5cc502f
 SHA512:
-  metadata.gz: 0bd42ba387e49d7355153d557bbc2f4723f5eae92ee733ed5e21f550ed6eda7c5b914f1a38bbfa569d7df3061c94d8c4da62e814ceb87d328ac8cfd9cb59c490
-  data.tar.gz: d4675c40d58c9bafddefef6ce6e71e2a0bf61bafe256db7e3512f889416f3ed1cd5388a651d7334dd7afc88da660677a7b684afcabfaed077ef4e5a49dab9221
+  metadata.gz: bf4a2b9e2ee7ba6d811b8d20202c2c0bc998afe5fcbd08c1325f573a38b422a5d94a21666efff14526d44071aa2faeb36e2157f72a097d36dad8309b30418aef
+  data.tar.gz: 28087756e716a4a6e2c8526b4920571118a3c6b2896ff2e302d52281fccfb277b5741b1a0c8b2be132ba7c405c6ea689767369b90c8db052562348739d3ba04d

data/.rubocop.yml

@@ -7,28 +7,28 @@ Layout/LineLength:
   Max: 175
 
 Metrics/AbcSize:
-  Max: 82
+  Max: 87
 
 Metrics/BlockLength:
-  Max: 160
+  Max: 182
 
 Metrics/BlockNesting:
   Max: 4
 
 Metrics/ClassLength:
-  Max: 275
+  Max: 285
 
 Metrics/CyclomaticComplexity:
-  Max: 20
+  Max: 21
 
 Metrics/MethodLength:
-  Max: 65
+  Max: 68
 
 Metrics/ParameterLists:
-  Max: 14
+  Max: 18
 
 Metrics/PerceivedComplexity:
-  Max: 23
+  Max: 24
 
 # We use `dc` as a parameter in many methods
 Naming/MethodParameterName:

data/.travis.yml

@@ -1,13 +1,13 @@
 language: ruby
 rvm:
-- 2.4.9
-- 2.5.7
-- 2.6.5
-- 2.7.0
+- 2.4.10
+- 2.5.8
+- 2.6.6
+- 2.7.1
 jobs:
   include:
     - stage: Gem release
-      rvm: 2.5.7
+      rvm: 2.5.8
       script: echo "Publishing consul-templaterb on rubygems.org ..."
       deploy:
         provider: rubygems

data/CHANGELOG.md

@@ -2,6 +2,49 @@
 
 ## (UNRELEASED)
 
+## 1.27.0 (June 5, 2020)
+
+NEW FEATURES:
+
+ * For Consul 1.7+, now support `checks_in_state(check_state, dc: nil, [agent: consul_agent_address])`,
+   fixes feature [#65](https://github.com/criteo/consul-templaterb/issues/65)
+ * New options to support/disable TLS validation thanks to [@jeromegn](https://github.com/jeromegn)
+   [#66](https://github.com/criteo/consul-templaterb/pull/66)
+
+## 1.26.3 (April 15, 2020)
+
+BUGFIX:
+
+ * Removed all Criteo specific decorators from Consul-UI
+
+## 1.26.2 (April 15, 2020)
+
+BUGFIX:
+
+ * Fixed broken Dockerfile (was missing the new `decorator.js.erb` file). Fixes #61 (Thanks to @ simongareste)
+
+NEW FEATURES:
+
+ * Added `consul_members_count` metric in [samples/metrics.erb](samples/metrics.erb)
+
+## 1.26.1 (March 27, 2020)
+
+BUGFIX:
+
+ * Using `agent: http://vault_or_consul_agent:port>` was not properly taken into account in some endpoints
+
+## 1.26.0 (March 5, 2020)
+
+NEW FEATURES:
+
+ * Using `agent: http://vault_or_consul_agent:port>` on most methods will now override the agent
+   used to perform Consul queries. It might be useful for very large clusters or large WAN federations,
+   because you can perform some requests on some agents, and some others on other agenrs. Might
+   also be useful to federate data from preprod/prod for instance.
+ * Added agent attribute for all methods in documentation
+ * Added [samples/all_services_multi_agents.txt.erb](samples/all_services_multi_agents.txt.erb) as an
+   example of new feature.
+
 ## 1.25.2 (February 29, 2020)
 
 BUGFIX:

data/Dockerfile

@@ -12,4 +12,4 @@ ENV LANG C.UTF-8
 ENV CONSUL_HTTP_ADDR http://consul-relay.service.consul.preprod.crto.in:8500
 
 ENTRYPOINT ["/usr/local/bin/bundle", "exec", "consul-templaterb"]
-CMD ["--template", "samples/consul-ui/consul-keys-ui.html.erb", "--template", "samples/consul-ui/consul-nodes-ui.html.erb", "--template", "samples/consul-ui/consul-services-ui.html.erb", "--template", "samples/consul-ui/consul-timeline-ui.html.erb", "--template", "samples/consul-ui/consul_keys.json.erb", "--template", "samples/consul-ui/consul_nodes.json.erb", "--template", "samples/consul-ui/consul_services.json.erb", "--template", "samples/consul-ui/timeline.json.erb", "--template", "samples/consul-ui/consul-services-ui.html.erb:samples/consul-ui/index.html:touch samples/consul-ui/ready", "--sig-reload=NONE", "--exec=nginx -c /usr/src/app/docker-nginx-conf/nginx.conf"]
+CMD ["--template", "samples/consul-ui/consul-keys-ui.html.erb", "--template", "samples/consul-ui/decorators.js.erb", "--template", "samples/consul-ui/consul-nodes-ui.html.erb", "--template", "samples/consul-ui/consul-services-ui.html.erb", "--template", "samples/consul-ui/consul-timeline-ui.html.erb", "--template", "samples/consul-ui/consul_keys.json.erb", "--template", "samples/consul-ui/consul_nodes.json.erb", "--template", "samples/consul-ui/consul_services.json.erb", "--template", "samples/consul-ui/timeline.json.erb", "--template", "samples/consul-ui/consul-services-ui.html.erb:samples/consul-ui/index.html:touch samples/consul-ui/ready", "--sig-reload=NONE", "--exec=nginx -c /usr/src/app/docker-nginx-conf/nginx.conf"]

data/INTERNALS.md

@@ -12,6 +12,8 @@ the list of services registered in the cluster).
 
 For each endpoint, `consul-templaterb` performs a watch by getting information about the last transaction on this endpoints, storing it locally and watching at Consul to get updates about the data within this endpoint.
 
+![Main Loop](docs/images/consul-templaterb.svg)
+
 ## How it works
 
 There is a loop in the engine, every second, all templates are rendered using the `ERB` template engine.

data/README.md

@@ -33,6 +33,8 @@ with fully [working examples](samples/).
 It also allows displaying a very nice and hi-performance HTML5 UI for Consul,
 see [consul-ui](samples/consul-ui) for details.
 
+There is an article [docs/article-06_Template-based_discovery_with_consul-templaterb.md](docs/article-06_Template-based_discovery_with_consul-templaterb.md) summarizing why we developped this tool (also available on [medium](https://medium.com/criteo-labs/template-based-discovery-with-consul-templaterb-8ff88434c457)).
+
 ## Video introduction to consul-templaterb
 
 [![Introduction to Consul-templaterb](docs/images/consul-ui_001.png)](https://youtu.be/zLzrLGLLl4Q)
@@ -330,7 +332,7 @@ Please consult [CHANGELOG.md](CHANGELOG.md) for fixed bugs.
 
 ## TODO
 
-* [x] Hashi's Vault support (EXPERIMENTAL)
+* [x] Hashi's Vault support
 * [ ] Implement automatic dynamic rate limit
 * [x] More samples: apache, nginx, a full website displaying consul information...
 * [x] Optimize rendering speed at start-up: an iteration is done every second by default, but it would be possible to speed

data/TemplateAPI.md

@@ -7,6 +7,10 @@ annoted with `[]`.
 Most of them support the optional dc attribute to use data from another datacenter. If the `dc`
 attribute is not specified, the function will output data from the current datacenter.
 
+Starting with version 1.26.0, most methods also support the
+`agent: "http://address_of_vault_or_consul_agent:port"` to use another agent than the one specified in command line. It opens the way to use different agents/DC or to create templates federating several different not-linked clusters (ex: prod/preprod).
+Example on how using it: [samples/all_services_multi_agents.txt.erb](samples/all_services_multi_agents.txt.erb).
+
 To ease template development, `consul-templaterb` supports HOT reload of templates, thus it is possible to
 develop the templates interactively. While developing, it is possible to use the switch `--hot-reload=keep`,
 thus the application will display a warning if the template is invalid and won't stop
@@ -44,154 +48,162 @@ to avoid having to write .result in all templates, some shortcuts have been adde
 
 Also available for all results:
 
-### Common methods available for all objects
-
- * .all?
- * .any?
- * .assoc
- * .chunk
- * .chunk_while
- * .class
- * .clear
- * .collect
- * .collect_concat
- * .compact
- * .count
- * .cycle
- * .detect
- * .dig
- * .display
- * .drop
- * .drop_while
- * .dup
- * .each
- * .each_cons
- * .each_entry
- * .each_slice
- * .each_with_index
- * .each_with_object
- * .empty?
- * .entries
- * .enum_for
- * .eql?
- * .equal?
- * .extend
- * .fetch
- * .find
- * .find_all
- * .find_index
- * .first
- * .flat_map
- * .flatten
- * .grep
- * .grep_v
- * .group_by
- * .hash
- * .include?
- * .index
- * .inject
- * .inspect
- * .is_a?
- * .itself
- * .keep_if
- * .kind_of?
- * .lazy
- * .length
- * .map
- * .max
- * .max_by
- * .member?
- * .min
- * .min_by
- * .minmax
- * .minmax_by
- * .nil?
- * .none?
- * .object_id
- * .one?
- * .partition
- * .pp
- * .rassoc
- * .reduce
- * .reject
- * .replace
- * .reverse_each
- * .select
- * .shift
- * .size
- * .slice
- * .slice_after
- * .slice_before
- * .slice_when
- * .sort
- * .sort_by
- * .sum
- * .take
- * .take_while
- * .tap
- * .to_a
- * .to_enum
- * .to_h
- * .to_s
- * .uniq
- * .values_at
- * .yield_self
- * .zip
-
- ### Methods available for Array objects
-
- * .append
- * .at
- * .bsearch
- * .bsearch_index
- * .combination
- * .concat
- * .each_index
- * .fill
- * .insert
- * .join
- * .last
- * .pack
- * .permutation
- * .pop
- * .prepend
- * .product
- * .push
- * .repeated_combination
- * .repeated_permutation
- * .reverse
- * .rindex
- * .rotate
- * .sample
- * .shuffle
- * .to_ary
- * .transpose
- * .unshift
-
- ### Methods available for hash objects
-
- * .compare_by_identity
- * .compare_by_identity?
- * .each_key
- * .each_pair
- * .each_value
- * .fetch_values
- * .has_key?
- * .has_value?
- * .invert
- * .key
- * .key?
- * .keys
- * .merge
- * .rehash
- * .store
- * .to_hash
- * .to_proc
- * .transform_keys
- * .transform_values
- * .update
- * .value?
- * .values
+<details id="common-methods-available-for-all-objects">
+<summary>Common methods available for all objects</summary>
+<div class="samples">
+<ul>
+<li>.all?</li>
+<li>.any?</li>
+<li>.assoc</li>
+<li>.chunk</li>
+<li>.chunk_while</li>
+<li>.class</li>
+<li>.clear</li>
+<li>.collect</li>
+<li>.collect_concat</li>
+<li>.compact</li>
+<li>.count</li>
+<li>.cycle</li>
+<li>.detect</li>
+<li>.dig</li>
+<li>.display</li>
+<li>.drop</li>
+<li>.drop_while</li>
+<li>.dup</li>
+<li>.each</li>
+<li>.each_cons</li>
+<li>.each_entry</li>
+<li>.each_slice</li>
+<li>.each_with_index</li>
+<li>.each_with_object</li>
+<li>.empty?</li>
+<li>.entries</li>
+<li>.enum_for</li>
+<li>.eql?</li>
+<li>.equal?</li>
+<li>.extend</li>
+<li>.fetch</li>
+<li>.find</li>
+<li>.find_all</li>
+<li>.find_index</li>
+<li>.first</li>
+<li>.flat_map</li>
+<li>.flatten</li>
+<li>.grep</li>
+<li>.grep_v</li>
+<li>.group_by</li>
+<li>.hash</li>
+<li>.include?</li>
+<li>.index</li>
+<li>.inject</li>
+<li>.inspect</li>
+<li>.is_a?</li>
+<li>.itself</li>
+<li>.keep_if</li>
+<li>.kind_of?</li>
+<li>.lazy</li>
+<li>.length</li>
+<li>.map</li>
+<li>.max</li>
+<li>.max_by</li>
+<li>.member?</li>
+<li>.min</li>
+<li>.min_by</li>
+<li>.minmax</li>
+<li>.minmax_by</li>
+<li>.nil?</li>
+<li>.none?</li>
+<li>.object_id</li>
+<li>.one?</li>
+<li>.partition</li>
+<li>.pp</li>
+<li>.rassoc</li>
+<li>.reduce</li>
+<li>.reject</li>
+<li>.replace</li>
+<li>.reverse_each</li>
+<li>.select</li>
+<li>.shift</li>
+<li>.size</li>
+<li>.slice</li>
+<li>.slice_after</li>
+<li>.slice_before</li>
+<li>.slice_when</li>
+<li>.sort</li>
+<li>.sort_by</li>
+<li>.sum</li>
+<li>.take</li>
+<li>.take_while</li>
+<li>.tap</li>
+<li>.to_a</li>
+<li>.to_enum</li>
+<li>.to_h</li>
+<li>.to_s</li>
+<li>.uniq</li>
+<li>.values_at</li>
+<li>.yield_self</li>
+<li>.zip</li>
+</ul></div></details>
+
+<details>
+<summary id="methods-available-for-array-objects">Methods available for Array objects</summary>
+<div class="samples">
+<ul>
+<li>.append</li>
+<li>.at</li>
+<li>.bsearch</li>
+<li>.bsearch_index</li>
+<li>.combination</li>
+<li>.concat</li>
+<li>.each_index</li>
+<li>.fill</li>
+<li>.insert</li>
+<li>.join</li>
+<li>.last</li>
+<li>.pack</li>
+<li>.permutation</li>
+<li>.pop</li>
+<li>.prepend</li>
+<li>.product</li>
+<li>.push</li>
+<li>.repeated_combination</li>
+<li>.repeated_permutation</li>
+<li>.reverse</li>
+<li>.rindex</li>
+<li>.rotate</li>
+<li>.sample</li>
+<li>.shuffle</li>
+<li>.to_ary</li>
+<li>.transpose</li>
+<li>.unshift</li>
+</ul></div></details>
+
+<details id="Methods-available-for-hash-objects">
+<summary>Methods available for hash objects</summary>
+<ul>
+<li>.compare_by_identity</li>
+<li>.compare_by_identity?</li>
+<li>.each_key</li>
+<li>.each_pair</li>
+<li>.each_value</li>
+<li>.fetch_values</li>
+<li>.has_key?</li>
+<li>.has_value?</li>
+<li>.invert</li>
+<li>.key</li>
+<li>.key?</li>
+<li>.keys</li>
+<li>.merge</li>
+<li>.rehash</li>
+<li>.store</li>
+<li>.to_hash</li>
+<li>.to_proc</li>
+<li>.transform_keys</li>
+<li>.transform_values</li>
+<li>.update</li>
+<li>.value?</li>
+<li>.values</li>
+</ul></div></details>
 
 ## coordinate
 
@@ -203,7 +215,7 @@ The coordinate object allow to interact with the coordinates of DCs and nodes as
 [List the Wan Coordinates](https://www.consul.io/api/coordinate.html#read-wan-coordinates) from local DC to
 other DCs. If dc is set, it will perform the same operation but from another DC.
 
-### coordinate.nodes([dc: datacenter])
+### coordinate.nodes([dc: datacenter], [agent: consul_agent_address])
 
 [Read all LAN nodes coordinates](https://www.consul.io/api/coordinate.html#read-lan-coordinates-for-all-nodes).
 If If dc is set, it will perform the same operation but for another DC.
@@ -213,7 +225,7 @@ If If dc is set, it will perform the same operation but for another DC.
 Computes the rtt between 2 nodes returned by `coordinate.nodes` or `coordinate.datacenters`. A re-implementation of Golang sample code
 [Working with Coordinates](https://www.consul.io/docs/internals/coordinates.html#working-with-coordinates).
 
-## datacenters()
+## datacenters([agent: consul_agent_address])
 
 [Get the list of datacenters as string array](https://www.consul.io/api/catalog.html#list-datacenters).
 
@@ -233,7 +245,7 @@ Full example: [samples/consul_template.txt.erb](samples/consul_template.txt.erb)
 </div>
 </details>
 
-## services([dc: datacenter], [tag: tagToFilterWith])
+## services([dc: datacenter], [tag: tagToFilterWith], [agent: consul_agent_address])
 
 [List the services matching the optional tag filter](https://www.consul.io/api/catalog.html#list-services),
 if tag is not specified, will match all the services. Note that this endpoint performs client side tag
@@ -270,7 +282,7 @@ Full example: [samples/consul_template.txt.erb](samples/consul_template.txt.erb)
 </div>
 </details>
 
-## service(serviceName, [dc: datacenter], [tag: tagToFilterWith], [passing: true])
+## service(serviceName, [dc: datacenter], [tag: tagToFilterWith], [passing: true], [agent: consul_agent_address])
 
 [List the instances](https://www.consul.io/api/health.html#list-nodes-for-service) of a service having the given
 optional tag. If no tag is specified, will return all instances of service. By default, it will return all the
@@ -342,7 +354,7 @@ Full example: [samples/consul_template.txt.erb](samples/consul_template.txt.erb)
 </div>
 </details>
 
-## nodes([dc: datacenter])
+## nodes([dc: datacenter], [agent: consul_agent_address])
 
 [List all the nodes of selected datacenter](https://www.consul.io/api/catalog.html#list-nodes). No filtering is
 applied.
@@ -363,20 +375,29 @@ Full example: [samples/consul_template.txt.erb](samples/consul_template.txt.erb)
 </div>
 </details>
 
-## node(nodeNameOrId, [dc: datacenter])
+## node(nodeNameOrId, [dc: datacenter], [agent: consul_agent_address])
 
 [List all the services of a given Node](https://www.consul.io/api/catalog.html#list-services-for-node) using its
 name or its ID. If DC is specified, will lookup for given node in another datacenter.
 
-## checks_for_node(name, dc: nil, passing: false, tag: nil)
+## checks_for_node(name, dc: nil, passing: false, tag: nil, [agent: consul_agent_address])
 
 [Find all the checks](https://www.consul.io/api/health.html#list-checks-for-node) of a given node name.
 
-## checks_for_service(name, dc: nil, passing: false, tag: nil)
+## checks_for_service(name, dc: nil, passing: false, tag: nil, [agent: consul_agent_address])
 
 [Find all the checks](https://www.consul.io/api/health.html#list-checks-for-service) of a given service.
 
-## kv(name, [dc: nil], [keys: false], [recurse: false])
+## def checks_in_state(check_state, dc: nil, [agent: consul_agent_address])
+
+[Find all the checks in a given state](https://www.consul.io/api-docs/health#list-checks-in-state) in the whole cluster.
+
+The filter check_state must be one of any|critical|warning|passing.
+
+Warning: this endpoint might be very frequently updated in a
+large cluster if you are using `any` value. This endpoint is supported with Consul 1.7+.
+
+## kv(name, [dc: nil], [keys: false], [recurse: false], [agent: consul_agent_address])
 
 [Read keys from KV Store](https://www.consul.io/api/kv.html#read-key). It can be used for both listing the keys and
 getting the values. See the file in samples [keys.html.erb](samples/keys.html.erb) for a working example.
@@ -451,7 +472,7 @@ value123 : <%= result.get_decoded('/my/multiple/values/value123') %>
 Since `kv('/my/multiple/values', recurse: true)` will retrieve all values at once, it might be more
 efficient in some cases than retrieving all values one by one.
 
-## agent_members(wan: false)
+## agent_members(wan: false, [agent: consul_agent_address])
 
 [Get the Serf information](https://www.consul.io/api/agent.html#list-members) from Consul Agent point of view.
 This is a list of Serf information containing serf information. This information is not consistent and should be used with care, most of the time, you should prefer `nodes()`.
@@ -470,13 +491,13 @@ The returned value is an array containing the following objects containing the f
 
  See [samples/members.json.erb](samples/members.json.erb) for example of usage.
 
-## agent_metrics()
+## agent_metrics([agent: consul_agent_address])
 
 [Get the metrics of Consul Agent](https://www.consul.io/api/agent.html#view-metrics). Since this endpoint does
 not support blocking queries, data will be refreshed every few seconds, but will not use blocking queries
 mechanism.
 
-## agent_self()
+## agent_self([agent: consul_agent_address])
 
 [Get the configuration of Consul Agent](https://www.consul.io/api/agent.html#read-configuration).
 Since this endpoint does not support blocking queries, data will be refreshed every few seconds,
@@ -542,7 +563,7 @@ consul-templaterb --template "source.html.erb:dest.html:reload_command:params.ya
 
 See [samples/consul-ui/consul-services-ui.html.erb](samples/consul-ui/consul-services-ui.html.erb) for example of usage.
 
-## secrets(prefix)
+## secrets(prefix, [agent: vault_agent_address])
 
 It requires that a Vault token is given either in parameter or in environment variable
 The [policies](https://www.vaultproject.io/docs/concepts/policies.html) should be properly set.
@@ -568,7 +589,7 @@ Full example: [samples/vault-ldap.txt.erb](samples/vault-ldap.txt.erb)
 </div>
 </details>
 
-## secret(path, [data = nil])
+## secret(path, [data = nil], [agent: vault_agent_address])
 
 It requires that a Vault token is given either in parameter or in environment variable
 The [policies](https://www.vaultproject.io/docs/concepts/policies.html) should be properly set.
@@ -689,5 +710,4 @@ Here are templates rendered by consul-templaterb:
 ```
 
 </div>
-</details>
-
+</details>