hiera-mysql-json-backend 1.1.0 → 1.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 484c8f90a35683ae4e4edfb5f8463b09200a7bcb
-  data.tar.gz: a0a8eeb6c9d7588cf911cc0d0f2b1254ffc6820f
+  metadata.gz: 53185036554e275062f16f0fab0556b80b2b1551
+  data.tar.gz: 8dafb494f356fdae2f623204778958987f092769
 SHA512:
-  metadata.gz: 21908ff7b64277542e964a9ff8a039616ba47f105cab9ae6821c7ce17de8aacef356a767de463880a6138b94e88b6a9b0d4937f5843afa2cda1acf3c07a712cd
-  data.tar.gz: 154cd16ab985dbd5562a49a46077b1bba0814ca5b41acbc07d172e248a582c8878c8380195aab87c805adad53a73e2c4366de8a678f7b0ae35c1e26a0a463937
+  metadata.gz: 0e0e476c2d32b10c2240d7fab945bc277ef67a6c2b99b8191c85b6b2fdb9f2f68a58ae0fb2bf8f8e9118f1a787fcdc9e9a396ef9f568be4c6b1f1036fb042d48
+  data.tar.gz: 4fa32e35527dec742eabbf565487d59b2eb4eab2ea170db8550e7acf749822ed15f342e301ba58c73af810d8b52e2e4391392bdfb69c80562f4d9ec848b094eb

data/README.md

@@ -1,90 +1,151 @@
 [![Gem Version](https://badge.fury.io/rb/hiera-mysql-json-backend.png)](http://badge.fury.io/rb/hiera-mysql-json-backend)
 
-## hiera-mysql-backend
+## hiera-mysql-json-backend
 
-Alternate MySQL backend for Hiera
+Alternate MySQL backend for Hiera with json support
 
-This is a MySQL backend for Hiera inspired by [hiera-mysql](https://github.com/crayfishx/hiera-mysql). Unfortunately no work has been done to that backend for the past 9 months and it was missing a couple of features I needed so I decided to pick up the torch and implement them myself.
+This is a backend for Hiera based on
+[hiera-mysql-backend](https://github.com/Telmo/hiera-mysql-backend).
 
-### What is different from hiera-mysql
+### What makes this backend different from the other mysql backends
 
-In [hiera-mysql](https://github.com/crayfishx/hiera-mysql) you define the queries in the hiera.yaml file. I felt this was too restricting so instead hiera-mysql-backend uses, poorly named, sql files. This sql files follow the Hiera hierarchy.
+This backend differs from
+[hiera-mysql](https://github.com/crayfishx/hiera-mysql) and 
+[hiera-mysql-backend](https://github.com/Telmo/hiera-mysql-backend) in that it
+expects queries to return json. This json is then parsed and the resulting data
+structure is given back to the lookup.
 
-[hiera-mysql](https://github.com/crayfishx/hiera-mysql) would also return the last matching query not the first one which I felt it was confusing.
+### Usage
 
-[hiera-mysql](https://github.com/crayfishx/hiera-mysql) used the mysql gem, I am partial to [mysql2](https://github.com/brianmario/mysql2)
+Puppet client:
 
-Exception handling. hiera-mysql would cause a puppet run to fail if one of the queries was incorrect. For example a fact that you are distributing with a module is needed for the query to return its data but that fact is not available outside the module having a `SELECT * from %{custom_fact}` would make puppet runs fail.
+`/opt/puppetlabs/puppet/bin/gem install hiera-mysql-json-backend mysql2`
 
-### What goes into the sql files.
+Puppet server:
 
-The poorly named sql files are really yaml files where the key is the lookup key and the value is the SQL statement (it accepts interpolation)
+`/opt/puppetlabs/bin/puppetserver gem install hiera-mysql-json-backend-jruby jdbc-mysql`
 
-As of version 0.0.4 you can also add connection information to these sql files, this allows you to connect to different databases. This is optional if no connection information is found it will use the default defined in your hiera.yaml config.
+### Configuring hiera
 
-Lets assume your _datadir_ is `/etc/puppet/hieradata/` and your hierarchy for hiera just have a common. hiera-mysql-backend would look for /etc/puppet/hieradata/common.sql the common.sql would look like:
+The backend is configured like any other in the hiera.conf. Here is the
+simplest possible example:
 
 ```yaml
 ---
-# This is optional, if not present it will use the default connection info from hiera.yaml
-:dbconfig:
-  :host: database.example.com
-  :user: hieratest
-  :pass: sekret
-  :database: testhieradb
-  :port: 44445
+:backends:
+  - yaml
+  - mysql_json
+
+:yaml:
+  :datadir: /etc/puppet/hieradata
+
+:mysql_json:
+  :datadir: /etc/puppet/hieradata
 
-applications: SELECT * FROM applications WHERE host='%{fqdn}';
-coats: SELECT cut,name,type FROM coats WHERE color='brown';
+:hierarchy:
+  - "%{::clientcert}"
+  - "%{::custom_location}"
+  - common
+
+:logger: console
 ```
 
-If `host` is not defined it will use `localhost` as default.
+This will cause it to try to connect on localhost:3306 with no username and
+password. This will probably not work. The following options can be set:
 
-If `port` is not defined it will use the default `3306` mysql port
+* `host`: mysql host (string)
+* `port`: mysql port (int)
+* `user`: mysql user (string)
+* `pass`: mysql password (string)
+* `database`: mysql database name (string)
+* `datadir`: root of your `mysql_json` hierarchy (string)
+* `only_for`: only perform queries if conditions in this section are met (hash)
+* `ignore_json_parse_errors`: Do not raise an exception when the database
+  contains invalid json (boolean, defaults to false)
 
-running `hiera applications` would run the query against the configured database.
+A more complete example might contain:
 
+```yaml
+:mysql_json:
+  :datadir: /etc/puppet/hierasql
+  :host: db042.example.com
+  :user: puppetserver
+  :pass: secret123
+  :port: 3306
+  :ignore_json_parse_errors: true # why why why...
+  :only_for:
+    :fqdn:
+      - '^vagrant.+'
+      - '^node\d+\.someservice\.example\.com$'
+    :domain:
+      - '^vagrant.+
 
-### Using
+```
 
-`gem install hiera-mysql-backend`
+This will perform lookups if any of the given conditions are met:
 
+* The nodes fqdn fact starts with 'vagrant'
+* The nodes fqdn fact looks like node01.someservice.example.com
+* The nodes domain fact starts with 'vagrant'
 
-### Configuring Hiera
+Any node fact may be used to build these conditions.
 
-Hiera configuration is pretty simple
+Note that putting something in the list that always matches makes the whole
+`only_for` block useless.
 
-```yaml
----
-:backends:
-  - yaml
-  - mysql2
+### Defining queries
 
-:yaml:
-  :datadir: /etc/puppet/hieradata
+Queries are defined in he poorly named sql files. These  are really yaml files
+where the key is the lookup key and the value is the SQL statement (it accepts
+interpolation)
 
-:mysql2:
-  :datadir: /etc/puppet/hieradata
-  :host: hostname
-  :user: username
-  :pass: password
-  :database: database
-  :port: 3306
+As of version 0.0.4 you can also add connection information to these sql files,
+this allows you to connect to different databases. This is optional if no
+connection information is found it will use the default defined in your
+hiera.yaml config.
 
-:hierarchy:
-  - "%{::clientcert}"
-  - "%{::custom_location}"
-  - common
+Lets assume your _datadir_ is `/etc/puppet/hieradata/` and your hierarchy for
+hiera just have a common. hiera-mysql-backend would look for
+/etc/puppet/hieradata/common.sql the common.sql would look like:
 
-:logger: console
+```yaml
+---
+# This is optional, if not present it will use the default connection info from hiera.yaml
+:dbconfig:
+  :host: database.example.com
+  :user: hieratest
+  :pass: sekret
+  :database: testhieradb
+  :port: 44445
+
+applications: SELECT value FROM applications WHERE host='%{fqdn}';
 ```
 
 If `host` is not defined it will use `localhost` as default.
 
 If `port` is not defined it will use the default `3306` mysql port
 
+Running `hiera applications` would run the query against the configured
+database, parse the result as json and return the resulting data structure. If
+all you want is a string, strings are valid json, too.
+
+
+### Error handling
+
+When encountering invalid json, it will raise an exception, which would in turn
+cause catalog compilation to fail. You can disable this behaviour by setting
+`ignore_json_parse_errors`.
+
+When no results are returned by your query, it will return nil.
+
+When multiple results are returned by your query, it will return nil. A future
+version will introduce `ignore_multiple_results`, defaulting to true, to make
+it possible to trigger a catalog compilation in this case.
+
+
 ## Known issues
 
+1. Multiple results are currently silently ignored.
 1. It always return an Array of hashes regardless of the number of items returned. (I did this on purpose because it is what I needed but I may be persuaded to do otherwise)
 2. This README is poorly written.
 

data/hiera-mysql-json-backend-puppetserver.gemspec

@@ -2,7 +2,7 @@
 
 Gem::Specification.new do |gem|
   gem.name          = "hiera-mysql-json-backend-jruby"
-  gem.version       = "1.1.0"
+  gem.version       = "1.1.1"
   gem.authors       = ["Hostnet"]
   gem.email         = ["opensource@hostnet.nl"]
   gem.description   = %q{Alternative MySQL backend with json support for hiera}

data/hiera-mysql-json-backend.gemspec

@@ -2,7 +2,7 @@
 
 Gem::Specification.new do |gem|
   gem.name          = "hiera-mysql-json-backend"
-  gem.version       = "1.1.0"
+  gem.version       = "1.1.1"
   gem.authors       = ["Hostnet"]
   gem.email         = ["opensource@hostnet.nl"]
   gem.description   = %q{Alternative MySQL backend with json support for hiera}

data/lib/hiera/backend/mysql_json_backend.rb

@@ -25,7 +25,7 @@ class Hiera
         return true unless constraints.is_a?(Hash)
         should_lookup = false
         constraints.each do |item, matchers|
-          next unless scope.exist?(item.to_s)
+          next unless scope[item.to_s]
           if scope[item.to_s] =~ compile_regexes(matchers)
             should_lookup = true
             break
@@ -85,7 +85,7 @@ class Hiera
           new_answer = Backend.parse_answer(data[key], scope)
 
           sql_results = query(connection_hash, new_answer)
-
+          # TODO: make sure we fail if we have more than 1 result, skip if less than 1.
           next if sql_results.length != 1
           begin
             new_answer = JSON.parse(sql_results[0]['value'])

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: hiera-mysql-json-backend
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.1.1
 platform: ruby
 authors:
 - Hostnet
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-07-18 00:00:00.000000000 Z
+date: 2017-08-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: mysql2