cmdb 2.6.2 → 3.0.0rc1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.dockerignore +7 -0
- data/.gitignore +3 -0
- data/CHANGELOG.md +71 -34
- data/Gemfile +4 -3
- data/Gemfile.lock +6 -19
- data/LICENSE +25 -0
- data/README.md +138 -166
- data/Rakefile +13 -0
- data/cmdb.gemspec +1 -3
- data/docker-compose.yml +15 -0
- data/exe/cmdb +36 -9
- data/lib/cmdb/commands/help.rb +21 -66
- data/lib/cmdb/commands/shell.rb +165 -0
- data/lib/cmdb/commands/shim.rb +13 -128
- data/lib/cmdb/commands.rb +1 -0
- data/lib/cmdb/interface.rb +51 -59
- data/lib/cmdb/shell/dsl.rb +73 -0
- data/lib/cmdb/shell/printer.rb +115 -0
- data/lib/cmdb/shell/text.rb +65 -0
- data/lib/cmdb/shell.rb +8 -0
- data/lib/cmdb/source/consul.rb +90 -0
- data/lib/cmdb/{file_source.rb → source/file.rb} +15 -34
- data/lib/cmdb/source/memory.rb +39 -0
- data/lib/cmdb/source/network.rb +104 -0
- data/lib/cmdb/source.rb +94 -0
- data/lib/cmdb/version.rb +1 -1
- data/lib/cmdb.rb +26 -17
- metadata +18 -36
- data/TODO.md +0 -3
- data/lib/cmdb/consul_source.rb +0 -83
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3e6aea8a8e330452b914eae78b2909200d62aca4
|
4
|
+
data.tar.gz: ffcb1a5762568246b37e3924c793f29d36b7e3ab
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f20fe6ab9ff05e6f3450117f21abbc91b215bdedb55cfee11b6ad8bfba2937826536862f8d7593a3ec60d845424288d6aab3340691bf58957d586dab37d0c4ca
|
7
|
+
data.tar.gz: 59dd521b9730b77d8acea1ba8587a04bc6cb172f67e79cfbfe2eda09825bc3107fd7bc799824f7a823ae00295f3509418523066cf3cc6d55694d727aadff30a8
|
data/.dockerignore
ADDED
data/.gitignore
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,46 +1,83 @@
|
|
1
|
-
###
|
1
|
+
### 3.0.0rc1
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
|
3
|
+
With this major version we introduce significant compatibility-breaking changes!
|
4
|
+
Functions and options have been removed and we have rethought how sources are
|
5
|
+
specified.
|
6
6
|
|
7
|
-
|
7
|
+
#### New: Interactive shell
|
8
8
|
|
9
|
-
|
10
|
-
|
9
|
+
A `shell` subcommand allows you to navigate the CMDB as if it were a
|
10
|
+
filesystem with `ls`, `cd` and other commands.
|
11
11
|
|
12
|
-
|
13
|
-
the shard identifier (shard403, shard93, etc) as the prefix. Also prefix is optional so
|
14
|
-
in dev environments if you don't have these shard identifier as prefixes, you don't have
|
15
|
-
to specify it. When replacing keys in configuration files, the requested keys are loaded
|
16
|
-
from consul on demand.
|
12
|
+
#### New: Read/write sources
|
17
13
|
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
--dir=config -- rainbows -c config/rainbows.rb
|
22
|
-
```
|
14
|
+
Non-file sources allow keys to be written as well as read. You can set the values
|
15
|
+
of keys from the shell, or programatically by calling the `#set` method of the
|
16
|
+
source.
|
23
17
|
|
24
|
-
|
25
|
-
------------------------------------------------
|
18
|
+
#### Removed: Environment sources
|
26
19
|
|
27
|
-
|
28
|
-
|
29
|
-
|
20
|
+
We no longer treat the process environment as a _source_ for CMDB data, but
|
21
|
+
rather a destination; the use cases for treating it as a source were spurious
|
22
|
+
and mostly concerned with testing of this gem.
|
30
23
|
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
```
|
24
|
+
The `--env` option to the `cmdb shim` command is now the default behavior and
|
25
|
+
there is no way _not_ to populate the environment. We retain the safeguards
|
26
|
+
that cause CMDB to raise an error if two sources would populate the environment
|
27
|
+
with the same key name; overlap/inheritance is still disallowed.
|
36
28
|
|
37
|
-
|
29
|
+
#### Removed: Dichotomy between key enumeration and single-get
|
38
30
|
|
39
|
-
|
31
|
+
In v2, calls to `#get` were required to include a source's prefix in the key
|
32
|
+
name (e.g. `common.sandwich.size` whereas the key names returned from
|
33
|
+
`#each_pair` lacked a prefix (e.g. `sandwich.size`). This was confusing and
|
34
|
+
served no real purpose, so all methods have been normalized to always include
|
35
|
+
the prefix in the key name.
|
40
36
|
|
41
|
-
|
37
|
+
Naturally, sources with a nil prefix do not enforce this requirement!
|
42
38
|
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
39
|
+
It is safe to call `#get` on a source whose prefix does not match; the source
|
40
|
+
will simply return nil. However, calling `#set` with an invalid key prefix
|
41
|
+
will cause settable sources to raise `CMDB::BadKey`.
|
42
|
+
|
43
|
+
#### Changed: Command-line interface
|
44
|
+
|
45
|
+
The `cmdb` command now supports a number of common options that apply to all
|
46
|
+
subcommands.
|
47
|
+
|
48
|
+
Rather than have a hodgepodge of CLI options for each type of source, all sources
|
49
|
+
are represented by URLs where the scheme tells the gem what type of source to
|
50
|
+
create: `file://`, or `consul://`. You can specify as many sources as
|
51
|
+
you'd like by passing `--source` option multiple times with different URLs.
|
52
|
+
|
53
|
+
If you omit `--source` entirely, the CLI will scan the network for common
|
54
|
+
locations of supported sources. If it finds nothing, it will exit with an
|
55
|
+
error.
|
56
|
+
|
57
|
+
#### Changed: data file locations
|
58
|
+
|
59
|
+
The gem no longer scans fixed directories for JSON/YML files on startup; you
|
60
|
+
must explicitly provide the locations of every file that has CMDB keys by
|
61
|
+
passing `--source=file:///foo/bar.yml` to the CLI.
|
62
|
+
|
63
|
+
### Changed: shim command
|
64
|
+
|
65
|
+
The `--dir` option has been renamed to `--rewrite` for clarity.
|
66
|
+
|
67
|
+
The `--reload` and --reload-signal` options are no longer supported; CMDB has
|
68
|
+
lost its ability to reload the app when your files change. As a result, we
|
69
|
+
no longer depend on the `listen` gem.
|
70
|
+
|
71
|
+
The option to specify a `--root` for the CMDB interface as a whole has been
|
72
|
+
removed.
|
73
|
+
|
74
|
+
#### Implementation changes
|
75
|
+
|
76
|
+
The dependency on Diplomat has been removed; we now speak to consul without a
|
77
|
+
middleman.
|
78
|
+
|
79
|
+
#### Design changes
|
80
|
+
|
81
|
+
All sources have a new common base class `Source`, which also serves as an
|
82
|
+
enclosing namespace for all derived classes (`Source::Consul`,
|
83
|
+
`Source::File`, etc).
|
data/Gemfile
CHANGED
@@ -4,11 +4,12 @@ source 'https://rubygems.org'
|
|
4
4
|
gemspec
|
5
5
|
|
6
6
|
gem 'rake'
|
7
|
-
gem 'backticks'
|
8
|
-
gem 'cucumber'
|
9
|
-
gem 'rspec'
|
10
7
|
|
11
8
|
group :test do
|
9
|
+
gem 'backticks', '1.0.0rc1'
|
10
|
+
gem 'docker-compose', '1.0.0rc2'
|
11
|
+
gem 'cucumber'
|
12
|
+
gem 'rspec'
|
12
13
|
gem 'rubocop'
|
13
14
|
gem 'webmock'
|
14
15
|
end
|
data/Gemfile.lock
CHANGED
@@ -1,9 +1,7 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
cmdb (
|
5
|
-
diplomat (~> 0.15)
|
6
|
-
listen (~> 3.0)
|
4
|
+
cmdb (3.0.0rc1)
|
7
5
|
trollop (~> 2.0)
|
8
6
|
|
9
7
|
GEM
|
@@ -11,7 +9,7 @@ GEM
|
|
11
9
|
specs:
|
12
10
|
addressable (2.4.0)
|
13
11
|
ast (2.2.0)
|
14
|
-
backticks (0.
|
12
|
+
backticks (1.0.0rc1)
|
15
13
|
builder (3.2.2)
|
16
14
|
byebug (8.2.2)
|
17
15
|
coderay (1.1.1)
|
@@ -29,22 +27,13 @@ GEM
|
|
29
27
|
gherkin (~> 3.2.0)
|
30
28
|
cucumber-wire (0.0.1)
|
31
29
|
diff-lcs (1.2.5)
|
32
|
-
|
33
|
-
|
34
|
-
json (~> 1.8)
|
35
|
-
faraday (0.9.2)
|
36
|
-
multipart-post (>= 1.2, < 3)
|
37
|
-
ffi (1.9.10)
|
30
|
+
docker-compose (1.0.0rc2)
|
31
|
+
backticks (>= 1.0.0rc1)
|
38
32
|
gherkin (3.2.0)
|
39
33
|
hashdiff (0.3.0)
|
40
|
-
json (1.8.3)
|
41
|
-
listen (3.0.6)
|
42
|
-
rb-fsevent (>= 0.9.3)
|
43
|
-
rb-inotify (>= 0.9.7)
|
44
34
|
method_source (0.8.2)
|
45
35
|
multi_json (1.11.2)
|
46
36
|
multi_test (0.1.2)
|
47
|
-
multipart-post (2.0.0)
|
48
37
|
parser (2.3.0.6)
|
49
38
|
ast (~> 2.2)
|
50
39
|
powerpack (0.1.1)
|
@@ -57,9 +46,6 @@ GEM
|
|
57
46
|
pry (~> 0.10)
|
58
47
|
rainbow (2.1.0)
|
59
48
|
rake (10.5.0)
|
60
|
-
rb-fsevent (0.9.7)
|
61
|
-
rb-inotify (0.9.7)
|
62
|
-
ffi (>= 0.5.0)
|
63
49
|
rspec (3.4.0)
|
64
50
|
rspec-core (~> 3.4.0)
|
65
51
|
rspec-expectations (~> 3.4.0)
|
@@ -94,10 +80,11 @@ PLATFORMS
|
|
94
80
|
ruby
|
95
81
|
|
96
82
|
DEPENDENCIES
|
97
|
-
backticks
|
83
|
+
backticks (= 1.0.0rc1)
|
98
84
|
bundler (~> 1.10)
|
99
85
|
cmdb!
|
100
86
|
cucumber
|
87
|
+
docker-compose (= 1.0.0rc2)
|
101
88
|
pry
|
102
89
|
pry-byebug
|
103
90
|
rake
|
data/LICENSE
ADDED
@@ -0,0 +1,25 @@
|
|
1
|
+
License
|
2
|
+
-------
|
3
|
+
|
4
|
+
(The MIT License)
|
5
|
+
|
6
|
+
Copyright (c) 2015-2016 RightScale, Inc.
|
7
|
+
|
8
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
9
|
+
a copy of this software and associated documentation files (the
|
10
|
+
'Software'), to deal in the Software without restriction, including
|
11
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
12
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
13
|
+
permit persons to whom the Software is furnished to do so, subject to
|
14
|
+
the following conditions:
|
15
|
+
|
16
|
+
The above copyright notice and this permission notice shall be
|
17
|
+
included in all copies or substantial portions of the Software.
|
18
|
+
|
19
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
20
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
21
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
22
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
23
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
24
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
25
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
CHANGED
@@ -3,17 +3,12 @@
|
|
3
3
|
[![TravisCI][travis_ci_img]](https://travis-ci.org/rightscale/cmdb)
|
4
4
|
[travis_ci_img]: https://travis-ci.org/rightscale/cmdb.svg?branch=master
|
5
5
|
|
6
|
-
*NOTE:* this gem is under heavy development and it is likely that v3 will contain several interface-breaking
|
7
|
-
changes and simplifications. We encourage you to play with our toys and give us feedback on how you would
|
8
|
-
like to see the project evolve, but if you use this gem for production-grade software, please make sure to
|
9
|
-
pin to version `~> 2.6` in your Gemfile to avoid breakage!
|
10
|
-
|
11
6
|
CMDB is a Ruby interface for consuming data from one or more configuration management databases
|
12
7
|
(CMDBs) and making that information available to Web applications.
|
13
8
|
|
14
9
|
It is intended to support multiple CM technologies, including:
|
15
|
-
- JSON/YAML files on a local disk
|
16
10
|
- consul
|
11
|
+
- JSON/YAML files on a local disk
|
17
12
|
- (someday) etcd
|
18
13
|
- (someday) ZooKeeper
|
19
14
|
|
@@ -26,39 +21,127 @@ CMDB supports two primary use cases:
|
|
26
21
|
|
27
22
|
1. Decouple your modern (12-factor) application from the CM mechanism that is used to deploy it,
|
28
23
|
transforming CMDB keys and values into the enviroment variables that your app expects.
|
29
|
-
2.
|
30
|
-
disk files
|
24
|
+
2. Deploy legacy applications that expect their configuration to be
|
25
|
+
written to disk files by rewriting files at app-load time, substituting
|
26
|
+
CMDB variables into the files as required.
|
27
|
+
|
28
|
+
CMDB has three primary interfaces:
|
29
|
+
|
30
|
+
1. The `cmdb shim` command populates the environment with values and/or rewrites hardcoded
|
31
|
+
config files, then spawns your application.
|
32
|
+
2. The `CMDB::Interface` object provides a programmatic API for querying CMDBs. Its `#to_h`
|
33
|
+
method transforms the whole configuration into an environment-friendly hash if you prefer to seed the
|
34
|
+
environment yourself, without using the shim.
|
35
|
+
3. The `cmdb shell` command navigates your k/v store using filesystem-like
|
36
|
+
metaphors (`ls`, `cd`, and so forth)
|
37
|
+
|
38
|
+
# Data Model
|
39
|
+
|
40
|
+
CMDB models all data sources as hierarchical trees whose nodes are named, and
|
41
|
+
whose leaf nodes can contain a piece of data: strings, numbers, booleans, or
|
42
|
+
lists are all supported data types. Maps are disallowed on order to prevent
|
43
|
+
ambiguity; a map always represents a subtree of the k/v store, never a value.
|
44
|
+
|
45
|
+
This model is a "least common denominator" simplification of the data models of
|
46
|
+
YML, JSON, ZooKeeper and etcd; by disallowing maps as the values of keys, it
|
47
|
+
avoids ambiguity over whether a map should be treated as a subtree or as a
|
48
|
+
distinct value.
|
49
|
+
|
50
|
+
## Source Prefixes
|
51
|
+
|
52
|
+
Some CMDB sources have a `prefix`, indicating that _all_ keys contained in
|
53
|
+
that source begin with the same prefix. No two sources may share a prefix,
|
54
|
+
ensuring that sources don't "hide" each others' data. The prefix of a source is
|
55
|
+
usually automatically determined by the final component of its URL, e.g. the
|
56
|
+
filename in the case of `file://` sources and the final path component in the
|
57
|
+
case of `consul://` or other network sources.
|
58
|
+
|
59
|
+
## Inheritance
|
60
|
+
|
61
|
+
The uniqueness constraint on prefixes means that all sources' keys are
|
62
|
+
disjoint; there is no such thing as "inheritance" in the CMDB data model.
|
31
63
|
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
64
|
+
When keys are exported to the environment, the prefix is stripped from the
|
65
|
+
key name; however, CMDB _still_ prevents overlap in this case.
|
66
|
+
|
67
|
+
Inheritance may be supported in future as an optional behavior, but is omitted
|
68
|
+
for the time being because in practice, it causes more problems than it solves.
|
69
|
+
|
70
|
+
## Ambiguous Key Names
|
71
|
+
|
72
|
+
Consider a file that defines the following variables:
|
73
|
+
|
74
|
+
# confusing.yml
|
75
|
+
this:
|
76
|
+
is:
|
77
|
+
ambiguous
|
78
|
+
was:
|
79
|
+
very: ambiguous
|
80
|
+
extremely: confusing
|
81
|
+
|
82
|
+
At first glance, ths file defines two CMDB keys:
|
83
|
+
- `confusing.this.is` (a string)
|
84
|
+
- `confusing.this.was` (a map)
|
85
|
+
|
86
|
+
However, an equally valid interpretation would be:
|
87
|
+
- `confusing.this.is`
|
88
|
+
- `confusing.this.was.very`
|
89
|
+
- `confusing.this.was.extremely`
|
90
|
+
|
91
|
+
Because CMDB keys cannot contain maps, the first interpretation is wrong. The second
|
92
|
+
interpretation is valid according to the data model, but results in a situation where the type
|
93
|
+
of the keys could change if the structure of the YML file changes.
|
94
|
+
|
95
|
+
For this reason, any YAML file that defines an "ambiguous" key name will cause an error at
|
96
|
+
initialization time. To avoid ambiguous key names, think of your YAML file as a tree and remember
|
97
|
+
that _leaf nodes must define data_ and _internal nodes must define structure_.
|
39
98
|
|
40
99
|
# Getting Started
|
41
100
|
|
42
|
-
##
|
101
|
+
## Determine sources
|
102
|
+
|
103
|
+
Sources are specified with the `--source` option when you run the CLI. This
|
104
|
+
option applies to all subcommands (`shim`, `shell`, etc) and must appear
|
105
|
+
before the subcommand name.
|
106
|
+
|
107
|
+
You can add as many sources as you'd like. All sources are specified as a URI,
|
108
|
+
where the scheme tells CMDB which driver to use and how to interpret the rest
|
109
|
+
of the URI.
|
110
|
+
|
111
|
+
|
43
112
|
|
44
|
-
|
113
|
+
Sources can optionally have a "prefix" which is used as a common prefix of all
|
114
|
+
key names under the source. When CMDB can identify the prefix for your source,
|
115
|
+
it makes merge operations more efficient, helps provide semantic context
|
116
|
+
for your key names, and makes it easier to identify and avoid naming collisions
|
117
|
+
between different sources.
|
45
118
|
|
46
|
-
|
47
|
-
2. `~/.cmdb` -- useful for developers when testing the app
|
119
|
+
Examples:
|
48
120
|
|
49
|
-
|
50
|
-
the
|
51
|
-
`
|
121
|
+
* `file:///var/lib/cmdb/myapp.yml` creates a file source with the prefix
|
122
|
+
`myapp`; the value `foo: bar` in the file would have the key `myapp.foo`
|
123
|
+
* `consul://localhost` creates a source with no key prefix that talks to a local
|
124
|
+
consul agent on the standard port (8500); a value `foo/bar` in Consul would
|
125
|
+
have the key `foo.bar
|
126
|
+
* `consul://kv:18500/myapp` creates a source with the prefix `myapp.` that
|
127
|
+
talks to a remote consul agent on a nonstandard port (18500); this source
|
128
|
+
only "sees" Consul values under the path `/myapp/` and their key
|
129
|
+
names always begin with `myapp.`
|
130
|
+
* `consul://localhost/mycorp/staging/myapp` creates a source with the prefix
|
131
|
+
`myapp.`; this source only "sees" Consul values under the path
|
132
|
+
`staging/myapp` and their key names always begin with `myapp.`
|
133
|
+
* `consul://localhost/mycorp/staging` creates a source with the prefix `staging.`
|
134
|
+
that has all keys in the staging environment. (It is probably a bad idea to
|
135
|
+
use this source with the `myapp` source in the example above!)
|
52
136
|
|
53
|
-
|
54
|
-
|
137
|
+
If no sources are specified on the command line, CMDB will run an auto-detect
|
138
|
+
algorithm to check for network agents listening at localhost.
|
55
139
|
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
- chocolate
|
140
|
+
To learn more about sources and prefixes, see "Data model," below.
|
141
|
+
|
142
|
+
## Invoke the CMDB Shell
|
143
|
+
|
144
|
+
To enter an interactive sh-like shell, just type `cmdb shell`.
|
62
145
|
|
63
146
|
## Invoke the CMDB Shim
|
64
147
|
|
@@ -75,23 +158,30 @@ your application. The shim can do several things for you:
|
|
75
158
|
If you have an app that uses 12-factor (dotenv) style configuration, the shim
|
76
159
|
can populate the environment with CMDB values:
|
77
160
|
|
78
|
-
bundle exec cmdb shim
|
161
|
+
bundle exec cmdb shim
|
79
162
|
|
80
163
|
# Now your app can refer to ENV['DB_HOSTNAME'] or ENV['WIDGETS_FLAVORS]
|
81
164
|
# Note missing "my_app" prefix that would be present if you asked for these using their CMDB key names
|
82
165
|
|
166
|
+
Note that when we export CMDB keys into the environment, we _remove_ the prefix of
|
167
|
+
each key; in the example above, the values could have come from `common.db.hostname`
|
168
|
+
and `myapp.widgets.flavors` but their names have been simplified. If any two sources
|
169
|
+
have keys whose simplified names are identical, CMDB prints a detailed error message
|
170
|
+
and fails rather than putting ambiguous data into the environment.
|
171
|
+
|
83
172
|
Note that the data type of CMDB inputs is preserved: lists remain lists, numbers remain numbers,
|
84
173
|
and so forth. This works irrespective of the format of your configuration files, and also holds true
|
85
174
|
for CMDB values that are serialized to the environment (as a JSON document, in the case of lists).
|
86
175
|
|
87
|
-
###
|
176
|
+
### Rewrite static configuration files with dynamic CMDB values
|
88
177
|
|
89
|
-
If the `--
|
90
|
-
|
178
|
+
If the `--rewrite` option is provided, the shim recursively scans the provided
|
179
|
+
subdirectory for data files that contain replacement tokens; when a token is
|
91
180
|
found, it substitutes the corresponding CMDB key's value.
|
92
181
|
|
93
182
|
Replacement tokens look like this: `<<name.of.my.key>>` and can appear anywhere in a file as a YAML
|
94
|
-
or JSON _value_ (but never a key).
|
183
|
+
or JSON _value_ (but never a key). Unlike environment variables, replacement tokens always use
|
184
|
+
the fully-qualified key name, including prefix.
|
95
185
|
|
96
186
|
Replacement tokens should appear inside string literals in your configuration files so they don't
|
97
187
|
invalidate syntax or render the files unparsable by other tools.
|
@@ -100,10 +190,6 @@ The shim performs replacement in-memory and saves all of the edits at once, maki
|
|
100
190
|
operation nearly atomic. If any keys are missing, then no files are changed on disk and the shim
|
101
191
|
exits with a helpful error message.
|
102
192
|
|
103
|
-
*NOTE:* the shim does not perform rewriting in development mode; the expectation is that your app's
|
104
|
-
configuration files will already provide reasonable dev-mode defaults and that rewriting them
|
105
|
-
is not necessary.
|
106
|
-
|
107
193
|
Given `my_app.yml` and an application with two configuration files:
|
108
194
|
|
109
195
|
# config/database.yml
|
@@ -136,135 +222,21 @@ can do this for you. Just add the `--user` flag when you invoke it:
|
|
136
222
|
|
137
223
|
bundle exec cmdb shim --user=www-data whoami
|
138
224
|
|
139
|
-
|
140
|
-
|
141
|
-
You can pass the `--reload=key.name` option to `cmdb shim` in order to enable filesystem
|
142
|
-
watching. The shim will signal your application server whenever files are created,
|
143
|
-
updated or deleted, generally causing a graceful restart of the server process.
|
144
|
-
|
145
|
-
Needless to say, your app server must support graceful restart upon receipt of
|
146
|
-
a certain signal! The CMDB gem uses SIGHUP by default, but you can override this
|
147
|
-
with --reload-signal=SIGWHATEVER.
|
225
|
+
# Data Sources
|
148
226
|
|
149
|
-
|
150
|
-
watching is enabled. It *should* be a boolean, but *may* be nil or any "truthy" value such as a
|
151
|
-
number or string. If the key is truthy, then the shim will perform filesystem-watching.
|
227
|
+
## Network Servers
|
152
228
|
|
153
|
-
|
229
|
+
To read CMDB data from a consul server, add a CLI parameter such as
|
230
|
+
`--source=consul://some-host/key/subkey`. This will create a source whose
|
231
|
+
prefix is `subkey` that encompasses the subtree of the k/v store that lies
|
232
|
+
underneath `/key/subkey`.
|
154
233
|
|
155
|
-
|
234
|
+
## Flat Files
|
156
235
|
|
157
|
-
|
158
|
-
|
236
|
+
To read CMDB data from a flat file on disk, add a CLI parameter such as
|
237
|
+
`--source=file:///var/lib/cmdb/mykeys.yml`. This will parse the YAML file
|
238
|
+
located in `/var/lib/cmdb` and present it as a source whose prefix is `mykeys`.
|
159
239
|
|
160
|
-
|
161
|
-
|
162
|
-
|
163
|
-
|
164
|
-
This allows you to read CMDB values from the directly from your code.
|
165
|
-
|
166
|
-
# Data Model
|
167
|
-
|
168
|
-
This library models all CMDBs as hierarchical key/value stores whose leaf nodes can be strings,
|
169
|
-
numbers, or arrays of like-typed objects. This model is a "least common denominator" simplification
|
170
|
-
of the data models of YML, JSON, ZooKeeper and etcd, allowing all of those technologies to be
|
171
|
-
treated as interchangeable sources of configuration information.
|
172
|
-
|
173
|
-
CMDB key names consist of a dot-separated string e.g. `my_app.category_of_settings.some_value`. The
|
174
|
-
value of a CMDB key can be a string, boolean, number, nil, or a list of any of those types.
|
175
|
-
|
176
|
-
CMDB keys *cannot* contain maps/hashes, nor can lists contain differently-typed data.
|
177
|
-
|
178
|
-
When a CMDB key is accessed through the Ruby API or referenced with a file-rewrite <<token>>, its
|
179
|
-
name always begins with the file or path name of its *source* (JSON file, consul path, etc).
|
180
|
-
|
181
|
-
When a CMDB key is written into the process environment or accessed via `Source#to_h`, its name
|
182
|
-
is "bare" and the source name is irrelevant.
|
183
|
-
|
184
|
-
If we use a `--consul-prefix` of `/kv/rightscale/intregration/shard403/common`
|
185
|
-
then a key names would look like `common.debug.enabled` and environment names
|
186
|
-
would look like `DEBUG_ENABLED`. The same is true if we load a `common.json`
|
187
|
-
file source from `/var/lib/cmdb`.
|
188
|
-
|
189
|
-
A future version of cmdb will harmonize the treatment of names; the prefix
|
190
|
-
will be insignificant to the key name and keys will look like environment
|
191
|
-
variables.
|
192
|
-
|
193
|
-
## Network Data Sources
|
194
|
-
|
195
|
-
To read from a consul server, pass `--consul-url` with a consul server address
|
196
|
-
and `--consul-prefix` one or more times with a top-level path to treat as a
|
197
|
-
named source.
|
198
|
-
|
199
|
-
## Disk-Based Data Sources
|
200
|
-
|
201
|
-
When the CMDB interface is initialized, it searches two directories for YAML files:
|
202
|
-
- /var/lib/cmdb
|
203
|
-
- ~/.cmdb
|
204
|
-
|
205
|
-
YAML files in these directories are assumed to contain CMDB values and loaded into memory in the
|
206
|
-
order they are encountered. The hierarchy of the YAML keys is flattened in order to derive
|
207
|
-
dot-separated key names. Consider the following YAML file:
|
208
|
-
|
209
|
-
# beverages.yml
|
210
|
-
coffee:
|
211
|
-
- latte
|
212
|
-
- cappucino
|
213
|
-
- mocha
|
214
|
-
- macchiato
|
215
|
-
tea:
|
216
|
-
- chai
|
217
|
-
- herbal
|
218
|
-
- black
|
219
|
-
to_go: true
|
220
|
-
|
221
|
-
This defines three CMDB values: `beverages.coffee` (a list of four items), `beverages.tea`
|
222
|
-
(a list of three items), and `beverages.to_go` (a boolean).
|
223
|
-
|
224
|
-
### Key Namespaces
|
225
|
-
|
226
|
-
The name of a CMDB file is important; it defines a namespace for all of the variables contained
|
227
|
-
inside. No two files may share a name; therefore, no two CMDB keys can have the same name.
|
228
|
-
Likewise, all keys with a given prefix are guaranteed to come from the same source.
|
229
|
-
|
230
|
-
### Overlapping Namespaces
|
231
|
-
|
232
|
-
Because CMDB files can come from several directories, it's possible for two same-named data files
|
233
|
-
to define values in the same namespace. In this case, the behavior of RightService varies depending
|
234
|
-
on the value of RACK_ENV or RAILS_ENV:
|
235
|
-
|
236
|
-
- unset, development or test: CMDB chooses the highest-precedence file and ignores the others
|
237
|
-
after printing a warning. Files in `/etc` win over files in `$HOME`, which win over
|
238
|
-
files in the working directory.
|
239
|
-
|
240
|
-
- any other environment: CMDB fails with an error message that describes the problem and
|
241
|
-
the locations of the overlapping files.
|
242
|
-
|
243
|
-
### Ambiguous Key Names
|
244
|
-
|
245
|
-
Consider a file that defines the following variables:
|
246
|
-
|
247
|
-
# confusing.yml
|
248
|
-
this:
|
249
|
-
is:
|
250
|
-
ambiguous
|
251
|
-
was:
|
252
|
-
very: ambiguous
|
253
|
-
extremely: confusing
|
254
|
-
|
255
|
-
At first glance, ths file defines two CMDB keys:
|
256
|
-
- `confusing.this.is` (a string)
|
257
|
-
- `confusing.this.was` (a map)
|
258
|
-
|
259
|
-
However, an equally valid interpretation would be:
|
260
|
-
- `confusing.this.is`
|
261
|
-
- `confusing.this.was.very`
|
262
|
-
- `confusing.this.was.extremely`
|
263
|
-
|
264
|
-
Because CMDB keys cannot contain maps, the first interpretation is wrong. The second
|
265
|
-
interpretation is valid according to the data model, but results in a situation where the type
|
266
|
-
of the keys could change if the structure of the YML file changes.
|
267
|
-
|
268
|
-
For this reason, any YAML file that defines an "ambiguous" key name will cause an error at
|
269
|
-
initialization time. To avoid ambiguous key names, think of your YAML file as a tree and remember
|
270
|
-
that _leaf nodes must define data_ and _internal nodes must define structure_.
|
240
|
+
JSON and YAML files are both supported. The structured data within each file
|
241
|
+
can contain arbitrarily-deep subtrees which are interpreted as subkeys,
|
242
|
+
sub-subkeys and so forth.
|
data/Rakefile
CHANGED
@@ -15,3 +15,16 @@ require 'rspec/core/rake_task'
|
|
15
15
|
RSpec::Core::RakeTask.new(:spec)
|
16
16
|
|
17
17
|
task default: [:spec, :cucumber]
|
18
|
+
|
19
|
+
|
20
|
+
require 'docker/compose'
|
21
|
+
desc 'Create Consul source using Docker Compose and invoke cmdb shell'
|
22
|
+
task :sandbox do
|
23
|
+
compose = Docker::Compose::Session.new
|
24
|
+
compose.up 'consul', detached:true
|
25
|
+
mapper = Docker::Compose::Mapper.new(compose)
|
26
|
+
url = mapper.map('consul://consul:8500/sandbox')
|
27
|
+
|
28
|
+
lib = File.expand_path('../lib', __FILE__)
|
29
|
+
exec "ruby -I#{lib} -rpry exe/cmdb --source=#{url} shell"
|
30
|
+
end
|
data/cmdb.gemspec
CHANGED
@@ -19,10 +19,8 @@ Gem::Specification.new do |spec|
|
|
19
19
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
20
|
spec.require_paths = ['lib']
|
21
21
|
|
22
|
-
spec.required_ruby_version = Gem::Requirement.new('~> 2.
|
22
|
+
spec.required_ruby_version = Gem::Requirement.new('~> 2.1')
|
23
23
|
|
24
|
-
spec.add_dependency 'listen', '~> 3.0'
|
25
|
-
spec.add_dependency 'diplomat', '~> 0.15'
|
26
24
|
spec.add_dependency 'trollop', '~> 2.0'
|
27
25
|
|
28
26
|
spec.add_development_dependency 'bundler', '~> 1.10'
|