cmdb 2.6.2 → 3.0.0rc1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.dockerignore +7 -0
- data/.gitignore +3 -0
- data/CHANGELOG.md +71 -34
- data/Gemfile +4 -3
- data/Gemfile.lock +6 -19
- data/LICENSE +25 -0
- data/README.md +138 -166
- data/Rakefile +13 -0
- data/cmdb.gemspec +1 -3
- data/docker-compose.yml +15 -0
- data/exe/cmdb +36 -9
- data/lib/cmdb/commands/help.rb +21 -66
- data/lib/cmdb/commands/shell.rb +165 -0
- data/lib/cmdb/commands/shim.rb +13 -128
- data/lib/cmdb/commands.rb +1 -0
- data/lib/cmdb/interface.rb +51 -59
- data/lib/cmdb/shell/dsl.rb +73 -0
- data/lib/cmdb/shell/printer.rb +115 -0
- data/lib/cmdb/shell/text.rb +65 -0
- data/lib/cmdb/shell.rb +8 -0
- data/lib/cmdb/source/consul.rb +90 -0
- data/lib/cmdb/{file_source.rb → source/file.rb} +15 -34
- data/lib/cmdb/source/memory.rb +39 -0
- data/lib/cmdb/source/network.rb +104 -0
- data/lib/cmdb/source.rb +94 -0
- data/lib/cmdb/version.rb +1 -1
- data/lib/cmdb.rb +26 -17
- metadata +18 -36
- data/TODO.md +0 -3
- data/lib/cmdb/consul_source.rb +0 -83
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3e6aea8a8e330452b914eae78b2909200d62aca4
|
4
|
+
data.tar.gz: ffcb1a5762568246b37e3924c793f29d36b7e3ab
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f20fe6ab9ff05e6f3450117f21abbc91b215bdedb55cfee11b6ad8bfba2937826536862f8d7593a3ec60d845424288d6aab3340691bf58957d586dab37d0c4ca
|
7
|
+
data.tar.gz: 59dd521b9730b77d8acea1ba8587a04bc6cb172f67e79cfbfe2eda09825bc3107fd7bc799824f7a823ae00295f3509418523066cf3cc6d55694d727aadff30a8
|
data/.dockerignore
ADDED
data/.gitignore
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,46 +1,83 @@
|
|
1
|
-
###
|
1
|
+
### 3.0.0rc1
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
|
3
|
+
With this major version we introduce significant compatibility-breaking changes!
|
4
|
+
Functions and options have been removed and we have rethought how sources are
|
5
|
+
specified.
|
6
6
|
|
7
|
-
|
7
|
+
#### New: Interactive shell
|
8
8
|
|
9
|
-
|
10
|
-
|
9
|
+
A `shell` subcommand allows you to navigate the CMDB as if it were a
|
10
|
+
filesystem with `ls`, `cd` and other commands.
|
11
11
|
|
12
|
-
|
13
|
-
the shard identifier (shard403, shard93, etc) as the prefix. Also prefix is optional so
|
14
|
-
in dev environments if you don't have these shard identifier as prefixes, you don't have
|
15
|
-
to specify it. When replacing keys in configuration files, the requested keys are loaded
|
16
|
-
from consul on demand.
|
12
|
+
#### New: Read/write sources
|
17
13
|
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
--dir=config -- rainbows -c config/rainbows.rb
|
22
|
-
```
|
14
|
+
Non-file sources allow keys to be written as well as read. You can set the values
|
15
|
+
of keys from the shell, or programatically by calling the `#set` method of the
|
16
|
+
source.
|
23
17
|
|
24
|
-
|
25
|
-
------------------------------------------------
|
18
|
+
#### Removed: Environment sources
|
26
19
|
|
27
|
-
|
28
|
-
|
29
|
-
|
20
|
+
We no longer treat the process environment as a _source_ for CMDB data, but
|
21
|
+
rather a destination; the use cases for treating it as a source were spurious
|
22
|
+
and mostly concerned with testing of this gem.
|
30
23
|
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
```
|
24
|
+
The `--env` option to the `cmdb shim` command is now the default behavior and
|
25
|
+
there is no way _not_ to populate the environment. We retain the safeguards
|
26
|
+
that cause CMDB to raise an error if two sources would populate the environment
|
27
|
+
with the same key name; overlap/inheritance is still disallowed.
|
36
28
|
|
37
|
-
|
29
|
+
#### Removed: Dichotomy between key enumeration and single-get
|
38
30
|
|
39
|
-
|
31
|
+
In v2, calls to `#get` were required to include a source's prefix in the key
|
32
|
+
name (e.g. `common.sandwich.size` whereas the key names returned from
|
33
|
+
`#each_pair` lacked a prefix (e.g. `sandwich.size`). This was confusing and
|
34
|
+
served no real purpose, so all methods have been normalized to always include
|
35
|
+
the prefix in the key name.
|
40
36
|
|
41
|
-
|
37
|
+
Naturally, sources with a nil prefix do not enforce this requirement!
|
42
38
|
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
39
|
+
It is safe to call `#get` on a source whose prefix does not match; the source
|
40
|
+
will simply return nil. However, calling `#set` with an invalid key prefix
|
41
|
+
will cause settable sources to raise `CMDB::BadKey`.
|
42
|
+
|
43
|
+
#### Changed: Command-line interface
|
44
|
+
|
45
|
+
The `cmdb` command now supports a number of common options that apply to all
|
46
|
+
subcommands.
|
47
|
+
|
48
|
+
Rather than have a hodgepodge of CLI options for each type of source, all sources
|
49
|
+
are represented by URLs where the scheme tells the gem what type of source to
|
50
|
+
create: `file://`, or `consul://`. You can specify as many sources as
|
51
|
+
you'd like by passing `--source` option multiple times with different URLs.
|
52
|
+
|
53
|
+
If you omit `--source` entirely, the CLI will scan the network for common
|
54
|
+
locations of supported sources. If it finds nothing, it will exit with an
|
55
|
+
error.
|
56
|
+
|
57
|
+
#### Changed: data file locations
|
58
|
+
|
59
|
+
The gem no longer scans fixed directories for JSON/YML files on startup; you
|
60
|
+
must explicitly provide the locations of every file that has CMDB keys by
|
61
|
+
passing `--source=file:///foo/bar.yml` to the CLI.
|
62
|
+
|
63
|
+
### Changed: shim command
|
64
|
+
|
65
|
+
The `--dir` option has been renamed to `--rewrite` for clarity.
|
66
|
+
|
67
|
+
The `--reload` and --reload-signal` options are no longer supported; CMDB has
|
68
|
+
lost its ability to reload the app when your files change. As a result, we
|
69
|
+
no longer depend on the `listen` gem.
|
70
|
+
|
71
|
+
The option to specify a `--root` for the CMDB interface as a whole has been
|
72
|
+
removed.
|
73
|
+
|
74
|
+
#### Implementation changes
|
75
|
+
|
76
|
+
The dependency on Diplomat has been removed; we now speak to consul without a
|
77
|
+
middleman.
|
78
|
+
|
79
|
+
#### Design changes
|
80
|
+
|
81
|
+
All sources have a new common base class `Source`, which also serves as an
|
82
|
+
enclosing namespace for all derived classes (`Source::Consul`,
|
83
|
+
`Source::File`, etc).
|
data/Gemfile
CHANGED
@@ -4,11 +4,12 @@ source 'https://rubygems.org'
|
|
4
4
|
gemspec
|
5
5
|
|
6
6
|
gem 'rake'
|
7
|
-
gem 'backticks'
|
8
|
-
gem 'cucumber'
|
9
|
-
gem 'rspec'
|
10
7
|
|
11
8
|
group :test do
|
9
|
+
gem 'backticks', '1.0.0rc1'
|
10
|
+
gem 'docker-compose', '1.0.0rc2'
|
11
|
+
gem 'cucumber'
|
12
|
+
gem 'rspec'
|
12
13
|
gem 'rubocop'
|
13
14
|
gem 'webmock'
|
14
15
|
end
|
data/Gemfile.lock
CHANGED
@@ -1,9 +1,7 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
cmdb (
|
5
|
-
diplomat (~> 0.15)
|
6
|
-
listen (~> 3.0)
|
4
|
+
cmdb (3.0.0rc1)
|
7
5
|
trollop (~> 2.0)
|
8
6
|
|
9
7
|
GEM
|
@@ -11,7 +9,7 @@ GEM
|
|
11
9
|
specs:
|
12
10
|
addressable (2.4.0)
|
13
11
|
ast (2.2.0)
|
14
|
-
backticks (0.
|
12
|
+
backticks (1.0.0rc1)
|
15
13
|
builder (3.2.2)
|
16
14
|
byebug (8.2.2)
|
17
15
|
coderay (1.1.1)
|
@@ -29,22 +27,13 @@ GEM
|
|
29
27
|
gherkin (~> 3.2.0)
|
30
28
|
cucumber-wire (0.0.1)
|
31
29
|
diff-lcs (1.2.5)
|
32
|
-
|
33
|
-
|
34
|
-
json (~> 1.8)
|
35
|
-
faraday (0.9.2)
|
36
|
-
multipart-post (>= 1.2, < 3)
|
37
|
-
ffi (1.9.10)
|
30
|
+
docker-compose (1.0.0rc2)
|
31
|
+
backticks (>= 1.0.0rc1)
|
38
32
|
gherkin (3.2.0)
|
39
33
|
hashdiff (0.3.0)
|
40
|
-
json (1.8.3)
|
41
|
-
listen (3.0.6)
|
42
|
-
rb-fsevent (>= 0.9.3)
|
43
|
-
rb-inotify (>= 0.9.7)
|
44
34
|
method_source (0.8.2)
|
45
35
|
multi_json (1.11.2)
|
46
36
|
multi_test (0.1.2)
|
47
|
-
multipart-post (2.0.0)
|
48
37
|
parser (2.3.0.6)
|
49
38
|
ast (~> 2.2)
|
50
39
|
powerpack (0.1.1)
|
@@ -57,9 +46,6 @@ GEM
|
|
57
46
|
pry (~> 0.10)
|
58
47
|
rainbow (2.1.0)
|
59
48
|
rake (10.5.0)
|
60
|
-
rb-fsevent (0.9.7)
|
61
|
-
rb-inotify (0.9.7)
|
62
|
-
ffi (>= 0.5.0)
|
63
49
|
rspec (3.4.0)
|
64
50
|
rspec-core (~> 3.4.0)
|
65
51
|
rspec-expectations (~> 3.4.0)
|
@@ -94,10 +80,11 @@ PLATFORMS
|
|
94
80
|
ruby
|
95
81
|
|
96
82
|
DEPENDENCIES
|
97
|
-
backticks
|
83
|
+
backticks (= 1.0.0rc1)
|
98
84
|
bundler (~> 1.10)
|
99
85
|
cmdb!
|
100
86
|
cucumber
|
87
|
+
docker-compose (= 1.0.0rc2)
|
101
88
|
pry
|
102
89
|
pry-byebug
|
103
90
|
rake
|
data/LICENSE
ADDED
@@ -0,0 +1,25 @@
|
|
1
|
+
License
|
2
|
+
-------
|
3
|
+
|
4
|
+
(The MIT License)
|
5
|
+
|
6
|
+
Copyright (c) 2015-2016 RightScale, Inc.
|
7
|
+
|
8
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
9
|
+
a copy of this software and associated documentation files (the
|
10
|
+
'Software'), to deal in the Software without restriction, including
|
11
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
12
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
13
|
+
permit persons to whom the Software is furnished to do so, subject to
|
14
|
+
the following conditions:
|
15
|
+
|
16
|
+
The above copyright notice and this permission notice shall be
|
17
|
+
included in all copies or substantial portions of the Software.
|
18
|
+
|
19
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
20
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
21
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
22
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
23
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
24
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
25
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
CHANGED
@@ -3,17 +3,12 @@
|
|
3
3
|
[![TravisCI][travis_ci_img]](https://travis-ci.org/rightscale/cmdb)
|
4
4
|
[travis_ci_img]: https://travis-ci.org/rightscale/cmdb.svg?branch=master
|
5
5
|
|
6
|
-
*NOTE:* this gem is under heavy development and it is likely that v3 will contain several interface-breaking
|
7
|
-
changes and simplifications. We encourage you to play with our toys and give us feedback on how you would
|
8
|
-
like to see the project evolve, but if you use this gem for production-grade software, please make sure to
|
9
|
-
pin to version `~> 2.6` in your Gemfile to avoid breakage!
|
10
|
-
|
11
6
|
CMDB is a Ruby interface for consuming data from one or more configuration management databases
|
12
7
|
(CMDBs) and making that information available to Web applications.
|
13
8
|
|
14
9
|
It is intended to support multiple CM technologies, including:
|
15
|
-
- JSON/YAML files on a local disk
|
16
10
|
- consul
|
11
|
+
- JSON/YAML files on a local disk
|
17
12
|
- (someday) etcd
|
18
13
|
- (someday) ZooKeeper
|
19
14
|
|
@@ -26,39 +21,127 @@ CMDB supports two primary use cases:
|
|
26
21
|
|
27
22
|
1. Decouple your modern (12-factor) application from the CM mechanism that is used to deploy it,
|
28
23
|
transforming CMDB keys and values into the enviroment variables that your app expects.
|
29
|
-
2.
|
30
|
-
disk files
|
24
|
+
2. Deploy legacy applications that expect their configuration to be
|
25
|
+
written to disk files by rewriting files at app-load time, substituting
|
26
|
+
CMDB variables into the files as required.
|
27
|
+
|
28
|
+
CMDB has three primary interfaces:
|
29
|
+
|
30
|
+
1. The `cmdb shim` command populates the environment with values and/or rewrites hardcoded
|
31
|
+
config files, then spawns your application.
|
32
|
+
2. The `CMDB::Interface` object provides a programmatic API for querying CMDBs. Its `#to_h`
|
33
|
+
method transforms the whole configuration into an environment-friendly hash if you prefer to seed the
|
34
|
+
environment yourself, without using the shim.
|
35
|
+
3. The `cmdb shell` command navigates your k/v store using filesystem-like
|
36
|
+
metaphors (`ls`, `cd`, and so forth)
|
37
|
+
|
38
|
+
# Data Model
|
39
|
+
|
40
|
+
CMDB models all data sources as hierarchical trees whose nodes are named, and
|
41
|
+
whose leaf nodes can contain a piece of data: strings, numbers, booleans, or
|
42
|
+
lists are all supported data types. Maps are disallowed on order to prevent
|
43
|
+
ambiguity; a map always represents a subtree of the k/v store, never a value.
|
44
|
+
|
45
|
+
This model is a "least common denominator" simplification of the data models of
|
46
|
+
YML, JSON, ZooKeeper and etcd; by disallowing maps as the values of keys, it
|
47
|
+
avoids ambiguity over whether a map should be treated as a subtree or as a
|
48
|
+
distinct value.
|
49
|
+
|
50
|
+
## Source Prefixes
|
51
|
+
|
52
|
+
Some CMDB sources have a `prefix`, indicating that _all_ keys contained in
|
53
|
+
that source begin with the same prefix. No two sources may share a prefix,
|
54
|
+
ensuring that sources don't "hide" each others' data. The prefix of a source is
|
55
|
+
usually automatically determined by the final component of its URL, e.g. the
|
56
|
+
filename in the case of `file://` sources and the final path component in the
|
57
|
+
case of `consul://` or other network sources.
|
58
|
+
|
59
|
+
## Inheritance
|
60
|
+
|
61
|
+
The uniqueness constraint on prefixes means that all sources' keys are
|
62
|
+
disjoint; there is no such thing as "inheritance" in the CMDB data model.
|
31
63
|
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
64
|
+
When keys are exported to the environment, the prefix is stripped from the
|
65
|
+
key name; however, CMDB _still_ prevents overlap in this case.
|
66
|
+
|
67
|
+
Inheritance may be supported in future as an optional behavior, but is omitted
|
68
|
+
for the time being because in practice, it causes more problems than it solves.
|
69
|
+
|
70
|
+
## Ambiguous Key Names
|
71
|
+
|
72
|
+
Consider a file that defines the following variables:
|
73
|
+
|
74
|
+
# confusing.yml
|
75
|
+
this:
|
76
|
+
is:
|
77
|
+
ambiguous
|
78
|
+
was:
|
79
|
+
very: ambiguous
|
80
|
+
extremely: confusing
|
81
|
+
|
82
|
+
At first glance, ths file defines two CMDB keys:
|
83
|
+
- `confusing.this.is` (a string)
|
84
|
+
- `confusing.this.was` (a map)
|
85
|
+
|
86
|
+
However, an equally valid interpretation would be:
|
87
|
+
- `confusing.this.is`
|
88
|
+
- `confusing.this.was.very`
|
89
|
+
- `confusing.this.was.extremely`
|
90
|
+
|
91
|
+
Because CMDB keys cannot contain maps, the first interpretation is wrong. The second
|
92
|
+
interpretation is valid according to the data model, but results in a situation where the type
|
93
|
+
of the keys could change if the structure of the YML file changes.
|
94
|
+
|
95
|
+
For this reason, any YAML file that defines an "ambiguous" key name will cause an error at
|
96
|
+
initialization time. To avoid ambiguous key names, think of your YAML file as a tree and remember
|
97
|
+
that _leaf nodes must define data_ and _internal nodes must define structure_.
|
39
98
|
|
40
99
|
# Getting Started
|
41
100
|
|
42
|
-
##
|
101
|
+
## Determine sources
|
102
|
+
|
103
|
+
Sources are specified with the `--source` option when you run the CLI. This
|
104
|
+
option applies to all subcommands (`shim`, `shell`, etc) and must appear
|
105
|
+
before the subcommand name.
|
106
|
+
|
107
|
+
You can add as many sources as you'd like. All sources are specified as a URI,
|
108
|
+
where the scheme tells CMDB which driver to use and how to interpret the rest
|
109
|
+
of the URI.
|
110
|
+
|
111
|
+
|
43
112
|
|
44
|
-
|
113
|
+
Sources can optionally have a "prefix" which is used as a common prefix of all
|
114
|
+
key names under the source. When CMDB can identify the prefix for your source,
|
115
|
+
it makes merge operations more efficient, helps provide semantic context
|
116
|
+
for your key names, and makes it easier to identify and avoid naming collisions
|
117
|
+
between different sources.
|
45
118
|
|
46
|
-
|
47
|
-
2. `~/.cmdb` -- useful for developers when testing the app
|
119
|
+
Examples:
|
48
120
|
|
49
|
-
|
50
|
-
the
|
51
|
-
`
|
121
|
+
* `file:///var/lib/cmdb/myapp.yml` creates a file source with the prefix
|
122
|
+
`myapp`; the value `foo: bar` in the file would have the key `myapp.foo`
|
123
|
+
* `consul://localhost` creates a source with no key prefix that talks to a local
|
124
|
+
consul agent on the standard port (8500); a value `foo/bar` in Consul would
|
125
|
+
have the key `foo.bar
|
126
|
+
* `consul://kv:18500/myapp` creates a source with the prefix `myapp.` that
|
127
|
+
talks to a remote consul agent on a nonstandard port (18500); this source
|
128
|
+
only "sees" Consul values under the path `/myapp/` and their key
|
129
|
+
names always begin with `myapp.`
|
130
|
+
* `consul://localhost/mycorp/staging/myapp` creates a source with the prefix
|
131
|
+
`myapp.`; this source only "sees" Consul values under the path
|
132
|
+
`staging/myapp` and their key names always begin with `myapp.`
|
133
|
+
* `consul://localhost/mycorp/staging` creates a source with the prefix `staging.`
|
134
|
+
that has all keys in the staging environment. (It is probably a bad idea to
|
135
|
+
use this source with the `myapp` source in the example above!)
|
52
136
|
|
53
|
-
|
54
|
-
|
137
|
+
If no sources are specified on the command line, CMDB will run an auto-detect
|
138
|
+
algorithm to check for network agents listening at localhost.
|
55
139
|
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
- chocolate
|
140
|
+
To learn more about sources and prefixes, see "Data model," below.
|
141
|
+
|
142
|
+
## Invoke the CMDB Shell
|
143
|
+
|
144
|
+
To enter an interactive sh-like shell, just type `cmdb shell`.
|
62
145
|
|
63
146
|
## Invoke the CMDB Shim
|
64
147
|
|
@@ -75,23 +158,30 @@ your application. The shim can do several things for you:
|
|
75
158
|
If you have an app that uses 12-factor (dotenv) style configuration, the shim
|
76
159
|
can populate the environment with CMDB values:
|
77
160
|
|
78
|
-
bundle exec cmdb shim
|
161
|
+
bundle exec cmdb shim
|
79
162
|
|
80
163
|
# Now your app can refer to ENV['DB_HOSTNAME'] or ENV['WIDGETS_FLAVORS]
|
81
164
|
# Note missing "my_app" prefix that would be present if you asked for these using their CMDB key names
|
82
165
|
|
166
|
+
Note that when we export CMDB keys into the environment, we _remove_ the prefix of
|
167
|
+
each key; in the example above, the values could have come from `common.db.hostname`
|
168
|
+
and `myapp.widgets.flavors` but their names have been simplified. If any two sources
|
169
|
+
have keys whose simplified names are identical, CMDB prints a detailed error message
|
170
|
+
and fails rather than putting ambiguous data into the environment.
|
171
|
+
|
83
172
|
Note that the data type of CMDB inputs is preserved: lists remain lists, numbers remain numbers,
|
84
173
|
and so forth. This works irrespective of the format of your configuration files, and also holds true
|
85
174
|
for CMDB values that are serialized to the environment (as a JSON document, in the case of lists).
|
86
175
|
|
87
|
-
###
|
176
|
+
### Rewrite static configuration files with dynamic CMDB values
|
88
177
|
|
89
|
-
If the `--
|
90
|
-
|
178
|
+
If the `--rewrite` option is provided, the shim recursively scans the provided
|
179
|
+
subdirectory for data files that contain replacement tokens; when a token is
|
91
180
|
found, it substitutes the corresponding CMDB key's value.
|
92
181
|
|
93
182
|
Replacement tokens look like this: `<<name.of.my.key>>` and can appear anywhere in a file as a YAML
|
94
|
-
or JSON _value_ (but never a key).
|
183
|
+
or JSON _value_ (but never a key). Unlike environment variables, replacement tokens always use
|
184
|
+
the fully-qualified key name, including prefix.
|
95
185
|
|
96
186
|
Replacement tokens should appear inside string literals in your configuration files so they don't
|
97
187
|
invalidate syntax or render the files unparsable by other tools.
|
@@ -100,10 +190,6 @@ The shim performs replacement in-memory and saves all of the edits at once, maki
|
|
100
190
|
operation nearly atomic. If any keys are missing, then no files are changed on disk and the shim
|
101
191
|
exits with a helpful error message.
|
102
192
|
|
103
|
-
*NOTE:* the shim does not perform rewriting in development mode; the expectation is that your app's
|
104
|
-
configuration files will already provide reasonable dev-mode defaults and that rewriting them
|
105
|
-
is not necessary.
|
106
|
-
|
107
193
|
Given `my_app.yml` and an application with two configuration files:
|
108
194
|
|
109
195
|
# config/database.yml
|
@@ -136,135 +222,21 @@ can do this for you. Just add the `--user` flag when you invoke it:
|
|
136
222
|
|
137
223
|
bundle exec cmdb shim --user=www-data whoami
|
138
224
|
|
139
|
-
|
140
|
-
|
141
|
-
You can pass the `--reload=key.name` option to `cmdb shim` in order to enable filesystem
|
142
|
-
watching. The shim will signal your application server whenever files are created,
|
143
|
-
updated or deleted, generally causing a graceful restart of the server process.
|
144
|
-
|
145
|
-
Needless to say, your app server must support graceful restart upon receipt of
|
146
|
-
a certain signal! The CMDB gem uses SIGHUP by default, but you can override this
|
147
|
-
with --reload-signal=SIGWHATEVER.
|
225
|
+
# Data Sources
|
148
226
|
|
149
|
-
|
150
|
-
watching is enabled. It *should* be a boolean, but *may* be nil or any "truthy" value such as a
|
151
|
-
number or string. If the key is truthy, then the shim will perform filesystem-watching.
|
227
|
+
## Network Servers
|
152
228
|
|
153
|
-
|
229
|
+
To read CMDB data from a consul server, add a CLI parameter such as
|
230
|
+
`--source=consul://some-host/key/subkey`. This will create a source whose
|
231
|
+
prefix is `subkey` that encompasses the subtree of the k/v store that lies
|
232
|
+
underneath `/key/subkey`.
|
154
233
|
|
155
|
-
|
234
|
+
## Flat Files
|
156
235
|
|
157
|
-
|
158
|
-
|
236
|
+
To read CMDB data from a flat file on disk, add a CLI parameter such as
|
237
|
+
`--source=file:///var/lib/cmdb/mykeys.yml`. This will parse the YAML file
|
238
|
+
located in `/var/lib/cmdb` and present it as a source whose prefix is `mykeys`.
|
159
239
|
|
160
|
-
|
161
|
-
|
162
|
-
|
163
|
-
|
164
|
-
This allows you to read CMDB values from the directly from your code.
|
165
|
-
|
166
|
-
# Data Model
|
167
|
-
|
168
|
-
This library models all CMDBs as hierarchical key/value stores whose leaf nodes can be strings,
|
169
|
-
numbers, or arrays of like-typed objects. This model is a "least common denominator" simplification
|
170
|
-
of the data models of YML, JSON, ZooKeeper and etcd, allowing all of those technologies to be
|
171
|
-
treated as interchangeable sources of configuration information.
|
172
|
-
|
173
|
-
CMDB key names consist of a dot-separated string e.g. `my_app.category_of_settings.some_value`. The
|
174
|
-
value of a CMDB key can be a string, boolean, number, nil, or a list of any of those types.
|
175
|
-
|
176
|
-
CMDB keys *cannot* contain maps/hashes, nor can lists contain differently-typed data.
|
177
|
-
|
178
|
-
When a CMDB key is accessed through the Ruby API or referenced with a file-rewrite <<token>>, its
|
179
|
-
name always begins with the file or path name of its *source* (JSON file, consul path, etc).
|
180
|
-
|
181
|
-
When a CMDB key is written into the process environment or accessed via `Source#to_h`, its name
|
182
|
-
is "bare" and the source name is irrelevant.
|
183
|
-
|
184
|
-
If we use a `--consul-prefix` of `/kv/rightscale/intregration/shard403/common`
|
185
|
-
then a key names would look like `common.debug.enabled` and environment names
|
186
|
-
would look like `DEBUG_ENABLED`. The same is true if we load a `common.json`
|
187
|
-
file source from `/var/lib/cmdb`.
|
188
|
-
|
189
|
-
A future version of cmdb will harmonize the treatment of names; the prefix
|
190
|
-
will be insignificant to the key name and keys will look like environment
|
191
|
-
variables.
|
192
|
-
|
193
|
-
## Network Data Sources
|
194
|
-
|
195
|
-
To read from a consul server, pass `--consul-url` with a consul server address
|
196
|
-
and `--consul-prefix` one or more times with a top-level path to treat as a
|
197
|
-
named source.
|
198
|
-
|
199
|
-
## Disk-Based Data Sources
|
200
|
-
|
201
|
-
When the CMDB interface is initialized, it searches two directories for YAML files:
|
202
|
-
- /var/lib/cmdb
|
203
|
-
- ~/.cmdb
|
204
|
-
|
205
|
-
YAML files in these directories are assumed to contain CMDB values and loaded into memory in the
|
206
|
-
order they are encountered. The hierarchy of the YAML keys is flattened in order to derive
|
207
|
-
dot-separated key names. Consider the following YAML file:
|
208
|
-
|
209
|
-
# beverages.yml
|
210
|
-
coffee:
|
211
|
-
- latte
|
212
|
-
- cappucino
|
213
|
-
- mocha
|
214
|
-
- macchiato
|
215
|
-
tea:
|
216
|
-
- chai
|
217
|
-
- herbal
|
218
|
-
- black
|
219
|
-
to_go: true
|
220
|
-
|
221
|
-
This defines three CMDB values: `beverages.coffee` (a list of four items), `beverages.tea`
|
222
|
-
(a list of three items), and `beverages.to_go` (a boolean).
|
223
|
-
|
224
|
-
### Key Namespaces
|
225
|
-
|
226
|
-
The name of a CMDB file is important; it defines a namespace for all of the variables contained
|
227
|
-
inside. No two files may share a name; therefore, no two CMDB keys can have the same name.
|
228
|
-
Likewise, all keys with a given prefix are guaranteed to come from the same source.
|
229
|
-
|
230
|
-
### Overlapping Namespaces
|
231
|
-
|
232
|
-
Because CMDB files can come from several directories, it's possible for two same-named data files
|
233
|
-
to define values in the same namespace. In this case, the behavior of RightService varies depending
|
234
|
-
on the value of RACK_ENV or RAILS_ENV:
|
235
|
-
|
236
|
-
- unset, development or test: CMDB chooses the highest-precedence file and ignores the others
|
237
|
-
after printing a warning. Files in `/etc` win over files in `$HOME`, which win over
|
238
|
-
files in the working directory.
|
239
|
-
|
240
|
-
- any other environment: CMDB fails with an error message that describes the problem and
|
241
|
-
the locations of the overlapping files.
|
242
|
-
|
243
|
-
### Ambiguous Key Names
|
244
|
-
|
245
|
-
Consider a file that defines the following variables:
|
246
|
-
|
247
|
-
# confusing.yml
|
248
|
-
this:
|
249
|
-
is:
|
250
|
-
ambiguous
|
251
|
-
was:
|
252
|
-
very: ambiguous
|
253
|
-
extremely: confusing
|
254
|
-
|
255
|
-
At first glance, ths file defines two CMDB keys:
|
256
|
-
- `confusing.this.is` (a string)
|
257
|
-
- `confusing.this.was` (a map)
|
258
|
-
|
259
|
-
However, an equally valid interpretation would be:
|
260
|
-
- `confusing.this.is`
|
261
|
-
- `confusing.this.was.very`
|
262
|
-
- `confusing.this.was.extremely`
|
263
|
-
|
264
|
-
Because CMDB keys cannot contain maps, the first interpretation is wrong. The second
|
265
|
-
interpretation is valid according to the data model, but results in a situation where the type
|
266
|
-
of the keys could change if the structure of the YML file changes.
|
267
|
-
|
268
|
-
For this reason, any YAML file that defines an "ambiguous" key name will cause an error at
|
269
|
-
initialization time. To avoid ambiguous key names, think of your YAML file as a tree and remember
|
270
|
-
that _leaf nodes must define data_ and _internal nodes must define structure_.
|
240
|
+
JSON and YAML files are both supported. The structured data within each file
|
241
|
+
can contain arbitrarily-deep subtrees which are interpreted as subkeys,
|
242
|
+
sub-subkeys and so forth.
|
data/Rakefile
CHANGED
@@ -15,3 +15,16 @@ require 'rspec/core/rake_task'
|
|
15
15
|
RSpec::Core::RakeTask.new(:spec)
|
16
16
|
|
17
17
|
task default: [:spec, :cucumber]
|
18
|
+
|
19
|
+
|
20
|
+
require 'docker/compose'
|
21
|
+
desc 'Create Consul source using Docker Compose and invoke cmdb shell'
|
22
|
+
task :sandbox do
|
23
|
+
compose = Docker::Compose::Session.new
|
24
|
+
compose.up 'consul', detached:true
|
25
|
+
mapper = Docker::Compose::Mapper.new(compose)
|
26
|
+
url = mapper.map('consul://consul:8500/sandbox')
|
27
|
+
|
28
|
+
lib = File.expand_path('../lib', __FILE__)
|
29
|
+
exec "ruby -I#{lib} -rpry exe/cmdb --source=#{url} shell"
|
30
|
+
end
|
data/cmdb.gemspec
CHANGED
@@ -19,10 +19,8 @@ Gem::Specification.new do |spec|
|
|
19
19
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
20
|
spec.require_paths = ['lib']
|
21
21
|
|
22
|
-
spec.required_ruby_version = Gem::Requirement.new('~> 2.
|
22
|
+
spec.required_ruby_version = Gem::Requirement.new('~> 2.1')
|
23
23
|
|
24
|
-
spec.add_dependency 'listen', '~> 3.0'
|
25
|
-
spec.add_dependency 'diplomat', '~> 0.15'
|
26
24
|
spec.add_dependency 'trollop', '~> 2.0'
|
27
25
|
|
28
26
|
spec.add_development_dependency 'bundler', '~> 1.10'
|