knife-topo 1.1.2 → 2.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +125 -148
- data/knife-topo.gemspec +12 -12
- data/lib/chef/knife/topo/bootstrap_helper.rb +75 -0
- data/lib/chef/knife/topo/command_helper.rb +71 -0
- data/lib/chef/knife/topo/consts.rb +4 -0
- data/lib/chef/knife/topo/loader.rb +137 -0
- data/lib/chef/knife/topo/node_update_helper.rb +111 -0
- data/lib/chef/knife/topo/processor.rb +76 -0
- data/lib/chef/knife/topo/processor/via_cookbook.rb +118 -0
- data/lib/chef/knife/topo/processor/via_cookbook_print.rb +97 -0
- data/lib/chef/knife/topo/version.rb +3 -4
- data/lib/chef/knife/topo_bootstrap.rb +107 -79
- data/lib/chef/knife/topo_cookbook_create.rb +41 -144
- data/lib/chef/knife/topo_cookbook_upload.rb +40 -54
- data/lib/chef/knife/topo_create.rb +120 -132
- data/lib/chef/knife/topo_delete.rb +67 -65
- data/lib/chef/knife/topo_export.rb +108 -133
- data/lib/chef/knife/topo_import.rb +62 -76
- data/lib/chef/knife/topo_list.rb +18 -25
- data/lib/chef/knife/topo_search.rb +61 -61
- data/lib/chef/knife/topo_update.rb +25 -110
- data/lib/chef/topo/converter.rb +75 -0
- data/lib/chef/topo/converter/topo_v1.rb +123 -0
- data/lib/chef/topology.rb +137 -0
- metadata +13 -3
- data/lib/chef/knife/topology_helper.rb +0 -366
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f48a892fd80f731049b38f49a1d872e3f66f8397
|
4
|
+
data.tar.gz: 04ee0c7a4dcbf577b41278e4efbc77439412799b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8fcc6ae3d4194d02c594549f7bbced00cab46b15e5caf320b0bf18f6141bd32bb48f7ac6d48d889ac768f508c6b2efb4d0d8ab58de6570d8ad8b447654146bdd
|
7
|
+
data.tar.gz: 5386a5e09a49ca90b4122a797cd54b56f002a8c57883065a0fcc9ee4e37971d46547b9fcae9df2c1250a3e893848d57771c6674e272828d7dda4e717e657188e
|
data/README.md
CHANGED
@@ -16,42 +16,63 @@ through a single (json) configuration file. It may also be useful
|
|
16
16
|
if you are regularly bringing up multi-node systems with similar
|
17
17
|
topologies but differences in their configuration details.
|
18
18
|
|
19
|
+
This plugin can be used in conjunction with the
|
20
|
+
[topo cookbook](http://github.com/christinedraper/topo-cookbook)
|
21
|
+
to configure dynamically deployed topologies of nodes (e.g. in AWS).
|
22
|
+
In this approach, use knife-topo to set up the topology details.
|
23
|
+
Use the 'topo' cookbook to let nodes pull their own detailed configuration
|
24
|
+
from the topology data bag on the Chef server.
|
19
25
|
|
20
|
-
#
|
26
|
+
# Changes from V1 #
|
27
|
+
|
28
|
+
## Attribute setting strategy
|
29
|
+
|
30
|
+
V2 introduces the notion of an attribute setting strategy. Instead of
|
31
|
+
specifying node attributes and cookbook attributes separately, you
|
32
|
+
specify one set of attributes and the method by which
|
33
|
+
they should be set on the node (e.g. 'direct_to_node' or 'via_cookbook').
|
34
|
+
|
35
|
+
The purpose of this change is to make it easier to support new
|
36
|
+
methods of setting attributes such as policy files, and also making it
|
37
|
+
easier to switch between methods.
|
21
38
|
|
22
|
-
|
23
|
-
unzip and copy `lib/chef/knife` into your plugin directory, e.g.:
|
39
|
+
## Node type
|
24
40
|
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
41
|
+
A node type can be specified for a node. The node type is used
|
42
|
+
by the 'topo' cookbook to identify the right configuration to use. It
|
43
|
+
is also used in the 'via cookbook' method to support attributes that
|
44
|
+
vary by node type.
|
29
45
|
|
30
|
-
|
46
|
+
## One topology per file
|
31
47
|
|
32
|
-
|
48
|
+
To reduce complexity, V2 no longer supports multiple topologies in a
|
49
|
+
JSON file.
|
33
50
|
|
34
|
-
|
35
|
-
embedded chefdk gem path), and you may need `sudo` depending on your setup.
|
51
|
+
## V1 -> V2 Migration
|
36
52
|
|
37
|
-
|
38
|
-
|
53
|
+
V1 topology JSON files can be converted to V2 by importing them and
|
54
|
+
then exporting them. knife-topo will auto-detect V1 format files on
|
55
|
+
import. You can also explicitly specify the input format using
|
56
|
+
`knife topo import sometopo.json --input-format 'topo_v1'`
|
39
57
|
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
the ruby plugin scripts into ~/.chef/plugins/knife) or install knife topo using the embedded gem:
|
58
|
+
# Installation #
|
59
|
+
|
60
|
+
Install knife-topo as a gem
|
44
61
|
|
45
|
-
|
62
|
+
```
|
63
|
+
$ chef gem install knife-topo
|
64
|
+
```
|
46
65
|
|
47
66
|
# Usage #
|
48
67
|
|
49
|
-
Define
|
68
|
+
Define each topology in a [topology file](#topology-file). Import
|
50
69
|
that file into your Chef workspace using [knife topo import](#import),
|
51
|
-
then create
|
52
|
-
|
53
|
-
|
54
|
-
|
70
|
+
then create the topology [knife topo create](#create), specifying
|
71
|
+
the '--bootstrap' option if you want to bootstrap all of the nodes.
|
72
|
+
|
73
|
+
Update the topology file as the configuration changes, import those
|
74
|
+
changes [knife topo import](#import) and run
|
75
|
+
[knife topo update](#update) to update the topology.
|
55
76
|
|
56
77
|
|
57
78
|
# Getting Started #
|
@@ -61,33 +82,32 @@ Try out this plugin using a [test-repo](test-repo) provided in the knife-topo gi
|
|
61
82
|
and unzip it, then follow the [Instructions](test-repo/Instructions.md) for the example.
|
62
83
|
|
63
84
|
The instructions assume you have [chefDK](https://downloads.chef.io/chef-dk/)
|
64
|
-
|
65
|
-
none of these are requirements to use the knife-topo plugin.
|
85
|
+
installed and working with Vagrant and VirtualBox.
|
66
86
|
|
67
87
|
If you're the sort of person who just wants to jump in and try it, here's some hints.
|
68
88
|
|
69
89
|
Generate a topology file for a topology called test1 from existing nodes node1 and node2:
|
70
90
|
|
71
|
-
|
91
|
+
knife topo export node1 node2 --topo test1 > test1.json
|
72
92
|
|
73
93
|
Import a topology json file, generating all of the necessary artifacts in your workspace:
|
74
94
|
|
75
|
-
knife topo import
|
95
|
+
knife topo import test1.json
|
76
96
|
|
77
97
|
Create the topology using existing nodes:
|
78
98
|
|
79
99
|
knife topo create test1
|
80
100
|
|
81
|
-
Create the topology bootstrapping new nodes in vagrant (you will need to add the
|
101
|
+
Create the topology, bootstrapping new nodes in vagrant (you will need to add the
|
82
102
|
host details for bootstrap to the file before importing):
|
83
103
|
|
84
104
|
knife topo create test1 --bootstrap -xvagrant -Pvagrant --sudo
|
85
105
|
|
86
106
|
# Topology File <a name="topology-file"></a>#
|
87
107
|
|
88
|
-
See the [example topology file](test-repo/
|
108
|
+
See the [example topology file](test-repo/test1.json)
|
89
109
|
|
90
|
-
The topology file contains a single topology
|
110
|
+
The topology file contains a single topology.
|
91
111
|
Each topology has some overall properties, an array of nodes and
|
92
112
|
an array defining topology cookbook attributes.
|
93
113
|
|
@@ -96,20 +116,20 @@ an array defining topology cookbook attributes.
|
|
96
116
|
```
|
97
117
|
{
|
98
118
|
"name": "test1",
|
119
|
+
"node_type": "appserver",
|
99
120
|
"chef_environment": "test",
|
100
|
-
"tags": [
|
121
|
+
"tags": [ "testsys" ],
|
101
122
|
"normal": {
|
102
|
-
|
103
|
-
|
104
|
-
|
123
|
+
"owner": {
|
124
|
+
"name": "Christine Draper"
|
125
|
+
}
|
105
126
|
},
|
106
127
|
"nodes" : [
|
107
128
|
...
|
108
|
-
],
|
109
|
-
"cookbook_attributes" : [
|
110
129
|
]
|
111
130
|
}
|
112
131
|
```
|
132
|
+
|
113
133
|
The `name` is how you will refer to the topology in the
|
114
134
|
`knife topo` subcommands.
|
115
135
|
|
@@ -117,6 +137,33 @@ The `chef-environment` and `normal` attributes defined
|
|
117
137
|
here will be applied to all nodes in the topology, unless alternative
|
118
138
|
values are provided for a specific node. The `tags`
|
119
139
|
will be added to each node.
|
140
|
+
|
141
|
+
### Attribute setting strategy
|
142
|
+
|
143
|
+
The default strategy for setting attributes is `direct_to_node`.
|
144
|
+
In this strategy, normal attributes are set directly on the nodes when
|
145
|
+
the topology is created, bootstrapped or updated. Attributes with
|
146
|
+
other priorities are ignored.
|
147
|
+
|
148
|
+
The `strategy` field can be set to 'via_cookbook', in which case
|
149
|
+
additional `strategy_data` can be provided to specify a cookbok
|
150
|
+
and attribute filename.
|
151
|
+
```
|
152
|
+
{
|
153
|
+
"name": "test1",
|
154
|
+
...
|
155
|
+
"strategy" : "via_cookbook",
|
156
|
+
"strategy_data": {
|
157
|
+
"cookbook": "topo_test1",
|
158
|
+
"filename": "softwareversion",
|
159
|
+
}
|
160
|
+
}
|
161
|
+
```
|
162
|
+
|
163
|
+
In this strategy, the cookbook and attribute file are generated
|
164
|
+
in the local workspace when the topology is imported, and uploaded
|
165
|
+
to the server when the topology is created or updated. Attributes
|
166
|
+
can have any valid priority.
|
120
167
|
|
121
168
|
## Node List <a name="node-list"></a>
|
122
169
|
Each topology contains a list of `nodes`.
|
@@ -127,47 +174,50 @@ Each topology contains a list of `nodes`.
|
|
127
174
|
...
|
128
175
|
"nodes": [
|
129
176
|
{
|
130
|
-
|
131
|
-
|
132
|
-
|
133
|
-
|
134
|
-
|
135
|
-
|
136
|
-
|
137
|
-
|
138
|
-
|
139
|
-
|
140
|
-
|
177
|
+
"name": "buildserver01",
|
178
|
+
"node_type" : "buildserver",
|
179
|
+
"ssh_host": "192.168.1.201",
|
180
|
+
"ssh_port": "2224",
|
181
|
+
"chef_environment": "dev",
|
182
|
+
"run_list": [
|
183
|
+
"role[base-ubuntu]",
|
184
|
+
"ypo::db",
|
185
|
+
"recipe[ypo::appserver]"
|
186
|
+
],
|
187
|
+
... node attributes, see below ...,
|
188
|
+
"tags": [ "build" ]
|
141
189
|
},
|
142
190
|
...
|
143
191
|
]
|
144
192
|
}
|
145
193
|
```
|
146
194
|
Within `nodes`, the `name` field is the node name that will be used in Chef.
|
147
|
-
The fields `chef_environment`, `run_list
|
148
|
-
|
195
|
+
The fields `chef_environment`, `run_list` and `tags`
|
196
|
+
will also be applied to the node in Chef. All of these
|
149
197
|
fields are optional.
|
150
198
|
|
151
|
-
The `
|
199
|
+
The `node_type` sets the node attribute `normal['topo']['node_type']`.
|
200
|
+
This attribute is used in the 'via_cookbook' strategy to specify
|
201
|
+
attributes that apply to only nodes of that type.
|
202
|
+
|
203
|
+
The `ssh_host` and `ssh_port` fields are used to
|
152
204
|
bootstrap a node.
|
153
205
|
|
154
|
-
##
|
206
|
+
## Node Attributes <a name="node-attributes"></a>
|
155
207
|
|
156
|
-
Each topology may have attributes that are set
|
157
|
-
|
158
|
-
|
159
|
-
array.
|
208
|
+
Each topology may have attributes that are set on each node
|
209
|
+
according to the attribute setting strategy. The attribute names and
|
210
|
+
values are specified by priority ('default', 'normal', 'override').
|
160
211
|
|
161
212
|
```
|
162
|
-
"
|
213
|
+
"nodes": [
|
163
214
|
{
|
164
|
-
|
165
|
-
"filename": "softwareversion",
|
215
|
+
"name": "buildserver01",
|
166
216
|
"normal":
|
167
217
|
{
|
168
218
|
"nodejs":
|
169
219
|
{
|
170
|
-
"version": "0.
|
220
|
+
"version": "0.10.40"
|
171
221
|
},
|
172
222
|
|
173
223
|
"testapp":
|
@@ -177,40 +227,13 @@ array.
|
|
177
227
|
|
178
228
|
"mongodb":
|
179
229
|
{
|
180
|
-
"package_version": "2.6.
|
181
|
-
}
|
182
|
-
},
|
183
|
-
"conditional" : [
|
184
|
-
{
|
185
|
-
"qualifier": "node_type",
|
186
|
-
"value" : "buildserver",
|
187
|
-
"normal":
|
188
|
-
{
|
189
|
-
"mongodb":
|
190
|
-
{
|
191
|
-
"package_version": "2.5.1"
|
192
|
-
}
|
193
|
-
}
|
230
|
+
"package_version": "2.6.9"
|
194
231
|
}
|
195
|
-
|
232
|
+
}
|
196
233
|
}
|
197
234
|
]
|
198
235
|
```
|
199
236
|
|
200
|
-
Attributes listed directly under an attribute priority (e.g. 'normal'
|
201
|
-
in the above) will generate an entry in the attribute file such as:
|
202
|
-
|
203
|
-
normal['mongodb'][package_version] = "2.6.1"
|
204
|
-
|
205
|
-
Attributes listed under the `conditional` property will generate an
|
206
|
-
entry in the attribute file such as:
|
207
|
-
|
208
|
-
```
|
209
|
-
if (node['topo']['node_type'] == "buildserver")
|
210
|
-
normal['mongodb']['package_version'] = "2.5.1"
|
211
|
-
end
|
212
|
-
```
|
213
|
-
|
214
237
|
# Subcommands <a name="subcommands"></a>
|
215
238
|
|
216
239
|
The main subcommands for `knife topo` are:
|
@@ -223,8 +246,6 @@ The additional subcommands can also be useful, depending on your
|
|
223
246
|
workflow:
|
224
247
|
|
225
248
|
* [knife topo bootstrap](#bootstrap)- Bootstraps a topology of nodes
|
226
|
-
* [knife topo cookbook create](#cookbook-create) - Generate the topology cookbooks
|
227
|
-
* [knife topo cookbook upload](#cookbook-upload) - Upload the topology cookbooks
|
228
249
|
* [knife export](#export) - Export data from a topology (or from nodes that you want in a topology)
|
229
250
|
* [knife topo list](#list) - List the topologies
|
230
251
|
* [knife topo search](#search) - Search for nodes that are in a topology, or in no topology
|
@@ -257,6 +278,7 @@ The knife topo bootstrap subcommand supports the following additional options.
|
|
257
278
|
|
258
279
|
Option | Description
|
259
280
|
------------ | -----------
|
281
|
+
--overwrite | Re-bootstrap existing nodes
|
260
282
|
See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html) | Options supported by `knife bootstrap` are passed through to the bootstrap command
|
261
283
|
|
262
284
|
|
@@ -265,49 +287,6 @@ The following will bootstrap nodes in the test1 topology, using a
|
|
265
287
|
user name of vagrant, password of vagrant, and running using sudo.
|
266
288
|
|
267
289
|
$ knife topo bootstrap test1 -x vagrant -P vagrant --sudo
|
268
|
-
|
269
|
-
## knife topo cookbook create <a name="cookbook-create"></a>
|
270
|
-
|
271
|
-
knife topo cookbook create TOPOLOGY
|
272
|
-
|
273
|
-
Generates the topology cookbook attribute files and attributes described in the
|
274
|
-
[cookbook_attributes](#cookbook-attributes) property.
|
275
|
-
|
276
|
-
### Options:
|
277
|
-
|
278
|
-
The knife topo cookbook create subcommand supports the following additional options.
|
279
|
-
|
280
|
-
Option | Description
|
281
|
-
------------ | -----------
|
282
|
-
See [knife cookbook create](http://docs.opscode.com/chef/knife.html#cookbook) | Options supported by `knife cookbook create` are passed through
|
283
|
-
|
284
|
-
### Examples:
|
285
|
-
The following will generate the topology cookbook attribute files for
|
286
|
-
topology test1.
|
287
|
-
|
288
|
-
$ knife topo cookbook create test1
|
289
|
-
|
290
|
-
## knife topo cookbook upload <a name="cookbook-upload"></a>
|
291
|
-
|
292
|
-
knife topo cookbook upload TOPOLOGY
|
293
|
-
|
294
|
-
Uploads the topology cookbook attribute files.
|
295
|
-
|
296
|
-
### Options:
|
297
|
-
|
298
|
-
The knife topo cookbook upload subcommand supports the following additional options.
|
299
|
-
|
300
|
-
Option | Description
|
301
|
-
------------ | -----------
|
302
|
-
See [knife cookbook upload](http://docs.opscode.com/chef/knife.html#cookbook) | Options supported by `knife cookbook upload` are passed through
|
303
|
-
|
304
|
-
|
305
|
-
### Examples:
|
306
|
-
The following will generate the topology cookbook attribute files for
|
307
|
-
topology test1.
|
308
|
-
|
309
|
-
$ knife topo cookbook create test1
|
310
|
-
|
311
290
|
|
312
291
|
## knife topo create <a name="create"></a>
|
313
292
|
|
@@ -315,8 +294,9 @@ topology test1.
|
|
315
294
|
|
316
295
|
Creates the specified topology in the chef server as an item in the
|
317
296
|
topology data bag. Creates the chef environment associated
|
318
|
-
with the topology, if it does not already exist. Uploads
|
319
|
-
topology
|
297
|
+
with the topology, if it does not already exist. Uploads the
|
298
|
+
topology cookbook, if using the 'via_cookbook' method.
|
299
|
+
Updates existing nodes based on the topology
|
320
300
|
information. New nodes will be created if the bootstrap option is
|
321
301
|
specified.
|
322
302
|
|
@@ -329,6 +309,7 @@ Option | Description
|
|
329
309
|
--bootstrap | Bootstrap the topology (see [topo bootstrap](#bootstrap))
|
330
310
|
See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html) | Options supported by `knife bootstrap` are passed through to the bootstrap command
|
331
311
|
--disable-upload | Do not upload topology cookbooks
|
312
|
+
--overwrite | Re-bootstrap existing nodes
|
332
313
|
|
333
314
|
### Examples:
|
334
315
|
The following will create the 'test1' topology, and bootstrap it.
|
@@ -350,7 +331,7 @@ which is used by `knife topo search`.
|
|
350
331
|
|
351
332
|
## knife topo export <a name="export"></a>
|
352
333
|
|
353
|
-
knife topo export [ NODE ... ]
|
334
|
+
knife topo export NODE [ NODE ... ]
|
354
335
|
|
355
336
|
Exports the nodes into a topology JSON.
|
356
337
|
|
@@ -370,7 +351,6 @@ The knife topo export subcommand supports the following additional options.
|
|
370
351
|
Option | Description
|
371
352
|
------------ | -----------
|
372
353
|
--topo | Name of the topology to export (defaults to 'topo1')
|
373
|
-
--all | Export all topologies
|
374
354
|
--min-priority | Only export attributes with a priority equal or above this priority.
|
375
355
|
|
376
356
|
### Examples:
|
@@ -378,12 +358,8 @@ Option | Description
|
|
378
358
|
The following will export the data for nodes n1 and n2 as part of a
|
379
359
|
topology called 'my_topo':
|
380
360
|
|
381
|
-
$ knife topo export n1 n2 --topo my_topo >
|
382
|
-
|
383
|
-
|
384
|
-
The following will export all existing topologies to a file called 'all_topos.json'.
|
361
|
+
$ knife topo export n1 n2 --topo my_topo > my_topo.json
|
385
362
|
|
386
|
-
$ knife topo export --all > all_topos.json
|
387
363
|
|
388
364
|
The following will create an outline for a new topology called
|
389
365
|
'christine_test', or export the current details if it already exists:
|
@@ -393,16 +369,17 @@ The following will create an outline for a new topology called
|
|
393
369
|
|
394
370
|
## knife topo import <a name="import"></a>
|
395
371
|
|
396
|
-
knife topo import [ TOPOLOGY_FILE
|
372
|
+
knife topo import [ TOPOLOGY_FILE ]
|
397
373
|
|
398
|
-
Imports the topologies from a
|
374
|
+
Imports data bag items containing the topologies from a
|
399
375
|
[topology file](#topology-file) into the local repo. If no topology
|
400
376
|
file is specified, attempts to read from a file called 'topology.json'
|
401
|
-
in the current directory. Generates
|
402
|
-
|
377
|
+
in the current directory. Generates additional artifacts (e.g.
|
378
|
+
topology cookbook attribute file) where needed.
|
403
379
|
|
404
380
|
### Examples:
|
405
|
-
The following will import
|
381
|
+
The following will import the topology or topologies defined in the
|
382
|
+
'topology.json' file.
|
406
383
|
|
407
384
|
$ knife topo import topology.json
|
408
385
|
|
@@ -484,7 +461,7 @@ The following will update all topologies in the 'topologies' data bag.
|
|
484
461
|
|
485
462
|
Author:: Christine Draper (christine_draper@thirdwaveinsights.com)
|
486
463
|
|
487
|
-
Copyright:: Copyright (c) 2014-
|
464
|
+
Copyright:: Copyright (c) 2014-2016 ThirdWave Insights, LLC
|
488
465
|
|
489
466
|
License:: Apache License, Version 2.0
|
490
467
|
|