knife-topo 1.1.2 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: a5982dbdd98f486734b60e5f8f7d6db46c18daf2
4
- data.tar.gz: 8ba53f9163e5c9883f7098319c9a3955d9c24d14
3
+ metadata.gz: f48a892fd80f731049b38f49a1d872e3f66f8397
4
+ data.tar.gz: 04ee0c7a4dcbf577b41278e4efbc77439412799b
5
5
  SHA512:
6
- metadata.gz: 1a273459319db01f84a3bdfd1507ef5a1c9f201829087d439473fc38aeb2c68c1eec8c4699971b551081bb4f1ce2e6d85d1e7630eae07ba945326d073d833613
7
- data.tar.gz: 644dd730a1e628f4adc3cf2d6f7b3e1f6dcacbb1f1e5e762df157aed9484a14aa0f998d583e760aa9337324ddca0ddc07d18bead4173488d59ed97a73cda9abb
6
+ metadata.gz: 8fcc6ae3d4194d02c594549f7bbced00cab46b15e5caf320b0bf18f6141bd32bb48f7ac6d48d889ac768f508c6b2efb4d0d8ab58de6570d8ad8b447654146bdd
7
+ data.tar.gz: 5386a5e09a49ca90b4122a797cd54b56f002a8c57883065a0fcc9ee4e37971d46547b9fcae9df2c1250a3e893848d57771c6674e272828d7dda4e717e657188e
data/README.md CHANGED
@@ -16,42 +16,63 @@ through a single (json) configuration file. It may also be useful
16
16
  if you are regularly bringing up multi-node systems with similar
17
17
  topologies but differences in their configuration details.
18
18
 
19
+ This plugin can be used in conjunction with the
20
+ [topo cookbook](http://github.com/christinedraper/topo-cookbook)
21
+ to configure dynamically deployed topologies of nodes (e.g. in AWS).
22
+ In this approach, use knife-topo to set up the topology details.
23
+ Use the 'topo' cookbook to let nodes pull their own detailed configuration
24
+ from the topology data bag on the Chef server.
19
25
 
20
- # Installation #
26
+ # Changes from V1 #
27
+
28
+ ## Attribute setting strategy
29
+
30
+ V2 introduces the notion of an attribute setting strategy. Instead of
31
+ specifying node attributes and cookbook attributes separately, you
32
+ specify one set of attributes and the method by which
33
+ they should be set on the node (e.g. 'direct_to_node' or 'via_cookbook').
34
+
35
+ The purpose of this change is to make it easier to support new
36
+ methods of setting attributes such as policy files, and also making it
37
+ easier to switch between methods.
21
38
 
22
- [Download the latest knife-topo release](http://github.com/christinedraper/knife-topo/releases/latest),
23
- unzip and copy `lib/chef/knife` into your plugin directory, e.g.:
39
+ ## Node type
24
40
 
25
- $ unzip knife-topo-1.0.0.zip -d ~
26
- $ cd ~/knife-topo-1.0.0
27
- $ mkdir -p ~/.chef/plugins/knife
28
- $ cp lib/chef/knife/* ~/.chef/plugins/knife
41
+ A node type can be specified for a node. The node type is used
42
+ by the 'topo' cookbook to identify the right configuration to use. It
43
+ is also used in the 'via cookbook' method to support attributes that
44
+ vary by node type.
29
45
 
30
- or install knife-topo as a gem
46
+ ## One topology per file
31
47
 
32
- $ gem install knife-topo
48
+ To reduce complexity, V2 no longer supports multiple topologies in a
49
+ JSON file.
33
50
 
34
- You may need to use `chef gem install knife-topo` (to install into the
35
- embedded chefdk gem path), and you may need `sudo` depending on your setup.
51
+ ## V1 -> V2 Migration
36
52
 
37
- This plugin has been tested with Chef Version 11.12 and 12.0.3 on Ubuntu 12.04 and 14.04 LTS,
38
- and run on Windows and Mac.
53
+ V1 topology JSON files can be converted to V2 by importing them and
54
+ then exporting them. knife-topo will auto-detect V1 format files on
55
+ import. You can also explicitly specify the input format using
56
+ `knife topo import sometopo.json --input-format 'topo_v1'`
39
57
 
40
- Note: I've encountered a case (on a Mac) where knife was not configured to use
41
- gems on the gem path. If the gem install succeeds but `knife topo`
42
- is not a recognized knife command, try the first approach (copy
43
- the ruby plugin scripts into ~/.chef/plugins/knife) or install knife topo using the embedded gem:
58
+ # Installation #
59
+
60
+ Install knife-topo as a gem
44
61
 
45
- $ /opt/chef/embedded/bin/gem install knife-topo
62
+ ```
63
+ $ chef gem install knife-topo
64
+ ```
46
65
 
47
66
  # Usage #
48
67
 
49
- Define one or more topologies in a [topology file](#topology-file). Import
68
+ Define each topology in a [topology file](#topology-file). Import
50
69
  that file into your Chef workspace using [knife topo import](#import),
51
- then create and bootstrap the nodes using a single command [knife topo create](#create).
52
- Update the topology file as the configuration changes (e.g., when you
53
- need to update software versions), import those changes and run one command
54
- [knife topo update](#update) to update all of the nodes.
70
+ then create the topology [knife topo create](#create), specifying
71
+ the '--bootstrap' option if you want to bootstrap all of the nodes.
72
+
73
+ Update the topology file as the configuration changes, import those
74
+ changes [knife topo import](#import) and run
75
+ [knife topo update](#update) to update the topology.
55
76
 
56
77
 
57
78
  # Getting Started #
@@ -61,33 +82,32 @@ Try out this plugin using a [test-repo](test-repo) provided in the knife-topo gi
61
82
  and unzip it, then follow the [Instructions](test-repo/Instructions.md) for the example.
62
83
 
63
84
  The instructions assume you have [chefDK](https://downloads.chef.io/chef-dk/)
64
- or equivalent installed and working with Vagrant and VirtualBox, but
65
- none of these are requirements to use the knife-topo plugin.
85
+ installed and working with Vagrant and VirtualBox.
66
86
 
67
87
  If you're the sort of person who just wants to jump in and try it, here's some hints.
68
88
 
69
89
  Generate a topology file for a topology called test1 from existing nodes node1 and node2:
70
90
 
71
- knife topo export test1 node1 node2 > topology.json
91
+ knife topo export node1 node2 --topo test1 > test1.json
72
92
 
73
93
  Import a topology json file, generating all of the necessary artifacts in your workspace:
74
94
 
75
- knife topo import topology1.json
95
+ knife topo import test1.json
76
96
 
77
97
  Create the topology using existing nodes:
78
98
 
79
99
  knife topo create test1
80
100
 
81
- Create the topology bootstrapping new nodes in vagrant (you will need to add the
101
+ Create the topology, bootstrapping new nodes in vagrant (you will need to add the
82
102
  host details for bootstrap to the file before importing):
83
103
 
84
104
  knife topo create test1 --bootstrap -xvagrant -Pvagrant --sudo
85
105
 
86
106
  # Topology File <a name="topology-file"></a>#
87
107
 
88
- See the [example topology file](test-repo/topology.json)
108
+ See the [example topology file](test-repo/test1.json)
89
109
 
90
- The topology file contains a single topology, or an array of topologies.
110
+ The topology file contains a single topology.
91
111
  Each topology has some overall properties, an array of nodes and
92
112
  an array defining topology cookbook attributes.
93
113
 
@@ -96,20 +116,20 @@ an array defining topology cookbook attributes.
96
116
  ```
97
117
  {
98
118
  "name": "test1",
119
+ "node_type": "appserver",
99
120
  "chef_environment": "test",
100
- "tags": ["system_sys1", "phase_test" ],
121
+ "tags": [ "testsys" ],
101
122
  "normal": {
102
- "owner": {
103
- "name": "Christine Draper"
104
- }
123
+ "owner": {
124
+ "name": "Christine Draper"
125
+ }
105
126
  },
106
127
  "nodes" : [
107
128
  ...
108
- ],
109
- "cookbook_attributes" : [
110
129
  ]
111
130
  }
112
131
  ```
132
+
113
133
  The `name` is how you will refer to the topology in the
114
134
  `knife topo` subcommands.
115
135
 
@@ -117,6 +137,33 @@ The `chef-environment` and `normal` attributes defined
117
137
  here will be applied to all nodes in the topology, unless alternative
118
138
  values are provided for a specific node. The `tags`
119
139
  will be added to each node.
140
+
141
+ ### Attribute setting strategy
142
+
143
+ The default strategy for setting attributes is `direct_to_node`.
144
+ In this strategy, normal attributes are set directly on the nodes when
145
+ the topology is created, bootstrapped or updated. Attributes with
146
+ other priorities are ignored.
147
+
148
+ The `strategy` field can be set to 'via_cookbook', in which case
149
+ additional `strategy_data` can be provided to specify a cookbok
150
+ and attribute filename.
151
+ ```
152
+ {
153
+ "name": "test1",
154
+ ...
155
+ "strategy" : "via_cookbook",
156
+ "strategy_data": {
157
+ "cookbook": "topo_test1",
158
+ "filename": "softwareversion",
159
+ }
160
+ }
161
+ ```
162
+
163
+ In this strategy, the cookbook and attribute file are generated
164
+ in the local workspace when the topology is imported, and uploaded
165
+ to the server when the topology is created or updated. Attributes
166
+ can have any valid priority.
120
167
 
121
168
  ## Node List <a name="node-list"></a>
122
169
  Each topology contains a list of `nodes`.
@@ -127,47 +174,50 @@ Each topology contains a list of `nodes`.
127
174
  ...
128
175
  "nodes": [
129
176
  {
130
- "name": "buildserver01",
131
- "ssh_host": "192.168.1.201",
132
- "ssh_port": "2224",
133
- "chef_environment": "dev",
134
- "run_list": ["role[base-ubuntu]", "ypo::db", "recipe[ypo::appserver]"],
135
- "normal": {
136
- "topo" : {
137
- node_type": "buildserver"
138
- }
139
- },
140
- "tags": ["build"]
177
+ "name": "buildserver01",
178
+ "node_type" : "buildserver",
179
+ "ssh_host": "192.168.1.201",
180
+ "ssh_port": "2224",
181
+ "chef_environment": "dev",
182
+ "run_list": [
183
+ "role[base-ubuntu]",
184
+ "ypo::db",
185
+ "recipe[ypo::appserver]"
186
+ ],
187
+ ... node attributes, see below ...,
188
+ "tags": [ "build" ]
141
189
  },
142
190
  ...
143
191
  ]
144
192
  }
145
193
  ```
146
194
  Within `nodes`, the `name` field is the node name that will be used in Chef.
147
- The fields `chef_environment`, `run_list`, `tags` and the attributes
148
- within `normal` will also be applied to the node in Chef. All of these
195
+ The fields `chef_environment`, `run_list` and `tags`
196
+ will also be applied to the node in Chef. All of these
149
197
  fields are optional.
150
198
 
151
- The `ssh_host` and `ssh_port` are optional fields that are used to
199
+ The `node_type` sets the node attribute `normal['topo']['node_type']`.
200
+ This attribute is used in the 'via_cookbook' strategy to specify
201
+ attributes that apply to only nodes of that type.
202
+
203
+ The `ssh_host` and `ssh_port` fields are used to
152
204
  bootstrap a node.
153
205
 
154
- ## Topology Cookbook Attributes <a name="cookbook-attributes"></a>
206
+ ## Node Attributes <a name="node-attributes"></a>
155
207
 
156
- Each topology may have attributes that are set via
157
- an attribute file in a topology-specific cookbook. Each
158
- attribute file is described in an entry in the 'cookbook_attributes'
159
- array.
208
+ Each topology may have attributes that are set on each node
209
+ according to the attribute setting strategy. The attribute names and
210
+ values are specified by priority ('default', 'normal', 'override').
160
211
 
161
212
  ```
162
- "cookbook_attributes": [
213
+ "nodes": [
163
214
  {
164
- "cookbook": "testsys_test1",
165
- "filename": "softwareversion",
215
+ "name": "buildserver01",
166
216
  "normal":
167
217
  {
168
218
  "nodejs":
169
219
  {
170
- "version": "0.28"
220
+ "version": "0.10.40"
171
221
  },
172
222
 
173
223
  "testapp":
@@ -177,40 +227,13 @@ array.
177
227
 
178
228
  "mongodb":
179
229
  {
180
- "package_version": "2.6.1"
181
- }
182
- },
183
- "conditional" : [
184
- {
185
- "qualifier": "node_type",
186
- "value" : "buildserver",
187
- "normal":
188
- {
189
- "mongodb":
190
- {
191
- "package_version": "2.5.1"
192
- }
193
- }
230
+ "package_version": "2.6.9"
194
231
  }
195
- ]
232
+ }
196
233
  }
197
234
  ]
198
235
  ```
199
236
 
200
- Attributes listed directly under an attribute priority (e.g. 'normal'
201
- in the above) will generate an entry in the attribute file such as:
202
-
203
- normal['mongodb'][package_version] = "2.6.1"
204
-
205
- Attributes listed under the `conditional` property will generate an
206
- entry in the attribute file such as:
207
-
208
- ```
209
- if (node['topo']['node_type'] == "buildserver")
210
- normal['mongodb']['package_version'] = "2.5.1"
211
- end
212
- ```
213
-
214
237
  # Subcommands <a name="subcommands"></a>
215
238
 
216
239
  The main subcommands for `knife topo` are:
@@ -223,8 +246,6 @@ The additional subcommands can also be useful, depending on your
223
246
  workflow:
224
247
 
225
248
  * [knife topo bootstrap](#bootstrap)- Bootstraps a topology of nodes
226
- * [knife topo cookbook create](#cookbook-create) - Generate the topology cookbooks
227
- * [knife topo cookbook upload](#cookbook-upload) - Upload the topology cookbooks
228
249
  * [knife export](#export) - Export data from a topology (or from nodes that you want in a topology)
229
250
  * [knife topo list](#list) - List the topologies
230
251
  * [knife topo search](#search) - Search for nodes that are in a topology, or in no topology
@@ -257,6 +278,7 @@ The knife topo bootstrap subcommand supports the following additional options.
257
278
 
258
279
  Option | Description
259
280
  ------------ | -----------
281
+ --overwrite | Re-bootstrap existing nodes
260
282
  See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html) | Options supported by `knife bootstrap` are passed through to the bootstrap command
261
283
 
262
284
 
@@ -265,49 +287,6 @@ The following will bootstrap nodes in the test1 topology, using a
265
287
  user name of vagrant, password of vagrant, and running using sudo.
266
288
 
267
289
  $ knife topo bootstrap test1 -x vagrant -P vagrant --sudo
268
-
269
- ## knife topo cookbook create <a name="cookbook-create"></a>
270
-
271
- knife topo cookbook create TOPOLOGY
272
-
273
- Generates the topology cookbook attribute files and attributes described in the
274
- [cookbook_attributes](#cookbook-attributes) property.
275
-
276
- ### Options:
277
-
278
- The knife topo cookbook create subcommand supports the following additional options.
279
-
280
- Option | Description
281
- ------------ | -----------
282
- See [knife cookbook create](http://docs.opscode.com/chef/knife.html#cookbook) | Options supported by `knife cookbook create` are passed through
283
-
284
- ### Examples:
285
- The following will generate the topology cookbook attribute files for
286
- topology test1.
287
-
288
- $ knife topo cookbook create test1
289
-
290
- ## knife topo cookbook upload <a name="cookbook-upload"></a>
291
-
292
- knife topo cookbook upload TOPOLOGY
293
-
294
- Uploads the topology cookbook attribute files.
295
-
296
- ### Options:
297
-
298
- The knife topo cookbook upload subcommand supports the following additional options.
299
-
300
- Option | Description
301
- ------------ | -----------
302
- See [knife cookbook upload](http://docs.opscode.com/chef/knife.html#cookbook) | Options supported by `knife cookbook upload` are passed through
303
-
304
-
305
- ### Examples:
306
- The following will generate the topology cookbook attribute files for
307
- topology test1.
308
-
309
- $ knife topo cookbook create test1
310
-
311
290
 
312
291
  ## knife topo create <a name="create"></a>
313
292
 
@@ -315,8 +294,9 @@ topology test1.
315
294
 
316
295
  Creates the specified topology in the chef server as an item in the
317
296
  topology data bag. Creates the chef environment associated
318
- with the topology, if it does not already exist. Uploads any
319
- topology cookbooks. Updates existing nodes based on the topology
297
+ with the topology, if it does not already exist. Uploads the
298
+ topology cookbook, if using the 'via_cookbook' method.
299
+ Updates existing nodes based on the topology
320
300
  information. New nodes will be created if the bootstrap option is
321
301
  specified.
322
302
 
@@ -329,6 +309,7 @@ Option | Description
329
309
  --bootstrap | Bootstrap the topology (see [topo bootstrap](#bootstrap))
330
310
  See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html) | Options supported by `knife bootstrap` are passed through to the bootstrap command
331
311
  --disable-upload | Do not upload topology cookbooks
312
+ --overwrite | Re-bootstrap existing nodes
332
313
 
333
314
  ### Examples:
334
315
  The following will create the 'test1' topology, and bootstrap it.
@@ -350,7 +331,7 @@ which is used by `knife topo search`.
350
331
 
351
332
  ## knife topo export <a name="export"></a>
352
333
 
353
- knife topo export [ NODE ... ]
334
+ knife topo export NODE [ NODE ... ]
354
335
 
355
336
  Exports the nodes into a topology JSON.
356
337
 
@@ -370,7 +351,6 @@ The knife topo export subcommand supports the following additional options.
370
351
  Option | Description
371
352
  ------------ | -----------
372
353
  --topo | Name of the topology to export (defaults to 'topo1')
373
- --all | Export all topologies
374
354
  --min-priority | Only export attributes with a priority equal or above this priority.
375
355
 
376
356
  ### Examples:
@@ -378,12 +358,8 @@ Option | Description
378
358
  The following will export the data for nodes n1 and n2 as part of a
379
359
  topology called 'my_topo':
380
360
 
381
- $ knife topo export n1 n2 --topo my_topo > new_topo.json
382
-
383
-
384
- The following will export all existing topologies to a file called 'all_topos.json'.
361
+ $ knife topo export n1 n2 --topo my_topo > my_topo.json
385
362
 
386
- $ knife topo export --all > all_topos.json
387
363
 
388
364
  The following will create an outline for a new topology called
389
365
  'christine_test', or export the current details if it already exists:
@@ -393,16 +369,17 @@ The following will create an outline for a new topology called
393
369
 
394
370
  ## knife topo import <a name="import"></a>
395
371
 
396
- knife topo import [ TOPOLOGY_FILE [ TOPOLOGY ... ]]
372
+ knife topo import [ TOPOLOGY_FILE ]
397
373
 
398
- Imports the topologies from a
374
+ Imports data bag items containing the topologies from a
399
375
  [topology file](#topology-file) into the local repo. If no topology
400
376
  file is specified, attempts to read from a file called 'topology.json'
401
- in the current directory. Generates the topology cookbook attribute
402
- files and attributes described in the 'cookbook_attributes' property.
377
+ in the current directory. Generates additional artifacts (e.g.
378
+ topology cookbook attribute file) where needed.
403
379
 
404
380
  ### Examples:
405
- The following will import all topologies defined in the 'topology.json' file.
381
+ The following will import the topology or topologies defined in the
382
+ 'topology.json' file.
406
383
 
407
384
  $ knife topo import topology.json
408
385
 
@@ -484,7 +461,7 @@ The following will update all topologies in the 'topologies' data bag.
484
461
 
485
462
  Author:: Christine Draper (christine_draper@thirdwaveinsights.com)
486
463
 
487
- Copyright:: Copyright (c) 2014-2015 ThirdWave Insights, LLC
464
+ Copyright:: Copyright (c) 2014-2016 ThirdWave Insights, LLC
488
465
 
489
466
  License:: Apache License, Version 2.0
490
467