dev-lxc 1.7.0 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,11 +1,10 @@
1
1
  # dev-lxc
2
2
 
3
- A tool for creating Chef Server clusters and Chef Analytics clusters using LXC containers.
3
+ A tool for building Chef Server clusters and Chef Analytics clusters using LXC containers.
4
4
 
5
- Using [ruby-lxc](https://github.com/lxc/ruby-lxc) it builds a standalone Chef Server or
6
- tier Chef Server cluster composed of a backend and multiple frontends with round-robin
7
- DNS resolution. It can also build a standalone or tier Chef Analytics server and connect
8
- it with the Chef Server.
5
+ Using [ruby-lxc](https://github.com/lxc/ruby-lxc) it builds servers and optionally installs and
6
+ configures many Chef products including a standalone Chef Server or tier Chef Server cluster
7
+ composed of a backend and multiple frontends with round-robin DNS resolution.
9
8
 
10
9
  dev-lxc also has commands to manipulate Chef node containers. For example, dev-lxc can bootstrap a
11
10
  container by installing Chef Client, configuring it for a Chef Server and running a specified run_list.
@@ -15,21 +14,21 @@ for demo purposes, as well as general experimentation and exploration of Chef pr
15
14
 
16
15
  ### Features
17
16
 
18
- 1. LXC 1.0 Containers - Resource efficient servers with fast start/stop times and standard init
17
+ 1. LXC 2.0 Containers - Resource efficient servers with fast start/stop times and standard init
19
18
  2. Btrfs - Efficient, persistent storage backend provides fast, lightweight container cloning
20
19
  3. Dnsmasq - DHCP networking and DNS resolution
21
- 4. Platform Images - Images that are built to resemble a traditional server
20
+ 4. Base Containers - Containers that are built to resemble a traditional server
22
21
  5. ruby-lxc - Ruby bindings for liblxc
23
22
  6. YAML - Simple, customizable definition of clusters; No more setting ENV variables
24
23
  7. Build process closely follows online installation documentation
25
- 8. Images - Images are created during the cluster's build process which makes rebuilding
24
+ 8. Snapshots - Snapshots are created during the cluster's build process which makes rebuilding
26
25
  a cluster very fast.
27
26
 
28
27
  Its containers, standard init, networking and build process are designed to be similar
29
28
  to what you would build if you follow the online installation documentation so the end
30
29
  result is a cluster that is relatively similar to a more traditionally built cluster.
31
30
 
32
- The Btrfs backed clones provide a quick clean slate which is helpful especially for
31
+ The Btrfs backed snapshots provide a quick clean slate which is helpful especially for
33
32
  experimenting and troubleshooting. Or it can be used to build a customized cluster
34
33
  for demo purposes and be able to bring it up quickly and reliably.
35
34
 
@@ -52,35 +51,9 @@ If you aren't familiar with using containers please read this introduction.
52
51
 
53
52
  Once you login to the Vagrant VM platform you should run `sudo -i` to login as the root user.
54
53
 
55
- Consider using `byobu` or `tmux` for a terminal multiplexer as [dev-lxc-platform README
56
- describes](https://github.com/jeremiahsnapp/dev-lxc-platform#use-a-terminal-multiplexer).
57
-
58
- * Setup Mounts and Packages
59
-
60
- As [described below](https://github.com/jeremiahsnapp/dev-lxc#cluster-config-files)
61
- `dev-lxc` uses a `dev-lxc.yml` config file for each cluster.
62
- Be sure that you configure the `mounts` and `packages` lists in `dev-lxc.yml` to match your
63
- particular environment.
64
-
65
- The package paths in dev-lxc's example configs assume that the packages are stored in the
66
- following directory structure in the dev-lxc-platform VM. I recommend creating that
67
- directory structure in the physical workstation and configuring dev-lxc-platform's `.knife.yml`
68
- to mount the structure into `/root/dev` in the dev-lxc-platform VM.
69
-
70
- ```
71
- /root/dev/chef-packages/
72
- ├── analytics
73
- ├── cs
74
- ├── ec
75
- ├── manage
76
- ├── osc
77
- ├── push-jobs-server
78
- └── reporting
79
- ```
80
-
81
54
  ## Update dev-lxc gem
82
55
 
83
- Run `gem update dev-lxc` inside the Vagrant VM platform to ensure you have the latest version.
56
+ Run `chef gem update dev-lxc` inside the Vagrant VM platform to ensure you have the latest version.
84
57
 
85
58
  ## Usage
86
59
 
@@ -89,69 +62,221 @@ Run `gem update dev-lxc` inside the Vagrant VM platform to ensure you have the l
89
62
  ```
90
63
  dev-lxc help
91
64
 
92
- dev-lxc -h
93
-
94
- dev-lxc --help
95
-
96
65
  dev-lxc help <subcommand>
97
66
  ```
98
67
 
99
68
  ### Shorter Commands are Faster (to type that is :)
100
69
 
101
70
  The dev-lxc-platform's root user's `~/.bashrc` file has aliased `dl` to `dev-lxc` for ease of use but
102
- for most instructions in this README I will use `dev-lxc` for clarity.
71
+ for most instructions this README will use `dev-lxc` for clarity.
103
72
 
104
73
  You only have to type enough of a `dev-lxc` subcommand to make it unique.
105
74
 
106
- The following commands are equivalent:
75
+ For example, the following commands are equivalent:
107
76
 
108
77
  ```
109
- dev-lxc init --chef > dev-lxc.yml
110
- dl i --chef > dev-lxc.yml
78
+ dev-lxc status
79
+ dl st
111
80
  ```
112
81
 
113
82
  ```
114
- dev-lxc up
115
- dl u
83
+ dev-lxc snapshot
84
+ dl sn
116
85
  ```
117
86
 
87
+ ### Base Containers
88
+
89
+ The container that is used as the base container for a cluster's containers must exist before
90
+ the cluster can be built. The cluster's containers are cloned from the base container using
91
+ the btrfs filesystem to very quickly provide a lightweight duplicate of the container.
92
+
93
+ This container provides the chosen OS platform and version (e.g. b-ubuntu-1404).
94
+
95
+ A typical LXC container has minimal packages installed so `dev-lxc` makes sure that the
96
+ same packages used in Chef's [bento boxes](https://github.com/opscode/bento) are
97
+ installed to provide a more typical server environment.
98
+ A few additional packages are also installed.
99
+
100
+ Base containers have openssh-server installed and running with unique SSH Host Keys.
101
+
102
+ Base containers have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
103
+
104
+ *Once this base container is created there is rarely a need to delete it.*
105
+
106
+ ### Create a dev-lxc Base Container
107
+
108
+ dev-lxc is able to create base containers that have openssh-server installed and running with unique SSH Host Keys.
109
+
110
+ dev-lxc base containers have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
111
+
112
+ You can see a menu of base containers that `dev-lxc` can create by using the following command.
113
+
118
114
  ```
119
- dev-lxc status
120
- dl st
115
+ dev-lxc create-base-container
121
116
  ```
122
117
 
118
+ The initial creation of base containers can take awhile so let's go ahead and start creating
119
+ an Ubuntu 14.04 container now.
120
+
123
121
  ```
124
- dev-lxc destroy
125
- dl d
122
+ dev-lxc create-base-container b-ubuntu-1404
123
+ ```
124
+
125
+ Note: It is possible to pass additional arguments to the underlying LXC create command.
126
+ For example:
127
+
128
+ ```
129
+ dev-lxc create-base-container b-ubuntu-1404 -o -- '--no-validate --keyserver http://my.key.server.com'
126
130
  ```
127
131
 
128
- ### Create and Manage a Cluster
132
+ ### dev-lxc.yml Config Files
133
+
134
+ dev-lxc uses a YAML configuration file named `dev-lxc.yml` to define a cluster.
129
135
 
130
- The following instructions will build a tier Chef Server with an Analytics server
131
- for demonstration purposes.
132
- The size of this cluster uses about 3GB ram and takes awhile for the first
133
- build of the servers. Feel free to try the standalone config first.
136
+ The `init` command generates sample config files for various server types.
134
137
 
135
- #### Define cluster
138
+ Let's generate a config for a Chef Server tier topology with one backend and one frontend
139
+ along with an Analytics Server, Supermarket Server and a node server.
136
140
 
137
- The following command saves a predefined config to dev-lxc.yml.
141
+ ```
142
+ dev-lxc init --chef-tier --analytics --supermarket --nodes > dev-lxc.yml
143
+ ```
138
144
 
139
- Be sure you configure the
140
- [mounts and packages entries](https://github.com/jeremiahsnapp/dev-lxc#cluster-config-files)
141
- appropriately.
145
+ The contents of `dev-lxc.yml` should look like this.
142
146
 
143
147
  ```
144
- dev-lxc init --tiered-chef --analytics > dev-lxc.yml
148
+ # base_container must be the name of an existing container
149
+ base_container: b-ubuntu-1404
150
+
151
+ # list any host directories you want mounted into the servers
152
+ #mounts:
153
+ # - /root/dev root/dev
154
+
155
+ # list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
156
+ #ssh-keys:
157
+ # - /root/dev/clusters/id_rsa.pub
158
+
159
+ # DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
160
+
161
+ chef-server:
162
+ servers:
163
+ chef.lxc:
164
+ ipaddress: 10.0.3.203
165
+ products:
166
+ chef-server:
167
+ manage:
168
+ push-jobs-server:
169
+ reporting:
170
+
171
+ analytics:
172
+ servers:
173
+ analytics.lxc:
174
+ ipaddress: 10.0.3.204
175
+ products:
176
+ analytics:
177
+
178
+ supermarket:
179
+ servers:
180
+ supermarket.lxc:
181
+ ipaddress: 10.0.3.206
182
+ products:
183
+ supermarket:
184
+
185
+ nodes:
186
+ servers:
187
+ node-1.lxc:
188
+ products:
189
+ chef:
145
190
  ```
146
191
 
147
- #### List Images
192
+ As you can see there are four server types represented by five servers.
193
+
194
+ 1. chef-server - chef.lxc
195
+ 2. analytics - analytics.lxc
196
+ 3. supermarket - supermarket.lxc
197
+ 4. nodes - node-1.lxc
148
198
 
149
- List of each servers' images created during the build process.
199
+ The global settings used by each of the server types are the `base_container`, a list of `mounts` and
200
+ a list of `ssh-keys`. These settings are described in the config comments.
201
+
202
+ Be sure to set `base_container` in the `dev-lxc.yml` to an existing container's name.
203
+ This container will be cloned to create each container in the cluster.
204
+ If you don't already have a container to use as a `base_container` then you can follow the instructions in the
205
+ [Create a dev-lxc Base Container section](https://github.com/jeremiahsnapp/dev-lxc#create-a-dev-lxc-base-container) to create one.
206
+
207
+ It is possible to define different values for `base_container`, `mounts` or `ssh-keys` for a particular server type as
208
+ you can see in the following snippet.
150
209
 
151
210
  ```
152
- dev-lxc list-images
211
+ nodes:
212
+ base_container: b-centos-6
213
+ servers:
214
+ node-1.lxc:
153
215
  ```
154
216
 
217
+ IP addresses from the range 10.0.3.150 - 254 can be assigned to the servers. If an IP address
218
+ is not specified then a dynamic IP address is assigned when the server starts.
219
+
220
+ dev-lxc uses the [mixlib-install](https://github.com/chef/mixlib-install) library to download Chef products
221
+ to a cache in `/var/dev-lxc` in the host VM. This cache is automatically mounted into each server when it starts.
222
+
223
+ A list of Chef products to be installed can be defined for each server
224
+ using [product names that mixlib-install understands](https://github.com/chef/mixlib-install/blob/master/PRODUCT_MATRIX.md).
225
+
226
+ The channel and version of the product can be defined also.
227
+
228
+ Channel can be `current`, `stable` or `unstable` with `stable` as the default.
229
+ Version can be `latest` or a version number with `latest` as the default.
230
+
231
+ For example, the following specifies the `current` channel and version `0.16.1` of the `chefdk` product.
232
+
233
+ ```
234
+ nodes:
235
+ servers:
236
+ node-1.lxc:
237
+ products:
238
+ chefdk:
239
+ channel: current
240
+ version: 0.16.1
241
+ ```
242
+
243
+ The `package_source` setting can be used to specify a package file on disk.
244
+
245
+ ```
246
+ nodes:
247
+ servers:
248
+ node-1.lxc:
249
+ products:
250
+ chefdk:
251
+ package_source: /root/chefdk_0.16.1-1_amd64.deb
252
+ ```
253
+
254
+ dev-lxc knows how to automatically configure Chef Server standalone, Chef Server tier topology,
255
+ Chef Server HA 2.0 as well as Chef Client, Analytics, Compliance and Supermarket.
256
+
257
+ If an Analytics server or Supermarket server is defined in the same config file as
258
+ a Chef Server then each server will automatically be integrated with that Chef Server.
259
+
260
+ If a node server with Chef Client or Chef DK installed is defined in the same config file as
261
+ a Chef Server then the Chef Client will automatically be configured to use that Chef Server.
262
+
263
+ Alternatively, values for `chef_server_url`, `validation_client_name` and `validation_key` can
264
+ be set in the config file.
265
+
266
+ ```
267
+ nodes:
268
+ servers:
269
+ node-1.lxc:
270
+ chef_server_url: https://api.chef.io/organizations/demo
271
+ validation_client_name: demo-validator
272
+ validation_key: /hosted-chef/chef-repo/.chef/demo-validator.pem
273
+ products:
274
+ chef:
275
+ ```
276
+
277
+ The dev-lxc.yml config file is very customizable. You can add or remove mounts, products or servers,
278
+ change ip addresses, server names, the base_container and more.
279
+
155
280
  #### Cluster status
156
281
 
157
282
  Run the following command to see the status of the cluster.
@@ -163,12 +288,13 @@ dev-lxc status
163
288
  This is an example of the output.
164
289
 
165
290
  ```
166
- Chef Server: chef.lxc
167
- Analytics: analytics.lxc
291
+ chef.lxc NOT_CREATED
292
+
293
+ analytics.lxc NOT_CREATED
168
294
 
169
- chef-be.lxc running 10.0.3.201
170
- chef-fe1.lxc running 10.0.3.202
171
- analytics.lxc running 10.0.3.204
295
+ supermarket.lxc NOT_CREATED
296
+
297
+ node-1.lxc NOT_CREATED
172
298
  ```
173
299
 
174
300
  #### cluster-view, tks, tls commands
@@ -178,6 +304,7 @@ tmux/byobu sessions to more easily see the state of a cluster.
178
304
 
179
305
  Running the `cluster-view` command in the same directory as a `dev-lxc.yml` file
180
306
  creates a tmux/byobu session with the same name as the cluster's directory.
307
+
181
308
  `cluster-view` can also be run with the parent directory of a `dev-lxc.yml` file
182
309
  as the first argument and `cluster-view` will change to that directory before
183
310
  creating the tmux/byobu session.
@@ -186,12 +313,7 @@ The session's first window is named "cluster".
186
313
 
187
314
  The left side is for running dev-lxc commands.
188
315
 
189
- The right side is made up of three vertically stacked panes with each pane's content
190
- updating every 0.5 seconds.
191
-
192
- * Top - system's memory usage provided by `free -h`
193
- * Middle - cluster's status provided by `dev-lxc status`
194
- * Bottom - list of the cluster's images provided by `dev-lxc list-images`
316
+ The right side updates every 0.5 seconds with the cluster's status provided by `dev-lxc status`.
195
317
 
196
318
  The session's second window is named "shell". It opens in the same directory as the
197
319
  cluster's `dev-lxc.yml` file.
@@ -211,16 +333,13 @@ Use the keyboard shortcuts Alt-Up/Down to easily switch between tmux/byobu sessi
211
333
 
212
334
  #### Start cluster
213
335
 
214
- Starting the cluster the first time takes awhile since it has a lot to build.
215
-
216
- The tool automatically creates images at appropriate times so future creation of the
217
- cluster's servers is very quick.
336
+ Starting the cluster the first time takes awhile since it has a lot to download and build.
218
337
 
219
338
  ```
220
339
  dev-lxc up
221
340
  ```
222
341
 
223
- A test org, user, knife.rb and keys are automatically created in
342
+ A test org, users, knife.rb and keys are automatically created in
224
343
  the bootstrap backend server in `/root/chef-repo/.chef` for testing purposes.
225
344
 
226
345
  The `knife-opc` plugin is installed in the embedded ruby environment of the
@@ -243,21 +362,9 @@ Now you can easily use knife to access the cluster.
243
362
 
244
363
  ```
245
364
  cd chef-repo
246
- knife ssl fetch
247
365
  knife client list
248
366
  ```
249
367
 
250
- #### Cheap cluster rebuilds
251
-
252
- Clones of the servers as they existed immediately after initial installation, configuration and
253
- test org and user creation are available so you can destroy the cluster and "rebuild" it within
254
- seconds effectively starting with a clean slate very easily.
255
-
256
- ```
257
- dev-lxc destroy
258
- dev-lxc up
259
- ```
260
-
261
368
  #### Stop and start the cluster
262
369
 
263
370
  ```
@@ -265,24 +372,10 @@ dev-lxc halt
265
372
  dev-lxc up
266
373
  ```
267
374
 
268
- #### Backdoor access to each server's filesystem
269
-
270
- The realpath subcommand can be used to prepend each server's rootfs path to a particular file.
271
-
272
- For example, you can use the following command to edit the Chef Servers' chef-server.rb
273
- file without logging into the containers.
274
-
275
- ```
276
- emacs $(dev-lxc realpath chef /etc/opscode/chef-server.rb)
277
- ```
278
-
279
375
  #### Run arbitrary commands in each server
280
376
 
281
- After modifying the chef-server.rb you could use the run-command subcommand to tell the backend and
282
- frontend servers to run `chef-server-ctl reconfigure`.
283
-
284
377
  ```
285
- dev-lxc run-command chef 'chef-server-ctl reconfigure'
378
+ dev-lxc run-command chef 'uptime'
286
379
  ```
287
380
 
288
381
  #### Attach the terminal to a server
@@ -290,41 +383,51 @@ dev-lxc run-command chef 'chef-server-ctl reconfigure'
290
383
  Attach the terminal to a server in the cluster that matches the REGEX pattern given.
291
384
 
292
385
  ```
293
- dev-lxc attach chef-be
386
+ dev-lxc attach chef
294
387
  ```
295
388
 
296
- #### Make a snapshot of the servers
389
+ #### Create a snapshot of the servers
297
390
 
298
- Save the changes in the servers to custom images.
391
+ Save the changes in the servers to snapshots with a comment.
299
392
 
300
393
  ```
301
394
  dev-lxc halt
302
- dev-lxc snapshot
395
+ dev-lxc snapshot -c 'this is a snapshot comment'
303
396
  ```
304
397
 
305
- Now the servers can be destroyed and recreated with the same changes captured at the time of the snapshot.
398
+ #### List snapshots
306
399
 
307
400
  ```
308
- dev-lxc destroy
401
+ dev-lxc snapshot -l
402
+ ```
403
+
404
+ #### Restore snapshots
405
+
406
+ Restore snapshots by name.
407
+
408
+ Leave out the snapshot name or specify `LAST` to restore the most recent snapshot.
409
+
410
+ ```
411
+ dev-lxc snapshot -r
309
412
  dev-lxc up
310
413
  ```
311
414
 
312
- #### Destroy cluster
415
+ #### Destroy snapshots
416
+
417
+ Destroy snapshots by name or destroy all snapshots by specifying `ALL`.
313
418
 
314
- Use the following command to destroy the cluster's servers and also destroy their custom, unique and shared
315
- images if you want to build them from scratch.
419
+ Leave out the snapshot name or specify `LAST` to destroy the most recent snapshots.
316
420
 
317
421
  ```
318
- dev-lxc destroy -c -u -s
422
+ dev-lxc snapshot -d
319
423
  ```
320
424
 
321
- #### Global status of all dev-lxc images and servers
425
+ #### Destroy cluster
322
426
 
323
- Use the `global-status` command to see the status of all dev-lxc images and servers stored in dev-lxc's
324
- default LXC config_path `/var/lib/dev-lxc`.
427
+ Use the following command to destroy the cluster's servers.
325
428
 
326
429
  ```
327
- dev-lxc global-status
430
+ dev-lxc destroy
328
431
  ```
329
432
 
330
433
  #### Use commands against specific servers
@@ -335,8 +438,7 @@ that matches a set of server names.
335
438
  dev-lxc <subcommand> [SERVER_NAME_REGEX]
336
439
  ```
337
440
 
338
- For example, to only start the Chef Servers named `chef-be.lxc` and `chef-fe1.lxc`
339
- you can run the following command.
441
+ For example, to only start the Chef Server you can run the following command.
340
442
 
341
443
  ```
342
444
  dev-lxc up chef
@@ -346,8 +448,8 @@ dev-lxc up chef
346
448
 
347
449
  dev-lxc can also manage an adhoc cluster of servers.
348
450
 
349
- An adhoc cluster is just a set of managed servers cloned from the specified platform
350
- image. The servers have SSH server running, a "dev-lxc" user with "dev-lxc" password and
451
+ An adhoc cluster is just a set of managed servers cloned from the specified base
452
+ container. The servers have SSH server running, a "dev-lxc" user with "dev-lxc" password and
351
453
  passwordless sudo access.
352
454
 
353
455
  This is particularly useful when you want to use something else, such as chef-provisioning,
@@ -356,228 +458,15 @@ to configure the servers.
356
458
  The number of servers, their names and their IP addresses can be changed to fit your
357
459
  particular requirements.
358
460
 
359
- The "adhoc" template provides an example cluster configuration suitable for creating a
360
- Chef Delivery cluster.
361
-
362
461
  ```
363
462
  mkdir -p /root/dev/clusters/delivery
364
463
  cd /root/dev/clusters/delivery
365
464
  dev-lxc init --adhoc > dev-lxc.yml
465
+ # edit dev-lxc.yml to have enough adhoc servers for a delivery cluster
366
466
  cluster-view
367
467
  dl up
368
468
  ```
369
469
 
370
- ### Managing Node Containers
371
-
372
- #### Manually Create a Platform Image
373
-
374
- Platform images can be used for purposes other than building clusters. For example, they can
375
- be used as Chef nodes for testing purposes.
376
-
377
- Platform images have openssh-server installed and running with unique SSH Host Keys.
378
-
379
- Platform images have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
380
-
381
- You can see a menu of platform images this tool can create by using the following command.
382
-
383
- ```
384
- dev-lxc create
385
- ```
386
-
387
- The initial creation of platform images can take awhile so let's go ahead and start creating
388
- an Ubuntu 14.04 image now.
389
-
390
- ```
391
- dev-lxc create p-ubuntu-1404
392
- ```
393
-
394
- Note: It is possible to pass additional arguments to the underlying LXC create command.
395
- For example:
396
-
397
- ```
398
- dev-lxc create p-ubuntu-1404 -o -- '--no-validate --keyserver http://my.key.server.com'
399
- ```
400
-
401
- #### Install Chef Client in a Container
402
-
403
- Use the `-v` option to specify a particular version of Chef Client.
404
-
405
- Use `-v latest` or leave out the `-v` option to install the latest version of Chef Client.
406
-
407
- For example, install the latest 11.x version of Chef Client.
408
-
409
- ```
410
- dev-lxc install-chef-client test-node.lxc -v 11
411
- ```
412
-
413
- #### Configure Chef Client in a Container
414
-
415
- Use the `-s`, `-u`, `-k` options to set `chef_server_url`, `validation_client_name` and
416
- `validation_key` in a container's `/etc/chef/client.rb` and copy the validator's key to
417
- `/etc/chef/validation.pem`.
418
-
419
- Or leave the options empty and it will default to using values from the cluster defined
420
- in `dev-lxc.yml`.
421
-
422
- ```
423
- dev-lxc config-chef-client test-node.lxc
424
- ```
425
-
426
- #### Bootstrap Chef Client in a Container
427
-
428
- Specifying a `BASE_CONTAINER_NAME` will clone the base container into a new container
429
- and bootstrap it. If no `BASE_CONTAINER_NAME` is given then the container to be bootstrapped
430
- needs to already exist.
431
-
432
- Use the `-v` option to specify a particular version of Chef Client.
433
-
434
- Use the `-s`, `-u`, `-k` options to set `chef_server_url`, `validation_client_name` and
435
- `validation_key` in a container's `/etc/chef/client.rb` and copy the validator's key to
436
- `/etc/chef/validation.pem`.
437
-
438
- Or leave the options empty and it will default to using values from the cluster defined
439
- in `dev-lxc.yml`.
440
-
441
- Use the `-r` option to specify the run_list for chef-client to use.
442
-
443
- ```
444
- dev-lxc bootstrap-container test-node.lxc -r my_run_list
445
- ```
446
-
447
- ### Using the dev-lxc library
448
-
449
- dev-lxc cli interface can be used as a library.
450
-
451
- ```
452
- require 'dev-lxc/cli'
453
-
454
- ARGV = [ 'up' ] # start all servers
455
- DevLXC::CLI::DevLXC.start
456
-
457
- ARGV = [ 'status' ] # show status of all servers
458
- DevLXC::CLI::DevLXC.start
459
-
460
- ARGV = [ 'run-command', 'uptime' ] # run `uptime` in all servers
461
- DevLXC::CLI::DevLXC.start
462
-
463
- ARGV = [ 'destroy' ] # destroy all servers
464
- DevLXC::CLI::DevLXC.start
465
- ```
466
-
467
- dev-lxc itself can also be used as a library
468
-
469
- ```
470
- require 'yaml'
471
- require 'dev-lxc'
472
-
473
- config = YAML.load(IO.read('dev-lxc.yml'))
474
- server = DevLXC::Server.new("chef-fe1.lxc", 'chef-server', config)
475
-
476
- server.start # start chef-fe1.lxc
477
- server.status # show status of chef-fe1.lxc
478
- server.run_command("chef-server-ctl reconfigure") # run command in chef-fe1.lxc
479
- server.stop # stop chef-fe1.lxc
480
- server.destroy # destroy chef-fe1.lxc
481
- ```
482
-
483
- ## Cluster Config Files
484
-
485
- dev-lxc uses a YAML configuration file named `dev-lxc.yml` to define a cluster.
486
-
487
- The following command generates sample config files for various cluster topologies.
488
-
489
- ```
490
- dev-lxc init
491
- ```
492
-
493
- `dev-lxc init --tiered-chef --analytics > dev-lxc.yml` creates a `dev-lxc.yml` file with the following content:
494
-
495
- ```
496
- ## platform_image can be one of the following:
497
- ## p-centos-5, p-centos-6, p-centos-7, p-ubuntu-1204, p-ubuntu-1404 or p-ubuntu-1604
498
- platform_image: p-ubuntu-1404
499
-
500
- ## platform_image_options can be set to provide additional arguments to the LXC create command
501
- ## reference arg examples: https://github.com/lxc/lxc/blob/lxc-2.0.0/templates/lxc-download.in#L200-L207
502
- #platform_image_options: --no-validate
503
-
504
- ## list any host directories you want mounted into the servers
505
- mounts:
506
- - /root/dev root/dev
507
-
508
- ## list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
509
- #ssh-keys:
510
- # - /root/dev/clusters/id_rsa.pub
511
-
512
- ## DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
513
-
514
- chef-server:
515
- packages:
516
- server: /root/dev/chef-packages/cs/chef-server-core_12.5.0-1_amd64.deb
517
- manage: /root/dev/chef-packages/manage/chef-manage_2.2.1-1_amd64.deb
518
- reporting: /root/dev/chef-packages/reporting/opscode-reporting_1.5.6-1_amd64.deb
519
- push-jobs-server: /root/dev/chef-packages/push-jobs-server/opscode-push-jobs-server_1.1.6-1_amd64.deb
520
- topology: tier
521
- api_fqdn: chef.lxc
522
- servers:
523
- chef-be.lxc:
524
- ipaddress: 10.0.3.201
525
- role: backend
526
- bootstrap: true
527
- chef-fe1.lxc:
528
- ipaddress: 10.0.3.202
529
- role: frontend
530
-
531
- analytics:
532
- packages:
533
- analytics: /root/dev/chef-packages/analytics/opscode-analytics_1.3.1-1_amd64.deb
534
- servers:
535
- analytics.lxc:
536
- ipaddress: 10.0.3.204
537
- ```
538
-
539
- This config defines a tier cluster consisting of a single backend and a single frontend.
540
-
541
- A second frontend is commented out to conserve resources. If you uncomment the second
542
- frontend then both frontends will be created and dnsmasq will resolve the `api_fqdn`
543
- [chef.lxc](chef.lxc) to both frontends using a round-robin policy.
544
-
545
- The config file is very customizable. You can add or remove mounts, packages or servers,
546
- change ip addresses, change server names, change the platform_image and more.
547
-
548
- The `mounts` list describes what directories get mounted from the Vagrant VM platform into
549
- each container. You need to make sure that you configure the mount entries to be
550
- appropriate for your environment.
551
-
552
- The same is true for the `packages` list. The paths that are provided in the default configs are just examples.
553
- You need to make sure that you have each package you want to use downloaded to appropriate directories
554
- that will be available to the container when it is started.
555
-
556
- I recommend downloading the packages to a directory on your workstation.
557
- Then configure the
558
- [dev-lxc-platform's .kitchen.yml](https://github.com/jeremiahsnapp/dev-lxc-platform#description)
559
- to mount that directory in the Vagrant VM platform.
560
- Then configure the cluster's mount entries in `dev-lxc.yml` to mount the Vagrant VM platform's
561
- directory into each container.
562
-
563
- Make sure the mounts and packages represent actual paths that are available in your environment.
564
-
565
- ### Managing Multiple Clusters
566
-
567
- By default, `dev-lxc` looks for a `dev-lxc.yml` file in the present working directory.
568
- You can also specify a particular config file as an option for most dev-lxc commands.
569
-
570
- The following is an example of managing multiple clusters while still avoiding specifying
571
- each cluster's config file.
572
-
573
- ```
574
- mkdir -p ~/clusters/{clusterA,clusterB}
575
- dev-lxc init --tiered-chef > ~/clusters/clusterA/dev-lxc.yml
576
- dev-lxc init --chef > ~/clusters/clusterB/dev-lxc.yml
577
- cd ~/clusters/clusterA && dev-lxc up # starts clusterA
578
- cd ~/clusters/clusterB && dev-lxc up # starts clusterB
579
- ```
580
-
581
470
  ### Maintain Uniqueness Across Multiple Clusters
582
471
 
583
472
  The default cluster configs are already designed to be unique from each other but as you build
@@ -596,129 +485,18 @@ more clusters you have to maintain uniqueness across the YAML config files for t
596
485
  If cluster B is started with the same `api_fqdn` or `analytics_fqdn` as an already running cluster A,
597
486
  then cluster B will overwrite cluster A's DNS resolution of `api_fqdn` or `analytics_fqdn`.
598
487
 
599
- It is easy to provide uniqueness in the server names, `api_fqdn` and `analytics_fqdn`.
600
- For example, you can use the following command to prefix the servers names with `1234-` when
601
- generating a cluster's config.
602
-
603
- dev-lxc init --tiered-chef 1234- > dev-lxc.yml
604
-
605
488
  * IP Addresses
606
489
 
607
490
  IP addresses uniqueness only matters when clusters with the same IP's are running.
608
-
491
+
609
492
  If cluster B is started with the same IP's as an already running cluster A, then cluster B
610
493
  will overwrite cluster A's DHCP reservation of the IP's but dnsmasq will still refuse to
611
494
  assign the IP's to cluster B because they already in use by cluster A. dnsmasq then assigns
612
495
  random IP's from the DHCP pool to cluster B leaving it in an unexpected state.
613
496
 
614
497
  The `dev-lxc-platform` creates the IP range 10.0.3.150 - 254 for DHCP reserved IP's.
615
-
616
- Use unique IP's from that range when configuring clusters.
617
-
618
- ## Images
619
-
620
- One of the key things this tool uses is the concept of images.
621
-
622
- `dev-lxc` creates images with a "p-", "s-", "u-" or "c-" prefix on the name to distinguish
623
- it as a "platform", "shared", "unique" or "custom" image.
624
-
625
- Images are then cloned using the btrfs filesystem to very quickly provide a lightweight duplicate
626
- of the image. This clone is either used to build the next image in the build process or the final
627
- container that will actually be run.
628
-
629
- By default, the cluster's images and final server containers are all stored in `/var/lib/dev-lxc`
630
- so they don't clutter the containers stored in the default LXC config_path `/var/lib/lxc`.
631
-
632
- The cluster's LXC config_path can be configured by setting `lxc_config_path` at the top of the
633
- `dev-lxc.yml` file to the desired directory.
634
-
635
- There are four image categories.
636
-
637
- 1. Platform Image
638
-
639
- The platform image is the first to get created and is identified by the
640
- "p-" prefix on the image name.
641
-
642
- `DevLXC#create_platform_image` controls the creation of a platform image.
643
498
 
644
- This image provides the chosen OS platform and version (e.g. p-ubuntu-1404).
645
- A typical LXC container has minimal packages installed so `dev-lxc` makes sure that the
646
- same packages used in Chef's [bento boxes](https://github.com/opscode/bento) are
647
- installed to provide a more typical server environment.
648
- A few additional packages are also installed.
649
-
650
- Platform images have openssh-server installed and running with unique SSH Host Keys.
651
-
652
- Platform images have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
653
-
654
- *Once this platform image is created there is rarely a need to delete it.*
655
-
656
- 2. Shared Image
657
-
658
- The shared image is the second to get created and is identified by the
659
- "s-" prefix on the image name.
660
-
661
- `DevLXC::Server#create_shared_image` controls the creation of a shared image.
662
-
663
- Chef packages that are common to all servers in a Chef cluster, such as chef-server-core,
664
- opscode-reporting and opscode-push-jobs-server are installed using `dpkg` or `rpm`.
665
-
666
- Note the manage package will not be installed at this point since it is not common to all
667
- servers (i.e. it does not get installed on backend servers).
668
-
669
- The name of this image is built from the names and versions of the Chef packages that
670
- get installed which makes this image easy to be reused by another cluster that is
671
- configured to use the same Chef packages.
672
-
673
- *Since no configuration actually happens yet there is rarely a need to delete this image.*
674
-
675
- 3. Unique Image
676
-
677
- The unique image is the last to get created and is identified by the
678
- "u-" prefix on the image name.
679
-
680
- `DevLXC::Server#create` controls the creation of a unique image.
681
-
682
- Each unique Chef server (e.g. standalone, backend or frontend) is created.
683
-
684
- * The specified hostname is assigned.
685
- * dnsmasq is configured to reserve the specified IP address for the image's MAC address.
686
- * A DNS entry is created in dnsmasq if appropriate.
687
- * All installed Chef packages are configured.
688
- * Test users and orgs are created.
689
- * The opscode-manage package is installed and configured if specified.
690
-
691
- After each server is fully configured a clone of it is made resulting in the server's
692
- unique image. These unique images make it very easy to quickly recreate
693
- a Chef cluster from a clean starting point.
694
-
695
- 4. Custom Image
696
-
697
- The custom image is only created when the `snapshot` command is used and is identified
698
- by the "c-" prefix on the image name.
699
-
700
- `DevLXC::Server#snapshot` controls the creation of a custom image.
701
-
702
- Custom images can be used to save the changes that have been made in servers.
703
- Later, when the servers are destroyed and recreated, they will start running with the changes
704
- that were captured at the time of the snapshot.
705
-
706
- ### Destroying Images
707
-
708
- When using `dev-lxc destroy` to destroy servers you have the option to also destroy any or all of
709
- the four types of images associated with the servers.
710
-
711
- The following command will list the options available.
712
-
713
- ```
714
- dev-lxc help destroy
715
- ```
716
-
717
- Of course, you can also just use the standard LXC commands to destroy any container.
718
-
719
- ```
720
- lxc-destroy -n [NAME]
721
- ```
499
+ Use unique IP's from that range when configuring clusters.
722
500
 
723
501
  ## Contributing
724
502