dev-lxc 1.7.0 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +230 -452
- data/dev-lxc.gemspec +2 -1
- data/lib/dev-lxc/cli.rb +174 -309
- data/lib/dev-lxc/cluster.rb +772 -113
- data/lib/dev-lxc/container.rb +1 -107
- data/lib/dev-lxc/server.rb +107 -420
- data/lib/dev-lxc/version.rb +1 -1
- data/lib/dev-lxc.rb +72 -76
- metadata +18 -4
data/README.md
CHANGED
@@ -1,11 +1,10 @@
|
|
1
1
|
# dev-lxc
|
2
2
|
|
3
|
-
A tool for
|
3
|
+
A tool for building Chef Server clusters and Chef Analytics clusters using LXC containers.
|
4
4
|
|
5
|
-
Using [ruby-lxc](https://github.com/lxc/ruby-lxc) it builds
|
6
|
-
|
7
|
-
|
8
|
-
it with the Chef Server.
|
5
|
+
Using [ruby-lxc](https://github.com/lxc/ruby-lxc) it builds servers and optionally installs and
|
6
|
+
configures many Chef products including a standalone Chef Server or tier Chef Server cluster
|
7
|
+
composed of a backend and multiple frontends with round-robin DNS resolution.
|
9
8
|
|
10
9
|
dev-lxc also has commands to manipulate Chef node containers. For example, dev-lxc can bootstrap a
|
11
10
|
container by installing Chef Client, configuring it for a Chef Server and running a specified run_list.
|
@@ -15,21 +14,21 @@ for demo purposes, as well as general experimentation and exploration of Chef pr
|
|
15
14
|
|
16
15
|
### Features
|
17
16
|
|
18
|
-
1. LXC
|
17
|
+
1. LXC 2.0 Containers - Resource efficient servers with fast start/stop times and standard init
|
19
18
|
2. Btrfs - Efficient, persistent storage backend provides fast, lightweight container cloning
|
20
19
|
3. Dnsmasq - DHCP networking and DNS resolution
|
21
|
-
4.
|
20
|
+
4. Base Containers - Containers that are built to resemble a traditional server
|
22
21
|
5. ruby-lxc - Ruby bindings for liblxc
|
23
22
|
6. YAML - Simple, customizable definition of clusters; No more setting ENV variables
|
24
23
|
7. Build process closely follows online installation documentation
|
25
|
-
8.
|
24
|
+
8. Snapshots - Snapshots are created during the cluster's build process which makes rebuilding
|
26
25
|
a cluster very fast.
|
27
26
|
|
28
27
|
Its containers, standard init, networking and build process are designed to be similar
|
29
28
|
to what you would build if you follow the online installation documentation so the end
|
30
29
|
result is a cluster that is relatively similar to a more traditionally built cluster.
|
31
30
|
|
32
|
-
The Btrfs backed
|
31
|
+
The Btrfs backed snapshots provide a quick clean slate which is helpful especially for
|
33
32
|
experimenting and troubleshooting. Or it can be used to build a customized cluster
|
34
33
|
for demo purposes and be able to bring it up quickly and reliably.
|
35
34
|
|
@@ -52,35 +51,9 @@ If you aren't familiar with using containers please read this introduction.
|
|
52
51
|
|
53
52
|
Once you login to the Vagrant VM platform you should run `sudo -i` to login as the root user.
|
54
53
|
|
55
|
-
Consider using `byobu` or `tmux` for a terminal multiplexer as [dev-lxc-platform README
|
56
|
-
describes](https://github.com/jeremiahsnapp/dev-lxc-platform#use-a-terminal-multiplexer).
|
57
|
-
|
58
|
-
* Setup Mounts and Packages
|
59
|
-
|
60
|
-
As [described below](https://github.com/jeremiahsnapp/dev-lxc#cluster-config-files)
|
61
|
-
`dev-lxc` uses a `dev-lxc.yml` config file for each cluster.
|
62
|
-
Be sure that you configure the `mounts` and `packages` lists in `dev-lxc.yml` to match your
|
63
|
-
particular environment.
|
64
|
-
|
65
|
-
The package paths in dev-lxc's example configs assume that the packages are stored in the
|
66
|
-
following directory structure in the dev-lxc-platform VM. I recommend creating that
|
67
|
-
directory structure in the physical workstation and configuring dev-lxc-platform's `.knife.yml`
|
68
|
-
to mount the structure into `/root/dev` in the dev-lxc-platform VM.
|
69
|
-
|
70
|
-
```
|
71
|
-
/root/dev/chef-packages/
|
72
|
-
├── analytics
|
73
|
-
├── cs
|
74
|
-
├── ec
|
75
|
-
├── manage
|
76
|
-
├── osc
|
77
|
-
├── push-jobs-server
|
78
|
-
└── reporting
|
79
|
-
```
|
80
|
-
|
81
54
|
## Update dev-lxc gem
|
82
55
|
|
83
|
-
Run `gem update dev-lxc` inside the Vagrant VM platform to ensure you have the latest version.
|
56
|
+
Run `chef gem update dev-lxc` inside the Vagrant VM platform to ensure you have the latest version.
|
84
57
|
|
85
58
|
## Usage
|
86
59
|
|
@@ -89,69 +62,221 @@ Run `gem update dev-lxc` inside the Vagrant VM platform to ensure you have the l
|
|
89
62
|
```
|
90
63
|
dev-lxc help
|
91
64
|
|
92
|
-
dev-lxc -h
|
93
|
-
|
94
|
-
dev-lxc --help
|
95
|
-
|
96
65
|
dev-lxc help <subcommand>
|
97
66
|
```
|
98
67
|
|
99
68
|
### Shorter Commands are Faster (to type that is :)
|
100
69
|
|
101
70
|
The dev-lxc-platform's root user's `~/.bashrc` file has aliased `dl` to `dev-lxc` for ease of use but
|
102
|
-
for most instructions
|
71
|
+
for most instructions this README will use `dev-lxc` for clarity.
|
103
72
|
|
104
73
|
You only have to type enough of a `dev-lxc` subcommand to make it unique.
|
105
74
|
|
106
|
-
|
75
|
+
For example, the following commands are equivalent:
|
107
76
|
|
108
77
|
```
|
109
|
-
dev-lxc
|
110
|
-
dl
|
78
|
+
dev-lxc status
|
79
|
+
dl st
|
111
80
|
```
|
112
81
|
|
113
82
|
```
|
114
|
-
dev-lxc
|
115
|
-
dl
|
83
|
+
dev-lxc snapshot
|
84
|
+
dl sn
|
116
85
|
```
|
117
86
|
|
87
|
+
### Base Containers
|
88
|
+
|
89
|
+
The container that is used as the base container for a cluster's containers must exist before
|
90
|
+
the cluster can be built. The cluster's containers are cloned from the base container using
|
91
|
+
the btrfs filesystem to very quickly provide a lightweight duplicate of the container.
|
92
|
+
|
93
|
+
This container provides the chosen OS platform and version (e.g. b-ubuntu-1404).
|
94
|
+
|
95
|
+
A typical LXC container has minimal packages installed so `dev-lxc` makes sure that the
|
96
|
+
same packages used in Chef's [bento boxes](https://github.com/opscode/bento) are
|
97
|
+
installed to provide a more typical server environment.
|
98
|
+
A few additional packages are also installed.
|
99
|
+
|
100
|
+
Base containers have openssh-server installed and running with unique SSH Host Keys.
|
101
|
+
|
102
|
+
Base containers have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
|
103
|
+
|
104
|
+
*Once this base container is created there is rarely a need to delete it.*
|
105
|
+
|
106
|
+
### Create a dev-lxc Base Container
|
107
|
+
|
108
|
+
dev-lxc is able to create base containers that have openssh-server installed and running with unique SSH Host Keys.
|
109
|
+
|
110
|
+
dev-lxc base containers have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
|
111
|
+
|
112
|
+
You can see a menu of base containers that `dev-lxc` can create by using the following command.
|
113
|
+
|
118
114
|
```
|
119
|
-
dev-lxc
|
120
|
-
dl st
|
115
|
+
dev-lxc create-base-container
|
121
116
|
```
|
122
117
|
|
118
|
+
The initial creation of base containers can take awhile so let's go ahead and start creating
|
119
|
+
an Ubuntu 14.04 container now.
|
120
|
+
|
123
121
|
```
|
124
|
-
dev-lxc
|
125
|
-
|
122
|
+
dev-lxc create-base-container b-ubuntu-1404
|
123
|
+
```
|
124
|
+
|
125
|
+
Note: It is possible to pass additional arguments to the underlying LXC create command.
|
126
|
+
For example:
|
127
|
+
|
128
|
+
```
|
129
|
+
dev-lxc create-base-container b-ubuntu-1404 -o -- '--no-validate --keyserver http://my.key.server.com'
|
126
130
|
```
|
127
131
|
|
128
|
-
###
|
132
|
+
### dev-lxc.yml Config Files
|
133
|
+
|
134
|
+
dev-lxc uses a YAML configuration file named `dev-lxc.yml` to define a cluster.
|
129
135
|
|
130
|
-
The
|
131
|
-
for demonstration purposes.
|
132
|
-
The size of this cluster uses about 3GB ram and takes awhile for the first
|
133
|
-
build of the servers. Feel free to try the standalone config first.
|
136
|
+
The `init` command generates sample config files for various server types.
|
134
137
|
|
135
|
-
|
138
|
+
Let's generate a config for a Chef Server tier topology with one backend and one frontend
|
139
|
+
along with an Analytics Server, Supermarket Server and a node server.
|
136
140
|
|
137
|
-
|
141
|
+
```
|
142
|
+
dev-lxc init --chef-tier --analytics --supermarket --nodes > dev-lxc.yml
|
143
|
+
```
|
138
144
|
|
139
|
-
|
140
|
-
[mounts and packages entries](https://github.com/jeremiahsnapp/dev-lxc#cluster-config-files)
|
141
|
-
appropriately.
|
145
|
+
The contents of `dev-lxc.yml` should look like this.
|
142
146
|
|
143
147
|
```
|
144
|
-
|
148
|
+
# base_container must be the name of an existing container
|
149
|
+
base_container: b-ubuntu-1404
|
150
|
+
|
151
|
+
# list any host directories you want mounted into the servers
|
152
|
+
#mounts:
|
153
|
+
# - /root/dev root/dev
|
154
|
+
|
155
|
+
# list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
|
156
|
+
#ssh-keys:
|
157
|
+
# - /root/dev/clusters/id_rsa.pub
|
158
|
+
|
159
|
+
# DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
|
160
|
+
|
161
|
+
chef-server:
|
162
|
+
servers:
|
163
|
+
chef.lxc:
|
164
|
+
ipaddress: 10.0.3.203
|
165
|
+
products:
|
166
|
+
chef-server:
|
167
|
+
manage:
|
168
|
+
push-jobs-server:
|
169
|
+
reporting:
|
170
|
+
|
171
|
+
analytics:
|
172
|
+
servers:
|
173
|
+
analytics.lxc:
|
174
|
+
ipaddress: 10.0.3.204
|
175
|
+
products:
|
176
|
+
analytics:
|
177
|
+
|
178
|
+
supermarket:
|
179
|
+
servers:
|
180
|
+
supermarket.lxc:
|
181
|
+
ipaddress: 10.0.3.206
|
182
|
+
products:
|
183
|
+
supermarket:
|
184
|
+
|
185
|
+
nodes:
|
186
|
+
servers:
|
187
|
+
node-1.lxc:
|
188
|
+
products:
|
189
|
+
chef:
|
145
190
|
```
|
146
191
|
|
147
|
-
|
192
|
+
As you can see there are four server types represented by five servers.
|
193
|
+
|
194
|
+
1. chef-server - chef.lxc
|
195
|
+
2. analytics - analytics.lxc
|
196
|
+
3. supermarket - supermarket.lxc
|
197
|
+
4. nodes - node-1.lxc
|
148
198
|
|
149
|
-
|
199
|
+
The global settings used by each of the server types are the `base_container`, a list of `mounts` and
|
200
|
+
a list of `ssh-keys`. These settings are described in the config comments.
|
201
|
+
|
202
|
+
Be sure to set `base_container` in the `dev-lxc.yml` to an existing container's name.
|
203
|
+
This container will be cloned to create each container in the cluster.
|
204
|
+
If you don't already have a container to use as a `base_container` then you can follow the instructions in the
|
205
|
+
[Create a dev-lxc Base Container section](https://github.com/jeremiahsnapp/dev-lxc#create-a-dev-lxc-base-container) to create one.
|
206
|
+
|
207
|
+
It is possible to define different values for `base_container`, `mounts` or `ssh-keys` for a particular server type as
|
208
|
+
you can see in the following snippet.
|
150
209
|
|
151
210
|
```
|
152
|
-
|
211
|
+
nodes:
|
212
|
+
base_container: b-centos-6
|
213
|
+
servers:
|
214
|
+
node-1.lxc:
|
153
215
|
```
|
154
216
|
|
217
|
+
IP addresses from the range 10.0.3.150 - 254 can be assigned to the servers. If an IP address
|
218
|
+
is not specified then a dynamic IP address is assigned when the server starts.
|
219
|
+
|
220
|
+
dev-lxc uses the [mixlib-install](https://github.com/chef/mixlib-install) library to download Chef products
|
221
|
+
to a cache in `/var/dev-lxc` in the host VM. This cache is automatically mounted into each server when it starts.
|
222
|
+
|
223
|
+
A list of Chef products to be installed can be defined for each server
|
224
|
+
using [product names that mixlib-install understands](https://github.com/chef/mixlib-install/blob/master/PRODUCT_MATRIX.md).
|
225
|
+
|
226
|
+
The channel and version of the product can be defined also.
|
227
|
+
|
228
|
+
Channel can be `current`, `stable` or `unstable` with `stable` as the default.
|
229
|
+
Version can be `latest` or a version number with `latest` as the default.
|
230
|
+
|
231
|
+
For example, the following specifies the `current` channel and version `0.16.1` of the `chefdk` product.
|
232
|
+
|
233
|
+
```
|
234
|
+
nodes:
|
235
|
+
servers:
|
236
|
+
node-1.lxc:
|
237
|
+
products:
|
238
|
+
chefdk:
|
239
|
+
channel: current
|
240
|
+
version: 0.16.1
|
241
|
+
```
|
242
|
+
|
243
|
+
The `package_source` setting can be used to specify a package file on disk.
|
244
|
+
|
245
|
+
```
|
246
|
+
nodes:
|
247
|
+
servers:
|
248
|
+
node-1.lxc:
|
249
|
+
products:
|
250
|
+
chefdk:
|
251
|
+
package_source: /root/chefdk_0.16.1-1_amd64.deb
|
252
|
+
```
|
253
|
+
|
254
|
+
dev-lxc knows how to automatically configure Chef Server standalone, Chef Server tier topology,
|
255
|
+
Chef Server HA 2.0 as well as Chef Client, Analytics, Compliance and Supermarket.
|
256
|
+
|
257
|
+
If an Analytics server or Supermarket server is defined in the same config file as
|
258
|
+
a Chef Server then each server will automatically be integrated with that Chef Server.
|
259
|
+
|
260
|
+
If a node server with Chef Client or Chef DK installed is defined in the same config file as
|
261
|
+
a Chef Server then the Chef Client will automatically be configured to use that Chef Server.
|
262
|
+
|
263
|
+
Alternatively, values for `chef_server_url`, `validation_client_name` and `validation_key` can
|
264
|
+
be set in the config file.
|
265
|
+
|
266
|
+
```
|
267
|
+
nodes:
|
268
|
+
servers:
|
269
|
+
node-1.lxc:
|
270
|
+
chef_server_url: https://api.chef.io/organizations/demo
|
271
|
+
validation_client_name: demo-validator
|
272
|
+
validation_key: /hosted-chef/chef-repo/.chef/demo-validator.pem
|
273
|
+
products:
|
274
|
+
chef:
|
275
|
+
```
|
276
|
+
|
277
|
+
The dev-lxc.yml config file is very customizable. You can add or remove mounts, products or servers,
|
278
|
+
change ip addresses, server names, the base_container and more.
|
279
|
+
|
155
280
|
#### Cluster status
|
156
281
|
|
157
282
|
Run the following command to see the status of the cluster.
|
@@ -163,12 +288,13 @@ dev-lxc status
|
|
163
288
|
This is an example of the output.
|
164
289
|
|
165
290
|
```
|
166
|
-
|
167
|
-
|
291
|
+
chef.lxc NOT_CREATED
|
292
|
+
|
293
|
+
analytics.lxc NOT_CREATED
|
168
294
|
|
169
|
-
|
170
|
-
|
171
|
-
|
295
|
+
supermarket.lxc NOT_CREATED
|
296
|
+
|
297
|
+
node-1.lxc NOT_CREATED
|
172
298
|
```
|
173
299
|
|
174
300
|
#### cluster-view, tks, tls commands
|
@@ -178,6 +304,7 @@ tmux/byobu sessions to more easily see the state of a cluster.
|
|
178
304
|
|
179
305
|
Running the `cluster-view` command in the same directory as a `dev-lxc.yml` file
|
180
306
|
creates a tmux/byobu session with the same name as the cluster's directory.
|
307
|
+
|
181
308
|
`cluster-view` can also be run with the parent directory of a `dev-lxc.yml` file
|
182
309
|
as the first argument and `cluster-view` will change to that directory before
|
183
310
|
creating the tmux/byobu session.
|
@@ -186,12 +313,7 @@ The session's first window is named "cluster".
|
|
186
313
|
|
187
314
|
The left side is for running dev-lxc commands.
|
188
315
|
|
189
|
-
The right side
|
190
|
-
updating every 0.5 seconds.
|
191
|
-
|
192
|
-
* Top - system's memory usage provided by `free -h`
|
193
|
-
* Middle - cluster's status provided by `dev-lxc status`
|
194
|
-
* Bottom - list of the cluster's images provided by `dev-lxc list-images`
|
316
|
+
The right side updates every 0.5 seconds with the cluster's status provided by `dev-lxc status`.
|
195
317
|
|
196
318
|
The session's second window is named "shell". It opens in the same directory as the
|
197
319
|
cluster's `dev-lxc.yml` file.
|
@@ -211,16 +333,13 @@ Use the keyboard shortcuts Alt-Up/Down to easily switch between tmux/byobu sessi
|
|
211
333
|
|
212
334
|
#### Start cluster
|
213
335
|
|
214
|
-
Starting the cluster the first time takes awhile since it has a lot to build.
|
215
|
-
|
216
|
-
The tool automatically creates images at appropriate times so future creation of the
|
217
|
-
cluster's servers is very quick.
|
336
|
+
Starting the cluster the first time takes awhile since it has a lot to download and build.
|
218
337
|
|
219
338
|
```
|
220
339
|
dev-lxc up
|
221
340
|
```
|
222
341
|
|
223
|
-
A test org,
|
342
|
+
A test org, users, knife.rb and keys are automatically created in
|
224
343
|
the bootstrap backend server in `/root/chef-repo/.chef` for testing purposes.
|
225
344
|
|
226
345
|
The `knife-opc` plugin is installed in the embedded ruby environment of the
|
@@ -243,21 +362,9 @@ Now you can easily use knife to access the cluster.
|
|
243
362
|
|
244
363
|
```
|
245
364
|
cd chef-repo
|
246
|
-
knife ssl fetch
|
247
365
|
knife client list
|
248
366
|
```
|
249
367
|
|
250
|
-
#### Cheap cluster rebuilds
|
251
|
-
|
252
|
-
Clones of the servers as they existed immediately after initial installation, configuration and
|
253
|
-
test org and user creation are available so you can destroy the cluster and "rebuild" it within
|
254
|
-
seconds effectively starting with a clean slate very easily.
|
255
|
-
|
256
|
-
```
|
257
|
-
dev-lxc destroy
|
258
|
-
dev-lxc up
|
259
|
-
```
|
260
|
-
|
261
368
|
#### Stop and start the cluster
|
262
369
|
|
263
370
|
```
|
@@ -265,24 +372,10 @@ dev-lxc halt
|
|
265
372
|
dev-lxc up
|
266
373
|
```
|
267
374
|
|
268
|
-
#### Backdoor access to each server's filesystem
|
269
|
-
|
270
|
-
The realpath subcommand can be used to prepend each server's rootfs path to a particular file.
|
271
|
-
|
272
|
-
For example, you can use the following command to edit the Chef Servers' chef-server.rb
|
273
|
-
file without logging into the containers.
|
274
|
-
|
275
|
-
```
|
276
|
-
emacs $(dev-lxc realpath chef /etc/opscode/chef-server.rb)
|
277
|
-
```
|
278
|
-
|
279
375
|
#### Run arbitrary commands in each server
|
280
376
|
|
281
|
-
After modifying the chef-server.rb you could use the run-command subcommand to tell the backend and
|
282
|
-
frontend servers to run `chef-server-ctl reconfigure`.
|
283
|
-
|
284
377
|
```
|
285
|
-
dev-lxc run-command chef '
|
378
|
+
dev-lxc run-command chef 'uptime'
|
286
379
|
```
|
287
380
|
|
288
381
|
#### Attach the terminal to a server
|
@@ -290,41 +383,51 @@ dev-lxc run-command chef 'chef-server-ctl reconfigure'
|
|
290
383
|
Attach the terminal to a server in the cluster that matches the REGEX pattern given.
|
291
384
|
|
292
385
|
```
|
293
|
-
dev-lxc attach chef
|
386
|
+
dev-lxc attach chef
|
294
387
|
```
|
295
388
|
|
296
|
-
####
|
389
|
+
#### Create a snapshot of the servers
|
297
390
|
|
298
|
-
Save the changes in the servers to
|
391
|
+
Save the changes in the servers to snapshots with a comment.
|
299
392
|
|
300
393
|
```
|
301
394
|
dev-lxc halt
|
302
|
-
dev-lxc snapshot
|
395
|
+
dev-lxc snapshot -c 'this is a snapshot comment'
|
303
396
|
```
|
304
397
|
|
305
|
-
|
398
|
+
#### List snapshots
|
306
399
|
|
307
400
|
```
|
308
|
-
dev-lxc
|
401
|
+
dev-lxc snapshot -l
|
402
|
+
```
|
403
|
+
|
404
|
+
#### Restore snapshots
|
405
|
+
|
406
|
+
Restore snapshots by name.
|
407
|
+
|
408
|
+
Leave out the snapshot name or specify `LAST` to restore the most recent snapshot.
|
409
|
+
|
410
|
+
```
|
411
|
+
dev-lxc snapshot -r
|
309
412
|
dev-lxc up
|
310
413
|
```
|
311
414
|
|
312
|
-
#### Destroy
|
415
|
+
#### Destroy snapshots
|
416
|
+
|
417
|
+
Destroy snapshots by name or destroy all snapshots by specifying `ALL`.
|
313
418
|
|
314
|
-
|
315
|
-
images if you want to build them from scratch.
|
419
|
+
Leave out the snapshot name or specify `LAST` to destroy the most recent snapshots.
|
316
420
|
|
317
421
|
```
|
318
|
-
dev-lxc
|
422
|
+
dev-lxc snapshot -d
|
319
423
|
```
|
320
424
|
|
321
|
-
####
|
425
|
+
#### Destroy cluster
|
322
426
|
|
323
|
-
Use the
|
324
|
-
default LXC config_path `/var/lib/dev-lxc`.
|
427
|
+
Use the following command to destroy the cluster's servers.
|
325
428
|
|
326
429
|
```
|
327
|
-
dev-lxc
|
430
|
+
dev-lxc destroy
|
328
431
|
```
|
329
432
|
|
330
433
|
#### Use commands against specific servers
|
@@ -335,8 +438,7 @@ that matches a set of server names.
|
|
335
438
|
dev-lxc <subcommand> [SERVER_NAME_REGEX]
|
336
439
|
```
|
337
440
|
|
338
|
-
For example, to only start the Chef
|
339
|
-
you can run the following command.
|
441
|
+
For example, to only start the Chef Server you can run the following command.
|
340
442
|
|
341
443
|
```
|
342
444
|
dev-lxc up chef
|
@@ -346,8 +448,8 @@ dev-lxc up chef
|
|
346
448
|
|
347
449
|
dev-lxc can also manage an adhoc cluster of servers.
|
348
450
|
|
349
|
-
An adhoc cluster is just a set of managed servers cloned from the specified
|
350
|
-
|
451
|
+
An adhoc cluster is just a set of managed servers cloned from the specified base
|
452
|
+
container. The servers have SSH server running, a "dev-lxc" user with "dev-lxc" password and
|
351
453
|
passwordless sudo access.
|
352
454
|
|
353
455
|
This is particularly useful when you want to use something else, such as chef-provisioning,
|
@@ -356,228 +458,15 @@ to configure the servers.
|
|
356
458
|
The number of servers, their names and their IP addresses can be changed to fit your
|
357
459
|
particular requirements.
|
358
460
|
|
359
|
-
The "adhoc" template provides an example cluster configuration suitable for creating a
|
360
|
-
Chef Delivery cluster.
|
361
|
-
|
362
461
|
```
|
363
462
|
mkdir -p /root/dev/clusters/delivery
|
364
463
|
cd /root/dev/clusters/delivery
|
365
464
|
dev-lxc init --adhoc > dev-lxc.yml
|
465
|
+
# edit dev-lxc.yml to have enough adhoc servers for a delivery cluster
|
366
466
|
cluster-view
|
367
467
|
dl up
|
368
468
|
```
|
369
469
|
|
370
|
-
### Managing Node Containers
|
371
|
-
|
372
|
-
#### Manually Create a Platform Image
|
373
|
-
|
374
|
-
Platform images can be used for purposes other than building clusters. For example, they can
|
375
|
-
be used as Chef nodes for testing purposes.
|
376
|
-
|
377
|
-
Platform images have openssh-server installed and running with unique SSH Host Keys.
|
378
|
-
|
379
|
-
Platform images have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
|
380
|
-
|
381
|
-
You can see a menu of platform images this tool can create by using the following command.
|
382
|
-
|
383
|
-
```
|
384
|
-
dev-lxc create
|
385
|
-
```
|
386
|
-
|
387
|
-
The initial creation of platform images can take awhile so let's go ahead and start creating
|
388
|
-
an Ubuntu 14.04 image now.
|
389
|
-
|
390
|
-
```
|
391
|
-
dev-lxc create p-ubuntu-1404
|
392
|
-
```
|
393
|
-
|
394
|
-
Note: It is possible to pass additional arguments to the underlying LXC create command.
|
395
|
-
For example:
|
396
|
-
|
397
|
-
```
|
398
|
-
dev-lxc create p-ubuntu-1404 -o -- '--no-validate --keyserver http://my.key.server.com'
|
399
|
-
```
|
400
|
-
|
401
|
-
#### Install Chef Client in a Container
|
402
|
-
|
403
|
-
Use the `-v` option to specify a particular version of Chef Client.
|
404
|
-
|
405
|
-
Use `-v latest` or leave out the `-v` option to install the latest version of Chef Client.
|
406
|
-
|
407
|
-
For example, install the latest 11.x version of Chef Client.
|
408
|
-
|
409
|
-
```
|
410
|
-
dev-lxc install-chef-client test-node.lxc -v 11
|
411
|
-
```
|
412
|
-
|
413
|
-
#### Configure Chef Client in a Container
|
414
|
-
|
415
|
-
Use the `-s`, `-u`, `-k` options to set `chef_server_url`, `validation_client_name` and
|
416
|
-
`validation_key` in a container's `/etc/chef/client.rb` and copy the validator's key to
|
417
|
-
`/etc/chef/validation.pem`.
|
418
|
-
|
419
|
-
Or leave the options empty and it will default to using values from the cluster defined
|
420
|
-
in `dev-lxc.yml`.
|
421
|
-
|
422
|
-
```
|
423
|
-
dev-lxc config-chef-client test-node.lxc
|
424
|
-
```
|
425
|
-
|
426
|
-
#### Bootstrap Chef Client in a Container
|
427
|
-
|
428
|
-
Specifying a `BASE_CONTAINER_NAME` will clone the base container into a new container
|
429
|
-
and bootstrap it. If no `BASE_CONTAINER_NAME` is given then the container to be bootstrapped
|
430
|
-
needs to already exist.
|
431
|
-
|
432
|
-
Use the `-v` option to specify a particular version of Chef Client.
|
433
|
-
|
434
|
-
Use the `-s`, `-u`, `-k` options to set `chef_server_url`, `validation_client_name` and
|
435
|
-
`validation_key` in a container's `/etc/chef/client.rb` and copy the validator's key to
|
436
|
-
`/etc/chef/validation.pem`.
|
437
|
-
|
438
|
-
Or leave the options empty and it will default to using values from the cluster defined
|
439
|
-
in `dev-lxc.yml`.
|
440
|
-
|
441
|
-
Use the `-r` option to specify the run_list for chef-client to use.
|
442
|
-
|
443
|
-
```
|
444
|
-
dev-lxc bootstrap-container test-node.lxc -r my_run_list
|
445
|
-
```
|
446
|
-
|
447
|
-
### Using the dev-lxc library
|
448
|
-
|
449
|
-
dev-lxc cli interface can be used as a library.
|
450
|
-
|
451
|
-
```
|
452
|
-
require 'dev-lxc/cli'
|
453
|
-
|
454
|
-
ARGV = [ 'up' ] # start all servers
|
455
|
-
DevLXC::CLI::DevLXC.start
|
456
|
-
|
457
|
-
ARGV = [ 'status' ] # show status of all servers
|
458
|
-
DevLXC::CLI::DevLXC.start
|
459
|
-
|
460
|
-
ARGV = [ 'run-command', 'uptime' ] # run `uptime` in all servers
|
461
|
-
DevLXC::CLI::DevLXC.start
|
462
|
-
|
463
|
-
ARGV = [ 'destroy' ] # destroy all servers
|
464
|
-
DevLXC::CLI::DevLXC.start
|
465
|
-
```
|
466
|
-
|
467
|
-
dev-lxc itself can also be used as a library
|
468
|
-
|
469
|
-
```
|
470
|
-
require 'yaml'
|
471
|
-
require 'dev-lxc'
|
472
|
-
|
473
|
-
config = YAML.load(IO.read('dev-lxc.yml'))
|
474
|
-
server = DevLXC::Server.new("chef-fe1.lxc", 'chef-server', config)
|
475
|
-
|
476
|
-
server.start # start chef-fe1.lxc
|
477
|
-
server.status # show status of chef-fe1.lxc
|
478
|
-
server.run_command("chef-server-ctl reconfigure") # run command in chef-fe1.lxc
|
479
|
-
server.stop # stop chef-fe1.lxc
|
480
|
-
server.destroy # destroy chef-fe1.lxc
|
481
|
-
```
|
482
|
-
|
483
|
-
## Cluster Config Files
|
484
|
-
|
485
|
-
dev-lxc uses a YAML configuration file named `dev-lxc.yml` to define a cluster.
|
486
|
-
|
487
|
-
The following command generates sample config files for various cluster topologies.
|
488
|
-
|
489
|
-
```
|
490
|
-
dev-lxc init
|
491
|
-
```
|
492
|
-
|
493
|
-
`dev-lxc init --tiered-chef --analytics > dev-lxc.yml` creates a `dev-lxc.yml` file with the following content:
|
494
|
-
|
495
|
-
```
|
496
|
-
## platform_image can be one of the following:
|
497
|
-
## p-centos-5, p-centos-6, p-centos-7, p-ubuntu-1204, p-ubuntu-1404 or p-ubuntu-1604
|
498
|
-
platform_image: p-ubuntu-1404
|
499
|
-
|
500
|
-
## platform_image_options can be set to provide additional arguments to the LXC create command
|
501
|
-
## reference arg examples: https://github.com/lxc/lxc/blob/lxc-2.0.0/templates/lxc-download.in#L200-L207
|
502
|
-
#platform_image_options: --no-validate
|
503
|
-
|
504
|
-
## list any host directories you want mounted into the servers
|
505
|
-
mounts:
|
506
|
-
- /root/dev root/dev
|
507
|
-
|
508
|
-
## list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
|
509
|
-
#ssh-keys:
|
510
|
-
# - /root/dev/clusters/id_rsa.pub
|
511
|
-
|
512
|
-
## DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
|
513
|
-
|
514
|
-
chef-server:
|
515
|
-
packages:
|
516
|
-
server: /root/dev/chef-packages/cs/chef-server-core_12.5.0-1_amd64.deb
|
517
|
-
manage: /root/dev/chef-packages/manage/chef-manage_2.2.1-1_amd64.deb
|
518
|
-
reporting: /root/dev/chef-packages/reporting/opscode-reporting_1.5.6-1_amd64.deb
|
519
|
-
push-jobs-server: /root/dev/chef-packages/push-jobs-server/opscode-push-jobs-server_1.1.6-1_amd64.deb
|
520
|
-
topology: tier
|
521
|
-
api_fqdn: chef.lxc
|
522
|
-
servers:
|
523
|
-
chef-be.lxc:
|
524
|
-
ipaddress: 10.0.3.201
|
525
|
-
role: backend
|
526
|
-
bootstrap: true
|
527
|
-
chef-fe1.lxc:
|
528
|
-
ipaddress: 10.0.3.202
|
529
|
-
role: frontend
|
530
|
-
|
531
|
-
analytics:
|
532
|
-
packages:
|
533
|
-
analytics: /root/dev/chef-packages/analytics/opscode-analytics_1.3.1-1_amd64.deb
|
534
|
-
servers:
|
535
|
-
analytics.lxc:
|
536
|
-
ipaddress: 10.0.3.204
|
537
|
-
```
|
538
|
-
|
539
|
-
This config defines a tier cluster consisting of a single backend and a single frontend.
|
540
|
-
|
541
|
-
A second frontend is commented out to conserve resources. If you uncomment the second
|
542
|
-
frontend then both frontends will be created and dnsmasq will resolve the `api_fqdn`
|
543
|
-
[chef.lxc](chef.lxc) to both frontends using a round-robin policy.
|
544
|
-
|
545
|
-
The config file is very customizable. You can add or remove mounts, packages or servers,
|
546
|
-
change ip addresses, change server names, change the platform_image and more.
|
547
|
-
|
548
|
-
The `mounts` list describes what directories get mounted from the Vagrant VM platform into
|
549
|
-
each container. You need to make sure that you configure the mount entries to be
|
550
|
-
appropriate for your environment.
|
551
|
-
|
552
|
-
The same is true for the `packages` list. The paths that are provided in the default configs are just examples.
|
553
|
-
You need to make sure that you have each package you want to use downloaded to appropriate directories
|
554
|
-
that will be available to the container when it is started.
|
555
|
-
|
556
|
-
I recommend downloading the packages to a directory on your workstation.
|
557
|
-
Then configure the
|
558
|
-
[dev-lxc-platform's .kitchen.yml](https://github.com/jeremiahsnapp/dev-lxc-platform#description)
|
559
|
-
to mount that directory in the Vagrant VM platform.
|
560
|
-
Then configure the cluster's mount entries in `dev-lxc.yml` to mount the Vagrant VM platform's
|
561
|
-
directory into each container.
|
562
|
-
|
563
|
-
Make sure the mounts and packages represent actual paths that are available in your environment.
|
564
|
-
|
565
|
-
### Managing Multiple Clusters
|
566
|
-
|
567
|
-
By default, `dev-lxc` looks for a `dev-lxc.yml` file in the present working directory.
|
568
|
-
You can also specify a particular config file as an option for most dev-lxc commands.
|
569
|
-
|
570
|
-
The following is an example of managing multiple clusters while still avoiding specifying
|
571
|
-
each cluster's config file.
|
572
|
-
|
573
|
-
```
|
574
|
-
mkdir -p ~/clusters/{clusterA,clusterB}
|
575
|
-
dev-lxc init --tiered-chef > ~/clusters/clusterA/dev-lxc.yml
|
576
|
-
dev-lxc init --chef > ~/clusters/clusterB/dev-lxc.yml
|
577
|
-
cd ~/clusters/clusterA && dev-lxc up # starts clusterA
|
578
|
-
cd ~/clusters/clusterB && dev-lxc up # starts clusterB
|
579
|
-
```
|
580
|
-
|
581
470
|
### Maintain Uniqueness Across Multiple Clusters
|
582
471
|
|
583
472
|
The default cluster configs are already designed to be unique from each other but as you build
|
@@ -596,129 +485,18 @@ more clusters you have to maintain uniqueness across the YAML config files for t
|
|
596
485
|
If cluster B is started with the same `api_fqdn` or `analytics_fqdn` as an already running cluster A,
|
597
486
|
then cluster B will overwrite cluster A's DNS resolution of `api_fqdn` or `analytics_fqdn`.
|
598
487
|
|
599
|
-
It is easy to provide uniqueness in the server names, `api_fqdn` and `analytics_fqdn`.
|
600
|
-
For example, you can use the following command to prefix the servers names with `1234-` when
|
601
|
-
generating a cluster's config.
|
602
|
-
|
603
|
-
dev-lxc init --tiered-chef 1234- > dev-lxc.yml
|
604
|
-
|
605
488
|
* IP Addresses
|
606
489
|
|
607
490
|
IP addresses uniqueness only matters when clusters with the same IP's are running.
|
608
|
-
|
491
|
+
|
609
492
|
If cluster B is started with the same IP's as an already running cluster A, then cluster B
|
610
493
|
will overwrite cluster A's DHCP reservation of the IP's but dnsmasq will still refuse to
|
611
494
|
assign the IP's to cluster B because they already in use by cluster A. dnsmasq then assigns
|
612
495
|
random IP's from the DHCP pool to cluster B leaving it in an unexpected state.
|
613
496
|
|
614
497
|
The `dev-lxc-platform` creates the IP range 10.0.3.150 - 254 for DHCP reserved IP's.
|
615
|
-
|
616
|
-
Use unique IP's from that range when configuring clusters.
|
617
|
-
|
618
|
-
## Images
|
619
|
-
|
620
|
-
One of the key things this tool uses is the concept of images.
|
621
|
-
|
622
|
-
`dev-lxc` creates images with a "p-", "s-", "u-" or "c-" prefix on the name to distinguish
|
623
|
-
it as a "platform", "shared", "unique" or "custom" image.
|
624
|
-
|
625
|
-
Images are then cloned using the btrfs filesystem to very quickly provide a lightweight duplicate
|
626
|
-
of the image. This clone is either used to build the next image in the build process or the final
|
627
|
-
container that will actually be run.
|
628
|
-
|
629
|
-
By default, the cluster's images and final server containers are all stored in `/var/lib/dev-lxc`
|
630
|
-
so they don't clutter the containers stored in the default LXC config_path `/var/lib/lxc`.
|
631
|
-
|
632
|
-
The cluster's LXC config_path can be configured by setting `lxc_config_path` at the top of the
|
633
|
-
`dev-lxc.yml` file to the desired directory.
|
634
|
-
|
635
|
-
There are four image categories.
|
636
|
-
|
637
|
-
1. Platform Image
|
638
|
-
|
639
|
-
The platform image is the first to get created and is identified by the
|
640
|
-
"p-" prefix on the image name.
|
641
|
-
|
642
|
-
`DevLXC#create_platform_image` controls the creation of a platform image.
|
643
498
|
|
644
|
-
|
645
|
-
A typical LXC container has minimal packages installed so `dev-lxc` makes sure that the
|
646
|
-
same packages used in Chef's [bento boxes](https://github.com/opscode/bento) are
|
647
|
-
installed to provide a more typical server environment.
|
648
|
-
A few additional packages are also installed.
|
649
|
-
|
650
|
-
Platform images have openssh-server installed and running with unique SSH Host Keys.
|
651
|
-
|
652
|
-
Platform images have a "dev-lxc" user with "dev-lxc" password and passwordless sudo.
|
653
|
-
|
654
|
-
*Once this platform image is created there is rarely a need to delete it.*
|
655
|
-
|
656
|
-
2. Shared Image
|
657
|
-
|
658
|
-
The shared image is the second to get created and is identified by the
|
659
|
-
"s-" prefix on the image name.
|
660
|
-
|
661
|
-
`DevLXC::Server#create_shared_image` controls the creation of a shared image.
|
662
|
-
|
663
|
-
Chef packages that are common to all servers in a Chef cluster, such as chef-server-core,
|
664
|
-
opscode-reporting and opscode-push-jobs-server are installed using `dpkg` or `rpm`.
|
665
|
-
|
666
|
-
Note the manage package will not be installed at this point since it is not common to all
|
667
|
-
servers (i.e. it does not get installed on backend servers).
|
668
|
-
|
669
|
-
The name of this image is built from the names and versions of the Chef packages that
|
670
|
-
get installed which makes this image easy to be reused by another cluster that is
|
671
|
-
configured to use the same Chef packages.
|
672
|
-
|
673
|
-
*Since no configuration actually happens yet there is rarely a need to delete this image.*
|
674
|
-
|
675
|
-
3. Unique Image
|
676
|
-
|
677
|
-
The unique image is the last to get created and is identified by the
|
678
|
-
"u-" prefix on the image name.
|
679
|
-
|
680
|
-
`DevLXC::Server#create` controls the creation of a unique image.
|
681
|
-
|
682
|
-
Each unique Chef server (e.g. standalone, backend or frontend) is created.
|
683
|
-
|
684
|
-
* The specified hostname is assigned.
|
685
|
-
* dnsmasq is configured to reserve the specified IP address for the image's MAC address.
|
686
|
-
* A DNS entry is created in dnsmasq if appropriate.
|
687
|
-
* All installed Chef packages are configured.
|
688
|
-
* Test users and orgs are created.
|
689
|
-
* The opscode-manage package is installed and configured if specified.
|
690
|
-
|
691
|
-
After each server is fully configured a clone of it is made resulting in the server's
|
692
|
-
unique image. These unique images make it very easy to quickly recreate
|
693
|
-
a Chef cluster from a clean starting point.
|
694
|
-
|
695
|
-
4. Custom Image
|
696
|
-
|
697
|
-
The custom image is only created when the `snapshot` command is used and is identified
|
698
|
-
by the "c-" prefix on the image name.
|
699
|
-
|
700
|
-
`DevLXC::Server#snapshot` controls the creation of a custom image.
|
701
|
-
|
702
|
-
Custom images can be used to save the changes that have been made in servers.
|
703
|
-
Later, when the servers are destroyed and recreated, they will start running with the changes
|
704
|
-
that were captured at the time of the snapshot.
|
705
|
-
|
706
|
-
### Destroying Images
|
707
|
-
|
708
|
-
When using `dev-lxc destroy` to destroy servers you have the option to also destroy any or all of
|
709
|
-
the four types of images associated with the servers.
|
710
|
-
|
711
|
-
The following command will list the options available.
|
712
|
-
|
713
|
-
```
|
714
|
-
dev-lxc help destroy
|
715
|
-
```
|
716
|
-
|
717
|
-
Of course, you can also just use the standard LXC commands to destroy any container.
|
718
|
-
|
719
|
-
```
|
720
|
-
lxc-destroy -n [NAME]
|
721
|
-
```
|
499
|
+
Use unique IP's from that range when configuring clusters.
|
722
500
|
|
723
501
|
## Contributing
|
724
502
|
|