bosh-bootstrap 0.5.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +18 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +22 -0
- data/README.md +318 -0
- data/Rakefile +1 -0
- data/bin/bosh-bootstrap +8 -0
- data/bosh-bootstrap.gemspec +34 -0
- data/lib/bosh-bootstrap.rb +10 -0
- data/lib/bosh-bootstrap/cli.rb +1024 -0
- data/lib/bosh-bootstrap/commander.rb +9 -0
- data/lib/bosh-bootstrap/commander/README.md +47 -0
- data/lib/bosh-bootstrap/commander/command.rb +25 -0
- data/lib/bosh-bootstrap/commander/commands.rb +80 -0
- data/lib/bosh-bootstrap/commander/local_server.rb +68 -0
- data/lib/bosh-bootstrap/commander/remote_script_command.rb +48 -0
- data/lib/bosh-bootstrap/commander/remote_server.rb +121 -0
- data/lib/bosh-bootstrap/commander/upload_command.rb +17 -0
- data/lib/bosh-bootstrap/helpers.rb +2 -0
- data/lib/bosh-bootstrap/helpers/fog_setup.rb +50 -0
- data/lib/bosh-bootstrap/helpers/settings.rb +36 -0
- data/lib/bosh-bootstrap/stages.rb +8 -0
- data/lib/bosh-bootstrap/stages/stage_micro_bosh_delete.rb +90 -0
- data/lib/bosh-bootstrap/stages/stage_micro_bosh_delete/bosh_micro_delete +19 -0
- data/lib/bosh-bootstrap/stages/stage_micro_bosh_deploy.rb +135 -0
- data/lib/bosh-bootstrap/stages/stage_micro_bosh_deploy/bosh_micro_deploy +36 -0
- data/lib/bosh-bootstrap/stages/stage_micro_bosh_deploy/download_micro_bosh_stemcell +132 -0
- data/lib/bosh-bootstrap/stages/stage_micro_bosh_deploy/install_key_pair_for_user +23 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm.rb +52 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm/convert_salted_password +9 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm/create_vcap_user +79 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm/install_base_packages +13 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm/install_bosh +54 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm/install_ruby +33 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm/install_useful_gems +24 -0
- data/lib/bosh-bootstrap/stages/stage_prepare_inception_vm/validate_bosh_deployer +21 -0
- data/lib/bosh-bootstrap/stages/stage_setup_new_bosh.rb +52 -0
- data/lib/bosh-bootstrap/stages/stage_setup_new_bosh/cleanup_permissions +14 -0
- data/lib/bosh-bootstrap/stages/stage_setup_new_bosh/setup_bosh_user +29 -0
- data/lib/bosh-bootstrap/stages/stage_validate_inception_vm.rb +39 -0
- data/lib/bosh-bootstrap/stages/stage_validate_inception_vm/validate_ubuntu +6 -0
- data/lib/bosh-bootstrap/version.rb +5 -0
- data/lib/bosh/providers.rb +21 -0
- data/lib/bosh/providers/README.md +5 -0
- data/lib/bosh/providers/aws.rb +77 -0
- data/lib/bosh/providers/base_provider.rb +20 -0
- data/lib/bosh/providers/openstack.rb +40 -0
- metadata +239 -0
data/.gitignore
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2012 Stark & Wayne LLC
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,318 @@
|
|
1
|
+
# Stark & Wayne's Bosh Bootstrapper
|
2
|
+
|
3
|
+
In order to deploy CloudFoundry, and a growing number of other complex systems, you will need a BOSH. BOSH provides a complete lifecycle manager/deployer for complex systems. CloudFoundry is a very complex system when it comes to deployment/upgrades.
|
4
|
+
|
5
|
+
The Stark & Wayne Bosh Bootstrapper is the simplest way to get a Micro BOSH running, to upgrade an existing Micro BOSH, and to delete it if you change your mind.
|
6
|
+
|
7
|
+
Bootstrap a Micro BOSH universe from one CLI command. Also allows SSH access and the ability to delete created Micro BOSHes.
|
8
|
+
|
9
|
+
```
|
10
|
+
$ bosh-bootstrap deploy --latest-stemcell
|
11
|
+
Creating inception VM...
|
12
|
+
Creating micro BOSH VM...
|
13
|
+
|
14
|
+
$ bosh-bootstrap ssh
|
15
|
+
Open SSH tunnel to inception VM...
|
16
|
+
|
17
|
+
$ bosh-bootstrap delete
|
18
|
+
Deleting micro BOSH VM...
|
19
|
+
```
|
20
|
+
|
21
|
+
It is now very simple to bootstrap a micro BOSH from a single, local CLI. The bootstrapper first creates an inception VM and then uses the `bosh_deployer` (`bosh micro deploy`) to deploy micro BOSH.
|
22
|
+
|
23
|
+
To be cute about it, the Stark & Wayne Bosh Bootstrapper aims to provide lifecycle management for the BOSH lifecycle manager. Zing! See the "Deep dive into deploy command" section below for greater understanding why the Stark & Wayne Bosh Bootstrapper is very useful.
|
24
|
+
|
25
|
+
## Installation
|
26
|
+
|
27
|
+
This bootstrapper for BOSH is distributed as a RubyGem for Ruby 1.8+.
|
28
|
+
|
29
|
+
```
|
30
|
+
$ gem install bosh-bootstrap
|
31
|
+
```
|
32
|
+
|
33
|
+
## Usage
|
34
|
+
|
35
|
+
### First time usage
|
36
|
+
|
37
|
+
The first time you use `bosh-bootstrap` it will create everything necessary. The example output below includes user prompts.
|
38
|
+
|
39
|
+
```
|
40
|
+
$ bosh-bootstrap deploy --latest-stemcell
|
41
|
+
|
42
|
+
Stage 1: Choose infrastructure
|
43
|
+
|
44
|
+
Found infrastructure API credentials at ~/.fog (override with --fog)
|
45
|
+
1. AWS (default)
|
46
|
+
2. AWS (bosh)
|
47
|
+
3. Rackspace (default)
|
48
|
+
Choose infrastructure: 1
|
49
|
+
|
50
|
+
Confirming: using AWS infrastructure.
|
51
|
+
|
52
|
+
1. ap-northeast-1
|
53
|
+
2. ap-southeast-1
|
54
|
+
3. eu-west-1
|
55
|
+
4. us-east-1
|
56
|
+
5. us-west-1
|
57
|
+
6. us-west-2
|
58
|
+
7. sa-east-1
|
59
|
+
Choose AWS region: 6
|
60
|
+
Confirming: Using AWS us-west-2 region.
|
61
|
+
|
62
|
+
|
63
|
+
Stage 2: Configuration
|
64
|
+
|
65
|
+
Confirming: Micro BOSH will be named microbosh_aws_us_east_1
|
66
|
+
|
67
|
+
BOSH username: drnic
|
68
|
+
BOSH password: ********
|
69
|
+
Confirming: After BOSH is created, your username will be drnic
|
70
|
+
|
71
|
+
Confirming: Micro BOSH will be assigned IP address 174.129.227.124
|
72
|
+
|
73
|
+
Confirming: Micro BOSH protected by security group named microbosh_aws_us_east_1, with ports [22, 6868, 25555, 25888]
|
74
|
+
|
75
|
+
Confirming: Micro BOSH accessible via key pair named microbosh_aws_us_east_1
|
76
|
+
|
77
|
+
Confirming: Micro BOSH will be created with stemcell micro-bosh-stemcell-aws-0.6.4.tgz
|
78
|
+
|
79
|
+
|
80
|
+
Stage 3: Create/Allocate the Inception VM
|
81
|
+
|
82
|
+
1. create new inception VM
|
83
|
+
2. use an existing Ubuntu server
|
84
|
+
3. use this server (must be ubuntu & on same network as bosh)
|
85
|
+
Create or specify an Inception VM: 1
|
86
|
+
|
87
|
+
Confirming: Inception VM has been created
|
88
|
+
|
89
|
+
Stage 4: Preparing the Inception VM
|
90
|
+
|
91
|
+
Successfully created vcap user
|
92
|
+
Successfully installed base packages
|
93
|
+
Successfully installed ruby 1.9.3
|
94
|
+
Successfully installed useful ruby gems
|
95
|
+
Successfully installed bosh
|
96
|
+
Successfully captured value of salted password
|
97
|
+
Successfully validated bosh deployer
|
98
|
+
|
99
|
+
Stage 5: Deploying micro BOSH
|
100
|
+
|
101
|
+
Successfully downloaded micro-bosh stemcell
|
102
|
+
Successfully uploaded micro-bosh deployment manifest file
|
103
|
+
Successfully installed key pair for user
|
104
|
+
Successfully deploy micro bosh
|
105
|
+
```
|
106
|
+
|
107
|
+
### Local usage
|
108
|
+
|
109
|
+
During the `bosh-bootstrap deploy` sequence above, you could choose to use the local VM as the Inception VM.
|
110
|
+
|
111
|
+
For AWS, it is important that you only use a VM that is on the same infrastructure region. The process of creating a micro-bosh VM is to use a local
|
112
|
+
EBS volume to create an AMI. The target region for the micro-bosh VM must therefore be in the same region.
|
113
|
+
|
114
|
+
### Repeat usage
|
115
|
+
|
116
|
+
The `deploy` command can be re-run and it will not prompt again for inputs. It aims to be idempotent. This means that if you see any errors when running `deploy`, such as unavailability of VMs or IP addresses, then when you resolve those issues you can re-run the `deploy` command and it will resume the bootstrap of micro-bosh (and the optional inception VM).
|
117
|
+
|
118
|
+
## SSH access
|
119
|
+
|
120
|
+
You can open an SSH shell with the Inception VM:
|
121
|
+
|
122
|
+
```
|
123
|
+
$ bosh-bootstrap ssh
|
124
|
+
```
|
125
|
+
|
126
|
+
You can also pass a COMMAND argument and that command will be run instead of the shell being opened.
|
127
|
+
|
128
|
+
```
|
129
|
+
$ bosh-bootstrap ssh 'whoami'
|
130
|
+
ubuntu
|
131
|
+
```
|
132
|
+
|
133
|
+
## Deleting micro BOSH
|
134
|
+
|
135
|
+
The `bosh-bootstrap delete` command will delete the target micro-bosh.
|
136
|
+
|
137
|
+
```
|
138
|
+
$ bosh-bootstrap delete
|
139
|
+
Stage 1: Target inception VM to use to delete micro-bosh
|
140
|
+
|
141
|
+
Confirming: Using inception VM ubuntu@ec2-184-73-231-239.compute-1.amazonaws.com
|
142
|
+
|
143
|
+
Stage 2: Deleting micro BOSH
|
144
|
+
Delete micro BOSH
|
145
|
+
stopping agent services (00:00:01)
|
146
|
+
unmount disk (00:00:10)
|
147
|
+
detach disk (00:00:13)
|
148
|
+
delete disk (00:02:35)
|
149
|
+
delete VM (00:00:37)
|
150
|
+
delete stemcell (00:00:00)
|
151
|
+
Done 6/6 00:03:37
|
152
|
+
Deleted deployment 'microbosh-aws-us-east-1', took 00:03:37 to complete
|
153
|
+
```
|
154
|
+
|
155
|
+
## Deep dive into the BOSH Bootstrap deploy command
|
156
|
+
|
157
|
+
What is actually happening when you run `bosh-bootstrap deploy`?
|
158
|
+
|
159
|
+
At the heart of `bosh-bootstrap deploy` is the execution of the BOSH Deployer, a command provided with BOSH to bootstrap a single VM with all the parts of BOSH running on it. If you ran this command yourself you would run:
|
160
|
+
|
161
|
+
```
|
162
|
+
$ gem install bosh-deployer
|
163
|
+
$ bosh download public stemcell some-microbosh-stemcell.tgz
|
164
|
+
$ bosh micro deploy some-microbosh-stemcell.tgz
|
165
|
+
```
|
166
|
+
|
167
|
+
Unfortunately for this simple scenario, there are many little prerequisite steps before those three commands. The Stark & Wayne Bosh Bootstrapper replaces pages and pages of step-by-step instructions with a single command line that does everything. It even allows you to upgrade your Micro BOSH with newer BOSH releases: both publicly available stemcells and custom stemcells generated from the BOSH source code.
|
168
|
+
|
169
|
+
To understand exactly what the `bosh-bootstrap deploy` command is doing, let's start with what the running parts of BOSH are and how `bosh micro deploy` deploys them.
|
170
|
+
|
171
|
+
### What is in BOSH?
|
172
|
+
|
173
|
+
A running BOSH, whether it is running on a single server or a cluster of servers, is a collection of processes. The core of BOSH is the Director and the Blobstore. The remaining processes provide support, storage or messaging.
|
174
|
+
|
175
|
+
* The Director, the public API for the bosh CLI and coordinator of BOSH behavior
|
176
|
+
* The Blobstore, to store and retrieve precompiled packages
|
177
|
+
* Agents, run on each server within deployments
|
178
|
+
* The Health Manager, to track the state of deployed systems (the infrastructure and running jobs)
|
179
|
+
* Internal DNS, called PowerDNS, for internal unique naming of servers within BOSH deployments
|
180
|
+
* Registry, for example AWS Registry, for tracking the infrastructure that has been provisioned (servers, persistent disks)
|
181
|
+
* PostgreSQL
|
182
|
+
* Redis
|
183
|
+
|
184
|
+
When you deploy a BOSH using the BOSH Deployer (`bosh micro deploy`) or indirectly via the BOSH Bootstrapper, you are actually deploying a BOSH release that describes a BOSH called [bosh-release](https://github.com/cloudfoundry/bosh-release). The processes listed above are called "jobs" and you can see the full list of jobs inside a BOSH within the [jobs/ directory](https://github.com/cloudfoundry/bosh-release/jobs) of `bosh-release`.
|
185
|
+
|
186
|
+
But you don't yet have a BOSH to deploy another BOSH.
|
187
|
+
|
188
|
+
### How to get your first BOSH?
|
189
|
+
|
190
|
+
The BOSH Deployer (`bosh micro deploy`) exists to spin you up a pre-baked server with all the packages and jobs running.
|
191
|
+
|
192
|
+
When you run the BOSH Deployer on a server, it does not convert that server into a BOSH. Rather, it provisions a single brand new server, with all the required packages, configuration and startup scripts. We call this pre-baked server a Micro BOSH.
|
193
|
+
|
194
|
+
A Micro BOSH server is a normal running server built from a base OS image that already contains all the packages, configuration and startup scripts for the jobs listed above.
|
195
|
+
|
196
|
+
In BOSH terminology, call these pre-packaged base OS images "stemcells".
|
197
|
+
|
198
|
+
For AWS, vSphere and OpenStack there are publicly available stemcells that can bootstrap a Micro BOSH for that infrastructure. To see the current list of all public Micro BOSH stemcells for all infrastructure providers; and to download one of them:
|
199
|
+
|
200
|
+
```
|
201
|
+
$ bosh public stemcells --tag micro
|
202
|
+
$ bosh download public stemcell micro-bosh-stemcell-aws-0.6.4.tgz
|
203
|
+
```
|
204
|
+
|
205
|
+
The CloudFoundry BOSH team will release new public stemcells overtime. The BOSH Deployer allows you to upgrade to newer stemcells as easily as it is to deploy a Micro BOSH initially.
|
206
|
+
|
207
|
+
```
|
208
|
+
$ bosh micro deploy micro-bosh-stemcell-aws-0.6.4.tgz
|
209
|
+
$ bosh micro deploy micro-stemcell-aws-0.7.0.tgz --update
|
210
|
+
```
|
211
|
+
|
212
|
+
### Configuring a Micro BOSH
|
213
|
+
|
214
|
+
The command above will not work without first providing BOSH Deployer with configuration details. The stemcell file alone is not sufficient information. When we deploy or update a Micro BOSH we need to provide the following:
|
215
|
+
|
216
|
+
* A static IP address - this IP address will be bound to the initial Micro BOSH server, and when the Micro BOSH is updated in future and the server is thrown away and replaced, then it is bound to the replacement servers
|
217
|
+
* Server properties - the instance type (such as m1.large on AWS) or RAM/CPU combination (on vSphere)
|
218
|
+
* Server persistent disk - a single persistent, attached disk volume will be provisioned and mounted at `/var/vcap/store`; when the Micro BOSH is updated is is unmounted, unattached from the current server and then reattached and remounted to the upgraded server
|
219
|
+
* Infrastructure API credentials - the magic permissions for the Micro BOSH to provision servers and persistent disks for its BOSH deployments
|
220
|
+
|
221
|
+
This information is to go into a file called `/path/to/deployments/NAME/micro_bosh.yml`. Before `bosh micro deploy` is run, we first need to tell BOSH Deployer which file contains the Micro BOSH deployment manifest.
|
222
|
+
|
223
|
+
In the Stark & Wayne Bosh Bootstrapper, the manifests are stored at `/var/vcap/store/microboshes/deployments/NAME/micro_bosh.yml`.
|
224
|
+
|
225
|
+
So the BOSH Deployer command that is run to specify the deployment manifest and run the deployment is:
|
226
|
+
|
227
|
+
```
|
228
|
+
$ bosh micro deployment `/var/vcap/store/microboshes/deployments/NAME/micro_bosh.yml`
|
229
|
+
$ bosh micro deploy /var/vcap/store/stemcells/micro-bosh-stemcell-aws-0.6.4.tgz
|
230
|
+
```
|
231
|
+
|
232
|
+
### Why does it take so long to deploy Micro BOSH on AWS?
|
233
|
+
|
234
|
+
On AWS it can take over 20 minutes to deploy or upgrade a Micro BOSH from a public stemcell. The majority of this time is taken with converting the stemcell file (such as `micro-bosh-stemcell-aws-0.6.4.tgz`) into an Amazon AMI.
|
235
|
+
|
236
|
+
When you boot a new server on AWS you provide the base machine image for the root filesystem. This is called the Amazon Machine Image (AMI). For our Micro BOSH, we need an AMI that contains all the packages, process configuration and startup scripts. That is, we need to convert our stemcell into an AMI; then use the AMI to boot the Micro BOSH server.
|
237
|
+
|
238
|
+
The BOSH Deployer performs all the hard work to create an AMI. Believe me, it is a lot of hard work.
|
239
|
+
|
240
|
+
The summary of the process of creating the Micro BOSH AMI is:
|
241
|
+
|
242
|
+
1. Create a new EBS volume (an attached disk) on the server running BOSH Deployer
|
243
|
+
2. Unpack/upload the stemcell onto the EBS volume
|
244
|
+
3. Create a snapshot of the EBS volume
|
245
|
+
4. Register the snapshot as an AMI
|
246
|
+
|
247
|
+
This process takes the majority of the time to deploy a new/replacement Micro BOSH server.
|
248
|
+
|
249
|
+
### Why can't I run BOSH Deployer from my laptop?
|
250
|
+
|
251
|
+
One of the feature of the BOSH Bootstrapper is that you can run it from your local laptop. BOSH Deployer itself cannot be run from your laptop. The reason is hidden in the step-by-step AMI example above. In AWS, to create an EBS volume, create a snapshot and register it as an AMI, you need to be running the commands on an AWS server in the same target region as your future Micro BOSH server.
|
252
|
+
|
253
|
+
The server that runs the BOSH Deployer is commonly called the Inception VM. For AWS you need an Inception VM in the same AWS region that you will provision your Micro BOSH server. Since a BOSH also manages a stemcell process similar to the above, your BOSH must be in the same AWS region that you a deploying BOSH releases.
|
254
|
+
|
255
|
+
### How does BOSH Bootstrapper get around this requirement?
|
256
|
+
|
257
|
+
The BOSH Bootstrapper can run from your laptop or locally from an Inception VM.
|
258
|
+
|
259
|
+
If you run it from your laptop, then it will prompt you to create a new Inception VM or for the host/username of a pre-existing Ubuntu server. The BOSH Bootstrapper will then use SSH to command the Inception VM to perform all the deployment steps discussed above.
|
260
|
+
|
261
|
+
That is the core of the service being provided by the BOSH Bootstrapper - to prepare an Inception VM and to command it to deploy and upgrade Micro BOSHes.
|
262
|
+
|
263
|
+
## Internal configuration/settings
|
264
|
+
|
265
|
+
Once you've used the CLI it stores your settings for your BOSH, so that you can re-run the tool for upgrades or other future functionality.
|
266
|
+
|
267
|
+
By default, the settings file is stored at `~/.bosh_bootstrap/manifest.yml`.
|
268
|
+
|
269
|
+
For an AWS BOSH it looks like:
|
270
|
+
|
271
|
+
``` yaml
|
272
|
+
---
|
273
|
+
fog_path: /Users/drnic/.fog
|
274
|
+
fog_credentials:
|
275
|
+
provider: AWS
|
276
|
+
aws_access_key_id: ACCESS_KEY
|
277
|
+
aws_secret_access_key: SECRET_KEY
|
278
|
+
region: us-east-1
|
279
|
+
bosh_cloud_properties:
|
280
|
+
aws:
|
281
|
+
access_key_id: ACCESS_KEY
|
282
|
+
secret_access_key: SECRET_KEY
|
283
|
+
default_key_name: microbosh
|
284
|
+
default_security_groups:
|
285
|
+
- microbosh
|
286
|
+
ec2_private_key: /home/vcap/.ssh/microbosh.pem
|
287
|
+
bosh_resources_cloud_properties:
|
288
|
+
instance_type: m1.medium
|
289
|
+
bosh_provider: aws
|
290
|
+
region_code: us-east-1
|
291
|
+
bosh_username: drnic
|
292
|
+
bosh_password: PASSWORD
|
293
|
+
bosh:
|
294
|
+
password: PASSWORD
|
295
|
+
salted_password: 'sdfkjhadsjkadsfjhdsf'
|
296
|
+
persistent_disk: 16384
|
297
|
+
ip_address: 107.22.247.45
|
298
|
+
micro_bosh_stemcell_name: "micro-bosh-stemcell-aws-0.6.4.tgz"
|
299
|
+
```
|
300
|
+
|
301
|
+
## Contributing
|
302
|
+
|
303
|
+
1. Fork it
|
304
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
305
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
306
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
307
|
+
5. Create new Pull Request
|
308
|
+
|
309
|
+
## Copyright
|
310
|
+
|
311
|
+
All documentation and source code is copyright of Stark & Wayne LLC.
|
312
|
+
|
313
|
+
## Subscription and Support
|
314
|
+
|
315
|
+
This documentation & tool is freely available to all people and companies coming to CloudFoundry and BOSH.
|
316
|
+
|
317
|
+
If you decide to run CloudFoundry and BOSH in production, please purchase a Subscription and Support Agreement with Stark & Wayne so we can continue to create and maintain top quality documentation and tools; and also provide you with bespoke support for your deployments. We want to help you be successfully.
|
318
|
+
|
data/Rakefile
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require "bundler/gem_tasks"
|
data/bin/bosh-bootstrap
ADDED
@@ -0,0 +1,34 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'bosh-bootstrap/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |gem|
|
7
|
+
gem.name = "bosh-bootstrap"
|
8
|
+
gem.version = Bosh::Bootstrap::VERSION
|
9
|
+
gem.authors = ["Dr Nic Williams"]
|
10
|
+
gem.email = ["drnicwilliams@gmail.com"]
|
11
|
+
gem.description = %q{Bootstrap a micro BOSH universe from one CLI}
|
12
|
+
gem.summary = <<-EOS
|
13
|
+
Now very simple to bootstrap a micro BOSH from a single, local CLI.
|
14
|
+
The bootstrapper first creates an inception VM and then uses
|
15
|
+
bosh_deployer (bosh micro deploy) to deploy micro BOSH from
|
16
|
+
an available stemcell.
|
17
|
+
EOS
|
18
|
+
gem.homepage = "https://github.com/StarkAndWayne/bosh-bootstrap"
|
19
|
+
|
20
|
+
gem.files = `git ls-files`.split($/)
|
21
|
+
gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
|
22
|
+
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
23
|
+
gem.require_paths = ["lib"]
|
24
|
+
|
25
|
+
gem.add_dependency "thor"
|
26
|
+
gem.add_dependency "highline"
|
27
|
+
gem.add_dependency "settingslogic"
|
28
|
+
gem.add_dependency "POpen4"
|
29
|
+
gem.add_dependency "net-scp"
|
30
|
+
gem.add_dependency "fog", "~>1.8.0"
|
31
|
+
gem.add_dependency "escape"
|
32
|
+
gem.add_dependency "bosh_cli"
|
33
|
+
gem.add_development_dependency "bosh_deployer"
|
34
|
+
end
|
@@ -0,0 +1,1024 @@
|
|
1
|
+
require "thor"
|
2
|
+
require "highline"
|
3
|
+
require "fileutils"
|
4
|
+
|
5
|
+
# for the #sh helper
|
6
|
+
require "rake"
|
7
|
+
require "rake/file_utils"
|
8
|
+
|
9
|
+
require "escape"
|
10
|
+
|
11
|
+
require "bosh-bootstrap/helpers"
|
12
|
+
|
13
|
+
module Bosh::Bootstrap
|
14
|
+
class Cli < Thor
|
15
|
+
include Thor::Actions
|
16
|
+
include Bosh::Bootstrap::Helpers::FogSetup
|
17
|
+
include Bosh::Bootstrap::Helpers::Settings
|
18
|
+
include FileUtils
|
19
|
+
|
20
|
+
attr_reader :fog_credentials
|
21
|
+
attr_reader :server
|
22
|
+
|
23
|
+
desc "deploy", "Bootstrap Micro BOSH, and optionally an Inception VM"
|
24
|
+
method_option :fog, :type => :string, :desc => "fog config file (default: ~/.fog)"
|
25
|
+
method_option :"private-key", :type => :string, :desc => "Local passphrase-less private key path"
|
26
|
+
method_option :"upgrade-deps", :type => :boolean, :desc => "Force upgrade dependencies, packages & gems"
|
27
|
+
method_option :"edge-deployer", :type => :boolean, :desc => "Install bosh deployer from git instead of rubygems"
|
28
|
+
method_option :"latest-stemcell", :type => :boolean, :desc => "Use latest micro-bosh stemcell; possibly not tagged stable"
|
29
|
+
method_option :"edge-stemcell", :type => :boolean, :desc => "Create custom stemcell from BOSH git source"
|
30
|
+
def deploy
|
31
|
+
load_deploy_options # from method_options above
|
32
|
+
|
33
|
+
deploy_stage_1_choose_infrastructure_provider
|
34
|
+
deploy_stage_2_bosh_configuration
|
35
|
+
deploy_stage_3_create_allocate_inception_vm
|
36
|
+
deploy_stage_4_prepare_inception_vm
|
37
|
+
deploy_stage_5_deploy_micro_bosh
|
38
|
+
deploy_stage_6_setup_new_bosh
|
39
|
+
end
|
40
|
+
|
41
|
+
# desc "delete", "Delete Micro BOSH"
|
42
|
+
# method_option :all, :type => :boolean, :desc => "Delete all micro-boshes and inception VM [coming soon]"
|
43
|
+
# def delete
|
44
|
+
# delete_stage_1_target_inception_vm
|
45
|
+
#
|
46
|
+
# if options[:all]
|
47
|
+
# error "I'm sorry; the awesome --all flag is not yet implemented"
|
48
|
+
# delete_all_stage_2_delete_micro_boshes
|
49
|
+
# delete_all_stage_3_delete_inception_vm
|
50
|
+
# else
|
51
|
+
# delete_one_stage_2_delete_micro_bosh
|
52
|
+
# end
|
53
|
+
# end
|
54
|
+
#
|
55
|
+
desc "ssh [COMMAND]", "Open an ssh session to the inception VM [do nothing if local machine is inception VM]"
|
56
|
+
long_desc <<-DESC
|
57
|
+
If a command is supplied, it will be run, otherwise a session will be
|
58
|
+
opened.
|
59
|
+
DESC
|
60
|
+
def ssh(cmd=nil)
|
61
|
+
run_ssh_command_or_open_tunnel(cmd)
|
62
|
+
end
|
63
|
+
|
64
|
+
no_tasks do
|
65
|
+
DEFAULT_INCEPTION_VOLUME_SIZE = 32 # Gb
|
66
|
+
|
67
|
+
def deploy_stage_1_choose_infrastructure_provider
|
68
|
+
header "Stage 1: Choose infrastructure"
|
69
|
+
unless settings[:fog_credentials]
|
70
|
+
choose_fog_provider
|
71
|
+
end
|
72
|
+
confirm "Using infrastructure provider #{settings.fog_credentials.provider}"
|
73
|
+
|
74
|
+
unless settings[:region_code]
|
75
|
+
choose_provider_region
|
76
|
+
end
|
77
|
+
if settings[:region_code]
|
78
|
+
confirm "Using #{settings.fog_credentials.provider} region #{settings.region_code}"
|
79
|
+
else
|
80
|
+
confirm "No specific region/data center for #{settings.fog_credentials.provider}"
|
81
|
+
end
|
82
|
+
end
|
83
|
+
|
84
|
+
def deploy_stage_2_bosh_configuration
|
85
|
+
header "Stage 2: BOSH configuration"
|
86
|
+
unless settings[:bosh_name]
|
87
|
+
provider, region = settings.bosh_provider, settings.region_code
|
88
|
+
if region
|
89
|
+
default_name = "microbosh-#{provider}-#{region}".gsub(/\W+/, '-')
|
90
|
+
else
|
91
|
+
default_name = "microbosh-#{provider}".gsub(/\W+/, '-')
|
92
|
+
end
|
93
|
+
bosh_name = hl.ask("Useful name for Micro BOSH? ") { |q| q.default = default_name }
|
94
|
+
settings[:bosh_name] = bosh_name
|
95
|
+
save_settings!
|
96
|
+
end
|
97
|
+
confirm "Micro BOSH will be named #{settings.bosh_name}"
|
98
|
+
|
99
|
+
unless settings[:bosh_username]
|
100
|
+
prompt_for_bosh_credentials
|
101
|
+
end
|
102
|
+
confirm "After BOSH is created, your username will be #{settings.bosh_username}"
|
103
|
+
|
104
|
+
unless settings[:bosh]
|
105
|
+
say "Defaulting to 16Gb persistent disk for BOSH"
|
106
|
+
password = settings.bosh_password # FIXME dual use of password?
|
107
|
+
settings[:bosh] = {}
|
108
|
+
settings[:bosh][:password] = password
|
109
|
+
settings[:bosh][:persistent_disk] = 16384
|
110
|
+
save_settings!
|
111
|
+
end
|
112
|
+
unless settings[:bosh]["ip_address"]
|
113
|
+
say "Acquiring IP address for micro BOSH..."
|
114
|
+
ip_address = acquire_ip_address
|
115
|
+
settings[:bosh]["ip_address"] = ip_address
|
116
|
+
end
|
117
|
+
unless settings[:bosh]["ip_address"]
|
118
|
+
error "IP address not available/provided currently"
|
119
|
+
else
|
120
|
+
confirm "Micro BOSH will be assigned IP address #{settings[:bosh]['ip_address']}"
|
121
|
+
end
|
122
|
+
save_settings!
|
123
|
+
|
124
|
+
unless settings[:bosh_security_group]
|
125
|
+
security_group_name = settings.bosh_name
|
126
|
+
create_security_group(security_group_name)
|
127
|
+
end
|
128
|
+
ports = settings.bosh_security_group.ports.values
|
129
|
+
confirm "Micro BOSH protected by security group " +
|
130
|
+
"named #{settings.bosh_security_group.name}, with ports #{ports}"
|
131
|
+
|
132
|
+
unless settings[:bosh_key_pair]
|
133
|
+
key_pair_name = settings.bosh_name
|
134
|
+
create_key_pair(key_pair_name)
|
135
|
+
end
|
136
|
+
confirm "Micro BOSH accessible via key pair named #{settings.bosh_key_pair.name}"
|
137
|
+
|
138
|
+
unless settings[:micro_bosh_stemcell_name]
|
139
|
+
settings[:micro_bosh_stemcell_name] = micro_bosh_stemcell_name
|
140
|
+
save_settings!
|
141
|
+
end
|
142
|
+
|
143
|
+
confirm "Micro BOSH will be created with stemcell #{settings.micro_bosh_stemcell_name}"
|
144
|
+
end
|
145
|
+
|
146
|
+
def deploy_stage_3_create_allocate_inception_vm
|
147
|
+
header "Stage 3: Create/Allocate the Inception VM"
|
148
|
+
unless settings["inception"] && settings["inception"]["host"]
|
149
|
+
hl.choose do |menu|
|
150
|
+
menu.prompt = "Create or specify an Inception VM: "
|
151
|
+
if aws? || openstack?
|
152
|
+
menu.choice("create new inception VM") do
|
153
|
+
aws? ? boot_aws_inception_vm : boot_openstack_inception_vm
|
154
|
+
end
|
155
|
+
end
|
156
|
+
menu.choice("use an existing Ubuntu server") do
|
157
|
+
settings["inception"] = {}
|
158
|
+
settings["inception"]["host"] = \
|
159
|
+
hl.ask("Host address (IP or domain) to inception VM? ")
|
160
|
+
settings["inception"]["username"] = \
|
161
|
+
hl.ask("Username that you have SSH access to? ") {|q| q.default = "ubuntu"}
|
162
|
+
end
|
163
|
+
menu.choice("use this server (must be ubuntu & on same network as bosh)") do
|
164
|
+
# dummy data for settings.inception
|
165
|
+
settings["inception"] = {}
|
166
|
+
settings["inception"]["username"] = `whoami`.strip
|
167
|
+
end
|
168
|
+
end
|
169
|
+
end
|
170
|
+
# If successfully validate inception VM, then save those settings.
|
171
|
+
save_settings!
|
172
|
+
|
173
|
+
if settings["inception"]["host"]
|
174
|
+
@server = Commander::RemoteServer.new(settings.inception.host)
|
175
|
+
confirm "Using inception VM #{settings.inception.username}@#{settings.inception.host}"
|
176
|
+
else
|
177
|
+
@server = Commander::LocalServer.new
|
178
|
+
confirm "Using this server as the inception VM"
|
179
|
+
end
|
180
|
+
unless settings["inception"]["validated"]
|
181
|
+
unless server.run(Bosh::Bootstrap::Stages::StageValidateInceptionVm.new(settings).commands)
|
182
|
+
error "Failed to complete Stage 3: Create/Allocate the Inception VM"
|
183
|
+
end
|
184
|
+
settings["inception"]["validated"] = true
|
185
|
+
end
|
186
|
+
# If successfully validate inception VM, then save those settings.
|
187
|
+
save_settings!
|
188
|
+
end
|
189
|
+
|
190
|
+
def deploy_stage_4_prepare_inception_vm
|
191
|
+
unless settings["inception"]["prepared"] && !settings["upgrade_deps"]
|
192
|
+
header "Stage 4: Preparing the Inception VM"
|
193
|
+
unless server.run(Bosh::Bootstrap::Stages::StagePrepareInceptionVm.new(settings).commands)
|
194
|
+
error "Failed to complete Stage 4: Preparing the Inception VM"
|
195
|
+
end
|
196
|
+
# Settings are updated by this stage
|
197
|
+
# it generates a salted password from settings.bosh.password
|
198
|
+
# and stores it in settings.bosh.salted_password
|
199
|
+
settings["inception"]["prepared"] = true
|
200
|
+
save_settings!
|
201
|
+
else
|
202
|
+
header "Stage 4: Preparing the Inception VM", :skipping => "Already prepared inception VM."
|
203
|
+
end
|
204
|
+
end
|
205
|
+
|
206
|
+
def deploy_stage_5_deploy_micro_bosh
|
207
|
+
header "Stage 5: Deploying micro BOSH"
|
208
|
+
unless server.run(Bosh::Bootstrap::Stages::MicroBoshDeploy.new(settings).commands)
|
209
|
+
error "Failed to complete Stage 5: Deploying micro BOSH"
|
210
|
+
end
|
211
|
+
|
212
|
+
confirm "Successfully built micro BOSH"
|
213
|
+
end
|
214
|
+
|
215
|
+
def deploy_stage_6_setup_new_bosh
|
216
|
+
# TODO change to a polling test of director being available
|
217
|
+
say "Pausing to wait for BOSH Director..."
|
218
|
+
sleep 5
|
219
|
+
|
220
|
+
header "Stage 6: Setup bosh"
|
221
|
+
unless server.run(Bosh::Bootstrap::Stages::SetupNewBosh.new(settings).commands)
|
222
|
+
error "Failed to complete Stage 6: Setup bosh"
|
223
|
+
end
|
224
|
+
|
225
|
+
say "Locally targeting and login to new BOSH..."
|
226
|
+
sh "bosh -u #{settings.bosh_username} -p #{settings.bosh_password} target #{settings.bosh.ip_address}"
|
227
|
+
sh "bosh login #{settings.bosh_username} #{settings.bosh_password}"
|
228
|
+
|
229
|
+
save_settings!
|
230
|
+
|
231
|
+
confirm "You are now targeting and logged in to your BOSH"
|
232
|
+
end
|
233
|
+
|
234
|
+
def delete_stage_1_target_inception_vm
|
235
|
+
header "Stage 1: Target inception VM to use to delete micro-bosh"
|
236
|
+
if settings["inception"]["host"]
|
237
|
+
@server = Commander::RemoteServer.new(settings.inception.host)
|
238
|
+
confirm "Using inception VM #{settings.inception.username}@#{settings.inception.host}"
|
239
|
+
else
|
240
|
+
@server = Commander::LocalServer.new
|
241
|
+
confirm "Using this server as the inception VM"
|
242
|
+
end
|
243
|
+
end
|
244
|
+
|
245
|
+
def delete_one_stage_2_delete_micro_bosh
|
246
|
+
header "Stage 2: Deleting micro BOSH"
|
247
|
+
unless server.run(Bosh::Bootstrap::Stages::MicroBoshDelete.new(settings).commands)
|
248
|
+
error "Failed to complete Stage 1: Delete micro BOSH"
|
249
|
+
end
|
250
|
+
save_settings!
|
251
|
+
end
|
252
|
+
|
253
|
+
def delete_all_stage_2_delete_micro_boshes
|
254
|
+
|
255
|
+
end
|
256
|
+
|
257
|
+
def delete_all_stage_3_delete_inception_vm
|
258
|
+
|
259
|
+
end
|
260
|
+
|
261
|
+
def run_ssh_command_or_open_tunnel(cmd)
|
262
|
+
unless settings[:inception]
|
263
|
+
say "No inception VM being used", :yellow
|
264
|
+
exit 0
|
265
|
+
end
|
266
|
+
unless host = settings.inception[:host]
|
267
|
+
exit "Inception VM has not finished launching; run to complete: #{self.class.banner_base} deploy"
|
268
|
+
end
|
269
|
+
username = 'vcap'
|
270
|
+
exit system Escape.shell_command(['ssh', "#{username}@#{host}", cmd].compact)
|
271
|
+
|
272
|
+
# TODO how to use the specific private_key_path as configured in settings
|
273
|
+
# _, private_key_path = local_ssh_key_paths
|
274
|
+
# exit system Escape.shell_command(['ssh', "-i #{private_key_path}", "#{username}@#{host}", cmd].compact)
|
275
|
+
#
|
276
|
+
# Currently this shows:
|
277
|
+
# Warning: Identity file /Users/drnic/.ssh/id_rsa not accessible: No such file or directory.
|
278
|
+
end
|
279
|
+
|
280
|
+
# Display header for a new section of the bootstrapper
|
281
|
+
def header(title, options={})
|
282
|
+
say "" # golden whitespace
|
283
|
+
if skipping = options[:skipping]
|
284
|
+
say "Skipping #{title}", [:yellow, :bold]
|
285
|
+
say skipping
|
286
|
+
else
|
287
|
+
say title, [:green, :bold]
|
288
|
+
end
|
289
|
+
say "" # more golden whitespace
|
290
|
+
end
|
291
|
+
|
292
|
+
def error(message)
|
293
|
+
say message, :red
|
294
|
+
exit 1
|
295
|
+
end
|
296
|
+
|
297
|
+
def confirm(message)
|
298
|
+
say "Confirming: #{message}", green
|
299
|
+
say "" # bonus golden whitespace
|
300
|
+
end
|
301
|
+
|
302
|
+
def load_deploy_options
|
303
|
+
settings["fog_path"] = File.expand_path(options[:fog] || "~/.fog")
|
304
|
+
|
305
|
+
settings["bosh_git_source"] = options[:"edge-deployer"] # use bosh git repo instead of rubygems
|
306
|
+
|
307
|
+
# determine which micro-bosh stemcell to download/create
|
308
|
+
if options[:"latest-stemcell"]
|
309
|
+
settings["micro_bosh_stemcell_type"] = "latest"
|
310
|
+
settings["micro_bosh_stemcell_name"] = nil # force name to be refetched
|
311
|
+
elsif options[:"edge-stemcell"]
|
312
|
+
settings["micro_bosh_stemcell_type"] = "custom"
|
313
|
+
settings["micro_bosh_stemcell_name"] = "custom"
|
314
|
+
else
|
315
|
+
# may have already been set from previous deploy run
|
316
|
+
settings["micro_bosh_stemcell_type"] ||= "stable"
|
317
|
+
end
|
318
|
+
|
319
|
+
# once a stemcell is downloaded or created; these fields above should
|
320
|
+
# be uploaded with values such as:
|
321
|
+
# -> settings["micro_bosh_stemcell_name"] = "micro-bosh-stemcell-aws-0.8.1.tgz"
|
322
|
+
|
323
|
+
if options["private-key"]
|
324
|
+
private_key_path = File.expand_path(options["private-key"])
|
325
|
+
unless File.exists?(private_key_path)
|
326
|
+
error "Cannot find a file at #{private_key_path}"
|
327
|
+
end
|
328
|
+
public_key_path = "#{private_key_path}.pub"
|
329
|
+
unless File.exists?(public_key_path)
|
330
|
+
error "Cannot find a file at #{public_key_path}"
|
331
|
+
end
|
332
|
+
|
333
|
+
settings["local"] ||= {}
|
334
|
+
settings["local"]["private_key_path"] = private_key_path
|
335
|
+
settings["local"]["public_key_path"] = public_key_path
|
336
|
+
end
|
337
|
+
|
338
|
+
if options["upgrade-deps"]
|
339
|
+
settings["upgrade_deps"] = options["upgrade-deps"]
|
340
|
+
else
|
341
|
+
settings.delete("upgrade_deps")
|
342
|
+
end
|
343
|
+
save_settings!
|
344
|
+
end
|
345
|
+
|
346
|
+
# Displays a prompt for known IaaS that are configured
|
347
|
+
# within .fog config file.
|
348
|
+
#
|
349
|
+
# For example:
|
350
|
+
#
|
351
|
+
# 1. AWS (default)
|
352
|
+
# 2. AWS (bosh)
|
353
|
+
# 3. Alternate credentials
|
354
|
+
# Choose infrastructure: 1
|
355
|
+
#
|
356
|
+
# If .fog config only contains one provider, do not prompt.
|
357
|
+
#
|
358
|
+
# fog config file looks like:
|
359
|
+
# :default:
|
360
|
+
# :aws_access_key_id: PERSONAL_ACCESS_KEY
|
361
|
+
# :aws_secret_access_key: PERSONAL_SECRET
|
362
|
+
# :bosh:
|
363
|
+
# :aws_access_key_id: SPECIAL_IAM_ACCESS_KEY
|
364
|
+
# :aws_secret_access_key: SPECIAL_IAM_SECRET_KEY
|
365
|
+
#
|
366
|
+
# Convert this into:
|
367
|
+
# { "AWS (default)" => {:aws_access_key_id => ...}, "AWS (bosh)" => {...} }
|
368
|
+
#
|
369
|
+
# Then display options to user to choose.
|
370
|
+
#
|
371
|
+
# Currently detects following fog providers:
|
372
|
+
# * AWS
|
373
|
+
# * OpenStack
|
374
|
+
#
|
375
|
+
# If "Alternate credentials" is selected, then user is prompted for fog
|
376
|
+
# credentials:
|
377
|
+
# * provider?
|
378
|
+
# * access keys?
|
379
|
+
# * API URI or region?
|
380
|
+
#
|
381
|
+
# At the end, settings.fog_credentials contains the credentials for target IaaS
|
382
|
+
# and :provider key for the IaaS name.
|
383
|
+
#
|
384
|
+
# {:provider=>"AWS",
|
385
|
+
# :aws_access_key_id=>"PERSONAL_ACCESS_KEY",
|
386
|
+
# :aws_secret_access_key=>"PERSONAL_SECRET"}
|
387
|
+
#
|
388
|
+
# settings.fog_credentials.provider is the provider name
|
389
|
+
# settings.bosh_provider is the BOSH name for the provider (aws,vsphere,openstack)
|
390
|
+
# so as to local stemcells (see +micro_bosh_stemcell_name+)
|
391
|
+
def choose_fog_provider
|
392
|
+
@fog_providers = {}
|
393
|
+
# Prepare menu options:
|
394
|
+
# each provider/profile name gets a menu choice option
|
395
|
+
fog_config.inject({}) do |iaas_options, fog_profile|
|
396
|
+
profile_name, profile = fog_profile
|
397
|
+
if profile[:aws_access_key_id]
|
398
|
+
# TODO does fog have inbuilt detection algorithm?
|
399
|
+
@fog_providers["AWS (#{profile_name})"] = {
|
400
|
+
"provider" => "AWS",
|
401
|
+
"aws_access_key_id" => profile[:aws_access_key_id],
|
402
|
+
"aws_secret_access_key" => profile[:aws_secret_access_key]
|
403
|
+
}
|
404
|
+
end
|
405
|
+
if profile[:openstack_username]
|
406
|
+
# TODO does fog have inbuilt detection algorithm?
|
407
|
+
@fog_providers["OpenStack (#{profile_name})"] = {
|
408
|
+
"provider" => "OpenStack",
|
409
|
+
"openstack_username" => profile[:openstack_username],
|
410
|
+
"openstack_api_key" => profile[:openstack_api_key],
|
411
|
+
"openstack_tenant" => profile[:openstack_tenant],
|
412
|
+
"openstack_auth_url" => profile[:openstack_auth_url]
|
413
|
+
}
|
414
|
+
end
|
415
|
+
end
|
416
|
+
# Display menu
|
417
|
+
# Include "Alternate credentials" as the last option
|
418
|
+
if @fog_providers.keys.size > 0
|
419
|
+
hl.choose do |menu|
|
420
|
+
menu.prompt = "Choose infrastructure: "
|
421
|
+
@fog_providers.each do |label, credentials|
|
422
|
+
menu.choice(label) { @fog_credentials = credentials }
|
423
|
+
end
|
424
|
+
menu.choice("Alternate credentials") { prompt_for_alternate_fog_credentials }
|
425
|
+
end
|
426
|
+
else
|
427
|
+
prompt_for_alternate_fog_credentials
|
428
|
+
end
|
429
|
+
settings[:fog_credentials] = {}
|
430
|
+
@fog_credentials.each do |key, value|
|
431
|
+
settings[:fog_credentials][key] = value
|
432
|
+
end
|
433
|
+
setup_bosh_cloud_properties
|
434
|
+
settings[:bosh_resources_cloud_properties] = bosh_resources_cloud_properties
|
435
|
+
settings[:bosh_provider] = settings.bosh_cloud_properties.keys.first # aws, vsphere...
|
436
|
+
save_settings!
|
437
|
+
end
|
438
|
+
|
439
|
+
# If no .fog file is found, or if user chooses "Alternate credentials",
|
440
|
+
# then this method prompts the user:
|
441
|
+
# * provider?
|
442
|
+
# * access keys?
|
443
|
+
# * API URI or region?
|
444
|
+
#
|
445
|
+
# Populates +@fog_credentials+ with a Hash that includes :provider key
|
446
|
+
# For example:
|
447
|
+
# {
|
448
|
+
# :provider => "AWS",
|
449
|
+
# :aws_access_key_id => ACCESS_KEY,
|
450
|
+
# :aws_secret_access_key => SECRET_KEY
|
451
|
+
# }
|
452
|
+
def prompt_for_alternate_fog_credentials
|
453
|
+
say "" # glorious whitespace
|
454
|
+
creds = {}
|
455
|
+
hl.choose do |menu|
|
456
|
+
menu.prompt = "Choose infrastructure: "
|
457
|
+
menu.choice("AWS") do
|
458
|
+
creds[:provider] = "AWS"
|
459
|
+
creds[:aws_access_key_id] = hl.ask("Access key: ")
|
460
|
+
creds[:aws_secret_access_key] = hl.ask("Secret key: ")
|
461
|
+
end
|
462
|
+
menu.choice("OpenStack") do
|
463
|
+
creds[:provider] = "OpenStack"
|
464
|
+
creds[:openstack_username] = hl.ask("Username: ")
|
465
|
+
creds[:openstack_api_key] = hl.ask("API key: ")
|
466
|
+
creds[:openstack_tenant] = hl.ask("Tenant: ")
|
467
|
+
creds[:openstack_auth_url] = hl.ask("Authorization URL: ")
|
468
|
+
end
|
469
|
+
end
|
470
|
+
@fog_credentials = creds
|
471
|
+
end
|
472
|
+
|
473
|
+
def setup_bosh_cloud_properties
|
474
|
+
if aws?
|
475
|
+
settings[:bosh_cloud_properties] = {}
|
476
|
+
settings[:bosh_cloud_properties][:aws] = {}
|
477
|
+
props = settings[:bosh_cloud_properties][:aws]
|
478
|
+
props[:access_key_id] = settings.fog_credentials.aws_access_key_id
|
479
|
+
props[:secret_access_key] = settings.fog_credentials.aws_secret_access_key
|
480
|
+
# props[:ec2_endpoint] = "ec2.REGION.amazonaws.com" - via +choose_aws_region+
|
481
|
+
# props[:region] = REGION - via +choose_aws_region+
|
482
|
+
# props[:default_key_name] = "microbosh" - via +create_aws_key_pair+
|
483
|
+
# props[:ec2_private_key] = "/home/vcap/.ssh/microbosh.pem" - via +create_aws_key_pair+
|
484
|
+
# props[:default_security_groups] = ["microbosh"], - via +create_aws_security_group+
|
485
|
+
elsif openstack?
|
486
|
+
settings[:bosh_cloud_properties] = {}
|
487
|
+
settings[:bosh_cloud_properties][:openstack] = {}
|
488
|
+
props = settings[:bosh_cloud_properties][:openstack]
|
489
|
+
props[:username] = settings.fog_credentials.openstack_username
|
490
|
+
props[:api_key] = settings.fog_credentials.openstack_api_key
|
491
|
+
props[:tenant] = settings.fog_credentials.openstack_tenant
|
492
|
+
props[:auth_url] = settings.fog_credentials.openstack_auth_url
|
493
|
+
# props[:default_key_name] = "microbosh" - via +create_openstack_key_pair+
|
494
|
+
# props[:private_key] = "/home/vcap/.ssh/microbosh.pem" - via +create_openstack_key_pair+
|
495
|
+
# props[:default_security_groups] = ["microbosh"], - via +create_openstack_security_group+
|
496
|
+
else
|
497
|
+
raise "implement #bosh_cloud_properties for #{settings.fog_credentials.provider}"
|
498
|
+
end
|
499
|
+
end
|
500
|
+
|
501
|
+
def bosh_resources_cloud_properties
|
502
|
+
if aws?
|
503
|
+
{"instance_type" => "m1.medium"}
|
504
|
+
elsif openstack?
|
505
|
+
# TODO: Ask for instance type
|
506
|
+
{"instance_type" => "m1.medium"}
|
507
|
+
else
|
508
|
+
raise "implement #bosh_resources_cloud_properties for #{settings.fog_credentials.provider}"
|
509
|
+
end
|
510
|
+
end
|
511
|
+
|
512
|
+
# Ask user to provide region information (URI)
|
513
|
+
# or choose from a known list of regions (e.g. AWS)
|
514
|
+
# Return true if region selected (@region_code is set)
|
515
|
+
# Else return false
|
516
|
+
def choose_provider_region
|
517
|
+
if aws?
|
518
|
+
choose_aws_region
|
519
|
+
else
|
520
|
+
settings["region_code"] = nil
|
521
|
+
false
|
522
|
+
end
|
523
|
+
end
|
524
|
+
|
525
|
+
def choose_aws_region
|
526
|
+
aws_regions = provider.region_labels
|
527
|
+
default_aws_region = provider.default_region_label
|
528
|
+
|
529
|
+
hl.choose do |menu|
|
530
|
+
menu.prompt = "Choose AWS region (default: #{default_aws_region}): "
|
531
|
+
aws_regions.each do |region|
|
532
|
+
menu.choice(region) do
|
533
|
+
settings["region_code"] = region
|
534
|
+
settings["fog_credentials"]["region"] = region
|
535
|
+
settings["bosh_cloud_properties"]["aws"]["region"] = region
|
536
|
+
settings["bosh_cloud_properties"]["aws"]["ec2_endpoint"] = "ec2.#{region}.amazonaws.com"
|
537
|
+
save_settings!
|
538
|
+
end
|
539
|
+
menu.default = default_aws_region
|
540
|
+
end
|
541
|
+
end
|
542
|
+
reset_fog_compute
|
543
|
+
true
|
544
|
+
end
|
545
|
+
|
546
|
+
# Creates a security group.
|
547
|
+
# Also sets up the bosh_cloud_properties for the remote server
|
548
|
+
#
|
549
|
+
# Adds settings:
|
550
|
+
# * bosh_security_group.name
|
551
|
+
# * bosh_security_group.ports
|
552
|
+
# * bosh_cloud_properties.<bosh_provider>.default_security_groups
|
553
|
+
def create_security_group(security_group_name)
|
554
|
+
ports = {
|
555
|
+
ssh_access: 22,
|
556
|
+
nats_server: 4222,
|
557
|
+
message_bus: 6868,
|
558
|
+
blobstore: 25250,
|
559
|
+
bosh_director: 25555
|
560
|
+
}
|
561
|
+
if aws?
|
562
|
+
ports[:aws_registry] = 25777
|
563
|
+
elsif openstack?
|
564
|
+
ports[:openstack_registry] = 25889
|
565
|
+
end
|
566
|
+
|
567
|
+
provider.create_security_group(security_group_name, "microbosh", ports)
|
568
|
+
|
569
|
+
settings["bosh_cloud_properties"][provider_name]["default_security_groups"] = [security_group_name]
|
570
|
+
settings["bosh_security_group"] = {}
|
571
|
+
settings["bosh_security_group"]["name"] = security_group_name
|
572
|
+
settings["bosh_security_group"]["ports"] = {}
|
573
|
+
ports.each { |name, port| settings["bosh_security_group"]["ports"][name.to_s] = port }
|
574
|
+
save_settings!
|
575
|
+
end
|
576
|
+
|
577
|
+
# Creates a key pair.
|
578
|
+
def create_key_pair(key_pair_name)
|
579
|
+
if aws?
|
580
|
+
create_aws_key_pair(key_pair_name)
|
581
|
+
elsif openstack?
|
582
|
+
create_openstack_key_pair(key_pair_name)
|
583
|
+
else
|
584
|
+
raise "implement #create_key_pair for #{settings.fog_credentials.provider}"
|
585
|
+
end
|
586
|
+
end
|
587
|
+
|
588
|
+
# Creates an AWS key pair, and stores the private key
|
589
|
+
# in settings manifest.
|
590
|
+
# Also sets up the bosh_cloud_properties for the remote server
|
591
|
+
# to have the .pem key installed.
|
592
|
+
#
|
593
|
+
# Adds settings:
|
594
|
+
# * bosh_key_pair.name
|
595
|
+
# * bosh_key_pair.private_key
|
596
|
+
# * bosh_key_pair.fingerprint
|
597
|
+
# * bosh_cloud_properties.aws.default_key_name
|
598
|
+
# * bosh_cloud_properties.aws.ec2_private_key
|
599
|
+
def create_aws_key_pair(key_pair_name)
|
600
|
+
unless fog_compute.key_pairs.get(key_pair_name)
|
601
|
+
say "creating key pair #{key_pair_name}..."
|
602
|
+
kp = fog_compute.key_pairs.create(:name => key_pair_name)
|
603
|
+
settings[:bosh_key_pair] = {}
|
604
|
+
settings[:bosh_key_pair][:name] = key_pair_name
|
605
|
+
settings[:bosh_key_pair][:private_key] = kp.private_key
|
606
|
+
settings[:bosh_key_pair][:fingerprint] = kp.fingerprint
|
607
|
+
settings["bosh_cloud_properties"]["aws"]["default_key_name"] = key_pair_name
|
608
|
+
settings["bosh_cloud_properties"]["aws"]["ec2_private_key"] = "/home/vcap/.ssh/#{key_pair_name}.pem"
|
609
|
+
save_settings!
|
610
|
+
else
|
611
|
+
error "AWS key pair '#{key_pair_name}' already exists. Rename BOSH or delete old key pair manually and re-run CLI."
|
612
|
+
end
|
613
|
+
end
|
614
|
+
|
615
|
+
# Creates an OpenStack key pair, and stores the private key
|
616
|
+
# in settings manifest.
|
617
|
+
# Also sets up the bosh_cloud_properties for the remote server
|
618
|
+
# to have the .pem key installed.
|
619
|
+
#
|
620
|
+
# Adds settings:
|
621
|
+
# * bosh_key_pair.name
|
622
|
+
# * bosh_key_pair.private_key
|
623
|
+
# * bosh_key_pair.fingerprint
|
624
|
+
# * bosh_cloud_properties.openstack.default_key_name
|
625
|
+
# * bosh_cloud_properties.openstack.ec2_private_key
|
626
|
+
def create_openstack_key_pair(key_pair_name)
|
627
|
+
unless fog_compute.key_pairs.get(key_pair_name)
|
628
|
+
say "creating key pair #{key_pair_name}..."
|
629
|
+
kp = fog_compute.key_pairs.create(:name => key_pair_name)
|
630
|
+
settings[:bosh_key_pair] = {}
|
631
|
+
settings[:bosh_key_pair][:name] = key_pair_name
|
632
|
+
settings[:bosh_key_pair][:private_key] = kp.private_key
|
633
|
+
settings[:bosh_key_pair][:fingerprint] = kp.fingerprint
|
634
|
+
settings["bosh_cloud_properties"]["openstack"]["default_key_name"] = key_pair_name
|
635
|
+
settings["bosh_cloud_properties"]["openstack"]["private_key"] = "/home/vcap/.ssh/#{key_pair_name}.pem"
|
636
|
+
save_settings!
|
637
|
+
else
|
638
|
+
error "OpenStack key pair '#{key_pair_name}' already exists. Rename BOSH or delete old key pair manually and re-run CLI."
|
639
|
+
end
|
640
|
+
end
|
641
|
+
|
642
|
+
# Provisions an AWS m1.small VM as the inception VM
|
643
|
+
# Updates settings.inception.host/username
|
644
|
+
#
|
645
|
+
# NOTE: if any stage fails, when the CLI is re-run
|
646
|
+
# and "create new server" is selected again, the process should
|
647
|
+
# complete
|
648
|
+
#
|
649
|
+
# Assumes that local CLI user has public/private keys at ~/.ssh/id_rsa.pub
|
650
|
+
def boot_aws_inception_vm
|
651
|
+
say "" # glowing whitespace
|
652
|
+
|
653
|
+
public_key_path, private_key_path = local_ssh_key_paths
|
654
|
+
unless settings["inception"] && settings["inception"]["server_id"]
|
655
|
+
username = "ubuntu"
|
656
|
+
size = "m1.small"
|
657
|
+
say "Provisioning #{size} for inception VM..."
|
658
|
+
server = fog_compute.servers.bootstrap({
|
659
|
+
:public_key_path => public_key_path,
|
660
|
+
:private_key_path => private_key_path,
|
661
|
+
:flavor_id => size,
|
662
|
+
:bits => 64,
|
663
|
+
:username => "ubuntu"
|
664
|
+
})
|
665
|
+
unless server
|
666
|
+
error "Something mysteriously cloudy happened and fog could not provision a VM. Please check your limits."
|
667
|
+
end
|
668
|
+
|
669
|
+
settings["inception"] = {}
|
670
|
+
settings["inception"]["server_id"] = server.id
|
671
|
+
settings["inception"]["username"] = username
|
672
|
+
save_settings!
|
673
|
+
end
|
674
|
+
|
675
|
+
server ||= fog_compute.servers.get(settings["inception"]["server_id"])
|
676
|
+
|
677
|
+
unless settings["inception"]["ip_address"]
|
678
|
+
say "Provisioning IP address for inception VM..."
|
679
|
+
ip_address = acquire_ip_address
|
680
|
+
associate_ip_address_with_server(ip_address, server)
|
681
|
+
host = server.dns_name
|
682
|
+
|
683
|
+
settings["inception"]["ip_address"] = ip_address
|
684
|
+
save_settings!
|
685
|
+
end
|
686
|
+
|
687
|
+
unless settings["inception"]["disk_size"]
|
688
|
+
disk_size = DEFAULT_INCEPTION_VOLUME_SIZE # Gb
|
689
|
+
device = "/dev/sdi"
|
690
|
+
provision_and_mount_volume(server, disk_size, device)
|
691
|
+
|
692
|
+
settings["inception"]["disk_size"] = disk_size
|
693
|
+
settings["inception"]["disk_device"] = device
|
694
|
+
save_settings!
|
695
|
+
end
|
696
|
+
|
697
|
+
# settings["inception"]["host"] is used externally to determine
|
698
|
+
# if an inception VM has been assigned already; so we leave it
|
699
|
+
# until last in this method to set this setting.
|
700
|
+
# This way we can always rerun the CLI and rerun this method
|
701
|
+
# and idempotently get an inception VM
|
702
|
+
unless settings["inception"]["host"]
|
703
|
+
settings["inception"]["host"] = server.dns_name
|
704
|
+
save_settings!
|
705
|
+
end
|
706
|
+
|
707
|
+
confirm "Inception VM has been created"
|
708
|
+
display_inception_ssh_access
|
709
|
+
end
|
710
|
+
|
711
|
+
# Provisions an OpenStack m1.small VM as the inception VM
|
712
|
+
# Updates settings.inception.host/username
|
713
|
+
#
|
714
|
+
# NOTE: if any stage fails, when the CLI is re-run
|
715
|
+
# and "create new server" is selected again, the process should
|
716
|
+
# complete
|
717
|
+
#
|
718
|
+
# Assumes that local CLI user has public/private keys at ~/.ssh/id_rsa.pub
|
719
|
+
def boot_openstack_inception_vm
|
720
|
+
say "" # glowing whitespace
|
721
|
+
|
722
|
+
public_key_path, private_key_path = local_ssh_key_paths
|
723
|
+
|
724
|
+
# make sure we've a fog key pair
|
725
|
+
key_pair_name = Fog.respond_to?(:credential) && Fog.credential || :default
|
726
|
+
unless key_pair = fog_compute.key_pairs.get("fog_#{key_pair_name}")
|
727
|
+
say "creating key pair fog_#{key_pair_name}..."
|
728
|
+
public_key = File.open(public_key_path, 'rb') { |f| f.read }
|
729
|
+
key_pair = fog_compute.key_pairs.create(
|
730
|
+
:name => "fog_#{key_pair_name}",
|
731
|
+
:public_key => public_key
|
732
|
+
)
|
733
|
+
end
|
734
|
+
confirm "Using key pair #{key_pair.name} for Inception VM"
|
735
|
+
|
736
|
+
# make sure port 22 is open in the default security group
|
737
|
+
security_group = fog_compute.security_groups.find { |sg| sg.name == 'default' }
|
738
|
+
authorized = security_group.rules.detect do |ip_permission|
|
739
|
+
ip_permission['ip_range'].first && ip_permission['ip_range']['cidr'] == '0.0.0.0/0' &&
|
740
|
+
ip_permission['from_port'] == 22 &&
|
741
|
+
ip_permission['ip_protocol'] == 'tcp' &&
|
742
|
+
ip_permission['to_port'] == 22
|
743
|
+
end
|
744
|
+
unless authorized
|
745
|
+
security_group.create_security_group_rule(22, 22)
|
746
|
+
end
|
747
|
+
confirm "Inception VM port 22 open"
|
748
|
+
|
749
|
+
unless settings["inception"] && settings["inception"]["server_id"]
|
750
|
+
username = "ubuntu"
|
751
|
+
say "Provisioning server for inception VM..."
|
752
|
+
settings["inception"] = {}
|
753
|
+
|
754
|
+
# Select OpenStack flavor
|
755
|
+
unless settings["inception"]["flavor_id"]
|
756
|
+
say ""
|
757
|
+
hl.choose do |menu|
|
758
|
+
menu.prompt = "Choose OpenStack flavor: "
|
759
|
+
fog_compute.flavors.each do |flavor|
|
760
|
+
menu.choice(flavor.name) do
|
761
|
+
settings["inception"]["flavor_id"] = flavor.id
|
762
|
+
save_settings!
|
763
|
+
end
|
764
|
+
end
|
765
|
+
end
|
766
|
+
end
|
767
|
+
|
768
|
+
# Select OpenStack image
|
769
|
+
unless settings["inception"]["image_id"]
|
770
|
+
say ""
|
771
|
+
hl.choose do |menu|
|
772
|
+
menu.prompt = "Choose OpenStack image (Ubuntu): "
|
773
|
+
fog_compute.images.each do |image|
|
774
|
+
menu.choice(image.name) do
|
775
|
+
settings["inception"]["image_id"] = image.id
|
776
|
+
save_settings!
|
777
|
+
end
|
778
|
+
end
|
779
|
+
end
|
780
|
+
end
|
781
|
+
|
782
|
+
# Boot OpenStack server
|
783
|
+
server = fog_compute.servers.create(
|
784
|
+
:name => "Inception VM",
|
785
|
+
:key_name => key_pair.name,
|
786
|
+
:public_key_path => public_key_path,
|
787
|
+
:private_key_path => private_key_path,
|
788
|
+
:flavor_ref => settings["inception"]["flavor_id"],
|
789
|
+
:image_ref => settings["inception"]["image_id"],
|
790
|
+
:username => username
|
791
|
+
)
|
792
|
+
unless server
|
793
|
+
error "Something mysteriously cloudy happened and fog could not provision a VM. Please check your limits."
|
794
|
+
end
|
795
|
+
server.wait_for { ready? }
|
796
|
+
|
797
|
+
settings["inception"]["server_id"] = server.id
|
798
|
+
settings["inception"]["username"] = username
|
799
|
+
save_settings!
|
800
|
+
end
|
801
|
+
|
802
|
+
server ||= fog_compute.servers.get(settings["inception"]["server_id"])
|
803
|
+
|
804
|
+
unless settings["inception"]["ip_address"]
|
805
|
+
say "Provisioning IP address for inception VM..."
|
806
|
+
ip_address = acquire_ip_address
|
807
|
+
associate_ip_address_with_server(ip_address, server)
|
808
|
+
|
809
|
+
settings["inception"]["ip_address"] = ip_address
|
810
|
+
save_settings!
|
811
|
+
end
|
812
|
+
|
813
|
+
unless settings["inception"]["disk_size"]
|
814
|
+
disk_size = DEFAULT_INCEPTION_VOLUME_SIZE # Gb
|
815
|
+
device = "/dev/vdc"
|
816
|
+
provision_and_mount_volume(server, disk_size, device)
|
817
|
+
|
818
|
+
settings["inception"]["disk_size"] = disk_size
|
819
|
+
settings["inception"]["disk_device"] = device
|
820
|
+
save_settings!
|
821
|
+
|
822
|
+
# TODO use provision_and_mount_volume
|
823
|
+
|
824
|
+
disk_size = 16 # Gb
|
825
|
+
va = fog_compute.get_server_volumes(server.id).body['volumeAttachments']
|
826
|
+
unless vol = va.find { |v| v["device"] == "/dev/vdc" }
|
827
|
+
say "Provisioning #{disk_size}Gb persistent disk for inception VM..."
|
828
|
+
volume = fog_compute.volumes.create(:name => "Inception Disk",
|
829
|
+
:description => "",
|
830
|
+
:size => disk_size,
|
831
|
+
:availability_zone => server.availability_zone)
|
832
|
+
volume.wait_for { volume.status == 'available' }
|
833
|
+
volume.attach(server.id, "/dev/vdc")
|
834
|
+
volume.wait_for { volume.status == 'in-use' }
|
835
|
+
end
|
836
|
+
|
837
|
+
# Format and mount the volume
|
838
|
+
# TODO: Hack
|
839
|
+
unless server.public_ip_address
|
840
|
+
server.addresses["public"] = [settings["inception"]["ip_address"]]
|
841
|
+
end
|
842
|
+
unless server.public_key_path
|
843
|
+
server.public_key_path = public_key_path
|
844
|
+
end
|
845
|
+
unless server.private_key_path
|
846
|
+
server.private_key_path = private_key_path
|
847
|
+
end
|
848
|
+
server.username = settings["inception"]["username"]
|
849
|
+
server.sshable?
|
850
|
+
|
851
|
+
say "Mounting persistent disk as volume on inception VM..."
|
852
|
+
# TODO if any of these ssh calls fail; retry
|
853
|
+
server.ssh(['sudo mkfs.ext4 /dev/vdc -F'])
|
854
|
+
server.ssh(['sudo mkdir -p /var/vcap/store'])
|
855
|
+
server.ssh(['sudo mount /dev/vdc /var/vcap/store'])
|
856
|
+
|
857
|
+
settings["inception"]["disk_size"] = disk_size
|
858
|
+
save_settings!
|
859
|
+
end
|
860
|
+
|
861
|
+
# settings["inception"]["host"] is used externally to determine
|
862
|
+
# if an inception VM has been assigned already; so we leave it
|
863
|
+
# until last in this method to set this setting.
|
864
|
+
# This way we can always rerun the CLI and rerun this method
|
865
|
+
# and idempotently get an inception VM
|
866
|
+
unless settings["inception"]["host"]
|
867
|
+
settings["inception"]["host"] = settings["inception"]["ip_address"]
|
868
|
+
save_settings!
|
869
|
+
end
|
870
|
+
|
871
|
+
confirm "Inception VM has been created"
|
872
|
+
display_inception_ssh_access
|
873
|
+
end
|
874
|
+
|
875
|
+
# Provision or provide an IP address to use
|
876
|
+
# For AWS, it will dynamically provision an elastic IP
|
877
|
+
# For OpenStack, it will dynamically provision a floating IP
|
878
|
+
def acquire_ip_address
|
879
|
+
unless public_ip = provider.provision_public_ip_address
|
880
|
+
say "Unable to acquire a public IP. Please check your account for capacity or service issues.".red
|
881
|
+
exit 1
|
882
|
+
end
|
883
|
+
public_ip
|
884
|
+
end
|
885
|
+
|
886
|
+
def associate_ip_address_with_server(ip_address, server)
|
887
|
+
address = fog_compute.addresses.get(ip_address)
|
888
|
+
address.server = server
|
889
|
+
server.reload
|
890
|
+
end
|
891
|
+
|
892
|
+
# Provision a volume for a specific device (unless already provisioned)
|
893
|
+
# Request that the +server+ mount the volume at the +device+ location.
|
894
|
+
#
|
895
|
+
# Requires that we can SSH into +server+.
|
896
|
+
def provision_and_mount_volume(server, disk_size, device)
|
897
|
+
unless volume = server.volumes.all.find {|v| v.device == device}
|
898
|
+
say "Provisioning #{disk_size}Gb persistent disk for inception VM..."
|
899
|
+
volume = fog_compute.volumes.create(
|
900
|
+
size: disk_size,
|
901
|
+
name: "Inception Disk",
|
902
|
+
description: '',
|
903
|
+
device: "/dev/sdi",
|
904
|
+
availability_zone: server.availability_zone)
|
905
|
+
# TODO: the following works in fog 1.9.0+ (but which has a bug in bootstrap)
|
906
|
+
# https://github.com/fog/fog/issues/1516
|
907
|
+
#
|
908
|
+
# volume.wait_for { volume.status == 'available' }
|
909
|
+
# volume.attach(server.id, "/dev/vdc")
|
910
|
+
# volume.wait_for { volume.status == 'in-use' }
|
911
|
+
#
|
912
|
+
# Instead, using:
|
913
|
+
volume.server = server
|
914
|
+
end
|
915
|
+
|
916
|
+
# Format and mount the volume
|
917
|
+
say "Mounting persistent disk as volume on inception VM..."
|
918
|
+
disk_mounted = false
|
919
|
+
until disk_mounted
|
920
|
+
begin
|
921
|
+
# TODO catch Errno::ETIMEDOUT and re-run ssh commands
|
922
|
+
server.ssh(["sudo mkfs.ext4 #{device} -F"])
|
923
|
+
server.ssh(["sudo mkdir -p /var/vcap/store"])
|
924
|
+
server.ssh(["sudo mount #{device} /var/vcap/store"])
|
925
|
+
disk_mounted = true
|
926
|
+
rescue Errno::ETIMEDOUT => e
|
927
|
+
say "Timeout error/warning mounting volume, retrying...", yellow
|
928
|
+
end
|
929
|
+
end
|
930
|
+
end
|
931
|
+
|
932
|
+
def display_inception_ssh_access
|
933
|
+
_, private_key_path = local_ssh_key_paths
|
934
|
+
say "SSH access: ssh -i #{private_key_path} #{settings["inception"]["username"]}@#{settings["inception"]["host"]}"
|
935
|
+
end
|
936
|
+
|
937
|
+
# Discover/create local passphrase-less SSH keys to allow
|
938
|
+
# communication with Inception VM
|
939
|
+
#
|
940
|
+
# Returns [public_key_path, private_key_path]
|
941
|
+
def local_ssh_key_paths
|
942
|
+
unless settings["local"] && settings["local"]["private_key_path"]
|
943
|
+
settings["local"] = {}
|
944
|
+
public_key_path = File.expand_path("~/.ssh/id_rsa.pub")
|
945
|
+
private_key_path = File.expand_path("~/.ssh/id_rsa")
|
946
|
+
raise "Please create public keys at ~/.ssh/id_rsa.pub or use --private-key flag" unless File.exists?(public_key_path)
|
947
|
+
|
948
|
+
settings["local"]["public_key_path"] = public_key_path
|
949
|
+
settings["local"]["private_key_path"] = private_key_path
|
950
|
+
save_settings!
|
951
|
+
end
|
952
|
+
[settings.local.public_key_path, settings.local.private_key_path]
|
953
|
+
end
|
954
|
+
|
955
|
+
def aws?
|
956
|
+
settings.fog_credentials.provider == "AWS"
|
957
|
+
end
|
958
|
+
|
959
|
+
def openstack?
|
960
|
+
settings.fog_credentials.provider == "OpenStack"
|
961
|
+
end
|
962
|
+
|
963
|
+
def prompt_for_bosh_credentials
|
964
|
+
prompt = hl
|
965
|
+
say "Please enter a user/password for the BOSH that will be created."
|
966
|
+
settings[:bosh_username] = prompt.ask("BOSH username: ") { |q| q.default = `whoami`.strip }
|
967
|
+
settings[:bosh_password] = prompt.ask("BOSH password: ") { |q| q.echo = "x" }
|
968
|
+
save_settings!
|
969
|
+
end
|
970
|
+
|
971
|
+
# Returns the latest micro-bosh stemcell
|
972
|
+
# for the target provider (aws, vsphere, openstack)
|
973
|
+
def micro_bosh_stemcell_name
|
974
|
+
@micro_bosh_stemcell_name ||= begin
|
975
|
+
provider = settings.bosh_provider.downcase # aws, vsphere, openstack
|
976
|
+
stemcell_filter_tags = ['micro', provider]
|
977
|
+
if openstack?
|
978
|
+
# FIXME remove this if when openstack has its first stable
|
979
|
+
else
|
980
|
+
if settings["micro_bosh_stemcell_type"] == "stable"
|
981
|
+
stemcell_filter_tags << "stable" # latest stable micro-bosh stemcell by default
|
982
|
+
end
|
983
|
+
end
|
984
|
+
tags = stemcell_filter_tags.join(",")
|
985
|
+
bosh_stemcells_cmd = "bosh public stemcells --tags #{tags}"
|
986
|
+
say "Locating micro-bosh stemcell, running '#{bosh_stemcells_cmd}'..."
|
987
|
+
#
|
988
|
+
# The +bosh_stemcells_cmd+ has an output that looks like:
|
989
|
+
# +-----------------------------------+--------------------+
|
990
|
+
# | Name | Tags |
|
991
|
+
# +-----------------------------------+--------------------+
|
992
|
+
# | micro-bosh-stemcell-aws-0.6.4.tgz | aws, micro, stable |
|
993
|
+
# | micro-bosh-stemcell-aws-0.7.0.tgz | aws, micro, test |
|
994
|
+
# +-----------------------------------+--------------------+
|
995
|
+
#
|
996
|
+
# So to get the latest version for the filter tags,
|
997
|
+
# get the Name field, reverse sort, and return the first item
|
998
|
+
`#{bosh_stemcells_cmd} | grep micro | awk '{ print $2 }' | sort -r | head -n 1`.strip
|
999
|
+
end
|
1000
|
+
end
|
1001
|
+
|
1002
|
+
def provider_name
|
1003
|
+
settings.bosh_provider
|
1004
|
+
end
|
1005
|
+
|
1006
|
+
# a helper object for the target BOSH provider
|
1007
|
+
def provider
|
1008
|
+
@provider ||= Bosh::Providers.for_bosh_provider_name(settings.bosh_provider, fog_compute)
|
1009
|
+
end
|
1010
|
+
|
1011
|
+
def cyan; "\033[36m" end
|
1012
|
+
def clear; "\033[0m" end
|
1013
|
+
def bold; "\033[1m" end
|
1014
|
+
def red; "\033[31m" end
|
1015
|
+
def green; "\033[32m" end
|
1016
|
+
def yellow; "\033[33m" end
|
1017
|
+
|
1018
|
+
# Helper to access HighLine for ask & menu prompts
|
1019
|
+
def hl
|
1020
|
+
@hl ||= HighLine.new
|
1021
|
+
end
|
1022
|
+
end
|
1023
|
+
end
|
1024
|
+
end
|