bosh-gen 0.17.0 → 0.17.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +8 -8
- data/ChangeLog.md +3 -1
- data/README.md +66 -2
- data/bosh-gen.gemspec +2 -1
- data/lib/bosh/cli/commands/share_release.rb +78 -0
- data/lib/bosh/gen/utils/file_with_progress_bar.rb +62 -0
- data/lib/bosh/gen/version.rb +1 -1
- data/spec/generators/new_release_generator_spec.rb +24 -3
- metadata +20 -6
- data/spec/support/fog.rb +0 -3
checksums.yaml
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
---
|
2
2
|
!binary "U0hBMQ==":
|
3
3
|
metadata.gz: !binary |-
|
4
|
-
|
4
|
+
N2JjZDA3YzU4NWRmYmE2YmNlZjYxZDU5YzZjNWU5NzQwODRlOWIxNA==
|
5
5
|
data.tar.gz: !binary |-
|
6
|
-
|
6
|
+
MjU1MGQzNDJjYTRkYjYzNjc5OTYwYTg5NGJhZTRjYjgzMDMxZGMwZQ==
|
7
7
|
SHA512:
|
8
8
|
metadata.gz: !binary |-
|
9
|
-
|
10
|
-
|
11
|
-
|
9
|
+
MTZmMDA4OTQxYWMxYjk0MDk1NzIzZDViNTUxNDQxYTI1ODI2ZWUyZTBmZTNm
|
10
|
+
ZTRiYTliNWVlY2IxNDlhNWJkOWZjMWY1Y2U2YjAzN2E2NTJiZWRhNTZkNGJj
|
11
|
+
ZWZmYWY0ZjUzZDJjZGQxOTRmNTMwMTM0ODhlN2MxMTg1OTE4ZWI=
|
12
12
|
data.tar.gz: !binary |-
|
13
|
-
|
14
|
-
|
15
|
-
|
13
|
+
MWFhMjE5ZDZjYTMwYzkyMWQ2MDQ3ZmQ4NjAwZGIwZGZjZmJlMmM2MTRkOWVk
|
14
|
+
ZDY4ZjJkYTZmMTIyOTA2NWRjNWIxZWFkOTFjMGU2NGJjOTNiY2VlZTczOTdj
|
15
|
+
NmE0ZmYxZDU5MmJhMTRkZWI1OGNhNjdkNWU5OTY4ZDkyOTQ3ZjI=
|
data/ChangeLog.md
CHANGED
@@ -4,7 +4,9 @@ Change Log
|
|
4
4
|
v0.17.0
|
5
5
|
-------
|
6
6
|
|
7
|
-
- `new
|
7
|
+
- `new` - no more read-only credentials; no more encryption keys
|
8
|
+
- `new` - no more --s3/--swift flags; prompts instead
|
9
|
+
- `share release` - easily upload a final release and get a public URL to share (v0.17.1)
|
8
10
|
|
9
11
|
v0.16.0
|
10
12
|
-------
|
data/README.md
CHANGED
@@ -1,9 +1,13 @@
|
|
1
1
|
BOSH Generators
|
2
2
|
===============
|
3
3
|
|
4
|
-
Generators for creating BOSH releases.
|
4
|
+
Generators for creating and sharing BOSH releases.
|
5
5
|
|
6
|
-
New in 0.17: Creates blobstore/bucket when creating new release.
|
6
|
+
New in 0.17: Creates blobstore/bucket when creating new release. AWS bucket is publicly readable.
|
7
|
+
|
8
|
+
If you would like to share your BOSH release with the world, you can use the [BOSH Community AWS S3 account](#share-bosh-releases).
|
9
|
+
|
10
|
+
![example](http://f.cl.ly/items/3v2F43020a3N0Q1g3Z0E/bosh-gen-new.gif)
|
7
11
|
|
8
12
|
Installation
|
9
13
|
------------
|
@@ -210,6 +214,66 @@ bosh upload release
|
|
210
214
|
|
211
215
|
When you create a final release, you will first need to setup your AWS credentials in `config/final.yml`
|
212
216
|
|
217
|
+
Share BOSH releases
|
218
|
+
-------------------
|
219
|
+
|
220
|
+
To share your BOSH release with other BOSH users you need ONLY:
|
221
|
+
|
222
|
+
- Use a public blobstore (such as AWS S3)
|
223
|
+
- Use a public source control repository
|
224
|
+
- Optionally, publicly share pre-created final release tarballs via a HTTP URL.
|
225
|
+
|
226
|
+
### Share release tarballs via HTTP
|
227
|
+
|
228
|
+
bosh-gen includes a BOSH CLI plugin to upload dev or final tarballs to your blobstore, and get a public URL back (if supported by your blobstore)
|
229
|
+
|
230
|
+
```
|
231
|
+
$ bosh create release --with-tarball
|
232
|
+
$ bosh share release releases/my-project-1.tgz
|
233
|
+
https://my-project-boshrelease.s3.amazonaws.com/boshrelease-my-project-1.tgz
|
234
|
+
```
|
235
|
+
|
236
|
+
The URL is displayed and can be given to other users and uploaded directly to their BOSH:
|
237
|
+
|
238
|
+
```
|
239
|
+
$ bosh upload release https://my-project-boshrelease.s3.amazonaws.com/boshrelease-my-project-1.tgz
|
240
|
+
```
|
241
|
+
|
242
|
+
They no longer require your BOSH release repo to access the BOSH release.
|
243
|
+
|
244
|
+
### BOSH community facilities
|
245
|
+
|
246
|
+
You are welcome to re-use the BOSH user community facilities:
|
247
|
+
|
248
|
+
- Use the shared AWS S3 account (currently over 30 BOSH release blobstores).
|
249
|
+
- Place your release git repository in the [@cloudfoundry-community](https://github.com/cloudfoundry-community) GitHub account (over 50 people have access).
|
250
|
+
|
251
|
+
One time only, please email [Dr Nic Williams](mailto:drnicwilliams@gmail.com) and he will set you up with access:
|
252
|
+
|
253
|
+
- Read/write credentials to the AWS S3 account for your BOSH release blobstores/buckets
|
254
|
+
- Access to create [@cloudfoundry-community](https://github.com/cloudfoundry-community) GitHub repositories for your BOSH releases
|
255
|
+
|
256
|
+
When he gives you the AWS S3 credentials, place them in the `~/.fog` file and you'll easily be able to reuse them for each new BOSH release:
|
257
|
+
|
258
|
+
```yaml
|
259
|
+
:community:
|
260
|
+
:aws_access_key_id: ACCESS
|
261
|
+
:aws_secret_access_key: SECRET
|
262
|
+
```
|
263
|
+
|
264
|
+
Then for your next BOSH release:
|
265
|
+
|
266
|
+
```
|
267
|
+
$ bosh-gen new my-project
|
268
|
+
create
|
269
|
+
Auto-detected infrastructure API credentials at ~/.fog (override with $FOG)
|
270
|
+
1. AWS (community)
|
271
|
+
2. Alternate credentials
|
272
|
+
Choose an auto-detected infrastructure: 2
|
273
|
+
```
|
274
|
+
|
275
|
+
You'll only need to do this once. Yes, it would be awesome if there was some public service to do this nicely. Want to build it?
|
276
|
+
|
213
277
|
Contributing
|
214
278
|
------------
|
215
279
|
|
data/bosh-gen.gemspec
CHANGED
@@ -18,8 +18,9 @@ Gem::Specification.new do |gem|
|
|
18
18
|
gem.add_dependency "thor"
|
19
19
|
gem.add_dependency "bosh_cli"
|
20
20
|
gem.add_dependency "bosh_common"
|
21
|
+
gem.add_dependency "progressbar"
|
21
22
|
|
22
|
-
gem.add_dependency "cyoi", "~> 0.10
|
23
|
+
gem.add_dependency "cyoi", "~> 0.10"
|
23
24
|
gem.add_dependency "fog", "~> 1.11"
|
24
25
|
gem.add_dependency "readwritesettings", "~> 3.0"
|
25
26
|
|
@@ -0,0 +1,78 @@
|
|
1
|
+
require 'bosh/gen/utils/file_with_progress_bar'
|
2
|
+
|
3
|
+
module Bosh::Cli::Command
|
4
|
+
class ShareRelease < Base
|
5
|
+
usage "share release"
|
6
|
+
desc "upload release tarball to blobstore"
|
7
|
+
def share_release(tarball_path)
|
8
|
+
# need to be in release dir to get blobstore credentials
|
9
|
+
check_if_release_dir
|
10
|
+
|
11
|
+
unless File.exist?(tarball_path)
|
12
|
+
err("Release tarball file doesn't exist")
|
13
|
+
end
|
14
|
+
|
15
|
+
tarball = Bosh::Cli::ReleaseTarball.new(tarball_path)
|
16
|
+
say("\nVerifying release...")
|
17
|
+
tarball.validate(:allow_sparse => true)
|
18
|
+
nl
|
19
|
+
|
20
|
+
unless tarball.valid?
|
21
|
+
err('Release is invalid, please fix, verify and upload again')
|
22
|
+
end
|
23
|
+
|
24
|
+
upload_name = "boshrelease-#{tarball.release_name}-#{tarball.version}.tgz"
|
25
|
+
|
26
|
+
f = ::FileWithProgressBar.open(tarball_path, 'r')
|
27
|
+
f.out = Bosh::Cli::Config.output
|
28
|
+
|
29
|
+
raw_blobstore_client = unwrap_blobstore_client(blobstore)
|
30
|
+
bucket_name = raw_blobstore_client.instance_variable_get("@bucket_name")
|
31
|
+
|
32
|
+
fog = fog_storage(raw_blobstore_client)
|
33
|
+
dir = fog.directories.get(bucket_name)
|
34
|
+
|
35
|
+
say("\nUploading release...")
|
36
|
+
if file = dir.files.new(key: upload_name, body: f)
|
37
|
+
file.public = true
|
38
|
+
file.save
|
39
|
+
nl
|
40
|
+
say(file.public_url)
|
41
|
+
else
|
42
|
+
err('Failed to upload file to blobstore')
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
private
|
47
|
+
def unwrap_blobstore_client(blobstore)
|
48
|
+
if blobstore.is_a?(Bosh::Blobstore::RetryableBlobstoreClient)
|
49
|
+
unwrap_blobstore_client(blobstore.instance_variable_get("@client"))
|
50
|
+
elsif blobstore.is_a?(Bosh::Blobstore::Sha1VerifiableBlobstoreClient)
|
51
|
+
unwrap_blobstore_client(blobstore.instance_variable_get("@client"))
|
52
|
+
else
|
53
|
+
blobstore
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
def fog_storage(blobstore)
|
58
|
+
blobstore_options = blobstore.instance_variable_get("@options")
|
59
|
+
if blobstore.is_a?(Bosh::Blobstore::S3BlobstoreClient)
|
60
|
+
require "fog/aws"
|
61
|
+
return Fog::Storage.new(
|
62
|
+
provider: 'AWS',
|
63
|
+
aws_access_key_id: blobstore_options[:access_key_id],
|
64
|
+
aws_secret_access_key: blobstore_options[:secret_access_key],
|
65
|
+
)
|
66
|
+
# elsif blobstore.is_a?(Bosh::Blobstore::SwiftBlobstoreClient)
|
67
|
+
# require "fog/openstack"
|
68
|
+
# return Fog::Storage.new(
|
69
|
+
# provider: 'OpenStack',
|
70
|
+
# aws_access_key_id: blobstore_options[:access_key_id],
|
71
|
+
# aws_secret_access_key: blobstore_options[:secret_access_key],
|
72
|
+
# )
|
73
|
+
else
|
74
|
+
err('Not yet implemented for #{blobstore.class} blobstore')
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
@@ -0,0 +1,62 @@
|
|
1
|
+
require "progressbar"
|
2
|
+
# f = FileWithProgressBar.open(file, 'r')
|
3
|
+
# f.out = $stdout
|
4
|
+
#
|
5
|
+
# Then pass f like any File object that invokes f.read
|
6
|
+
#
|
7
|
+
# To upload with Fog:
|
8
|
+
# d = Storage[:aws].directories.create(key: 'drnic-test-upload')
|
9
|
+
# d.files.create(key: 'test.tgz', body: f)
|
10
|
+
# redis-2.8.12.: 100% |oooooooooooooo| 1.2MB 243.4KB/s Time: 00:00:04
|
11
|
+
class FileWithProgressBar < ::File
|
12
|
+
|
13
|
+
def out=(out)
|
14
|
+
@out = out
|
15
|
+
end
|
16
|
+
|
17
|
+
def progress_bar
|
18
|
+
return @progress_bar if @progress_bar
|
19
|
+
@out ||= StringIO.new
|
20
|
+
@progress_bar = ProgressBar.new(file_name, size, @out)
|
21
|
+
@progress_bar.file_transfer_mode
|
22
|
+
@progress_bar
|
23
|
+
end
|
24
|
+
|
25
|
+
def file_name
|
26
|
+
File.basename(self.path)
|
27
|
+
end
|
28
|
+
|
29
|
+
def stop_progress_bar
|
30
|
+
progress_bar.halt unless progress_bar.finished?
|
31
|
+
end
|
32
|
+
|
33
|
+
def size
|
34
|
+
@size || File.size(self.path)
|
35
|
+
end
|
36
|
+
|
37
|
+
def size=(size)
|
38
|
+
@size=size
|
39
|
+
end
|
40
|
+
|
41
|
+
def read(*args)
|
42
|
+
result = super(*args)
|
43
|
+
|
44
|
+
if result && result.size > 0
|
45
|
+
progress_bar.inc(result.size)
|
46
|
+
else
|
47
|
+
progress_bar.finish
|
48
|
+
end
|
49
|
+
|
50
|
+
result
|
51
|
+
end
|
52
|
+
|
53
|
+
def write(*args)
|
54
|
+
count = super(*args)
|
55
|
+
if count
|
56
|
+
progress_bar.inc(count)
|
57
|
+
else
|
58
|
+
progress_bar.finish
|
59
|
+
end
|
60
|
+
count
|
61
|
+
end
|
62
|
+
end
|
data/lib/bosh/gen/version.rb
CHANGED
@@ -4,9 +4,31 @@ require "bosh/gen/generators/new_release_generator"
|
|
4
4
|
describe Bosh::Gen::Generators::NewReleaseGenerator do
|
5
5
|
include Bosh::Gen::Settings
|
6
6
|
|
7
|
-
|
7
|
+
def cyoi_provider
|
8
|
+
provider = instance_double("Cyoi::Cli::Provider")
|
9
|
+
expect(provider).to receive(:execute!)
|
10
|
+
expect(Cyoi::Cli::Provider).to receive(:new).with([settings_dir]).and_return(provider)
|
11
|
+
provider
|
12
|
+
end
|
13
|
+
|
14
|
+
def cyoi_blobstore(blobstore_name)
|
15
|
+
blobstore = instance_double("Cyoi::Cli::Blobstore")
|
16
|
+
expect(blobstore).to receive(:execute!)
|
17
|
+
expect(Cyoi::Cli::Blobstore).to receive(:new).with([blobstore_name, settings_dir]).and_return(blobstore)
|
18
|
+
blobstore
|
19
|
+
end
|
20
|
+
|
21
|
+
before do
|
8
22
|
in_home_folder do
|
9
23
|
self.settings_dir = "redis-boshrelease/config"
|
24
|
+
cyoi_provider
|
25
|
+
cyoi_blobstore "redis-boshrelease"
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
it "generates with s3 blobstore" do
|
30
|
+
in_home_folder do
|
31
|
+
|
10
32
|
setting "provider.name", "aws"
|
11
33
|
setting "provider.region", "us-west-2"
|
12
34
|
setting "provider.credentials.aws_access_key_id", "ACCESS"
|
@@ -32,8 +54,7 @@ describe Bosh::Gen::Generators::NewReleaseGenerator do
|
|
32
54
|
end
|
33
55
|
end
|
34
56
|
|
35
|
-
|
36
|
-
xit "generates with swift blobstore" do
|
57
|
+
it "generates with swift blobstore" do
|
37
58
|
in_home_folder do
|
38
59
|
setting "provider.name", "openstack"
|
39
60
|
setting "provider.credentials.openstack_auth_url", "http://10.0.0.2:5000/v2.0/tokens"
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: bosh-gen
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.17.
|
4
|
+
version: 0.17.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Dr Nic Williams
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-06-
|
11
|
+
date: 2014-06-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: thor
|
@@ -52,20 +52,34 @@ dependencies:
|
|
52
52
|
- - ! '>='
|
53
53
|
- !ruby/object:Gem::Version
|
54
54
|
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: progressbar
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - ! '>='
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :runtime
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ! '>='
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '0'
|
55
69
|
- !ruby/object:Gem::Dependency
|
56
70
|
name: cyoi
|
57
71
|
requirement: !ruby/object:Gem::Requirement
|
58
72
|
requirements:
|
59
73
|
- - ~>
|
60
74
|
- !ruby/object:Gem::Version
|
61
|
-
version: 0.10
|
75
|
+
version: '0.10'
|
62
76
|
type: :runtime
|
63
77
|
prerelease: false
|
64
78
|
version_requirements: !ruby/object:Gem::Requirement
|
65
79
|
requirements:
|
66
80
|
- - ~>
|
67
81
|
- !ruby/object:Gem::Version
|
68
|
-
version: 0.10
|
82
|
+
version: '0.10'
|
69
83
|
- !ruby/object:Gem::Dependency
|
70
84
|
name: fog
|
71
85
|
requirement: !ruby/object:Gem::Requirement
|
@@ -141,6 +155,7 @@ files:
|
|
141
155
|
- Rakefile
|
142
156
|
- bin/bosh-gen
|
143
157
|
- bosh-gen.gemspec
|
158
|
+
- lib/bosh/cli/commands/share_release.rb
|
144
159
|
- lib/bosh/gen.rb
|
145
160
|
- lib/bosh/gen/cli.rb
|
146
161
|
- lib/bosh/gen/generators/deployment_manifest_generator.rb
|
@@ -210,6 +225,7 @@ files:
|
|
210
225
|
- lib/bosh/gen/models/deployment_manifest.rb
|
211
226
|
- lib/bosh/gen/models/release_detection.rb
|
212
227
|
- lib/bosh/gen/settings.rb
|
228
|
+
- lib/bosh/gen/utils/file_with_progress_bar.rb
|
213
229
|
- lib/bosh/gen/version.rb
|
214
230
|
- spec/fixtures/bosh_config/multiple_boshes.yml
|
215
231
|
- spec/fixtures/deployment_manifests/2_jobs_1_ip_8196_disk.yml
|
@@ -285,7 +301,6 @@ files:
|
|
285
301
|
- spec/models/deployment_manifest_spec.rb
|
286
302
|
- spec/models/release_detection_spec.rb
|
287
303
|
- spec/spec_helper.rb
|
288
|
-
- spec/support/fog.rb
|
289
304
|
- spec/support/generator_spec_helper.rb
|
290
305
|
homepage: https://github.com/drnic/bosh-gen
|
291
306
|
licenses: []
|
@@ -385,5 +400,4 @@ test_files:
|
|
385
400
|
- spec/models/deployment_manifest_spec.rb
|
386
401
|
- spec/models/release_detection_spec.rb
|
387
402
|
- spec/spec_helper.rb
|
388
|
-
- spec/support/fog.rb
|
389
403
|
- spec/support/generator_spec_helper.rb
|
data/spec/support/fog.rb
DELETED