bosh-gen 0.17.0 → 0.17.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,15 +1,15 @@
1
1
  ---
2
2
  !binary "U0hBMQ==":
3
3
  metadata.gz: !binary |-
4
- NmY5YjI3MTFiNTQ5YWVlNzZiODVjNTkzMjc5ZTUyOWI3YjcwOGZiNA==
4
+ N2JjZDA3YzU4NWRmYmE2YmNlZjYxZDU5YzZjNWU5NzQwODRlOWIxNA==
5
5
  data.tar.gz: !binary |-
6
- N2UxN2QxZTQ2MjNhMTY0YjNlMWRhZjRjYTg0ZWJmMjE5ZmQxYmZiYw==
6
+ MjU1MGQzNDJjYTRkYjYzNjc5OTYwYTg5NGJhZTRjYjgzMDMxZGMwZQ==
7
7
  SHA512:
8
8
  metadata.gz: !binary |-
9
- MWMwN2NmNTc5OTYwYTYwMjMxODdkODMwNTRlOGE1NjhjYmNkMjViZTJhZDQx
10
- NjUxOWVlNzZhMDAyMWMyOTFjNGFjN2Q4YmExNGVlZmQzNGQwMmE1MjY1OGFi
11
- Njg4YTE5ODAxMWNlNGE2ZDEwZmQ2ZTA4ZmI4YWNkNWVjNDhlMzk=
9
+ MTZmMDA4OTQxYWMxYjk0MDk1NzIzZDViNTUxNDQxYTI1ODI2ZWUyZTBmZTNm
10
+ ZTRiYTliNWVlY2IxNDlhNWJkOWZjMWY1Y2U2YjAzN2E2NTJiZWRhNTZkNGJj
11
+ ZWZmYWY0ZjUzZDJjZGQxOTRmNTMwMTM0ODhlN2MxMTg1OTE4ZWI=
12
12
  data.tar.gz: !binary |-
13
- YjBmODA3NzM3MzVhYmI4MTg3ZDYyMzQ3ZWUyMzZjODMwNGRmNGEyMjRmMzNj
14
- NmRmNGZlODk2Zjk4Njc3OTllMzJjYjY1NTAyMjEyYjAyNmM1ZjZiMzM5NjU3
15
- OGJhNDE3OGM2MmM2NDc5YzcyYTFiODY1YTQ4MWUwZWQ0NzM5NTg=
13
+ MWFhMjE5ZDZjYTMwYzkyMWQ2MDQ3ZmQ4NjAwZGIwZGZjZmJlMmM2MTRkOWVk
14
+ ZDY4ZjJkYTZmMTIyOTA2NWRjNWIxZWFkOTFjMGU2NGJjOTNiY2VlZTczOTdj
15
+ NmE0ZmYxZDU5MmJhMTRkZWI1OGNhNjdkNWU5OTY4ZDkyOTQ3ZjI=
@@ -4,7 +4,9 @@ Change Log
4
4
  v0.17.0
5
5
  -------
6
6
 
7
- - `new --s3` - no more read-only credentials; no more encryption keys
7
+ - `new` - no more read-only credentials; no more encryption keys
8
+ - `new` - no more --s3/--swift flags; prompts instead
9
+ - `share release` - easily upload a final release and get a public URL to share (v0.17.1)
8
10
 
9
11
  v0.16.0
10
12
  -------
data/README.md CHANGED
@@ -1,9 +1,13 @@
1
1
  BOSH Generators
2
2
  ===============
3
3
 
4
- Generators for creating BOSH releases.
4
+ Generators for creating and sharing BOSH releases.
5
5
 
6
- New in 0.17: Creates blobstore/bucket when creating new release.
6
+ New in 0.17: Creates blobstore/bucket when creating new release. AWS bucket is publicly readable.
7
+
8
+ If you would like to share your BOSH release with the world, you can use the [BOSH Community AWS S3 account](#share-bosh-releases).
9
+
10
+ ![example](http://f.cl.ly/items/3v2F43020a3N0Q1g3Z0E/bosh-gen-new.gif)
7
11
 
8
12
  Installation
9
13
  ------------
@@ -210,6 +214,66 @@ bosh upload release
210
214
 
211
215
  When you create a final release, you will first need to setup your AWS credentials in `config/final.yml`
212
216
 
217
+ Share BOSH releases
218
+ -------------------
219
+
220
+ To share your BOSH release with other BOSH users you need ONLY:
221
+
222
+ - Use a public blobstore (such as AWS S3)
223
+ - Use a public source control repository
224
+ - Optionally, publicly share pre-created final release tarballs via a HTTP URL.
225
+
226
+ ### Share release tarballs via HTTP
227
+
228
+ bosh-gen includes a BOSH CLI plugin to upload dev or final tarballs to your blobstore, and get a public URL back (if supported by your blobstore)
229
+
230
+ ```
231
+ $ bosh create release --with-tarball
232
+ $ bosh share release releases/my-project-1.tgz
233
+ https://my-project-boshrelease.s3.amazonaws.com/boshrelease-my-project-1.tgz
234
+ ```
235
+
236
+ The URL is displayed and can be given to other users and uploaded directly to their BOSH:
237
+
238
+ ```
239
+ $ bosh upload release https://my-project-boshrelease.s3.amazonaws.com/boshrelease-my-project-1.tgz
240
+ ```
241
+
242
+ They no longer require your BOSH release repo to access the BOSH release.
243
+
244
+ ### BOSH community facilities
245
+
246
+ You are welcome to re-use the BOSH user community facilities:
247
+
248
+ - Use the shared AWS S3 account (currently over 30 BOSH release blobstores).
249
+ - Place your release git repository in the [@cloudfoundry-community](https://github.com/cloudfoundry-community) GitHub account (over 50 people have access).
250
+
251
+ One time only, please email [Dr Nic Williams](mailto:drnicwilliams@gmail.com) and he will set you up with access:
252
+
253
+ - Read/write credentials to the AWS S3 account for your BOSH release blobstores/buckets
254
+ - Access to create [@cloudfoundry-community](https://github.com/cloudfoundry-community) GitHub repositories for your BOSH releases
255
+
256
+ When he gives you the AWS S3 credentials, place them in the `~/.fog` file and you'll easily be able to reuse them for each new BOSH release:
257
+
258
+ ```yaml
259
+ :community:
260
+ :aws_access_key_id: ACCESS
261
+ :aws_secret_access_key: SECRET
262
+ ```
263
+
264
+ Then for your next BOSH release:
265
+
266
+ ```
267
+ $ bosh-gen new my-project
268
+ create
269
+ Auto-detected infrastructure API credentials at ~/.fog (override with $FOG)
270
+ 1. AWS (community)
271
+ 2. Alternate credentials
272
+ Choose an auto-detected infrastructure: 2
273
+ ```
274
+
275
+ You'll only need to do this once. Yes, it would be awesome if there was some public service to do this nicely. Want to build it?
276
+
213
277
  Contributing
214
278
  ------------
215
279
 
@@ -18,8 +18,9 @@ Gem::Specification.new do |gem|
18
18
  gem.add_dependency "thor"
19
19
  gem.add_dependency "bosh_cli"
20
20
  gem.add_dependency "bosh_common"
21
+ gem.add_dependency "progressbar"
21
22
 
22
- gem.add_dependency "cyoi", "~> 0.10.0"
23
+ gem.add_dependency "cyoi", "~> 0.10"
23
24
  gem.add_dependency "fog", "~> 1.11"
24
25
  gem.add_dependency "readwritesettings", "~> 3.0"
25
26
 
@@ -0,0 +1,78 @@
1
+ require 'bosh/gen/utils/file_with_progress_bar'
2
+
3
+ module Bosh::Cli::Command
4
+ class ShareRelease < Base
5
+ usage "share release"
6
+ desc "upload release tarball to blobstore"
7
+ def share_release(tarball_path)
8
+ # need to be in release dir to get blobstore credentials
9
+ check_if_release_dir
10
+
11
+ unless File.exist?(tarball_path)
12
+ err("Release tarball file doesn't exist")
13
+ end
14
+
15
+ tarball = Bosh::Cli::ReleaseTarball.new(tarball_path)
16
+ say("\nVerifying release...")
17
+ tarball.validate(:allow_sparse => true)
18
+ nl
19
+
20
+ unless tarball.valid?
21
+ err('Release is invalid, please fix, verify and upload again')
22
+ end
23
+
24
+ upload_name = "boshrelease-#{tarball.release_name}-#{tarball.version}.tgz"
25
+
26
+ f = ::FileWithProgressBar.open(tarball_path, 'r')
27
+ f.out = Bosh::Cli::Config.output
28
+
29
+ raw_blobstore_client = unwrap_blobstore_client(blobstore)
30
+ bucket_name = raw_blobstore_client.instance_variable_get("@bucket_name")
31
+
32
+ fog = fog_storage(raw_blobstore_client)
33
+ dir = fog.directories.get(bucket_name)
34
+
35
+ say("\nUploading release...")
36
+ if file = dir.files.new(key: upload_name, body: f)
37
+ file.public = true
38
+ file.save
39
+ nl
40
+ say(file.public_url)
41
+ else
42
+ err('Failed to upload file to blobstore')
43
+ end
44
+ end
45
+
46
+ private
47
+ def unwrap_blobstore_client(blobstore)
48
+ if blobstore.is_a?(Bosh::Blobstore::RetryableBlobstoreClient)
49
+ unwrap_blobstore_client(blobstore.instance_variable_get("@client"))
50
+ elsif blobstore.is_a?(Bosh::Blobstore::Sha1VerifiableBlobstoreClient)
51
+ unwrap_blobstore_client(blobstore.instance_variable_get("@client"))
52
+ else
53
+ blobstore
54
+ end
55
+ end
56
+
57
+ def fog_storage(blobstore)
58
+ blobstore_options = blobstore.instance_variable_get("@options")
59
+ if blobstore.is_a?(Bosh::Blobstore::S3BlobstoreClient)
60
+ require "fog/aws"
61
+ return Fog::Storage.new(
62
+ provider: 'AWS',
63
+ aws_access_key_id: blobstore_options[:access_key_id],
64
+ aws_secret_access_key: blobstore_options[:secret_access_key],
65
+ )
66
+ # elsif blobstore.is_a?(Bosh::Blobstore::SwiftBlobstoreClient)
67
+ # require "fog/openstack"
68
+ # return Fog::Storage.new(
69
+ # provider: 'OpenStack',
70
+ # aws_access_key_id: blobstore_options[:access_key_id],
71
+ # aws_secret_access_key: blobstore_options[:secret_access_key],
72
+ # )
73
+ else
74
+ err('Not yet implemented for #{blobstore.class} blobstore')
75
+ end
76
+ end
77
+ end
78
+ end
@@ -0,0 +1,62 @@
1
+ require "progressbar"
2
+ # f = FileWithProgressBar.open(file, 'r')
3
+ # f.out = $stdout
4
+ #
5
+ # Then pass f like any File object that invokes f.read
6
+ #
7
+ # To upload with Fog:
8
+ # d = Storage[:aws].directories.create(key: 'drnic-test-upload')
9
+ # d.files.create(key: 'test.tgz', body: f)
10
+ # redis-2.8.12.: 100% |oooooooooooooo| 1.2MB 243.4KB/s Time: 00:00:04
11
+ class FileWithProgressBar < ::File
12
+
13
+ def out=(out)
14
+ @out = out
15
+ end
16
+
17
+ def progress_bar
18
+ return @progress_bar if @progress_bar
19
+ @out ||= StringIO.new
20
+ @progress_bar = ProgressBar.new(file_name, size, @out)
21
+ @progress_bar.file_transfer_mode
22
+ @progress_bar
23
+ end
24
+
25
+ def file_name
26
+ File.basename(self.path)
27
+ end
28
+
29
+ def stop_progress_bar
30
+ progress_bar.halt unless progress_bar.finished?
31
+ end
32
+
33
+ def size
34
+ @size || File.size(self.path)
35
+ end
36
+
37
+ def size=(size)
38
+ @size=size
39
+ end
40
+
41
+ def read(*args)
42
+ result = super(*args)
43
+
44
+ if result && result.size > 0
45
+ progress_bar.inc(result.size)
46
+ else
47
+ progress_bar.finish
48
+ end
49
+
50
+ result
51
+ end
52
+
53
+ def write(*args)
54
+ count = super(*args)
55
+ if count
56
+ progress_bar.inc(count)
57
+ else
58
+ progress_bar.finish
59
+ end
60
+ count
61
+ end
62
+ end
@@ -1,5 +1,5 @@
1
1
  module Bosh
2
2
  module Gen
3
- VERSION = "0.17.0"
3
+ VERSION = "0.17.1"
4
4
  end
5
5
  end
@@ -4,9 +4,31 @@ require "bosh/gen/generators/new_release_generator"
4
4
  describe Bosh::Gen::Generators::NewReleaseGenerator do
5
5
  include Bosh::Gen::Settings
6
6
 
7
- it "generates with s3 blobstore" do
7
+ def cyoi_provider
8
+ provider = instance_double("Cyoi::Cli::Provider")
9
+ expect(provider).to receive(:execute!)
10
+ expect(Cyoi::Cli::Provider).to receive(:new).with([settings_dir]).and_return(provider)
11
+ provider
12
+ end
13
+
14
+ def cyoi_blobstore(blobstore_name)
15
+ blobstore = instance_double("Cyoi::Cli::Blobstore")
16
+ expect(blobstore).to receive(:execute!)
17
+ expect(Cyoi::Cli::Blobstore).to receive(:new).with([blobstore_name, settings_dir]).and_return(blobstore)
18
+ blobstore
19
+ end
20
+
21
+ before do
8
22
  in_home_folder do
9
23
  self.settings_dir = "redis-boshrelease/config"
24
+ cyoi_provider
25
+ cyoi_blobstore "redis-boshrelease"
26
+ end
27
+ end
28
+
29
+ it "generates with s3 blobstore" do
30
+ in_home_folder do
31
+
10
32
  setting "provider.name", "aws"
11
33
  setting "provider.region", "us-west-2"
12
34
  setting "provider.credentials.aws_access_key_id", "ACCESS"
@@ -32,8 +54,7 @@ describe Bosh::Gen::Generators::NewReleaseGenerator do
32
54
  end
33
55
  end
34
56
 
35
- # TODO: Fog::Errors::MockNotImplemented: Contributions welcome!
36
- xit "generates with swift blobstore" do
57
+ it "generates with swift blobstore" do
37
58
  in_home_folder do
38
59
  setting "provider.name", "openstack"
39
60
  setting "provider.credentials.openstack_auth_url", "http://10.0.0.2:5000/v2.0/tokens"
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: bosh-gen
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.17.0
4
+ version: 0.17.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dr Nic Williams
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-06-29 00:00:00.000000000 Z
11
+ date: 2014-06-30 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: thor
@@ -52,20 +52,34 @@ dependencies:
52
52
  - - ! '>='
53
53
  - !ruby/object:Gem::Version
54
54
  version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: progressbar
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ! '>='
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ! '>='
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
55
69
  - !ruby/object:Gem::Dependency
56
70
  name: cyoi
57
71
  requirement: !ruby/object:Gem::Requirement
58
72
  requirements:
59
73
  - - ~>
60
74
  - !ruby/object:Gem::Version
61
- version: 0.10.0
75
+ version: '0.10'
62
76
  type: :runtime
63
77
  prerelease: false
64
78
  version_requirements: !ruby/object:Gem::Requirement
65
79
  requirements:
66
80
  - - ~>
67
81
  - !ruby/object:Gem::Version
68
- version: 0.10.0
82
+ version: '0.10'
69
83
  - !ruby/object:Gem::Dependency
70
84
  name: fog
71
85
  requirement: !ruby/object:Gem::Requirement
@@ -141,6 +155,7 @@ files:
141
155
  - Rakefile
142
156
  - bin/bosh-gen
143
157
  - bosh-gen.gemspec
158
+ - lib/bosh/cli/commands/share_release.rb
144
159
  - lib/bosh/gen.rb
145
160
  - lib/bosh/gen/cli.rb
146
161
  - lib/bosh/gen/generators/deployment_manifest_generator.rb
@@ -210,6 +225,7 @@ files:
210
225
  - lib/bosh/gen/models/deployment_manifest.rb
211
226
  - lib/bosh/gen/models/release_detection.rb
212
227
  - lib/bosh/gen/settings.rb
228
+ - lib/bosh/gen/utils/file_with_progress_bar.rb
213
229
  - lib/bosh/gen/version.rb
214
230
  - spec/fixtures/bosh_config/multiple_boshes.yml
215
231
  - spec/fixtures/deployment_manifests/2_jobs_1_ip_8196_disk.yml
@@ -285,7 +301,6 @@ files:
285
301
  - spec/models/deployment_manifest_spec.rb
286
302
  - spec/models/release_detection_spec.rb
287
303
  - spec/spec_helper.rb
288
- - spec/support/fog.rb
289
304
  - spec/support/generator_spec_helper.rb
290
305
  homepage: https://github.com/drnic/bosh-gen
291
306
  licenses: []
@@ -385,5 +400,4 @@ test_files:
385
400
  - spec/models/deployment_manifest_spec.rb
386
401
  - spec/models/release_detection_spec.rb
387
402
  - spec/spec_helper.rb
388
- - spec/support/fog.rb
389
403
  - spec/support/generator_spec_helper.rb
@@ -1,3 +0,0 @@
1
- require "fog"
2
-
3
- Fog.mock!