omnibus 5.2.0 → 5.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d909e83257599b77c57d11ad3f4cad644286d005
-  data.tar.gz: ff63967a1b6a5189f29bb6524e73af08ecfc2d29
+  metadata.gz: a17a0e151fcd48e38827a9504daafcbdeb58abe5
+  data.tar.gz: 599d4ab2fb191028721c42fb912d836a05ef6e69
 SHA512:
-  metadata.gz: 5c02188f92090b922772ffc0c9c50a92ae32224a90ae36d0c5b3fe3bf79c56e3e32043ef797ade533ac783700665ea8b8874bfea24954c8db5190a21321565ef
-  data.tar.gz: 67be7899e3af5eff14c57d54c2d414165daa3b4811a65e8f4491a355a2725274f6522ca6daa4a7b8a3df1251657659e71b1ed8d664afecdd6e7561522ff16726
+  metadata.gz: 829c377378e984fabdbc83f3e23e7f092a25c8dda2823391fa7c4198a873529c8f8750367abeca0b908ad9c3b6073633a4308aa90814cb0e114fe6a2d6a0b39f
+  data.tar.gz: 6262ddf08f75fc3d49735e2f223a63c0eced1bf6050ab912f9a79113589c4dfd69ccc30abcc97f4c5caca60609435dabd861d09f0a6b88dd4ae61b586de77c32

data/.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,26 @@
+### Description
+
+Briefly describe the issue
+
+### Omnibus Version
+
+Tell us which version of Omnibus you are using
+
+### Platform Version
+
+Tell us which Operating System distribution, version and architecture the Omnibus build is being executed on.
+
+### Replication Case
+
+Tell us what steps to take to replicate your problem.  See [How to create a Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve)
+for information on how to create a good replication case.
+
+### Build Output
+
+The relevant output of the Omnibus build or a link to a gist of the entire build, if there is one.
+
+The verbose build output (`omnibus build <PROJECT> -l internal`) may be useful, but please link to a gist, or truncate it.
+
+--------------------------------------------------
+/cc @chef/omnibus-maintainers <- This ensures the Omnibus Maintainers team are notified to review this PR.
+

data/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,6 @@
+### Description
+
+Briefly describe the new feature or fix here
+
+--------------------------------------------------
+/cc @chef/omnibus-maintainers <- This ensures the Omnibus Maintainers team are notified to review this PR.

data/.travis.yml

@@ -1,3 +1,4 @@
+sudo: false
 rvm:
   - 2.0.0
   - 2.1
@@ -8,7 +9,12 @@ branches:
     - 2.0-stable
     - 3.0-stable
     - master
-before_install: gem install bundler
+before_install:
+  - gem update --system
+  - gem install bundler
+before_script:
+  - git config --global user.email "clouseau@chef.io"
+  - git config --global user.name "Pink Panther"
 script: bundle exec rake travis:ci
 notifications:
   slack:

data/CHANGELOG.md

@@ -1,6 +1,30 @@
 Omnibus CHANGELOG
 =================
 
+v5.3.0 (March 25, 2016)
+-----------------------
+### New Features
+
+- Get correct architecture name for Debian environments from `dpkg` (#646)
+- Update fauxhai to latest (#652)
+- Use project.license in deb and rpm packagers if available (#653)
+- Whitelist additional Solaris11 libraries (#650)
+- Use gcc 4.9 on freebsd (#649)
+- Per-Platform manifest generation (#645)
+  - Support omnibus manifest arguments: `omnibus manifest --os=OS --platform-family=FAMILY --platform=PLATFORM --platform-version=VERSION`
+  - Can be used to generate manifests for other platforms than the one you are on, allowing a multi-platform build to be quickly set up.
+  - Affects chef-sugar's `windows?`, `solaris?` and similar functions, which are used by software in `omnibus-software`
+- Performance: Ignore non-libraries in health checks (#642)
+- Add `github` source (#643)
+- Add `suse` support (#647)
+- Exclude files with whitespace names from AIX packages (#641)
+- Fix compilation problems on Windows (#640)
+  - Disable `sse` to avoid compilation segfaults
+  - Don't change newlines when `git pull`ing
+- Warn when software licenses are missing for software components (#638)
+- Add Chef MLSA to the list of recognized licenses (#639)
+- Download remote license files into the LICENSES directory (#637)
+
 v5.2.0 (March 15, 2016)
 -----------------------
 ### New Features

data/MAINTAINERS.md

@@ -1,8 +1,8 @@
 # Maintainers
 
-This file lists how the Omnibus project is maintained. When making changes
-to the system, this file tells you who needs to review your patch - you need a
-simple majority of maintainers for the relevant subsystems to provide a :+1: on
+This file lists how the `omnibus` project is maintained.
+
+When making changes to the system, you need two maintainers to provide a :+1: on
 your pull request. Additionally, you need to not receive a veto from a
 Lieutenant or the Project Lead.
 
@@ -19,10 +19,10 @@ project lead.
 
 * [Daniel DeLeo](https://github.com/danielsdeleo)
 * [Jay Mundrawala](https://github.com/jaym)
+* [Kartik Null Cating-Subramanian](https://github.com/ksubrama)
 * [Lamont Granquist](https://github.com/lamont-granquist)
 * [Scott Hain](https://github.com/scotthain)
 * [Seth Vargo](http://github.com/sethvargo)
 * [Steven Danna](https://github.com/stevendanna)
-* [Yvonne Lam](http://github.com/yzl)
 * [Thom May](https://github.com/thommay)
-* [Kartik Null Cating-Subramanian](https://github.com/ksubrama)
+* [Yvonne Lam](http://github.com/yzl)

data/README.md

@@ -4,7 +4,7 @@
 [![Build Status](http://img.shields.io/travis/chef/omnibus.svg)][travis]
 
 [gem]: https://rubygems.org/gems/omnibus
-[travis]: http://travis-ci.org/chef/omnibus
+[travis]: https://travis-ci.org/chef/omnibus
 
 Easily create full-stack installers for your project across a variety of platforms.
 
@@ -341,7 +341,7 @@ use_git_caching false
 License
 -------
 ```text
-Copyright 2012-2014 Chef Software, Inc.
+Copyright 2012-2016 Chef Software, Inc.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

data/appveyor.yml

@@ -25,6 +25,8 @@ install:
   - cp C:\Ruby21%ruby_version:~3%\Devkit\mingw\bin\bsdtar.exe C:\Ruby21%ruby_version:~3%\Devkit\mingw\bin\tar.exe
   - appveyor DownloadFile http://curl.haxx.se/ca/cacert.pem -FileName C:\cacert.pem
   - set SSL_CERT_FILE=C:\cacert.pem
+  - git config --global user.email "clouseau@chef.io"
+  - git config --global user.name "Pink Panther"
 
 build_script:
   - bundle install

data/features/commands/build.feature

@@ -1,8 +1,42 @@
 Feature: omnibus build
+  Background:
+    Given I have an omnibus project named "hamlet"
+
   Scenario: When the project does not exist
-    * I have an omnibus project named "hamlet"
-    * I run `omnibus build bacon`
-    * the output should contain:
-      """
-      I could not find a project named `bacon' in any of the project locations:
-      """
+    When I run `omnibus build bacon`
+
+    Then the output should contain:
+         """
+         I could not find a project named `bacon' in any of the project locations:
+         """
+    And  the exit status should not be 0
+
+# These scenarios don't work on appveyor due to no heat.exe/candle.exe/light.exe
+#  Scenario: When the project has no software definitions
+#    When I run `omnibus build hamlet`
+#
+#    Then it should pass with "[Project: hamlet] I | Building version manifest"
+#    And  the file "output/version-manifest.json" should exist
+#    And  the file "output/version-manifest.txt" should exist
+#    And  the file "output/LICENSE" should exist
+#    And  the directory "output/LICENSES" should exist
+#
+#  Scenario: When the project has a software definition
+#    Given a file "config/software/ophelia.rb" with:
+#          """
+#          name "ophelia"
+#          default_version "1.0.0"
+#          build do
+#            command "echo true > #{install_dir}/blah.txt"
+#          end
+#          """
+#    And   I append to "config/projects/hamlet.rb" with "dependency 'ophelia'"
+#
+#    When  I run `omnibus build hamlet`
+#
+#    Then it should pass with "[Builder: ophelia] I | $ echo true"
+#    And  the file "output/blah.txt" should contain "true"
+#    And  the file "output/version-manifest.json" should exist
+#    And  the file "output/version-manifest.txt" should exist
+#    And  the file "output/LICENSE" should exist
+#    And  the directory "output/LICENSES" should exist

data/features/commands/clean.feature

@@ -10,12 +10,12 @@ Feature: omnibus clean
   Scenario: When the --purge option is given
     * I have an omnibus project named "hamlet"
     * I successfully run `omnibus clean hamlet --purge`
-    * the output should contain "remove  local/build/hamlet"
+    * the output should contain "remove  output"
 
   Scenario: When no options are given
     * I have an omnibus project named "hamlet"
     * I successfully run `omnibus clean hamlet`
     * the output should not contain:
       """
-      remove  local/build/hamlet
+      remove  output
       """

data/features/commands/manifest.feature

@@ -0,0 +1,131 @@
+Feature: omnibus manifest
+  Background:
+    Given I have an omnibus project named "hamlet"
+
+  Scenario: When the project does not exist
+    When I run `omnibus manifest bacon`
+
+    Then the output should contain:
+         """
+         I could not find a project named `bacon' in any of the project locations:
+         """
+    And  the exit status should not be 0
+
+  Scenario: When the project has no software definitions
+    When I run `omnibus manifest hamlet`
+
+    Then it should pass with "[Project: hamlet] I | Building version manifest"
+    And  the output should contain:
+         """
+           "software": {
+           },
+         """
+
+  Scenario: When the project has a software definition
+    Given a file "config/software/ophelia.rb" with:
+          """
+          name "ophelia"
+          default_version "1.0.0"
+          build do
+            command "echo true > #{install_dir}/blah.txt"
+          end
+          """
+    And   I append to "config/projects/hamlet.rb" with "dependency 'ophelia'"
+
+    When  I run `omnibus manifest hamlet`
+
+    Then it should pass with "[Project: hamlet] I | Building version manifest"
+    And  the output should contain:
+         """
+           "software": {
+             "ophelia": {
+               "locked_version": "1.0.0",
+               "locked_source": null,
+               "source_type": "project_local",
+               "described_version": "1.0.0",
+               "license": "Unspecified"
+             }
+           },
+         """
+
+  Scenario: When the project has a software definition
+    Given a file "config/software/ophelia.rb" with:
+          """
+          name "ophelia"
+          default_version "1.0.0"
+          build do
+            command "echo true > #{install_dir}/blah.txt"
+          end
+          """
+    And   I append to "config/projects/hamlet.rb" with "dependency 'ophelia'"
+
+    When  I run `omnibus manifest hamlet`
+
+    Then it should pass with "[Project: hamlet] I | Building version manifest"
+    And  the output should contain:
+         """
+           "software": {
+             "ophelia": {
+               "locked_version": "1.0.0",
+               "locked_source": null,
+               "source_type": "project_local",
+               "described_version": "1.0.0",
+               "license": "Unspecified"
+             }
+           },
+         """
+    And  the exit status should be 0
+
+  Scenario: When the project has a software definition whose version depends on the OS
+    Given a file "config/software/ophelia.rb" with:
+          """
+          name "ophelia"
+          default_version (windows? ? "2.0.0" : "1.0.0")
+          build do
+            command "echo true > #{install_dir}/blah.txt"
+          end
+          """
+    And   I append to "config/projects/hamlet.rb" with "dependency 'ophelia'"
+
+    When  I run `omnibus manifest hamlet --os=linux --platform_family=debian --platform=ubuntu --platform_version=14.04`
+
+    Then it should pass with "[Project: hamlet] I | Building version manifest"
+    And  the output should contain:
+         """
+           "software": {
+             "ophelia": {
+               "locked_version": "1.0.0",
+               "locked_source": null,
+               "source_type": "project_local",
+               "described_version": "1.0.0",
+               "license": "Unspecified"
+             }
+           },
+         """
+
+  Scenario: When the project has a software definition whose version depends on the OS
+    Given a file "config/software/ophelia.rb" with:
+          """
+          name "ophelia"
+          default_version (windows? ? "2.0.0" : "1.0.0")
+          build do
+            command "echo true > #{install_dir}/blah.txt"
+          end
+          """
+    And   I append to "config/projects/hamlet.rb" with "dependency 'ophelia'"
+
+    When  I run `omnibus manifest hamlet --os=windows --platform_family=windows --platform=windows --platform_version=2012r2`
+
+    Then it should pass with "[Project: hamlet] I | Building version manifest"
+    And  the output should contain:
+         """
+           "software": {
+             "ophelia": {
+               "locked_version": "2.0.0",
+               "locked_source": null,
+               "source_type": "project_local",
+               "described_version": "2.0.0",
+               "license": "Unspecified"
+             }
+           },
+         """

data/features/step_definitions/generator_steps.rb

@@ -4,30 +4,46 @@ Given(/^I have an omnibus project named "(.+)"$/) do |name|
   create_directory(name)
   cd(name)
 
+  # Build target dir must be created
+  abs_path = expand_path(".")
+
+  # Single top level output dir
+  create_directory("output")
+
   write_file("config/projects/#{name}.rb", <<-EOH.gsub(/^ {4}/, ''))
     name '#{name}'
     maintainer 'Mrs. Maintainer'
     homepage 'https://example.com'
-    install_dir './local/build/#{name}'
+    install_dir "#{abs_path}/output"
 
     build_version '1.0.0'
 
     exclude '\.git*'
     exclude 'bundler\/git'
+
+    # This is necessary for Windows to pass.
+    package :msi do
+      upgrade_code "102FDF98-B9BF-4CE1-A716-2AB9CBCDA403"
+    end
   EOH
 
   write_file('omnibus.rb', <<-EOH.gsub(/^ {4}/, ''))
     # Build configuration
     append_timestamp false
-    cache_dir     './local/omnibus/cache'
-    git_cache_dir './local/omnibus/cache/git_cache'
-    source_dir    './local/omnibus/src'
-    build_dir     './local/omnibus/build'
-    package_dir   './local/omnibus/pkg'
-    package_tmp   './local/omnibus/pkg-tmp'
+    cache_dir     '#{abs_path}/local/omnibus/cache'
+    git_cache_dir '#{abs_path}/local/omnibus/cache/git_cache'
+    source_dir    '#{abs_path}/local/omnibus/src'
+    build_dir     '#{abs_path}/local/omnibus/build'
+    package_dir   '#{abs_path}/local/omnibus/pkg'
+    package_tmp   '#{abs_path}/local/omnibus/pkg-tmp'
   EOH
 end
 
+Given(/^I debug$/) do
+  require 'pry'
+  binding.pry
+end
+
 Given(/^I have a platform mappings file named "(.+)"$/) do |name|
   write_file(name, <<-EOH.gsub(/^ {4}/, ''))
     {

data/lib/omnibus.rb

@@ -112,6 +112,9 @@ module Omnibus
       end
 
       Config.reset!
+      # Clear caches on Project and Software
+      Project.reset!
+      Software.reset!
     end
 
     #

data/lib/omnibus/cli.rb

@@ -99,8 +99,28 @@ module Omnibus
     #
     #   $ omnibus manifest PROJECT
     #
+    method_option :os,
+      desc: "An os name in Ohai form. Defaults to the current os.",
+      type: :string
+    method_option :platform_family,
+      desc: "A platform family string in Ohai form. Defaults to the current platform_family.",
+      type: :string
+    method_option :platform,
+      desc: "A platform string in Ohai form. Defaults to the current platform.",
+      type: :string
+    method_option :platform_version,
+      desc: "A platform version string in Ohai form. Defaults to the current platform.",
+      type: :string
+    method_option :architecture,
+      desc: "The target architecture: x86, x64, powerpc. Defaults to the current architecture."
     desc 'manifest PROJECT', 'Print a manifest for the given Omnibus project'
     def manifest(name)
+      # Override ohai information
+      Ohai['os'] = @options[:os] if @options[:os]
+      Ohai['platform_family'] = @options[:platform_family] if @options[:platform_family]
+      Ohai['platform'] = @options[:platform] if @options[:platform]
+      Ohai['platform_version'] = @options[:platform_version] if @options[:platform_version]
+      Ohai['kernel']['machine'] = @options[:architecture] if @options[:architecture]
       puts JSON.pretty_generate(Project.load(name).built_manifest.to_hash)
     end
 

data/lib/omnibus/download_helpers.rb

@@ -0,0 +1,137 @@
+#
+# Copyright 2015 Chef Software, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'open-uri'
+require 'ruby-progressbar'
+
+module Omnibus
+  module DownloadHelpers
+    def self.included(base)
+      base.send(:include, InstanceMethods)
+    end
+
+    module InstanceMethods
+      private
+      #
+      # Downloads a from a given url to a given path using Ruby's
+      # +OpenURI+ implementation.
+      #
+      # @param [String] from_url
+      # @param [String] to_path
+      # @param [Hash] options
+      #   +options+ compatible with Ruby's +OpenURI+ implementation.
+      #   You can also use special option +enable_progress_bar+ which will
+      #   display a progress bar during download.
+      #
+      # @raise [SocketError]
+      # @raise [Errno::ECONNREFUSED]
+      # @raise [Errno::ECONNRESET]
+      # @raise [Errno::ENETUNREACH]
+      # @raise [Timeout::Error]
+      # @raise [OpenURI::HTTPError]
+      #
+      # @return [void]
+      #
+      def download_file!(from_url, to_path, download_options = {})
+        options = download_options.dup
+
+        # :enable_progress_bar is a special option we handle.
+        # by default we enable the progress bar.
+        enable_progress_bar = options.delete(:enable_progress_bar)
+        enable_progress_bar = true if enable_progress_bar.nil?
+
+        options.merge!(download_headers)
+        options[:read_timeout] = Omnibus::Config.fetcher_read_timeout
+
+        fetcher_retries ||= Omnibus::Config.fetcher_retries
+
+        reported_total = 0
+        if enable_progress_bar
+          progress_bar = ProgressBar.create(
+            output: $stdout,
+            format: '%e %B %p%% (%r KB/sec)',
+            rate_scale: ->(rate) { rate / 1024 },
+          )
+
+          options[:content_length_proc] = ->(total) {
+            reported_total = total
+            progress_bar.total = total
+          }
+          options[:progress_proc] = ->(step) {
+            downloaded_amount = [step, reported_total].min
+            progress_bar.progress = downloaded_amount
+          }
+        end
+
+        file = open(from_url, options)
+        # This is a temporary file. Close and flush it before attempting to copy
+        # it over.
+        file.close
+        FileUtils.cp(file.path, to_path)
+        file.unlink
+      rescue SocketError,
+             Errno::ECONNREFUSED,
+             Errno::ECONNRESET,
+             Errno::ENETUNREACH,
+             Timeout::Error,
+             OpenURI::HTTPError => e
+        if fetcher_retries != 0
+          log.info(log_key) { "Retrying failed download due to #{e} (#{fetcher_retries} retries left)..." }
+          fetcher_retries -= 1
+          retry
+        else
+          log.error(log_key) { "Download failed - #{e.class}!" }
+          raise
+        end
+      end
+
+      #
+      # The list of headers to pass to the download.
+      #
+      # @return [Hash]
+      #
+      def download_headers
+        {}.tap do |h|
+          # Alright kids, sit down while grandpa tells you a story. Back when the
+          # Internet was just a series of tubes, and you had to "dial in" using
+          # this thing called a "modem", ancient astronaunt theorists (computer
+          # scientists) invented gzip to compress requests sent over said tubes
+          # and make the Internet faster.
+          #
+          # Fast forward to the year of broadband - ungzipping these files was
+          # tedious and hard, so Ruby and other client libraries decided to do it
+          # for you:
+          #
+          #   https://github.com/ruby/ruby/blob/c49ae7/lib/net/http.rb#L1031-L1033
+          #
+          # Meanwhile, software manufacturers began automatically compressing
+          # their software for distribution as a +.tar.gz+, publishing the
+          # appropriate checksums accordingly.
+          #
+          # But consider... If a software manufacturer is publishing the checksum
+          # for a gzipped tarball, and the client is automatically ungzipping its
+          # responses, then checksums can (read: should) never match! Herein lies
+          # the bug that took many hours away from the lives of a once-happy
+          # developer.
+          #
+          # TL;DR - Do not let Ruby ungzip our file
+          #
+          h['Accept-Encoding'] = 'identity'
+        end
+      end
+    end
+  end
+end