omnibus 3.1.1 → 3.2.0.rc.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 47494584e1a03e4d60e11d7e184ef6fdacfb3fcf
-  data.tar.gz: 6afbd1c06cb80f127ed9d1ef153e6cc66e41a41f
+  metadata.gz: 4812c396c2ec933378d32542df7c8245700ecf52
+  data.tar.gz: 51791434059dcada2197bf2bb8693e6c080669f2
 SHA512:
-  metadata.gz: 909e063a81130bc42ddcda0ec238276768b40a12089b26e9ef98ae422ab1ec19cc2b9a888964ef22930d7ce77293dfd2696d4dd8b8d9a2a8ec617d8c88e38f1f
-  data.tar.gz: 627dbb3379bf4642618770455455a977cdf7256c6d97c0e88d50ec999defc77ad532616cb782fe41295fc227613ee672df82718e1b530b837623ac6b5f33e19b
+  metadata.gz: 805e85738934c39e9d849fa6dd93d0877b17c60aab484e788616e1e143bd72df59971a8fce9c02cc15f2c3402717a565883425058a6fba20c0e3231c4508a6c8
+  data.tar.gz: 9351cf29fb353525d4e705bd0fdde1884ccaac3d393c3c78b96ed88f6963467a92ed32b40f5aad9e8ef7ccda0e460df0a8ed0954b39818e2d0a715d96261aca6

data/CHANGELOG.md

@@ -1,6 +1,77 @@
 Omnibus Ruby CHANGELOG
 ======================
 
+v3.2.0.rc.1 (July 14, 2014)
+-----------==--------------
+- Make build commands output during `log.info` instead of `log.debug`
+- Refactor Chef Sugar into an includable module, permitting DSL methods in both Software and Project definitions
+- Refactor `omnibus release` into a non-S3-specific backend "publisher"
+- Add support for specifying a dir glob to the `publish` command to upload multiple packages
+- "Package" is now a public API
+- Generate a real omnibus configuration file (no more `omnibus.rb.example`)
+- Add a releaser for Artifactory
+- Add additional information to package metdata (such as shasums)
+- Remove uses of Omnibus.config and use the Config object directly
+- Add the ability to define multiple `software_gems` in the config
+- Add the ability to define `local_software_paths` in the config
+- Add the ability to disable git caching in the config
+- Omnibus.load_configuration now requires a file path
+- Add new API for loading a project - `Project.load`
+- Add new API for loading a software - `Software.load`
+- Add publish APIs for dirtying the git cache
+- Add test coverage for the "public" API
+- Add validation to `source` in software DSL
+- Update generator templates to use the new APIs
+- Upgrade to Ohai 7.2
+- Improve YARDoc
+
+### Deprecations
+- Remove deprecated `Omnibus.configure` method
+- Deprecate `Omnibus.config.value` in favor of `Config.value` instead
+- Deprecate `Omnibus.project_root` in favor of `Config.project_root`
+- Deprecate [DSL] `platform` in favor of `Ohai['platform']`
+- Deprecate [DSL] `platform_family` in favor of `Ohai['platform_family']`
+- Deprecate [DSL] `platform_version` in favor of `Ohai['platform_version']`
+- Deprecate [DSL] `build_dir` in favor of `Config.build_dir`
+- Deprecate [DSL] `cache_dir` in favor of `Config.cache_dir`
+- Deprecate [DSL] `source_dir` in favor of `Config.source_dir`
+- Deprecate [DSL] `config` in favor of `Config` (capitalized)
+- Deprecate `Ohai.value` in favor of `Ohai['value']`
+- Deprecate `Project#install_path` in favor of `Project.install_dir`
+- Deprecate [DSL] `install_path` in favor of `install_dir`
+- Rename `Config.install_path_cache_dir` to `git_cache_dir`
+
+### DSL Changes
+- Add `with_embedded_path` to software
+- Add `with_standard_compiler_flags` to software
+- Add `package_scripts_path` to project
+
+### Bug fixes
+- Fix a small typo in the project generator (come -> some)
+- Update sample software definition for libpng to 1.5.18
+- Improved logging output
+- Include Chef Sugar in both software and project DSLs
+- Documentation updates and typographical fixes
+- Change the generated omnibus.rb to use a default homepage that includes the protocol
+- Ensure that software fetched via the PathFetcher are cached correctly
+- Downgrade FPM to ~> 0.4 - FPM 1.0.0+ uses FFI to attach to some libc functions. This fails on RHEL 5 & 6. As we don’t need a bleeding edge FPM the easiest fix is to just downgrade to the most recent pre-1.0.0 version.
+- Always print backtraces when errors occur
+- Do not sent ldd/otool to the same file - first steps in allowing parallel builds
+- Only rescue `Omnibus::Error` when invoked through the CLI - this will allow other bugs to actually raise at the Ruby level
+- Refactor the algorithm for git caching to take into account overrides and missing versions
+- Remove nested git directories before incremental caching occurs
+- Intelligently parse the project's homepage because Ruby's native URI implementation is buggy
+- Fetch all software at the start of the build - this fixes a bug where a build would fail halfway through because of a tiny typo of GitHub outage. Now, all required software is downloaded **before** the build starts, lowering the feedback time for a failure due to networking issues
+- Use the fetcher's `version_for_cache` method directly, falling back to `0.0.0` (and a warining) if no version is given
+
+### Potentially breaking changes
+- Merged `Package` and `Artifact` into the same class and updated API - this was considered an **internal** API so it is not a violation of semver
+- Use a common class for Omnibus exceptions - if you were rescuing Omnibus::Error, you might be rescuing all exceptions now
+- Use a cleanroom object when evaluating the DSL - prior to this release, Omnibus did not declare a public API. Project and software definitions had unrestricted access to the entire project.rb and software.rb methods respectively. This poses two problems - first, it makes it impossible to guarantee a public DSL API over a public (code) API. Second, it permits a developer to change the behavior of project.rb or software.rb accidentially, simply by defining a new method. The introducing of a cleanroom fixes both these bugs, however, it was impossible to know what was formerly considered a public API. Thus, it is possible that a previously-relied-on method is now unavaiable using the cleanroom. Please open an issue if you encounter such a case.
+- Remove mixlib-config - if you were relying on mixlib-config as a transitive dependency, it is no longer available
+- Remove the ability to use an overrides file - this was for internal use only and was never exposed as a public API. However, if you dug into the code and found it, it has now been removed. For BC purposes, the value still exists in the configuration object, but is essentially a no-op
+
+
 v3.1.1 (May 20, 2014)
 ---------------------
 - Update project generators to use new APIs. The old project generators created a project that issued deprecation warnings!

data/Gemfile

@@ -6,10 +6,3 @@ group :docs do
   gem 'redcarpet',     '~> 2.2.2'
   gem 'github-markup', '~> 0.7'
 end
-
-group :local do
-  gem 'guard'
-  gem 'guard-rspec'
-  gem 'guard-rubocop'
-  gem 'ruby_gntp'
-end

data/README.md

@@ -47,7 +47,9 @@ $ bundle install --binstubs
 $ bin/omnibus build $MY_PROJECT_NAME
 ```
 
-More details can be found in the generated project README file.
+More details can be found in the generated project's README file.
+
+Omnibus determines the platform for which to build an installer based on **the platform it is currently running on**. That is, you can only generate a `.deb` file on a Debian-based system. To alleviate this caveat, the generated project includes a [Test Kitchen](http://kitchen.ci) setup suitable for generating a series of Omnibus projects.
 
 
 More documentation
@@ -56,18 +58,79 @@ If you are creating OSX packages, please see the [OSX-specific documentation](do
 
 
 Configuration DSL
-------------------
-Though the template project will build, it won't do anything exciting. For that, you'll need to use the Omnibus DSL to define the specifics of your own application.
+-----------------
+Though the template project will build, it will not do anything exciting. For that, you need to use the Omnibus DSL to define the specifics of your application.
+
+### Config
+If present, Omnibus will use a top-level configuration file named `omnibus.rb` at the root of your repository. This file is loaded at runtime and includes a number of configuration tunables. Here is an example:
+
+```ruby
+# Build locally (instead of /var)
+# -------------------------------
+base_dir './local'
+
+# Disable git caching
+# ------------------------------
+use_git_caching false
+
+# Enable S3 asset caching
+# ------------------------------
+use_s3_caching true
+s3_access_key  ENV['S3_ACCESS_KEY']
+s3_secret_key  ENV['S3_SECRET_KEY']
+s3_bucket      ENV['S3_BUCKET']
+```
+
+For more information, please see the [`Config` documentation](http://rubydoc.info/github/opscode/omnibus-ruby/Omnibus/Config).
+
+You can tell Omnibus to load a difference configuration file by passing the `--config` option to any command:
+
+```shell
+$ bin/omnibus --config /path/to/config.rb
+```
+
+Finally, you can override a specific configuration option at the command line using the `--override` flag. This takes ultimate precedence over any configuration file values:
+
+```shell
+$ bin/omnibus --override use_git_caching:false
+```
+
+### Projects
+A Project DSL file defines your actual application; this is the thing you are creating a full-stack installer for in the first place. It provides a means to define the dependencies of the project (again, as specified in Software DSL definition files), as well as ways to set installer package metadata.
+
+All project definitions must be in the `config/projects` directory of your Omnibus repository.
+
+```ruby
+name            'chef-full'
+maintainer      'YOUR NAME'
+homepage        'http://yoursite.com'
+
+install_dir     '/opt/chef'
+build_version   '0.10.8'
+build_iteration 4
+
+dependency 'chef'
+```
+
+Some DSL methods available include:
+
+| DSL Method        | Description                                 |
+| :---------------: | --------------------------------------------|
+| `name`            | The name of the project                     |
+| `install_dir`     | The desired install location of the package |
+| `build_version`   | The package version                         |
+| `build_iteration` | The package iteration number                |
+| `dependency`      | An Omnibus software-defined component to include in this package |
+
+For more information, please see the [`Project` documentation](http://rubydoc.info/github/opscode/omnibus-ruby/Omnibus/Project).
+
 
 ### Software
 Omnibus "software" files define individual software components that go into making your overall package. They are the building blocks of your application. The Software DSL provides a way to define where to retrieve the software sources, how to build them, and what dependencies they have. These dependencies are also defined in their own Software DSL files, thus forming the basis for a dependency-aware build ordering.
 
 All Software definitions should go in the `config/software` directory of your Omnibus project repository.
 
-CHEF has created software definitions for a number of commonly-needed components, available in the [omnibus-software](https://github.com/opscode/omnibus-software.git)
-repository. When you create a new project skeleton using Omnibus, this is automatically added to the project's Gemfile, making all these software definitions available to you.  (If you prefer, however, you can write your own versions of these same definitions in your project repository; local copies in `config/software` have precedence over anything from the `omnibus-software` repository.)
-
-An example:
+Here is an example:
 
 ```ruby
 name 'ruby'
@@ -118,48 +181,48 @@ version '2.1.1' do
 end
 ```
 
-Since the software definitions are simply ruby code, you can conditionally execute anything by wrapping it with pure ruby that tests for the version number.
+Since the software definitions are simply ruby code, you can conditionally execute anything by wrapping it with pure Ruby that tests for the version number.
 
-For more DSL methods, please consult the documentation.
+For more DSL methods, please consult the [`Software` documentation](http://rubydoc.info/github/opscode/omnibus-ruby/Omnibus/Software).
 
-### Projects
-A Project DSL file defines your actual application; this is the thing you are creating a full-stack installer for in the first place. It provides a means to define the dependencies of the project (again, as specified in Software DSL definition files), as well as ways to set installer package metadata.
+#### Sharing software definitions
+The easiest way to share organization-wide software is via bundler and Rubygems. For an example software repository, look at Chef's [omnibus-software](https://github.com/opscode/omnibus-software). For more information, please see the [Rubygems documentation](http://guides.rubygems.org/publishing/).
 
-All Project definitions (yes, you can have more than one) should go in the `config/projects` directory of your Omnibus project repository.
+It is recommended you use bundler to pull down these gems (as bundler also permits pulling softare directly from GitHub):
 
 ```ruby
-name            'chef-full'
-maintainer      'YOUR NAME'
-homepage        'http://yoursite.com'
+gem 'my-company-omnibus-software'
+gem 'omnibus-software', github: 'my-company/omnibus-software'
+```
 
-install_path    '/opt/chef'
-build_version   '0.10.8'
-build_iteration 4
+Then add the name of the software to the list of `software_gems` in your Omnibus config:
 
-dependency 'chef'
+```ruby
+software_gems %w(my-company-omnibus-software omnibus-software)
 ```
 
-Some DSL methods available include:
+You may also specify local paths on disk (but be warned this may make sharing the project among teams difficult):
 
-| DSL Method        | Description                                 |
-| :---------------: | --------------------------------------------|
-| `name`            | The name of the project                     |
-| `install_path`    | The desired install location of the package |
-| `build_version`   | The package version                         |
-| `build_iteration` | The package iteration number                |
-| `dependency`      | An Omnibus software-defined component to include in this package |
+```ruby
+local_software_dirs %w(/path/to/software /other/path/to/software)
+```
 
-For more information, please see the documentation.
+For all of these paths, **order matters**, so it is possible to depend on local software version while still retaining a remote software repo. Given the above example, Omnibus will search for a software definition named `foo` in this order:
 
 
-Caveats
--------
-### A note on builds
-As stated above, the generated project skeleton can run "as-is". However, Omnibus determines the platform for which to build an installer based on *the platform it is currently running on*. That is, you can only generate a `.deb` file for Ubuntu if you're actually running Omnibus *on Ubuntu*.
+```text
+$PWD/config/software/foo.rb
+/path/to/software/config/software/foo.rb
+/other/path/to/software/config/software/foo.rb
+/Users/sethvargo/.gems/.../my-comany-omnibus-software/config/software/foo.rb
+/Users/sethvargo/.gems/.../omnibus-software/config/software/foo.rb
+```
 
-This is currently achieved using [Test Kitchen](http://kitchen.ci), which is included with any newly generated Omnibus project.
+The first instance of `foo.rb` that is encountered will be used. Please note that **local** (vendored) softare definitions take precedence!
 
 
+Caveats
+-------
 ### Overrides
 The project definitions can override specific software dependencies by passing in `override` to use the correct version:
 
@@ -173,12 +236,21 @@ override :chef, version: '2.1.1'
 dependency 'chef'
 ```
 
-There is no checking that the version override that you supply has been provided in a version override block in the software definition.
+**The overridden version must be defined in the associated software!**
+
+### Debugging
+By default, Omnibus will log at the `warn` level. You can override this by passing the `--log-level` flag to your Omnibus call:
+
+```shell
+$ bin/omnibus build <project> --log-level info // or 'debug'
+```
 
 ### Git caching
-As of Omnibus 3.0.0, projects are no longer built using rake. Instead, we have rewritten the software dependencies to leverage git caching. This means we cache compiled software definitions, so future Omnibus project builds are much faster.
+by default, Omnibus caches compiled software definitions, so n+1 Omnibus project builds are much faster. This functionality can be disabled by adding the following to your `omnibus.rb`:
 
-For more information on potential breaking changes, please see the CHANGELOG entry for Omnibus 3.0.0.
+```ruby
+use_git_caching false
+```
 
 
 License

data/Rakefile

@@ -20,13 +20,9 @@ Cucumber::Rake::Task.new(:acceptance) do |t|
   end.join(' ')
 end
 
-require 'rubocop/rake_task'
-desc 'Run Ruby style checks'
-Rubocop::RakeTask.new(:style)
-
 namespace :travis do
   desc 'Run tests on Travis'
-  task ci: %w(unit acceptance style)
+  task ci: %w(unit acceptance)
 end
 
 task default: %w(travis:ci)

data/docs/omnibus-build-cache.md

@@ -5,7 +5,7 @@ reduces the time it takes to rebuild a project when only a few
 components need to be rebuilt.
 
 The cache uses git to snapshot the project tree after each software
-component build. The entire contents of the project's `install_path`
+component build. The entire contents of the project's `install_dir`
 is included in the snaphot.
 
 When rebuilding, omnibus walks the linearized component list and
@@ -18,11 +18,11 @@ config/software/$COMPONENT).
 
 The default location of the cache (which is just a bare git
 repository) is
-`/var/cache/omnibus/cache/install_path/$INSTALL_PATH`. You can
+`/var/cache/omnibus/cache/git_cache/$INSTALL_DIR`. You can
 customize the location of the cache in the `omnibus.rb` config file
-using the key `install_path_cache_dir`. For example:
+using the key `git_cache_dir`. For example:
 
-    install_path_cache_dir "/opt/ominbus-caches"
+    git_cache_dir "/opt/ominbus-caches"
 
 ## How It Works ##
 
@@ -44,7 +44,7 @@ component list, you want to have your most frequently changed
 components last to minimize rebuild churn.
 
 Lightweight git tags are used as cache keys. After building a
-component, a tag name is computed in `install_path_cache.rb#tag`. The
+component, a tag name is computed in `GitCache#tag`. The
 tag name has the format `#{name}-#{version}-#{digest}` where name and
 version map to the component that was just built and digest is a
 SHA256 value. The digest is computed by string-ifying the name/version

data/features/commands/_deprecated.feature

@@ -1,7 +1,4 @@
 Feature: Backwards-compatible deprecated commands
-  Background:
-    * I have an omnibus project named "hamlet"
-
   Scenario: When "build project" is given
     * I run `omnibus build project hamlet`
     * the output should contain:
@@ -64,3 +61,24 @@ Feature: Backwards-compatible deprecated commands
       """
       The environment variable 'OMNIBUS_APPEND_TIMESTAMP' is deprecated. Please use '--override append_timestamp:false' instead.
       """
+
+  Scenario: When "release package" is given
+    * I run `omnibus release package /path/to/package`
+    * the output should contain:
+      """
+      The interface for releasing a project has changed. Please use 'omnibus publish BACKEND [COMAMND]' instead.
+      """
+
+  Scenario: When "release package --public" is given
+    * I run `omnibus release package /path/to/package --public`
+    * the output should contain:
+      """
+      The '--public' option has been deprecated! Please use '--acl public' instead.
+      """
+
+  Scenario: When "release package --no-public" is given
+    * I run `omnibus release package /path/to/package --no-public`
+    * the output should contain:
+      """
+      The '--no-public' option has been deprecated! Please use '--acl private' instead.
+      """

data/features/step_definitions/generator_steps.rb

@@ -8,7 +8,7 @@ Given(/^I have an omnibus project named "(.+)"$/) do |name|
     name '#{name}'
     maintainer 'Mrs. Maintainer'
     homepage 'https://example.com'
-    install_path './local/build/#{name}'
+    install_dir './local/build/#{name}'
 
     build_version '1.0.0'
 
@@ -18,11 +18,11 @@ Given(/^I have an omnibus project named "(.+)"$/) do |name|
 
   write_file('omnibus.rb', <<-EOH.gsub(/^ {4}/, ''))
     # Build configuration
-    cache_dir              './local/omnibus/cache'
-    install_path_cache_dir './local/omnibus/cache/install_path'
-    source_dir             './local/omnibus/src'
-    build_dir              './local/omnibus/build'
-    package_dir            './local/omnibus/pkg'
-    package_tmp            './local/omnibus/pkg-tmp'
+    cache_dir     './local/omnibus/cache'
+    git_cache_dir './local/omnibus/cache/git_cache'
+    source_dir    './local/omnibus/src'
+    build_dir     './local/omnibus/build'
+    package_dir   './local/omnibus/pkg'
+    package_tmp   './local/omnibus/pkg-tmp'
   EOH
 end

data/lib/omnibus.rb

@@ -15,6 +15,7 @@
 #
 
 require 'pathname'
+require 'json'
 
 require 'omnibus/exceptions'
 require 'omnibus/version'
@@ -25,46 +26,51 @@ module Omnibus
   #
   # @return [String]
   #
-  DEFAULT_CONFIG = 'omnibus.rb'
+  DEFAULT_CONFIG = 'omnibus.rb'.freeze
 
-  autoload :Artifact,         'omnibus/artifact'
   autoload :Builder,          'omnibus/builder'
   autoload :BuildVersion,     'omnibus/build_version'
   autoload :BuildVersionDSL,  'omnibus/build_version_dsl'
   autoload :Cleaner,          'omnibus/cleaner'
+  autoload :Cleanroom,        'omnibus/cleanroom'
   autoload :Config,           'omnibus/config'
+  autoload :Digestable,       'omnibus/digestable'
   autoload :Error,            'omnibus/exceptions'
   autoload :Fetcher,          'omnibus/fetcher'
   autoload :Generator,        'omnibus/generator'
+  autoload :GitCache,         'omnibus/git_cache'
   autoload :HealthCheck,      'omnibus/health_check'
-  autoload :InstallPathCache, 'omnibus/install_path_cache'
   autoload :Library,          'omnibus/library'
   autoload :Logger,           'omnibus/logger'
   autoload :Logging,          'omnibus/logging'
+  autoload :NullArgumentable, 'omnibus/null_argumentable'
   autoload :NullBuilder,      'omnibus/null_builder'
   autoload :Ohai,             'omnibus/ohai'
-  autoload :Overrides,        'omnibus/overrides'
-  autoload :PackageRelease,   'omnibus/package_release'
+  autoload :Package,          'omnibus/package'
   autoload :Project,          'omnibus/project'
+  autoload :Publisher,        'omnibus/publisher'
   autoload :Reports,          'omnibus/reports'
   autoload :S3Cache,          'omnibus/s3_cache'
   autoload :Software,         'omnibus/software'
-  autoload :SoftwareS3URLs,   'omnibus/software_s3_urls'
+  autoload :Sugarable,        'omnibus/sugarable'
   autoload :Util,             'omnibus/util'
 
   # @todo Remove this in the next major release
   autoload :OHAI, 'omnibus/ohai'
 
-  # @todo Refactor these under a +Fetcher module
   autoload :GitFetcher,     'omnibus/fetchers/git_fetcher'
   autoload :NetFetcher,     'omnibus/fetchers/net_fetcher'
   autoload :PathFetcher,    'omnibus/fetchers/path_fetcher'
   autoload :S3CacheFetcher, 'omnibus/fetchers/s3_cache_fetcher'
 
+  autoload :ArtifactoryPublisher, 'omnibus/publishers/artifactory_publisher'
+  autoload :NullPublisher,        'omnibus/publishers/null_publisher'
+  autoload :S3Publisher,          'omnibus/publishers/s3_publisher'
+
   module Command
     autoload :Base,    'omnibus/cli/base'
     autoload :Cache,   'omnibus/cli/cache'
-    autoload :Release, 'omnibus/cli/release'
+    autoload :Publish, 'omnibus/cli/publish'
   end
 
   module Packager
@@ -111,48 +117,48 @@ module Omnibus
       @logger ||= Logger.new
     end
 
+    #
+    # @api private
+    #
+    # Programatically set the logger for Omnibus.
+    #
+    # @param [Logger] logger
+    #
     def logger=(logger)
       @logger = logger
     end
 
-    def ui
-      @ui ||= Thor::Base.shell.new
-    end
-
-    # Configure Omnibus.
     #
-    # After this has been called, the {Omnibus::Config} object is
-    # available as `Omnibus.config`.
+    # The UI class for Omnibus.
     #
-    # @return [void]
+    # @return [Thor::Shell]
     #
-    # @deprecated Use {#load_configuration} if you need to process a
-    #   config file, followed by {#process_configuration} to act upon it.
-    def configure
-      load_configuration
-      process_configuration
+    def ui
+      @ui ||= Thor::Base.shell.new
     end
 
-    # Convenience method for access to the Omnibus::Config object.
+    # Convenience method for access to the {Config} object.
     # Provided for backward compatibility.
     #
-    # @ return [Omnibus::Config]
+    # @return [Config]
     #
-    # @deprecated Just refer to {Omnibus::Config} directly.
+    # @deprecated Just refer to {Config} directly.
     def config
+      Omnibus.logger.deprecated('Omnibus') do
+        'Omnibus.config. Please use Config.(thing) instead.'
+      end
+
       Config
     end
 
     # Load in an Omnibus configuration file.  Values will be merged with
-    # and override the defaults defined in {Omnibus::Config}.
+    # and override the defaults defined in {Config}.
     #
-    # @param file [String] path to a configuration file to load
+    # @param [String] file path to a configuration file to load
     #
     # @return [void]
-    def load_configuration(file = nil)
-      if file
-        Config.from_file(file)
-      end
+    def load_configuration(file)
+      Config.load(file)
     end
 
     # Processes the configuration to construct the dependency tree of
@@ -160,227 +166,282 @@ module Omnibus
     #
     # @return [void]
     def process_configuration
-      Config.validate
       process_dsl_files
     end
 
-    # All {Omnibus::Project} instances that have been created.
     #
-    # @return [Array<Omnibus::Project>]
+    # All {Project} instances that have been loaded.
+    #
+    # @return [Array<:Project>]
+    #
     def projects
-      @projects ||= []
+      _projects.values
     end
 
-    # Names of all the {Omnibus::Project} instances that have been created.
     #
-    # @return [Array<String>]
-    def project_names
-      projects.map { |p| p.name }
-    end
-
-    # Load the {Omnibus::Project} instance with the given name.
+    # Load the {Project} instance with the given name.
+    #
+    # @param [String] name
+    #   the name of the project to get
+    #
+    # @return [Project]
     #
-    # @param name [String]
-    # @return {Omnibus::Project}
     def project(name)
-      projects.find { |p| p.name == name }
+      _projects[name.to_s]
     end
 
-    # The absolute path to the Omnibus project/repository directory.
+    #
+    # The absolute path to the Omnibus project/reository directory.
+    #
+    # @deprecated Use {Config.project_root} instead.
     #
     # @return [String]
+    #
     def project_root
+      Omnibus.logger.deprecated('Omnibus') do
+        'Omnibus.project_root. Please use Config.project_root instead.'
+      end
+
+      Config.project_root
+    end
+
+    #
+    # Backward compat alias.
+    #
+    # @deprecated Use {Config.project_root} instead.
+    #
+    # @see (Omnibus.project_root)
+    #
+    def root
+      Omnibus.logger.deprecated('Omnibus') do
+        'Omnibus.root. Please use Omnibus.project_root instead.'
+      end
+
       Config.project_root
     end
 
+    #
     # The source root is the path to the root directory of the `omnibus` gem.
     #
     # @return [Pathname]
+    #
     def source_root
       @source_root ||= Pathname.new(File.expand_path('../..', __FILE__))
     end
 
-    # The source root is the path to the root directory of the `omnibus-software`
-    # gem.
+    # Processes all configured {Omnibus::Project} and
+    # {Omnibus::Software} DSL files.
     #
-    # @return [Pathname]
-    def omnibus_software_root
-      @omnibus_software_root ||= begin
-        if (spec = Gem::Specification.find_all_by_name(Config.software_gem).first)
-          Pathname.new(spec.gem_dir)
-        else
-          nil
-        end
-      end
+    # @return [void]
+    def process_dsl_files
+      expand_software
     end
 
-    # Return paths to all configured {Omnibus::Project} DSL files.
     #
-    # @return [Array<String>]
-    def project_files
-      ruby_files(File.join(project_root, Config.project_dir))
-    end
-
-    # Return paths to all configured {Omnibus::Software} DSL files.
+    # The list of directories to search for {Software} files. These paths are
+    # returned **in order** of specifity.
     #
-    # @return [Array<String>]
-    def software_files
-      ruby_files(File.join(project_root, Config.software_dir))
-    end
-
-    # Return directories to search for {Omnibus::Software} DSL files.
+    # @see (Config#project_root)
+    # @see (Config#software_dir)
+    # @see (Config#software_gems)
+    # @see (Config#local_software_dirs)
     #
     # @return [Array<String>]
+    #
     def software_dirs
-      @software_dirs ||= begin
-        software_dirs = [File.join(project_root, Config.software_dir)]
-        software_dirs << File.join(omnibus_software_root, 'config', 'software') if omnibus_software_root
-        software_dirs
+      directories = [
+        paths_from_project_root,
+        paths_from_local_software_dirs,
+        paths_from_software_gems,
+      ].flatten
+
+      directories.inject([]) do |array, directory|
+        softwares_path = File.join(directory, Config.software_dir)
+
+        if File.directory?(softwares_path)
+          array << softwares_path
+        else
+          Omnibus.logger.warn('Omnibus') do
+            "`#{directory}' does not contain a valid directory structure. " \
+            "Does it contain a folder at `#{Config.software_dir}'?"
+          end
+        end
+
+        array
       end
     end
 
-    # Backward compat alias
     #
-    # @todo Remve this in the next major release (4.0)
+    # A hash of all softwares (by name) and their respective path on disk. These
+    # files are **in order**, meaning the software path is the **first**
+    # occurrence of the software in the list. If the same software is
+    # encountered a second time, it will be skipped.
     #
-    # @see (Omnibus.project_root)
-    def root
-      Omnibus.logger.deprecated('Omnibus') do
-        'Omnibus.root. Please use Omnibus.project_root instead.'
-      end
-
-      project_root
-    end
-
-    # Processes all configured {Omnibus::Project} and
-    # {Omnibus::Software} DSL files.
+    # @example
+    #   { 'preparation' => '/home/omnibus/project/config/software/preparation.rb' }
     #
-    # @return [void]
-    def process_dsl_files
-      # Do projects first
-      expand_projects
-
-      # Then do software
-      final_software_map = prefer_local_software(omnibus_software_files, software_files)
-
-      overrides = Config.override_file ? Omnibus::Overrides.overrides : {}
+    # @return [Hash<String, String>]
+    #
+    def software_map
+      software_dirs.inject({}) do |hash, directory|
+        Dir.glob("#{directory}/*.rb").each do |path|
+          name = File.basename(path, '.rb')
+
+          if hash[name].nil?
+            Omnibus.logger.info('Omnibus#software_map') do
+              "Using software `#{name}' from `#{path}'."
+            end
+
+            hash[name] = path
+          else
+            Omnibus.logger.debug('Omnibus#software_map') do
+              "Skipping software `#{name}' because it was loaded from an " \
+              "earlier path."
+            end
+          end
+        end
 
-      expand_software(overrides, final_software_map)
+        hash
+      end
     end
 
     private
 
-    # Generates {Omnibus::Project}s for each project DSL file in
-    # `project_specs`.  All projects are then accessible at
-    # {Omnibus#projects}
     #
-    # @return [void]
+    # @api private
     #
-    # @see Omnibus::Project
-    def expand_projects
-      project_files.each do |spec|
-        Omnibus.projects << Omnibus::Project.load(spec)
-      end
-    end
-
-    # Generate {Omnibus::Software} objects for all software DSL files in
-    # `software_specs`.
+    # The list of omnibus projects. This is an internal API that maps a
+    # project's name to the actual project object.
     #
-    # @param overrides [Hash] a hash of version override information.
-    # @param software_files [Array<String>]
-    # @return [void]
+    # @return [Hash<String, Project>]
     #
-    # @see Omnibus::Overrides#overrides
-    def expand_software(overrides, software_map)
-      unless overrides.is_a? Hash
-        raise ArgumentError, "Overrides argument must be a hash!  You passed #{overrides.inspect}."
-      end
+    def _projects
+      return @_projects if @_projects
 
-      Omnibus.projects.each do |project|
-        project.dependencies.each do |dependency|
-          recursively_load_dependency(dependency, project, overrides, software_map)
+      path = File.expand_path(Config.project_dir, Config.project_root)
+      @_projects = Dir.glob("#{path}/*.rb").inject({}) do |hash, path|
+        name = File.basename(path, '.rb')
+
+        if hash[name].nil?
+          Omnibus.logger.info('Omnibus#projects') do
+            "Using project `#{name}' from `#{path}'."
+          end
+
+          hash[name] = Project.load(path)
+        else
+          Omnibus.logger.debug('Omnibus#projects') do
+            "Skipping project `#{name}' because it was already loaded."
+          end
         end
+
+        hash
       end
+
+      @_projects
     end
 
-    # Return a list of all the Ruby files (i.e., those with an "rb"
-    # extension) in the given directory
     #
-    # @param dir [String]
+    # The list of all software paths to software from the project root. This is
+    # always a single value, but an array is returned for consistency with the
+    # other +software_paths_*+ methods.
+    #
+    # @see (Config#project_root)
+    # @see (Config#software_dir)
+    #
     # @return [Array<String>]
-    def ruby_files(dir)
-      Dir.glob("#{dir}/*.rb")
+    #
+    def paths_from_project_root
+      [Config.project_root]
     end
 
-    # Retrieve the fully-qualified paths to every software definition
-    # file bundled in the {https://github.com/opscode/omnibus-software omnibus-software} gem.
     #
-    # @return [Array<String>] the list of paths. Will be empty if the
-    #   `omnibus-software` gem is not in the gem path.
-    def omnibus_software_files
-      if omnibus_software_root
-        Dir.glob(File.join(omnibus_software_root, 'config', 'software', '*.rb'))
-      else
-        []
+    # The list of all software paths on disk to software files. If relative
+    # paths are given, they are expanded relative to {Config#project_root}.
+    #
+    # @see (Config#local_software_dirs)
+    #
+    # @return [Array<String>]
+    #
+    def paths_from_local_software_dirs
+      Array(Config.local_software_dirs).inject([]) do |array, path|
+        fullpath = File.expand_path(path, Config.project_root)
+
+        if File.directory?(fullpath)
+          array << fullpath
+        else
+          Omnibus.logger.warn('Omnibus') do
+            "Could not load softwares from path `#{fullpath}'. Does it exist?"
+          end
+        end
+
+        array
       end
     end
 
-    # Given a list of software definitions from `omnibus-software` itself, and a
-    # list of software files local to the current project, create a
-    # single list of software definitions.  If the software was defined
-    # in both sets, the locally-defined one ends up in the final list.
     #
-    # The base name of the software file determines what software it
-    # defines.
+    # The list of software paths from within the list of gems. These gems paths
+    # are loaded from disk using +Gem::Specification+. The latest version of
+    # the gem on disk is loaded. For this reason, it is recommended that you
+    # add these gems to your bundle and be nice to your co-workers.
+    #
+    # @see (Config#software_gems)
     #
-    # @param omnibus_files [Array<String>]
-    # @param local_files [Array<String>]
     # @return [Array<String>]
-    def prefer_local_software(omnibus_files, local_files)
-      base = software_map(omnibus_files)
-      local = software_map(local_files)
-      base.merge(local)
+    #
+    def paths_from_software_gems
+      Array(Config.software_gems).inject([]) do |array, name|
+        if (spec = Gem::Specification.find_all_by_name(name).first)
+          array << File.expand_path(spec.gem_dir)
+        else
+          Omnibus.logger.warn('Omnibus') do
+            "Could not load softwares from gem `#{name}'. Is it installed?"
+          end
+        end
+
+        array
+      end
     end
 
-    # Given a list of file paths, create a map of the basename (without
-    # extension) to the complete path.
     #
-    # @param files [Array<String>]
-    # @return [Hash<String, String>]
-    def software_map(files)
-      files.each_with_object({}) do |file, collection|
-        software_name = File.basename(file, '.*')
-        collection[software_name] = file
+    # Generate {Software} objects for all software DSL files in
+    # +software_specs+.
+    #
+    # @return [void]
+    #
+    def expand_software
+      Omnibus.projects.each do |project|
+        project.dependencies.each do |dependency|
+          recursively_load_dependency(dependency, project)
+        end
       end
     end
 
-    # Loads a project's dependency recursively, ensuring all transitive dependencies
-    # are also loaded.
     #
-    # @param dependency_name [String]
-    # @param project [Omnibus::Project]
-    # @param overrides [Hash] a hash of version override information.
-    # @param software_map [Hash<String, String>]
+    # Loads a project's dependencies recursively, ensuring all transitive
+    # dependencies are also loaded in the correct order.
+    #
+    # @param [String] dependency
+    #   the name of the dependency
+    # @param [Project] project
+    #   the project that loaded the software
     #
     # @return [void]
-    def recursively_load_dependency(dependency_name, project, overrides, software_map)
-      dep_file = software_map[dependency_name]
+    #
+    def recursively_load_dependency(dependency, project)
+      filepath = software_map[dependency]
 
-      unless dep_file
-        raise MissingProjectDependency.new(dependency_name, software_dirs)
+      if filepath.nil?
+        raise MissingProjectDependency.new(dependency, software_dirs)
       end
 
-      dep_software = Omnibus::Software.load(dep_file, project, overrides)
+      software = Software.load(project, filepath)
 
       # load any transitive deps for the component into the library also
-      dep_software.dependencies.each do |dep|
-        recursively_load_dependency(dep, project, overrides, software_map)
+      software.dependencies.each do |transitive_dependency|
+        recursively_load_dependency(transitive_dependency, project)
       end
 
-      project.library.component_added(dep_software)
+      project.library.component_added(software)
     end
   end
 end
-
-# Sugars must be loaded after everything else has been registered
-require 'omnibus/sugar'