bundler 0.5.0 → 0.6.0

data/README.markdown

@@ -12,12 +12,6 @@ and all child dependencies specified in this manifest. It can manage any update
 to the gem manifest file and update the bundled gems accordingly. It also lets
 you run any ruby code in context of the bundled gem environment.
 
-## Disclaimer
-
-This project is under rapid development. It is usable today, but there will be
-many changes in the near future, including to the Gemfile DSL. We will bump up
-versions with changes though. We greatly appreciate feedback.
-
 ## Installation
 
 Bundler has no dependencies. Just clone the git repository and install the gem
@@ -25,6 +19,10 @@ with the following rake task:
 
     rake install
 
+You can also install the gem with
+
+    gem install bundler
+
 ## Usage
 
 Bundler requires a gem manifest file to be created. This should be a file named
@@ -50,11 +48,30 @@ information, please refer to Bundler::ManifestBuilder.
     # it can be specified the same way as with rubygems' #gem method.
     gem "rack", "1.0.0"
 
-    # Specify a dependency rspec, but only activate that gem in the "testing"
-    # environment (read more about environments later). :except is also a valid
-    # option to specify environment restrictions.
+    # Specify a dependency rspec, but only require that gem in the "testing"
+    # environment. :except is also a valid option to specify environment
+    # restrictions.
     gem "rspec", :only => :testing
 
+    # Specify a dependency, but specify that it is already present and expanded
+    # at vendor/rspec. Bundler will treat rspec as though it was the rspec gem
+    # for the purpose of gem resolution: if another gem depends on a version
+    # of rspec satisfied by "1.1.6", it will be used.
+    #
+    # If a gemspec is found in the directory, it will be used to specify load
+    # paths and supply additional dependencies.
+    #
+    # Bundler will also recursively search for *.gemspec, and assume that
+    # gemspecs it finds represent gems that are rooted in the same directory
+    # the gemspec is found in.
+    gem "rspec", "1.1.6", :vendored_at => "vendor/rspec"
+
+    # Works exactly like :vendored_at, but first downloads the repo from
+    # git and handles stashing the files for you. As with :vendored_at,
+    # Bundler will automatically use *.gemspec files in the root or anywhere
+    # in the repository.
+    gem "rails", "3.0.pre", :git => "git://github.com/rails/rails.git"
+
     # Add http://gems.github.com as a source that the bundler will use
     # to find gems listed in the manifest. By default,
     # http://gems.rubyforge.org is already added to the list.
@@ -84,11 +101,43 @@ information, please refer to Bundler::ManifestBuilder.
     # the ones that have been bundled.
     disable_rubygems
 
+### Gem Resolution
+
+One of the most important things that the bundler does is do a
+dependency resolution on the full list of gems that you specify, all
+at once. This differs from the one-at-a-time dependency resolution that
+Rubygems does, which can result in the following problem:
+
+    # On my system:
+    #   activesupport 3.0.pre
+    #   activesupport 2.3.4
+    #   activemerchant 1.4.2
+    #   rails 2.3.4
+    #
+    # activemerchant 1.4.2 depends on activesupport >= 2.3.2
+
+    gem "activemerchant", "1.4.2"
+    # results in activating activemerchant, as well as
+    # activesupport 3.0.pre, since it is >= 2.3.2
+
+    gem "rails", "2.3.4"
+    # results in:
+    #   can't activate activesupport (= 2.3.4, runtime)
+    #   for ["rails-2.3.4"], already activated
+    #   activesupport-3.0.pre for ["activemerchant-1.4.2"]
+
+This is because activemerchant has a broader dependency, which results
+in the activation of a version of activesupport that does not satisfy
+a more narrow dependency.
+
+Bundler solves this problem by evaluating all dependencies at once,
+so it can detect that all gems *together* require activesupport "2.3.4".
+
 ### Running Bundler
 
 Once a manifest file has been created, the only thing that needs to be done
 is to run the `gem bundle` command anywhere in your application. The script
-will load the manifest file, resole all the dependencies, download all
+will load the manifest file, resolve all the dependencies, download all
 needed gems, and install them into the specified directory.
 
 Every time an update is made to the manifest file, run `gem bundle` again to
@@ -96,6 +145,41 @@ get the changes installed. This will only check the remote sources if your
 currently installed gems do not satisfy the `Gemfile`. If you want to force
 checking for updates on the remote sources, use the `--update` option.
 
+### Remote deploys
+
+When you run `gem bundle`, the following steps occur:
+
+1. Gemfile is read in
+2. The gems specified in the Gemfile are resolved against the gems
+   already in your bundle. If the dependencies resolve, skip to step 5.
+3. If the dependencies in your Gemfile cannot be fully resolved
+   against the gems already in the bundle, the metadata for each
+   source is fetched.
+4. The gems in the Gemfile are resolved against the full list of
+   available gems in all sources, and the resulting gems are downloaded
+5. Each gem that has been downloaded but not yet expanded is expanded
+   into the local directory. This expansion process also installs
+   native gems.
+
+As you can see, if you run gem bundle twice in a row, it will do nothing the
+second time, since the gems obviously resolve against the installed gems,
+and they are all expanded.
+
+This also means that if you run `gem bundle`, and .gitignore the expanded
+copies, leaving only the cached `.gem` files, you can run `gem bundle` again
+on the remote system, and it will only expand out the gems (but not
+resolve or download `.gem` files). This also means that native gems
+will be compiled for the target platform without requiring that the
+`.gem` file itself be downloaded from a remote gem server.
+
+Assuming a Rails app with Bundler's standard setup, add something like
+this to your top-level `.gitignore` to only keep the cache:
+
+    vendor/gems/
+    !vendor/gems/cache/
+
+Make sure that you explicitly `git add vendor/gems/cache` before you commit.
+
 ### Running your application
 
 The easiest way to run your application is to start it with an executable
@@ -108,6 +192,9 @@ run it with the `gem exec` command. For example:
 
     gem exec ruby my_ruby_script.rb
 
+You can use `gem exec bash` to enter a shell that will run all binaries in
+the current context.
+
 Yet another way is to manually require the environment file first. This is
 located in `[bundle_path]/environments/default.rb`. For example:
 
@@ -133,9 +220,9 @@ to follow.
 * You can now use rails if you prepend `gem exec` to every call to `script/*`
   but that isn't fun.
 
-* At the top of `config/boot.rb`, add the following line:
+* At the top of `config/preinitializer.rb`, add the following line:
 
-        require File.expand_path(File.join(File.dirname(__FILE__), '..', 'vendor', 'gems', 'environment'))
+    require "#{RAILS_ROOT}/vendor/gems/environment"
 
 In theory, this should be enough to get going.
 
@@ -143,12 +230,17 @@ In theory, this should be enough to get going.
 
 Ideally, no gem would assume the presence of rubygems at runtime. Rubygems provides
 enough features so that this isn't necessary. However, there are a number of gems
-that require specific rubygem features.
+that require specific rubygems features.
 
 If the `disable_rubygems` option is used, Bundler will stub out the most common
 of these features, but it is possible that things will not go as intended quite
 yet. So, if you are brave, try your code without rubygems at runtime.
 
+This is different from the `disable_system_gems` option, which uses the rubygems
+library, but prevents system gems from being loaded; only gems that are bundled
+will be available to your application. This option guarantees that dependencies
+of your application will be available to a remote system.
+
 ## Known Issues
 
 * When a gem points to a git repository, the git repository will be cloned
@@ -159,4 +251,4 @@ yet. So, if you are brave, try your code without rubygems at runtime.
 Please report all bugs on the github issue tracker for the project located
 at:
 
-    http://github.com/wycats/bundler/issues/
+    http://github.com/wycats/bundler/issues/

data/Rakefile

@@ -1,12 +1,14 @@
+$:.unshift File.join(File.dirname(__FILE__), 'lib')
 require 'rubygems' unless ENV['NO_RUBYGEMS']
 require 'rubygems/specification'
+require 'bundler'
 require 'date'
 
 spec = Gem::Specification.new do |s|
   s.name    = "bundler"
-  s.version = "0.5.0"
-  s.author  = "Yehuda Katz"
-  s.email   = "wycats@gmail.com"
+  s.version = Bundler::VERSION
+  s.authors = ["Yehuda Katz", "Carl Lerche"]
+  s.email   = ["wycats@gmail.com", "clerche@engineyard.com"]
   s.homepage = "http://github.com/wycats/bundler"
   s.description = s.summary = "An easy way to vendor gem dependencies"
 
@@ -31,6 +33,7 @@ else
   Spec::Rake::SpecTask.new do |t|
     t.spec_files = FileList['spec/**/*_spec.rb'] - FileList['spec/fixtures/**/*_spec.rb']
     t.spec_opts = %w(-fs --color)
+    t.warning = true
   end
 end
 

data/lib/bundler.rb

@@ -16,9 +16,10 @@ require "bundler/dsl"
 require "bundler/cli"
 require "bundler/repository"
 require "bundler/dependency"
+require "bundler/remote_specification"
 
 module Bundler
-  VERSION = "0.5.0"
+  VERSION = "0.6.0"
 
   class << self
     attr_writer :logger

data/lib/bundler/cli.rb

@@ -6,19 +6,25 @@ module Bundler
       new(options).run(command)
     rescue DefaultManifestNotFound => e
       Bundler.logger.error "Could not find a Gemfile to use"
-      exit 2
+      exit 3
     rescue InvalidEnvironmentName => e
       Bundler.logger.error "Gemfile error: #{e.message}"
-      exit
+      exit 4
     rescue InvalidRepository => e
       Bundler.logger.error e.message
-      exit
+      exit 5
     rescue VersionConflict => e
       Bundler.logger.error e.message
-      exit
+      exit 6
     rescue GemNotFound => e
       Bundler.logger.error e.message
-      exit
+      exit 7
+    rescue InvalidCacheArgument => e
+      Bundler.logger.error e.message
+      exit 8
+    rescue SourceNotCached => e
+      Bundler.logger.error e.message
+      exit 9
     end
 
     def initialize(options)
@@ -27,7 +33,19 @@ module Bundler
     end
 
     def bundle
-      @manifest.install(@options[:update])
+      @manifest.install(@options)
+    end
+
+    def cache
+      @manifest.cache(@options)
+    end
+
+    def prune
+      @manifest.prune(@options)
+    end
+
+    def list
+      @manifest.list(@options)
     end
 
     def exec

data/lib/bundler/commands/bundle_command.rb

@@ -10,6 +10,22 @@ class Gem::Commands::BundleCommand < Gem::Command
     add_option('-u', '--update', "Force a remote check for newer gems") do
       options[:update] = true
     end
+
+    add_option('--cached', "Only use cached gems when expanding the bundle") do
+      options[:cached] = true
+    end
+
+    add_option('--cache GEM', "Specify a path to a .gem file to add to the bundled gem cache") do |gem, options|
+      options[:cache] = gem
+    end
+
+    add_option('--prune-cache', "Removes all .gem files from the bundle's cache") do
+      options[:prune] = true
+    end
+
+    add_option('--list', "List all gems that are part of the active bundle") do
+      options[:list] = true
+    end
   end
 
   def usage
@@ -25,7 +41,15 @@ Bundle stuff
   def execute
     # Prevent the bundler from getting required unless it is actually being used
     require 'bundler'
-    Bundler::CLI.run(:bundle, options)
+    if options[:cache]
+      Bundler::CLI.run(:cache, options)
+    elsif options[:prune]
+      Bundler::CLI.run(:prune, options)
+    elsif options[:list]
+      Bundler::CLI.run(:list, options)
+    else
+      Bundler::CLI.run(:bundle, options)
+    end
   end
 
 end

data/lib/bundler/dsl.rb

@@ -5,6 +5,7 @@ module Bundler
     def initialize(environment)
       @environment = environment
       @sources = Hash.new { |h,k| h[k] = {} }
+      @only, @except = nil, nil
     end
 
     def bundle_path(path)

data/lib/bundler/environment.rb

@@ -2,13 +2,16 @@ require "rubygems/source_index"
 
 module Bundler
   class DefaultManifestNotFound < StandardError; end
+  class InvalidCacheArgument < StandardError; end
+  class SourceNotCached < StandardError; end
 
   class Environment
     attr_reader :filename, :dependencies
-    attr_accessor :rubygems, :system_gems, :gem_path, :bindir
+    attr_accessor :rubygems, :system_gems
+    attr_writer :gem_path, :bindir
 
     def self.load(gemfile = nil)
-      gemfile = gemfile ? Pathname.new(gemfile) : default_manifest_file
+      gemfile = gemfile ? Pathname.new(gemfile).expand_path : default_manifest_file
 
       unless gemfile.file?
         raise ManifestFileNotFound, "#{filename.inspect} does not exist"
@@ -31,7 +34,7 @@ module Bundler
 
     def initialize(filename) #, sources, dependencies, bindir, path, rubygems, system_gems)
       @filename         = filename
-      @default_sources  = [GemSource.new(:uri => "http://gems.rubyforge.org")]
+      @default_sources  = [GemSource.new(:uri => "http://gems.rubyforge.org"), SystemGemSource.new({})]
       @sources          = []
       @priority_sources = []
       @dependencies     = []
@@ -43,23 +46,61 @@ module Bundler
       builder.instance_eval(File.read(filename))
     end
 
-    def install(update = false)
-      begin
-        tmp_path = filename.dirname.join(".tmp")
-        FileUtils.mkdir_p(tmp_path)
-        sources.each { |s| s.tmp_path = tmp_path }
-        repository.install(gem_dependencies, sources,
-          :rubygems    => rubygems,
-          :system_gems => system_gems,
-          :manifest    => filename,
-          :update      => update
-        )
-      ensure
-        FileUtils.rm_rf(tmp_path)
-      end
+    def install(options = {})
+      update = options[:update]
+      cached = options[:cached]
+
+      repository.install(gem_dependencies, sources,
+        :rubygems    => rubygems,
+        :system_gems => system_gems,
+        :manifest    => filename,
+        :update      => update,
+        :cached      => cached
+      )
       Bundler.logger.info "Done."
     end
 
+    def cache(options = {})
+      gemfile = options[:cache]
+
+      if File.extname(gemfile) == ".gem"
+        if !File.exist?(gemfile)
+          raise InvalidCacheArgument, "'#{gemfile}' does not exist."
+        end
+        repository.cache(gemfile)
+      elsif File.directory?(gemfile) || gemfile.include?('/')
+        if !File.directory?(gemfile)
+          raise InvalidCacheArgument, "'#{gemfile}' does not exist."
+        end
+        gemfiles = Dir["#{gemfile}/*.gem"]
+        if gemfiles.empty?
+          raise InvalidCacheArgument, "'#{gemfile}' contains no gemfiles"
+        end
+        repository.cache(*gemfiles)
+      else
+        local = Gem::SourceIndex.from_installed_gems.find_name(gemfile).last
+
+        if !local
+          raise InvalidCacheArgument, "w0t? '#{gemfile}' means nothing to me."
+        end
+
+        gemfile = Pathname.new(local.loaded_from)
+        gemfile = gemfile.dirname.join('..', 'cache', "#{local.full_name}.gem").expand_path
+        repository.cache(gemfile)
+      end
+    end
+
+    def prune(options = {})
+      repository.prune(gem_dependencies, sources)
+    end
+
+    def list(options = {})
+      Bundler.logger.info "Currently bundled gems:"
+      repository.gems.each do |spec|
+        Bundler.logger.info " * #{spec.name} (#{spec.version})"
+      end
+    end
+
     def setup_environment
       unless system_gems
         ENV["GEM_HOME"] = gem_path

data/lib/bundler/gem_bundle.rb

@@ -1,8 +1,8 @@
 module Bundler
   class GemBundle < Array
-    def download(repository)
+    def download
       sort_by {|s| s.full_name.downcase }.each do |spec|
-        spec.source.download(spec, repository)
+        spec.source.download(spec)
       end
 
       self

data/lib/bundler/gem_ext.rb

@@ -1,5 +1,7 @@
 module Gem
   class Installer
+    remove_method(:app_script_text) if method_defined?(:app_script_text)
+
     def app_script_text(bin_file_name)
       path = @gem_home
       template = File.read(File.join(File.dirname(__FILE__), "templates", "app_script.erb"))
@@ -12,6 +14,8 @@ module Gem
     attr_accessor :source
     attr_accessor :location
 
+    remove_method(:specification_version) if method_defined?(:specification_version)
+
     # Hack to fix github's strange marshal file
     def specification_version
       @specification_version && @specification_version.to_i
@@ -19,7 +23,11 @@ module Gem
 
     alias full_gem_path_without_location full_gem_path
     def full_gem_path
-      @location ? @location : full_gem_path_without_location
+      if defined?(@location) && @location
+        @location
+      else
+        full_gem_path_without_location
+      end
     end
   end
 end

data/lib/bundler/remote_specification.rb

@@ -0,0 +1,50 @@
+module Bundler
+  # Represents a lazily loaded gem specification, where the full specification
+  # is on the source server in rubygems' "quick" index. The proxy object is to
+  # be seeded with what we're given from the source's abbreviated index - the
+  # full specification will only be fetched when necesary.
+  class RemoteSpecification
+    attr_reader :name, :version, :platform
+    attr_accessor :source
+
+    def initialize(name, version, platform, source_uri)
+      @name     = name
+      @version  = version
+      @platform = platform
+      @source_uri = source_uri
+    end
+
+    def full_name
+      if platform == Gem::Platform::RUBY or platform.nil? then
+        "#{@name}-#{@version}"
+      else
+        "#{@name}-#{@version}-#{platform}"
+      end
+    end
+
+    # Because Rubyforge cannot be trusted to provide valid specifications
+    # once the remote gem is donwloaded, the backend specification will
+    # be swapped out.
+    def __swap__(spec)
+      @specification = spec
+    end
+
+    private
+
+    def _remote_uri
+      "#{@source_uri}/quick/Marshal.4.8/#{@name}-#{@version}.gemspec.rz"
+    end
+
+    def _remote_specification
+      @specification ||= begin
+        deflated = Gem::RemoteFetcher.fetcher.fetch_path(_remote_uri)
+        inflated = Gem.inflate(deflated)
+        Marshal.load(inflated)
+      end
+    end
+
+    def method_missing(method, *args, &blk)
+      _remote_specification.send(method, *args, &blk)
+    end
+  end
+end

data/lib/bundler/repository.rb

@@ -1,6 +1,3 @@
-require "bundler/repository/gem_repository"
-require "bundler/repository/directory_repository"
-
 module Bundler
   class InvalidRepository < StandardError ; end
 
@@ -13,51 +10,67 @@ module Bundler
       @path   = Pathname.new(path)
       @bindir = Pathname.new(bindir)
 
-      @repos = {
-        :gem       => Gems.new(@path, @bindir),
-        :directory => Directory.new(@path.join("dirs"), @bindir)
-      }
+      @cache = GemDirectorySource.new(:location => @path.join("cache"))
     end
 
     def install(dependencies, sources, options = {})
-      if options[:update] || !satisfies?(dependencies)
-        fetch(dependencies, sources)
-        expand(options)
-      else
-        # Remove any gems that are still around if the Gemfile changed without
-        # requiring new gems to be download (e.g. a line in the Gemfile was
-        # removed)
-        cleanup(Resolver.resolve(dependencies, [source_index]))
+      # TODO: clean this up
+      sources.each do |s|
+        s.repository = self
+        s.local = options[:cached]
       end
-      configure(options)
-      sync
-    end
 
-    def gems
-      gems = []
-      each_repo do |repo|
-        gems.concat repo.gems
+      begin
+        valid = Resolver.resolve(dependencies, [source_index])
+      rescue Bundler::GemNotFound
       end
-      gems
+
+      if options[:cached]
+        sources = sources.select { |s| s.can_be_local? }
+      end
+
+      if options[:update] || !valid
+        Bundler.logger.info "Calculating dependencies..."
+        bundle = Resolver.resolve(dependencies, [@cache] + sources)
+        do_install(bundle, options)
+        valid = bundle
+      end
+      cleanup(valid)
+      configure(valid, options)
     end
 
-    def satisfies?(dependencies)
-      index = source_index
-      dependencies.all? { |dep| index.search(dep).size > 0 }
+    def cache(*gemfiles)
+      FileUtils.mkdir_p(@path.join("cache"))
+      gemfiles.each do |gemfile|
+        Bundler.logger.info "Caching: #{File.basename(gemfile)}"
+        FileUtils.cp(gemfile, @path.join("cache"))
+      end
     end
 
-    def source_index
-      index = Gem::SourceIndex.new
+    def prune(dependencies, sources)
+      sources.each do |s|
+        s.repository = self
+        s.local = true
+      end
 
-      each_repo do |repo|
-        index.gems.merge!(repo.source_index.gems)
+      sources = sources.select { |s| s.can_be_local? }
+      bundle = Resolver.resolve(dependencies, [@cache] + sources)
+      @cache.gems.each do |name, spec|
+        unless bundle.any? { |s| s.name == spec.name && s.version == spec.version }
+          Bundler.logger.info "Pruning #{spec.name} (#{spec.version}) from the cache"
+          FileUtils.rm @path.join("cache", "#{spec.full_name}.gem")
+        end
       end
+    end
 
-      index
+    def gems
+      source_index.gems.values
     end
 
-    def add_spec(type, spec)
-      @repos[type].add_spec(spec)
+    def source_index
+      index = Gem::SourceIndex.from_gems_in(@path.join("specifications"))
+      index.each { |n, spec| spec.loaded_from = @path.join("specifications", "#{spec.full_name}.gemspec") }
+      index
     end
 
     def download_path_for(type)
@@ -66,34 +79,84 @@ module Bundler
 
   private
 
-    def cleanup(bundle)
-      each_repo do |repo|
-        repo.cleanup(bundle)
+    def do_install(bundle, options)
+      bundle.download
+
+      bundle.each do |spec|
+        spec.loaded_from = @path.join("specifications", "#{spec.full_name}.gemspec")
+        # Do nothing if the gem is already expanded
+        next if @path.join("gems", spec.full_name).directory?
+
+        case spec.source
+        when GemSource, GemDirectorySource, SystemGemSource
+          expand_gemfile(spec, options)
+        else
+          expand_vendored_gem(spec, options)
+        end
       end
     end
 
-    def each_repo
-      @repos.each do |k, repo|
-        yield repo
+    def expand_gemfile(spec, options)
+      Bundler.logger.info "Installing #{spec.name} (#{spec.version})"
+
+      gemfile = @path.join("cache", "#{spec.full_name}.gem").to_s
+
+      installer = Gem::Installer.new(gemfile, options.merge(
+        :install_dir         => @path,
+        :ignore_dependencies => true,
+        :env_shebang         => true,
+        :wrappers            => true,
+        :bin_dir             => @bindir
+      ))
+      installer.install
+    end
+
+    def expand_vendored_gem(spec, options)
+      add_spec(spec)
+      FileUtils.mkdir_p(@path.join("gems"))
+      File.symlink(spec.location, @path.join("gems", spec.full_name))
+
+      # HAX -- Generate the bin
+      bin_dir = @bindir
+      path    = @path
+      installer = Gem::Installer.allocate
+      installer.instance_eval do
+        @spec     = spec
+        @bin_dir  = bin_dir
+        @gem_dir  = path.join("gems", "#{spec.full_name}")
+        @gem_home = path
+        @wrappers = true
       end
+      installer.generate_bin
     end
 
-    def fetch(dependencies, sources)
-      bundle = Resolver.resolve(dependencies, sources)
-      # Cleanup here to remove any gems that could cause problem in the expansion
-      # phase
-      #
-      # TODO: Try to avoid double cleanup
-      cleanup(bundle)
-      bundle.download(self)
+    def add_spec(spec)
+      destination = path.join('specifications')
+      destination.mkdir unless destination.exist?
+
+      File.open(destination.join("#{spec.full_name}.gemspec"), 'w') do |f|
+        f.puts spec.to_ruby
+      end
     end
 
-    def sync
-      glob = gems.map { |g| g.executables }.flatten.join(',')
+    def cleanup(valid)
+      to_delete = gems
+      to_delete.delete_if do |spec|
+        valid.any? { |other| spec.name == other.name && spec.version == other.version }
+      end
 
-      (Dir[@bindir.join("*")] - Dir[@bindir.join("{#{glob}}")]).each do |file|
-        Bundler.logger.info "Deleting bin file: #{File.basename(file)}"
-        FileUtils.rm_rf(file)
+      valid_executables = valid.map { |s| s.executables }.flatten.compact
+
+      to_delete.each do |spec|
+        Bundler.logger.info "Deleting gem: #{spec.name} (#{spec.version})"
+        FileUtils.rm_rf(@path.join("specifications", "#{spec.full_name}.gemspec"))
+        FileUtils.rm_rf(@path.join("gems", spec.full_name))
+        # Cleanup the bin directory
+        spec.executables.each do |bin|
+          next if valid_executables.include?(bin)
+          Bundler.logger.info "Deleting bin file: #{bin}"
+          FileUtils.rm_rf(@bindir.join(bin))
+        end
       end
     end
 
@@ -103,14 +166,13 @@ module Bundler
       end
     end
 
-    def configure(options)
-      generate_environment(options)
+    def configure(specs, options)
+      generate_environment(specs, options)
     end
 
-    def generate_environment(options)
+    def generate_environment(specs, options)
       FileUtils.mkdir_p(path)
 
-      specs      = gems
       load_paths = load_paths_for_specs(specs)
       bindir     = @bindir.relative_path_from(path).to_s
       filename   = options[:manifest].relative_path_from(path).to_s
@@ -148,4 +210,4 @@ module Bundler
       "require #{file.inspect}#{constraint}"
     end
   end
-end
+end

data/lib/bundler/resolver.rb

@@ -37,8 +37,6 @@ module Bundler
     # <GemBundle>,nil:: If the list of dependencies can be resolved, a
     #   collection of gemspecs is returned. Otherwise, nil is returned.
     def self.resolve(requirements, sources)
-      Bundler.logger.info "Calculating dependencies..."
-
       resolver = new(sources)
       result = catch(:success) do
         resolver.resolve(requirements, {})
@@ -181,7 +179,9 @@ module Bundler
     def search(dependency)
       @cache[dependency.hash] ||= begin
         @specs[dependency.name].values.select do |spec|
-          dependency =~ spec
+          match = dependency =~ spec
+          match &= dependency.version_requirements.prerelease? if spec.version.prerelease?
+          match
         end.sort_by {|s| s.version }
       end
     end

data/lib/bundler/source.rb

@@ -2,7 +2,7 @@ module Bundler
   # Represents a source of rubygems. Initially, this is only gem repositories, but
   # eventually, this will be git, svn, HTTP
   class Source
-    attr_accessor :tmp_path
+    attr_accessor :repository, :local
   end
 
   class GemSource < Source
@@ -14,6 +14,10 @@ module Bundler
       raise ArgumentError, "The source must be an absolute URI" unless @uri.absolute?
     end
 
+    def can_be_local?
+      false
+    end
+
     def gems
       @specs ||= fetch_specs
     end
@@ -28,16 +32,22 @@ module Bundler
 
     class RubygemsRetardation < StandardError; end
 
-    def download(spec, repository)
+    def download(spec)
       Bundler.logger.info "Downloading #{spec.full_name}.gem"
 
-      destination = repository.download_path_for(:gem)
+      destination = repository.path
 
       unless destination.writable?
         raise RubygemsRetardation
       end
 
-      Gem::RemoteFetcher.fetcher.download(spec, uri, repository.download_path_for(:gem))
+      # Download the gem
+      Gem::RemoteFetcher.fetcher.download(spec, uri, destination)
+
+      # Re-read the gemspec from the downloaded gem to correct
+      # any errors that were present in the Rubyforge specification.
+      new_spec = Gem::Format.from_file_by_path(destination.join('cache', "#{spec.full_name}.gem")).spec
+      spec.__swap__(new_spec)
     end
 
   private
@@ -45,17 +55,109 @@ module Bundler
     def fetch_specs
       Bundler.logger.info "Updating source: #{to_s}"
 
-      deflated = Gem::RemoteFetcher.fetcher.fetch_path("#{uri}/Marshal.4.8.Z")
-      inflated = Gem.inflate deflated
+      fetcher = Gem::RemoteFetcher.fetcher
+      main_index = fetcher.fetch_path("#{uri}/specs.4.8.gz")
+      begin
+        prerelease_index = fetcher.fetch_path("#{uri}/prerelease_specs.4.8.gz")
+        index = Marshal.load(main_index) + Marshal.load(prerelease_index)
+      rescue Gem::RemoteFetcher::FetchError
+        Bundler.logger.warn "Source '#{uri}' does not support prerelease gems"
+        index = Marshal.load(main_index)
+      end
 
-      index = Marshal.load(inflated)
-      index.gems
+      gems = {}
+      index.each do |name, version, platform|
+        spec = RemoteSpecification.new(name, version, platform, @uri)
+        gems[spec.full_name] = spec
+      end
+      gems
     rescue Gem::RemoteFetcher::FetchError => e
       raise ArgumentError, "#{to_s} is not a valid source: #{e.message}"
     end
   end
 
+  class SystemGemSource < Source
+    def initialize(options)
+      # Nothing to do
+    end
+
+    def can_be_local?
+      false
+    end
+
+    def gems
+      @specs ||= Gem::SourceIndex.from_installed_gems.gems
+    end
+
+    def ==(other)
+      other.is_a?(SystemGemSource)
+    end
+
+    def to_s
+      "system"
+    end
+
+    def download(spec)
+      # gemfile = Pathname.new(local.loaded_from)
+      # gemfile = gemfile.dirname.join('..', 'cache', "#{local.full_name}.gem").expand_path
+      # repository.cache(File.join(Gem.dir, "cache", "#{local.full_name}.gem"))
+      gemfile = Pathname.new(spec.loaded_from)
+      gemfile = gemfile.dirname.join('..', 'cache', "#{spec.full_name}.gem")
+      repository.cache(gemfile)
+    end
+
+  private
+
+    def fetch_specs
+
+    end
+
+  end
+
+  class GemDirectorySource < Source
+    attr_reader :location
+
+    def initialize(options)
+      @location = options[:location]
+    end
+
+    def can_be_local?
+      true
+    end
+
+    def gems
+      @specs ||= fetch_specs
+    end
+
+    def ==(other)
+      location == other.location
+    end
+
+    def to_s
+      location.to_s
+    end
+
+    def download(spec)
+      # raise NotImplementedError
+    end
+
+  private
+
+    def fetch_specs
+      specs = {}
+
+      Dir["#{@location}/*.gem"].each do |gemfile|
+        spec = Gem::Format.from_file_by_path(gemfile).spec
+        specs[spec.full_name] = spec
+      end
+
+      specs
+    end
+  end
+
   class DirectorySource < Source
+    attr_reader :location
+
     def initialize(options)
       @name          = options[:name]
       @version       = options[:version]
@@ -63,32 +165,38 @@ module Bundler
       @require_paths = options[:require_paths] || %w(lib)
     end
 
+    def can_be_local?
+      true
+    end
+
     def gems
       @gems ||= begin
         specs = {}
 
         # Find any gemspec files in the directory and load those specs
-        Dir[@location.join('**', '*.gemspec')].each do |file|
-          path = Pathname.new(file).relative_path_from(@location).dirname
-          spec = eval(File.read(file))
-          spec.require_paths.map! { |p| path.join(p) }
-          specs[spec.full_name] = spec
+        Dir["#{location}/**/*.gemspec"].each do |file|
+          file = Pathname.new(file)
+          if spec = eval(File.read(file)) and validate_gemspec(file, spec)
+            spec.location = file.dirname.expand_path
+            specs[spec.full_name] = spec
+          end
         end
 
         # If a gemspec for the dependency was not found, add it to the list
         if specs.keys.grep(/^#{Regexp.escape(@name)}/).empty?
           case
           when @version.nil?
-            raise ArgumentError, "If you use :at, you must specify the gem" \
+            raise ArgumentError, "If you use :at, you must specify the gem " \
               "and version you wish to stand in for"
           when !Gem::Version.correct?(@version)
-            raise ArgumentError, "If you use :at, you must specify a gem and" \
+            raise ArgumentError, "If you use :at, you must specify a gem and " \
               "version. You specified #{@version} for the version"
           end
 
           default = Gem::Specification.new do |s|
-            s.name = @name
-            s.version = Gem::Version.new(@version) if @version
+            s.name     = @name
+            s.version  = Gem::Version.new(@version) if @version
+            s.location = location
           end
           specs[default.full_name] = default
         end
@@ -97,18 +205,52 @@ module Bundler
       end
     end
 
+    # Too aggressive apparently.
+    # ===
+    # def validate_gemspec(file, spec)
+    #   file = Pathname.new(file)
+    #   Dir.chdir(file.dirname) do
+    #     spec.validate
+    #   end
+    # rescue Gem::InvalidSpecificationException => e
+    #   file = file.relative_path_from(repository.path)
+    #   Bundler.logger.warn e.message
+    #   Bundler.logger.warn "Gemspec #{spec.name} (#{spec.version}) found at '#{file}' is not valid"
+    #   false
+    # end
+    def validate_gemspec(file, spec)
+      base = file.dirname
+      msg  = "Gemspec for #{spec.name} (#{spec.version}) is invalid:"
+      # Check the require_paths
+      (spec.require_paths || []).each do |path|
+        unless base.join(path).directory?
+          Bundler.logger.warn "#{msg} Missing require path: '#{path}'"
+          return false
+        end
+      end
+
+      # Check the executables
+      (spec.executables || []).each do |exec|
+        unless base.join(spec.bindir, exec).file?
+          Bundler.logger.warn "#{msg} Missing executable: '#{File.join(spec.bindir, exec)}'"
+          return false
+        end
+      end
+
+      true
+    end
+
     def ==(other)
       # TMP HAX
       other.is_a?(DirectorySource)
     end
 
     def to_s
-      "#{@name} (#{@version}) Located at: '#{@location}'"
+      "#{@name} (#{@version}) Located at: '#{location}'"
     end
 
-    def download(spec, repository)
-      spec.require_paths.map! { |p| File.join(@location, p) }
-      repository.add_spec(:directory, spec)
+    def download(spec)
+      # Nothing needed here
     end
   end
 
@@ -120,31 +262,35 @@ module Bundler
       @branch = options[:branch]
     end
 
+    def location
+      # TMP HAX to get the *.gemspec reading to work
+      repository.path.join('dirs', File.basename(@uri, '.git'))
+    end
+
     def gems
-      FileUtils.mkdir_p(tmp_path.join("gitz"))
+      unless location.directory?
+        # Raise an error if the source should run in local mode,
+        # but it has not been cached yet.
+        if local
+          raise SourceNotCached, "Git repository '#{@uri}' has not been cloned yet"
+        end
 
-      # TMP HAX to get the *.gemspec reading to work
-      @location = tmp_path.join("gitz", @name)
+        FileUtils.mkdir_p(location.dirname)
 
-      Bundler.logger.info "Cloning git repository at: #{@uri}"
-      `git clone #{@uri} #{@location} --no-hardlinks`
+        Bundler.logger.info "Cloning git repository at: #{@uri}"
+        `git clone #{@uri} #{location} --no-hardlinks`
 
-      if @ref
-        Dir.chdir(@location) { `git checkout #{@ref}` }
-      elsif @branch && @branch != "master"
-        Dir.chdir(@location) { `git checkout origin/#{@branch}` }
+        if @ref
+          Dir.chdir(location) { `git checkout #{@ref}` }
+        elsif @branch && @branch != "master"
+          Dir.chdir(location) { `git checkout origin/#{@branch}` }
+        end
       end
       super
     end
 
-    def download(spec, repository)
-      dest = repository.download_path_for(:directory).join(@name)
-      spec.require_paths.map! { |p| File.join(dest, p) }
-      repository.add_spec(:directory, spec)
-      if spec.name == @name
-        FileUtils.mkdir_p(dest.dirname)
-        FileUtils.mv(tmp_path.join("gitz", spec.name), dest)
-      end
+    def download(spec)
+      # Nothing needed here
     end
   end
-end
+end

data/lib/bundler/templates/environment.erb

@@ -1,13 +1,14 @@
 # DO NOT MODIFY THIS FILE
 module Bundler
-  dir = File.dirname(__FILE__)
+ file = File.expand_path(__FILE__)
+ dir = File.dirname(file)
 
 <% unless options[:system_gems] -%>
   ENV["GEM_HOME"] = dir
   ENV["GEM_PATH"] = dir
 <% end -%>
   ENV["PATH"]     = "#{dir}/<%= bindir %>:#{ENV["PATH"]}"
-  ENV["RUBYOPT"]  = "-r#{__FILE__} #{ENV["RUBYOPT"]}"
+  ENV["RUBYOPT"]  = "-r#{file} #{ENV["RUBYOPT"]}"
 
 <% load_paths.each do |load_path| -%>
   $LOAD_PATH.unshift File.expand_path("#{dir}/<%= load_path %>")
@@ -57,6 +58,7 @@ module Bundler
         only = _combine_onlys(opt[:only] || opt["only"])
         except = _combine_excepts(opt[:except] || opt["except"])
         files = opt[:require_as] || opt["require_as"] || name
+        files = [files] unless files.respond_to?(:each)
 
         return unless !only || only.any? {|e| e == @env }
         return if except && except.any? {|e| e == @env }
@@ -124,4 +126,4 @@ module Gem
   class VerificationError < Exception; end
   class SystemExitException < SystemExit; end
 end
-<% end -%>
+<% end -%>

metadata

@@ -1,20 +1,23 @@
 --- !ruby/object:Gem::Specification 
 name: bundler
 version: !ruby/object:Gem::Version 
-  version: 0.5.0
+  version: 0.6.0
 platform: ruby
 authors: 
 - Yehuda Katz
+- Carl Lerche
 autorequire: 
 bindir: bin
 cert_chain: []
 
-date: 2009-09-03 00:00:00 -07:00
+date: 2009-09-23 00:00:00 -07:00
 default_executable: 
 dependencies: []
 
 description: An easy way to vendor gem dependencies
-email: wycats@gmail.com
+email: 
+- wycats@gmail.com
+- clerche@engineyard.com
 executables: []
 
 extensions: []
@@ -35,8 +38,7 @@ files:
 - lib/bundler/finder.rb
 - lib/bundler/gem_bundle.rb
 - lib/bundler/gem_ext.rb
-- lib/bundler/repository/directory_repository.rb
-- lib/bundler/repository/gem_repository.rb
+- lib/bundler/remote_specification.rb
 - lib/bundler/repository.rb
 - lib/bundler/resolver.rb
 - lib/bundler/runtime.rb

data/lib/bundler/repository/directory_repository.rb

@@ -1,46 +0,0 @@
-module Bundler
-  class Repository
-    class Directory
-      attr_reader :path, :bindir
-
-      def initialize(path, bindir)
-        @path   = path
-        @bindir = bindir
-
-        FileUtils.mkdir_p(path.to_s)
-      end
-
-      def source_index
-        index = Gem::SourceIndex.from_gems_in(@path.join("specifications"))
-        index.each { |n, spec| spec.loaded_from = @path.join("specifications", "#{spec.full_name}.gemspec") }
-        index
-      end
-
-      def gems
-        source_index.gems.values
-      end
-
-      def add_spec(spec)
-        destination = path.join('specifications')
-        destination.mkdir unless destination.exist?
-
-        File.open(destination.join("#{spec.full_name}.gemspec"), 'w') do |f|
-          f.puts spec.to_ruby
-        end
-      end
-
-      def download_path_for
-        @path.join("dirs")
-      end
-
-      # Checks whether a gem is installed
-      def expand(options)
-        # raise NotImplementedError
-      end
-
-      def cleanup(gems)
-        # raise NotImplementedError
-      end
-    end
-  end
-end

data/lib/bundler/repository/gem_repository.rb

@@ -1,108 +0,0 @@
-module Bundler
-  class Repository
-    class Gems
-      attr_reader :path, :bindir
-
-      def initialize(path, bindir)
-        @path   = path
-        @bindir = bindir
-      end
-
-      # Returns the source index for all gems installed in the
-      # repository
-      def source_index
-        index = Gem::SourceIndex.from_gems_in(@path.join("specifications"))
-        index.each { |n, spec| spec.loaded_from = @path.join("specifications", "#{spec.full_name}.gemspec") }
-        index
-      end
-
-      def gems
-        source_index.gems.values
-      end
-
-      # Checks whether a gem is installed
-      def expand(options)
-        cached_gems.each do |name, version|
-          unless installed?(name, version)
-            install_cached_gem(name, version, options)
-          end
-        end
-      end
-
-      def cleanup(gems)
-        glob = gems.map { |g| g.full_name }.join(',')
-        base = path.join("{cache,specifications,gems}")
-
-        (Dir[base.join("*")] - Dir[base.join("{#{glob}}{.gemspec,.gem,}")]).each do |file|
-          if File.basename(file) =~ /\.gem$/
-            name = File.basename(file, '.gem')
-            Bundler.logger.info "Deleting gem: #{name}"
-          end
-          FileUtils.rm_rf(file)
-        end
-      end
-
-      def add_spec(spec)
-        raise NotImplementedError
-      end
-
-      def download_path_for
-        path
-      end
-
-    private
-
-      def cache_path
-        @path.join("cache")
-      end
-
-      def cache_files
-        Dir[cache_path.join("*.gem")]
-      end
-
-      def cached_gems
-        cache_files.map do |f|
-          full_name = File.basename(f).gsub(/\.gem$/, '')
-          full_name.split(/-(?=[^-]+$)/)
-        end
-      end
-
-      def spec_path
-        @path.join("specifications")
-      end
-
-      def spec_files
-        Dir[spec_path.join("*.gemspec")]
-      end
-
-      def gem_path
-        @path.join("gems")
-      end
-
-      def gem_paths
-        Dir[gem_path.join("*")]
-      end
-
-      def installed?(name, version)
-        spec_files.any? { |g| File.basename(g) == "#{name}-#{version}.gemspec" } &&
-          gem_paths.any? { |g| File.basename(g) == "#{name}-#{version}" }
-      end
-
-      def install_cached_gem(name, version, options = {})
-        cached_gem = cache_path.join("#{name}-#{version}.gem")
-        # TODO: Add a warning if cached_gem is not a file
-        if cached_gem.file?
-          Bundler.logger.info "Installing #{name}-#{version}.gem"
-          installer = Gem::Installer.new(cached_gem.to_s, options.merge(
-            :install_dir         => @path,
-            :ignore_dependencies => true,
-            :env_shebang         => true,
-            :wrappers            => true,
-            :bin_dir             => @bindir
-          ))
-          installer.install
-        end
-      end
-    end
-  end
-end