fig 0.1.62 → 0.1.64

data/Changes

@@ -1,3 +1,159 @@
+v0.1.64
+
+  Backwards incompatibilities:
+
+  - --set and --append command-line options now take priority over package.fig
+    files and --include options are now processed after any "include"
+    statements.
+
+    Previously, "fig foo/1.2.3 --set SET=command-line --append
+    APPEND=command-line --include command-line/1.2.3" was equivalent to
+
+      config default
+        set SET=command-line
+        add APPEND=command-line
+        include command-line/1.2.3
+
+        include foo/1.2.3
+      end
+
+    Now it is equivalent to
+
+      config default
+        include foo/1.2.3
+
+        set SET=command-line
+        add APPEND=command-line
+        include command-line/1.2.3
+      end
+
+    (It really is like that: a package is synthesized with statements
+    equivalent to the command-line options and that package is run through the
+    whole process.)
+
+  - Retrieve variable names can no longer contain "@", "/", or ".".  Since
+    environment variable statements only allow alphanumerics and underscore in
+    the variable names, any retrieve variable with those characters in its name
+    would never have any effect.
+
+  - Running fig --publish or --publish-local with --resource or --archive
+    options without also specifying a --set or --append option is now an error.
+
+    What would previously happen is that the --resource and --archive options
+    were ignored and package.fig locating would happen and the publish would be
+    based only on that.  Probably not what was intended.
+
+    E.g.:
+
+      fig --publish foo/1 --resource=something.txt
+
+    would ignore the --resource option, look for a package.fig file and publish
+    based upon that.
+
+  - Now fails if you specify both a descriptor and the --file option and you
+    aren't publishing.  Previously it would silently ignore the --file option,
+    which could be a bit more than a little surprising.
+
+  New features:
+
+  - Overrides are now independent statements, no longer attached to includes.
+    E.g. this
+
+      config default
+        include A/1 override C/3
+        include B/1 override C/3
+      end
+
+    is now equivalent to this
+
+      config default
+        override C/3
+        include A/1
+        include B/1
+      end
+
+    No changes to existing (functioning) package.fig files will be necessary.
+    In fact, this will fix some that didn't work due to versionless includes.
+
+  - You can now specify overrides on the command-line using "--override".
+
+  - Now checks repository format version and fails if it's different from what
+    it knows about.
+
+    There are no immediate plans to change the repository format, but
+    considering the inability to fix the non-unique archive name issue below,
+    it behooves us to allow for a format change in the future.  If the layout
+    does change, then current code could possibly corrupt the future
+    repository.
+
+  - Warns if you attempt to use a "retrieve" statement to write to an absolute
+    path; all retrieve destinations are relative.
+
+  - Now warns about ineffectual "retrieve" statements.
+
+  - A lot of existence checks have been added, resulting in nicer error
+    messages and fewer stack traces.
+
+  - README.md has had most of its content ripped out.  Documentation has been
+    greatly expanded and is available at https://github.com/mfoemmel/fig/wiki.
+
+  - Includes the Fig version in the comments in published .fig files.
+
+  Bug fixes:
+
+  - Better command-line parsing.  There were a number of scenarios where
+    missing or malformed arguments to options would cause incorrect behavior or
+    stack traces.
+
+  - Handling of symlinks is better.
+
+  - Now checks that archive base names are unique before allowing publishing.
+    Previously, if you had something like
+
+        archive http://somewhere/archive.tar.gz
+        archive foo/archive.tar.gz
+
+    everything would work locally, but the version of the package published to
+    the remote repository was corrupt.  Given the current repository layout,
+    this is not fixable, so we now disallow publishing packages like this.
+
+    Similarly, we now complain about archives named "resources.tar.gz" because
+    Fig creates an archive by that name to hold resources.
+
+  - Updates should be more robust in the face of simultaneous runs of fig.
+
+  - Retrieves:
+
+    * Now saves retrieve metadata prior to an exec(2).  Previously, if you ran
+      a command, Fig would lose track of what you had in your current
+      directory.
+
+    * Cleanup of retrieves for no-longer-referenced packages now happens, i.e.,
+      if a dependency removes a dependency, files are correctly removed.
+
+      Note that, depending upon the packages involved, switching between two
+      base packages can be slower than it was before.  In other words, if
+      packages "foo" and "bar" have a lot of retrieves of dependences NOT in
+      common, running
+
+          fig --update-if-missing foo/v1.2.3
+          fig --update-if-missing bar/v6.5.4
+
+      will be slower.  On the other hand, you won't have a lot of stuff from
+      "foo" hanging around that shouldn't be there.
+
+v0.1.63.beta.2
+v0.1.63.beta.1
+
+  - Test releases
+
+v0.1.62
+
+  Bug fixes:
+
+  - Fixed stack trace when encountering conflicting dependency package
+    versions.
+
 v0.1.61
 
   New features:

data/VERSION

@@ -1 +1 @@
-0.1.62
+0.1.64

data/bin/fig

@@ -1,10 +1,17 @@
 #!/usr/bin/env ruby
 
+if ENV['FIG_COVERAGE']
+  require File.expand_path(
+    File.join(
+      File.dirname(__FILE__), %w< .. lib fig command coveragesupport.rb >
+    )
+  )
+end
+
 $LOAD_PATH << File.expand_path(File.join(File.dirname(__FILE__), %w< .. lib > ))
 
 require 'rubygems'
 
 require 'fig/command'
 
-return_code = Fig::Command.new.run_with_exception_handling(ARGV)
-exit return_code
+exit Fig::Command.new.run_with_exception_handling ARGV

data/bin/fig-debug

@@ -1,5 +1,13 @@
 #!/usr/bin/env ruby
 
+if ENV['FIG_COVERAGE']
+  require File.expand_path(
+    File.join(
+      File.dirname(__FILE__), %w< .. lib fig command coveragesupport.rb >
+    )
+  )
+end
+
 $LOAD_PATH << File.expand_path(File.join(File.dirname(__FILE__), %w< .. lib > ))
 
 require 'rubygems'
@@ -8,5 +16,4 @@ require 'fig/command'
 
 # Identical to regular fig, but doesn't use exception handling so you can see
 # stack traces.
-return_code = Fig::Command.new.run_fig(ARGV)
-exit return_code
+exit Fig::Command.new.run_fig ARGV

data/lib/fig/applicationconfiguration.rb

@@ -1,8 +1,9 @@
 module Fig; end
 
-# Configuration for the Fig program, as opposed to the configuration in a
-# package.
+# Configuration for the Fig program, as opposed to a config in a package.
 class Fig::ApplicationConfiguration
+  attr_reader :remote_repository_url
+
   def initialize(remote_repository_url)
     @data = []
     @remote_repository_url = remote_repository_url

data/lib/fig/atexit.rb

@@ -0,0 +1,37 @@
+module Fig; end
+
+# This exists because standard Kernel#at_exit blocks don't get run before
+# Kernel#exec.
+class Fig::AtExit
+  def self.add(&block)
+    EXIT_PROCS << block
+
+    return
+  end
+
+  def self.execute()
+    EXIT_PROCS.each do
+      |proc|
+
+      begin
+        proc.call()
+      rescue StandardError => exception
+        $stderr.puts(
+          [
+            %q<Got exception from "at exit" processing.>,
+            exception.message,
+            exception.backtrace
+          ].flatten.join("\n")
+        )
+      end
+    end
+
+    return
+  end
+
+  private
+
+  EXIT_PROCS = []
+
+  at_exit { Fig::AtExit.execute() }
+end

data/lib/fig/backtrace.rb

@@ -1,7 +1,8 @@
+require 'fig/repositoryerror'
+
 module Fig; end
 
-# Contains traces of file inclusions so that the user can track down which file
-# an error occurred in.
+# Keeps track of overrides and can produce package definition stack traces.
 class Fig::Backtrace
   attr_reader :overrides
 
@@ -11,11 +12,21 @@ class Fig::Backtrace
     @overrides  = {}
   end
 
-  def add_override(package_name, version)
+  def add_override(statement)
+    package_name = statement.package_name
     # Don't replace an existing override on the stack
-    return if get_override(package_name)
+    return if @parent && @parent.get_override(package_name)
+
+    new_version = statement.version
+    existing_version = @overrides[package_name]
+    if existing_version && existing_version != new_version
+      stacktrace = dump_to_string()
+      raise Fig::RepositoryError.new(
+        "Override #{package_name} version conflict (#{existing_version} vs #{new_version})#{statement.position_string}." + ( stacktrace.empty? ? '' : "\n#{stacktrace}" )
+      )
+    end
 
-    @overrides[package_name] = version
+    @overrides[package_name] = new_version
   end
 
   # Returns a version.
@@ -31,7 +42,7 @@ class Fig::Backtrace
   def dump(out)
     stack = []
     collect(stack)
-    i=0
+    i = 0
     for descriptor in stack
       indent=''
       i.times { indent += '  ' }
@@ -42,6 +53,12 @@ class Fig::Backtrace
 
   protected
 
+  def dump_to_string()
+    string_handle = StringIO.new
+    dump(string_handle)
+    return string_handle.string
+  end
+
   def collect(stack)
     if @parent
       @parent.collect(stack)

data/lib/fig/command.rb

@@ -2,27 +2,28 @@ require 'rubygems'
 require 'net/ftp'
 require 'set'
 
+require 'fig/atexit'
+require 'fig/command/options'
 require 'fig/environment'
 require 'fig/figrc'
 require 'fig/logging'
 require 'fig/operatingsystem'
-require 'fig/options'
 require 'fig/package'
 require 'fig/parser'
 require 'fig/repository'
 require 'fig/repositoryerror'
-require 'fig/retriever'
 require 'fig/statement/configuration'
-require 'fig/statement/publish'
 require 'fig/userinputerror'
+require 'fig/workingdirectorymaintainer'
 
-# These are a breakout of parts of this class simply to keep the file size down.
+# The following are a break out of parts of this class simply to keep the file
+# size down.
 
 # You will need to look in this file for any stuff related to --list-* options.
 require 'fig/command/listing'
 
 # You will need to look in this file for any stuff related to loading the
-# primary Package object.
+# base Package object.
 require 'fig/command/packageload'
 
 module Fig; end
@@ -32,17 +33,59 @@ class Fig::Command
   include Fig::Command::Listing
   include Fig::Command::PackageLoad
 
+  def self.get_version()
+    line = nil
+
+    begin
+      File.open(
+        "#{File.expand_path(File.dirname(__FILE__) + '/../../VERSION')}"
+      ) do |file|
+        line = file.gets
+      end
+    rescue
+      $stderr.puts 'Could not retrieve version number. Something has mucked with your Fig install.'
+
+      return nil
+    end
+
+    # Note that we accept anything that contains three decimal numbers
+    # seperated by periods.  This allows for versions like
+    # "4.3.2-super-special-version-in-3D".
+    if line !~ %r< \b \d+ [.] \d+ [.] \d+ \b >x
+      $stderr.puts %Q<"#{line}" does not look like a version number. Something has mucked with your Fig install.>
+
+      return nil
+    end
+
+    return line
+  end
+
   def run_fig(argv)
-    @options = Fig::Options.new(argv)
+    begin
+      @options = Fig::Command::Options.new(argv)
+    rescue Fig::UserInputError => error
+      $stderr.puts error.to_s # Logging isn't set up yet.
+      return 1
+    end
+
     if not @options.exit_code.nil?
       return @options.exit_code
     end
     @descriptor = @options.descriptor
 
+    if @options.help?
+      return @options.help
+    end
+
+    if @options.version?
+      return emit_version()
+    end
+
     configure()
 
     if @options.clean?
       check_required_package_descriptor('to clean')
+      ensure_descriptor_and_file_were_not_both_specified()
       @repository.clean(@descriptor)
       return 0
     end
@@ -55,26 +98,30 @@ class Fig::Command
       return publish()
     end
 
+    ensure_descriptor_and_file_were_not_both_specified()
+
     load_package_object()
 
     if @options.listing()
       handle_post_parse_list_options()
     elsif @options.get()
-      puts @environment[@options.get()]
+      # Ruby v1.8 emits "nil" for nil, whereas ruby v1.9 emits the empty
+      # string, so, for consistency, we need to ensure that we always emit the
+      # empty string.
+      puts @environment[@options.get()] || ''
     elsif @options.shell_command
       @environment.execute_shell(@options.shell_command) do
         |command| @operating_system.shell_exec command
       end
     elsif @descriptor
-      @environment.include_config(@package, @descriptor, {}, nil)
+      @environment.include_config(@base_package, @descriptor, nil)
       @environment.execute_config(
-        @package,
+        @base_package,
         @descriptor,
         @options.command_extra_argv || []
       ) { |cmd| @operating_system.shell_exec cmd }
     elsif not @repository.updating?
-      $stderr.puts "Nothing to do.\n"
-      $stderr.puts Fig::Options::USAGE
+      $stderr.puts "Nothing to do.\n\n"
       $stderr.puts %q<Run "fig --help" for a full list of commands.>
       return 1
     end
@@ -95,7 +142,7 @@ class Fig::Command
       return 1
     rescue OptionParser::InvalidOption => error
       $stderr.puts error.to_s
-      $stderr.puts Fig::Options::USAGE
+      $stderr.puts Fig::Command::Options::USAGE
       return 1
     rescue Fig::RepositoryError => error
       log_error_message(error)
@@ -103,6 +150,15 @@ class Fig::Command
     end
   end
 
+  def emit_version()
+    version = Fig::Command.get_version()
+    return 1 if version.nil?
+
+    puts File.basename($0) + ' v' + version
+
+    return 0
+  end
+
   private
 
   def derive_remote_url()
@@ -128,11 +184,10 @@ class Fig::Command
   def configure()
     Fig::Logging.initialize_pre_configuration(@options.log_level())
 
-    remote_url = derive_remote_url()
 
     @configuration = Fig::FigRC.find(
       @options.figrc(),
-      remote_url,
+      derive_remote_url(),
       @options.login?,
       @options.home(),
       @options.no_figrc?
@@ -146,8 +201,7 @@ class Fig::Command
     @operating_system = Fig::OperatingSystem.new(@options.login?)
     @repository = Fig::Repository.new(
       @operating_system,
-      File.expand_path(File.join(@options.home(), 'repos')),
-      remote_url,
+      @options.home(),
       @configuration,
       nil, # remote_user
       @options.update?,
@@ -155,15 +209,13 @@ class Fig::Command
       check_include_statements_versions?
     )
 
-    @retriever = Fig::Retriever.new('.')
-
-    at_exit { @retriever.save_metadata() }
+    @working_directory_maintainer = Fig::WorkingDirectoryMaintainer.new('.')
 
-    @environment = prepare_environment
-
-    @options.non_command_package_statements().each do |statement|
-      @environment.apply_config_statement(nil, statement, nil)
+    Fig::AtExit.add do
+      @working_directory_maintainer.prepare_for_shutdown(@options.updating?)
     end
+
+    prepare_environment()
   end
 
   def prepare_environment()
@@ -172,7 +224,18 @@ class Fig::Command
       environment_variables = Fig::OperatingSystem.get_environment_variables({})
     end
 
-    return Fig::Environment.new(@repository, environment_variables, @retriever)
+    @environment = Fig::Environment.new(
+      @repository, environment_variables, @working_directory_maintainer
+    )
+
+    Fig::AtExit.add { @environment.check_unused_retrieves() }
+
+    return
+  end
+
+  def config_was_specified_by_user()
+    return ! @options.config().nil?                   ||
+           @descriptor && ! @descriptor.config().nil?
   end
 
   def base_config()
@@ -181,6 +244,29 @@ class Fig::Command
            Fig::Package::DEFAULT_CONFIG
   end
 
+  # If the user has specified a descriptor, than any package.fig or --file
+  # option is ignored.  Thus, in order to avoid confusing the user, we make
+  # specifying both an error.
+  #
+  # The one exception to this rule is when we are publishing, which should
+  # already have been invoked by the time this is called.
+  def ensure_descriptor_and_file_were_not_both_specified()
+    file = @options.package_definition_file()
+
+    # If the user specified --no-file, even though it's kind of superfluous,
+    # we'll let it slide because the user doesn't think that any file will be
+    # processed.
+    file_specified = ! file.nil? && file != :none
+
+    if @descriptor and file_specified
+      raise Fig::UserInputError.new(
+        %Q<Cannot specify both a package descriptor (#{@descriptor.original_string}) and the --file option (#{file}).>
+      )
+    end
+
+    return
+  end
+
   def check_required_package_descriptor(operation_description)
     if not @descriptor
       raise Fig::UserInputError.new(
@@ -209,31 +295,45 @@ class Fig::Command
     check_required_package_descriptor('to publish')
 
     if @descriptor.name.nil? || @descriptor.version.nil?
-      raise Fig::UserInputError.new('Please specify a package name and a version name.')
+      raise Fig::UserInputError.new(
+        'Please specify a package name and a version name.'
+      )
+    end
+    if @descriptor.name == '_meta'
+      raise Fig::UserInputError.new(
+        %q<Due to implementation issues, cannot create a package named "_meta".>
+      )
     end
 
-    if not @options.non_command_package_statements().empty?
+    publish_statements = nil
+    if not @options.environment_statements().empty?
       publish_statements =
         @options.resources() +
         @options.archives() +
         [
           Fig::Statement::Configuration.new(
+            nil,
             nil,
             Fig::Package::DEFAULT_CONFIG,
-            @options.non_command_package_statements()
+            @options.environment_statements()
           )
         ]
-      publish_statements << Fig::Statement::Publish.new()
+    elsif not @options.resources().empty? or not @options.archives().empty?
+      raise Fig::UserInputError.new(
+        '--resource/--archive options were specified, but no --set/--append option was given. Will not publish.'
+      )
     else
-      load_package_file()
-      if not @package.statements.empty?
-        publish_statements = @package.statements
+      load_package_object_from_file()
+      if not @base_package.statements.empty?
+        publish_statements = @base_package.statements
       else
         $stderr.puts 'Nothing to publish.'
         return 1
       end
     end
 
+    apply_base_config_to_environment(:ignore_base_package)
+
     if @options.publish?
       Fig::Logging.info "Checking status of #{@descriptor.to_string()}..."