autoproj 2.0.3 → 2.1.0.rc1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 225ebf408df3b402e49ba4fb88968d99ca7516d4
-  data.tar.gz: 5e6c520db390621a7fc4f5b9a6ac772d9b08e459
+  metadata.gz: cc8cc412594a39f08dcc1262a09309892711bbd1
+  data.tar.gz: 1a2c7d45beefce2c25c6a8b48ec65785c8c437df
 SHA512:
-  metadata.gz: be6bdb315d178d9980fa5f769fc044ba2dfdc1a9cfd605e3a184142f8d7a6132777778ecf75e3f33bcf0c4184139e8dac3a4aafdf5668efd9903d94be4488ee5
-  data.tar.gz: 89e2b7a9ada6aff9be121e91c087f6d476f94a78dd022bbb2bdb8a59dfc866bf1bafa463ce8df47b47404a0a654a38b5e42bf53b98e1aa1bc446e138a31d9b21
+  metadata.gz: 44c014c39dba092b57e00f878338b25507d2962b8b492cf962bc514269c226ca45eaf087e2fb07979c89cd3c9134bc12c8c469af259805f5d92ed61f82b6b218
+  data.tar.gz: 95fe2cfc145dd25669e5359947b5ffbf939b9df7ac011af73e650492fdf6e8f7b758ad5f372cfc7e5efd0aa86efa25607b8f84e3744f2ef9784377575b0798b3

data/README.md

@@ -2,14 +2,14 @@
 [![Gem Version](https://badge.fury.io/rb/autoproj.svg)](http://badge.fury.io/rb/autoproj)
 [![Documentation](http://b.repl.ca/v1/yard-docs-blue.png)](http://rubydoc.info/gems/autoproj/frames)
 
-What is Autoproj
-----------------
+# What is Autoproj
+
 Autoproj allows to easily install and maintain software that is under source
 code form (usually from a version control system). It has been designed to support a
 package-oriented development process, where each package can have its own
 version control repository (think "distributed version control"). It also
 provides an easy integration of the local operating system (Debian, Ubuntu,
-Fedora, maybe MacOSX at some point).
+Fedora, MacOSX).
 
 This tool has been developed over the years. It is now maintained in the frame of the Rock
 robotics project (http://rock-robotics.org), to install robotics-related
@@ -19,8 +19,8 @@ One main design direction for autoproj is that packages can be built with
 autoproj without having been designed to be built with autoproj.
 
 The philosophy behind autoproj is:
-* supports any type of build system (CMake and autotools are built-in)
-* supports different VCS: cvs, svn, git, plain tarballs.
+* supports any type of build system (CMake, autotools, ruby packages, ...)
+* supports different VCS: git, plain archives, cvs, svn, ...
 * software packages are plain packages, meaning that they can be built and
   installed /outside/ an autoproj tree, and are not tied *at all* to the
   autoproj build system.
@@ -29,8 +29,7 @@ The philosophy behind autoproj is:
   access to.
 * handle code generation properly
 
-Overview of an autoproj installation
--------------------------------------
+## Overview of an autoproj installation
 
 The idea in an autoproj installation is that people share _definitions_ for a
 set of packages that can depend on each other. Then, anyone can cherry-pick in
@@ -47,8 +46,8 @@ Each package definition includes:
 See this
 page[http://www.rock-robotics.org/stable/documentation/autoproj/writing_manifest.html] for more information.
 
-Software packages in Autoproj
------------------------------
+## Software packages in Autoproj
+
 In the realm of autoproj, a software package should be a self-contained build
 system, that could be built outside of an autoproj tree. In practice, it means
 that the package writer should leverage its build system (for instance, cmake)
@@ -57,9 +56,196 @@ appropriate build options that should be given (for instance, include
 directories or library names).
 
 As a guideline, we recommend that inter-package dependencies are managed by
-using pkg-config.
+using pkg-config for C/C++ packages.
 
 To describe the package, and more importantly to setup cross-package
 dependencies, an optional manifest file can be
 added[http://www.rock-robotics.org/stable/documentation/autoproj/advanced/manifest-xml.html].
 
+# Migrating from v1 to v2
+
+Autoproj 2.0 has been released on the 22nd December 2016 and brought
+significant changes both to its internals and to its workflow. What follows
+describes the changes brought by v2, from the point of view of someone that
+already knows autoproj v1
+
+### Upgrade process
+
+Autoproj 2.x is backward incompatible in two ways: first, it requires
+ruby 2.0+. Second, both the workspace layout and the way RubyGems are
+handled changed and therefore the upgrade process is pretty complex.
+
+For these reasons, the latest version of autoproj 1.x (1.13.3) will
+not automatically upgrade to 2.x. The update to 2.x will have to be
+manual. This is going to be a general policy going forward (2.x will
+not upgrade to 3.x and so on). Moreover, it will remain possible to
+bootstrap an autoproj 1.x workspace using this bootstrap script:
+
+   https://raw.githubusercontent.com/rock-core/autoproj/v1.13.3/bin/autoproj_bootstrap
+
+To bootstrap a new install using autoproj 2.0, just follow the
+standard bootstrap process, which did not change. Note that by using
+the bootstrap from the 'master' branch on github you *will* bootstrap
+using autoproj 2.0.
+
+To upgrade an existing autoproj 1.x install, run the following script
+from the root of the installation:
+
+   https://raw.githubusercontent.com/rock-core/autoproj/master/bin/autoproj_install
+
+After the upgrade, one can "downgrade" by simply replacing the new
+autoproj-generated env.sh script by the backup the upgrade process did
+(env.sh-autoproj-v1). Open a new console, et voila.
+
+### New 2.x workspace
+
+Under 2.x, all autoproj-generated files related to the workspace are
+saved under a new .autoproj directory (as e.g. the config files and
+remotes). The upgrade process does not delete these to allow for
+"downgrading".
+
+In addition, autoproj now uses [bundler](bundler.io) to manage the gems. This means
+that, by default, the gems are shared between all autoproj installs,
+bundler making sure that upgrading a gem on one install does not
+affect another. This makes bootstrapping a lot faster (since already
+present gems will be reused) and in the future will allow 'autoproj
+versions' and related commands (tag and commit) to pin the gem
+versions. It also resolves the gem dependencies globally, thus
+detecting and/or handling problems with gems that have conflicting
+dependencies (an issue we currently have / had with webgen)
+
+This also means that using the `gem` command to manage the gems is not allowed
+anymore.
+
+To add a gem to the workspace, create or edit `autoproj/Gemfile` and add `gem`
+entries following [the bundler documentation](http://bundler.io/gemfile.html).
+Once the file is edited, run `autoproj osdeps` and reload the updated `env.sh`. 
+
+To remove gems, remove the corresponding line in `autoproj/Gemfile`, run
+`autoproj osdeps` and reload env.sh.
+
+Alternatively to the main `autoproj/Gemfile`, files with the `.gemfile`
+extension in `autoproj/overrides.d` are also considered
+
+### Proper help for the command-line interface
+
+'autoproj help' is useful (which was definitely NOT the case on 1.x),
+showing details about each subcommand as well as all existing command
+line options.
+
+### Parallel import
+
+autoproj update and status can now operate in parallel. The default is
+to spawn 10 process, but this can be controlled through the
+'parallel_import_level' option in .autoproj/config.yml.
+
+Note that for this feature to work well, one has to specify if a
+repository needs user interaction (e.g. github private repositories
+requiring a password) or not. The generic interactive: option can be
+given for any importer, and the private: option can be used for this
+purpose in the github handler. The latter also allows to use a
+different pull method than for public repositories (and defaults to
+the push handler, e.g. http,ssh leads to private repositories using
+ssh to pull by default)
+
+### Improved workflow for overrides
+
+To increase convenience, the git importer sets up a remote for each
+package set, with the version control information that this package
+set expects. So, if you have a rock.core package that is overriden by
+myproject, the rock.core remote will point to the URL defined in
+rock.core and the myproject remote will point to the overriden URL.
+The 'autobuild' remote always points to the final one.
+
+The status and show subcommands learned the --mainline option. With
+--mainline, the status is displayed against the VCS without any
+overrides applied. With --mainline=rock.core, only the overrides up to
+and including rock.core will be applied:
+
+autoproj status --mainline # compare the local state against the
+"upstream" state
+autoproj status --mainline=my.set # compare the local state against
+someone that would have overrides only until 'my.set'
+
+The option is also available to "autoproj show"
+
+### Environment handling
+
+The environment is not global anymore, but per-package. This means
+that builds that were missing dependencies could be previously passing
+and will fail. It also means that the environment of packages that
+have been used but are not anymore will not pollute env.sh.
+
+### Separate build directories
+
+Autoproj.builddir can be set to a full path, in which case every
+package's build directory will be placed into a subdirectory of this
+path (instead of within the package checkout). This is great to avoid
+backing up build files (separating the build/install and source
+folders), or to share source files for different build setups.
+autoproj locate and acd learned -b or --build to resp. locate or cd
+into a package's build directory. This should also improve the usage
+of Eclipse in an autoproj environment.
+
+### Improved workflow with heavy branching
+
+Principally when using pull requests, we have a tendency to push a lot
+of code on branches. 'autoproj status' becomes a lot more difficult to
+interpret as we always have to ask ourselves "is this code on a PR
+already ?"
+
+2.0 improves this workflow in two ways:
+- "autoproj versions" now looks by default remotely to determine the
+"best" remote branch. It can be turned off with the --local option
+- autoproj status also checks by default if there is a remote branch,
+and displays that information. Additionally, the --snapshot option
+tells status to display the status against this "best" branch.
+
+### Plugin support
+
+The autoproj CLI can now be extended with plugins. Two such plugins
+are already available:
+   autoproj-stats: compute statistics about authorship and
+copyright/license information
+   autoproj-git: git-aware subcommand (for now, only knows "clean")
+
+A plugin is a gem, but must be installed using the `autoproj plugin` subcommand
+to make it available. For instance, `autoproj plugin install autoproj-stats`
+will add the 'stats' subcommand to the autoproj command, whose documentation
+is available with autoproj help stats
+
+### Other changes
+
+update and build have a working --no-deps option (this was broken pre-2.0)
+
+All subcommands for which it makes sense now accept a package
+selection on the command line.
+
+bootstrap learned --seed-config=PATH to provide a base configuration
+for the build, useful for automated build environments.
+
+Autoproj pre-2.0 had widely inconsistent behaviour between source and
+osdep packages. Hopefully all of them have been fixed. For all intent
+and purposes, osdeps and source packages look the same. For instance,
+osdep can be excluded or ignored now.
+
+autoproj update learned --force-reset to reset to the expected commit,
+bypassing any check. Great for CI environments.
+
+# Developing autoproj 2.x
+
+The best way to use autoproj 2.x from git is to checkout autoproj and
+autobuild manually, and write a Gemfile.autoproj-2.0 containing
+
+   source "https://rubygems.org"
+   gem "autoproj", path: '/home/doudou/dev/gems/autoproj'
+   gem "autobuild", path: '/home/doudou/dev/gems/autobuild'
+   gem "utilrb", ">= 3.0.0.a"
+
+Then, pass this gemfile to the --gemfile argument to autoproj_install
+or autoproj_bootstrap. Note that one can re-run autoproj_install in an
+already bootstrapped autoproj workspace, e.g.
+
+   wget https://raw.githubusercontent.com/rock-core/autoproj/master/bin/autoproj_install
+   ruby autoproj_install --gemfile=../Gemfile.autoproj-2.0
+

data/autoproj.gemspec

@@ -23,15 +23,15 @@ Gem::Specification.new do |s|
     s.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
 
     s.add_runtime_dependency "bundler"
-    s.add_runtime_dependency "autobuild", ">= 1.10.0.a"
-    s.add_runtime_dependency "utilrb", ">= 3.0"
+    s.add_runtime_dependency "autobuild", ">= 1.11.0.a"
+    s.add_runtime_dependency "utilrb", '~> 3.0.0', ">= 3.0.0"
     s.add_runtime_dependency "thor", '~> 0.19.0', '>= 0.19.1'
-    s.add_runtime_dependency 'concurrent-ruby'
-    s.add_runtime_dependency 'tty-color', '~> 0.3.0', '>= 0.3.0'
-    s.add_runtime_dependency 'tty-prompt'
-    s.add_development_dependency "flexmock", ">= 2.0.0"
-    s.add_development_dependency "minitest", ">= 5.0", "~> 5.0"
-    s.add_development_dependency "fakefs"
+    s.add_runtime_dependency 'concurrent-ruby', '~> 1.0.0', '>= 1.0.0'
+    s.add_runtime_dependency 'tty-color', '~> 0.4.0', '>= 0.4.0'
+    s.add_runtime_dependency 'tty-prompt', '~> 0.12.0', '>= 0.12.0'
+    s.add_runtime_dependency 'tty-spinner', '~> 0.4.0', '>= 0.4.0'
+    s.add_development_dependency "flexmock", '~> 2.0', ">= 2.0.0"
+    s.add_development_dependency "minitest", "~> 5.0", ">= 5.0"
     s.add_development_dependency "simplecov"
 end
 

data/bin/alocate

@@ -2,5 +2,16 @@
 
 require 'autoproj/cli/main'
 Autoproj::CLI.basic_setup
-Autoproj::CLI::Main.start(['locate', *ARGV])
+
+class Alocate < Autoproj::CLI::Main
+    def self.banner(*)
+        "alocate [options]"
+    end
+end
+
+if ARGV.include?('--help') || ARGV.include?('help')
+    Alocate.start(['help', 'locate'])
+else
+    Alocate.start(['locate', *ARGV])
+end
 

data/bin/alog

@@ -3,6 +3,14 @@
 require 'autoproj/cli/main'
 require 'autoproj/cli/locate'
 
+if ARGV.include?('--help') || ARGV.include?('help')
+    puts "Usage:"
+    puts "  alog [package]"
+    puts
+    puts "Shows log available for the given package and allows to select and display one"
+    exit 0
+end
+
 Autoproj::CLI.basic_setup
 Autoproj.report(silent: true) do
     cli = Autoproj::CLI::Locate.new

data/bin/amake

@@ -2,5 +2,16 @@
 
 require 'autoproj/cli/main'
 Autoproj::CLI.basic_setup
-Autoproj::CLI::Main.start(['build', '--amake', *ARGV])
+
+class Amake < Autoproj::CLI::Main
+    def self.banner(*)
+        "amake [options]"
+    end
+end
+
+if ARGV.include?('--help') || ARGV.include?('help')
+    Amake.start(['help', 'build'])
+else
+    Amake.start(['build', '--amake', *ARGV])
+end
 

data/bin/aup

@@ -2,4 +2,15 @@
 
 require 'autoproj/cli/main'
 Autoproj::CLI.basic_setup
-Autoproj::CLI::Main.start(['update', '--aup', *ARGV])
+
+class Aup < Autoproj::CLI::Main
+    def self.banner(*)
+        "aup [options]"
+    end
+end
+
+if ARGV.include?('--help') || ARGV.include?('help')
+    Aup.start(['help', 'update'])
+else
+    Aup.start(['update', '--aup', *ARGV])
+end

data/bin/autoproj_bootstrap

@@ -48,6 +48,7 @@ module Autoproj
                 @autoproj_options = Array.new
 
                 @env = Hash.new
+                env['RUBYOPT'] = []
                 env['RUBYLIB'] = []
                 env['GEM_PATH'] = []
                 env['GEM_HOME'] = []
@@ -190,10 +191,11 @@ module Autoproj
             # @param [String] autoproj_version a constraint on the autoproj version
             #   that should be used
             # @return [String]
-            def default_gemfile_contents(autoproj_version = ">= 2.0.0.a")
+            def default_gemfile_contents(autoproj_version = ">= 2.0.0")
                 ["source \"#{gem_source}\"",
+                 "ruby \"#{RUBY_VERSION}\" if respond_to?(:ruby)",
                  "gem \"autoproj\", \"#{autoproj_version}\"",
-                 "gem \"utilrb\", \">= 3.0.0.a\""].join("\n")
+                 "gem \"utilrb\", \">= 3.0.1\""].join("\n")
             end
 
             # Parse the provided command line options and returns the non-options
@@ -260,14 +262,19 @@ module Autoproj
                 end
             end
 
-            def install_bundler(gem_program)
+            def install_bundler(gem_program, silent: false)
                 local = ['--local'] if local?
 
+                redirection = Hash.new
+                if silent
+                    redirection = Hash[out: :close]
+                end
+
                 result = system(
-                    env_for_child,
+                    env_for_child.merge('GEM_HOME' => gems_gem_home),
                     Gem.ruby, gem_program, 'install', '--env-shebang', '--no-document', '--no-format-executable', '--clear-sources', '--source', gem_source,
                         *local,
-                        "--bindir=#{File.join(gems_gem_home, 'bin')}", 'bundler')
+                        "--bindir=#{File.join(gems_gem_home, 'bin')}", 'bundler', **redirection)
 
                 if !result
                     STDERR.puts "FATAL: failed to install bundler in #{gems_gem_home}"
@@ -573,14 +580,16 @@ require 'bundler/setup'
 
             def stage2(*vars)
                 require 'autobuild'
-                puts "saving env.sh and .autoproj/env.sh"
+                puts "saving temporary env.sh and .autoproj/env.sh"
                 save_env_sh(*vars)
+                puts "running 'autoproj envsh' to generate a proper env.sh"
                 if !system(Gem.ruby, autoproj_path, 'envsh', *autoproj_options)
                     STDERR.puts "failed to run autoproj envsh on the newly installed autoproj (#{autoproj_path})"
                     exit 1
                 end
                 # This is really needed on an existing install to install the
                 # gems that were present in the v1 layout
+                puts "running 'autoproj osdeps' to re-install missing gems"
                 if !system(Gem.ruby, autoproj_path, 'osdeps')
                     STDERR.puts "failed to run autoproj osdeps on the newly installed autoproj (#{autoproj_path})"
                     exit 1

data/bin/autoproj_install

@@ -48,6 +48,7 @@ module Autoproj
                 @autoproj_options = Array.new
 
                 @env = Hash.new
+                env['RUBYOPT'] = []
                 env['RUBYLIB'] = []
                 env['GEM_PATH'] = []
                 env['GEM_HOME'] = []
@@ -190,10 +191,11 @@ module Autoproj
             # @param [String] autoproj_version a constraint on the autoproj version
             #   that should be used
             # @return [String]
-            def default_gemfile_contents(autoproj_version = ">= 2.0.0.a")
+            def default_gemfile_contents(autoproj_version = ">= 2.0.0")
                 ["source \"#{gem_source}\"",
+                 "ruby \"#{RUBY_VERSION}\" if respond_to?(:ruby)",
                  "gem \"autoproj\", \"#{autoproj_version}\"",
-                 "gem \"utilrb\", \">= 3.0.0.a\""].join("\n")
+                 "gem \"utilrb\", \">= 3.0.1\""].join("\n")
             end
 
             # Parse the provided command line options and returns the non-options
@@ -260,14 +262,19 @@ module Autoproj
                 end
             end
 
-            def install_bundler(gem_program)
+            def install_bundler(gem_program, silent: false)
                 local = ['--local'] if local?
 
+                redirection = Hash.new
+                if silent
+                    redirection = Hash[out: :close]
+                end
+
                 result = system(
-                    env_for_child,
+                    env_for_child.merge('GEM_HOME' => gems_gem_home),
                     Gem.ruby, gem_program, 'install', '--env-shebang', '--no-document', '--no-format-executable', '--clear-sources', '--source', gem_source,
                         *local,
-                        "--bindir=#{File.join(gems_gem_home, 'bin')}", 'bundler')
+                        "--bindir=#{File.join(gems_gem_home, 'bin')}", 'bundler', **redirection)
 
                 if !result
                     STDERR.puts "FATAL: failed to install bundler in #{gems_gem_home}"
@@ -573,14 +580,16 @@ require 'bundler/setup'
 
             def stage2(*vars)
                 require 'autobuild'
-                puts "saving env.sh and .autoproj/env.sh"
+                puts "saving temporary env.sh and .autoproj/env.sh"
                 save_env_sh(*vars)
+                puts "running 'autoproj envsh' to generate a proper env.sh"
                 if !system(Gem.ruby, autoproj_path, 'envsh', *autoproj_options)
                     STDERR.puts "failed to run autoproj envsh on the newly installed autoproj (#{autoproj_path})"
                     exit 1
                 end
                 # This is really needed on an existing install to install the
                 # gems that were present in the v1 layout
+                puts "running 'autoproj osdeps' to re-install missing gems"
                 if !system(Gem.ruby, autoproj_path, 'osdeps')
                     STDERR.puts "failed to run autoproj osdeps on the newly installed autoproj (#{autoproj_path})"
                     exit 1