autoproj 1.3.4 → 1.4.0

data/doc/guide/src/package_sets/osdeps.page

@@ -0,0 +1,123 @@
+---
+title: Operating System dependencies
+sort_info: 300
+---
+
+Each package set can have one or multiple .osdeps files. These files declare
+how to install prepackaged dependencies, based on the underlying operating system.
+
+Defining dependencies between source packages and OS packages
+-------------------------------------------------------------
+
+These dependencies can be defined in two ways:
+
+ * by adding a <tt>&lt;osdep name="depname" /&gt;</tt> tag in the package's
+   [manifest.xml](manifest-xml.html)
+ * by calling <tt>pkg.depends_on_os_package("depname")</tt> [in the autobuild
+   file](autobuild.html).
+
+In both cases, depname is used as a key to find the OS package definition in the
+osdeps files (see below). It does not have to be the actual package name.
+
+OS packages
+-----------
+
+The general format of the an OS package definition:
+
+{coderay:: yaml}
+name:
+    distribution1,distribution2:
+        version1,version2: [package_name1, package_name2]
+{coderay}
+
+Where 'name' is the name used to declare the dependency (see above),
+<tt>distribution*</tt> the name of the distribution and <tt>version*</tt> the
+distribution's version, and <tt>package_name*</tt> the name of the package in
+the underlying OS.
+
+Since the osdeps file is a YAML file, <tt>[package_name1, package_name2]</tt> can also be written
+
+{coderay:: yaml}
+name:
+    distribution1,distribution2:
+        version1,version2:
+            - package_name1
+            - package_name2
+{coderay}
+
+If only one package needs to be installed, one can use the shortcut
+
+{coderay:: yaml}
+name:
+    distribution1,distribution2:
+        version1,version2: package_name
+{coderay}
+
+Finally, if the package name is version-independent, the version can be omitted:
+
+
+{coderay:: yaml}
+name:
+    distribution1,distribution2: package_name
+{coderay}
+
+Examples:
+
+{coderay:: yaml}
+ruby:
+    debian,ubuntu:
+        9.04,10.04,sid: libruby-dev
+boost:
+    debian:
+        - libboost1.38-dev
+        - libboost-program1.38-dev
+    ubuntu:
+        9.04,10.04: libboost-dev
+{coderay}
+
+At the time of this writing, autoproj is able to install packages on
+Ubuntu/Debian and Gentoo. Support for other operating systems can be easily
+added, so [contact me](http://github.com/doudou) if you want to do so.
+
+RubyGems packages
+-----------------
+
+RubyGems packages are OS-independent. In the osdeps files, they can be referred
+to by replacing the OS distribution name by 'gem'.
+
+Example:
+
+{coderay:: yaml}
+hoe:
+    gem:
+        hoe
+{coderay}
+
+If the OS dep name and the RubyGems name are the same, one can use the shortcut
+
+{coderay:: yaml}
+hoe: gem
+{coderay}
+
+Note that it is possible to use a mixture of RubyGems and OS packages. For
+instance, the following snippet will both install the gnuplot package and the
+gnuplot RubyGems whenever an osdep on 'gnuplot' is declared.
+
+{coderay:: yaml}
+gnuplot:
+    gem: gnuplot
+    debian: gnuplot
+{coderay}
+
+Ignoring some dependencies
+--------------------------
+It is possible that, on some operating systems, a given package should simply be
+ignored. To do so, simply use the 'ignore' keyword. Example:
+
+{coderay:: yaml}
+gnuplot:
+    gem: gnuplot
+    debian: ignore
+    ubuntu: gnuplot
+{coderay}
+

data/doc/guide/src/structure.page

@@ -4,16 +4,37 @@ sort_info: 50
 ---
 
 This page describes the structure of a autoproj installation, and describes how
-to manage one. See [the introduction](index.page) for the bootstrapping process.
+to manage one. See [the introduction](index.html) for the bootstrapping process.
 
 Structure
 ---------
 
-A autoproj installation is characterized by a autoproj/ directory in which all
-autoproj-related configuration is saved. The contained files are as follows:
-
- * autoproj/manifest: list of available package sets, customization options.
-   The first part is covered here, the second in the next page.
+!overview.png!
+<img class="full" src="overview.png" />
+
+The autoproj configuration and build process goes like this:
+
+ * a set of packages are declared in _package sets_. These package sets can
+   either be local (saved along with the rest of the autoproj configuration) or
+   remote (imported from a VCS). They declare how to get, build and install
+   a certain number of packages. On the image above, there are three of those
+   sets: rubim.base, rubim.orocos and rubim.drivers.
+ * packages that are relevant to the local installation are cherry-picked from
+   the package sets. One can either select packages one-by-one (the case for the
+   two packages of rubim.drivers above), or a package set can be imported as a
+   whole (rubim.orocos and rubim.base).
+ * autoproj will then import the selected packages, auto-select their
+   dependencies and import them as well, build and install all of this.
+ * the selected packages can be imported and built in subdirectories of the
+   installation tree. In the above example, all packages from rubim.orocos are
+   built in the tools/ subdirectory, one package of rubim.drivers in the
+   perception/ subdirectory and so on.
+
+In practice, the autoproj configuration is saved in an autoproj/ directory. It
+is split like this:
+
+ * autoproj/manifest: list of available package sets, package selection and
+   installation layout (where to put what).
  * .remotes/\*: package sets that are imported from a remote version
    control system
  * autoproj/\*/: local sets, i.e. sets that have not been imported from a remote
@@ -29,8 +50,10 @@ The build is done in two steps:
    installation
 
 Moreover, the <tt>build/log</tt> directory contains the output of all commands
-that have been run during the build. Finally, a <tt>env.sh</tt> script is
-generated to set up your shell for the use of the installed software.
+that have been run during the build.
+
+Finally, a <tt>env.sh</tt> script is generated to set up your shell for the use
+of the installed software.
 
 Listing and adding package sets
 -------------------------------
@@ -64,7 +87,9 @@ For the git importer, one of 'branch' or 'tag' options can be provided as well:
 {coderay}
 
 Imported package sets are saved in the <tt>.remotes</tt> directory of the
-autoproj installation.
+autoproj installation. The importers that are available for configuration are
+the same than the ones available for the packages themselves, so see [this
+page](package_sets/importers.html#all_importers) for the list of available importers.
 
 Management
 ----------
@@ -84,12 +109,18 @@ autoproj build --reconfigure
 Alternatively, you can edit the autoproj/config.yml file directly.
 
 If you are in a disconnected environment (i.e. no access to remote
-repositories), use the <tt>--no-update</tt> option to skip the update phase.
-autoproj will still have to checkout new packages, though:
+repositories), use the fast-build mode, to skip the package update and OS
+packages installation phases (autoproj will still check out new packages,
+though):
 
-autoproj build --no-update
+autoproj fast-build
 {.commandline}
 
+The package update can be disabled with --no-update and the OS dependencies
+installation with --no-osdeps. In practice, "autoproj fast-build" is equivalent
+to autoproj build --no-update --no-osdeps
+{.block}
+
 If, on the other hand, you only want to update the source code, do
 
 autoproj update

data/lib/autoproj/autobuild.rb

@@ -1,13 +1,17 @@
+require 'find'
+require 'fileutils'
 require 'autobuild'
 require 'set'
 
 class Autobuild::Package
-    def autoproj_name
+    def autoproj_name # :nodoc:
         srcdir.gsub /^#{Regexp.quote(Autoproj.root_dir)}\//, ''
     end
 end
 
 module Autoproj
+    # Subclass of Autobuild::Reporter, used to display a message when the build
+    # finishes/fails.
     class Reporter < Autobuild::Reporter
         def error(error)
             error_lines = error.to_s.split("\n")
@@ -22,15 +26,12 @@ module Autoproj
         end
     end
 
+    # Displays a warning message
     def self.warn(message)
         STDERR.puts Autoproj.console.color("  WARN: #{message}", :magenta)
     end
 
-    @definition_files = Hash.new
     @file_stack       = Array.new
-    class << self
-        attr_reader :definition_files
-    end
 
     def self.package_name_from_options(spec)
         if spec.kind_of?(Hash)
@@ -78,16 +79,18 @@ module Autoproj
     end
 end
 
+# Sets up a documentation target on pkg that runs 'rake <target>'
 def ruby_doc(pkg, target = 'doc')
     pkg.doc_task do
+        pkg.progress "generating documentation for %s"
         pkg.doc_disabled unless File.file?('Rakefile')
         Autobuild::Subprocess.run pkg.name, 'doc', 'rake', target
     end
 
 end
 
-# Common setup for packages hosted on groupfiles/Autonomy
-def package_common(package_type, spec)
+# Common setup for packages
+def package_common(package_type, spec) # :nodoc:
     package_name = Autoproj.package_name_from_options(spec)
 
     begin
@@ -108,6 +111,20 @@ def package_common(package_type, spec)
     end
 end
 
+def import_package(options, &block)
+    package_common(:import, options, &block)
+end
+
+# Define a cmake package
+#
+# Example:
+#
+#   cmake_package 'package_name' do |pkg|
+#       pkg.define "CMAKE_BUILD_TYPE", "Release"
+#   end
+#
+# +pkg+ is an Autobuild::CMake instance. See the Autobuild API for more
+# information.
 def cmake_package(options, &block)
     package_common(:cmake, options) do |pkg|
         Autoproj.add_build_system_dependency 'cmake'
@@ -123,8 +140,15 @@ def cmake_package(options, &block)
     end
 end
 
-# Use this method to import and build CMake packages that are hosted on the
-# groupfiles server, on the autonomy project folder
+# Define an autotools package
+#
+# Example:
+#   autotools_package 'package_name' do |pkg|
+#       pkg.configureflags << "--enable-llvm"
+#   end
+#
+# +pkg+ is an Autobuild::Autotools instance. See the Autobuild API for more
+# information.
 def autotools_package(options, &block)
     package_common(:autotools, options) do |pkg|
         Autoproj.add_build_system_dependency 'autotools'
@@ -140,20 +164,32 @@ def autotools_package(options, &block)
     end
 end
 
-def ruby_common(pkg)
+# Common setup for Ruby packages
+def ruby_common(pkg) # :nodoc:
     def pkg.prepare_for_forced_build
         super
         extdir = File.join(srcdir, 'ext')
         if File.directory?(extdir)
-            FileUtils.rm_rf File.join(extdir, "build", "CMakeCache.txt")
-            FileUtils.rm_rf File.join(extdir, "Makefile")
+            Find.find(extdir) do |file|
+                next if file !~ /\<Makefile\>|\<CMakeCache.txt\>$/
+                FileUtils.rm_rf file
+            end
         end
     end
     def pkg.prepare_for_rebuild
         super
         extdir = File.join(srcdir, 'ext')
         if File.directory?(extdir)
-            FileUtils.rm_rf File.join(extdir, "build")
+            Find.find(extdir) do |file|
+                if file =~ /Makefile/
+                    Autobuild::Subprocess.run self, 'build', Autobuild.tool("make"), "-C", File.dirname(file), "clean"
+                end
+            end
+            Find.find(extdir) do |file|
+                if File.directory?(file) && file == "build"
+                    FileUtils.rm_rf file
+                end
+            end
         end
     end
 
@@ -180,6 +216,17 @@ def env_add(name, value)
     Autoproj.env_add(name, value)
 end
 
+
+# Defines a Ruby package
+#
+# Example:
+#   
+#   ruby_package 'package_name' do |pkg|
+#       pkg.doc_target = 'doc'
+#   end
+#
+# +pkg+ is an Autobuild::Importer instance. See the Autobuild API for more
+# information.
 def ruby_package(options)
     package_common(:import, options) do |pkg|
         class << pkg
@@ -194,12 +241,25 @@ def ruby_package(options)
     end
 end
 
+# Defines an oroGen package. By default, autoproj will look for an orogen file
+# called package_basename.orogen if the package is called dir/package_basename
+#
+# Example:
+#   orogen_package 'package_name' do |pkg|
+#       pkg.orogen_file = "my.orogen"
+#       pkg.corba = false
+#   end
+#
+# +pkg+ is an Autobuild::Orogen instance. See the Autobuild API for more
+# information.
 def orogen_package(options, &block)
     package_common(:orogen, options) do |pkg|
         yield(pkg) if block_given?
     end
 end
 
+# Defines an import-only package, i.e. a package that is simply checked out but
+# not built in any way
 def source_package(options)
     package_common(options) do |pkg|
         pkg.srcdir   = pkg.name
@@ -207,11 +267,17 @@ def source_package(options)
     end
 end
 
+# Define a configuration option
+#
+# See Autoproj.configuration_option
 def configuration_option(*opts, &block)
     Autoproj.configuration_option(*opts, &block)
 end
 
-def user_config(*opts)
-    Autoproj.user_config(*opts)
+# Retrieves the configuration value for the given option
+#
+# See Autoproj.user_config
+def user_config(key)
+    Autoproj.user_config(key)
 end
 

data/lib/autoproj/default.osdeps

@@ -1,24 +1,44 @@
 # The following definitions are needed to bootstrap autoproj
 none: ignore
 
-ruby:
+ruby18:
     debian,ubuntu:
         - ruby1.8-dev
         - ruby1.8
+        - rubygems1.8
+        - ri1.8
         - libopenssl-ruby1.8
     gentoo:
         - dev-lang/ruby:1.8
 
+ruby19:
+    debian,ubuntu:
+        - ruby1.9.1
+        - ruby1.9.1-dev
+        - rubygems1.9.1
+        - ri1.9.1
+        - libopenssl-ruby1.9.1
+    gentoo:
+        - dev-lang/ruby:1.9
+    arch:
+        - ruby
+
+rdoc: gem
+
 build-essential:
     debian,ubuntu: build-essential
     gentoo:
+    arch:
 
 libxml2:
     debian,ubuntu: libxml2-dev
     gentoo: dev-libs/libxml2
+    arch: libxml2
+
 libxslt:
     debian,ubuntu: libxslt1-dev
     gentoo: dev-libs/libxslt
+    arch: libxslt
 
 autobuild: gem
 autoproj: gem
@@ -27,26 +47,30 @@ autoproj: gem
 git:
     debian,ubuntu: git-core
     gentoo: dev-util/git
-
+    arch: git
 svn:
     debian,ubuntu: subversion
     gentoo: dev-util/subversion
-
+    arch: subversion
 cmake:
     debian,ubuntu: cmake
     gentoo: dev-util/cmake
+    arch: cmake
 
 autotools:
     debian,ubuntu: [automake1.9, autoconf]
     gentoo: [sys-devel/automake:1.9, sys-devel/autoconf]
+    arch: automake autoconf
 
 lsb_release:
     debian,ubuntu: lsb-release
     gentoo: sys-apps/lsb-release
+    arch:
     
 archive:
     debian,ubuntu: [tar, unzip]
     gentoo: [app-arch/tar, app-arch/unzip]
+    arch: [tar, unzip]
 
 # vim: expandtab