autoproj 1.4.4 → 1.5.0

data/doc/guide/src/package_sets/autobuild.page

@@ -31,7 +31,7 @@ The above snippet being equivalent to calling <tt>cmake -DVAR=VALUE</tt>
 
 The "pkg" variable in the example above is an instance of
 [Autobuild::CMake](http://doudou.github.com/autobuild/Autobuild/CMake.html)
-{.block}
+{: .block}
 
 Defining autotools packages {#autotools}
 ---------------------------
@@ -46,7 +46,7 @@ end
 
 The 'pkg' variable in the example above is an instance of
 [Autobuild::Autotools](http://doudou.github.com/autobuild/Autobuild/Autotools.html)
-{.block}
+{: .block}
 
 Defining Ruby packages
 ----------------------
@@ -64,8 +64,36 @@ convention:
 
  * programs are in bin/
  * the library itself is in lib/
- * if an extension needs to be built, a Rakefile has to be there with a "setup"
-   target to do the job.
+
+If a Rakefile is present in the root of the source package, its <tt>default</tt>
+task will be called during the build, and its <tt>redocs</tt> task will be used
+for documentation generation. The <tt>rake_setup_task</tt> and
+<tt>rake_doc_task</tt> package properties can be used to override this default
+setting:
+
+{coderay::ruby}
+ruby_package "package_name" do |pkg|
+    pkg.rake_setup_task = "setup"
+    pkg.rake_doc_task = "doc:all"
+end
+{coderay}
+
+Additionally, they can be set to <tt>nil</tt> to disable either setup or documentation
+generation. For instance, the following code disables documentation generation
+and uses the +setup+ task at build time:
+
+{coderay::ruby}
+ruby_package "package_name" do |pkg|
+    pkg.rake_setup_task = "setup"
+    pkg.rake_doc_task = nil
+end
+{coderay}
+
+The 'pkg' variable in the example above is an instance of
+[Autobuild::ImporterPackage](http://doudou.github.com/autobuild/Autobuild/ImporterPackage.html),
+with additional methods coming from [the RubyPackage
+module](http://doudou.github.com/autoproj/api/Autoproj/RubyPackage.html)
+{: .block}
 
 Defining oroGen packages
 ------------------------
@@ -82,7 +110,7 @@ documentation](http://doudou.github.com/orogen) for more information.
 
 The 'pkg' variable in the example above is an instance of
 [Autobuild::Orogen](http://doudou.github.com/autobuild/Autobuild/Orogen.html)
-{.block}
+{: .block}
 
 Custom package building
 -----------------------
@@ -119,17 +147,13 @@ Inter-package dependencies can be defined with
 pkg.depends_on "package_name"
 {coderay}
 
-In the same way, if the source package depends on an OS package (see
-[Prepackaged dependencies](osdeps.html) for details), you can use
-
-{coderay:: ruby}
-pkg.depends_on_os_package "package_name"
-{coderay}
+Where package name is either the name of another autoproj-built package, or the
+name of a package that is to be [provided by the operating system](osdeps.html).
 
 Both methods should be used only for dynamic dependencies, i.e. dependencies
 that are dependent on build options (see below). Static dependencies should be
 defined in [the package's manifest.xml](manifest-xml.html)
-{.warning}
+{: .warning}
 
 Finally, it is possible to give aliases to a package's name, by using the
 Autobuild::Package#provides method. If one does
@@ -173,5 +197,5 @@ Do not try to have too many options, that is in general bad policy as
 non-advanced users won't be able to know what to answer. Advanced users will
 always have the option to override your autobuild definitions to tweak the
 builds to their needs.
-{.warning}
+{: .warning}
 

data/doc/guide/src/package_sets/importers.page

@@ -103,7 +103,7 @@ version_control:
 
 Note that the repository *must* be accessible without any password, and the
 import will fail if a password was needed.
-{.warning}
+{: .warning}
 
 Tar archives
 ------------

data/doc/guide/src/package_sets/manifest-xml.page

@@ -21,9 +21,9 @@ The general form of the file is:
   <url>http://sites.google.com/site/rubyinmotion</url>
   <logo>http://sites.google.com/site/rubyinmotion</logo>
 
-  <depend package="pkgname"/> <!-- add dependency on another autoproj-built package -->
+  <depend package="pkgname"/> <!-- add dependency on either another
+                               autoproj-built package, or an OS-provided one -->
   <depend package="common"/>
-  <rosdep name="python" /> <!-- add dependency on a prepackaged operating system package -->
 </package>
 {coderay}
 

data/doc/guide/src/package_sets/osdeps.page

@@ -3,21 +3,26 @@ title: Operating System dependencies
 sort_info: 300
 ---
 
-Each package set can have one or multiple .osdeps files. These files declare
-how to install prepackaged dependencies, based on the underlying operating system.
+Autoproj offers the possibility to use OS-packaged software instead of building
+it, to leverage the underlying platform's software. This page details how it's
+done.
 
 Defining dependencies between source packages and OS packages
 -------------------------------------------------------------
 
-These dependencies can be defined in two ways:
+If a source package depends on an OS package, this dependency should be declared
+in the same way than between  source packages:
 
- * by adding a `<osdep name="depname" />`  tag in the package's
-   [manifest.xml](manifest-xml.html)
- * by calling `pkg.depends_on_os_package("depname")` [in the autobuild
-   file](autobuild.html).
+ * either by adding the <depend package="os_dep_name" /> tag in the
+   manifest.xml, or
+ * by calling #depends_on("os_dep_name") in the autobuild file.
 
-In both cases, depname is used as a key to find the OS package definition in the
-osdeps files (see below). It does not have to be the actual package name.
+The os_dep_name above being the name of the package as declared in the OS
+dependencies files defined below.
+
+During the build, autoproj looks first for OS dependencies. If no dependency is
+available for that particular platform, it will then look for a source package
+definition and build it. If none of the two exist, an error is returned.
 
 OS packages
 -----------

data/doc/guide/src/quick_start.page

@@ -0,0 +1,110 @@
+---
+title: Quick start
+sort_info: 25
+---
+
+Bootstrapping
+-------------
+"Bootstrapping" means getting autoproj itself before it can work its magic ...
+The canonical way is the following:
+
+ * install Ruby by yourself. On Debian or Ubuntu, this is done with
+   done with
+
+   sudo apt-get install wget ruby
+   {: .cmdline}
+
+ * then, [download this script](autoproj_bootstrap) *in the directory where
+   you want to create an autoproj installation*, and run it. This can be done with
+
+   wget http://doudou.github.com/autoproj/autoproj\_bootstrap <br />
+   ruby autoproj\_bootstrap
+   {: .cmdline}
+
+ * follow the instructions printed by the script<tt>manifest</tt>.
+
+Additionally, if you are given a reference to a source code repository in which
+an autoproj configuration is stored (i.e. a directory in which a manifest is
+present), you can bootstrap this configuration directly:
+
+   wget http://doudou.github.com/autoproj/autoproj\_bootstrap <br />
+   ruby autoproj\_bootstrap VCS 
+   {: .cmdline}
+
+For instance, to build all packages made available by the RubyInMotion project,
+do
+
+   wget http://doudou.github.com/autoproj/autoproj\_bootstrap <br />
+   ruby autoproj\_bootstrap git git://github.com/doudou/rubim.all.git
+   {: .cmdline}
+
+Additional options can be given for the version control system. For instance,
+
+   wget http://doudou.github.com/autoproj/autoproj\_bootstrap <br />
+   ruby autoproj\_bootstrap git git://github.com/doudou/rubim.all.git branch=stable
+   {: .cmdline}
+
+Management
+----------
+
+To build the packages, simply do:
+
+autoproj build
+{: .commandline}
+
+It will ask the value of newly defined configuration options, import code hosted
+remotely that is not yet present, install OS packages (e.g. Ubuntu packages on
+an Ubunut installation) and build everything.
+
+autoproj will *not* ask you again about the configuration questions you already
+answered, so if you want to change them, do:
+
+autoproj build --reconfigure
+{: .commandline}
+
+Alternatively, you can edit the autoproj/config.yml file directly.
+
+By default, autoproj does not automatically update the package sources. To do
+that, you have to explicitely call
+
+autoproj update
+{: .commandline}
+
+Finally, if you want to make sure that your build is fresh, then do
+
+autoproj rebuild
+{: .commandline}
+
+A less intrusive version of it only forces all tools to reconsider building. It
+is mainly useful for CMake when the build environment changed -- cmake caches a
+lot of values. To trigger this, do
+
+autoproj force-build
+{: .commandline}
+
+To add a new set, one edits the <tt>autoproj/manifest</tt> file and adds it
+there. Then, simply starting the build will update everything and rebuild what
+is needed.
+
+Documentation is generated only when asked explicitely:
+
+autoproj doc
+{: .commandline}
+
+It generates documentation of packages that have some, and copies that
+documentation into build/doc/, following the same layout than the source
+directory.
+
+All these commands (i.e. build, doc, and update) accept a package name as
+argument, thus triggering build only for this package and its dependencies. For
+instance:
+
+autoproj build orocos/rtt
+{: .commandline}
+
+**Exception**: to avoid long builds, the rebuild and force-build commands apply only
+to the packages given on the command line. E.g., autoproj rebuild orocos/rtt
+will only rebuild the orocos/rtt packages. If you want to rebuild both the
+packages and its dependencies, use the --with-depends options.
+{: .warning}
+

data/doc/guide/src/{structure.page → writing_manifest.page}

@@ -1,5 +1,5 @@
 ---
-title: Managing autoproj installations
+title: The manifest file.
 sort_info: 50
 ---
 
@@ -9,8 +9,8 @@ to manage one. See [the introduction](index.html) for the bootstrapping process.
 Structure
 ---------
 
-!overview.png!
-<img class="full" src="overview.png" />
+![Structure overview](overview.png)!
+{: .full}
 
 The autoproj configuration and build process goes like this:
 
@@ -35,11 +35,11 @@ is split like this:
 
  * autoproj/manifest: list of available package sets, package selection and
    installation layout (where to put what).
- * .remotes/\*: package sets that are imported from a remote version
-   control system
  * autoproj/\*/: local sets, i.e. sets that have not been imported from a remote
    version control system.
- * autoproj\/init.rb, autoproj\/overrides.rb and autoproj\/overrides.yml:
+ * autoproj/remotes/\*/: package sets that are imported from a remote version
+   control system
+ * autoproj/init.rb, autoproj/overrides.rb and autoproj/overrides.yml:
    installation customization
 
 The build is done in two steps:
@@ -91,66 +91,56 @@ autoproj installation. The importers that are available for configuration are
 the same than the ones available for the packages themselves, so see [this
 page](package_sets/importers.html#all_importers) for the list of available importers.
 
-Management
-----------
-
-To update and build a autoproj installation, simply do:
-
-autoproj build
-{.commandline}
-
-It will ask the value of newly defined configuration options, import (or update)
-code hosted remotely, and build it. autoproj will *not* ask you again about the
-configuration questions you already answered, so if you want to change them, do:
-
-autoproj build --reconfigure
-{.commandline}
+Listing the available packages.
+-----------------------------
+Once you have updated your manifest file to list all the package sets that you
+want to use, you can list all the packages that are now available with
 
-Alternatively, you can edit the autoproj/config.yml file directly.
+autoproj list-sets
+{: .commandline}
 
-If you are in a disconnected environment (i.e. no access to remote
-repositories), use the fast-build mode, to skip the package update and OS
-packages installation phases (autoproj will still check out new packages,
-though):
+Its output looks like this:
 
-autoproj fast-build
-{.commandline}
+<pre>
+rubim.orocos
+  local source in /home/doudou/dfki/imoby1.9/autoproj/orocos
+  packages:
+  | orocos/base      | git:/home/doudou/dfki/dfki-share/projects/all/Intelligent_Mobility/development/software/git/tools/orocos/base.git                           |
+  | orocos/logger    | git:/home/doudou/dfki/dfki-share/projects/all/Intelligent_Mobility/development/software/git/tools/orocos/logger.git branch=fast_log         |
+  | orocos/ocl       | git:/home/doudou/dfki/dfki-share/projects/all/Intelligent_Mobility/development/software/git/tools/orocos/ocl.git branch=rubim               |
+</pre>
 
-The package update can be disabled with --no-update and the OS dependencies
-installation with --no-osdeps. In practice, "autoproj fast-build" is equivalent
-to autoproj build --no-update --no-osdeps
-{.block}
+The first line is the **package set name**. It is defined in the package set's
+source.yml file and *does not have to be identical to the set's directory
+name*. In the example above, the package set is saved in autoproj/orocos but is
+called dfki.orocos.
 
-If, on the other hand, you only want to update the source code, do
+The second line tells you where this set comes from. It is local if it comes
+along with the main autoproj configuration (manifest and so on). It is remote
+if it is imported from a version control system.
 
-autoproj update
-{.commandline}
+Finally comes the list of packages that are defined in this set, along with the
+importer setup (URL and options, as for instance branches for orocos/logger
+above).
 
-Finally, if you want to make sure that your build is fresh, then do
+Picking the packages to build
+-----------------------------
+If you do not wish to build all the packages that are available (you rarely
+wish that), you have to list the desired packages in your manifest file.
 
-autoproj rebuild
-{.commandline}
+To do so, you will have to create a <tt>layout</tt> section and list the
+desired packages:
 
-A less intrusive version of it only forces all tools to reconsider building. It
-is mainly useful for CMake when the build environment changed -- cmake caches a
-lot of values. To trigger this, do
-
-autoproj force-build
-{.commandline}
-
-To add a new set, one edits the <tt>autoproj/manifest</tt> file and adds it
-there. Then, simply starting the build will update everything and rebuild what
-is needed.
-
-Documentation is generated only when asked explicitely:
-
-autoproj doc
-{.commandline}
+{coderay:: yaml}
+layout:
+    - rubim.base
+    - orocos/rtt
+    - orocos/logger
+{coderay}
 
-All these commands (i.e. build, doc, and update) accept a package name as
-argument, thus triggering build only for this package and its dependencies. For
-instance:
+This layout can either list packages one by one, but complete package sets can
+also be selected (the rubim.base above)
 
-autoproj build orocos/rtt
-{.commandline}
+More advanced mechanisms are available to customize this list. These mechanisms
+are [detailed here](customization.html)
 

data/lib/autoproj.rb

@@ -4,6 +4,7 @@ module Autoproj
 end
 
 require "enumerator"
+require 'autoproj/version'
 require 'autoproj/manifest'
 require 'autoproj/osdeps'
 require 'autoproj/system'

data/lib/autoproj/autobuild.rb

@@ -3,9 +3,29 @@ require 'fileutils'
 require 'autobuild'
 require 'set'
 
-class Autobuild::Package
-    def autoproj_name # :nodoc:
-        srcdir.gsub /^#{Regexp.quote(Autoproj.root_dir)}\//, ''
+module Autobuild
+    class Package
+        def autoproj_name # :nodoc:
+            srcdir.gsub /^#{Regexp.quote(Autoproj.root_dir)}\//, ''
+        end
+
+        alias __depends_on__ depends_on
+        def depends_on(name)
+            if Autoproj.osdeps.has?(name)
+                @os_packages ||= Set.new
+                @os_packages << name
+            else
+                __depends_on__(name)
+            end
+        end
+
+        def depends_on_os_package(name)
+            depends_on(name)
+        end
+
+        def os_packages
+            @os_packages ||= Set.new
+        end
     end
 end
 
@@ -80,16 +100,6 @@ module Autoproj
     end
 end
 
-# Sets up a documentation target on pkg that runs 'rake <target>'
-def ruby_doc(pkg, target = 'doc')
-    pkg.doc_task do
-        pkg.progress "generating documentation for %s"
-        pkg.doc_disabled unless File.file?('Rakefile')
-        Autobuild::Subprocess.run pkg.name, 'doc', 'rake', target
-    end
-
-end
-
 # Common setup for packages
 def package_common(package_type, spec, &block) # :nodoc:
     package_name = Autoproj.package_name_from_options(spec)
@@ -164,9 +174,10 @@ def autotools_package(options, &block)
     end
 end
 
-# Common setup for Ruby packages
-def ruby_common(pkg) # :nodoc:
-    def pkg.prepare_for_forced_build
+# This module is used to extend importer packages to handle ruby packages
+# properly
+module Autoproj::RubyPackage
+    def prepare_for_forced_build # :nodoc:
         super
         extdir = File.join(srcdir, 'ext')
         if File.directory?(extdir)
@@ -176,7 +187,8 @@ def ruby_common(pkg) # :nodoc:
             end
         end
     end
-    def pkg.prepare_for_rebuild
+
+    def prepare_for_rebuild # :nodoc:
         super
         extdir = File.join(srcdir, 'ext')
         if File.directory?(extdir)
@@ -194,20 +206,22 @@ def ruby_common(pkg) # :nodoc:
         end
     end
 
-    def pkg.prepare
+    def import
         super
-        Autobuild.update_environment srcdir
-    end
 
-    pkg.post_install do
-        Autobuild.progress "setting up Ruby package #{pkg.name}"
-        Autobuild.update_environment pkg.srcdir
-        if File.file?('Rakefile')
-            if File.directory?('ext')
-                Autobuild::Subprocess.run pkg.name, 'post-install', 'rake', 'setup'
-            end
+        Autobuild.update_environment srcdir
+        libdir = File.join(srcdir, 'lib')
+        if File.directory?(libdir)
+            Autobuild.env_add_path 'RUBYLIB', libdir
         end
     end
+
+    # The Rake task that is used to set up the package. Defaults to "default".
+    # Set to nil to disable documentation generation
+    attr_accessor :rake_setup_task
+    # The Rake task that is used to generate documentation. Defaults to "doc".
+    # Set to nil to disable documentation generation
+    attr_accessor :rake_doc_task
 end
 
 def env_set(name, value)
@@ -230,14 +244,42 @@ end
 # information.
 def ruby_package(options)
     package_common(:import, options) do |pkg|
-        class << pkg
-            attr_accessor :doc_target
+        pkg.exclude << /\.so$/
+        pkg.exclude << /Makefile$/
+        pkg.exclude << /mkmf.log$/
+        pkg.exclude << /\.o$/
+
+        pkg.extend Autoproj::RubyPackage
+        pkg.rake_setup_task = "default"
+        pkg.rake_doc_task   = "redocs"
+
+        # Set up code
+        pkg.post_install do
+            Autobuild.progress "setting up Ruby package #{pkg.name}"
+            Autobuild.update_environment pkg.srcdir
+            # Add lib/ unconditionally, as we know that it is a ruby package.
+            # Autobuild will add it only if there is a .rb file in the directory
+            libdir = File.join(pkg.srcdir, 'lib')
+            if File.directory?(libdir)
+                Autobuild.env_add_path 'RUBYLIB', libdir
+            end
+
+            if pkg.rake_setup_task && File.file?(File.join(pkg.srcdir, 'Rakefile'))
+                Autobuild::Subprocess.run pkg, 'post-install',
+                    'rake', pkg.rake_setup_task
+            end
         end
 
-        ruby_common(pkg)
         yield(pkg) if block_given?
-        unless pkg.has_doc?
-            ruby_doc(pkg, pkg.doc_target || 'redocs')
+
+        # Documentation code. Ignore if the user provided its own documentation
+        # task, or disabled the documentation generation altogether by setting
+        # rake_doc_task to nil
+        if !pkg.has_doc? && pkg.rake_doc_task
+            pkg.doc_task do
+                pkg.progress "generating documentation for %s"
+                Autobuild::Subprocess.run pkg, 'doc', 'rake', pkg.rake_doc_task
+            end
         end
     end
 end