autoproj 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,139 @@
1
+ ---
2
+ title: Creating a package set
3
+ sort_info: 100
4
+ ---
5
+ A package set is made of three things:
6
+
7
+ * the description file (source.yml)
8
+ * an optional initialization script (init.rb)
9
+ * autobuild scripts (\*.autobuild)
10
+
11
+ Autobuild scripts
12
+ -----------------
13
+ The autobuild scripts lists all the packages defined by this set. It is a
14
+ Ruby script (but you don't need to know Ruby to write one). In its most simple
15
+ form, it looks like:
16
+
17
+ {coderay:: ruby}
18
+ cmake_package "orocos/rtt"
19
+ autotools_package "drivers/imu"
20
+ {coderay}
21
+
22
+ The above snippet defines two packages. The first one uses CMake as a build
23
+ system and will be installed in the <tt>orocos/rtt</tt> subdirectory of the
24
+ autoproj installation. The second one is an autotools package. There is, for
25
+ now, only support for these two build systems, and for Ruby packages. Package
26
+ definitions can be tweaked quite a lot, including the ability to generate
27
+ documentation. See [the next page](autobuild.html) for more information on how to write autobuild
28
+ scripts.
29
+
30
+ source.yml
31
+ ----------
32
+ The source.yml file gives generic information on the package set itself (most
33
+ importantly its name), and version control information (i.e. where to get the
34
+ packages). It is a YAML file, and looks like:
35
+
36
+ {coderay:: yaml}
37
+ name: dfki.orocos
38
+ constants:
39
+ ROOT_DIR: $HOME/share
40
+
41
+ version_control:
42
+ - "modules/.*":
43
+ type: git
44
+ url: $ROOT_DIR/$PACKAGE.git
45
+ - "drivers/.*":
46
+ type: svn
47
+ url: svn+ssh://rlbsvn.informatik.uni-bremen.de/trunk/$PACKAGE.git
48
+ {coderay}
49
+
50
+ The name field gives a name for the set. It is arbitrary, but the guideline
51
+ is to give a name that is java-like for namespaces, i.e. origin.name.
52
+
53
+ The <tt>constants:</tt> section lists values that can be reused for different
54
+ packages. Autoproj defines the HOME constant which is the user's home directory,
55
+ and the PACKAGE constant that is the actual package name. We will see later that
56
+ these values can also be configuration variables that are asked to the user.
57
+
58
+ Finally, the <tt>version_control:</tt> section describes how to import each
59
+ software package. Its general format is:
60
+ {coderay:: yaml}
61
+ package_name:
62
+ type: version_control_type # git, svn, cvs, darcs
63
+ url: repository_url
64
+ {coderay}
65
+
66
+ Where package\_name is a regular expression that matches the package name, i.e.
67
+ use ".\*" to match all packages for instance. The package name is the one given
68
+ to the <tt>blabla_package</tt> stanza in the autobuild file. In the above
69
+ autobuild example, the package names are for instance "orocos/rtt" and
70
+ "drivers/imu".
71
+
72
+ For the git importer, one of 'branch' or 'tag' options can be provided as well:
73
+ {coderay:: yaml}
74
+ package_name:
75
+ branch: branch_to_track
76
+ tag: tag_to_stick_to # it is branch OR tag
77
+ {coderay}
78
+
79
+ The options are applied in order, meaning that the toplevel options override the
80
+ lower level ones. In general, one will have a ".\*" entry to give options for all
81
+ packages, and then override for specific packages:
82
+
83
+ {coderay:: yaml}
84
+ version_control:
85
+ - "modules/logger": # we don't follow master on this module
86
+ branch: imoby
87
+
88
+ - .*: # common options for all packages
89
+ type: git
90
+ url: $ROOT_DIR/$PACKAGE.git
91
+ {coderay}
92
+
93
+ Interaction between package sets definition files
94
+ -------------------------------------------
95
+ When multiple package sets are used, it is possible to override the version
96
+ control definitions in low-priority sets with a higher priority one. Autoproj
97
+ has the following rules when searching for version control information:
98
+
99
+ * autoproj looks at the package sets *in the order they appear in the
100
+ installation manifest*
101
+ * autoproj searches a relevant version\_control field, and stops at the first
102
+ package set that has one.
103
+ * autoproj *stops searching* at the package set that defines the package.
104
+ Consequence: this set *must* have a version\_control field for it, and an
105
+ error will be issued otherwise.
106
+
107
+ Using configuration options
108
+ ---------------------------
109
+ autoproj offers a configuration system that allows the user to tweak the build
110
+ to its needs. If the version control definitions depend on such configuration
111
+ options (as, for instance, because you want to parametrize the importer URLs),
112
+ then create an init.rb file next to the source.yml file, and add lines looking
113
+ like:
114
+
115
+ {coderay:: ruby}
116
+ configuration_option "option_name", "option_type",
117
+ :default => "default_value",
118
+ :values => ["set", "of", "possible", "values"],
119
+ :doc => "description of the option"
120
+ {coderay}
121
+
122
+ where the option\_type field can either be "string" or "boolean".
123
+
124
+ Then, you can use the option\_name as an expansion in the source.yml file.
125
+
126
+ For instance, with an init.rb looking like:
127
+ {coderay:: ruby}
128
+ configuration_option "MOUNT_POINT", "string",
129
+ :default => "$HOME/nfs",
130
+ :doc => "mount point of the NFS server"
131
+ {coderay}
132
+
133
+ {coderay:: yaml}
134
+ version_control:
135
+ ".*":
136
+ url: $MOUNT_POINT/git/$PACKAGE.git
137
+ type: git
138
+ {coderay}
139
+
@@ -0,0 +1,153 @@
1
+ ---
2
+ title: Managing autoproj installations
3
+ sort_info: 50
4
+ ---
5
+
6
+ This page describes the structure of a autoproj installation, and describes how
7
+ to manage one. See [the introduction](index.page) for the bootstrapping process.
8
+
9
+ Structure
10
+ ---------
11
+
12
+ A autoproj installation is characterized by a autoproj/ directory in which all
13
+ autoproj-related configuration is saved. The contained files are as follows:
14
+
15
+ * autoproj/manifest: list of available package sets, layout of the
16
+ installation. See "Management" below.
17
+ * autoproj/remotes/\*: package sets that are imported from a remote version
18
+ control system
19
+ * autoproj/\*/: local sets, i.e. sets that have not been imported from a remote
20
+ version control system.
21
+
22
+ The build is done in two steps:
23
+
24
+ * each package is being built in a <tt>build</tt> subdirectory of the package's
25
+ source (<tt>package_directory/build/</tt>)
26
+ * it is then installed in the build/ directory at the toplevel of the autoproj
27
+ installation
28
+
29
+ Moreover, the <tt>build/log</tt> directory contains the output of all commands
30
+ that have been run during the build. Finally, a <tt>env.sh</tt> script is
31
+ generated to set up your shell for the use of the installed software.
32
+
33
+ Toplevel manifest
34
+ -----------------
35
+ The <tt>autoproj/manifest</tt> file is a yaml file which describes the package
36
+ sets that are available, and how packages are organized in the installation. It
37
+ looks like this:
38
+
39
+ {coderay:: yaml}
40
+ layout:
41
+ - autoproj.orocos
42
+ - asguard:
43
+ - dfki.imoby
44
+
45
+ package_sets:
46
+ - imoby
47
+ - type: git
48
+ url: git://github.com/doudou/autoproj-orocos.git
49
+ {coderay}
50
+
51
+ The first section, the <tt>layout</tt> section, lists the packages or package
52
+ sets we want to have in this installation. It is a layout, meaning that some packages
53
+ can be built and installed in a subdirectory of the main installation
54
+ (see "Layouts and dependencies" below). Packages are referred to by the name they
55
+ have in the autobuild file (see [this page for more details](source_yml.hml)).
56
+ Package sets are referred to by the name given in the [set's <tt>source.yml</tt>
57
+ file](source_yml.html), and are interpreted as "build all packages of the given
58
+ set".
59
+
60
+ The second section, the <tt>package_sets</tt> section, lists both local and remote
61
+ sets that are available to this installation. Local sets are
62
+ subdirectories of the <tt>autoproj/</tt> directory: for instance, in the above
63
+ example, autoproj will look at the <tt>autoproj/imoby/</tt> directory. Remote
64
+ sets are taken from remote version control systems. Its general format is:
65
+
66
+ {coderay:: yaml}
67
+ - type: version_control_type # git, svn, cvs, darcs
68
+ url: repository_url
69
+ {coderay}
70
+
71
+ For the git importer, one of 'branch' or 'tag' options can be provided as well:
72
+ {coderay:: yaml}
73
+ - type: version_control_type # git, svn, cvs, darcs
74
+ url: repository_url
75
+ branch: branch_to_track
76
+ tag: tag_to_stick_to # it is branch OR tag
77
+ {coderay}
78
+
79
+ Management
80
+ ----------
81
+
82
+ To update and build a autoproj installation, simply do:
83
+
84
+ autoproj build
85
+ {.commandline}
86
+
87
+ It will ask the value of newly defined configuration options, import (or update)
88
+ code hosted remotely, and build it. autoproj will *not* ask you again about the
89
+ configuration questions you already answered, so if you want to change them, do:
90
+
91
+ autoproj build --reconfigure
92
+ {.commandline}
93
+
94
+ Alternatively, you can edit the autoproj/config.yml file directly.
95
+
96
+ If you are in a disconnected environment (i.e. no access to remote
97
+ repositories), use the <tt>--no-update</tt> option to skip the update phase.
98
+ autoproj will still have to checkout new packages, though:
99
+
100
+ autoproj build --no-update
101
+ {.commandline}
102
+
103
+ If, on the other hand, you only want to update the sources, do
104
+
105
+ autoproj update
106
+ {.commandline}
107
+
108
+ To add a new set, one edits the <tt>autoproj/manifest</tt> file and adds it
109
+ there. Then, simply starting the build will update everything and rebuild what
110
+ is needed.
111
+
112
+ Documentation is generated only when asked explicitely:
113
+
114
+ autoproj doc
115
+ {.commandline}
116
+
117
+ Building packages selectively on the command line
118
+ -------------------------------------------------
119
+
120
+ All the build-related commands given above (i.e. build, doc, and update) can be
121
+ given a package name or a name used as a subdirectory in the layout section of
122
+ the manifest.
123
+
124
+ In the first case, only the package and the dependencies _that are on the same
125
+ level on the installation layout_ are built. It means that with the following
126
+ layout:
127
+
128
+ {coderay:: yaml}
129
+ layout:
130
+ - typelib
131
+ - asguard:
132
+ - modules/base
133
+ - modules/logger
134
+ {coderay}
135
+
136
+ If the command line is
137
+
138
+ autoproj build modules/logger
139
+ {.cmdline}
140
+
141
+ then only modules/logger and modules/base will be built -- assuming
142
+ modules/logger depends on modules/base -- but typelib will be left alone
143
+ _regardless of its state_. It may speed up the build process tremendously, but
144
+ also may generate errors if other packages needed to be updated.
145
+
146
+ Idem, if the command line is:
147
+
148
+ autoproj build asguard
149
+ {.cmdline}
150
+
151
+ then all packages or asguard/ are built _but none of the dependencies that are
152
+ defined in other places in the layout_.
153
+
data/lib/autoproj.rb ADDED
@@ -0,0 +1,23 @@
1
+ module Autoproj
2
+ class ConfigError < RuntimeError; end
3
+ class InternalError < RuntimeError; end
4
+ end
5
+
6
+ require "enumerator"
7
+ require 'autoproj/manifest'
8
+ require 'autoproj/osdeps'
9
+ require 'autoproj/system'
10
+ require 'autoproj/options'
11
+ require 'logger'
12
+ require 'utilrb/logger'
13
+
14
+ module Autoproj
15
+ class << self
16
+ attr_reader :logger
17
+ end
18
+ @logger = Logger.new(STDOUT)
19
+ logger.level = Logger::WARN
20
+ logger.formatter = lambda { |severity, time, progname, msg| "#{severity}: #{msg}\n" }
21
+ extend Logger::Forward
22
+ end
23
+
@@ -0,0 +1,173 @@
1
+ require 'autobuild'
2
+ require 'set'
3
+
4
+ module Autoproj
5
+ class Reporter < Autobuild::Reporter
6
+ def error(error)
7
+ error_lines = error.to_s.split("\n")
8
+ STDERR.puts color("Build failed: #{error_lines.shift}", :bold, :red)
9
+ STDERR.puts error_lines.join("\n")
10
+ end
11
+ def success
12
+ STDERR.puts color("Build finished successfully at #{Time.now}", :bold, :green)
13
+ if Autobuild.post_success_message
14
+ puts Autobuild.post_success_message
15
+ end
16
+ end
17
+ end
18
+
19
+ def self.warn(message)
20
+ STDERR.puts Autoproj.console.color(" WARN: #{message}", :magenta)
21
+ end
22
+
23
+ @definition_files = Hash.new
24
+ @file_stack = Array.new
25
+ class << self
26
+ attr_reader :definition_files
27
+ end
28
+
29
+ def self.package_name_from_options(spec)
30
+ if spec.kind_of?(Hash)
31
+ spec.to_a.first.first.to_str
32
+ else
33
+ spec.to_str
34
+ end
35
+ end
36
+
37
+ def self.current_file
38
+ @file_stack.last
39
+ end
40
+
41
+ def self.define(package_type, spec, &block)
42
+ package = Autobuild.send(package_type, spec, &block)
43
+ Autoproj.manifest.register_package package, *current_file
44
+ end
45
+
46
+ @loaded_autobuild_files = Set.new
47
+ def self.import_autobuild_file(source, path)
48
+ return if @loaded_autobuild_files.include?(path)
49
+
50
+ @file_stack.push([source, File.basename(path)])
51
+ load path
52
+ @loaded_autobuild_files << path
53
+
54
+ ensure
55
+ @file_stack.pop
56
+ end
57
+ end
58
+
59
+ def ruby_doc(pkg, target = 'doc')
60
+ pkg.doc_task do
61
+ pkg.doc_disabled unless File.file?('Rakefile')
62
+ Autobuild::Subprocess.run pkg.name, 'doc', 'rake', target
63
+ end
64
+
65
+ end
66
+
67
+ # Common setup for packages hosted on groupfiles/Autonomy
68
+ def package_common(package_type, spec)
69
+ package_name = Autoproj.package_name_from_options(spec)
70
+
71
+ begin
72
+ Rake::Task[package_name]
73
+ Autoproj.warn "#{package_name} in #{Autoproj.current_file[0]} has been overriden in #{Autoproj.definition_source(package_name)}"
74
+ rescue
75
+ end
76
+
77
+ Autoproj.define(package_type, spec) do |pkg|
78
+ pkg.srcdir = pkg.name
79
+ yield(pkg) if block_given?
80
+ end
81
+ end
82
+
83
+ def cmake_package(options, &block)
84
+ package_common(:cmake, options) do |pkg|
85
+ Autoproj.add_build_system_dependency 'cmake'
86
+ yield(pkg) if block_given?
87
+ unless pkg.has_doc?
88
+ pkg.with_doc do
89
+ doc_html = File.join('doc', 'html')
90
+ if File.directory? doc_html
91
+ pkg.doc_dir = doc_html
92
+ end
93
+ end
94
+ end
95
+ end
96
+ end
97
+
98
+ # Use this method to import and build CMake packages that are hosted on the
99
+ # groupfiles server, on the autonomy project folder
100
+ def autotools_package(options, &block)
101
+ package_common(:autotools, options) do |pkg|
102
+ Autoproj.add_build_system_dependency 'autotools'
103
+ yield(pkg) if block_given?
104
+ unless pkg.has_doc?
105
+ pkg.with_doc do
106
+ doc_html = File.join('doc', 'html')
107
+ if File.directory? doc_html
108
+ pkg.doc_dir = doc_html
109
+ end
110
+ end
111
+ end
112
+ end
113
+ end
114
+
115
+ def ruby_common(pkg)
116
+ def pkg.prepare
117
+ super
118
+ Autobuild.update_environment srcdir
119
+ end
120
+
121
+ pkg.post_install do
122
+ Autobuild.progress " setting up Ruby package #{pkg.name}"
123
+ Autobuild.update_environment pkg.srcdir
124
+ if File.file?('Rakefile')
125
+ if File.directory?('ext')
126
+ Autobuild::Subprocess.run pkg.name, 'post-install', 'rake', 'setup'
127
+ end
128
+ end
129
+ end
130
+ end
131
+
132
+ def env_set(name, value)
133
+ Autoproj.env_set(name, value)
134
+ end
135
+ def env_add(name, value)
136
+ Autoproj.env_add(name, value)
137
+ end
138
+
139
+ def ruby_package(options)
140
+ package_common(:import, options) do |pkg|
141
+ class << pkg
142
+ attr_accessor :doc_target
143
+ end
144
+
145
+ ruby_common(pkg)
146
+ yield(pkg) if block_given?
147
+ unless pkg.has_doc?
148
+ ruby_doc(pkg, pkg.doc_target || 'redocs')
149
+ end
150
+ end
151
+ end
152
+
153
+ def orogen_package(options, &block)
154
+ package_common(:orogen, options) do |pkg|
155
+ yield(pkg) if block_given?
156
+ end
157
+ end
158
+
159
+ def source_package(options)
160
+ package_common(options) do |pkg|
161
+ pkg.srcdir = pkg.name
162
+ yield(pkg) if block_given?
163
+ end
164
+ end
165
+
166
+ def configuration_option(*opts, &block)
167
+ Autoproj.configuration_option(*opts, &block)
168
+ end
169
+
170
+ def user_config(*opts)
171
+ Autoproj.user_config(*opts)
172
+ end
173
+