roby 0.7

data/TODO.txt

@@ -0,0 +1,146 @@
+= TODO
+
+== Refactoring
+=== Small stuff
+* remote_id => network_id
+* incremental_dump? => dump_reference?
+
+=== Big stuff
+* fix the Roby singleton methods/Roby::Control/Roby::Propagation mess. Maybe move
+  all the execution-related stuff in a subclass of Plan, MainPlan. That would allow
+  per-MainPlan execution management and in the end having multiple executable
+  plan in the same plan manager (I'm thinking bi-simulation)
+* have separate relation graphs in each Plan object. That would be possible
+  if we don't allow anymore to change any event or task relations if the task
+  is not included in a plan, which would greatly simply logging-related stuff
+  (for instance, rebuilding a representation of plans from logged data).
+  Moreover, we would not have tasks hanging around anymore because they have
+  never been removed from any plan.
+* in dRoby, separate the implementation of the RX/TX and
+  connection/disconnection protocol, including the management of the TX queue
+  (callbacks, synchronous/asynchronous).
+* have a thread-safe handling of transactions. This requires a few things:
+  - separate relation graphs: have one relation graph per plan object: see previous
+    point.
+  - stop using method forwarding, which is not the good way to do that.
+    Instead, do as for distributed: update the task status in the transaction
+    thread. Need to define a notion of "transaction thread" and how messages
+    can get propagated there, though.
+
+== Core
+* dynamic parameters. State prediction in both plan and transaction context:
+  allow to predict the effect of changing the value of dynamic parameters in
+  transaction context.
+
+* when predicates like happened? are called in a propagation context, and
+  we don't know the result yet (happened? = false), register a continuation and
+  its dependency on the event. Include this continuation in the event set and
+  sort it accordingly. 
+
+* in #replace, use Hierarchy#fullfilled_events to check that
+  all needed events are provided. If some are missing, use define_event to try
+  to add a new one dynamically
+
+* when replacing an event generator, what to do with its handlers ?
+  - either we discard them, because some handlers are used in a specific way
+  - either we apply them on the new task
+  - unfortunately there is no good answer... Both are useful and it is
+    difficult (if not impossible) to know what to do. Having no solution for
+    this problem reduces the usefulness of event handlers greatly.
+
+* Need to tag relations with a name so that we can find what is what (kind of
+  'roles'). For instance, in PlanningLoop, we would tag the seeds (i.e. the
+  planned tasks) with a certain name which would allow to add any child without
+  the loop code noticing
+
+* add a multiplicity parameter for hierarchy relations which tells
+  how many child task of a given kind are expected by the parent task. Add a
+  'realized_by' transition for that. For instance, in case of
+  Pom::Localization, we can tell that the task expects at least one
+  Pom::Estimator. If the last estimator breaks, we can repair the plan online
+  by adding a transition.
+
+* we NEED plan merging: if we reuse a task which is already running, it should
+  be transparent for the new process: this new task tree will call start!, but
+  the task is running.  Moreover, if it is synchronizing on the start event, it
+  should appear "as if" the event has been emitted
+
+* Check the capability we have to put every application block in transaction
+  context. This would allow for instance to discard **all** plan modification
+  done by an exception handler (or plan command, event handler, you get my
+  drift) if an exception is raised. I think it would be a worthy feature [This
+  can't be done because of the transaction creation cost]
+
+* Kill local tasks that are in force_gc even if they have parents, if their
+  parents are remote tasks
+
+* rename Roby::TaskModelTag::argument into 'arguments' (beware of name clash
+  with the accessor)
+
+* Fix transaction proxying. Better use the same kind of model than in dRoby:
+  create slightly modified tasks of the same model than the proxied event, do
+  not proxy task events
+
+* Find a way to remove the "return unless proxying?" in top of proxy_code. It sucks
+
+* fix handling of execution agents: we should re-spawn the execution agents
+  only if there is an execution agent model on the task model. We currently use
+  respawn on the execution agent instance.  The problem lies in, for instance,
+  ConnectionTask which are respawned for all pending remote tasks.
+  ConnectionTask should never be created because of the exected_by relation.
+
+== Planning
+* fix memory leak regarding planning loops: for now, make_loop creates a new
+  planning method each time make_loop is called. The problem is that the
+  created method should actually be local to the planner object, not the
+  planner model (we are doing the latter right now).
+
+== Distributed
+* extension of communication to really have a multi-robot system. It is for now
+  too much peer-to-peer
+  - a pDB should always send the messages about objects it owns. For now, we
+    check two things: that the involved objects are owned and that the message
+    comes from the inside (we never forward a message coming from outside).
+    There is a synchronization problem here:
+
+	A sends a message M about a joint ab object to B
+	B does not send the message to C
+	but B can send other messages to C, including the /consequences/ of the M message
+
+	Result: C knows about the consequences of M, but not about M itself.
+
+    The solution is to mark all messages with an UID, and forward new messages
+    when we decode them (in PeerServer#demux)
+  - make sure unknown models are properly handled in 3-peers scenario. In
+    particular, the following situation should work
+
+	peer 1 and 2 know about a SpecificModel task model
+	peer 3 does not
+
+    If 1 sees tasks of 2 *through* the plan of 3, and if these tasks are of the
+    SpecificModel model, then they should be instances of the SpecificModel
+    class 
+
+* There is a race condition when removing objects: if a peer A is building a
+  transaction involving tasks from a peer B, and if the peer B removes the
+  tasks, then there is a race condition between the time B sends the
+  transaction and the time it processes the remove_object update from A. The
+  net effect is that when the transaction sibling is created, some tasks it
+  includes do not exist anymore.  It is more general, since the same problem
+  exists when wrapping tasks in a transaction for instance.
+
+  I don't think we should really fix the race condition per se. We should in
+  general check if the object has not been removed when A processes updates
+  from B and have some ways to feedback B from A.
+
+* Better handling of errors during transactions, better management of new
+  peers, ... DecisionControl should really be the central component handling
+  all that. Moreover, defining management policies for distributed transactions
+  should also be defined.
+
+== UI
+* log snapshot: build a set of logging messages which represent the current
+  state of the plan. Useful for display (it would allow to seek a lot faster)
+  and for runtime display of the plan state (send the snapshot on connection)
+
+ vim: tw=100

data/app/README.txt

@@ -0,0 +1,24 @@
+== Directories
+A basic Roby application has the following directories:
+config:: configuration files. config/init.rb is the main configuration file (loaded
+	 by all robots). Robot-specific configuration is in config/ROBOTNAME.rb.
+	 The main Roby configuration file is config/roby.yml. The default file
+         describes all available configuration options.
+planners:: planner models. Global planners (shared by all robots) are in
+	   planners/. Robot-specific planners are in planners/ROBOTNAME/
+controllers:: robot controllers. These files are supposed to start the basic robot
+	      services, to make the robot ready. A robot shall have a controllers/ROBOTNAME.rb
+	      file which does that.
+tasks:: task models
+data:: where all data files are. See #find_data.
+scripts:: various scripts needed to run and debug a Roby application
+
+The basic directory structure, and the global files, are installed by <tt>roby init</tt>. Basic
+robot files can be added by <tt>roby robot ROBOTNAME</tt>
+
+== Genom/Pocosim integration
+An application can use the Genom/Pocosim integration by calling <tt>roby init --module genom</tt>.
+The following files and directories are added:
+config/ROBOTNAME-genom.rb:: Genom-specific configuration for ROBOTNAME
+tasks/genom/:: per-Genom module tasks
+

data/app/Rakefile

@@ -0,0 +1,8 @@
+require 'rake/rdoctask'
+
+Rake::RDocTask.new('doc') do |doc|
+    doc.rdoc_files.include 'README'
+    doc.rdoc_files.include 'tasks/**/*.rb'
+    doc.rdoc_files.include 'planners/**/*.rb'
+    doc.rdoc_files.include 'controllers/**/*.rb'
+end

data/app/config/ROBOT.rb

@@ -0,0 +1,5 @@
+# Add robot-specific configuration here. This file is loaded after config/init.rb
+# and after Roby has been set up
+Roby::State.update do |s|
+end
+

data/app/config/app.yml

@@ -0,0 +1,91 @@
+# Control configuration
+control:
+  # Abort if a task/event exception is not caught 
+  # abort_on_exception: false
+  # Abort if an application exception occurs 
+  # abort_on_application_exception: false
+  
+  # The executive. Set to 'none' to disable.
+  # The executive object determines, in the current plan, what are
+  # the tasks which must be started. The 'simple' executive starts
+  # all tasks that have no predecessors.
+  executive: simple
+
+  # The length of a cycle (in seconds)
+  # cycle: 0.1
+
+# Logging configuration
+log:
+  # Where log files go
+  # dir: log/
+  
+  # Log system events (default: false)
+  # events: true
+  
+  # Disables the log server (default: enabled)
+  # server: false
+  #
+  # ... or configure it
+  # server:
+  #     # The port to listen on (for service discovery)
+  #     port: 48933
+  #     # The discovery period
+  #     period: 10
+
+  # Logging levels. It is a hash of component_name: level (where level is one
+  # of DEBUG, INFO, WARN and FATAL). The output can be redirected in a file
+  # using level:filename, in which case 'filename' is relative to the log
+  # directory. Any occurence of 'ROBOT' in filename is replaced by the robot
+  # name
+  #
+  # levels:
+  #   roby: INFO
+  #   roby/planning: DEBUG
+  #   roby/distributed: DEBUG:ROBOT-distributed.log
+  #   genom: INFO
+
+# dRoby neighbour discovery
+discovery:
+  # The host:port at which we can connect to a discovery tuplespace
+  # (default: no discovery tuplespace)
+  # tuplespace: "localhost:29568"
+  # The port at which we do ring discovery (default: disabled)
+  # ring: 42865
+
+# dRoby host configuration
+droby:
+  # How many errors are allowed before disconnecting from a peer
+  # max_errors: 1
+  # The ring discovery period if we do ring discovery
+  # period: 0.5
+  # The hostname at the dRoby server must listen to. It is either
+  # 'host:port', or just ':port'
+  # host: ":48902"
+
+# Genom configuration
+genom:
+  # The memory size allocated for H2
+  # mem_size: 4000000
+  # If true, do not remove H2 devices when the application quit
+  # keep_h2: false
+  
+# Pocosim configuration
+pocosim:
+  # The host of the GDHE display
+  # display: localhost
+
+  # An initialization script to display the scene on GDHE. The file is searched
+  # in the data/ director
+  # gdhe: lousa.gdhe
+  
+  # The Gazebo world file for pocosim/gazebo simulations. The file is searched
+  # in the data/ directory 
+  # gazebo: lousa.world
+
+# Per-robot configuration. In these, you can override global options, or set
+# new options.
+# dala:
+#   droby:
+#     host: ":1287"
+
+# vim: sw=2

data/app/config/init.rb

@@ -0,0 +1,7 @@
+# This file is called to do application-global configuration. For configuration
+# specific to a robot, edit config/NAME.rb, where NAME is the robot name.
+#
+# Enable some of the standard plugins
+# Roby.app.using 'fault_injection'
+# Roby.app.using 'subsystems'
+

data/app/config/roby.yml

@@ -0,0 +1,3 @@
+# DO NOT CHANGE THIS FILE
+# It is automatically updated by the roby tool. Any change will be erased
+plugins: []

data/app/controllers/ROBOT.rb

@@ -0,0 +1,2 @@
+# This is the robot controller. This file is required last, after Roby has been
+# fully set up. If you have to initialize some services at startup, to it here

data/app/planners/ROBOT/main.rb

@@ -0,0 +1,6 @@
+require 'planners/main'
+
+class MainPlanner
+    # Add here robot-specific methods for MainPlanner
+end
+

data/app/planners/main.rb

@@ -0,0 +1,5 @@
+# The main planner. A planner of this model is automatically added in the
+# Interface planner list.
+class MainPlanner < Roby::Planning::Planner
+end
+

data/app/scripts/distributed

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/distributed'

data/app/scripts/generate/bookmarks

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/generate/bookmarks'

data/app/scripts/replay

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/replay'

data/app/scripts/results

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/results'

data/app/scripts/run

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/run'

data/app/scripts/server

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/server'

data/app/scripts/shell

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/shell'

data/app/scripts/test

@@ -0,0 +1,3 @@
+#! /usr/bin/env ruby
+APP_DIR = File.expand_path('..', File.dirname(__FILE__))
+require 'roby/app/scripts/test'

data/bin/roby

@@ -0,0 +1,210 @@
+#! /usr/bin/env ruby
+
+APP_DIR  = Dir.pwd
+
+require 'optparse'
+require 'ostruct'
+require 'fileutils'
+require 'yaml'
+require 'find'
+
+require 'roby/app'
+
+MODES = %w{init robot}
+config = OpenStruct.new
+parser = OptionParser.new do |opt|
+    opt.banner = "Usage: roby mode [options]"
+    opt.on_tail('-h', '--help', 'this help message') do
+	if !config.mode
+	    global_help(opt)
+	end
+	STDERR.puts opt
+	exit
+    end
+end
+
+def global_help(opt)
+    opt.separator ""
+    opt.separator "  Available modes:"
+    opt.separator "    init   creates new roby application, or adds the files needed by plugins"
+    opt.separator "    robot  creates a new robot type in this application"
+    opt.separator ""
+    opt.separator "  roby <mode> --help displays mode-specific help"
+    opt.separator ""
+    opt.separator "  Global options"
+end
+
+def mode_init(opt, config)
+    config.enabled_plugins = []
+
+    opt.banner = "Usage: roby [global options] init [options]"
+    opt.separator ""
+    opt.separator "  Creates an application template in the current directory, or"
+    opt.separator "  installs the files needed by a specific plugin in an already"
+    opt.separator "  existing application"
+    opt.separator ""
+    opt.separator "  Available options"
+    known_plugins = Roby.app.available_plugins.map { |name, _, _| name }
+    opt.on('-p', '--plugin NAME,NAME', Array, "install the files needed by the given plugins. Known plugins are: #{known_plugins.join(", ")}") do |new_plugins|
+	new_plugins.each do |name|
+	    if !known_plugins.include?(name)
+		STDERR.puts "unknown plugin #{name}. Known plugins are #{known_plugins}"
+	    end
+	    config.enabled_plugins << name
+	end
+    end
+
+    opt.separator ""
+    opt.separator "  Global options"
+end
+def mode_robot(opt, config)
+    opt.banner = "Usage: roby [global options] robot NAME"
+    opt.separator ""
+    opt.separator "  Creates the template for a new robot type"
+    opt.separator ""
+    opt.separator "  Global options"
+end
+
+remaining = []
+parser.order(ARGV) do |arg|
+    if !config.mode
+	begin
+	    send("mode_#{arg}", parser, config)
+	    config.mode = arg
+	rescue NoMethodError
+	    STDERR.puts "Invalid mode of operation '#{arg}'. Valid modes are: #{MODES.join(", ")}"
+	    STDERR.puts parser
+	    exit(1)
+	end
+    else
+	remaining << arg
+    end
+end
+
+module Roby
+    class Installer
+	# The directory in which we are installing
+	attr_reader :app_dir
+
+	# The configuration hash saved in config/roby.yml
+	attr_reader :config
+
+	def config_path; File.join(app_dir, 'config', 'roby.yml') end
+	def installed_plugins; config['plugins'] || [] end
+
+	def check_plugins(plugins)
+	    if name = plugins.find { |name| !Roby.app.defined_plugin?(name) }
+		known_plugins = Roby.app.available_plugins.map { |name, _| name }
+		raise ArgumentError, "unknown plugin #{name}. Available plugins are #{known_plugins.join(", ")}"
+	    end
+	end
+
+	def initialize(app_dir)
+	    @app_dir = File.expand_path(app_dir)
+
+	    # Read the application configuration from config/roby.yml if the file exists,
+	    @config = if File.file?(config_path)
+			  YAML.load_file(config_path)
+		      else
+			  Hash['plugins', []]
+		      end
+	    check_plugins(config['plugins'])
+	end
+
+	def save_config
+	    File.open(config_path, 'w') do |io|
+		io << YAML.dump(config)
+	    end
+	end
+
+	# Install the template files for a core Roby application and the provided
+	# plugins
+	def install(plugins)
+	    check_plugins(plugins)
+	    install_dir(plugins) do |file|
+		next if file =~ /ROBOT/
+		file
+	    end
+
+	    config['plugins'] |= plugins
+	    save_config
+	end
+
+	# Installs the template files for a new robot named +name+
+	def robot(name)
+	    install_dir(installed_plugins) do |file|
+		next if file !~ /ROBOT/
+		file.gsub /ROBOT/, name
+	    end
+	end
+
+	# Copies the template files into the application directory, without erasing
+	# already existing files. +plugins+ is the list of plugins we should copy
+	# files from
+	def install_dir(plugins = [], &filter)
+	    Installer.copy_tree(File.join(Roby::ROBY_ROOT_DIR, 'app'), app_dir, &filter)
+	    plugins.each do |enabled_name|
+		plugin_desc = Roby.app.available_plugins.find { |name, dir, _, _| enabled_name == name }
+
+		plugin_app_dir = File.join(plugin_desc[1], 'app')
+		next unless File.directory?(plugin_app_dir)
+		Installer.copy_tree(plugin_app_dir, app_dir, &filter)
+	    end
+	end
+
+	# Copy all files that are in +basedir+ into +destdir+. If a block is given,
+	# it is called with each file relative path. The block must then return the
+	# destination name, or nil if the file is to be skipped.
+	def self.copy_tree(basedir, destdir)
+	    basedir = File.expand_path(basedir)
+	    destdir = File.expand_path(destdir)
+
+	    Find.find(basedir) do |file|
+		relative = file.gsub /#{Regexp.quote("#{basedir}")}\/?/, ''
+		relative = yield(relative) if block_given?
+		# The block can return nil if the file shouldn't be installed
+		next unless relative
+
+		destfile = File.join(destdir, relative)
+		if File.directory?(file)
+		    if !File.exists?(destfile)
+			puts "creating #{relative}/"
+			Dir.mkdir destfile
+		    elsif !File.directory?(destfile)
+			STDERR.puts "#{destfile} exists but it is not a directory"
+			exit(1)
+		    end
+		else
+		    if !File.exists?(destfile)
+			FileUtils.cp file, destfile
+			puts "creating #{relative}"
+		    elsif !File.file?(destfile)
+			STDERR.puts "#{destfile} exists but it is not a file"
+			exit(1)
+		    end
+		end
+	    end
+	end
+    end
+end
+
+if !config.mode
+    global_help(parser)
+    STDERR.puts parser
+    exit
+end
+
+installer = Roby::Installer.new(APP_DIR)
+if config.mode == 'init'
+    installer.install(config.enabled_plugins)
+
+elsif config.mode == 'robot'
+    unless robotname = remaining.shift
+	STDERR.puts "No robot name given on command line"
+	STDERR.puts parser
+	exit(1)
+    end
+
+    installer.robot(robotname)
+end
+