rsmp 0.8.6 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 437183789b08f12b23264bc2716b84fd6c2298c2abab58dfb0b64529d004ac7d
-  data.tar.gz: 5595afbb4edb5d3e66320cae10c92c98a13fd56848095bb8ec4bb3615a097b05
+  metadata.gz: dde978c380def64dfcad00d0a7c9845a7c57476828623664dcf2ace7f3f6b669
+  data.tar.gz: 6dadc564b21089271b9921f876b95c899ad42c5bbc27b9250d34b47648447937
 SHA512:
-  metadata.gz: f3cdc4723d2d4616cc703d18aca6432c354bdbd7e4084c443a5f17aa8c420463d567813e73dc3fb454f02461b773149970932ee1d6d84f0434d8eaf21324a0ff
-  data.tar.gz: b90f7f2c8edc5722e0044cb31107779caabc4a9acbfce8bb713dd9a7d84e7272dd96fb2aad293c7d30e2350701758fe5300cfed776a774a754798cf8ac91037c
+  metadata.gz: 7bc25b5125c599a5c46db4c919e8bf6891ddad0631de6264dfe5df97eac5cdf85768f099df8f41bff26203fb6fb15d95f4b09c8e2d4b32e80b24b3065be88d1e
+  data.tar.gz: 8b58da0f830cc4e4fa7f0906a2d691a3d1548b364c21e320c0f597ce4da873dd16b2db0d5e196c70f21fa9126c24a4e3f4c951586768cad0f99b78047ed7eb18

data/.github/workflows/rspec.yaml

@@ -0,0 +1,14 @@
+# This workflow runs RSpec tests
+
+name: RSpec
+on: [push, pull_request]
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - uses: ruby/setup-ruby@v1
+      with:
+        # ruby-version is not needed because we have a .ruby-version file
+        bundler-cache: true # runs 'bundle install' and caches installed gems automatically
+    - run: bundle exec rspec

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    rsmp (0.8.6)
+    rsmp (0.9.0)
       async (~> 1.29.1)
       async-io (~> 1.32.2)
       colorize (~> 0.8.1)
@@ -103,6 +103,7 @@ GEM
 PLATFORMS
   x86_64-darwin-20
   x86_64-darwin-21
+  x86_64-linux
 
 DEPENDENCIES
   aruba (~> 2.0.0)

data/README.md

@@ -23,7 +23,7 @@ $ git submodule update   # fetch submodules
 
 Alternatively, you can pass --recurse-submodules to the git clone command, and it will automatically initialize and update each submodule in the repository.
 
-## Usage 
+## Usage
 ### Site and Supervisor
 The RSMP::Site and RSMP::Supervisor classes can be used to run a RSMP site.
 
@@ -172,7 +172,7 @@ Use the ```tlc``` site type to run an emulation of a traffic light controller. T
 ### CLI help and options.
 Use ```--help <command>``` to get a list of available options.
 
-Use ```--config <path>``` to point to a .yaml config file, controlling things like IP adresses, ports, and log output. Examples of config files can be found the folder ```config/```. 
+Use ```--config <path>``` to point to a .yaml config file, controlling things like IP adresses, ports, and log output. Examples of config files can be found the folder ```config/```.
 
 ## Tests
 ### RSpec

data/bin/console

@@ -1,7 +1,7 @@
 #!/usr/bin/env ruby
 
 # Make IRB run inside Async, so async task
-# will run the the background. 
+# will run the the background.
 
 require 'bundler/setup'
 require 'irb'

data/documentation/classes_and_modules.md

@@ -14,13 +14,13 @@ SupervisorProxy    SiteProxy - - include Components, SiteProxyWait
 
 ## Modules
 ### Logging
-Handle logging.
+Handles logging.
 
-### Wait
-Handles waiting for an async condition and block.
+### Task
+Handles async tasks.
 
 ### Components
-Component handling.
+Handles RSMP components.
 
 ## Classes
 ### Node

data/documentation/collecting_message.md

@@ -1,7 +1,7 @@
 # Collection
 You often need to collect messages or responses. The collector classes are used to collect message asyncronously. Other tasks continue until the collection completes, time outs or is cancelled.
 
-A collector can collect ingoing and/or outgoing messages. 
+A collector can collect ingoing and/or outgoing messages.
 
 An object that includes the Notifier module (or implements the same functionality) must be provided when you construct a Collected. The collector will attach itself to this notifier when it starts collecting, to receive messages. The SiteProxy and SupervisorProxy classes both include the Notifier module, and can therefore be used as message sources.
 
@@ -32,7 +32,7 @@ outgoing: Whether to collect outgoing messages. Defaults to true
 component: An RSMP component id.
 
 ### Collecting
-Use collect() to start collecting and wait for completion or timeout. The status will be returned. 
+Use collect() to start collecting and wait for completion or timeout. The status will be returned.
 
 ```ruby
 result = collector.collect # => :ok, :timeout or :cancelled

data/documentation/tasks.md

@@ -0,0 +1,149 @@
+# Tasks
+
+## Concurrency
+When you use a site or a supervisor, it runs asyncronously so you can run several concurrently, or do other things concurrently, like sending messages and waiting for reponses.
+
+Concurrency is provided by Ruby Fiber Scheduler and the Async gem. and used to:
+
+- Runn multiple sites or supervisors
+- Run multiple connections from each site or supervisor
+- Wait for messages, connections or states
+
+Classes don't inherit from Async::Task. Instead they include task as instance variables. This means that the hierachy of objects and tasks can be different.
+
+## Task module
+The `Task` module is used by the `Node` and `Proxy` classes and defines a life cycle for handling async tasks.
+
+A single main tasks is kept in `@task`. If subclasses need subtask, they can start them as needed, inside the main task.
+
+You first call `start`. If `@task` already exists, it will return immedatiately.
+Otherwise an async task is created and stored in `@task`, and `run` is called inside this task, to handle any long-running jobs, like listening for incoming messages. By default `run` calls `start_subtasks`, but should be overriden to do actual work as well.
+The call to `start` returns immediately, with the async task running concurently.
+
+If you want to stop the task, call `stop`. It will fist call `stop_subtasks` and then `stop_task` which calls Async#stop on `@task` and sets `@task` to `nil`.
+
+## Stopping tasks
+If a task stops itself, code after the call to `stop` will not be run, unless you use an ensure block:
+
+```ruby
+require 'async'
+
+Async do |task|
+	task.stop
+	puts "I just stopped"   # this will not be reaced, because the task was stopped
+end
+
+Async do |task|
+	task.stop
+ensure
+	puts "I just stopped"   # this will be reached
+end
+```
+This can happen if either the task stops itself directly, or if it stops task higher up in the task hierarchy.
+
+For example, when a timer task finds that an acknowledgement was not received in time, and then closes the connection by calling stop() on the Proxy, the proxy which will then in turn stop the timer task. So any code that should be run after that must be inside an ensure block.
+
+## Proxies
+A site connects to one of more supervisors. A proxy is created and run to handle each connection.
+
+A site can connect to one or more supervisor. It creates a proxy for each and runs them.
+
+When the proxy is run, it creates an async task to read incoming messages. This reader task is stored in `@reader`and will be a sub task of the site/supervisor main task.
+
+A proxy also creates a timer task, stored in `@timer`. This task is used to check watchdog and acknowledgement timeouts.
+
+Proxies build on the Task module functionality by handling RSMP communication via a TCP socket.
+
+The TCP socket can be open or closed. The RSMP communication first goes through a handshake sequence before being ready. The `status` attribute keeps track of this, and can be either `:disconnected`, `:connected` or `:ready`
+
+Proxies provide `connect` and `close` for starting and stopping commununication.
+
+A supervisor proxies connects actively to a site proxy, whereas a site proxy waits for the supervisor proxy to connect.
+
+A supervisor proxy is created at startup, and is responsible for creating the TCP socket and connecting to the supervisor.
+
+A site proxy is also created at startup, but the socket is created in the supervisor by `Aync::Endpoint#accept`when a site connects.
+
+## Object Hierarchy
+A Site has one or more SupervisorProxies, which represent connections to the remote supervisor. 
+
+A Supervisor has one or more SiteProxies, which represent connections to the remote site.
+
+SiteProxy and SupervisorProxy both inherit from Proxy and have a reader task and a timer task. These are just Async tasks, not separate classes.
+
+## Task Hierachy
+When you start an Async task, the parent will be the currently running task, unless you specify a different parent task.
+
+The Async task hierarchy is similar to the object hierachy, with the difference that proxies have reader and timer tasks. And Async::IO which is used to handle TPC connections concurently, will create some intermediate tasks:
+
+```
+Supervisor @task
+	accepting connections      # this task is created by Async::IO
+		incoming connection 1    # this task is created by Async::IO
+			SiteProxy @task
+				SiteProxy @reader
+				SiteProxy @timer
+		incoming connection 2    # this task is created by Async::IO
+			SiteProxy @task
+				SiteProxy @reader
+				SiteProxy @timer
+
+```
+
+
+A Site run from the CLI, before it connects to a supervisor:
+
+```
+#<Async::Reactor:0xb90 1 children (running)>
+	#<Async::Task:0xba4 cli (running)>
+		#<Async::Task:0xbcc RSMP::TLC::TrafficControllerSite main task (running)>
+			#<Async::Task:0xbe0 RSMP::SupervisorProxy main task (running)>
+			#<Async::Task:0xbf4 tlc timer (running)>
+```
+
+After the site connects to a supervisor:
+
+```
+#<Async::Reactor:0xadc 1 children (running)>
+	#<Async::Task:0xaf0 cli (running)>
+		#<Async::Task:0xb18 RSMP::TLC::TrafficControllerSite main task (running)>
+			#<Async::Task:0xb2c RSMP::SupervisorProxy main task (running)>
+				#<Async::Task:0xb40 reader (running)>
+				#<Async::Task:0xb54 timer (running)>
+			#<Async::Task:0xb68 tlc timer (running)>
+```
+
+
+
+A supervisor run from the CLI, before any sites have connected:
+
+```
+#<Async::Reactor:0x94c 1 children (running)>
+	#<Async::Task:0x960 cli (running)>
+		#<Async::Task:0x988 RSMP::Supervisor main task (running)>
+			#<Async::Task:0x99c accepting connections #<Addrinfo: 0.0.0.0:14111 TCP> [fd=12] (running)>
+```
+
+The supervisor after a site has connected
+
+```
+#<Async::Reactor:0x94c 1 children (running)>
+	#<Async::Task:0x960 cli (running)>
+		#<Async::Task:0x988 RSMP::Supervisor main task (running)>
+			#<Async::Task:0x99c accepting connections #<Addrinfo: 0.0.0.0:13111 TCP> [fd=12] (running)>
+				#<Async::Task:0xac8 incoming connection #<Addrinfo: 127.0.0.1:51778 TCP> [fd=13] (running)>
+					#<Async::Task:0xadc RSMP::SiteProxy main task (running)>
+						#<Async::Task:0xaf0 reader (running)>
+						#<Async::Task:0xb04 timer (running)>
+```
+
+The task hierachy matters when you stop a task or iterate on subtasks. Note that calling `Task#wait`
+ does not wait for subtasks, whereas Task#stop stops all subtasks.
+
+## Transient tasks
+If you mark an Async task with `transient: true` when you created it, that task will be stopped aas soon as all normal task are completed. They will also be moved up the hierarchy if the parent is complete, but not the grandparent.
+
+Transient tasks can be used to cleanup, but using an `ensure` block in the transient task. When you call `stop` on a task, an Async::Stop exception is raised, which will run the code in the `ensure` block.
+
+Some of the RSpec tests runs tests in in transient task. As soon as the main test code is complete, any subtasks like Sites or Supervisors that might otherwise prevent the test from completing, will be stopped automatically.
+

data/lib/rsmp/archive.rb

@@ -20,11 +20,11 @@ module RSMP
 
     def self.prepare_item item
       raise ArgumentError unless item.is_a? Hash
-    
+
       cleaned = item.select { |k,v| [:author,:level,:ip,:port,:site_id,:component,:text,:message,:exception].include? k }
       cleaned[:timestamp] = Clock.now
       if item[:message]
-        cleaned[:direction] = item[:message].direction 
+        cleaned[:direction] = item[:message].direction
         cleaned[:component] = item[:message].attributes['cId']
       end
 
@@ -54,7 +54,7 @@ module RSMP
         @items.shift
       end
     end
-    
+
     private
 
     def find options, &block

data/lib/rsmp/cli.rb

@@ -11,7 +11,7 @@ module RSMP
     desc "site", "Run RSMP site"
     method_option :config, :type => :string, :aliases => "-c", banner: 'Path to .yaml config file'
     method_option :id, :type => :string, :aliases => "-i", banner: 'RSMP site id'
-    method_option :supervisors, :type => :string, :aliases => "-s", banner: 'ip:port,... list of supervisor to connect to'      
+    method_option :supervisors, :type => :string, :aliases => "-s", banner: 'ip:port,... list of supervisor to connect to'
     method_option :log, :type => :string, :aliases => "-l", banner: 'Path to log file'
     method_option :json, :type => :boolean, :aliases => "-j", banner: 'Show JSON messages in log'
     method_option :type, :type => :string, :aliases => "-t", banner: 'Type of site: [tlc]'
@@ -60,19 +60,35 @@ module RSMP
             site_class = RSMP::Site
         end
       end
-      site_class.new(site_settings:settings, log_settings: log_settings).start
+      Async do |task|
+        task.annotate 'cli'
+        loop do
+          begin
+            site = site_class.new(site_settings:settings, log_settings: log_settings)
+            site.start
+            site.wait
+          rescue RSMP::Restart
+            site.stop
+          end
+        end
+      end
+    rescue Interrupt
+      # cntr-c
     rescue RSMP::Schemer::UnknownSchemaTypeError => e
       puts "Cannot start site: #{e}"
     rescue RSMP::Schemer::UnknownSchemaVersionError => e
       puts "Cannot start site: #{e}"
     rescue Psych::SyntaxError => e
       puts "Cannot read config file #{e}"
+    rescue Exception => e
+      puts "Uncaught error: #{e}"
+      puts caller.join("\n")
     end
 
     desc "supervisor", "Run RSMP supervisor"
     method_option :config, :type => :string, :aliases => "-c", banner: 'Path to .yaml config file'
     method_option :id, :type => :string, :aliases => "-i", banner: 'RSMP site id'
-    method_option :ip, :type => :numeric, banner: 'IP address to listen on'      
+    method_option :ip, :type => :numeric, banner: 'IP address to listen on'
     method_option :port, :type => :string, :aliases => "-p", banner: 'Port to listen on'
     method_option :log, :type => :string, :aliases => "-l", banner: 'Path to log file'
     method_option :json, :type => :boolean, :aliases => "-j", banner: 'Show JSON messages in log'
@@ -110,7 +126,14 @@ module RSMP
         log_settings['json'] = options[:json]
       end
 
-      RSMP::Supervisor.new(supervisor_settings:settings,log_settings:log_settings).start
+      Async do |task|
+        task.annotate 'cli'
+        supervisor = RSMP::Supervisor.new(supervisor_settings:settings,log_settings:log_settings)
+        supervisor.start
+        supervisor.wait
+      end
+    rescue Interrupt
+      # ctrl-c
     rescue RSMP::ConfigurationError => e
       puts "Cannot start supervisor: #{e}"
     end

data/lib/rsmp/collect/state_collector.rb

@@ -25,7 +25,7 @@ module RSMP
   # matches this input:
   #
   # {"cCI"=>"M0104", "n"=>"month", "v"=>"9", "age"=>"recent"}
-  # 
+  #
   # And the result is stored as:
   # {
   #   {"cCI"=>"M0104", "cO"=>"setDate", "n"=>"month", "v"=>/\d+/} =>

data/lib/rsmp/components.rb

@@ -3,7 +3,7 @@
 module RSMP
   module Components
     attr_reader :components
-    
+
     def initialize_components
       @components = {}
     end
@@ -25,10 +25,10 @@ module RSMP
 
     def check_main_component settings
       unless settings['main'] && settings['main'].size >= 1
-        raise ConfigurationError.new("main component must be defined") 
+        raise ConfigurationError.new("main component must be defined")
       end
       if settings['main'].size > 1
-        raise ConfigurationError.new("only one main component can be defined, found #{settings['main'].keys.join(', ')}") 
+        raise ConfigurationError.new("only one main component can be defined, found #{settings['main'].keys.join(', ')}")
       end
     end
 

data/lib/rsmp/convert/export/json_schema.rb

@@ -23,7 +23,7 @@ module RSMP
 
         def self.build_value item
           out = {}
-          
+
           if item['description']
             out["description"] = item['description']
           end
@@ -95,7 +95,7 @@ module RSMP
             }
           end
           json = {
-            "properties" => { 
+            "properties" => {
               "aCId" => { "enum" => items.keys.sort },
               "rvs" => { "items" => { "allOf" => list } }
             }
@@ -175,7 +175,7 @@ module RSMP
               }
             ]
           }
-          out["sxl.json"] = output_json json     
+          out["sxl.json"] = output_json json
         end
 
         def self.generate sxl
@@ -192,7 +192,7 @@ module RSMP
           out.each_pair do |relative_path,str|
             path = File.join(folder, relative_path)
             FileUtils.mkdir_p File.dirname(path)      # create folders if needed
-            file = File.open(path, 'w+')      # w+ means truncate or create new file 
+            file = File.open(path, 'w+')      # w+ means truncate or create new file
             file.puts str
           end
         end

data/lib/rsmp/convert/import/yaml.rb

@@ -24,7 +24,7 @@ module RSMP
             commands: {}
           }
 
-          yaml['objects'].each_pair do |type,object|  
+          yaml['objects'].each_pair do |type,object|
             object["alarms"].each { |id,item| sxl[:alarms][id] = item }
             object["statuses"].each { |id,item| sxl[:statuses][id] = item }
             object["commands"].each { |id,item| sxl[:commands][id] = item }

data/lib/rsmp/error.rb

@@ -64,7 +64,4 @@ module RSMP
 
 	class RepeatedStatusError < Error
 	end
-
-	class TimestampError < Error
-	end
 end

data/lib/rsmp/inspect.rb

@@ -1,5 +1,5 @@
 # Costume inspect, to reduce noise
-# 
+#
 # Instance variables of classes starting with Async or RSMP are shown
 # with only their class name and object id, which reduces output,
 # especially for deep object structures.

data/lib/rsmp/logger.rb

@@ -2,7 +2,7 @@ module RSMP
   class Logger
 
     attr_accessor :settings
-    
+
     def initialize settings={}
       defaults = {
         'active'=>true,
@@ -115,7 +115,7 @@ module RSMP
             end
           end
         end
-        return false if ack && @settings["acknowledgements"] == false && 
+        return false if ack && @settings["acknowledgements"] == false &&
           [:not_acknowledged,:warning,:error].include?(item[:level]) == false
       end
       true
@@ -159,7 +159,7 @@ module RSMP
 
     def log item, force:false
       if output?(item, force)
-        output item[:level], build_output(item) 
+        output item[:level], build_output(item)
       end
     end
 
@@ -169,7 +169,7 @@ module RSMP
       else
         ' '*length
       end
-    end 
+    end
 
     def dump archive, force:false, num:nil
       num ||= archive.items.size
@@ -183,7 +183,7 @@ module RSMP
     def build_part parts, item, key, &block
       skey = key.to_s
       return unless @settings[skey]
-      
+
       part = item[key]
       part = yield part if block
       part = part.to_s

data/lib/rsmp/logging.rb

@@ -8,7 +8,7 @@ module RSMP
 
     def initialize_logging options
       @archive = options[:archive] || RSMP::Archive.new
-      @logger = options[:logger] || RSMP::Logger.new(options[:log_settings]) 
+      @logger = options[:logger] || RSMP::Logger.new(options[:log_settings])
     end
 
     def author

data/lib/rsmp/message.rb

@@ -188,7 +188,7 @@ module RSMP
   class Unknown < Message
   end
 
-  class AggregatedStatus < Message 
+  class AggregatedStatus < Message
     def initialize attributes = {}
       super({
         "type" => "AggregatedStatus",

data/lib/rsmp/node.rb

@@ -3,14 +3,14 @@
 module RSMP
   class Node
     include Logging
-    include Wait
     include Inspect
+    include Task
 
     attr_reader :archive, :logger, :task, :deferred, :error_queue, :clock, :collector
 
     def initialize options
       initialize_logging options
-      @task = options[:task]
+      initialize_task
       @deferred = []
       @clock = Clock.new
       @error_queue = Async::Queue.new
@@ -18,6 +18,13 @@ module RSMP
       @collect = options[:collect]
     end
 
+    # stop proxies, then call super
+    def stop_subtasks
+      @proxies.each { |proxy| proxy.stop }
+      @proxies.clear
+      super
+    end
+
     def ignore_errors classes, &block
       was, @ignore_errors = @ignore_errors, [classes].flatten
       yield
@@ -50,55 +57,13 @@ module RSMP
 
     def clear_deferred
       @deferred.clear
-    end    
-
-    def do_start task
-      task.annotate self.class.to_s
-      @task = task
-      start_action
-      idle
-    end
-
-    def start
-      starting
-      if @task
-        do_start @task
-      else
-        Async do |task|
-          do_start task
-        end
-      end
-    rescue Errno::EADDRINUSE => e
-      log "Cannot start: #{e.to_s}", level: :error
-    rescue SystemExit, SignalException, Interrupt
-      @logger.unmute_all
-      exiting
-    end
-
-    def idle
-      loop do
-        @task.sleep 60
-      end
-    end
-
-    def stop
-      @task.stop if @task
-    end
-
-    def restart
-      stop
-      start
-    end
-
-    def exiting
-      log "Exiting", level: :info
     end
 
     def check_required_settings settings, required
       raise ArgumentError.new "Settings is empty" unless settings
       required.each do |setting|
         raise ArgumentError.new "Missing setting: #{setting}" unless settings.include? setting.to_s
-      end 
+      end
     end
 
     def author