rtext 0.2.0

data/CHANGELOG

@@ -0,0 +1,4 @@
+=0.2.0
+
+* First public release
+

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2012 Martin Thiede
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README

@@ -0,0 +1,29 @@
+== RText - Ruby Textual Modelling 
+
+RText can be used to derive textual languages from an RGen metamodel with very little effort. 
+
+RText features include:
+* Generic textual syntax
+* Generic instantiator (parser) and serializer
+* Consistency checks derived from the metamodel
+* Designed to work with lightweight editor plugins providing
+  - Syntax Highlighting
+  - Auto Completion on language commands and model references
+  - Error Annotations 
+  - Model Navigation
+
+
+= Installation
+
+> gem install rtext
+
+
+= Documentation
+
+
+= Examples
+
+	
+= License
+
+RGen is released under the MIT license.

data/RText_Plugin_Implementation_Guide

@@ -0,0 +1,247 @@
+=RText Plugin Implementation Guide
+
+RText editors consist of a frontend and a backend. Most of the logic is implemented
+within the backend while the frontend is kept as thin as possible. This should simplify
+the task of writing a frontend and promote development of RText frontends for different
+editing environments.
+
+This guide explains how to build a RText frontend which will typically be a plugin for an 
+existing editor.
+
+
+==Plugin Responsibilies
+
+Here is a rough overview of what a RText plugin has to do:
+
+* find the .rtext config file for a given RText model file
+* start the corresponding backend process if not started already
+* provide a way to reload the model and issue the refresh command to the backend
+* implement syntax highlighting for the generic RText syntax
+* forward auto completion requests to the backend, let the user choose from the returned options 
+  and insert the option chosen
+* forward element search requests to the backend, let the user choose from the matched elements 
+  and jump to the element chosen
+* forward reference target requests to the backend, let the user choose from the targets found 
+  and jump to the target chosen 
+* query the backend for model problems and display them to the user
+* stop the backend process when it's no longer needed
+
+
+==Plugin Implementation Hints
+
+The plugin implementation could include the following entities.
+
+===Command
+
+A command issued by the plugin and sent to the backend service. See the section about the
+Frontend/Backend Communication Protocol below for details about available commands.
+
+
+===Connector
+
+A connector represents the connection to a running backend process.
+Each connector/process is associated with a specific .rtext file and a specific service specification
+within this file. See the RText Users Guide for details about the .rtext config file.
+
+When the backend service is started, it outputs the UDP port it is listening on. The connector
+should use this port to send commands to the backend.
+
+The connector should send commands to the backend. It should assign invocation ids and keep the
+invocation until a response is received or a timeout has occured.
+
+When the backend responds, the connector should associate the response packages with the previous
+request using the invocation id. It should also join fragmented response packages and send an
+internal notification when a response has been fully received.
+
+
+===Connector Manager
+
+The connector manager should be able to return a connector for a given RText model file. 
+
+It needs to identify the backend process to which to connect by means of the .rtext config file and
+a contained service specification (one .rtext files may contain service specifications for starting
+different backend services depending on the model file name).
+
+For this, it needs to find the first .rtext config file containing a service specification matching 
+the given RText model file name. See the RText Users guide for details about the .rtext config file
+and the way how they are found in the filesystem.
+
+Once the .rtext file/service specification is found it can be used as a key for a lookup for 
+already established connections. The plugin should avoid starting the same process twice. All 
+RText model files with the same file extension and belonging to the same .rtext file should 
+use the same process.
+
+The process should be kept running as long as editing is in progress. This will greatly 
+reduce response times as the model can be kept in memory and only the changed parts need
+to be reloaded.
+
+When the plugin lifecycle ends, the connector manager should send the "stop" command to the 
+backend processes.
+
+
+==Frontend/Backend Communication Protocol
+
+Communication takes place via plain text transmitted via UDP. There are requests by
+the frontend and responses by the backend. In both cases the text is interpreted as
+a set of lines (separated by \n or \r\n).
+
+
+===Request
+
+The request consists of a command id, an invocation id and the command parameters.
+The invocation id is repeated within the response to allow proper association with the
+request. Command parameters are command specific (see below).
+
+Requests can not span more than one UDP package and thus are limited to the UDP payload size.
+
+  line 1:    <command id>
+  line 2:    <invocation id>
+  line 3..n: <command parameters>
+
+
+===Response
+
+Each response contains the invocation id of the request in the first line.
+Since it could be splitted over several UDP packages it contains an indication 
+in the second line if more packages are following ("more") or this is the last package 
+("last"). The rest of the response is a fragment of the command result data. All fragments
+of a set of response packages with the same invocation id need to be joined.
+
+Note that there are no mechanisms to protect agains package loss or reordering since
+this protocol is intended to be used on local socket connections only.
+
+  line 1:    <invocation id>
+  line 2:    "more" | "last"
+  line 3..n: <command result data>
+
+
+===Command: Refresh
+
+The refresh command tells the backend to reload the model from the file system. Frontends
+could issue this command after a model file has been changed in the file system or on
+an explicit user request.
+
+  Request Format:
+    command id:          "refresh"
+    no parameters 
+
+  Response Format:
+    no result data
+
+
+===Command: Complete
+
+This command is a request by the frontend to show auto completion options at a given
+location within a file. The location is expressed using a set of context lines and the cursor
+column position in the current line.
+
+Context lines are lines from an RText file which contain a (context) command and all 
+the parent commands wrapped around it. Any sibling commands can be omitted as well as
+any lines containing closing braces and brackets. The order of lines is the same as in the RText file.
+
+Here is an example. Consider the following RText file with the cursor in the line of "Command3" at 
+the time when the auto completion command is issued.
+
+  Command1 {
+    Command2 {
+      role1: [
+        Command3          <== cursor in this line
+        Command4
+      ]
+    }
+    Command5
+  }
+
+The context lines in this case would be the following.
+
+  Command1 {
+    Command2 {
+      role1: [
+        Command3
+
+The current line is always the last of the context lines.
+
+Note that all siblings of the command and parent commands have been stripped off, as well as
+any closing braces or brackets.
+
+The purpose of this special context line format is to keep the task of extracting the
+context in the frontend simple and the amount of data transmitted to the backend low.
+It's also a way to keep the parsing time of the context low in the backend and thus to minimize
+the user noticable delay.
+
+  Request Format:
+    command id:          "complete"
+    param line 1:        cursor column position in the current line 
+    param line 2..n:     context lines
+
+  Response Format:
+    result line 1..n:    <completion string>;<extra info string>
+
+
+===Command: Show Problems
+
+This command is a request by the frontend to determine and return all problems present in the current model.
+The command implicitly reloads the model from the filesystem before checking for problems.
+
+  Request Format:
+    command id:          "show_problems"
+    no parameters 
+
+  Response Format:
+    result line n:       <file name>
+    result line n+1..n+m <line number>;<message>
+
+Note that the file name is repeated only once for all problems within a file to reduce the amout of
+result data which needs to be transmitted.
+
+
+===Command: Reference Targets
+
+This command is used to retrieve the targets of a given reference. The reference location is expressed
+by means of a set of context lines and the cursor column position in the current line. The cursor 
+column position must be within the string representing the reference. The format of the context lines
+is the same as the one described with the "Complete" command (see above).
+
+Note that the service provider is free to define how a reference is represented. In particular it
+can interpret the command identifier as a reference and use this mechanism to show incoming references
+(reverse references).
+
+As references can be ambiguous, the result is a list of possible targets. Each target consists of the
+filename and line number of the target element as well as the text to be displayed to the user in case
+there are more than one targets to choose from.
+
+  Request Format:
+    command id:          "get_reference_targets" 
+    param line 1:        cursor column position in the current line 
+    param line 2..n:     context lines
+
+  Response Format:
+    result line 1..n:    <file name>;<line number>;<display name>
+
+
+===Command: Find Elements
+
+This command returns a set of model elements matching a search pattern. The format of the
+pattern depends on the service provider. In a simple case, for example, the pattern could be the 
+beginning of the element names.
+
+  Request Format:
+    command id:          "get_elements"
+    param line 1:        <seach pattern>
+
+  Response Format:
+    result line 1..n:    <display name>;<file name>;<line number>
+
+
+===Command: Stop
+
+This command is normally invoked when the frontend terminates or otherwise needs to terminate
+the backend service. When receiving this command, the backend will terminate.
+
+  Request Format:
+    command id:          "stop"
+    no parameters
+
+  Response Format:
+    no result data
+

data/RText_Users_Guide

@@ -0,0 +1,31 @@
+=RText Users Guide
+
+== The .rtext config file
+
+A .rtext config file is used to specify which backend service is to be started for a given
+RText model file.
+
+In order to be found by an RText plugin, the .rtext file must be located in the same directory
+as the RText model file being edited or in any of its parent directories. 
+
+Here is the grammar for the config file syntax:
+
+  rtext_config      ::= <service_spec>+
+  service_spec      ::= <file_pattern_list>:\n<command line string>\n
+  file_pattern_list ::= <file_pattern> | <file_pattern>, <file_pattern_list>
+  file_pattern      ::= <filename without path> | *.<extension>
+
+Here is an example:
+
+  *.ext1:
+  ruby service1.rb 
+  *.ext2, *.ext3:
+  ruby service2.rb *.ext2 *.ext3
+  noext:
+  service3.sh
+
+This example contains specifications for three different backend services. service1.rb is started
+when a model file with the extension "ext1" is being edited. service2.rb is started when a 
+model file with extension "ext2" or "ext3" is edited. service2.rb also receives additional command
+line parameters. service3.sh is started whenever a file named "noext" is edited.
+

data/Rakefile

@@ -0,0 +1,46 @@
+require 'rake/gempackagetask'
+require 'rake/rdoctask'
+
+DocFiles = [
+  "README", "CHANGELOG", "MIT-LICENSE", 
+  "RText_Users_Guide", 
+  "RText_Plugin_Implementation_Guide"]
+
+RTextGemSpec = Gem::Specification.new do |s|
+  s.name = %q{rtext}
+  s.version = "0.2.0"
+  s.date = Time.now.strftime("%Y-%m-%d")
+  s.summary = %q{Ruby Textual Modelling}
+  s.email = %q{martin dot thiede at gmx de}
+  s.homepage = %q{http://ruby-gen.org}
+  s.description = %q{RText can be used to derive textual languages from an RGen metamodel with very little effort.}
+  s.authors = ["Martin Thiede"]
+  s.add_dependency('rgen', '>= 0.6.0')
+  gemfiles = Rake::FileList.new
+  gemfiles.include("{lib,test}/**/*")
+  gemfiles.include(DocFiles)
+  gemfiles.include("Rakefile") 
+  gemfiles.exclude(/\b\.bak\b/)
+  s.files = gemfiles
+  s.rdoc_options = ["--main", "README", "-x", "test"]
+  s.extra_rdoc_files = DocFiles
+end
+
+Rake::RDocTask.new do |rd|
+  rd.main = "README"
+  rd.rdoc_files.include(DocFiles)
+  rd.rdoc_files.include("lib/**/*.rb")
+  rd.rdoc_dir = "doc"
+end
+
+RTextPackageTask = Rake::GemPackageTask.new(RTextGemSpec) do |p|
+  p.need_zip = false
+end	
+
+task :prepare_package_rdoc => :rdoc do
+  RTextPackageTask.package_files.include("doc/**/*")
+end
+
+task :release => [:prepare_package_rdoc, :package]
+
+task :clobber => [:clobber_rdoc, :clobber_package]

data/lib/rtext/completer.rb

@@ -0,0 +1,152 @@
+require 'rgen/ecore/ecore_ext'
+
+module RText
+
+class Completer
+
+  CompletionOption = Struct.new(:text, :extra)
+
+  # Creates a completer for RText::Language +language+.
+  #
+  def initialize(language)
+    @lang = language
+  end
+
+  # Provides completion options
+  #
+  #  :linestart
+  #    the content of the current line before the cursor
+  #
+  #  :prev_line_provider
+  #    is a proc which must return lines above the current line
+  #    it receives an index parameter in the range 1..n
+  #    1 is the line just above the current one, 2 is the second line above, etc.
+  #    the proc must return the line as a string or nil if there is no more line
+  #
+  #  :ref_completion_option_provider
+  #    a proc which receives a EReference and should return
+  #    the possible completion options as CompletionOption objects 
+  #    note, that the context element may be nil if this information is unavailable
+  #
+  def complete(line, linepos, prev_line_provider, ref_completion_option_provider=nil)
+    linestart = line[0..linepos-1]
+    # command
+    if linestart =~ /^\s*(\w*)$/ 
+      prefix = $1
+      classes = completion_classes(prev_line_provider)
+      classes = classes.select{|c| c.name.index(prefix) == 0} if prefix
+      classes.sort{|a,b| a.name <=> b.name}.collect do |c| 
+        uargs = @lang.unlabled_arguments(c).collect{|a| "<#{a.name}>"}.join(", ")
+        CompletionOption.new(c.name, uargs)
+      end
+    # attribute
+    elsif linestart =~ /^\s*(\w+)\s+(?:[^,]+,)*\s*(\w*)$/
+      command, prefix = $1, $2
+      clazz = @lang.class_by_command(command)
+      if clazz
+        features = @lang.labled_arguments(clazz.ecore)
+        features = features.select{|f| f.name.index(prefix) == 0} if prefix
+        features.sort{|a,b| a.name <=> b.name}.collect do |f| 
+          CompletionOption.new("#{f.name}:", "<#{f.eType.name}>")
+        end
+      else
+        []
+      end
+    # value
+    elsif linestart =~ /\s*(\w+)\s+(?:[^,]+,)*\s*(\w+):\s*(\S*)$/
+      command, fn, prefix = $1, $2, $3
+      clazz = @lang.class_by_command(command)
+      feature = clazz && @lang.non_containments(clazz.ecore).find{|f| f.name == fn}
+      if feature
+        if feature.is_a?(RGen::ECore::EReference)
+          if ref_completion_option_provider
+            ref_completion_option_provider.call(feature)
+          else
+            []
+          end
+        elsif feature.eType.is_a?(RGen::ECore::EEnum)
+          feature.eType.eLiterals.collect do |l|
+            CompletionOption.new("#{l.name}")
+          end 
+        elsif feature.eType.instanceClass == String
+          [ CompletionOption.new("\"\"") ]
+        elsif feature.eType.instanceClass == Integer 
+          (0..4).collect{|i| CompletionOption.new("#{i}") }
+        elsif feature.eType.instanceClass == Float 
+          (0..4).collect{|i| CompletionOption.new("#{i}.0") }
+        elsif feature.eType.instanceClass == RGen::MetamodelBuilder::DataTypes::Boolean
+          [true, false].collect{|b| CompletionOption.new("#{b}") }
+        else
+          []
+        end
+      else
+        []
+      end
+    else
+      []
+    end
+  end
+
+  private
+
+  def completion_classes(prev_line_provider)
+    clazz, feature = context(prev_line_provider)
+    if clazz
+      if feature
+        @lang.concrete_types(feature.eType)
+      else
+        refs_by_class = {}
+        clazz.eAllReferences.select{|r| r.containment}.each do |r|
+          @lang.concrete_types(r.eType).each { |c| (refs_by_class[c] ||= []) << r }
+        end
+        refs_by_class.keys.select{|c| refs_by_class[c].size == 1}
+      end
+    else
+      @lang.root_epackage.eAllClasses.select{|c| !c.abstract &&
+        !c.eAllReferences.any?{|r| r.eOpposite && r.eOpposite.containment}}
+    end
+  end
+
+  def context(prev_line_provider)
+    command, role = parse_context(prev_line_provider)
+    clazz = command && @lang.root_epackage.eAllClasses.find{|c| c.name == command}
+    feature = role && clazz && clazz.eAllReferences.find{|r| r.containment && r.name == role}
+    [clazz, feature]
+  end
+
+  def parse_context(prev_line_provider)
+    block_nesting = 0
+    array_nesting = 0
+    non_empty_lines = 0
+    role = nil
+    i = 0
+    while line = prev_line_provider.call(i+=1)
+      # empty or comment
+      next if line =~ /^\s*$/ || line =~ /^\s*#/
+      # role
+      if line =~ /^\s*(\w+):\s*$/
+        role = $1 if non_empty_lines == 0
+      # block open
+      elsif line =~ /^\s*(\S+).*\{\s*$/
+        block_nesting -= 1
+        return [$1, role] if block_nesting < 0
+      # block close
+      elsif line =~ /^\s*\}\s*$/
+        block_nesting += 1
+      # array open
+      elsif line =~ /^\s*(\w+):\s*\[\s*$/
+        array_nesting -= 1
+        role = $1 if array_nesting < 0
+      # array close
+      elsif line =~ /^\s*\]\s*$/
+        array_nesting += 1
+      end
+      non_empty_lines += 1
+    end
+    [nil, nil]
+  end
+
+end
+
+end
+