sexpr 0.4.0 → 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 67f9fda1b0281f22e49fc267f432c1bed5ba6f270911b0470786662533d257d0
+  data.tar.gz: 97b7fc7fce491b594f4fb52a06daa5837407a5d5db59ab6bb6ae56e517e38492
+SHA512:
+  metadata.gz: 808fbbb1bd28d417addfd740a85c441e022208ed87111a713bd4cd1920254b606181c3c8a2bbabd2c4fcaa707543b11d3baa1db6e661378fa7f7b350256599f7
+  data.tar.gz: 2c3795497e39ededc4cec8897fd178a34911fa3647a40561d8218da19bc9b72671de4b6d86bbd5854beb1d9ebc2907d86536d33531fe7c8b38074338c8824263

data/CHANGELOG.md

@@ -1,14 +1,88 @@
+# 1.0.0 / 2021-12-11
+
+* Upgrade project structure to modern Enspirit practices
+
+* Citrus dependency upgraded to 3.x and code fixed accordingly
+
+# 0.6.0 / 2012-09-13
+
+* Major enhancements (possibly breaking changes)
+
+  * All grammar rules are now proper Matcher::Rule instances. A NonTerminal
+    matcher is added and is used for non-terminal rule productions and matching
+    (all but Terminal and Alternative).
+  * `Rewriter.sexpr_grammar` and `Rewriter#sexpr_grammar` have been removed.
+    `Processor.grammar` and `Processor#grammar` are provided instead.
+
+* Minor enhancements
+
+  * Added `Grammar#tagging_module_for(rulename)` that returns the user-defined
+    module used to tag a given grammar rule production.
+  * `Grammar#looks_a_sexpr?(arg)` became public.
+
+# 0.5.1 / 2012-03-13
+
+* Minor enhancements
+
+  * Modules and Classes are now recognized for terminals in the same way as Regexp, true,
+    false and nil. As there are quite a few bugs in Psych/Sick to put class names in .yaml
+    files, Sexpr recognizes them also as strings starting with '::' (e.g. ::Symbol).
+
+* Bug fixes
+
+  * Use `reference.const_get(name, *false*)` when looking for tagging modules to avoid
+    finding ruby classes such as Array when tagging sexpr such as [:array, ...].
+
+# 0.5.0 / 2012-02-25
+
+* Major enhancements
+
+  * `Processor#apply` is introduced and is intended to be used to apply processing rules
+    from the inside of the processor class. This method is equivalent to `#call` when no
+    preprocessor is installed (see below).
+  * Preprocessors can now be installed on a Processor through its `use` class method.
+    Preprocessors are applied ala 'Enumerable#inject' on the s-expression passed at
+    `Prcoessor#call`. The preprocessed result is then passed to `#apply` for further
+    processing by the processor itself.
+    (Please note that preprocessing is *not* performed by `#apply` itself; preprocessing
+    is typically applied once one the whole abstract syntax tree instead of successively
+    when its nodes are encountered. Tip: use helpers for such behavior).
+  * `Processor.use` can also be used to make specific computations ahead of processing
+    (such as a symbol table). For this, simply pass a Hash that maps a computation name
+    to a processor class. Attribute readers are automatically installed on the processor
+    class and instance variables set by `#call` accordingly.
+    (Please note such preprocessors do not participate to the rewriting chain described
+    above.)
+
+* Minor enhancements
+
+  * `Processor.call(sexpr, opts)` is a shortcut for `Processor.new(opts).call(sexpr)`
+  * `Grammar#default_taggging_module` is introduced for cases where either no fine-grained
+    tagging is needed (e.g. all nodes tagged with the same Node module) or fine-grained
+    tagging is needed but is not a total function (e.g. not all nodes have a
+    specialization of Node).
+
+* Bug fixes
+
+  * Processor options taken at construction are now correctly kept under @options (an
+    attribute reader is provided)
+
+* Breaking changes
+
+  * The `Processor#main_processor` feature (undocumented and unused in examples) has been
+    removed. Using preprocessors is much cleaner that linking processors to each other.
+
 # 0.4.0 / 2012-02-23
 
 * Major enhancements
 
-  * A processing/rewriting framework has been added to Sexpr. See the `Processor` and `Rewriter`
-    classes, as well as the boolean expression example.
-  * Tracking markers can now decorate s-expressions, provided they include the `Sexpr` module.
-    Tracking markers are a simple Hash of meta-information (i.e. not taken into account for
-    equality for s-expressions). Such markers can be set with `Grammar#sexpr(sexpr, markers)`.
-    Default markers are typically provided by parsers for traceability of the s-expression
-    with the source text it comes from.
+  * A processing/rewriting framework has been added to Sexpr. See the `Processor` and
+    `Rewriter` classes, as well as the boolean expression example.
+  * Tracking markers can now decorate s-expressions, provided they include the `Sexpr`
+    module. Tracking markers are a simple Hash of meta-information (i.e. not taken into
+    account for equality for s-expressions). Such markers can be set with
+    `Grammar#sexpr(sexpr, markers)`. Default markers are typically provided by parsers for
+    traceability of the s-expression with the source text it comes from.
 
 * Minor enhancements
 
@@ -20,12 +94,12 @@
 
 * Breaking changes
 
-  * `Parser.factor` does no longer accept options. This is to avoid the 'yet another options'
-    symptom and favor convention over configuration.
+  * `Parser.factor` does no longer accept options. This is to avoid the 'yet another
+    options' symptom and favor convention over configuration.
   * Accordingly, `Sexpr::Citrus::Parser` no longer takes options at construction either.
-  * `Grammar#sexpr` does no longer allow parsing options as second argument, but takes tracking
-    markers (see enhancements). To palliate to this, default parsing options can now be
-    specified through `Grammar#default_parse_options` (see enhancements).
+  * `Grammar#sexpr` does no longer allow parsing options as second argument, but takes
+    tracking markers (see enhancements). To palliate to this, default parsing options can
+    now be specified through `Grammar#default_parse_options` (see enhancements).
 
 # 0.3.0 / 2012-02-21
 
@@ -44,11 +118,11 @@
     the result to a constant makes perfect sense and benefits from the ruby's magic naming
     feature.
   * A loaded grammar now respond to a :sexpr method that parses (if needed) and returns a
-    s-expression. The latter, and all its sub-expressions are automatically tagged with the
-    Sexpr module, as well as with user-defined modules. The latter are automatically discovered
-    with a convention over configuration heuristics that associates rule names to module names.
-    That convention may however be overridden with specific grammar methods (see the BoolExpr
-    example).
+    s-expression. The latter, and all its sub-expressions are automatically tagged with
+    the Sexpr module, as well as with user-defined modules. The latter are automatically
+    discovered with a convention over configuration heuristics that associates rule names
+    to module names. That convention may however be overridden with specific grammar
+    methods (see the BoolExpr example).
 
 # 0.2.0 / 2012-02-21
 

data/Gemfile

@@ -1,9 +1,2 @@
 source 'http://rubygems.org'
-
-group :development do
-  gem "epath", "~> 0.0.1"
-  gem "citrus", "~> 2.4"
-  gem "rake", "~> 0.9.2"
-  gem "rspec", "~> 2.8.0"
-  gem "wlang", "~> 0.10.2"
-end
+gemspec

data/README.md

@@ -84,4 +84,29 @@ https://github.com/blambeau/sexpr
 
 ### Where to read next?
 
-Have a look at the examples directory.
+Have a look at the examples directory.
+
+## Public API
+
+`sexpr` uses Semver and reached 1.0. The public API is as follows:
+
+* The structure of YAML grammar files
+* The `Sexpr` module and its public methods
+* The behavior of the `Grammar` class through its public methods
+* The behavior of the `Node` class (public methods)
+* The behavior of the `Processor` and `Rewriter` classes (public & protected methods)
+* The list of error classes and when they are raised
+
+## Contribute
+
+Please use github issues and pull requests for all questions, bug reports,
+and contributions. Don't hesitate to get in touch with us with an early code
+spike if you plan to add non trivial features.
+
+## Licence
+
+This software is distributed by Enspirit SRL under a MIT Licence. Please
+contact Bernard Lambeau (blambeau@gmail.com) with any question.
+
+Enspirit (https://enspirit.be) and Klaro App (https://klaro.cards) are both
+actively using and contributing to the library.

data/Rakefile

@@ -1,15 +1,3 @@
-begin
-  gem "bundler", "~> 1.0"
-  require "bundler/setup"
-rescue LoadError => ex
-  puts ex.message
-  abort "Bundler failed to load, (did you run 'gem install bundler' ?)"
-end
-
-# Dynamically load the gem spec
-$gemspec_file = File.expand_path('../sexpr.gemspec', __FILE__)
-$gemspec      = Kernel.eval(File.read($gemspec_file))
-
 # We run tests by default
 task :default => :test
 

data/lib/sexpr/grammar/matching.rb

@@ -15,7 +15,7 @@ module Sexpr
       end
       alias :=== :match?
 
-      private
+    private
 
       def compile_rules(rules)
         Hash[rules.map{|k,v|
@@ -26,17 +26,19 @@ module Sexpr
       def compile_rule(name, defn)
         case rule = compile_rule_defn(defn)
         when Matcher::Terminal, Matcher::Alternative
-          rule
         else
-          Matcher::Rule.new(name, rule)
+          rule = Matcher::NonTerminal.new(name, rule)
         end
+        Matcher::Rule.new(name, rule)
       end
 
       def compile_rule_defn(arg, grammar = self)
         case arg
         when Matcher
           arg
-        when Regexp, TrueClass, FalseClass, NilClass
+        when Regexp, Module
+          Matcher::Terminal.new arg
+        when TrueClass, FalseClass, NilClass
           Matcher::Terminal.new arg
         when lambda{|x| x.is_a?(Array) && x.size == 1 && x.first.is_a?(Array)}
           Matcher::Sequence.new arg.first.map{|s| compile_rule_defn(s) }
@@ -46,6 +48,9 @@ module Sexpr
           Matcher::Many.new compile_rule_defn($`), $1
         when /^[a-z][a-z_]+$/
           Matcher::Reference.new arg.to_sym, grammar
+        when /^::([A-Z][a-z]*.*)$/
+          found = $1.split('::').inject(Kernel){|cl,n| cl.const_get(n)}
+          Matcher::Terminal.new found
         else
           raise ArgumentError, "Invalid rule definition: #{arg.inspect}", caller
         end

data/lib/sexpr/grammar/parsing.rb

@@ -11,7 +11,7 @@ module Sexpr
         parser!.parse(input, options)
       end
 
-      private
+    private
 
       def parser!
         unless p = parser

data/lib/sexpr/grammar/tagging.rb

@@ -15,25 +15,42 @@ module Sexpr
         nil
       end
 
+      def default_tagging_module
+        nil
+      end
+
+      def tagging_module_for(rulename)
+        if ref = tagging_reference
+          modname = rule2modname(rulename)
+          ref.const_get(modname, false) rescue default_tagging_module
+        else
+          default_tagging_module
+        end
+      end
+
       def sexpr(input, markers = nil)
         case input
         when Array
-          tag_sexpr input, tagging_reference, markers
+          tag_sexpr input, markers
         else
           sexpr = parser!.to_sexpr(parse(input))
-          tag_sexpr sexpr, tagging_reference, markers, true
+          tag_sexpr sexpr, markers, true
         end
       end
 
-      private
+      def looks_a_sexpr?(arg)
+        arg.is_a?(Array) and arg.first.is_a?(Symbol)
+      end
+
+    private
 
-      def tag_sexpr(sexpr, reference, markers = nil, force = false)
+      def tag_sexpr(sexpr, markers = nil, force = false)
         return sexpr unless looks_a_sexpr?(sexpr)
         return sexpr if Sexpr===sexpr and not(force) and markers.nil?
 
         # set the Sexpr modules
         sexpr.extend(Sexpr) unless Sexpr===sexpr
-        tag_sexpr_with_user_module(sexpr, reference) if reference
+        tag_sexpr_with_user_module(sexpr)
 
         # set the markers if any
         if markers
@@ -43,20 +60,15 @@ module Sexpr
 
         # recurse
         sexpr[1..-1].each do |child|
-          tag_sexpr(child, reference, nil, force)
+          tag_sexpr(child, nil, force)
         end
         sexpr
       end
 
-      def tag_sexpr_with_user_module(sexpr, reference)
-        rulename = sexpr.first
-        modname  = rule2modname(rulename)
-        mod      = reference.const_get(modname) rescue nil
-        sexpr.extend(mod) if mod
-      end
-
-      def looks_a_sexpr?(arg)
-        arg.is_a?(Array) and arg.first.is_a?(Symbol)
+      def tag_sexpr_with_user_module(sexpr)
+        tag = tagging_module_for(sexpr.first)
+        sexpr.extend(tag) if tag
+        sexpr
       end
 
       def parser!

data/lib/sexpr/matcher/many.rb

@@ -28,11 +28,26 @@ module Sexpr
         i >= @min ? last : nil
       end
 
+      def parse(sexp, to = [])
+        i, last, got = 0, sexp, []
+        while sexp && (@max.nil? || i < @max)
+          if res = @term.parse(sexp, got)
+            last = res
+            i += 1
+          end
+          sexp = res
+        end
+        if i >= @min
+          got.each{|x| to << x }
+          last
+        end
+      end
+
       def inspect
         "(many #{term.inspect}, #{min}, #{max})"
       end
 
-      private
+    private
 
       def minmax(min, max)
         case min

data/lib/sexpr/matcher/non_terminal.rb

@@ -0,0 +1,31 @@
+module Sexpr
+  module Matcher
+    class NonTerminal
+      include Matcher
+
+      attr_reader :name
+      attr_reader :defn
+
+      def initialize(name, defn)
+        @name = name
+        @defn = defn
+      end
+
+      def match?(sexp)
+        return nil   unless sexp.is_a?(Array)
+        return false unless sexp.first && (sexp.first == name)
+        defn.match?(sexp[1..-1])
+      end
+
+      def eat(sexp)
+        return nil unless match?(sexp.first)
+        sexp[1..-1]
+      end
+
+      def inspect
+        "(non-terminal #{name}, #{defn.inspect})"
+      end
+
+    end # class NonTerminal
+  end # module Matcher
+end # module Sexpr

data/lib/sexpr/matcher/rule.rb

@@ -12,14 +12,11 @@ module Sexpr
       end
 
       def match?(sexp)
-        return nil unless sexp.is_a?(Array)
-        return false unless sexp.first && (sexp.first == name)
-        defn.match?(sexp[1..-1])
+        defn.match?(sexp)
       end
 
       def eat(sexp)
-        return nil unless match?(sexp.first)
-        sexp[1..-1]
+        defn.eat(sexp)
       end
 
       def inspect
@@ -28,4 +25,4 @@ module Sexpr
 
     end # class Rule
   end # module Matcher
-end # module Sexpr
+end # module Sexpr

data/lib/sexpr/matcher/terminal.rb

@@ -25,7 +25,7 @@ module Sexpr
 
       def terminal_match?(term)
         case @value
-        when Regexp
+        when Regexp, Module
           @value === term rescue false
         when TrueClass, FalseClass, NilClass
           @value == term

data/lib/sexpr/matcher.rb

@@ -10,6 +10,7 @@ end # module Sexpr
 require_relative "matcher/alternative"
 require_relative "matcher/many"
 require_relative "matcher/reference"
-require_relative "matcher/rule"
 require_relative "matcher/sequence"
-require_relative "matcher/terminal"
+require_relative "matcher/terminal"
+require_relative "matcher/non_terminal"
+require_relative "matcher/rule"

data/lib/sexpr/node.rb

@@ -31,7 +31,7 @@ module Sexpr
     end
     alias :dup :sexpr_copy
 
-    private
+  private
 
     def sexpr_copy_tagging(copy)
       (class << self; self; end).included_modules.each do |mod|

data/lib/sexpr/parser.rb

@@ -28,7 +28,7 @@ module Sexpr
 
     end # class methods
 
-    private
+  private
 
     def input_text(input)
       case input

data/lib/sexpr/processor/helper.rb

@@ -16,7 +16,7 @@ module Sexpr
         yield(processor, sexpr)
       end
 
-      private
+    private
 
       def next_call(processor, sexpr, toplevel)
         if nic = next_in_chain

data/lib/sexpr/processor/sexpr_coercions.rb

@@ -5,40 +5,32 @@ module Sexpr
       module Methods
 
         def parse(*args)
-          sexpr_grammar.parse(*args)
+          grammar.parse(*args)
         end
 
         def sexpr(*args)
-          sexpr_grammar.sexpr(*args)
+          grammar.sexpr(*args)
         end
 
       end
 
       def call(processor, sexpr, &bl)
+        g = processor.class.grammar
+
         # input coercion
-        sexpr = grammar(processor).sexpr(sexpr)
+        sexpr = g.sexpr(sexpr)
 
         # recursive call
         sexpr = next_call(processor, sexpr, bl)
 
         # output coercion
         if sexpr.is_a?(Array) and sexpr.first.is_a?(Symbol)
-          grammar(processor).sexpr(sexpr)
+          g.sexpr(sexpr)
         else
           sexpr
         end
       end
 
-      private
-
-      def grammar(processor)
-        if processor.respond_to?(:sexpr_grammar)
-          processor.sexpr_grammar
-        else
-          Sexpr
-        end
-      end
-
     end # class SexprCoercions
   end  # class Processor
 end # module Sexpr

data/lib/sexpr/processor.rb

@@ -4,36 +4,63 @@ require_relative 'processor/sexpr_coercions'
 module Sexpr
   class Processor
 
-    ### class methods
+    class << self
 
-    def self.helpers
-      @helpers ||= superclass.helpers.dup rescue [ ]
-    end
+      def grammar(grammar = nil)
+        @grammar = grammar if grammar
+        @grammar ||= (@grammar || superclass.grammar) rescue Sexpr
+        @grammar
+      end
 
-    def self.helper(helper_class)
-      methods = helper_class.const_get(:Methods) rescue nil
-      module_eval{ include methods } if methods
-      helpers << helper_class
-    end
+      def preprocessors
+        @preprocessors ||= superclass.preprocessors.dup rescue [ ]
+      end
 
-    def self.build_helper_chain(helpers = self.helpers)
-      return NullHelper.new if helpers.empty?
-      helpers[0...-1].reverse.inject(helpers.last.new) do |chain, h_class|
-        prepended = h_class.new
-        prepended.next_in_chain = chain
-        prepended
+      def use(preprocessor, options = nil)
+        preprocessor.keys.each{|k| attr_reader(k)} if preprocessor.is_a?(Hash)
+        preprocessors << [preprocessor, options]
+      end
+
+      def helpers
+        @helpers ||= superclass.helpers.dup rescue [ ]
+      end
+
+      def helper(helper_class)
+        methods = helper_class.const_get(:Methods) rescue nil
+        module_eval{ include methods } if methods
+        helpers << helper_class
       end
-    end
 
-    ### instance methods
+      def build_helper_chain(helpers = self.helpers)
+        return NullHelper.new if helpers.empty?
+        helpers[0...-1].reverse.inject(helpers.last.new) do |chain, h_class|
+          prepended = h_class.new
+          prepended.next_in_chain = chain
+          prepended
+        end
+      end
+
+      def call(sexpr, options = nil)
+        new(options).call(sexpr)
+      end
 
-    attr_reader :main_processor
+    end # class << self
 
-    def initialize(options = {})
-      @main_processor = options.delete(:main_processor) || self
+    attr_reader :options
+
+    def initialize(options = nil)
+      @options = options || {}
+    end
+
+    def grammar
+      self.class.grammar
     end
 
     def call(sexpr)
+      apply(preprocess(sexpr))
+    end
+
+    def apply(sexpr)
       help(sexpr) do |n|
         meth = :"on_#{n.first}"
         meth = :"on_missing" unless respond_to?(meth)
@@ -45,7 +72,28 @@ module Sexpr
       raise UnexpectedSexprError, "Unexpected sexpr: #{sexpr.inspect}"
     end
 
-    private
+  private
+
+    def preprocess(sexpr)
+      preprocessors = self.class.preprocessors
+      preprocessors.each do |pre|
+        sexpr = _preprocess(sexpr, pre)
+      end
+      sexpr
+    end
+
+    def _preprocess(sexpr, preprocessing)
+      pre_class, options = preprocessing
+      if Hash===pre_class
+        pre_class.each_pair do |k,v|
+          computed = v.new(options).call(sexpr)
+          self.instance_variable_set(:"@#{k}", computed)
+        end
+        sexpr
+      else
+        pre_class.new(options).call(sexpr)
+      end
+    end
 
     def helper_chain
       @helper_chain ||= self.class.build_helper_chain

data/lib/sexpr/rewriter.rb

@@ -2,17 +2,9 @@ module Sexpr
   class Rewriter < Processor
     helper SexprCoercions
 
-    def self.grammar(sexpr_grammar)
-      @sexpr_grammar = sexpr_grammar
-    end
-
-    def sexpr_grammar
-      (self.class.instance_variable_get(:"@sexpr_grammar") || super) rescue Sexpr
-    end
-
     def copy_and_apply(sexpr)
       sexpr.sexpr_copy do |copy, child|
-        copy << (Sexpr===child ? call(child) : child)
+        copy << (Sexpr===child ? apply(child) : child)
       end
     end
 

data/lib/sexpr/version.rb

@@ -1,8 +1,8 @@
 module Sexpr
   module Version
 
-    MAJOR = 0
-    MINOR = 4
+    MAJOR = 1
+    MINOR = 0
     TINY  = 0
 
     def self.to_s
@@ -11,4 +11,4 @@ module Sexpr
 
   end
   VERSION = Version.to_s
-end
+end