ruby_speech 0.4.0 → 0.5.0

data/CHANGELOG.md

@@ -1,5 +1,20 @@
 # develop
 
+# 0.5.0 - 2012-01-03
+  * Feature: Add a whole bunch more SSML elements:
+  ** p & s
+  ** mark
+  ** desc
+  ** sub
+  ** phoneme
+  * Feature: Added the ability to inline grammar rule references in both destructive and non-destructive modes
+  * Feature: Added the ability to tokenize a grammar, turning all tokens into unambiguous `<token/>` elements
+  * Feature: Added the ability to whitespace normalize a grammar
+  * Feature: Added the ability to match an input string against a Grammar
+  * Feature: Constructing a GRXML grammar with a root rule specified but not provided will raise an exception
+  * Feature: Embedding a GRXML grammar of a mode different from the host will raise an exception
+  * Bugfix: Fix upward traversal through a document via #parent
+
 # 0.4.0 - 2011-12-30
   * Feature: Add the ability to look up child elements by name/attributes easily
   * Feature: Allow easy access to a GRXML grammar's root rule element

data/README.md

@@ -46,15 +46,15 @@ Once your `Speak` is fully prepared and you're ready to send it off for processi
 You may also then need to call `to_s`.
 
 
-Contruct a GRXML (SRGS) document like this:
+Construct a GRXML (SRGS) document like this:
 
 ```ruby
 require 'ruby_speech'
 
-grammy = RubySpeech::GRXML.draw :mode => 'dtmf', :root => 'digits' do
-  rule id: 'digits' do
+grammy = RubySpeech::GRXML.draw mode: :dtmf, root: 'pin' do
+  rule id: 'digit' do
     one_of do
-      0.upto(9) {|d| item { d.to_s } }
+      ('0'..'9').map { |d| item { d } }
     end
   end
 
@@ -79,8 +79,8 @@ grammy.to_s
 which becomes
 
 ```xml
-<grammar xmlns="http://www.w3.org/2001/06/grammar" version="1.0" xml:lang="en-US" mode="dtmf" root="digits">
-  <rule id="digits">
+<grammar xmlns="http://www.w3.org/2001/06/grammar" version="1.0" xml:lang="en-US" mode="dtmf" root="pin">
+  <rule id="digit">
     <one-of>
       <item>0</item>
       <item>1</item>
@@ -103,6 +103,101 @@ which becomes
 </grammar>
 ```
 
+### Grammar matching
+
+It is possible to match some arbitrary input against a GRXML grammar. In order to do so, certain normalization routines should first be run on the grammar in order to prepare it for matching. These are reference inlining, tokenization and whitespace normalization, and are described [in the SRGS spec](http://www.w3.org/TR/speech-grammar/#S2.1). This process will transform the above grammar like so:
+
+```ruby
+grammy.inline!
+grammy.tokenize!
+grammy.normalize_whitespace
+```
+
+```xml
+<grammar xmlns="http://www.w3.org/2001/06/grammar" version="1.0" xml:lang="en-US" mode="dtmf" root="pin">
+  <rule id="pin" scope="public">
+    <one-of>
+      <item>
+        <item repeat="4">
+          <one-of>
+            <item>
+              <token>0</token>
+            </item>
+            <item>
+              <token>1</token>
+            </item>
+            <item>
+              <token>2</token>
+            </item>
+            <item>
+              <token>3</token>
+            </item>
+            <item>
+              <token>4</token>
+            </item>
+            <item>
+              <token>5</token>
+            </item>
+            <item>
+              <token>6</token>
+            </item>
+            <item>
+              <token>7</token>
+            </item>
+            <item>
+              <token>8</token>
+            </item>
+            <item>
+              <token>9</token>
+            </item>
+          </one-of>
+        </item>
+        <token>#</token>
+      </item>
+      <item>
+        <token>*</token>
+        <token>9</token>
+      </item>
+    </one-of>
+  </rule>
+</grammar>
+```
+
+Matching against some sample input strings then returns the following results:
+
+```ruby
+>> subject.match '*9'
+=> #<RubySpeech::GRXML::Match:0x00000100ae5d98
+      @mode = :dtmf,
+      @confidence = 1,
+      @utterance = "*9",
+      @interpretation = "*9"
+    >
+>> subject.match '1234#'
+=> #<RubySpeech::GRXML::Match:0x00000100b7e020
+      @mode = :dtmf,
+      @confidence = 1,
+      @utterance = "1234#",
+      @interpretation = "1234#"
+    >
+>> subject.match '5678#'
+=> #<RubySpeech::GRXML::Match:0x00000101218688
+      @mode = :dtmf,
+      @confidence = 1,
+      @utterance = "5678#",
+      @interpretation = "5678#"
+    >
+>> subject.match '1111#'
+=> #<RubySpeech::GRXML::Match:0x000001012f69d8
+      @mode = :dtmf,
+      @confidence = 1,
+      @utterance = "1111#",
+      @interpretation = "1111#"
+    >
+>> subject.match '111'
+=> #<RubySpeech::GRXML::NoMatch:0x00000101371660>
+```
+
 Check out the [YARD documentation](http://rdoc.info/github/benlangfeld/ruby_speech/master/frames) for more
 
 ## Features:
@@ -114,6 +209,13 @@ Check out the [YARD documentation](http://rdoc.info/github/benlangfeld/ruby_spee
 * `<say-as/>`
 * `<break/>`
 * `<audio/>`
+* `<p/>` and `<s/>`
+* `<phoneme/>`
+* `<sub/>`
+
+#### Misc
+* `<mark/>`
+* `<desc/>`
 
 ### GRXML
 * Document construction
@@ -126,17 +228,9 @@ Check out the [YARD documentation](http://rdoc.info/github/benlangfeld/ruby_spee
 
 ## TODO:
 ### SSML
-#### Document Structure
-* `<p/>` and `<s/>`
-* `<phoneme/>`
-* `<sub/>`
 * `<lexicon/>`
 * `<meta/>` and `<metadata/>`
 
-#### Misc
-* `<mark/>`
-* `<desc/>`
-
 ### GRXML
 * `<meta/>` and `<metadata/>`
 * `<example/>`

data/lib/ruby_speech/generic_element.rb

@@ -4,7 +4,7 @@ module RubySpeech
   module GenericElement
 
     def self.included(klass)
-      klass.class_attribute :registered_ns, :registered_name
+      klass.class_attribute :registered_ns, :registered_name, :defaults
       klass.extend ClassMethods
     end
 
@@ -43,7 +43,7 @@ module RubySpeech
       def import(node)
         node = Nokogiri::XML.parse(node, nil, nil, Nokogiri::XML::ParseOptions::NOBLANKS).root unless node.is_a?(Nokogiri::XML::Node)
         return node.content if node.is_a?(Nokogiri::XML::Text)
-        klass = class_from_registration(node.element_name)
+        klass = class_from_registration node.element_name
         if klass && klass != self
           klass.import node
         else
@@ -51,26 +51,73 @@ module RubySpeech
         end
       end
 
-      def new(element_name, atts = {}, &block)
+      def new(atts = {}, &block)
         blk_proc = lambda do |new_node|
-          atts.each_pair { |k, v| new_node.send :"#{k}=", v }
+          (self.defaults || {}).merge(atts).each_pair { |k, v| new_node.send :"#{k}=", v }
           block_return = new_node.eval_dsl_block &block
-          new_node << new_node.encode_special_chars(block_return) if block_return.is_a?(String)
+          new_node << block_return if block_return.is_a?(String)
         end
 
         case RUBY_VERSION.split('.')[0,2].join.to_i
         when 18
-          super(element_name).tap do |n|
+          super(self.registered_name, nil, self.namespace).tap do |n|
             blk_proc[n]
           end
         else
-          super(element_name) do |n|
+          super(self.registered_name, nil, self.namespace) do |n|
             blk_proc[n]
           end
         end
       end
     end
 
+    attr_writer :parent
+
+    def parent
+      @parent || super
+    end
+
+    def inherit(node)
+      self.parent = node.parent
+      super
+    end
+
+    def version
+      read_attr :version
+    end
+
+    def version=(other)
+      write_attr :version, other
+    end
+
+    ##
+    # @return [String] the base URI to which relative URLs are resolved
+    #
+    def base_uri
+      read_attr :base
+    end
+
+    ##
+    # @param [String] uri the base URI to which relative URLs are resolved
+    #
+    def base_uri=(uri)
+      write_attr 'xml:base', uri
+    end
+
+    def to_doc
+      Nokogiri::XML::Document.new.tap do |doc|
+        doc << self
+      end
+    end
+
+    def +(other)
+      self.class.new(:base_uri => base_uri).tap do |new_element|
+        (self.children + other.children).each do |child|
+          new_element << child
+        end
+      end
+    end
+
     def eval_dsl_block(&block)
       return unless block_given?
       @block_binding = eval "self", block.binding
@@ -106,19 +153,24 @@ module RubySpeech
       when self.class.module::Element
         self << other
       else
-        raise ArgumentError, "Can only embed a String or an SSML element"
+        raise ArgumentError, "Can only embed a String or a #{self.class.module} element, not a #{other}"
       end
     end
 
     def string(other)
-      self << encode_special_chars(other)
+      self << other
+    end
+
+    def <<(other)
+      other = encode_special_chars other if other.is_a? String
+      super other
     end
 
     def method_missing(method_name, *args, &block)
       const_name = method_name.to_s.sub('ssml', '').titleize.gsub(' ', '')
       if self.class.module.const_defined?(const_name)
         const = self.class.module.const_get const_name
-        self << const.new(*args, &block)
+        embed const.new(*args, &block)
       elsif @block_binding && @block_binding.respond_to?(method_name)
         @block_binding.send method_name, *args, &block
       else
@@ -126,6 +178,15 @@ module RubySpeech
       end
     end
 
+    def clone
+      GRXML.import to_xml
+    end
+
+    def traverse(&block)
+      nokogiri_children.each { |j| j.traverse &block }
+      block.call self
+    end
+
     def eql?(o, *args)
       super o, :content, :children, *args
     end

data/lib/ruby_speech/grxml.rb

@@ -13,6 +13,9 @@ module RubySpeech
       autoload :Token
     end
 
+    autoload :Match
+    autoload :NoMatch
+
     InvalidChildError = Class.new StandardError
 
     GRXML_NAMESPACE = 'http://www.w3.org/2001/06/grammar'
@@ -21,7 +24,7 @@ module RubySpeech
       Grammar.new(attributes).tap do |grammar|
         block_return = grammar.eval_dsl_block &block
         grammar << block_return if block_return.is_a?(String)
-      end
+      end.assert_has_matching_root_rule
     end
 
     def self.import(other)

data/lib/ruby_speech/grxml/element.rb

@@ -18,6 +18,10 @@ module RubySpeech
       alias_method :nokogiri_children, :children
 
       include GenericElement
+
+      def regexp_content # :nodoc:
+        children.map(&:regexp_content).join
+      end
     end # Element
   end # GRXML
 end # RubySpeech

data/lib/ruby_speech/grxml/grammar.rb

@@ -18,37 +18,9 @@ module RubySpeech
 
       register :grammar
 
-      VALID_CHILD_TYPES = [Nokogiri::XML::Element, Nokogiri::XML::Text, Rule, Tag].freeze
-
-      ##
-      # Create a new GRXML grammar root element
-      #
-      # @param [Hash] atts Key-value pairs of options mapping to setter methods
-      #
-      # @return [Grammar] an element for use in an GRXML document
-      #
-      def self.new(atts = {}, &block)
-        new_node = super('grammar', atts)
-        new_node[:version] = '1.0'
-        new_node.namespace = GRXML_NAMESPACE
-        new_node.language ||= "en-US"
-        new_node.eval_dsl_block &block
-        new_node
-      end
-
-      ##
-      # @return [String] the base URI to which relative URLs are resolved
-      #
-      def base_uri
-        read_attr :base
-      end
+      self.defaults = { :version => '1.0', :language => "en-US" }
 
-      ##
-      # @param [String] uri the base URI to which relative URLs are resolved
-      #
-      def base_uri=(uri)
-        write_attr 'xml:base', uri
-      end
+      VALID_CHILD_TYPES = [Nokogiri::XML::Element, Nokogiri::XML::Text, Rule, Tag].freeze
 
       ##
       #
@@ -57,7 +29,7 @@ module RubySpeech
       # @return [String]
       #
       def mode
-        read_attr :mode
+        read_attr :mode, :to_sym
       end
 
       ##
@@ -84,17 +56,6 @@ module RubySpeech
         write_attr :root, ia
       end
 
-      def <<(arg)
-        raise InvalidChildError, "A Grammar can only accept Rule and Tag as children" unless VALID_CHILD_TYPES.include? arg.class
-        super
-      end
-
-      def to_doc
-        Nokogiri::XML::Document.new.tap do |doc|
-          doc << self
-        end
-      end
-
       ##
       #
       # @return [String]
@@ -110,11 +71,39 @@ module RubySpeech
         write_attr :'tag-format', s
       end
 
+      ##
+      # @return [Rule] The root rule node for the document
+      #
       def root_rule
         children(:rule, :id => root).first
       end
 
+      ##
+      # Checks for a root rule matching the value of the root tag
+      #
+      # @raises [InvalidChildError] if there is not a rule present in the document with the correct ID
+      #
+      # @return [Grammar] self
+      #
+      def assert_has_matching_root_rule
+        raise InvalidChildError, "A GRXML document must have a rule matching the root rule name" unless has_matching_root_rule?
+        self
+      end
+
+      ##
+      # @return [Grammar] an inlined copy of self
+      #
       def inline
+        clone.inline!
+      end
+
+      ##
+      # Replaces rulerefs in the document with a copy of the original rule.
+      # Removes all top level rules except the root rule
+      #
+      # @return self
+      #
+      def inline!
         find("//ns:ruleref", :ns => namespace_href).each do |ref|
           rule = children(:rule, :id => ref[:uri].sub(/^#/, '')).first
           ref.swap rule.nokogiri_children
@@ -126,17 +115,159 @@ module RubySpeech
         self
       end
 
-      def +(other)
-        self.class.new(:base_uri => base_uri).tap do |new_grammar|
-          (self.children + other.children).each do |child|
-            new_grammar << child
+      ##
+      # Replaces textual content of the document with token elements containing such content.
+      # This homogenises all tokens in the document to a consistent format for processing.
+      #
+      def tokenize!
+        traverse do |element|
+          next unless element.is_a? Nokogiri::XML::Text
+
+          next if self.class.import(element.parent).is_a? Token
+
+          tokens = split_tokens(element).map do |string|
+            Token.new.tap { |token| token << string }
           end
+
+          element.swap Nokogiri::XML::NodeSet.new(Nokogiri::XML::Document.new, tokens)
         end
       end
 
+      ##
+      # Normalizes whitespace within tokens in the document according to the rules in the SRGS spec (http://www.w3.org/TR/speech-grammar/#S2.1)
+      # Leading and trailing whitespace is removed, and multiple spaces within the string are collapsed down to single spaces.
+      #
+      def normalize_whitespace
+        traverse do |element|
+          next if element === self
+
+          imported_element = self.class.import element
+          next unless imported_element.respond_to? :normalize_whitespace
+
+          imported_element.normalize_whitespace
+          element.swap imported_element
+        end
+      end
+
+      ##
+      # Checks the grammar for a match against an input string
+      #
+      # @param [String] other the input string to check for a match with the grammar
+      #
+      # @return [NoMatch, Match] depending on the result of a match attempt. If a match can be found, it will be returned with appropriate mode/confidence/utterance and interpretation attributes
+      #
+      # @example A grammar that takes a 4 digit pin terminated by hash, or the *9 escape sequence
+      #     ```ruby
+      #       grammar = RubySpeech::GRXML.draw :mode => :dtmf, :root => 'pin' do
+      #         rule :id => 'digit' do
+      #           one_of do
+      #             ('0'..'9').map { |d| item { d } }
+      #           end
+      #         end
+      #
+      #         rule :id => 'pin', :scope => 'public' do
+      #           one_of do
+      #             item do
+      #               item :repeat => '4' do
+      #                 ruleref :uri => '#digit'
+      #               end
+      #               "#"
+      #             end
+      #             item do
+      #               "\* 9"
+      #             end
+      #           end
+      #         end
+      #       end
+      #
+      #       >> subject.match '*9'
+      #       => #<RubySpeech::GRXML::Match:0x00000100ae5d98
+      #             @mode = :dtmf,
+      #             @confidence = 1,
+      #             @utterance = "*9",
+      #             @interpretation = "*9"
+      #           >
+      #       >> subject.match '1234#'
+      #       => #<RubySpeech::GRXML::Match:0x00000100b7e020
+      #             @mode = :dtmf,
+      #             @confidence = 1,
+      #             @utterance = "1234#",
+      #             @interpretation = "1234#"
+      #           >
+      #       >> subject.match '111'
+      #       => #<RubySpeech::GRXML::NoMatch:0x00000101371660>
+      #
+      #     ```
+      #
+      def match(other)
+        regex = to_regexp
+        return NoMatch.new if regex == //
+        match = regex.match other
+        return NoMatch.new unless match
+
+        Match.new :mode           => mode,
+                  :confidence     => dtmf? ? 1 : 0,
+                  :utterance      => other,
+                  :interpretation => interpret_utterance(other)
+      end
+
+      ##
+      # Converts the grammar into a regular expression for matching
+      #
+      # @return [Regexp] a regular expression which is equivalent to the grammar
+      #
+      def to_regexp
+        /^#{regexp_content.join}$/
+      end
+
+      def regexp_content
+        root_rule.children.map &:regexp_content
+      end
+
+      def dtmf?
+        mode == :dtmf
+      end
+
+      def voice?
+        mode == :voice
+      end
+
+      def <<(arg)
+        raise InvalidChildError, "A Grammar can only accept Rule and Tag as children" unless VALID_CHILD_TYPES.include? arg.class
+        super
+      end
+
       def eql?(o)
         super o, :language, :base_uri, :mode, :root
       end
+
+      def embed(other)
+        raise InvalidChildError, "Embedded grammars must have the same mode" if other.is_a?(self.class) && other.mode != mode
+        super
+      end
+
+      private
+
+      def has_matching_root_rule?
+        !root || root_rule
+      end
+
+      def interpret_utterance(utterance)
+        conversion = Hash.new { |hash, key| hash[key] = key }
+        conversion['*'] = 'star'
+        conversion['#'] = 'pound'
+
+        utterance.chars.inject [] do |array, digit|
+          array << "dtmf-#{conversion[digit]}"
+        end.join ' '
+      end
+
+      def split_tokens(element)
+        element.to_s.split(/(\".*\")/).reject(&:empty?).map do |string|
+          match = string.match /^\"(.*)\"$/
+          match ? match[1] : string.split(' ')
+        end.flatten
+      end
     end # Grammar
   end # GRXML
 end # RubySpeech