tomparse 0.3.0 → 0.4.0

data/.index

@@ -64,5 +64,5 @@ description: ! 'TomParse provides no other functionality than to take a code com
   and parse it in to a convenient object-oriented structure in accordance
 
   with TomDoc standard.'
-version: 0.3.0
-date: '2013-01-21'
+version: 0.4.0
+date: '2013-02-11'

data/.yardopts

@@ -0,0 +1,8 @@
+--title "TomParse"
+--readme README.md
+--plugin tomdoc
+--private
+lib/**/*.rb
+-
+*.md
+*.txt

data/HISTORY.md

@@ -1,15 +1,40 @@
 # Release History
 
+## 0.4.0 / 2013-02-10
+
+For this release, the document parser has been substantially refactored.
+It should be both more robust and more flexible. Arguments can now
+have an optional `Arguments` heading, and the arguments list can be
+indented. A new `Options` section is supported to handle Ruby 2.0 style
+keyword options. Tag labels ony need to be capitalized. They no longer
+have to be all-caps. And lastly, the `Signature` section has been
+repurposed for describing variant *parameter signatures*, not dynamic
+methods. As such `signature_fields` has been deprecated. Yes, these
+are additions and some deviation from the original TomDoc spec, but
+when purely conceptual specs go into practice they have a tendency to 
+adapt to practical requirements.
+
+Changes:
+
+* Refactor document parser.
+* Add support for optional Arguments section header.
+* Add support for Options section.
+* Modify purpose of Signatures section.
+* Deprecate signature field list.
+* Tag labels only need to be capitialized, not all-caps.
+
+
 ## 0.3.0 / 2012-01-21
 
 This release fixes a bug which prevented descriptions from having
 multiple paragraphs. In addition it adds support for section tags.
-See the README for more information of tags.
+See the README for more information on tags.
 
 Changes:
 
 * Fix multi-paragraph description parsing.
 * Add support for tags.
+* Fix support for option hashes. (Jonas Oberschweiber)
 
 
 ## 0.2.1 / 2012-04-30

data/README.md

@@ -13,7 +13,7 @@
 
 TomParse is a TomDoc parser for Ruby. It provides no other functionality than
 to take a code comment and parse it in to a convenient object-oriented
-structure in accordance with TomDoc standard. See [TomDoc](https://github.com/mojombo/tomdoc)
+structure in accordance with the TomDoc standard. See [TomDoc](https://github.com/mojombo/tomdoc)
 for more information about the TomDoc format.
 
 
@@ -59,12 +59,59 @@ looks something like this:
       text * count
     end
 
-### Extra Features
+## Extra Features
 
-Okay, we told a little white lie in the description. TomParse does take a tiny 
-bit of liberty with the specification to offer up some additional documentation
-goodness. In particular, TomParse recoginizes paragraphs starting with an all-caps
-word, followed by a colon and a space, as special tags. Here is an example:
+Okay, we told a little white lie in the description. TomParse does take some
+liberties with the specification to offer up some additional documentation
+goodness.
+
+### Arguments
+
+The TomDoc specification is rather strict about how arguments are written --they
+have to be the second section just after the description. TomParse offers a little
+more flexability by allowing for an `Arguments` header to be used.
+
+```ruby
+  # Method to do fooey.
+  #
+  # Examples
+  #   foo(name)
+  #
+  # Arguments
+  #   name - Name of the fooey.
+  #
+  # Returns nothing.
+  def foo(name)
+    ...
+```
+
+We still recommend putting the arguments right after the desciption in most cases,
+but there are times when its reads better to put them lower, such when using a
+`Signatures` section (see below).
+
+### Options
+
+Ruby 2.0 finally introduces *keyword arguments* to the language. TomDoc's current
+support of argument options, as a secondary argument list of a Hash argument,
+doesn't take keyoword arguments into good account. To remedy this TomParse provides
+and `Options` section, and it written just like one would an `Arguments` section.
+
+```ruby
+  # Method to do fooey.
+  #
+  # Options
+  #   debug - Turn on debug mode? (default: false)
+  #
+  # Returns nothing.
+  def foo(**options)
+    ...
+```
+
+### Tags
+
+One really nice new feature of TomParse is it's ability to recoginize sections
+starting with a capitalized word followed by a colon and a space, as special *tags*.
+Here is an example:
 
 ```ruby
   # Method to do something.
@@ -77,7 +124,44 @@ word, followed by a colon and a space, as special tags. Here is an example:
 ```
 
 When this is parsed, rather then lumping the TODO line in with the description,
-the TomDoc instance will have a `tags` property containing `{'todo'=>'This is a todo note.'}`.
+the TomDoc instance will have a `tags` entry containing `['TODO', 'This is a todo note.']`.
+It is important for consumer applications to recognize this. They can either just
+add the tags back into the description when generating documentation, or handle
+them  separately. But tags don't have to occur right after the description. They
+can occur any place in the documentation.
+
+### Signatures
+
+TomParse does not support the Signature sections in exactly the same fashion as
+the TomDoc specification describes. Rather then define dynamic methods, signatures
+are used to specify alternate argument patterns, one signature per line.
+
+```ruby
+  # Method to do something.
+  #
+  # name - The name of the thing.
+  # pattern - The pattern of the thing.
+  #
+  # Signatures
+  #
+  #   dosomething(name)
+  #   dosomething(name=>pattern)
+  #
+  # Returns nothing.
+  def dosomething(*args)
+    ...
+```
+
+Technically the Signatures section can still be used to designate a dynamic method,
+but as you can see by the above example, TomParse does not support the *field* list.
+If needed just use the arguments list for these as well.
+
+This choice was made, btw, because support for signatures as defined by the spec,
+leads to very non-optimal code. It requires scanning every chunk of documentation
+for a `Signature` section in order to determine how to treat it. What is needed
+is a more universal syntax, that can be easily recognized by some clear identifier
+at the top of a comment --and one that doesn't confuse dynamic method definitions
+with the more traditional concept of method signatures.
 
 
 ## Resources
@@ -107,6 +191,7 @@ the TomDoc instance will have a `tags` property containing `{'todo'=>'This is a
 
 * [Citron](http://rubyworks.github.com/citron) (testing)
 * [AE](http://rubyworks.github.com/ae) (testing)
+* [Fire](http://detroit.github.com/fire) (building)
 * [Detroit](http://detroit.github.com/detroit) (building)
 
 

data/lib/tomparse.rb

@@ -1,4 +1,8 @@
 module TomParse
+  require 'tomparse/parser'
+  require 'tomparse/parse_error'
+  require 'tomparse/argument'
+  require 'tomparse/option'
 
   # Main interface to parser.
   #
@@ -11,9 +15,10 @@ module TomParse
   # Encapsulate parsed tomdoc documentation.
   #
   # TODO: Currently uses lazy evaluation, eventually this should
-  # be removed and simply parsed at initialization time.
+  # be removed and simply parsed all at once.
   #
   class TomDoc
+
     attr_accessor :raw
 
     # Public: Initialize a TomDoc object.
@@ -22,25 +27,15 @@ module TomParse
     #
     # Returns new TomDoc instance.
     def initialize(text, parse_options={})
-      @raw = text.to_s.strip
-
-      @arguments        = []
-      @options          = []  # TODO
-      @examples         = []
-      @returns          = []
-      @raises           = []
-      @signatures       = []
-      @signature_fields = []
-      @tags             = {}
-
-      parse unless @raw.empty?
+      @parser = Parser.new(text, parse_options)
+      @parser.parse
     end
 
     # Raw documentation text.
     #
     # Returns String of raw documentation text.
     def to_s
-      @raw
+      @parser.raw
     end
 
     # Validate given comment text.
@@ -54,9 +49,7 @@ module TomParse
     #
     # Returns true if comment is valid, otherwise false.
     def valid?
-      return false if !raw.include?('Returns')
-      return false if sections.size < 2
-      true
+      @parser.valid?
     end
 
     # Validate raw comment.
@@ -64,553 +57,121 @@ module TomParse
     # Returns true if comment is valid.
     # Raises ParseError if comment is not valid.
     def validate
-      if !raw.include?('Returns')
-        raise ParseError.new("No `Returns' statement.")
-      end
-
-      if sections.size < 2
-        raise ParseError.new("No description section found.")
-      end
-
-      true
+      @parser.validate
     end
 
+    # TODO: Should we clean the raw documentation here and then pass it on to the parser?
+
     # The raw comment text cleaned-up and ready for section parsing.
     #
     # Returns cleaned-up comment String.
     def tomdoc
-      lines = raw.split("\n")
-
-      # remove remark symbol
-      if lines.all?{ |line| /^\s*#/ =~ line }   
-        lines = lines.map do |line|
-          line =~ /^(\s*#)/ ? line.sub($1, '') : nil
-        end
-      end
-
-      # for some reason the first line is coming in without indention
-      # regardless, so we temporary remove it
-      first = lines.shift
-
-      # remove indention
-      spaces = lines.map do |line|
-        next if line.strip.empty?
-        md = /^(\s*)/.match(line)
-        md ? md[1].size : nil
-      end.compact
-
-      space = spaces.min || 0
-      lines = lines.map do |line|
-        if line.strip.empty?
-          line.strip
-        else
-          line[space..-1]
-        end
-      end
-
-      # put first line back
-      lines.unshift(first.sub(/^\s*/,'')) if first
-
-      lines.compact.join("\n")
+      return @parser.tomdoc
     end
 
     # List of comment sections. These are divided simply on "\n\n".
     #
     # Returns Array of comment sections.
     def sections
-      parsed {
-        @sections
-      }
+      @parser.sections
     end
 
     # Description of method or class/module.
     #
     # Returns description String.
     def description
-      parsed {
-        @description
-      }
+      @parser.description
     end
 
-    # Description of method or class/module.
+    # Arguments list.
     #
-    # Returns description String.
+    # Returns list of arguments.
     def arguments
-      parsed {
-        @arguments
-      }
+      @parser.arguments
     end
     alias args arguments
 
+    # Keyword arguments, aka Options.
+    #
+    # Returns list of options.
+    def options
+      @parser.options
+    end
+    alias keyword_arguments options
+
     # List of use examples of a method or class/module.
     #
     # Returns String of examples.
     def examples
-      parsed {
-        @examples
-      }
+      @parser.examples
     end
 
     # Description of a methods yield procedure.
     #
     # Returns String decription of yield procedure.
     def yields
-      parsed {
-        @yields
-      }
+      @parser.yields
     end
 
     # The list of retrun values a method can return.
     #
     # Returns Array of method return descriptions.
     def returns
-      parsed {
-        @returns
-      }
+      @parser.returns
     end
 
     # A list of errors a method might raise.
     #
     # Returns Array of method raises descriptions.
     def raises
-      parsed {
-        @raises
-      }
+      @parser.raises
     end
 
     # A list of alternate method signatures.
     #
     # Returns Array of signatures.
     def signatures
-      parsed {
-        @signatures 
-      }
+      @parser.signatures 
     end
 
-    # A list of signature fields.
+    # Deprecated: A list of signature fields.
+    #
+    # TODO: Presently this will always return an empty list. It will either
+    # be removed or renamed in future version.
     #
     # Returns Array of field definitions.
     def signature_fields
-      parsed {
-        @signature_fields
-      }
+      @parser.signature_fields
     end
 
-    # A mapping of tags.
+    # List of tags.
     #
-    # Returns Hash of tags.
+    # Returns an associatve array of tags. [Array<Array<String>>]
     def tags
-      parsed {
-        @tags
-      }
+      @parser.tags
     end
 
     # Check if method is public.
     #
     # Returns true if method is public.
     def public?
-      parsed {
-        @status == 'Public'
-      }
+      @parser.public?
     end
 
     # Check if method is internal.
     #
     # Returns true if method is internal.
     def internal?
-      parsed {
-        @status == 'Internal'
-      }
+      @parser.internal?
     end
 
     # Check if method is deprecated.
     #
     # Returns true if method is deprecated.
     def deprecated?
-      parsed {
-        @status == 'Deprecated'
-      }
+      @parser.deprecated?
     end
 
-  private
-
-    # Has the comment been parsed yet?
-    def parsed(&block)
-      parse unless @parsed
-      block.call
-    end
-
-    # Internal: Parse the Tomdoc formatted comment.
-    #
-    # Returns true if there was a comment to parse.
-    def parse
-      @parsed = true
-
-      sections = tomdoc.split("\n\n")
-
-      return false if sections.empty?
-
-      # The description is always the first section, but it may have
-      # multiple paragraphs. This routine collects those together.
-      desc = [sections.shift]
-      loop do
-         s = sections.first
-         break if s.nil?
-         break if s =~ /^\w+\s+\-/m
-         break if section_type(s) != nil
-         desc << sections.shift
-      end
-      sections = [desc.join("\n\n")] + sections
-  
-      @sections = sections.dup
-
-      parse_description(sections.shift)
-
-      if sections.first && sections.first =~ /^\w+\s+\-/m
-        parse_arguments(sections.shift)
-      end
-
-      current = sections.shift
-      while current
-        case type = section_type(current)
-        when :examples
-          parse_examples(current, sections)
-        when :yields
-          parse_yields(current)
-        when :returns
-          parse_returns(current)  # also does raises
-        when :raises
-          parse_returns(current)  # also does returns
-        when :signature
-          parse_signature(current, sections)
-        when Symbol
-          parse_tag(current)
-        end
-        current = sections.shift
-      end
-
-      return @parsed
-    end
-
-    #
-    #
-    def section_type(section)
-      case section
-      when /^Examples/
-        :examples
-      when /^Yields/
-        :yields
-      when /^Returns/
-        :returns
-      when /^Raises/
-        :raises
-      when /^Signature/
-        :signature
-      when /^([A-Z]*)\:\ /
-        $1.downcase.to_sym
-      else
-        nil
-      end
-    end
-
-    # Recognized description status.
-    TOMDOC_STATUS = ['Internal', 'Public', 'Deprecated']
-
-    # Parse description.
-    #
-    # section - String containig description.
-    #
-    # Returns nothing.
-    def parse_description(section)
-      if md = /^([A-Z]\w+\:)/.match(section)
-        @status = md[1].chomp(':')
-        if TOMDOC_STATUS.include?(@status)
-          @description = md.post_match.strip
-        else
-          @description = section.strip
-        end
-      else
-        @description = section.strip
-      end   
-    end
-
-    # Parse arguments section. Arguments occur subsequent to
-    # the description.
-    #
-    # section - String containing argument definitions.
-    #
-    # Returns nothing.
-    def parse_arguments(section)
-      args = []
-      last_indent = nil
-
-      section.lines.each do |line|
-        next if line.strip.empty?
-        indent = line.scan(/^\s*/)[0].to_s.size
-
-        if last_indent && indent > 0 && indent >= last_indent
-          args.last.description << "\r\n" + line
-        else
-          param, desc = line.split(" - ")
-          args << Argument.new(param.strip, desc.strip) if param && desc
-        end
-
-        last_indent = indent
-      end
-
-      args.each do |arg|
-        arg.parse(arg.description)
-      end
-
-      @arguments = args
-    end
-
-    # Parse examples.
-    #
-    # section  - String starting with `Examples`.
-    # sections - All sections subsequent to section.
-    #
-    # Returns nothing.
-    def parse_examples(section, sections)
-      examples = []
-
-      section = section.sub('Examples', '').gsub(/^\s{2}/,'')
-
-      examples << section unless section.empty?
-      while sections.first && sections.first !~ /^\S/
-        examples << sections.shift.gsub(/^\s{2}/,'')
-      end
-
-      @examples = examples
-    end
-
-    # Parse yields section.
-    #
-    # section - String contaning Yields line.
-    #
-    # Returns nothing.
-    def parse_yields(section)
-      @yields = section.strip
-    end
-
-    # Parse returns section.
-    #
-    # section - String contaning Returns and/or Raises lines.
-    #
-    # Returns nothing.
-    def parse_returns(section)
-      returns, raises, current = [], [], []
-
-      lines = section.split("\n")  
-      lines.each do |line|
-        case line
-        when /^Returns/
-          returns << line
-          current = returns
-        when /^Raises/
-          raises << line
-          current = raises
-        when /^\s+/
-          current.last << line.squeeze(' ')
-        else
-          current << line  # TODO: What to do with non-compliant line?
-        end
-      end
-
-      @returns.concat(returns)
-      @raises.concat(raises)
-    end
-
-    # Parse signature section.
-    #
-    # section  - String starting with `Signature`.
-    # sections - All sections subsequent to section.
-    #
-    # Returns nothing.
-    def parse_signature(section, sections=[])
-      signatures = []
-
-      section = section.sub('Signature', '').strip
-
-      signatures << section unless section.empty?
-
-      while sections.first && sections.first !~ /^\S/
-        sigs = sections.shift
-        sigs.split("\n").each do |s|
-          signatures << s.strip
-        end
-      end
-
-      @signatures = signatures
-
-      if sections.first && sections.first =~ /^\w+\s*\-/m
-        parse_signature_fields(sections.shift)
-      end
-    end
-
-    # Subsequent to Signature section there can be field
-    # definitions.
-    #
-    # section  - String subsequent to signatures.
-    #
-    # Returns nothing.
-    def parse_signature_fields(section)
-      args = []
-      last_indent = nil
-
-      section.split("\n").each do |line|
-        next if line.strip.empty?
-        indent = line.scan(/^\s*/)[0].to_s.size
-
-        if last_indent && indent > last_indent
-          args.last.description << line.squeeze(" ")
-        else
-          param, desc = line.split(" - ")
-          args << Argument.new(param.strip, desc.strip) if param && desc
-        end
-
-        last_indent = indent
-      end
-
-      @signature_fields = args
-    end
-
-    # Tags are arbitrary sections designated by all cap labels and a colon.
-    #
-    # label   - String name of the tag.
-    # section - String of the tag section.
-    #
-    # Returns nothing.
-    def parse_tag(section)
-
-      md = /^([A-Z]*)\:\ /.match(section)
-
-      label = md[1]
-      text  = md.post_match
-
-      @tags[label.downcase] = text
-    end
-
-  end
-
-  # Encapsulate a method argument.
-  #
-  class Argument
-
-    attr_accessor :name
-
-    attr_accessor :description
-
-    attr_accessor :options
-
-    # Create new Argument object.
-    #
-    # name        - name of argument
-    # description - argument description
-    #
-    def initialize(name, description = '')
-      @name = name.to_s.intern
-      parse(description)
-    end
-
-    # Is this an optional argument?
-    #
-    # Returns Boolean.
-    def optional?
-      @description.downcase.include? 'optional'
-    end
-
-    # Parse arguments section. Arguments occur subsequent to
-    # the description.
-    #
-    # section - String containing argument definitions.
-    #
-    # Returns nothing.
-    def parse(description)
-      desc = []
-      opts = []
-
-      lines = description.lines.to_a
-
-      until lines.empty? or /^\s+\:(\w+)\s+-\s+(.*?)$/ =~ lines.first
-        desc << lines.shift.chomp.squeeze(" ")
-      end
-
-      opts = []
-      last_indent = nil
-
-      lines.each do |line|
-        next if line.strip.empty?
-        indent = line.scan(/^\s*/)[0].to_s.size
-
-        if last_indent && indent > last_indent
-          opts.last.description << line.squeeze(" ")
-        else
-          param, d = line.split(" - ")
-          opts << Option.new(param.strip, d.strip) if param && d
-        end
-
-        last_indent = indent
-      end
-
-      @description = desc.join
-      @options     = opts
-    end
-
-  end
-
-  # Encapsulate a named parameter.
-  #
-  class Option
-
-    attr_accessor :name
-
-    attr_accessor :description
-
-    # Create new Argument object.
-    #
-    # name        - name of option
-    # description - option description
-    #
-    def initialize(name, description = '')
-      @name = name.to_s.intern
-      @description = description
-    end
-
-    # Is this a required option?
-    #
-    # Returns Boolean.
-    def required?
-      @description.downcase.include? 'required'
-    end
-
-  end
-
-  # Raised when comment can't be parsed, which means it's most
-  # likely not valid TomDoc.
-  #
-  class ParseError < RuntimeError
-    # Create new ParseError object.
-    #
-    # doc - document string
-    #
-    def initialize(doc)
-      @doc = doc
-    end
-
-    # Provide access to document string.
-    #
-    # Returns String.
-    def message
-      @doc
-    end
-
-    # Provide access to document string.
-    #
-    # Returns String.
-    def to_s
-      @doc
-    end
   end
 
 end