yard 0.2.2 → 0.2.3

data/docs/WHATSNEW.markdown

@@ -0,0 +1,96 @@
+YARD: What's New in 0.2.3?
+==========================
+
+1. **Full Ruby 1.9 support**
+2. **New parser code and handler API for 1.9**
+3. **A new `@overload` tag**
+4. **Better documentation**
+5. **Template changes and bug fixes**
+
+Full Ruby 1.9 support
+---------------------
+
+YARD's development actually focuses primarily on 1.9 from the get-go, so it is 
+not an afterthought. All features are first implemented for compatibility with 
+1.9, but of course all functionality is also tested in 1.8.x. YARD 0.2.2 was
+mostly compatible with 1.9, but the new release improves and extends in certain
+areas where compatibility was lacking. The new release should be fully functional
+in Ruby 1.9.
+  
+New parser code and handler API for 1.9
+---------------------------------------
+
+Using Ruby 1.9 also gives YARD the advantage of using the new `ripper` library 
+which was added to stdlib. The ripper parser is Ruby's official answer to 
+projects like ParseTree and ruby2ruby. Ripper allows access to the AST as it 
+is parsed by the Ruby compiler. This has some large benefits over alternative 
+projects: 
+
+  1. It is officially supported and maintained by the Ruby core team.
+  2. The AST is generated directly from the exact same code that drives the
+     compiler, meaning anything that compiles is guaranteed to generate the
+     equivalent AST.
+  3. It needs no hacks, gems or extra libs and works out of the box in 1.9.
+  4. It's *fast*.
+  
+Having the AST means that developers looking to extend YARD have much better
+access to the parsed code than in previous versions. The only caveat is that
+this library is not back-compatible to 1.8.x. Because of this, there are
+subtle changes to the handler extension API that developers use to extend YARD.
+Namely, there is now a standard API for 1.9 and a "legacy" API that can run in
+both 1.8.x and 1.9 if needed. A developer can still use the legacy API to write
+handlers that are compatible for both 1.8.x and 1.9 in one shot, or decide to
+implement the handler using both APIs. Realize that the benefit of using the new
+API means 1.9 users will get a 2.5x parsing speed increase over running the legacy
+handlers (this is *in addition to* the ~1.8x speed increase of using YARV over MRI).
+    
+A new `@overload` tag
+---------------------
+
+The new `@overload` tag enables users to document methods that take multiple 
+parameters depending on context. This is basically equivalent to RDoc's call-seq,
+but with a name that is more akin to the OOP concept of method overloading
+that is actually being employed. Here's an example:
+
+      # @overload def to_html(html, autolink = true)
+      #   This docstring describes the specific overload only.
+      #   @param [String] html the HTML
+      #   @param [Boolean] autolink whether or not to atuomatically link
+      #     URL references
+      # @overload def to_html(html, opts = {})
+      #   @param [String] html the HTML
+      #   @param [Hash] opts any attributes to add to the root HTML node
+      def to_html(*args)
+        # split args depending on context
+      end
+      
+As you can see each overload takes its own nested tags (including a docstring)
+as if it were its own method. This allows "virtual" overloading behaviour at
+the API level to make Ruby look like overload-aware languages without caring
+about the implementation details required to add the behaviour.
+
+It is still recommended practice, however, to stay away from overloading when
+possible and document the types of each method's real parameters. This allows
+toolkits making use of YARD to get accurate type information for your methods,
+for instance, allowing IDE autocompletion. There are, of course, situations
+where overload just makes more sense.
+
+Better documentation
+--------------------
+
+The first few iterations of YARD were very much a proof of concept. Few people
+were paying attention and it was really just pieced together to see what was
+feasible. Now that YARD is gaining interest, there are many developers that
+want to take advantage of its extensibility support to do some really cool stuff.
+Considerable time was spent for this release documenting, at a high level, what
+YARD can do and how it can be done. Expect this documentation to be extended and
+improved in future releases.
+
+Template changes and bug fixes
+------------------------------
+
+Of course no new release would be complete without fixing the old broken code.
+Some tags existed but were not present in generated documentation. The templates
+were mostly fixed to add the major omitted tags. In addition to template adjustments,
+many parsing bugs were ironed out to make YARD much more stable with existing projects
+(Rails, HAML, Sinatra, Ramaze, etc.).

data/lib/yard.rb

@@ -1,11 +1,14 @@
 module YARD
-  VERSION = "0.2.2"
+  VERSION = "0.2.3"
   ROOT = File.dirname(__FILE__)
   TEMPLATE_ROOT = File.join(File.dirname(__FILE__), '..', 'templates')
   
   def self.parse(*args) Parser::SourceParser.parse(*args) end
 end
 
+# Keep track of Ruby version for compatibility code
+RUBY19, RUBY18 = *(RUBY_VERSION >= "1.9" ? [true, false] : [false, true])
+
 $:.unshift(YARD::ROOT)
 
 files  = ['yard/logging', 'yard/autoload']

data/lib/yard/autoload.rb

@@ -10,6 +10,7 @@ module YARD
     autoload :ClassObject,          'yard/code_objects/class_object'
     autoload :ClassVariableObject,  'yard/code_objects/class_variable_object'
     autoload :ConstantObject,       'yard/code_objects/constant_object'
+    autoload :ExtendedMethodObject, 'yard/code_objects/extended_method_object'
     autoload :MethodObject,         'yard/code_objects/method_object'
     autoload :ModuleObject,         'yard/code_objects/module_object'
     autoload :NamespaceObject,      'yard/code_objects/namespace_object'
@@ -22,20 +23,26 @@ module YARD
     autoload :BUILTIN_MODULES,      'yard/code_objects/base'
     autoload :BUILTIN_EXCEPTIONS,   'yard/code_objects/base'
     autoload :CONSTANTMATCH,        'yard/code_objects/base'
-    autoload :ISEP,                 'yard/code_objects/base'
     autoload :METHODMATCH,          'yard/code_objects/base'
     autoload :METHODNAMEMATCH,      'yard/code_objects/base'
     autoload :NAMESPACEMATCH,       'yard/code_objects/base'
     autoload :NSEP,                 'yard/code_objects/base'
+    autoload :NSEPQ,                'yard/code_objects/base'
+    autoload :ISEP,                 'yard/code_objects/base'
+    autoload :ISEPQ,                'yard/code_objects/base'
+    autoload :CSEP,                 'yard/code_objects/base'
+    autoload :CSEPQ,                'yard/code_objects/base'
   end
   
   module Generators
     module Helpers
-      autoload :BaseHelper,             'yard/generators/helpers/base_helper'
-      autoload :FilterHelper,           'yard/generators/helpers/filter_helper'
-      autoload :HtmlHelper,             'yard/generators/helpers/html_helper'
-      autoload :MethodHelper,           'yard/generators/helpers/method_helper'
-      autoload :UMLHelper,              'yard/generators/helpers/uml_helper'
+      autoload :BaseHelper,                 'yard/generators/helpers/base_helper'
+      autoload :FilterHelper,               'yard/generators/helpers/filter_helper'
+      autoload :HtmlHelper,                 'yard/generators/helpers/html_helper'
+      autoload :HtmlSyntaxHighlightHelper,  'yard/generators/helpers/html_syntax_highlight_helper'
+      autoload :MarkupHelper,               'yard/generators/helpers/markup_helper'
+      autoload :MethodHelper,               'yard/generators/helpers/method_helper'
+      autoload :UMLHelper,                  'yard/generators/helpers/uml_helper'
     end
     
     autoload :AttributesGenerator,      'yard/generators/attributes_generator'
@@ -56,37 +63,74 @@ module YARD
     autoload :MixinsGenerator,          'yard/generators/mixins_generator'
     autoload :ModuleGenerator,          'yard/generators/module_generator'
     autoload :QuickDocGenerator,        'yard/generators/quick_doc_generator'
+    autoload :RootGenerator,            'yard/generators/root_generator'
     autoload :SourceGenerator,          'yard/generators/source_generator'
     autoload :TagsGenerator,            'yard/generators/tags_generator'
     autoload :UMLGenerator,             'yard/generators/uml_generator'
     autoload :VisibilityGroupGenerator, 'yard/generators/visibility_group_generator'
+    autoload :OverloadsGenerator,       'yard/generators/overloads_generator'
   end
   
   module Handlers
-    autoload :AliasHandler,         'yard/handlers/alias_handler'
-    autoload :AttributeHandler,     'yard/handlers/attribute_handler'
-    autoload :Base,                 'yard/handlers/base'
-    autoload :ClassHandler,         'yard/handlers/class_handler'
-    autoload :ClassVariableHandler, 'yard/handlers/class_variable_handler'
-    autoload :ConstantHandler,      'yard/handlers/constant_handler'
-    autoload :ExceptionHandler,     'yard/handlers/exception_handler'
-    autoload :MethodHandler,        'yard/handlers/method_handler'
-    autoload :MixinHandler,         'yard/handlers/mixin_handler'
-    autoload :ModuleHandler,        'yard/handlers/module_handler'
-    autoload :VisibilityHandler,    'yard/handlers/visibility_handler'
-    autoload :UndocumentableError,  'yard/handlers/base'
-    autoload :YieldHandler,         'yard/handlers/yield_handler'
+    module Ruby
+      module Legacy
+        autoload :Base,                 'yard/handlers/ruby/legacy/base'
+
+        autoload :AliasHandler,         'yard/handlers/ruby/legacy/alias_handler'
+        autoload :AttributeHandler,     'yard/handlers/ruby/legacy/attribute_handler'
+        autoload :ClassHandler,         'yard/handlers/ruby/legacy/class_handler'
+        autoload :ClassVariableHandler, 'yard/handlers/ruby/legacy/class_variable_handler'
+        autoload :ConstantHandler,      'yard/handlers/ruby/legacy/constant_handler'
+        autoload :ExceptionHandler,     'yard/handlers/ruby/legacy/exception_handler'
+        autoload :ExtendHandler,        'yard/handlers/ruby/legacy/extend_handler'
+        autoload :MethodHandler,        'yard/handlers/ruby/legacy/method_handler'
+        autoload :MixinHandler,         'yard/handlers/ruby/legacy/mixin_handler'
+        autoload :ModuleHandler,        'yard/handlers/ruby/legacy/module_handler'
+        autoload :VisibilityHandler,    'yard/handlers/ruby/legacy/visibility_handler'
+        autoload :YieldHandler,         'yard/handlers/ruby/legacy/yield_handler'
+      end
+
+      autoload :Base,                   'yard/handlers/ruby/base'
+
+      autoload :AliasHandler,           'yard/handlers/ruby/alias_handler'
+      autoload :AttributeHandler,       'yard/handlers/ruby/attribute_handler'
+      autoload :ClassHandler,           'yard/handlers/ruby/class_handler'
+      autoload :ClassConditionHandler,  'yard/handlers/ruby/class_condition_handler'
+      autoload :ClassVariableHandler,   'yard/handlers/ruby/class_variable_handler'
+      autoload :ConstantHandler,        'yard/handlers/ruby/constant_handler'
+      autoload :ExceptionHandler,       'yard/handlers/ruby/exception_handler'
+      autoload :ExtendHandler,          'yard/handlers/ruby/extend_handler'
+      autoload :MethodHandler,          'yard/handlers/ruby/method_handler'
+      autoload :MethodConditionHandler, 'yard/handlers/ruby/method_condition_handler'
+      autoload :MixinHandler,           'yard/handlers/ruby/mixin_handler'
+      autoload :ModuleHandler,          'yard/handlers/ruby/module_handler'
+      autoload :VisibilityHandler,      'yard/handlers/ruby/visibility_handler'
+      autoload :YieldHandler,           'yard/handlers/ruby/yield_handler'
+    end
+
+    autoload :Base,                       'yard/handlers/base'
+    autoload :Processor,                  'yard/handlers/processor'
   end
 
   module Parser
-    module RubyToken
-      require 'yard/parser/ruby_lex' # Too much to include manually
+    module Ruby
+      module Legacy
+        autoload :Statement,      'yard/parser/ruby/legacy/statement'
+        autoload :StatementList,  'yard/parser/ruby/legacy/statement_list'
+        autoload :TokenList,      'yard/parser/ruby/legacy/token_list'
+
+        module RubyToken
+          require 'yard/parser/ruby/legacy/ruby_lex' # Too much to include manually
+        end
+      end
+
+      autoload :AstNode,           'yard/parser/ruby/ast_node'
+      autoload :ParserSyntaxError, 'yard/parser/ruby/ruby_parser'
+      autoload :RubyParser,        'yard/parser/ruby/ruby_parser'
     end
-    
-    autoload :SourceParser,   'yard/parser/source_parser'
-    autoload :Statement,      'yard/parser/statement'
-    autoload :StatementList,  'yard/parser/statement_list'
-    autoload :TokenList,      'yard/parser/token_list'
+
+    autoload :SourceParser,        'yard/parser/source_parser'
+    autoload :UndocumentableError, 'yard/parser/source_parser'
   end
   
   module Rake
@@ -102,15 +146,19 @@ module YARD
   
   module Tags
     autoload :DefaultFactory, 'yard/tags/default_factory'
+    autoload :DefaultTag,     'yard/tags/default_tag'
     autoload :Library,        'yard/tags/library'
+    autoload :OptionTag,      'yard/tags/option_tag'
+    autoload :OverloadTag,    'yard/tags/overload_tag'
+    autoload :RefTag,         'yard/tags/ref_tag'
+    autoload :RefTagList,     'yard/tags/ref_tag_list'
     autoload :Tag,            'yard/tags/tag'
+    autoload :TagFormatError, 'yard/tags/tag_format_error'
   end
 
-  autoload :Registry, 'yard/registry'
+  autoload :Docstring, 'yard/docstring'
+  autoload :Registry,  'yard/registry'
 end
 
-# Load handlers immediately
-YARD::Handlers.constants.each {|c| YARD::Handlers.const_get(c) }
-
 # P() needs to be loaded right away
-YARD::CodeObjects::Proxy
+YARD::CodeObjects::Proxy

data/lib/yard/cli/yard_graph.rb

@@ -4,6 +4,7 @@ module YARD
   module CLI
     class YardGraph
       attr_reader :options, :visibilities
+      attr_reader :objects
       
       def self.run(*args) new.run(*args) end
         
@@ -17,9 +18,9 @@ module YARD
       end
       
       def run(*args)
-        optparse(*args)
         Registry.load
-        Generators::UMLGenerator.new(options).generate Registry.root
+        optparse(*args)
+        Generators::UMLGenerator.new(options).generate(*objects)
       end
       
       private
@@ -75,6 +76,11 @@ module YARD
 
         begin
           opts.parse!(args)
+          if args.first
+            @objects = args.map {|o| Registry.at(o) }.compact
+          else
+            @objects = [Registry.root]
+          end
         rescue => e
           STDERR.puts e.message
           STDERR << "\n" << opts

data/lib/yard/cli/yardoc.rb

@@ -3,8 +3,11 @@ require 'optparse'
 module YARD
   module CLI
     class Yardoc
+      DEFAULT_YARDOPTS_FILE = ".yardopts"
+      
       attr_reader :options, :visibilities
       attr_accessor :files, :reload, :generate
+      attr_accessor :options_file
       
       def self.run(*args) new.run(*args) end
         
@@ -13,7 +16,6 @@ module YARD
           :format => :html, 
           :template => :default, 
           :serializer => YARD::Serializers::FileSystemSerializer.new, 
-          :readme => ['README', 'README.txt'],
           :verifier => lambda do |gen, obj| 
             return false if gen.respond_to?(:visibility) && !visibilities.include?(gen.visibility) 
           end
@@ -22,19 +24,38 @@ module YARD
         @reload = true
         @generate = true
         @files = ['lib/**/*.rb']
+        @options_file = DEFAULT_YARDOPTS_FILE
       end
       
       def run(*args)
+        args += support_rdoc_document_file!
+        optparse(*yardopts)
         optparse(*args)
         Registry.load(files, reload)
         
         if generate
-          Generators::FullDocGenerator.new(options).generate Registry.all(:module, :class)
+          Generators::FullDocGenerator.new(options).generate(all_objects)
         end
       end
+
+      def all_objects
+        Registry.all(:root, :module, :class)
+      end
+      
+      def yardopts
+        IO.read(options_file).split(/\s+/)
+      rescue Errno::ENOENT
+        []
+      end
       
       private
       
+      def support_rdoc_document_file!
+        IO.read(".document").split(/\s+/)
+      rescue Errno::ENOENT
+        []
+      end
+      
       def optparse(*args)
         serialopts = SymbolHash.new
         
@@ -43,6 +64,10 @@ module YARD
 
         opts.separator "(if a list of source files is omitted, lib/**/*.rb is used.)"
         opts.separator ""
+        opts.separator "A base set of options can be specified by adding a .yardopts"
+        opts.separator "file to your base path containing all extra options separated"
+        opts.separator "by whitespace."
+        opts.separator ""
         opts.separator "General Options:"
 
         opts.on('-c', '--use-cache [FILE]', 
@@ -65,6 +90,10 @@ module YARD
             exit
           end
         end
+        
+        opts.on('--legacy', 'Use old style parser and handlers. Unavailable under Ruby 1.8.x') do
+          YARD::Parser::SourceParser.parser_type = :ruby18
+        end
 
         opts.separator ""
         opts.separator "Output options:"
@@ -80,13 +109,37 @@ module YARD
         opts.on('--private', "Show or don't show private methods. (default hides private)") do 
           visibilities.push(:private) 
         end
-        
+
+        opts.on('--no-highlight', "Don't highlight code in docs as Ruby.") do 
+          options[:no_highlight] = true
+        end
+
         opts.on('-r', '--readme FILE', 'The readme file used as the title page of documentation.') do |readme|
+          raise Errno::ENOENT, readme unless File.file?(readme)
           options[:readme] = readme
         end
         
-        opts.on('-d', '--output-dir PATH', 
+        opts.on('--files FILE1,FILE2,...', 'Any extra comma separated static files to be included (eg. FAQ)') do |files|
+          options[:files] = []
+          files.split(",").each do |file|
+            raise Errno::ENOENT, file unless File.file?(file)
+            options[:files] << file
+          end
+        end
+
+        opts.on('-m', '--markup MARKUP', 
+                'Markup style used in documentation, like textile, markdown or rdoc. (defaults to rdoc)') do |markup|
+          options[:markup] = markup.to_sym
+        end
+
+        opts.on('-M', '--markup-provider MARKUP_PROVIDER', 
+                'Overrides the library used to process markup formatting (specify the gem name)') do |markup_provider|
+          options[:markup_provider] = markup_provider.to_sym
+        end
+        
+        opts.on('-o', '--output-dir PATH', 
                 'The output directory. (defaults to ./doc)') do |dir|
+          options[:serializer] = nil
           serialopts[:basepath] = dir
         end
 
@@ -97,7 +150,7 @@ module YARD
 
         opts.on('-p', '--template-path PATH', 
                 'The template path to look for templates in. (used with -t).') do |path|
-          YARD::Generator::Base.register_template_path(path)
+          YARD::Generators::Base.register_template_path(path)
         end
         
         opts.on('-f', '--format FORMAT', 
@@ -123,9 +176,9 @@ module YARD
         # Last minute modifications
         self.files = args unless args.empty?
         self.reload = false if self.files.empty?
-        visibilities.uniq!
-        options[:serializer] = Serializers::FileSystemSerializer.new(serialopts)
+        self.visibilities.uniq!
+        options[:serializer] ||= Serializers::FileSystemSerializer.new(serialopts)
       end
     end
   end
-end
+end